[DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

[dmagda]

Igniters,

I'd like us to move on and finish our conversation on the IGFS [1] and
Hadoop Accelerator [2] support.

To my knowledge, there is no single committer who maintains the
integrations; they are no longer tested and, even more, the community
stopped providing the binaries since Ignite 2.6.0 release (look for
In-Memory Hadoop Accelerator table [3]).

Why all of that happened? Because of a little value, something succeeds
while something fails. Does it mean that Ignite cannot be used for Hadoop
acceleration, in general? No, quite the opposite, it CAN be used, but a
solution is different. Have Ignite with native persistence deployed close
to your Hadoop cluster (replace GridGain with Ignite) [4].

So, I propose we remove IGFS and In-Memory Hadoop Accelerator from our
master repository and rework existing public documentation showing how to
achieve the acceleration with Ignite.

Any supporters or objections?


[1] https://apacheignite-fs.readme.io/docs/in-memory-file-system
[2] https://apacheignite-fs.readme.io/docs/hadoop-accelerator
[3] https://ignite.apache.org/download.cgi#binaries
[4]
https://docs.gridgain.com/docs/bdb-getting-started#section-gridgain-data-lake-accelerator

-
Denis

[Ivan Pavlukhin]

Denis,

I must say that aforementioned solutions for a Hadoop ecosystem appear
from time to time in questions on a user mailing list. So, it seems
that there is a practical need for such solutions.

But of course it does not mean that we should continue a support of
IGFS and Hadoop Accelerator. If both are not solutions that fit well
common use cases then we should discontinue it. If any of them is very
good for it's purposes but we do not have a capacity to support it
without sacrificing main Ignite goals then we still should discontinue
it in my mind.

P.S. Personally I am a fan of UNIX way. I like ideas of a single
responsibility and integrations. And I suppose there are other Ignite
features which could be dropped.

ср, 12 июн. 2019 г. в 21:04, Denis Magda <[hidden email]>:

>
> Igniters,
>
> I'd like us to move on and finish our conversation on the IGFS [1] and
> Hadoop Accelerator [2] support.
>
> To my knowledge, there is no single committer who maintains the
> integrations; they are no longer tested and, even more, the community
> stopped providing the binaries since Ignite 2.6.0 release (look for
> In-Memory Hadoop Accelerator table [3]).
>
> Why all of that happened? Because of a little value, something succeeds
> while something fails. Does it mean that Ignite cannot be used for Hadoop
> acceleration, in general? No, quite the opposite, it CAN be used, but a
> solution is different. Have Ignite with native persistence deployed close
> to your Hadoop cluster (replace GridGain with Ignite) [4].
>
> So, I propose we remove IGFS and In-Memory Hadoop Accelerator from our
> master repository and rework existing public documentation showing how to
> achieve the acceleration with Ignite.
>
> Any supporters or objections?
>
>
> [1] https://apacheignite-fs.readme.io/docs/in-memory-file-system
> [2] https://apacheignite-fs.readme.io/docs/hadoop-accelerator
> [3] https://ignite.apache.org/download.cgi#binaries
> [4]
> https://docs.gridgain.com/docs/bdb-getting-started#section-gridgain-data-lake-accelerator
>
> -
> Denis

--
Best regards,
Ivan Pavlukhin

[Alexey Goncharuk]

Denis,

I fully support this idea.

First, looking back, I do not think it was a good design in the first place
to build IGFS on top of Ignite caches. Second, I have never seen a case
where IGFS provided significant performance boost. Usually it's either all
data already fits buffer cache, and IGFS caching is not needed; or data
does not fit buffer cache, and access pattern is close to full scan and
additional caching in IGFS does not make sense.

пн, 17 июн. 2019 г. в 11:28, Павлухин Иван <[hidden email]>:

> Denis,
>
> I must say that aforementioned solutions for a Hadoop ecosystem appear
> from time to time in questions on a user mailing list. So, it seems
> that there is a practical need for such solutions.
>
> But of course it does not mean that we should continue a support of
> IGFS and Hadoop Accelerator. If both are not solutions that fit well
> common use cases then we should discontinue it. If any of them is very
> good for it's purposes but we do not have a capacity to support it
> without sacrificing main Ignite goals then we still should discontinue
> it in my mind.
>
> P.S. Personally I am a fan of UNIX way. I like ideas of a single
> responsibility and integrations. And I suppose there are other Ignite
> features which could be dropped.
>
> ср, 12 июн. 2019 г. в 21:04, Denis Magda <[hidden email]>:
>
> >
> > Igniters,
> >
> > I'd like us to move on and finish our conversation on the IGFS [1] and
> > Hadoop Accelerator [2] support.
> >
> > To my knowledge, there is no single committer who maintains the
> > integrations; they are no longer tested and, even more, the community
> > stopped providing the binaries since Ignite 2.6.0 release (look for
> > In-Memory Hadoop Accelerator table [3]).
> >
> > Why all of that happened? Because of a little value, something succeeds
> > while something fails. Does it mean that Ignite cannot be used for Hadoop
> > acceleration, in general? No, quite the opposite, it CAN be used, but a
> > solution is different. Have Ignite with native persistence deployed close
> > to your Hadoop cluster (replace GridGain with Ignite) [4].
> >
> > So, I propose we remove IGFS and In-Memory Hadoop Accelerator from our
> > master repository and rework existing public documentation showing how to
> > achieve the acceleration with Ignite.
> >
> > Any supporters or objections?
> >
> >
> > [1] https://apacheignite-fs.readme.io/docs/in-memory-file-system
> > [2] https://apacheignite-fs.readme.io/docs/hadoop-accelerator
> > [3] https://ignite.apache.org/download.cgi#binaries
> > [4]
> >
> https://docs.gridgain.com/docs/bdb-getting-started#section-gridgain-data-lake-accelerator
> >
> > -
> > Denis
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>

[Nikolay Izhikov-2]

+1 from me to reduce supported feature list.

Guys, are we talking about Ignite 3.0?


В Пн, 17/06/2019 в 11:57 +0300, Alexey Goncharuk пишет:

> Denis,
>
> I fully support this idea.
>
> First, looking back, I do not think it was a good design in the first place
> to build IGFS on top of Ignite caches. Second, I have never seen a case
> where IGFS provided significant performance boost. Usually it's either all
> data already fits buffer cache, and IGFS caching is not needed; or data
> does not fit buffer cache, and access pattern is close to full scan and
> additional caching in IGFS does not make sense.
>
> пн, 17 июн. 2019 г. в 11:28, Павлухин Иван <[hidden email]>:
>
> > Denis,
> >
> > I must say that aforementioned solutions for a Hadoop ecosystem appear
> > from time to time in questions on a user mailing list. So, it seems
> > that there is a practical need for such solutions.
> >
> > But of course it does not mean that we should continue a support of
> > IGFS and Hadoop Accelerator. If both are not solutions that fit well
> > common use cases then we should discontinue it. If any of them is very
> > good for it's purposes but we do not have a capacity to support it
> > without sacrificing main Ignite goals then we still should discontinue
> > it in my mind.
> >
> > P.S. Personally I am a fan of UNIX way. I like ideas of a single
> > responsibility and integrations. And I suppose there are other Ignite
> > features which could be dropped.
> >
> > ср, 12 июн. 2019 г. в 21:04, Denis Magda <[hidden email]>:
> >
> > >
> > > Igniters,
> > >
> > > I'd like us to move on and finish our conversation on the IGFS [1] and
> > > Hadoop Accelerator [2] support.
> > >
> > > To my knowledge, there is no single committer who maintains the
> > > integrations; they are no longer tested and, even more, the community
> > > stopped providing the binaries since Ignite 2.6.0 release (look for
> > > In-Memory Hadoop Accelerator table [3]).
> > >
> > > Why all of that happened? Because of a little value, something succeeds
> > > while something fails. Does it mean that Ignite cannot be used for Hadoop
> > > acceleration, in general? No, quite the opposite, it CAN be used, but a
> > > solution is different. Have Ignite with native persistence deployed close
> > > to your Hadoop cluster (replace GridGain with Ignite) [4].
> > >
> > > So, I propose we remove IGFS and In-Memory Hadoop Accelerator from our
> > > master repository and rework existing public documentation showing how to
> > > achieve the acceleration with Ignite.
> > >
> > > Any supporters or objections?
> > >
> > >
> > > [1] https://apacheignite-fs.readme.io/docs/in-memory-file-system
> > > [2] https://apacheignite-fs.readme.io/docs/hadoop-accelerator
> > > [3] https://ignite.apache.org/download.cgi#binaries
> > > [4]
> > >
> >
> > https://docs.gridgain.com/docs/bdb-getting-started#section-gridgain-data-lake-accelerator
> > >
> > > -
> > > Denis
> >
> >
> >
> > --
> > Best regards,
> > Ivan Pavlukhin
> >

[Igor Sapego-2]

+1 from me. Hadoop Accelerator seems like an outdated solution,
and I believe IGFS was only added to support Hadoop Accelerator.

Best Regards,
Igor


On Mon, Jun 17, 2019 at 12:00 PM Nikolay Izhikov <[hidden email]>
wrote:

> +1 from me to reduce supported feature list.
>
> Guys, are we talking about Ignite 3.0?
>
>
> В Пн, 17/06/2019 в 11:57 +0300, Alexey Goncharuk пишет:
> > Denis,
> >
> > I fully support this idea.
> >
> > First, looking back, I do not think it was a good design in the first
> place
> > to build IGFS on top of Ignite caches. Second, I have never seen a case
> > where IGFS provided significant performance boost. Usually it's either
> all
> > data already fits buffer cache, and IGFS caching is not needed; or data
> > does not fit buffer cache, and access pattern is close to full scan and
> > additional caching in IGFS does not make sense.
> >
> > пн, 17 июн. 2019 г. в 11:28, Павлухин Иван <[hidden email]>:
> >
> > > Denis,
> > >
> > > I must say that aforementioned solutions for a Hadoop ecosystem appear
> > > from time to time in questions on a user mailing list. So, it seems
> > > that there is a practical need for such solutions.
> > >
> > > But of course it does not mean that we should continue a support of
> > > IGFS and Hadoop Accelerator. If both are not solutions that fit well
> > > common use cases then we should discontinue it. If any of them is very
> > > good for it's purposes but we do not have a capacity to support it
> > > without sacrificing main Ignite goals then we still should discontinue
> > > it in my mind.
> > >
> > > P.S. Personally I am a fan of UNIX way. I like ideas of a single
> > > responsibility and integrations. And I suppose there are other Ignite
> > > features which could be dropped.
> > >
> > > ср, 12 июн. 2019 г. в 21:04, Denis Magda <[hidden email]>:
> > >
> > > >
> > > > Igniters,
> > > >
> > > > I'd like us to move on and finish our conversation on the IGFS [1]
> and
> > > > Hadoop Accelerator [2] support.
> > > >
> > > > To my knowledge, there is no single committer who maintains the
> > > > integrations; they are no longer tested and, even more, the community
> > > > stopped providing the binaries since Ignite 2.6.0 release (look for
> > > > In-Memory Hadoop Accelerator table [3]).
> > > >
> > > > Why all of that happened? Because of a little value, something
> succeeds
> > > > while something fails. Does it mean that Ignite cannot be used for
> Hadoop
> > > > acceleration, in general? No, quite the opposite, it CAN be used,
> but a
> > > > solution is different. Have Ignite with native persistence deployed
> close
> > > > to your Hadoop cluster (replace GridGain with Ignite) [4].
> > > >
> > > > So, I propose we remove IGFS and In-Memory Hadoop Accelerator from
> our
> > > > master repository and rework existing public documentation showing
> how to
> > > > achieve the acceleration with Ignite.
> > > >
> > > > Any supporters or objections?
> > > >
> > > >
> > > > [1] https://apacheignite-fs.readme.io/docs/in-memory-file-system
> > > > [2] https://apacheignite-fs.readme.io/docs/hadoop-accelerator
> > > > [3] https://ignite.apache.org/download.cgi#binaries
> > > > [4]
> > > >
> > >
> > >
> https://docs.gridgain.com/docs/bdb-getting-started#section-gridgain-data-lake-accelerator
> > > >
> > > > -
> > > > Denis
> > >
> > >
> > >
> > > --
> > > Best regards,
> > > Ivan Pavlukhin
> > >
>

[Dmitry Pavlov]

Hi Igniters,

+1 from me provided that we move sources to some supplementary repository.
If someone later would like to maintain/fix this module, he/she should be
able to access sources with current state of the master.

Sincerely,
Dmitriy Pavlov

пн, 17 июн. 2019 г. в 18:25, Igor Sapego <[hidden email]>:

> +1 from me. Hadoop Accelerator seems like an outdated solution,
> and I believe IGFS was only added to support Hadoop Accelerator.
>
> Best Regards,
> Igor
>
>
> On Mon, Jun 17, 2019 at 12:00 PM Nikolay Izhikov <[hidden email]>
> wrote:
>
> > +1 from me to reduce supported feature list.
> >
> > Guys, are we talking about Ignite 3.0?
> >
> >
> > В Пн, 17/06/2019 в 11:57 +0300, Alexey Goncharuk пишет:
> > > Denis,
> > >
> > > I fully support this idea.
> > >
> > > First, looking back, I do not think it was a good design in the first
> > place
> > > to build IGFS on top of Ignite caches. Second, I have never seen a case
> > > where IGFS provided significant performance boost. Usually it's either
> > all
> > > data already fits buffer cache, and IGFS caching is not needed; or data
> > > does not fit buffer cache, and access pattern is close to full scan and
> > > additional caching in IGFS does not make sense.
> > >
> > > пн, 17 июн. 2019 г. в 11:28, Павлухин Иван <[hidden email]>:
> > >
> > > > Denis,
> > > >
> > > > I must say that aforementioned solutions for a Hadoop ecosystem
> appear
> > > > from time to time in questions on a user mailing list. So, it seems
> > > > that there is a practical need for such solutions.
> > > >
> > > > But of course it does not mean that we should continue a support of
> > > > IGFS and Hadoop Accelerator. If both are not solutions that fit well
> > > > common use cases then we should discontinue it. If any of them is
> very
> > > > good for it's purposes but we do not have a capacity to support it
> > > > without sacrificing main Ignite goals then we still should
> discontinue
> > > > it in my mind.
> > > >
> > > > P.S. Personally I am a fan of UNIX way. I like ideas of a single
> > > > responsibility and integrations. And I suppose there are other Ignite
> > > > features which could be dropped.
> > > >
> > > > ср, 12 июн. 2019 г. в 21:04, Denis Magda <[hidden email]>:
> > > >
> > > > >
> > > > > Igniters,
> > > > >
> > > > > I'd like us to move on and finish our conversation on the IGFS [1]
> > and
> > > > > Hadoop Accelerator [2] support.
> > > > >
> > > > > To my knowledge, there is no single committer who maintains the
> > > > > integrations; they are no longer tested and, even more, the
> community
> > > > > stopped providing the binaries since Ignite 2.6.0 release (look for
> > > > > In-Memory Hadoop Accelerator table [3]).
> > > > >
> > > > > Why all of that happened? Because of a little value, something
> > succeeds
> > > > > while something fails. Does it mean that Ignite cannot be used for
> > Hadoop
> > > > > acceleration, in general? No, quite the opposite, it CAN be used,
> > but a
> > > > > solution is different. Have Ignite with native persistence deployed
> > close
> > > > > to your Hadoop cluster (replace GridGain with Ignite) [4].
> > > > >
> > > > > So, I propose we remove IGFS and In-Memory Hadoop Accelerator from
> > our
> > > > > master repository and rework existing public documentation showing
> > how to
> > > > > achieve the acceleration with Ignite.
> > > > >
> > > > > Any supporters or objections?
> > > > >
> > > > >
> > > > > [1] https://apacheignite-fs.readme.io/docs/in-memory-file-system
> > > > > [2] https://apacheignite-fs.readme.io/docs/hadoop-accelerator
> > > > > [3] https://ignite.apache.org/download.cgi#binaries
> > > > > [4]
> > > > >
> > > >
> > > >
> >
> https://docs.gridgain.com/docs/bdb-getting-started#section-gridgain-data-lake-accelerator
> > > > >
> > > > > -
> > > > > Denis
> > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Ivan Pavlukhin
> > > >
> >
>

[dmagda]

>
> +1 from me to reduce supported feature list.
> Guys, are we talking about Ignite 3.0?


Nikolay, I would discontinue IGFS before 3.0. Let's do this in the next
release? As for other features and integrations, 3.0 should be considered
as a version.


> +1 from me provided that we move sources to some supplementary repository.
> If someone later would like to maintain/fix this module, he/she should be
> able to access sources with current state of the master.


Dmitry, are you suggesting to move the sources to Github and abandon them
there? Sort of legacy code cemetery.

-
Denis


On Mon, Jun 17, 2019 at 2:00 AM Nikolay Izhikov <[hidden email]> wrote:

> +1 from me to reduce supported feature list.
>
> Guys, are we talking about Ignite 3.0?
>
>
> В Пн, 17/06/2019 в 11:57 +0300, Alexey Goncharuk пишет:
> > Denis,
> >
> > I fully support this idea.
> >
> > First, looking back, I do not think it was a good design in the first
> place
> > to build IGFS on top of Ignite caches. Second, I have never seen a case
> > where IGFS provided significant performance boost. Usually it's either
> all
> > data already fits buffer cache, and IGFS caching is not needed; or data
> > does not fit buffer cache, and access pattern is close to full scan and
> > additional caching in IGFS does not make sense.
> >
> > пн, 17 июн. 2019 г. в 11:28, Павлухин Иван <[hidden email]>:
> >
> > > Denis,
> > >
> > > I must say that aforementioned solutions for a Hadoop ecosystem appear
> > > from time to time in questions on a user mailing list. So, it seems
> > > that there is a practical need for such solutions.
> > >
> > > But of course it does not mean that we should continue a support of
> > > IGFS and Hadoop Accelerator. If both are not solutions that fit well
> > > common use cases then we should discontinue it. If any of them is very
> > > good for it's purposes but we do not have a capacity to support it
> > > without sacrificing main Ignite goals then we still should discontinue
> > > it in my mind.
> > >
> > > P.S. Personally I am a fan of UNIX way. I like ideas of a single
> > > responsibility and integrations. And I suppose there are other Ignite
> > > features which could be dropped.
> > >
> > > ср, 12 июн. 2019 г. в 21:04, Denis Magda <[hidden email]>:
> > >
> > > >
> > > > Igniters,
> > > >
> > > > I'd like us to move on and finish our conversation on the IGFS [1]
> and
> > > > Hadoop Accelerator [2] support.
> > > >
> > > > To my knowledge, there is no single committer who maintains the
> > > > integrations; they are no longer tested and, even more, the community
> > > > stopped providing the binaries since Ignite 2.6.0 release (look for
> > > > In-Memory Hadoop Accelerator table [3]).
> > > >
> > > > Why all of that happened? Because of a little value, something
> succeeds
> > > > while something fails. Does it mean that Ignite cannot be used for
> Hadoop
> > > > acceleration, in general? No, quite the opposite, it CAN be used,
> but a
> > > > solution is different. Have Ignite with native persistence deployed
> close
> > > > to your Hadoop cluster (replace GridGain with Ignite) [4].
> > > >
> > > > So, I propose we remove IGFS and In-Memory Hadoop Accelerator from
> our
> > > > master repository and rework existing public documentation showing
> how to
> > > > achieve the acceleration with Ignite.
> > > >
> > > > Any supporters or objections?
> > > >
> > > >
> > > > [1] https://apacheignite-fs.readme.io/docs/in-memory-file-system
> > > > [2] https://apacheignite-fs.readme.io/docs/hadoop-accelerator
> > > > [3] https://ignite.apache.org/download.cgi#binaries
> > > > [4]
> > > >
> > >
> > >
> https://docs.gridgain.com/docs/bdb-getting-started#section-gridgain-data-lake-accelerator
> > > >
> > > > -
> > > > Denis
> > >
> > >
> > >
> > > --
> > > Best regards,
> > > Ivan Pavlukhin
> > >
>

[Mmuzaf]

Folks,

+1 to reduce the number of supported features.

Probably, the best solution will be removing IGFS from core module and
making it as an Ignite plugin (will require some efforts to do this).
I've also think we can move IGFS to the separate branch (from the
master one) if someone will decide merge to latest changes from the
master branch to build Ignite from scratch with IGFS feature.

On Mon, 17 Jun 2019 at 22:42, Denis Magda <[hidden email]> wrote:

>
> >
> > +1 from me to reduce supported feature list.
> > Guys, are we talking about Ignite 3.0?
>
>
> Nikolay, I would discontinue IGFS before 3.0. Let's do this in the next
> release? As for other features and integrations, 3.0 should be considered
> as a version.
>
>
> > +1 from me provided that we move sources to some supplementary repository.
> > If someone later would like to maintain/fix this module, he/she should be
> > able to access sources with current state of the master.
>
>
> Dmitry, are you suggesting to move the sources to Github and abandon them
> there? Sort of legacy code cemetery.
>
> -
> Denis
>
>
> On Mon, Jun 17, 2019 at 2:00 AM Nikolay Izhikov <[hidden email]> wrote:
>
> > +1 from me to reduce supported feature list.
> >
> > Guys, are we talking about Ignite 3.0?
> >
> >
> > В Пн, 17/06/2019 в 11:57 +0300, Alexey Goncharuk пишет:
> > > Denis,
> > >
> > > I fully support this idea.
> > >
> > > First, looking back, I do not think it was a good design in the first
> > place
> > > to build IGFS on top of Ignite caches. Second, I have never seen a case
> > > where IGFS provided significant performance boost. Usually it's either
> > all
> > > data already fits buffer cache, and IGFS caching is not needed; or data
> > > does not fit buffer cache, and access pattern is close to full scan and
> > > additional caching in IGFS does not make sense.
> > >
> > > пн, 17 июн. 2019 г. в 11:28, Павлухин Иван <[hidden email]>:
> > >
> > > > Denis,
> > > >
> > > > I must say that aforementioned solutions for a Hadoop ecosystem appear
> > > > from time to time in questions on a user mailing list. So, it seems
> > > > that there is a practical need for such solutions.
> > > >
> > > > But of course it does not mean that we should continue a support of
> > > > IGFS and Hadoop Accelerator. If both are not solutions that fit well
> > > > common use cases then we should discontinue it. If any of them is very
> > > > good for it's purposes but we do not have a capacity to support it
> > > > without sacrificing main Ignite goals then we still should discontinue
> > > > it in my mind.
> > > >
> > > > P.S. Personally I am a fan of UNIX way. I like ideas of a single
> > > > responsibility and integrations. And I suppose there are other Ignite
> > > > features which could be dropped.
> > > >
> > > > ср, 12 июн. 2019 г. в 21:04, Denis Magda <[hidden email]>:
> > > >
> > > > >
> > > > > Igniters,
> > > > >
> > > > > I'd like us to move on and finish our conversation on the IGFS [1]
> > and
> > > > > Hadoop Accelerator [2] support.
> > > > >
> > > > > To my knowledge, there is no single committer who maintains the
> > > > > integrations; they are no longer tested and, even more, the community
> > > > > stopped providing the binaries since Ignite 2.6.0 release (look for
> > > > > In-Memory Hadoop Accelerator table [3]).
> > > > >
> > > > > Why all of that happened? Because of a little value, something
> > succeeds
> > > > > while something fails. Does it mean that Ignite cannot be used for
> > Hadoop
> > > > > acceleration, in general? No, quite the opposite, it CAN be used,
> > but a
> > > > > solution is different. Have Ignite with native persistence deployed
> > close
> > > > > to your Hadoop cluster (replace GridGain with Ignite) [4].
> > > > >
> > > > > So, I propose we remove IGFS and In-Memory Hadoop Accelerator from
> > our
> > > > > master repository and rework existing public documentation showing
> > how to
> > > > > achieve the acceleration with Ignite.
> > > > >
> > > > > Any supporters or objections?
> > > > >
> > > > >
> > > > > [1] https://apacheignite-fs.readme.io/docs/in-memory-file-system
> > > > > [2] https://apacheignite-fs.readme.io/docs/hadoop-accelerator
> > > > > [3] https://ignite.apache.org/download.cgi#binaries
> > > > > [4]
> > > > >
> > > >
> > > >
> > https://docs.gridgain.com/docs/bdb-getting-started#section-gridgain-data-lake-accelerator
> > > > >
> > > > > -
> > > > > Denis
> > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Ivan Pavlukhin
> > > >
> >

[dmagda]

Igniters,

Thanks a lot for sharing your opinion. As I see, there is a consensus that
IGFS and Hadoop Accelerator are to be discontinued and no longer supported
by the community.

As for the source code, if the community prefers moving the source code to
another repository rather than removing it, then let's do it. I see 3
solutions here:

   - The simplest - just point out to the latest Ignite release branch that
   has the source code. This should be Ignite 2.6.0. Remove from Ignite master.
   - Decouple from the master and move to a 3rd party Github repo. More
   complicated and time-consuming.
   - See if we should move the component to Apache Attic (
   http://attic.apache.org): the Attic is designed for projects to be
   retired but not for the components. Thus, that might be not an option.

Personally, I'm for the first approach. Does it sound reasonable?

-
Denis


On Tue, Jun 18, 2019 at 7:39 AM Maxim Muzafarov <[hidden email]> wrote:

> Folks,
>
> +1 to reduce the number of supported features.
>
> Probably, the best solution will be removing IGFS from core module and
> making it as an Ignite plugin (will require some efforts to do this).
> I've also think we can move IGFS to the separate branch (from the
> master one) if someone will decide merge to latest changes from the
> master branch to build Ignite from scratch with IGFS feature.
>
> On Mon, 17 Jun 2019 at 22:42, Denis Magda <[hidden email]> wrote:
> >
> > >
> > > +1 from me to reduce supported feature list.
> > > Guys, are we talking about Ignite 3.0?
> >
> >
> > Nikolay, I would discontinue IGFS before 3.0. Let's do this in the next
> > release? As for other features and integrations, 3.0 should be considered
> > as a version.
> >
> >
> > > +1 from me provided that we move sources to some supplementary
> repository.
> > > If someone later would like to maintain/fix this module, he/she should
> be
> > > able to access sources with current state of the master.
> >
> >
> > Dmitry, are you suggesting to move the sources to Github and abandon them
> > there? Sort of legacy code cemetery.
> >
> > -
> > Denis
> >
> >
> > On Mon, Jun 17, 2019 at 2:00 AM Nikolay Izhikov <[hidden email]>
> wrote:
> >
> > > +1 from me to reduce supported feature list.
> > >
> > > Guys, are we talking about Ignite 3.0?
> > >
> > >
> > > В Пн, 17/06/2019 в 11:57 +0300, Alexey Goncharuk пишет:
> > > > Denis,
> > > >
> > > > I fully support this idea.
> > > >
> > > > First, looking back, I do not think it was a good design in the first
> > > place
> > > > to build IGFS on top of Ignite caches. Second, I have never seen a
> case
> > > > where IGFS provided significant performance boost. Usually it's
> either
> > > all
> > > > data already fits buffer cache, and IGFS caching is not needed; or
> data
> > > > does not fit buffer cache, and access pattern is close to full scan
> and
> > > > additional caching in IGFS does not make sense.
> > > >
> > > > пн, 17 июн. 2019 г. в 11:28, Павлухин Иван <[hidden email]>:
> > > >
> > > > > Denis,
> > > > >
> > > > > I must say that aforementioned solutions for a Hadoop ecosystem
> appear
> > > > > from time to time in questions on a user mailing list. So, it seems
> > > > > that there is a practical need for such solutions.
> > > > >
> > > > > But of course it does not mean that we should continue a support of
> > > > > IGFS and Hadoop Accelerator. If both are not solutions that fit
> well
> > > > > common use cases then we should discontinue it. If any of them is
> very
> > > > > good for it's purposes but we do not have a capacity to support it
> > > > > without sacrificing main Ignite goals then we still should
> discontinue
> > > > > it in my mind.
> > > > >
> > > > > P.S. Personally I am a fan of UNIX way. I like ideas of a single
> > > > > responsibility and integrations. And I suppose there are other
> Ignite
> > > > > features which could be dropped.
> > > > >
> > > > > ср, 12 июн. 2019 г. в 21:04, Denis Magda <[hidden email]>:
> > > > >
> > > > > >
> > > > > > Igniters,
> > > > > >
> > > > > > I'd like us to move on and finish our conversation on the IGFS
> [1]
> > > and
> > > > > > Hadoop Accelerator [2] support.
> > > > > >
> > > > > > To my knowledge, there is no single committer who maintains the
> > > > > > integrations; they are no longer tested and, even more, the
> community
> > > > > > stopped providing the binaries since Ignite 2.6.0 release (look
> for
> > > > > > In-Memory Hadoop Accelerator table [3]).
> > > > > >
> > > > > > Why all of that happened? Because of a little value, something
> > > succeeds
> > > > > > while something fails. Does it mean that Ignite cannot be used
> for
> > > Hadoop
> > > > > > acceleration, in general? No, quite the opposite, it CAN be used,
> > > but a
> > > > > > solution is different. Have Ignite with native persistence
> deployed
> > > close
> > > > > > to your Hadoop cluster (replace GridGain with Ignite) [4].
> > > > > >
> > > > > > So, I propose we remove IGFS and In-Memory Hadoop Accelerator
> from
> > > our
> > > > > > master repository and rework existing public documentation
> showing
> > > how to
> > > > > > achieve the acceleration with Ignite.
> > > > > >
> > > > > > Any supporters or objections?
> > > > > >
> > > > > >
> > > > > > [1] https://apacheignite-fs.readme.io/docs/in-memory-file-system
> > > > > > [2] https://apacheignite-fs.readme.io/docs/hadoop-accelerator
> > > > > > [3] https://ignite.apache.org/download.cgi#binaries
> > > > > > [4]
> > > > > >
> > > > >
> > > > >
> > >
> https://docs.gridgain.com/docs/bdb-getting-started#section-gridgain-data-lake-accelerator
> > > > > >
> > > > > > -
> > > > > > Denis
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Ivan Pavlukhin
> > > > >
> > >
>

[Dmitry Pavlov]

Hi Denis,

I like the first approach with applying Maxim's idea of creating a branch
named -igfs-Hadoop (not release, but current master state).
2nd) 3rd party repo can be Apache repo just like ignite-release. But it's
true it is time-consuming to move code.
3rd) Attic is for projects, I hope no one here wants to Ignite to be there
:) I'm not sure it is possible to move just one component there. But if it
is possible, we should anyway start from 2nd option and create standalone
repo ignite-igfs-hadoop (and it will become later attic-ignite-igfs-hadoop).

But if someone could stand up and say he/she wants to do migration from one
repo to another (option 2), I like it as well.

Sincerely

вт, 18 июн. 2019 г. в 21:05, Denis Magda <[hidden email]>:

> Igniters,
>
> Thanks a lot for sharing your opinion. As I see, there is a consensus that
> IGFS and Hadoop Accelerator are to be discontinued and no longer supported
> by the community.
>
> As for the source code, if the community prefers moving the source code to
> another repository rather than removing it, then let's do it. I see 3
> solutions here:
>
>    - The simplest - just point out to the latest Ignite release branch that
>    has the source code. This should be Ignite 2.6.0. Remove from Ignite
> master.
>    - Decouple from the master and move to a 3rd party Github repo. More
>    complicated and time-consuming.
>    - See if we should move the component to Apache Attic (
>    http://attic.apache.org): the Attic is designed for projects to be
>    retired but not for the components. Thus, that might be not an option.
>
> Personally, I'm for the first approach. Does it sound reasonable?
>
> -
> Denis
>
>
> On Tue, Jun 18, 2019 at 7:39 AM Maxim Muzafarov <[hidden email]>
> wrote:
>
> > Folks,
> >
> > +1 to reduce the number of supported features.
> >
> > Probably, the best solution will be removing IGFS from core module and
> > making it as an Ignite plugin (will require some efforts to do this).
> > I've also think we can move IGFS to the separate branch (from the
> > master one) if someone will decide merge to latest changes from the
> > master branch to build Ignite from scratch with IGFS feature.
> >
> > On Mon, 17 Jun 2019 at 22:42, Denis Magda <[hidden email]> wrote:
> > >
> > > >
> > > > +1 from me to reduce supported feature list.
> > > > Guys, are we talking about Ignite 3.0?
> > >
> > >
> > > Nikolay, I would discontinue IGFS before 3.0. Let's do this in the next
> > > release? As for other features and integrations, 3.0 should be
> considered
> > > as a version.
> > >
> > >
> > > > +1 from me provided that we move sources to some supplementary
> > repository.
> > > > If someone later would like to maintain/fix this module, he/she
> should
> > be
> > > > able to access sources with current state of the master.
> > >
> > >
> > > Dmitry, are you suggesting to move the sources to Github and abandon
> them
> > > there? Sort of legacy code cemetery.
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Mon, Jun 17, 2019 at 2:00 AM Nikolay Izhikov <[hidden email]>
> > wrote:
> > >
> > > > +1 from me to reduce supported feature list.
> > > >
> > > > Guys, are we talking about Ignite 3.0?
> > > >
> > > >
> > > > В Пн, 17/06/2019 в 11:57 +0300, Alexey Goncharuk пишет:
> > > > > Denis,
> > > > >
> > > > > I fully support this idea.
> > > > >
> > > > > First, looking back, I do not think it was a good design in the
> first
> > > > place
> > > > > to build IGFS on top of Ignite caches. Second, I have never seen a
> > case
> > > > > where IGFS provided significant performance boost. Usually it's
> > either
> > > > all
> > > > > data already fits buffer cache, and IGFS caching is not needed; or
> > data
> > > > > does not fit buffer cache, and access pattern is close to full scan
> > and
> > > > > additional caching in IGFS does not make sense.
> > > > >
> > > > > пн, 17 июн. 2019 г. в 11:28, Павлухин Иван <[hidden email]>:
> > > > >
> > > > > > Denis,
> > > > > >
> > > > > > I must say that aforementioned solutions for a Hadoop ecosystem
> > appear
> > > > > > from time to time in questions on a user mailing list. So, it
> seems
> > > > > > that there is a practical need for such solutions.
> > > > > >
> > > > > > But of course it does not mean that we should continue a support
> of
> > > > > > IGFS and Hadoop Accelerator. If both are not solutions that fit
> > well
> > > > > > common use cases then we should discontinue it. If any of them is
> > very
> > > > > > good for it's purposes but we do not have a capacity to support
> it
> > > > > > without sacrificing main Ignite goals then we still should
> > discontinue
> > > > > > it in my mind.
> > > > > >
> > > > > > P.S. Personally I am a fan of UNIX way. I like ideas of a single
> > > > > > responsibility and integrations. And I suppose there are other
> > Ignite
> > > > > > features which could be dropped.
> > > > > >
> > > > > > ср, 12 июн. 2019 г. в 21:04, Denis Magda <[hidden email]>:
> > > > > >
> > > > > > >
> > > > > > > Igniters,
> > > > > > >
> > > > > > > I'd like us to move on and finish our conversation on the IGFS
> > [1]
> > > > and
> > > > > > > Hadoop Accelerator [2] support.
> > > > > > >
> > > > > > > To my knowledge, there is no single committer who maintains the
> > > > > > > integrations; they are no longer tested and, even more, the
> > community
> > > > > > > stopped providing the binaries since Ignite 2.6.0 release (look
> > for
> > > > > > > In-Memory Hadoop Accelerator table [3]).
> > > > > > >
> > > > > > > Why all of that happened? Because of a little value, something
> > > > succeeds
> > > > > > > while something fails. Does it mean that Ignite cannot be used
> > for
> > > > Hadoop
> > > > > > > acceleration, in general? No, quite the opposite, it CAN be
> used,
> > > > but a
> > > > > > > solution is different. Have Ignite with native persistence
> > deployed
> > > > close
> > > > > > > to your Hadoop cluster (replace GridGain with Ignite) [4].
> > > > > > >
> > > > > > > So, I propose we remove IGFS and In-Memory Hadoop Accelerator
> > from
> > > > our
> > > > > > > master repository and rework existing public documentation
> > showing
> > > > how to
> > > > > > > achieve the acceleration with Ignite.
> > > > > > >
> > > > > > > Any supporters or objections?
> > > > > > >
> > > > > > >
> > > > > > > [1]
> https://apacheignite-fs.readme.io/docs/in-memory-file-system
> > > > > > > [2] https://apacheignite-fs.readme.io/docs/hadoop-accelerator
> > > > > > > [3] https://ignite.apache.org/download.cgi#binaries
> > > > > > > [4]
> > > > > > >
> > > > > >
> > > > > >
> > > >
> >
> https://docs.gridgain.com/docs/bdb-getting-started#section-gridgain-data-lake-accelerator
> > > > > > >
> > > > > > > -
> > > > > > > Denis
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Best regards,
> > > > > > Ivan Pavlukhin
> > > > > >
> > > >
> >
>

[dmagda]

Dmitriy,

I like the first approach with applying Maxim's idea of creating a branch
> named -igfs-Hadoop (not release, but current master state).


+1 for this approach. Any other opinions before we finalize this discussion?

-
Denis


On Tue, Jun 18, 2019 at 11:28 AM Dmitriy Pavlov <[hidden email]> wrote:

> Hi Denis,
>
> I like the first approach with applying Maxim's idea of creating a branch
> named -igfs-Hadoop (not release, but current master state).
> 2nd) 3rd party repo can be Apache repo just like ignite-release. But it's
> true it is time-consuming to move code.
> 3rd) Attic is for projects, I hope no one here wants to Ignite to be there
> :) I'm not sure it is possible to move just one component there. But if it
> is possible, we should anyway start from 2nd option and create standalone
> repo ignite-igfs-hadoop (and it will become later
> attic-ignite-igfs-hadoop).
>
> But if someone could stand up and say he/she wants to do migration from one
> repo to another (option 2), I like it as well.
>
> Sincerely
>
> вт, 18 июн. 2019 г. в 21:05, Denis Magda <[hidden email]>:
>
> > Igniters,
> >
> > Thanks a lot for sharing your opinion. As I see, there is a consensus
> that
> > IGFS and Hadoop Accelerator are to be discontinued and no longer
> supported
> > by the community.
> >
> > As for the source code, if the community prefers moving the source code
> to
> > another repository rather than removing it, then let's do it. I see 3
> > solutions here:
> >
> >    - The simplest - just point out to the latest Ignite release branch
> that
> >    has the source code. This should be Ignite 2.6.0. Remove from Ignite
> > master.
> >    - Decouple from the master and move to a 3rd party Github repo. More
> >    complicated and time-consuming.
> >    - See if we should move the component to Apache Attic (
> >    http://attic.apache.org): the Attic is designed for projects to be
> >    retired but not for the components. Thus, that might be not an option.
> >
> > Personally, I'm for the first approach. Does it sound reasonable?
> >
> > -
> > Denis
> >
> >
> > On Tue, Jun 18, 2019 at 7:39 AM Maxim Muzafarov <[hidden email]>
> > wrote:
> >
> > > Folks,
> > >
> > > +1 to reduce the number of supported features.
> > >
> > > Probably, the best solution will be removing IGFS from core module and
> > > making it as an Ignite plugin (will require some efforts to do this).
> > > I've also think we can move IGFS to the separate branch (from the
> > > master one) if someone will decide merge to latest changes from the
> > > master branch to build Ignite from scratch with IGFS feature.
> > >
> > > On Mon, 17 Jun 2019 at 22:42, Denis Magda <[hidden email]> wrote:
> > > >
> > > > >
> > > > > +1 from me to reduce supported feature list.
> > > > > Guys, are we talking about Ignite 3.0?
> > > >
> > > >
> > > > Nikolay, I would discontinue IGFS before 3.0. Let's do this in the
> next
> > > > release? As for other features and integrations, 3.0 should be
> > considered
> > > > as a version.
> > > >
> > > >
> > > > > +1 from me provided that we move sources to some supplementary
> > > repository.
> > > > > If someone later would like to maintain/fix this module, he/she
> > should
> > > be
> > > > > able to access sources with current state of the master.
> > > >
> > > >
> > > > Dmitry, are you suggesting to move the sources to Github and abandon
> > them
> > > > there? Sort of legacy code cemetery.
> > > >
> > > > -
> > > > Denis
> > > >
> > > >
> > > > On Mon, Jun 17, 2019 at 2:00 AM Nikolay Izhikov <[hidden email]
> >
> > > wrote:
> > > >
> > > > > +1 from me to reduce supported feature list.
> > > > >
> > > > > Guys, are we talking about Ignite 3.0?
> > > > >
> > > > >
> > > > > В Пн, 17/06/2019 в 11:57 +0300, Alexey Goncharuk пишет:
> > > > > > Denis,
> > > > > >
> > > > > > I fully support this idea.
> > > > > >
> > > > > > First, looking back, I do not think it was a good design in the
> > first
> > > > > place
> > > > > > to build IGFS on top of Ignite caches. Second, I have never seen
> a
> > > case
> > > > > > where IGFS provided significant performance boost. Usually it's
> > > either
> > > > > all
> > > > > > data already fits buffer cache, and IGFS caching is not needed;
> or
> > > data
> > > > > > does not fit buffer cache, and access pattern is close to full
> scan
> > > and
> > > > > > additional caching in IGFS does not make sense.
> > > > > >
> > > > > > пн, 17 июн. 2019 г. в 11:28, Павлухин Иван <[hidden email]
> >:
> > > > > >
> > > > > > > Denis,
> > > > > > >
> > > > > > > I must say that aforementioned solutions for a Hadoop ecosystem
> > > appear
> > > > > > > from time to time in questions on a user mailing list. So, it
> > seems
> > > > > > > that there is a practical need for such solutions.
> > > > > > >
> > > > > > > But of course it does not mean that we should continue a
> support
> > of
> > > > > > > IGFS and Hadoop Accelerator. If both are not solutions that fit
> > > well
> > > > > > > common use cases then we should discontinue it. If any of them
> is
> > > very
> > > > > > > good for it's purposes but we do not have a capacity to support
> > it
> > > > > > > without sacrificing main Ignite goals then we still should
> > > discontinue
> > > > > > > it in my mind.
> > > > > > >
> > > > > > > P.S. Personally I am a fan of UNIX way. I like ideas of a
> single
> > > > > > > responsibility and integrations. And I suppose there are other
> > > Ignite
> > > > > > > features which could be dropped.
> > > > > > >
> > > > > > > ср, 12 июн. 2019 г. в 21:04, Denis Magda <[hidden email]>:
> > > > > > >
> > > > > > > >
> > > > > > > > Igniters,
> > > > > > > >
> > > > > > > > I'd like us to move on and finish our conversation on the
> IGFS
> > > [1]
> > > > > and
> > > > > > > > Hadoop Accelerator [2] support.
> > > > > > > >
> > > > > > > > To my knowledge, there is no single committer who maintains
> the
> > > > > > > > integrations; they are no longer tested and, even more, the
> > > community
> > > > > > > > stopped providing the binaries since Ignite 2.6.0 release
> (look
> > > for
> > > > > > > > In-Memory Hadoop Accelerator table [3]).
> > > > > > > >
> > > > > > > > Why all of that happened? Because of a little value,
> something
> > > > > succeeds
> > > > > > > > while something fails. Does it mean that Ignite cannot be
> used
> > > for
> > > > > Hadoop
> > > > > > > > acceleration, in general? No, quite the opposite, it CAN be
> > used,
> > > > > but a
> > > > > > > > solution is different. Have Ignite with native persistence
> > > deployed
> > > > > close
> > > > > > > > to your Hadoop cluster (replace GridGain with Ignite) [4].
> > > > > > > >
> > > > > > > > So, I propose we remove IGFS and In-Memory Hadoop Accelerator
> > > from
> > > > > our
> > > > > > > > master repository and rework existing public documentation
> > > showing
> > > > > how to
> > > > > > > > achieve the acceleration with Ignite.
> > > > > > > >
> > > > > > > > Any supporters or objections?
> > > > > > > >
> > > > > > > >
> > > > > > > > [1]
> > https://apacheignite-fs.readme.io/docs/in-memory-file-system
> > > > > > > > [2]
> https://apacheignite-fs.readme.io/docs/hadoop-accelerator
> > > > > > > > [3] https://ignite.apache.org/download.cgi#binaries
> > > > > > > > [4]
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > >
> > >
> >
> https://docs.gridgain.com/docs/bdb-getting-started#section-gridgain-data-lake-accelerator
> > > > > > > >
> > > > > > > > -
> > > > > > > > Denis
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Best regards,
> > > > > > > Ivan Pavlukhin
> > > > > > >
> > > > >
> > >
> >
>

[Alexey Goncharuk]

Folks,

I think there is a consensus here, but we did not remove IGFS neither in
2.7 nor in 2.8, did we? Should we schedule a corresponding ticket for 2.9?

[Anton Kalashnikov]

Hello.

I created a ticket for this activity - https://issues.apache.org/jira/browse/IGNITE-12647. And if we are still in consensus I'll do it at the nearest time(I've already had the prepared code).

-- 
Best regards,
Anton Kalashnikov


10.02.2020, 12:07, "Alexey Goncharuk" <[hidden email]>:
> Folks,
>
> I think there is a consensus here, but we did not remove IGFS neither in
> 2.7 nor in 2.8, did we? Should we schedule a corresponding ticket for 2.9?

[Anton Kalashnikov]

I found the correct ticket for such activity - https://issues.apache.org/jira/browse/IGNITE-11942

-- 
Best regards,
Anton Kalashnikov


10.02.2020, 12:16, "Anton Kalashnikov" <[hidden email]>:

> Hello.
>
> I created a ticket for this activity - https://issues.apache.org/jira/browse/IGNITE-12647. And if we are still in consensus I'll do it at the nearest time(I've already had the prepared code).
>
> --
> Best regards,
> Anton Kalashnikov
>
> 10.02.2020, 12:07, "Alexey Goncharuk" <[hidden email]>:
>>  Folks,
>>
>>  I think there is a consensus here, but we did not remove IGFS neither in
>>  2.7 nor in 2.8, did we? Should we schedule a corresponding ticket for 2.9?

[Ivan Pavlukhin]

Is not it blocked by
https://issues.apache.org/jira/browse/IGNITE-10292 as stated in JIRA?

@Alex Zinoviev could you please shed some light on this?

Best regards,
Ivan Pavlukhin

пн, 10 февр. 2020 г. в 12:46, Anton Kalashnikov <[hidden email]>:

>
> I found the correct ticket for such activity - https://issues.apache.org/jira/browse/IGNITE-11942
>
> --
> Best regards,
> Anton Kalashnikov
>
>
> 10.02.2020, 12:16, "Anton Kalashnikov" <[hidden email]>:
> > Hello.
> >
> > I created a ticket for this activity - https://issues.apache.org/jira/browse/IGNITE-12647. And if we are still in consensus I'll do it at the nearest time(I've already had the prepared code).
> >
> > --
> > Best regards,
> > Anton Kalashnikov
> >
> > 10.02.2020, 12:07, "Alexey Goncharuk" <[hidden email]>:
> >>  Folks,
> >>
> >>  I think there is a consensus here, but we did not remove IGFS neither in
> >>  2.7 nor in 2.8, did we? Should we schedule a corresponding ticket for 2.9?

[Alexey Zinoviev]

Tensorflow integration uses IGFS, if you have any idea how to store files
in memory by another way, please suggest something.
I hope to decouple Ignite-TF integration to the separate repository before
release 2.9 with its own file system over Ignite Caches

пн, 10 февр. 2020 г. в 12:49, Ivan Pavlukhin <[hidden email]>:

> Is not it blocked by
> https://issues.apache.org/jira/browse/IGNITE-10292 as stated in JIRA?
>
> @Alex Zinoviev could you please shed some light on this?
>
> Best regards,
> Ivan Pavlukhin
>
> пн, 10 февр. 2020 г. в 12:46, Anton Kalashnikov <[hidden email]>:
>
> >
> > I found the correct ticket for such activity -
> https://issues.apache.org/jira/browse/IGNITE-11942
> >
> > --
> > Best regards,
> > Anton Kalashnikov
> >
> >
> > 10.02.2020, 12:16, "Anton Kalashnikov" <[hidden email]>:
> > > Hello.
> > >
> > > I created a ticket for this activity -
> https://issues.apache.org/jira/browse/IGNITE-12647. And if we are still
> in consensus I'll do it at the nearest time(I've already had the prepared
> code).
> > >
> > > --
> > > Best regards,
> > > Anton Kalashnikov
> > >
> > > 10.02.2020, 12:07, "Alexey Goncharuk" <[hidden email]>:
> > >>  Folks,
> > >>
> > >>  I think there is a consensus here, but we did not remove IGFS
> neither in
> > >>  2.7 nor in 2.8, did we? Should we schedule a corresponding ticket
> for 2.9?
>

[Alexey Goncharuk]

Got it, then no need to rush, let's wait for the TF-IGFS decoupling.

пн, 10 февр. 2020 г. в 13:15, Alexey Zinoviev <[hidden email]>:

> Tensorflow integration uses IGFS, if you have any idea how to store files
> in memory by another way, please suggest something.
> I hope to decouple Ignite-TF integration to the separate repository before
> release 2.9 with its own file system over Ignite Caches
>
> пн, 10 февр. 2020 г. в 12:49, Ivan Pavlukhin <[hidden email]>:
>
> > Is not it blocked by
> > https://issues.apache.org/jira/browse/IGNITE-10292 as stated in JIRA?
> >
> > @Alex Zinoviev could you please shed some light on this?
> >
> > Best regards,
> > Ivan Pavlukhin
> >
> > пн, 10 февр. 2020 г. в 12:46, Anton Kalashnikov <[hidden email]>:
> >
> > >
> > > I found the correct ticket for such activity -
> > https://issues.apache.org/jira/browse/IGNITE-11942
> > >
> > > --
> > > Best regards,
> > > Anton Kalashnikov
> > >
> > >
> > > 10.02.2020, 12:16, "Anton Kalashnikov" <[hidden email]>:
> > > > Hello.
> > > >
> > > > I created a ticket for this activity -
> > https://issues.apache.org/jira/browse/IGNITE-12647. And if we are still
> > in consensus I'll do it at the nearest time(I've already had the prepared
> > code).
> > > >
> > > > --
> > > > Best regards,
> > > > Anton Kalashnikov
> > > >
> > > > 10.02.2020, 12:07, "Alexey Goncharuk" <[hidden email]>:
> > > >>  Folks,
> > > >>
> > > >>  I think there is a consensus here, but we did not remove IGFS
> > neither in
> > > >>  2.7 nor in 2.8, did we? Should we schedule a corresponding ticket
> > for 2.9?
> >
>

[Alexey Zinoviev]

Thank you so you much! Will wait:)

пн, 10 февр. 2020 г. в 15:13, Alexey Goncharuk <[hidden email]>:

> Got it, then no need to rush, let's wait for the TF-IGFS decoupling.
>
> пн, 10 февр. 2020 г. в 13:15, Alexey Zinoviev <[hidden email]>:
>
> > Tensorflow integration uses IGFS, if you have any idea how to store files
> > in memory by another way, please suggest something.
> > I hope to decouple Ignite-TF integration to the separate repository
> before
> > release 2.9 with its own file system over Ignite Caches
> >
> > пн, 10 февр. 2020 г. в 12:49, Ivan Pavlukhin <[hidden email]>:
> >
> > > Is not it blocked by
> > > https://issues.apache.org/jira/browse/IGNITE-10292 as stated in JIRA?
> > >
> > > @Alex Zinoviev could you please shed some light on this?
> > >
> > > Best regards,
> > > Ivan Pavlukhin
> > >
> > > пн, 10 февр. 2020 г. в 12:46, Anton Kalashnikov <[hidden email]>:
> > >
> > > >
> > > > I found the correct ticket for such activity -
> > > https://issues.apache.org/jira/browse/IGNITE-11942
> > > >
> > > > --
> > > > Best regards,
> > > > Anton Kalashnikov
> > > >
> > > >
> > > > 10.02.2020, 12:16, "Anton Kalashnikov" <[hidden email]>:
> > > > > Hello.
> > > > >
> > > > > I created a ticket for this activity -
> > > https://issues.apache.org/jira/browse/IGNITE-12647. And if we are
> still
> > > in consensus I'll do it at the nearest time(I've already had the
> prepared
> > > code).
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Anton Kalashnikov
> > > > >
> > > > > 10.02.2020, 12:07, "Alexey Goncharuk" <[hidden email]
> >:
> > > > >>  Folks,
> > > > >>
> > > > >>  I think there is a consensus here, but we did not remove IGFS
> > > neither in
> > > > >>  2.7 nor in 2.8, did we? Should we schedule a corresponding ticket
> > > for 2.9?
> > >
> >
>

[Anton Kalashnikov]

Hi everyone,

The task of removal IGFS and Hadoop accelerator is ready to review.(https://issues.apache.org/jira/browse/IGNITE-11942)
I've already asked some guys to take a look at it but if somebody familiar with this part of code, feel free to take a look at the changes too(especially scripts changes).

I also think it is good to decide which release it should be planned on. This task planned for 2.9 right now but I should notice that first of all there are a lot of changes and secondly there are some changes in public API(removed some methods from configuration). So maybe it makes sense to move this ticket to the next release. What do you think?

-- 
Best regards,
Anton Kalashnikov


10.02.2020, 15:45, "Alexey Zinoviev" <[hidden email]>:

> Thank you so you much! Will wait:)
>
> пн, 10 февр. 2020 г. в 15:13, Alexey Goncharuk <[hidden email]>:
>
>>  Got it, then no need to rush, let's wait for the TF-IGFS decoupling.
>>
>>  пн, 10 февр. 2020 г. в 13:15, Alexey Zinoviev <[hidden email]>:
>>
>>  > Tensorflow integration uses IGFS, if you have any idea how to store files
>>  > in memory by another way, please suggest something.
>>  > I hope to decouple Ignite-TF integration to the separate repository
>>  before
>>  > release 2.9 with its own file system over Ignite Caches
>>  >
>>  > пн, 10 февр. 2020 г. в 12:49, Ivan Pavlukhin <[hidden email]>:
>>  >
>>  > > Is not it blocked by
>>  > > https://issues.apache.org/jira/browse/IGNITE-10292 as stated in JIRA?
>>  > >
>>  > > @Alex Zinoviev could you please shed some light on this?
>>  > >
>>  > > Best regards,
>>  > > Ivan Pavlukhin
>>  > >
>>  > > пн, 10 февр. 2020 г. в 12:46, Anton Kalashnikov <[hidden email]>:
>>  > >
>>  > > >
>>  > > > I found the correct ticket for such activity -
>>  > > https://issues.apache.org/jira/browse/IGNITE-11942
>>  > > >
>>  > > > --
>>  > > > Best regards,
>>  > > > Anton Kalashnikov
>>  > > >
>>  > > >
>>  > > > 10.02.2020, 12:16, "Anton Kalashnikov" <[hidden email]>:
>>  > > > > Hello.
>>  > > > >
>>  > > > > I created a ticket for this activity -
>>  > > https://issues.apache.org/jira/browse/IGNITE-12647. And if we are
>>  still
>>  > > in consensus I'll do it at the nearest time(I've already had the
>>  prepared
>>  > > code).
>>  > > > >
>>  > > > > --
>>  > > > > Best regards,
>>  > > > > Anton Kalashnikov
>>  > > > >
>>  > > > > 10.02.2020, 12:07, "Alexey Goncharuk" <[hidden email]
>>  >:
>>  > > > >> Folks,
>>  > > > >>
>>  > > > >> I think there is a consensus here, but we did not remove IGFS
>>  > > neither in
>>  > > > >> 2.7 nor in 2.8, did we? Should we schedule a corresponding ticket
>>  > > for 2.9?
>>  > >
>>  >

[Pavel Tupitsyn]

We are breaking backwards compatibility,
so this can be only done for Ignite 3.0, am I right?

On Thu, Jul 9, 2020 at 1:46 PM Anton Kalashnikov <[hidden email]> wrote:

> Hi everyone,
>
> The task of removal IGFS and Hadoop accelerator is ready to review.(
> https://issues.apache.org/jira/browse/IGNITE-11942)
> I've already asked some guys to take a look at it but if somebody familiar
> with this part of code, feel free to take a look at the changes
> too(especially scripts changes).
>
> I also think it is good to decide which release it should be planned on.
> This task planned for 2.9 right now but I should notice that first of all
> there are a lot of changes and secondly there are some changes in public
> API(removed some methods from configuration). So maybe it makes sense to
> move this ticket to the next release. What do you think?
>
> --
> Best regards,
> Anton Kalashnikov
>
>
> 10.02.2020, 15:45, "Alexey Zinoviev" <[hidden email]>:
> > Thank you so you much! Will wait:)
> >
> > пн, 10 февр. 2020 г. в 15:13, Alexey Goncharuk <
> [hidden email]>:
> >
> >>  Got it, then no need to rush, let's wait for the TF-IGFS decoupling.
> >>
> >>  пн, 10 февр. 2020 г. в 13:15, Alexey Zinoviev <[hidden email]
> >:
> >>
> >>  > Tensorflow integration uses IGFS, if you have any idea how to store
> files
> >>  > in memory by another way, please suggest something.
> >>  > I hope to decouple Ignite-TF integration to the separate repository
> >>  before
> >>  > release 2.9 with its own file system over Ignite Caches
> >>  >
> >>  > пн, 10 февр. 2020 г. в 12:49, Ivan Pavlukhin <[hidden email]>:
> >>  >
> >>  > > Is not it blocked by
> >>  > > https://issues.apache.org/jira/browse/IGNITE-10292 as stated in
> JIRA?
> >>  > >
> >>  > > @Alex Zinoviev could you please shed some light on this?
> >>  > >
> >>  > > Best regards,
> >>  > > Ivan Pavlukhin
> >>  > >
> >>  > > пн, 10 февр. 2020 г. в 12:46, Anton Kalashnikov <[hidden email]
> >:
> >>  > >
> >>  > > >
> >>  > > > I found the correct ticket for such activity -
> >>  > > https://issues.apache.org/jira/browse/IGNITE-11942
> >>  > > >
> >>  > > > --
> >>  > > > Best regards,
> >>  > > > Anton Kalashnikov
> >>  > > >
> >>  > > >
> >>  > > > 10.02.2020, 12:16, "Anton Kalashnikov" <[hidden email]>:
> >>  > > > > Hello.
> >>  > > > >
> >>  > > > > I created a ticket for this activity -
> >>  > > https://issues.apache.org/jira/browse/IGNITE-12647. And if we are
> >>  still
> >>  > > in consensus I'll do it at the nearest time(I've already had the
> >>  prepared
> >>  > > code).
> >>  > > > >
> >>  > > > > --
> >>  > > > > Best regards,
> >>  > > > > Anton Kalashnikov
> >>  > > > >
> >>  > > > > 10.02.2020, 12:07, "Alexey Goncharuk" <
> [hidden email]
> >>  >:
> >>  > > > >> Folks,
> >>  > > > >>
> >>  > > > >> I think there is a consensus here, but we did not remove IGFS
> >>  > > neither in
> >>  > > > >> 2.7 nor in 2.8, did we? Should we schedule a corresponding
> ticket
> >>  > > for 2.9?
> >>  > >
> >>  >
>