Apache Ignite 2.4 release

[dsetrakyan]

Agree on both, the performance fix and the spark data frames. Let's get
them into the release.

However, Raymond is right. We should know how long the performance fix will
take. If it adds another month to the development, we should include it
into the next release. I am hoping that it can be done faster though.


Alexey Goncharuk, Dmitriy Pavlov, any ideas?

D.

On Wed, Feb 7, 2018 at 9:07 AM, Nikolay Izhikov <[hidden email]> wrote:

> Hello, Igniters.
>
> Please, consider including IGNITE-7337 - Spark Data Frames: support saving
> a data frame in Ignite [1] in the 2.4 release.
> It seems we can merge it into the master in a day or few.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-7337
>
>
> В Ср, 07/02/2018 в 08:35 -0800, Denis Magda пишет:
> > I’m voting for the blocker addition into the release. Sergey K. how will
> it affect your testing cycles? Do you need to re-run everything from
> scratch and how many days you need?
> >
> > —
> > Denis
> >
> > > On Feb 6, 2018, at 11:29 PM, Alexey Goncharuk <
> [hidden email]> wrote:
> > >
> > > Guys,
> > >
> > > Thanks to Dmitriy Pavlov we found the ticket [1] which causes a major
> > > slowdown when page replacement starts. Even though it's not a
> regression, I
> > > suggest we consider it a blocker for 2.4 because this is a huge
> performance
> > > issue which can make it virtually impossible to use native persistence
> when
> > > data size is significantly larger than memory size.
> > >
> > > Any objections?
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-7638
> > >
> > > 2018-01-30 17:10 GMT+03:00 Pavel Tupitsyn <[hidden email]>:
> > >
> > > > Igniters, I will handle 2.4 release if there are no objections.
> > > > Let's take a bit more time for testing and start vote by the end of
> this
> > > > week.
> > > >
> > > > Pavel
> > > >
> > > > On Sat, Jan 27, 2018 at 3:32 AM, Denis Magda <[hidden email]>
> wrote:
> > > >
> > > > > Hi Vyacheslav,
> > > > >
> > > > > According to the previous review notes the impact of the changes
> might be
> > > > > significant, thus, I would recommend us to move the changes to the
> next
> > > > > release.
> > > > >
> > > > > BTW, don’t hesitate to ping reviewers more frequently if there is a
> > > > > pending/abandon review. We are all the people who are tend to
> forget or
> > > > > miss notifications ;)
> > > > >
> > > > > > On Jan 26, 2018, at 2:04 AM, Vyacheslav Daradur <
> [hidden email]>
> > > > >
> > > > > wrote:
> > > > > >
> > > > > > Hi, Vladimir, it's good news. I'm looking forward to new Ignite
> > > >
> > > > release!
> > > > > >
> > > > > > Could you please share a release schedule for 'varint'
> optimizations?
> > > > > >
> > > > > > The task [1] is waiting for review for 5 months.
> > > > > >
> > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-5097
> > > > > >
> > > > > >
> > > > > > On Fri, Jan 26, 2018 at 12:51 PM, Vladimir Ozerov <
> > > >
> > > > [hidden email]>
> > > > > wrote:
> > > > > > > Hi Igniters,
> > > > > > >
> > > > > > > As far as I can see all required tasks and fixes were merged. I
> > > >
> > > > propose
> > > > > to
> > > > > > > take several days of silence to test what we've done and start
> vote at
> > > > >
> > > > > the
> > > > > > > beginning of the next week.
> > > > > > >
> > > > > > > Makes sense?
> > > > > > >
> > > > > > > On Mon, Jan 22, 2018 at 8:39 PM, Denis Magda <
> [hidden email]>
> > > >
> > > > wrote:
> > > > > > >
> > > > > > > > Ok, let’s target Wednesday as a code freeze date.
> > > > > > > >
> > > > > > > > Community members who are involved in 2.4 release please
> merge you
> > > > >
> > > > > fixes
> > > > > > > > and optimizations by that time.
> > > > > > > >
> > > > > > > > —
> > > > > > > > Denis
> > > > > > > >
> > > > > > > > > On Jan 22, 2018, at 8:24 AM, Anton Vinogradov <
> [hidden email]>
> > > >
> > > > wrote:
> > > > > > > > >
> > > > > > > > > Issues
> > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-6711
> > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-6902
> > > > > > > > >
> > > > > > > > > are in master and ignite-2.4
> > > > > > > > >
> > > > > > > > > On Mon, Jan 22, 2018 at 6:43 PM, Denis Magda <
> [hidden email]>
> > > > >
> > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Igniters,
> > > > > > > > > >
> > > > > > > > > > What’s left and what prevents us from passing 2.4
> through the QA
> > > > >
> > > > > phase?
> > > > > > > > > >
> > > > > > > > > > Petr, Anton, Vladimir, Alex G., others please chime in.
> > > > > > > > > >
> > > > > > > > > > —
> > > > > > > > > > Denis
> > > > > > > > > >
> > > > > > > > > > > On Jan 17, 2018, at 7:05 PM, Dmitriy Setrakyan <
> > > > >
> > > > > [hidden email]>
> > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > Great news, indeed! Looking forward to 2.4 release.
> > > > > > > > > > >
> > > > > > > > > > > On Wed, Jan 17, 2018 at 2:55 AM, Vladimir Ozerov <
> > > > >
> > > > > [hidden email]
> > > > > > > > >
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Igniters,
> > > > > > > > > > > >
> > > > > > > > > > > > Great news - two very important tickets - baseline
> topology and
> > > > >
> > > > > Java 8
> > > > > > > > > > > > support - were merged to master. At this point it
> makes sense to
> > > > > > > >
> > > > > > > > create
> > > > > > > > > > a
> > > > > > > > > > > > branch to stabilize the release and finalize
> outstanding
> > > >
> > > > tickets. I
> > > > > > > > > > created
> > > > > > > > > > > > the branch *ignite-2.4*. Please follow the rules
> below when
> > > >
> > > > merging
> > > > > > > > new
> > > > > > > > > > > > commits to master:
> > > > > > > > > > > > 1) If ticket is targeted for 2.4 release, please
> merge to master,
> > > > >
> > > > > then
> > > > > > > > > > > > cherry-pick to ignite-2.4
> > > > > > > > > > > > 2) Otherwise just merge to master.
> > > > > > > > > > > >
> > > > > > > > > > > > Vladimir.
> > > > > > > > > > > >
> > > > > > > > > > > > On Tue, Jan 16, 2018 at 12:32 PM, Anton Vinogradov <
> > > > > > > > > > > > [hidden email]
> > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > Denis,
> > > > > > > > > > > > >
> > > > > > > > > > > > > I'll review https://issues.apache.org/
> jira/browse/IGNITE-6902
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Fri, Jan 12, 2018 at 11:45 PM, Denis Magda <
> > > >
> > > > [hidden email]>
> > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Here is the page create before:
> > > > > > > > > > > > > > https://cwiki.apache.org/
> confluence/display/IGNITE/
> > > > > > > >
> > > > > > > > Apache+Ignite+2.4
> > > > > > > > > > <
> > > > > > > > > > > > > > https://cwiki.apache.org/
> confluence/display/IGNITE/
> > > > > > > >
> > > > > > > > Apache+Ignite+2.4>
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > This is good news. It will be perfect if we also
> release as
> > > >
> > > > many
> > > > > > > > > > > > > "required
> > > > > > > > > > > > > > tickets" as we can:
> > > > > > > > > > > > > > https://cwiki.apache.org/
> confluence/display/IGNITE/
> > > > > > > >
> > > > > > > > Apache+Ignite+2.4
> > > > > > > > > > <
> > > > > > > > > > > > > > https://cwiki.apache.org/
> confluence/display/IGNITE/
> > > > > > > >
> > > > > > > > Apache+Ignite+2.4>
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > My favorite is memory metrics in bytes:
> > > > >
> > > > > https://issues.apache.org/
> > > > > > > > > > > > > > jira/browse/IGNITE-6902 <
> https://issues.apache.org/
> > > > > > > > > > > > >
> > > > > > > > > > > > > jira/browse/IGNITE-6902
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > The discussion lists are flooded with the
> demands for this
> > > > >
> > > > > essential
> > > > > > > > > > > > > > capability. Who is going to review it?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > -
> > > > > > > > > > > > > > Denis
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On Jan 12, 2018, at 11:22 AM, Dmitriy
> Setrakyan <
> > > > > > > > > > > >
> > > > > > > > > > > > [hidden email]
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > +1
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Do we have a list of features for 2.4
> documented anywhere?
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > D.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On Fri, Jan 12, 2018 at 7:43 AM, Pavel
> Tupitsyn <
> > > > > > > > > > > >
> > > > > > > > > > > > [hidden email]>
> > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > +1
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > But some stuff is not yet in master (like
> baseline topology),
> > > > >
> > > > > so
> > > > > > > > it
> > > > > > > > > > > > is
> > > > > > > > > > > > > > too
> > > > > > > > > > > > > > > > soon to create a branch, I think.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Pavel
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > On Fri, Jan 12, 2018 at 5:31 PM, Vladimir
> Ozerov <
> > > > > > > > > > > > >
> > > > > > > > > > > > > [hidden email]>
> > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Igniters,
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > A number of major improvements have been
> made (or being
> > > > >
> > > > > finalized
> > > > > > > > > > > > at
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > moment) to Ignite since recent 2.3
> release. This includes
> > > > > > > >
> > > > > > > > baseline
> > > > > > > > > > > > > > > > > topology, new DDL commands, migration to
> Java 8 and Java 9
> > > > > > > >
> > > > > > > > support,
> > > > > > > > > > > > > > > > > critical performance improvements to WAL,
> etc..
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > I think it makes sense to make a new
> release. What do you
> > > > >
> > > > > think
> > > > > > > > if
> > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > release AI 2.4 with all this great stuff
> by the end of Jan?
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > If there are no objections, I'll create a
> branch to start
> > > > > > > > > > > > >
> > > > > > > > > > > > > stabilization
> > > > > > > > > > > > > > > > > process.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Vladimir.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Best Regards, Vyacheslav D.
> > > > >
> > > > >
> >
> >
>

[Nikolay Izhikov-2]

Hello, Dmitriy.

IGNITE-7337 are merged to master [1]

Do I need to do something more to include this feature into 2.4. release?

[1] https://github.com/apache/ignite/commit/7c01452990ad0de0fb84ab4c0424a6d71e5bccba

В Ср, 07/02/2018 в 11:26 -0800, Dmitriy Setrakyan пишет:

> Agree on both, the performance fix and the spark data frames. Let's get
> them into the release.
>
> However, Raymond is right. We should know how long the performance fix will
> take. If it adds another month to the development, we should include it
> into the next release. I am hoping that it can be done faster though.
>
>
> Alexey Goncharuk, Dmitriy Pavlov, any ideas?
>
> D.
>
> On Wed, Feb 7, 2018 at 9:07 AM, Nikolay Izhikov <[hidden email]> wrote:
>
> > Hello, Igniters.
> >
> > Please, consider including IGNITE-7337 - Spark Data Frames: support saving
> > a data frame in Ignite [1] in the 2.4 release.
> > It seems we can merge it into the master in a day or few.
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-7337
> >
> >
> > В Ср, 07/02/2018 в 08:35 -0800, Denis Magda пишет:
> > > I’m voting for the blocker addition into the release. Sergey K. how will
> >
> > it affect your testing cycles? Do you need to re-run everything from
> > scratch and how many days you need?
> > >
> > > —
> > > Denis
> > >
> > > > On Feb 6, 2018, at 11:29 PM, Alexey Goncharuk <
> >
> > [hidden email]> wrote:
> > > >
> > > > Guys,
> > > >
> > > > Thanks to Dmitriy Pavlov we found the ticket [1] which causes a major
> > > > slowdown when page replacement starts. Even though it's not a
> >
> > regression, I
> > > > suggest we consider it a blocker for 2.4 because this is a huge
> >
> > performance
> > > > issue which can make it virtually impossible to use native persistence
> >
> > when
> > > > data size is significantly larger than memory size.
> > > >
> > > > Any objections?
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IGNITE-7638
> > > >
> > > > 2018-01-30 17:10 GMT+03:00 Pavel Tupitsyn <[hidden email]>:
> > > >
> > > > > Igniters, I will handle 2.4 release if there are no objections.
> > > > > Let's take a bit more time for testing and start vote by the end of
> >
> > this
> > > > > week.
> > > > >
> > > > > Pavel
> > > > >
> > > > > On Sat, Jan 27, 2018 at 3:32 AM, Denis Magda <[hidden email]>
> >
> > wrote:
> > > > >
> > > > > > Hi Vyacheslav,
> > > > > >
> > > > > > According to the previous review notes the impact of the changes
> >
> > might be
> > > > > > significant, thus, I would recommend us to move the changes to the
> >
> > next
> > > > > > release.
> > > > > >
> > > > > > BTW, don’t hesitate to ping reviewers more frequently if there is a
> > > > > > pending/abandon review. We are all the people who are tend to
> >
> > forget or
> > > > > > miss notifications ;)
> > > > > >
> > > > > > > On Jan 26, 2018, at 2:04 AM, Vyacheslav Daradur <
> >
> > [hidden email]>
> > > > > >
> > > > > > wrote:
> > > > > > >
> > > > > > > Hi, Vladimir, it's good news. I'm looking forward to new Ignite
> > > > >
> > > > > release!
> > > > > > >
> > > > > > > Could you please share a release schedule for 'varint'
> >
> > optimizations?
> > > > > > >
> > > > > > > The task [1] is waiting for review for 5 months.
> > > > > > >
> > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-5097
> > > > > > >
> > > > > > >
> > > > > > > On Fri, Jan 26, 2018 at 12:51 PM, Vladimir Ozerov <
> > > > >
> > > > > [hidden email]>
> > > > > > wrote:
> > > > > > > > Hi Igniters,
> > > > > > > >
> > > > > > > > As far as I can see all required tasks and fixes were merged. I
> > > > >
> > > > > propose
> > > > > > to
> > > > > > > > take several days of silence to test what we've done and start
> >
> > vote at
> > > > > >
> > > > > > the
> > > > > > > > beginning of the next week.
> > > > > > > >
> > > > > > > > Makes sense?
> > > > > > > >
> > > > > > > > On Mon, Jan 22, 2018 at 8:39 PM, Denis Magda <
> >
> > [hidden email]>
> > > > >
> > > > > wrote:
> > > > > > > >
> > > > > > > > > Ok, let’s target Wednesday as a code freeze date.
> > > > > > > > >
> > > > > > > > > Community members who are involved in 2.4 release please
> >
> > merge you
> > > > > >
> > > > > > fixes
> > > > > > > > > and optimizations by that time.
> > > > > > > > >
> > > > > > > > > —
> > > > > > > > > Denis
> > > > > > > > >
> > > > > > > > > > On Jan 22, 2018, at 8:24 AM, Anton Vinogradov <
> >
> > [hidden email]>
> > > > >
> > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > Issues
> > > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-6711
> > > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-6902
> > > > > > > > > >
> > > > > > > > > > are in master and ignite-2.4
> > > > > > > > > >
> > > > > > > > > > On Mon, Jan 22, 2018 at 6:43 PM, Denis Magda <
> >
> > [hidden email]>
> > > > > >
> > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Igniters,
> > > > > > > > > > >
> > > > > > > > > > > What’s left and what prevents us from passing 2.4
> >
> > through the QA
> > > > > >
> > > > > > phase?
> > > > > > > > > > >
> > > > > > > > > > > Petr, Anton, Vladimir, Alex G., others please chime in.
> > > > > > > > > > >
> > > > > > > > > > > —
> > > > > > > > > > > Denis
> > > > > > > > > > >
> > > > > > > > > > > > On Jan 17, 2018, at 7:05 PM, Dmitriy Setrakyan <
> > > > > >
> > > > > > [hidden email]>
> > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > Great news, indeed! Looking forward to 2.4 release.
> > > > > > > > > > > >
> > > > > > > > > > > > On Wed, Jan 17, 2018 at 2:55 AM, Vladimir Ozerov <
> > > > > >
> > > > > > [hidden email]
> > > > > > > > > >
> > > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Igniters,
> > > > > > > > > > > > >
> > > > > > > > > > > > > Great news - two very important tickets - baseline
> >
> > topology and
> > > > > >
> > > > > > Java 8
> > > > > > > > > > > > > support - were merged to master. At this point it
> >
> > makes sense to
> > > > > > > > >
> > > > > > > > > create
> > > > > > > > > > > a
> > > > > > > > > > > > > branch to stabilize the release and finalize
> >
> > outstanding
> > > > >
> > > > > tickets. I
> > > > > > > > > > > created
> > > > > > > > > > > > > the branch *ignite-2.4*. Please follow the rules
> >
> > below when
> > > > >
> > > > > merging
> > > > > > > > > new
> > > > > > > > > > > > > commits to master:
> > > > > > > > > > > > > 1) If ticket is targeted for 2.4 release, please
> >
> > merge to master,
> > > > > >
> > > > > > then
> > > > > > > > > > > > > cherry-pick to ignite-2.4
> > > > > > > > > > > > > 2) Otherwise just merge to master.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Vladimir.
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Tue, Jan 16, 2018 at 12:32 PM, Anton Vinogradov <
> > > > > > > > > > > > > [hidden email]
> > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > Denis,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I'll review https://issues.apache.org/
> >
> > jira/browse/IGNITE-6902
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Fri, Jan 12, 2018 at 11:45 PM, Denis Magda <
> > > > >
> > > > > [hidden email]>
> > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Here is the page create before:
> > > > > > > > > > > > > > > https://cwiki.apache.org/
> >
> > confluence/display/IGNITE/
> > > > > > > > >
> > > > > > > > > Apache+Ignite+2.4
> > > > > > > > > > > <
> > > > > > > > > > > > > > > https://cwiki.apache.org/
> >
> > confluence/display/IGNITE/
> > > > > > > > >
> > > > > > > > > Apache+Ignite+2.4>
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > This is good news. It will be perfect if we also
> >
> > release as
> > > > >
> > > > > many
> > > > > > > > > > > > > > "required
> > > > > > > > > > > > > > > tickets" as we can:
> > > > > > > > > > > > > > > https://cwiki.apache.org/
> >
> > confluence/display/IGNITE/
> > > > > > > > >
> > > > > > > > > Apache+Ignite+2.4
> > > > > > > > > > > <
> > > > > > > > > > > > > > > https://cwiki.apache.org/
> >
> > confluence/display/IGNITE/
> > > > > > > > >
> > > > > > > > > Apache+Ignite+2.4>
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > My favorite is memory metrics in bytes:
> > > > > >
> > > > > > https://issues.apache.org/
> > > > > > > > > > > > > > > jira/browse/IGNITE-6902 <
> >
> > https://issues.apache.org/
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > jira/browse/IGNITE-6902
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > The discussion lists are flooded with the
> >
> > demands for this
> > > > > >
> > > > > > essential
> > > > > > > > > > > > > > > capability. Who is going to review it?
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > -
> > > > > > > > > > > > > > > Denis
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > On Jan 12, 2018, at 11:22 AM, Dmitriy
> >
> > Setrakyan <
> > > > > > > > > > > > >
> > > > > > > > > > > > > [hidden email]
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > +1
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Do we have a list of features for 2.4
> >
> > documented anywhere?
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > D.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > On Fri, Jan 12, 2018 at 7:43 AM, Pavel
> >
> > Tupitsyn <
> > > > > > > > > > > > >
> > > > > > > > > > > > > [hidden email]>
> > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > +1
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > But some stuff is not yet in master (like
> >
> > baseline topology),
> > > > > >
> > > > > > so
> > > > > > > > > it
> > > > > > > > > > > > > is
> > > > > > > > > > > > > > > too
> > > > > > > > > > > > > > > > > soon to create a branch, I think.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Pavel
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > On Fri, Jan 12, 2018 at 5:31 PM, Vladimir
> >
> > Ozerov <
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > [hidden email]>
> > > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Igniters,
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > A number of major improvements have been
> >
> > made (or being
> > > > > >
> > > > > > finalized
> > > > > > > > > > > > > at
> > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > moment) to Ignite since recent 2.3
> >
> > release. This includes
> > > > > > > > >
> > > > > > > > > baseline
> > > > > > > > > > > > > > > > > > topology, new DDL commands, migration to
> >
> > Java 8 and Java 9
> > > > > > > > >
> > > > > > > > > support,
> > > > > > > > > > > > > > > > > > critical performance improvements to WAL,
> >
> > etc..
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > I think it makes sense to make a new
> >
> > release. What do you
> > > > > >
> > > > > > think
> > > > > > > > > if
> > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > > release AI 2.4 with all this great stuff
> >
> > by the end of Jan?
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > If there are no objections, I'll create a
> >
> > branch to start
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > stabilization
> > > > > > > > > > > > > > > > > > process.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Vladimir.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Best Regards, Vyacheslav D.
> > > > > >
> > > > > >
> > >
> > >

[Valentin Kulichenko]

Nikolay,

To merge it to 2.4, you need to merge the change to ignite-2.4 release.
Let's do this if we come to an agreement in the neighbor thread.

-Val

On Thu, Feb 8, 2018 at 8:21 PM, Nikolay Izhikov <[hidden email]> wrote:

> Hello, Dmitriy.
>
> IGNITE-7337 are merged to master [1]
>
> Do I need to do something more to include this feature into 2.4. release?
>
> [1] https://github.com/apache/ignite/commit/7c01452990ad0de0fb84ab4c0424a6
> d71e5bccba
>
> В Ср, 07/02/2018 в 11:26 -0800, Dmitriy Setrakyan пишет:
> > Agree on both, the performance fix and the spark data frames. Let's get
> > them into the release.
> >
> > However, Raymond is right. We should know how long the performance fix
> will
> > take. If it adds another month to the development, we should include it
> > into the next release. I am hoping that it can be done faster though.
> >
> >
> > Alexey Goncharuk, Dmitriy Pavlov, any ideas?
> >
> > D.
> >
> > On Wed, Feb 7, 2018 at 9:07 AM, Nikolay Izhikov <[hidden email]>
> wrote:
> >
> > > Hello, Igniters.
> > >
> > > Please, consider including IGNITE-7337 - Spark Data Frames: support
> saving
> > > a data frame in Ignite [1] in the 2.4 release.
> > > It seems we can merge it into the master in a day or few.
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-7337
> > >
> > >
> > > В Ср, 07/02/2018 в 08:35 -0800, Denis Magda пишет:
> > > > I’m voting for the blocker addition into the release. Sergey K. how
> will
> > >
> > > it affect your testing cycles? Do you need to re-run everything from
> > > scratch and how many days you need?
> > > >
> > > > —
> > > > Denis
> > > >
> > > > > On Feb 6, 2018, at 11:29 PM, Alexey Goncharuk <
> > >
> > > [hidden email]> wrote:
> > > > >
> > > > > Guys,
> > > > >
> > > > > Thanks to Dmitriy Pavlov we found the ticket [1] which causes a
> major
> > > > > slowdown when page replacement starts. Even though it's not a
> > >
> > > regression, I
> > > > > suggest we consider it a blocker for 2.4 because this is a huge
> > >
> > > performance
> > > > > issue which can make it virtually impossible to use native
> persistence
> > >
> > > when
> > > > > data size is significantly larger than memory size.
> > > > >
> > > > > Any objections?
> > > > >
> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-7638
> > > > >
> > > > > 2018-01-30 17:10 GMT+03:00 Pavel Tupitsyn <[hidden email]>:
> > > > >
> > > > > > Igniters, I will handle 2.4 release if there are no objections.
> > > > > > Let's take a bit more time for testing and start vote by the end
> of
> > >
> > > this
> > > > > > week.
> > > > > >
> > > > > > Pavel
> > > > > >
> > > > > > On Sat, Jan 27, 2018 at 3:32 AM, Denis Magda <[hidden email]>
> > >
> > > wrote:
> > > > > >
> > > > > > > Hi Vyacheslav,
> > > > > > >
> > > > > > > According to the previous review notes the impact of the
> changes
> > >
> > > might be
> > > > > > > significant, thus, I would recommend us to move the changes to
> the
> > >
> > > next
> > > > > > > release.
> > > > > > >
> > > > > > > BTW, don’t hesitate to ping reviewers more frequently if there
> is a
> > > > > > > pending/abandon review. We are all the people who are tend to
> > >
> > > forget or
> > > > > > > miss notifications ;)
> > > > > > >
> > > > > > > > On Jan 26, 2018, at 2:04 AM, Vyacheslav Daradur <
> > >
> > > [hidden email]>
> > > > > > >
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > Hi, Vladimir, it's good news. I'm looking forward to new
> Ignite
> > > > > >
> > > > > > release!
> > > > > > > >
> > > > > > > > Could you please share a release schedule for 'varint'
> > >
> > > optimizations?
> > > > > > > >
> > > > > > > > The task [1] is waiting for review for 5 months.
> > > > > > > >
> > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-5097
> > > > > > > >
> > > > > > > >
> > > > > > > > On Fri, Jan 26, 2018 at 12:51 PM, Vladimir Ozerov <
> > > > > >
> > > > > > [hidden email]>
> > > > > > > wrote:
> > > > > > > > > Hi Igniters,
> > > > > > > > >
> > > > > > > > > As far as I can see all required tasks and fixes were
> merged. I
> > > > > >
> > > > > > propose
> > > > > > > to
> > > > > > > > > take several days of silence to test what we've done and
> start
> > >
> > > vote at
> > > > > > >
> > > > > > > the
> > > > > > > > > beginning of the next week.
> > > > > > > > >
> > > > > > > > > Makes sense?
> > > > > > > > >
> > > > > > > > > On Mon, Jan 22, 2018 at 8:39 PM, Denis Magda <
> > >
> > > [hidden email]>
> > > > > >
> > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Ok, let’s target Wednesday as a code freeze date.
> > > > > > > > > >
> > > > > > > > > > Community members who are involved in 2.4 release please
> > >
> > > merge you
> > > > > > >
> > > > > > > fixes
> > > > > > > > > > and optimizations by that time.
> > > > > > > > > >
> > > > > > > > > > —
> > > > > > > > > > Denis
> > > > > > > > > >
> > > > > > > > > > > On Jan 22, 2018, at 8:24 AM, Anton Vinogradov <
> > >
> > > [hidden email]>
> > > > > >
> > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > Issues
> > > > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-6711
> > > > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-6902
> > > > > > > > > > >
> > > > > > > > > > > are in master and ignite-2.4
> > > > > > > > > > >
> > > > > > > > > > > On Mon, Jan 22, 2018 at 6:43 PM, Denis Magda <
> > >
> > > [hidden email]>
> > > > > > >
> > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Igniters,
> > > > > > > > > > > >
> > > > > > > > > > > > What’s left and what prevents us from passing 2.4
> > >
> > > through the QA
> > > > > > >
> > > > > > > phase?
> > > > > > > > > > > >
> > > > > > > > > > > > Petr, Anton, Vladimir, Alex G., others please chime
> in.
> > > > > > > > > > > >
> > > > > > > > > > > > —
> > > > > > > > > > > > Denis
> > > > > > > > > > > >
> > > > > > > > > > > > > On Jan 17, 2018, at 7:05 PM, Dmitriy Setrakyan <
> > > > > > >
> > > > > > > [hidden email]>
> > > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > Great news, indeed! Looking forward to 2.4 release.
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Wed, Jan 17, 2018 at 2:55 AM, Vladimir Ozerov <
> > > > > > >
> > > > > > > [hidden email]
> > > > > > > > > > >
> > > > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Igniters,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Great news - two very important tickets -
> baseline
> > >
> > > topology and
> > > > > > >
> > > > > > > Java 8
> > > > > > > > > > > > > > support - were merged to master. At this point it
> > >
> > > makes sense to
> > > > > > > > > >
> > > > > > > > > > create
> > > > > > > > > > > > a
> > > > > > > > > > > > > > branch to stabilize the release and finalize
> > >
> > > outstanding
> > > > > >
> > > > > > tickets. I
> > > > > > > > > > > > created
> > > > > > > > > > > > > > the branch *ignite-2.4*. Please follow the rules
> > >
> > > below when
> > > > > >
> > > > > > merging
> > > > > > > > > > new
> > > > > > > > > > > > > > commits to master:
> > > > > > > > > > > > > > 1) If ticket is targeted for 2.4 release, please
> > >
> > > merge to master,
> > > > > > >
> > > > > > > then
> > > > > > > > > > > > > > cherry-pick to ignite-2.4
> > > > > > > > > > > > > > 2) Otherwise just merge to master.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Vladimir.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Tue, Jan 16, 2018 at 12:32 PM, Anton
> Vinogradov <
> > > > > > > > > > > > > > [hidden email]
> > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > Denis,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > I'll review https://issues.apache.org/
> > >
> > > jira/browse/IGNITE-6902
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On Fri, Jan 12, 2018 at 11:45 PM, Denis Magda <
> > > > > >
> > > > > > [hidden email]>
> > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Here is the page create before:
> > > > > > > > > > > > > > > > https://cwiki.apache.org/
> > >
> > > confluence/display/IGNITE/
> > > > > > > > > >
> > > > > > > > > > Apache+Ignite+2.4
> > > > > > > > > > > > <
> > > > > > > > > > > > > > > > https://cwiki.apache.org/
> > >
> > > confluence/display/IGNITE/
> > > > > > > > > >
> > > > > > > > > > Apache+Ignite+2.4>
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > This is good news. It will be perfect if we
> also
> > >
> > > release as
> > > > > >
> > > > > > many
> > > > > > > > > > > > > > > "required
> > > > > > > > > > > > > > > > tickets" as we can:
> > > > > > > > > > > > > > > > https://cwiki.apache.org/
> > >
> > > confluence/display/IGNITE/
> > > > > > > > > >
> > > > > > > > > > Apache+Ignite+2.4
> > > > > > > > > > > > <
> > > > > > > > > > > > > > > > https://cwiki.apache.org/
> > >
> > > confluence/display/IGNITE/
> > > > > > > > > >
> > > > > > > > > > Apache+Ignite+2.4>
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > My favorite is memory metrics in bytes:
> > > > > > >
> > > > > > > https://issues.apache.org/
> > > > > > > > > > > > > > > > jira/browse/IGNITE-6902 <
> > >
> > > https://issues.apache.org/
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > jira/browse/IGNITE-6902
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > The discussion lists are flooded with the
> > >
> > > demands for this
> > > > > > >
> > > > > > > essential
> > > > > > > > > > > > > > > > capability. Who is going to review it?
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > -
> > > > > > > > > > > > > > > > Denis
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > On Jan 12, 2018, at 11:22 AM, Dmitriy
> > >
> > > Setrakyan <
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > [hidden email]
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > +1
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Do we have a list of features for 2.4
> > >
> > > documented anywhere?
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > D.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > On Fri, Jan 12, 2018 at 7:43 AM, Pavel
> > >
> > > Tupitsyn <
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > [hidden email]>
> > > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > +1
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > But some stuff is not yet in master (like
> > >
> > > baseline topology),
> > > > > > >
> > > > > > > so
> > > > > > > > > > it
> > > > > > > > > > > > > > is
> > > > > > > > > > > > > > > > too
> > > > > > > > > > > > > > > > > > soon to create a branch, I think.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Pavel
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > On Fri, Jan 12, 2018 at 5:31 PM, Vladimir
> > >
> > > Ozerov <
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > [hidden email]>
> > > > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Igniters,
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > A number of major improvements have
> been
> > >
> > > made (or being
> > > > > > >
> > > > > > > finalized
> > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > moment) to Ignite since recent 2.3
> > >
> > > release. This includes
> > > > > > > > > >
> > > > > > > > > > baseline
> > > > > > > > > > > > > > > > > > > topology, new DDL commands, migration
> to
> > >
> > > Java 8 and Java 9
> > > > > > > > > >
> > > > > > > > > > support,
> > > > > > > > > > > > > > > > > > > critical performance improvements to
> WAL,
> > >
> > > etc..
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > I think it makes sense to make a new
> > >
> > > release. What do you
> > > > > > >
> > > > > > > think
> > > > > > > > > > if
> > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > > > release AI 2.4 with all this great
> stuff
> > >
> > > by the end of Jan?
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > If there are no objections, I'll
> create a
> > >
> > > branch to start
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > stabilization
> > > > > > > > > > > > > > > > > > > process.
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Vladimir.
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Best Regards, Vyacheslav D.
> > > > > > >
> > > > > > >
> > > >
> > > >
>

[Valentin Kulichenko]

IGNITE-7337 (Data frame save functionality) is merged into 2.4.

-Val

On Fri, Feb 9, 2018 at 11:47 AM, Valentin Kulichenko <
[hidden email]> wrote:

> Nikolay,
>
> To merge it to 2.4, you need to merge the change to ignite-2.4 release.
> Let's do this if we come to an agreement in the neighbor thread.
>
> -Val
>
> On Thu, Feb 8, 2018 at 8:21 PM, Nikolay Izhikov <[hidden email]>
> wrote:
>
>> Hello, Dmitriy.
>>
>> IGNITE-7337 are merged to master [1]
>>
>> Do I need to do something more to include this feature into 2.4. release?
>>
>> [1] https://github.com/apache/ignite/commit/7c01452990ad0de0fb84
>> ab4c0424a6d71e5bccba
>>
>> В Ср, 07/02/2018 в 11:26 -0800, Dmitriy Setrakyan пишет:
>> > Agree on both, the performance fix and the spark data frames. Let's get
>> > them into the release.
>> >
>> > However, Raymond is right. We should know how long the performance fix
>> will
>> > take. If it adds another month to the development, we should include it
>> > into the next release. I am hoping that it can be done faster though.
>> >
>> >
>> > Alexey Goncharuk, Dmitriy Pavlov, any ideas?
>> >
>> > D.
>> >
>> > On Wed, Feb 7, 2018 at 9:07 AM, Nikolay Izhikov <[hidden email]>
>> wrote:
>> >
>> > > Hello, Igniters.
>> > >
>> > > Please, consider including IGNITE-7337 - Spark Data Frames: support
>> saving
>> > > a data frame in Ignite [1] in the 2.4 release.
>> > > It seems we can merge it into the master in a day or few.
>> > >
>> > > [1] https://issues.apache.org/jira/browse/IGNITE-7337
>> > >
>> > >
>> > > В Ср, 07/02/2018 в 08:35 -0800, Denis Magda пишет:
>> > > > I’m voting for the blocker addition into the release. Sergey K. how
>> will
>> > >
>> > > it affect your testing cycles? Do you need to re-run everything from
>> > > scratch and how many days you need?
>> > > >
>> > > > —
>> > > > Denis
>> > > >
>> > > > > On Feb 6, 2018, at 11:29 PM, Alexey Goncharuk <
>> > >
>> > > [hidden email]> wrote:
>> > > > >
>> > > > > Guys,
>> > > > >
>> > > > > Thanks to Dmitriy Pavlov we found the ticket [1] which causes a
>> major
>> > > > > slowdown when page replacement starts. Even though it's not a
>> > >
>> > > regression, I
>> > > > > suggest we consider it a blocker for 2.4 because this is a huge
>> > >
>> > > performance
>> > > > > issue which can make it virtually impossible to use native
>> persistence
>> > >
>> > > when
>> > > > > data size is significantly larger than memory size.
>> > > > >
>> > > > > Any objections?
>> > > > >
>> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-7638
>> > > > >
>> > > > > 2018-01-30 17:10 GMT+03:00 Pavel Tupitsyn <[hidden email]>:
>> > > > >
>> > > > > > Igniters, I will handle 2.4 release if there are no objections.
>> > > > > > Let's take a bit more time for testing and start vote by the
>> end of
>> > >
>> > > this
>> > > > > > week.
>> > > > > >
>> > > > > > Pavel
>> > > > > >
>> > > > > > On Sat, Jan 27, 2018 at 3:32 AM, Denis Magda <[hidden email]
>> >
>> > >
>> > > wrote:
>> > > > > >
>> > > > > > > Hi Vyacheslav,
>> > > > > > >
>> > > > > > > According to the previous review notes the impact of the
>> changes
>> > >
>> > > might be
>> > > > > > > significant, thus, I would recommend us to move the changes
>> to the
>> > >
>> > > next
>> > > > > > > release.
>> > > > > > >
>> > > > > > > BTW, don’t hesitate to ping reviewers more frequently if
>> there is a
>> > > > > > > pending/abandon review. We are all the people who are tend to
>> > >
>> > > forget or
>> > > > > > > miss notifications ;)
>> > > > > > >
>> > > > > > > > On Jan 26, 2018, at 2:04 AM, Vyacheslav Daradur <
>> > >
>> > > [hidden email]>
>> > > > > > >
>> > > > > > > wrote:
>> > > > > > > >
>> > > > > > > > Hi, Vladimir, it's good news. I'm looking forward to new
>> Ignite
>> > > > > >
>> > > > > > release!
>> > > > > > > >
>> > > > > > > > Could you please share a release schedule for 'varint'
>> > >
>> > > optimizations?
>> > > > > > > >
>> > > > > > > > The task [1] is waiting for review for 5 months.
>> > > > > > > >
>> > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-5097
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > On Fri, Jan 26, 2018 at 12:51 PM, Vladimir Ozerov <
>> > > > > >
>> > > > > > [hidden email]>
>> > > > > > > wrote:
>> > > > > > > > > Hi Igniters,
>> > > > > > > > >
>> > > > > > > > > As far as I can see all required tasks and fixes were
>> merged. I
>> > > > > >
>> > > > > > propose
>> > > > > > > to
>> > > > > > > > > take several days of silence to test what we've done and
>> start
>> > >
>> > > vote at
>> > > > > > >
>> > > > > > > the
>> > > > > > > > > beginning of the next week.
>> > > > > > > > >
>> > > > > > > > > Makes sense?
>> > > > > > > > >
>> > > > > > > > > On Mon, Jan 22, 2018 at 8:39 PM, Denis Magda <
>> > >
>> > > [hidden email]>
>> > > > > >
>> > > > > > wrote:
>> > > > > > > > >
>> > > > > > > > > > Ok, let’s target Wednesday as a code freeze date.
>> > > > > > > > > >
>> > > > > > > > > > Community members who are involved in 2.4 release please
>> > >
>> > > merge you
>> > > > > > >
>> > > > > > > fixes
>> > > > > > > > > > and optimizations by that time.
>> > > > > > > > > >
>> > > > > > > > > > —
>> > > > > > > > > > Denis
>> > > > > > > > > >
>> > > > > > > > > > > On Jan 22, 2018, at 8:24 AM, Anton Vinogradov <
>> > >
>> > > [hidden email]>
>> > > > > >
>> > > > > > wrote:
>> > > > > > > > > > >
>> > > > > > > > > > > Issues
>> > > > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-6711
>> > > > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-6902
>> > > > > > > > > > >
>> > > > > > > > > > > are in master and ignite-2.4
>> > > > > > > > > > >
>> > > > > > > > > > > On Mon, Jan 22, 2018 at 6:43 PM, Denis Magda <
>> > >
>> > > [hidden email]>
>> > > > > > >
>> > > > > > > wrote:
>> > > > > > > > > > >
>> > > > > > > > > > > > Igniters,
>> > > > > > > > > > > >
>> > > > > > > > > > > > What’s left and what prevents us from passing 2.4
>> > >
>> > > through the QA
>> > > > > > >
>> > > > > > > phase?
>> > > > > > > > > > > >
>> > > > > > > > > > > > Petr, Anton, Vladimir, Alex G., others please chime
>> in.
>> > > > > > > > > > > >
>> > > > > > > > > > > > —
>> > > > > > > > > > > > Denis
>> > > > > > > > > > > >
>> > > > > > > > > > > > > On Jan 17, 2018, at 7:05 PM, Dmitriy Setrakyan <
>> > > > > > >
>> > > > > > > [hidden email]>
>> > > > > > > > > > > > wrote:
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > Great news, indeed! Looking forward to 2.4
>> release.
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > On Wed, Jan 17, 2018 at 2:55 AM, Vladimir Ozerov <
>> > > > > > >
>> > > > > > > [hidden email]
>> > > > > > > > > > >
>> > > > > > > > > > > > > wrote:
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > > Igniters,
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > Great news - two very important tickets -
>> baseline
>> > >
>> > > topology and
>> > > > > > >
>> > > > > > > Java 8
>> > > > > > > > > > > > > > support - were merged to master. At this point
>> it
>> > >
>> > > makes sense to
>> > > > > > > > > >
>> > > > > > > > > > create
>> > > > > > > > > > > > a
>> > > > > > > > > > > > > > branch to stabilize the release and finalize
>> > >
>> > > outstanding
>> > > > > >
>> > > > > > tickets. I
>> > > > > > > > > > > > created
>> > > > > > > > > > > > > > the branch *ignite-2.4*. Please follow the rules
>> > >
>> > > below when
>> > > > > >
>> > > > > > merging
>> > > > > > > > > > new
>> > > > > > > > > > > > > > commits to master:
>> > > > > > > > > > > > > > 1) If ticket is targeted for 2.4 release, please
>> > >
>> > > merge to master,
>> > > > > > >
>> > > > > > > then
>> > > > > > > > > > > > > > cherry-pick to ignite-2.4
>> > > > > > > > > > > > > > 2) Otherwise just merge to master.
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > Vladimir.
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > On Tue, Jan 16, 2018 at 12:32 PM, Anton
>> Vinogradov <
>> > > > > > > > > > > > > > [hidden email]
>> > > > > > > > > > > > > > > wrote:
>> > > > > > > > > > > > > > > Denis,
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > I'll review https://issues.apache.org/
>> > >
>> > > jira/browse/IGNITE-6902
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > On Fri, Jan 12, 2018 at 11:45 PM, Denis Magda
>> <
>> > > > > >
>> > > > > > [hidden email]>
>> > > > > > > > > > > > wrote:
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > Here is the page create before:
>> > > > > > > > > > > > > > > > https://cwiki.apache.org/
>> > >
>> > > confluence/display/IGNITE/
>> > > > > > > > > >
>> > > > > > > > > > Apache+Ignite+2.4
>> > > > > > > > > > > > <
>> > > > > > > > > > > > > > > > https://cwiki.apache.org/
>> > >
>> > > confluence/display/IGNITE/
>> > > > > > > > > >
>> > > > > > > > > > Apache+Ignite+2.4>
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > This is good news. It will be perfect if we
>> also
>> > >
>> > > release as
>> > > > > >
>> > > > > > many
>> > > > > > > > > > > > > > > "required
>> > > > > > > > > > > > > > > > tickets" as we can:
>> > > > > > > > > > > > > > > > https://cwiki.apache.org/
>> > >
>> > > confluence/display/IGNITE/
>> > > > > > > > > >
>> > > > > > > > > > Apache+Ignite+2.4
>> > > > > > > > > > > > <
>> > > > > > > > > > > > > > > > https://cwiki.apache.org/
>> > >
>> > > confluence/display/IGNITE/
>> > > > > > > > > >
>> > > > > > > > > > Apache+Ignite+2.4>
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > My favorite is memory metrics in bytes:
>> > > > > > >
>> > > > > > > https://issues.apache.org/
>> > > > > > > > > > > > > > > > jira/browse/IGNITE-6902 <
>> > >
>> > > https://issues.apache.org/
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > jira/browse/IGNITE-6902
>> > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > The discussion lists are flooded with the
>> > >
>> > > demands for this
>> > > > > > >
>> > > > > > > essential
>> > > > > > > > > > > > > > > > capability. Who is going to review it?
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > -
>> > > > > > > > > > > > > > > > Denis
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > On Jan 12, 2018, at 11:22 AM, Dmitriy
>> > >
>> > > Setrakyan <
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > [hidden email]
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > wrote:
>> > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > +1
>> > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > Do we have a list of features for 2.4
>> > >
>> > > documented anywhere?
>> > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > D.
>> > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > On Fri, Jan 12, 2018 at 7:43 AM, Pavel
>> > >
>> > > Tupitsyn <
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > [hidden email]>
>> > > > > > > > > > > > > > > > > wrote:
>> > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > +1
>> > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > But some stuff is not yet in master
>> (like
>> > >
>> > > baseline topology),
>> > > > > > >
>> > > > > > > so
>> > > > > > > > > > it
>> > > > > > > > > > > > > > is
>> > > > > > > > > > > > > > > > too
>> > > > > > > > > > > > > > > > > > soon to create a branch, I think.
>> > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > Pavel
>> > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > On Fri, Jan 12, 2018 at 5:31 PM,
>> Vladimir
>> > >
>> > > Ozerov <
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > [hidden email]>
>> > > > > > > > > > > > > > > > > > wrote:
>> > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > Igniters,
>> > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > A number of major improvements have
>> been
>> > >
>> > > made (or being
>> > > > > > >
>> > > > > > > finalized
>> > > > > > > > > > > > > > at
>> > > > > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > > > > > moment) to Ignite since recent 2.3
>> > >
>> > > release. This includes
>> > > > > > > > > >
>> > > > > > > > > > baseline
>> > > > > > > > > > > > > > > > > > > topology, new DDL commands, migration
>> to
>> > >
>> > > Java 8 and Java 9
>> > > > > > > > > >
>> > > > > > > > > > support,
>> > > > > > > > > > > > > > > > > > > critical performance improvements to
>> WAL,
>> > >
>> > > etc..
>> > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > I think it makes sense to make a new
>> > >
>> > > release. What do you
>> > > > > > >
>> > > > > > > think
>> > > > > > > > > > if
>> > > > > > > > > > > > > > we
>> > > > > > > > > > > > > > > > > > > release AI 2.4 with all this great
>> stuff
>> > >
>> > > by the end of Jan?
>> > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > If there are no objections, I'll
>> create a
>> > >
>> > > branch to start
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > stabilization
>> > > > > > > > > > > > > > > > > > > process.
>> > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > Vladimir.
>> > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > --
>> > > > > > > > Best Regards, Vyacheslav D.
>> > > > > > >
>> > > > > > >
>> > > >
>> > > >
>>
>
>

[dsetrakyan]

On Sun, Feb 11, 2018 at 2:23 PM, Valentin Kulichenko <
[hidden email]> wrote:

> IGNITE-7337 (Data frame save functionality) is merged into 2.4.
>

Thanks Val, this is great news! Also, thanks for Nikolai Izhikov who
implemented most of the code.

Has this been documented already. Where can we point the community to read
about this new functionality. What is possible now that was not possible
before?

[Nikolay Izhikov-2]

Hello, Dmitriy.


Data Frame Read feature is documented in https://issues.apache.org/jira/browse/IGNITE-7345
AFAIK page isn't published yet.

I will document Data Frame Save feature in a few days.
There is a ticket for it - https://issues.apache.org/jira/browse/IGNITE-7655


В Вс, 11/02/2018 в 15:09 -0800, Dmitriy Setrakyan пишет:

> On Sun, Feb 11, 2018 at 2:23 PM, Valentin Kulichenko <
> [hidden email]> wrote:
>
> > IGNITE-7337 (Data frame save functionality) is merged into 2.4.
> >
>
> Thanks Val, this is great news! Also, thanks for Nikolai Izhikov who
> implemented most of the code.
>
> Has this been documented already. Where can we point the community to read
> about this new functionality. What is possible now that was not possible
> before?

[Dmitriy Pavlov]

Hi,

Unfortunately, a quick fix did not give us too much performance boost.

I'm going to implement a complete algorithm change for storing the page
identifier. But this change is quite significant and will require
re-testing. I suggest including
https://issues.apache.org/jira/browse/IGNITE-7638 in the next version, for
example, to 2.5.

Sincerely,
Dmitriy Pavlov


ср, 7 февр. 2018 г. в 22:27, Dmitriy Setrakyan <[hidden email]>:

> Agree on both, the performance fix and the spark data frames. Let's get
> them into the release.
>
> However, Raymond is right. We should know how long the performance fix will
> take. If it adds another month to the development, we should include it
> into the next release. I am hoping that it can be done faster though.
>
>
> Alexey Goncharuk, Dmitriy Pavlov, any ideas?
>
> D.
>
> On Wed, Feb 7, 2018 at 9:07 AM, Nikolay Izhikov <[hidden email]>
> wrote:
>
> > Hello, Igniters.
> >
> > Please, consider including IGNITE-7337 - Spark Data Frames: support
> saving
> > a data frame in Ignite [1] in the 2.4 release.
> > It seems we can merge it into the master in a day or few.
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-7337
> >
> >
> > В Ср, 07/02/2018 в 08:35 -0800, Denis Magda пишет:
> > > I’m voting for the blocker addition into the release. Sergey K. how
> will
> > it affect your testing cycles? Do you need to re-run everything from
> > scratch and how many days you need?
> > >
> > > —
> > > Denis
> > >
> > > > On Feb 6, 2018, at 11:29 PM, Alexey Goncharuk <
> > [hidden email]> wrote:
> > > >
> > > > Guys,
> > > >
> > > > Thanks to Dmitriy Pavlov we found the ticket [1] which causes a major
> > > > slowdown when page replacement starts. Even though it's not a
> > regression, I
> > > > suggest we consider it a blocker for 2.4 because this is a huge
> > performance
> > > > issue which can make it virtually impossible to use native
> persistence
> > when
> > > > data size is significantly larger than memory size.
> > > >
> > > > Any objections?
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IGNITE-7638
> > > >
> > > > 2018-01-30 17:10 GMT+03:00 Pavel Tupitsyn <[hidden email]>:
> > > >
> > > > > Igniters, I will handle 2.4 release if there are no objections.
> > > > > Let's take a bit more time for testing and start vote by the end of
> > this
> > > > > week.
> > > > >
> > > > > Pavel
> > > > >
> > > > > On Sat, Jan 27, 2018 at 3:32 AM, Denis Magda <[hidden email]>
> > wrote:
> > > > >
> > > > > > Hi Vyacheslav,
> > > > > >
> > > > > > According to the previous review notes the impact of the changes
> > might be
> > > > > > significant, thus, I would recommend us to move the changes to
> the
> > next
> > > > > > release.
> > > > > >
> > > > > > BTW, don’t hesitate to ping reviewers more frequently if there
> is a
> > > > > > pending/abandon review. We are all the people who are tend to
> > forget or
> > > > > > miss notifications ;)
> > > > > >
> > > > > > > On Jan 26, 2018, at 2:04 AM, Vyacheslav Daradur <
> > [hidden email]>
> > > > > >
> > > > > > wrote:
> > > > > > >
> > > > > > > Hi, Vladimir, it's good news. I'm looking forward to new Ignite
> > > > >
> > > > > release!
> > > > > > >
> > > > > > > Could you please share a release schedule for 'varint'
> > optimizations?
> > > > > > >
> > > > > > > The task [1] is waiting for review for 5 months.
> > > > > > >
> > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-5097
> > > > > > >
> > > > > > >
> > > > > > > On Fri, Jan 26, 2018 at 12:51 PM, Vladimir Ozerov <
> > > > >
> > > > > [hidden email]>
> > > > > > wrote:
> > > > > > > > Hi Igniters,
> > > > > > > >
> > > > > > > > As far as I can see all required tasks and fixes were
> merged. I
> > > > >
> > > > > propose
> > > > > > to
> > > > > > > > take several days of silence to test what we've done and
> start
> > vote at
> > > > > >
> > > > > > the
> > > > > > > > beginning of the next week.
> > > > > > > >
> > > > > > > > Makes sense?
> > > > > > > >
> > > > > > > > On Mon, Jan 22, 2018 at 8:39 PM, Denis Magda <
> > [hidden email]>
> > > > >
> > > > > wrote:
> > > > > > > >
> > > > > > > > > Ok, let’s target Wednesday as a code freeze date.
> > > > > > > > >
> > > > > > > > > Community members who are involved in 2.4 release please
> > merge you
> > > > > >
> > > > > > fixes
> > > > > > > > > and optimizations by that time.
> > > > > > > > >
> > > > > > > > > —
> > > > > > > > > Denis
> > > > > > > > >
> > > > > > > > > > On Jan 22, 2018, at 8:24 AM, Anton Vinogradov <
> > [hidden email]>
> > > > >
> > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > Issues
> > > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-6711
> > > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-6902
> > > > > > > > > >
> > > > > > > > > > are in master and ignite-2.4
> > > > > > > > > >
> > > > > > > > > > On Mon, Jan 22, 2018 at 6:43 PM, Denis Magda <
> > [hidden email]>
> > > > > >
> > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Igniters,
> > > > > > > > > > >
> > > > > > > > > > > What’s left and what prevents us from passing 2.4
> > through the QA
> > > > > >
> > > > > > phase?
> > > > > > > > > > >
> > > > > > > > > > > Petr, Anton, Vladimir, Alex G., others please chime in.
> > > > > > > > > > >
> > > > > > > > > > > —
> > > > > > > > > > > Denis
> > > > > > > > > > >
> > > > > > > > > > > > On Jan 17, 2018, at 7:05 PM, Dmitriy Setrakyan <
> > > > > >
> > > > > > [hidden email]>
> > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > Great news, indeed! Looking forward to 2.4 release.
> > > > > > > > > > > >
> > > > > > > > > > > > On Wed, Jan 17, 2018 at 2:55 AM, Vladimir Ozerov <
> > > > > >
> > > > > > [hidden email]
> > > > > > > > > >
> > > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Igniters,
> > > > > > > > > > > > >
> > > > > > > > > > > > > Great news - two very important tickets - baseline
> > topology and
> > > > > >
> > > > > > Java 8
> > > > > > > > > > > > > support - were merged to master. At this point it
> > makes sense to
> > > > > > > > >
> > > > > > > > > create
> > > > > > > > > > > a
> > > > > > > > > > > > > branch to stabilize the release and finalize
> > outstanding
> > > > >
> > > > > tickets. I
> > > > > > > > > > > created
> > > > > > > > > > > > > the branch *ignite-2.4*. Please follow the rules
> > below when
> > > > >
> > > > > merging
> > > > > > > > > new
> > > > > > > > > > > > > commits to master:
> > > > > > > > > > > > > 1) If ticket is targeted for 2.4 release, please
> > merge to master,
> > > > > >
> > > > > > then
> > > > > > > > > > > > > cherry-pick to ignite-2.4
> > > > > > > > > > > > > 2) Otherwise just merge to master.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Vladimir.
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Tue, Jan 16, 2018 at 12:32 PM, Anton Vinogradov
> <
> > > > > > > > > > > > > [hidden email]
> > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > Denis,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I'll review https://issues.apache.org/
> > jira/browse/IGNITE-6902
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Fri, Jan 12, 2018 at 11:45 PM, Denis Magda <
> > > > >
> > > > > [hidden email]>
> > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Here is the page create before:
> > > > > > > > > > > > > > > https://cwiki.apache.org/
> > confluence/display/IGNITE/
> > > > > > > > >
> > > > > > > > > Apache+Ignite+2.4
> > > > > > > > > > > <
> > > > > > > > > > > > > > > https://cwiki.apache.org/
> > confluence/display/IGNITE/
> > > > > > > > >
> > > > > > > > > Apache+Ignite+2.4>
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > This is good news. It will be perfect if we
> also
> > release as
> > > > >
> > > > > many
> > > > > > > > > > > > > > "required
> > > > > > > > > > > > > > > tickets" as we can:
> > > > > > > > > > > > > > > https://cwiki.apache.org/
> > confluence/display/IGNITE/
> > > > > > > > >
> > > > > > > > > Apache+Ignite+2.4
> > > > > > > > > > > <
> > > > > > > > > > > > > > > https://cwiki.apache.org/
> > confluence/display/IGNITE/
> > > > > > > > >
> > > > > > > > > Apache+Ignite+2.4>
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > My favorite is memory metrics in bytes:
> > > > > >
> > > > > > https://issues.apache.org/
> > > > > > > > > > > > > > > jira/browse/IGNITE-6902 <
> > https://issues.apache.org/
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > jira/browse/IGNITE-6902
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > The discussion lists are flooded with the
> > demands for this
> > > > > >
> > > > > > essential
> > > > > > > > > > > > > > > capability. Who is going to review it?
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > -
> > > > > > > > > > > > > > > Denis
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > On Jan 12, 2018, at 11:22 AM, Dmitriy
> > Setrakyan <
> > > > > > > > > > > > >
> > > > > > > > > > > > > [hidden email]
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > +1
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Do we have a list of features for 2.4
> > documented anywhere?
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > D.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > On Fri, Jan 12, 2018 at 7:43 AM, Pavel
> > Tupitsyn <
> > > > > > > > > > > > >
> > > > > > > > > > > > > [hidden email]>
> > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > +1
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > But some stuff is not yet in master (like
> > baseline topology),
> > > > > >
> > > > > > so
> > > > > > > > > it
> > > > > > > > > > > > > is
> > > > > > > > > > > > > > > too
> > > > > > > > > > > > > > > > > soon to create a branch, I think.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Pavel
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > On Fri, Jan 12, 2018 at 5:31 PM, Vladimir
> > Ozerov <
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > [hidden email]>
> > > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Igniters,
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > A number of major improvements have been
> > made (or being
> > > > > >
> > > > > > finalized
> > > > > > > > > > > > > at
> > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > moment) to Ignite since recent 2.3
> > release. This includes
> > > > > > > > >
> > > > > > > > > baseline
> > > > > > > > > > > > > > > > > > topology, new DDL commands, migration to
> > Java 8 and Java 9
> > > > > > > > >
> > > > > > > > > support,
> > > > > > > > > > > > > > > > > > critical performance improvements to WAL,
> > etc..
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > I think it makes sense to make a new
> > release. What do you
> > > > > >
> > > > > > think
> > > > > > > > > if
> > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > > release AI 2.4 with all this great stuff
> > by the end of Jan?
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > If there are no objections, I'll create a
> > branch to start
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > stabilization
> > > > > > > > > > > > > > > > > > process.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Vladimir.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Best Regards, Vyacheslav D.
> > > > > >
> > > > > >
> > >
> > >
> >
>

[dsetrakyan]

On Mon, Feb 12, 2018 at 9:22 AM, Dmitry Pavlov <[hidden email]>
wrote:

> Hi,
>
> Unfortunately, a quick fix did not give us too much performance boost.
>
> I'm going to implement a complete algorithm change for storing the page
> identifier. But this change is quite significant and will require
> re-testing. I suggest including
> https://issues.apache.org/jira/browse/IGNITE-7638 in the next version, for
> example, to 2.5.
>
> Sincerely,
> Dmitriy Pavlov
>
>
>

Dmitriy, thanks for the update! Are there other tickets that are holding
the release at this point? I remember that there was a performance
degradation issue in FULL_SYNC mode, but I cannot find a ticket.

D.

[Vladimir Ozerov]

Igniters,

AI 2.4 release was shifted a bit and over this time we implemented two
important SQL features:
1) COPY command for fast file upload to the cluster [1]
2) Streaming mode for thin driver [2]

Both commands are very important for fast data ingestion into Ignite
through SQL. I would like to ask community to consider to include these two
features into AI 2.4 in *experimental* state because both of them will be
improved in various ways in the nearest time. If we do so, we will be able
to collect some feedback from the users before AI 2.5 release. What do you
think?

Vladimir.

[1] https://issues.apache.org/jira/browse/IGNITE-6917
[2] https://issues.apache.org/jira/browse/IGNITE-7253

On Tue, Feb 13, 2018 at 1:22 AM, Dmitriy Setrakyan <[hidden email]>
wrote:

> On Mon, Feb 12, 2018 at 9:22 AM, Dmitry Pavlov <[hidden email]>
> wrote:
>
> > Hi,
> >
> > Unfortunately, a quick fix did not give us too much performance boost.
> >
> > I'm going to implement a complete algorithm change for storing the page
> > identifier. But this change is quite significant and will require
> > re-testing. I suggest including
> > https://issues.apache.org/jira/browse/IGNITE-7638 in the next version,
> for
> > example, to 2.5.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> >
> >
> Dmitriy, thanks for the update! Are there other tickets that are holding
> the release at this point? I remember that there was a performance
> degradation issue in FULL_SYNC mode, but I cannot find a ticket.
>
> D.
>

[Nikolay Izhikov-2]

+1

В Чт, 15/02/2018 в 17:27 +0300, Vladimir Ozerov пишет:

> Igniters,
>
> AI 2.4 release was shifted a bit and over this time we implemented two
> important SQL features:
> 1) COPY command for fast file upload to the cluster [1]
> 2) Streaming mode for thin driver [2]
>
> Both commands are very important for fast data ingestion into Ignite
> through SQL. I would like to ask community to consider to include these two
> features into AI 2.4 in *experimental* state because both of them will be
> improved in various ways in the nearest time. If we do so, we will be able
> to collect some feedback from the users before AI 2.5 release. What do you
> think?
>
> Vladimir.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-6917
> [2] https://issues.apache.org/jira/browse/IGNITE-7253
>
> On Tue, Feb 13, 2018 at 1:22 AM, Dmitriy Setrakyan <[hidden email]>
> wrote:
>
> > On Mon, Feb 12, 2018 at 9:22 AM, Dmitry Pavlov <[hidden email]>
> > wrote:
> >
> > > Hi,
> > >
> > > Unfortunately, a quick fix did not give us too much performance boost.
> > >
> > > I'm going to implement a complete algorithm change for storing the page
> > > identifier. But this change is quite significant and will require
> > > re-testing. I suggest including
> > > https://issues.apache.org/jira/browse/IGNITE-7638 in the next version,
> >
> > for
> > > example, to 2.5.
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > >
> > >
> >
> > Dmitriy, thanks for the update! Are there other tickets that are holding
> > the release at this point? I remember that there was a performance
> > degradation issue in FULL_SYNC mode, but I cannot find a ticket.
> >
> > D.
> >

[dmagda]

Vladimir,

I would suggest not to do this because we still need to spend time on
testing, documentation, etc. If someone shows interest in this features
they can assemble binaries from the master.

--
Denis

On Thu, Feb 15, 2018 at 6:43 AM, Nikolay Izhikov <[hidden email]>
wrote:

> +1
>
> В Чт, 15/02/2018 в 17:27 +0300, Vladimir Ozerov пишет:
> > Igniters,
> >
> > AI 2.4 release was shifted a bit and over this time we implemented two
> > important SQL features:
> > 1) COPY command for fast file upload to the cluster [1]
> > 2) Streaming mode for thin driver [2]
> >
> > Both commands are very important for fast data ingestion into Ignite
> > through SQL. I would like to ask community to consider to include these
> two
> > features into AI 2.4 in *experimental* state because both of them will be
> > improved in various ways in the nearest time. If we do so, we will be
> able
> > to collect some feedback from the users before AI 2.5 release. What do
> you
> > think?
> >
> > Vladimir.
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-6917
> > [2] https://issues.apache.org/jira/browse/IGNITE-7253
> >
> > On Tue, Feb 13, 2018 at 1:22 AM, Dmitriy Setrakyan <
> [hidden email]>
> > wrote:
> >
> > > On Mon, Feb 12, 2018 at 9:22 AM, Dmitry Pavlov <[hidden email]>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > Unfortunately, a quick fix did not give us too much performance
> boost.
> > > >
> > > > I'm going to implement a complete algorithm change for storing the
> page
> > > > identifier. But this change is quite significant and will require
> > > > re-testing. I suggest including
> > > > https://issues.apache.org/jira/browse/IGNITE-7638 in the next
> version,
> > >
> > > for
> > > > example, to 2.5.
> > > >
> > > > Sincerely,
> > > > Dmitriy Pavlov
> > > >
> > > >
> > > >
> > >
> > > Dmitriy, thanks for the update! Are there other tickets that are
> holding
> > > the release at this point? I remember that there was a performance
> > > degradation issue in FULL_SYNC mode, but I cannot find a ticket.
> > >
> > > D.
> > >
>

[Ilya Lantukh]

Hi all,

I'd like to suggest to change default WALMode. Currently we have:
DEFAULT (write and fsync),
LOG_ONLY (write without fsync),
BACKGROUND,
NONE.

It turns out that fsyncs in current DEFAULT mode significantly restricts
Ignite performance. Compared to LOG_ONLY, it offers additional guarantees
that data won't be lost in case of OS or hardware failure, but such
guarantees aren't needed very often, and tradeoff is too big.

I suggest to rename current DEFAULT to STRICT and make LOG_ONLY new default
mode. We can leave DEFAULT as @Deprecated and treat it as STRICT, so that
users with old configs will have the same behaviour.

What do you think?

On Fri, Feb 16, 2018 at 12:35 AM, Denis Magda <[hidden email]> wrote:

> Vladimir,
>
> I would suggest not to do this because we still need to spend time on
> testing, documentation, etc. If someone shows interest in this features
> they can assemble binaries from the master.
>
> --
> Denis
>
> On Thu, Feb 15, 2018 at 6:43 AM, Nikolay Izhikov <[hidden email]>
> wrote:
>
> > +1
> >
> > В Чт, 15/02/2018 в 17:27 +0300, Vladimir Ozerov пишет:
> > > Igniters,
> > >
> > > AI 2.4 release was shifted a bit and over this time we implemented two
> > > important SQL features:
> > > 1) COPY command for fast file upload to the cluster [1]
> > > 2) Streaming mode for thin driver [2]
> > >
> > > Both commands are very important for fast data ingestion into Ignite
> > > through SQL. I would like to ask community to consider to include these
> > two
> > > features into AI 2.4 in *experimental* state because both of them will
> be
> > > improved in various ways in the nearest time. If we do so, we will be
> > able
> > > to collect some feedback from the users before AI 2.5 release. What do
> > you
> > > think?
> > >
> > > Vladimir.
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-6917
> > > [2] https://issues.apache.org/jira/browse/IGNITE-7253
> > >
> > > On Tue, Feb 13, 2018 at 1:22 AM, Dmitriy Setrakyan <
> > [hidden email]>
> > > wrote:
> > >
> > > > On Mon, Feb 12, 2018 at 9:22 AM, Dmitry Pavlov <
> [hidden email]>
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > Unfortunately, a quick fix did not give us too much performance
> > boost.
> > > > >
> > > > > I'm going to implement a complete algorithm change for storing the
> > page
> > > > > identifier. But this change is quite significant and will require
> > > > > re-testing. I suggest including
> > > > > https://issues.apache.org/jira/browse/IGNITE-7638 in the next
> > version,
> > > >
> > > > for
> > > > > example, to 2.5.
> > > > >
> > > > > Sincerely,
> > > > > Dmitriy Pavlov
> > > > >
> > > > >
> > > > >
> > > >
> > > > Dmitriy, thanks for the update! Are there other tickets that are
> > holding
> > > > the release at this point? I remember that there was a performance
> > > > degradation issue in FULL_SYNC mode, but I cannot find a ticket.
> > > >
> > > > D.
> > > >
> >
>

--
Best regards,
Ilya

[dsetrakyan]

Naming one of the enum constants DEFAULT was a huge mistake. Not sure how
it passed a code review, but let us all be more careful going forward.

I agree with Ilya. The only remedy right now is to deprecate the DEFAULT
constant.

D.

On Fri, Feb 16, 2018 at 5:37 AM, Ilya Lantukh <[hidden email]> wrote:

> Hi all,
>
> I'd like to suggest to change default WALMode. Currently we have:
> DEFAULT (write and fsync),
> LOG_ONLY (write without fsync),
> BACKGROUND,
> NONE.
>
> It turns out that fsyncs in current DEFAULT mode significantly restricts
> Ignite performance. Compared to LOG_ONLY, it offers additional guarantees
> that data won't be lost in case of OS or hardware failure, but such
> guarantees aren't needed very often, and tradeoff is too big.
>
> I suggest to rename current DEFAULT to STRICT and make LOG_ONLY new default
> mode. We can leave DEFAULT as @Deprecated and treat it as STRICT, so that
> users with old configs will have the same behaviour.
>
> What do you think?
>
> On Fri, Feb 16, 2018 at 12:35 AM, Denis Magda <[hidden email]> wrote:
>
> > Vladimir,
> >
> > I would suggest not to do this because we still need to spend time on
> > testing, documentation, etc. If someone shows interest in this features
> > they can assemble binaries from the master.
> >
> > --
> > Denis
> >
> > On Thu, Feb 15, 2018 at 6:43 AM, Nikolay Izhikov <[hidden email]>
> > wrote:
> >
> > > +1
> > >
> > > В Чт, 15/02/2018 в 17:27 +0300, Vladimir Ozerov пишет:
> > > > Igniters,
> > > >
> > > > AI 2.4 release was shifted a bit and over this time we implemented
> two
> > > > important SQL features:
> > > > 1) COPY command for fast file upload to the cluster [1]
> > > > 2) Streaming mode for thin driver [2]
> > > >
> > > > Both commands are very important for fast data ingestion into Ignite
> > > > through SQL. I would like to ask community to consider to include
> these
> > > two
> > > > features into AI 2.4 in *experimental* state because both of them
> will
> > be
> > > > improved in various ways in the nearest time. If we do so, we will be
> > > able
> > > > to collect some feedback from the users before AI 2.5 release. What
> do
> > > you
> > > > think?
> > > >
> > > > Vladimir.
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IGNITE-6917
> > > > [2] https://issues.apache.org/jira/browse/IGNITE-7253
> > > >
> > > > On Tue, Feb 13, 2018 at 1:22 AM, Dmitriy Setrakyan <
> > > [hidden email]>
> > > > wrote:
> > > >
> > > > > On Mon, Feb 12, 2018 at 9:22 AM, Dmitry Pavlov <
> > [hidden email]>
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > Unfortunately, a quick fix did not give us too much performance
> > > boost.
> > > > > >
> > > > > > I'm going to implement a complete algorithm change for storing
> the
> > > page
> > > > > > identifier. But this change is quite significant and will require
> > > > > > re-testing. I suggest including
> > > > > > https://issues.apache.org/jira/browse/IGNITE-7638 in the next
> > > version,
> > > > >
> > > > > for
> > > > > > example, to 2.5.
> > > > > >
> > > > > > Sincerely,
> > > > > > Dmitriy Pavlov
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > > Dmitriy, thanks for the update! Are there other tickets that are
> > > holding
> > > > > the release at this point? I remember that there was a performance
> > > > > degradation issue in FULL_SYNC mode, but I cannot find a ticket.
> > > > >
> > > > > D.
> > > > >
> > >
> >
>
>
>
> --
> Best regards,
> Ilya
>

[dsetrakyan]

BTW, Ilya, why not name the enum value FULL_SYNC instead of STRICT?

On Fri, Feb 16, 2018 at 5:43 AM, Dmitriy Setrakyan <[hidden email]>
wrote:

> Naming one of the enum constants DEFAULT was a huge mistake. Not sure how
> it passed a code review, but let us all be more careful going forward.
>
> I agree with Ilya. The only remedy right now is to deprecate the DEFAULT
> constant.
>
> D.
>
> On Fri, Feb 16, 2018 at 5:37 AM, Ilya Lantukh <[hidden email]>
> wrote:
>
>> Hi all,
>>
>> I'd like to suggest to change default WALMode. Currently we have:
>> DEFAULT (write and fsync),
>> LOG_ONLY (write without fsync),
>> BACKGROUND,
>> NONE.
>>
>> It turns out that fsyncs in current DEFAULT mode significantly restricts
>> Ignite performance. Compared to LOG_ONLY, it offers additional guarantees
>> that data won't be lost in case of OS or hardware failure, but such
>> guarantees aren't needed very often, and tradeoff is too big.
>>
>> I suggest to rename current DEFAULT to STRICT and make LOG_ONLY new
>> default
>> mode. We can leave DEFAULT as @Deprecated and treat it as STRICT, so that
>> users with old configs will have the same behaviour.
>>
>> What do you think?
>>
>> On Fri, Feb 16, 2018 at 12:35 AM, Denis Magda <[hidden email]> wrote:
>>
>> > Vladimir,
>> >
>> > I would suggest not to do this because we still need to spend time on
>> > testing, documentation, etc. If someone shows interest in this features
>> > they can assemble binaries from the master.
>> >
>> > --
>> > Denis
>> >
>> > On Thu, Feb 15, 2018 at 6:43 AM, Nikolay Izhikov <[hidden email]>
>> > wrote:
>> >
>> > > +1
>> > >
>> > > В Чт, 15/02/2018 в 17:27 +0300, Vladimir Ozerov пишет:
>> > > > Igniters,
>> > > >
>> > > > AI 2.4 release was shifted a bit and over this time we implemented
>> two
>> > > > important SQL features:
>> > > > 1) COPY command for fast file upload to the cluster [1]
>> > > > 2) Streaming mode for thin driver [2]
>> > > >
>> > > > Both commands are very important for fast data ingestion into Ignite
>> > > > through SQL. I would like to ask community to consider to include
>> these
>> > > two
>> > > > features into AI 2.4 in *experimental* state because both of them
>> will
>> > be
>> > > > improved in various ways in the nearest time. If we do so, we will
>> be
>> > > able
>> > > > to collect some feedback from the users before AI 2.5 release. What
>> do
>> > > you
>> > > > think?
>> > > >
>> > > > Vladimir.
>> > > >
>> > > > [1] https://issues.apache.org/jira/browse/IGNITE-6917
>> > > > [2] https://issues.apache.org/jira/browse/IGNITE-7253
>> > > >
>> > > > On Tue, Feb 13, 2018 at 1:22 AM, Dmitriy Setrakyan <
>> > > [hidden email]>
>> > > > wrote:
>> > > >
>> > > > > On Mon, Feb 12, 2018 at 9:22 AM, Dmitry Pavlov <
>> > [hidden email]>
>> > > > > wrote:
>> > > > >
>> > > > > > Hi,
>> > > > > >
>> > > > > > Unfortunately, a quick fix did not give us too much performance
>> > > boost.
>> > > > > >
>> > > > > > I'm going to implement a complete algorithm change for storing
>> the
>> > > page
>> > > > > > identifier. But this change is quite significant and will
>> require
>> > > > > > re-testing. I suggest including
>> > > > > > https://issues.apache.org/jira/browse/IGNITE-7638 in the next
>> > > version,
>> > > > >
>> > > > > for
>> > > > > > example, to 2.5.
>> > > > > >
>> > > > > > Sincerely,
>> > > > > > Dmitriy Pavlov
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > >
>> > > > > Dmitriy, thanks for the update! Are there other tickets that are
>> > > holding
>> > > > > the release at this point? I remember that there was a performance
>> > > > > degradation issue in FULL_SYNC mode, but I cannot find a ticket.
>> > > > >
>> > > > > D.
>> > > > >
>> > >
>> >
>>
>>
>>
>> --
>> Best regards,
>> Ilya
>>
>
>

[Ivan Rakov]

 From my point of view, STRICT is the best option. The name signalizes
to user that this mode provides optional strict guarantees.
FULL_SYNC can be messed with CacheWriteSynchronizationMode#FULL_SYNC. I
don't like the idea of naming different things with same names.

Best Regards,
Ivan Rakov

On 16.02.2018 15:01, Dmitriy Setrakyan wrote:

> BTW, Ilya, why not name the enum value FULL_SYNC instead of STRICT?
>
> On Fri, Feb 16, 2018 at 5:43 AM, Dmitriy Setrakyan <[hidden email]>
> wrote:
>
>> Naming one of the enum constants DEFAULT was a huge mistake. Not sure how
>> it passed a code review, but let us all be more careful going forward.
>>
>> I agree with Ilya. The only remedy right now is to deprecate the DEFAULT
>> constant.
>>
>> D.
>>
>> On Fri, Feb 16, 2018 at 5:37 AM, Ilya Lantukh <[hidden email]>
>> wrote:
>>
>>> Hi all,
>>>
>>> I'd like to suggest to change default WALMode. Currently we have:
>>> DEFAULT (write and fsync),
>>> LOG_ONLY (write without fsync),
>>> BACKGROUND,
>>> NONE.
>>>
>>> It turns out that fsyncs in current DEFAULT mode significantly restricts
>>> Ignite performance. Compared to LOG_ONLY, it offers additional guarantees
>>> that data won't be lost in case of OS or hardware failure, but such
>>> guarantees aren't needed very often, and tradeoff is too big.
>>>
>>> I suggest to rename current DEFAULT to STRICT and make LOG_ONLY new
>>> default
>>> mode. We can leave DEFAULT as @Deprecated and treat it as STRICT, so that
>>> users with old configs will have the same behaviour.
>>>
>>> What do you think?
>>>
>>> On Fri, Feb 16, 2018 at 12:35 AM, Denis Magda <[hidden email]> wrote:
>>>
>>>> Vladimir,
>>>>
>>>> I would suggest not to do this because we still need to spend time on
>>>> testing, documentation, etc. If someone shows interest in this features
>>>> they can assemble binaries from the master.
>>>>
>>>> --
>>>> Denis
>>>>
>>>> On Thu, Feb 15, 2018 at 6:43 AM, Nikolay Izhikov <[hidden email]>
>>>> wrote:
>>>>
>>>>> +1
>>>>>
>>>>> В Чт, 15/02/2018 в 17:27 +0300, Vladimir Ozerov пишет:
>>>>>> Igniters,
>>>>>>
>>>>>> AI 2.4 release was shifted a bit and over this time we implemented
>>> two
>>>>>> important SQL features:
>>>>>> 1) COPY command for fast file upload to the cluster [1]
>>>>>> 2) Streaming mode for thin driver [2]
>>>>>>
>>>>>> Both commands are very important for fast data ingestion into Ignite
>>>>>> through SQL. I would like to ask community to consider to include
>>> these
>>>>> two
>>>>>> features into AI 2.4 in *experimental* state because both of them
>>> will
>>>> be
>>>>>> improved in various ways in the nearest time. If we do so, we will
>>> be
>>>>> able
>>>>>> to collect some feedback from the users before AI 2.5 release. What
>>> do
>>>>> you
>>>>>> think?
>>>>>>
>>>>>> Vladimir.
>>>>>>
>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-6917
>>>>>> [2] https://issues.apache.org/jira/browse/IGNITE-7253
>>>>>>
>>>>>> On Tue, Feb 13, 2018 at 1:22 AM, Dmitriy Setrakyan <
>>>>> [hidden email]>
>>>>>> wrote:
>>>>>>
>>>>>>> On Mon, Feb 12, 2018 at 9:22 AM, Dmitry Pavlov <
>>>> [hidden email]>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Unfortunately, a quick fix did not give us too much performance
>>>>> boost.
>>>>>>>> I'm going to implement a complete algorithm change for storing
>>> the
>>>>> page
>>>>>>>> identifier. But this change is quite significant and will
>>> require
>>>>>>>> re-testing. I suggest including
>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-7638 in the next
>>>>> version,
>>>>>>> for
>>>>>>>> example, to 2.5.
>>>>>>>>
>>>>>>>> Sincerely,
>>>>>>>> Dmitriy Pavlov
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> Dmitriy, thanks for the update! Are there other tickets that are
>>>>> holding
>>>>>>> the release at this point? I remember that there was a performance
>>>>>>> degradation issue in FULL_SYNC mode, but I cannot find a ticket.
>>>>>>>
>>>>>>> D.
>>>>>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Ilya
>>>
>>

[Anton Vinogradov]

What about
FULL_SYNC
SYNC -> default
BACKGROUND
NODE
?

On Fri, Feb 16, 2018 at 3:09 PM, Ivan Rakov <[hidden email]> wrote:

> From my point of view, STRICT is the best option. The name signalizes to
> user that this mode provides optional strict guarantees.
> FULL_SYNC can be messed with CacheWriteSynchronizationMode#FULL_SYNC. I
> don't like the idea of naming different things with same names.
>
> Best Regards,
> Ivan Rakov
>
>
> On 16.02.2018 15:01, Dmitriy Setrakyan wrote:
>
>> BTW, Ilya, why not name the enum value FULL_SYNC instead of STRICT?
>>
>> On Fri, Feb 16, 2018 at 5:43 AM, Dmitriy Setrakyan <[hidden email]
>> >
>> wrote:
>>
>> Naming one of the enum constants DEFAULT was a huge mistake. Not sure how
>>> it passed a code review, but let us all be more careful going forward.
>>>
>>> I agree with Ilya. The only remedy right now is to deprecate the DEFAULT
>>> constant.
>>>
>>> D.
>>>
>>> On Fri, Feb 16, 2018 at 5:37 AM, Ilya Lantukh <[hidden email]>
>>> wrote:
>>>
>>> Hi all,
>>>>
>>>> I'd like to suggest to change default WALMode. Currently we have:
>>>> DEFAULT (write and fsync),
>>>> LOG_ONLY (write without fsync),
>>>> BACKGROUND,
>>>> NONE.
>>>>
>>>> It turns out that fsyncs in current DEFAULT mode significantly restricts
>>>> Ignite performance. Compared to LOG_ONLY, it offers additional
>>>> guarantees
>>>> that data won't be lost in case of OS or hardware failure, but such
>>>> guarantees aren't needed very often, and tradeoff is too big.
>>>>
>>>> I suggest to rename current DEFAULT to STRICT and make LOG_ONLY new
>>>> default
>>>> mode. We can leave DEFAULT as @Deprecated and treat it as STRICT, so
>>>> that
>>>> users with old configs will have the same behaviour.
>>>>
>>>> What do you think?
>>>>
>>>> On Fri, Feb 16, 2018 at 12:35 AM, Denis Magda <[hidden email]>
>>>> wrote:
>>>>
>>>> Vladimir,
>>>>>
>>>>> I would suggest not to do this because we still need to spend time on
>>>>> testing, documentation, etc. If someone shows interest in this features
>>>>> they can assemble binaries from the master.
>>>>>
>>>>> --
>>>>> Denis
>>>>>
>>>>> On Thu, Feb 15, 2018 at 6:43 AM, Nikolay Izhikov <[hidden email]>
>>>>> wrote:
>>>>>
>>>>> +1
>>>>>>
>>>>>> В Чт, 15/02/2018 в 17:27 +0300, Vladimir Ozerov пишет:
>>>>>>
>>>>>>> Igniters,
>>>>>>>
>>>>>>> AI 2.4 release was shifted a bit and over this time we implemented
>>>>>>>
>>>>>> two
>>>>
>>>>> important SQL features:
>>>>>>> 1) COPY command for fast file upload to the cluster [1]
>>>>>>> 2) Streaming mode for thin driver [2]
>>>>>>>
>>>>>>> Both commands are very important for fast data ingestion into Ignite
>>>>>>> through SQL. I would like to ask community to consider to include
>>>>>>>
>>>>>> these
>>>>
>>>>> two
>>>>>>
>>>>>>> features into AI 2.4 in *experimental* state because both of them
>>>>>>>
>>>>>> will
>>>>
>>>>> be
>>>>>
>>>>>> improved in various ways in the nearest time. If we do so, we will
>>>>>>>
>>>>>> be
>>>>
>>>>> able
>>>>>>
>>>>>>> to collect some feedback from the users before AI 2.5 release. What
>>>>>>>
>>>>>> do
>>>>
>>>>> you
>>>>>>
>>>>>>> think?
>>>>>>>
>>>>>>> Vladimir.
>>>>>>>
>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-6917
>>>>>>> [2] https://issues.apache.org/jira/browse/IGNITE-7253
>>>>>>>
>>>>>>> On Tue, Feb 13, 2018 at 1:22 AM, Dmitriy Setrakyan <
>>>>>>>
>>>>>> [hidden email]>
>>>>>>
>>>>>>> wrote:
>>>>>>>
>>>>>>> On Mon, Feb 12, 2018 at 9:22 AM, Dmitry Pavlov <
>>>>>>>>
>>>>>>> [hidden email]>
>>>>>
>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Unfortunately, a quick fix did not give us too much performance
>>>>>>>>>
>>>>>>>> boost.
>>>>>>
>>>>>>> I'm going to implement a complete algorithm change for storing
>>>>>>>>>
>>>>>>>> the
>>>>
>>>>> page
>>>>>>
>>>>>>> identifier. But this change is quite significant and will
>>>>>>>>>
>>>>>>>> require
>>>>
>>>>> re-testing. I suggest including
>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-7638 in the next
>>>>>>>>>
>>>>>>>> version,
>>>>>>
>>>>>>> for
>>>>>>>>
>>>>>>>>> example, to 2.5.
>>>>>>>>>
>>>>>>>>> Sincerely,
>>>>>>>>> Dmitriy Pavlov
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Dmitriy, thanks for the update! Are there other tickets that are
>>>>>>>>
>>>>>>> holding
>>>>>>
>>>>>>> the release at this point? I remember that there was a performance
>>>>>>>> degradation issue in FULL_SYNC mode, but I cannot find a ticket.
>>>>>>>>
>>>>>>>> D.
>>>>>>>>
>>>>>>>>
>>>>
>>>> --
>>>> Best regards,
>>>> Ilya
>>>>
>>>>
>>>
>

[Anton Vinogradov]

typo
NODE -> NONE

On Fri, Feb 16, 2018 at 3:21 PM, Anton Vinogradov <[hidden email]>
wrote:

> What about
> FULL_SYNC
> SYNC -> default
> BACKGROUND
> NODE
> ?
>
> On Fri, Feb 16, 2018 at 3:09 PM, Ivan Rakov <[hidden email]> wrote:
>
>> From my point of view, STRICT is the best option. The name signalizes to
>> user that this mode provides optional strict guarantees.
>> FULL_SYNC can be messed with CacheWriteSynchronizationMode#FULL_SYNC. I
>> don't like the idea of naming different things with same names.
>>
>> Best Regards,
>> Ivan Rakov
>>
>>
>> On 16.02.2018 15:01, Dmitriy Setrakyan wrote:
>>
>>> BTW, Ilya, why not name the enum value FULL_SYNC instead of STRICT?
>>>
>>> On Fri, Feb 16, 2018 at 5:43 AM, Dmitriy Setrakyan <
>>> [hidden email]>
>>> wrote:
>>>
>>> Naming one of the enum constants DEFAULT was a huge mistake. Not sure how
>>>> it passed a code review, but let us all be more careful going forward.
>>>>
>>>> I agree with Ilya. The only remedy right now is to deprecate the DEFAULT
>>>> constant.
>>>>
>>>> D.
>>>>
>>>> On Fri, Feb 16, 2018 at 5:37 AM, Ilya Lantukh <[hidden email]>
>>>> wrote:
>>>>
>>>> Hi all,
>>>>>
>>>>> I'd like to suggest to change default WALMode. Currently we have:
>>>>> DEFAULT (write and fsync),
>>>>> LOG_ONLY (write without fsync),
>>>>> BACKGROUND,
>>>>> NONE.
>>>>>
>>>>> It turns out that fsyncs in current DEFAULT mode significantly
>>>>> restricts
>>>>> Ignite performance. Compared to LOG_ONLY, it offers additional
>>>>> guarantees
>>>>> that data won't be lost in case of OS or hardware failure, but such
>>>>> guarantees aren't needed very often, and tradeoff is too big.
>>>>>
>>>>> I suggest to rename current DEFAULT to STRICT and make LOG_ONLY new
>>>>> default
>>>>> mode. We can leave DEFAULT as @Deprecated and treat it as STRICT, so
>>>>> that
>>>>> users with old configs will have the same behaviour.
>>>>>
>>>>> What do you think?
>>>>>
>>>>> On Fri, Feb 16, 2018 at 12:35 AM, Denis Magda <[hidden email]>
>>>>> wrote:
>>>>>
>>>>> Vladimir,
>>>>>>
>>>>>> I would suggest not to do this because we still need to spend time on
>>>>>> testing, documentation, etc. If someone shows interest in this
>>>>>> features
>>>>>> they can assemble binaries from the master.
>>>>>>
>>>>>> --
>>>>>> Denis
>>>>>>
>>>>>> On Thu, Feb 15, 2018 at 6:43 AM, Nikolay Izhikov <[hidden email]
>>>>>> >
>>>>>> wrote:
>>>>>>
>>>>>> +1
>>>>>>>
>>>>>>> В Чт, 15/02/2018 в 17:27 +0300, Vladimir Ozerov пишет:
>>>>>>>
>>>>>>>> Igniters,
>>>>>>>>
>>>>>>>> AI 2.4 release was shifted a bit and over this time we implemented
>>>>>>>>
>>>>>>> two
>>>>>
>>>>>> important SQL features:
>>>>>>>> 1) COPY command for fast file upload to the cluster [1]
>>>>>>>> 2) Streaming mode for thin driver [2]
>>>>>>>>
>>>>>>>> Both commands are very important for fast data ingestion into Ignite
>>>>>>>> through SQL. I would like to ask community to consider to include
>>>>>>>>
>>>>>>> these
>>>>>
>>>>>> two
>>>>>>>
>>>>>>>> features into AI 2.4 in *experimental* state because both of them
>>>>>>>>
>>>>>>> will
>>>>>
>>>>>> be
>>>>>>
>>>>>>> improved in various ways in the nearest time. If we do so, we will
>>>>>>>>
>>>>>>> be
>>>>>
>>>>>> able
>>>>>>>
>>>>>>>> to collect some feedback from the users before AI 2.5 release. What
>>>>>>>>
>>>>>>> do
>>>>>
>>>>>> you
>>>>>>>
>>>>>>>> think?
>>>>>>>>
>>>>>>>> Vladimir.
>>>>>>>>
>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-6917
>>>>>>>> [2] https://issues.apache.org/jira/browse/IGNITE-7253
>>>>>>>>
>>>>>>>> On Tue, Feb 13, 2018 at 1:22 AM, Dmitriy Setrakyan <
>>>>>>>>
>>>>>>> [hidden email]>
>>>>>>>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> On Mon, Feb 12, 2018 at 9:22 AM, Dmitry Pavlov <
>>>>>>>>>
>>>>>>>> [hidden email]>
>>>>>>
>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> Unfortunately, a quick fix did not give us too much performance
>>>>>>>>>>
>>>>>>>>> boost.
>>>>>>>
>>>>>>>> I'm going to implement a complete algorithm change for storing
>>>>>>>>>>
>>>>>>>>> the
>>>>>
>>>>>> page
>>>>>>>
>>>>>>>> identifier. But this change is quite significant and will
>>>>>>>>>>
>>>>>>>>> require
>>>>>
>>>>>> re-testing. I suggest including
>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-7638 in the next
>>>>>>>>>>
>>>>>>>>> version,
>>>>>>>
>>>>>>>> for
>>>>>>>>>
>>>>>>>>>> example, to 2.5.
>>>>>>>>>>
>>>>>>>>>> Sincerely,
>>>>>>>>>> Dmitriy Pavlov
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Dmitriy, thanks for the update! Are there other tickets that are
>>>>>>>>>
>>>>>>>> holding
>>>>>>>
>>>>>>>> the release at this point? I remember that there was a performance
>>>>>>>>> degradation issue in FULL_SYNC mode, but I cannot find a ticket.
>>>>>>>>>
>>>>>>>>> D.
>>>>>>>>>
>>>>>>>>>
>>>>>
>>>>> --
>>>>> Best regards,
>>>>> Ilya
>>>>>
>>>>>
>>>>
>>
>

[Dmitriy Pavlov]

I had idea to name old default as FSYNC, but it would be too scientific.

For old DEFAULT, STRICT or STRICT_SYNC - IMO are best options, so I agree
with Ivan.

пт, 16 февр. 2018 г. в 15:21, Anton Vinogradov <[hidden email]>:

> typo
> NODE -> NONE
>
> On Fri, Feb 16, 2018 at 3:21 PM, Anton Vinogradov <
> [hidden email]>
> wrote:
>
> > What about
> > FULL_SYNC
> > SYNC -> default
> > BACKGROUND
> > NODE
> > ?
> >
> > On Fri, Feb 16, 2018 at 3:09 PM, Ivan Rakov <[hidden email]>
> wrote:
> >
> >> From my point of view, STRICT is the best option. The name signalizes to
> >> user that this mode provides optional strict guarantees.
> >> FULL_SYNC can be messed with CacheWriteSynchronizationMode#FULL_SYNC. I
> >> don't like the idea of naming different things with same names.
> >>
> >> Best Regards,
> >> Ivan Rakov
> >>
> >>
> >> On 16.02.2018 15:01, Dmitriy Setrakyan wrote:
> >>
> >>> BTW, Ilya, why not name the enum value FULL_SYNC instead of STRICT?
> >>>
> >>> On Fri, Feb 16, 2018 at 5:43 AM, Dmitriy Setrakyan <
> >>> [hidden email]>
> >>> wrote:
> >>>
> >>> Naming one of the enum constants DEFAULT was a huge mistake. Not sure
> how
> >>>> it passed a code review, but let us all be more careful going forward.
> >>>>
> >>>> I agree with Ilya. The only remedy right now is to deprecate the
> DEFAULT
> >>>> constant.
> >>>>
> >>>> D.
> >>>>
> >>>> On Fri, Feb 16, 2018 at 5:37 AM, Ilya Lantukh <[hidden email]>
> >>>> wrote:
> >>>>
> >>>> Hi all,
> >>>>>
> >>>>> I'd like to suggest to change default WALMode. Currently we have:
> >>>>> DEFAULT (write and fsync),
> >>>>> LOG_ONLY (write without fsync),
> >>>>> BACKGROUND,
> >>>>> NONE.
> >>>>>
> >>>>> It turns out that fsyncs in current DEFAULT mode significantly
> >>>>> restricts
> >>>>> Ignite performance. Compared to LOG_ONLY, it offers additional
> >>>>> guarantees
> >>>>> that data won't be lost in case of OS or hardware failure, but such
> >>>>> guarantees aren't needed very often, and tradeoff is too big.
> >>>>>
> >>>>> I suggest to rename current DEFAULT to STRICT and make LOG_ONLY new
> >>>>> default
> >>>>> mode. We can leave DEFAULT as @Deprecated and treat it as STRICT, so
> >>>>> that
> >>>>> users with old configs will have the same behaviour.
> >>>>>
> >>>>> What do you think?
> >>>>>
> >>>>> On Fri, Feb 16, 2018 at 12:35 AM, Denis Magda <[hidden email]>
> >>>>> wrote:
> >>>>>
> >>>>> Vladimir,
> >>>>>>
> >>>>>> I would suggest not to do this because we still need to spend time
> on
> >>>>>> testing, documentation, etc. If someone shows interest in this
> >>>>>> features
> >>>>>> they can assemble binaries from the master.
> >>>>>>
> >>>>>> --
> >>>>>> Denis
> >>>>>>
> >>>>>> On Thu, Feb 15, 2018 at 6:43 AM, Nikolay Izhikov <
> [hidden email]
> >>>>>> >
> >>>>>> wrote:
> >>>>>>
> >>>>>> +1
> >>>>>>>
> >>>>>>> В Чт, 15/02/2018 в 17:27 +0300, Vladimir Ozerov пишет:
> >>>>>>>
> >>>>>>>> Igniters,
> >>>>>>>>
> >>>>>>>> AI 2.4 release was shifted a bit and over this time we implemented
> >>>>>>>>
> >>>>>>> two
> >>>>>
> >>>>>> important SQL features:
> >>>>>>>> 1) COPY command for fast file upload to the cluster [1]
> >>>>>>>> 2) Streaming mode for thin driver [2]
> >>>>>>>>
> >>>>>>>> Both commands are very important for fast data ingestion into
> Ignite
> >>>>>>>> through SQL. I would like to ask community to consider to include
> >>>>>>>>
> >>>>>>> these
> >>>>>
> >>>>>> two
> >>>>>>>
> >>>>>>>> features into AI 2.4 in *experimental* state because both of them
> >>>>>>>>
> >>>>>>> will
> >>>>>
> >>>>>> be
> >>>>>>
> >>>>>>> improved in various ways in the nearest time. If we do so, we will
> >>>>>>>>
> >>>>>>> be
> >>>>>
> >>>>>> able
> >>>>>>>
> >>>>>>>> to collect some feedback from the users before AI 2.5 release.
> What
> >>>>>>>>
> >>>>>>> do
> >>>>>
> >>>>>> you
> >>>>>>>
> >>>>>>>> think?
> >>>>>>>>
> >>>>>>>> Vladimir.
> >>>>>>>>
> >>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-6917
> >>>>>>>> [2] https://issues.apache.org/jira/browse/IGNITE-7253
> >>>>>>>>
> >>>>>>>> On Tue, Feb 13, 2018 at 1:22 AM, Dmitriy Setrakyan <
> >>>>>>>>
> >>>>>>> [hidden email]>
> >>>>>>>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> On Mon, Feb 12, 2018 at 9:22 AM, Dmitry Pavlov <
> >>>>>>>>>
> >>>>>>>> [hidden email]>
> >>>>>>
> >>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> Hi,
> >>>>>>>>>>
> >>>>>>>>>> Unfortunately, a quick fix did not give us too much performance
> >>>>>>>>>>
> >>>>>>>>> boost.
> >>>>>>>
> >>>>>>>> I'm going to implement a complete algorithm change for storing
> >>>>>>>>>>
> >>>>>>>>> the
> >>>>>
> >>>>>> page
> >>>>>>>
> >>>>>>>> identifier. But this change is quite significant and will
> >>>>>>>>>>
> >>>>>>>>> require
> >>>>>
> >>>>>> re-testing. I suggest including
> >>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-7638 in the next
> >>>>>>>>>>
> >>>>>>>>> version,
> >>>>>>>
> >>>>>>>> for
> >>>>>>>>>
> >>>>>>>>>> example, to 2.5.
> >>>>>>>>>>
> >>>>>>>>>> Sincerely,
> >>>>>>>>>> Dmitriy Pavlov
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Dmitriy, thanks for the update! Are there other tickets that are
> >>>>>>>>>
> >>>>>>>> holding
> >>>>>>>
> >>>>>>>> the release at this point? I remember that there was a performance
> >>>>>>>>> degradation issue in FULL_SYNC mode, but I cannot find a ticket.
> >>>>>>>>>
> >>>>>>>>> D.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>
> >>>>> --
> >>>>> Best regards,
> >>>>> Ilya
> >>>>>
> >>>>>
> >>>>
> >>
> >
>

[dsetrakyan]

On Fri, Feb 16, 2018 at 6:26 AM, Dmitry Pavlov <[hidden email]>
wrote:

> I had idea to name old default as FSYNC, but it would be too scientific.
>

I like FSYNC, I do not think it is too scientific. Definitely not more
scientific than LOG_ONLY.

For old DEFAULT, STRICT or STRICT_SYNC - IMO are best options, so I agree
> with Ivan.
>

Not sure I like STRICT. In Linux world, fsync is a well known command which
does exactly what our FSYNC mode will do:
http://man7.org/linux/man-pages/man2/fsync.2.html . STRICT, on the other
hand, is not a commonly understood term. Why create a new term to define
something that has already been defined?

Also, what if tomorrow we need to add an even stricter parameter? Then we
are back to the same problem we are trying to fix right now.

D.

[Anton Vinogradov]

 >> I had idea to name old default as FSYNC, but it would be too scientific.
So, then it can be  FSYNC, SYNC, BACKGROUND and NONE!

On Fri, Feb 16, 2018 at 3:49 PM, Dmitriy Setrakyan <[hidden email]>
wrote:

> On Fri, Feb 16, 2018 at 6:26 AM, Dmitry Pavlov <[hidden email]>
> wrote:
>
> > I had idea to name old default as FSYNC, but it would be too scientific.
> >
>
> I like FSYNC, I do not think it is too scientific. Definitely not more
> scientific than LOG_ONLY.
>
> For old DEFAULT, STRICT or STRICT_SYNC - IMO are best options, so I agree
> > with Ivan.
> >
>
> Not sure I like STRICT. In Linux world, fsync is a well known command which
> does exactly what our FSYNC mode will do:
> http://man7.org/linux/man-pages/man2/fsync.2.html . STRICT, on the other
> hand, is not a commonly understood term. Why create a new term to define
> something that has already been defined?
>
> Also, what if tomorrow we need to add an even stricter parameter? Then we
> are back to the same problem we are trying to fix right now.
>
> D.
>