Apache Ignite Developers - Legacy Mail Archive

Apache Ignite 2.4 release

Classic

List

Threaded

69 messages Options

1234

Vladimir Ozerov

Re: Apache Ignite 2.4 release

Alex,

You get me right. DEFAULT -> LOG_ONLY doesn't introduce any dramatic
changes when comparing 2.3 to 2.4 - Ignite was unsafe out of the box in
2.3, and it is unsafe in 2.4 as well.

The very problem is that we claim ourselves to be ACID, while in reality we
are only "AI" out of the box, because durability is not guaranteed due to
zero backups and LOG_ONLY and consistency is not guaranteed due to
PRIMARY_SYNC. Neither Cassandra, nor Mongo or any others claim themselves
to be ACID, so it is not valid to refer to their defaults.

On Mon, Feb 19, 2018 at 10:06 AM, Alexey Goncharuk <
[hidden email]> wrote:

> In terms of 'safety', Ignite default settings are far beyond optimal. For
> in-memory mode, we have 0 backups by default, which means partition loss in
> a case of node failure, we have readFromBackup=true and PRIMARY_SYNC by
> default which effectively cancels linearizability property for cache
> updates, so setting the default WAL mode to LOG_ONLY does not seem to be a
> bigger evil than it currently is. If we are to move to safer defaults, we
> should change all of the affected sides.
>
> I also want to clarify the difference between guarantees in
> non-fsync modes. We should distinguish the loss of durability (the loss of
> the last update) because the update did not make it to disk and data loss
> because the disk content was shuffled due to an incomplete page write. In
> my understanding, the current situation is:
> FSYNC: loss of durability: not possible, data loss: not possible
> LOG_ONLY: loss of durability: possible only if OS/power fails, data loss:
> possible only if OS/power fails
> BACKGROUND: loss of durability: possible if Ignite process fails, data
> loss: possible only if OS/power fails
>
> The data loss situation can be mitigated in the cluster using a large
> enough replication factor (this is what Dmitriy was describing in the case
> of LOG_ONLY and 3 backups configuration).
>
> Denis,
> I do not think it is fair to compare Ignite defaults to Cassandra's
> defaults because Cassandra is _not_ transactional _eventually consistent_
> datastore, they claim much weaker guarantees than Ignite.
>
> All in all, I'm ok to change the WAL default right now, but I would revisit
> all those settings in 3.0 and made Ignite safe-first.
>
> 2018-02-17 3:24 GMT+03:00 Denis Magda <[hidden email]>:
>
> > Classic relational databases have no choice rather than to use FSYNC by
> > default. RDBMS is all about consistency.
> >
> > Distributed databases try to balance consistency and performance. For
> > instance, why to fsync every update if there is usually 1 backup copy?
> > This is probably why VoltDB [1] and Cassandra use the modes comparable to
> > Ignite's LOG_ONLY.
> >
> > Ignite as a distributed database should care of both consistency and
> > performance.
> >
> > My vote goes to FSYNC, LOG_ONLY (default), BACKGROUND, NONE.
> >
> >
> > [1] https://docs.voltdb.com/UsingVoltDB/CmdLogConfig.php
> >
> > --
> > Denis
> >
> >
> > On Fri, Feb 16, 2018 at 2:14 PM, Dmitriy Setrakyan <
> [hidden email]>
> > wrote:
> >
> > > Vova,
> > >
> > > I hear your concerns, but at the same time I know that one of the
> largest
> > > banks in eastern Europe is using Ignite in LOG_ONLY mode with 3 backups
> > to
> > > move money. The rational is that the probability of failure of 4
> servers
> > at
> > > hardware level at the same time is very low. However, if the JVM
> process
> > > fails on any server, then it can be safely restarted without loosing
> > data.
> > > In my view, this is why LOG_ONLY mode makes sense as a default.
> > >
> > > I still vote to change the default to LOG_ONLY, deprecate the DEFAULT
> > name
> > > altogether and add FSYNC mode instead.
> > >
> > > D.
> > >
> > > On Fri, Feb 16, 2018 at 4:05 PM, Vladimir Ozerov <[hidden email]
> >
> > > wrote:
> > >
> > > > Sergey,
> > > >
> > > > We do not have backups by default either, so essentially we are
> loosing
> > > > data by default. Moreover, backups are less reliable option than
> fsync
> > > > because a lot of users cannot afford putting servers into separate
> > power
> > > > circuits, so a single power failure may easily lead to poweroff of
> the
> > > > whole cluster at once, so data is lost still. This is normal practice
> > > even
> > > > for enterprise deployments (e.g. asynchronous replication).
> > > >
> > > > To make things even worse, we employ PRIMARY_SYNC mode by default! So
> > > even
> > > > if you configured backups, you still may loose data due to a single
> > node
> > > > failure - just shutdown the PRIMARY after commit is confirmed to the
> > > client
> > > > and your recent update will disappers.
> > > >
> > > > So this is what user should do to make himself safe:
> > > > 1) Learn about WAL modes
> > > > 2) Learn about backups
> > > > 3) Learn about synchronization modes
> > > > 4) Cross his fingers that he understood everything correctly and that
> > > there
> > > > are no other hidden surprises in Ignite which could lead to data
> loss.
> > > >
> > > > Way to much for a product, claiming to be A*C*ID and persistent,
> don't
> > > you
> > > > think so?
> > > >
> > > > Leaving deafult WAL mode with fsync resolves all these issues.
> > > >
> > > > Vladimir.
> > > >
> > > >
> > > > On Fri, Feb 16, 2018 at 11:43 PM, Sergey Kozlov <
> [hidden email]>
> > > > wrote:
> > > >
> > > > > I suppose some approaches used by classic databases makes no sense
> > for
> > > > > Ignite. FSYNC requirement for databases has the nature of single
> host
> > > > > solution. If you have corrupted db files you have corrupted (lost)
> > > data.
> > > > >
> > > > > For Ignite the enough number of backups and the failure detecting
> > logic
> > > > can
> > > > > provide the data consistency in term "cluster data consistency".
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Feb 16, 2018 at 8:57 PM, Dmitry Pavlov <
> > [hidden email]>
> > > > > wrote:
> > > > >
> > > > > > Hi, all WAL modes except NONE protects from data consistency
> > problem
> > > > > > (B+Tree, pages, etc), which is why I suggest to avoid saying
> > > > 'corrupted'
> > > > > > about 'unapplied updates'.
> > > > > >
> > > > > > Log Only and Background may cause unapplied updates in case of
> > > > OS/process
> > > > > > failures.
> > > > > >
> > > > > > None mode IMO is not an option in case data consistency is
> needed.
> > > > > >
> > > > > > пт, 16 февр. 2018 г. в 20:49, Valentin Kulichenko <
> > > > > > [hidden email]>:
> > > > > >
> > > > > > > Guys,
> > > > > > >
> > > > > > > While we're on this topic, what is the difference between
> > > BACKGROUND
> > > > > and
> > > > > > > NONE in terms of semantics and provided guarantees? To me it
> > looks
> > > > like
> > > > > > > both guarantee to recover the state since last checkpoint and
> > > > anything
> > > > > > else
> > > > > > > can potentially be lost, so from user perspective they are the
> > > same.
> > > > > Am I
> > > > > > > missing something here?
> > > > > > >
> > > > > > > Also there is the following in Javadoc for NONE: "If an Ignite
> > node
> > > > is
> > > > > > > terminated in NONE mode abruptly, it is likely that the data
> > stored
> > > > on
> > > > > > disk
> > > > > > > is corrupted and work directory will need to be cleared for a
> > node
> > > > > > > restart.". If this is really the case, I'm not sure NONE makes
> > > sense
> > > > at
> > > > > > > all. Why would I enable persistence if I'm likely to clear the
> > > > storage
> > > > > on
> > > > > > > restart?
> > > > > > >
> > > > > > > -Val
> > > > > > >
> > > > > > > On Fri, Feb 16, 2018 at 8:39 AM, Vladimir Ozerov <
> > > > [hidden email]
> > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > What is the reason to have DEFAULT mode at all if you claim
> > > > LOG_ONLY
> > > > > to
> > > > > > > be
> > > > > > > > completely safe? :)
> > > > > > > >
> > > > > > > > And how it could be safe provided that without fsync we loose
> > > part
> > > > of
> > > > > > WAL
> > > > > > > > itself in case of crash?
> > > > > > > >
> > > > > > > > пт, 16 февр. 2018 г. в 19:32, Dmitry Pavlov <
> > > [hidden email]
> > > > >:
> > > > > > > >
> > > > > > > > > Thank you. Data can't be corrupted in case crash because of
> > WAL
> > > > > > replay
> > > > > > > > > (since completed checkpoint). Physical records are used to
> > > > restore
> > > > > > > > probably
> > > > > > > > > corrupted pages in persistent store (we overwrite so called
> > > 'grey
> > > > > > > zone' -
> > > > > > > > > pages we don't know for sure if they have been written).
> > > > > > > > >
> > > > > > > > > Only one effect is unwritten one or several last
> > transactions.
> > > It
> > > > > is
> > > > > > > not
> > > > > > > > > the same with corrupted data.
> > > > > > > > >
> > > > > > > > > пт, 16 февр. 2018 г. в 19:19, Vladimir Ozerov <
> > > > > [hidden email]
> > > > > > >:
> > > > > > > > >
> > > > > > > > > > Log only mode is not safe - data might be corrupted in
> case
> > > of
> > > > > > system
> > > > > > > > > > crash. Oracle - fsync, Postgres - fsync, SQL Server -
> > fsync,
> > > > > > > Cassandra
> > > > > > > > -
> > > > > > > > > > similar to our “background”.
> > > > > > > > > >
> > > > > > > > > > пт, 16 февр. 2018 г. в 19:11, Dmitry Pavlov <
> > > > > [hidden email]
> > > > > > >:
> > > > > > > > > >
> > > > > > > > > > > Hi Vladimir,
> > > > > > > > > > >
> > > > > > > > > > > What you saying is defenetely make sence.
> > > > > > > > > > >
> > > > > > > > > > > In the same time LOG_ONLY is also safe mode, user will
> be
> > > > able
> > > > > to
> > > > > > > > > restore
> > > > > > > > > > > system after crash. If it is not true, we should create
> > > > > critical
> > > > > > > > ticket
> > > > > > > > > > and
> > > > > > > > > > > fix it.
> > > > > > > > > > >
> > > > > > > > > > > Do you know other databases defaults, such as
> Cassandra,
> > > > > Oracle,
> > > > > > > > > Postgre?
> > > > > > > > > > >
> > > > > > > > > > > Sincerely,
> > > > > > > > > > > Dmitriy Pavlov
> > > > > > > > > > >
> > > > > > > > > > > пт, 16 февр. 2018 г. в 18:41, Vladimir Ozerov <
> > > > > > > [hidden email]
> > > > > > > > >:
> > > > > > > > > > >
> > > > > > > > > > > > Igniters,
> > > > > > > > > > > >
> > > > > > > > > > > > Sorry for pouring oil on the flames, but from
> database
> > > > > > > perspective
> > > > > > > > > > moving
> > > > > > > > > > > > from FSYNC to non-FSYNC mode appears to be a mistake.
> > > When
> > > > > you
> > > > > > > work
> > > > > > > > > > with
> > > > > > > > > > > > database, your main expectation is that it will save
> > your
> > > > > data.
> > > > > > > All
> > > > > > > > > > > > production database vendor make sure that you are
> safe,
> > > not
> > > > > > that
> > > > > > > > you
> > > > > > > > > > are
> > > > > > > > > > > > fast. Moreover, some vendors even prevent you from
> > being
> > > in
> > > > > > > unsafe
> > > > > > > > > mode
> > > > > > > > > > > > (e.g. you cannot disable fsync in SQL Server at all).
> > > > > > > > > > > >
> > > > > > > > > > > > If we continue going in this direction, we will end
> up
> > > > with a
> > > > > > > > > product,
> > > > > > > > > > > > which is unsafe out of the box and require tons of
> > > > > > documentation
> > > > > > > to
> > > > > > > > > > > > understand how to make it safe. Definitely not the
> > right
> > > > > > message
> > > > > > > to
> > > > > > > > > the
> > > > > > > > > > > > market. This is like a car without brakes - would you
> > > like
> > > > to
> > > > > > > drive
> > > > > > > > > it?
> > > > > > > > > > > If
> > > > > > > > > > > > this is Need For Speed game and you have unlimited
> > lives
> > > > > > > (in-memory
> > > > > > > > > > cache
> > > > > > > > > > > > with backing store), then yes. If this is a real life
> > > with
> > > > > > > > > > (persistence)
> > > > > > > > > > > -
> > > > > > > > > > > > then no.
> > > > > > > > > > > >
> > > > > > > > > > > > On Fri, Feb 16, 2018 at 5:20 PM, Dmitriy Setrakyan <
> > > > > > > > > > > [hidden email]>
> > > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Well, I cannot say that I like the name LOG_ONLY,
> > but I
> > > > > would
> > > > > > > > vote
> > > > > > > > > to
> > > > > > > > > > > > keep
> > > > > > > > > > > > > it for now, given that it is already documented in
> > many
> > > > > > places,
> > > > > > > > > > blogs,
> > > > > > > > > > > > and
> > > > > > > > > > > > > examples.
> > > > > > > > > > > > >
> > > > > > > > > > > > > D.
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Fri, Feb 16, 2018 at 8:13 AM, Ivan Rakov <
> > > > > > > > [hidden email]
> > > > > > > > > >
> > > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Looks like it's an Ignite term - I've never heard
> > of
> > > it
> > > > > > > outside
> > > > > > > > > > > Ignite
> > > > > > > > > > > > > > scope.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Though, renaming existing enum value requires
> > keeping
> > > > old
> > > > > > as
> > > > > > > > > > > > deprecated.
> > > > > > > > > > > > > > DEFAULT is confusing enough to pay this price.
> > > > > > > > > > > > > > As for LOG_ONLY, I think we can keep it as long
> as
> > it
> > > > has
> > > > > > > good
> > > > > > > > > and
> > > > > > > > > > > > > > definitive javadoc.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Best Regards,
> > > > > > > > > > > > > > Ivan Rakov
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On 16.02.2018 17:07, Dmitriy Setrakyan wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >> Igniters, just to clarify, does the term
> LOG_ONLY
> > > mean
> > > > > > > > anything
> > > > > > > > > in
> > > > > > > > > > > the
> > > > > > > > > > > > > >> industry or is this just an Ignite term?
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> D.
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> On Fri, Feb 16, 2018 at 8:03 AM, Anton
> Vinogradov
> > <
> > > > > > > > > > > > > >> [hidden email]>
> > > > > > > > > > > > > >> wrote:
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> Log only mode: flushes application buffers.
> > > > > > > > > > > > > >>> So, in synced mode without fsync guarantee.
> > That's
> > > > why
> > > > > I
> > > > > > > > > propose
> > > > > > > > > > to
> > > > > > > > > > > > > >>> rename
> > > > > > > > > > > > > >>> it as SYNC.
> > > > > > > > > > > > > >>>
> > > > > > > > > > > > > >>> On Fri, Feb 16, 2018 at 4:49 PM, Ilya Lantukh <
> > > > > > > > > > > [hidden email]
> > > > > > > > > > > > >
> > > > > > > > > > > > > >>> wrote:
> > > > > > > > > > > > > >>>
> > > > > > > > > > > > > >>> I am OK with either FSYNC or STRICT variant.
> > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > >>>> LOG_ONLY name means "log without fsync".
> > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > >>>> On Fri, Feb 16, 2018 at 4:05 PM, Dmitriy
> > > Setrakyan <
> > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > >>> [hidden email]>
> > > > > > > > > > > > > >>>
> > > > > > > > > > > > > >>>> wrote:
> > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > >>>> On Fri, Feb 16, 2018 at 7:02 AM, Ivan Rakov <
> > > > > > > > > > > [hidden email]>
> > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > >>>> wrote:
> > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > >>>>> Why create a new term to define something
> that
> > > has
> > > > > > > already
> > > > > > > > > been
> > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > >>>>> defined?
> > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > >>>>> That makes sense. I'm ok with FSYNC.
> > > > > > > > > > > > > >>>>>> Anton, I don't understand why we should
> rename
> > > > > > LOG_ONLY
> > > > > > > to
> > > > > > > > > > SYNC.
> > > > > > > > > > > > We
> > > > > > > > > > > > > >>>>>> started this discussion with bad naming of
> > > > DEFAULT,
> > > > > > but
> > > > > > > > this
> > > > > > > > > > has
> > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > >>>>> nothing
> > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > >>>>> to
> > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > >>>>>> do with LOG_ONLY (even though it may be
> > > > scientific -
> > > > > > but
> > > > > > > > > SYNC
> > > > > > > > > > > > sounds
> > > > > > > > > > > > > >>>>>> scientific as well).
> > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > >>>>>> I agree with Ivan, we should not go wild
> with
> > > > > > renaming.
> > > > > > > > > > > However, I
> > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > >>>> would
> > > > > > > > > > > > > >>>
> > > > > > > > > > > > > >>>> like to find out what is the meaning behind
> the
> > > > > LOG_ONLY
> > > > > > > > name.
> > > > > > > > > > Can
> > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > >>>> someone
> > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > >>>>> explain?
> > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > >>>>> D.
> > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > >>>> --
> > > > > > > > > > > > > >>>> Best regards,
> > > > > > > > > > > > > >>>> Ilya
> > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Sergey Kozlov
> > > > > GridGain Systems
> > > > > www.gridgain.com
> > > > >
> > > >
> > >
> >
>

dmagda

Re: Apache Ignite 2.4 release

In reply to this post by Alexey Goncharuk

Alex, please clarify some of the points

data loss because the disk content was shuffled due to an incomplete page
> write

are you talking about some phase of the checkpointing process? Do you mean
that the process just passed over changes to the file but didn't call
fsync, thus, offloading this to OS to decide?

BACKGROUND: loss of durability: possible if Ignite process fails, data
> loss: possible only if OS/power fails

This should be true if mmap files are not used for BACKGROUND, otherwise,
it provides the same guarantees as LOG_ONLY, right?

--
Denis

On Sun, Feb 18, 2018 at 11:06 PM, Alexey Goncharuk <
[hidden email]> wrote:

dmagda

Re: Apache Ignite 2.4 release

In reply to this post by Vladimir Ozerov

>
> Neither Cassandra, nor Mongo or any others claim themselves
> to be ACID, so it is not valid to refer to their defaults.

This is why I referred to VoltDB as to an example. It supports ACID
transactions and persists changes to disk in a mode similar to LOG_ONLY.

--
Denis

On Sun, Feb 18, 2018 at 11:37 PM, Vladimir Ozerov <[hidden email]>
wrote:

> Alex,
>
> You get me right. DEFAULT -> LOG_ONLY doesn't introduce any dramatic
> changes when comparing 2.3 to 2.4 - Ignite was unsafe out of the box in
> 2.3, and it is unsafe in 2.4 as well.
>
> The very problem is that we claim ourselves to be ACID, while in reality we
> are only "AI" out of the box, because durability is not guaranteed due to
> zero backups and LOG_ONLY and consistency is not guaranteed due to
> PRIMARY_SYNC. Neither Cassandra, nor Mongo or any others claim themselves
> to be ACID, so it is not valid to refer to their defaults.
>
> On Mon, Feb 19, 2018 at 10:06 AM, Alexey Goncharuk <
> [hidden email]> wrote:
>
> > In terms of 'safety', Ignite default settings are far beyond optimal. For
> > in-memory mode, we have 0 backups by default, which means partition loss
> in
> > a case of node failure, we have readFromBackup=true and PRIMARY_SYNC by
> > default which effectively cancels linearizability property for cache
> > updates, so setting the default WAL mode to LOG_ONLY does not seem to be
> a
> > bigger evil than it currently is. If we are to move to safer defaults, we
> > should change all of the affected sides.
> >
> > I also want to clarify the difference between guarantees in
> > non-fsync modes. We should distinguish the loss of durability (the loss
> of
> > the last update) because the update did not make it to disk and data loss
> > because the disk content was shuffled due to an incomplete page write. In
> > my understanding, the current situation is:
> > FSYNC: loss of durability: not possible, data loss: not possible
> > LOG_ONLY: loss of durability: possible only if OS/power fails, data loss:
> > possible only if OS/power fails
> > BACKGROUND: loss of durability: possible if Ignite process fails, data
> > loss: possible only if OS/power fails
> >
> > The data loss situation can be mitigated in the cluster using a large
> > enough replication factor (this is what Dmitriy was describing in the
> case
> > of LOG_ONLY and 3 backups configuration).
> >
> > Denis,
> > I do not think it is fair to compare Ignite defaults to Cassandra's
> > defaults because Cassandra is _not_ transactional _eventually consistent_
> > datastore, they claim much weaker guarantees than Ignite.
> >
> > All in all, I'm ok to change the WAL default right now, but I would
> revisit
> > all those settings in 3.0 and made Ignite safe-first.
> >
> > 2018-02-17 3:24 GMT+03:00 Denis Magda <[hidden email]>:
> >
> > > Classic relational databases have no choice rather than to use FSYNC by
> > > default. RDBMS is all about consistency.
> > >
> > > Distributed databases try to balance consistency and performance. For
> > > instance, why to fsync every update if there is usually 1 backup copy?
> > > This is probably why VoltDB [1] and Cassandra use the modes comparable
> to
> > > Ignite's LOG_ONLY.
> > >
> > > Ignite as a distributed database should care of both consistency and
> > > performance.
> > >
> > > My vote goes to FSYNC, LOG_ONLY (default), BACKGROUND, NONE.
> > >
> > >
> > > [1] https://docs.voltdb.com/UsingVoltDB/CmdLogConfig.php
> > >
> > > --
> > > Denis
> > >
> > >
> > > On Fri, Feb 16, 2018 at 2:14 PM, Dmitriy Setrakyan <
> > [hidden email]>
> > > wrote:
> > >
> > > > Vova,
> > > >
> > > > I hear your concerns, but at the same time I know that one of the
> > largest
> > > > banks in eastern Europe is using Ignite in LOG_ONLY mode with 3
> backups
> > > to
> > > > move money. The rational is that the probability of failure of 4
> > servers
> > > at
> > > > hardware level at the same time is very low. However, if the JVM
> > process
> > > > fails on any server, then it can be safely restarted without loosing
> > > data.
> > > > In my view, this is why LOG_ONLY mode makes sense as a default.
> > > >
> > > > I still vote to change the default to LOG_ONLY, deprecate the DEFAULT
> > > name
> > > > altogether and add FSYNC mode instead.
> > > >
> > > > D.
> > > >
> > > > On Fri, Feb 16, 2018 at 4:05 PM, Vladimir Ozerov <
> [hidden email]
> > >
> > > > wrote:
> > > >
> > > > > Sergey,
> > > > >
> > > > > We do not have backups by default either, so essentially we are
> > loosing
> > > > > data by default. Moreover, backups are less reliable option than
> > fsync
> > > > > because a lot of users cannot afford putting servers into separate
> > > power
> > > > > circuits, so a single power failure may easily lead to poweroff of
> > the
> > > > > whole cluster at once, so data is lost still. This is normal
> practice
> > > > even
> > > > > for enterprise deployments (e.g. asynchronous replication).
> > > > >
> > > > > To make things even worse, we employ PRIMARY_SYNC mode by default!
> So
> > > > even
> > > > > if you configured backups, you still may loose data due to a single
> > > node
> > > > > failure - just shutdown the PRIMARY after commit is confirmed to
> the
> > > > client
> > > > > and your recent update will disappers.
> > > > >
> > > > > So this is what user should do to make himself safe:
> > > > > 1) Learn about WAL modes
> > > > > 2) Learn about backups
> > > > > 3) Learn about synchronization modes
> > > > > 4) Cross his fingers that he understood everything correctly and
> that
> > > > there
> > > > > are no other hidden surprises in Ignite which could lead to data
> > loss.
> > > > >
> > > > > Way to much for a product, claiming to be A*C*ID and persistent,
> > don't
> > > > you
> > > > > think so?
> > > > >
> > > > > Leaving deafult WAL mode with fsync resolves all these issues.
> > > > >
> > > > > Vladimir.
> > > > >
> > > > >
> > > > > On Fri, Feb 16, 2018 at 11:43 PM, Sergey Kozlov <
> > [hidden email]>
> > > > > wrote:
> > > > >
> > > > > > I suppose some approaches used by classic databases makes no
> sense
> > > for
> > > > > > Ignite. FSYNC requirement for databases has the nature of single
> > host
> > > > > > solution. If you have corrupted db files you have corrupted
> (lost)
> > > > data.
> > > > > >
> > > > > > For Ignite the enough number of backups and the failure detecting
> > > logic
> > > > > can
> > > > > > provide the data consistency in term "cluster data consistency".
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Fri, Feb 16, 2018 at 8:57 PM, Dmitry Pavlov <
> > > [hidden email]>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi, all WAL modes except NONE protects from data consistency
> > > problem
> > > > > > > (B+Tree, pages, etc), which is why I suggest to avoid saying
> > > > > 'corrupted'
> > > > > > > about 'unapplied updates'.
> > > > > > >
> > > > > > > Log Only and Background may cause unapplied updates in case of
> > > > > OS/process
> > > > > > > failures.
> > > > > > >
> > > > > > > None mode IMO is not an option in case data consistency is
> > needed.
> > > > > > >
> > > > > > > пт, 16 февр. 2018 г. в 20:49, Valentin Kulichenko <
> > > > > > > [hidden email]>:
> > > > > > >
> > > > > > > > Guys,
> > > > > > > >
> > > > > > > > While we're on this topic, what is the difference between
> > > > BACKGROUND
> > > > > > and
> > > > > > > > NONE in terms of semantics and provided guarantees? To me it
> > > looks
> > > > > like
> > > > > > > > both guarantee to recover the state since last checkpoint and
> > > > > anything
> > > > > > > else
> > > > > > > > can potentially be lost, so from user perspective they are
> the
> > > > same.
> > > > > > Am I
> > > > > > > > missing something here?
> > > > > > > >
> > > > > > > > Also there is the following in Javadoc for NONE: "If an
> Ignite
> > > node
> > > > > is
> > > > > > > > terminated in NONE mode abruptly, it is likely that the data
> > > stored
> > > > > on
> > > > > > > disk
> > > > > > > > is corrupted and work directory will need to be cleared for a
> > > node
> > > > > > > > restart.". If this is really the case, I'm not sure NONE
> makes
> > > > sense
> > > > > at
> > > > > > > > all. Why would I enable persistence if I'm likely to clear
> the
> > > > > storage
> > > > > > on
> > > > > > > > restart?
> > > > > > > >
> > > > > > > > -Val
> > > > > > > >
> > > > > > > > On Fri, Feb 16, 2018 at 8:39 AM, Vladimir Ozerov <
> > > > > [hidden email]
> > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > What is the reason to have DEFAULT mode at all if you claim
> > > > > LOG_ONLY
> > > > > > to
> > > > > > > > be
> > > > > > > > > completely safe? :)
> > > > > > > > >
> > > > > > > > > And how it could be safe provided that without fsync we
> loose
> > > > part
> > > > > of
> > > > > > > WAL
> > > > > > > > > itself in case of crash?
> > > > > > > > >
> > > > > > > > > пт, 16 февр. 2018 г. в 19:32, Dmitry Pavlov <
> > > > [hidden email]
> > > > > >:
> > > > > > > > >
> > > > > > > > > > Thank you. Data can't be corrupted in case crash because
> of
> > > WAL
> > > > > > > replay
> > > > > > > > > > (since completed checkpoint). Physical records are used
> to
> > > > > restore
> > > > > > > > > probably
> > > > > > > > > > corrupted pages in persistent store (we overwrite so
> called
> > > > 'grey
> > > > > > > > zone' -
> > > > > > > > > > pages we don't know for sure if they have been written).
> > > > > > > > > >
> > > > > > > > > > Only one effect is unwritten one or several last
> > > transactions.
> > > > It
> > > > > > is
> > > > > > > > not
> > > > > > > > > > the same with corrupted data.
> > > > > > > > > >
> > > > > > > > > > пт, 16 февр. 2018 г. в 19:19, Vladimir Ozerov <
> > > > > > [hidden email]
> > > > > > > >:
> > > > > > > > > >
> > > > > > > > > > > Log only mode is not safe - data might be corrupted in
> > case
> > > > of
> > > > > > > system
> > > > > > > > > > > crash. Oracle - fsync, Postgres - fsync, SQL Server -
> > > fsync,
> > > > > > > > Cassandra
> > > > > > > > > -
> > > > > > > > > > > similar to our “background”.
> > > > > > > > > > >
> > > > > > > > > > > пт, 16 февр. 2018 г. в 19:11, Dmitry Pavlov <
> > > > > > [hidden email]
> > > > > > > >:
> > > > > > > > > > >
> > > > > > > > > > > > Hi Vladimir,
> > > > > > > > > > > >
> > > > > > > > > > > > What you saying is defenetely make sence.
> > > > > > > > > > > >
> > > > > > > > > > > > In the same time LOG_ONLY is also safe mode, user
> will
> > be
> > > > > able
> > > > > > to
> > > > > > > > > > restore
> > > > > > > > > > > > system after crash. If it is not true, we should
> create
> > > > > > critical
> > > > > > > > > ticket
> > > > > > > > > > > and
> > > > > > > > > > > > fix it.
> > > > > > > > > > > >
> > > > > > > > > > > > Do you know other databases defaults, such as
> > Cassandra,
> > > > > > Oracle,
> > > > > > > > > > Postgre?
> > > > > > > > > > > >
> > > > > > > > > > > > Sincerely,
> > > > > > > > > > > > Dmitriy Pavlov
> > > > > > > > > > > >
> > > > > > > > > > > > пт, 16 февр. 2018 г. в 18:41, Vladimir Ozerov <
> > > > > > > > [hidden email]
> > > > > > > > > >:
> > > > > > > > > > > >
> > > > > > > > > > > > > Igniters,
> > > > > > > > > > > > >
> > > > > > > > > > > > > Sorry for pouring oil on the flames, but from
> > database
> > > > > > > > perspective
> > > > > > > > > > > moving
> > > > > > > > > > > > > from FSYNC to non-FSYNC mode appears to be a
> mistake.
> > > > When
> > > > > > you
> > > > > > > > work
> > > > > > > > > > > with
> > > > > > > > > > > > > database, your main expectation is that it will
> save
> > > your
> > > > > > data.
> > > > > > > > All
> > > > > > > > > > > > > production database vendor make sure that you are
> > safe,
> > > > not
> > > > > > > that
> > > > > > > > > you
> > > > > > > > > > > are
> > > > > > > > > > > > > fast. Moreover, some vendors even prevent you from
> > > being
> > > > in
> > > > > > > > unsafe
> > > > > > > > > > mode
> > > > > > > > > > > > > (e.g. you cannot disable fsync in SQL Server at
> all).
> > > > > > > > > > > > >
> > > > > > > > > > > > > If we continue going in this direction, we will end
> > up
> > > > > with a
> > > > > > > > > > product,
> > > > > > > > > > > > > which is unsafe out of the box and require tons of
> > > > > > > documentation
> > > > > > > > to
> > > > > > > > > > > > > understand how to make it safe. Definitely not the
> > > right
> > > > > > > message
> > > > > > > > to
> > > > > > > > > > the
> > > > > > > > > > > > > market. This is like a car without brakes - would
> you
> > > > like
> > > > > to
> > > > > > > > drive
> > > > > > > > > > it?
> > > > > > > > > > > > If
> > > > > > > > > > > > > this is Need For Speed game and you have unlimited
> > > lives
> > > > > > > > (in-memory
> > > > > > > > > > > cache
> > > > > > > > > > > > > with backing store), then yes. If this is a real
> life
> > > > with
> > > > > > > > > > > (persistence)
> > > > > > > > > > > > -
> > > > > > > > > > > > > then no.
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Fri, Feb 16, 2018 at 5:20 PM, Dmitriy Setrakyan
> <
> > > > > > > > > > > > [hidden email]>
> > > > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Well, I cannot say that I like the name LOG_ONLY,
> > > but I
> > > > > > would
> > > > > > > > > vote
> > > > > > > > > > to
> > > > > > > > > > > > > keep
> > > > > > > > > > > > > > it for now, given that it is already documented
> in
> > > many
> > > > > > > places,
> > > > > > > > > > > blogs,
> > > > > > > > > > > > > and
> > > > > > > > > > > > > > examples.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > D.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Fri, Feb 16, 2018 at 8:13 AM, Ivan Rakov <
> > > > > > > > > [hidden email]
> > > > > > > > > > >
> > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Looks like it's an Ignite term - I've never
> heard
> > > of
> > > > it
> > > > > > > > outside
> > > > > > > > > > > > Ignite
> > > > > > > > > > > > > > > scope.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Though, renaming existing enum value requires
> > > keeping
> > > > > old
> > > > > > > as
> > > > > > > > > > > > > deprecated.
> > > > > > > > > > > > > > > DEFAULT is confusing enough to pay this price.
> > > > > > > > > > > > > > > As for LOG_ONLY, I think we can keep it as long
> > as
> > > it
> > > > > has
> > > > > > > > good
> > > > > > > > > > and
> > > > > > > > > > > > > > > definitive javadoc.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Best Regards,
> > > > > > > > > > > > > > > Ivan Rakov
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On 16.02.2018 17:07, Dmitriy Setrakyan wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >> Igniters, just to clarify, does the term
> > LOG_ONLY
> > > > mean
> > > > > > > > > anything
> > > > > > > > > > in
> > > > > > > > > > > > the
> > > > > > > > > > > > > > >> industry or is this just an Ignite term?
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> D.
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> On Fri, Feb 16, 2018 at 8:03 AM, Anton
> > Vinogradov
> > > <
> > > > > > > > > > > > > > >> [hidden email]>
> > > > > > > > > > > > > > >> wrote:
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> Log only mode: flushes application buffers.
> > > > > > > > > > > > > > >>> So, in synced mode without fsync guarantee.
> > > That's
> > > > > why
> > > > > > I
> > > > > > > > > > propose
> > > > > > > > > > > to
> > > > > > > > > > > > > > >>> rename
> > > > > > > > > > > > > > >>> it as SYNC.
> > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > >>> On Fri, Feb 16, 2018 at 4:49 PM, Ilya
> Lantukh <
> > > > > > > > > > > > [hidden email]
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >>> wrote:
> > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > >>> I am OK with either FSYNC or STRICT variant.
> > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > >>>> LOG_ONLY name means "log without fsync".
> > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > >>>> On Fri, Feb 16, 2018 at 4:05 PM, Dmitriy
> > > > Setrakyan <
> > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > >>> [hidden email]>
> > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > >>>> wrote:
> > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > >>>> On Fri, Feb 16, 2018 at 7:02 AM, Ivan Rakov
> <
> > > > > > > > > > > > [hidden email]>
> > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > >>>> wrote:
> > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > >>>>> Why create a new term to define something
> > that
> > > > has
> > > > > > > > already
> > > > > > > > > > been
> > > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > > >>>>> defined?
> > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > >>>>> That makes sense. I'm ok with FSYNC.
> > > > > > > > > > > > > > >>>>>> Anton, I don't understand why we should
> > rename
> > > > > > > LOG_ONLY
> > > > > > > > to
> > > > > > > > > > > SYNC.
> > > > > > > > > > > > > We
> > > > > > > > > > > > > > >>>>>> started this discussion with bad naming of
> > > > > DEFAULT,
> > > > > > > but
> > > > > > > > > this
> > > > > > > > > > > has
> > > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > > >>>>> nothing
> > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > >>>>> to
> > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > >>>>>> do with LOG_ONLY (even though it may be
> > > > > scientific -
> > > > > > > but
> > > > > > > > > > SYNC
> > > > > > > > > > > > > sounds
> > > > > > > > > > > > > > >>>>>> scientific as well).
> > > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > > >>>>>> I agree with Ivan, we should not go wild
> > with
> > > > > > > renaming.
> > > > > > > > > > > > However, I
> > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > >>>> would
> > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > >>>> like to find out what is the meaning behind
> > the
> > > > > > LOG_ONLY
> > > > > > > > > name.
> > > > > > > > > > > Can
> > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > >>>> someone
> > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > >>>>> explain?
> > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > >>>>> D.
> > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > >>>> --
> > > > > > > > > > > > > > >>>> Best regards,
> > > > > > > > > > > > > > >>>> Ilya
> > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Sergey Kozlov
> > > > > > GridGain Systems
> > > > > > www.gridgain.com
> > > > > >
> > > > >
> > > >
> > >
> >
>

dsetrakyan

Re: Apache Ignite 2.4 release

In reply to this post by Vladimir Ozerov

On Mon, Feb 19, 2018 at 1:37 AM, Vladimir Ozerov <[hidden email]>
wrote:

Vladimir,
Ignite can be fully ACID, but at the same time have non-ACID defaults, as
long as we clearly document how to get ACID behavior. I do not see an issue
with it.

D.

Pavel Tupitsyn

Re: Apache Ignite 2.4 release

Igniters,

Any update on the 2.4 release status? Anything else to merge there?

Pavel

On Mon, Feb 19, 2018 at 9:30 PM, Dmitriy Setrakyan <[hidden email]>
wrote:

> On Mon, Feb 19, 2018 at 1:37 AM, Vladimir Ozerov <[hidden email]>
> wrote:
>
> > Alex,
> >
> > You get me right. DEFAULT -> LOG_ONLY doesn't introduce any dramatic
> > changes when comparing 2.3 to 2.4 - Ignite was unsafe out of the box in
> > 2.3, and it is unsafe in 2.4 as well.
> >
> > The very problem is that we claim ourselves to be ACID, while in reality
> we
> > are only "AI" out of the box, because durability is not guaranteed due to
> > zero backups and LOG_ONLY and consistency is not guaranteed due to
> > PRIMARY_SYNC. Neither Cassandra, nor Mongo or any others claim themselves
> > to be ACID, so it is not valid to refer to their defaults.
> >
>
> Vladimir,
> Ignite can be fully ACID, but at the same time have non-ACID defaults, as
> long as we clearly document how to get ACID behavior. I do not see an issue
> with it.
>
> D.
>

dmagda

Re: Apache Ignite 2.4 release

Good question!

We've been waiting for a sign-off from Alexey Goncharuk and Sergey Kozlov
who are benchmarking and trying to address performance issues in
coopeartion.

Guys, please share your results and forecast.

--
Denis

On Sun, Feb 25, 2018 at 12:10 AM, Pavel Tupitsyn <[hidden email]>
wrote:

> Igniters,
>
> Any update on the 2.4 release status? Anything else to merge there?
>
> Pavel
>
> On Mon, Feb 19, 2018 at 9:30 PM, Dmitriy Setrakyan <[hidden email]>
> wrote:
>
> > On Mon, Feb 19, 2018 at 1:37 AM, Vladimir Ozerov <[hidden email]>
> > wrote:
> >
> > > Alex,
> > >
> > > You get me right. DEFAULT -> LOG_ONLY doesn't introduce any dramatic
> > > changes when comparing 2.3 to 2.4 - Ignite was unsafe out of the box in
> > > 2.3, and it is unsafe in 2.4 as well.
> > >
> > > The very problem is that we claim ourselves to be ACID, while in
> reality
> > we
> > > are only "AI" out of the box, because durability is not guaranteed due
> to
> > > zero backups and LOG_ONLY and consistency is not guaranteed due to
> > > PRIMARY_SYNC. Neither Cassandra, nor Mongo or any others claim
> themselves
> > > to be ACID, so it is not valid to refer to their defaults.
> > >
> >
> > Vladimir,
> > Ignite can be fully ACID, but at the same time have non-ACID defaults, as
> > long as we clearly document how to get ACID behavior. I do not see an
> issue
> > with it.
> >
> > D.
> >
>

Vladimir Ozerov

Re: Apache Ignite 2.4 release

Hi,

We found one nasty regression in DROP COLUMN feature and fixed it today.
Another problem is performance - we found that recent changes to TCP
communication metrics caused considerable slowdown in in-memory mode. I
almost fixed it and will merge the fix in the nearest day if benchmarks are
ok.

That said I think we will be able to start vote in the beginning of the
next week.

Any objections?

Vladimir.

On Mon, Feb 26, 2018 at 11:22 PM, Denis Magda <[hidden email]> wrote:

> Good question!
>
> We've been waiting for a sign-off from Alexey Goncharuk and Sergey Kozlov
> who are benchmarking and trying to address performance issues in
> coopeartion.
>
> Guys, please share your results and forecast.
>
> --
> Denis
>
> On Sun, Feb 25, 2018 at 12:10 AM, Pavel Tupitsyn <[hidden email]>
> wrote:
>
> > Igniters,
> >
> > Any update on the 2.4 release status? Anything else to merge there?
> >
> > Pavel
> >
> > On Mon, Feb 19, 2018 at 9:30 PM, Dmitriy Setrakyan <
> [hidden email]>
> > wrote:
> >
> > > On Mon, Feb 19, 2018 at 1:37 AM, Vladimir Ozerov <[hidden email]
> >
> > > wrote:
> > >
> > > > Alex,
> > > >
> > > > You get me right. DEFAULT -> LOG_ONLY doesn't introduce any dramatic
> > > > changes when comparing 2.3 to 2.4 - Ignite was unsafe out of the box
> in
> > > > 2.3, and it is unsafe in 2.4 as well.
> > > >
> > > > The very problem is that we claim ourselves to be ACID, while in
> > reality
> > > we
> > > > are only "AI" out of the box, because durability is not guaranteed
> due
> > to
> > > > zero backups and LOG_ONLY and consistency is not guaranteed due to
> > > > PRIMARY_SYNC. Neither Cassandra, nor Mongo or any others claim
> > themselves
> > > > to be ACID, so it is not valid to refer to their defaults.
> > > >
> > >
> > > Vladimir,
> > > Ignite can be fully ACID, but at the same time have non-ACID defaults,
> as
> > > long as we clearly document how to get ACID behavior. I do not see an
> > issue
> > > with it.
> > >
> > > D.
> > >
> >
>

dmagda

Re: Apache Ignite 2.4 release

Hi Vladimir,

The regression, as well as the performance, is definitely bad. Please work
with Sergey to see how much time he needs to rerun the whole test set if
you merge the changes.

I would target the end of the next week as a release date if the vote goes
well. Probably, we can run the test suites over the weekend if your changes
get to the branch.

Nikolay Izhikov, follow up with Vladimir and Sergey tomorrow. If they
decide to merge the fixes then you are free to merge your fixed for Spark
examples.

--
Denis

On Thu, Mar 1, 2018 at 7:46 AM, Vladimir Ozerov <[hidden email]>
wrote:

> Hi,
>
> We found one nasty regression in DROP COLUMN feature and fixed it today.
> Another problem is performance - we found that recent changes to TCP
> communication metrics caused considerable slowdown in in-memory mode. I
> almost fixed it and will merge the fix in the nearest day if benchmarks are
> ok.
>
> That said I think we will be able to start vote in the beginning of the
> next week.
>
> Any objections?
>
> Vladimir.
>
> On Mon, Feb 26, 2018 at 11:22 PM, Denis Magda <[hidden email]> wrote:
>
> > Good question!
> >
> > We've been waiting for a sign-off from Alexey Goncharuk and Sergey Kozlov
> > who are benchmarking and trying to address performance issues in
> > coopeartion.
> >
> > Guys, please share your results and forecast.
> >
> > --
> > Denis
> >
> > On Sun, Feb 25, 2018 at 12:10 AM, Pavel Tupitsyn <[hidden email]>
> > wrote:
> >
> > > Igniters,
> > >
> > > Any update on the 2.4 release status? Anything else to merge there?
> > >
> > > Pavel
> > >
> > > On Mon, Feb 19, 2018 at 9:30 PM, Dmitriy Setrakyan <
> > [hidden email]>
> > > wrote:
> > >
> > > > On Mon, Feb 19, 2018 at 1:37 AM, Vladimir Ozerov <
> [hidden email]
> > >
> > > > wrote:
> > > >
> > > > > Alex,
> > > > >
> > > > > You get me right. DEFAULT -> LOG_ONLY doesn't introduce any
> dramatic
> > > > > changes when comparing 2.3 to 2.4 - Ignite was unsafe out of the
> box
> > in
> > > > > 2.3, and it is unsafe in 2.4 as well.
> > > > >
> > > > > The very problem is that we claim ourselves to be ACID, while in
> > > reality
> > > > we
> > > > > are only "AI" out of the box, because durability is not guaranteed
> > due
> > > to
> > > > > zero backups and LOG_ONLY and consistency is not guaranteed due to
> > > > > PRIMARY_SYNC. Neither Cassandra, nor Mongo or any others claim
> > > themselves
> > > > > to be ACID, so it is not valid to refer to their defaults.
> > > > >
> > > >
> > > > Vladimir,
> > > > Ignite can be fully ACID, but at the same time have non-ACID
> defaults,
> > as
> > > > long as we clearly document how to get ACID behavior. I do not see an
> > > issue
> > > > with it.
> > > >
> > > > D.
> > > >
> > >
> >
>

Vladimir Ozerov

Re: Apache Ignite 2.4 release

Hi,
I merged performance fix for TCP metrics.

Nikolay,
I think it is OK to merge your fix today. Let's agree on a final code
freeze on Monday, 5 Mar.

On Thu, Mar 1, 2018 at 8:24 PM, Denis Magda <[hidden email]> wrote:

> Hi Vladimir,
>
> The regression, as well as the performance, is definitely bad. Please work
> with Sergey to see how much time he needs to rerun the whole test set if
> you merge the changes.
>
> I would target the end of the next week as a release date if the vote goes
> well. Probably, we can run the test suites over the weekend if your changes
> get to the branch.
>
> Nikolay Izhikov, follow up with Vladimir and Sergey tomorrow. If they
> decide to merge the fixes then you are free to merge your fixed for Spark
> examples.
>
> --
> Denis
>
> On Thu, Mar 1, 2018 at 7:46 AM, Vladimir Ozerov <[hidden email]>
> wrote:
>
> > Hi,
> >
> > We found one nasty regression in DROP COLUMN feature and fixed it today.
> > Another problem is performance - we found that recent changes to TCP
> > communication metrics caused considerable slowdown in in-memory mode. I
> > almost fixed it and will merge the fix in the nearest day if benchmarks
> are
> > ok.
> >
> > That said I think we will be able to start vote in the beginning of the
> > next week.
> >
> > Any objections?
> >
> > Vladimir.
> >
> > On Mon, Feb 26, 2018 at 11:22 PM, Denis Magda <[hidden email]> wrote:
> >
> > > Good question!
> > >
> > > We've been waiting for a sign-off from Alexey Goncharuk and Sergey
> Kozlov
> > > who are benchmarking and trying to address performance issues in
> > > coopeartion.
> > >
> > > Guys, please share your results and forecast.
> > >
> > > --
> > > Denis
> > >
> > > On Sun, Feb 25, 2018 at 12:10 AM, Pavel Tupitsyn <[hidden email]
> >
> > > wrote:
> > >
> > > > Igniters,
> > > >
> > > > Any update on the 2.4 release status? Anything else to merge there?
> > > >
> > > > Pavel
> > > >
> > > > On Mon, Feb 19, 2018 at 9:30 PM, Dmitriy Setrakyan <
> > > [hidden email]>
> > > > wrote:
> > > >
> > > > > On Mon, Feb 19, 2018 at 1:37 AM, Vladimir Ozerov <
> > [hidden email]
> > > >
> > > > > wrote:
> > > > >
> > > > > > Alex,
> > > > > >
> > > > > > You get me right. DEFAULT -> LOG_ONLY doesn't introduce any
> > dramatic
> > > > > > changes when comparing 2.3 to 2.4 - Ignite was unsafe out of the
> > box
> > > in
> > > > > > 2.3, and it is unsafe in 2.4 as well.
> > > > > >
> > > > > > The very problem is that we claim ourselves to be ACID, while in
> > > > reality
> > > > > we
> > > > > > are only "AI" out of the box, because durability is not
> guaranteed
> > > due
> > > > to
> > > > > > zero backups and LOG_ONLY and consistency is not guaranteed due
> to
> > > > > > PRIMARY_SYNC. Neither Cassandra, nor Mongo or any others claim
> > > > themselves
> > > > > > to be ACID, so it is not valid to refer to their defaults.
> > > > > >
> > > > >
> > > > > Vladimir,
> > > > > Ignite can be fully ACID, but at the same time have non-ACID
> > defaults,
> > > as
> > > > > long as we clearly document how to get ACID behavior. I do not see
> an
> > > > issue
> > > > > with it.
> > > > >
> > > > > D.
> > > > >
> > > >
> > >
> >
>

1234