snapshot transaction isolation

[dsetrakyan]

Igniters,

I keep hearing questions from users about the snapshot isolation. Currently
ignite provides Optimistic and Pessimistic
<https://apacheignite.readme.io/docs/transactions#optimistic-and-pessimistic>
transactions [1]. This modes ensure that transactional values are
consistent with each other on 1st access of each value.

However, some use cases require that transactional values are consistent
with each other not at 1st access, but at transaction start time. After
giving it some thought, I think we can support it with minimal effort, if
we add a few restrictions. For example, we can easily support it if users
specify all the keys at the beginning of the transaction, for example

   1. User tells Ignite which keys he/she plans to transact on
   2. Ignite preemptively acquires locks on all these keys
   3. After locks are acquired, user has assurance that values will not
   change outside of this transaction and are consistent with teacher.
   4. Locks are released upon commit

The above algorithm will also perform better, as the initial looks will be
acquired in bulk, and not individually.

Thoughts?

[1]
https://apacheignite.readme.io/docs/transactions#optimistic-and-pessimistic

[Alexey Goncharuk]

If all keys are known in advance, how is it different from starting a
pessimistic transaction and invoking getAll() on those keys? Introducing a
new concept with such restrictions does not makes sense to me.

2016-02-04 1:27 GMT+03:00 Dmitriy Setrakyan <[hidden email]>:

> Igniters,
>
> I keep hearing questions from users about the snapshot isolation. Currently
> ignite provides Optimistic and Pessimistic
> <
> https://apacheignite.readme.io/docs/transactions#optimistic-and-pessimistic
> >
> transactions [1]. This modes ensure that transactional values are
> consistent with each other on 1st access of each value.
>
> However, some use cases require that transactional values are consistent
> with each other not at 1st access, but at transaction start time. After
> giving it some thought, I think we can support it with minimal effort, if
> we add a few restrictions. For example, we can easily support it if users
> specify all the keys at the beginning of the transaction, for example
>
>    1. User tells Ignite which keys he/she plans to transact on
>    2. Ignite preemptively acquires locks on all these keys
>    3. After locks are acquired, user has assurance that values will not
>    change outside of this transaction and are consistent with teacher.
>    4. Locks are released upon commit
>
> The above algorithm will also perform better, as the initial looks will be
> acquired in bulk, and not individually.
>
> Thoughts?
>
> [1]
> https://apacheignite.readme.io/docs/transactions#optimistic-and-pessimistic
>

[dsetrakyan]

I think the whole point is to lock 1st and get 2nd. If you do a getAll,
isn’t it possible that some value will be updated before you get it? If
yes, then user’s logic will potentially be based on a wrong value, no?

D.

On Thu, Feb 4, 2016 at 1:29 AM, Alexey Goncharuk <[hidden email]
> wrote:

> If all keys are known in advance, how is it different from starting a
> pessimistic transaction and invoking getAll() on those keys? Introducing a
> new concept with such restrictions does not makes sense to me.
>
> 2016-02-04 1:27 GMT+03:00 Dmitriy Setrakyan <[hidden email]>:
>
> > Igniters,
> >
> > I keep hearing questions from users about the snapshot isolation.
> Currently
> > ignite provides Optimistic and Pessimistic
> > <
> >
> https://apacheignite.readme.io/docs/transactions#optimistic-and-pessimistic
> > >
> > transactions [1]. This modes ensure that transactional values are
> > consistent with each other on 1st access of each value.
> >
> > However, some use cases require that transactional values are consistent
> > with each other not at 1st access, but at transaction start time. After
> > giving it some thought, I think we can support it with minimal effort, if
> > we add a few restrictions. For example, we can easily support it if users
> > specify all the keys at the beginning of the transaction, for example
> >
> >    1. User tells Ignite which keys he/she plans to transact on
> >    2. Ignite preemptively acquires locks on all these keys
> >    3. After locks are acquired, user has assurance that values will not
> >    change outside of this transaction and are consistent with teacher.
> >    4. Locks are released upon commit
> >
> > The above algorithm will also perform better, as the initial looks will
> be
> > acquired in bulk, and not individually.
> >
> > Thoughts?
> >
> > [1]
> >
> https://apacheignite.readme.io/docs/transactions#optimistic-and-pessimistic
> >
>

[Alexey Goncharuk]

In PESSIMISTIC transaction a value is always read after a lock is acquired,
so a locked value cannot be updated. Am I missing something? Do you have a
specific scenario in mind?

[yzhdanov]

> If you do a getAll,
> isn’t it possible that some value will be updated before you get it? If
> yes, then user’s logic will potentially be based on a wrong value, no?

1. What if any value gets updated before you lock it? It seems this is the
strongest guarantee we can provide with this approach.

> However, some use cases require that transactional values are consistent
> with each other not at 1st access, but at transaction start time. After
>giving it some thought, I think we can support it with minimal effort, if
> we add a few restrictions. For example, we can easily support it if users
> specify all the keys at the beginning of the transaction, for example

>   1. User tells Ignite which keys he/she plans to transact on
>   2. Ignite preemptively acquires locks on all these keys
>   3. After locks are acquired, user has assurance that values will not
>   change outside of this transaction and are consistent with teacher.
>   4. Locks are released upon commit

I think that it will also be very good to add tx-awareness to cache lock we
currently have. GETALL may be very heavy which may not be needed + we
support all TX not only pessimistic. So, the logic will be:

START_TX()

LOCK_ALL(KEYS);

INVOKE/PUT/GET/ETC

COMMIT()/ROLLBACK()


--Yakov

2016-02-04 12:31 GMT+03:00 Dmitriy Setrakyan <[hidden email]>:

> I think the whole point is to lock 1st and get 2nd. If you do a getAll,
> isn’t it possible that some value will be updated before you get it? If
> yes, then user’s logic will potentially be based on a wrong value, no?
>
> D.
>
> On Thu, Feb 4, 2016 at 1:29 AM, Alexey Goncharuk <
> [hidden email]
> > wrote:
>
> > If all keys are known in advance, how is it different from starting a
> > pessimistic transaction and invoking getAll() on those keys? Introducing
> a
> > new concept with such restrictions does not makes sense to me.
> >
> > 2016-02-04 1:27 GMT+03:00 Dmitriy Setrakyan <[hidden email]>:
> >
> > > Igniters,
> > >
> > > I keep hearing questions from users about the snapshot isolation.
> > Currently
> > > ignite provides Optimistic and Pessimistic
> > > <
> > >
> >
> https://apacheignite.readme.io/docs/transactions#optimistic-and-pessimistic
> > > >
> > > transactions [1]. This modes ensure that transactional values are
> > > consistent with each other on 1st access of each value.
> > >
> > > However, some use cases require that transactional values are
> consistent
> > > with each other not at 1st access, but at transaction start time. After
> > > giving it some thought, I think we can support it with minimal effort,
> if
> > > we add a few restrictions. For example, we can easily support it if
> users
> > > specify all the keys at the beginning of the transaction, for example
> > >
> > >    1. User tells Ignite which keys he/she plans to transact on
> > >    2. Ignite preemptively acquires locks on all these keys
> > >    3. After locks are acquired, user has assurance that values will not
> > >    change outside of this transaction and are consistent with teacher.
> > >    4. Locks are released upon commit
> > >
> > > The above algorithm will also perform better, as the initial looks will
> > be
> > > acquired in bulk, and not individually.
> > >
> > > Thoughts?
> > >
> > > [1]
> > >
> >
> https://apacheignite.readme.io/docs/transactions#optimistic-and-pessimistic
> > >
> >
>

[Alexey Goncharuk]

So, basically, we want to add lockAll() method that locks entries without
returning their values to a client - this is a good idea. I do not want,
however, to call it SNAPSHOT isolation, because this is not what it is.

[Vladimir Ozerov]

+1
This should help us with IGFS performance as we currently use getAll() in
PESSIMISTIC mode mainly to lock keys, not to get their values.

On Thu, Feb 4, 2016 at 12:56 PM, Alexey Goncharuk <
[hidden email]> wrote:

> So, basically, we want to add lockAll() method that locks entries without
> returning their values to a client - this is a good idea. I do not want,
> however, to call it SNAPSHOT isolation, because this is not what it is.
>

[dsetrakyan]

On Thu, Feb 4, 2016 at 1:56 AM, Alexey Goncharuk <[hidden email]
> wrote:

> So, basically, we want to add lockAll() method that locks entries without
> returning their values to a client - this is a good idea. I do not want,
> however, to call it SNAPSHOT isolation, because this is not what it is.
>

I think I see your point. All we need to do is allow for lock() and
lockAll() invocations within a transaction, no? Do you know why we
currently prohibit it?

[Alexey Goncharuk]

Currently lock-only functionality is exposed via j.u.c.Lock interface on
IgniteCache. We have two choices here:
 * Release such locks on transaction commit, which would break the contract
of j.u.c.Lock
 * Do not release such locks on transaction commit, which, in my opinion,
conflicts with the expectation of transaction locks.

Either way looks dirty to me, so I would vote for adding a new
properly-named method on IgniteCache specifically for this case.

[dsetrakyan]

Hm… how about allowing to acquire locks within a transaction and just
releasing locks on transaction commit? It won’t break current compatibility
because we simply do not allow acquiring locks within transactions right
now. I am wrong?

D.

On Mon, Feb 8, 2016 at 4:10 AM, Alexey Goncharuk <[hidden email]
> wrote:

> Currently lock-only functionality is exposed via j.u.c.Lock interface on
> IgniteCache. We have two choices here:
>  * Release such locks on transaction commit, which would break the contract
> of j.u.c.Lock
>  * Do not release such locks on transaction commit, which, in my opinion,
> conflicts with the expectation of transaction locks.
>
> Either way looks dirty to me, so I would vote for adding a new
> properly-named method on IgniteCache specifically for this case.
>