DeadLocked while calling .Net ThinClient PutAll ?

[Tâm Nguyễn Mạnh]

Hi Igniters,

I heard that It could lead to DeadLocked when I pass non sorted collection
into putAll method. Is it true ?

From document i see that method requires a IEnumerable<KeyValuePair<TK,
TV>> as input. I think pass a sorted collection into API is just a
workaround way. Should it be handle natively in Ignite Core, shouldn't it ?

How do you thing ?

--
Thanks & Best Regards

Tam, Nguyen Manh

[Ivan Pavlukhin]

Hi,

When you are working with TRANSACTIONAL cache you should be aware that
deadlocks might happen. I see the following options to prevent
deadlocks to happen:
1. Explicitly order all keys involved in any transaction using the
same comparator.
2. Use OPTIMISTIC transactions.

Also, in many databases deadlocks can happen as well. So, instead of
preventing deadlocks it is quite common to provide some means of
breaking them. In Ignite it is possible to configure a transaction
timeout. In that case deadlocked transaction will be aborted sooner or
later and another transaction will be able to make a progress.

> Should it be handle natively in Ignite Core, shouldn't it ?

Unfortunately there is no general way to prevent deadlocks when using
PutAll with unordered collection. Ignite does not enforce all keys to
be Comparable which means that there could be keys without defined
consistent order. And if we cannot order keys consistently then we can
catch a deadlock.

пн, 18 мар. 2019 г. в 05:53, Tâm Nguyễn Mạnh <[hidden email]>:

>
> Hi Igniters,
>
> I heard that It could lead to DeadLocked when I pass non sorted collection
> into putAll method. Is it true ?
>
> From document i see that method requires a IEnumerable<KeyValuePair<TK,
> TV>> as input. I think pass a sorted collection into API is just a
> workaround way. Should it be handle natively in Ignite Core, shouldn't it ?
>
> How do you thing ?
>
> --
> Thanks & Best Regards
>
> Tam, Nguyen Manh

--
Best regards,
Ivan Pavlukhin

[Ilya Kasnacheev]

Hello!

Maybe we could issue a developer warning as soon as we detect putAll() with
something which has more than one element and which is not a sorted map?

Like we do with indexed when they don't fit inline, etc.

Regards,
--
Ilya Kasnacheev


пн, 18 мар. 2019 г. в 09:13, Павлухин Иван <[hidden email]>:

> Hi,
>
> When you are working with TRANSACTIONAL cache you should be aware that
> deadlocks might happen. I see the following options to prevent
> deadlocks to happen:
> 1. Explicitly order all keys involved in any transaction using the
> same comparator.
> 2. Use OPTIMISTIC transactions.
>
> Also, in many databases deadlocks can happen as well. So, instead of
> preventing deadlocks it is quite common to provide some means of
> breaking them. In Ignite it is possible to configure a transaction
> timeout. In that case deadlocked transaction will be aborted sooner or
> later and another transaction will be able to make a progress.
>
> > Should it be handle natively in Ignite Core, shouldn't it ?
>
> Unfortunately there is no general way to prevent deadlocks when using
> PutAll with unordered collection. Ignite does not enforce all keys to
> be Comparable which means that there could be keys without defined
> consistent order. And if we cannot order keys consistently then we can
> catch a deadlock.
>
> пн, 18 мар. 2019 г. в 05:53, Tâm Nguyễn Mạnh <[hidden email]>:
> >
> > Hi Igniters,
> >
> > I heard that It could lead to DeadLocked when I pass non sorted
> collection
> > into putAll method. Is it true ?
> >
> > From document i see that method requires a IEnumerable<KeyValuePair<TK,
> > TV>> as input. I think pass a sorted collection into API is just a
> > workaround way. Should it be handle natively in Ignite Core, shouldn't
> it ?
> >
> > How do you thing ?
> >
> > --
> > Thanks & Best Regards
> >
> > Tam, Nguyen Manh
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>

[Pavel Tupitsyn]

To avoid deadlocks you have to always take locks in the same order.
* That order is not always going to be "sorted with default comparer".
* Not every type has a default comparer

So Ignite can't sort the keys for you, nor can it check if they are sorted.
And even if it could, it would be making unnecessary assumptions about user
code.
And it would be potentially expensive to sort or check order.

Ilya, if you mean `instanceof` check, that does not look like a good API to
me to require a particular interface implementation.

On Mon, Mar 18, 2019 at 10:25 AM Ilya Kasnacheev <[hidden email]>
wrote:

> Hello!
>
> Maybe we could issue a developer warning as soon as we detect putAll() with
> something which has more than one element and which is not a sorted map?
>
> Like we do with indexed when they don't fit inline, etc.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пн, 18 мар. 2019 г. в 09:13, Павлухин Иван <[hidden email]>:
>
> > Hi,
> >
> > When you are working with TRANSACTIONAL cache you should be aware that
> > deadlocks might happen. I see the following options to prevent
> > deadlocks to happen:
> > 1. Explicitly order all keys involved in any transaction using the
> > same comparator.
> > 2. Use OPTIMISTIC transactions.
> >
> > Also, in many databases deadlocks can happen as well. So, instead of
> > preventing deadlocks it is quite common to provide some means of
> > breaking them. In Ignite it is possible to configure a transaction
> > timeout. In that case deadlocked transaction will be aborted sooner or
> > later and another transaction will be able to make a progress.
> >
> > > Should it be handle natively in Ignite Core, shouldn't it ?
> >
> > Unfortunately there is no general way to prevent deadlocks when using
> > PutAll with unordered collection. Ignite does not enforce all keys to
> > be Comparable which means that there could be keys without defined
> > consistent order. And if we cannot order keys consistently then we can
> > catch a deadlock.
> >
> > пн, 18 мар. 2019 г. в 05:53, Tâm Nguyễn Mạnh <[hidden email]
> >:
> > >
> > > Hi Igniters,
> > >
> > > I heard that It could lead to DeadLocked when I pass non sorted
> > collection
> > > into putAll method. Is it true ?
> > >
> > > From document i see that method requires a IEnumerable<KeyValuePair<TK,
> > > TV>> as input. I think pass a sorted collection into API is just a
> > > workaround way. Should it be handle natively in Ignite Core, shouldn't
> > it ?
> > >
> > > How do you thing ?
> > >
> > > --
> > > Thanks & Best Regards
> > >
> > > Tam, Nguyen Manh
> >
> >
> >
> > --
> > Best regards,
> > Ivan Pavlukhin
> >
>

[Ilya Kasnacheev]

Hello!

We can defnitely do a heuristic check that passed map is HashMap (therefore
unordered), since HashMap is basically "the" map. If we do a warning in 99%
of problematic cases but fail to detect the remaining 1%, it's still a big
improvement over the current behavior.

Same thing in .Net with their default Dictionary.

I have created a ticket: https://issues.apache.org/jira/browse/IGNITE-11558

Regards,
--
Ilya Kasnacheev


пн, 18 мар. 2019 г. в 11:12, Pavel Tupitsyn <[hidden email]>:

> To avoid deadlocks you have to always take locks in the same order.
> * That order is not always going to be "sorted with default comparer".
> * Not every type has a default comparer
>
> So Ignite can't sort the keys for you, nor can it check if they are sorted.
> And even if it could, it would be making unnecessary assumptions about user
> code.
> And it would be potentially expensive to sort or check order.
>
> Ilya, if you mean `instanceof` check, that does not look like a good API to
> me to require a particular interface implementation.
>
> On Mon, Mar 18, 2019 at 10:25 AM Ilya Kasnacheev <
> [hidden email]>
> wrote:
>
> > Hello!
> >
> > Maybe we could issue a developer warning as soon as we detect putAll()
> with
> > something which has more than one element and which is not a sorted map?
> >
> > Like we do with indexed when they don't fit inline, etc.
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > пн, 18 мар. 2019 г. в 09:13, Павлухин Иван <[hidden email]>:
> >
> > > Hi,
> > >
> > > When you are working with TRANSACTIONAL cache you should be aware that
> > > deadlocks might happen. I see the following options to prevent
> > > deadlocks to happen:
> > > 1. Explicitly order all keys involved in any transaction using the
> > > same comparator.
> > > 2. Use OPTIMISTIC transactions.
> > >
> > > Also, in many databases deadlocks can happen as well. So, instead of
> > > preventing deadlocks it is quite common to provide some means of
> > > breaking them. In Ignite it is possible to configure a transaction
> > > timeout. In that case deadlocked transaction will be aborted sooner or
> > > later and another transaction will be able to make a progress.
> > >
> > > > Should it be handle natively in Ignite Core, shouldn't it ?
> > >
> > > Unfortunately there is no general way to prevent deadlocks when using
> > > PutAll with unordered collection. Ignite does not enforce all keys to
> > > be Comparable which means that there could be keys without defined
> > > consistent order. And if we cannot order keys consistently then we can
> > > catch a deadlock.
> > >
> > > пн, 18 мар. 2019 г. в 05:53, Tâm Nguyễn Mạnh <
> [hidden email]
> > >:
> > > >
> > > > Hi Igniters,
> > > >
> > > > I heard that It could lead to DeadLocked when I pass non sorted
> > > collection
> > > > into putAll method. Is it true ?
> > > >
> > > > From document i see that method requires a
> IEnumerable<KeyValuePair<TK,
> > > > TV>> as input. I think pass a sorted collection into API is just a
> > > > workaround way. Should it be handle natively in Ignite Core,
> shouldn't
> > > it ?
> > > >
> > > > How do you thing ?
> > > >
> > > > --
> > > > Thanks & Best Regards
> > > >
> > > > Tam, Nguyen Manh
> > >
> > >
> > >
> > > --
> > > Best regards,
> > > Ivan Pavlukhin
> > >
> >
>