Re: Ignite 2.0.0 GridUnsafe unmonitor

[dmagda]

+ dev list

Igniters, that’s a relevant point below. Newcomers to Ignite tend to stumble on deadlocks simply because the keys are passed in an unordered HashMap. Propose to do the following:
- update bulk operations Java doc.
- print out a warning if a HashMap is used and its exceeds one element.

Thoughts?

—
Denis

> On Oct 21, 2017, at 6:16 PM, dark <[hidden email]> wrote:
>
> Many people seem to be more likely to send Cache entries in bulk via a
> HashMap.
> How do you expose a warning statement by checking if the TreeMap is putAll
> inside the code?
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/

[dsetrakyan]

Denis,

We should definitely print out a thorough warning if HashMap is passed into
a bulk method (instead of SortedMap). However, we should make sure that we
only print that warning once and not ever time the API is called.

Can you please file a ticket for 2.4?

D.

On Thu, Oct 26, 2017 at 11:05 AM, Denis Magda <[hidden email]> wrote:

> + dev list
>
> Igniters, that’s a relevant point below. Newcomers to Ignite tend to
> stumble on deadlocks simply because the keys are passed in an unordered
> HashMap. Propose to do the following:
> - update bulk operations Java doc.
> - print out a warning if a HashMap is used and its exceeds one element.

> Thoughts?
>
> —
> Denis
>
> > On Oct 21, 2017, at 6:16 PM, dark <[hidden email]> wrote:
> >
> > Many people seem to be more likely to send Cache entries in bulk via a
> > HashMap.
> > How do you expose a warning statement by checking if the TreeMap is
> putAll
> > inside the code?
> >
> >
> >
> >
> > --
> > Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
>

[Dmitriy Pavlov]

I agree with Denis, if we don't have such warning we should continiously
warn users in wiki pages/blogs/presentations. It is simpler to warn from
code.

What do you think if we will issue warning only if size > 1. HashMap with 1
item will not cause deadlock. Moreover where can be some custom singleton
Map provided by user.

Sincerely,
Dmitriy Pavlov

вт, 31 окт. 2017 г. в 7:18, Dmitriy Setrakyan <[hidden email]>:

> Denis,
>
> We should definitely print out a thorough warning if HashMap is passed into
> a bulk method (instead of SortedMap). However, we should make sure that we
> only print that warning once and not ever time the API is called.
>
> Can you please file a ticket for 2.4?
>
> D.
>
> On Thu, Oct 26, 2017 at 11:05 AM, Denis Magda <[hidden email]> wrote:
>
> > + dev list
> >
> > Igniters, that’s a relevant point below. Newcomers to Ignite tend to
> > stumble on deadlocks simply because the keys are passed in an unordered
> > HashMap. Propose to do the following:
> > - update bulk operations Java doc.
> > - print out a warning if a HashMap is used and its exceeds one element.
>
>
> > Thoughts?
> >
> > —
> > Denis
> >
> > > On Oct 21, 2017, at 6:16 PM, dark <[hidden email]> wrote:
> > >
> > > Many people seem to be more likely to send Cache entries in bulk via a
> > > HashMap.
> > > How do you expose a warning statement by checking if the TreeMap is
> > putAll
> > > inside the code?
> > >
> > >
> > >
> > >
> > > --
> > > Sent from: http://apache-ignite-users.70518.x6.nabble.com/
> >
> >
>

[dmagda]

Here is a ticket for the improvement:
https://issues.apache.org/jira/browse/IGNITE-6804

—
Denis

> On Oct 31, 2017, at 3:55 AM, Dmitry Pavlov <[hidden email]> wrote:
>
> I agree with Denis, if we don't have such warning we should continiously warn users in wiki pages/blogs/presentations. It is simpler to warn from code.
>
> What do you think if we will issue warning only if size > 1. HashMap with 1 item will not cause deadlock. Moreover where can be some custom singleton Map provided by user.
>
> Sincerely,
> Dmitriy Pavlov
>
> вт, 31 окт. 2017 г. в 7:18, Dmitriy Setrakyan <[hidden email] <mailto:[hidden email]>>:
> Denis,
>
> We should definitely print out a thorough warning if HashMap is passed into
> a bulk method (instead of SortedMap). However, we should make sure that we
> only print that warning once and not ever time the API is called.
>
> Can you please file a ticket for 2.4?
>
> D.
>
> On Thu, Oct 26, 2017 at 11:05 AM, Denis Magda <[hidden email] <mailto:[hidden email]>> wrote:
>
> > + dev list
> >
> > Igniters, that’s a relevant point below. Newcomers to Ignite tend to
> > stumble on deadlocks simply because the keys are passed in an unordered
> > HashMap. Propose to do the following:
> > - update bulk operations Java doc.
> > - print out a warning if a HashMap is used and its exceeds one element.
>
>
> > Thoughts?
> >
> > —
> > Denis
> >
> > > On Oct 21, 2017, at 6:16 PM, dark <[hidden email] <mailto:[hidden email]>> wrote:
> > >
> > > Many people seem to be more likely to send Cache entries in bulk via a
> > > HashMap.
> > > How do you expose a warning statement by checking if the TreeMap is
> > putAll
> > > inside the code?
> > >
> > >
> > >
> > >
> > > --
> > > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ <http://apache-ignite-users.70518.x6.nabble.com/>
> >
> >

[Vladimir Ozerov]

Guys,

Printing a warning in this case is really strange idea. First, how would
explain it in case of OPTIMISTIC/SERIALIZABLE transactions where deadlocks
are impossible? Second, what would you do in case tow sorted maps are
passed one by one in a transaction? User still may have a deadlock. :Last,
we are going towards SQL world, where "maps" simply do not exist, and
virtually any update could eailty lead to a deadlock.

Let's avoid strange warnings for normal usage scenario. Denis, please close
the ticket :-)))

Vladimir.

On Tue, Oct 31, 2017 at 8:34 PM, Denis Magda <[hidden email]> wrote:

> Here is a ticket for the improvement:
> https://issues.apache.org/jira/browse/IGNITE-6804
>
> —
> Denis
>
> > On Oct 31, 2017, at 3:55 AM, Dmitry Pavlov <[hidden email]>
> wrote:
> >
> > I agree with Denis, if we don't have such warning we should continiously
> warn users in wiki pages/blogs/presentations. It is simpler to warn from
> code.
> >
> > What do you think if we will issue warning only if size > 1. HashMap
> with 1 item will not cause deadlock. Moreover where can be some custom
> singleton Map provided by user.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > вт, 31 окт. 2017 г. в 7:18, Dmitriy Setrakyan <[hidden email]
> <mailto:[hidden email]>>:
> > Denis,
> >
> > We should definitely print out a thorough warning if HashMap is passed
> into
> > a bulk method (instead of SortedMap). However, we should make sure that
> we
> > only print that warning once and not ever time the API is called.
> >
> > Can you please file a ticket for 2.4?
> >
> > D.
> >
> > On Thu, Oct 26, 2017 at 11:05 AM, Denis Magda <[hidden email]
> <mailto:[hidden email]>> wrote:
> >
> > > + dev list
> > >
> > > Igniters, that’s a relevant point below. Newcomers to Ignite tend to
> > > stumble on deadlocks simply because the keys are passed in an unordered
> > > HashMap. Propose to do the following:
> > > - update bulk operations Java doc.
> > > - print out a warning if a HashMap is used and its exceeds one element.
> >
> >
> > > Thoughts?
> > >
> > > —
> > > Denis
> > >
> > > > On Oct 21, 2017, at 6:16 PM, dark <[hidden email] <mailto:
> [hidden email]>> wrote:
> > > >
> > > > Many people seem to be more likely to send Cache entries in bulk via
> a
> > > > HashMap.
> > > > How do you expose a warning statement by checking if the TreeMap is
> > > putAll
> > > > inside the code?
> > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ <
> http://apache-ignite-users.70518.x6.nabble.com/>
> > >
> > >
>
>

[dmagda]

Vladimir,

That’s an oversight and lack of explanation on our side. The goal is to avoid unexpected deadlocks when a user passed a HashMap in cache.putAll. Before printing out a warning we can filter out OPTIMISTIC/SERIALIZABLE and other suitable scenarios.

So you’re free to offer another solution aside from closing the ticket :)

—
Denis

 

> On Oct 31, 2017, at 10:55 AM, Vladimir Ozerov <[hidden email]> wrote:
>
> Guys,
>
> Printing a warning in this case is really strange idea. First, how would explain it in case of OPTIMISTIC/SERIALIZABLE transactions where deadlocks are impossible? Second, what would you do in case tow sorted maps are passed one by one in a transaction? User still may have a deadlock. :Last, we are going towards SQL world, where "maps" simply do not exist, and virtually any update could eailty lead to a deadlock.
>
> Let's avoid strange warnings for normal usage scenario. Denis, please close the ticket :-)))
>
> Vladimir.
>
> On Tue, Oct 31, 2017 at 8:34 PM, Denis Magda <[hidden email] <mailto:[hidden email]>> wrote:
> Here is a ticket for the improvement:
> https://issues.apache.org/jira/browse/IGNITE-6804 <https://issues.apache.org/jira/browse/IGNITE-6804>
>
> —
> Denis
>
> > On Oct 31, 2017, at 3:55 AM, Dmitry Pavlov <[hidden email] <mailto:[hidden email]>> wrote:
> >
> > I agree with Denis, if we don't have such warning we should continiously warn users in wiki pages/blogs/presentations. It is simpler to warn from code.
> >
> > What do you think if we will issue warning only if size > 1. HashMap with 1 item will not cause deadlock. Moreover where can be some custom singleton Map provided by user.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > вт, 31 окт. 2017 г. в 7:18, Dmitriy Setrakyan <[hidden email] <mailto:[hidden email]> <mailto:[hidden email] <mailto:[hidden email]>>>:
> > Denis,
> >
> > We should definitely print out a thorough warning if HashMap is passed into
> > a bulk method (instead of SortedMap). However, we should make sure that we
> > only print that warning once and not ever time the API is called.
> >
> > Can you please file a ticket for 2.4?
> >
> > D.
> >
> > On Thu, Oct 26, 2017 at 11:05 AM, Denis Magda <[hidden email] <mailto:[hidden email]> <mailto:[hidden email] <mailto:[hidden email]>>> wrote:
> >
> > > + dev list
> > >
> > > Igniters, that’s a relevant point below. Newcomers to Ignite tend to
> > > stumble on deadlocks simply because the keys are passed in an unordered
> > > HashMap. Propose to do the following:
> > > - update bulk operations Java doc.
> > > - print out a warning if a HashMap is used and its exceeds one element.
> >
> >
> > > Thoughts?
> > >
> > > —
> > > Denis
> > >
> > > > On Oct 21, 2017, at 6:16 PM, dark <[hidden email] <mailto:[hidden email]> <mailto:[hidden email] <mailto:[hidden email]>>> wrote:
> > > >
> > > > Many people seem to be more likely to send Cache entries in bulk via a
> > > > HashMap.
> > > > How do you expose a warning statement by checking if the TreeMap is
> > > putAll
> > > > inside the code?
> > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ <http://apache-ignite-users.70518.x6.nabble.com/> <http://apache-ignite-users.70518.x6.nabble.com/ <http://apache-ignite-users.70518.x6.nabble.com/>>
> > >
> > >

[Dmitriy Pavlov]

Vladimir, thank you. Good point for optimistic tx, but still putAll usage
require using sorted collections.
User, of course, may broke this scenario also by using sorted maps with
incorrect custom comparators (one asc, one desc).
Producing warning in code for Igntie for potential deadlock is good
usability feature.
Please look to user list, deadlock is very often question.

вт, 31 окт. 2017 г. в 21:02, Denis Magda <[hidden email]>:

> Vladimir,
>
> That’s an oversight and lack of explanation on our side. The goal is to
> avoid unexpected deadlocks when a user passed a HashMap in cache.putAll.
> Before printing out a warning we can filter out OPTIMISTIC/SERIALIZABLE and
> other suitable scenarios.
>
> So you’re free to offer another solution aside from closing the ticket :)
>
> —
> Denis
>
>
>
> On Oct 31, 2017, at 10:55 AM, Vladimir Ozerov <[hidden email]>
> wrote:
>
> Guys,
>
> Printing a warning in this case is really strange idea. First, how would
> explain it in case of OPTIMISTIC/SERIALIZABLE transactions where deadlocks
> are impossible? Second, what would you do in case tow sorted maps are
> passed one by one in a transaction? User still may have a deadlock. :Last,
> we are going towards SQL world, where "maps" simply do not exist, and
> virtually any update could eailty lead to a deadlock.
>
> Let's avoid strange warnings for normal usage scenario. Denis, please
> close the ticket :-)))
>
> Vladimir.
>
> On Tue, Oct 31, 2017 at 8:34 PM, Denis Magda <[hidden email]> wrote:
>
>> Here is a ticket for the improvement:
>> https://issues.apache.org/jira/browse/IGNITE-6804
>>
>> —
>> Denis
>>
>> > On Oct 31, 2017, at 3:55 AM, Dmitry Pavlov <[hidden email]>
>> wrote:
>> >
>> > I agree with Denis, if we don't have such warning we should
>> continiously warn users in wiki pages/blogs/presentations. It is simpler to
>> warn from code.
>> >
>> > What do you think if we will issue warning only if size > 1. HashMap
>> with 1 item will not cause deadlock. Moreover where can be some custom
>> singleton Map provided by user.
>> >
>> > Sincerely,
>> > Dmitriy Pavlov
>> >
>> > вт, 31 окт. 2017 г. в 7:18, Dmitriy Setrakyan <[hidden email]
>> <mailto:[hidden email]>>:
>> > Denis,
>> >
>> > We should definitely print out a thorough warning if HashMap is passed
>> into
>> > a bulk method (instead of SortedMap). However, we should make sure that
>> we
>> > only print that warning once and not ever time the API is called.
>> >
>> > Can you please file a ticket for 2.4?
>> >
>> > D.
>> >
>> > On Thu, Oct 26, 2017 at 11:05 AM, Denis Magda <[hidden email]
>> <mailto:[hidden email]>> wrote:
>> >
>> > > + dev list
>> > >
>> > > Igniters, that’s a relevant point below. Newcomers to Ignite tend to
>> > > stumble on deadlocks simply because the keys are passed in an
>> unordered
>> > > HashMap. Propose to do the following:
>> > > - update bulk operations Java doc.
>> > > - print out a warning if a HashMap is used and its exceeds one
>> element.
>> >
>> >
>> > > Thoughts?
>> > >
>> > > —
>> > > Denis
>> > >
>> > > > On Oct 21, 2017, at 6:16 PM, dark <[hidden email] <mailto:
>> [hidden email]>> wrote:
>> > > >
>> > > > Many people seem to be more likely to send Cache entries in bulk
>> via a
>> > > > HashMap.
>> > > > How do you expose a warning statement by checking if the TreeMap is
>> > > putAll
>> > > > inside the code?
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ <
>> http://apache-ignite-users.70518.x6.nabble.com/>
>> > >
>> > >
>
>
>

[Valentin Kulichenko]

I like the idea to print out a warning if unsorted map is provided. The
fact that there are tons of other ways to get a deadlock doesn't mean that
we should ignore this case which is actually very common.

-Val

On Tue, Oct 31, 2017 at 12:34 PM, Dmitry Pavlov <[hidden email]>
wrote:

> Vladimir, thank you. Good point for optimistic tx, but still putAll usage
> require using sorted collections.
> User, of course, may broke this scenario also by using sorted maps with
> incorrect custom comparators (one asc, one desc).
> Producing warning in code for Igntie for potential deadlock is good
> usability feature.
> Please look to user list, deadlock is very often question.
>
> вт, 31 окт. 2017 г. в 21:02, Denis Magda <[hidden email]>:
>
>> Vladimir,
>>
>> That’s an oversight and lack of explanation on our side. The goal is to
>> avoid unexpected deadlocks when a user passed a HashMap in cache.putAll.
>> Before printing out a warning we can filter out OPTIMISTIC/SERIALIZABLE and
>> other suitable scenarios.
>>
>> So you’re free to offer another solution aside from closing the ticket :)
>>
>> —
>> Denis
>>
>>
>>
>> On Oct 31, 2017, at 10:55 AM, Vladimir Ozerov <[hidden email]>
>> wrote:
>>
>> Guys,
>>
>> Printing a warning in this case is really strange idea. First, how would
>> explain it in case of OPTIMISTIC/SERIALIZABLE transactions where deadlocks
>> are impossible? Second, what would you do in case tow sorted maps are
>> passed one by one in a transaction? User still may have a deadlock. :Last,
>> we are going towards SQL world, where "maps" simply do not exist, and
>> virtually any update could eailty lead to a deadlock.
>>
>> Let's avoid strange warnings for normal usage scenario. Denis, please
>> close the ticket :-)))
>>
>> Vladimir.
>>
>> On Tue, Oct 31, 2017 at 8:34 PM, Denis Magda <[hidden email]> wrote:
>>
>>> Here is a ticket for the improvement:
>>> https://issues.apache.org/jira/browse/IGNITE-6804
>>>
>>> —
>>> Denis
>>>
>>> > On Oct 31, 2017, at 3:55 AM, Dmitry Pavlov <[hidden email]>
>>> wrote:
>>> >
>>> > I agree with Denis, if we don't have such warning we should
>>> continiously warn users in wiki pages/blogs/presentations. It is simpler to
>>> warn from code.
>>> >
>>> > What do you think if we will issue warning only if size > 1. HashMap
>>> with 1 item will not cause deadlock. Moreover where can be some custom
>>> singleton Map provided by user.
>>> >
>>> > Sincerely,
>>> > Dmitriy Pavlov
>>> >
>>> > вт, 31 окт. 2017 г. в 7:18, Dmitriy Setrakyan <[hidden email]
>>> <mailto:[hidden email]>>:
>>> > Denis,
>>> >
>>> > We should definitely print out a thorough warning if HashMap is passed
>>> into
>>> > a bulk method (instead of SortedMap). However, we should make sure
>>> that we
>>> > only print that warning once and not ever time the API is called.
>>> >
>>> > Can you please file a ticket for 2.4?
>>> >
>>> > D.
>>> >
>>> > On Thu, Oct 26, 2017 at 11:05 AM, Denis Magda <[hidden email]
>>> <mailto:[hidden email]>> wrote:
>>> >
>>> > > + dev list
>>> > >
>>> > > Igniters, that’s a relevant point below. Newcomers to Ignite tend to
>>> > > stumble on deadlocks simply because the keys are passed in an
>>> unordered
>>> > > HashMap. Propose to do the following:
>>> > > - update bulk operations Java doc.
>>> > > - print out a warning if a HashMap is used and its exceeds one
>>> element.
>>> >
>>> >
>>> > > Thoughts?
>>> > >
>>> > > —
>>> > > Denis
>>> > >
>>> > > > On Oct 21, 2017, at 6:16 PM, dark <[hidden email] <mailto:
>>> [hidden email]>> wrote:
>>> > > >
>>> > > > Many people seem to be more likely to send Cache entries in bulk
>>> via a
>>> > > > HashMap.
>>> > > > How do you expose a warning statement by checking if the TreeMap is
>>> > > putAll
>>> > > > inside the code?
>>> > > >
>>> > > >
>>> > > >
>>> > > >
>>> > > > --
>>> > > > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ <http:
>>> //apache-ignite-users.70518.x6.nabble.com/>
>>> > >
>>> > >
>>
>>
>>

[yzhdanov]

+1 for warning about potential deadlock and improving javadocs

--Yakov