Fwd: Distributed queue problem with peerClassLoading enabled

[dsetrakyan]

Igniters,

I want to bring your attention to my email response on the user list. I
just want to confirm that we are all on the same page and that my
understanding is correct.

I think that once we merge the BinaryObject to the release branch, we
should remove the restriction on SHARED or CONTINUOUS deployment modes in
cache, assuming that our own Binary marshaling is used.

Alexey G, given that you are in charge of this merge, can you please make
this change as well?

Thanks,
D.

---------- Forwarded message ----------
From: Dmitriy Setrakyan <[hidden email]>
Date: Tue, Nov 3, 2015 at 11:13 AM
Subject: Re: Distributed queue problem with peerClassLoading enabled
To: user <[hidden email]>


Hi Mateusz,

In the upcoming Ignite 1.5 release, we will not be storing user-classes on
the server side at all, so the need for deployment of enqueued objects goes
away.

With that in mind, we will be removing the requirement for caches to work
only with SHARED and CONTINUOUS deployment modes, so you will be able to
use PRIVATE or ISOLATED deployment modes to deploy your computations.

We are currently actively working on finalizing the 1.5 release, so it
should be coming real soon.

D.

On Tue, Nov 3, 2015 at 3:05 AM, mp <[hidden email]> wrote:

> Hi Denis,
>
> Thanks a lot for the clarifications.
>
> So, is there any way in Ignite to achieve my goal: different versions of
> the same class running on the cluster (they can be deployed at the same
> time from different client nodes)?
> If yes, I can think of a way in which I can achieve my goals without using
> the distributed queue.
>
> Cheers,
> -Mateusz
>
>
> On Tue, Nov 3, 2015 at 11:53 AM, Denis Magda <[hidden email]> wrote:
>
>> Hi Mateusz,
>>
>> Unfortunately I don't see any way to specify userVersion programmatically.
>> In any case CONTINIOUS and SHARED mode is not suitable for your task
>> because
>> the developers may want to have different version of the same class.
>> Please
>> refer to DeploymentMode documentation for more info on different mode
>> types.
>>
>> Do you really need to use IgniteQueue in your tests because, yes, neither
>> PRIVATE nor ISOLATED modes are supported for user caches.
>>
>> --
>> Denis
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p1820.html
>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>
>
>

[Alexey Goncharuk]

I am not sure what behavior we want to achieve here. Do we want to disable
cache peer class loading for binary marshalling altogether? This
restriction is safe to remove only if user chooses to work exclusively with
Binary object representation.

If user still wants to work with class object representation, SHARED or
CONTINUOUS deployment mode is still a requirement for the same reasons it
was a requirement before.

2015-11-04 11:48 GMT+03:00 Dmitriy Setrakyan <[hidden email]>:

> Igniters,
>
> I want to bring your attention to my email response on the user list. I
> just want to confirm that we are all on the same page and that my
> understanding is correct.
>
> I think that once we merge the BinaryObject to the release branch, we
> should remove the restriction on SHARED or CONTINUOUS deployment modes in
> cache, assuming that our own Binary marshaling is used.
>
> Alexey G, given that you are in charge of this merge, can you please make
> this change as well?
>
> Thanks,
> D.
>
> ---------- Forwarded message ----------
> From: Dmitriy Setrakyan <[hidden email]>
> Date: Tue, Nov 3, 2015 at 11:13 AM
> Subject: Re: Distributed queue problem with peerClassLoading enabled
> To: user <[hidden email]>
>
>
> Hi Mateusz,
>
> In the upcoming Ignite 1.5 release, we will not be storing user-classes on
> the server side at all, so the need for deployment of enqueued objects goes
> away.
>
> With that in mind, we will be removing the requirement for caches to work
> only with SHARED and CONTINUOUS deployment modes, so you will be able to
> use PRIVATE or ISOLATED deployment modes to deploy your computations.
>
> We are currently actively working on finalizing the 1.5 release, so it
> should be coming real soon.
>
> D.
>
> On Tue, Nov 3, 2015 at 3:05 AM, mp <[hidden email]> wrote:
>
> > Hi Denis,
> >
> > Thanks a lot for the clarifications.
> >
> > So, is there any way in Ignite to achieve my goal: different versions of
> > the same class running on the cluster (they can be deployed at the same
> > time from different client nodes)?
> > If yes, I can think of a way in which I can achieve my goals without
> using
> > the distributed queue.
> >
> > Cheers,
> > -Mateusz
> >
> >
> > On Tue, Nov 3, 2015 at 11:53 AM, Denis Magda <[hidden email]>
> wrote:
> >
> >> Hi Mateusz,
> >>
> >> Unfortunately I don't see any way to specify userVersion
> programmatically.
> >> In any case CONTINIOUS and SHARED mode is not suitable for your task
> >> because
> >> the developers may want to have different version of the same class.
> >> Please
> >> refer to DeploymentMode documentation for more info on different mode
> >> types.
> >>
> >> Do you really need to use IgniteQueue in your tests because, yes,
> neither
> >> PRIVATE nor ISOLATED modes are supported for user caches.
> >>
> >> --
> >> Denis
> >>
> >>
> >>
> >> --
> >> View this message in context:
> >>
> http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p1820.html
> >> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
> >>
> >
> >
>

[dsetrakyan]

On Wed, Nov 4, 2015 at 4:39 AM, Alexey Goncharuk <[hidden email]
> wrote:

> I am not sure what behavior we want to achieve here. Do we want to
> disable cache peer class loading for binary marshaling altogether?
> This restriction is safe to remove only if user chooses to work exclusively
> with Binary object representation.
>

Alexey, yes, you are right. Currently we have a restriction for PRIVATE or
ISOLATED deployment modes in caches. I am suggesting we should remove this
restriction and add a warning in the log about using only Binary
representation.


> If user still wants to work with class object representation, SHARED
> or CONTINUOUS deployment mode is still a requirement for the same reasons
> it was a requirement before.
>

Agree.

>
> 2015-11-04 11:48 GMT+03:00 Dmitriy Setrakyan <[hidden email]>:
>
> > Igniters,
> >
> > I want to bring your attention to my email response on the user list. I
> > just want to confirm that we are all on the same page and that my
> > understanding is correct.
> >
> > I think that once we merge the BinaryObject to the release branch, we
> > should remove the restriction on SHARED or CONTINUOUS deployment modes in
> > cache, assuming that our own Binary marshaling is used.
> >
> > Alexey G, given that you are in charge of this merge, can you please make
> > this change as well?
> >
> > Thanks,
> > D.
> >
> > ---------- Forwarded message ----------
> > From: Dmitriy Setrakyan <[hidden email]>
> > Date: Tue, Nov 3, 2015 at 11:13 AM
> > Subject: Re: Distributed queue problem with peerClassLoading enabled
> > To: user <[hidden email]>
> >
> >
> > Hi Mateusz,
> >
> > In the upcoming Ignite 1.5 release, we will not be storing user-classes
> on
> > the server side at all, so the need for deployment of enqueued objects
> goes
> > away.
> >
> > With that in mind, we will be removing the requirement for caches to work
> > only with SHARED and CONTINUOUS deployment modes, so you will be able to
> > use PRIVATE or ISOLATED deployment modes to deploy your computations.
> >
> > We are currently actively working on finalizing the 1.5 release, so it
> > should be coming real soon.
> >
> > D.
> >
> > On Tue, Nov 3, 2015 at 3:05 AM, mp <[hidden email]> wrote:
> >
> > > Hi Denis,
> > >
> > > Thanks a lot for the clarifications.
> > >
> > > So, is there any way in Ignite to achieve my goal: different versions
> of
> > > the same class running on the cluster (they can be deployed at the same
> > > time from different client nodes)?
> > > If yes, I can think of a way in which I can achieve my goals without
> > using
> > > the distributed queue.
> > >
> > > Cheers,
> > > -Mateusz
> > >
> > >
> > > On Tue, Nov 3, 2015 at 11:53 AM, Denis Magda <[hidden email]>
> > wrote:
> > >
> > >> Hi Mateusz,
> > >>
> > >> Unfortunately I don't see any way to specify userVersion
> > programmatically.
> > >> In any case CONTINIOUS and SHARED mode is not suitable for your task
> > >> because
> > >> the developers may want to have different version of the same class.
> > >> Please
> > >> refer to DeploymentMode documentation for more info on different mode
> > >> types.
> > >>
> > >> Do you really need to use IgniteQueue in your tests because, yes,
> > neither
> > >> PRIVATE nor ISOLATED modes are supported for user caches.
> > >>
> > >> --
> > >> Denis
> > >>
> > >>
> > >>
> > >> --
> > >> View this message in context:
> > >>
> >
> http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p1820.html
> > >> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
> > >>
> > >
> > >
> >
>

[Denis Magda]

Alexey, Dmitriy,

So the restriction (see discussion above) is still there in 1.5 release.

If we still want to remove it then it makes sense to create a JIRA ticket. Thoughts?

In addition if the restriction is removed then we have to take care of this bug that is related to unintentional deserialization of binary objects stored in a distributed queue
https://issues.apache.org/jira/browse/IGNITE-2339

--
Denis

[Denis Magda]

Ticket is created
https://issues.apache.org/jira/browse/IGNITE-2667

If anyone is ready to pick it up and fix then go ahead ;)

--
Denis