Improvements for P2P class loading

[Denis Garus]

Igniters!

I would like to propose a few improvements for the P2P class loading
feature in Ignite.
These improvements have the aim to reduce the number of requests that may
be needing to get a class on a remote node.

a. We should send a request for a class to the node initiator firstly.
Currently, GridDeploymentClassLoader sends a request for a class to all
participated nodes one by one until it gets a successful response. However,
in most cases, the required byte code would be loaded from the node that
initiates execution. To find out what is the node initiator, we can use the
way that we use to determine a security context to execute a user-defined
code (see GridIoManager#invokeListener). If the node initiator doesn't have
a required class, we should ask other participated nodes as it is currently.

b. Some classes in a single response.
When a required class contains anonymous and/or inner classes, Ignite tries
to get these classes using separate requests. However, we can determine
that case and send data, that we send anyway, in a single response as an
answer for the first request. In this way, we may significantly reduce the
number of communications required for a class loading.

What do you think?

I would like to create a separate task for a. and b.

[dmagda]

Denis,

Why do we broadcast the request to all the nodes in case of scenario a)?
There should be a reason for that.

Alex Goncharuk, do you remember we planned to revisit the P2P as part of
Ignite 3.0 to remove certain limitations? How about linking all the P2P
tasks together and create either an IEP or an umbrella ticket.

-
Denis


On Tue, Sep 24, 2019 at 12:55 AM Denis Garus <[hidden email]> wrote:

> Igniters!
>
> I would like to propose a few improvements for the P2P class loading
> feature in Ignite.
> These improvements have the aim to reduce the number of requests that may
> be needing to get a class on a remote node.
>
> a. We should send a request for a class to the node initiator firstly.
> Currently, GridDeploymentClassLoader sends a request for a class to all
> participated nodes one by one until it gets a successful response. However,
> in most cases, the required byte code would be loaded from the node that
> initiates execution. To find out what is the node initiator, we can use the
> way that we use to determine a security context to execute a user-defined
> code (see GridIoManager#invokeListener). If the node initiator doesn't have
> a required class, we should ask other participated nodes as it is
> currently.
>
> b. Some classes in a single response.
> When a required class contains anonymous and/or inner classes, Ignite tries
> to get these classes using separate requests. However, we can determine
> that case and send data, that we send anyway, in a single response as an
> answer for the first request. In this way, we may significantly reduce the
> number of communications required for a class loading.
>
> What do you think?
>
> I would like to create a separate task for a. and b.
>

[Denis Garus]

Denis, yes you are right, there is a reason for this.
It's the case when one of a participated node leaves a cluster and loader
send a request to the next node.
Also, the list of participated nodes is limited enough.
Thus, the proposed improvement a) is wrong.

But I think the point b) is still actual.

Thank you for the feedback!

ср, 25 сент. 2019 г. в 01:32, Denis Magda <[hidden email]>:

> Denis,
>
> Why do we broadcast the request to all the nodes in case of scenario a)?
> There should be a reason for that.
>
> Alex Goncharuk, do you remember we planned to revisit the P2P as part of
> Ignite 3.0 to remove certain limitations? How about linking all the P2P
> tasks together and create either an IEP or an umbrella ticket.
>
> -
> Denis
>
>
> On Tue, Sep 24, 2019 at 12:55 AM Denis Garus <[hidden email]> wrote:
>
> > Igniters!
> >
> > I would like to propose a few improvements for the P2P class loading
> > feature in Ignite.
> > These improvements have the aim to reduce the number of requests that may
> > be needing to get a class on a remote node.
> >
> > a. We should send a request for a class to the node initiator firstly.
> > Currently, GridDeploymentClassLoader sends a request for a class to all
> > participated nodes one by one until it gets a successful response.
> However,
> > in most cases, the required byte code would be loaded from the node that
> > initiates execution. To find out what is the node initiator, we can use
> the
> > way that we use to determine a security context to execute a user-defined
> > code (see GridIoManager#invokeListener). If the node initiator doesn't
> have
> > a required class, we should ask other participated nodes as it is
> > currently.
> >
> > b. Some classes in a single response.
> > When a required class contains anonymous and/or inner classes, Ignite
> tries
> > to get these classes using separate requests. However, we can determine
> > that case and send data, that we send anyway, in a single response as an
> > answer for the first request. In this way, we may significantly reduce
> the
> > number of communications required for a class loading.
> >
> > What do you think?
> >
> > I would like to create a separate task for a. and b.
> >
>