[ML] Deployment of user-defined preprocessors

[aplatonov]

Hi, Igniters!
Currently we don't have an ability to deploy automatically user-defined
preprocessors and vectorizers. Client's code should be deployed manually to
Ignite server nodes.

I have an idea how to fix it. If we pass user's classloader and one of
user-defined classes from fit-level to
ComputeUtils.affinityCallWithRetries() then we wiil be able to use
GridPeerDeployAware interface to send informtation about this classloader
to server nodes.

To support this ability we can define interfaces like these:

public interface DeployableObject {
    public List<Object> getDependencies();
}

and

public interface DeployingContext {
    public Class<?> userClass();
    public ClassLoader clientClassLoader();
}

DeployableObject will be mark for our ignite-ml final classes like trainers
or concrete preprocessors and it can be able to return all dependencies
that should be deployed to server nodes if it's needed. If these
dependencies are DeployableObjects too then depenndencies will be unfolded
recursively. Classes that isn't defined as DeployableObject will be
recognized as user-defined (NOTE: all leaf classes in our hierarchy will be
DeployableObject).

This list of DeployableObjects will be user for define user class loader
and one of these objects will be used for passing to GridPeerDeployAware.

So, this logic allows to pass user-defined Preprocessors and Vectorizers to
training algorithms and pipelines.

What do you think?

Sincerely
Alexey Platonov

[Alexey Zinoviev]

It sounds great, could you please explain what are you going to change in
the main Vectorizer/Preprocessor API to support it?

пт, 31 мая 2019 г. в 14:20, Алексей Платонов <[hidden email]>:

> Hi, Igniters!
> Currently we don't have an ability to deploy automatically user-defined
> preprocessors and vectorizers. Client's code should be deployed manually to
> Ignite server nodes.
>
> I have an idea how to fix it. If we pass user's classloader and one of
> user-defined classes from fit-level to
> ComputeUtils.affinityCallWithRetries() then we wiil be able to use
> GridPeerDeployAware interface to send informtation about this classloader
> to server nodes.
>
> To support this ability we can define interfaces like these:
>
> public interface DeployableObject {
>     public List<Object> getDependencies();
> }
>
> and
>
> public interface DeployingContext {
>     public Class<?> userClass();
>     public ClassLoader clientClassLoader();
> }
>
> DeployableObject will be mark for our ignite-ml final classes like trainers
> or concrete preprocessors and it can be able to return all dependencies
> that should be deployed to server nodes if it's needed. If these
> dependencies are DeployableObjects too then depenndencies will be unfolded
> recursively. Classes that isn't defined as DeployableObject will be
> recognized as user-defined (NOTE: all leaf classes in our hierarchy will be
> DeployableObject).
>
> This list of DeployableObjects will be user for define user class loader
> and one of these objects will be used for passing to GridPeerDeployAware.
>
> So, this logic allows to pass user-defined Preprocessors and Vectorizers to
> training algorithms and pipelines.
>
> What do you think?
>
> Sincerely
> Alexey Platonov
>

[aplatonov]

No, I won't change them.
In context of this architecture we should pass learning environment to
ComputeUtils and this concerns just DatasetBuilders and ComputeUtils APIs.
APIs of Preprocessors and Trainers won't be changed. We just set
DeploymentContext in fit method but it doens't require to change APIs of
these classes.

Sincerely
Alexey Platonov

пт, 31 мая 2019 г. в 14:51, Alexey Zinoviev <[hidden email]>:

> It sounds great, could you please explain what are you going to change in
> the main Vectorizer/Preprocessor API to support it?
>
> пт, 31 мая 2019 г. в 14:20, Алексей Платонов <[hidden email]>:
>
> > Hi, Igniters!
> > Currently we don't have an ability to deploy automatically user-defined
> > preprocessors and vectorizers. Client's code should be deployed manually
> to
> > Ignite server nodes.
> >
> > I have an idea how to fix it. If we pass user's classloader and one of
> > user-defined classes from fit-level to
> > ComputeUtils.affinityCallWithRetries() then we wiil be able to use
> > GridPeerDeployAware interface to send informtation about this classloader
> > to server nodes.
> >
> > To support this ability we can define interfaces like these:
> >
> > public interface DeployableObject {
> >     public List<Object> getDependencies();
> > }
> >
> > and
> >
> > public interface DeployingContext {
> >     public Class<?> userClass();
> >     public ClassLoader clientClassLoader();
> > }
> >
> > DeployableObject will be mark for our ignite-ml final classes like
> trainers
> > or concrete preprocessors and it can be able to return all dependencies
> > that should be deployed to server nodes if it's needed. If these
> > dependencies are DeployableObjects too then depenndencies will be
> unfolded
> > recursively. Classes that isn't defined as DeployableObject will be
> > recognized as user-defined (NOTE: all leaf classes in our hierarchy will
> be
> > DeployableObject).
> >
> > This list of DeployableObjects will be user for define user class loader
> > and one of these objects will be used for passing to GridPeerDeployAware.
> >
> > So, this logic allows to pass user-defined Preprocessors and Vectorizers
> to
> > training algorithms and pipelines.
> >
> > What do you think?
> >
> > Sincerely
> > Alexey Platonov
> >
>

[Alexey Zinoviev]

I'd like this approach, can't wait to review your PR!

пт, 31 мая 2019 г. в 15:33, Алексей Платонов <[hidden email]>:

> No, I won't change them.
> In context of this architecture we should pass learning environment to
> ComputeUtils and this concerns just DatasetBuilders and ComputeUtils APIs.
> APIs of Preprocessors and Trainers won't be changed. We just set
> DeploymentContext in fit method but it doens't require to change APIs of
> these classes.
>
> Sincerely
> Alexey Platonov
>
> пт, 31 мая 2019 г. в 14:51, Alexey Zinoviev <[hidden email]>:
>
> > It sounds great, could you please explain what are you going to change in
> > the main Vectorizer/Preprocessor API to support it?
> >
> > пт, 31 мая 2019 г. в 14:20, Алексей Платонов <[hidden email]>:
> >
> > > Hi, Igniters!
> > > Currently we don't have an ability to deploy automatically user-defined
> > > preprocessors and vectorizers. Client's code should be deployed
> manually
> > to
> > > Ignite server nodes.
> > >
> > > I have an idea how to fix it. If we pass user's classloader and one of
> > > user-defined classes from fit-level to
> > > ComputeUtils.affinityCallWithRetries() then we wiil be able to use
> > > GridPeerDeployAware interface to send informtation about this
> classloader
> > > to server nodes.
> > >
> > > To support this ability we can define interfaces like these:
> > >
> > > public interface DeployableObject {
> > >     public List<Object> getDependencies();
> > > }
> > >
> > > and
> > >
> > > public interface DeployingContext {
> > >     public Class<?> userClass();
> > >     public ClassLoader clientClassLoader();
> > > }
> > >
> > > DeployableObject will be mark for our ignite-ml final classes like
> > trainers
> > > or concrete preprocessors and it can be able to return all dependencies
> > > that should be deployed to server nodes if it's needed. If these
> > > dependencies are DeployableObjects too then depenndencies will be
> > unfolded
> > > recursively. Classes that isn't defined as DeployableObject will be
> > > recognized as user-defined (NOTE: all leaf classes in our hierarchy
> will
> > be
> > > DeployableObject).
> > >
> > > This list of DeployableObjects will be user for define user class
> loader
> > > and one of these objects will be used for passing to
> GridPeerDeployAware.
> > >
> > > So, this logic allows to pass user-defined Preprocessors and
> Vectorizers
> > to
> > > training algorithms and pipelines.
> > >
> > > What do you think?
> > >
> > > Sincerely
> > > Alexey Platonov
> > >
> >
>

[dmitrievanthony]

It's an amazing idea! I faced this problem several times and it's really
annoying, I'd be glad to this problem fixed.

Best regards,
Anton Dmitriev.



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

[Yuriy Babak]

Alexey,

This is a cool change, do you create a ticket for it?

If no I can create one.

Best regards,
Yuriy Babak


пт, 31 мая 2019 г. в 14:20, Алексей Платонов <[hidden email]>:

> Hi, Igniters!
> Currently we don't have an ability to deploy automatically user-defined
> preprocessors and vectorizers. Client's code should be deployed manually to
> Ignite server nodes.
>
> I have an idea how to fix it. If we pass user's classloader and one of
> user-defined classes from fit-level to
> ComputeUtils.affinityCallWithRetries() then we wiil be able to use
> GridPeerDeployAware interface to send informtation about this classloader
> to server nodes.
>
> To support this ability we can define interfaces like these:
>
> public interface DeployableObject {
>     public List<Object> getDependencies();
> }
>
> and
>
> public interface DeployingContext {
>     public Class<?> userClass();
>     public ClassLoader clientClassLoader();
> }
>
> DeployableObject will be mark for our ignite-ml final classes like trainers
> or concrete preprocessors and it can be able to return all dependencies
> that should be deployed to server nodes if it's needed. If these
> dependencies are DeployableObjects too then depenndencies will be unfolded
> recursively. Classes that isn't defined as DeployableObject will be
> recognized as user-defined (NOTE: all leaf classes in our hierarchy will be
> DeployableObject).
>
> This list of DeployableObjects will be user for define user class loader
> and one of these objects will be used for passing to GridPeerDeployAware.
>
> So, this logic allows to pass user-defined Preprocessors and Vectorizers to
> training algorithms and pipelines.
>
> What do you think?
>
> Sincerely
> Alexey Platonov
>

[aplatonov]

Hi, Igniters!
I've prepared a PR with fixes to correct peer class loading for ML-lambdas.
Here it is: https://github.com/apache/ignite/pull/6853

It will be great if someone reviews my PR. Alexey, Yuriy, could you please
to see this contribution.

Thanks!

Best regards.
Alexey Platonov

ср, 5 июн. 2019 г. в 17:20, Yuriy Babak <[hidden email]>:

> Alexey,
>
> This is a cool change, do you create a ticket for it?
>
> If no I can create one.
>
> Best regards,
> Yuriy Babak
>
>
> пт, 31 мая 2019 г. в 14:20, Алексей Платонов <[hidden email]>:
>
> > Hi, Igniters!
> > Currently we don't have an ability to deploy automatically user-defined
> > preprocessors and vectorizers. Client's code should be deployed manually
> to
> > Ignite server nodes.
> >
> > I have an idea how to fix it. If we pass user's classloader and one of
> > user-defined classes from fit-level to
> > ComputeUtils.affinityCallWithRetries() then we wiil be able to use
> > GridPeerDeployAware interface to send informtation about this classloader
> > to server nodes.
> >
> > To support this ability we can define interfaces like these:
> >
> > public interface DeployableObject {
> >     public List<Object> getDependencies();
> > }
> >
> > and
> >
> > public interface DeployingContext {
> >     public Class<?> userClass();
> >     public ClassLoader clientClassLoader();
> > }
> >
> > DeployableObject will be mark for our ignite-ml final classes like
> trainers
> > or concrete preprocessors and it can be able to return all dependencies
> > that should be deployed to server nodes if it's needed. If these
> > dependencies are DeployableObjects too then depenndencies will be
> unfolded
> > recursively. Classes that isn't defined as DeployableObject will be
> > recognized as user-defined (NOTE: all leaf classes in our hierarchy will
> be
> > DeployableObject).
> >
> > This list of DeployableObjects will be user for define user class loader
> > and one of these objects will be used for passing to GridPeerDeployAware.
> >
> > So, this logic allows to pass user-defined Preprocessors and Vectorizers
> to
> > training algorithms and pipelines.
> >
> > What do you think?
> >
> > Sincerely
> > Alexey Platonov
> >
>

[Alexey Zinoviev]

Thank you for the contribution, will make a review this week

вс, 8 сент. 2019 г., 10:05 Алексей Платонов <[hidden email]>:

> Hi, Igniters!
> I've prepared a PR with fixes to correct peer class loading for ML-lambdas.
> Here it is: https://github.com/apache/ignite/pull/6853
>
> It will be great if someone reviews my PR. Alexey, Yuriy, could you please
> to see this contribution.
>
> Thanks!
>
> Best regards.
> Alexey Platonov
>
> ср, 5 июн. 2019 г. в 17:20, Yuriy Babak <[hidden email]>:
>
> > Alexey,
> >
> > This is a cool change, do you create a ticket for it?
> >
> > If no I can create one.
> >
> > Best regards,
> > Yuriy Babak
> >
> >
> > пт, 31 мая 2019 г. в 14:20, Алексей Платонов <[hidden email]>:
> >
> > > Hi, Igniters!
> > > Currently we don't have an ability to deploy automatically user-defined
> > > preprocessors and vectorizers. Client's code should be deployed
> manually
> > to
> > > Ignite server nodes.
> > >
> > > I have an idea how to fix it. If we pass user's classloader and one of
> > > user-defined classes from fit-level to
> > > ComputeUtils.affinityCallWithRetries() then we wiil be able to use
> > > GridPeerDeployAware interface to send informtation about this
> classloader
> > > to server nodes.
> > >
> > > To support this ability we can define interfaces like these:
> > >
> > > public interface DeployableObject {
> > >     public List<Object> getDependencies();
> > > }
> > >
> > > and
> > >
> > > public interface DeployingContext {
> > >     public Class<?> userClass();
> > >     public ClassLoader clientClassLoader();
> > > }
> > >
> > > DeployableObject will be mark for our ignite-ml final classes like
> > trainers
> > > or concrete preprocessors and it can be able to return all dependencies
> > > that should be deployed to server nodes if it's needed. If these
> > > dependencies are DeployableObjects too then depenndencies will be
> > unfolded
> > > recursively. Classes that isn't defined as DeployableObject will be
> > > recognized as user-defined (NOTE: all leaf classes in our hierarchy
> will
> > be
> > > DeployableObject).
> > >
> > > This list of DeployableObjects will be user for define user class
> loader
> > > and one of these objects will be used for passing to
> GridPeerDeployAware.
> > >
> > > So, this logic allows to pass user-defined Preprocessors and
> Vectorizers
> > to
> > > training algorithms and pipelines.
> > >
> > > What do you think?
> > >
> > > Sincerely
> > > Alexey Platonov
> > >
> >
>