changes in marshaling and SQL

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

changes in marshaling and SQL

dsetrakyan
Igniters,

As a part of Denis Magda's work on IGNITE-950
<https://issues.apache.org/jira/browse/IGNITE-950>, we are not going to
require that server side nodes have class definitions of user classes, as
we are going to keep them in the binary format. This is a BIG deal, as now
users will not have to copy their application JAR files to the server nodes.

However, this change also affects the way we support queries. Previously,
we simply extracted fields using reflection to do index lookups, but now we
cannot use reflection, since we do not have class definitions on the
servers anymore.

As a result, the following restrictions are going to be introduced to the
query processing:


   1. @QuerySqlField annotation can't be set to a getter method anymore,
   only to fields.
   2. Comparable on _key and _value type will not work anymore. However,
   "_key" and "_value" types are auto-generated and mostly used internally by
   Ignite itself.


Generally, I believe that both (1) and (2) are not a big deal. However, I
would like to confirm that for the (2), we can still retrieve and sort
primitive values and Strings natively.

Serj, given that you are working on the query changes, can you confirm the
(2)?

D.
Reply | Threaded
Open this post in threaded view
|

Re: changes in marshaling and SQL

Alexey Goncharuk
I have a concern regarding (1): there was a couple of user use-cases in the
past that we were able to solve by putting the annotation to a getter
method, in particular this one:
http://stackoverflow.com/questions/27352482/gridgain-query-an-object-which-has-a-list-of-items/

Will there be another workaround to address questions like that given that
the annotation is not allowed for methods?

2015-06-25 9:28 GMT-07:00 Dmitriy Setrakyan <[hidden email]>:

> Igniters,
>
> As a part of Denis Magda's work on IGNITE-950
> <https://issues.apache.org/jira/browse/IGNITE-950>, we are not going to
> require that server side nodes have class definitions of user classes, as
> we are going to keep them in the binary format. This is a BIG deal, as now
> users will not have to copy their application JAR files to the server
> nodes.
>
> However, this change also affects the way we support queries. Previously,
> we simply extracted fields using reflection to do index lookups, but now we
> cannot use reflection, since we do not have class definitions on the
> servers anymore.
>
> As a result, the following restrictions are going to be introduced to the
> query processing:
>
>
>    1. @QuerySqlField annotation can't be set to a getter method anymore,
>    only to fields.
>    2. Comparable on _key and _value type will not work anymore. However,
>    "_key" and "_value" types are auto-generated and mostly used internally
> by
>    Ignite itself.
>
>
> Generally, I believe that both (1) and (2) are not a big deal. However, I
> would like to confirm that for the (2), we can still retrieve and sort
> primitive values and Strings natively.
>
> Serj, given that you are working on the query changes, can you confirm the
> (2)?
>
> D.
>
Reply | Threaded
Open this post in threaded view
|

Re: changes in marshaling and SQL

Semyon Boikov
In reply to this post by dsetrakyan
Just want to remind that deserialized value is needed not only for
indexing, now deserialized value is passed to EntryProcessors, cache
interceptors, cache store, cache events, scan query, continuous query
filter, so if user uses one of these features class still will be needed on
servers.

On Thu, Jun 25, 2015 at 7:28 PM, Dmitriy Setrakyan <[hidden email]>
wrote:

> Igniters,
>
> As a part of Denis Magda's work on IGNITE-950
> <https://issues.apache.org/jira/browse/IGNITE-950>, we are not going to
> require that server side nodes have class definitions of user classes, as
> we are going to keep them in the binary format. This is a BIG deal, as now
> users will not have to copy their application JAR files to the server
> nodes.
>
> However, this change also affects the way we support queries. Previously,
> we simply extracted fields using reflection to do index lookups, but now we
> cannot use reflection, since we do not have class definitions on the
> servers anymore.
>
> As a result, the following restrictions are going to be introduced to the
> query processing:
>
>
>    1. @QuerySqlField annotation can't be set to a getter method anymore,
>    only to fields.
>    2. Comparable on _key and _value type will not work anymore. However,
>    "_key" and "_value" types are auto-generated and mostly used internally
> by
>    Ignite itself.
>
>
> Generally, I believe that both (1) and (2) are not a big deal. However, I
> would like to confirm that for the (2), we can still retrieve and sort
> primitive values and Strings natively.
>
> Serj, given that you are working on the query changes, can you confirm the
> (2)?
>
> D.
>
Reply | Threaded
Open this post in threaded view
|

Re: changes in marshaling and SQL

Atri Sharma
I am not too well versed with this area yet, but am not sure why we need
the entire class definition to be present for getting the deserialized
value of a class object.

It should be enough to register serialier/deserializer to/from binary
format and let server nodes call these on demand (maybe some sort of global
catalog).

Unless I am missing something.

On Fri, Jun 26, 2015 at 11:51 AM, Semyon Boikov <[hidden email]>
wrote:

> Just want to remind that deserialized value is needed not only for
> indexing, now deserialized value is passed to EntryProcessors, cache
> interceptors, cache store, cache events, scan query, continuous query
> filter, so if user uses one of these features class still will be needed on
> servers.
>
> On Thu, Jun 25, 2015 at 7:28 PM, Dmitriy Setrakyan <[hidden email]>
> wrote:
>
> > Igniters,
> >
> > As a part of Denis Magda's work on IGNITE-950
> > <https://issues.apache.org/jira/browse/IGNITE-950>, we are not going to
> > require that server side nodes have class definitions of user classes, as
> > we are going to keep them in the binary format. This is a BIG deal, as
> now
> > users will not have to copy their application JAR files to the server
> > nodes.
> >
> > However, this change also affects the way we support queries. Previously,
> > we simply extracted fields using reflection to do index lookups, but now
> we
> > cannot use reflection, since we do not have class definitions on the
> > servers anymore.
> >
> > As a result, the following restrictions are going to be introduced to the
> > query processing:
> >
> >
> >    1. @QuerySqlField annotation can't be set to a getter method anymore,
> >    only to fields.
> >    2. Comparable on _key and _value type will not work anymore. However,
> >    "_key" and "_value" types are auto-generated and mostly used
> internally
> > by
> >    Ignite itself.
> >
> >
> > Generally, I believe that both (1) and (2) are not a big deal. However, I
> > would like to confirm that for the (2), we can still retrieve and sort
> > primitive values and Strings natively.
> >
> > Serj, given that you are working on the query changes, can you confirm
> the
> > (2)?
> >
> > D.
> >
>



--
Regards,

Atri
*l'apprenant*
Reply | Threaded
Open this post in threaded view
|

Re: changes in marshaling and SQL

dsetrakyan
In reply to this post by Semyon Boikov
On Thu, Jun 25, 2015 at 11:21 PM, Semyon Boikov <[hidden email]>
wrote:

> Just want to remind that deserialized value is needed not only for
> indexing, now deserialized value is passed to EntryProcessors, cache
> interceptors, cache store, cache events, scan query, continuous query
> filter, so if user uses one of these features class still will be needed on
> servers.
>

Semyon, I don't think we need to deserialize in this case, but rather pass
some CacheObject wrapper around the binary format.

D.


>
> On Thu, Jun 25, 2015 at 7:28 PM, Dmitriy Setrakyan <[hidden email]>
> wrote:
>
> > Igniters,
> >
> > As a part of Denis Magda's work on IGNITE-950
> > <https://issues.apache.org/jira/browse/IGNITE-950>, we are not going to
> > require that server side nodes have class definitions of user classes, as
> > we are going to keep them in the binary format. This is a BIG deal, as
> now
> > users will not have to copy their application JAR files to the server
> > nodes.
> >
> > However, this change also affects the way we support queries. Previously,
> > we simply extracted fields using reflection to do index lookups, but now
> we
> > cannot use reflection, since we do not have class definitions on the
> > servers anymore.
> >
> > As a result, the following restrictions are going to be introduced to the
> > query processing:
> >
> >
> >    1. @QuerySqlField annotation can't be set to a getter method anymore,
> >    only to fields.
> >    2. Comparable on _key and _value type will not work anymore. However,
> >    "_key" and "_value" types are auto-generated and mostly used
> internally
> > by
> >    Ignite itself.
> >
> >
> > Generally, I believe that both (1) and (2) are not a big deal. However, I
> > would like to confirm that for the (2), we can still retrieve and sort
> > primitive values and Strings natively.
> >
> > Serj, given that you are working on the query changes, can you confirm
> the
> > (2)?
> >
> > D.
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: changes in marshaling and SQL

Semyon Boikov
Ok, in this case we need to design this wrapper API, also this should be
somehow configurable so user still should be able to work with real classes.

On Fri, Jun 26, 2015 at 10:50 AM, Dmitriy Setrakyan <[hidden email]>
wrote:

> On Thu, Jun 25, 2015 at 11:21 PM, Semyon Boikov <[hidden email]>
> wrote:
>
> > Just want to remind that deserialized value is needed not only for
> > indexing, now deserialized value is passed to EntryProcessors, cache
> > interceptors, cache store, cache events, scan query, continuous query
> > filter, so if user uses one of these features class still will be needed
> on
> > servers.
> >
>
> Semyon, I don't think we need to deserialize in this case, but rather pass
> some CacheObject wrapper around the binary format.
>
> D.
>
>
> >
> > On Thu, Jun 25, 2015 at 7:28 PM, Dmitriy Setrakyan <
> [hidden email]>
> > wrote:
> >
> > > Igniters,
> > >
> > > As a part of Denis Magda's work on IGNITE-950
> > > <https://issues.apache.org/jira/browse/IGNITE-950>, we are not going
> to
> > > require that server side nodes have class definitions of user classes,
> as
> > > we are going to keep them in the binary format. This is a BIG deal, as
> > now
> > > users will not have to copy their application JAR files to the server
> > > nodes.
> > >
> > > However, this change also affects the way we support queries.
> Previously,
> > > we simply extracted fields using reflection to do index lookups, but
> now
> > we
> > > cannot use reflection, since we do not have class definitions on the
> > > servers anymore.
> > >
> > > As a result, the following restrictions are going to be introduced to
> the
> > > query processing:
> > >
> > >
> > >    1. @QuerySqlField annotation can't be set to a getter method
> anymore,
> > >    only to fields.
> > >    2. Comparable on _key and _value type will not work anymore.
> However,
> > >    "_key" and "_value" types are auto-generated and mostly used
> > internally
> > > by
> > >    Ignite itself.
> > >
> > >
> > > Generally, I believe that both (1) and (2) are not a big deal.
> However, I
> > > would like to confirm that for the (2), we can still retrieve and sort
> > > primitive values and Strings natively.
> > >
> > > Serj, given that you are working on the query changes, can you confirm
> > the
> > > (2)?
> > >
> > > D.
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: changes in marshaling and SQL

Vladimir Ozerov
+1 for Semen point.

Is it really popular scenario when user does not have classes on server?
Looks like a corner case for me.

On Fri, Jun 26, 2015 at 11:40 AM, Semyon Boikov <[hidden email]>
wrote:

> Ok, in this case we need to design this wrapper API, also this should be
> somehow configurable so user still should be able to work with real
> classes.
>
> On Fri, Jun 26, 2015 at 10:50 AM, Dmitriy Setrakyan <[hidden email]
> >
> wrote:
>
> > On Thu, Jun 25, 2015 at 11:21 PM, Semyon Boikov <[hidden email]>
> > wrote:
> >
> > > Just want to remind that deserialized value is needed not only for
> > > indexing, now deserialized value is passed to EntryProcessors, cache
> > > interceptors, cache store, cache events, scan query, continuous query
> > > filter, so if user uses one of these features class still will be
> needed
> > on
> > > servers.
> > >
> >
> > Semyon, I don't think we need to deserialize in this case, but rather
> pass
> > some CacheObject wrapper around the binary format.
> >
> > D.
> >
> >
> > >
> > > On Thu, Jun 25, 2015 at 7:28 PM, Dmitriy Setrakyan <
> > [hidden email]>
> > > wrote:
> > >
> > > > Igniters,
> > > >
> > > > As a part of Denis Magda's work on IGNITE-950
> > > > <https://issues.apache.org/jira/browse/IGNITE-950>, we are not going
> > to
> > > > require that server side nodes have class definitions of user
> classes,
> > as
> > > > we are going to keep them in the binary format. This is a BIG deal,
> as
> > > now
> > > > users will not have to copy their application JAR files to the server
> > > > nodes.
> > > >
> > > > However, this change also affects the way we support queries.
> > Previously,
> > > > we simply extracted fields using reflection to do index lookups, but
> > now
> > > we
> > > > cannot use reflection, since we do not have class definitions on the
> > > > servers anymore.
> > > >
> > > > As a result, the following restrictions are going to be introduced to
> > the
> > > > query processing:
> > > >
> > > >
> > > >    1. @QuerySqlField annotation can't be set to a getter method
> > anymore,
> > > >    only to fields.
> > > >    2. Comparable on _key and _value type will not work anymore.
> > However,
> > > >    "_key" and "_value" types are auto-generated and mostly used
> > > internally
> > > > by
> > > >    Ignite itself.
> > > >
> > > >
> > > > Generally, I believe that both (1) and (2) are not a big deal.
> > However, I
> > > > would like to confirm that for the (2), we can still retrieve and
> sort
> > > > primitive values and Strings natively.
> > > >
> > > > Serj, given that you are working on the query changes, can you
> confirm
> > > the
> > > > (2)?
> > > >
> > > > D.
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: changes in marshaling and SQL

dsetrakyan
Can we repurpose CacheObject API for this?

On Fri, Jun 26, 2015 at 1:55 AM, Vladimir Ozerov <[hidden email]>
wrote:

> +1 for Semen point.
>
> Is it really popular scenario when user does not have classes on server?
> Looks like a corner case for me.
>
> On Fri, Jun 26, 2015 at 11:40 AM, Semyon Boikov <[hidden email]>
> wrote:
>
> > Ok, in this case we need to design this wrapper API, also this should be
> > somehow configurable so user still should be able to work with real
> > classes.
> >
> > On Fri, Jun 26, 2015 at 10:50 AM, Dmitriy Setrakyan <
> [hidden email]
> > >
> > wrote:
> >
> > > On Thu, Jun 25, 2015 at 11:21 PM, Semyon Boikov <[hidden email]>
> > > wrote:
> > >
> > > > Just want to remind that deserialized value is needed not only for
> > > > indexing, now deserialized value is passed to EntryProcessors, cache
> > > > interceptors, cache store, cache events, scan query, continuous query
> > > > filter, so if user uses one of these features class still will be
> > needed
> > > on
> > > > servers.
> > > >
> > >
> > > Semyon, I don't think we need to deserialize in this case, but rather
> > pass
> > > some CacheObject wrapper around the binary format.
> > >
> > > D.
> > >
> > >
> > > >
> > > > On Thu, Jun 25, 2015 at 7:28 PM, Dmitriy Setrakyan <
> > > [hidden email]>
> > > > wrote:
> > > >
> > > > > Igniters,
> > > > >
> > > > > As a part of Denis Magda's work on IGNITE-950
> > > > > <https://issues.apache.org/jira/browse/IGNITE-950>, we are not
> going
> > > to
> > > > > require that server side nodes have class definitions of user
> > classes,
> > > as
> > > > > we are going to keep them in the binary format. This is a BIG deal,
> > as
> > > > now
> > > > > users will not have to copy their application JAR files to the
> server
> > > > > nodes.
> > > > >
> > > > > However, this change also affects the way we support queries.
> > > Previously,
> > > > > we simply extracted fields using reflection to do index lookups,
> but
> > > now
> > > > we
> > > > > cannot use reflection, since we do not have class definitions on
> the
> > > > > servers anymore.
> > > > >
> > > > > As a result, the following restrictions are going to be introduced
> to
> > > the
> > > > > query processing:
> > > > >
> > > > >
> > > > >    1. @QuerySqlField annotation can't be set to a getter method
> > > anymore,
> > > > >    only to fields.
> > > > >    2. Comparable on _key and _value type will not work anymore.
> > > However,
> > > > >    "_key" and "_value" types are auto-generated and mostly used
> > > > internally
> > > > > by
> > > > >    Ignite itself.
> > > > >
> > > > >
> > > > > Generally, I believe that both (1) and (2) are not a big deal.
> > > However, I
> > > > > would like to confirm that for the (2), we can still retrieve and
> > sort
> > > > > primitive values and Strings natively.
> > > > >
> > > > > Serj, given that you are working on the query changes, can you
> > confirm
> > > > the
> > > > > (2)?
> > > > >
> > > > > D.
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: changes in marshaling and SQL

Semyon Boikov
I don't think so, this is internal API and now it doesn't even have method
to get field by name.

On Fri, Jun 26, 2015 at 12:02 PM, Dmitriy Setrakyan <[hidden email]>
wrote:

> Can we repurpose CacheObject API for this?
>
> On Fri, Jun 26, 2015 at 1:55 AM, Vladimir Ozerov <[hidden email]>
> wrote:
>
> > +1 for Semen point.
> >
> > Is it really popular scenario when user does not have classes on server?
> > Looks like a corner case for me.
> >
> > On Fri, Jun 26, 2015 at 11:40 AM, Semyon Boikov <[hidden email]>
> > wrote:
> >
> > > Ok, in this case we need to design this wrapper API, also this should
> be
> > > somehow configurable so user still should be able to work with real
> > > classes.
> > >
> > > On Fri, Jun 26, 2015 at 10:50 AM, Dmitriy Setrakyan <
> > [hidden email]
> > > >
> > > wrote:
> > >
> > > > On Thu, Jun 25, 2015 at 11:21 PM, Semyon Boikov <
> [hidden email]>
> > > > wrote:
> > > >
> > > > > Just want to remind that deserialized value is needed not only for
> > > > > indexing, now deserialized value is passed to EntryProcessors,
> cache
> > > > > interceptors, cache store, cache events, scan query, continuous
> query
> > > > > filter, so if user uses one of these features class still will be
> > > needed
> > > > on
> > > > > servers.
> > > > >
> > > >
> > > > Semyon, I don't think we need to deserialize in this case, but rather
> > > pass
> > > > some CacheObject wrapper around the binary format.
> > > >
> > > > D.
> > > >
> > > >
> > > > >
> > > > > On Thu, Jun 25, 2015 at 7:28 PM, Dmitriy Setrakyan <
> > > > [hidden email]>
> > > > > wrote:
> > > > >
> > > > > > Igniters,
> > > > > >
> > > > > > As a part of Denis Magda's work on IGNITE-950
> > > > > > <https://issues.apache.org/jira/browse/IGNITE-950>, we are not
> > going
> > > > to
> > > > > > require that server side nodes have class definitions of user
> > > classes,
> > > > as
> > > > > > we are going to keep them in the binary format. This is a BIG
> deal,
> > > as
> > > > > now
> > > > > > users will not have to copy their application JAR files to the
> > server
> > > > > > nodes.
> > > > > >
> > > > > > However, this change also affects the way we support queries.
> > > > Previously,
> > > > > > we simply extracted fields using reflection to do index lookups,
> > but
> > > > now
> > > > > we
> > > > > > cannot use reflection, since we do not have class definitions on
> > the
> > > > > > servers anymore.
> > > > > >
> > > > > > As a result, the following restrictions are going to be
> introduced
> > to
> > > > the
> > > > > > query processing:
> > > > > >
> > > > > >
> > > > > >    1. @QuerySqlField annotation can't be set to a getter method
> > > > anymore,
> > > > > >    only to fields.
> > > > > >    2. Comparable on _key and _value type will not work anymore.
> > > > However,
> > > > > >    "_key" and "_value" types are auto-generated and mostly used
> > > > > internally
> > > > > > by
> > > > > >    Ignite itself.
> > > > > >
> > > > > >
> > > > > > Generally, I believe that both (1) and (2) are not a big deal.
> > > > However, I
> > > > > > would like to confirm that for the (2), we can still retrieve and
> > > sort
> > > > > > primitive values and Strings natively.
> > > > > >
> > > > > > Serj, given that you are working on the query changes, can you
> > > confirm
> > > > > the
> > > > > > (2)?
> > > > > >
> > > > > > D.
> > > > > >
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: changes in marshaling and SQL

Sergi
Guys,

Now I see that idea of not having classes on server node is definitely a
corner case. Why do we even want that other than for interoperability with
other platforms like .NET? I think we should carefully reconsider the whole
approach before we brake something.

Sergi

2015-06-26 13:13 GMT+03:00 Semyon Boikov <[hidden email]>:

> I don't think so, this is internal API and now it doesn't even have method
> to get field by name.
>
> On Fri, Jun 26, 2015 at 12:02 PM, Dmitriy Setrakyan <[hidden email]
> >
> wrote:
>
> > Can we repurpose CacheObject API for this?
> >
> > On Fri, Jun 26, 2015 at 1:55 AM, Vladimir Ozerov <[hidden email]>
> > wrote:
> >
> > > +1 for Semen point.
> > >
> > > Is it really popular scenario when user does not have classes on
> server?
> > > Looks like a corner case for me.
> > >
> > > On Fri, Jun 26, 2015 at 11:40 AM, Semyon Boikov <[hidden email]>
> > > wrote:
> > >
> > > > Ok, in this case we need to design this wrapper API, also this should
> > be
> > > > somehow configurable so user still should be able to work with real
> > > > classes.
> > > >
> > > > On Fri, Jun 26, 2015 at 10:50 AM, Dmitriy Setrakyan <
> > > [hidden email]
> > > > >
> > > > wrote:
> > > >
> > > > > On Thu, Jun 25, 2015 at 11:21 PM, Semyon Boikov <
> > [hidden email]>
> > > > > wrote:
> > > > >
> > > > > > Just want to remind that deserialized value is needed not only
> for
> > > > > > indexing, now deserialized value is passed to EntryProcessors,
> > cache
> > > > > > interceptors, cache store, cache events, scan query, continuous
> > query
> > > > > > filter, so if user uses one of these features class still will be
> > > > needed
> > > > > on
> > > > > > servers.
> > > > > >
> > > > >
> > > > > Semyon, I don't think we need to deserialize in this case, but
> rather
> > > > pass
> > > > > some CacheObject wrapper around the binary format.
> > > > >
> > > > > D.
> > > > >
> > > > >
> > > > > >
> > > > > > On Thu, Jun 25, 2015 at 7:28 PM, Dmitriy Setrakyan <
> > > > > [hidden email]>
> > > > > > wrote:
> > > > > >
> > > > > > > Igniters,
> > > > > > >
> > > > > > > As a part of Denis Magda's work on IGNITE-950
> > > > > > > <https://issues.apache.org/jira/browse/IGNITE-950>, we are not
> > > going
> > > > > to
> > > > > > > require that server side nodes have class definitions of user
> > > > classes,
> > > > > as
> > > > > > > we are going to keep them in the binary format. This is a BIG
> > deal,
> > > > as
> > > > > > now
> > > > > > > users will not have to copy their application JAR files to the
> > > server
> > > > > > > nodes.
> > > > > > >
> > > > > > > However, this change also affects the way we support queries.
> > > > > Previously,
> > > > > > > we simply extracted fields using reflection to do index
> lookups,
> > > but
> > > > > now
> > > > > > we
> > > > > > > cannot use reflection, since we do not have class definitions
> on
> > > the
> > > > > > > servers anymore.
> > > > > > >
> > > > > > > As a result, the following restrictions are going to be
> > introduced
> > > to
> > > > > the
> > > > > > > query processing:
> > > > > > >
> > > > > > >
> > > > > > >    1. @QuerySqlField annotation can't be set to a getter method
> > > > > anymore,
> > > > > > >    only to fields.
> > > > > > >    2. Comparable on _key and _value type will not work anymore.
> > > > > However,
> > > > > > >    "_key" and "_value" types are auto-generated and mostly used
> > > > > > internally
> > > > > > > by
> > > > > > >    Ignite itself.
> > > > > > >
> > > > > > >
> > > > > > > Generally, I believe that both (1) and (2) are not a big deal.
> > > > > However, I
> > > > > > > would like to confirm that for the (2), we can still retrieve
> and
> > > > sort
> > > > > > > primitive values and Strings natively.
> > > > > > >
> > > > > > > Serj, given that you are working on the query changes, can you
> > > > confirm
> > > > > > the
> > > > > > > (2)?
> > > > > > >
> > > > > > > D.
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: changes in marshaling and SQL

yzhdanov
Btw, not having classes on servers makes impossible affinity routed
computations. In this case all data should be processed on clients and then
marhsalled and send back to clients. Thoughts?

--Yakov

2015-06-26 14:16 GMT+03:00 Sergi Vladykin <[hidden email]>:

> Guys,
>
> Now I see that idea of not having classes on server node is definitely a
> corner case. Why do we even want that other than for interoperability with
> other platforms like .NET? I think we should carefully reconsider the whole
> approach before we brake something.
>
> Sergi
>
> 2015-06-26 13:13 GMT+03:00 Semyon Boikov <[hidden email]>:
>
> > I don't think so, this is internal API and now it doesn't even have
> method
> > to get field by name.
> >
> > On Fri, Jun 26, 2015 at 12:02 PM, Dmitriy Setrakyan <
> [hidden email]
> > >
> > wrote:
> >
> > > Can we repurpose CacheObject API for this?
> > >
> > > On Fri, Jun 26, 2015 at 1:55 AM, Vladimir Ozerov <[hidden email]
> >
> > > wrote:
> > >
> > > > +1 for Semen point.
> > > >
> > > > Is it really popular scenario when user does not have classes on
> > server?
> > > > Looks like a corner case for me.
> > > >
> > > > On Fri, Jun 26, 2015 at 11:40 AM, Semyon Boikov <
> [hidden email]>
> > > > wrote:
> > > >
> > > > > Ok, in this case we need to design this wrapper API, also this
> should
> > > be
> > > > > somehow configurable so user still should be able to work with real
> > > > > classes.
> > > > >
> > > > > On Fri, Jun 26, 2015 at 10:50 AM, Dmitriy Setrakyan <
> > > > [hidden email]
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > On Thu, Jun 25, 2015 at 11:21 PM, Semyon Boikov <
> > > [hidden email]>
> > > > > > wrote:
> > > > > >
> > > > > > > Just want to remind that deserialized value is needed not only
> > for
> > > > > > > indexing, now deserialized value is passed to EntryProcessors,
> > > cache
> > > > > > > interceptors, cache store, cache events, scan query, continuous
> > > query
> > > > > > > filter, so if user uses one of these features class still will
> be
> > > > > needed
> > > > > > on
> > > > > > > servers.
> > > > > > >
> > > > > >
> > > > > > Semyon, I don't think we need to deserialize in this case, but
> > rather
> > > > > pass
> > > > > > some CacheObject wrapper around the binary format.
> > > > > >
> > > > > > D.
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > On Thu, Jun 25, 2015 at 7:28 PM, Dmitriy Setrakyan <
> > > > > > [hidden email]>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Igniters,
> > > > > > > >
> > > > > > > > As a part of Denis Magda's work on IGNITE-950
> > > > > > > > <https://issues.apache.org/jira/browse/IGNITE-950>, we are
> not
> > > > going
> > > > > > to
> > > > > > > > require that server side nodes have class definitions of user
> > > > > classes,
> > > > > > as
> > > > > > > > we are going to keep them in the binary format. This is a BIG
> > > deal,
> > > > > as
> > > > > > > now
> > > > > > > > users will not have to copy their application JAR files to
> the
> > > > server
> > > > > > > > nodes.
> > > > > > > >
> > > > > > > > However, this change also affects the way we support queries.
> > > > > > Previously,
> > > > > > > > we simply extracted fields using reflection to do index
> > lookups,
> > > > but
> > > > > > now
> > > > > > > we
> > > > > > > > cannot use reflection, since we do not have class definitions
> > on
> > > > the
> > > > > > > > servers anymore.
> > > > > > > >
> > > > > > > > As a result, the following restrictions are going to be
> > > introduced
> > > > to
> > > > > > the
> > > > > > > > query processing:
> > > > > > > >
> > > > > > > >
> > > > > > > >    1. @QuerySqlField annotation can't be set to a getter
> method
> > > > > > anymore,
> > > > > > > >    only to fields.
> > > > > > > >    2. Comparable on _key and _value type will not work
> anymore.
> > > > > > However,
> > > > > > > >    "_key" and "_value" types are auto-generated and mostly
> used
> > > > > > > internally
> > > > > > > > by
> > > > > > > >    Ignite itself.
> > > > > > > >
> > > > > > > >
> > > > > > > > Generally, I believe that both (1) and (2) are not a big
> deal.
> > > > > > However, I
> > > > > > > > would like to confirm that for the (2), we can still retrieve
> > and
> > > > > sort
> > > > > > > > primitive values and Strings natively.
> > > > > > > >
> > > > > > > > Serj, given that you are working on the query changes, can
> you
> > > > > confirm
> > > > > > > the
> > > > > > > > (2)?
> > > > > > > >
> > > > > > > > D.
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: changes in marshaling and SQL

dsetrakyan
In reply to this post by Sergi
Sergey,

Classes do not disappear, they become optional. Without having classes,
certain things, like CacheInterceptor may not work, but not everyone uses
it anyway.

The work you are doing is to make sure that we are not deserializing
objects only for indexing. For  your purposes, I think you should assume
that there are no class definitions on the server side.

D.

On Fri, Jun 26, 2015 at 4:16 AM, Sergi Vladykin <[hidden email]>
wrote:

> Guys,
>
> Now I see that idea of not having classes on server node is definitely a
> corner case. Why do we even want that other than for interoperability with
> other platforms like .NET? I think we should carefully reconsider the whole
> approach before we brake something.
>
> Sergi
>
> 2015-06-26 13:13 GMT+03:00 Semyon Boikov <[hidden email]>:
>
> > I don't think so, this is internal API and now it doesn't even have
> method
> > to get field by name.
> >
> > On Fri, Jun 26, 2015 at 12:02 PM, Dmitriy Setrakyan <
> [hidden email]
> > >
> > wrote:
> >
> > > Can we repurpose CacheObject API for this?
> > >
> > > On Fri, Jun 26, 2015 at 1:55 AM, Vladimir Ozerov <[hidden email]
> >
> > > wrote:
> > >
> > > > +1 for Semen point.
> > > >
> > > > Is it really popular scenario when user does not have classes on
> > server?
> > > > Looks like a corner case for me.
> > > >
> > > > On Fri, Jun 26, 2015 at 11:40 AM, Semyon Boikov <
> [hidden email]>
> > > > wrote:
> > > >
> > > > > Ok, in this case we need to design this wrapper API, also this
> should
> > > be
> > > > > somehow configurable so user still should be able to work with real
> > > > > classes.
> > > > >
> > > > > On Fri, Jun 26, 2015 at 10:50 AM, Dmitriy Setrakyan <
> > > > [hidden email]
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > On Thu, Jun 25, 2015 at 11:21 PM, Semyon Boikov <
> > > [hidden email]>
> > > > > > wrote:
> > > > > >
> > > > > > > Just want to remind that deserialized value is needed not only
> > for
> > > > > > > indexing, now deserialized value is passed to EntryProcessors,
> > > cache
> > > > > > > interceptors, cache store, cache events, scan query, continuous
> > > query
> > > > > > > filter, so if user uses one of these features class still will
> be
> > > > > needed
> > > > > > on
> > > > > > > servers.
> > > > > > >
> > > > > >
> > > > > > Semyon, I don't think we need to deserialize in this case, but
> > rather
> > > > > pass
> > > > > > some CacheObject wrapper around the binary format.
> > > > > >
> > > > > > D.
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > On Thu, Jun 25, 2015 at 7:28 PM, Dmitriy Setrakyan <
> > > > > > [hidden email]>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Igniters,
> > > > > > > >
> > > > > > > > As a part of Denis Magda's work on IGNITE-950
> > > > > > > > <https://issues.apache.org/jira/browse/IGNITE-950>, we are
> not
> > > > going
> > > > > > to
> > > > > > > > require that server side nodes have class definitions of user
> > > > > classes,
> > > > > > as
> > > > > > > > we are going to keep them in the binary format. This is a BIG
> > > deal,
> > > > > as
> > > > > > > now
> > > > > > > > users will not have to copy their application JAR files to
> the
> > > > server
> > > > > > > > nodes.
> > > > > > > >
> > > > > > > > However, this change also affects the way we support queries.
> > > > > > Previously,
> > > > > > > > we simply extracted fields using reflection to do index
> > lookups,
> > > > but
> > > > > > now
> > > > > > > we
> > > > > > > > cannot use reflection, since we do not have class definitions
> > on
> > > > the
> > > > > > > > servers anymore.
> > > > > > > >
> > > > > > > > As a result, the following restrictions are going to be
> > > introduced
> > > > to
> > > > > > the
> > > > > > > > query processing:
> > > > > > > >
> > > > > > > >
> > > > > > > >    1. @QuerySqlField annotation can't be set to a getter
> method
> > > > > > anymore,
> > > > > > > >    only to fields.
> > > > > > > >    2. Comparable on _key and _value type will not work
> anymore.
> > > > > > However,
> > > > > > > >    "_key" and "_value" types are auto-generated and mostly
> used
> > > > > > > internally
> > > > > > > > by
> > > > > > > >    Ignite itself.
> > > > > > > >
> > > > > > > >
> > > > > > > > Generally, I believe that both (1) and (2) are not a big
> deal.
> > > > > > However, I
> > > > > > > > would like to confirm that for the (2), we can still retrieve
> > and
> > > > > sort
> > > > > > > > primitive values and Strings natively.
> > > > > > > >
> > > > > > > > Serj, given that you are working on the query changes, can
> you
> > > > > confirm
> > > > > > > the
> > > > > > > > (2)?
> > > > > > > >
> > > > > > > > D.
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>