Hello,
I found that every BinaryObject created by BinaryObjectBuilder has hashcode == 0 by default. This can cause situation that all objects created by code similar to: *BinaryObject key = builder.setField("id", i).build();* *streamer.addData(key, key);* will be stored at one partition and this cause grid performance drop. Of course user can set hashcode: *BinaryObject key = builder.setField("id", i).hashCode(random()).build();* but there is no guarantee that he will. I propose to generate random hashcode or hashcode based on field's hashcodes in case no hashcode set or to warn user somehow that he have to specify it. Thoughts? |
Random hash code is not an option.
I would suggest cache throws exception on update if binary object created with builder does not have hash code initialized. Vladimir, can we somehow make it possible to know whether hash code was or was not inited on a binary object? --Yakov 2016-03-24 11:43 GMT+03:00 Anton Vinogradov <[hidden email]>: > Hello, > > I found that every BinaryObject created by BinaryObjectBuilder has hashcode > == 0 by default. > This can cause situation that all objects created by code similar to: > > *BinaryObject key = builder.setField("id", i).build();* > *streamer.addData(key, key);* > > will be stored at one partition and this cause grid performance drop. > > Of course user can set hashcode: > > *BinaryObject key = builder.setField("id", i).hashCode(random()).build();* > > but there is no guarantee that he will. > > I propose to generate random hashcode or hashcode based on field's > hashcodes in case no hashcode set or to warn user somehow that he have to > specify it. > > Thoughts? > |
When object has been built, there is no way to know, whether hash code was
set or not. Zero could be real hash code, as well as not set hash code. Random is not an option. Any kind of automatic generation is not an option as well, because we do not know, how hash code of real class instance is calculated. I honestly do not see a big problem here. We can simply improve JavaDocs. It could be a problem in some specific cases, like JDBC store. This situation require the same solution as already mentioned problems with equals and comparators. Vladimir. 24 марта 2016 г. 11:52 пользователь "Yakov Zhdanov" <[hidden email]> написал: > Random hash code is not an option. > > I would suggest cache throws exception on update if binary object created > with builder does not have hash code initialized. > > Vladimir, can we somehow make it possible to know whether hash code was or > was not inited on a binary object? > > --Yakov > > 2016-03-24 11:43 GMT+03:00 Anton Vinogradov <[hidden email]>: > > > Hello, > > > > I found that every BinaryObject created by BinaryObjectBuilder has > hashcode > > == 0 by default. > > This can cause situation that all objects created by code similar to: > > > > *BinaryObject key = builder.setField("id", i).build();* > > *streamer.addData(key, key);* > > > > will be stored at one partition and this cause grid performance drop. > > > > Of course user can set hashcode: > > > > *BinaryObject key = builder.setField("id", > i).hashCode(random()).build();* > > > > but there is no guarantee that he will. > > > > I propose to generate random hashcode or hashcode based on field's > > hashcodes in case no hashcode set or to warn user somehow that he have to > > specify it. > > > > Thoughts? > > > |
+1 for the improvement of JavaDoc because the problem arises only when
BinaryObjects are used as keys. Readme.io already has a warning regarding this https://apacheignite.readme.io/docs/binary-marshaller#modifying-binary-objects-using-binaryobjectbuilder -- Denis On 3/24/2016 12:38 PM, Vladimir Ozerov wrote: > When object has been built, there is no way to know, whether hash code was > set or not. Zero could be real hash code, as well as not set hash code. > > Random is not an option. Any kind of automatic generation is not an option > as well, because we do not know, how hash code of real class instance is > calculated. > > I honestly do not see a big problem here. We can simply improve JavaDocs. > > It could be a problem in some specific cases, like JDBC store. This > situation require the same solution as already mentioned problems with > equals and comparators. > > Vladimir. > 24 марта 2016 г. 11:52 пользователь "Yakov Zhdanov" <[hidden email]> > написал: > >> Random hash code is not an option. >> >> I would suggest cache throws exception on update if binary object created >> with builder does not have hash code initialized. >> >> Vladimir, can we somehow make it possible to know whether hash code was or >> was not inited on a binary object? >> >> --Yakov >> >> 2016-03-24 11:43 GMT+03:00 Anton Vinogradov <[hidden email]>: >> >>> Hello, >>> >>> I found that every BinaryObject created by BinaryObjectBuilder has >> hashcode >>> == 0 by default. >>> This can cause situation that all objects created by code similar to: >>> >>> *BinaryObject key = builder.setField("id", i).build();* >>> *streamer.addData(key, key);* >>> >>> will be stored at one partition and this cause grid performance drop. >>> >>> Of course user can set hashcode: >>> >>> *BinaryObject key = builder.setField("id", >> i).hashCode(random()).build();* >>> but there is no guarantee that he will. >>> >>> I propose to generate random hashcode or hashcode based on field's >>> hashcodes in case no hashcode set or to warn user somehow that he have to >>> specify it. >>> >>> Thoughts? >>> |
As for JdbcPojoStore it has an option to configure "hasher".
On Thu, Mar 24, 2016 at 5:20 PM, Denis Magda <[hidden email]> wrote: > +1 for the improvement of JavaDoc because the problem arises only when > BinaryObjects are used as keys. > Readme.io already has a warning regarding this > > https://apacheignite.readme.io/docs/binary-marshaller#modifying-binary-objects-using-binaryobjectbuilder > > -- > Denis > > > On 3/24/2016 12:38 PM, Vladimir Ozerov wrote: > >> When object has been built, there is no way to know, whether hash code was >> set or not. Zero could be real hash code, as well as not set hash code. >> >> Random is not an option. Any kind of automatic generation is not an option >> as well, because we do not know, how hash code of real class instance is >> calculated. >> >> I honestly do not see a big problem here. We can simply improve JavaDocs. >> >> It could be a problem in some specific cases, like JDBC store. This >> situation require the same solution as already mentioned problems with >> equals and comparators. >> >> Vladimir. >> 24 марта 2016 г. 11:52 пользователь "Yakov Zhdanov" <[hidden email]> >> написал: >> >> Random hash code is not an option. >>> >>> I would suggest cache throws exception on update if binary object created >>> with builder does not have hash code initialized. >>> >>> Vladimir, can we somehow make it possible to know whether hash code was >>> or >>> was not inited on a binary object? >>> >>> --Yakov >>> >>> 2016-03-24 11:43 GMT+03:00 Anton Vinogradov <[hidden email]>: >>> >>> Hello, >>>> >>>> I found that every BinaryObject created by BinaryObjectBuilder has >>>> >>> hashcode >>> >>>> == 0 by default. >>>> This can cause situation that all objects created by code similar to: >>>> >>>> *BinaryObject key = builder.setField("id", i).build();* >>>> *streamer.addData(key, key);* >>>> >>>> will be stored at one partition and this cause grid performance drop. >>>> >>>> Of course user can set hashcode: >>>> >>>> *BinaryObject key = builder.setField("id", >>>> >>> i).hashCode(random()).build();* >>> >>>> but there is no guarantee that he will. >>>> >>>> I propose to generate random hashcode or hashcode based on field's >>>> hashcodes in case no hashcode set or to warn user somehow that he have >>>> to >>>> specify it. >>>> >>>> Thoughts? >>>> >>>> > -- Alexey Kuznetsov GridGain Systems www.gridgain.com |
This is a nasty problem that is very difficult to track especially for beginners. But I think it's easy to solve: since the BinaryObjects are built exclusively using the BinaryObjectBuilder, you can simply implement a check in the BinaryObjectBuilder.build() to see whether or not BinaryObjectBuilder.hashCode(int hashCode) has been called. If it has not, Ignite may log one of those helpful warnings. (Goes without saying it should be possible to suppress the warning before it gets too annoying.)
Regards Andrey > Date: Thu, 24 Mar 2016 17:39:45 +0700 > Subject: Re: All BinaryObjects created by BinaryObjectBuilder stored at the same partition by default > From: [hidden email] > To: [hidden email] > > As for JdbcPojoStore it has an option to configure "hasher". > > On Thu, Mar 24, 2016 at 5:20 PM, Denis Magda <[hidden email]> wrote: > > > +1 for the improvement of JavaDoc because the problem arises only when > > BinaryObjects are used as keys. > > Readme.io already has a warning regarding this > > > > https://apacheignite.readme.io/docs/binary-marshaller#modifying-binary-objects-using-binaryobjectbuilder > > > > -- > > Denis > > > > > > On 3/24/2016 12:38 PM, Vladimir Ozerov wrote: > > > >> When object has been built, there is no way to know, whether hash code was > >> set or not. Zero could be real hash code, as well as not set hash code. > >> > >> Random is not an option. Any kind of automatic generation is not an option > >> as well, because we do not know, how hash code of real class instance is > >> calculated. > >> > >> I honestly do not see a big problem here. We can simply improve JavaDocs. > >> > >> It could be a problem in some specific cases, like JDBC store. This > >> situation require the same solution as already mentioned problems with > >> equals and comparators. > >> > >> Vladimir. > >> 24 марта 2016 г. 11:52 пользователь "Yakov Zhdanov" <[hidden email]> > >> написал: > >> > >> Random hash code is not an option. > >>> > >>> I would suggest cache throws exception on update if binary object created > >>> with builder does not have hash code initialized. > >>> > >>> Vladimir, can we somehow make it possible to know whether hash code was > >>> or > >>> was not inited on a binary object? > >>> > >>> --Yakov > >>> > >>> 2016-03-24 11:43 GMT+03:00 Anton Vinogradov <[hidden email]>: > >>> > >>> Hello, > >>>> > >>>> I found that every BinaryObject created by BinaryObjectBuilder has > >>>> > >>> hashcode > >>> > >>>> == 0 by default. > >>>> This can cause situation that all objects created by code similar to: > >>>> > >>>> *BinaryObject key = builder.setField("id", i).build();* > >>>> *streamer.addData(key, key);* > >>>> > >>>> will be stored at one partition and this cause grid performance drop. > >>>> > >>>> Of course user can set hashcode: > >>>> > >>>> *BinaryObject key = builder.setField("id", > >>>> > >>> i).hashCode(random()).build();* > >>> > >>>> but there is no guarantee that he will. > >>>> > >>>> I propose to generate random hashcode or hashcode based on field's > >>>> hashcodes in case no hashcode set or to warn user somehow that he have > >>>> to > >>>> specify it. > >>>> > >>>> Thoughts? > >>>> > >>>> > > > > > -- > Alexey Kuznetsov > GridGain Systems > www.gridgain.com |
In reply to this post by Alexey Kuznetsov-2
This is a nasty problem that is very difficult to track especially for beginners. But I think it's easy to solve: since the BinaryObjects are built exclusively using the BinaryObjectBuilder, you can simply implement a check in the BinaryObjectBuilder.build() to see whether or not BinaryObjectBuilder.hashCode(int hashCode) has been called. If it has not, Ignite may log one of those helpful warnings. (Goes without saying it should be possible to suppress the warning before it gets too annoying.)
Regards Andrey > Date: Thu, 24 Mar 2016 17:39:45 +0700 > Subject: Re: All BinaryObjects created by BinaryObjectBuilder stored at the same partition by default > From: [hidden email] > To: [hidden email] > > As for JdbcPojoStore it has an option to configure "hasher". > > On Thu, Mar 24, 2016 at 5:20 PM, Denis Magda <[hidden email]> wrote: > > > +1 for the improvement of JavaDoc because the problem arises only when > > BinaryObjects are used as keys. > > Readme.io already has a warning regarding this > > > > https://apacheignite.readme.io/docs/binary-marshaller#modifying-binary-objects-using-binaryobjectbuilder > > > > -- > > Denis > > > > > > On 3/24/2016 12:38 PM, Vladimir Ozerov wrote: > > > >> When object has been built, there is no way to know, whether hash code was > >> set or not. Zero could be real hash code, as well as not set hash code. > >> > >> Random is not an option. Any kind of automatic generation is not an option > >> as well, because we do not know, how hash code of real class instance is > >> calculated. > >> > >> I honestly do not see a big problem here. We can simply improve JavaDocs. > >> > >> It could be a problem in some specific cases, like JDBC store. This > >> situation require the same solution as already mentioned problems with > >> equals and comparators. > >> > >> Vladimir. > >> 24 марта 2016 г. 11:52 пользователь "Yakov Zhdanov" <[hidden email]> > >> написал: > >> > >> Random hash code is not an option. > >>> > >>> I would suggest cache throws exception on update if binary object created > >>> with builder does not have hash code initialized. > >>> > >>> Vladimir, can we somehow make it possible to know whether hash code was > >>> or > >>> was not inited on a binary object? > >>> > >>> --Yakov > >>> > >>> 2016-03-24 11:43 GMT+03:00 Anton Vinogradov <[hidden email]>: > >>> > >>> Hello, > >>>> > >>>> I found that every BinaryObject created by BinaryObjectBuilder has > >>>> > >>> hashcode > >>> > >>>> == 0 by default. > >>>> This can cause situation that all objects created by code similar to: > >>>> > >>>> *BinaryObject key = builder.setField("id", i).build();* > >>>> *streamer.addData(key, key);* > >>>> > >>>> will be stored at one partition and this cause grid performance drop. > >>>> > >>>> Of course user can set hashcode: > >>>> > >>>> *BinaryObject key = builder.setField("id", > >>>> > >>> i).hashCode(random()).build();* > >>> > >>>> but there is no guarantee that he will. > >>>> > >>>> I propose to generate random hashcode or hashcode based on field's > >>>> hashcodes in case no hashcode set or to warn user somehow that he have > >>>> to > >>>> specify it. > >>>> > >>>> Thoughts? > >>>> > >>>> > > > > > -- > Alexey Kuznetsov > GridGain Systems > www.gridgain.com |
Guys,
It seems like this problem has become an important one once again. In the course of working on https://issues.apache.org/jira/browse/IGNITE-2294 (DML support) there's need to support binary marshaller. And, although we can build just BinaryObject and put it to cache, without adequate hash code it won't be stored properly. Currently SQL MERGE works simply by deserializing newly built object, but it's obviously wrong and is just a workaround rather a solution. Has anyone come with possible design proposals for this problem's solution? Thanks. - Alex |
Alex,
In your case, why not just explicitly set hashcode every time you create an object? There is BinaryObjectBuilder.hashCode(...) method. D. On Mon, Aug 1, 2016 at 7:42 AM, al.psc <[hidden email]> wrote: > Guys, > > It seems like this problem has become an important one once again. > In the course of working on > https://issues.apache.org/jira/browse/IGNITE-2294 (DML support) there's > need > to support binary marshaller. And, although we can build just BinaryObject > and put it to cache, without adequate hash code it won't be stored > properly. > Currently SQL MERGE works simply by deserializing newly built object, but > it's obviously wrong and is just a workaround rather a solution. > Has anyone come with possible design proposals for this problem's solution? > > Thanks. > > - Alex > > > > -- > View this message in context: > http://apache-ignite-developers.2346864.n4.nabble.com/All-BinaryObjects-created-by-BinaryObjectBuilder-stored-at-the-same-partition-by-default-tp8042p10304.html > Sent from the Apache Ignite Developers mailing list archive at Nabble.com. > |
Dmitriy,
The question is how do you calculate the value of the hashCode? Do you want it to be specified explicitly in INSERT statement? 2016-08-01 19:47 GMT+03:00 Dmitriy Setrakyan <[hidden email]>: > Alex, > > In your case, why not just explicitly set hashcode every time you create an > object? There is BinaryObjectBuilder.hashCode(...) method. > > D. > > On Mon, Aug 1, 2016 at 7:42 AM, al.psc <[hidden email]> > wrote: > > > Guys, > > > > It seems like this problem has become an important one once again. > > In the course of working on > > https://issues.apache.org/jira/browse/IGNITE-2294 (DML support) there's > > need > > to support binary marshaller. And, although we can build just > BinaryObject > > and put it to cache, without adequate hash code it won't be stored > > properly. > > Currently SQL MERGE works simply by deserializing newly built object, but > > it's obviously wrong and is just a workaround rather a solution. > > Has anyone come with possible design proposals for this problem's > solution? > > > > Thanks. > > > > - Alex > > > > > > > > -- > > View this message in context: > > > http://apache-ignite-developers.2346864.n4.nabble.com/All-BinaryObjects-created-by-BinaryObjectBuilder-stored-at-the-same-partition-by-default-tp8042p10304.html > > Sent from the Apache Ignite Developers mailing list archive at > Nabble.com. > > > |
On Mon, Aug 1, 2016 at 10:01 AM, Alexey Goncharuk <
[hidden email]> wrote: > Dmitriy, > > The question is how do you calculate the value of the hashCode? Do you want > it to be specified explicitly in INSERT statement? > I think optionally we should allow to specify hashCode as part of the INSERT statement. However, if it is not specified, we should calculate it automatically based in the key fields defined in the schema/type. Agree? > > 2016-08-01 19:47 GMT+03:00 Dmitriy Setrakyan <[hidden email]>: > > > Alex, > > > > In your case, why not just explicitly set hashcode every time you create > an > > object? There is BinaryObjectBuilder.hashCode(...) method. > > > > D. > > > > On Mon, Aug 1, 2016 at 7:42 AM, al.psc <[hidden email]> > > wrote: > > > > > Guys, > > > > > > It seems like this problem has become an important one once again. > > > In the course of working on > > > https://issues.apache.org/jira/browse/IGNITE-2294 (DML support) > there's > > > need > > > to support binary marshaller. And, although we can build just > > BinaryObject > > > and put it to cache, without adequate hash code it won't be stored > > > properly. > > > Currently SQL MERGE works simply by deserializing newly built object, > but > > > it's obviously wrong and is just a workaround rather a solution. > > > Has anyone come with possible design proposals for this problem's > > solution? > > > > > > Thanks. > > > > > > - Alex > > > > > > > > > > > > -- > > > View this message in context: > > > > > > http://apache-ignite-developers.2346864.n4.nabble.com/All-BinaryObjects-created-by-BinaryObjectBuilder-stored-at-the-same-partition-by-default-tp8042p10304.html > > > Sent from the Apache Ignite Developers mailing list archive at > > Nabble.com. > > > > > > |
I think we also should print some warning in case when hashCode() wasn't
called on BinaryObject explicitly. On Tue, Aug 2, 2016 at 2:20 AM, Dmitriy Setrakyan <[hidden email]> wrote: > On Mon, Aug 1, 2016 at 10:01 AM, Alexey Goncharuk < > [hidden email]> wrote: > > > Dmitriy, > > > > The question is how do you calculate the value of the hashCode? Do you > want > > it to be specified explicitly in INSERT statement? > > > > I think optionally we should allow to specify hashCode as part of the > INSERT statement. However, if it is not specified, we should calculate it > automatically based in the key fields defined in the schema/type. Agree? > > > > > > 2016-08-01 19:47 GMT+03:00 Dmitriy Setrakyan <[hidden email]>: > > > > > Alex, > > > > > > In your case, why not just explicitly set hashcode every time you > create > > an > > > object? There is BinaryObjectBuilder.hashCode(...) method. > > > > > > D. > > > > > > On Mon, Aug 1, 2016 at 7:42 AM, al.psc < > [hidden email]> > > > wrote: > > > > > > > Guys, > > > > > > > > It seems like this problem has become an important one once again. > > > > In the course of working on > > > > https://issues.apache.org/jira/browse/IGNITE-2294 (DML support) > > there's > > > > need > > > > to support binary marshaller. And, although we can build just > > > BinaryObject > > > > and put it to cache, without adequate hash code it won't be stored > > > > properly. > > > > Currently SQL MERGE works simply by deserializing newly built object, > > but > > > > it's obviously wrong and is just a workaround rather a solution. > > > > Has anyone come with possible design proposals for this problem's > > > solution? > > > > > > > > Thanks. > > > > > > > > - Alex > > > > > > > > > > > > > > > > -- > > > > View this message in context: > > > > > > > > > > http://apache-ignite-developers.2346864.n4.nabble.com/All-BinaryObjects-created-by-BinaryObjectBuilder-stored-at-the-same-partition-by-default-tp8042p10304.html > > > > Sent from the Apache Ignite Developers mailing list archive at > > > Nabble.com. > > > > > > > > > > -- Andrey Gura GridGain Systems, Inc. www.gridgain.com |
Andrey,
The question is when to print this warning. I doubt we can print a warning when calling *BinaryObjectBuilder.build() *method, because an object without a hash code is normal situation. On Tue, Aug 2, 2016 at 9:00 AM, Andrey Gura <[hidden email]> wrote: > I think we also should print some warning in case when hashCode() wasn't > called on BinaryObject explicitly. > > On Tue, Aug 2, 2016 at 2:20 AM, Dmitriy Setrakyan <[hidden email]> > wrote: > > > On Mon, Aug 1, 2016 at 10:01 AM, Alexey Goncharuk < > > [hidden email]> wrote: > > > > > Dmitriy, > > > > > > The question is how do you calculate the value of the hashCode? Do you > > want > > > it to be specified explicitly in INSERT statement? > > > > > > > I think optionally we should allow to specify hashCode as part of the > > INSERT statement. However, if it is not specified, we should calculate it > > automatically based in the key fields defined in the schema/type. Agree? > > > > > > > > > > 2016-08-01 19:47 GMT+03:00 Dmitriy Setrakyan <[hidden email]>: > > > > > > > Alex, > > > > > > > > In your case, why not just explicitly set hashcode every time you > > create > > > an > > > > object? There is BinaryObjectBuilder.hashCode(...) method. > > > > > > > > D. > > > > > > > > On Mon, Aug 1, 2016 at 7:42 AM, al.psc < > > [hidden email]> > > > > wrote: > > > > > > > > > Guys, > > > > > > > > > > It seems like this problem has become an important one once again. > > > > > In the course of working on > > > > > https://issues.apache.org/jira/browse/IGNITE-2294 (DML support) > > > there's > > > > > need > > > > > to support binary marshaller. And, although we can build just > > > > BinaryObject > > > > > and put it to cache, without adequate hash code it won't be stored > > > > > properly. > > > > > Currently SQL MERGE works simply by deserializing newly built > object, > > > but > > > > > it's obviously wrong and is just a workaround rather a solution. > > > > > Has anyone come with possible design proposals for this problem's > > > > solution? > > > > > > > > > > Thanks. > > > > > > > > > > - Alex > > > > > > > > > > > > > > > > > > > > -- > > > > > View this message in context: > > > > > > > > > > > > > > > http://apache-ignite-developers.2346864.n4.nabble.com/All-BinaryObjects-created-by-BinaryObjectBuilder-stored-at-the-same-partition-by-default-tp8042p10304.html > > > > > Sent from the Apache Ignite Developers mailing list archive at > > > > Nabble.com. > > > > > > > > > > > > > > > > > > -- > Andrey Gura > GridGain Systems, Inc. > www.gridgain.com > |
On Mon, Aug 1, 2016 at 11:38 PM, Vladimir Ozerov <[hidden email]>
wrote: > Andrey, > > The question is when to print this warning. I doubt we can print a warning > when calling *BinaryObjectBuilder.build() *method, because an object > without a hash code is normal situation. > > I would not only print warning, but throw exception, if an object without a hashCode ends up on a put or read operation in cache. > On Tue, Aug 2, 2016 at 9:00 AM, Andrey Gura <[hidden email]> wrote: > > > I think we also should print some warning in case when hashCode() wasn't > > called on BinaryObject explicitly. > > > > On Tue, Aug 2, 2016 at 2:20 AM, Dmitriy Setrakyan <[hidden email] > > > > wrote: > > > > > On Mon, Aug 1, 2016 at 10:01 AM, Alexey Goncharuk < > > > [hidden email]> wrote: > > > > > > > Dmitriy, > > > > > > > > The question is how do you calculate the value of the hashCode? Do > you > > > want > > > > it to be specified explicitly in INSERT statement? > > > > > > > > > > I think optionally we should allow to specify hashCode as part of the > > > INSERT statement. However, if it is not specified, we should calculate > it > > > automatically based in the key fields defined in the schema/type. > Agree? > > > > > > > > > > > > > > 2016-08-01 19:47 GMT+03:00 Dmitriy Setrakyan <[hidden email] > >: > > > > > > > > > Alex, > > > > > > > > > > In your case, why not just explicitly set hashcode every time you > > > create > > > > an > > > > > object? There is BinaryObjectBuilder.hashCode(...) method. > > > > > > > > > > D. > > > > > > > > > > On Mon, Aug 1, 2016 at 7:42 AM, al.psc < > > > [hidden email]> > > > > > wrote: > > > > > > > > > > > Guys, > > > > > > > > > > > > It seems like this problem has become an important one once > again. > > > > > > In the course of working on > > > > > > https://issues.apache.org/jira/browse/IGNITE-2294 (DML support) > > > > there's > > > > > > need > > > > > > to support binary marshaller. And, although we can build just > > > > > BinaryObject > > > > > > and put it to cache, without adequate hash code it won't be > stored > > > > > > properly. > > > > > > Currently SQL MERGE works simply by deserializing newly built > > object, > > > > but > > > > > > it's obviously wrong and is just a workaround rather a solution. > > > > > > Has anyone come with possible design proposals for this problem's > > > > > solution? > > > > > > > > > > > > Thanks. > > > > > > > > > > > > - Alex > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > View this message in context: > > > > > > > > > > > > > > > > > > > > > http://apache-ignite-developers.2346864.n4.nabble.com/All-BinaryObjects-created-by-BinaryObjectBuilder-stored-at-the-same-partition-by-default-tp8042p10304.html > > > > > > Sent from the Apache Ignite Developers mailing list archive at > > > > > Nabble.com. > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > Andrey Gura > > GridGain Systems, Inc. > > www.gridgain.com > > > |
Dmitriy,
Good point, however, currently there's no way to distinguish hash code of zero which is a valid case from missing hash code. We probably should enhance binary builder for it to handle this case. - Alex 2016-08-02 9:47 GMT+03:00 Dmitriy Setrakyan <[hidden email]>: > On Mon, Aug 1, 2016 at 11:38 PM, Vladimir Ozerov <[hidden email]> > wrote: > >> Andrey, >> >> The question is when to print this warning. I doubt we can print a warning >> when calling *BinaryObjectBuilder.build() *method, because an object >> without a hash code is normal situation. >> >> > I would not only print warning, but throw exception, if an object without a > hashCode ends up on a put or read operation in cache. > > >> On Tue, Aug 2, 2016 at 9:00 AM, Andrey Gura <[hidden email]> wrote: >> >> > I think we also should print some warning in case when hashCode() wasn't >> > called on BinaryObject explicitly. >> > >> > On Tue, Aug 2, 2016 at 2:20 AM, Dmitriy Setrakyan <[hidden email] >> > >> > wrote: >> > >> > > On Mon, Aug 1, 2016 at 10:01 AM, Alexey Goncharuk < >> > > [hidden email]> wrote: >> > > >> > > > Dmitriy, >> > > > >> > > > The question is how do you calculate the value of the hashCode? Do >> you >> > > want >> > > > it to be specified explicitly in INSERT statement? >> > > > >> > > >> > > I think optionally we should allow to specify hashCode as part of the >> > > INSERT statement. However, if it is not specified, we should calculate >> it >> > > automatically based in the key fields defined in the schema/type. >> Agree? >> > > >> > > >> > > > >> > > > 2016-08-01 19:47 GMT+03:00 Dmitriy Setrakyan <[hidden email] >> >: >> > > > >> > > > > Alex, >> > > > > >> > > > > In your case, why not just explicitly set hashcode every time you >> > > create >> > > > an >> > > > > object? There is BinaryObjectBuilder.hashCode(...) method. >> > > > > >> > > > > D. >> > > > > >> > > > > On Mon, Aug 1, 2016 at 7:42 AM, al.psc < >> > > [hidden email]> >> > > > > wrote: >> > > > > >> > > > > > Guys, >> > > > > > >> > > > > > It seems like this problem has become an important one once >> again. >> > > > > > In the course of working on >> > > > > > https://issues.apache.org/jira/browse/IGNITE-2294 (DML support) >> > > > there's >> > > > > > need >> > > > > > to support binary marshaller. And, although we can build just >> > > > > BinaryObject >> > > > > > and put it to cache, without adequate hash code it won't be >> stored >> > > > > > properly. >> > > > > > Currently SQL MERGE works simply by deserializing newly built >> > object, >> > > > but >> > > > > > it's obviously wrong and is just a workaround rather a solution. >> > > > > > Has anyone come with possible design proposals for this problem's >> > > > > solution? >> > > > > > >> > > > > > Thanks. >> > > > > > >> > > > > > - Alex >> > > > > > >> > > > > > >> > > > > > >> > > > > > -- >> > > > > > View this message in context: >> > > > > > >> > > > > >> > > > >> > > >> > >> http://apache-ignite-developers.2346864.n4.nabble.com/All-BinaryObjects-created-by-BinaryObjectBuilder-stored-at-the-same-partition-by-default-tp8042p10304.html >> > > > > > Sent from the Apache Ignite Developers mailing list archive at >> > > > > Nabble.com. >> > > > > > >> > > > > >> > > > >> > > >> > >> > >> > >> > -- >> > Andrey Gura >> > GridGain Systems, Inc. >> > www.gridgain.com >> > >> |
In reply to this post by dsetrakyan
I still fail to understand the use case then. Do we want to support insert
via SQL and get via regular cache API? If yes, then specifying hashCode in INSERT statement is not optional, because otherwise the next cache.get() for the inserted key will not work. So, we either enforce user to always specify a hashCode in insert statement, or force him to use our "automatic" algorithm of hashCode calculation in his key objects. Both ways look fishy to me. 2016-08-02 2:20 GMT+03:00 Dmitriy Setrakyan <[hidden email]>: > On Mon, Aug 1, 2016 at 10:01 AM, Alexey Goncharuk < > [hidden email]> wrote: > > > Dmitriy, > > > > The question is how do you calculate the value of the hashCode? Do you > want > > it to be specified explicitly in INSERT statement? > > > > I think optionally we should allow to specify hashCode as part of the > INSERT statement. However, if it is not specified, we should calculate it > automatically based in the key fields defined in the schema/type. Agree? > > > > > > 2016-08-01 19:47 GMT+03:00 Dmitriy Setrakyan <[hidden email]>: > > > > > Alex, > > > > > > In your case, why not just explicitly set hashcode every time you > create > > an > > > object? There is BinaryObjectBuilder.hashCode(...) method. > > > > > > D. > > > > > > On Mon, Aug 1, 2016 at 7:42 AM, al.psc < > [hidden email]> > > > wrote: > > > > > > > Guys, > > > > > > > > It seems like this problem has become an important one once again. > > > > In the course of working on > > > > https://issues.apache.org/jira/browse/IGNITE-2294 (DML support) > > there's > > > > need > > > > to support binary marshaller. And, although we can build just > > > BinaryObject > > > > and put it to cache, without adequate hash code it won't be stored > > > > properly. > > > > Currently SQL MERGE works simply by deserializing newly built object, > > but > > > > it's obviously wrong and is just a workaround rather a solution. > > > > Has anyone come with possible design proposals for this problem's > > > solution? > > > > > > > > Thanks. > > > > > > > > - Alex > > > > > > > > > > > > > > > > -- > > > > View this message in context: > > > > > > > > > > http://apache-ignite-developers.2346864.n4.nabble.com/All-BinaryObjects-created-by-BinaryObjectBuilder-stored-at-the-same-partition-by-default-tp8042p10304.html > > > > Sent from the Apache Ignite Developers mailing list archive at > > > Nabble.com. > > > > > > > > > > |
How about we supply hash code evaluation routine via, say, JavaScript?
Yakov has already suggested that in the beginning of this thread. - Alex 2016-08-02 10:46 GMT+03:00 Alexey Goncharuk <[hidden email]>: > I still fail to understand the use case then. Do we want to support insert > via SQL and get via regular cache API? If yes, then specifying hashCode in > INSERT statement is not optional, because otherwise the next cache.get() > for the inserted key will not work. So, we either enforce user to always > specify a hashCode in insert statement, or force him to use our "automatic" > algorithm of hashCode calculation in his key objects. Both ways look fishy > to me. > > 2016-08-02 2:20 GMT+03:00 Dmitriy Setrakyan <[hidden email]>: > >> On Mon, Aug 1, 2016 at 10:01 AM, Alexey Goncharuk < >> [hidden email]> wrote: >> >> > Dmitriy, >> > >> > The question is how do you calculate the value of the hashCode? Do you >> want >> > it to be specified explicitly in INSERT statement? >> > >> >> I think optionally we should allow to specify hashCode as part of the >> INSERT statement. However, if it is not specified, we should calculate it >> automatically based in the key fields defined in the schema/type. Agree? >> >> >> > >> > 2016-08-01 19:47 GMT+03:00 Dmitriy Setrakyan <[hidden email]>: >> > >> > > Alex, >> > > >> > > In your case, why not just explicitly set hashcode every time you >> create >> > an >> > > object? There is BinaryObjectBuilder.hashCode(...) method. >> > > >> > > D. >> > > >> > > On Mon, Aug 1, 2016 at 7:42 AM, al.psc < >> [hidden email]> >> > > wrote: >> > > >> > > > Guys, >> > > > >> > > > It seems like this problem has become an important one once again. >> > > > In the course of working on >> > > > https://issues.apache.org/jira/browse/IGNITE-2294 (DML support) >> > there's >> > > > need >> > > > to support binary marshaller. And, although we can build just >> > > BinaryObject >> > > > and put it to cache, without adequate hash code it won't be stored >> > > > properly. >> > > > Currently SQL MERGE works simply by deserializing newly built object, >> > but >> > > > it's obviously wrong and is just a workaround rather a solution. >> > > > Has anyone come with possible design proposals for this problem's >> > > solution? >> > > > >> > > > Thanks. >> > > > >> > > > - Alex >> > > > >> > > > >> > > > >> > > > -- >> > > > View this message in context: >> > > > >> > > >> > >> http://apache-ignite-developers.2346864.n4.nabble.com/All-BinaryObjects-created-by-BinaryObjectBuilder-stored-at-the-same-partition-by-default-tp8042p10304.html >> > > > Sent from the Apache Ignite Developers mailing list archive at >> > > Nabble.com. >> > > > >> > > >> > >> |
In reply to this post by al.psc
We can distinguish this situation using boolean flag that initializes
during object building. But it will increase object size. On Tue, Aug 2, 2016 at 10:14 AM, Alexander Paschenko < [hidden email]> wrote: > Dmitriy, > > Good point, however, currently there's no way to distinguish hash code > of zero which is a valid case from missing hash code. We probably > should enhance binary builder for it to handle this case. > > - Alex > > 2016-08-02 9:47 GMT+03:00 Dmitriy Setrakyan <[hidden email]>: > > On Mon, Aug 1, 2016 at 11:38 PM, Vladimir Ozerov <[hidden email]> > > wrote: > > > >> Andrey, > >> > >> The question is when to print this warning. I doubt we can print a > warning > >> when calling *BinaryObjectBuilder.build() *method, because an object > >> without a hash code is normal situation. > >> > >> > > I would not only print warning, but throw exception, if an object > without a > > hashCode ends up on a put or read operation in cache. > > > > > >> On Tue, Aug 2, 2016 at 9:00 AM, Andrey Gura <[hidden email]> wrote: > >> > >> > I think we also should print some warning in case when hashCode() > wasn't > >> > called on BinaryObject explicitly. > >> > > >> > On Tue, Aug 2, 2016 at 2:20 AM, Dmitriy Setrakyan < > [hidden email] > >> > > >> > wrote: > >> > > >> > > On Mon, Aug 1, 2016 at 10:01 AM, Alexey Goncharuk < > >> > > [hidden email]> wrote: > >> > > > >> > > > Dmitriy, > >> > > > > >> > > > The question is how do you calculate the value of the hashCode? Do > >> you > >> > > want > >> > > > it to be specified explicitly in INSERT statement? > >> > > > > >> > > > >> > > I think optionally we should allow to specify hashCode as part of > the > >> > > INSERT statement. However, if it is not specified, we should > calculate > >> it > >> > > automatically based in the key fields defined in the schema/type. > >> Agree? > >> > > > >> > > > >> > > > > >> > > > 2016-08-01 19:47 GMT+03:00 Dmitriy Setrakyan < > [hidden email] > >> >: > >> > > > > >> > > > > Alex, > >> > > > > > >> > > > > In your case, why not just explicitly set hashcode every time > you > >> > > create > >> > > > an > >> > > > > object? There is BinaryObjectBuilder.hashCode(...) method. > >> > > > > > >> > > > > D. > >> > > > > > >> > > > > On Mon, Aug 1, 2016 at 7:42 AM, al.psc < > >> > > [hidden email]> > >> > > > > wrote: > >> > > > > > >> > > > > > Guys, > >> > > > > > > >> > > > > > It seems like this problem has become an important one once > >> again. > >> > > > > > In the course of working on > >> > > > > > https://issues.apache.org/jira/browse/IGNITE-2294 (DML > support) > >> > > > there's > >> > > > > > need > >> > > > > > to support binary marshaller. And, although we can build just > >> > > > > BinaryObject > >> > > > > > and put it to cache, without adequate hash code it won't be > >> stored > >> > > > > > properly. > >> > > > > > Currently SQL MERGE works simply by deserializing newly built > >> > object, > >> > > > but > >> > > > > > it's obviously wrong and is just a workaround rather a > solution. > >> > > > > > Has anyone come with possible design proposals for this > problem's > >> > > > > solution? > >> > > > > > > >> > > > > > Thanks. > >> > > > > > > >> > > > > > - Alex > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > -- > >> > > > > > View this message in context: > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > http://apache-ignite-developers.2346864.n4.nabble.com/All-BinaryObjects-created-by-BinaryObjectBuilder-stored-at-the-same-partition-by-default-tp8042p10304.html > >> > > > > > Sent from the Apache Ignite Developers mailing list archive at > >> > > > > Nabble.com. > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > > >> > > >> > -- > >> > Andrey Gura > >> > GridGain Systems, Inc. > >> > www.gridgain.com > >> > > >> > -- Andrey Gura GridGain Systems, Inc. www.gridgain.com |
In reply to this post by al.psc
There is a technique we already use to see if a field is initialized by application code. By default a field has to be a reference to a predefined object and the reference comparison (not equals) is used to check if the field is initialized by the user.
Refer to IgniteConfiguration.failureDetectionTimeout and IgniteSpiAdapter.initializeFailureDetectionTimeout for more details. — Denis > On Aug 2, 2016, at 12:14 AM, Alexander Paschenko <[hidden email]> wrote: > > Dmitriy, > > Good point, however, currently there's no way to distinguish hash code > of zero which is a valid case from missing hash code. We probably > should enhance binary builder for it to handle this case. > > - Alex > > 2016-08-02 9:47 GMT+03:00 Dmitriy Setrakyan <[hidden email]>: >> On Mon, Aug 1, 2016 at 11:38 PM, Vladimir Ozerov <[hidden email]> >> wrote: >> >>> Andrey, >>> >>> The question is when to print this warning. I doubt we can print a warning >>> when calling *BinaryObjectBuilder.build() *method, because an object >>> without a hash code is normal situation. >>> >>> >> I would not only print warning, but throw exception, if an object without a >> hashCode ends up on a put or read operation in cache. >> >> >>> On Tue, Aug 2, 2016 at 9:00 AM, Andrey Gura <[hidden email]> wrote: >>> >>>> I think we also should print some warning in case when hashCode() wasn't >>>> called on BinaryObject explicitly. >>>> >>>> On Tue, Aug 2, 2016 at 2:20 AM, Dmitriy Setrakyan <[hidden email] >>>> >>>> wrote: >>>> >>>>> On Mon, Aug 1, 2016 at 10:01 AM, Alexey Goncharuk < >>>>> [hidden email]> wrote: >>>>> >>>>>> Dmitriy, >>>>>> >>>>>> The question is how do you calculate the value of the hashCode? Do >>> you >>>>> want >>>>>> it to be specified explicitly in INSERT statement? >>>>>> >>>>> >>>>> I think optionally we should allow to specify hashCode as part of the >>>>> INSERT statement. However, if it is not specified, we should calculate >>> it >>>>> automatically based in the key fields defined in the schema/type. >>> Agree? >>>>> >>>>> >>>>>> >>>>>> 2016-08-01 19:47 GMT+03:00 Dmitriy Setrakyan <[hidden email] >>>> : >>>>>> >>>>>>> Alex, >>>>>>> >>>>>>> In your case, why not just explicitly set hashcode every time you >>>>> create >>>>>> an >>>>>>> object? There is BinaryObjectBuilder.hashCode(...) method. >>>>>>> >>>>>>> D. >>>>>>> >>>>>>> On Mon, Aug 1, 2016 at 7:42 AM, al.psc < >>>>> [hidden email]> >>>>>>> wrote: >>>>>>> >>>>>>>> Guys, >>>>>>>> >>>>>>>> It seems like this problem has become an important one once >>> again. >>>>>>>> In the course of working on >>>>>>>> https://issues.apache.org/jira/browse/IGNITE-2294 (DML support) >>>>>> there's >>>>>>>> need >>>>>>>> to support binary marshaller. And, although we can build just >>>>>>> BinaryObject >>>>>>>> and put it to cache, without adequate hash code it won't be >>> stored >>>>>>>> properly. >>>>>>>> Currently SQL MERGE works simply by deserializing newly built >>>> object, >>>>>> but >>>>>>>> it's obviously wrong and is just a workaround rather a solution. >>>>>>>> Has anyone come with possible design proposals for this problem's >>>>>>> solution? >>>>>>>> >>>>>>>> Thanks. >>>>>>>> >>>>>>>> - Alex >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> View this message in context: >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> http://apache-ignite-developers.2346864.n4.nabble.com/All-BinaryObjects-created-by-BinaryObjectBuilder-stored-at-the-same-partition-by-default-tp8042p10304.html >>>>>>>> Sent from the Apache Ignite Developers mailing list archive at >>>>>>> Nabble.com. >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> Andrey Gura >>>> GridGain Systems, Inc. >>>> www.gridgain.com >>>> >>> |
Denis,
I hardly can imagine how it could work in our binary protocol. Can you please elaborate? On Tue, Aug 2, 2016 at 5:06 PM, Denis Magda <[hidden email]> wrote: > There is a technique we already use to see if a field is initialized by > application code. By default a field has to be a reference to a predefined > object and the reference comparison (not equals) is used to check if the > field is initialized by the user. > > Refer to IgniteConfiguration.failureDetectionTimeout and > IgniteSpiAdapter.initializeFailureDetectionTimeout for more details. > > — > Denis > > > On Aug 2, 2016, at 12:14 AM, Alexander Paschenko < > [hidden email]> wrote: > > > > Dmitriy, > > > > Good point, however, currently there's no way to distinguish hash code > > of zero which is a valid case from missing hash code. We probably > > should enhance binary builder for it to handle this case. > > > > - Alex > > > > 2016-08-02 9:47 GMT+03:00 Dmitriy Setrakyan <[hidden email]>: > >> On Mon, Aug 1, 2016 at 11:38 PM, Vladimir Ozerov <[hidden email]> > >> wrote: > >> > >>> Andrey, > >>> > >>> The question is when to print this warning. I doubt we can print a > warning > >>> when calling *BinaryObjectBuilder.build() *method, because an object > >>> without a hash code is normal situation. > >>> > >>> > >> I would not only print warning, but throw exception, if an object > without a > >> hashCode ends up on a put or read operation in cache. > >> > >> > >>> On Tue, Aug 2, 2016 at 9:00 AM, Andrey Gura <[hidden email]> > wrote: > >>> > >>>> I think we also should print some warning in case when hashCode() > wasn't > >>>> called on BinaryObject explicitly. > >>>> > >>>> On Tue, Aug 2, 2016 at 2:20 AM, Dmitriy Setrakyan < > [hidden email] > >>>> > >>>> wrote: > >>>> > >>>>> On Mon, Aug 1, 2016 at 10:01 AM, Alexey Goncharuk < > >>>>> [hidden email]> wrote: > >>>>> > >>>>>> Dmitriy, > >>>>>> > >>>>>> The question is how do you calculate the value of the hashCode? Do > >>> you > >>>>> want > >>>>>> it to be specified explicitly in INSERT statement? > >>>>>> > >>>>> > >>>>> I think optionally we should allow to specify hashCode as part of the > >>>>> INSERT statement. However, if it is not specified, we should > calculate > >>> it > >>>>> automatically based in the key fields defined in the schema/type. > >>> Agree? > >>>>> > >>>>> > >>>>>> > >>>>>> 2016-08-01 19:47 GMT+03:00 Dmitriy Setrakyan <[hidden email] > >>>> : > >>>>>> > >>>>>>> Alex, > >>>>>>> > >>>>>>> In your case, why not just explicitly set hashcode every time you > >>>>> create > >>>>>> an > >>>>>>> object? There is BinaryObjectBuilder.hashCode(...) method. > >>>>>>> > >>>>>>> D. > >>>>>>> > >>>>>>> On Mon, Aug 1, 2016 at 7:42 AM, al.psc < > >>>>> [hidden email]> > >>>>>>> wrote: > >>>>>>> > >>>>>>>> Guys, > >>>>>>>> > >>>>>>>> It seems like this problem has become an important one once > >>> again. > >>>>>>>> In the course of working on > >>>>>>>> https://issues.apache.org/jira/browse/IGNITE-2294 (DML support) > >>>>>> there's > >>>>>>>> need > >>>>>>>> to support binary marshaller. And, although we can build just > >>>>>>> BinaryObject > >>>>>>>> and put it to cache, without adequate hash code it won't be > >>> stored > >>>>>>>> properly. > >>>>>>>> Currently SQL MERGE works simply by deserializing newly built > >>>> object, > >>>>>> but > >>>>>>>> it's obviously wrong and is just a workaround rather a solution. > >>>>>>>> Has anyone come with possible design proposals for this problem's > >>>>>>> solution? > >>>>>>>> > >>>>>>>> Thanks. > >>>>>>>> > >>>>>>>> - Alex > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> -- > >>>>>>>> View this message in context: > >>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>>> > >>> > http://apache-ignite-developers.2346864.n4.nabble.com/All-BinaryObjects-created-by-BinaryObjectBuilder-stored-at-the-same-partition-by-default-tp8042p10304.html > >>>>>>>> Sent from the Apache Ignite Developers mailing list archive at > >>>>>>> Nabble.com. > >>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> Andrey Gura > >>>> GridGain Systems, Inc. > >>>> www.gridgain.com > >>>> > >>> > > |
Free forum by Nabble | Edit this page |