Portables hash code.
Locked
 
|
Igniters,
I'm working on [1] "IGNITE-1753 Rework CacheJdbcPojoStore to new API." And one of subtasks is to support portable objects with JDBC store. I implemented this and during tests found a huge performance drop when I have PortableObject as key. After some debugging I found that all my portable objects have hashCode = 0. I'm using PortableBuilder to build my portable objects. And I expected that PortableBuilder will calculate proper hash code for me out of the box. I think we should at least describe in PortableBuilder javadocs that by default PortableBuilder will return zero hashcode? Or we should calculate hashcode in PortableBuilder.build() method? Or may be we could add boolean argument PortableBuilder.build(boolean generateHashCode)? Thoughts? P.S. After I added manual hashcode calculation to my CacheJdbcPojoStore performance drop is gone away. [1] https://issues.apache.org/jira/browse/IGNITE-1753 -- Alexey Kuznetsov GridGain Systems www.gridgain.com |
|
|
Alexey,
PortableBuilder has an (un-documented) method hashCode(int hashCode) that should be used to explicitly set the hashCode for the portable instance being built. I'm not sure why this has been designed this way, but I'm guessing that since the PortableBuilder is pretty dumb and it wouldn't know which fields to use for hash code computation (in some cases you'd only want to include specific portable fields rather than all fields). Regards Andrey > Date: Wed, 28 Oct 2015 09:49:07 +0700 > Subject: Portables hash code. > From: [hidden email] > To: [hidden email] > > Igniters, > > I'm working on [1] "IGNITE-1753 Rework CacheJdbcPojoStore to new API." > > And one of subtasks is to support portable objects with JDBC store. > > I implemented this and during tests found a huge performance drop when I > have PortableObject as key. > > After some debugging I found that all my portable objects have hashCode = 0. > I'm using PortableBuilder to build my portable objects. > > And I expected that PortableBuilder will calculate proper hash code for me > out of the box. > > I think we should at least describe in PortableBuilder javadocs that by > default PortableBuilder will return zero hashcode? > > Or we should calculate hashcode in PortableBuilder.build() method? > > Or may be we could add boolean argument PortableBuilder.build(boolean > generateHashCode)? > > Thoughts? > > P.S. After I added manual hashcode calculation to my CacheJdbcPojoStore > performance drop is gone away. > > > [1] https://issues.apache.org/jira/browse/IGNITE-1753 > -- > Alexey Kuznetsov > GridGain Systems > www.gridgain.com |
|
|
Andrey, thanks.
Actually I use this not well documented method to solve my problem. And I'm proposed to at least add more info for PortableBuilder about this method. On Wed, Oct 28, 2015 at 10:19 AM, Andrey Kornev <[hidden email]> wrote: > Alexey, > > PortableBuilder has an (un-documented) method hashCode(int hashCode) that > should be used to explicitly set the hashCode for the portable instance > being built. I'm not sure why this has been designed this way, but I'm > guessing that since the PortableBuilder is pretty dumb and it wouldn't know > which fields to use for hash code computation (in some cases you'd only > want to include specific portable fields rather than all fields). > > Regards > Andrey > > > Date: Wed, 28 Oct 2015 09:49:07 +0700 > > Subject: Portables hash code. > > From: [hidden email] > > To: [hidden email] > > > > Igniters, > > > > I'm working on [1] "IGNITE-1753 Rework CacheJdbcPojoStore to new API." > > > > And one of subtasks is to support portable objects with JDBC store. > > > > I implemented this and during tests found a huge performance drop when I > > have PortableObject as key. > > > > After some debugging I found that all my portable objects have hashCode > = 0. > > I'm using PortableBuilder to build my portable objects. > > > > And I expected that PortableBuilder will calculate proper hash code for > me > > out of the box. > > > > I think we should at least describe in PortableBuilder javadocs that by > > default PortableBuilder will return zero hashcode? > > > > Or we should calculate hashcode in PortableBuilder.build() method? > > > > Or may be we could add boolean argument PortableBuilder.build(boolean > > generateHashCode)? > > > > Thoughts? > > > > P.S. After I added manual hashcode calculation to my CacheJdbcPojoStore > > performance drop is gone away. > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-1753 > > -- > > Alexey Kuznetsov > > GridGain Systems > > www.gridgain.com > > -- Alexey Kuznetsov GridGain Systems www.gridgain.com |
|
How about we check that dashcode is not 0 when storing portable keys in
cache? On Tue, Oct 27, 2015 at 8:42 PM, Alexey Kuznetsov <[hidden email]> wrote: > Andrey, thanks. > > Actually I use this not well documented method to solve my problem. > > And I'm proposed to at least add more info for PortableBuilder about this > method. > > On Wed, Oct 28, 2015 at 10:19 AM, Andrey Kornev <[hidden email]> > wrote: > > > Alexey, > > > > PortableBuilder has an (un-documented) method hashCode(int hashCode) that > > should be used to explicitly set the hashCode for the portable instance > > being built. I'm not sure why this has been designed this way, but I'm > > guessing that since the PortableBuilder is pretty dumb and it wouldn't > know > > which fields to use for hash code computation (in some cases you'd only > > want to include specific portable fields rather than all fields). > > > > Regards > > Andrey > > > > > Date: Wed, 28 Oct 2015 09:49:07 +0700 > > > Subject: Portables hash code. > > > From: [hidden email] > > > To: [hidden email] > > > > > > Igniters, > > > > > > I'm working on [1] "IGNITE-1753 Rework CacheJdbcPojoStore to new API." > > > > > > And one of subtasks is to support portable objects with JDBC store. > > > > > > I implemented this and during tests found a huge performance drop when > I > > > have PortableObject as key. > > > > > > After some debugging I found that all my portable objects have hashCode > > = 0. > > > I'm using PortableBuilder to build my portable objects. > > > > > > And I expected that PortableBuilder will calculate proper hash code for > > me > > > out of the box. > > > > > > I think we should at least describe in PortableBuilder javadocs that by > > > default PortableBuilder will return zero hashcode? > > > > > > Or we should calculate hashcode in PortableBuilder.build() method? > > > > > > Or may be we could add boolean argument PortableBuilder.build(boolean > > > generateHashCode)? > > > > > > Thoughts? > > > > > > P.S. After I added manual hashcode calculation to my CacheJdbcPojoStore > > > performance drop is gone away. > > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-1753 > > > -- > > > Alexey Kuznetsov > > > GridGain Systems > > > www.gridgain.com > > > > > > > > -- > Alexey Kuznetsov > GridGain Systems > www.gridgain.com > |
|
|
Dmitriy,
>> How about we check that dashcode is not 0 when storing portable keys in cache? And print warning in log? Sounds good for me. ----- But I will try to describe once again my concern. I'm don't know anything about portables. So, I open javadoc and see some idiomatic code like this: builder = ... for ( fileds) { builder.setField(..) } portable = builder.build(); And I in my PojoStore I do as described. If there was a NOTE in javadoc about hashCode() it will save 4 hours of debug for me :) On Wed, Oct 28, 2015 at 11:32 AM, Dmitriy Setrakyan <[hidden email]> wrote: > How about we check that dashcode is not 0 when storing portable keys in > cache? > > On Tue, Oct 27, 2015 at 8:42 PM, Alexey Kuznetsov <[hidden email] > > > wrote: > > > Andrey, thanks. > > > > Actually I use this not well documented method to solve my problem. > > > > And I'm proposed to at least add more info for PortableBuilder about this > > method. > > > > On Wed, Oct 28, 2015 at 10:19 AM, Andrey Kornev < > [hidden email]> > > wrote: > > > > > Alexey, > > > > > > PortableBuilder has an (un-documented) method hashCode(int hashCode) > that > > > should be used to explicitly set the hashCode for the portable instance > > > being built. I'm not sure why this has been designed this way, but I'm > > > guessing that since the PortableBuilder is pretty dumb and it wouldn't > > know > > > which fields to use for hash code computation (in some cases you'd only > > > want to include specific portable fields rather than all fields). > > > > > > Regards > > > Andrey > > > > > > > Date: Wed, 28 Oct 2015 09:49:07 +0700 > > > > Subject: Portables hash code. > > > > From: [hidden email] > > > > To: [hidden email] > > > > > > > > Igniters, > > > > > > > > I'm working on [1] "IGNITE-1753 Rework CacheJdbcPojoStore to new > API." > > > > > > > > And one of subtasks is to support portable objects with JDBC store. > > > > > > > > I implemented this and during tests found a huge performance drop > when > > I > > > > have PortableObject as key. > > > > > > > > After some debugging I found that all my portable objects have > hashCode > > > = 0. > > > > I'm using PortableBuilder to build my portable objects. > > > > > > > > And I expected that PortableBuilder will calculate proper hash code > for > > > me > > > > out of the box. > > > > > > > > I think we should at least describe in PortableBuilder javadocs that > by > > > > default PortableBuilder will return zero hashcode? > > > > > > > > Or we should calculate hashcode in PortableBuilder.build() method? > > > > > > > > Or may be we could add boolean argument PortableBuilder.build(boolean > > > > generateHashCode)? > > > > > > > > Thoughts? > > > > > > > > P.S. After I added manual hashcode calculation to my > CacheJdbcPojoStore > > > > performance drop is gone away. > > > > > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-1753 > > > > -- > > > > Alexey Kuznetsov > > > > GridGain Systems > > > > www.gridgain.com > > > > > > > > > > > > > > -- > > Alexey Kuznetsov > > GridGain Systems > > www.gridgain.com > > > -- Alexey Kuznetsov GridGain Systems www.gridgain.com |
|
|
Sure, makes sense to add a documentation section to PortableObject describing default implementation of hashCode/equals.
However, I don’t see any significant reason why the end user needs to use portable objects as keys. — Denis > On 28 окт. 2015 г., at 7:42, Alexey Kuznetsov <[hidden email]> wrote: > > Dmitriy, > >>> How about we check that dashcode is not 0 when storing portable keys in > cache? > > And print warning in log? Sounds good for me. > > ----- > > But I will try to describe once again my concern. > > I'm don't know anything about portables. So, I open javadoc and see some > idiomatic code like this: > > builder = ... > > for ( fileds) { > builder.setField(..) > } > > portable = builder.build(); > > And I in my PojoStore I do as described. > > If there was a NOTE in javadoc about hashCode() it will save 4 hours of > debug for me :) > > > > > On Wed, Oct 28, 2015 at 11:32 AM, Dmitriy Setrakyan <[hidden email]> > wrote: > >> How about we check that dashcode is not 0 when storing portable keys in >> cache? >> >> On Tue, Oct 27, 2015 at 8:42 PM, Alexey Kuznetsov <[hidden email] >>> >> wrote: >> >>> Andrey, thanks. >>> >>> Actually I use this not well documented method to solve my problem. >>> >>> And I'm proposed to at least add more info for PortableBuilder about this >>> method. >>> >>> On Wed, Oct 28, 2015 at 10:19 AM, Andrey Kornev < >> [hidden email]> >>> wrote: >>> >>>> Alexey, >>>> >>>> PortableBuilder has an (un-documented) method hashCode(int hashCode) >> that >>>> should be used to explicitly set the hashCode for the portable instance >>>> being built. I'm not sure why this has been designed this way, but I'm >>>> guessing that since the PortableBuilder is pretty dumb and it wouldn't >>> know >>>> which fields to use for hash code computation (in some cases you'd only >>>> want to include specific portable fields rather than all fields). >>>> >>>> Regards >>>> Andrey >>>> >>>>> Date: Wed, 28 Oct 2015 09:49:07 +0700 >>>>> Subject: Portables hash code. >>>>> From: [hidden email] >>>>> To: [hidden email] >>>>> >>>>> Igniters, >>>>> >>>>> I'm working on [1] "IGNITE-1753 Rework CacheJdbcPojoStore to new >> API." >>>>> >>>>> And one of subtasks is to support portable objects with JDBC store. >>>>> >>>>> I implemented this and during tests found a huge performance drop >> when >>> I >>>>> have PortableObject as key. >>>>> >>>>> After some debugging I found that all my portable objects have >> hashCode >>>> = 0. >>>>> I'm using PortableBuilder to build my portable objects. >>>>> >>>>> And I expected that PortableBuilder will calculate proper hash code >> for >>>> me >>>>> out of the box. >>>>> >>>>> I think we should at least describe in PortableBuilder javadocs that >> by >>>>> default PortableBuilder will return zero hashcode? >>>>> >>>>> Or we should calculate hashcode in PortableBuilder.build() method? >>>>> >>>>> Or may be we could add boolean argument PortableBuilder.build(boolean >>>>> generateHashCode)? >>>>> >>>>> Thoughts? >>>>> >>>>> P.S. After I added manual hashcode calculation to my >> CacheJdbcPojoStore >>>>> performance drop is gone away. >>>>> >>>>> >>>>> [1] https://issues.apache.org/jira/browse/IGNITE-1753 >>>>> -- >>>>> Alexey Kuznetsov >>>>> GridGain Systems >>>>> www.gridgain.com >>>> >>>> >>> >>> >>> >>> -- >>> Alexey Kuznetsov >>> GridGain Systems >>> www.gridgain.com >>> >> > > > > -- > Alexey Kuznetsov > GridGain Systems > www.gridgain.com |
|
|
Denis,
My use case: In my legacy database I have table Organization with compound ID: (sbjID; orgID). For Pojo store will be generated key with 2 fields inside: OrganizationKey(int sbjID; int orgID). And store will load this key into cache. Of course usual scenario will be when key is primitive (int or long), but portable also could be a key. And one more case: several types in same cache: for store we should have different key types. On Wed, Oct 28, 2015 at 12:58 PM, Denis Magda <[hidden email]> wrote: > Sure, makes sense to add a documentation section to PortableObject > describing default implementation of hashCode/equals. > > However, I don’t see any significant reason why the end user needs to use > portable objects as keys. > > — > Denis > > > On 28 окт. 2015 г., at 7:42, Alexey Kuznetsov <[hidden email]> > wrote: > > > > Dmitriy, > > > >>> How about we check that dashcode is not 0 when storing portable keys in > > cache? > > > > And print warning in log? Sounds good for me. > > > > ----- > > > > But I will try to describe once again my concern. > > > > I'm don't know anything about portables. So, I open javadoc and see some > > idiomatic code like this: > > > > builder = ... > > > > for ( fileds) { > > builder.setField(..) > > } > > > > portable = builder.build(); > > > > And I in my PojoStore I do as described. > > > > If there was a NOTE in javadoc about hashCode() it will save 4 hours of > > debug for me :) > > > > > > > > > > On Wed, Oct 28, 2015 at 11:32 AM, Dmitriy Setrakyan < > [hidden email]> > > wrote: > > > >> How about we check that dashcode is not 0 when storing portable keys in > >> cache? > >> > >> On Tue, Oct 27, 2015 at 8:42 PM, Alexey Kuznetsov < > [hidden email] > >>> > >> wrote: > >> > >>> Andrey, thanks. > >>> > >>> Actually I use this not well documented method to solve my problem. > >>> > >>> And I'm proposed to at least add more info for PortableBuilder about > this > >>> method. > >>> > >>> On Wed, Oct 28, 2015 at 10:19 AM, Andrey Kornev < > >> [hidden email]> > >>> wrote: > >>> > >>>> Alexey, > >>>> > >>>> PortableBuilder has an (un-documented) method hashCode(int hashCode) > >> that > >>>> should be used to explicitly set the hashCode for the portable > instance > >>>> being built. I'm not sure why this has been designed this way, but I'm > >>>> guessing that since the PortableBuilder is pretty dumb and it wouldn't > >>> know > >>>> which fields to use for hash code computation (in some cases you'd > only > >>>> want to include specific portable fields rather than all fields). > >>>> > >>>> Regards > >>>> Andrey > >>>> > >>>>> Date: Wed, 28 Oct 2015 09:49:07 +0700 > >>>>> Subject: Portables hash code. > >>>>> From: [hidden email] > >>>>> To: [hidden email] > >>>>> > >>>>> Igniters, > >>>>> > >>>>> I'm working on [1] "IGNITE-1753 Rework CacheJdbcPojoStore to new > >> API." > >>>>> > >>>>> And one of subtasks is to support portable objects with JDBC store. > >>>>> > >>>>> I implemented this and during tests found a huge performance drop > >> when > >>> I > >>>>> have PortableObject as key. > >>>>> > >>>>> After some debugging I found that all my portable objects have > >> hashCode > >>>> = 0. > >>>>> I'm using PortableBuilder to build my portable objects. > >>>>> > >>>>> And I expected that PortableBuilder will calculate proper hash code > >> for > >>>> me > >>>>> out of the box. > >>>>> > >>>>> I think we should at least describe in PortableBuilder javadocs that > >> by > >>>>> default PortableBuilder will return zero hashcode? > >>>>> > >>>>> Or we should calculate hashcode in PortableBuilder.build() method? > >>>>> > >>>>> Or may be we could add boolean argument PortableBuilder.build(boolean > >>>>> generateHashCode)? > >>>>> > >>>>> Thoughts? > >>>>> > >>>>> P.S. After I added manual hashcode calculation to my > >> CacheJdbcPojoStore > >>>>> performance drop is gone away. > >>>>> > >>>>> > >>>>> [1] https://issues.apache.org/jira/browse/IGNITE-1753 > >>>>> -- > >>>>> Alexey Kuznetsov > >>>>> GridGain Systems > >>>>> www.gridgain.com > >>>> > >>>> > >>> > >>> > >>> > >>> -- > >>> Alexey Kuznetsov > >>> GridGain Systems > >>> www.gridgain.com > >>> > >> > > > > > > > > -- > > Alexey Kuznetsov > > GridGain Systems > > www.gridgain.com > > -- Alexey Kuznetsov GridGain Systems www.gridgain.com |
|
|
Igniters,
After some thinking on hash code generation for portables and POJO store we could do the following: 1) As Dmitriy suggested in [1] a boolean flag "isHashCode" to JdbcTypeField. If true then this field will be used for hash code calculation by pojo store. 2) Add to all PortableBuilder.setField() methods one more boolean argument isHashCode. 3) PortableBuilder should store fields in order they were added. 4) PortableBuilder will calculate hash code if at least one field was added with this flag. I think we could use algorithm like this (pseudocode): int hashCode = 0; for (field : fields) { if (field.isHashCode) hashCode = 31 * hashCode + (field.value != null ? field.value.hashCode() : 0); } . 5) Describe in documentation that ORDER of fields is VERY IMPORTANT for hash code calculation and equality checks. Thoughs? [1] https://cwiki.apache.org/confluence/display/IGNITE/Jdbc+Store+Configuration On Wed, Oct 28, 2015 at 1:06 PM, Alexey Kuznetsov <[hidden email]> wrote: > Denis, > > My use case: > > In my legacy database I have table Organization with compound ID: (sbjID; > orgID). > For Pojo store will be generated key with 2 fields inside: > OrganizationKey(int sbjID; int orgID). > > And store will load this key into cache. > > Of course usual scenario will be when key is primitive (int or long), but > portable also could be a key. > > And one more case: several types in same cache: for store we should have > different key types. > > > On Wed, Oct 28, 2015 at 12:58 PM, Denis Magda <[hidden email]> wrote: > >> Sure, makes sense to add a documentation section to PortableObject >> describing default implementation of hashCode/equals. >> >> However, I don’t see any significant reason why the end user needs to use >> portable objects as keys. >> >> — >> Denis >> >> > On 28 окт. 2015 г., at 7:42, Alexey Kuznetsov <[hidden email]> >> wrote: >> > >> > Dmitriy, >> > >> >>> How about we check that dashcode is not 0 when storing portable keys >> in >> > cache? >> > >> > And print warning in log? Sounds good for me. >> > >> > ----- >> > >> > But I will try to describe once again my concern. >> > >> > I'm don't know anything about portables. So, I open javadoc and see some >> > idiomatic code like this: >> > >> > builder = ... >> > >> > for ( fileds) { >> > builder.setField(..) >> > } >> > >> > portable = builder.build(); >> > >> > And I in my PojoStore I do as described. >> > >> > If there was a NOTE in javadoc about hashCode() it will save 4 hours of >> > debug for me :) >> > >> > >> > >> > >> > On Wed, Oct 28, 2015 at 11:32 AM, Dmitriy Setrakyan < >> [hidden email]> >> > wrote: >> > >> >> How about we check that dashcode is not 0 when storing portable keys in >> >> cache? >> >> >> >> On Tue, Oct 27, 2015 at 8:42 PM, Alexey Kuznetsov < >> [hidden email] >> >>> >> >> wrote: >> >> >> >>> Andrey, thanks. >> >>> >> >>> Actually I use this not well documented method to solve my problem. >> >>> >> >>> And I'm proposed to at least add more info for PortableBuilder about >> this >> >>> method. >> >>> >> >>> On Wed, Oct 28, 2015 at 10:19 AM, Andrey Kornev < >> >> [hidden email]> >> >>> wrote: >> >>> >> >>>> Alexey, >> >>>> >> >>>> PortableBuilder has an (un-documented) method hashCode(int hashCode) >> >> that >> >>>> should be used to explicitly set the hashCode for the portable >> instance >> >>>> being built. I'm not sure why this has been designed this way, but >> I'm >> >>>> guessing that since the PortableBuilder is pretty dumb and it >> wouldn't >> >>> know >> >>>> which fields to use for hash code computation (in some cases you'd >> only >> >>>> want to include specific portable fields rather than all fields). >> >>>> >> >>>> Regards >> >>>> Andrey >> >>>> >> >>>>> Date: Wed, 28 Oct 2015 09:49:07 +0700 >> >>>>> Subject: Portables hash code. >> >>>>> From: [hidden email] >> >>>>> To: [hidden email] >> >>>>> >> >>>>> Igniters, >> >>>>> >> >>>>> I'm working on [1] "IGNITE-1753 Rework CacheJdbcPojoStore to new >> >> API." >> >>>>> >> >>>>> And one of subtasks is to support portable objects with JDBC store. >> >>>>> >> >>>>> I implemented this and during tests found a huge performance drop >> >> when >> >>> I >> >>>>> have PortableObject as key. >> >>>>> >> >>>>> After some debugging I found that all my portable objects have >> >> hashCode >> >>>> = 0. >> >>>>> I'm using PortableBuilder to build my portable objects. >> >>>>> >> >>>>> And I expected that PortableBuilder will calculate proper hash code >> >> for >> >>>> me >> >>>>> out of the box. >> >>>>> >> >>>>> I think we should at least describe in PortableBuilder javadocs that >> >> by >> >>>>> default PortableBuilder will return zero hashcode? >> >>>>> >> >>>>> Or we should calculate hashcode in PortableBuilder.build() method? >> >>>>> >> >>>>> Or may be we could add boolean argument >> PortableBuilder.build(boolean >> >>>>> generateHashCode)? >> >>>>> >> >>>>> Thoughts? >> >>>>> >> >>>>> P.S. After I added manual hashcode calculation to my >> >> CacheJdbcPojoStore >> >>>>> performance drop is gone away. >> >>>>> >> >>>>> >> >>>>> [1] https://issues.apache.org/jira/browse/IGNITE-1753 >> >>>>> -- >> >>>>> Alexey Kuznetsov >> >>>>> GridGain Systems >> >>>>> www.gridgain.com >> >>>> >> >>>> >> >>> >> >>> >> >>> >> >>> -- >> >>> Alexey Kuznetsov >> >>> GridGain Systems >> >>> www.gridgain.com >> >>> >> >> >> > >> > >> > >> > -- >> > Alexey Kuznetsov >> > GridGain Systems >> > www.gridgain.com >> >> > > > -- > Alexey Kuznetsov > GridGain Systems > www.gridgain.com > -- Alexey Kuznetsov GridGain Systems www.gridgain.com |
|
|
I do not like idea of predefined alogrithms, because we will lack
flexibility. What if user has backing Java/.Net class where hash code is calculated differently? What if user *cannot change* this class for some reason? Any hard-coded hash code algorithm looks like a shortcut for me. Moreover, hash code can be different on different paltforms for the same base type. E.g. in Java hash code of string is more or less constant, while in .Net it changes from release to release. Java UUID has one hash code, .Net counterpart Guid has another hash code. Etc.. As a result user will be able to use this solution without any problems only if he operates on primitive types. Why not abstracting out this to some interface and provide default implementation with proposed algorithm? On Wed, Oct 28, 2015 at 5:49 PM, Alexey Kuznetsov <[hidden email]> wrote: > Igniters, > > After some thinking on hash code generation for portables and POJO store we > could do the following: > > 1) As Dmitriy suggested in [1] a boolean flag "isHashCode" to > JdbcTypeField. If true then this field will be used for hash code > calculation by pojo store. > 2) Add to all PortableBuilder.setField() methods one more boolean argument > isHashCode. > 3) PortableBuilder should store fields in order they were added. > 4) PortableBuilder will calculate hash code if at least one field was added > with this flag. > I think we could use algorithm like this (pseudocode): > int hashCode = 0; > > for (field : fields) { > if (field.isHashCode) > hashCode = 31 * hashCode + (field.value != null > ? field.value.hashCode() : 0); > } > . > 5) Describe in documentation that ORDER of fields is VERY IMPORTANT for > hash code calculation and equality checks. > > Thoughs? > > [1] > https://cwiki.apache.org/confluence/display/IGNITE/Jdbc+Store+Configuration > > On Wed, Oct 28, 2015 at 1:06 PM, Alexey Kuznetsov <[hidden email] > > > wrote: > > > Denis, > > > > My use case: > > > > In my legacy database I have table Organization with compound ID: (sbjID; > > orgID). > > For Pojo store will be generated key with 2 fields inside: > > OrganizationKey(int sbjID; int orgID). > > > > And store will load this key into cache. > > > > Of course usual scenario will be when key is primitive (int or long), but > > portable also could be a key. > > > > And one more case: several types in same cache: for store we should have > > different key types. > > > > > > On Wed, Oct 28, 2015 at 12:58 PM, Denis Magda <[hidden email]> > wrote: > > > >> Sure, makes sense to add a documentation section to PortableObject > >> describing default implementation of hashCode/equals. > >> > >> However, I don’t see any significant reason why the end user needs to > use > >> portable objects as keys. > >> > >> — > >> Denis > >> > >> > On 28 окт. 2015 г., at 7:42, Alexey Kuznetsov < > [hidden email]> > >> wrote: > >> > > >> > Dmitriy, > >> > > >> >>> How about we check that dashcode is not 0 when storing portable keys > >> in > >> > cache? > >> > > >> > And print warning in log? Sounds good for me. > >> > > >> > ----- > >> > > >> > But I will try to describe once again my concern. > >> > > >> > I'm don't know anything about portables. So, I open javadoc and see > some > >> > idiomatic code like this: > >> > > >> > builder = ... > >> > > >> > for ( fileds) { > >> > builder.setField(..) > >> > } > >> > > >> > portable = builder.build(); > >> > > >> > And I in my PojoStore I do as described. > >> > > >> > If there was a NOTE in javadoc about hashCode() it will save 4 hours > of > >> > debug for me :) > >> > > >> > > >> > > >> > > >> > On Wed, Oct 28, 2015 at 11:32 AM, Dmitriy Setrakyan < > >> [hidden email]> > >> > wrote: > >> > > >> >> How about we check that dashcode is not 0 when storing portable keys > in > >> >> cache? > >> >> > >> >> On Tue, Oct 27, 2015 at 8:42 PM, Alexey Kuznetsov < > >> [hidden email] > >> >>> > >> >> wrote: > >> >> > >> >>> Andrey, thanks. > >> >>> > >> >>> Actually I use this not well documented method to solve my problem. > >> >>> > >> >>> And I'm proposed to at least add more info for PortableBuilder about > >> this > >> >>> method. > >> >>> > >> >>> On Wed, Oct 28, 2015 at 10:19 AM, Andrey Kornev < > >> >> [hidden email]> > >> >>> wrote: > >> >>> > >> >>>> Alexey, > >> >>>> > >> >>>> PortableBuilder has an (un-documented) method hashCode(int > hashCode) > >> >> that > >> >>>> should be used to explicitly set the hashCode for the portable > >> instance > >> >>>> being built. I'm not sure why this has been designed this way, but > >> I'm > >> >>>> guessing that since the PortableBuilder is pretty dumb and it > >> wouldn't > >> >>> know > >> >>>> which fields to use for hash code computation (in some cases you'd > >> only > >> >>>> want to include specific portable fields rather than all fields). > >> >>>> > >> >>>> Regards > >> >>>> Andrey > >> >>>> > >> >>>>> Date: Wed, 28 Oct 2015 09:49:07 +0700 > >> >>>>> Subject: Portables hash code. > >> >>>>> From: [hidden email] > >> >>>>> To: [hidden email] > >> >>>>> > >> >>>>> Igniters, > >> >>>>> > >> >>>>> I'm working on [1] "IGNITE-1753 Rework CacheJdbcPojoStore to new > >> >> API." > >> >>>>> > >> >>>>> And one of subtasks is to support portable objects with JDBC > store. > >> >>>>> > >> >>>>> I implemented this and during tests found a huge performance drop > >> >> when > >> >>> I > >> >>>>> have PortableObject as key. > >> >>>>> > >> >>>>> After some debugging I found that all my portable objects have > >> >> hashCode > >> >>>> = 0. > >> >>>>> I'm using PortableBuilder to build my portable objects. > >> >>>>> > >> >>>>> And I expected that PortableBuilder will calculate proper hash > code > >> >> for > >> >>>> me > >> >>>>> out of the box. > >> >>>>> > >> >>>>> I think we should at least describe in PortableBuilder javadocs > that > >> >> by > >> >>>>> default PortableBuilder will return zero hashcode? > >> >>>>> > >> >>>>> Or we should calculate hashcode in PortableBuilder.build() method? > >> >>>>> > >> >>>>> Or may be we could add boolean argument > >> PortableBuilder.build(boolean > >> >>>>> generateHashCode)? > >> >>>>> > >> >>>>> Thoughts? > >> >>>>> > >> >>>>> P.S. After I added manual hashcode calculation to my > >> >> CacheJdbcPojoStore > >> >>>>> performance drop is gone away. > >> >>>>> > >> >>>>> > >> >>>>> [1] https://issues.apache.org/jira/browse/IGNITE-1753 > >> >>>>> -- > >> >>>>> Alexey Kuznetsov > >> >>>>> GridGain Systems > >> >>>>> www.gridgain.com > >> >>>> > >> >>>> > >> >>> > >> >>> > >> >>> > >> >>> -- > >> >>> Alexey Kuznetsov > >> >>> GridGain Systems > >> >>> www.gridgain.com > >> >>> > >> >> > >> > > >> > > >> > > >> > -- > >> > Alexey Kuznetsov > >> > GridGain Systems > >> > www.gridgain.com > >> > >> > > > > > > -- > > Alexey Kuznetsov > > GridGain Systems > > www.gridgain.com > > > > > > -- > Alexey Kuznetsov > GridGain Systems > www.gridgain.com > |
|
|
On Wed, Oct 28, 2015 at 8:12 AM, Vladimir Ozerov <[hidden email]>
wrote: > I do not like idea of predefined alogrithms, because we will lack > flexibility. What if user has backing Java/.Net class where hash code is > calculated differently? What if user *cannot change* this class for some > reason? > Any hard-coded hash code algorithm looks like a shortcut for me. > > Moreover, hash code can be different on different paltforms for the same > base type. E.g. in Java hash code of string is more or less constant, while > in .Net it changes from release to release. Java UUID has one hash code, > .Net counterpart Guid has another hash code. Etc.. As a result user will be > able to use this solution without any problems only if he operates on > primitive types. > > Why not abstracting out this to some interface and provide default > implementation with proposed algorithm? > How about adding JdbcTypeHasher interface, which will look like this: interface JdbcTypeHasher { public int hashCode(IgniteObject o, Collection<String> fields); } User should be optionally set this interface at the JdbcType level. If not set, then we use our default implementation. Will this work? > > On Wed, Oct 28, 2015 at 5:49 PM, Alexey Kuznetsov <[hidden email] > > > wrote: > > > Igniters, > > > > After some thinking on hash code generation for portables and POJO store > we > > could do the following: > > > > 1) As Dmitriy suggested in [1] a boolean flag "isHashCode" to > > JdbcTypeField. If true then this field will be used for hash code > > calculation by pojo store. > > 2) Add to all PortableBuilder.setField() methods one more boolean > argument > > isHashCode. > > 3) PortableBuilder should store fields in order they were added. > > 4) PortableBuilder will calculate hash code if at least one field was > added > > with this flag. > > I think we could use algorithm like this (pseudocode): > > int hashCode = 0; > > > > for (field : fields) { > > if (field.isHashCode) > > hashCode = 31 * hashCode + (field.value != null > > ? field.value.hashCode() : 0); > > } > > . > > 5) Describe in documentation that ORDER of fields is VERY IMPORTANT for > > hash code calculation and equality checks. > > > > Thoughs? > > > > [1] > > > https://cwiki.apache.org/confluence/display/IGNITE/Jdbc+Store+Configuration > > > > On Wed, Oct 28, 2015 at 1:06 PM, Alexey Kuznetsov < > [hidden email] > > > > > wrote: > > > > > Denis, > > > > > > My use case: > > > > > > In my legacy database I have table Organization with compound ID: > (sbjID; > > > orgID). > > > For Pojo store will be generated key with 2 fields inside: > > > OrganizationKey(int sbjID; int orgID). > > > > > > And store will load this key into cache. > > > > > > Of course usual scenario will be when key is primitive (int or long), > but > > > portable also could be a key. > > > > > > And one more case: several types in same cache: for store we should > have > > > different key types. > > > > > > > > > On Wed, Oct 28, 2015 at 12:58 PM, Denis Magda <[hidden email]> > > wrote: > > > > > >> Sure, makes sense to add a documentation section to PortableObject > > >> describing default implementation of hashCode/equals. > > >> > > >> However, I don’t see any significant reason why the end user needs to > > use > > >> portable objects as keys. > > >> > > >> — > > >> Denis > > >> > > >> > On 28 окт. 2015 г., at 7:42, Alexey Kuznetsov < > > [hidden email]> > > >> wrote: > > >> > > > >> > Dmitriy, > > >> > > > >> >>> How about we check that dashcode is not 0 when storing portable > keys > > >> in > > >> > cache? > > >> > > > >> > And print warning in log? Sounds good for me. > > >> > > > >> > ----- > > >> > > > >> > But I will try to describe once again my concern. > > >> > > > >> > I'm don't know anything about portables. So, I open javadoc and see > > some > > >> > idiomatic code like this: > > >> > > > >> > builder = ... > > >> > > > >> > for ( fileds) { > > >> > builder.setField(..) > > >> > } > > >> > > > >> > portable = builder.build(); > > >> > > > >> > And I in my PojoStore I do as described. > > >> > > > >> > If there was a NOTE in javadoc about hashCode() it will save 4 hours > > of > > >> > debug for me :) > > >> > > > >> > > > >> > > > >> > > > >> > On Wed, Oct 28, 2015 at 11:32 AM, Dmitriy Setrakyan < > > >> [hidden email]> > > >> > wrote: > > >> > > > >> >> How about we check that dashcode is not 0 when storing portable > keys > > in > > >> >> cache? > > >> >> > > >> >> On Tue, Oct 27, 2015 at 8:42 PM, Alexey Kuznetsov < > > >> [hidden email] > > >> >>> > > >> >> wrote: > > >> >> > > >> >>> Andrey, thanks. > > >> >>> > > >> >>> Actually I use this not well documented method to solve my > problem. > > >> >>> > > >> >>> And I'm proposed to at least add more info for PortableBuilder > about > > >> this > > >> >>> method. > > >> >>> > > >> >>> On Wed, Oct 28, 2015 at 10:19 AM, Andrey Kornev < > > >> >> [hidden email]> > > >> >>> wrote: > > >> >>> > > >> >>>> Alexey, > > >> >>>> > > >> >>>> PortableBuilder has an (un-documented) method hashCode(int > > hashCode) > > >> >> that > > >> >>>> should be used to explicitly set the hashCode for the portable > > >> instance > > >> >>>> being built. I'm not sure why this has been designed this way, > but > > >> I'm > > >> >>>> guessing that since the PortableBuilder is pretty dumb and it > > >> wouldn't > > >> >>> know > > >> >>>> which fields to use for hash code computation (in some cases > you'd > > >> only > > >> >>>> want to include specific portable fields rather than all fields). > > >> >>>> > > >> >>>> Regards > > >> >>>> Andrey > > >> >>>> > > >> >>>>> Date: Wed, 28 Oct 2015 09:49:07 +0700 > > >> >>>>> Subject: Portables hash code. > > >> >>>>> From: [hidden email] > > >> >>>>> To: [hidden email] > > >> >>>>> > > >> >>>>> Igniters, > > >> >>>>> > > >> >>>>> I'm working on [1] "IGNITE-1753 Rework CacheJdbcPojoStore to > new > > >> >> API." > > >> >>>>> > > >> >>>>> And one of subtasks is to support portable objects with JDBC > > store. > > >> >>>>> > > >> >>>>> I implemented this and during tests found a huge performance > drop > > >> >> when > > >> >>> I > > >> >>>>> have PortableObject as key. > > >> >>>>> > > >> >>>>> After some debugging I found that all my portable objects have > > >> >> hashCode > > >> >>>> = 0. > > >> >>>>> I'm using PortableBuilder to build my portable objects. > > >> >>>>> > > >> >>>>> And I expected that PortableBuilder will calculate proper hash > > code > > >> >> for > > >> >>>> me > > >> >>>>> out of the box. > > >> >>>>> > > >> >>>>> I think we should at least describe in PortableBuilder javadocs > > that > > >> >> by > > >> >>>>> default PortableBuilder will return zero hashcode? > > >> >>>>> > > >> >>>>> Or we should calculate hashcode in PortableBuilder.build() > method? > > >> >>>>> > > >> >>>>> Or may be we could add boolean argument > > >> PortableBuilder.build(boolean > > >> >>>>> generateHashCode)? > > >> >>>>> > > >> >>>>> Thoughts? > > >> >>>>> > > >> >>>>> P.S. After I added manual hashcode calculation to my > > >> >> CacheJdbcPojoStore > > >> >>>>> performance drop is gone away. > > >> >>>>> > > >> >>>>> > > >> >>>>> [1] https://issues.apache.org/jira/browse/IGNITE-1753 > > >> >>>>> -- > > >> >>>>> Alexey Kuznetsov > > >> >>>>> GridGain Systems > > >> >>>>> www.gridgain.com > > >> >>>> > > >> >>>> > > >> >>> > > >> >>> > > >> >>> > > >> >>> -- > > >> >>> Alexey Kuznetsov > > >> >>> GridGain Systems > > >> >>> www.gridgain.com > > >> >>> > > >> >> > > >> > > > >> > > > >> > > > >> > -- > > >> > Alexey Kuznetsov > > >> > GridGain Systems > > >> > www.gridgain.com > > >> > > >> > > > > > > > > > -- > > > Alexey Kuznetsov > > > GridGain Systems > > > www.gridgain.com > > > > > > > > > > > -- > > Alexey Kuznetsov > > GridGain Systems > > www.gridgain.com > > > |
|
|
Dmitriy,
> How about adding JdbcTypeHasher interface, which will look like this: > interface JdbcTypeHasher { > public int hashCode(IgniteObject o, Collection<String> fields); >} > User should be optionally set this interface at the JdbcType level. If not > set, then we use our default implementation. > Will this work? First I thought it will work, but when I started to implement hasher support I faced several problems: 1) IgniteObject/PortableObject is actually immutable, so setting hashcode for already constructed object looks dirty (set several bytes in array at certain offset). 2) It is not very efficient from implementation point of jdbc pojo store (on more cycle over object fields with not very efficient access by name) So, I came to this solution: // Hash builder. public interface JdbcTypeHashBuilder { // Calculate hash code for specified value and field name. public int toHash(Object val, String typeName, String fieldName); // Calculated hash code. public int hash(); } // Default implementation of JdbcTypeHashBuilder. User could implement its own if needed. public class JdbcTypeDefaultHashBuilder implements JdbcTypeHashBuilder { private int hash = 0; @Override public int toHash(Object val, String typeName, String fieldName) { hash = 31 * hash + (val != null ? val.hashCode() : 0); return hash; } @Override public int hash() { return hash; } } // Factory for creating default hash builder: public class JdbcTypeDefaultHashBuilderFactory implements Factory<JdbcTypeHashBuilder> { private static final long serialVersionUID = 0L; /** Instance of factory for reuse. */ public final static JdbcTypeDefaultHashBuilderFactory INSTANCE = new JdbcTypeDefaultHashBuilderFactory(); @Override public JdbcTypeHashBuilder create() { return new JdbcTypeDefaultHashBuilder(); } } And I add factory to CacheJdbcPojoStoreConfiguration { .... private Factory<JdbcTypeHashBuilder> hashBuilderFactory = JdbcTypeDefaultHashBuilderFactory.INSTANCE; } Now I could build PortableObject and calculate its hash at the same time: ... portableBuilder = ... hashBuilder = factory.create(); for (dbField: dbFields) { Object val = dbField.getValue(); portableBuilder.setField(dbField.getJavaName(), val) ; hashBuilder.toHash(val, typeName, dbField.getJavaName()); } return portableBuilder.hashCode(hashBuilder.hash()).build(); Thoughts? -- Alexey Kuznetsov GridGain Systems www.gridgain.com |
|
|
Alexey,
What if we create a thin wrapper around PortableBuilder that implements IgniteObject? Since PortableBuilder already has getField() method, you only need to delegate to a builder in this wrapper. Then we can invoke the interface Dmitriy suggested, obtain hashCode and set it back to the builder. In pseudo-code: PortableBuilder bld = getBuilder(); populateFromResultSet(bld); IgniteObject bldView = new BuilderIgniteObjectView(bld); int hashCode = hashBuilder.hashCode(bldView); bld.hashCode(hashCode); bld.build(); Thoughts? 2015-10-29 16:34 GMT+03:00 Alexey Kuznetsov <[hidden email]>: > Dmitriy, > > > How about adding JdbcTypeHasher interface, which will look like this: > > interface JdbcTypeHasher { > > public int hashCode(IgniteObject o, Collection<String> fields); > >} > > User should be optionally set this interface at the JdbcType level. If > not > > set, then we use our default implementation. > > Will this work? > > First I thought it will work, but when I started to implement hasher > support I faced several problems: > 1) IgniteObject/PortableObject is actually immutable, so setting hashcode > for already constructed object looks dirty (set several bytes in array at > certain offset). > 2) It is not very efficient from implementation point of jdbc pojo store > (on more cycle over object fields with not very efficient access by name) > > So, I came to this solution: > > // Hash builder. > public interface JdbcTypeHashBuilder { > // Calculate hash code for specified value and field name. > public int toHash(Object val, String typeName, String fieldName); > > // Calculated hash code. > public int hash(); > } > > // Default implementation of JdbcTypeHashBuilder. User could implement its > own if needed. > public class JdbcTypeDefaultHashBuilder implements JdbcTypeHashBuilder { > private int hash = 0; > > @Override public int toHash(Object val, String typeName, String > fieldName) { > hash = 31 * hash + (val != null ? val.hashCode() : 0); > > return hash; > } > > @Override public int hash() { > return hash; > } > } > > // Factory for creating default hash builder: > public class JdbcTypeDefaultHashBuilderFactory implements > Factory<JdbcTypeHashBuilder> { > private static final long serialVersionUID = 0L; > > /** Instance of factory for reuse. */ > public final static JdbcTypeDefaultHashBuilderFactory INSTANCE = new > JdbcTypeDefaultHashBuilderFactory(); > > @Override public JdbcTypeHashBuilder create() { > return new JdbcTypeDefaultHashBuilder(); > } > } > > And I add factory to CacheJdbcPojoStoreConfiguration { > .... > private Factory<JdbcTypeHashBuilder> hashBuilderFactory = > JdbcTypeDefaultHashBuilderFactory.INSTANCE; > } > > Now I could build PortableObject and calculate its hash at the same time: > ... > portableBuilder = ... > > hashBuilder = factory.create(); > > for (dbField: dbFields) { > Object val = dbField.getValue(); > > portableBuilder.setField(dbField.getJavaName(), val) ; > hashBuilder.toHash(val, typeName, dbField.getJavaName()); > } > > return portableBuilder.hashCode(hashBuilder.hash()).build(); > > Thoughts? > > -- > Alexey Kuznetsov > GridGain Systems > www.gridgain.com > |
|
|
Alexey,
I like this very much, this almost solve my problems and I do not need any fabrics! Thanks! On Thu, Oct 29, 2015 at 9:19 PM, Alexey Goncharuk < [hidden email]> wrote: > Alexey, > > What if we create a thin wrapper around PortableBuilder that implements > IgniteObject? Since PortableBuilder already has getField() method, you only > need to delegate to a builder in this wrapper. Then we can invoke the > interface Dmitriy suggested, obtain hashCode and set it back to the > builder. > > In pseudo-code: > > PortableBuilder bld = getBuilder(); > > populateFromResultSet(bld); > > IgniteObject bldView = new BuilderIgniteObjectView(bld); > > int hashCode = hashBuilder.hashCode(bldView); > bld.hashCode(hashCode); > > bld.build(); > > > Thoughts? > > 2015-10-29 16:34 GMT+03:00 Alexey Kuznetsov <[hidden email]>: > > > Dmitriy, > > > > > How about adding JdbcTypeHasher interface, which will look like this: > > > interface JdbcTypeHasher { > > > public int hashCode(IgniteObject o, Collection<String> fields); > > >} > > > User should be optionally set this interface at the JdbcType level. If > > not > > > set, then we use our default implementation. > > > Will this work? > > > > First I thought it will work, but when I started to implement hasher > > support I faced several problems: > > 1) IgniteObject/PortableObject is actually immutable, so setting > hashcode > > for already constructed object looks dirty (set several bytes in array at > > certain offset). > > 2) It is not very efficient from implementation point of jdbc pojo > store > > (on more cycle over object fields with not very efficient access by name) > > > > So, I came to this solution: > > > > // Hash builder. > > public interface JdbcTypeHashBuilder { > > // Calculate hash code for specified value and field name. > > public int toHash(Object val, String typeName, String fieldName); > > > > // Calculated hash code. > > public int hash(); > > } > > > > // Default implementation of JdbcTypeHashBuilder. User could implement > its > > own if needed. > > public class JdbcTypeDefaultHashBuilder implements JdbcTypeHashBuilder { > > private int hash = 0; > > > > @Override public int toHash(Object val, String typeName, String > > fieldName) { > > hash = 31 * hash + (val != null ? val.hashCode() : 0); > > > > return hash; > > } > > > > @Override public int hash() { > > return hash; > > } > > } > > > > // Factory for creating default hash builder: > > public class JdbcTypeDefaultHashBuilderFactory implements > > Factory<JdbcTypeHashBuilder> { > > private static final long serialVersionUID = 0L; > > > > /** Instance of factory for reuse. */ > > public final static JdbcTypeDefaultHashBuilderFactory INSTANCE = new > > JdbcTypeDefaultHashBuilderFactory(); > > > > @Override public JdbcTypeHashBuilder create() { > > return new JdbcTypeDefaultHashBuilder(); > > } > > } > > > > And I add factory to CacheJdbcPojoStoreConfiguration { > > .... > > private Factory<JdbcTypeHashBuilder> hashBuilderFactory = > > JdbcTypeDefaultHashBuilderFactory.INSTANCE; > > } > > > > Now I could build PortableObject and calculate its hash at the same time: > > ... > > portableBuilder = ... > > > > hashBuilder = factory.create(); > > > > for (dbField: dbFields) { > > Object val = dbField.getValue(); > > > > portableBuilder.setField(dbField.getJavaName(), val) ; > > hashBuilder.toHash(val, typeName, dbField.getJavaName()); > > } > > > > return portableBuilder.hashCode(hashBuilder.hash()).build(); > > > > Thoughts? > > > > -- > > Alexey Kuznetsov > > GridGain Systems > > www.gridgain.com > > > -- Alexey Kuznetsov GridGain Systems www.gridgain.com |
|
|
I am not sure I like this. We now require creation of some sort of
BuilderIgniteObjectView, which looks a bit over the top for me. How about using the hasher to calculate the hash-code and setting it into the builder manually in your implementation? To my knowledge, builder allows you to get individual field values, so you can get all the values required to calculate the hash value: For example, here is what a hasher can look like: —————- interface JdbcTypeHasher { int hashCode(Collection<?> values); } —————- Then, somewhere in your code, you will set the hash-code on the builder yourself: ————— IgniteObjectBuilder builder = …; // initialize Collection<?> c = new ArrayList<>(); for (each field required for hash) { c.add(builder.getField(name)); } int hash = hasher.hashCode(c); builder.hashCode(hash); // Set hash-code on the builder. —————— Thoughts? D. On Thu, Oct 29, 2015 at 7:23 AM, Alexey Kuznetsov <[hidden email]> wrote: > Alexey, > > I like this very much, this almost solve my problems and I do not need any > fabrics! > > Thanks! > > On Thu, Oct 29, 2015 at 9:19 PM, Alexey Goncharuk < > [hidden email]> wrote: > > > Alexey, > > > > What if we create a thin wrapper around PortableBuilder that implements > > IgniteObject? Since PortableBuilder already has getField() method, you > only > > need to delegate to a builder in this wrapper. Then we can invoke the > > interface Dmitriy suggested, obtain hashCode and set it back to the > > builder. > > > > In pseudo-code: > > > > PortableBuilder bld = getBuilder(); > > > > populateFromResultSet(bld); > > > > IgniteObject bldView = new BuilderIgniteObjectView(bld); > > > > int hashCode = hashBuilder.hashCode(bldView); > > bld.hashCode(hashCode); > > > > bld.build(); > > > > > > Thoughts? > > > > 2015-10-29 16:34 GMT+03:00 Alexey Kuznetsov <[hidden email]>: > > > > > Dmitriy, > > > > > > > How about adding JdbcTypeHasher interface, which will look like this: > > > > interface JdbcTypeHasher { > > > > public int hashCode(IgniteObject o, Collection<String> fields); > > > >} > > > > User should be optionally set this interface at the JdbcType level. > If > > > not > > > > set, then we use our default implementation. > > > > Will this work? > > > > > > First I thought it will work, but when I started to implement hasher > > > support I faced several problems: > > > 1) IgniteObject/PortableObject is actually immutable, so setting > > hashcode > > > for already constructed object looks dirty (set several bytes in array > at > > > certain offset). > > > 2) It is not very efficient from implementation point of jdbc pojo > > store > > > (on more cycle over object fields with not very efficient access by > name) > > > > > > So, I came to this solution: > > > > > > // Hash builder. > > > public interface JdbcTypeHashBuilder { > > > // Calculate hash code for specified value and field name. > > > public int toHash(Object val, String typeName, String fieldName); > > > > > > // Calculated hash code. > > > public int hash(); > > > } > > > > > > // Default implementation of JdbcTypeHashBuilder. User could implement > > its > > > own if needed. > > > public class JdbcTypeDefaultHashBuilder implements JdbcTypeHashBuilder > { > > > private int hash = 0; > > > > > > @Override public int toHash(Object val, String typeName, String > > > fieldName) { > > > hash = 31 * hash + (val != null ? val.hashCode() : 0); > > > > > > return hash; > > > } > > > > > > @Override public int hash() { > > > return hash; > > > } > > > } > > > > > > // Factory for creating default hash builder: > > > public class JdbcTypeDefaultHashBuilderFactory implements > > > Factory<JdbcTypeHashBuilder> { > > > private static final long serialVersionUID = 0L; > > > > > > /** Instance of factory for reuse. */ > > > public final static JdbcTypeDefaultHashBuilderFactory INSTANCE = > new > > > JdbcTypeDefaultHashBuilderFactory(); > > > > > > @Override public JdbcTypeHashBuilder create() { > > > return new JdbcTypeDefaultHashBuilder(); > > > } > > > } > > > > > > And I add factory to CacheJdbcPojoStoreConfiguration { > > > .... > > > private Factory<JdbcTypeHashBuilder> hashBuilderFactory = > > > JdbcTypeDefaultHashBuilderFactory.INSTANCE; > > > } > > > > > > Now I could build PortableObject and calculate its hash at the same > time: > > > ... > > > portableBuilder = ... > > > > > > hashBuilder = factory.create(); > > > > > > for (dbField: dbFields) { > > > Object val = dbField.getValue(); > > > > > > portableBuilder.setField(dbField.getJavaName(), val) ; > > > hashBuilder.toHash(val, typeName, dbField.getJavaName()); > > > } > > > > > > return portableBuilder.hashCode(hashBuilder.hash()).build(); > > > > > > Thoughts? > > > > > > -- > > > Alexey Kuznetsov > > > GridGain Systems > > > www.gridgain.com > > > > > > > > > -- > Alexey Kuznetsov > GridGain Systems > www.gridgain.com > |
|
|
Thanks, Dmitriy.
I think your solution is more straightforward, no need to create wrappers. Will implement this way. On Fri, Oct 30, 2015 at 6:00 AM, Dmitriy Setrakyan <[hidden email]> wrote: > I am not sure I like this. We now require creation of some sort of > BuilderIgniteObjectView, which looks a bit over the top for me. > > How about using the hasher to calculate the hash-code and setting it into > the builder manually in your implementation? To my knowledge, builder > allows you to get individual field values, so you can get all the values > required to calculate the hash value: > > For example, here is what a hasher can look like: > > —————- > interface JdbcTypeHasher { > int hashCode(Collection<?> values); > } > —————- > > Then, somewhere in your code, you will set the hash-code on the builder > yourself: > > ————— > IgniteObjectBuilder builder = …; // initialize > > Collection<?> c = new ArrayList<>(); > > for (each field required for hash) { > c.add(builder.getField(name)); > } > > int hash = hasher.hashCode(c); > > builder.hashCode(hash); // Set hash-code on the builder. > —————— > > Thoughts? > > D. > > On Thu, Oct 29, 2015 at 7:23 AM, Alexey Kuznetsov <[hidden email] > > > wrote: > > > Alexey, > > > > I like this very much, this almost solve my problems and I do not need > any > > fabrics! > > > > Thanks! > > > > On Thu, Oct 29, 2015 at 9:19 PM, Alexey Goncharuk < > > [hidden email]> wrote: > > > > > Alexey, > > > > > > What if we create a thin wrapper around PortableBuilder that implements > > > IgniteObject? Since PortableBuilder already has getField() method, you > > only > > > need to delegate to a builder in this wrapper. Then we can invoke the > > > interface Dmitriy suggested, obtain hashCode and set it back to the > > > builder. > > > > > > In pseudo-code: > > > > > > PortableBuilder bld = getBuilder(); > > > > > > populateFromResultSet(bld); > > > > > > IgniteObject bldView = new BuilderIgniteObjectView(bld); > > > > > > int hashCode = hashBuilder.hashCode(bldView); > > > bld.hashCode(hashCode); > > > > > > bld.build(); > > > > > > > > > Thoughts? > > > > > > 2015-10-29 16:34 GMT+03:00 Alexey Kuznetsov <[hidden email]>: > > > > > > > Dmitriy, > > > > > > > > > How about adding JdbcTypeHasher interface, which will look like > this: > > > > > interface JdbcTypeHasher { > > > > > public int hashCode(IgniteObject o, Collection<String> fields); > > > > >} > > > > > User should be optionally set this interface at the JdbcType level. > > If > > > > not > > > > > set, then we use our default implementation. > > > > > Will this work? > > > > > > > > First I thought it will work, but when I started to implement hasher > > > > support I faced several problems: > > > > 1) IgniteObject/PortableObject is actually immutable, so setting > > > hashcode > > > > for already constructed object looks dirty (set several bytes in > array > > at > > > > certain offset). > > > > 2) It is not very efficient from implementation point of jdbc pojo > > > store > > > > (on more cycle over object fields with not very efficient access by > > name) > > > > > > > > So, I came to this solution: > > > > > > > > // Hash builder. > > > > public interface JdbcTypeHashBuilder { > > > > // Calculate hash code for specified value and field name. > > > > public int toHash(Object val, String typeName, String fieldName); > > > > > > > > // Calculated hash code. > > > > public int hash(); > > > > } > > > > > > > > // Default implementation of JdbcTypeHashBuilder. User could > implement > > > its > > > > own if needed. > > > > public class JdbcTypeDefaultHashBuilder implements > JdbcTypeHashBuilder > > { > > > > private int hash = 0; > > > > > > > > @Override public int toHash(Object val, String typeName, String > > > > fieldName) { > > > > hash = 31 * hash + (val != null ? val.hashCode() : 0); > > > > > > > > return hash; > > > > } > > > > > > > > @Override public int hash() { > > > > return hash; > > > > } > > > > } > > > > > > > > // Factory for creating default hash builder: > > > > public class JdbcTypeDefaultHashBuilderFactory implements > > > > Factory<JdbcTypeHashBuilder> { > > > > private static final long serialVersionUID = 0L; > > > > > > > > /** Instance of factory for reuse. */ > > > > public final static JdbcTypeDefaultHashBuilderFactory INSTANCE = > > new > > > > JdbcTypeDefaultHashBuilderFactory(); > > > > > > > > @Override public JdbcTypeHashBuilder create() { > > > > return new JdbcTypeDefaultHashBuilder(); > > > > } > > > > } > > > > > > > > And I add factory to CacheJdbcPojoStoreConfiguration { > > > > .... > > > > private Factory<JdbcTypeHashBuilder> hashBuilderFactory = > > > > JdbcTypeDefaultHashBuilderFactory.INSTANCE; > > > > } > > > > > > > > Now I could build PortableObject and calculate its hash at the same > > time: > > > > ... > > > > portableBuilder = ... > > > > > > > > hashBuilder = factory.create(); > > > > > > > > for (dbField: dbFields) { > > > > Object val = dbField.getValue(); > > > > > > > > portableBuilder.setField(dbField.getJavaName(), val) ; > > > > hashBuilder.toHash(val, typeName, dbField.getJavaName()); > > > > } > > > > > > > > return portableBuilder.hashCode(hashBuilder.hash()).build(); > > > > > > > > Thoughts? > > > > > > > > -- > > > > Alexey Kuznetsov > > > > GridGain Systems > > > > www.gridgain.com > > > > > > > > > > > > > > > -- > > Alexey Kuznetsov > > GridGain Systems > > www.gridgain.com > > > -- Alexey Kuznetsov GridGain Systems www.gridgain.com |
«
Return to Apache Ignite Developers - Legacy Mail Archive
|
1 view|%1 views
| Free forum by Nabble | Edit this page |