Binary object hash code resolution in Apache Ignite 2.0

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Binary object hash code resolution in Apache Ignite 2.0

Vladimir Ozerov
Folks,

Historically we have fundamental flaw in how we deal with hashCode/equals
for BinaryObjects:
1) hashCode() is taken from original deserialized object.
2) But equals() is performed through binary array comparison.

We recently introduced BinaryIdentityResolver [1] as a part of DML feature,
but we didn't change any existing semantics, because it might broke
applications running AI 1.x.

Now I am thinking on how it should be designed properly in AI 2.0, and here
is my idea:
1) Every type will use BinaryArrayIdentityResolver [2] unless it is
overridden in config
2) Hash code is calculated only using resolver
3) Equality is determined only using resolver
4) Never ever use Object.hashCode().

This way we will have clean and consistent hashCode/equals semantics.

Any objections or comments?

Vladimir.

[1]
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/binary/BinaryIdentityResolver.java
[2]
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/binary/BinaryArrayIdentityResolver.java
Reply | Threaded
Open this post in threaded view
|

Re: Binary object hash code resolution in Apache Ignite 2.0

Pavel Tupitsyn
Makes sense to me.

Ticket for reference: https://issues.apache.org/jira/browse/IGNITE-4558

On Fri, Jan 27, 2017 at 1:37 PM, Vladimir Ozerov <[hidden email]>
wrote:

> Folks,
>
> Historically we have fundamental flaw in how we deal with hashCode/equals
> for BinaryObjects:
> 1) hashCode() is taken from original deserialized object.
> 2) But equals() is performed through binary array comparison.
>
> We recently introduced BinaryIdentityResolver [1] as a part of DML feature,
> but we didn't change any existing semantics, because it might broke
> applications running AI 1.x.
>
> Now I am thinking on how it should be designed properly in AI 2.0, and here
> is my idea:
> 1) Every type will use BinaryArrayIdentityResolver [2] unless it is
> overridden in config
> 2) Hash code is calculated only using resolver
> 3) Equality is determined only using resolver
> 4) Never ever use Object.hashCode().
>
> This way we will have clean and consistent hashCode/equals semantics.
>
> Any objections or comments?
>
> Vladimir.
>
> [1]
> https://github.com/apache/ignite/blob/master/modules/
> core/src/main/java/org/apache/ignite/binary/BinaryIdentityResolver.java
> [2]
> https://github.com/apache/ignite/blob/master/modules/
> core/src/main/java/org/apache/ignite/binary/BinaryArrayIdentityResolver.
> java
>
Reply | Threaded
Open this post in threaded view
|

Re: Binary object hash code resolution in Apache Ignite 2.0

dsetrakyan
Why shouldn't we use Object.hashCode() by default, even if the equals check
is done by comparing byte arrays? Wouldn't it be faster?

On Fri, Jan 27, 2017 at 4:38 AM, Pavel Tupitsyn <[hidden email]>
wrote:

> Makes sense to me.
>
> Ticket for reference: https://issues.apache.org/jira/browse/IGNITE-4558
>
> On Fri, Jan 27, 2017 at 1:37 PM, Vladimir Ozerov <[hidden email]>
> wrote:
>
> > Folks,
> >
> > Historically we have fundamental flaw in how we deal with hashCode/equals
> > for BinaryObjects:
> > 1) hashCode() is taken from original deserialized object.
> > 2) But equals() is performed through binary array comparison.
> >
> > We recently introduced BinaryIdentityResolver [1] as a part of DML
> feature,
> > but we didn't change any existing semantics, because it might broke
> > applications running AI 1.x.
> >
> > Now I am thinking on how it should be designed properly in AI 2.0, and
> here
> > is my idea:
> > 1) Every type will use BinaryArrayIdentityResolver [2] unless it is
> > overridden in config
> > 2) Hash code is calculated only using resolver
> > 3) Equality is determined only using resolver
> > 4) Never ever use Object.hashCode().
> >
> > This way we will have clean and consistent hashCode/equals semantics.
> >
> > Any objections or comments?
> >
> > Vladimir.
> >
> > [1]
> > https://github.com/apache/ignite/blob/master/modules/
> > core/src/main/java/org/apache/ignite/binary/BinaryIdentityResolver.java
> > [2]
> > https://github.com/apache/ignite/blob/master/modules/
> > core/src/main/java/org/apache/ignite/binary/BinaryArrayIdentityResolver.
> > java
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Binary object hash code resolution in Apache Ignite 2.0

Vladimir Ozerov
Because it breaks equals/hashCode contract and cannot work with DML (we use
ugly hack as a workaround now).

27 янв. 2017 г. 20:33 пользователь "Dmitriy Setrakyan" <
[hidden email]> написал:

> Why shouldn't we use Object.hashCode() by default, even if the equals check
> is done by comparing byte arrays? Wouldn't it be faster?
>
> On Fri, Jan 27, 2017 at 4:38 AM, Pavel Tupitsyn <[hidden email]>
> wrote:
>
> > Makes sense to me.
> >
> > Ticket for reference: https://issues.apache.org/jira/browse/IGNITE-4558
> >
> > On Fri, Jan 27, 2017 at 1:37 PM, Vladimir Ozerov <[hidden email]>
> > wrote:
> >
> > > Folks,
> > >
> > > Historically we have fundamental flaw in how we deal with
> hashCode/equals
> > > for BinaryObjects:
> > > 1) hashCode() is taken from original deserialized object.
> > > 2) But equals() is performed through binary array comparison.
> > >
> > > We recently introduced BinaryIdentityResolver [1] as a part of DML
> > feature,
> > > but we didn't change any existing semantics, because it might broke
> > > applications running AI 1.x.
> > >
> > > Now I am thinking on how it should be designed properly in AI 2.0, and
> > here
> > > is my idea:
> > > 1) Every type will use BinaryArrayIdentityResolver [2] unless it is
> > > overridden in config
> > > 2) Hash code is calculated only using resolver
> > > 3) Equality is determined only using resolver
> > > 4) Never ever use Object.hashCode().
> > >
> > > This way we will have clean and consistent hashCode/equals semantics.
> > >
> > > Any objections or comments?
> > >
> > > Vladimir.
> > >
> > > [1]
> > > https://github.com/apache/ignite/blob/master/modules/
> > > core/src/main/java/org/apache/ignite/binary/
> BinaryIdentityResolver.java
> > > [2]
> > > https://github.com/apache/ignite/blob/master/modules/
> > > core/src/main/java/org/apache/ignite/binary/
> BinaryArrayIdentityResolver.
> > > java
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Binary object hash code resolution in Apache Ignite 2.0

dmagda
In reply to this post by Vladimir Ozerov
Sounds reasonable to me. Actually, I thought that BinaryArrayIdentityResolver was already enabled by default since 1.8.


Denis

> On Jan 27, 2017, at 2:37 AM, Vladimir Ozerov <[hidden email]> wrote:
>
> Folks,
>
> Historically we have fundamental flaw in how we deal with hashCode/equals
> for BinaryObjects:
> 1) hashCode() is taken from original deserialized object.
> 2) But equals() is performed through binary array comparison.
>
> We recently introduced BinaryIdentityResolver [1] as a part of DML feature,
> but we didn't change any existing semantics, because it might broke
> applications running AI 1.x.
>
> Now I am thinking on how it should be designed properly in AI 2.0, and here
> is my idea:
> 1) Every type will use BinaryArrayIdentityResolver [2] unless it is
> overridden in config
> 2) Hash code is calculated only using resolver
> 3) Equality is determined only using resolver
> 4) Never ever use Object.hashCode().
>
> This way we will have clean and consistent hashCode/equals semantics.
>
> Any objections or comments?
>
> Vladimir.
>
> [1]
> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/binary/BinaryIdentityResolver.java
> [2]
> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/binary/BinaryArrayIdentityResolver.java

Reply | Threaded
Open this post in threaded view
|

Re: Binary object hash code resolution in Apache Ignite 2.0

Vladimir Ozerov
No, we didn't do that because it could break existing user code. Not a
problem for AI 2.0.

On Fri, Jan 27, 2017 at 9:50 PM, Denis Magda <[hidden email]> wrote:

> Sounds reasonable to me. Actually, I thought that
> BinaryArrayIdentityResolver was already enabled by default since 1.8.
>
> —
> Denis
>
> > On Jan 27, 2017, at 2:37 AM, Vladimir Ozerov <[hidden email]>
> wrote:
> >
> > Folks,
> >
> > Historically we have fundamental flaw in how we deal with hashCode/equals
> > for BinaryObjects:
> > 1) hashCode() is taken from original deserialized object.
> > 2) But equals() is performed through binary array comparison.
> >
> > We recently introduced BinaryIdentityResolver [1] as a part of DML
> feature,
> > but we didn't change any existing semantics, because it might broke
> > applications running AI 1.x.
> >
> > Now I am thinking on how it should be designed properly in AI 2.0, and
> here
> > is my idea:
> > 1) Every type will use BinaryArrayIdentityResolver [2] unless it is
> > overridden in config
> > 2) Hash code is calculated only using resolver
> > 3) Equality is determined only using resolver
> > 4) Never ever use Object.hashCode().
> >
> > This way we will have clean and consistent hashCode/equals semantics.
> >
> > Any objections or comments?
> >
> > Vladimir.
> >
> > [1]
> > https://github.com/apache/ignite/blob/master/modules/
> core/src/main/java/org/apache/ignite/binary/BinaryIdentityResolver.java
> > [2]
> > https://github.com/apache/ignite/blob/master/modules/
> core/src/main/java/org/apache/ignite/binary/BinaryArrayIdentityResolver.
> java
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Binary object hash code resolution in Apache Ignite 2.0

dsetrakyan
In reply to this post by Vladimir Ozerov
On Fri, Jan 27, 2017 at 9:56 AM, Vladimir Ozerov <[hidden email]>
wrote:

> Because it breaks equals/hashCode contract and cannot work with DML (we use
> ugly hack as a workaround now).
>

Got it. Perhaps we can calculate hashCode only over the first 10 bytes or
so, for performance reasons.


>
> 27 янв. 2017 г. 20:33 пользователь "Dmitriy Setrakyan" <
> [hidden email]> написал:
>
> > Why shouldn't we use Object.hashCode() by default, even if the equals
> check
> > is done by comparing byte arrays? Wouldn't it be faster?
> >
> > On Fri, Jan 27, 2017 at 4:38 AM, Pavel Tupitsyn <[hidden email]>
> > wrote:
> >
> > > Makes sense to me.
> > >
> > > Ticket for reference: https://issues.apache.org/jira
> /browse/IGNITE-4558
> > >
> > > On Fri, Jan 27, 2017 at 1:37 PM, Vladimir Ozerov <[hidden email]
> >
> > > wrote:
> > >
> > > > Folks,
> > > >
> > > > Historically we have fundamental flaw in how we deal with
> > hashCode/equals
> > > > for BinaryObjects:
> > > > 1) hashCode() is taken from original deserialized object.
> > > > 2) But equals() is performed through binary array comparison.
> > > >
> > > > We recently introduced BinaryIdentityResolver [1] as a part of DML
> > > feature,
> > > > but we didn't change any existing semantics, because it might broke
> > > > applications running AI 1.x.
> > > >
> > > > Now I am thinking on how it should be designed properly in AI 2.0,
> and
> > > here
> > > > is my idea:
> > > > 1) Every type will use BinaryArrayIdentityResolver [2] unless it is
> > > > overridden in config
> > > > 2) Hash code is calculated only using resolver
> > > > 3) Equality is determined only using resolver
> > > > 4) Never ever use Object.hashCode().
> > > >
> > > > This way we will have clean and consistent hashCode/equals semantics.
> > > >
> > > > Any objections or comments?
> > > >
> > > > Vladimir.
> > > >
> > > > [1]
> > > > https://github.com/apache/ignite/blob/master/modules/
> > > > core/src/main/java/org/apache/ignite/binary/
> > BinaryIdentityResolver.java
> > > > [2]
> > > > https://github.com/apache/ignite/blob/master/modules/
> > > > core/src/main/java/org/apache/ignite/binary/
> > BinaryArrayIdentityResolver.
> > > > java
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Binary object hash code resolution in Apache Ignite 2.0

Valentin Kulichenko
I think hash code can be calculated during serialization, while we write
bytes. Obviously, there is no reason to calculate it each time hashCode()
is called on the same binary object.

-Val

On Fri, Jan 27, 2017 at 10:59 AM, Dmitriy Setrakyan <[hidden email]>
wrote:

> On Fri, Jan 27, 2017 at 9:56 AM, Vladimir Ozerov <[hidden email]>
> wrote:
>
> > Because it breaks equals/hashCode contract and cannot work with DML (we
> use
> > ugly hack as a workaround now).
> >
>
> Got it. Perhaps we can calculate hashCode only over the first 10 bytes or
> so, for performance reasons.
>
>
> >
> > 27 янв. 2017 г. 20:33 пользователь "Dmitriy Setrakyan" <
> > [hidden email]> написал:
> >
> > > Why shouldn't we use Object.hashCode() by default, even if the equals
> > check
> > > is done by comparing byte arrays? Wouldn't it be faster?
> > >
> > > On Fri, Jan 27, 2017 at 4:38 AM, Pavel Tupitsyn <[hidden email]>
> > > wrote:
> > >
> > > > Makes sense to me.
> > > >
> > > > Ticket for reference: https://issues.apache.org/jira
> > /browse/IGNITE-4558
> > > >
> > > > On Fri, Jan 27, 2017 at 1:37 PM, Vladimir Ozerov <
> [hidden email]
> > >
> > > > wrote:
> > > >
> > > > > Folks,
> > > > >
> > > > > Historically we have fundamental flaw in how we deal with
> > > hashCode/equals
> > > > > for BinaryObjects:
> > > > > 1) hashCode() is taken from original deserialized object.
> > > > > 2) But equals() is performed through binary array comparison.
> > > > >
> > > > > We recently introduced BinaryIdentityResolver [1] as a part of DML
> > > > feature,
> > > > > but we didn't change any existing semantics, because it might broke
> > > > > applications running AI 1.x.
> > > > >
> > > > > Now I am thinking on how it should be designed properly in AI 2.0,
> > and
> > > > here
> > > > > is my idea:
> > > > > 1) Every type will use BinaryArrayIdentityResolver [2] unless it is
> > > > > overridden in config
> > > > > 2) Hash code is calculated only using resolver
> > > > > 3) Equality is determined only using resolver
> > > > > 4) Never ever use Object.hashCode().
> > > > >
> > > > > This way we will have clean and consistent hashCode/equals
> semantics.
> > > > >
> > > > > Any objections or comments?
> > > > >
> > > > > Vladimir.
> > > > >
> > > > > [1]
> > > > > https://github.com/apache/ignite/blob/master/modules/
> > > > > core/src/main/java/org/apache/ignite/binary/
> > > BinaryIdentityResolver.java
> > > > > [2]
> > > > > https://github.com/apache/ignite/blob/master/modules/
> > > > > core/src/main/java/org/apache/ignite/binary/
> > > BinaryArrayIdentityResolver.
> > > > > java
> > > > >
> > > >
> > >
> >
>