Binary object hash code resolution in Apache Ignite 2.0

[Vladimir Ozerov]

Folks,

Historically we have fundamental flaw in how we deal with hashCode/equals
for BinaryObjects:
1) hashCode() is taken from original deserialized object.
2) But equals() is performed through binary array comparison.

We recently introduced BinaryIdentityResolver [1] as a part of DML feature,
but we didn't change any existing semantics, because it might broke
applications running AI 1.x.

Now I am thinking on how it should be designed properly in AI 2.0, and here
is my idea:
1) Every type will use BinaryArrayIdentityResolver [2] unless it is
overridden in config
2) Hash code is calculated only using resolver
3) Equality is determined only using resolver
4) Never ever use Object.hashCode().

This way we will have clean and consistent hashCode/equals semantics.

Any objections or comments?

Vladimir.

[1]
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/binary/BinaryIdentityResolver.java
[2]
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/binary/BinaryArrayIdentityResolver.java

[Pavel Tupitsyn]

Makes sense to me.

Ticket for reference: https://issues.apache.org/jira/browse/IGNITE-4558

On Fri, Jan 27, 2017 at 1:37 PM, Vladimir Ozerov <[hidden email]>
wrote:

> Folks,
>
> Historically we have fundamental flaw in how we deal with hashCode/equals
> for BinaryObjects:
> 1) hashCode() is taken from original deserialized object.
> 2) But equals() is performed through binary array comparison.
>
> We recently introduced BinaryIdentityResolver [1] as a part of DML feature,
> but we didn't change any existing semantics, because it might broke
> applications running AI 1.x.
>
> Now I am thinking on how it should be designed properly in AI 2.0, and here
> is my idea:
> 1) Every type will use BinaryArrayIdentityResolver [2] unless it is
> overridden in config
> 2) Hash code is calculated only using resolver
> 3) Equality is determined only using resolver
> 4) Never ever use Object.hashCode().
>
> This way we will have clean and consistent hashCode/equals semantics.
>
> Any objections or comments?
>
> Vladimir.
>
> [1]
> https://github.com/apache/ignite/blob/master/modules/
> core/src/main/java/org/apache/ignite/binary/BinaryIdentityResolver.java
> [2]
> https://github.com/apache/ignite/blob/master/modules/
> core/src/main/java/org/apache/ignite/binary/BinaryArrayIdentityResolver.
> java
>

[dsetrakyan]

Why shouldn't we use Object.hashCode() by default, even if the equals check
is done by comparing byte arrays? Wouldn't it be faster?

On Fri, Jan 27, 2017 at 4:38 AM, Pavel Tupitsyn <[hidden email]>
wrote:

> Makes sense to me.
>
> Ticket for reference: https://issues.apache.org/jira/browse/IGNITE-4558
>
> On Fri, Jan 27, 2017 at 1:37 PM, Vladimir Ozerov <[hidden email]>
> wrote:
>
> > Folks,
> >
> > Historically we have fundamental flaw in how we deal with hashCode/equals
> > for BinaryObjects:
> > 1) hashCode() is taken from original deserialized object.
> > 2) But equals() is performed through binary array comparison.
> >
> > We recently introduced BinaryIdentityResolver [1] as a part of DML
> feature,
> > but we didn't change any existing semantics, because it might broke
> > applications running AI 1.x.
> >
> > Now I am thinking on how it should be designed properly in AI 2.0, and
> here
> > is my idea:
> > 1) Every type will use BinaryArrayIdentityResolver [2] unless it is
> > overridden in config
> > 2) Hash code is calculated only using resolver
> > 3) Equality is determined only using resolver
> > 4) Never ever use Object.hashCode().
> >
> > This way we will have clean and consistent hashCode/equals semantics.
> >
> > Any objections or comments?
> >
> > Vladimir.
> >
> > [1]
> > https://github.com/apache/ignite/blob/master/modules/
> > core/src/main/java/org/apache/ignite/binary/BinaryIdentityResolver.java
> > [2]
> > https://github.com/apache/ignite/blob/master/modules/
> > core/src/main/java/org/apache/ignite/binary/BinaryArrayIdentityResolver.
> > java
> >
>

[Vladimir Ozerov]

Because it breaks equals/hashCode contract and cannot work with DML (we use
ugly hack as a workaround now).

27 янв. 2017 г. 20:33 пользователь "Dmitriy Setrakyan" <
[hidden email]> написал:

> Why shouldn't we use Object.hashCode() by default, even if the equals check
> is done by comparing byte arrays? Wouldn't it be faster?
>
> On Fri, Jan 27, 2017 at 4:38 AM, Pavel Tupitsyn <[hidden email]>
> wrote:
>
> > Makes sense to me.
> >
> > Ticket for reference: https://issues.apache.org/jira/browse/IGNITE-4558
> >
> > On Fri, Jan 27, 2017 at 1:37 PM, Vladimir Ozerov <[hidden email]>
> > wrote:
> >
> > > Folks,
> > >
> > > Historically we have fundamental flaw in how we deal with
> hashCode/equals
> > > for BinaryObjects:
> > > 1) hashCode() is taken from original deserialized object.
> > > 2) But equals() is performed through binary array comparison.
> > >
> > > We recently introduced BinaryIdentityResolver [1] as a part of DML
> > feature,
> > > but we didn't change any existing semantics, because it might broke
> > > applications running AI 1.x.
> > >
> > > Now I am thinking on how it should be designed properly in AI 2.0, and
> > here
> > > is my idea:
> > > 1) Every type will use BinaryArrayIdentityResolver [2] unless it is
> > > overridden in config
> > > 2) Hash code is calculated only using resolver
> > > 3) Equality is determined only using resolver
> > > 4) Never ever use Object.hashCode().
> > >
> > > This way we will have clean and consistent hashCode/equals semantics.
> > >
> > > Any objections or comments?
> > >
> > > Vladimir.
> > >
> > > [1]
> > > https://github.com/apache/ignite/blob/master/modules/
> > > core/src/main/java/org/apache/ignite/binary/
> BinaryIdentityResolver.java
> > > [2]
> > > https://github.com/apache/ignite/blob/master/modules/
> > > core/src/main/java/org/apache/ignite/binary/
> BinaryArrayIdentityResolver.
> > > java
> > >
> >
>

[dmagda]

Sounds reasonable to me. Actually, I thought that BinaryArrayIdentityResolver was already enabled by default since 1.8.

—
Denis

> On Jan 27, 2017, at 2:37 AM, Vladimir Ozerov <[hidden email]> wrote:
>
> Folks,
>
> Historically we have fundamental flaw in how we deal with hashCode/equals
> for BinaryObjects:
> 1) hashCode() is taken from original deserialized object.
> 2) But equals() is performed through binary array comparison.
>
> We recently introduced BinaryIdentityResolver [1] as a part of DML feature,
> but we didn't change any existing semantics, because it might broke
> applications running AI 1.x.
>
> Now I am thinking on how it should be designed properly in AI 2.0, and here
> is my idea:
> 1) Every type will use BinaryArrayIdentityResolver [2] unless it is
> overridden in config
> 2) Hash code is calculated only using resolver
> 3) Equality is determined only using resolver
> 4) Never ever use Object.hashCode().
>
> This way we will have clean and consistent hashCode/equals semantics.
>
> Any objections or comments?
>
> Vladimir.
>
> [1]
> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/binary/BinaryIdentityResolver.java
> [2]
> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/binary/BinaryArrayIdentityResolver.java

[Vladimir Ozerov]

No, we didn't do that because it could break existing user code. Not a
problem for AI 2.0.

On Fri, Jan 27, 2017 at 9:50 PM, Denis Magda <[hidden email]> wrote:

> Sounds reasonable to me. Actually, I thought that
> BinaryArrayIdentityResolver was already enabled by default since 1.8.
>
> —
> Denis
>
> > On Jan 27, 2017, at 2:37 AM, Vladimir Ozerov <[hidden email]>
> wrote:
> >
> > Folks,
> >
> > Historically we have fundamental flaw in how we deal with hashCode/equals
> > for BinaryObjects:
> > 1) hashCode() is taken from original deserialized object.
> > 2) But equals() is performed through binary array comparison.
> >
> > We recently introduced BinaryIdentityResolver [1] as a part of DML
> feature,
> > but we didn't change any existing semantics, because it might broke
> > applications running AI 1.x.
> >
> > Now I am thinking on how it should be designed properly in AI 2.0, and
> here
> > is my idea:
> > 1) Every type will use BinaryArrayIdentityResolver [2] unless it is
> > overridden in config
> > 2) Hash code is calculated only using resolver
> > 3) Equality is determined only using resolver
> > 4) Never ever use Object.hashCode().
> >
> > This way we will have clean and consistent hashCode/equals semantics.
> >
> > Any objections or comments?
> >
> > Vladimir.
> >
> > [1]
> > https://github.com/apache/ignite/blob/master/modules/
> core/src/main/java/org/apache/ignite/binary/BinaryIdentityResolver.java
> > [2]
> > https://github.com/apache/ignite/blob/master/modules/
> core/src/main/java/org/apache/ignite/binary/BinaryArrayIdentityResolver.
> java
>
>

[dsetrakyan]

On Fri, Jan 27, 2017 at 9:56 AM, Vladimir Ozerov <[hidden email]>
wrote:

> Because it breaks equals/hashCode contract and cannot work with DML (we use
> ugly hack as a workaround now).
>

Got it. Perhaps we can calculate hashCode only over the first 10 bytes or
so, for performance reasons.

>
> 27 янв. 2017 г. 20:33 пользователь "Dmitriy Setrakyan" <
> [hidden email]> написал:
>
> > Why shouldn't we use Object.hashCode() by default, even if the equals
> check
> > is done by comparing byte arrays? Wouldn't it be faster?
> >
> > On Fri, Jan 27, 2017 at 4:38 AM, Pavel Tupitsyn <[hidden email]>
> > wrote:
> >
> > > Makes sense to me.
> > >
> > > Ticket for reference: https://issues.apache.org/jira
> /browse/IGNITE-4558
> > >
> > > On Fri, Jan 27, 2017 at 1:37 PM, Vladimir Ozerov <[hidden email]
> >
> > > wrote:
> > >
> > > > Folks,
> > > >
> > > > Historically we have fundamental flaw in how we deal with
> > hashCode/equals
> > > > for BinaryObjects:
> > > > 1) hashCode() is taken from original deserialized object.
> > > > 2) But equals() is performed through binary array comparison.
> > > >
> > > > We recently introduced BinaryIdentityResolver [1] as a part of DML
> > > feature,
> > > > but we didn't change any existing semantics, because it might broke
> > > > applications running AI 1.x.
> > > >
> > > > Now I am thinking on how it should be designed properly in AI 2.0,
> and
> > > here
> > > > is my idea:
> > > > 1) Every type will use BinaryArrayIdentityResolver [2] unless it is
> > > > overridden in config
> > > > 2) Hash code is calculated only using resolver
> > > > 3) Equality is determined only using resolver
> > > > 4) Never ever use Object.hashCode().
> > > >
> > > > This way we will have clean and consistent hashCode/equals semantics.
> > > >
> > > > Any objections or comments?
> > > >
> > > > Vladimir.
> > > >
> > > > [1]
> > > > https://github.com/apache/ignite/blob/master/modules/
> > > > core/src/main/java/org/apache/ignite/binary/
> > BinaryIdentityResolver.java
> > > > [2]
> > > > https://github.com/apache/ignite/blob/master/modules/
> > > > core/src/main/java/org/apache/ignite/binary/
> > BinaryArrayIdentityResolver.
> > > > java
> > > >
> > >
> >
>

[Valentin Kulichenko]

I think hash code can be calculated during serialization, while we write
bytes. Obviously, there is no reason to calculate it each time hashCode()
is called on the same binary object.

-Val

On Fri, Jan 27, 2017 at 10:59 AM, Dmitriy Setrakyan <[hidden email]>
wrote:

> On Fri, Jan 27, 2017 at 9:56 AM, Vladimir Ozerov <[hidden email]>
> wrote:
>
> > Because it breaks equals/hashCode contract and cannot work with DML (we
> use
> > ugly hack as a workaround now).
> >
>
> Got it. Perhaps we can calculate hashCode only over the first 10 bytes or
> so, for performance reasons.
>
>
> >
> > 27 янв. 2017 г. 20:33 пользователь "Dmitriy Setrakyan" <
> > [hidden email]> написал:
> >
> > > Why shouldn't we use Object.hashCode() by default, even if the equals
> > check
> > > is done by comparing byte arrays? Wouldn't it be faster?
> > >
> > > On Fri, Jan 27, 2017 at 4:38 AM, Pavel Tupitsyn <[hidden email]>
> > > wrote:
> > >
> > > > Makes sense to me.
> > > >
> > > > Ticket for reference: https://issues.apache.org/jira
> > /browse/IGNITE-4558
> > > >
> > > > On Fri, Jan 27, 2017 at 1:37 PM, Vladimir Ozerov <
> [hidden email]
> > >
> > > > wrote:
> > > >
> > > > > Folks,
> > > > >
> > > > > Historically we have fundamental flaw in how we deal with
> > > hashCode/equals
> > > > > for BinaryObjects:
> > > > > 1) hashCode() is taken from original deserialized object.
> > > > > 2) But equals() is performed through binary array comparison.
> > > > >
> > > > > We recently introduced BinaryIdentityResolver [1] as a part of DML
> > > > feature,
> > > > > but we didn't change any existing semantics, because it might broke
> > > > > applications running AI 1.x.
> > > > >
> > > > > Now I am thinking on how it should be designed properly in AI 2.0,
> > and
> > > > here
> > > > > is my idea:
> > > > > 1) Every type will use BinaryArrayIdentityResolver [2] unless it is
> > > > > overridden in config
> > > > > 2) Hash code is calculated only using resolver
> > > > > 3) Equality is determined only using resolver
> > > > > 4) Never ever use Object.hashCode().
> > > > >
> > > > > This way we will have clean and consistent hashCode/equals
> semantics.
> > > > >
> > > > > Any objections or comments?
> > > > >
> > > > > Vladimir.
> > > > >
> > > > > [1]
> > > > > https://github.com/apache/ignite/blob/master/modules/
> > > > > core/src/main/java/org/apache/ignite/binary/
> > > BinaryIdentityResolver.java
> > > > > [2]
> > > > > https://github.com/apache/ignite/blob/master/modules/
> > > > > core/src/main/java/org/apache/ignite/binary/
> > > BinaryArrayIdentityResolver.
> > > > > java
> > > > >
> > > >
> > >
> >
>