Interoperable Ignite.NET Dates

[Alexey Kukushkin]

Igniters,

What do you think about changing .NET API to read/write portable dates by default and making that really portable?

The Problem
Presently .NET API writes dates as composite Ignite objects. Only .NET clients can read such dates: any other client (JDBC, Java, etc) does not understand it without custom deserialization.

It is still possible to configure .NET serialization to write dates as Ignite dates - see DateTime Serialization note. But then Ignite accepts only UTC dates, requiring the application developers to convert local dates to UTC dates and back. This task is not trivial: DateTime.ToUniversalTime uses calendars different from Java (and the .NET calendars are the invalid ones - especially for pre-1990 dates).

The motivation for the current default behavior was probably the desire to keep the Time Zone information: Ignite dates do not store time zones.

In our experience interoperability is more important than storing time zone info.

The Proposal

1. Always write .NET dates as portable Ignite dates: get rid of the BinaryReflectiveSerializer.ForceTimestamp flag that currently triggers this behavior.
    - We could still keep the ForceTimestamp flag if saving .NET dates as transparent objects seems a useful case. We do not think it is useful.

2. Automatically convert Local dates to UTC and back inside Ignite.NET.
    - In this case we lose the DateTime.Kind of UTC dates: we write a UTC date but we would read a Local date since Ignite would always convert UTC to Local when reading.
      We could add a UtcDate date flag to QuerySqlFieldAttribute and BinaryReflectiveSerializer to control the deserialization behavior if keeping dates in UTC format use case seems important.

3. Use NodaTime for UTC<->Local conversions. Noda time uses Java calendars making the conversion truely portable.

The Benefits
1. We think portable dates are much more important than storing time zone info. Why do we store time zones for every date on the server anyway?
2. Simpler application code: no more manual configuration to trigger the portable behavior.
3. Non-trivial code to make the truly portable UTC<->Local conversion is implemented once inside Ignite instead of having every Ignite.NET application implementing it.

Thank you!

--
Best regards,
Alexey

[Pavel Tupitsyn]

Alexey,

Just to clarify before we start the discussion:
this proposal seems to introduce some breaking changes, so we are talking
about Ignite 3.0, correct?

Pavel

On Tue, Nov 3, 2020 at 12:13 AM Alexey Kukushkin <[hidden email]>
wrote:

> Igniters,
>
> What do you think about changing .NET API to read/write portable dates by
> default and making that really portable?
>
> *The Problem*
> Presently .NET API writes dates as composite Ignite objects. Only .NET
> clients can read such dates: any other client (JDBC, Java, etc) does not
> understand it without custom deserialization.
>
> It is still possible to configure .NET serialization to write dates as
> Ignite dates - see DateTime Serialization note
> <
> https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility
> >.
> But then Ignite accepts only UTC dates, requiring the application
> developers to convert local dates to UTC dates and back. This task is not
> trivial: DateTime.ToUniversalTime
> <
> https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1
> >
> uses
> calendars different from Java (and the .NET calendars are the invalid ones
> - especially for pre-1990 dates).
>
> The motivation for the current default behavior was probably the desire to
> keep the Time Zone information: Ignite dates do not store time zones.
>
> In our experience interoperability is more important than storing time zone
> info.
>
> *The Proposal*
>
>    1. Always write .NET dates as portable Ignite dates: get rid of the
>    BinaryReflectiveSerializer.ForceTimestamp flag that currently triggers
> this
>    behavior.
>       - We could still keep the ForceTimestamp flag if saving .NET dates as
>       transparent objects seems a useful case. We do not think it is
> useful.
>    2. Automatically convert Local dates to UTC and back *inside*
>    Ignite.NET.
>       - In this case we lose the DateTime.Kind of UTC dates: we write a UTC
>       date but we would read a Local date since Ignite would always
> convert UTC
>       to Local when reading.
>       We could add a UtcDate date flag to QuerySqlFieldAttribute
>       and BinaryReflectiveSerializer to control the deserialization
> behavior if
>       keeping dates in UTC format use case seems important.
>    3. Use NodaTime <https://nodatime.org/> for UTC<->Local conversions.
>    Noda time uses Java calendars making the conversion truely portable.
>
> *The Benefits*
>
>    1. We think portable dates are much more important than storing time
>    zone info. Why do we store time zones for every date on the server
> anyway?
>    Time zone is client-side info.
>    2. Simpler application code: no more manual configuration to trigger the
>    portable behavior.
>    3. Non-trivial code to make the truly portable UTC<->Local conversion is
>    implemented once inside Ignite instead of having every Ignite.NET
>    application implementing it.
>
> Thank you!
>
> --
> Best regards,
> Alexey
>

[Alexey Kukushkin]

Pavel,

We propose changing public API so this is for Ignite 3.0.

Thanks!

вт, 3 нояб. 2020 г. в 02:17, Pavel Tupitsyn <[hidden email]>:

> Alexey,
>
> Just to clarify before we start the discussion:
> this proposal seems to introduce some breaking changes, so we are talking
> about Ignite 3.0, correct?
>
> Pavel
>
> On Tue, Nov 3, 2020 at 12:13 AM Alexey Kukushkin <
> [hidden email]>
> wrote:
>
> > Igniters,
> >
> > What do you think about changing .NET API to read/write portable dates by
> > default and making that really portable?
> >
> > *The Problem*
> > Presently .NET API writes dates as composite Ignite objects. Only .NET
> > clients can read such dates: any other client (JDBC, Java, etc) does not
> > understand it without custom deserialization.
> >
> > It is still possible to configure .NET serialization to write dates as
> > Ignite dates - see DateTime Serialization note
> > <
> >
> https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility
> > >.
> > But then Ignite accepts only UTC dates, requiring the application
> > developers to convert local dates to UTC dates and back. This task is not
> > trivial: DateTime.ToUniversalTime
> > <
> >
> https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1
> > >
> > uses
> > calendars different from Java (and the .NET calendars are the invalid
> ones
> > - especially for pre-1990 dates).
> >
> > The motivation for the current default behavior was probably the desire
> to
> > keep the Time Zone information: Ignite dates do not store time zones.
> >
> > In our experience interoperability is more important than storing time
> zone
> > info.
> >
> > *The Proposal*
> >
> >    1. Always write .NET dates as portable Ignite dates: get rid of the
> >    BinaryReflectiveSerializer.ForceTimestamp flag that currently triggers
> > this
> >    behavior.
> >       - We could still keep the ForceTimestamp flag if saving .NET dates
> as
> >       transparent objects seems a useful case. We do not think it is
> > useful.
> >    2. Automatically convert Local dates to UTC and back *inside*
> >    Ignite.NET.
> >       - In this case we lose the DateTime.Kind of UTC dates: we write a
> UTC
> >       date but we would read a Local date since Ignite would always
> > convert UTC
> >       to Local when reading.
> >       We could add a UtcDate date flag to QuerySqlFieldAttribute
> >       and BinaryReflectiveSerializer to control the deserialization
> > behavior if
> >       keeping dates in UTC format use case seems important.
> >    3. Use NodaTime <https://nodatime.org/> for UTC<->Local conversions.
> >    Noda time uses Java calendars making the conversion truely portable.
> >
> > *The Benefits*
> >
> >    1. We think portable dates are much more important than storing time
> >    zone info. Why do we store time zones for every date on the server
> > anyway?
> >    Time zone is client-side info.
> >    2. Simpler application code: no more manual configuration to trigger
> the
> >    portable behavior.
> >    3. Non-trivial code to make the truly portable UTC<->Local conversion
> is
> >    implemented once inside Ignite instead of having every Ignite.NET
> >    application implementing it.
> >
> > Thank you!
> >
> > --
> > Best regards,
> > Alexey
> >
>

--
Best regards,
Alexey

[Valentin Kulichenko]

Hi Alexey,

The IEP-54 [1] describes the data layout proposed for Ignite 3.0, it
includes various date/time types. Can you please take a look and check if
this addresses your concerns? We can then discuss further if needed.

https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach

-Val

On Tue, Nov 3, 2020 at 12:05 AM Alexey Kukushkin <[hidden email]>
wrote:

> Pavel,
>
> We propose changing public API so this is for Ignite 3.0.
>
> Thanks!
>
> вт, 3 нояб. 2020 г. в 02:17, Pavel Tupitsyn <[hidden email]>:
>
> > Alexey,
> >
> > Just to clarify before we start the discussion:
> > this proposal seems to introduce some breaking changes, so we are talking
> > about Ignite 3.0, correct?
> >
> > Pavel
> >
> > On Tue, Nov 3, 2020 at 12:13 AM Alexey Kukushkin <
> > [hidden email]>
> > wrote:
> >
> > > Igniters,
> > >
> > > What do you think about changing .NET API to read/write portable dates
> by
> > > default and making that really portable?
> > >
> > > *The Problem*
> > > Presently .NET API writes dates as composite Ignite objects. Only .NET
> > > clients can read such dates: any other client (JDBC, Java, etc) does
> not
> > > understand it without custom deserialization.
> > >
> > > It is still possible to configure .NET serialization to write dates as
> > > Ignite dates - see DateTime Serialization note
> > > <
> > >
> >
> https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility
> > > >.
> > > But then Ignite accepts only UTC dates, requiring the application
> > > developers to convert local dates to UTC dates and back. This task is
> not
> > > trivial: DateTime.ToUniversalTime
> > > <
> > >
> >
> https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1
> > > >
> > > uses
> > > calendars different from Java (and the .NET calendars are the invalid
> > ones
> > > - especially for pre-1990 dates).
> > >
> > > The motivation for the current default behavior was probably the desire
> > to
> > > keep the Time Zone information: Ignite dates do not store time zones.
> > >
> > > In our experience interoperability is more important than storing time
> > zone
> > > info.
> > >
> > > *The Proposal*
> > >
> > >    1. Always write .NET dates as portable Ignite dates: get rid of the
> > >    BinaryReflectiveSerializer.ForceTimestamp flag that currently
> triggers
> > > this
> > >    behavior.
> > >       - We could still keep the ForceTimestamp flag if saving .NET
> dates
> > as
> > >       transparent objects seems a useful case. We do not think it is
> > > useful.
> > >    2. Automatically convert Local dates to UTC and back *inside*
> > >    Ignite.NET.
> > >       - In this case we lose the DateTime.Kind of UTC dates: we write a
> > UTC
> > >       date but we would read a Local date since Ignite would always
> > > convert UTC
> > >       to Local when reading.
> > >       We could add a UtcDate date flag to QuerySqlFieldAttribute
> > >       and BinaryReflectiveSerializer to control the deserialization
> > > behavior if
> > >       keeping dates in UTC format use case seems important.
> > >    3. Use NodaTime <https://nodatime.org/> for UTC<->Local
> conversions.
> > >    Noda time uses Java calendars making the conversion truely portable.
> > >
> > > *The Benefits*
> > >
> > >    1. We think portable dates are much more important than storing time
> > >    zone info. Why do we store time zones for every date on the server
> > > anyway?
> > >    Time zone is client-side info.
> > >    2. Simpler application code: no more manual configuration to trigger
> > the
> > >    portable behavior.
> > >    3. Non-trivial code to make the truly portable UTC<->Local
> conversion
> > is
> > >    implemented once inside Ignite instead of having every Ignite.NET
> > >    application implementing it.
> > >
> > > Thank you!
> > >
> > > --
> > > Best regards,
> > > Alexey
> > >
> >
>
>
> --
> Best regards,
> Alexey
>

[Alexey Kukushkin]

Hi Val,

Our proposal does not overlap with IEP-54
<https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach>,
which proposes changing Ignite date storage format. Our proposal is
enhancing Ignite to handle .NET dates in a truly portable way instead of
requiring the application developers to repeat this task every time:

   - Write .NET dates as Ignite dates (today .NET dates are written as
   generic Ignite objects)
   - Convert Local .NET dates to UTC inside Ignite and to it using Java
   calendars.


вт, 3 нояб. 2020 г. в 11:10, Valentin Kulichenko <
[hidden email]>:

> Hi Alexey,
>
> The IEP-54 [1] describes the data layout proposed for Ignite 3.0, it
> includes various date/time types. Can you please take a look and check if
> this addresses your concerns? We can then discuss further if needed.
>
>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
>
> -Val
>
> On Tue, Nov 3, 2020 at 12:05 AM Alexey Kukushkin <
> [hidden email]>
> wrote:
>
> > Pavel,
> >
> > We propose changing public API so this is for Ignite 3.0.
> >
> > Thanks!
> >
> > вт, 3 нояб. 2020 г. в 02:17, Pavel Tupitsyn <[hidden email]>:
> >
> > > Alexey,
> > >
> > > Just to clarify before we start the discussion:
> > > this proposal seems to introduce some breaking changes, so we are
> talking
> > > about Ignite 3.0, correct?
> > >
> > > Pavel
> > >
> > > On Tue, Nov 3, 2020 at 12:13 AM Alexey Kukushkin <
> > > [hidden email]>
> > > wrote:
> > >
> > > > Igniters,
> > > >
> > > > What do you think about changing .NET API to read/write portable
> dates
> > by
> > > > default and making that really portable?
> > > >
> > > > *The Problem*
> > > > Presently .NET API writes dates as composite Ignite objects. Only
> .NET
> > > > clients can read such dates: any other client (JDBC, Java, etc) does
> > not
> > > > understand it without custom deserialization.
> > > >
> > > > It is still possible to configure .NET serialization to write dates
> as
> > > > Ignite dates - see DateTime Serialization note
> > > > <
> > > >
> > >
> >
> https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility
> > > > >.
> > > > But then Ignite accepts only UTC dates, requiring the application
> > > > developers to convert local dates to UTC dates and back. This task is
> > not
> > > > trivial: DateTime.ToUniversalTime
> > > > <
> > > >
> > >
> >
> https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1
> > > > >
> > > > uses
> > > > calendars different from Java (and the .NET calendars are the invalid
> > > ones
> > > > - especially for pre-1990 dates).
> > > >
> > > > The motivation for the current default behavior was probably the
> desire
> > > to
> > > > keep the Time Zone information: Ignite dates do not store time zones.
> > > >
> > > > In our experience interoperability is more important than storing
> time
> > > zone
> > > > info.
> > > >
> > > > *The Proposal*
> > > >
> > > >    1. Always write .NET dates as portable Ignite dates: get rid of
> the
> > > >    BinaryReflectiveSerializer.ForceTimestamp flag that currently
> > triggers
> > > > this
> > > >    behavior.
> > > >       - We could still keep the ForceTimestamp flag if saving .NET
> > dates
> > > as
> > > >       transparent objects seems a useful case. We do not think it is
> > > > useful.
> > > >    2. Automatically convert Local dates to UTC and back *inside*
> > > >    Ignite.NET.
> > > >       - In this case we lose the DateTime.Kind of UTC dates: we
> write a
> > > UTC
> > > >       date but we would read a Local date since Ignite would always
> > > > convert UTC
> > > >       to Local when reading.
> > > >       We could add a UtcDate date flag to QuerySqlFieldAttribute
> > > >       and BinaryReflectiveSerializer to control the deserialization
> > > > behavior if
> > > >       keeping dates in UTC format use case seems important.
> > > >    3. Use NodaTime <https://nodatime.org/> for UTC<->Local
> > conversions.
> > > >    Noda time uses Java calendars making the conversion truely
> portable.
> > > >
> > > > *The Benefits*
> > > >
> > > >    1. We think portable dates are much more important than storing
> time
> > > >    zone info. Why do we store time zones for every date on the server
> > > > anyway?
> > > >    Time zone is client-side info.
> > > >    2. Simpler application code: no more manual configuration to
> trigger
> > > the
> > > >    portable behavior.
> > > >    3. Non-trivial code to make the truly portable UTC<->Local
> > conversion
> > > is
> > > >    implemented once inside Ignite instead of having every Ignite.NET
> > > >    application implementing it.
> > > >
> > > > Thank you!
> > > >
> > > > --
> > > > Best regards,
> > > > Alexey
> > > >
> > >
> >
> >
> > --
> > Best regards,
> > Alexey
> >
>

--
Best regards,
Alexey

[Valentin Kulichenko]

Got it. So it only affects the .NET side, correct?

-Val

On Tue, Nov 3, 2020 at 3:52 AM Alexey Kukushkin <[hidden email]>
wrote:

> Hi Val,
>
> Our proposal does not overlap with IEP-54
> <
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> >,
> which proposes changing Ignite date storage format. Our proposal is
> enhancing Ignite to handle .NET dates in a truly portable way instead of
> requiring the application developers to repeat this task every time:
>
>    - Write .NET dates as Ignite dates (today .NET dates are written as
>    generic Ignite objects)
>    - Convert Local .NET dates to UTC inside Ignite and to it using Java
>    calendars.
>
>
> вт, 3 нояб. 2020 г. в 11:10, Valentin Kulichenko <
> [hidden email]>:
>
> > Hi Alexey,
> >
> > The IEP-54 [1] describes the data layout proposed for Ignite 3.0, it
> > includes various date/time types. Can you please take a look and check if
> > this addresses your concerns? We can then discuss further if needed.
> >
> >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> >
> > -Val
> >
> > On Tue, Nov 3, 2020 at 12:05 AM Alexey Kukushkin <
> > [hidden email]>
> > wrote:
> >
> > > Pavel,
> > >
> > > We propose changing public API so this is for Ignite 3.0.
> > >
> > > Thanks!
> > >
> > > вт, 3 нояб. 2020 г. в 02:17, Pavel Tupitsyn <[hidden email]>:
> > >
> > > > Alexey,
> > > >
> > > > Just to clarify before we start the discussion:
> > > > this proposal seems to introduce some breaking changes, so we are
> > talking
> > > > about Ignite 3.0, correct?
> > > >
> > > > Pavel
> > > >
> > > > On Tue, Nov 3, 2020 at 12:13 AM Alexey Kukushkin <
> > > > [hidden email]>
> > > > wrote:
> > > >
> > > > > Igniters,
> > > > >
> > > > > What do you think about changing .NET API to read/write portable
> > dates
> > > by
> > > > > default and making that really portable?
> > > > >
> > > > > *The Problem*
> > > > > Presently .NET API writes dates as composite Ignite objects. Only
> > .NET
> > > > > clients can read such dates: any other client (JDBC, Java, etc)
> does
> > > not
> > > > > understand it without custom deserialization.
> > > > >
> > > > > It is still possible to configure .NET serialization to write dates
> > as
> > > > > Ignite dates - see DateTime Serialization note
> > > > > <
> > > > >
> > > >
> > >
> >
> https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility
> > > > > >.
> > > > > But then Ignite accepts only UTC dates, requiring the application
> > > > > developers to convert local dates to UTC dates and back. This task
> is
> > > not
> > > > > trivial: DateTime.ToUniversalTime
> > > > > <
> > > > >
> > > >
> > >
> >
> https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1
> > > > > >
> > > > > uses
> > > > > calendars different from Java (and the .NET calendars are the
> invalid
> > > > ones
> > > > > - especially for pre-1990 dates).
> > > > >
> > > > > The motivation for the current default behavior was probably the
> > desire
> > > > to
> > > > > keep the Time Zone information: Ignite dates do not store time
> zones.
> > > > >
> > > > > In our experience interoperability is more important than storing
> > time
> > > > zone
> > > > > info.
> > > > >
> > > > > *The Proposal*
> > > > >
> > > > >    1. Always write .NET dates as portable Ignite dates: get rid of
> > the
> > > > >    BinaryReflectiveSerializer.ForceTimestamp flag that currently
> > > triggers
> > > > > this
> > > > >    behavior.
> > > > >       - We could still keep the ForceTimestamp flag if saving .NET
> > > dates
> > > > as
> > > > >       transparent objects seems a useful case. We do not think it
> is
> > > > > useful.
> > > > >    2. Automatically convert Local dates to UTC and back *inside*
> > > > >    Ignite.NET.
> > > > >       - In this case we lose the DateTime.Kind of UTC dates: we
> > write a
> > > > UTC
> > > > >       date but we would read a Local date since Ignite would always
> > > > > convert UTC
> > > > >       to Local when reading.
> > > > >       We could add a UtcDate date flag to QuerySqlFieldAttribute
> > > > >       and BinaryReflectiveSerializer to control the deserialization
> > > > > behavior if
> > > > >       keeping dates in UTC format use case seems important.
> > > > >    3. Use NodaTime <https://nodatime.org/> for UTC<->Local
> > > conversions.
> > > > >    Noda time uses Java calendars making the conversion truely
> > portable.
> > > > >
> > > > > *The Benefits*
> > > > >
> > > > >    1. We think portable dates are much more important than storing
> > time
> > > > >    zone info. Why do we store time zones for every date on the
> server
> > > > > anyway?
> > > > >    Time zone is client-side info.
> > > > >    2. Simpler application code: no more manual configuration to
> > trigger
> > > > the
> > > > >    portable behavior.
> > > > >    3. Non-trivial code to make the truly portable UTC<->Local
> > > conversion
> > > > is
> > > > >    implemented once inside Ignite instead of having every
> Ignite.NET
> > > > >    application implementing it.
> > > > >
> > > > > Thank you!
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Alexey
> > > > >
> > > >
> > >
> > >
> > > --
> > > Best regards,
> > > Alexey
> > >
> >
>
>
> --
> Best regards,
> Alexey
>

[Alexey Kukushkin]

Val, absolutely - our proposal affects .NET API only.

вт, 3 нояб. 2020 г. в 15:00, Valentin Kulichenko <
[hidden email]>:

> Got it. So it only affects the .NET side, correct?
>
> -Val
>
> On Tue, Nov 3, 2020 at 3:52 AM Alexey Kukushkin <[hidden email]
> >
> wrote:
>
> > Hi Val,
> >
> > Our proposal does not overlap with IEP-54
> > <
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > >,
> > which proposes changing Ignite date storage format. Our proposal is
> > enhancing Ignite to handle .NET dates in a truly portable way instead of
> > requiring the application developers to repeat this task every time:
> >
> >    - Write .NET dates as Ignite dates (today .NET dates are written as
> >    generic Ignite objects)
> >    - Convert Local .NET dates to UTC inside Ignite and to it using Java
> >    calendars.
> >
> >
> > вт, 3 нояб. 2020 г. в 11:10, Valentin Kulichenko <
> > [hidden email]>:
> >
> > > Hi Alexey,
> > >
> > > The IEP-54 [1] describes the data layout proposed for Ignite 3.0, it
> > > includes various date/time types. Can you please take a look and check
> if
> > > this addresses your concerns? We can then discuss further if needed.
> > >
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > >
> > > -Val
> > >
> > > On Tue, Nov 3, 2020 at 12:05 AM Alexey Kukushkin <
> > > [hidden email]>
> > > wrote:
> > >
> > > > Pavel,
> > > >
> > > > We propose changing public API so this is for Ignite 3.0.
> > > >
> > > > Thanks!
> > > >
> > > > вт, 3 нояб. 2020 г. в 02:17, Pavel Tupitsyn <[hidden email]>:
> > > >
> > > > > Alexey,
> > > > >
> > > > > Just to clarify before we start the discussion:
> > > > > this proposal seems to introduce some breaking changes, so we are
> > > talking
> > > > > about Ignite 3.0, correct?
> > > > >
> > > > > Pavel
> > > > >
> > > > > On Tue, Nov 3, 2020 at 12:13 AM Alexey Kukushkin <
> > > > > [hidden email]>
> > > > > wrote:
> > > > >
> > > > > > Igniters,
> > > > > >
> > > > > > What do you think about changing .NET API to read/write portable
> > > dates
> > > > by
> > > > > > default and making that really portable?
> > > > > >
> > > > > > *The Problem*
> > > > > > Presently .NET API writes dates as composite Ignite objects. Only
> > > .NET
> > > > > > clients can read such dates: any other client (JDBC, Java, etc)
> > does
> > > > not
> > > > > > understand it without custom deserialization.
> > > > > >
> > > > > > It is still possible to configure .NET serialization to write
> dates
> > > as
> > > > > > Ignite dates - see DateTime Serialization note
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility
> > > > > > >.
> > > > > > But then Ignite accepts only UTC dates, requiring the application
> > > > > > developers to convert local dates to UTC dates and back. This
> task
> > is
> > > > not
> > > > > > trivial: DateTime.ToUniversalTime
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1
> > > > > > >
> > > > > > uses
> > > > > > calendars different from Java (and the .NET calendars are the
> > invalid
> > > > > ones
> > > > > > - especially for pre-1990 dates).
> > > > > >
> > > > > > The motivation for the current default behavior was probably the
> > > desire
> > > > > to
> > > > > > keep the Time Zone information: Ignite dates do not store time
> > zones.
> > > > > >
> > > > > > In our experience interoperability is more important than storing
> > > time
> > > > > zone
> > > > > > info.
> > > > > >
> > > > > > *The Proposal*
> > > > > >
> > > > > >    1. Always write .NET dates as portable Ignite dates: get rid
> of
> > > the
> > > > > >    BinaryReflectiveSerializer.ForceTimestamp flag that currently
> > > > triggers
> > > > > > this
> > > > > >    behavior.
> > > > > >       - We could still keep the ForceTimestamp flag if saving
> .NET
> > > > dates
> > > > > as
> > > > > >       transparent objects seems a useful case. We do not think it
> > is
> > > > > > useful.
> > > > > >    2. Automatically convert Local dates to UTC and back *inside*
> > > > > >    Ignite.NET.
> > > > > >       - In this case we lose the DateTime.Kind of UTC dates: we
> > > write a
> > > > > UTC
> > > > > >       date but we would read a Local date since Ignite would
> always
> > > > > > convert UTC
> > > > > >       to Local when reading.
> > > > > >       We could add a UtcDate date flag to QuerySqlFieldAttribute
> > > > > >       and BinaryReflectiveSerializer to control the
> deserialization
> > > > > > behavior if
> > > > > >       keeping dates in UTC format use case seems important.
> > > > > >    3. Use NodaTime <https://nodatime.org/> for UTC<->Local
> > > > conversions.
> > > > > >    Noda time uses Java calendars making the conversion truely
> > > portable.
> > > > > >
> > > > > > *The Benefits*
> > > > > >
> > > > > >    1. We think portable dates are much more important than
> storing
> > > time
> > > > > >    zone info. Why do we store time zones for every date on the
> > server
> > > > > > anyway?
> > > > > >    Time zone is client-side info.
> > > > > >    2. Simpler application code: no more manual configuration to
> > > trigger
> > > > > the
> > > > > >    portable behavior.
> > > > > >    3. Non-trivial code to make the truly portable UTC<->Local
> > > > conversion
> > > > > is
> > > > > >    implemented once inside Ignite instead of having every
> > Ignite.NET
> > > > > >    application implementing it.
> > > > > >
> > > > > > Thank you!
> > > > > >
> > > > > > --
> > > > > > Best regards,
> > > > > > Alexey
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Alexey
> > > >
> > >
> >
> >
> > --
> > Best regards,
> > Alexey
> >
>

--
Best regards,
Alexey

[Valentin Kulichenko]

Cool, makes sense to me then. Please feel free to create a ticket and mark
it with the "ignite-3" label.

-Val

On Tue, Nov 3, 2020 at 4:03 AM Alexey Kukushkin <[hidden email]>
wrote:

> Val, absolutely - our proposal affects .NET API only.
>
> вт, 3 нояб. 2020 г. в 15:00, Valentin Kulichenko <
> [hidden email]>:
>
> > Got it. So it only affects the .NET side, correct?
> >
> > -Val
> >
> > On Tue, Nov 3, 2020 at 3:52 AM Alexey Kukushkin <
> [hidden email]
> > >
> > wrote:
> >
> > > Hi Val,
> > >
> > > Our proposal does not overlap with IEP-54
> > > <
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > > >,
> > > which proposes changing Ignite date storage format. Our proposal is
> > > enhancing Ignite to handle .NET dates in a truly portable way instead
> of
> > > requiring the application developers to repeat this task every time:
> > >
> > >    - Write .NET dates as Ignite dates (today .NET dates are written as
> > >    generic Ignite objects)
> > >    - Convert Local .NET dates to UTC inside Ignite and to it using Java
> > >    calendars.
> > >
> > >
> > > вт, 3 нояб. 2020 г. в 11:10, Valentin Kulichenko <
> > > [hidden email]>:
> > >
> > > > Hi Alexey,
> > > >
> > > > The IEP-54 [1] describes the data layout proposed for Ignite 3.0, it
> > > > includes various date/time types. Can you please take a look and
> check
> > if
> > > > this addresses your concerns? We can then discuss further if needed.
> > > >
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > > >
> > > > -Val
> > > >
> > > > On Tue, Nov 3, 2020 at 12:05 AM Alexey Kukushkin <
> > > > [hidden email]>
> > > > wrote:
> > > >
> > > > > Pavel,
> > > > >
> > > > > We propose changing public API so this is for Ignite 3.0.
> > > > >
> > > > > Thanks!
> > > > >
> > > > > вт, 3 нояб. 2020 г. в 02:17, Pavel Tupitsyn <[hidden email]
> >:
> > > > >
> > > > > > Alexey,
> > > > > >
> > > > > > Just to clarify before we start the discussion:
> > > > > > this proposal seems to introduce some breaking changes, so we are
> > > > talking
> > > > > > about Ignite 3.0, correct?
> > > > > >
> > > > > > Pavel
> > > > > >
> > > > > > On Tue, Nov 3, 2020 at 12:13 AM Alexey Kukushkin <
> > > > > > [hidden email]>
> > > > > > wrote:
> > > > > >
> > > > > > > Igniters,
> > > > > > >
> > > > > > > What do you think about changing .NET API to read/write
> portable
> > > > dates
> > > > > by
> > > > > > > default and making that really portable?
> > > > > > >
> > > > > > > *The Problem*
> > > > > > > Presently .NET API writes dates as composite Ignite objects.
> Only
> > > > .NET
> > > > > > > clients can read such dates: any other client (JDBC, Java, etc)
> > > does
> > > > > not
> > > > > > > understand it without custom deserialization.
> > > > > > >
> > > > > > > It is still possible to configure .NET serialization to write
> > dates
> > > > as
> > > > > > > Ignite dates - see DateTime Serialization note
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility
> > > > > > > >.
> > > > > > > But then Ignite accepts only UTC dates, requiring the
> application
> > > > > > > developers to convert local dates to UTC dates and back. This
> > task
> > > is
> > > > > not
> > > > > > > trivial: DateTime.ToUniversalTime
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1
> > > > > > > >
> > > > > > > uses
> > > > > > > calendars different from Java (and the .NET calendars are the
> > > invalid
> > > > > > ones
> > > > > > > - especially for pre-1990 dates).
> > > > > > >
> > > > > > > The motivation for the current default behavior was probably
> the
> > > > desire
> > > > > > to
> > > > > > > keep the Time Zone information: Ignite dates do not store time
> > > zones.
> > > > > > >
> > > > > > > In our experience interoperability is more important than
> storing
> > > > time
> > > > > > zone
> > > > > > > info.
> > > > > > >
> > > > > > > *The Proposal*
> > > > > > >
> > > > > > >    1. Always write .NET dates as portable Ignite dates: get rid
> > of
> > > > the
> > > > > > >    BinaryReflectiveSerializer.ForceTimestamp flag that
> currently
> > > > > triggers
> > > > > > > this
> > > > > > >    behavior.
> > > > > > >       - We could still keep the ForceTimestamp flag if saving
> > .NET
> > > > > dates
> > > > > > as
> > > > > > >       transparent objects seems a useful case. We do not think
> it
> > > is
> > > > > > > useful.
> > > > > > >    2. Automatically convert Local dates to UTC and back
> *inside*
> > > > > > >    Ignite.NET.
> > > > > > >       - In this case we lose the DateTime.Kind of UTC dates: we
> > > > write a
> > > > > > UTC
> > > > > > >       date but we would read a Local date since Ignite would
> > always
> > > > > > > convert UTC
> > > > > > >       to Local when reading.
> > > > > > >       We could add a UtcDate date flag to
> QuerySqlFieldAttribute
> > > > > > >       and BinaryReflectiveSerializer to control the
> > deserialization
> > > > > > > behavior if
> > > > > > >       keeping dates in UTC format use case seems important.
> > > > > > >    3. Use NodaTime <https://nodatime.org/> for UTC<->Local
> > > > > conversions.
> > > > > > >    Noda time uses Java calendars making the conversion truely
> > > > portable.
> > > > > > >
> > > > > > > *The Benefits*
> > > > > > >
> > > > > > >    1. We think portable dates are much more important than
> > storing
> > > > time
> > > > > > >    zone info. Why do we store time zones for every date on the
> > > server
> > > > > > > anyway?
> > > > > > >    Time zone is client-side info.
> > > > > > >    2. Simpler application code: no more manual configuration to
> > > > trigger
> > > > > > the
> > > > > > >    portable behavior.
> > > > > > >    3. Non-trivial code to make the truly portable UTC<->Local
> > > > > conversion
> > > > > > is
> > > > > > >    implemented once inside Ignite instead of having every
> > > Ignite.NET
> > > > > > >    application implementing it.
> > > > > > >
> > > > > > > Thank you!
> > > > > > >
> > > > > > > --
> > > > > > > Best regards,
> > > > > > > Alexey
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Alexey
> > > > >
> > > >
> > >
> > >
> > > --
> > > Best regards,
> > > Alexey
> > >
> >
>
>
> --
> Best regards,
> Alexey
>

[Alexey Kukushkin]

We already created tickets and tested the proposed solution with our
applications (already in PROD) and an internal fork of Ignite. This
discussion is a proposal to donate this change to open source Apache Ignite
is the community accepts it:

   - IGNITE-12824 .NET: Write DateTime in interoperable format by default
   <https://issues.apache.org/jira/browse/IGNITE-12824>
   - IGNITE-12825 Serialize Java and .NET dates using same calendars
   <https://issues.apache.org/jira/browse/IGNITE-12825>


вт, 3 нояб. 2020 г. в 15:08, Valentin Kulichenko <
[hidden email]>:

> Cool, makes sense to me then. Please feel free to create a ticket and mark
> it with the "ignite-3" label.
>
> -Val
>
> On Tue, Nov 3, 2020 at 4:03 AM Alexey Kukushkin <[hidden email]
> >
> wrote:
>
> > Val, absolutely - our proposal affects .NET API only.
> >
> > вт, 3 нояб. 2020 г. в 15:00, Valentin Kulichenko <
> > [hidden email]>:
> >
> > > Got it. So it only affects the .NET side, correct?
> > >
> > > -Val
> > >
> > > On Tue, Nov 3, 2020 at 3:52 AM Alexey Kukushkin <
> > [hidden email]
> > > >
> > > wrote:
> > >
> > > > Hi Val,
> > > >
> > > > Our proposal does not overlap with IEP-54
> > > > <
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > > > >,
> > > > which proposes changing Ignite date storage format. Our proposal is
> > > > enhancing Ignite to handle .NET dates in a truly portable way instead
> > of
> > > > requiring the application developers to repeat this task every time:
> > > >
> > > >    - Write .NET dates as Ignite dates (today .NET dates are written
> as
> > > >    generic Ignite objects)
> > > >    - Convert Local .NET dates to UTC inside Ignite and to it using
> Java
> > > >    calendars.
> > > >
> > > >
> > > > вт, 3 нояб. 2020 г. в 11:10, Valentin Kulichenko <
> > > > [hidden email]>:
> > > >
> > > > > Hi Alexey,
> > > > >
> > > > > The IEP-54 [1] describes the data layout proposed for Ignite 3.0,
> it
> > > > > includes various date/time types. Can you please take a look and
> > check
> > > if
> > > > > this addresses your concerns? We can then discuss further if
> needed.
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > > > >
> > > > > -Val
> > > > >
> > > > > On Tue, Nov 3, 2020 at 12:05 AM Alexey Kukushkin <
> > > > > [hidden email]>
> > > > > wrote:
> > > > >
> > > > > > Pavel,
> > > > > >
> > > > > > We propose changing public API so this is for Ignite 3.0.
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > вт, 3 нояб. 2020 г. в 02:17, Pavel Tupitsyn <
> [hidden email]
> > >:
> > > > > >
> > > > > > > Alexey,
> > > > > > >
> > > > > > > Just to clarify before we start the discussion:
> > > > > > > this proposal seems to introduce some breaking changes, so we
> are
> > > > > talking
> > > > > > > about Ignite 3.0, correct?
> > > > > > >
> > > > > > > Pavel
> > > > > > >
> > > > > > > On Tue, Nov 3, 2020 at 12:13 AM Alexey Kukushkin <
> > > > > > > [hidden email]>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Igniters,
> > > > > > > >
> > > > > > > > What do you think about changing .NET API to read/write
> > portable
> > > > > dates
> > > > > > by
> > > > > > > > default and making that really portable?
> > > > > > > >
> > > > > > > > *The Problem*
> > > > > > > > Presently .NET API writes dates as composite Ignite objects.
> > Only
> > > > > .NET
> > > > > > > > clients can read such dates: any other client (JDBC, Java,
> etc)
> > > > does
> > > > > > not
> > > > > > > > understand it without custom deserialization.
> > > > > > > >
> > > > > > > > It is still possible to configure .NET serialization to write
> > > dates
> > > > > as
> > > > > > > > Ignite dates - see DateTime Serialization note
> > > > > > > > <
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility
> > > > > > > > >.
> > > > > > > > But then Ignite accepts only UTC dates, requiring the
> > application
> > > > > > > > developers to convert local dates to UTC dates and back. This
> > > task
> > > > is
> > > > > > not
> > > > > > > > trivial: DateTime.ToUniversalTime
> > > > > > > > <
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1
> > > > > > > > >
> > > > > > > > uses
> > > > > > > > calendars different from Java (and the .NET calendars are the
> > > > invalid
> > > > > > > ones
> > > > > > > > - especially for pre-1990 dates).
> > > > > > > >
> > > > > > > > The motivation for the current default behavior was probably
> > the
> > > > > desire
> > > > > > > to
> > > > > > > > keep the Time Zone information: Ignite dates do not store
> time
> > > > zones.
> > > > > > > >
> > > > > > > > In our experience interoperability is more important than
> > storing
> > > > > time
> > > > > > > zone
> > > > > > > > info.
> > > > > > > >
> > > > > > > > *The Proposal*
> > > > > > > >
> > > > > > > >    1. Always write .NET dates as portable Ignite dates: get
> rid
> > > of
> > > > > the
> > > > > > > >    BinaryReflectiveSerializer.ForceTimestamp flag that
> > currently
> > > > > > triggers
> > > > > > > > this
> > > > > > > >    behavior.
> > > > > > > >       - We could still keep the ForceTimestamp flag if saving
> > > .NET
> > > > > > dates
> > > > > > > as
> > > > > > > >       transparent objects seems a useful case. We do not
> think
> > it
> > > > is
> > > > > > > > useful.
> > > > > > > >    2. Automatically convert Local dates to UTC and back
> > *inside*
> > > > > > > >    Ignite.NET.
> > > > > > > >       - In this case we lose the DateTime.Kind of UTC dates:
> we
> > > > > write a
> > > > > > > UTC
> > > > > > > >       date but we would read a Local date since Ignite would
> > > always
> > > > > > > > convert UTC
> > > > > > > >       to Local when reading.
> > > > > > > >       We could add a UtcDate date flag to
> > QuerySqlFieldAttribute
> > > > > > > >       and BinaryReflectiveSerializer to control the
> > > deserialization
> > > > > > > > behavior if
> > > > > > > >       keeping dates in UTC format use case seems important.
> > > > > > > >    3. Use NodaTime <https://nodatime.org/> for UTC<->Local
> > > > > > conversions.
> > > > > > > >    Noda time uses Java calendars making the conversion truely
> > > > > portable.
> > > > > > > >
> > > > > > > > *The Benefits*
> > > > > > > >
> > > > > > > >    1. We think portable dates are much more important than
> > > storing
> > > > > time
> > > > > > > >    zone info. Why do we store time zones for every date on
> the
> > > > server
> > > > > > > > anyway?
> > > > > > > >    Time zone is client-side info.
> > > > > > > >    2. Simpler application code: no more manual configuration
> to
> > > > > trigger
> > > > > > > the
> > > > > > > >    portable behavior.
> > > > > > > >    3. Non-trivial code to make the truly portable UTC<->Local
> > > > > > conversion
> > > > > > > is
> > > > > > > >    implemented once inside Ignite instead of having every
> > > > Ignite.NET
> > > > > > > >    application implementing it.
> > > > > > > >
> > > > > > > > Thank you!
> > > > > > > >
> > > > > > > > --
> > > > > > > > Best regards,
> > > > > > > > Alexey
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Best regards,
> > > > > > Alexey
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Alexey
> > > >
> > >
> >
> >
> > --
> > Best regards,
> > Alexey
> >
>

--
Best regards,
Alexey

[Pavel Tupitsyn]

Alexey,

The proposal looks good to me.

Usually I would try to avoid extra dependencies in the core assembly,
but NodaTime seems to be worth it.

Would you like to prepare an IEP?

Thanks,
Pavel



On Tue, Nov 3, 2020 at 3:15 PM Alexey Kukushkin <[hidden email]>
wrote:

> We already created tickets and tested the proposed solution with our
> applications (already in PROD) and an internal fork of Ignite. This
> discussion is a proposal to donate this change to open source Apache Ignite
> is the community accepts it:
>
>    - IGNITE-12824 .NET: Write DateTime in interoperable format by default
>    <https://issues.apache.org/jira/browse/IGNITE-12824>
>    - IGNITE-12825 Serialize Java and .NET dates using same calendars
>    <https://issues.apache.org/jira/browse/IGNITE-12825>
>
>
> вт, 3 нояб. 2020 г. в 15:08, Valentin Kulichenko <
> [hidden email]>:
>
> > Cool, makes sense to me then. Please feel free to create a ticket and
> mark
> > it with the "ignite-3" label.
> >
> > -Val
> >
> > On Tue, Nov 3, 2020 at 4:03 AM Alexey Kukushkin <
> [hidden email]
> > >
> > wrote:
> >
> > > Val, absolutely - our proposal affects .NET API only.
> > >
> > > вт, 3 нояб. 2020 г. в 15:00, Valentin Kulichenko <
> > > [hidden email]>:
> > >
> > > > Got it. So it only affects the .NET side, correct?
> > > >
> > > > -Val
> > > >
> > > > On Tue, Nov 3, 2020 at 3:52 AM Alexey Kukushkin <
> > > [hidden email]
> > > > >
> > > > wrote:
> > > >
> > > > > Hi Val,
> > > > >
> > > > > Our proposal does not overlap with IEP-54
> > > > > <
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > > > > >,
> > > > > which proposes changing Ignite date storage format. Our proposal is
> > > > > enhancing Ignite to handle .NET dates in a truly portable way
> instead
> > > of
> > > > > requiring the application developers to repeat this task every
> time:
> > > > >
> > > > >    - Write .NET dates as Ignite dates (today .NET dates are written
> > as
> > > > >    generic Ignite objects)
> > > > >    - Convert Local .NET dates to UTC inside Ignite and to it using
> > Java
> > > > >    calendars.
> > > > >
> > > > >
> > > > > вт, 3 нояб. 2020 г. в 11:10, Valentin Kulichenko <
> > > > > [hidden email]>:
> > > > >
> > > > > > Hi Alexey,
> > > > > >
> > > > > > The IEP-54 [1] describes the data layout proposed for Ignite 3.0,
> > it
> > > > > > includes various date/time types. Can you please take a look and
> > > check
> > > > if
> > > > > > this addresses your concerns? We can then discuss further if
> > needed.
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > > > > >
> > > > > > -Val
> > > > > >
> > > > > > On Tue, Nov 3, 2020 at 12:05 AM Alexey Kukushkin <
> > > > > > [hidden email]>
> > > > > > wrote:
> > > > > >
> > > > > > > Pavel,
> > > > > > >
> > > > > > > We propose changing public API so this is for Ignite 3.0.
> > > > > > >
> > > > > > > Thanks!
> > > > > > >
> > > > > > > вт, 3 нояб. 2020 г. в 02:17, Pavel Tupitsyn <
> > [hidden email]
> > > >:
> > > > > > >
> > > > > > > > Alexey,
> > > > > > > >
> > > > > > > > Just to clarify before we start the discussion:
> > > > > > > > this proposal seems to introduce some breaking changes, so we
> > are
> > > > > > talking
> > > > > > > > about Ignite 3.0, correct?
> > > > > > > >
> > > > > > > > Pavel
> > > > > > > >
> > > > > > > > On Tue, Nov 3, 2020 at 12:13 AM Alexey Kukushkin <
> > > > > > > > [hidden email]>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Igniters,
> > > > > > > > >
> > > > > > > > > What do you think about changing .NET API to read/write
> > > portable
> > > > > > dates
> > > > > > > by
> > > > > > > > > default and making that really portable?
> > > > > > > > >
> > > > > > > > > *The Problem*
> > > > > > > > > Presently .NET API writes dates as composite Ignite
> objects.
> > > Only
> > > > > > .NET
> > > > > > > > > clients can read such dates: any other client (JDBC, Java,
> > etc)
> > > > > does
> > > > > > > not
> > > > > > > > > understand it without custom deserialization.
> > > > > > > > >
> > > > > > > > > It is still possible to configure .NET serialization to
> write
> > > > dates
> > > > > > as
> > > > > > > > > Ignite dates - see DateTime Serialization note
> > > > > > > > > <
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility
> > > > > > > > > >.
> > > > > > > > > But then Ignite accepts only UTC dates, requiring the
> > > application
> > > > > > > > > developers to convert local dates to UTC dates and back.
> This
> > > > task
> > > > > is
> > > > > > > not
> > > > > > > > > trivial: DateTime.ToUniversalTime
> > > > > > > > > <
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1
> > > > > > > > > >
> > > > > > > > > uses
> > > > > > > > > calendars different from Java (and the .NET calendars are
> the
> > > > > invalid
> > > > > > > > ones
> > > > > > > > > - especially for pre-1990 dates).
> > > > > > > > >
> > > > > > > > > The motivation for the current default behavior was
> probably
> > > the
> > > > > > desire
> > > > > > > > to
> > > > > > > > > keep the Time Zone information: Ignite dates do not store
> > time
> > > > > zones.
> > > > > > > > >
> > > > > > > > > In our experience interoperability is more important than
> > > storing
> > > > > > time
> > > > > > > > zone
> > > > > > > > > info.
> > > > > > > > >
> > > > > > > > > *The Proposal*
> > > > > > > > >
> > > > > > > > >    1. Always write .NET dates as portable Ignite dates: get
> > rid
> > > > of
> > > > > > the
> > > > > > > > >    BinaryReflectiveSerializer.ForceTimestamp flag that
> > > currently
> > > > > > > triggers
> > > > > > > > > this
> > > > > > > > >    behavior.
> > > > > > > > >       - We could still keep the ForceTimestamp flag if
> saving
> > > > .NET
> > > > > > > dates
> > > > > > > > as
> > > > > > > > >       transparent objects seems a useful case. We do not
> > think
> > > it
> > > > > is
> > > > > > > > > useful.
> > > > > > > > >    2. Automatically convert Local dates to UTC and back
> > > *inside*
> > > > > > > > >    Ignite.NET.
> > > > > > > > >       - In this case we lose the DateTime.Kind of UTC
> dates:
> > we
> > > > > > write a
> > > > > > > > UTC
> > > > > > > > >       date but we would read a Local date since Ignite
> would
> > > > always
> > > > > > > > > convert UTC
> > > > > > > > >       to Local when reading.
> > > > > > > > >       We could add a UtcDate date flag to
> > > QuerySqlFieldAttribute
> > > > > > > > >       and BinaryReflectiveSerializer to control the
> > > > deserialization
> > > > > > > > > behavior if
> > > > > > > > >       keeping dates in UTC format use case seems important.
> > > > > > > > >    3. Use NodaTime <https://nodatime.org/> for UTC<->Local
> > > > > > > conversions.
> > > > > > > > >    Noda time uses Java calendars making the conversion
> truely
> > > > > > portable.
> > > > > > > > >
> > > > > > > > > *The Benefits*
> > > > > > > > >
> > > > > > > > >    1. We think portable dates are much more important than
> > > > storing
> > > > > > time
> > > > > > > > >    zone info. Why do we store time zones for every date on
> > the
> > > > > server
> > > > > > > > > anyway?
> > > > > > > > >    Time zone is client-side info.
> > > > > > > > >    2. Simpler application code: no more manual
> configuration
> > to
> > > > > > trigger
> > > > > > > > the
> > > > > > > > >    portable behavior.
> > > > > > > > >    3. Non-trivial code to make the truly portable
> UTC<->Local
> > > > > > > conversion
> > > > > > > > is
> > > > > > > > >    implemented once inside Ignite instead of having every
> > > > > Ignite.NET
> > > > > > > > >    application implementing it.
> > > > > > > > >
> > > > > > > > > Thank you!
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Best regards,
> > > > > > > > > Alexey
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Best regards,
> > > > > > > Alexey
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Alexey
> > > > >
> > > >
> > >
> > >
> > > --
> > > Best regards,
> > > Alexey
> > >
> >
>
>
> --
> Best regards,
> Alexey
>

[Alexey Kukushkin]

Pavel,

Thank you for your positive feedback! I will create an IEP.

вт, 3 нояб. 2020 г. в 15:48, Pavel Tupitsyn <[hidden email]>:

> Alexey,
>
> The proposal looks good to me.
>
> Usually I would try to avoid extra dependencies in the core assembly,
> but NodaTime seems to be worth it.
>
> Would you like to prepare an IEP?
>
> Thanks,
> Pavel
>
>
>
> On Tue, Nov 3, 2020 at 3:15 PM Alexey Kukushkin <[hidden email]
> >
> wrote:
>
> > We already created tickets and tested the proposed solution with our
> > applications (already in PROD) and an internal fork of Ignite. This
> > discussion is a proposal to donate this change to open source Apache
> Ignite
> > is the community accepts it:
> >
> >    - IGNITE-12824 .NET: Write DateTime in interoperable format by default
> >    <https://issues.apache.org/jira/browse/IGNITE-12824>
> >    - IGNITE-12825 Serialize Java and .NET dates using same calendars
> >    <https://issues.apache.org/jira/browse/IGNITE-12825>
> >
> >
> > вт, 3 нояб. 2020 г. в 15:08, Valentin Kulichenko <
> > [hidden email]>:
> >
> > > Cool, makes sense to me then. Please feel free to create a ticket and
> > mark
> > > it with the "ignite-3" label.
> > >
> > > -Val
> > >
> > > On Tue, Nov 3, 2020 at 4:03 AM Alexey Kukushkin <
> > [hidden email]
> > > >
> > > wrote:
> > >
> > > > Val, absolutely - our proposal affects .NET API only.
> > > >
> > > > вт, 3 нояб. 2020 г. в 15:00, Valentin Kulichenko <
> > > > [hidden email]>:
> > > >
> > > > > Got it. So it only affects the .NET side, correct?
> > > > >
> > > > > -Val
> > > > >
> > > > > On Tue, Nov 3, 2020 at 3:52 AM Alexey Kukushkin <
> > > > [hidden email]
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > Hi Val,
> > > > > >
> > > > > > Our proposal does not overlap with IEP-54
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > > > > > >,
> > > > > > which proposes changing Ignite date storage format. Our proposal
> is
> > > > > > enhancing Ignite to handle .NET dates in a truly portable way
> > instead
> > > > of
> > > > > > requiring the application developers to repeat this task every
> > time:
> > > > > >
> > > > > >    - Write .NET dates as Ignite dates (today .NET dates are
> written
> > > as
> > > > > >    generic Ignite objects)
> > > > > >    - Convert Local .NET dates to UTC inside Ignite and to it
> using
> > > Java
> > > > > >    calendars.
> > > > > >
> > > > > >
> > > > > > вт, 3 нояб. 2020 г. в 11:10, Valentin Kulichenko <
> > > > > > [hidden email]>:
> > > > > >
> > > > > > > Hi Alexey,
> > > > > > >
> > > > > > > The IEP-54 [1] describes the data layout proposed for Ignite
> 3.0,
> > > it
> > > > > > > includes various date/time types. Can you please take a look
> and
> > > > check
> > > > > if
> > > > > > > this addresses your concerns? We can then discuss further if
> > > needed.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > > > > > >
> > > > > > > -Val
> > > > > > >
> > > > > > > On Tue, Nov 3, 2020 at 12:05 AM Alexey Kukushkin <
> > > > > > > [hidden email]>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Pavel,
> > > > > > > >
> > > > > > > > We propose changing public API so this is for Ignite 3.0.
> > > > > > > >
> > > > > > > > Thanks!
> > > > > > > >
> > > > > > > > вт, 3 нояб. 2020 г. в 02:17, Pavel Tupitsyn <
> > > [hidden email]
> > > > >:
> > > > > > > >
> > > > > > > > > Alexey,
> > > > > > > > >
> > > > > > > > > Just to clarify before we start the discussion:
> > > > > > > > > this proposal seems to introduce some breaking changes, so
> we
> > > are
> > > > > > > talking
> > > > > > > > > about Ignite 3.0, correct?
> > > > > > > > >
> > > > > > > > > Pavel
> > > > > > > > >
> > > > > > > > > On Tue, Nov 3, 2020 at 12:13 AM Alexey Kukushkin <
> > > > > > > > > [hidden email]>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Igniters,
> > > > > > > > > >
> > > > > > > > > > What do you think about changing .NET API to read/write
> > > > portable
> > > > > > > dates
> > > > > > > > by
> > > > > > > > > > default and making that really portable?
> > > > > > > > > >
> > > > > > > > > > *The Problem*
> > > > > > > > > > Presently .NET API writes dates as composite Ignite
> > objects.
> > > > Only
> > > > > > > .NET
> > > > > > > > > > clients can read such dates: any other client (JDBC,
> Java,
> > > etc)
> > > > > > does
> > > > > > > > not
> > > > > > > > > > understand it without custom deserialization.
> > > > > > > > > >
> > > > > > > > > > It is still possible to configure .NET serialization to
> > write
> > > > > dates
> > > > > > > as
> > > > > > > > > > Ignite dates - see DateTime Serialization note
> > > > > > > > > > <
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility
> > > > > > > > > > >.
> > > > > > > > > > But then Ignite accepts only UTC dates, requiring the
> > > > application
> > > > > > > > > > developers to convert local dates to UTC dates and back.
> > This
> > > > > task
> > > > > > is
> > > > > > > > not
> > > > > > > > > > trivial: DateTime.ToUniversalTime
> > > > > > > > > > <
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1
> > > > > > > > > > >
> > > > > > > > > > uses
> > > > > > > > > > calendars different from Java (and the .NET calendars are
> > the
> > > > > > invalid
> > > > > > > > > ones
> > > > > > > > > > - especially for pre-1990 dates).
> > > > > > > > > >
> > > > > > > > > > The motivation for the current default behavior was
> > probably
> > > > the
> > > > > > > desire
> > > > > > > > > to
> > > > > > > > > > keep the Time Zone information: Ignite dates do not store
> > > time
> > > > > > zones.
> > > > > > > > > >
> > > > > > > > > > In our experience interoperability is more important than
> > > > storing
> > > > > > > time
> > > > > > > > > zone
> > > > > > > > > > info.
> > > > > > > > > >
> > > > > > > > > > *The Proposal*
> > > > > > > > > >
> > > > > > > > > >    1. Always write .NET dates as portable Ignite dates:
> get
> > > rid
> > > > > of
> > > > > > > the
> > > > > > > > > >    BinaryReflectiveSerializer.ForceTimestamp flag that
> > > > currently
> > > > > > > > triggers
> > > > > > > > > > this
> > > > > > > > > >    behavior.
> > > > > > > > > >       - We could still keep the ForceTimestamp flag if
> > saving
> > > > > .NET
> > > > > > > > dates
> > > > > > > > > as
> > > > > > > > > >       transparent objects seems a useful case. We do not
> > > think
> > > > it
> > > > > > is
> > > > > > > > > > useful.
> > > > > > > > > >    2. Automatically convert Local dates to UTC and back
> > > > *inside*
> > > > > > > > > >    Ignite.NET.
> > > > > > > > > >       - In this case we lose the DateTime.Kind of UTC
> > dates:
> > > we
> > > > > > > write a
> > > > > > > > > UTC
> > > > > > > > > >       date but we would read a Local date since Ignite
> > would
> > > > > always
> > > > > > > > > > convert UTC
> > > > > > > > > >       to Local when reading.
> > > > > > > > > >       We could add a UtcDate date flag to
> > > > QuerySqlFieldAttribute
> > > > > > > > > >       and BinaryReflectiveSerializer to control the
> > > > > deserialization
> > > > > > > > > > behavior if
> > > > > > > > > >       keeping dates in UTC format use case seems
> important.
> > > > > > > > > >    3. Use NodaTime <https://nodatime.org/> for
> UTC<->Local
> > > > > > > > conversions.
> > > > > > > > > >    Noda time uses Java calendars making the conversion
> > truely
> > > > > > > portable.
> > > > > > > > > >
> > > > > > > > > > *The Benefits*
> > > > > > > > > >
> > > > > > > > > >    1. We think portable dates are much more important
> than
> > > > > storing
> > > > > > > time
> > > > > > > > > >    zone info. Why do we store time zones for every date
> on
> > > the
> > > > > > server
> > > > > > > > > > anyway?
> > > > > > > > > >    Time zone is client-side info.
> > > > > > > > > >    2. Simpler application code: no more manual
> > configuration
> > > to
> > > > > > > trigger
> > > > > > > > > the
> > > > > > > > > >    portable behavior.
> > > > > > > > > >    3. Non-trivial code to make the truly portable
> > UTC<->Local
> > > > > > > > conversion
> > > > > > > > > is
> > > > > > > > > >    implemented once inside Ignite instead of having every
> > > > > > Ignite.NET
> > > > > > > > > >    application implementing it.
> > > > > > > > > >
> > > > > > > > > > Thank you!
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > Best regards,
> > > > > > > > > > Alexey
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Best regards,
> > > > > > > > Alexey
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Best regards,
> > > > > > Alexey
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Alexey
> > > >
> > >
> >
> >
> > --
> > Best regards,
> > Alexey
> >
>

--
Best regards,
Alexey