Binary compatibility of persistent storage

[Vladimir Ozerov]

igniters,

Ignite doesn't have compatibility for binary protocols between different
versions, as this would make development harder and slower. On the other
hand we maintain API compatibility what helps us move users to new versions
faster.

As native persistence is implemented, new challenge appeared - whether to
maintain binary compatibility of stored data. Many approaches exist:

1) No compatibility at all - easy for us, nightmare for users (IMO)
2) No compatibility, but provide migration instruments
3) Maintain compatibility between N latest minor versions
4) Maintain compatibility between all versions within major release

The more guarantees we offer, the harder them to maintain, the better UX.

Let's think on what compatibility mode we can offer to our users if any.
Any ideas?

Vladimir.

[Anton Vinogradov-2]

Vote for case #4.

As far as I know, Vyacheslav Daradur already works on framework allows to
check compatibility between different version of Ignite.

Vyacheslav,
Could you provide us more details?

On Tue, Sep 19, 2017 at 2:16 PM, Vladimir Ozerov <[hidden email]>
wrote:

> igniters,
>
> Ignite doesn't have compatibility for binary protocols between different
> versions, as this would make development harder and slower. On the other
> hand we maintain API compatibility what helps us move users to new versions
> faster.
>
> As native persistence is implemented, new challenge appeared - whether to
> maintain binary compatibility of stored data. Many approaches exist:
>
> 1) No compatibility at all - easy for us, nightmare for users (IMO)
> 2) No compatibility, but provide migration instruments
> 3) Maintain compatibility between N latest minor versions
> 4) Maintain compatibility between all versions within major release
>
> The more guarantees we offer, the harder them to maintain, the better UX.
>
> Let's think on what compatibility mode we can offer to our users if any.
> Any ideas?
>
> Vladimir.
>

[Nikolay Izhikov]

Hello, Vladimir.

I prefer 2 option.

 > 2) No compatibility, but provide migration instruments

I think we have to provide some tool for converting WAL files from any
older format to current.

19.09.2017 14:16, Vladimir Ozerov пишет:

> igniters,
>
> Ignite doesn't have compatibility for binary protocols between different
> versions, as this would make development harder and slower. On the other
> hand we maintain API compatibility what helps us move users to new versions
> faster.
>
> As native persistence is implemented, new challenge appeared - whether to
> maintain binary compatibility of stored data. Many approaches exist:
>
> 1) No compatibility at all - easy for us, nightmare for users (IMO)
> 2) No compatibility, but provide migration instruments
> 3) Maintain compatibility between N latest minor versions
> 4) Maintain compatibility between all versions within major release
>
> The more guarantees we offer, the harder them to maintain, the better UX.
>
> Let's think on what compatibility mode we can offer to our users if any.
> Any ideas?
>
> Vladimir.
>

[yzhdanov]

Vladimir,

Can you please describe the situation when 2 is possible, but 4 is not?

--Yakov

[daradurvs]

I vote for: “4) Maintain compatibility between all versions within major
release”.
I think this is a trade-off between the complexity of implementing new
features and UX.

We will be able to get rid of all legacy tools every major release.

I’m working on a testing framework, which helps us testing compatibility
features between different Ignite version.
Testing in the early stages will help us to consider the impact of changes
in other releases and to be closer to the end user.

Here is a dummy unit-test example:

void testNodeStartByOldVersionPersistenceData() throws Exception {
    try {
        startGrid(1, "2.1.0", new PostConfigurationClosure(), new
PostActionClosure());

        stopAllGrids(); // Stopping 2.1.0

        IgniteEx ignite = startGrid(0); // Starting current version

        ignite.active(true);

        IgniteCache<Integer, String> cache =
ignite.getOrCreateCache(TEST_CACHE_NAME);

        for (int i = 0; i < 10; i++)
            assertEquals("data" + i, cache.get(i));
    }
    finally {
        stopAllGrids();
    }
}

class PostActionClosure implements IgniteInClosure<Ignite> {
    @Override public void apply(Ignite ignite) {
        ignite.active(true);

        CacheConfiguration<Integer, String> cacheCfg = new
CacheConfiguration<>();

        IgniteCache<Integer, String> cache = ignite.createCache(cacheCfg);

        for (int i = 0; i < 10; i++)
            cache.put(i, "data" + i);
    }
}

class PostConfigurationClosure implements
IgniteInClosure<IgniteConfiguration> {
    @Override public void apply(IgniteConfiguration cfg) {
        // Post configuration actions
        cfg.setPersistentStoreConfiguration(new
PersistentStoreConfiguration());
    }
}

[1] https://issues.apache.org/jira/browse/IGNITE-5732 - Provide API to test
compatibility with old releases

On Tue, Sep 19, 2017 at 2:51 PM, Yakov Zhdanov <[hidden email]> wrote:

> Vladimir,
>
> Can you please describe the situation when 2 is possible, but 4 is not?
>
> --Yakov
>



--
Best Regards, Vyacheslav D.

[Dmitriy Pavlov]

I have same idea about testing of WAL compatibilty between releases. It
will be actual since 2.3 release (as 2.2 was emergency release without
impact changes).

I look forward to IGNITE-5732 completion and merge. And then I able to
start writting first test.

I think it is quite unexpected for user if Ignite is not able to start from
existing database after updating to next release. That is why I vote for
backward compatibilty with automatic transparent migration (if needed).

вт, 19 сент. 2017 г. в 15:16, Vyacheslav Daradur <[hidden email]>:

> I vote for: “4) Maintain compatibility between all versions within major
> release”.
> I think this is a trade-off between the complexity of implementing new
> features and UX.
>
> We will be able to get rid of all legacy tools every major release.
>
> I’m working on a testing framework, which helps us testing compatibility
> features between different Ignite version.
> Testing in the early stages will help us to consider the impact of changes
> in other releases and to be closer to the end user.
>
> Here is a dummy unit-test example:
>
> void testNodeStartByOldVersionPersistenceData() throws Exception {
>     try {
>         startGrid(1, "2.1.0", new PostConfigurationClosure(), new
> PostActionClosure());
>
>         stopAllGrids(); // Stopping 2.1.0
>
>         IgniteEx ignite = startGrid(0); // Starting current version
>
>         ignite.active(true);
>
>         IgniteCache<Integer, String> cache =
> ignite.getOrCreateCache(TEST_CACHE_NAME);
>
>         for (int i = 0; i < 10; i++)
>             assertEquals("data" + i, cache.get(i));
>     }
>     finally {
>         stopAllGrids();
>     }
> }
>
> class PostActionClosure implements IgniteInClosure<Ignite> {
>     @Override public void apply(Ignite ignite) {
>         ignite.active(true);
>
>         CacheConfiguration<Integer, String> cacheCfg = new
> CacheConfiguration<>();
>
>         IgniteCache<Integer, String> cache = ignite.createCache(cacheCfg);
>
>         for (int i = 0; i < 10; i++)
>             cache.put(i, "data" + i);
>     }
> }
>
> class PostConfigurationClosure implements
> IgniteInClosure<IgniteConfiguration> {
>     @Override public void apply(IgniteConfiguration cfg) {
>         // Post configuration actions
>         cfg.setPersistentStoreConfiguration(new
> PersistentStoreConfiguration());
>     }
> }
>
> [1] https://issues.apache.org/jira/browse/IGNITE-5732 - Provide API to
> test
> compatibility with old releases
>
> On Tue, Sep 19, 2017 at 2:51 PM, Yakov Zhdanov <[hidden email]>
> wrote:
>
> > Vladimir,
> >
> > Can you please describe the situation when 2 is possible, but 4 is not?
> >
> > --Yakov
> >
>
>
>
> --
> Best Regards, Vyacheslav D.
>

[Vladimir Ozerov]

Yakov,

Any major change in data/index page format. E.g. this could happen once
transactional SQL is ready.

On Tue, Sep 19, 2017 at 2:51 PM, Yakov Zhdanov <[hidden email]> wrote:

> Vladimir,
>
> Can you please describe the situation when 2 is possible, but 4 is not?
>
> --Yakov
>

[alexzaitzev]

I vote for the 4th variant, because it is the most common approach to the versioning.

For example, SemVer says that you can make incompatible API changes only in major versions.

19.09.2017, 14:52, "Vladimir Ozerov" <[hidden email]>:

> Yakov,
>
> Any major change in data/index page format. E.g. this could happen once
> transactional SQL is ready.
>
> On Tue, Sep 19, 2017 at 2:51 PM, Yakov Zhdanov <[hidden email]> wrote:
>
>>  Vladimir,
>>
>>  Can you please describe the situation when 2 is possible, but 4 is not?
>>
>>  --Yakov

[yzhdanov]

>Any major change in data/index page format. E.g. this could happen once
transactional SQL is ready.

I would suggest we automatically disable this feature for databases created
with older versions.

--Yakov

[Valentin Kulichenko]

In my view, there are two different scenarios.

First - user just upgrades the version (to get some bug fix, for example),
but does not intend to change anything in their project and/or use any new
features. In this case it should work transparently and cluster must be
able to work with older format. Ideally, we should detect this
automatically, but if it's not possible we can introduce some kind of
'compatibility mode' enabled by a system property.

Second - user upgrades to get new features that require data format change.
In this case, I think it's OK to suggest using a conversion tool. Or
probably we can apply it automatically on node startup?

-Val

On Tue, Sep 19, 2017 at 6:38 AM, Yakov Zhdanov <[hidden email]> wrote:

> >Any major change in data/index page format. E.g. this could happen once
> transactional SQL is ready.
>
> I would suggest we automatically disable this feature for databases created
> with older versions.
>
> --Yakov
>

[daradurvs]

Igniters,

The compatibility testing framework [1] will be completed soon.

We will be able to cover all the cases suggested by Vladimir, for example:

>> 2) No compatibility, but provide migration instruments
It will be possible to test migration tools and scenarios provided to the
end users.
Some usual steps:
    * start(2.2), putAll(data), stop(2.2)  - working with previous versions
of Ignite cluster;
    * startMigration(...) - migration using provided tools;
    * start(curVer), checkAll(data) - data validation;

>> 3) Maintain compatibility between N latest minor versions
Every change triggers compatibility test update, so, the latest version
will be compatible with a previous format. It will be possible to perform
cascade update (eg. 2.2 -> 2.6 -> 2.9):
    * changes in 2.3 -> add unit tests to check compatibility with 2.2;
    * changes in 2.7 -> rewrite previous unit tests to check compatibility
with 2.6;
    * changes in 2.9 -> rewrite previous unit tests to check compatibility
with 2.8;
    * etc.;
    * major release -> delete all compatibility tests, for example, for the
version 3.0;

>> 4) Maintain compatibility between all versions within major release
Every change triggers compatibility test creation, so, the latest version
will be compatible with every previous within major release:
    * changes in 2.3 -> add compatibility unit tests with 2.2;
    * changes in 2.7 -> add compatibility unit tests with 2.6;
    * changes in 2.9 -> add compatibility unit tests with 2.8;
    * etc.;
    * major release -> delete all compatibility tests, for example, for the
version 3.0;

[1] https://issues.apache.org/jira/browse/IGNITE-5732 - Provide API to test
compatibility with old releases


On Tue, Sep 19, 2017 at 8:06 PM, Valentin Kulichenko <
[hidden email]> wrote:

> In my view, there are two different scenarios.
>
> First - user just upgrades the version (to get some bug fix, for example),
> but does not intend to change anything in their project and/or use any new
> features. In this case it should work transparently and cluster must be
> able to work with older format. Ideally, we should detect this
> automatically, but if it's not possible we can introduce some kind of
> 'compatibility mode' enabled by a system property.
>
> Second - user upgrades to get new features that require data format change.
> In this case, I think it's OK to suggest using a conversion tool. Or
> probably we can apply it automatically on node startup?
>
> -Val
>
> On Tue, Sep 19, 2017 at 6:38 AM, Yakov Zhdanov <[hidden email]>
> wrote:
>
> > >Any major change in data/index page format. E.g. this could happen once
> > transactional SQL is ready.
> >
> > I would suggest we automatically disable this feature for databases
> created
> > with older versions.
> >
> > --Yakov
> >
>

--
Best Regards, Vyacheslav D.

[dmagda]

Either 3 or 4 looks as an appropriate approach for me. Ignite is no longer just an in-memory storage and we can not afford to force our users to migrate the data or configuration just because of the new cool feature in a new version. We should provide the same level of compatibility as RDBMS vendors do.

—
Denis

> On Sep 19, 2017, at 4:16 AM, Vladimir Ozerov <[hidden email]> wrote:
>
> igniters,
>
> Ignite doesn't have compatibility for binary protocols between different
> versions, as this would make development harder and slower. On the other
> hand we maintain API compatibility what helps us move users to new versions
> faster.
>
> As native persistence is implemented, new challenge appeared - whether to
> maintain binary compatibility of stored data. Many approaches exist:
>
> 1) No compatibility at all - easy for us, nightmare for users (IMO)
> 2) No compatibility, but provide migration instruments
> 3) Maintain compatibility between N latest minor versions
> 4) Maintain compatibility between all versions within major release
>
> The more guarantees we offer, the harder them to maintain, the better UX.
>
> Let's think on what compatibility mode we can offer to our users if any.
> Any ideas?
>
> Vladimir.

[Dmitriy Pavlov]

Denis, the argument sounds convincing to me. When you use 3rd party DB, you
rarely have to care how to migrate DB data.

Igniters, I suggest keeping compatibility as long as we can, even with the
transition to a new major release. If incompatibility will become
unavoidable we will consider migration procedures.

чт, 21 сент. 2017 г. в 0:21, Denis Magda <[hidden email]>:

> Either 3 or 4 looks as an appropriate approach for me. Ignite is no longer
> just an in-memory storage and we can not afford to force our users to
> migrate the data or configuration just because of the new cool feature in a
> new version. We should provide the same level of compatibility as RDBMS
> vendors do.
>
> —
> Denis
>
> > On Sep 19, 2017, at 4:16 AM, Vladimir Ozerov <[hidden email]>
> wrote:
> >
> > igniters,
> >
> > Ignite doesn't have compatibility for binary protocols between different
> > versions, as this would make development harder and slower. On the other
> > hand we maintain API compatibility what helps us move users to new
> versions
> > faster.
> >
> > As native persistence is implemented, new challenge appeared - whether to
> > maintain binary compatibility of stored data. Many approaches exist:
> >
> > 1) No compatibility at all - easy for us, nightmare for users (IMO)
> > 2) No compatibility, but provide migration instruments
> > 3) Maintain compatibility between N latest minor versions
> > 4) Maintain compatibility between all versions within major release
> >
> > The more guarantees we offer, the harder them to maintain, the better UX.
> >
> > Let's think on what compatibility mode we can offer to our users if any.
> > Any ideas?
> >
> > Vladimir.
>
>

[Dmitriy Pavlov]

Hi Guys,


I want to share the first experience of using the utility for compatibility
testing.


As part of the task to improve storage folders in PDS, I wrote 2 tests
(IGNITE 6285, FoldersReuseCompatibilityTest). These tests run versions 2.1
and 2.2, create storage, and then run the grid on new folders. It's very
easy to create such a test based on DummyPersistenceTest.


Since the test runs a grid on a separate JVM, it is always important to
call stopAllGrids ()


I want to thank the Vyacheslav D. for this improvement.


Tests are performed in the Compatiblity suite. The suite is run in the
chain RunAll, and daily for master.


Sincerely,

Dmitriy Pavlov

чт, 21 сент. 2017 г. в 0:39, Dmitry Pavlov <[hidden email]>:

> Denis, the argument sounds convincing to me. When you use 3rd party DB,
> you rarely have to care how to migrate DB data.
>
> Igniters, I suggest keeping compatibility as long as we can, even with the
> transition to a new major release. If incompatibility will become
> unavoidable we will consider migration procedures.
>
> чт, 21 сент. 2017 г. в 0:21, Denis Magda <[hidden email]>:
>
>> Either 3 or 4 looks as an appropriate approach for me. Ignite is no
>> longer just an in-memory storage and we can not afford to force our users
>> to migrate the data or configuration just because of the new cool feature
>> in a new version. We should provide the same level of compatibility as
>> RDBMS vendors do.
>>
>> —
>> Denis
>>
>> > On Sep 19, 2017, at 4:16 AM, Vladimir Ozerov <[hidden email]>
>> wrote:
>> >
>> > igniters,
>> >
>> > Ignite doesn't have compatibility for binary protocols between different
>> > versions, as this would make development harder and slower. On the other
>> > hand we maintain API compatibility what helps us move users to new
>> versions
>> > faster.
>> >
>> > As native persistence is implemented, new challenge appeared - whether
>> to
>> > maintain binary compatibility of stored data. Many approaches exist:
>> >
>> > 1) No compatibility at all - easy for us, nightmare for users (IMO)
>> > 2) No compatibility, but provide migration instruments
>> > 3) Maintain compatibility between N latest minor versions
>> > 4) Maintain compatibility between all versions within major release
>> >
>> > The more guarantees we offer, the harder them to maintain, the better
>> UX.
>> >
>> > Let's think on what compatibility mode we can offer to our users if any.
>> > Any ideas?
>> >
>> > Vladimir.
>>
>>