New API for changing configuration of persistent caches

classic Classic list List threaded Threaded
32 messages Options
12
Reply | Threaded
Open this post in threaded view
|

New API for changing configuration of persistent caches

Eduard Shangareev
Igniters,

I propose new public API to change the cache configuration of persistent
caches with keeping data.

It would look like:

Ignite ignite = ...;
ignite.restartCaches(cfg1, ... cfgN);

where cfgX is a new cache configuration, which contains the name of an
existing persistent cache.

The obvious limitation:
- affinity key mapping couldn't be changed;
- count of partitions couldn't be changed;
- MVCC couldn't be turned off/on;
- persistent couldn't be turned off;
- group settings couldn't be changed (group name);
- if cache belongs to group it's needed to restart all of them.

Failure scenario is the crucial thing (and most difficult):
- initiator fail;
- cluster restart at any stage;
- joining/starting offline nodes.

Some thoughts about implementation:
- stop cache with destroy=false;
- start cache dynamically with new configuration;
- if indexes settings changed - remove index.bin to start indexation;
- change blt-history when start cache initiated to not allow join nodes
with old configuration;
- use restartId (IGNITE-8911) to not allow to start cache in between.

Your thoughts? Would it be a useful feature?
Reply | Threaded
Open this post in threaded view
|

Re: New API for changing configuration of persistent caches

Vladimir Ozerov
Hi Ed,

Several questions from my side:
1) If we do not allow to change the most demanded by users things like
affinity or persistence/in-memory, then what kind of configuration
properties do we expect to be changed? Can we have several examples?
2) How will it interact with PME?
3) How will it interact with CREATE INDEX and ALTER TABLE commands?

On Wed, Nov 21, 2018 at 4:48 PM Eduard Shangareev <
[hidden email]> wrote:

> Igniters,
>
> I propose new public API to change the cache configuration of persistent
> caches with keeping data.
>
> It would look like:
>
> Ignite ignite = ...;
> ignite.restartCaches(cfg1, ... cfgN);
>
> where cfgX is a new cache configuration, which contains the name of an
> existing persistent cache.
>
> The obvious limitation:
> - affinity key mapping couldn't be changed;
> - count of partitions couldn't be changed;
> - MVCC couldn't be turned off/on;
> - persistent couldn't be turned off;
> - group settings couldn't be changed (group name);
> - if cache belongs to group it's needed to restart all of them.
>
> Failure scenario is the crucial thing (and most difficult):
> - initiator fail;
> - cluster restart at any stage;
> - joining/starting offline nodes.
>
> Some thoughts about implementation:
> - stop cache with destroy=false;
> - start cache dynamically with new configuration;
> - if indexes settings changed - remove index.bin to start indexation;
> - change blt-history when start cache initiated to not allow join nodes
> with old configuration;
> - use restartId (IGNITE-8911) to not allow to start cache in between.
>
> Your thoughts? Would it be a useful feature?
>
Reply | Threaded
Open this post in threaded view
|

Re: New API for changing configuration of persistent caches

Dmitry Pavlov
In reply to this post by Eduard Shangareev
Hi Eduard,

Can we just change cache parameters after calling cache() or
getOrCreateCache() by user? The method is blocked until restart finished.

We can set up new cache parameters according to the provided configuration.
If it is not possible to change some parameter, a method will fail.

I guess it would be the easiest way for users. So do we need a new method?

Sincerely,
Dmitriy Pavlov

ср, 21 нояб. 2018 г. в 16:48, Eduard Shangareev <[hidden email]
>:

> Igniters,
>
> I propose new public API to change the cache configuration of persistent
> caches with keeping data.
>
> It would look like:
>
> Ignite ignite = ...;
> ignite.restartCaches(cfg1, ... cfgN);
>
> where cfgX is a new cache configuration, which contains the name of an
> existing persistent cache.
>
> The obvious limitation:
> - affinity key mapping couldn't be changed;
> - count of partitions couldn't be changed;
> - MVCC couldn't be turned off/on;
> - persistent couldn't be turned off;
> - group settings couldn't be changed (group name);
> - if cache belongs to group it's needed to restart all of them.
>
> Failure scenario is the crucial thing (and most difficult):
> - initiator fail;
> - cluster restart at any stage;
> - joining/starting offline nodes.
>
> Some thoughts about implementation:
> - stop cache with destroy=false;
> - start cache dynamically with new configuration;
> - if indexes settings changed - remove index.bin to start indexation;
> - change blt-history when start cache initiated to not allow join nodes
> with old configuration;
> - use restartId (IGNITE-8911) to not allow to start cache in between.
>
> Your thoughts? Would it be a useful feature?
>
Reply | Threaded
Open this post in threaded view
|

Re: New API for changing configuration of persistent caches

Eduard Shangareev-2
In reply to this post by Vladimir Ozerov
Vladimir,

1) Affinity could be changed, but count of partition couldn't be.
2) So it would trigger 2 PME. Dynamic start and stop.
3) In theory, should cancel them and new setting should be applied. How it
works now? Create an index and stop node, for example.

On Wed, Nov 21, 2018 at 4:56 PM Vladimir Ozerov <[hidden email]>
wrote:

> Hi Ed,
>
> Several questions from my side:
> 1) If we do not allow to change the most demanded by users things like
> affinity or persistence/in-memory, then what kind of configuration
> properties do we expect to be changed? Can we have several examples?
> 2) How will it interact with PME?
> 3) How will it interact with CREATE INDEX and ALTER TABLE commands?
>
> On Wed, Nov 21, 2018 at 4:48 PM Eduard Shangareev <
> [hidden email]> wrote:
>
> > Igniters,
> >
> > I propose new public API to change the cache configuration of persistent
> > caches with keeping data.
> >
> > It would look like:
> >
> > Ignite ignite = ...;
> > ignite.restartCaches(cfg1, ... cfgN);
> >
> > where cfgX is a new cache configuration, which contains the name of an
> > existing persistent cache.
> >
> > The obvious limitation:
> > - affinity key mapping couldn't be changed;
> > - count of partitions couldn't be changed;
> > - MVCC couldn't be turned off/on;
> > - persistent couldn't be turned off;
> > - group settings couldn't be changed (group name);
> > - if cache belongs to group it's needed to restart all of them.
> >
> > Failure scenario is the crucial thing (and most difficult):
> > - initiator fail;
> > - cluster restart at any stage;
> > - joining/starting offline nodes.
> >
> > Some thoughts about implementation:
> > - stop cache with destroy=false;
> > - start cache dynamically with new configuration;
> > - if indexes settings changed - remove index.bin to start indexation;
> > - change blt-history when start cache initiated to not allow join nodes
> > with old configuration;
> > - use restartId (IGNITE-8911) to not allow to start cache in between.
> >
> > Your thoughts? Would it be a useful feature?
> >
>


--
Best regards,
Eduard.
Reply | Threaded
Open this post in threaded view
|

Re: New API for changing configuration of persistent caches

Eduard Shangareev
Dmitriy,

From my point of view, it's not good to have such unexpected behavior (and
change old one) for cache and getOrCreateCache method.
It should be a conscious decision, not an accident.

On Wed, Nov 21, 2018 at 5:01 PM Eduard Shangareev <[hidden email]>
wrote:

> Vladimir,
>
> 1) Affinity could be changed, but count of partition couldn't be.
> 2) So it would trigger 2 PME. Dynamic start and stop.
> 3) In theory, should cancel them and new setting should be applied. How it
> works now? Create an index and stop node, for example.
>
> On Wed, Nov 21, 2018 at 4:56 PM Vladimir Ozerov <[hidden email]>
> wrote:
>
> > Hi Ed,
> >
> > Several questions from my side:
> > 1) If we do not allow to change the most demanded by users things like
> > affinity or persistence/in-memory, then what kind of configuration
> > properties do we expect to be changed? Can we have several examples?
> > 2) How will it interact with PME?
> > 3) How will it interact with CREATE INDEX and ALTER TABLE commands?
> >
> > On Wed, Nov 21, 2018 at 4:48 PM Eduard Shangareev <
> > [hidden email]> wrote:
> >
> > > Igniters,
> > >
> > > I propose new public API to change the cache configuration of
> persistent
> > > caches with keeping data.
> > >
> > > It would look like:
> > >
> > > Ignite ignite = ...;
> > > ignite.restartCaches(cfg1, ... cfgN);
> > >
> > > where cfgX is a new cache configuration, which contains the name of an
> > > existing persistent cache.
> > >
> > > The obvious limitation:
> > > - affinity key mapping couldn't be changed;
> > > - count of partitions couldn't be changed;
> > > - MVCC couldn't be turned off/on;
> > > - persistent couldn't be turned off;
> > > - group settings couldn't be changed (group name);
> > > - if cache belongs to group it's needed to restart all of them.
> > >
> > > Failure scenario is the crucial thing (and most difficult):
> > > - initiator fail;
> > > - cluster restart at any stage;
> > > - joining/starting offline nodes.
> > >
> > > Some thoughts about implementation:
> > > - stop cache with destroy=false;
> > > - start cache dynamically with new configuration;
> > > - if indexes settings changed - remove index.bin to start indexation;
> > > - change blt-history when start cache initiated to not allow join nodes
> > > with old configuration;
> > > - use restartId (IGNITE-8911) to not allow to start cache in between.
> > >
> > > Your thoughts? Would it be a useful feature?
> > >
> >
>
>
> --
> Best regards,
> Eduard.
>
Reply | Threaded
Open this post in threaded view
|

Re: New API for changing configuration of persistent caches

Dmitry Pavlov
Sure. thanks for the reply. I think in this case we should mention a new
method in all error messages like 'cache create failed because of different
configs'.

So a user can easily find a new method to apply. So product will point
itself to a method how to get the evolution of cache configs without
re-loading all data.

ср, 21 нояб. 2018 г. в 17:05, Eduard Shangareev <[hidden email]
>:

> Dmitriy,
>
> From my point of view, it's not good to have such unexpected behavior (and
> change old one) for cache and getOrCreateCache method.
> It should be a conscious decision, not an accident.
>
> On Wed, Nov 21, 2018 at 5:01 PM Eduard Shangareev <
> [hidden email]>
> wrote:
>
> > Vladimir,
> >
> > 1) Affinity could be changed, but count of partition couldn't be.
> > 2) So it would trigger 2 PME. Dynamic start and stop.
> > 3) In theory, should cancel them and new setting should be applied. How
> it
> > works now? Create an index and stop node, for example.
> >
> > On Wed, Nov 21, 2018 at 4:56 PM Vladimir Ozerov <[hidden email]>
> > wrote:
> >
> > > Hi Ed,
> > >
> > > Several questions from my side:
> > > 1) If we do not allow to change the most demanded by users things like
> > > affinity or persistence/in-memory, then what kind of configuration
> > > properties do we expect to be changed? Can we have several examples?
> > > 2) How will it interact with PME?
> > > 3) How will it interact with CREATE INDEX and ALTER TABLE commands?
> > >
> > > On Wed, Nov 21, 2018 at 4:48 PM Eduard Shangareev <
> > > [hidden email]> wrote:
> > >
> > > > Igniters,
> > > >
> > > > I propose new public API to change the cache configuration of
> > persistent
> > > > caches with keeping data.
> > > >
> > > > It would look like:
> > > >
> > > > Ignite ignite = ...;
> > > > ignite.restartCaches(cfg1, ... cfgN);
> > > >
> > > > where cfgX is a new cache configuration, which contains the name of
> an
> > > > existing persistent cache.
> > > >
> > > > The obvious limitation:
> > > > - affinity key mapping couldn't be changed;
> > > > - count of partitions couldn't be changed;
> > > > - MVCC couldn't be turned off/on;
> > > > - persistent couldn't be turned off;
> > > > - group settings couldn't be changed (group name);
> > > > - if cache belongs to group it's needed to restart all of them.
> > > >
> > > > Failure scenario is the crucial thing (and most difficult):
> > > > - initiator fail;
> > > > - cluster restart at any stage;
> > > > - joining/starting offline nodes.
> > > >
> > > > Some thoughts about implementation:
> > > > - stop cache with destroy=false;
> > > > - start cache dynamically with new configuration;
> > > > - if indexes settings changed - remove index.bin to start indexation;
> > > > - change blt-history when start cache initiated to not allow join
> nodes
> > > > with old configuration;
> > > > - use restartId (IGNITE-8911) to not allow to start cache in between.
> > > >
> > > > Your thoughts? Would it be a useful feature?
> > > >
> > >
> >
> >
> > --
> > Best regards,
> > Eduard.
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: New API for changing configuration of persistent caches

Vladimir Ozerov
In reply to this post by Eduard Shangareev-2
Eduard,

Got it. Please take the following things in count during design:

1) Two distinct PMEs might not work well. Consider a situation w1hen I
decided to move a cache with index "MY_INDEX" from schema A to schema B.
While cache was stopped, another cache with index "MY_INDEX" was created in
schema B. Now first cache cannot start due to index name conflict.
2) Cancelling index creation is also bad idea because this is potentially
long operation. Instead, most likely that we should wait to concurrent
schema operations to finish first. That is, all operations on cache should
be ordered wrt each other somehow
3) Why do we think that cache restart will be needed at all? We have a lot
of configuration properties which could be changed safely either without
PME or with a single PME. - rebalance properties, cache store properties
(especially write-behind stuff), some query properties (e.g. "detail
metrics"), etc.. In essence, it seems that >50% of properties could be
changed without cache restart, other 25% will not be supported, and the
rest may require restart.
4) Client nodes and thin client may not have necessary classes in
classpath. E.g. consider a user which want to change rebalance timeout for
cache, but do not have configured interceptor in classpath. With proposed
API it will be impossible. This is especially true for non-Java clients.

That said, I think we should consider another API which will not require
full CacheConfiguration object. This might be a kind of builder or so. And
once user set properties he want to change to the builder, we can analyze
them and either change them in runtime without PME, change with a single
PME or change with full cache restart.

What do you think?

Vladimir.

On Wed, Nov 21, 2018 at 5:01 PM Eduard Shangareev <[hidden email]>
wrote:

> Vladimir,
>
> 1) Affinity could be changed, but count of partition couldn't be.
> 2) So it would trigger 2 PME. Dynamic start and stop.
> 3) In theory, should cancel them and new setting should be applied. How it
> works now? Create an index and stop node, for example.
>
> On Wed, Nov 21, 2018 at 4:56 PM Vladimir Ozerov <[hidden email]>
> wrote:
>
> > Hi Ed,
> >
> > Several questions from my side:
> > 1) If we do not allow to change the most demanded by users things like
> > affinity or persistence/in-memory, then what kind of configuration
> > properties do we expect to be changed? Can we have several examples?
> > 2) How will it interact with PME?
> > 3) How will it interact with CREATE INDEX and ALTER TABLE commands?
> >
> > On Wed, Nov 21, 2018 at 4:48 PM Eduard Shangareev <
> > [hidden email]> wrote:
> >
> > > Igniters,
> > >
> > > I propose new public API to change the cache configuration of
> persistent
> > > caches with keeping data.
> > >
> > > It would look like:
> > >
> > > Ignite ignite = ...;
> > > ignite.restartCaches(cfg1, ... cfgN);
> > >
> > > where cfgX is a new cache configuration, which contains the name of an
> > > existing persistent cache.
> > >
> > > The obvious limitation:
> > > - affinity key mapping couldn't be changed;
> > > - count of partitions couldn't be changed;
> > > - MVCC couldn't be turned off/on;
> > > - persistent couldn't be turned off;
> > > - group settings couldn't be changed (group name);
> > > - if cache belongs to group it's needed to restart all of them.
> > >
> > > Failure scenario is the crucial thing (and most difficult):
> > > - initiator fail;
> > > - cluster restart at any stage;
> > > - joining/starting offline nodes.
> > >
> > > Some thoughts about implementation:
> > > - stop cache with destroy=false;
> > > - start cache dynamically with new configuration;
> > > - if indexes settings changed - remove index.bin to start indexation;
> > > - change blt-history when start cache initiated to not allow join nodes
> > > with old configuration;
> > > - use restartId (IGNITE-8911) to not allow to start cache in between.
> > >
> > > Your thoughts? Would it be a useful feature?
> > >
> >
>
>
> --
> Best regards,
> Eduard.
>
Reply | Threaded
Open this post in threaded view
|

Re: New API for changing configuration of persistent caches

Eduard Shangareev
Vladimir,

I propose not to change cache configuration in runtime but restart cache
with the new compatible configuration on data which we have underfoot.

What we could change:
-backup count;
-TRANSACTIONAL <-> ATOMIC;
-REPLICATED - PARTITIONED;
-other settings.

So, yeah, it would be great to have a possibility to change some properties
in runtime. But right we don't any way to change anything except indexes
and SQL fields.

We already have all mechanism to do this.
The main issue is to make it reliable and exclude cases when we could come
to the unrecoverable state.

So, I suggest keeping the solution as simple as possible.
For indexes clashes and ClassNotFoundException we could revert
configuration update and start with the old configuration.


On Wed, Nov 21, 2018 at 5:44 PM Vladimir Ozerov <[hidden email]>
wrote:

> Eduard,
>
> Got it. Please take the following things in count during design:
>
> 1) Two distinct PMEs might not work well. Consider a situation w1hen I
> decided to move a cache with index "MY_INDEX" from schema A to schema B.
> While cache was stopped, another cache with index "MY_INDEX" was created in
> schema B. Now first cache cannot start due to index name conflict.
> 2) Cancelling index creation is also bad idea because this is potentially
> long operation. Instead, most likely that we should wait to concurrent
> schema operations to finish first. That is, all operations on cache should
> be ordered wrt each other somehow
> 3) Why do we think that cache restart will be needed at all? We have a lot
> of configuration properties which could be changed safely either without
> PME or with a single PME. - rebalance properties, cache store properties
> (especially write-behind stuff), some query properties (e.g. "detail
> metrics"), etc.. In essence, it seems that >50% of properties could be
> changed without cache restart, other 25% will not be supported, and the
> rest may require restart.
> 4) Client nodes and thin client may not have necessary classes in
> classpath. E.g. consider a user which want to change rebalance timeout for
> cache, but do not have configured interceptor in classpath. With proposed
> API it will be impossible. This is especially true for non-Java clients.
>
> That said, I think we should consider another API which will not require
> full CacheConfiguration object. This might be a kind of builder or so. And
> once user set properties he want to change to the builder, we can analyze
> them and either change them in runtime without PME, change with a single
> PME or change with full cache restart.
>
> What do you think?
>
> Vladimir.
>
> On Wed, Nov 21, 2018 at 5:01 PM Eduard Shangareev <
> [hidden email]>
> wrote:
>
> > Vladimir,
> >
> > 1) Affinity could be changed, but count of partition couldn't be.
> > 2) So it would trigger 2 PME. Dynamic start and stop.
> > 3) In theory, should cancel them and new setting should be applied. How
> it
> > works now? Create an index and stop node, for example.
> >
> > On Wed, Nov 21, 2018 at 4:56 PM Vladimir Ozerov <[hidden email]>
> > wrote:
> >
> > > Hi Ed,
> > >
> > > Several questions from my side:
> > > 1) If we do not allow to change the most demanded by users things like
> > > affinity or persistence/in-memory, then what kind of configuration
> > > properties do we expect to be changed? Can we have several examples?
> > > 2) How will it interact with PME?
> > > 3) How will it interact with CREATE INDEX and ALTER TABLE commands?
> > >
> > > On Wed, Nov 21, 2018 at 4:48 PM Eduard Shangareev <
> > > [hidden email]> wrote:
> > >
> > > > Igniters,
> > > >
> > > > I propose new public API to change the cache configuration of
> > persistent
> > > > caches with keeping data.
> > > >
> > > > It would look like:
> > > >
> > > > Ignite ignite = ...;
> > > > ignite.restartCaches(cfg1, ... cfgN);
> > > >
> > > > where cfgX is a new cache configuration, which contains the name of
> an
> > > > existing persistent cache.
> > > >
> > > > The obvious limitation:
> > > > - affinity key mapping couldn't be changed;
> > > > - count of partitions couldn't be changed;
> > > > - MVCC couldn't be turned off/on;
> > > > - persistent couldn't be turned off;
> > > > - group settings couldn't be changed (group name);
> > > > - if cache belongs to group it's needed to restart all of them.
> > > >
> > > > Failure scenario is the crucial thing (and most difficult):
> > > > - initiator fail;
> > > > - cluster restart at any stage;
> > > > - joining/starting offline nodes.
> > > >
> > > > Some thoughts about implementation:
> > > > - stop cache with destroy=false;
> > > > - start cache dynamically with new configuration;
> > > > - if indexes settings changed - remove index.bin to start indexation;
> > > > - change blt-history when start cache initiated to not allow join
> nodes
> > > > with old configuration;
> > > > - use restartId (IGNITE-8911) to not allow to start cache in between.
> > > >
> > > > Your thoughts? Would it be a useful feature?
> > > >
> > >
> >
> >
> > --
> > Best regards,
> > Eduard.
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: New API for changing configuration of persistent caches

Vladimir Ozerov
Eduard,

Simple != correct. Let’s consider a simple use case: user want to change
PARTITIONED -> REPLICATED from .NET, but do not some classes from
CacheConfiguration. How do we solve this?

Vladimir.

ср, 21 нояб. 2018 г. в 18:02, Eduard Shangareev <[hidden email]
>:

> Vladimir,
>
> I propose not to change cache configuration in runtime but restart cache
> with the new compatible configuration on data which we have underfoot.
>
> What we could change:
> -backup count;
> -TRANSACTIONAL <-> ATOMIC;
> -REPLICATED - PARTITIONED;
> -other settings.
>
> So, yeah, it would be great to have a possibility to change some properties
> in runtime. But right we don't any way to change anything except indexes
> and SQL fields.
>
> We already have all mechanism to do this.
> The main issue is to make it reliable and exclude cases when we could come
> to the unrecoverable state.
>
> So, I suggest keeping the solution as simple as possible.
> For indexes clashes and ClassNotFoundException we could revert
> configuration update and start with the old configuration.
>
>
> On Wed, Nov 21, 2018 at 5:44 PM Vladimir Ozerov <[hidden email]>
> wrote:
>
> > Eduard,
> >
> > Got it. Please take the following things in count during design:
> >
> > 1) Two distinct PMEs might not work well. Consider a situation w1hen I
> > decided to move a cache with index "MY_INDEX" from schema A to schema B.
> > While cache was stopped, another cache with index "MY_INDEX" was created
> in
> > schema B. Now first cache cannot start due to index name conflict.
> > 2) Cancelling index creation is also bad idea because this is potentially
> > long operation. Instead, most likely that we should wait to concurrent
> > schema operations to finish first. That is, all operations on cache
> should
> > be ordered wrt each other somehow
> > 3) Why do we think that cache restart will be needed at all? We have a
> lot
> > of configuration properties which could be changed safely either without
> > PME or with a single PME. - rebalance properties, cache store properties
> > (especially write-behind stuff), some query properties (e.g. "detail
> > metrics"), etc.. In essence, it seems that >50% of properties could be
> > changed without cache restart, other 25% will not be supported, and the
> > rest may require restart.
> > 4) Client nodes and thin client may not have necessary classes in
> > classpath. E.g. consider a user which want to change rebalance timeout
> for
> > cache, but do not have configured interceptor in classpath. With proposed
> > API it will be impossible. This is especially true for non-Java clients.
> >
> > That said, I think we should consider another API which will not require
> > full CacheConfiguration object. This might be a kind of builder or so.
> And
> > once user set properties he want to change to the builder, we can analyze
> > them and either change them in runtime without PME, change with a single
> > PME or change with full cache restart.
> >
> > What do you think?
> >
> > Vladimir.
> >
> > On Wed, Nov 21, 2018 at 5:01 PM Eduard Shangareev <
> > [hidden email]>
> > wrote:
> >
> > > Vladimir,
> > >
> > > 1) Affinity could be changed, but count of partition couldn't be.
> > > 2) So it would trigger 2 PME. Dynamic start and stop.
> > > 3) In theory, should cancel them and new setting should be applied. How
> > it
> > > works now? Create an index and stop node, for example.
> > >
> > > On Wed, Nov 21, 2018 at 4:56 PM Vladimir Ozerov <[hidden email]>
> > > wrote:
> > >
> > > > Hi Ed,
> > > >
> > > > Several questions from my side:
> > > > 1) If we do not allow to change the most demanded by users things
> like
> > > > affinity or persistence/in-memory, then what kind of configuration
> > > > properties do we expect to be changed? Can we have several examples?
> > > > 2) How will it interact with PME?
> > > > 3) How will it interact with CREATE INDEX and ALTER TABLE commands?
> > > >
> > > > On Wed, Nov 21, 2018 at 4:48 PM Eduard Shangareev <
> > > > [hidden email]> wrote:
> > > >
> > > > > Igniters,
> > > > >
> > > > > I propose new public API to change the cache configuration of
> > > persistent
> > > > > caches with keeping data.
> > > > >
> > > > > It would look like:
> > > > >
> > > > > Ignite ignite = ...;
> > > > > ignite.restartCaches(cfg1, ... cfgN);
> > > > >
> > > > > where cfgX is a new cache configuration, which contains the name of
> > an
> > > > > existing persistent cache.
> > > > >
> > > > > The obvious limitation:
> > > > > - affinity key mapping couldn't be changed;
> > > > > - count of partitions couldn't be changed;
> > > > > - MVCC couldn't be turned off/on;
> > > > > - persistent couldn't be turned off;
> > > > > - group settings couldn't be changed (group name);
> > > > > - if cache belongs to group it's needed to restart all of them.
> > > > >
> > > > > Failure scenario is the crucial thing (and most difficult):
> > > > > - initiator fail;
> > > > > - cluster restart at any stage;
> > > > > - joining/starting offline nodes.
> > > > >
> > > > > Some thoughts about implementation:
> > > > > - stop cache with destroy=false;
> > > > > - start cache dynamically with new configuration;
> > > > > - if indexes settings changed - remove index.bin to start
> indexation;
> > > > > - change blt-history when start cache initiated to not allow join
> > nodes
> > > > > with old configuration;
> > > > > - use restartId (IGNITE-8911) to not allow to start cache in
> between.
> > > > >
> > > > > Your thoughts? Would it be a useful feature?
> > > > >
> > > >
> > >
> > >
> > > --
> > > Best regards,
> > > Eduard.
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: New API for changing configuration of persistent caches

Vladimir Ozerov
Correction: “do not *have* some classes”.

ср, 21 нояб. 2018 г. в 18:09, Vladimir Ozerov <[hidden email]>:

> Eduard,
>
> Simple != correct. Let’s consider a simple use case: user want to change
> PARTITIONED -> REPLICATED from .NET, but do not some classes from
> CacheConfiguration. How do we solve this?
>
> Vladimir.
>
> ср, 21 нояб. 2018 г. в 18:02, Eduard Shangareev <
> [hidden email]>:
>
>> Vladimir,
>>
>> I propose not to change cache configuration in runtime but restart cache
>> with the new compatible configuration on data which we have underfoot.
>>
>> What we could change:
>> -backup count;
>> -TRANSACTIONAL <-> ATOMIC;
>> -REPLICATED - PARTITIONED;
>> -other settings.
>>
>> So, yeah, it would be great to have a possibility to change some
>> properties
>> in runtime. But right we don't any way to change anything except indexes
>> and SQL fields.
>>
>> We already have all mechanism to do this.
>> The main issue is to make it reliable and exclude cases when we could come
>> to the unrecoverable state.
>>
>> So, I suggest keeping the solution as simple as possible.
>> For indexes clashes and ClassNotFoundException we could revert
>> configuration update and start with the old configuration.
>>
>>
>> On Wed, Nov 21, 2018 at 5:44 PM Vladimir Ozerov <[hidden email]>
>> wrote:
>>
>> > Eduard,
>> >
>> > Got it. Please take the following things in count during design:
>> >
>> > 1) Two distinct PMEs might not work well. Consider a situation w1hen I
>> > decided to move a cache with index "MY_INDEX" from schema A to schema B.
>> > While cache was stopped, another cache with index "MY_INDEX" was
>> created in
>> > schema B. Now first cache cannot start due to index name conflict.
>> > 2) Cancelling index creation is also bad idea because this is
>> potentially
>> > long operation. Instead, most likely that we should wait to concurrent
>> > schema operations to finish first. That is, all operations on cache
>> should
>> > be ordered wrt each other somehow
>> > 3) Why do we think that cache restart will be needed at all? We have a
>> lot
>> > of configuration properties which could be changed safely either without
>> > PME or with a single PME. - rebalance properties, cache store properties
>> > (especially write-behind stuff), some query properties (e.g. "detail
>> > metrics"), etc.. In essence, it seems that >50% of properties could be
>> > changed without cache restart, other 25% will not be supported, and the
>> > rest may require restart.
>> > 4) Client nodes and thin client may not have necessary classes in
>> > classpath. E.g. consider a user which want to change rebalance timeout
>> for
>> > cache, but do not have configured interceptor in classpath. With
>> proposed
>> > API it will be impossible. This is especially true for non-Java clients.
>> >
>> > That said, I think we should consider another API which will not require
>> > full CacheConfiguration object. This might be a kind of builder or so.
>> And
>> > once user set properties he want to change to the builder, we can
>> analyze
>> > them and either change them in runtime without PME, change with a single
>> > PME or change with full cache restart.
>> >
>> > What do you think?
>> >
>> > Vladimir.
>> >
>> > On Wed, Nov 21, 2018 at 5:01 PM Eduard Shangareev <
>> > [hidden email]>
>> > wrote:
>> >
>> > > Vladimir,
>> > >
>> > > 1) Affinity could be changed, but count of partition couldn't be.
>> > > 2) So it would trigger 2 PME. Dynamic start and stop.
>> > > 3) In theory, should cancel them and new setting should be applied.
>> How
>> > it
>> > > works now? Create an index and stop node, for example.
>> > >
>> > > On Wed, Nov 21, 2018 at 4:56 PM Vladimir Ozerov <[hidden email]
>> >
>> > > wrote:
>> > >
>> > > > Hi Ed,
>> > > >
>> > > > Several questions from my side:
>> > > > 1) If we do not allow to change the most demanded by users things
>> like
>> > > > affinity or persistence/in-memory, then what kind of configuration
>> > > > properties do we expect to be changed? Can we have several examples?
>> > > > 2) How will it interact with PME?
>> > > > 3) How will it interact with CREATE INDEX and ALTER TABLE commands?
>> > > >
>> > > > On Wed, Nov 21, 2018 at 4:48 PM Eduard Shangareev <
>> > > > [hidden email]> wrote:
>> > > >
>> > > > > Igniters,
>> > > > >
>> > > > > I propose new public API to change the cache configuration of
>> > > persistent
>> > > > > caches with keeping data.
>> > > > >
>> > > > > It would look like:
>> > > > >
>> > > > > Ignite ignite = ...;
>> > > > > ignite.restartCaches(cfg1, ... cfgN);
>> > > > >
>> > > > > where cfgX is a new cache configuration, which contains the name
>> of
>> > an
>> > > > > existing persistent cache.
>> > > > >
>> > > > > The obvious limitation:
>> > > > > - affinity key mapping couldn't be changed;
>> > > > > - count of partitions couldn't be changed;
>> > > > > - MVCC couldn't be turned off/on;
>> > > > > - persistent couldn't be turned off;
>> > > > > - group settings couldn't be changed (group name);
>> > > > > - if cache belongs to group it's needed to restart all of them.
>> > > > >
>> > > > > Failure scenario is the crucial thing (and most difficult):
>> > > > > - initiator fail;
>> > > > > - cluster restart at any stage;
>> > > > > - joining/starting offline nodes.
>> > > > >
>> > > > > Some thoughts about implementation:
>> > > > > - stop cache with destroy=false;
>> > > > > - start cache dynamically with new configuration;
>> > > > > - if indexes settings changed - remove index.bin to start
>> indexation;
>> > > > > - change blt-history when start cache initiated to not allow join
>> > nodes
>> > > > > with old configuration;
>> > > > > - use restartId (IGNITE-8911) to not allow to start cache in
>> between.
>> > > > >
>> > > > > Your thoughts? Would it be a useful feature?
>> > > > >
>> > > >
>> > >
>> > >
>> > > --
>> > > Best regards,
>> > > Eduard.
>> > >
>> >
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: New API for changing configuration of persistent caches

Eduard Shangareev
In reply to this post by Vladimir Ozerov
Vladimir,

How does .Net user start caches right now?

On Wed, Nov 21, 2018 at 6:10 PM Vladimir Ozerov <[hidden email]>
wrote:

> Eduard,
>
> Simple != correct. Let’s consider a simple use case: user want to change
> PARTITIONED -> REPLICATED from .NET, but do not some classes from
> CacheConfiguration. How do we solve this?
>
> Vladimir.
>
> ср, 21 нояб. 2018 г. в 18:02, Eduard Shangareev <
> [hidden email]
> >:
>
> > Vladimir,
> >
> > I propose not to change cache configuration in runtime but restart cache
> > with the new compatible configuration on data which we have underfoot.
> >
> > What we could change:
> > -backup count;
> > -TRANSACTIONAL <-> ATOMIC;
> > -REPLICATED - PARTITIONED;
> > -other settings.
> >
> > So, yeah, it would be great to have a possibility to change some
> properties
> > in runtime. But right we don't any way to change anything except indexes
> > and SQL fields.
> >
> > We already have all mechanism to do this.
> > The main issue is to make it reliable and exclude cases when we could
> come
> > to the unrecoverable state.
> >
> > So, I suggest keeping the solution as simple as possible.
> > For indexes clashes and ClassNotFoundException we could revert
> > configuration update and start with the old configuration.
> >
> >
> > On Wed, Nov 21, 2018 at 5:44 PM Vladimir Ozerov <[hidden email]>
> > wrote:
> >
> > > Eduard,
> > >
> > > Got it. Please take the following things in count during design:
> > >
> > > 1) Two distinct PMEs might not work well. Consider a situation w1hen I
> > > decided to move a cache with index "MY_INDEX" from schema A to schema
> B.
> > > While cache was stopped, another cache with index "MY_INDEX" was
> created
> > in
> > > schema B. Now first cache cannot start due to index name conflict.
> > > 2) Cancelling index creation is also bad idea because this is
> potentially
> > > long operation. Instead, most likely that we should wait to concurrent
> > > schema operations to finish first. That is, all operations on cache
> > should
> > > be ordered wrt each other somehow
> > > 3) Why do we think that cache restart will be needed at all? We have a
> > lot
> > > of configuration properties which could be changed safely either
> without
> > > PME or with a single PME. - rebalance properties, cache store
> properties
> > > (especially write-behind stuff), some query properties (e.g. "detail
> > > metrics"), etc.. In essence, it seems that >50% of properties could be
> > > changed without cache restart, other 25% will not be supported, and the
> > > rest may require restart.
> > > 4) Client nodes and thin client may not have necessary classes in
> > > classpath. E.g. consider a user which want to change rebalance timeout
> > for
> > > cache, but do not have configured interceptor in classpath. With
> proposed
> > > API it will be impossible. This is especially true for non-Java
> clients.
> > >
> > > That said, I think we should consider another API which will not
> require
> > > full CacheConfiguration object. This might be a kind of builder or so.
> > And
> > > once user set properties he want to change to the builder, we can
> analyze
> > > them and either change them in runtime without PME, change with a
> single
> > > PME or change with full cache restart.
> > >
> > > What do you think?
> > >
> > > Vladimir.
> > >
> > > On Wed, Nov 21, 2018 at 5:01 PM Eduard Shangareev <
> > > [hidden email]>
> > > wrote:
> > >
> > > > Vladimir,
> > > >
> > > > 1) Affinity could be changed, but count of partition couldn't be.
> > > > 2) So it would trigger 2 PME. Dynamic start and stop.
> > > > 3) In theory, should cancel them and new setting should be applied.
> How
> > > it
> > > > works now? Create an index and stop node, for example.
> > > >
> > > > On Wed, Nov 21, 2018 at 4:56 PM Vladimir Ozerov <
> [hidden email]>
> > > > wrote:
> > > >
> > > > > Hi Ed,
> > > > >
> > > > > Several questions from my side:
> > > > > 1) If we do not allow to change the most demanded by users things
> > like
> > > > > affinity or persistence/in-memory, then what kind of configuration
> > > > > properties do we expect to be changed? Can we have several
> examples?
> > > > > 2) How will it interact with PME?
> > > > > 3) How will it interact with CREATE INDEX and ALTER TABLE commands?
> > > > >
> > > > > On Wed, Nov 21, 2018 at 4:48 PM Eduard Shangareev <
> > > > > [hidden email]> wrote:
> > > > >
> > > > > > Igniters,
> > > > > >
> > > > > > I propose new public API to change the cache configuration of
> > > > persistent
> > > > > > caches with keeping data.
> > > > > >
> > > > > > It would look like:
> > > > > >
> > > > > > Ignite ignite = ...;
> > > > > > ignite.restartCaches(cfg1, ... cfgN);
> > > > > >
> > > > > > where cfgX is a new cache configuration, which contains the name
> of
> > > an
> > > > > > existing persistent cache.
> > > > > >
> > > > > > The obvious limitation:
> > > > > > - affinity key mapping couldn't be changed;
> > > > > > - count of partitions couldn't be changed;
> > > > > > - MVCC couldn't be turned off/on;
> > > > > > - persistent couldn't be turned off;
> > > > > > - group settings couldn't be changed (group name);
> > > > > > - if cache belongs to group it's needed to restart all of them.
> > > > > >
> > > > > > Failure scenario is the crucial thing (and most difficult):
> > > > > > - initiator fail;
> > > > > > - cluster restart at any stage;
> > > > > > - joining/starting offline nodes.
> > > > > >
> > > > > > Some thoughts about implementation:
> > > > > > - stop cache with destroy=false;
> > > > > > - start cache dynamically with new configuration;
> > > > > > - if indexes settings changed - remove index.bin to start
> > indexation;
> > > > > > - change blt-history when start cache initiated to not allow join
> > > nodes
> > > > > > with old configuration;
> > > > > > - use restartId (IGNITE-8911) to not allow to start cache in
> > between.
> > > > > >
> > > > > > Your thoughts? Would it be a useful feature?
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Eduard.
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: New API for changing configuration of persistent caches

Alexey Kuznetsov
In reply to this post by Vladimir Ozerov
Vova, Ed

I think in case when classes not available we can sent a string with cache
XML configuration (bean) to server node that has all classes in classpath.

What do you think?

On Wed, Nov 21, 2018 at 10:10 PM Vladimir Ozerov <[hidden email]>
wrote:

> Eduard,
>
> Simple != correct. Let’s consider a simple use case: user want to change
> PARTITIONED -> REPLICATED from .NET, but do not some classes from
> CacheConfiguration. How do we solve this?
>
> Vladimir.
>
> ср, 21 нояб. 2018 г. в 18:02, Eduard Shangareev <
> [hidden email]
> >:
>
> > Vladimir,
> >
> > I propose not to change cache configuration in runtime but restart cache
> > with the new compatible configuration on data which we have underfoot.
> >
> > What we could change:
> > -backup count;
> > -TRANSACTIONAL <-> ATOMIC;
> > -REPLICATED - PARTITIONED;
> > -other settings.
> >
> > So, yeah, it would be great to have a possibility to change some
> properties
> > in runtime. But right we don't any way to change anything except indexes
> > and SQL fields.
> >
> > We already have all mechanism to do this.
> > The main issue is to make it reliable and exclude cases when we could
> come
> > to the unrecoverable state.
> >
> > So, I suggest keeping the solution as simple as possible.
> > For indexes clashes and ClassNotFoundException we could revert
> > configuration update and start with the old configuration.
> >
> >
> > On Wed, Nov 21, 2018 at 5:44 PM Vladimir Ozerov <[hidden email]>
> > wrote:
> >
> > > Eduard,
> > >
> > > Got it. Please take the following things in count during design:
> > >
> > > 1) Two distinct PMEs might not work well. Consider a situation w1hen I
> > > decided to move a cache with index "MY_INDEX" from schema A to schema
> B.
> > > While cache was stopped, another cache with index "MY_INDEX" was
> created
> > in
> > > schema B. Now first cache cannot start due to index name conflict.
> > > 2) Cancelling index creation is also bad idea because this is
> potentially
> > > long operation. Instead, most likely that we should wait to concurrent
> > > schema operations to finish first. That is, all operations on cache
> > should
> > > be ordered wrt each other somehow
> > > 3) Why do we think that cache restart will be needed at all? We have a
> > lot
> > > of configuration properties which could be changed safely either
> without
> > > PME or with a single PME. - rebalance properties, cache store
> properties
> > > (especially write-behind stuff), some query properties (e.g. "detail
> > > metrics"), etc.. In essence, it seems that >50% of properties could be
> > > changed without cache restart, other 25% will not be supported, and the
> > > rest may require restart.
> > > 4) Client nodes and thin client may not have necessary classes in
> > > classpath. E.g. consider a user which want to change rebalance timeout
> > for
> > > cache, but do not have configured interceptor in classpath. With
> proposed
> > > API it will be impossible. This is especially true for non-Java
> clients.
> > >
> > > That said, I think we should consider another API which will not
> require
> > > full CacheConfiguration object. This might be a kind of builder or so.
> > And
> > > once user set properties he want to change to the builder, we can
> analyze
> > > them and either change them in runtime without PME, change with a
> single
> > > PME or change with full cache restart.
> > >
> > > What do you think?
> > >
> > > Vladimir.
> > >
> > > On Wed, Nov 21, 2018 at 5:01 PM Eduard Shangareev <
> > > [hidden email]>
> > > wrote:
> > >
> > > > Vladimir,
> > > >
> > > > 1) Affinity could be changed, but count of partition couldn't be.
> > > > 2) So it would trigger 2 PME. Dynamic start and stop.
> > > > 3) In theory, should cancel them and new setting should be applied.
> How
> > > it
> > > > works now? Create an index and stop node, for example.
> > > >
> > > > On Wed, Nov 21, 2018 at 4:56 PM Vladimir Ozerov <
> [hidden email]>
> > > > wrote:
> > > >
> > > > > Hi Ed,
> > > > >
> > > > > Several questions from my side:
> > > > > 1) If we do not allow to change the most demanded by users things
> > like
> > > > > affinity or persistence/in-memory, then what kind of configuration
> > > > > properties do we expect to be changed? Can we have several
> examples?
> > > > > 2) How will it interact with PME?
> > > > > 3) How will it interact with CREATE INDEX and ALTER TABLE commands?
> > > > >
> > > > > On Wed, Nov 21, 2018 at 4:48 PM Eduard Shangareev <
> > > > > [hidden email]> wrote:
> > > > >
> > > > > > Igniters,
> > > > > >
> > > > > > I propose new public API to change the cache configuration of
> > > > persistent
> > > > > > caches with keeping data.
> > > > > >
> > > > > > It would look like:
> > > > > >
> > > > > > Ignite ignite = ...;
> > > > > > ignite.restartCaches(cfg1, ... cfgN);
> > > > > >
> > > > > > where cfgX is a new cache configuration, which contains the name
> of
> > > an
> > > > > > existing persistent cache.
> > > > > >
> > > > > > The obvious limitation:
> > > > > > - affinity key mapping couldn't be changed;
> > > > > > - count of partitions couldn't be changed;
> > > > > > - MVCC couldn't be turned off/on;
> > > > > > - persistent couldn't be turned off;
> > > > > > - group settings couldn't be changed (group name);
> > > > > > - if cache belongs to group it's needed to restart all of them.
> > > > > >
> > > > > > Failure scenario is the crucial thing (and most difficult):
> > > > > > - initiator fail;
> > > > > > - cluster restart at any stage;
> > > > > > - joining/starting offline nodes.
> > > > > >
> > > > > > Some thoughts about implementation:
> > > > > > - stop cache with destroy=false;
> > > > > > - start cache dynamically with new configuration;
> > > > > > - if indexes settings changed - remove index.bin to start
> > indexation;
> > > > > > - change blt-history when start cache initiated to not allow join
> > > nodes
> > > > > > with old configuration;
> > > > > > - use restartId (IGNITE-8911) to not allow to start cache in
> > between.
> > > > > >
> > > > > > Your thoughts? Would it be a useful feature?
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Eduard.
> > > >
> > >
> >
>


--
Alexey Kuznetsov
Reply | Threaded
Open this post in threaded view
|

Re: New API for changing configuration of persistent caches

Vladimir Ozerov
Alex,

Spring XML is not easily available from API. Sometimes it simply doesn’t
exist.
Why not use builder instead?

Ignite.changeCache(“my_cache”,
Builder.create().setCacheMode(REPLICATED).build())

Advantages:
1) Available on all platforms
2) Expose only properties we support
3) Avoid race condition when configuration changes between configuration
read and method call (what could lead to a number of strange effects).

Makes sense?

ср, 21 нояб. 2018 г. в 18:16, Alexey Kuznetsov <[hidden email]>:

> Vova, Ed
>
> I think in case when classes not available we can sent a string with cache
> XML configuration (bean) to server node that has all classes in classpath.
>
> What do you think?
>
> On Wed, Nov 21, 2018 at 10:10 PM Vladimir Ozerov <[hidden email]>
> wrote:
>
> > Eduard,
> >
> > Simple != correct. Let’s consider a simple use case: user want to change
> > PARTITIONED -> REPLICATED from .NET, but do not some classes from
> > CacheConfiguration. How do we solve this?
> >
> > Vladimir.
> >
> > ср, 21 нояб. 2018 г. в 18:02, Eduard Shangareev <
> > [hidden email]
> > >:
> >
> > > Vladimir,
> > >
> > > I propose not to change cache configuration in runtime but restart
> cache
> > > with the new compatible configuration on data which we have underfoot.
> > >
> > > What we could change:
> > > -backup count;
> > > -TRANSACTIONAL <-> ATOMIC;
> > > -REPLICATED - PARTITIONED;
> > > -other settings.
> > >
> > > So, yeah, it would be great to have a possibility to change some
> > properties
> > > in runtime. But right we don't any way to change anything except
> indexes
> > > and SQL fields.
> > >
> > > We already have all mechanism to do this.
> > > The main issue is to make it reliable and exclude cases when we could
> > come
> > > to the unrecoverable state.
> > >
> > > So, I suggest keeping the solution as simple as possible.
> > > For indexes clashes and ClassNotFoundException we could revert
> > > configuration update and start with the old configuration.
> > >
> > >
> > > On Wed, Nov 21, 2018 at 5:44 PM Vladimir Ozerov <[hidden email]>
> > > wrote:
> > >
> > > > Eduard,
> > > >
> > > > Got it. Please take the following things in count during design:
> > > >
> > > > 1) Two distinct PMEs might not work well. Consider a situation w1hen
> I
> > > > decided to move a cache with index "MY_INDEX" from schema A to schema
> > B.
> > > > While cache was stopped, another cache with index "MY_INDEX" was
> > created
> > > in
> > > > schema B. Now first cache cannot start due to index name conflict.
> > > > 2) Cancelling index creation is also bad idea because this is
> > potentially
> > > > long operation. Instead, most likely that we should wait to
> concurrent
> > > > schema operations to finish first. That is, all operations on cache
> > > should
> > > > be ordered wrt each other somehow
> > > > 3) Why do we think that cache restart will be needed at all? We have
> a
> > > lot
> > > > of configuration properties which could be changed safely either
> > without
> > > > PME or with a single PME. - rebalance properties, cache store
> > properties
> > > > (especially write-behind stuff), some query properties (e.g. "detail
> > > > metrics"), etc.. In essence, it seems that >50% of properties could
> be
> > > > changed without cache restart, other 25% will not be supported, and
> the
> > > > rest may require restart.
> > > > 4) Client nodes and thin client may not have necessary classes in
> > > > classpath. E.g. consider a user which want to change rebalance
> timeout
> > > for
> > > > cache, but do not have configured interceptor in classpath. With
> > proposed
> > > > API it will be impossible. This is especially true for non-Java
> > clients.
> > > >
> > > > That said, I think we should consider another API which will not
> > require
> > > > full CacheConfiguration object. This might be a kind of builder or
> so.
> > > And
> > > > once user set properties he want to change to the builder, we can
> > analyze
> > > > them and either change them in runtime without PME, change with a
> > single
> > > > PME or change with full cache restart.
> > > >
> > > > What do you think?
> > > >
> > > > Vladimir.
> > > >
> > > > On Wed, Nov 21, 2018 at 5:01 PM Eduard Shangareev <
> > > > [hidden email]>
> > > > wrote:
> > > >
> > > > > Vladimir,
> > > > >
> > > > > 1) Affinity could be changed, but count of partition couldn't be.
> > > > > 2) So it would trigger 2 PME. Dynamic start and stop.
> > > > > 3) In theory, should cancel them and new setting should be applied.
> > How
> > > > it
> > > > > works now? Create an index and stop node, for example.
> > > > >
> > > > > On Wed, Nov 21, 2018 at 4:56 PM Vladimir Ozerov <
> > [hidden email]>
> > > > > wrote:
> > > > >
> > > > > > Hi Ed,
> > > > > >
> > > > > > Several questions from my side:
> > > > > > 1) If we do not allow to change the most demanded by users things
> > > like
> > > > > > affinity or persistence/in-memory, then what kind of
> configuration
> > > > > > properties do we expect to be changed? Can we have several
> > examples?
> > > > > > 2) How will it interact with PME?
> > > > > > 3) How will it interact with CREATE INDEX and ALTER TABLE
> commands?
> > > > > >
> > > > > > On Wed, Nov 21, 2018 at 4:48 PM Eduard Shangareev <
> > > > > > [hidden email]> wrote:
> > > > > >
> > > > > > > Igniters,
> > > > > > >
> > > > > > > I propose new public API to change the cache configuration of
> > > > > persistent
> > > > > > > caches with keeping data.
> > > > > > >
> > > > > > > It would look like:
> > > > > > >
> > > > > > > Ignite ignite = ...;
> > > > > > > ignite.restartCaches(cfg1, ... cfgN);
> > > > > > >
> > > > > > > where cfgX is a new cache configuration, which contains the
> name
> > of
> > > > an
> > > > > > > existing persistent cache.
> > > > > > >
> > > > > > > The obvious limitation:
> > > > > > > - affinity key mapping couldn't be changed;
> > > > > > > - count of partitions couldn't be changed;
> > > > > > > - MVCC couldn't be turned off/on;
> > > > > > > - persistent couldn't be turned off;
> > > > > > > - group settings couldn't be changed (group name);
> > > > > > > - if cache belongs to group it's needed to restart all of them.
> > > > > > >
> > > > > > > Failure scenario is the crucial thing (and most difficult):
> > > > > > > - initiator fail;
> > > > > > > - cluster restart at any stage;
> > > > > > > - joining/starting offline nodes.
> > > > > > >
> > > > > > > Some thoughts about implementation:
> > > > > > > - stop cache with destroy=false;
> > > > > > > - start cache dynamically with new configuration;
> > > > > > > - if indexes settings changed - remove index.bin to start
> > > indexation;
> > > > > > > - change blt-history when start cache initiated to not allow
> join
> > > > nodes
> > > > > > > with old configuration;
> > > > > > > - use restartId (IGNITE-8911) to not allow to start cache in
> > > between.
> > > > > > >
> > > > > > > Your thoughts? Would it be a useful feature?
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Eduard.
> > > > >
> > > >
> > >
> >
>
>
> --
> Alexey Kuznetsov
>
Reply | Threaded
Open this post in threaded view
|

Re: New API for changing configuration of persistent caches

Vladimir Ozerov
In reply to this post by Eduard Shangareev
Ed,

Caches in .NET could be started programmatically, from XML which .NET API
has no access to, or dynamically from remote nodes (eg Java node).

ср, 21 нояб. 2018 г. в 18:24, Eduard Shangareev <[hidden email]
>:

> Vladimir,
>
> How does .Net user start caches right now?
>
> On Wed, Nov 21, 2018 at 6:10 PM Vladimir Ozerov <[hidden email]>
> wrote:
>
> > Eduard,
> >
> > Simple != correct. Let’s consider a simple use case: user want to change
> > PARTITIONED -> REPLICATED from .NET, but do not some classes from
> > CacheConfiguration. How do we solve this?
> >
> > Vladimir.
> >
> > ср, 21 нояб. 2018 г. в 18:02, Eduard Shangareev <
> > [hidden email]
> > >:
> >
> > > Vladimir,
> > >
> > > I propose not to change cache configuration in runtime but restart
> cache
> > > with the new compatible configuration on data which we have underfoot.
> > >
> > > What we could change:
> > > -backup count;
> > > -TRANSACTIONAL <-> ATOMIC;
> > > -REPLICATED - PARTITIONED;
> > > -other settings.
> > >
> > > So, yeah, it would be great to have a possibility to change some
> > properties
> > > in runtime. But right we don't any way to change anything except
> indexes
> > > and SQL fields.
> > >
> > > We already have all mechanism to do this.
> > > The main issue is to make it reliable and exclude cases when we could
> > come
> > > to the unrecoverable state.
> > >
> > > So, I suggest keeping the solution as simple as possible.
> > > For indexes clashes and ClassNotFoundException we could revert
> > > configuration update and start with the old configuration.
> > >
> > >
> > > On Wed, Nov 21, 2018 at 5:44 PM Vladimir Ozerov <[hidden email]>
> > > wrote:
> > >
> > > > Eduard,
> > > >
> > > > Got it. Please take the following things in count during design:
> > > >
> > > > 1) Two distinct PMEs might not work well. Consider a situation w1hen
> I
> > > > decided to move a cache with index "MY_INDEX" from schema A to schema
> > B.
> > > > While cache was stopped, another cache with index "MY_INDEX" was
> > created
> > > in
> > > > schema B. Now first cache cannot start due to index name conflict.
> > > > 2) Cancelling index creation is also bad idea because this is
> > potentially
> > > > long operation. Instead, most likely that we should wait to
> concurrent
> > > > schema operations to finish first. That is, all operations on cache
> > > should
> > > > be ordered wrt each other somehow
> > > > 3) Why do we think that cache restart will be needed at all? We have
> a
> > > lot
> > > > of configuration properties which could be changed safely either
> > without
> > > > PME or with a single PME. - rebalance properties, cache store
> > properties
> > > > (especially write-behind stuff), some query properties (e.g. "detail
> > > > metrics"), etc.. In essence, it seems that >50% of properties could
> be
> > > > changed without cache restart, other 25% will not be supported, and
> the
> > > > rest may require restart.
> > > > 4) Client nodes and thin client may not have necessary classes in
> > > > classpath. E.g. consider a user which want to change rebalance
> timeout
> > > for
> > > > cache, but do not have configured interceptor in classpath. With
> > proposed
> > > > API it will be impossible. This is especially true for non-Java
> > clients.
> > > >
> > > > That said, I think we should consider another API which will not
> > require
> > > > full CacheConfiguration object. This might be a kind of builder or
> so.
> > > And
> > > > once user set properties he want to change to the builder, we can
> > analyze
> > > > them and either change them in runtime without PME, change with a
> > single
> > > > PME or change with full cache restart.
> > > >
> > > > What do you think?
> > > >
> > > > Vladimir.
> > > >
> > > > On Wed, Nov 21, 2018 at 5:01 PM Eduard Shangareev <
> > > > [hidden email]>
> > > > wrote:
> > > >
> > > > > Vladimir,
> > > > >
> > > > > 1) Affinity could be changed, but count of partition couldn't be.
> > > > > 2) So it would trigger 2 PME. Dynamic start and stop.
> > > > > 3) In theory, should cancel them and new setting should be applied.
> > How
> > > > it
> > > > > works now? Create an index and stop node, for example.
> > > > >
> > > > > On Wed, Nov 21, 2018 at 4:56 PM Vladimir Ozerov <
> > [hidden email]>
> > > > > wrote:
> > > > >
> > > > > > Hi Ed,
> > > > > >
> > > > > > Several questions from my side:
> > > > > > 1) If we do not allow to change the most demanded by users things
> > > like
> > > > > > affinity or persistence/in-memory, then what kind of
> configuration
> > > > > > properties do we expect to be changed? Can we have several
> > examples?
> > > > > > 2) How will it interact with PME?
> > > > > > 3) How will it interact with CREATE INDEX and ALTER TABLE
> commands?
> > > > > >
> > > > > > On Wed, Nov 21, 2018 at 4:48 PM Eduard Shangareev <
> > > > > > [hidden email]> wrote:
> > > > > >
> > > > > > > Igniters,
> > > > > > >
> > > > > > > I propose new public API to change the cache configuration of
> > > > > persistent
> > > > > > > caches with keeping data.
> > > > > > >
> > > > > > > It would look like:
> > > > > > >
> > > > > > > Ignite ignite = ...;
> > > > > > > ignite.restartCaches(cfg1, ... cfgN);
> > > > > > >
> > > > > > > where cfgX is a new cache configuration, which contains the
> name
> > of
> > > > an
> > > > > > > existing persistent cache.
> > > > > > >
> > > > > > > The obvious limitation:
> > > > > > > - affinity key mapping couldn't be changed;
> > > > > > > - count of partitions couldn't be changed;
> > > > > > > - MVCC couldn't be turned off/on;
> > > > > > > - persistent couldn't be turned off;
> > > > > > > - group settings couldn't be changed (group name);
> > > > > > > - if cache belongs to group it's needed to restart all of them.
> > > > > > >
> > > > > > > Failure scenario is the crucial thing (and most difficult):
> > > > > > > - initiator fail;
> > > > > > > - cluster restart at any stage;
> > > > > > > - joining/starting offline nodes.
> > > > > > >
> > > > > > > Some thoughts about implementation:
> > > > > > > - stop cache with destroy=false;
> > > > > > > - start cache dynamically with new configuration;
> > > > > > > - if indexes settings changed - remove index.bin to start
> > > indexation;
> > > > > > > - change blt-history when start cache initiated to not allow
> join
> > > > nodes
> > > > > > > with old configuration;
> > > > > > > - use restartId (IGNITE-8911) to not allow to start cache in
> > > between.
> > > > > > >
> > > > > > > Your thoughts? Would it be a useful feature?
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Eduard.
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: New API for changing configuration of persistent caches

Eduard Shangareev
Vladimir,

I don't see any difference here.

The same possibilities would be available as with normal cache start:
-XML;
-remote node.

>3) Avoid race condition when configuration changes between configuration
read and method call (what could lead to a number of strange effects).

Well, we could add *old* configuration parameter for CAS-like semantic.

On Wed, Nov 21, 2018 at 6:26 PM Vladimir Ozerov <[hidden email]>
wrote:

> Ed,
>
> Caches in .NET could be started programmatically, from XML which .NET API
> has no access to, or dynamically from remote nodes (eg Java node).
>
> ср, 21 нояб. 2018 г. в 18:24, Eduard Shangareev <
> [hidden email]
> >:
>
> > Vladimir,
> >
> > How does .Net user start caches right now?
> >
> > On Wed, Nov 21, 2018 at 6:10 PM Vladimir Ozerov <[hidden email]>
> > wrote:
> >
> > > Eduard,
> > >
> > > Simple != correct. Let’s consider a simple use case: user want to
> change
> > > PARTITIONED -> REPLICATED from .NET, but do not some classes from
> > > CacheConfiguration. How do we solve this?
> > >
> > > Vladimir.
> > >
> > > ср, 21 нояб. 2018 г. в 18:02, Eduard Shangareev <
> > > [hidden email]
> > > >:
> > >
> > > > Vladimir,
> > > >
> > > > I propose not to change cache configuration in runtime but restart
> > cache
> > > > with the new compatible configuration on data which we have
> underfoot.
> > > >
> > > > What we could change:
> > > > -backup count;
> > > > -TRANSACTIONAL <-> ATOMIC;
> > > > -REPLICATED - PARTITIONED;
> > > > -other settings.
> > > >
> > > > So, yeah, it would be great to have a possibility to change some
> > > properties
> > > > in runtime. But right we don't any way to change anything except
> > indexes
> > > > and SQL fields.
> > > >
> > > > We already have all mechanism to do this.
> > > > The main issue is to make it reliable and exclude cases when we could
> > > come
> > > > to the unrecoverable state.
> > > >
> > > > So, I suggest keeping the solution as simple as possible.
> > > > For indexes clashes and ClassNotFoundException we could revert
> > > > configuration update and start with the old configuration.
> > > >
> > > >
> > > > On Wed, Nov 21, 2018 at 5:44 PM Vladimir Ozerov <
> [hidden email]>
> > > > wrote:
> > > >
> > > > > Eduard,
> > > > >
> > > > > Got it. Please take the following things in count during design:
> > > > >
> > > > > 1) Two distinct PMEs might not work well. Consider a situation
> w1hen
> > I
> > > > > decided to move a cache with index "MY_INDEX" from schema A to
> schema
> > > B.
> > > > > While cache was stopped, another cache with index "MY_INDEX" was
> > > created
> > > > in
> > > > > schema B. Now first cache cannot start due to index name conflict.
> > > > > 2) Cancelling index creation is also bad idea because this is
> > > potentially
> > > > > long operation. Instead, most likely that we should wait to
> > concurrent
> > > > > schema operations to finish first. That is, all operations on cache
> > > > should
> > > > > be ordered wrt each other somehow
> > > > > 3) Why do we think that cache restart will be needed at all? We
> have
> > a
> > > > lot
> > > > > of configuration properties which could be changed safely either
> > > without
> > > > > PME or with a single PME. - rebalance properties, cache store
> > > properties
> > > > > (especially write-behind stuff), some query properties (e.g.
> "detail
> > > > > metrics"), etc.. In essence, it seems that >50% of properties could
> > be
> > > > > changed without cache restart, other 25% will not be supported, and
> > the
> > > > > rest may require restart.
> > > > > 4) Client nodes and thin client may not have necessary classes in
> > > > > classpath. E.g. consider a user which want to change rebalance
> > timeout
> > > > for
> > > > > cache, but do not have configured interceptor in classpath. With
> > > proposed
> > > > > API it will be impossible. This is especially true for non-Java
> > > clients.
> > > > >
> > > > > That said, I think we should consider another API which will not
> > > require
> > > > > full CacheConfiguration object. This might be a kind of builder or
> > so.
> > > > And
> > > > > once user set properties he want to change to the builder, we can
> > > analyze
> > > > > them and either change them in runtime without PME, change with a
> > > single
> > > > > PME or change with full cache restart.
> > > > >
> > > > > What do you think?
> > > > >
> > > > > Vladimir.
> > > > >
> > > > > On Wed, Nov 21, 2018 at 5:01 PM Eduard Shangareev <
> > > > > [hidden email]>
> > > > > wrote:
> > > > >
> > > > > > Vladimir,
> > > > > >
> > > > > > 1) Affinity could be changed, but count of partition couldn't be.
> > > > > > 2) So it would trigger 2 PME. Dynamic start and stop.
> > > > > > 3) In theory, should cancel them and new setting should be
> applied.
> > > How
> > > > > it
> > > > > > works now? Create an index and stop node, for example.
> > > > > >
> > > > > > On Wed, Nov 21, 2018 at 4:56 PM Vladimir Ozerov <
> > > [hidden email]>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Ed,
> > > > > > >
> > > > > > > Several questions from my side:
> > > > > > > 1) If we do not allow to change the most demanded by users
> things
> > > > like
> > > > > > > affinity or persistence/in-memory, then what kind of
> > configuration
> > > > > > > properties do we expect to be changed? Can we have several
> > > examples?
> > > > > > > 2) How will it interact with PME?
> > > > > > > 3) How will it interact with CREATE INDEX and ALTER TABLE
> > commands?
> > > > > > >
> > > > > > > On Wed, Nov 21, 2018 at 4:48 PM Eduard Shangareev <
> > > > > > > [hidden email]> wrote:
> > > > > > >
> > > > > > > > Igniters,
> > > > > > > >
> > > > > > > > I propose new public API to change the cache configuration of
> > > > > > persistent
> > > > > > > > caches with keeping data.
> > > > > > > >
> > > > > > > > It would look like:
> > > > > > > >
> > > > > > > > Ignite ignite = ...;
> > > > > > > > ignite.restartCaches(cfg1, ... cfgN);
> > > > > > > >
> > > > > > > > where cfgX is a new cache configuration, which contains the
> > name
> > > of
> > > > > an
> > > > > > > > existing persistent cache.
> > > > > > > >
> > > > > > > > The obvious limitation:
> > > > > > > > - affinity key mapping couldn't be changed;
> > > > > > > > - count of partitions couldn't be changed;
> > > > > > > > - MVCC couldn't be turned off/on;
> > > > > > > > - persistent couldn't be turned off;
> > > > > > > > - group settings couldn't be changed (group name);
> > > > > > > > - if cache belongs to group it's needed to restart all of
> them.
> > > > > > > >
> > > > > > > > Failure scenario is the crucial thing (and most difficult):
> > > > > > > > - initiator fail;
> > > > > > > > - cluster restart at any stage;
> > > > > > > > - joining/starting offline nodes.
> > > > > > > >
> > > > > > > > Some thoughts about implementation:
> > > > > > > > - stop cache with destroy=false;
> > > > > > > > - start cache dynamically with new configuration;
> > > > > > > > - if indexes settings changed - remove index.bin to start
> > > > indexation;
> > > > > > > > - change blt-history when start cache initiated to not allow
> > join
> > > > > nodes
> > > > > > > > with old configuration;
> > > > > > > > - use restartId (IGNITE-8911) to not allow to start cache in
> > > > between.
> > > > > > > >
> > > > > > > > Your thoughts? Would it be a useful feature?
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Best regards,
> > > > > > Eduard.
> > > > > >
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: New API for changing configuration of persistent caches

Vladimir Ozerov
Ed,

Could you please demonstrate how .NET node or .NET will change cache
configuration with proposed API? Taking in count that XML is not available
in most cases, and custom Java classes from cache configuration are
available only on server nodes and only from Java.

ср, 21 нояб. 2018 г. в 18:36, Eduard Shangareev <[hidden email]
>:

> Vladimir,
>
> I don't see any difference here.
>
> The same possibilities would be available as with normal cache start:
> -XML;
> -remote node.
>
> >3) Avoid race condition when configuration changes between configuration
> read and method call (what could lead to a number of strange effects).
>
> Well, we could add *old* configuration parameter for CAS-like semantic.
>
> On Wed, Nov 21, 2018 at 6:26 PM Vladimir Ozerov <[hidden email]>
> wrote:
>
> > Ed,
> >
> > Caches in .NET could be started programmatically, from XML which .NET API
> > has no access to, or dynamically from remote nodes (eg Java node).
> >
> > ср, 21 нояб. 2018 г. в 18:24, Eduard Shangareev <
> > [hidden email]
> > >:
> >
> > > Vladimir,
> > >
> > > How does .Net user start caches right now?
> > >
> > > On Wed, Nov 21, 2018 at 6:10 PM Vladimir Ozerov <[hidden email]>
> > > wrote:
> > >
> > > > Eduard,
> > > >
> > > > Simple != correct. Let’s consider a simple use case: user want to
> > change
> > > > PARTITIONED -> REPLICATED from .NET, but do not some classes from
> > > > CacheConfiguration. How do we solve this?
> > > >
> > > > Vladimir.
> > > >
> > > > ср, 21 нояб. 2018 г. в 18:02, Eduard Shangareev <
> > > > [hidden email]
> > > > >:
> > > >
> > > > > Vladimir,
> > > > >
> > > > > I propose not to change cache configuration in runtime but restart
> > > cache
> > > > > with the new compatible configuration on data which we have
> > underfoot.
> > > > >
> > > > > What we could change:
> > > > > -backup count;
> > > > > -TRANSACTIONAL <-> ATOMIC;
> > > > > -REPLICATED - PARTITIONED;
> > > > > -other settings.
> > > > >
> > > > > So, yeah, it would be great to have a possibility to change some
> > > > properties
> > > > > in runtime. But right we don't any way to change anything except
> > > indexes
> > > > > and SQL fields.
> > > > >
> > > > > We already have all mechanism to do this.
> > > > > The main issue is to make it reliable and exclude cases when we
> could
> > > > come
> > > > > to the unrecoverable state.
> > > > >
> > > > > So, I suggest keeping the solution as simple as possible.
> > > > > For indexes clashes and ClassNotFoundException we could revert
> > > > > configuration update and start with the old configuration.
> > > > >
> > > > >
> > > > > On Wed, Nov 21, 2018 at 5:44 PM Vladimir Ozerov <
> > [hidden email]>
> > > > > wrote:
> > > > >
> > > > > > Eduard,
> > > > > >
> > > > > > Got it. Please take the following things in count during design:
> > > > > >
> > > > > > 1) Two distinct PMEs might not work well. Consider a situation
> > w1hen
> > > I
> > > > > > decided to move a cache with index "MY_INDEX" from schema A to
> > schema
> > > > B.
> > > > > > While cache was stopped, another cache with index "MY_INDEX" was
> > > > created
> > > > > in
> > > > > > schema B. Now first cache cannot start due to index name
> conflict.
> > > > > > 2) Cancelling index creation is also bad idea because this is
> > > > potentially
> > > > > > long operation. Instead, most likely that we should wait to
> > > concurrent
> > > > > > schema operations to finish first. That is, all operations on
> cache
> > > > > should
> > > > > > be ordered wrt each other somehow
> > > > > > 3) Why do we think that cache restart will be needed at all? We
> > have
> > > a
> > > > > lot
> > > > > > of configuration properties which could be changed safely either
> > > > without
> > > > > > PME or with a single PME. - rebalance properties, cache store
> > > > properties
> > > > > > (especially write-behind stuff), some query properties (e.g.
> > "detail
> > > > > > metrics"), etc.. In essence, it seems that >50% of properties
> could
> > > be
> > > > > > changed without cache restart, other 25% will not be supported,
> and
> > > the
> > > > > > rest may require restart.
> > > > > > 4) Client nodes and thin client may not have necessary classes in
> > > > > > classpath. E.g. consider a user which want to change rebalance
> > > timeout
> > > > > for
> > > > > > cache, but do not have configured interceptor in classpath. With
> > > > proposed
> > > > > > API it will be impossible. This is especially true for non-Java
> > > > clients.
> > > > > >
> > > > > > That said, I think we should consider another API which will not
> > > > require
> > > > > > full CacheConfiguration object. This might be a kind of builder
> or
> > > so.
> > > > > And
> > > > > > once user set properties he want to change to the builder, we can
> > > > analyze
> > > > > > them and either change them in runtime without PME, change with a
> > > > single
> > > > > > PME or change with full cache restart.
> > > > > >
> > > > > > What do you think?
> > > > > >
> > > > > > Vladimir.
> > > > > >
> > > > > > On Wed, Nov 21, 2018 at 5:01 PM Eduard Shangareev <
> > > > > > [hidden email]>
> > > > > > wrote:
> > > > > >
> > > > > > > Vladimir,
> > > > > > >
> > > > > > > 1) Affinity could be changed, but count of partition couldn't
> be.
> > > > > > > 2) So it would trigger 2 PME. Dynamic start and stop.
> > > > > > > 3) In theory, should cancel them and new setting should be
> > applied.
> > > > How
> > > > > > it
> > > > > > > works now? Create an index and stop node, for example.
> > > > > > >
> > > > > > > On Wed, Nov 21, 2018 at 4:56 PM Vladimir Ozerov <
> > > > [hidden email]>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Ed,
> > > > > > > >
> > > > > > > > Several questions from my side:
> > > > > > > > 1) If we do not allow to change the most demanded by users
> > things
> > > > > like
> > > > > > > > affinity or persistence/in-memory, then what kind of
> > > configuration
> > > > > > > > properties do we expect to be changed? Can we have several
> > > > examples?
> > > > > > > > 2) How will it interact with PME?
> > > > > > > > 3) How will it interact with CREATE INDEX and ALTER TABLE
> > > commands?
> > > > > > > >
> > > > > > > > On Wed, Nov 21, 2018 at 4:48 PM Eduard Shangareev <
> > > > > > > > [hidden email]> wrote:
> > > > > > > >
> > > > > > > > > Igniters,
> > > > > > > > >
> > > > > > > > > I propose new public API to change the cache configuration
> of
> > > > > > > persistent
> > > > > > > > > caches with keeping data.
> > > > > > > > >
> > > > > > > > > It would look like:
> > > > > > > > >
> > > > > > > > > Ignite ignite = ...;
> > > > > > > > > ignite.restartCaches(cfg1, ... cfgN);
> > > > > > > > >
> > > > > > > > > where cfgX is a new cache configuration, which contains the
> > > name
> > > > of
> > > > > > an
> > > > > > > > > existing persistent cache.
> > > > > > > > >
> > > > > > > > > The obvious limitation:
> > > > > > > > > - affinity key mapping couldn't be changed;
> > > > > > > > > - count of partitions couldn't be changed;
> > > > > > > > > - MVCC couldn't be turned off/on;
> > > > > > > > > - persistent couldn't be turned off;
> > > > > > > > > - group settings couldn't be changed (group name);
> > > > > > > > > - if cache belongs to group it's needed to restart all of
> > them.
> > > > > > > > >
> > > > > > > > > Failure scenario is the crucial thing (and most difficult):
> > > > > > > > > - initiator fail;
> > > > > > > > > - cluster restart at any stage;
> > > > > > > > > - joining/starting offline nodes.
> > > > > > > > >
> > > > > > > > > Some thoughts about implementation:
> > > > > > > > > - stop cache with destroy=false;
> > > > > > > > > - start cache dynamically with new configuration;
> > > > > > > > > - if indexes settings changed - remove index.bin to start
> > > > > indexation;
> > > > > > > > > - change blt-history when start cache initiated to not
> allow
> > > join
> > > > > > nodes
> > > > > > > > > with old configuration;
> > > > > > > > > - use restartId (IGNITE-8911) to not allow to start cache
> in
> > > > > between.
> > > > > > > > >
> > > > > > > > > Your thoughts? Would it be a useful feature?
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Best regards,
> > > > > > > Eduard.
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: New API for changing configuration of persistent caches

Eduard Shangareev-2
Vladimir,

It would be Java API specific.
For a user, we would add a new command for console.sh which would take an
XML-file path as a parameter.

We could add other possibilities: for example, with the builder which would
finally call this Ignite.restartCaches method. But it's nice to have, not a
mandatory one.

On Wed, Nov 21, 2018 at 6:43 PM Vladimir Ozerov <[hidden email]>
wrote:

> Ed,
>
> Could you please demonstrate how .NET node or .NET will change cache
> configuration with proposed API? Taking in count that XML is not available
> in most cases, and custom Java classes from cache configuration are
> available only on server nodes and only from Java.
>
> ср, 21 нояб. 2018 г. в 18:36, Eduard Shangareev <
> [hidden email]
> >:
>
> > Vladimir,
> >
> > I don't see any difference here.
> >
> > The same possibilities would be available as with normal cache start:
> > -XML;
> > -remote node.
> >
> > >3) Avoid race condition when configuration changes between configuration
> > read and method call (what could lead to a number of strange effects).
> >
> > Well, we could add *old* configuration parameter for CAS-like semantic.
> >
> > On Wed, Nov 21, 2018 at 6:26 PM Vladimir Ozerov <[hidden email]>
> > wrote:
> >
> > > Ed,
> > >
> > > Caches in .NET could be started programmatically, from XML which .NET
> API
> > > has no access to, or dynamically from remote nodes (eg Java node).
> > >
> > > ср, 21 нояб. 2018 г. в 18:24, Eduard Shangareev <
> > > [hidden email]
> > > >:
> > >
> > > > Vladimir,
> > > >
> > > > How does .Net user start caches right now?
> > > >
> > > > On Wed, Nov 21, 2018 at 6:10 PM Vladimir Ozerov <
> [hidden email]>
> > > > wrote:
> > > >
> > > > > Eduard,
> > > > >
> > > > > Simple != correct. Let’s consider a simple use case: user want to
> > > change
> > > > > PARTITIONED -> REPLICATED from .NET, but do not some classes from
> > > > > CacheConfiguration. How do we solve this?
> > > > >
> > > > > Vladimir.
> > > > >
> > > > > ср, 21 нояб. 2018 г. в 18:02, Eduard Shangareev <
> > > > > [hidden email]
> > > > > >:
> > > > >
> > > > > > Vladimir,
> > > > > >
> > > > > > I propose not to change cache configuration in runtime but
> restart
> > > > cache
> > > > > > with the new compatible configuration on data which we have
> > > underfoot.
> > > > > >
> > > > > > What we could change:
> > > > > > -backup count;
> > > > > > -TRANSACTIONAL <-> ATOMIC;
> > > > > > -REPLICATED - PARTITIONED;
> > > > > > -other settings.
> > > > > >
> > > > > > So, yeah, it would be great to have a possibility to change some
> > > > > properties
> > > > > > in runtime. But right we don't any way to change anything except
> > > > indexes
> > > > > > and SQL fields.
> > > > > >
> > > > > > We already have all mechanism to do this.
> > > > > > The main issue is to make it reliable and exclude cases when we
> > could
> > > > > come
> > > > > > to the unrecoverable state.
> > > > > >
> > > > > > So, I suggest keeping the solution as simple as possible.
> > > > > > For indexes clashes and ClassNotFoundException we could revert
> > > > > > configuration update and start with the old configuration.
> > > > > >
> > > > > >
> > > > > > On Wed, Nov 21, 2018 at 5:44 PM Vladimir Ozerov <
> > > [hidden email]>
> > > > > > wrote:
> > > > > >
> > > > > > > Eduard,
> > > > > > >
> > > > > > > Got it. Please take the following things in count during
> design:
> > > > > > >
> > > > > > > 1) Two distinct PMEs might not work well. Consider a situation
> > > w1hen
> > > > I
> > > > > > > decided to move a cache with index "MY_INDEX" from schema A to
> > > schema
> > > > > B.
> > > > > > > While cache was stopped, another cache with index "MY_INDEX"
> was
> > > > > created
> > > > > > in
> > > > > > > schema B. Now first cache cannot start due to index name
> > conflict.
> > > > > > > 2) Cancelling index creation is also bad idea because this is
> > > > > potentially
> > > > > > > long operation. Instead, most likely that we should wait to
> > > > concurrent
> > > > > > > schema operations to finish first. That is, all operations on
> > cache
> > > > > > should
> > > > > > > be ordered wrt each other somehow
> > > > > > > 3) Why do we think that cache restart will be needed at all? We
> > > have
> > > > a
> > > > > > lot
> > > > > > > of configuration properties which could be changed safely
> either
> > > > > without
> > > > > > > PME or with a single PME. - rebalance properties, cache store
> > > > > properties
> > > > > > > (especially write-behind stuff), some query properties (e.g.
> > > "detail
> > > > > > > metrics"), etc.. In essence, it seems that >50% of properties
> > could
> > > > be
> > > > > > > changed without cache restart, other 25% will not be supported,
> > and
> > > > the
> > > > > > > rest may require restart.
> > > > > > > 4) Client nodes and thin client may not have necessary classes
> in
> > > > > > > classpath. E.g. consider a user which want to change rebalance
> > > > timeout
> > > > > > for
> > > > > > > cache, but do not have configured interceptor in classpath.
> With
> > > > > proposed
> > > > > > > API it will be impossible. This is especially true for non-Java
> > > > > clients.
> > > > > > >
> > > > > > > That said, I think we should consider another API which will
> not
> > > > > require
> > > > > > > full CacheConfiguration object. This might be a kind of builder
> > or
> > > > so.
> > > > > > And
> > > > > > > once user set properties he want to change to the builder, we
> can
> > > > > analyze
> > > > > > > them and either change them in runtime without PME, change
> with a
> > > > > single
> > > > > > > PME or change with full cache restart.
> > > > > > >
> > > > > > > What do you think?
> > > > > > >
> > > > > > > Vladimir.
> > > > > > >
> > > > > > > On Wed, Nov 21, 2018 at 5:01 PM Eduard Shangareev <
> > > > > > > [hidden email]>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Vladimir,
> > > > > > > >
> > > > > > > > 1) Affinity could be changed, but count of partition couldn't
> > be.
> > > > > > > > 2) So it would trigger 2 PME. Dynamic start and stop.
> > > > > > > > 3) In theory, should cancel them and new setting should be
> > > applied.
> > > > > How
> > > > > > > it
> > > > > > > > works now? Create an index and stop node, for example.
> > > > > > > >
> > > > > > > > On Wed, Nov 21, 2018 at 4:56 PM Vladimir Ozerov <
> > > > > [hidden email]>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi Ed,
> > > > > > > > >
> > > > > > > > > Several questions from my side:
> > > > > > > > > 1) If we do not allow to change the most demanded by users
> > > things
> > > > > > like
> > > > > > > > > affinity or persistence/in-memory, then what kind of
> > > > configuration
> > > > > > > > > properties do we expect to be changed? Can we have several
> > > > > examples?
> > > > > > > > > 2) How will it interact with PME?
> > > > > > > > > 3) How will it interact with CREATE INDEX and ALTER TABLE
> > > > commands?
> > > > > > > > >
> > > > > > > > > On Wed, Nov 21, 2018 at 4:48 PM Eduard Shangareev <
> > > > > > > > > [hidden email]> wrote:
> > > > > > > > >
> > > > > > > > > > Igniters,
> > > > > > > > > >
> > > > > > > > > > I propose new public API to change the cache
> configuration
> > of
> > > > > > > > persistent
> > > > > > > > > > caches with keeping data.
> > > > > > > > > >
> > > > > > > > > > It would look like:
> > > > > > > > > >
> > > > > > > > > > Ignite ignite = ...;
> > > > > > > > > > ignite.restartCaches(cfg1, ... cfgN);
> > > > > > > > > >
> > > > > > > > > > where cfgX is a new cache configuration, which contains
> the
> > > > name
> > > > > of
> > > > > > > an
> > > > > > > > > > existing persistent cache.
> > > > > > > > > >
> > > > > > > > > > The obvious limitation:
> > > > > > > > > > - affinity key mapping couldn't be changed;
> > > > > > > > > > - count of partitions couldn't be changed;
> > > > > > > > > > - MVCC couldn't be turned off/on;
> > > > > > > > > > - persistent couldn't be turned off;
> > > > > > > > > > - group settings couldn't be changed (group name);
> > > > > > > > > > - if cache belongs to group it's needed to restart all of
> > > them.
> > > > > > > > > >
> > > > > > > > > > Failure scenario is the crucial thing (and most
> difficult):
> > > > > > > > > > - initiator fail;
> > > > > > > > > > - cluster restart at any stage;
> > > > > > > > > > - joining/starting offline nodes.
> > > > > > > > > >
> > > > > > > > > > Some thoughts about implementation:
> > > > > > > > > > - stop cache with destroy=false;
> > > > > > > > > > - start cache dynamically with new configuration;
> > > > > > > > > > - if indexes settings changed - remove index.bin to start
> > > > > > indexation;
> > > > > > > > > > - change blt-history when start cache initiated to not
> > allow
> > > > join
> > > > > > > nodes
> > > > > > > > > > with old configuration;
> > > > > > > > > > - use restartId (IGNITE-8911) to not allow to start cache
> > in
> > > > > > between.
> > > > > > > > > >
> > > > > > > > > > Your thoughts? Would it be a useful feature?
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Best regards,
> > > > > > > > Eduard.
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>


--
Best regards,
Eduard.
Reply | Threaded
Open this post in threaded view
|

Re: New API for changing configuration of persistent caches

Vladimir Ozerov
Ed,

We are not Java product. We support 6 platforms at the moment. Why do we
implement a feature which can only be used in Java, when it is very easy to
make it available from everywhere?

ср, 21 нояб. 2018 г. в 18:50, Eduard Shangareev <[hidden email]>:

> Vladimir,
>
> It would be Java API specific.
> For a user, we would add a new command for console.sh which would take an
> XML-file path as a parameter.
>
> We could add other possibilities: for example, with the builder which would
> finally call this Ignite.restartCaches method. But it's nice to have, not a
> mandatory one.
>
> On Wed, Nov 21, 2018 at 6:43 PM Vladimir Ozerov <[hidden email]>
> wrote:
>
> > Ed,
> >
> > Could you please demonstrate how .NET node or .NET will change cache
> > configuration with proposed API? Taking in count that XML is not
> available
> > in most cases, and custom Java classes from cache configuration are
> > available only on server nodes and only from Java.
> >
> > ср, 21 нояб. 2018 г. в 18:36, Eduard Shangareev <
> > [hidden email]
> > >:
> >
> > > Vladimir,
> > >
> > > I don't see any difference here.
> > >
> > > The same possibilities would be available as with normal cache start:
> > > -XML;
> > > -remote node.
> > >
> > > >3) Avoid race condition when configuration changes between
> configuration
> > > read and method call (what could lead to a number of strange effects).
> > >
> > > Well, we could add *old* configuration parameter for CAS-like semantic.
> > >
> > > On Wed, Nov 21, 2018 at 6:26 PM Vladimir Ozerov <[hidden email]>
> > > wrote:
> > >
> > > > Ed,
> > > >
> > > > Caches in .NET could be started programmatically, from XML which .NET
> > API
> > > > has no access to, or dynamically from remote nodes (eg Java node).
> > > >
> > > > ср, 21 нояб. 2018 г. в 18:24, Eduard Shangareev <
> > > > [hidden email]
> > > > >:
> > > >
> > > > > Vladimir,
> > > > >
> > > > > How does .Net user start caches right now?
> > > > >
> > > > > On Wed, Nov 21, 2018 at 6:10 PM Vladimir Ozerov <
> > [hidden email]>
> > > > > wrote:
> > > > >
> > > > > > Eduard,
> > > > > >
> > > > > > Simple != correct. Let’s consider a simple use case: user want to
> > > > change
> > > > > > PARTITIONED -> REPLICATED from .NET, but do not some classes from
> > > > > > CacheConfiguration. How do we solve this?
> > > > > >
> > > > > > Vladimir.
> > > > > >
> > > > > > ср, 21 нояб. 2018 г. в 18:02, Eduard Shangareev <
> > > > > > [hidden email]
> > > > > > >:
> > > > > >
> > > > > > > Vladimir,
> > > > > > >
> > > > > > > I propose not to change cache configuration in runtime but
> > restart
> > > > > cache
> > > > > > > with the new compatible configuration on data which we have
> > > > underfoot.
> > > > > > >
> > > > > > > What we could change:
> > > > > > > -backup count;
> > > > > > > -TRANSACTIONAL <-> ATOMIC;
> > > > > > > -REPLICATED - PARTITIONED;
> > > > > > > -other settings.
> > > > > > >
> > > > > > > So, yeah, it would be great to have a possibility to change
> some
> > > > > > properties
> > > > > > > in runtime. But right we don't any way to change anything
> except
> > > > > indexes
> > > > > > > and SQL fields.
> > > > > > >
> > > > > > > We already have all mechanism to do this.
> > > > > > > The main issue is to make it reliable and exclude cases when we
> > > could
> > > > > > come
> > > > > > > to the unrecoverable state.
> > > > > > >
> > > > > > > So, I suggest keeping the solution as simple as possible.
> > > > > > > For indexes clashes and ClassNotFoundException we could revert
> > > > > > > configuration update and start with the old configuration.
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Nov 21, 2018 at 5:44 PM Vladimir Ozerov <
> > > > [hidden email]>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Eduard,
> > > > > > > >
> > > > > > > > Got it. Please take the following things in count during
> > design:
> > > > > > > >
> > > > > > > > 1) Two distinct PMEs might not work well. Consider a
> situation
> > > > w1hen
> > > > > I
> > > > > > > > decided to move a cache with index "MY_INDEX" from schema A
> to
> > > > schema
> > > > > > B.
> > > > > > > > While cache was stopped, another cache with index "MY_INDEX"
> > was
> > > > > > created
> > > > > > > in
> > > > > > > > schema B. Now first cache cannot start due to index name
> > > conflict.
> > > > > > > > 2) Cancelling index creation is also bad idea because this is
> > > > > > potentially
> > > > > > > > long operation. Instead, most likely that we should wait to
> > > > > concurrent
> > > > > > > > schema operations to finish first. That is, all operations on
> > > cache
> > > > > > > should
> > > > > > > > be ordered wrt each other somehow
> > > > > > > > 3) Why do we think that cache restart will be needed at all?
> We
> > > > have
> > > > > a
> > > > > > > lot
> > > > > > > > of configuration properties which could be changed safely
> > either
> > > > > > without
> > > > > > > > PME or with a single PME. - rebalance properties, cache store
> > > > > > properties
> > > > > > > > (especially write-behind stuff), some query properties (e.g.
> > > > "detail
> > > > > > > > metrics"), etc.. In essence, it seems that >50% of properties
> > > could
> > > > > be
> > > > > > > > changed without cache restart, other 25% will not be
> supported,
> > > and
> > > > > the
> > > > > > > > rest may require restart.
> > > > > > > > 4) Client nodes and thin client may not have necessary
> classes
> > in
> > > > > > > > classpath. E.g. consider a user which want to change
> rebalance
> > > > > timeout
> > > > > > > for
> > > > > > > > cache, but do not have configured interceptor in classpath.
> > With
> > > > > > proposed
> > > > > > > > API it will be impossible. This is especially true for
> non-Java
> > > > > > clients.
> > > > > > > >
> > > > > > > > That said, I think we should consider another API which will
> > not
> > > > > > require
> > > > > > > > full CacheConfiguration object. This might be a kind of
> builder
> > > or
> > > > > so.
> > > > > > > And
> > > > > > > > once user set properties he want to change to the builder, we
> > can
> > > > > > analyze
> > > > > > > > them and either change them in runtime without PME, change
> > with a
> > > > > > single
> > > > > > > > PME or change with full cache restart.
> > > > > > > >
> > > > > > > > What do you think?
> > > > > > > >
> > > > > > > > Vladimir.
> > > > > > > >
> > > > > > > > On Wed, Nov 21, 2018 at 5:01 PM Eduard Shangareev <
> > > > > > > > [hidden email]>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Vladimir,
> > > > > > > > >
> > > > > > > > > 1) Affinity could be changed, but count of partition
> couldn't
> > > be.
> > > > > > > > > 2) So it would trigger 2 PME. Dynamic start and stop.
> > > > > > > > > 3) In theory, should cancel them and new setting should be
> > > > applied.
> > > > > > How
> > > > > > > > it
> > > > > > > > > works now? Create an index and stop node, for example.
> > > > > > > > >
> > > > > > > > > On Wed, Nov 21, 2018 at 4:56 PM Vladimir Ozerov <
> > > > > > [hidden email]>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi Ed,
> > > > > > > > > >
> > > > > > > > > > Several questions from my side:
> > > > > > > > > > 1) If we do not allow to change the most demanded by
> users
> > > > things
> > > > > > > like
> > > > > > > > > > affinity or persistence/in-memory, then what kind of
> > > > > configuration
> > > > > > > > > > properties do we expect to be changed? Can we have
> several
> > > > > > examples?
> > > > > > > > > > 2) How will it interact with PME?
> > > > > > > > > > 3) How will it interact with CREATE INDEX and ALTER TABLE
> > > > > commands?
> > > > > > > > > >
> > > > > > > > > > On Wed, Nov 21, 2018 at 4:48 PM Eduard Shangareev <
> > > > > > > > > > [hidden email]> wrote:
> > > > > > > > > >
> > > > > > > > > > > Igniters,
> > > > > > > > > > >
> > > > > > > > > > > I propose new public API to change the cache
> > configuration
> > > of
> > > > > > > > > persistent
> > > > > > > > > > > caches with keeping data.
> > > > > > > > > > >
> > > > > > > > > > > It would look like:
> > > > > > > > > > >
> > > > > > > > > > > Ignite ignite = ...;
> > > > > > > > > > > ignite.restartCaches(cfg1, ... cfgN);
> > > > > > > > > > >
> > > > > > > > > > > where cfgX is a new cache configuration, which contains
> > the
> > > > > name
> > > > > > of
> > > > > > > > an
> > > > > > > > > > > existing persistent cache.
> > > > > > > > > > >
> > > > > > > > > > > The obvious limitation:
> > > > > > > > > > > - affinity key mapping couldn't be changed;
> > > > > > > > > > > - count of partitions couldn't be changed;
> > > > > > > > > > > - MVCC couldn't be turned off/on;
> > > > > > > > > > > - persistent couldn't be turned off;
> > > > > > > > > > > - group settings couldn't be changed (group name);
> > > > > > > > > > > - if cache belongs to group it's needed to restart all
> of
> > > > them.
> > > > > > > > > > >
> > > > > > > > > > > Failure scenario is the crucial thing (and most
> > difficult):
> > > > > > > > > > > - initiator fail;
> > > > > > > > > > > - cluster restart at any stage;
> > > > > > > > > > > - joining/starting offline nodes.
> > > > > > > > > > >
> > > > > > > > > > > Some thoughts about implementation:
> > > > > > > > > > > - stop cache with destroy=false;
> > > > > > > > > > > - start cache dynamically with new configuration;
> > > > > > > > > > > - if indexes settings changed - remove index.bin to
> start
> > > > > > > indexation;
> > > > > > > > > > > - change blt-history when start cache initiated to not
> > > allow
> > > > > join
> > > > > > > > nodes
> > > > > > > > > > > with old configuration;
> > > > > > > > > > > - use restartId (IGNITE-8911) to not allow to start
> cache
> > > in
> > > > > > > between.
> > > > > > > > > > >
> > > > > > > > > > > Your thoughts? Would it be a useful feature?
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Best regards,
> > > > > > > > > Eduard.
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
>
> --
> Best regards,
> Eduard.
>
Reply | Threaded
Open this post in threaded view
|

Re: New API for changing configuration of persistent caches

Eduard Shangareev-2
Vladimir,

I didn't get how does .Net user start caches right now? XML and remote
node? Right?


On Wed, Nov 21, 2018 at 6:55 PM Vladimir Ozerov <[hidden email]>
wrote:

> Ed,
>
> We are not Java product. We support 6 platforms at the moment. Why do we
> implement a feature which can only be used in Java, when it is very easy to
> make it available from everywhere?
>
> ср, 21 нояб. 2018 г. в 18:50, Eduard Shangareev <[hidden email]
> >:
>
> > Vladimir,
> >
> > It would be Java API specific.
> > For a user, we would add a new command for console.sh which would take an
> > XML-file path as a parameter.
> >
> > We could add other possibilities: for example, with the builder which
> would
> > finally call this Ignite.restartCaches method. But it's nice to have,
> not a
> > mandatory one.
> >
> > On Wed, Nov 21, 2018 at 6:43 PM Vladimir Ozerov <[hidden email]>
> > wrote:
> >
> > > Ed,
> > >
> > > Could you please demonstrate how .NET node or .NET will change cache
> > > configuration with proposed API? Taking in count that XML is not
> > available
> > > in most cases, and custom Java classes from cache configuration are
> > > available only on server nodes and only from Java.
> > >
> > > ср, 21 нояб. 2018 г. в 18:36, Eduard Shangareev <
> > > [hidden email]
> > > >:
> > >
> > > > Vladimir,
> > > >
> > > > I don't see any difference here.
> > > >
> > > > The same possibilities would be available as with normal cache start:
> > > > -XML;
> > > > -remote node.
> > > >
> > > > >3) Avoid race condition when configuration changes between
> > configuration
> > > > read and method call (what could lead to a number of strange
> effects).
> > > >
> > > > Well, we could add *old* configuration parameter for CAS-like
> semantic.
> > > >
> > > > On Wed, Nov 21, 2018 at 6:26 PM Vladimir Ozerov <
> [hidden email]>
> > > > wrote:
> > > >
> > > > > Ed,
> > > > >
> > > > > Caches in .NET could be started programmatically, from XML which
> .NET
> > > API
> > > > > has no access to, or dynamically from remote nodes (eg Java node).
> > > > >
> > > > > ср, 21 нояб. 2018 г. в 18:24, Eduard Shangareev <
> > > > > [hidden email]
> > > > > >:
> > > > >
> > > > > > Vladimir,
> > > > > >
> > > > > > How does .Net user start caches right now?
> > > > > >
> > > > > > On Wed, Nov 21, 2018 at 6:10 PM Vladimir Ozerov <
> > > [hidden email]>
> > > > > > wrote:
> > > > > >
> > > > > > > Eduard,
> > > > > > >
> > > > > > > Simple != correct. Let’s consider a simple use case: user want
> to
> > > > > change
> > > > > > > PARTITIONED -> REPLICATED from .NET, but do not some classes
> from
> > > > > > > CacheConfiguration. How do we solve this?
> > > > > > >
> > > > > > > Vladimir.
> > > > > > >
> > > > > > > ср, 21 нояб. 2018 г. в 18:02, Eduard Shangareev <
> > > > > > > [hidden email]
> > > > > > > >:
> > > > > > >
> > > > > > > > Vladimir,
> > > > > > > >
> > > > > > > > I propose not to change cache configuration in runtime but
> > > restart
> > > > > > cache
> > > > > > > > with the new compatible configuration on data which we have
> > > > > underfoot.
> > > > > > > >
> > > > > > > > What we could change:
> > > > > > > > -backup count;
> > > > > > > > -TRANSACTIONAL <-> ATOMIC;
> > > > > > > > -REPLICATED - PARTITIONED;
> > > > > > > > -other settings.
> > > > > > > >
> > > > > > > > So, yeah, it would be great to have a possibility to change
> > some
> > > > > > > properties
> > > > > > > > in runtime. But right we don't any way to change anything
> > except
> > > > > > indexes
> > > > > > > > and SQL fields.
> > > > > > > >
> > > > > > > > We already have all mechanism to do this.
> > > > > > > > The main issue is to make it reliable and exclude cases when
> we
> > > > could
> > > > > > > come
> > > > > > > > to the unrecoverable state.
> > > > > > > >
> > > > > > > > So, I suggest keeping the solution as simple as possible.
> > > > > > > > For indexes clashes and ClassNotFoundException we could
> revert
> > > > > > > > configuration update and start with the old configuration.
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Nov 21, 2018 at 5:44 PM Vladimir Ozerov <
> > > > > [hidden email]>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Eduard,
> > > > > > > > >
> > > > > > > > > Got it. Please take the following things in count during
> > > design:
> > > > > > > > >
> > > > > > > > > 1) Two distinct PMEs might not work well. Consider a
> > situation
> > > > > w1hen
> > > > > > I
> > > > > > > > > decided to move a cache with index "MY_INDEX" from schema A
> > to
> > > > > schema
> > > > > > > B.
> > > > > > > > > While cache was stopped, another cache with index
> "MY_INDEX"
> > > was
> > > > > > > created
> > > > > > > > in
> > > > > > > > > schema B. Now first cache cannot start due to index name
> > > > conflict.
> > > > > > > > > 2) Cancelling index creation is also bad idea because this
> is
> > > > > > > potentially
> > > > > > > > > long operation. Instead, most likely that we should wait to
> > > > > > concurrent
> > > > > > > > > schema operations to finish first. That is, all operations
> on
> > > > cache
> > > > > > > > should
> > > > > > > > > be ordered wrt each other somehow
> > > > > > > > > 3) Why do we think that cache restart will be needed at
> all?
> > We
> > > > > have
> > > > > > a
> > > > > > > > lot
> > > > > > > > > of configuration properties which could be changed safely
> > > either
> > > > > > > without
> > > > > > > > > PME or with a single PME. - rebalance properties, cache
> store
> > > > > > > properties
> > > > > > > > > (especially write-behind stuff), some query properties
> (e.g.
> > > > > "detail
> > > > > > > > > metrics"), etc.. In essence, it seems that >50% of
> properties
> > > > could
> > > > > > be
> > > > > > > > > changed without cache restart, other 25% will not be
> > supported,
> > > > and
> > > > > > the
> > > > > > > > > rest may require restart.
> > > > > > > > > 4) Client nodes and thin client may not have necessary
> > classes
> > > in
> > > > > > > > > classpath. E.g. consider a user which want to change
> > rebalance
> > > > > > timeout
> > > > > > > > for
> > > > > > > > > cache, but do not have configured interceptor in classpath.
> > > With
> > > > > > > proposed
> > > > > > > > > API it will be impossible. This is especially true for
> > non-Java
> > > > > > > clients.
> > > > > > > > >
> > > > > > > > > That said, I think we should consider another API which
> will
> > > not
> > > > > > > require
> > > > > > > > > full CacheConfiguration object. This might be a kind of
> > builder
> > > > or
> > > > > > so.
> > > > > > > > And
> > > > > > > > > once user set properties he want to change to the builder,
> we
> > > can
> > > > > > > analyze
> > > > > > > > > them and either change them in runtime without PME, change
> > > with a
> > > > > > > single
> > > > > > > > > PME or change with full cache restart.
> > > > > > > > >
> > > > > > > > > What do you think?
> > > > > > > > >
> > > > > > > > > Vladimir.
> > > > > > > > >
> > > > > > > > > On Wed, Nov 21, 2018 at 5:01 PM Eduard Shangareev <
> > > > > > > > > [hidden email]>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Vladimir,
> > > > > > > > > >
> > > > > > > > > > 1) Affinity could be changed, but count of partition
> > couldn't
> > > > be.
> > > > > > > > > > 2) So it would trigger 2 PME. Dynamic start and stop.
> > > > > > > > > > 3) In theory, should cancel them and new setting should
> be
> > > > > applied.
> > > > > > > How
> > > > > > > > > it
> > > > > > > > > > works now? Create an index and stop node, for example.
> > > > > > > > > >
> > > > > > > > > > On Wed, Nov 21, 2018 at 4:56 PM Vladimir Ozerov <
> > > > > > > [hidden email]>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hi Ed,
> > > > > > > > > > >
> > > > > > > > > > > Several questions from my side:
> > > > > > > > > > > 1) If we do not allow to change the most demanded by
> > users
> > > > > things
> > > > > > > > like
> > > > > > > > > > > affinity or persistence/in-memory, then what kind of
> > > > > > configuration
> > > > > > > > > > > properties do we expect to be changed? Can we have
> > several
> > > > > > > examples?
> > > > > > > > > > > 2) How will it interact with PME?
> > > > > > > > > > > 3) How will it interact with CREATE INDEX and ALTER
> TABLE
> > > > > > commands?
> > > > > > > > > > >
> > > > > > > > > > > On Wed, Nov 21, 2018 at 4:48 PM Eduard Shangareev <
> > > > > > > > > > > [hidden email]> wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Igniters,
> > > > > > > > > > > >
> > > > > > > > > > > > I propose new public API to change the cache
> > > configuration
> > > > of
> > > > > > > > > > persistent
> > > > > > > > > > > > caches with keeping data.
> > > > > > > > > > > >
> > > > > > > > > > > > It would look like:
> > > > > > > > > > > >
> > > > > > > > > > > > Ignite ignite = ...;
> > > > > > > > > > > > ignite.restartCaches(cfg1, ... cfgN);
> > > > > > > > > > > >
> > > > > > > > > > > > where cfgX is a new cache configuration, which
> contains
> > > the
> > > > > > name
> > > > > > > of
> > > > > > > > > an
> > > > > > > > > > > > existing persistent cache.
> > > > > > > > > > > >
> > > > > > > > > > > > The obvious limitation:
> > > > > > > > > > > > - affinity key mapping couldn't be changed;
> > > > > > > > > > > > - count of partitions couldn't be changed;
> > > > > > > > > > > > - MVCC couldn't be turned off/on;
> > > > > > > > > > > > - persistent couldn't be turned off;
> > > > > > > > > > > > - group settings couldn't be changed (group name);
> > > > > > > > > > > > - if cache belongs to group it's needed to restart
> all
> > of
> > > > > them.
> > > > > > > > > > > >
> > > > > > > > > > > > Failure scenario is the crucial thing (and most
> > > difficult):
> > > > > > > > > > > > - initiator fail;
> > > > > > > > > > > > - cluster restart at any stage;
> > > > > > > > > > > > - joining/starting offline nodes.
> > > > > > > > > > > >
> > > > > > > > > > > > Some thoughts about implementation:
> > > > > > > > > > > > - stop cache with destroy=false;
> > > > > > > > > > > > - start cache dynamically with new configuration;
> > > > > > > > > > > > - if indexes settings changed - remove index.bin to
> > start
> > > > > > > > indexation;
> > > > > > > > > > > > - change blt-history when start cache initiated to
> not
> > > > allow
> > > > > > join
> > > > > > > > > nodes
> > > > > > > > > > > > with old configuration;
> > > > > > > > > > > > - use restartId (IGNITE-8911) to not allow to start
> > cache
> > > > in
> > > > > > > > between.
> > > > > > > > > > > >
> > > > > > > > > > > > Your thoughts? Would it be a useful feature?
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > Best regards,
> > > > > > > > > > Eduard.
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> >
> > --
> > Best regards,
> > Eduard.
> >
>


--
Best regards,
Eduard.
Reply | Threaded
Open this post in threaded view
|

Re: New API for changing configuration of persistent caches

Vladimir Ozerov
Ed,

He can do that programmatically. But I meant another case - Java node
creates a cache. Then .NET node wants to change it. Proposed API cannot
handle it.

ср, 21 нояб. 2018 г. в 19:03, Eduard Shangareev <[hidden email]>:

> Vladimir,
>
> I didn't get how does .Net user start caches right now? XML and remote
> node? Right?
>
>
> On Wed, Nov 21, 2018 at 6:55 PM Vladimir Ozerov <[hidden email]>
> wrote:
>
> > Ed,
> >
> > We are not Java product. We support 6 platforms at the moment. Why do we
> > implement a feature which can only be used in Java, when it is very easy
> to
> > make it available from everywhere?
> >
> > ср, 21 нояб. 2018 г. в 18:50, Eduard Shangareev <
> [hidden email]
> > >:
> >
> > > Vladimir,
> > >
> > > It would be Java API specific.
> > > For a user, we would add a new command for console.sh which would take
> an
> > > XML-file path as a parameter.
> > >
> > > We could add other possibilities: for example, with the builder which
> > would
> > > finally call this Ignite.restartCaches method. But it's nice to have,
> > not a
> > > mandatory one.
> > >
> > > On Wed, Nov 21, 2018 at 6:43 PM Vladimir Ozerov <[hidden email]>
> > > wrote:
> > >
> > > > Ed,
> > > >
> > > > Could you please demonstrate how .NET node or .NET will change cache
> > > > configuration with proposed API? Taking in count that XML is not
> > > available
> > > > in most cases, and custom Java classes from cache configuration are
> > > > available only on server nodes and only from Java.
> > > >
> > > > ср, 21 нояб. 2018 г. в 18:36, Eduard Shangareev <
> > > > [hidden email]
> > > > >:
> > > >
> > > > > Vladimir,
> > > > >
> > > > > I don't see any difference here.
> > > > >
> > > > > The same possibilities would be available as with normal cache
> start:
> > > > > -XML;
> > > > > -remote node.
> > > > >
> > > > > >3) Avoid race condition when configuration changes between
> > > configuration
> > > > > read and method call (what could lead to a number of strange
> > effects).
> > > > >
> > > > > Well, we could add *old* configuration parameter for CAS-like
> > semantic.
> > > > >
> > > > > On Wed, Nov 21, 2018 at 6:26 PM Vladimir Ozerov <
> > [hidden email]>
> > > > > wrote:
> > > > >
> > > > > > Ed,
> > > > > >
> > > > > > Caches in .NET could be started programmatically, from XML which
> > .NET
> > > > API
> > > > > > has no access to, or dynamically from remote nodes (eg Java
> node).
> > > > > >
> > > > > > ср, 21 нояб. 2018 г. в 18:24, Eduard Shangareev <
> > > > > > [hidden email]
> > > > > > >:
> > > > > >
> > > > > > > Vladimir,
> > > > > > >
> > > > > > > How does .Net user start caches right now?
> > > > > > >
> > > > > > > On Wed, Nov 21, 2018 at 6:10 PM Vladimir Ozerov <
> > > > [hidden email]>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Eduard,
> > > > > > > >
> > > > > > > > Simple != correct. Let’s consider a simple use case: user
> want
> > to
> > > > > > change
> > > > > > > > PARTITIONED -> REPLICATED from .NET, but do not some classes
> > from
> > > > > > > > CacheConfiguration. How do we solve this?
> > > > > > > >
> > > > > > > > Vladimir.
> > > > > > > >
> > > > > > > > ср, 21 нояб. 2018 г. в 18:02, Eduard Shangareev <
> > > > > > > > [hidden email]
> > > > > > > > >:
> > > > > > > >
> > > > > > > > > Vladimir,
> > > > > > > > >
> > > > > > > > > I propose not to change cache configuration in runtime but
> > > > restart
> > > > > > > cache
> > > > > > > > > with the new compatible configuration on data which we have
> > > > > > underfoot.
> > > > > > > > >
> > > > > > > > > What we could change:
> > > > > > > > > -backup count;
> > > > > > > > > -TRANSACTIONAL <-> ATOMIC;
> > > > > > > > > -REPLICATED - PARTITIONED;
> > > > > > > > > -other settings.
> > > > > > > > >
> > > > > > > > > So, yeah, it would be great to have a possibility to change
> > > some
> > > > > > > > properties
> > > > > > > > > in runtime. But right we don't any way to change anything
> > > except
> > > > > > > indexes
> > > > > > > > > and SQL fields.
> > > > > > > > >
> > > > > > > > > We already have all mechanism to do this.
> > > > > > > > > The main issue is to make it reliable and exclude cases
> when
> > we
> > > > > could
> > > > > > > > come
> > > > > > > > > to the unrecoverable state.
> > > > > > > > >
> > > > > > > > > So, I suggest keeping the solution as simple as possible.
> > > > > > > > > For indexes clashes and ClassNotFoundException we could
> > revert
> > > > > > > > > configuration update and start with the old configuration.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Wed, Nov 21, 2018 at 5:44 PM Vladimir Ozerov <
> > > > > > [hidden email]>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Eduard,
> > > > > > > > > >
> > > > > > > > > > Got it. Please take the following things in count during
> > > > design:
> > > > > > > > > >
> > > > > > > > > > 1) Two distinct PMEs might not work well. Consider a
> > > situation
> > > > > > w1hen
> > > > > > > I
> > > > > > > > > > decided to move a cache with index "MY_INDEX" from
> schema A
> > > to
> > > > > > schema
> > > > > > > > B.
> > > > > > > > > > While cache was stopped, another cache with index
> > "MY_INDEX"
> > > > was
> > > > > > > > created
> > > > > > > > > in
> > > > > > > > > > schema B. Now first cache cannot start due to index name
> > > > > conflict.
> > > > > > > > > > 2) Cancelling index creation is also bad idea because
> this
> > is
> > > > > > > > potentially
> > > > > > > > > > long operation. Instead, most likely that we should wait
> to
> > > > > > > concurrent
> > > > > > > > > > schema operations to finish first. That is, all
> operations
> > on
> > > > > cache
> > > > > > > > > should
> > > > > > > > > > be ordered wrt each other somehow
> > > > > > > > > > 3) Why do we think that cache restart will be needed at
> > all?
> > > We
> > > > > > have
> > > > > > > a
> > > > > > > > > lot
> > > > > > > > > > of configuration properties which could be changed safely
> > > > either
> > > > > > > > without
> > > > > > > > > > PME or with a single PME. - rebalance properties, cache
> > store
> > > > > > > > properties
> > > > > > > > > > (especially write-behind stuff), some query properties
> > (e.g.
> > > > > > "detail
> > > > > > > > > > metrics"), etc.. In essence, it seems that >50% of
> > properties
> > > > > could
> > > > > > > be
> > > > > > > > > > changed without cache restart, other 25% will not be
> > > supported,
> > > > > and
> > > > > > > the
> > > > > > > > > > rest may require restart.
> > > > > > > > > > 4) Client nodes and thin client may not have necessary
> > > classes
> > > > in
> > > > > > > > > > classpath. E.g. consider a user which want to change
> > > rebalance
> > > > > > > timeout
> > > > > > > > > for
> > > > > > > > > > cache, but do not have configured interceptor in
> classpath.
> > > > With
> > > > > > > > proposed
> > > > > > > > > > API it will be impossible. This is especially true for
> > > non-Java
> > > > > > > > clients.
> > > > > > > > > >
> > > > > > > > > > That said, I think we should consider another API which
> > will
> > > > not
> > > > > > > > require
> > > > > > > > > > full CacheConfiguration object. This might be a kind of
> > > builder
> > > > > or
> > > > > > > so.
> > > > > > > > > And
> > > > > > > > > > once user set properties he want to change to the
> builder,
> > we
> > > > can
> > > > > > > > analyze
> > > > > > > > > > them and either change them in runtime without PME,
> change
> > > > with a
> > > > > > > > single
> > > > > > > > > > PME or change with full cache restart.
> > > > > > > > > >
> > > > > > > > > > What do you think?
> > > > > > > > > >
> > > > > > > > > > Vladimir.
> > > > > > > > > >
> > > > > > > > > > On Wed, Nov 21, 2018 at 5:01 PM Eduard Shangareev <
> > > > > > > > > > [hidden email]>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Vladimir,
> > > > > > > > > > >
> > > > > > > > > > > 1) Affinity could be changed, but count of partition
> > > couldn't
> > > > > be.
> > > > > > > > > > > 2) So it would trigger 2 PME. Dynamic start and stop.
> > > > > > > > > > > 3) In theory, should cancel them and new setting should
> > be
> > > > > > applied.
> > > > > > > > How
> > > > > > > > > > it
> > > > > > > > > > > works now? Create an index and stop node, for example.
> > > > > > > > > > >
> > > > > > > > > > > On Wed, Nov 21, 2018 at 4:56 PM Vladimir Ozerov <
> > > > > > > > [hidden email]>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hi Ed,
> > > > > > > > > > > >
> > > > > > > > > > > > Several questions from my side:
> > > > > > > > > > > > 1) If we do not allow to change the most demanded by
> > > users
> > > > > > things
> > > > > > > > > like
> > > > > > > > > > > > affinity or persistence/in-memory, then what kind of
> > > > > > > configuration
> > > > > > > > > > > > properties do we expect to be changed? Can we have
> > > several
> > > > > > > > examples?
> > > > > > > > > > > > 2) How will it interact with PME?
> > > > > > > > > > > > 3) How will it interact with CREATE INDEX and ALTER
> > TABLE
> > > > > > > commands?
> > > > > > > > > > > >
> > > > > > > > > > > > On Wed, Nov 21, 2018 at 4:48 PM Eduard Shangareev <
> > > > > > > > > > > > [hidden email]> wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Igniters,
> > > > > > > > > > > > >
> > > > > > > > > > > > > I propose new public API to change the cache
> > > > configuration
> > > > > of
> > > > > > > > > > > persistent
> > > > > > > > > > > > > caches with keeping data.
> > > > > > > > > > > > >
> > > > > > > > > > > > > It would look like:
> > > > > > > > > > > > >
> > > > > > > > > > > > > Ignite ignite = ...;
> > > > > > > > > > > > > ignite.restartCaches(cfg1, ... cfgN);
> > > > > > > > > > > > >
> > > > > > > > > > > > > where cfgX is a new cache configuration, which
> > contains
> > > > the
> > > > > > > name
> > > > > > > > of
> > > > > > > > > > an
> > > > > > > > > > > > > existing persistent cache.
> > > > > > > > > > > > >
> > > > > > > > > > > > > The obvious limitation:
> > > > > > > > > > > > > - affinity key mapping couldn't be changed;
> > > > > > > > > > > > > - count of partitions couldn't be changed;
> > > > > > > > > > > > > - MVCC couldn't be turned off/on;
> > > > > > > > > > > > > - persistent couldn't be turned off;
> > > > > > > > > > > > > - group settings couldn't be changed (group name);
> > > > > > > > > > > > > - if cache belongs to group it's needed to restart
> > all
> > > of
> > > > > > them.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Failure scenario is the crucial thing (and most
> > > > difficult):
> > > > > > > > > > > > > - initiator fail;
> > > > > > > > > > > > > - cluster restart at any stage;
> > > > > > > > > > > > > - joining/starting offline nodes.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Some thoughts about implementation:
> > > > > > > > > > > > > - stop cache with destroy=false;
> > > > > > > > > > > > > - start cache dynamically with new configuration;
> > > > > > > > > > > > > - if indexes settings changed - remove index.bin to
> > > start
> > > > > > > > > indexation;
> > > > > > > > > > > > > - change blt-history when start cache initiated to
> > not
> > > > > allow
> > > > > > > join
> > > > > > > > > > nodes
> > > > > > > > > > > > > with old configuration;
> > > > > > > > > > > > > - use restartId (IGNITE-8911) to not allow to start
> > > cache
> > > > > in
> > > > > > > > > between.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Your thoughts? Would it be a useful feature?
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > --
> > > > > > > > > > > Best regards,
> > > > > > > > > > > Eduard.
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > >
> > > --
> > > Best regards,
> > > Eduard.
> > >
> >
>
>
> --
> Best regards,
> Eduard.
>
12