Ignite: configuration changes at runtime

[Alexey Kuznetsov]

Guys,

Why not to let user to decide whether to persist changed properties or not?
We could have a Boolean flag on API "persist=true|false".

Make sense?

On Tue, Aug 22, 2017 at 6:52 AM, Dmitriy Setrakyan <[hidden email]>
wrote:

> On Mon, Aug 21, 2017 at 7:28 AM, Alexey Dmitriev <[hidden email]>
> wrote:
>
> > I believe it should be persistent.
> > We can have a parameter about "reset configuration on restart", which can
> > be also OLCC-compatible
> >
>
> Alexey, the problem with persisting at all times is that some properties
> may be set directly from code. What is wrong with persisting when user
> explicitly calls saveToFile method?
>
>
> >
> > On Mon, Aug 21, 2017 at 5:22 PM, Dmitriy Setrakyan <
> [hidden email]>
> > wrote:
> >
> > > On Mon, Aug 21, 2017 at 7:15 AM, Alexey Dmitriev <
> [hidden email]
> > >
> > > wrote:
> > >
> > > > I would say it should as soon as we moving to "persistence".
> > > > It will require to look at the things a bit different than before,
> but
> > I
> > > > would say that's an evolution for the product.
> > > > We probably also should think how our configuration system should be
> > > > changed to make it more obvious.
> > > >
> > >
> > > But in that case, what to do with XML configuration? Should it be
> updated
> > > as well? Or even worse, what if user added configuration from code?
> > >
> > > On Mon, Aug 21, 2017 at 5:11 PM, Dmitriy Setrakyan <
> > [hidden email]>
> > > > wrote:
> > > >
> > > > > I think the main question I have is whether this configuration
> change
> > > > will
> > > > > survive restarts. Part of me says it should, and the other part
> says
> > it
> > > > > shouldn't.
> > > > >
> > > > > Does anyone have a strong opinion about this?
> > > > >
> > > > > D.
> > > > >
> > > > > On Mon, Aug 21, 2017 at 7:07 AM, Alexey Dmitriev <
> > > [hidden email]
> > > > >
> > > > > wrote:
> > > > >
> > > > > > It looks like very useful and natural thing having the parameters
> > > > change
> > > > > on
> > > > > > the fly.
> > > > > > Maybe we should design something like OLCC (on-line configuration
> > > > change)
> > > > > > module, which will request different procedures for different
> bunch
> > > of
> > > > > > parameters.
> > > > > > All the parameters, this way, will be splitted into several
> groups.
> > > > > > For example in the worst case, when all the grid has to be
> > > synchronized
> > > > > for
> > > > > > particular parameter change, some preparation on the grid has to
> be
> > > > > > performed, than when all the nodes reports readiness, the
> parameter
> > > > > change
> > > > > > could be initiated.
> > > > > > On the opposite if it is parameter configured per node, OLCC
> state
> > > > > machine
> > > > > > will work simpler without inter-node communication.
> > > > > >
> > > > > > On Mon, Aug 21, 2017 at 4:21 PM, Yakov Zhdanov <
> > [hidden email]>
> > > > > > wrote:
> > > > > >
> > > > > > > >I see your point. In this case, we should have a special
> package
> > > > > > > containing
> > > > > > > >all the runtime config properties.
> > > > > > >
> > > > > > > Dmitry, I think this will be a mess.
> > > > > > >
> > > > > > > Igniters, any more opinions?
> > > > > > >
> > > > > > > --Yakov
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Alexey Dmitriev, VP Engineering
> > > > > > *GridGain Systems*
> > > > > > www.gridgain.com
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Alexey Dmitriev, VP Engineering
> > > > *GridGain Systems*
> > > > www.gridgain.com
> > > >
> > >
> >
> >
> >
> > --
> > Alexey Dmitriev, VP Engineering
> > *GridGain Systems*
> > www.gridgain.com
> >
>

--
Alexey Kuznetsov

[dsetrakyan]

On Mon, Aug 21, 2017 at 6:20 PM, Alexey Kuznetsov <[hidden email]>
wrote:

> Guys,
>
> Why not to let user to decide whether to persist changed properties or not?
> We could have a Boolean flag on API "persist=true|false".
>
> Make sense?
>

Not really. What if the user configured Ignite from Java API, without any
XML. What will be saved? To prevent this ambiguity, I think the cleanest
way is to ask user to save the configuration to a file explicitly.

Makes sense?

> On Tue, Aug 22, 2017 at 6:52 AM, Dmitriy Setrakyan <[hidden email]>
> wrote:
>
> > On Mon, Aug 21, 2017 at 7:28 AM, Alexey Dmitriev <[hidden email]
> >
> > wrote:
> >
> > > I believe it should be persistent.
> > > We can have a parameter about "reset configuration on restart", which
> can
> > > be also OLCC-compatible
> > >
> >
> > Alexey, the problem with persisting at all times is that some properties
> > may be set directly from code. What is wrong with persisting when user
> > explicitly calls saveToFile method?
> >
> >
> > >
> > > On Mon, Aug 21, 2017 at 5:22 PM, Dmitriy Setrakyan <
> > [hidden email]>
> > > wrote:
> > >
> > > > On Mon, Aug 21, 2017 at 7:15 AM, Alexey Dmitriev <
> > [hidden email]
> > > >
> > > > wrote:
> > > >
> > > > > I would say it should as soon as we moving to "persistence".
> > > > > It will require to look at the things a bit different than before,
> > but
> > > I
> > > > > would say that's an evolution for the product.
> > > > > We probably also should think how our configuration system should
> be
> > > > > changed to make it more obvious.
> > > > >
> > > >
> > > > But in that case, what to do with XML configuration? Should it be
> > updated
> > > > as well? Or even worse, what if user added configuration from code?
> > > >
> > > > On Mon, Aug 21, 2017 at 5:11 PM, Dmitriy Setrakyan <
> > > [hidden email]>
> > > > > wrote:
> > > > >
> > > > > > I think the main question I have is whether this configuration
> > change
> > > > > will
> > > > > > survive restarts. Part of me says it should, and the other part
> > says
> > > it
> > > > > > shouldn't.
> > > > > >
> > > > > > Does anyone have a strong opinion about this?
> > > > > >
> > > > > > D.
> > > > > >
> > > > > > On Mon, Aug 21, 2017 at 7:07 AM, Alexey Dmitriev <
> > > > [hidden email]
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > It looks like very useful and natural thing having the
> parameters
> > > > > change
> > > > > > on
> > > > > > > the fly.
> > > > > > > Maybe we should design something like OLCC (on-line
> configuration
> > > > > change)
> > > > > > > module, which will request different procedures for different
> > bunch
> > > > of
> > > > > > > parameters.
> > > > > > > All the parameters, this way, will be splitted into several
> > groups.
> > > > > > > For example in the worst case, when all the grid has to be
> > > > synchronized
> > > > > > for
> > > > > > > particular parameter change, some preparation on the grid has
> to
> > be
> > > > > > > performed, than when all the nodes reports readiness, the
> > parameter
> > > > > > change
> > > > > > > could be initiated.
> > > > > > > On the opposite if it is parameter configured per node, OLCC
> > state
> > > > > > machine
> > > > > > > will work simpler without inter-node communication.
> > > > > > >
> > > > > > > On Mon, Aug 21, 2017 at 4:21 PM, Yakov Zhdanov <
> > > [hidden email]>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > >I see your point. In this case, we should have a special
> > package
> > > > > > > > containing
> > > > > > > > >all the runtime config properties.
> > > > > > > >
> > > > > > > > Dmitry, I think this will be a mess.
> > > > > > > >
> > > > > > > > Igniters, any more opinions?
> > > > > > > >
> > > > > > > > --Yakov
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Alexey Dmitriev, VP Engineering
> > > > > > > *GridGain Systems*
> > > > > > > www.gridgain.com
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Alexey Dmitriev, VP Engineering
> > > > > *GridGain Systems*
> > > > > www.gridgain.com
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Alexey Dmitriev, VP Engineering
> > > *GridGain Systems*
> > > www.gridgain.com
> > >
> >
>
>
>
> --
> Alexey Kuznetsov
>

[yzhdanov]

>I think the main question I have is whether this configuration change will
>survive restarts. Part of me says it should, and the other part says it
>shouldn't.

Dmitry,

I think that configuration changes should survive restarts if persistence
is enabled. Moreover, newly joining nodes should get the current settings
of the running nodes.

--Yakov

[yzhdanov]

>Not really. What if the user configured Ignite from Java API, without any
>XML. What will be saved? To prevent this ambiguity, I think the cleanest
>way is to ask user to save the configuration to a file explicitly.

I think it does not matter too much how the grid was configured. In any
case IgniteConfiguration object is created. And we will be allowing to
change only very few parameters that are mostly of primitive type or enum.

Once changed configuration is saved to metastore. When restarting we should
apply it by default or allow user to override it with provided one.

--Yakov

[dsetrakyan]

On Tue, Aug 22, 2017 at 4:43 AM, Yakov Zhdanov <[hidden email]> wrote:

> >Not really. What if the user configured Ignite from Java API, without any
> >XML. What will be saved? To prevent this ambiguity, I think the cleanest
> >way is to ask user to save the configuration to a file explicitly.
>
> I think it does not matter too much how the grid was configured. In any
> case IgniteConfiguration object is created. And we will be allowing to
> change only very few parameters that are mostly of primitive type or enum.
>
> Once changed configuration is saved to metastore. When restarting we should
> apply it by default or allow user to override it with provided one.
>

I would prefer that users explicitly specify which configuration file to
use on startup. This will prevent any magic. Still think that explicit call
to saveConfigurationFile(...) is the best approach.

[npordash]

The notion of saving changes to a configuration file does not make sense
for lots of use cases, Kubernetes is one example. Anything in the
container's filesystem is considered transient since it will get destroyed
when a pod is restarted.

I would think that runtime changes should be managed in the cluster
metadata somewhere instead of expecting them to be exported elsewhere like
a file. If new nodes join the cluster then runtime changes received as a
part of a node join operation would get applied prior to node activation
from completing. These would override any node configuration provided on
node startup, regardless of how it was configured.

IMO if someone wants to make runtime changes AND propagate those changes to
wherever they are doing external configuration management then perhaps that
should be left as an exercise for the user. There are so many ways that
this could be done and often what is sitting on the filesystem would easily
get overridden by some other mechanism (f.e ansible, salt, puppet, chef,
container orchestration systems, etc).

-Nick

On Tue, Aug 22, 2017, 6:54 PM Dmitriy Setrakyan <[hidden email]>
wrote:

> On Tue, Aug 22, 2017 at 4:43 AM, Yakov Zhdanov <[hidden email]>
> wrote:
>
> > >Not really. What if the user configured Ignite from Java API, without
> any
> > >XML. What will be saved? To prevent this ambiguity, I think the cleanest
> > >way is to ask user to save the configuration to a file explicitly.
> >
> > I think it does not matter too much how the grid was configured. In any
> > case IgniteConfiguration object is created. And we will be allowing to
> > change only very few parameters that are mostly of primitive type or
> enum.
> >
> > Once changed configuration is saved to metastore. When restarting we
> should
> > apply it by default or allow user to override it with provided one.
> >
>
> I would prefer that users explicitly specify which configuration file to
> use on startup. This will prevent any magic. Still think that explicit call
> to saveConfigurationFile(...) is the best approach.
>

[dmagda]

I’m fully share Nick’s point of view on this. If you change any configuration parameter in runtime then go ahead and update your static configuration or a set of DDL statements that are only used during a cluster restart.

—
Denis

> On Aug 22, 2017, at 7:13 PM, Nick Pordash <[hidden email]> wrote:
>
> The notion of saving changes to a configuration file does not make sense
> for lots of use cases, Kubernetes is one example. Anything in the
> container's filesystem is considered transient since it will get destroyed
> when a pod is restarted.
>
> I would think that runtime changes should be managed in the cluster
> metadata somewhere instead of expecting them to be exported elsewhere like
> a file. If new nodes join the cluster then runtime changes received as a
> part of a node join operation would get applied prior to node activation
> from completing. These would override any node configuration provided on
> node startup, regardless of how it was configured.
>
> IMO if someone wants to make runtime changes AND propagate those changes to
> wherever they are doing external configuration management then perhaps that
> should be left as an exercise for the user. There are so many ways that
> this could be done and often what is sitting on the filesystem would easily
> get overridden by some other mechanism (f.e ansible, salt, puppet, chef,
> container orchestration systems, etc).
>
> -Nick
>
> On Tue, Aug 22, 2017, 6:54 PM Dmitriy Setrakyan <[hidden email]>
> wrote:
>
>> On Tue, Aug 22, 2017 at 4:43 AM, Yakov Zhdanov <[hidden email]>
>> wrote:
>>
>>>> Not really. What if the user configured Ignite from Java API, without
>> any
>>>> XML. What will be saved? To prevent this ambiguity, I think the cleanest
>>>> way is to ask user to save the configuration to a file explicitly.
>>>
>>> I think it does not matter too much how the grid was configured. In any
>>> case IgniteConfiguration object is created. And we will be allowing to
>>> change only very few parameters that are mostly of primitive type or
>> enum.
>>>
>>> Once changed configuration is saved to metastore. When restarting we
>> should
>>> apply it by default or allow user to override it with provided one.
>>>
>>
>> I would prefer that users explicitly specify which configuration file to
>> use on startup. This will prevent any magic. Still think that explicit call
>> to saveConfigurationFile(...) is the best approach.
>>