Cluster auto activation design proposal

[Sergey Chugunov]

Going down "node set" road:

-fixed node set
-established node set
-base node set

On Thu, Aug 10, 2017 at 5:23 PM, Dmitriy Setrakyan <[hidden email]>
wrote:

> Can we brainstorm on the names again, I am not sure we have a consensus on
> the name "baseline topology". This will be included in Ignite
> configuration, so the name has to be clear.
>
> Some of the proposals were:
>
> - baseline topology
> - minimal node set
> - node restart set
> - minimal topology
>
> Any other suggestions?
>
> D.
>
> On Thu, Aug 10, 2017 at 2:13 AM, Alexey Goncharuk <
> [hidden email]> wrote:
>
> > Denis,
> >
> > This should be handled by the BT triggers. If I have 3 backups
> configured,
> > I actually won't care if my cluster will live 6 hours without an
> additional
> > backup. If for a partition there is only one backup left - a new BT
> should
> > be triggered automatically.
> >
> > 2017-08-10 0:33 GMT+03:00 Denis Magda <[hidden email]>:
> >
> > > Sergey,
> > >
> > > That’s the only concern I have:
> > >
> > > * 5. User takes out nodes from cluster (e.g. for maintenance purposes):
> > no
> > >   rebalance happens until user recreates BLT on new cluster topology.*
> > >
> > > What if a node is crashed (or some other kind of outage) in the middle
> of
> > > the night and the user has to be sure that survived nodes will
> rearrange
> > > and rebalancing partitions?
> > >
> > > —
> > > Denis
> > >
> > >
> > > > On Aug 4, 2017, at 9:21 AM, Sergey Chugunov <
> [hidden email]
> > >
> > > wrote:
> > > >
> > > > Folks,
> > > >
> > > > I've summarized all results from our discussion so far on wiki page:
> > > > https://cwiki.apache.org/confluence/display/IGNITE/
> > > Automatic+activation+design+-+draft
> > > >
> > > > I hope I reflected the most important details and going to add API
> > > > suggestions for all use cases soon.
> > > >
> > > > Feel free to give feedback here or in comments under the page.
> > > >
> > > > Thanks,
> > > > Sergey.
> > > >
> > > > On Thu, Aug 3, 2017 at 5:40 PM, Alexey Kuznetsov <
> > [hidden email]>
> > > > wrote:
> > > >
> > > >> Hi,
> > > >>
> > > >>>> 1. User creates new BLT using WebConsole or other tool and
> "applies"
> > > it
> > > >> to brand-new cluster.
> > > >>
> > > >> Good idea, but we also should implement *command-line utility* for
> the
> > > same
> > > >> use case.
> > > >>
> > > >> --
> > > >> Alexey Kuznetsov
> > > >>
> > >
> > >
> >
>

[Sergey Chugunov]

More options to name the concept (one may put "node set" or "topology" at
the end - for me these are interchangeable):


   1. essential
   2. basic
   3. completed
   4. prepared
   5. solid
   6. enduring

To me the name must stress the fact that "node set" or "topology" contains
all nodes user is going to work with for a long run.
It is what user plans to work with and base the work on.

Any thoughts on this?

Thanks,
Sergey.

On Thu, Aug 10, 2017 at 5:41 PM, Sergey Chugunov <[hidden email]>
wrote:

> Going down "node set" road:
>
> -fixed node set
> -established node set
> -base node set
>
> On Thu, Aug 10, 2017 at 5:23 PM, Dmitriy Setrakyan <[hidden email]>
> wrote:
>
>> Can we brainstorm on the names again, I am not sure we have a consensus on
>> the name "baseline topology". This will be included in Ignite
>> configuration, so the name has to be clear.
>>
>> Some of the proposals were:
>>
>> - baseline topology
>> - minimal node set
>> - node restart set
>> - minimal topology
>>
>> Any other suggestions?
>>
>> D.
>>
>> On Thu, Aug 10, 2017 at 2:13 AM, Alexey Goncharuk <
>> [hidden email]> wrote:
>>
>> > Denis,
>> >
>> > This should be handled by the BT triggers. If I have 3 backups
>> configured,
>> > I actually won't care if my cluster will live 6 hours without an
>> additional
>> > backup. If for a partition there is only one backup left - a new BT
>> should
>> > be triggered automatically.
>> >
>> > 2017-08-10 0:33 GMT+03:00 Denis Magda <[hidden email]>:
>> >
>> > > Sergey,
>> > >
>> > > That’s the only concern I have:
>> > >
>> > > * 5. User takes out nodes from cluster (e.g. for maintenance
>> purposes):
>> > no
>> > >   rebalance happens until user recreates BLT on new cluster topology.*
>> > >
>> > > What if a node is crashed (or some other kind of outage) in the
>> middle of
>> > > the night and the user has to be sure that survived nodes will
>> rearrange
>> > > and rebalancing partitions?
>> > >
>> > > —
>> > > Denis
>> > >
>> > >
>> > > > On Aug 4, 2017, at 9:21 AM, Sergey Chugunov <
>> [hidden email]
>> > >
>> > > wrote:
>> > > >
>> > > > Folks,
>> > > >
>> > > > I've summarized all results from our discussion so far on wiki page:
>> > > > https://cwiki.apache.org/confluence/display/IGNITE/
>> > > Automatic+activation+design+-+draft
>> > > >
>> > > > I hope I reflected the most important details and going to add API
>> > > > suggestions for all use cases soon.
>> > > >
>> > > > Feel free to give feedback here or in comments under the page.
>> > > >
>> > > > Thanks,
>> > > > Sergey.
>> > > >
>> > > > On Thu, Aug 3, 2017 at 5:40 PM, Alexey Kuznetsov <
>> > [hidden email]>
>> > > > wrote:
>> > > >
>> > > >> Hi,
>> > > >>
>> > > >>>> 1. User creates new BLT using WebConsole or other tool and
>> "applies"
>> > > it
>> > > >> to brand-new cluster.
>> > > >>
>> > > >> Good idea, but we also should implement *command-line utility* for
>> the
>> > > same
>> > > >> use case.
>> > > >>
>> > > >> --
>> > > >> Alexey Kuznetsov
>> > > >>
>> > >
>> > >
>> >
>>
>
>

[dsetrakyan]

Sergey,

As it is becoming clear from another thread, the only configuration users
will need to provide is the initial set of nodes. Everything else will be
handled by Ignite automatically.

In this case, the name Baseline Topology will appear only in documentation,
in which case the name is OK (that is if I am understanding the design
correctly).

However, the list initial set of nodes can be called initialActivationNodes.

Makes sense?

D.

On Fri, Aug 11, 2017 at 2:46 AM, Sergey Chugunov <[hidden email]>
wrote:

> More options to name the concept (one may put "node set" or "topology" at
> the end - for me these are interchangeable):
>
>
>    1. essential
>    2. basic
>    3. completed
>    4. prepared
>    5. solid
>    6. enduring
>
> To me the name must stress the fact that "node set" or "topology" contains
> all nodes user is going to work with for a long run.
> It is what user plans to work with and base the work on.
>
> Any thoughts on this?
>
> Thanks,
> Sergey.
>
> On Thu, Aug 10, 2017 at 5:41 PM, Sergey Chugunov <
> [hidden email]>
> wrote:
>
> > Going down "node set" road:
> >
> > -fixed node set
> > -established node set
> > -base node set
> >
> > On Thu, Aug 10, 2017 at 5:23 PM, Dmitriy Setrakyan <
> [hidden email]>
> > wrote:
> >
> >> Can we brainstorm on the names again, I am not sure we have a consensus
> on
> >> the name "baseline topology". This will be included in Ignite
> >> configuration, so the name has to be clear.
> >>
> >> Some of the proposals were:
> >>
> >> - baseline topology
> >> - minimal node set
> >> - node restart set
> >> - minimal topology
> >>
> >> Any other suggestions?
> >>
> >> D.
> >>
> >> On Thu, Aug 10, 2017 at 2:13 AM, Alexey Goncharuk <
> >> [hidden email]> wrote:
> >>
> >> > Denis,
> >> >
> >> > This should be handled by the BT triggers. If I have 3 backups
> >> configured,
> >> > I actually won't care if my cluster will live 6 hours without an
> >> additional
> >> > backup. If for a partition there is only one backup left - a new BT
> >> should
> >> > be triggered automatically.
> >> >
> >> > 2017-08-10 0:33 GMT+03:00 Denis Magda <[hidden email]>:
> >> >
> >> > > Sergey,
> >> > >
> >> > > That’s the only concern I have:
> >> > >
> >> > > * 5. User takes out nodes from cluster (e.g. for maintenance
> >> purposes):
> >> > no
> >> > >   rebalance happens until user recreates BLT on new cluster
> topology.*
> >> > >
> >> > > What if a node is crashed (or some other kind of outage) in the
> >> middle of
> >> > > the night and the user has to be sure that survived nodes will
> >> rearrange
> >> > > and rebalancing partitions?
> >> > >
> >> > > —
> >> > > Denis
> >> > >
> >> > >
> >> > > > On Aug 4, 2017, at 9:21 AM, Sergey Chugunov <
> >> [hidden email]
> >> > >
> >> > > wrote:
> >> > > >
> >> > > > Folks,
> >> > > >
> >> > > > I've summarized all results from our discussion so far on wiki
> page:
> >> > > > https://cwiki.apache.org/confluence/display/IGNITE/
> >> > > Automatic+activation+design+-+draft
> >> > > >
> >> > > > I hope I reflected the most important details and going to add API
> >> > > > suggestions for all use cases soon.
> >> > > >
> >> > > > Feel free to give feedback here or in comments under the page.
> >> > > >
> >> > > > Thanks,
> >> > > > Sergey.
> >> > > >
> >> > > > On Thu, Aug 3, 2017 at 5:40 PM, Alexey Kuznetsov <
> >> > [hidden email]>
> >> > > > wrote:
> >> > > >
> >> > > >> Hi,
> >> > > >>
> >> > > >>>> 1. User creates new BLT using WebConsole or other tool and
> >> "applies"
> >> > > it
> >> > > >> to brand-new cluster.
> >> > > >>
> >> > > >> Good idea, but we also should implement *command-line utility*
> for
> >> the
> >> > > same
> >> > > >> use case.
> >> > > >>
> >> > > >> --
> >> > > >> Alexey Kuznetsov
> >> > > >>
> >> > >
> >> > >
> >> >
> >>
> >
> >
>

[Sergey Chugunov]

Dmitriy,

As automatic creation of Baseline Topology is just another use case about
the whole concept, lets discuss it in this thread and forget about parallel
one.

*initialActivationNodes* for desired configuration of Baseline Topology
sounds good to me, I would like to hear from other Igniters.

However other options to create Baseline Topology were proposed:

   - Prepare it manually with WebConsole or visor-console (no
   initialActivationNodes configuration is involved).
   - Create it automatically on the first manual activation (again no
   initial configuration is involved).

Thus we still need Baseline Topology entity not only in documentation.

Also please take into account that at some point in time user may want to
recreate BLT (e.g. to add nodes to existing cluster). User will have to
interact with some entity; so we need it not only internally but on public
API.

Thanks,
Sergey.



On Sat, Aug 12, 2017 at 4:07 AM, Dmitriy Setrakyan <[hidden email]>
wrote:

> Sergey,
>
> As it is becoming clear from another thread, the only configuration users
> will need to provide is the initial set of nodes. Everything else will be
> handled by Ignite automatically.
>
> In this case, the name Baseline Topology will appear only in documentation,
> in which case the name is OK (that is if I am understanding the design
> correctly).
>
> However, the list initial set of nodes can be called
> initialActivationNodes.
>
> Makes sense?
>
> D.
>
> On Fri, Aug 11, 2017 at 2:46 AM, Sergey Chugunov <
> [hidden email]>
> wrote:
>
> > More options to name the concept (one may put "node set" or "topology" at
> > the end - for me these are interchangeable):
> >
> >
> >    1. essential
> >    2. basic
> >    3. completed
> >    4. prepared
> >    5. solid
> >    6. enduring
> >
> > To me the name must stress the fact that "node set" or "topology"
> contains
> > all nodes user is going to work with for a long run.
> > It is what user plans to work with and base the work on.
> >
> > Any thoughts on this?
> >
> > Thanks,
> > Sergey.
> >
> > On Thu, Aug 10, 2017 at 5:41 PM, Sergey Chugunov <
> > [hidden email]>
> > wrote:
> >
> > > Going down "node set" road:
> > >
> > > -fixed node set
> > > -established node set
> > > -base node set
> > >
> > > On Thu, Aug 10, 2017 at 5:23 PM, Dmitriy Setrakyan <
> > [hidden email]>
> > > wrote:
> > >
> > >> Can we brainstorm on the names again, I am not sure we have a
> consensus
> > on
> > >> the name "baseline topology". This will be included in Ignite
> > >> configuration, so the name has to be clear.
> > >>
> > >> Some of the proposals were:
> > >>
> > >> - baseline topology
> > >> - minimal node set
> > >> - node restart set
> > >> - minimal topology
> > >>
> > >> Any other suggestions?
> > >>
> > >> D.
> > >>
> > >> On Thu, Aug 10, 2017 at 2:13 AM, Alexey Goncharuk <
> > >> [hidden email]> wrote:
> > >>
> > >> > Denis,
> > >> >
> > >> > This should be handled by the BT triggers. If I have 3 backups
> > >> configured,
> > >> > I actually won't care if my cluster will live 6 hours without an
> > >> additional
> > >> > backup. If for a partition there is only one backup left - a new BT
> > >> should
> > >> > be triggered automatically.
> > >> >
> > >> > 2017-08-10 0:33 GMT+03:00 Denis Magda <[hidden email]>:
> > >> >
> > >> > > Sergey,
> > >> > >
> > >> > > That’s the only concern I have:
> > >> > >
> > >> > > * 5. User takes out nodes from cluster (e.g. for maintenance
> > >> purposes):
> > >> > no
> > >> > >   rebalance happens until user recreates BLT on new cluster
> > topology.*
> > >> > >
> > >> > > What if a node is crashed (or some other kind of outage) in the
> > >> middle of
> > >> > > the night and the user has to be sure that survived nodes will
> > >> rearrange
> > >> > > and rebalancing partitions?
> > >> > >
> > >> > > —
> > >> > > Denis
> > >> > >
> > >> > >
> > >> > > > On Aug 4, 2017, at 9:21 AM, Sergey Chugunov <
> > >> [hidden email]
> > >> > >
> > >> > > wrote:
> > >> > > >
> > >> > > > Folks,
> > >> > > >
> > >> > > > I've summarized all results from our discussion so far on wiki
> > page:
> > >> > > > https://cwiki.apache.org/confluence/display/IGNITE/
> > >> > > Automatic+activation+design+-+draft
> > >> > > >
> > >> > > > I hope I reflected the most important details and going to add
> API
> > >> > > > suggestions for all use cases soon.
> > >> > > >
> > >> > > > Feel free to give feedback here or in comments under the page.
> > >> > > >
> > >> > > > Thanks,
> > >> > > > Sergey.
> > >> > > >
> > >> > > > On Thu, Aug 3, 2017 at 5:40 PM, Alexey Kuznetsov <
> > >> > [hidden email]>
> > >> > > > wrote:
> > >> > > >
> > >> > > >> Hi,
> > >> > > >>
> > >> > > >>>> 1. User creates new BLT using WebConsole or other tool and
> > >> "applies"
> > >> > > it
> > >> > > >> to brand-new cluster.
> > >> > > >>
> > >> > > >> Good idea, but we also should implement *command-line utility*
> > for
> > >> the
> > >> > > same
> > >> > > >> use case.
> > >> > > >>
> > >> > > >> --
> > >> > > >> Alexey Kuznetsov
> > >> > > >>
> > >> > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>

[dsetrakyan]

On Thu, Aug 17, 2017 at 7:34 AM, Sergey Chugunov <[hidden email]>
wrote:

> Dmitriy,
>
> As automatic creation of Baseline Topology is just another use case about
> the whole concept, lets discuss it in this thread and forget about parallel
> one.
>
> *initialActivationNodes* for desired configuration of Baseline Topology
> sounds good to me, I would like to hear from other Igniters.
>

I don't think this sounds right. It should be either "inititialActiveNodes"
or "initialNodes". I am also open to other suggestions.


>
> However other options to create Baseline Topology were proposed:
>
>    - Prepare it manually with WebConsole or visor-console (no
>    initialActivationNodes configuration is involved).
>    - Create it automatically on the first manual activation (again no
>    initial configuration is involved).
>

> Thus we still need Baseline Topology entity not only in documentation.
>

In either case, wouldn't it involve creation of initialActiveNodes
implicitly?


>
> Also please take into account that at some point in time user may want to
> recreate BLT (e.g. to add nodes to existing cluster). User will have to
> interact with some entity; so we need it not only internally but on public
> API.
>

What would the public API look like and who would user interact with it?

[Alexey Dmitriev]

API is proposed in the head of the thread by Sergey, as I understood:
______________________________________________________

API for BaselineTopology manipulation may look like this:

*Ignite::activation::establishBaselineTopology();*
*Ignite::activation::establishBaselineTopology(BaselineTopology bltTop);*

Both methods will establish BT and activate cluster once it is established.

The first one allows user to establish BT using current topology. If any
changes happen to the topology during establishing process, user will be
notified and allowed to proceed or abort the procedure.

Second method allows to use some monitoring'n'management tools like
WebConsole where user can prepare a list of nodes, using them create a BT
and send to the cluster a command to finally establish it.

From high level BaselineTopology entity contains only collection of nodes:

*BaselineTopology {*
*  Collection<TopologyNode> nodes;*
*}*

*TopologyNode* here contains information about node - its consistent id and
set of user attributes used to calculate affinity function.
____________________________________________

[dsetrakyan]

I still do not see why anyone would explicitly call these 2 methods:

*Ignite::activation::establishBaselineTopology();*
*Ignite::activation::establishBaselineTopology(BaselineTopology bltTop);*

For example, if a web console, or some other admin process, want to
automatically set currently started nodes as the baseline topology,
shouldn't they just call a setter for minimalActivationNodes?

D.

On Fri, Aug 18, 2017 at 10:18 AM, Alexey Dmitriev <[hidden email]>
wrote:

> API is proposed in the head of the thread by Sergey, as I understood:
> ______________________________________________________
>
> API for BaselineTopology manipulation may look like this:
>
> *Ignite::activation::establishBaselineTopology();*
> *Ignite::activation::establishBaselineTopology(BaselineTopology bltTop);*
>
> Both methods will establish BT and activate cluster once it is established.
>
> The first one allows user to establish BT using current topology. If any
> changes happen to the topology during establishing process, user will be
> notified and allowed to proceed or abort the procedure.
>
> Second method allows to use some monitoring'n'management tools like
> WebConsole where user can prepare a list of nodes, using them create a BT
> and send to the cluster a command to finally establish it.
>
> From high level BaselineTopology entity contains only collection of nodes:
>
> *BaselineTopology {*
> *  Collection<TopologyNode> nodes;*
> *}*
>
> *TopologyNode* here contains information about node - its consistent id and
> set of user attributes used to calculate affinity function.
> ____________________________________________
>
>
>
> --
> View this message in context: http://apache-ignite-
> developers.2346864.n4.nabble.com/Cluster-auto-activation-
> design-proposal-tp20295p21066.html
> Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
>

[Sergey Chugunov]

Dmitriy,

As I understand you use the term "minimalActivationNodes" as a synonym for
BaselineTopology concept.
In that case I agree with you that we can replace both "establish*" methods
with a simple setter method (see below in summary).

Summing up the whole discussion I see the functionality as following:

New concept BaselineTopology is introduced. The main features it enables
are:

   1. automatic activation of cluster;

   2. easy management of cluster topology changes (planned nodes
   maintenance, adding new nodes etc);

   3. eliminating of rebalancing traffic on short-term node failures.


Use cases to create BLT:

   1. user starts up new cluster of desired number of nodes and activates
   it using existing API. BLT is created with all nodes presented in the
   cluster at the moment of activation, no API is needed;

   2. user prepares BLT using web-console or visor CMD tools and sets it to
   the cluster. New API setter is needed:
   Ignite.activation().setBaselineTopology(BaselineTopology blt);

   3. user provides via static configuration a list of nodes that are
   expected to be in the cluster.
   User starts nodes one by one; when all preconfigured nodes are started
   cluster is activated and BLT is created.
   As list of nodes may be huge it is provided via separate file to avoid
   flooding main configuration.


Igniters, does this description match with your understanding of
functionality? If it does I'll create a set of tickets and start working on
implementation.

Thanks,
Sergey.


On Sat, Aug 19, 2017 at 5:41 PM, Dmitriy Setrakyan <[hidden email]>
wrote:

> I still do not see why anyone would explicitly call these 2 methods:
>
> *Ignite::activation::establishBaselineTopology();*
> *Ignite::activation::establishBaselineTopology(BaselineTopology bltTop);*
>
> For example, if a web console, or some other admin process, want to
> automatically set currently started nodes as the baseline topology,
> shouldn't they just call a setter for minimalActivationNodes?
>
> D.
>
> On Fri, Aug 18, 2017 at 10:18 AM, Alexey Dmitriev <[hidden email]>
> wrote:
>
> > API is proposed in the head of the thread by Sergey, as I understood:
> > ______________________________________________________
> >
> > API for BaselineTopology manipulation may look like this:
> >
> > *Ignite::activation::establishBaselineTopology();*
> > *Ignite::activation::establishBaselineTopology(BaselineTopology
> bltTop);*
> >
> > Both methods will establish BT and activate cluster once it is
> established.
> >
> > The first one allows user to establish BT using current topology. If any
> > changes happen to the topology during establishing process, user will be
> > notified and allowed to proceed or abort the procedure.
> >
> > Second method allows to use some monitoring'n'management tools like
> > WebConsole where user can prepare a list of nodes, using them create a BT
> > and send to the cluster a command to finally establish it.
> >
> > From high level BaselineTopology entity contains only collection of
> nodes:
> >
> > *BaselineTopology {*
> > *  Collection<TopologyNode> nodes;*
> > *}*
> >
> > *TopologyNode* here contains information about node - its consistent id
> and
> > set of user attributes used to calculate affinity function.
> > ____________________________________________
> >
> >
> >
> > --
> > View this message in context: http://apache-ignite-
> > developers.2346864.n4.nabble.com/Cluster-auto-activation-
> > design-proposal-tp20295p21066.html
> > Sent from the Apache Ignite Developers mailing list archive at
> Nabble.com.
> >
>

[dsetrakyan]

Sergey, I am still confused. What is the BaselineTopology interface in your
example? I thought that you agreed with me that we simply need a setter for
activation nodes, no?

D.

On Tue, Aug 22, 2017 at 4:47 AM, Sergey Chugunov <[hidden email]>
wrote:

> Dmitriy,
>
> As I understand you use the term "minimalActivationNodes" as a synonym for
> BaselineTopology concept.
> In that case I agree with you that we can replace both "establish*" methods
> with a simple setter method (see below in summary).
>
> Summing up the whole discussion I see the functionality as following:
>
> New concept BaselineTopology is introduced. The main features it enables
> are:
>
>    1. automatic activation of cluster;
>
>    2. easy management of cluster topology changes (planned nodes
>    maintenance, adding new nodes etc);
>
>    3. eliminating of rebalancing traffic on short-term node failures.
>
>
> Use cases to create BLT:
>
>    1. user starts up new cluster of desired number of nodes and activates
>    it using existing API. BLT is created with all nodes presented in the
>    cluster at the moment of activation, no API is needed;
>
>    2. user prepares BLT using web-console or visor CMD tools and sets it to
>    the cluster. New API setter is needed:
>    Ignite.activation().setBaselineTopology(BaselineTopology blt);
>
>    3. user provides via static configuration a list of nodes that are
>    expected to be in the cluster.
>    User starts nodes one by one; when all preconfigured nodes are started
>    cluster is activated and BLT is created.
>    As list of nodes may be huge it is provided via separate file to avoid
>    flooding main configuration.
>
>
> Igniters, does this description match with your understanding of
> functionality? If it does I'll create a set of tickets and start working on
> implementation.
>
> Thanks,
> Sergey.
>
>
> On Sat, Aug 19, 2017 at 5:41 PM, Dmitriy Setrakyan <[hidden email]>
> wrote:
>
> > I still do not see why anyone would explicitly call these 2 methods:
> >
> > *Ignite::activation::establishBaselineTopology();*
> > *Ignite::activation::establishBaselineTopology(BaselineTopology
> bltTop);*
> >
> > For example, if a web console, or some other admin process, want to
> > automatically set currently started nodes as the baseline topology,
> > shouldn't they just call a setter for minimalActivationNodes?
> >
> > D.
> >
> > On Fri, Aug 18, 2017 at 10:18 AM, Alexey Dmitriev <
> [hidden email]>
> > wrote:
> >
> > > API is proposed in the head of the thread by Sergey, as I understood:
> > > ______________________________________________________
> > >
> > > API for BaselineTopology manipulation may look like this:
> > >
> > > *Ignite::activation::establishBaselineTopology();*
> > > *Ignite::activation::establishBaselineTopology(BaselineTopology
> > bltTop);*
> > >
> > > Both methods will establish BT and activate cluster once it is
> > established.
> > >
> > > The first one allows user to establish BT using current topology. If
> any
> > > changes happen to the topology during establishing process, user will
> be
> > > notified and allowed to proceed or abort the procedure.
> > >
> > > Second method allows to use some monitoring'n'management tools like
> > > WebConsole where user can prepare a list of nodes, using them create a
> BT
> > > and send to the cluster a command to finally establish it.
> > >
> > > From high level BaselineTopology entity contains only collection of
> > nodes:
> > >
> > > *BaselineTopology {*
> > > *  Collection<TopologyNode> nodes;*
> > > *}*
> > >
> > > *TopologyNode* here contains information about node - its consistent id
> > and
> > > set of user attributes used to calculate affinity function.
> > > ____________________________________________
> > >
> > >
> > >
> > > --
> > > View this message in context: http://apache-ignite-
> > > developers.2346864.n4.nabble.com/Cluster-auto-activation-
> > > design-proposal-tp20295p21066.html
> > > Sent from the Apache Ignite Developers mailing list archive at
> > Nabble.com.
> > >
> >
>

[Sergey Chugunov]

Dmitriy,

It was my misunderstanding, I believe that setter is not enough and we need
a full-fledged entity.

We should also be able to check if BLTs are compatible. Interface looks
like this and use case for this functionality is described below.

interface BaselineTopology {
   Collection<ClusterNode> nodes();
   boolean isCompatibleWith(BaselineTopology blt);
}

Let's consider the following scenario:

   1. We have a grid with N nodes: it is up, active and has data in it. ->
   BLT #1 created.
   2. We shutdown the grid. Then divide it into two parts: Part1_grid and
   Part2_grid.
   3. We start and activate Part1_grid . Topology has changed -> BLT#2
   created.
   After that we shutdown that Part1_grid.
   4. We start and activate Part2_grid. Topology also has changed -> BLT#3
   created.
   5. Then we start Part1_grid and it's nodes try to join Part2_grid.


If join is successful we have an undefined state of the resulting grid:
values for the same key may (and will) differ between grid parts.

So to prevent this we should keep nodes with BLT#2 from joining the grid
with BLT#3. And we should fail nodes with an error message.

Thanks,
Sergey.




On Wed, Aug 23, 2017 at 5:47 AM, Dmitriy Setrakyan <[hidden email]>
wrote:

> Sergey, I am still confused. What is the BaselineTopology interface in your
> example? I thought that you agreed with me that we simply need a setter for
> activation nodes, no?
>
> D.
>
> On Tue, Aug 22, 2017 at 4:47 AM, Sergey Chugunov <
> [hidden email]>
> wrote:
>
> > Dmitriy,
> >
> > As I understand you use the term "minimalActivationNodes" as a synonym
> for
> > BaselineTopology concept.
> > In that case I agree with you that we can replace both "establish*"
> methods
> > with a simple setter method (see below in summary).
> >
> > Summing up the whole discussion I see the functionality as following:
> >
> > New concept BaselineTopology is introduced. The main features it enables
> > are:
> >
> >    1. automatic activation of cluster;
> >
> >    2. easy management of cluster topology changes (planned nodes
> >    maintenance, adding new nodes etc);
> >
> >    3. eliminating of rebalancing traffic on short-term node failures.
> >
> >
> > Use cases to create BLT:
> >
> >    1. user starts up new cluster of desired number of nodes and activates
> >    it using existing API. BLT is created with all nodes presented in the
> >    cluster at the moment of activation, no API is needed;
> >
> >    2. user prepares BLT using web-console or visor CMD tools and sets it
> to
> >    the cluster. New API setter is needed:
> >    Ignite.activation().setBaselineTopology(BaselineTopology blt);
> >
> >    3. user provides via static configuration a list of nodes that are
> >    expected to be in the cluster.
> >    User starts nodes one by one; when all preconfigured nodes are started
> >    cluster is activated and BLT is created.
> >    As list of nodes may be huge it is provided via separate file to avoid
> >    flooding main configuration.
> >
> >
> > Igniters, does this description match with your understanding of
> > functionality? If it does I'll create a set of tickets and start working
> on
> > implementation.
> >
> > Thanks,
> > Sergey.
> >
> >
> > On Sat, Aug 19, 2017 at 5:41 PM, Dmitriy Setrakyan <
> [hidden email]>
> > wrote:
> >
> > > I still do not see why anyone would explicitly call these 2 methods:
> > >
> > > *Ignite::activation::establishBaselineTopology();*
> > > *Ignite::activation::establishBaselineTopology(BaselineTopology
> > bltTop);*
> > >
> > > For example, if a web console, or some other admin process, want to
> > > automatically set currently started nodes as the baseline topology,
> > > shouldn't they just call a setter for minimalActivationNodes?
> > >
> > > D.
> > >
> > > On Fri, Aug 18, 2017 at 10:18 AM, Alexey Dmitriev <
> > [hidden email]>
> > > wrote:
> > >
> > > > API is proposed in the head of the thread by Sergey, as I understood:
> > > > ______________________________________________________
> > > >
> > > > API for BaselineTopology manipulation may look like this:
> > > >
> > > > *Ignite::activation::establishBaselineTopology();*
> > > > *Ignite::activation::establishBaselineTopology(BaselineTopology
> > > bltTop);*
> > > >
> > > > Both methods will establish BT and activate cluster once it is
> > > established.
> > > >
> > > > The first one allows user to establish BT using current topology. If
> > any
> > > > changes happen to the topology during establishing process, user will
> > be
> > > > notified and allowed to proceed or abort the procedure.
> > > >
> > > > Second method allows to use some monitoring'n'management tools like
> > > > WebConsole where user can prepare a list of nodes, using them create
> a
> > BT
> > > > and send to the cluster a command to finally establish it.
> > > >
> > > > From high level BaselineTopology entity contains only collection of
> > > nodes:
> > > >
> > > > *BaselineTopology {*
> > > > *  Collection<TopologyNode> nodes;*
> > > > *}*
> > > >
> > > > *TopologyNode* here contains information about node - its consistent
> id
> > > and
> > > > set of user attributes used to calculate affinity function.
> > > > ____________________________________________
> > > >
> > > >
> > > >
> > > > --
> > > > View this message in context: http://apache-ignite-
> > > > developers.2346864.n4.nabble.com/Cluster-auto-activation-
> > > > design-proposal-tp20295p21066.html
> > > > Sent from the Apache Ignite Developers mailing list archive at
> > > Nabble.com.
> > > >
> > >
> >
>

[dsetrakyan]

Sergey, the interface you are suggesting is internal, not external. Why
should user ever see it or care about it?

D.

On Mon, Aug 28, 2017 at 3:18 PM, Sergey Chugunov <[hidden email]>
wrote:

> Dmitriy,
>
> It was my misunderstanding, I believe that setter is not enough and we need
> a full-fledged entity.
>
> We should also be able to check if BLTs are compatible. Interface looks
> like this and use case for this functionality is described below.
>
> interface BaselineTopology {
>    Collection<ClusterNode> nodes();
>    boolean isCompatibleWith(BaselineTopology blt);
> }
>
> Let's consider the following scenario:
>
>    1. We have a grid with N nodes: it is up, active and has data in it. ->
>    BLT #1 created.
>    2. We shutdown the grid. Then divide it into two parts: Part1_grid and
>    Part2_grid.
>    3. We start and activate Part1_grid . Topology has changed -> BLT#2
>    created.
>    After that we shutdown that Part1_grid.
>    4. We start and activate Part2_grid. Topology also has changed -> BLT#3
>    created.
>    5. Then we start Part1_grid and it's nodes try to join Part2_grid.
>
>
> If join is successful we have an undefined state of the resulting grid:
> values for the same key may (and will) differ between grid parts.
>
> So to prevent this we should keep nodes with BLT#2 from joining the grid
> with BLT#3. And we should fail nodes with an error message.
>
> Thanks,
> Sergey.
>
>
>
>
> On Wed, Aug 23, 2017 at 5:47 AM, Dmitriy Setrakyan <[hidden email]>
> wrote:
>
> > Sergey, I am still confused. What is the BaselineTopology interface in
> your
> > example? I thought that you agreed with me that we simply need a setter
> for
> > activation nodes, no?
> >
> > D.
> >
> > On Tue, Aug 22, 2017 at 4:47 AM, Sergey Chugunov <
> > [hidden email]>
> > wrote:
> >
> > > Dmitriy,
> > >
> > > As I understand you use the term "minimalActivationNodes" as a synonym
> > for
> > > BaselineTopology concept.
> > > In that case I agree with you that we can replace both "establish*"
> > methods
> > > with a simple setter method (see below in summary).
> > >
> > > Summing up the whole discussion I see the functionality as following:
> > >
> > > New concept BaselineTopology is introduced. The main features it
> enables
> > > are:
> > >
> > >    1. automatic activation of cluster;
> > >
> > >    2. easy management of cluster topology changes (planned nodes
> > >    maintenance, adding new nodes etc);
> > >
> > >    3. eliminating of rebalancing traffic on short-term node failures.
> > >
> > >
> > > Use cases to create BLT:
> > >
> > >    1. user starts up new cluster of desired number of nodes and
> activates
> > >    it using existing API. BLT is created with all nodes presented in
> the
> > >    cluster at the moment of activation, no API is needed;
> > >
> > >    2. user prepares BLT using web-console or visor CMD tools and sets
> it
> > to
> > >    the cluster. New API setter is needed:
> > >    Ignite.activation().setBaselineTopology(BaselineTopology blt);
> > >
> > >    3. user provides via static configuration a list of nodes that are
> > >    expected to be in the cluster.
> > >    User starts nodes one by one; when all preconfigured nodes are
> started
> > >    cluster is activated and BLT is created.
> > >    As list of nodes may be huge it is provided via separate file to
> avoid
> > >    flooding main configuration.
> > >
> > >
> > > Igniters, does this description match with your understanding of
> > > functionality? If it does I'll create a set of tickets and start
> working
> > on
> > > implementation.
> > >
> > > Thanks,
> > > Sergey.
> > >
> > >
> > > On Sat, Aug 19, 2017 at 5:41 PM, Dmitriy Setrakyan <
> > [hidden email]>
> > > wrote:
> > >
> > > > I still do not see why anyone would explicitly call these 2 methods:
> > > >
> > > > *Ignite::activation::establishBaselineTopology();*
> > > > *Ignite::activation::establishBaselineTopology(BaselineTopology
> > > bltTop);*
> > > >
> > > > For example, if a web console, or some other admin process, want to
> > > > automatically set currently started nodes as the baseline topology,
> > > > shouldn't they just call a setter for minimalActivationNodes?
> > > >
> > > > D.
> > > >
> > > > On Fri, Aug 18, 2017 at 10:18 AM, Alexey Dmitriev <
> > > [hidden email]>
> > > > wrote:
> > > >
> > > > > API is proposed in the head of the thread by Sergey, as I
> understood:
> > > > > ______________________________________________________
> > > > >
> > > > > API for BaselineTopology manipulation may look like this:
> > > > >
> > > > > *Ignite::activation::establishBaselineTopology();*
> > > > > *Ignite::activation::establishBaselineTopology(BaselineTopology
> > > > bltTop);*
> > > > >
> > > > > Both methods will establish BT and activate cluster once it is
> > > > established.
> > > > >
> > > > > The first one allows user to establish BT using current topology.
> If
> > > any
> > > > > changes happen to the topology during establishing process, user
> will
> > > be
> > > > > notified and allowed to proceed or abort the procedure.
> > > > >
> > > > > Second method allows to use some monitoring'n'management tools like
> > > > > WebConsole where user can prepare a list of nodes, using them
> create
> > a
> > > BT
> > > > > and send to the cluster a command to finally establish it.
> > > > >
> > > > > From high level BaselineTopology entity contains only collection of
> > > > nodes:
> > > > >
> > > > > *BaselineTopology {*
> > > > > *  Collection<TopologyNode> nodes;*
> > > > > *}*
> > > > >
> > > > > *TopologyNode* here contains information about node - its
> consistent
> > id
> > > > and
> > > > > set of user attributes used to calculate affinity function.
> > > > > ____________________________________________
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > View this message in context: http://apache-ignite-
> > > > > developers.2346864.n4.nabble.com/Cluster-auto-activation-
> > > > > design-proposal-tp20295p21066.html
> > > > > Sent from the Apache Ignite Developers mailing list archive at
> > > > Nabble.com.
> > > > >
> > > >
> > >
> >
>

[Sergey Chugunov]

Dmitriy,

Now I see your point and I think you're right.

We can give end-user a simple setter like
Ignite::activation::setInitialActivationNodes(Collection<ClusterNode>
nodes) to provide collection of nodes that grid must reach to activate
automatically.

And then using the collection we'll create BaselineTopology abstraction
internally.

As a result user won't be exposed to our internal abstractions and will
work with intuitive concept of collection of nodes.

Thanks,
Sergey.


On Mon, Aug 28, 2017 at 4:39 PM, Dmitriy Setrakyan <[hidden email]>
wrote:

> Sergey, the interface you are suggesting is internal, not external. Why
> should user ever see it or care about it?
>
> D.
>
> On Mon, Aug 28, 2017 at 3:18 PM, Sergey Chugunov <
> [hidden email]>
> wrote:
>
> > Dmitriy,
> >
> > It was my misunderstanding, I believe that setter is not enough and we
> need
> > a full-fledged entity.
> >
> > We should also be able to check if BLTs are compatible. Interface looks
> > like this and use case for this functionality is described below.
> >
> > interface BaselineTopology {
> >    Collection<ClusterNode> nodes();
> >    boolean isCompatibleWith(BaselineTopology blt);
> > }
> >
> > Let's consider the following scenario:
> >
> >    1. We have a grid with N nodes: it is up, active and has data in it.
> ->
> >    BLT #1 created.
> >    2. We shutdown the grid. Then divide it into two parts: Part1_grid and
> >    Part2_grid.
> >    3. We start and activate Part1_grid . Topology has changed -> BLT#2
> >    created.
> >    After that we shutdown that Part1_grid.
> >    4. We start and activate Part2_grid. Topology also has changed ->
> BLT#3
> >    created.
> >    5. Then we start Part1_grid and it's nodes try to join Part2_grid.
> >
> >
> > If join is successful we have an undefined state of the resulting grid:
> > values for the same key may (and will) differ between grid parts.
> >
> > So to prevent this we should keep nodes with BLT#2 from joining the grid
> > with BLT#3. And we should fail nodes with an error message.
> >
> > Thanks,
> > Sergey.
> >
> >
> >
> >
> > On Wed, Aug 23, 2017 at 5:47 AM, Dmitriy Setrakyan <
> [hidden email]>
> > wrote:
> >
> > > Sergey, I am still confused. What is the BaselineTopology interface in
> > your
> > > example? I thought that you agreed with me that we simply need a setter
> > for
> > > activation nodes, no?
> > >
> > > D.
> > >
> > > On Tue, Aug 22, 2017 at 4:47 AM, Sergey Chugunov <
> > > [hidden email]>
> > > wrote:
> > >
> > > > Dmitriy,
> > > >
> > > > As I understand you use the term "minimalActivationNodes" as a
> synonym
> > > for
> > > > BaselineTopology concept.
> > > > In that case I agree with you that we can replace both "establish*"
> > > methods
> > > > with a simple setter method (see below in summary).
> > > >
> > > > Summing up the whole discussion I see the functionality as following:
> > > >
> > > > New concept BaselineTopology is introduced. The main features it
> > enables
> > > > are:
> > > >
> > > >    1. automatic activation of cluster;
> > > >
> > > >    2. easy management of cluster topology changes (planned nodes
> > > >    maintenance, adding new nodes etc);
> > > >
> > > >    3. eliminating of rebalancing traffic on short-term node failures.
> > > >
> > > >
> > > > Use cases to create BLT:
> > > >
> > > >    1. user starts up new cluster of desired number of nodes and
> > activates
> > > >    it using existing API. BLT is created with all nodes presented in
> > the
> > > >    cluster at the moment of activation, no API is needed;
> > > >
> > > >    2. user prepares BLT using web-console or visor CMD tools and sets
> > it
> > > to
> > > >    the cluster. New API setter is needed:
> > > >    Ignite.activation().setBaselineTopology(BaselineTopology blt);
> > > >
> > > >    3. user provides via static configuration a list of nodes that are
> > > >    expected to be in the cluster.
> > > >    User starts nodes one by one; when all preconfigured nodes are
> > started
> > > >    cluster is activated and BLT is created.
> > > >    As list of nodes may be huge it is provided via separate file to
> > avoid
> > > >    flooding main configuration.
> > > >
> > > >
> > > > Igniters, does this description match with your understanding of
> > > > functionality? If it does I'll create a set of tickets and start
> > working
> > > on
> > > > implementation.
> > > >
> > > > Thanks,
> > > > Sergey.
> > > >
> > > >
> > > > On Sat, Aug 19, 2017 at 5:41 PM, Dmitriy Setrakyan <
> > > [hidden email]>
> > > > wrote:
> > > >
> > > > > I still do not see why anyone would explicitly call these 2
> methods:
> > > > >
> > > > > *Ignite::activation::establishBaselineTopology();*
> > > > > *Ignite::activation::establishBaselineTopology(BaselineTopology
> > > > bltTop);*
> > > > >
> > > > > For example, if a web console, or some other admin process, want to
> > > > > automatically set currently started nodes as the baseline topology,
> > > > > shouldn't they just call a setter for minimalActivationNodes?
> > > > >
> > > > > D.
> > > > >
> > > > > On Fri, Aug 18, 2017 at 10:18 AM, Alexey Dmitriev <
> > > > [hidden email]>
> > > > > wrote:
> > > > >
> > > > > > API is proposed in the head of the thread by Sergey, as I
> > understood:
> > > > > > ______________________________________________________
> > > > > >
> > > > > > API for BaselineTopology manipulation may look like this:
> > > > > >
> > > > > > *Ignite::activation::establishBaselineTopology();*
> > > > > > *Ignite::activation::establishBaselineTopology(BaselineTopology
> > > > > bltTop);*
> > > > > >
> > > > > > Both methods will establish BT and activate cluster once it is
> > > > > established.
> > > > > >
> > > > > > The first one allows user to establish BT using current topology.
> > If
> > > > any
> > > > > > changes happen to the topology during establishing process, user
> > will
> > > > be
> > > > > > notified and allowed to proceed or abort the procedure.
> > > > > >
> > > > > > Second method allows to use some monitoring'n'management tools
> like
> > > > > > WebConsole where user can prepare a list of nodes, using them
> > create
> > > a
> > > > BT
> > > > > > and send to the cluster a command to finally establish it.
> > > > > >
> > > > > > From high level BaselineTopology entity contains only collection
> of
> > > > > nodes:
> > > > > >
> > > > > > *BaselineTopology {*
> > > > > > *  Collection<TopologyNode> nodes;*
> > > > > > *}*
> > > > > >
> > > > > > *TopologyNode* here contains information about node - its
> > consistent
> > > id
> > > > > and
> > > > > > set of user attributes used to calculate affinity function.
> > > > > > ____________________________________________
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > View this message in context: http://apache-ignite-
> > > > > > developers.2346864.n4.nabble.com/Cluster-auto-activation-
> > > > > > design-proposal-tp20295p21066.html
> > > > > > Sent from the Apache Ignite Developers mailing list archive at
> > > > > Nabble.com.
> > > > > >
> > > > >
> > > >
> > >
> >
>

[npordash]

How is a user expected to produce a collection of ClusterNode prior to all
of the expected nodes joining? Users don't create instances of this, so as
far as I can tell it would have to be retrieved from IgniteCluster.
However, would doing that and calling the proposed method be really any
different than calling Ignite.activate and using the current set of server
nodes as that collection?

From a user's perspective is it really necessary that specific nodes need
to be identified vs saying that they expect N server nodes to be in the
cluster for auto activation?

-Nick

On Wed, Aug 30, 2017, 1:23 AM Sergey Chugunov <[hidden email]>
wrote:

> Dmitriy,
>
> Now I see your point and I think you're right.
>
> We can give end-user a simple setter like
> Ignite::activation::setInitialActivationNodes(Collection<ClusterNode>
> nodes) to provide collection of nodes that grid must reach to activate
> automatically.
>
> And then using the collection we'll create BaselineTopology abstraction
> internally.
>
> As a result user won't be exposed to our internal abstractions and will
> work with intuitive concept of collection of nodes.
>
> Thanks,
> Sergey.
>
>
> On Mon, Aug 28, 2017 at 4:39 PM, Dmitriy Setrakyan <[hidden email]>
> wrote:
>
> > Sergey, the interface you are suggesting is internal, not external. Why
> > should user ever see it or care about it?
> >
> > D.
> >
> > On Mon, Aug 28, 2017 at 3:18 PM, Sergey Chugunov <
> > [hidden email]>
> > wrote:
> >
> > > Dmitriy,
> > >
> > > It was my misunderstanding, I believe that setter is not enough and we
> > need
> > > a full-fledged entity.
> > >
> > > We should also be able to check if BLTs are compatible. Interface looks
> > > like this and use case for this functionality is described below.
> > >
> > > interface BaselineTopology {
> > >    Collection<ClusterNode> nodes();
> > >    boolean isCompatibleWith(BaselineTopology blt);
> > > }
> > >
> > > Let's consider the following scenario:
> > >
> > >    1. We have a grid with N nodes: it is up, active and has data in it.
> > ->
> > >    BLT #1 created.
> > >    2. We shutdown the grid. Then divide it into two parts: Part1_grid
> and
> > >    Part2_grid.
> > >    3. We start and activate Part1_grid . Topology has changed -> BLT#2
> > >    created.
> > >    After that we shutdown that Part1_grid.
> > >    4. We start and activate Part2_grid. Topology also has changed ->
> > BLT#3
> > >    created.
> > >    5. Then we start Part1_grid and it's nodes try to join Part2_grid.
> > >
> > >
> > > If join is successful we have an undefined state of the resulting grid:
> > > values for the same key may (and will) differ between grid parts.
> > >
> > > So to prevent this we should keep nodes with BLT#2 from joining the
> grid
> > > with BLT#3. And we should fail nodes with an error message.
> > >
> > > Thanks,
> > > Sergey.
> > >
> > >
> > >
> > >
> > > On Wed, Aug 23, 2017 at 5:47 AM, Dmitriy Setrakyan <
> > [hidden email]>
> > > wrote:
> > >
> > > > Sergey, I am still confused. What is the BaselineTopology interface
> in
> > > your
> > > > example? I thought that you agreed with me that we simply need a
> setter
> > > for
> > > > activation nodes, no?
> > > >
> > > > D.
> > > >
> > > > On Tue, Aug 22, 2017 at 4:47 AM, Sergey Chugunov <
> > > > [hidden email]>
> > > > wrote:
> > > >
> > > > > Dmitriy,
> > > > >
> > > > > As I understand you use the term "minimalActivationNodes" as a
> > synonym
> > > > for
> > > > > BaselineTopology concept.
> > > > > In that case I agree with you that we can replace both "establish*"
> > > > methods
> > > > > with a simple setter method (see below in summary).
> > > > >
> > > > > Summing up the whole discussion I see the functionality as
> following:
> > > > >
> > > > > New concept BaselineTopology is introduced. The main features it
> > > enables
> > > > > are:
> > > > >
> > > > >    1. automatic activation of cluster;
> > > > >
> > > > >    2. easy management of cluster topology changes (planned nodes
> > > > >    maintenance, adding new nodes etc);
> > > > >
> > > > >    3. eliminating of rebalancing traffic on short-term node
> failures.
> > > > >
> > > > >
> > > > > Use cases to create BLT:
> > > > >
> > > > >    1. user starts up new cluster of desired number of nodes and
> > > activates
> > > > >    it using existing API. BLT is created with all nodes presented
> in
> > > the
> > > > >    cluster at the moment of activation, no API is needed;
> > > > >
> > > > >    2. user prepares BLT using web-console or visor CMD tools and
> sets
> > > it
> > > > to
> > > > >    the cluster. New API setter is needed:
> > > > >    Ignite.activation().setBaselineTopology(BaselineTopology blt);
> > > > >
> > > > >    3. user provides via static configuration a list of nodes that
> are
> > > > >    expected to be in the cluster.
> > > > >    User starts nodes one by one; when all preconfigured nodes are
> > > started
> > > > >    cluster is activated and BLT is created.
> > > > >    As list of nodes may be huge it is provided via separate file to
> > > avoid
> > > > >    flooding main configuration.
> > > > >
> > > > >
> > > > > Igniters, does this description match with your understanding of
> > > > > functionality? If it does I'll create a set of tickets and start
> > > working
> > > > on
> > > > > implementation.
> > > > >
> > > > > Thanks,
> > > > > Sergey.
> > > > >
> > > > >
> > > > > On Sat, Aug 19, 2017 at 5:41 PM, Dmitriy Setrakyan <
> > > > [hidden email]>
> > > > > wrote:
> > > > >
> > > > > > I still do not see why anyone would explicitly call these 2
> > methods:
> > > > > >
> > > > > > *Ignite::activation::establishBaselineTopology();*
> > > > > > *Ignite::activation::establishBaselineTopology(BaselineTopology
> > > > > bltTop);*
> > > > > >
> > > > > > For example, if a web console, or some other admin process, want
> to
> > > > > > automatically set currently started nodes as the baseline
> topology,
> > > > > > shouldn't they just call a setter for minimalActivationNodes?
> > > > > >
> > > > > > D.
> > > > > >
> > > > > > On Fri, Aug 18, 2017 at 10:18 AM, Alexey Dmitriev <
> > > > > [hidden email]>
> > > > > > wrote:
> > > > > >
> > > > > > > API is proposed in the head of the thread by Sergey, as I
> > > understood:
> > > > > > > ______________________________________________________
> > > > > > >
> > > > > > > API for BaselineTopology manipulation may look like this:
> > > > > > >
> > > > > > > *Ignite::activation::establishBaselineTopology();*
> > > > > > > *Ignite::activation::establishBaselineTopology(BaselineTopology
> > > > > > bltTop);*
> > > > > > >
> > > > > > > Both methods will establish BT and activate cluster once it is
> > > > > > established.
> > > > > > >
> > > > > > > The first one allows user to establish BT using current
> topology.
> > > If
> > > > > any
> > > > > > > changes happen to the topology during establishing process,
> user
> > > will
> > > > > be
> > > > > > > notified and allowed to proceed or abort the procedure.
> > > > > > >
> > > > > > > Second method allows to use some monitoring'n'management tools
> > like
> > > > > > > WebConsole where user can prepare a list of nodes, using them
> > > create
> > > > a
> > > > > BT
> > > > > > > and send to the cluster a command to finally establish it.
> > > > > > >
> > > > > > > From high level BaselineTopology entity contains only
> collection
> > of
> > > > > > nodes:
> > > > > > >
> > > > > > > *BaselineTopology {*
> > > > > > > *  Collection<TopologyNode> nodes;*
> > > > > > > *}*
> > > > > > >
> > > > > > > *TopologyNode* here contains information about node - its
> > > consistent
> > > > id
> > > > > > and
> > > > > > > set of user attributes used to calculate affinity function.
> > > > > > > ____________________________________________
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > View this message in context: http://apache-ignite-
> > > > > > > developers.2346864.n4.nabble.com/Cluster-auto-activation-
> > > > > > > design-proposal-tp20295p21066.html
> > > > > > > Sent from the Apache Ignite Developers mailing list archive at
> > > > > > Nabble.com.
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

[Sergey Chugunov]

Nick,

As I summed up in this thread above, calling setter for initial activation
nodes is not the only option:

   1. user starts up new cluster of desired number of nodes and activates
   it using existing API.
   BLT is created with all nodes presented in the cluster at the moment of
   activation, no API is needed;

   2. user prepares BLT using web-console or visor CMD tools and sets it to
   the cluster. New API setter is needed:
   Ignite.activation().setInitialActivationNodes(Collection<ClusterNode>
   nodes);

   3. user provides via static configuration a list of nodes that are
   expected to be in the cluster.
   User starts nodes one by one; when all preconfigured nodes are started
   cluster is activated and BLT is created.
   As list of nodes may be huge it is provided via separate file to avoid
   flooding main configuration.

So the option you proposed is already in the list.

As for idea of activating cluster based only on number of nodes may be
risky.
E.g. if user starts up with data stored on disk and unexpected node joins
the topology.
Cluster will get activated with N-1 nodes where all the data is presented
and one node completely empty. Data loss may happen in such scenario.

Thanks,
Sergey.


On Wed, Aug 30, 2017 at 4:23 PM, Nick Pordash <[hidden email]> wrote:

> How is a user expected to produce a collection of ClusterNode prior to all
> of the expected nodes joining? Users don't create instances of this, so as
> far as I can tell it would have to be retrieved from IgniteCluster.
> However, would doing that and calling the proposed method be really any
> different than calling Ignite.activate and using the current set of server
> nodes as that collection?
>
> From a user's perspective is it really necessary that specific nodes need
> to be identified vs saying that they expect N server nodes to be in the
> cluster for auto activation?
>
> -Nick
>
> On Wed, Aug 30, 2017, 1:23 AM Sergey Chugunov <[hidden email]>
> wrote:
>
> > Dmitriy,
> >
> > Now I see your point and I think you're right.
> >
> > We can give end-user a simple setter like
> > Ignite::activation::setInitialActivationNodes(Collection<ClusterNode>
> > nodes) to provide collection of nodes that grid must reach to activate
> > automatically.
> >
> > And then using the collection we'll create BaselineTopology abstraction
> > internally.
> >
> > As a result user won't be exposed to our internal abstractions and will
> > work with intuitive concept of collection of nodes.
> >
> > Thanks,
> > Sergey.
> >
> >
> > On Mon, Aug 28, 2017 at 4:39 PM, Dmitriy Setrakyan <
> [hidden email]>
> > wrote:
> >
> > > Sergey, the interface you are suggesting is internal, not external. Why
> > > should user ever see it or care about it?
> > >
> > > D.
> > >
> > > On Mon, Aug 28, 2017 at 3:18 PM, Sergey Chugunov <
> > > [hidden email]>
> > > wrote:
> > >
> > > > Dmitriy,
> > > >
> > > > It was my misunderstanding, I believe that setter is not enough and
> we
> > > need
> > > > a full-fledged entity.
> > > >
> > > > We should also be able to check if BLTs are compatible. Interface
> looks
> > > > like this and use case for this functionality is described below.
> > > >
> > > > interface BaselineTopology {
> > > >    Collection<ClusterNode> nodes();
> > > >    boolean isCompatibleWith(BaselineTopology blt);
> > > > }
> > > >
> > > > Let's consider the following scenario:
> > > >
> > > >    1. We have a grid with N nodes: it is up, active and has data in
> it.
> > > ->
> > > >    BLT #1 created.
> > > >    2. We shutdown the grid. Then divide it into two parts: Part1_grid
> > and
> > > >    Part2_grid.
> > > >    3. We start and activate Part1_grid . Topology has changed ->
> BLT#2
> > > >    created.
> > > >    After that we shutdown that Part1_grid.
> > > >    4. We start and activate Part2_grid. Topology also has changed ->
> > > BLT#3
> > > >    created.
> > > >    5. Then we start Part1_grid and it's nodes try to join Part2_grid.
> > > >
> > > >
> > > > If join is successful we have an undefined state of the resulting
> grid:
> > > > values for the same key may (and will) differ between grid parts.
> > > >
> > > > So to prevent this we should keep nodes with BLT#2 from joining the
> > grid
> > > > with BLT#3. And we should fail nodes with an error message.
> > > >
> > > > Thanks,
> > > > Sergey.
> > > >
> > > >
> > > >
> > > >
> > > > On Wed, Aug 23, 2017 at 5:47 AM, Dmitriy Setrakyan <
> > > [hidden email]>
> > > > wrote:
> > > >
> > > > > Sergey, I am still confused. What is the BaselineTopology interface
> > in
> > > > your
> > > > > example? I thought that you agreed with me that we simply need a
> > setter
> > > > for
> > > > > activation nodes, no?
> > > > >
> > > > > D.
> > > > >
> > > > > On Tue, Aug 22, 2017 at 4:47 AM, Sergey Chugunov <
> > > > > [hidden email]>
> > > > > wrote:
> > > > >
> > > > > > Dmitriy,
> > > > > >
> > > > > > As I understand you use the term "minimalActivationNodes" as a
> > > synonym
> > > > > for
> > > > > > BaselineTopology concept.
> > > > > > In that case I agree with you that we can replace both
> "establish*"
> > > > > methods
> > > > > > with a simple setter method (see below in summary).
> > > > > >
> > > > > > Summing up the whole discussion I see the functionality as
> > following:
> > > > > >
> > > > > > New concept BaselineTopology is introduced. The main features it
> > > > enables
> > > > > > are:
> > > > > >
> > > > > >    1. automatic activation of cluster;
> > > > > >
> > > > > >    2. easy management of cluster topology changes (planned nodes
> > > > > >    maintenance, adding new nodes etc);
> > > > > >
> > > > > >    3. eliminating of rebalancing traffic on short-term node
> > failures.
> > > > > >
> > > > > >
> > > > > > Use cases to create BLT:
> > > > > >
> > > > > >    1. user starts up new cluster of desired number of nodes and
> > > > activates
> > > > > >    it using existing API. BLT is created with all nodes presented
> > in
> > > > the
> > > > > >    cluster at the moment of activation, no API is needed;
> > > > > >
> > > > > >    2. user prepares BLT using web-console or visor CMD tools and
> > sets
> > > > it
> > > > > to
> > > > > >    the cluster. New API setter is needed:
> > > > > >    Ignite.activation().setBaselineTopology(BaselineTopology
> blt);
> > > > > >
> > > > > >    3. user provides via static configuration a list of nodes that
> > are
> > > > > >    expected to be in the cluster.
> > > > > >    User starts nodes one by one; when all preconfigured nodes are
> > > > started
> > > > > >    cluster is activated and BLT is created.
> > > > > >    As list of nodes may be huge it is provided via separate file
> to
> > > > avoid
> > > > > >    flooding main configuration.
> > > > > >
> > > > > >
> > > > > > Igniters, does this description match with your understanding of
> > > > > > functionality? If it does I'll create a set of tickets and start
> > > > working
> > > > > on
> > > > > > implementation.
> > > > > >
> > > > > > Thanks,
> > > > > > Sergey.
> > > > > >
> > > > > >
> > > > > > On Sat, Aug 19, 2017 at 5:41 PM, Dmitriy Setrakyan <
> > > > > [hidden email]>
> > > > > > wrote:
> > > > > >
> > > > > > > I still do not see why anyone would explicitly call these 2
> > > methods:
> > > > > > >
> > > > > > > *Ignite::activation::establishBaselineTopology();*
> > > > > > > *Ignite::activation::establishBaselineTopology(
> BaselineTopology
> > > > > > bltTop);*
> > > > > > >
> > > > > > > For example, if a web console, or some other admin process,
> want
> > to
> > > > > > > automatically set currently started nodes as the baseline
> > topology,
> > > > > > > shouldn't they just call a setter for minimalActivationNodes?
> > > > > > >
> > > > > > > D.
> > > > > > >
> > > > > > > On Fri, Aug 18, 2017 at 10:18 AM, Alexey Dmitriev <
> > > > > > [hidden email]>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > API is proposed in the head of the thread by Sergey, as I
> > > > understood:
> > > > > > > > ______________________________________________________
> > > > > > > >
> > > > > > > > API for BaselineTopology manipulation may look like this:
> > > > > > > >
> > > > > > > > *Ignite::activation::establishBaselineTopology();*
> > > > > > > > *Ignite::activation::establishBaselineTopology(
> BaselineTopology
> > > > > > > bltTop);*
> > > > > > > >
> > > > > > > > Both methods will establish BT and activate cluster once it
> is
> > > > > > > established.
> > > > > > > >
> > > > > > > > The first one allows user to establish BT using current
> > topology.
> > > > If
> > > > > > any
> > > > > > > > changes happen to the topology during establishing process,
> > user
> > > > will
> > > > > > be
> > > > > > > > notified and allowed to proceed or abort the procedure.
> > > > > > > >
> > > > > > > > Second method allows to use some monitoring'n'management
> tools
> > > like
> > > > > > > > WebConsole where user can prepare a list of nodes, using them
> > > > create
> > > > > a
> > > > > > BT
> > > > > > > > and send to the cluster a command to finally establish it.
> > > > > > > >
> > > > > > > > From high level BaselineTopology entity contains only
> > collection
> > > of
> > > > > > > nodes:
> > > > > > > >
> > > > > > > > *BaselineTopology {*
> > > > > > > > *  Collection<TopologyNode> nodes;*
> > > > > > > > *}*
> > > > > > > >
> > > > > > > > *TopologyNode* here contains information about node - its
> > > > consistent
> > > > > id
> > > > > > > and
> > > > > > > > set of user attributes used to calculate affinity function.
> > > > > > > > ____________________________________________
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > View this message in context: http://apache-ignite-
> > > > > > > > developers.2346864.n4.nabble.com/Cluster-auto-activation-
> > > > > > > > design-proposal-tp20295p21066.html
> > > > > > > > Sent from the Apache Ignite Developers mailing list archive
> at
> > > > > > > Nabble.com.
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

[dsetrakyan]

Hm... Can we also ask user to optionally provide a predicate which will
receive a collection of nodes started so far and return true if the
activation should happen? Will it be useful?

On Wed, Aug 30, 2017 at 6:28 PM, Sergey Chugunov <[hidden email]>
wrote:

> Nick,
>
> As I summed up in this thread above, calling setter for initial activation
> nodes is not the only option:
>
>    1. user starts up new cluster of desired number of nodes and activates
>    it using existing API.
>    BLT is created with all nodes presented in the cluster at the moment of
>    activation, no API is needed;
>
>    2. user prepares BLT using web-console or visor CMD tools and sets it to
>    the cluster. New API setter is needed:
>    Ignite.activation().setInitialActivationNodes(Collection<ClusterNode>
>    nodes);
>
>    3. user provides via static configuration a list of nodes that are
>    expected to be in the cluster.
>    User starts nodes one by one; when all preconfigured nodes are started
>    cluster is activated and BLT is created.
>    As list of nodes may be huge it is provided via separate file to avoid
>    flooding main configuration.
>
> So the option you proposed is already in the list.
>
> As for idea of activating cluster based only on number of nodes may be
> risky.
> E.g. if user starts up with data stored on disk and unexpected node joins
> the topology.
> Cluster will get activated with N-1 nodes where all the data is presented
> and one node completely empty. Data loss may happen in such scenario.
>
> Thanks,
> Sergey.
>
>
> On Wed, Aug 30, 2017 at 4:23 PM, Nick Pordash <[hidden email]>
> wrote:
>
> > How is a user expected to produce a collection of ClusterNode prior to
> all
> > of the expected nodes joining? Users don't create instances of this, so
> as
> > far as I can tell it would have to be retrieved from IgniteCluster.
> > However, would doing that and calling the proposed method be really any
> > different than calling Ignite.activate and using the current set of
> server
> > nodes as that collection?
> >
> > From a user's perspective is it really necessary that specific nodes need
> > to be identified vs saying that they expect N server nodes to be in the
> > cluster for auto activation?
> >
> > -Nick
> >
> > On Wed, Aug 30, 2017, 1:23 AM Sergey Chugunov <[hidden email]
> >
> > wrote:
> >
> > > Dmitriy,
> > >
> > > Now I see your point and I think you're right.
> > >
> > > We can give end-user a simple setter like
> > > Ignite::activation::setInitialActivationNodes(Collection<ClusterNode>
> > > nodes) to provide collection of nodes that grid must reach to activate
> > > automatically.
> > >
> > > And then using the collection we'll create BaselineTopology abstraction
> > > internally.
> > >
> > > As a result user won't be exposed to our internal abstractions and will
> > > work with intuitive concept of collection of nodes.
> > >
> > > Thanks,
> > > Sergey.
> > >
> > >
> > > On Mon, Aug 28, 2017 at 4:39 PM, Dmitriy Setrakyan <
> > [hidden email]>
> > > wrote:
> > >
> > > > Sergey, the interface you are suggesting is internal, not external.
> Why
> > > > should user ever see it or care about it?
> > > >
> > > > D.
> > > >
> > > > On Mon, Aug 28, 2017 at 3:18 PM, Sergey Chugunov <
> > > > [hidden email]>
> > > > wrote:
> > > >
> > > > > Dmitriy,
> > > > >
> > > > > It was my misunderstanding, I believe that setter is not enough and
> > we
> > > > need
> > > > > a full-fledged entity.
> > > > >
> > > > > We should also be able to check if BLTs are compatible. Interface
> > looks
> > > > > like this and use case for this functionality is described below.
> > > > >
> > > > > interface BaselineTopology {
> > > > >    Collection<ClusterNode> nodes();
> > > > >    boolean isCompatibleWith(BaselineTopology blt);
> > > > > }
> > > > >
> > > > > Let's consider the following scenario:
> > > > >
> > > > >    1. We have a grid with N nodes: it is up, active and has data in
> > it.
> > > > ->
> > > > >    BLT #1 created.
> > > > >    2. We shutdown the grid. Then divide it into two parts:
> Part1_grid
> > > and
> > > > >    Part2_grid.
> > > > >    3. We start and activate Part1_grid . Topology has changed ->
> > BLT#2
> > > > >    created.
> > > > >    After that we shutdown that Part1_grid.
> > > > >    4. We start and activate Part2_grid. Topology also has changed
> ->
> > > > BLT#3
> > > > >    created.
> > > > >    5. Then we start Part1_grid and it's nodes try to join
> Part2_grid.
> > > > >
> > > > >
> > > > > If join is successful we have an undefined state of the resulting
> > grid:
> > > > > values for the same key may (and will) differ between grid parts.
> > > > >
> > > > > So to prevent this we should keep nodes with BLT#2 from joining the
> > > grid
> > > > > with BLT#3. And we should fail nodes with an error message.
> > > > >
> > > > > Thanks,
> > > > > Sergey.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Aug 23, 2017 at 5:47 AM, Dmitriy Setrakyan <
> > > > [hidden email]>
> > > > > wrote:
> > > > >
> > > > > > Sergey, I am still confused. What is the BaselineTopology
> interface
> > > in
> > > > > your
> > > > > > example? I thought that you agreed with me that we simply need a
> > > setter
> > > > > for
> > > > > > activation nodes, no?
> > > > > >
> > > > > > D.
> > > > > >
> > > > > > On Tue, Aug 22, 2017 at 4:47 AM, Sergey Chugunov <
> > > > > > [hidden email]>
> > > > > > wrote:
> > > > > >
> > > > > > > Dmitriy,
> > > > > > >
> > > > > > > As I understand you use the term "minimalActivationNodes" as a
> > > > synonym
> > > > > > for
> > > > > > > BaselineTopology concept.
> > > > > > > In that case I agree with you that we can replace both
> > "establish*"
> > > > > > methods
> > > > > > > with a simple setter method (see below in summary).
> > > > > > >
> > > > > > > Summing up the whole discussion I see the functionality as
> > > following:
> > > > > > >
> > > > > > > New concept BaselineTopology is introduced. The main features
> it
> > > > > enables
> > > > > > > are:
> > > > > > >
> > > > > > >    1. automatic activation of cluster;
> > > > > > >
> > > > > > >    2. easy management of cluster topology changes (planned
> nodes
> > > > > > >    maintenance, adding new nodes etc);
> > > > > > >
> > > > > > >    3. eliminating of rebalancing traffic on short-term node
> > > failures.
> > > > > > >
> > > > > > >
> > > > > > > Use cases to create BLT:
> > > > > > >
> > > > > > >    1. user starts up new cluster of desired number of nodes and
> > > > > activates
> > > > > > >    it using existing API. BLT is created with all nodes
> presented
> > > in
> > > > > the
> > > > > > >    cluster at the moment of activation, no API is needed;
> > > > > > >
> > > > > > >    2. user prepares BLT using web-console or visor CMD tools
> and
> > > sets
> > > > > it
> > > > > > to
> > > > > > >    the cluster. New API setter is needed:
> > > > > > >    Ignite.activation().setBaselineTopology(BaselineTopology
> > blt);
> > > > > > >
> > > > > > >    3. user provides via static configuration a list of nodes
> that
> > > are
> > > > > > >    expected to be in the cluster.
> > > > > > >    User starts nodes one by one; when all preconfigured nodes
> are
> > > > > started
> > > > > > >    cluster is activated and BLT is created.
> > > > > > >    As list of nodes may be huge it is provided via separate
> file
> > to
> > > > > avoid
> > > > > > >    flooding main configuration.
> > > > > > >
> > > > > > >
> > > > > > > Igniters, does this description match with your understanding
> of
> > > > > > > functionality? If it does I'll create a set of tickets and
> start
> > > > > working
> > > > > > on
> > > > > > > implementation.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Sergey.
> > > > > > >
> > > > > > >
> > > > > > > On Sat, Aug 19, 2017 at 5:41 PM, Dmitriy Setrakyan <
> > > > > > [hidden email]>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > I still do not see why anyone would explicitly call these 2
> > > > methods:
> > > > > > > >
> > > > > > > > *Ignite::activation::establishBaselineTopology();*
> > > > > > > > *Ignite::activation::establishBaselineTopology(
> > BaselineTopology
> > > > > > > bltTop);*
> > > > > > > >
> > > > > > > > For example, if a web console, or some other admin process,
> > want
> > > to
> > > > > > > > automatically set currently started nodes as the baseline
> > > topology,
> > > > > > > > shouldn't they just call a setter for minimalActivationNodes?
> > > > > > > >
> > > > > > > > D.
> > > > > > > >
> > > > > > > > On Fri, Aug 18, 2017 at 10:18 AM, Alexey Dmitriev <
> > > > > > > [hidden email]>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > API is proposed in the head of the thread by Sergey, as I
> > > > > understood:
> > > > > > > > > ______________________________________________________
> > > > > > > > >
> > > > > > > > > API for BaselineTopology manipulation may look like this:
> > > > > > > > >
> > > > > > > > > *Ignite::activation::establishBaselineTopology();*
> > > > > > > > > *Ignite::activation::establishBaselineTopology(
> > BaselineTopology
> > > > > > > > bltTop);*
> > > > > > > > >
> > > > > > > > > Both methods will establish BT and activate cluster once it
> > is
> > > > > > > > established.
> > > > > > > > >
> > > > > > > > > The first one allows user to establish BT using current
> > > topology.
> > > > > If
> > > > > > > any
> > > > > > > > > changes happen to the topology during establishing process,
> > > user
> > > > > will
> > > > > > > be
> > > > > > > > > notified and allowed to proceed or abort the procedure.
> > > > > > > > >
> > > > > > > > > Second method allows to use some monitoring'n'management
> > tools
> > > > like
> > > > > > > > > WebConsole where user can prepare a list of nodes, using
> them
> > > > > create
> > > > > > a
> > > > > > > BT
> > > > > > > > > and send to the cluster a command to finally establish it.
> > > > > > > > >
> > > > > > > > > From high level BaselineTopology entity contains only
> > > collection
> > > > of
> > > > > > > > nodes:
> > > > > > > > >
> > > > > > > > > *BaselineTopology {*
> > > > > > > > > *  Collection<TopologyNode> nodes;*
> > > > > > > > > *}*
> > > > > > > > >
> > > > > > > > > *TopologyNode* here contains information about node - its
> > > > > consistent
> > > > > > id
> > > > > > > > and
> > > > > > > > > set of user attributes used to calculate affinity function.
> > > > > > > > > ____________________________________________
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > View this message in context: http://apache-ignite-
> > > > > > > > > developers.2346864.n4.nabble.com/Cluster-auto-activation-
> > > > > > > > > design-proposal-tp20295p21066.html
> > > > > > > > > Sent from the Apache Ignite Developers mailing list archive
> > at
> > > > > > > > Nabble.com.
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

[Sergey Chugunov]

Dmitriy,

The idea is interesting however I cannot come up with a clear use case
which can be widely adopted.
I would give users a simple API at first to cover 80% of their needs and
then collect some feedback and start thinking about adding new
functionality.

Makes sense?

Sergey.

On Thu, Aug 31, 2017 at 3:55 AM, Dmitriy Setrakyan <[hidden email]>
wrote:

> Hm... Can we also ask user to optionally provide a predicate which will
> receive a collection of nodes started so far and return true if the
> activation should happen? Will it be useful?
>
> On Wed, Aug 30, 2017 at 6:28 PM, Sergey Chugunov <
> [hidden email]>
> wrote:
>
> > Nick,
> >
> > As I summed up in this thread above, calling setter for initial
> activation
> > nodes is not the only option:
> >
> >    1. user starts up new cluster of desired number of nodes and activates
> >    it using existing API.
> >    BLT is created with all nodes presented in the cluster at the moment
> of
> >    activation, no API is needed;
> >
> >    2. user prepares BLT using web-console or visor CMD tools and sets it
> to
> >    the cluster. New API setter is needed:
> >    Ignite.activation().setInitialActivationNodes(Collection<ClusterNode>
> >    nodes);
> >
> >    3. user provides via static configuration a list of nodes that are
> >    expected to be in the cluster.
> >    User starts nodes one by one; when all preconfigured nodes are started
> >    cluster is activated and BLT is created.
> >    As list of nodes may be huge it is provided via separate file to avoid
> >    flooding main configuration.
> >
> > So the option you proposed is already in the list.
> >
> > As for idea of activating cluster based only on number of nodes may be
> > risky.
> > E.g. if user starts up with data stored on disk and unexpected node joins
> > the topology.
> > Cluster will get activated with N-1 nodes where all the data is presented
> > and one node completely empty. Data loss may happen in such scenario.
> >
> > Thanks,
> > Sergey.
> >
> >
> > On Wed, Aug 30, 2017 at 4:23 PM, Nick Pordash <[hidden email]>
> > wrote:
> >
> > > How is a user expected to produce a collection of ClusterNode prior to
> > all
> > > of the expected nodes joining? Users don't create instances of this, so
> > as
> > > far as I can tell it would have to be retrieved from IgniteCluster.
> > > However, would doing that and calling the proposed method be really any
> > > different than calling Ignite.activate and using the current set of
> > server
> > > nodes as that collection?
> > >
> > > From a user's perspective is it really necessary that specific nodes
> need
> > > to be identified vs saying that they expect N server nodes to be in the
> > > cluster for auto activation?
> > >
> > > -Nick
> > >
> > > On Wed, Aug 30, 2017, 1:23 AM Sergey Chugunov <
> [hidden email]
> > >
> > > wrote:
> > >
> > > > Dmitriy,
> > > >
> > > > Now I see your point and I think you're right.
> > > >
> > > > We can give end-user a simple setter like
> > > > Ignite::activation::setInitialActivationNodes(
> Collection<ClusterNode>
> > > > nodes) to provide collection of nodes that grid must reach to
> activate
> > > > automatically.
> > > >
> > > > And then using the collection we'll create BaselineTopology
> abstraction
> > > > internally.
> > > >
> > > > As a result user won't be exposed to our internal abstractions and
> will
> > > > work with intuitive concept of collection of nodes.
> > > >
> > > > Thanks,
> > > > Sergey.
> > > >
> > > >
> > > > On Mon, Aug 28, 2017 at 4:39 PM, Dmitriy Setrakyan <
> > > [hidden email]>
> > > > wrote:
> > > >
> > > > > Sergey, the interface you are suggesting is internal, not external.
> > Why
> > > > > should user ever see it or care about it?
> > > > >
> > > > > D.
> > > > >
> > > > > On Mon, Aug 28, 2017 at 3:18 PM, Sergey Chugunov <
> > > > > [hidden email]>
> > > > > wrote:
> > > > >
> > > > > > Dmitriy,
> > > > > >
> > > > > > It was my misunderstanding, I believe that setter is not enough
> and
> > > we
> > > > > need
> > > > > > a full-fledged entity.
> > > > > >
> > > > > > We should also be able to check if BLTs are compatible. Interface
> > > looks
> > > > > > like this and use case for this functionality is described below.
> > > > > >
> > > > > > interface BaselineTopology {
> > > > > >    Collection<ClusterNode> nodes();
> > > > > >    boolean isCompatibleWith(BaselineTopology blt);
> > > > > > }
> > > > > >
> > > > > > Let's consider the following scenario:
> > > > > >
> > > > > >    1. We have a grid with N nodes: it is up, active and has data
> in
> > > it.
> > > > > ->
> > > > > >    BLT #1 created.
> > > > > >    2. We shutdown the grid. Then divide it into two parts:
> > Part1_grid
> > > > and
> > > > > >    Part2_grid.
> > > > > >    3. We start and activate Part1_grid . Topology has changed ->
> > > BLT#2
> > > > > >    created.
> > > > > >    After that we shutdown that Part1_grid.
> > > > > >    4. We start and activate Part2_grid. Topology also has changed
> > ->
> > > > > BLT#3
> > > > > >    created.
> > > > > >    5. Then we start Part1_grid and it's nodes try to join
> > Part2_grid.
> > > > > >
> > > > > >
> > > > > > If join is successful we have an undefined state of the resulting
> > > grid:
> > > > > > values for the same key may (and will) differ between grid parts.
> > > > > >
> > > > > > So to prevent this we should keep nodes with BLT#2 from joining
> the
> > > > grid
> > > > > > with BLT#3. And we should fail nodes with an error message.
> > > > > >
> > > > > > Thanks,
> > > > > > Sergey.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Wed, Aug 23, 2017 at 5:47 AM, Dmitriy Setrakyan <
> > > > > [hidden email]>
> > > > > > wrote:
> > > > > >
> > > > > > > Sergey, I am still confused. What is the BaselineTopology
> > interface
> > > > in
> > > > > > your
> > > > > > > example? I thought that you agreed with me that we simply need
> a
> > > > setter
> > > > > > for
> > > > > > > activation nodes, no?
> > > > > > >
> > > > > > > D.
> > > > > > >
> > > > > > > On Tue, Aug 22, 2017 at 4:47 AM, Sergey Chugunov <
> > > > > > > [hidden email]>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Dmitriy,
> > > > > > > >
> > > > > > > > As I understand you use the term "minimalActivationNodes" as
> a
> > > > > synonym
> > > > > > > for
> > > > > > > > BaselineTopology concept.
> > > > > > > > In that case I agree with you that we can replace both
> > > "establish*"
> > > > > > > methods
> > > > > > > > with a simple setter method (see below in summary).
> > > > > > > >
> > > > > > > > Summing up the whole discussion I see the functionality as
> > > > following:
> > > > > > > >
> > > > > > > > New concept BaselineTopology is introduced. The main features
> > it
> > > > > > enables
> > > > > > > > are:
> > > > > > > >
> > > > > > > >    1. automatic activation of cluster;
> > > > > > > >
> > > > > > > >    2. easy management of cluster topology changes (planned
> > nodes
> > > > > > > >    maintenance, adding new nodes etc);
> > > > > > > >
> > > > > > > >    3. eliminating of rebalancing traffic on short-term node
> > > > failures.
> > > > > > > >
> > > > > > > >
> > > > > > > > Use cases to create BLT:
> > > > > > > >
> > > > > > > >    1. user starts up new cluster of desired number of nodes
> and
> > > > > > activates
> > > > > > > >    it using existing API. BLT is created with all nodes
> > presented
> > > > in
> > > > > > the
> > > > > > > >    cluster at the moment of activation, no API is needed;
> > > > > > > >
> > > > > > > >    2. user prepares BLT using web-console or visor CMD tools
> > and
> > > > sets
> > > > > > it
> > > > > > > to
> > > > > > > >    the cluster. New API setter is needed:
> > > > > > > >    Ignite.activation().setBaselineTopology(BaselineTopology
> > > blt);
> > > > > > > >
> > > > > > > >    3. user provides via static configuration a list of nodes
> > that
> > > > are
> > > > > > > >    expected to be in the cluster.
> > > > > > > >    User starts nodes one by one; when all preconfigured nodes
> > are
> > > > > > started
> > > > > > > >    cluster is activated and BLT is created.
> > > > > > > >    As list of nodes may be huge it is provided via separate
> > file
> > > to
> > > > > > avoid
> > > > > > > >    flooding main configuration.
> > > > > > > >
> > > > > > > >
> > > > > > > > Igniters, does this description match with your understanding
> > of
> > > > > > > > functionality? If it does I'll create a set of tickets and
> > start
> > > > > > working
> > > > > > > on
> > > > > > > > implementation.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Sergey.
> > > > > > > >
> > > > > > > >
> > > > > > > > On Sat, Aug 19, 2017 at 5:41 PM, Dmitriy Setrakyan <
> > > > > > > [hidden email]>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > I still do not see why anyone would explicitly call these 2
> > > > > methods:
> > > > > > > > >
> > > > > > > > > *Ignite::activation::establishBaselineTopology();*
> > > > > > > > > *Ignite::activation::establishBaselineTopology(
> > > BaselineTopology
> > > > > > > > bltTop);*
> > > > > > > > >
> > > > > > > > > For example, if a web console, or some other admin process,
> > > want
> > > > to
> > > > > > > > > automatically set currently started nodes as the baseline
> > > > topology,
> > > > > > > > > shouldn't they just call a setter for
> minimalActivationNodes?
> > > > > > > > >
> > > > > > > > > D.
> > > > > > > > >
> > > > > > > > > On Fri, Aug 18, 2017 at 10:18 AM, Alexey Dmitriev <
> > > > > > > > [hidden email]>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > API is proposed in the head of the thread by Sergey, as I
> > > > > > understood:
> > > > > > > > > > ______________________________________________________
> > > > > > > > > >
> > > > > > > > > > API for BaselineTopology manipulation may look like this:
> > > > > > > > > >
> > > > > > > > > > *Ignite::activation::establishBaselineTopology();*
> > > > > > > > > > *Ignite::activation::establishBaselineTopology(
> > > BaselineTopology
> > > > > > > > > bltTop);*
> > > > > > > > > >
> > > > > > > > > > Both methods will establish BT and activate cluster once
> it
> > > is
> > > > > > > > > established.
> > > > > > > > > >
> > > > > > > > > > The first one allows user to establish BT using current
> > > > topology.
> > > > > > If
> > > > > > > > any
> > > > > > > > > > changes happen to the topology during establishing
> process,
> > > > user
> > > > > > will
> > > > > > > > be
> > > > > > > > > > notified and allowed to proceed or abort the procedure.
> > > > > > > > > >
> > > > > > > > > > Second method allows to use some monitoring'n'management
> > > tools
> > > > > like
> > > > > > > > > > WebConsole where user can prepare a list of nodes, using
> > them
> > > > > > create
> > > > > > > a
> > > > > > > > BT
> > > > > > > > > > and send to the cluster a command to finally establish
> it.
> > > > > > > > > >
> > > > > > > > > > From high level BaselineTopology entity contains only
> > > > collection
> > > > > of
> > > > > > > > > nodes:
> > > > > > > > > >
> > > > > > > > > > *BaselineTopology {*
> > > > > > > > > > *  Collection<TopologyNode> nodes;*
> > > > > > > > > > *}*
> > > > > > > > > >
> > > > > > > > > > *TopologyNode* here contains information about node - its
> > > > > > consistent
> > > > > > > id
> > > > > > > > > and
> > > > > > > > > > set of user attributes used to calculate affinity
> function.
> > > > > > > > > > ____________________________________________
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > View this message in context: http://apache-ignite-
> > > > > > > > > > developers.2346864.n4.nabble.
> com/Cluster-auto-activation-
> > > > > > > > > > design-proposal-tp20295p21066.html
> > > > > > > > > > Sent from the Apache Ignite Developers mailing list
> archive
> > > at
> > > > > > > > > Nabble.com.
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

[dsetrakyan]

How about this:


> *interface ClusterActivator {*
> *    boolean activate(Collection<IgniteNode> nodes);**}*


Out of the box, we can provide this implementation of the activation filter:


>
> *ClusterInitialActiveSet implements ClusterActivator { *
> *    InigeInitialActiveSet(String... addresses);**}*


Then user configuration can look as follows:

*IgniteConfiguration.setActivationFilter(new
> ClusterInitialActiveSet("1.2.3.4", "4.3.2.1", etc));*


Thoughts?

D.

On Fri, Sep 1, 2017 at 1:47 PM, Sergey Chugunov <[hidden email]>
wrote:

> Dmitriy,
>
> The idea is interesting however I cannot come up with a clear use case
> which can be widely adopted.
> I would give users a simple API at first to cover 80% of their needs and
> then collect some feedback and start thinking about adding new
> functionality.
>
> Makes sense?
>
> Sergey.
>
> On Thu, Aug 31, 2017 at 3:55 AM, Dmitriy Setrakyan <[hidden email]>
> wrote:
>
> > Hm... Can we also ask user to optionally provide a predicate which will
> > receive a collection of nodes started so far and return true if the
> > activation should happen? Will it be useful?
> >
> > On Wed, Aug 30, 2017 at 6:28 PM, Sergey Chugunov <
> > [hidden email]>
> > wrote:
> >
> > > Nick,
> > >
> > > As I summed up in this thread above, calling setter for initial
> > activation
> > > nodes is not the only option:
> > >
> > >    1. user starts up new cluster of desired number of nodes and
> activates
> > >    it using existing API.
> > >    BLT is created with all nodes presented in the cluster at the moment
> > of
> > >    activation, no API is needed;
> > >
> > >    2. user prepares BLT using web-console or visor CMD tools and sets
> it
> > to
> > >    the cluster. New API setter is needed:
> > >    Ignite.activation().setInitialActivationNodes(Collection<
> ClusterNode>
> > >    nodes);
> > >
> > >    3. user provides via static configuration a list of nodes that are
> > >    expected to be in the cluster.
> > >    User starts nodes one by one; when all preconfigured nodes are
> started
> > >    cluster is activated and BLT is created.
> > >    As list of nodes may be huge it is provided via separate file to
> avoid
> > >    flooding main configuration.
> > >
> > > So the option you proposed is already in the list.
> > >
> > > As for idea of activating cluster based only on number of nodes may be
> > > risky.
> > > E.g. if user starts up with data stored on disk and unexpected node
> joins
> > > the topology.
> > > Cluster will get activated with N-1 nodes where all the data is
> presented
> > > and one node completely empty. Data loss may happen in such scenario.
> > >
> > > Thanks,
> > > Sergey.
> > >
> > >
> > > On Wed, Aug 30, 2017 at 4:23 PM, Nick Pordash <[hidden email]>
> > > wrote:
> > >
> > > > How is a user expected to produce a collection of ClusterNode prior
> to
> > > all
> > > > of the expected nodes joining? Users don't create instances of this,
> so
> > > as
> > > > far as I can tell it would have to be retrieved from IgniteCluster.
> > > > However, would doing that and calling the proposed method be really
> any
> > > > different than calling Ignite.activate and using the current set of
> > > server
> > > > nodes as that collection?
> > > >
> > > > From a user's perspective is it really necessary that specific nodes
> > need
> > > > to be identified vs saying that they expect N server nodes to be in
> the
> > > > cluster for auto activation?
> > > >
> > > > -Nick
> > > >
> > > > On Wed, Aug 30, 2017, 1:23 AM Sergey Chugunov <
> > [hidden email]
> > > >
> > > > wrote:
> > > >
> > > > > Dmitriy,
> > > > >
> > > > > Now I see your point and I think you're right.
> > > > >
> > > > > We can give end-user a simple setter like
> > > > > Ignite::activation::setInitialActivationNodes(
> > Collection<ClusterNode>
> > > > > nodes) to provide collection of nodes that grid must reach to
> > activate
> > > > > automatically.
> > > > >
> > > > > And then using the collection we'll create BaselineTopology
> > abstraction
> > > > > internally.
> > > > >
> > > > > As a result user won't be exposed to our internal abstractions and
> > will
> > > > > work with intuitive concept of collection of nodes.
> > > > >
> > > > > Thanks,
> > > > > Sergey.
> > > > >
> > > > >
> > > > > On Mon, Aug 28, 2017 at 4:39 PM, Dmitriy Setrakyan <
> > > > [hidden email]>
> > > > > wrote:
> > > > >
> > > > > > Sergey, the interface you are suggesting is internal, not
> external.
> > > Why
> > > > > > should user ever see it or care about it?
> > > > > >
> > > > > > D.
> > > > > >
> > > > > > On Mon, Aug 28, 2017 at 3:18 PM, Sergey Chugunov <
> > > > > > [hidden email]>
> > > > > > wrote:
> > > > > >
> > > > > > > Dmitriy,
> > > > > > >
> > > > > > > It was my misunderstanding, I believe that setter is not enough
> > and
> > > > we
> > > > > > need
> > > > > > > a full-fledged entity.
> > > > > > >
> > > > > > > We should also be able to check if BLTs are compatible.
> Interface
> > > > looks
> > > > > > > like this and use case for this functionality is described
> below.
> > > > > > >
> > > > > > > interface BaselineTopology {
> > > > > > >    Collection<ClusterNode> nodes();
> > > > > > >    boolean isCompatibleWith(BaselineTopology blt);
> > > > > > > }
> > > > > > >
> > > > > > > Let's consider the following scenario:
> > > > > > >
> > > > > > >    1. We have a grid with N nodes: it is up, active and has
> data
> > in
> > > > it.
> > > > > > ->
> > > > > > >    BLT #1 created.
> > > > > > >    2. We shutdown the grid. Then divide it into two parts:
> > > Part1_grid
> > > > > and
> > > > > > >    Part2_grid.
> > > > > > >    3. We start and activate Part1_grid . Topology has changed
> ->
> > > > BLT#2
> > > > > > >    created.
> > > > > > >    After that we shutdown that Part1_grid.
> > > > > > >    4. We start and activate Part2_grid. Topology also has
> changed
> > > ->
> > > > > > BLT#3
> > > > > > >    created.
> > > > > > >    5. Then we start Part1_grid and it's nodes try to join
> > > Part2_grid.
> > > > > > >
> > > > > > >
> > > > > > > If join is successful we have an undefined state of the
> resulting
> > > > grid:
> > > > > > > values for the same key may (and will) differ between grid
> parts.
> > > > > > >
> > > > > > > So to prevent this we should keep nodes with BLT#2 from joining
> > the
> > > > > grid
> > > > > > > with BLT#3. And we should fail nodes with an error message.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Sergey.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Aug 23, 2017 at 5:47 AM, Dmitriy Setrakyan <
> > > > > > [hidden email]>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Sergey, I am still confused. What is the BaselineTopology
> > > interface
> > > > > in
> > > > > > > your
> > > > > > > > example? I thought that you agreed with me that we simply
> need
> > a
> > > > > setter
> > > > > > > for
> > > > > > > > activation nodes, no?
> > > > > > > >
> > > > > > > > D.
> > > > > > > >
> > > > > > > > On Tue, Aug 22, 2017 at 4:47 AM, Sergey Chugunov <
> > > > > > > > [hidden email]>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Dmitriy,
> > > > > > > > >
> > > > > > > > > As I understand you use the term "minimalActivationNodes"
> as
> > a
> > > > > > synonym
> > > > > > > > for
> > > > > > > > > BaselineTopology concept.
> > > > > > > > > In that case I agree with you that we can replace both
> > > > "establish*"
> > > > > > > > methods
> > > > > > > > > with a simple setter method (see below in summary).
> > > > > > > > >
> > > > > > > > > Summing up the whole discussion I see the functionality as
> > > > > following:
> > > > > > > > >
> > > > > > > > > New concept BaselineTopology is introduced. The main
> features
> > > it
> > > > > > > enables
> > > > > > > > > are:
> > > > > > > > >
> > > > > > > > >    1. automatic activation of cluster;
> > > > > > > > >
> > > > > > > > >    2. easy management of cluster topology changes (planned
> > > nodes
> > > > > > > > >    maintenance, adding new nodes etc);
> > > > > > > > >
> > > > > > > > >    3. eliminating of rebalancing traffic on short-term node
> > > > > failures.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Use cases to create BLT:
> > > > > > > > >
> > > > > > > > >    1. user starts up new cluster of desired number of nodes
> > and
> > > > > > > activates
> > > > > > > > >    it using existing API. BLT is created with all nodes
> > > presented
> > > > > in
> > > > > > > the
> > > > > > > > >    cluster at the moment of activation, no API is needed;
> > > > > > > > >
> > > > > > > > >    2. user prepares BLT using web-console or visor CMD
> tools
> > > and
> > > > > sets
> > > > > > > it
> > > > > > > > to
> > > > > > > > >    the cluster. New API setter is needed:
> > > > > > > > >    Ignite.activation().setBaselin
> eTopology(BaselineTopology
> > > > blt);
> > > > > > > > >
> > > > > > > > >    3. user provides via static configuration a list of
> nodes
> > > that
> > > > > are
> > > > > > > > >    expected to be in the cluster.
> > > > > > > > >    User starts nodes one by one; when all preconfigured
> nodes
> > > are
> > > > > > > started
> > > > > > > > >    cluster is activated and BLT is created.
> > > > > > > > >    As list of nodes may be huge it is provided via separate
> > > file
> > > > to
> > > > > > > avoid
> > > > > > > > >    flooding main configuration.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Igniters, does this description match with your
> understanding
> > > of
> > > > > > > > > functionality? If it does I'll create a set of tickets and
> > > start
> > > > > > > working
> > > > > > > > on
> > > > > > > > > implementation.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Sergey.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Sat, Aug 19, 2017 at 5:41 PM, Dmitriy Setrakyan <
> > > > > > > > [hidden email]>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > I still do not see why anyone would explicitly call
> these 2
> > > > > > methods:
> > > > > > > > > >
> > > > > > > > > > *Ignite::activation::establishBaselineTopology();*
> > > > > > > > > > *Ignite::activation::establishBaselineTopology(
> > > > BaselineTopology
> > > > > > > > > bltTop);*
> > > > > > > > > >
> > > > > > > > > > For example, if a web console, or some other admin
> process,
> > > > want
> > > > > to
> > > > > > > > > > automatically set currently started nodes as the baseline
> > > > > topology,
> > > > > > > > > > shouldn't they just call a setter for
> > minimalActivationNodes?
> > > > > > > > > >
> > > > > > > > > > D.
> > > > > > > > > >
> > > > > > > > > > On Fri, Aug 18, 2017 at 10:18 AM, Alexey Dmitriev <
> > > > > > > > > [hidden email]>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > API is proposed in the head of the thread by Sergey,
> as I
> > > > > > > understood:
> > > > > > > > > > > ______________________________________________________
> > > > > > > > > > >
> > > > > > > > > > > API for BaselineTopology manipulation may look like
> this:
> > > > > > > > > > >
> > > > > > > > > > > *Ignite::activation::establishBaselineTopology();*
> > > > > > > > > > > *Ignite::activation::establishBaselineTopology(
> > > > BaselineTopology
> > > > > > > > > > bltTop);*
> > > > > > > > > > >
> > > > > > > > > > > Both methods will establish BT and activate cluster
> once
> > it
> > > > is
> > > > > > > > > > established.
> > > > > > > > > > >
> > > > > > > > > > > The first one allows user to establish BT using current
> > > > > topology.
> > > > > > > If
> > > > > > > > > any
> > > > > > > > > > > changes happen to the topology during establishing
> > process,
> > > > > user
> > > > > > > will
> > > > > > > > > be
> > > > > > > > > > > notified and allowed to proceed or abort the procedure.
> > > > > > > > > > >
> > > > > > > > > > > Second method allows to use some
> monitoring'n'management
> > > > tools
> > > > > > like
> > > > > > > > > > > WebConsole where user can prepare a list of nodes,
> using
> > > them
> > > > > > > create
> > > > > > > > a
> > > > > > > > > BT
> > > > > > > > > > > and send to the cluster a command to finally establish
> > it.
> > > > > > > > > > >
> > > > > > > > > > > From high level BaselineTopology entity contains only
> > > > > collection
> > > > > > of
> > > > > > > > > > nodes:
> > > > > > > > > > >
> > > > > > > > > > > *BaselineTopology {*
> > > > > > > > > > > *  Collection<TopologyNode> nodes;*
> > > > > > > > > > > *}*
> > > > > > > > > > >
> > > > > > > > > > > *TopologyNode* here contains information about node -
> its
> > > > > > > consistent
> > > > > > > > id
> > > > > > > > > > and
> > > > > > > > > > > set of user attributes used to calculate affinity
> > function.
> > > > > > > > > > > ____________________________________________
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > --
> > > > > > > > > > > View this message in context: http://apache-ignite-
> > > > > > > > > > > developers.2346864.n4.nabble.
> > com/Cluster-auto-activation-
> > > > > > > > > > > design-proposal-tp20295p21066.html
> > > > > > > > > > > Sent from the Apache Ignite Developers mailing list
> > archive
> > > > at
> > > > > > > > > > Nabble.com.
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

[Sergey Chugunov]

Dmitriy,

I like the idea of ClusterActivator interface.

From user perspective it provides the same functionality as the setter but
in more clear and intuitive way.

Also it gives us a good place to put all the documentation about the
feature.

Any other opinions?

On Fri, Sep 1, 2017 at 2:35 PM, Dmitriy Setrakyan <[hidden email]>
wrote:

> How about this:
>
>
> > *interface ClusterActivator {*
> > *    boolean activate(Collection<IgniteNode> nodes);**}*
>
>
> Out of the box, we can provide this implementation of the activation
> filter:
>
>
> >
> > *ClusterInitialActiveSet implements ClusterActivator { *
> > *    InigeInitialActiveSet(String... addresses);**}*
>
>
> Then user configuration can look as follows:
>
> *IgniteConfiguration.setActivationFilter(new
> > ClusterInitialActiveSet("1.2.3.4", "4.3.2.1", etc));*
>
>
> Thoughts?
>
> D.
>
> On Fri, Sep 1, 2017 at 1:47 PM, Sergey Chugunov <[hidden email]
> >
> wrote:
>
> > Dmitriy,
> >
> > The idea is interesting however I cannot come up with a clear use case
> > which can be widely adopted.
> > I would give users a simple API at first to cover 80% of their needs and
> > then collect some feedback and start thinking about adding new
> > functionality.
> >
> > Makes sense?
> >
> > Sergey.
> >
> > On Thu, Aug 31, 2017 at 3:55 AM, Dmitriy Setrakyan <
> [hidden email]>
> > wrote:
> >
> > > Hm... Can we also ask user to optionally provide a predicate which will
> > > receive a collection of nodes started so far and return true if the
> > > activation should happen? Will it be useful?
> > >
> > > On Wed, Aug 30, 2017 at 6:28 PM, Sergey Chugunov <
> > > [hidden email]>
> > > wrote:
> > >
> > > > Nick,
> > > >
> > > > As I summed up in this thread above, calling setter for initial
> > > activation
> > > > nodes is not the only option:
> > > >
> > > >    1. user starts up new cluster of desired number of nodes and
> > activates
> > > >    it using existing API.
> > > >    BLT is created with all nodes presented in the cluster at the
> moment
> > > of
> > > >    activation, no API is needed;
> > > >
> > > >    2. user prepares BLT using web-console or visor CMD tools and sets
> > it
> > > to
> > > >    the cluster. New API setter is needed:
> > > >    Ignite.activation().setInitialActivationNodes(Collection<
> > ClusterNode>
> > > >    nodes);
> > > >
> > > >    3. user provides via static configuration a list of nodes that are
> > > >    expected to be in the cluster.
> > > >    User starts nodes one by one; when all preconfigured nodes are
> > started
> > > >    cluster is activated and BLT is created.
> > > >    As list of nodes may be huge it is provided via separate file to
> > avoid
> > > >    flooding main configuration.
> > > >
> > > > So the option you proposed is already in the list.
> > > >
> > > > As for idea of activating cluster based only on number of nodes may
> be
> > > > risky.
> > > > E.g. if user starts up with data stored on disk and unexpected node
> > joins
> > > > the topology.
> > > > Cluster will get activated with N-1 nodes where all the data is
> > presented
> > > > and one node completely empty. Data loss may happen in such scenario.
> > > >
> > > > Thanks,
> > > > Sergey.
> > > >
> > > >
> > > > On Wed, Aug 30, 2017 at 4:23 PM, Nick Pordash <[hidden email]
> >
> > > > wrote:
> > > >
> > > > > How is a user expected to produce a collection of ClusterNode prior
> > to
> > > > all
> > > > > of the expected nodes joining? Users don't create instances of
> this,
> > so
> > > > as
> > > > > far as I can tell it would have to be retrieved from IgniteCluster.
> > > > > However, would doing that and calling the proposed method be really
> > any
> > > > > different than calling Ignite.activate and using the current set of
> > > > server
> > > > > nodes as that collection?
> > > > >
> > > > > From a user's perspective is it really necessary that specific
> nodes
> > > need
> > > > > to be identified vs saying that they expect N server nodes to be in
> > the
> > > > > cluster for auto activation?
> > > > >
> > > > > -Nick
> > > > >
> > > > > On Wed, Aug 30, 2017, 1:23 AM Sergey Chugunov <
> > > [hidden email]
> > > > >
> > > > > wrote:
> > > > >
> > > > > > Dmitriy,
> > > > > >
> > > > > > Now I see your point and I think you're right.
> > > > > >
> > > > > > We can give end-user a simple setter like
> > > > > > Ignite::activation::setInitialActivationNodes(
> > > Collection<ClusterNode>
> > > > > > nodes) to provide collection of nodes that grid must reach to
> > > activate
> > > > > > automatically.
> > > > > >
> > > > > > And then using the collection we'll create BaselineTopology
> > > abstraction
> > > > > > internally.
> > > > > >
> > > > > > As a result user won't be exposed to our internal abstractions
> and
> > > will
> > > > > > work with intuitive concept of collection of nodes.
> > > > > >
> > > > > > Thanks,
> > > > > > Sergey.
> > > > > >
> > > > > >
> > > > > > On Mon, Aug 28, 2017 at 4:39 PM, Dmitriy Setrakyan <
> > > > > [hidden email]>
> > > > > > wrote:
> > > > > >
> > > > > > > Sergey, the interface you are suggesting is internal, not
> > external.
> > > > Why
> > > > > > > should user ever see it or care about it?
> > > > > > >
> > > > > > > D.
> > > > > > >
> > > > > > > On Mon, Aug 28, 2017 at 3:18 PM, Sergey Chugunov <
> > > > > > > [hidden email]>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Dmitriy,
> > > > > > > >
> > > > > > > > It was my misunderstanding, I believe that setter is not
> enough
> > > and
> > > > > we
> > > > > > > need
> > > > > > > > a full-fledged entity.
> > > > > > > >
> > > > > > > > We should also be able to check if BLTs are compatible.
> > Interface
> > > > > looks
> > > > > > > > like this and use case for this functionality is described
> > below.
> > > > > > > >
> > > > > > > > interface BaselineTopology {
> > > > > > > >    Collection<ClusterNode> nodes();
> > > > > > > >    boolean isCompatibleWith(BaselineTopology blt);
> > > > > > > > }
> > > > > > > >
> > > > > > > > Let's consider the following scenario:
> > > > > > > >
> > > > > > > >    1. We have a grid with N nodes: it is up, active and has
> > data
> > > in
> > > > > it.
> > > > > > > ->
> > > > > > > >    BLT #1 created.
> > > > > > > >    2. We shutdown the grid. Then divide it into two parts:
> > > > Part1_grid
> > > > > > and
> > > > > > > >    Part2_grid.
> > > > > > > >    3. We start and activate Part1_grid . Topology has changed
> > ->
> > > > > BLT#2
> > > > > > > >    created.
> > > > > > > >    After that we shutdown that Part1_grid.
> > > > > > > >    4. We start and activate Part2_grid. Topology also has
> > changed
> > > > ->
> > > > > > > BLT#3
> > > > > > > >    created.
> > > > > > > >    5. Then we start Part1_grid and it's nodes try to join
> > > > Part2_grid.
> > > > > > > >
> > > > > > > >
> > > > > > > > If join is successful we have an undefined state of the
> > resulting
> > > > > grid:
> > > > > > > > values for the same key may (and will) differ between grid
> > parts.
> > > > > > > >
> > > > > > > > So to prevent this we should keep nodes with BLT#2 from
> joining
> > > the
> > > > > > grid
> > > > > > > > with BLT#3. And we should fail nodes with an error message.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Sergey.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Aug 23, 2017 at 5:47 AM, Dmitriy Setrakyan <
> > > > > > > [hidden email]>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Sergey, I am still confused. What is the BaselineTopology
> > > > interface
> > > > > > in
> > > > > > > > your
> > > > > > > > > example? I thought that you agreed with me that we simply
> > need
> > > a
> > > > > > setter
> > > > > > > > for
> > > > > > > > > activation nodes, no?
> > > > > > > > >
> > > > > > > > > D.
> > > > > > > > >
> > > > > > > > > On Tue, Aug 22, 2017 at 4:47 AM, Sergey Chugunov <
> > > > > > > > > [hidden email]>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Dmitriy,
> > > > > > > > > >
> > > > > > > > > > As I understand you use the term "minimalActivationNodes"
> > as
> > > a
> > > > > > > synonym
> > > > > > > > > for
> > > > > > > > > > BaselineTopology concept.
> > > > > > > > > > In that case I agree with you that we can replace both
> > > > > "establish*"
> > > > > > > > > methods
> > > > > > > > > > with a simple setter method (see below in summary).
> > > > > > > > > >
> > > > > > > > > > Summing up the whole discussion I see the functionality
> as
> > > > > > following:
> > > > > > > > > >
> > > > > > > > > > New concept BaselineTopology is introduced. The main
> > features
> > > > it
> > > > > > > > enables
> > > > > > > > > > are:
> > > > > > > > > >
> > > > > > > > > >    1. automatic activation of cluster;
> > > > > > > > > >
> > > > > > > > > >    2. easy management of cluster topology changes
> (planned
> > > > nodes
> > > > > > > > > >    maintenance, adding new nodes etc);
> > > > > > > > > >
> > > > > > > > > >    3. eliminating of rebalancing traffic on short-term
> node
> > > > > > failures.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Use cases to create BLT:
> > > > > > > > > >
> > > > > > > > > >    1. user starts up new cluster of desired number of
> nodes
> > > and
> > > > > > > > activates
> > > > > > > > > >    it using existing API. BLT is created with all nodes
> > > > presented
> > > > > > in
> > > > > > > > the
> > > > > > > > > >    cluster at the moment of activation, no API is needed;
> > > > > > > > > >
> > > > > > > > > >    2. user prepares BLT using web-console or visor CMD
> > tools
> > > > and
> > > > > > sets
> > > > > > > > it
> > > > > > > > > to
> > > > > > > > > >    the cluster. New API setter is needed:
> > > > > > > > > >    Ignite.activation().setBaselin
> > eTopology(BaselineTopology
> > > > > blt);
> > > > > > > > > >
> > > > > > > > > >    3. user provides via static configuration a list of
> > nodes
> > > > that
> > > > > > are
> > > > > > > > > >    expected to be in the cluster.
> > > > > > > > > >    User starts nodes one by one; when all preconfigured
> > nodes
> > > > are
> > > > > > > > started
> > > > > > > > > >    cluster is activated and BLT is created.
> > > > > > > > > >    As list of nodes may be huge it is provided via
> separate
> > > > file
> > > > > to
> > > > > > > > avoid
> > > > > > > > > >    flooding main configuration.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Igniters, does this description match with your
> > understanding
> > > > of
> > > > > > > > > > functionality? If it does I'll create a set of tickets
> and
> > > > start
> > > > > > > > working
> > > > > > > > > on
> > > > > > > > > > implementation.
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > Sergey.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Sat, Aug 19, 2017 at 5:41 PM, Dmitriy Setrakyan <
> > > > > > > > > [hidden email]>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > I still do not see why anyone would explicitly call
> > these 2
> > > > > > > methods:
> > > > > > > > > > >
> > > > > > > > > > > *Ignite::activation::establishBaselineTopology();*
> > > > > > > > > > > *Ignite::activation::establishBaselineTopology(
> > > > > BaselineTopology
> > > > > > > > > > bltTop);*
> > > > > > > > > > >
> > > > > > > > > > > For example, if a web console, or some other admin
> > process,
> > > > > want
> > > > > > to
> > > > > > > > > > > automatically set currently started nodes as the
> baseline
> > > > > > topology,
> > > > > > > > > > > shouldn't they just call a setter for
> > > minimalActivationNodes?
> > > > > > > > > > >
> > > > > > > > > > > D.
> > > > > > > > > > >
> > > > > > > > > > > On Fri, Aug 18, 2017 at 10:18 AM, Alexey Dmitriev <
> > > > > > > > > > [hidden email]>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > API is proposed in the head of the thread by Sergey,
> > as I
> > > > > > > > understood:
> > > > > > > > > > > > ______________________________
> ________________________
> > > > > > > > > > > >
> > > > > > > > > > > > API for BaselineTopology manipulation may look like
> > this:
> > > > > > > > > > > >
> > > > > > > > > > > > *Ignite::activation::establishBaselineTopology();*
> > > > > > > > > > > > *Ignite::activation::establishBaselineTopology(
> > > > > BaselineTopology
> > > > > > > > > > > bltTop);*
> > > > > > > > > > > >
> > > > > > > > > > > > Both methods will establish BT and activate cluster
> > once
> > > it
> > > > > is
> > > > > > > > > > > established.
> > > > > > > > > > > >
> > > > > > > > > > > > The first one allows user to establish BT using
> current
> > > > > > topology.
> > > > > > > > If
> > > > > > > > > > any
> > > > > > > > > > > > changes happen to the topology during establishing
> > > process,
> > > > > > user
> > > > > > > > will
> > > > > > > > > > be
> > > > > > > > > > > > notified and allowed to proceed or abort the
> procedure.
> > > > > > > > > > > >
> > > > > > > > > > > > Second method allows to use some
> > monitoring'n'management
> > > > > tools
> > > > > > > like
> > > > > > > > > > > > WebConsole where user can prepare a list of nodes,
> > using
> > > > them
> > > > > > > > create
> > > > > > > > > a
> > > > > > > > > > BT
> > > > > > > > > > > > and send to the cluster a command to finally
> establish
> > > it.
> > > > > > > > > > > >
> > > > > > > > > > > > From high level BaselineTopology entity contains only
> > > > > > collection
> > > > > > > of
> > > > > > > > > > > nodes:
> > > > > > > > > > > >
> > > > > > > > > > > > *BaselineTopology {*
> > > > > > > > > > > > *  Collection<TopologyNode> nodes;*
> > > > > > > > > > > > *}*
> > > > > > > > > > > >
> > > > > > > > > > > > *TopologyNode* here contains information about node -
> > its
> > > > > > > > consistent
> > > > > > > > > id
> > > > > > > > > > > and
> > > > > > > > > > > > set of user attributes used to calculate affinity
> > > function.
> > > > > > > > > > > > ____________________________________________
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > --
> > > > > > > > > > > > View this message in context: http://apache-ignite-
> > > > > > > > > > > > developers.2346864.n4.nabble.
> > > com/Cluster-auto-activation-
> > > > > > > > > > > > design-proposal-tp20295p21066.html
> > > > > > > > > > > > Sent from the Apache Ignite Developers mailing list
> > > archive
> > > > > at
> > > > > > > > > > > Nabble.com.
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

[dsetrakyan]

On Mon, Sep 4, 2017 at 6:13 AM, Sergey Chugunov <[hidden email]>
wrote:

> Dmitriy,
>
> I like the idea of ClusterActivator interface.
>
> From user perspective it provides the same functionality as the setter but
> in more clear and intuitive way.
>

BTW, I made a naming mistake in the original email. The setter name should
be "setClusterActivator(...).


>
> Also it gives us a good place to put all the documentation about the
> feature.
>

Agree. Another advantage is that users can now provide custom logic for the
initial cluster activation.


>
> Any other opinions?
>

Alexey Goncharuk, given that you provided the initial implementation of the
cluster activation, can you please take a look at this design and provide
comments?

>
> On Fri, Sep 1, 2017 at 2:35 PM, Dmitriy Setrakyan <[hidden email]>
> wrote:
>
> > How about this:
> >
> >
> > > *interface ClusterActivator {*
> > > *    boolean activate(Collection<IgniteNode> nodes);**}*
> >
> >
> > Out of the box, we can provide this implementation of the activation
> > filter:
> >
> >
> > >
> > > *ClusterInitialActiveSet implements ClusterActivator { *
> > > *    InigeInitialActiveSet(String... addresses);**}*
> >
> >
> > Then user configuration can look as follows:
> >
> > *IgniteConfiguration.setActivationFilter(new
> > > ClusterInitialActiveSet("1.2.3.4", "4.3.2.1", etc));*
> >
> >
> > Thoughts?
> >
> > D.
> >
> > On Fri, Sep 1, 2017 at 1:47 PM, Sergey Chugunov <
> [hidden email]
> > >
> > wrote:
> >
> > > Dmitriy,
> > >
> > > The idea is interesting however I cannot come up with a clear use case
> > > which can be widely adopted.
> > > I would give users a simple API at first to cover 80% of their needs
> and
> > > then collect some feedback and start thinking about adding new
> > > functionality.
> > >
> > > Makes sense?
> > >
> > > Sergey.
> > >
> > > On Thu, Aug 31, 2017 at 3:55 AM, Dmitriy Setrakyan <
> > [hidden email]>
> > > wrote:
> > >
> > > > Hm... Can we also ask user to optionally provide a predicate which
> will
> > > > receive a collection of nodes started so far and return true if the
> > > > activation should happen? Will it be useful?
> > > >
> > > > On Wed, Aug 30, 2017 at 6:28 PM, Sergey Chugunov <
> > > > [hidden email]>
> > > > wrote:
> > > >
> > > > > Nick,
> > > > >
> > > > > As I summed up in this thread above, calling setter for initial
> > > > activation
> > > > > nodes is not the only option:
> > > > >
> > > > >    1. user starts up new cluster of desired number of nodes and
> > > activates
> > > > >    it using existing API.
> > > > >    BLT is created with all nodes presented in the cluster at the
> > moment
> > > > of
> > > > >    activation, no API is needed;
> > > > >
> > > > >    2. user prepares BLT using web-console or visor CMD tools and
> sets
> > > it
> > > > to
> > > > >    the cluster. New API setter is needed:
> > > > >    Ignite.activation().setInitialActivationNodes(Collection<
> > > ClusterNode>
> > > > >    nodes);
> > > > >
> > > > >    3. user provides via static configuration a list of nodes that
> are
> > > > >    expected to be in the cluster.
> > > > >    User starts nodes one by one; when all preconfigured nodes are
> > > started
> > > > >    cluster is activated and BLT is created.
> > > > >    As list of nodes may be huge it is provided via separate file to
> > > avoid
> > > > >    flooding main configuration.
> > > > >
> > > > > So the option you proposed is already in the list.
> > > > >
> > > > > As for idea of activating cluster based only on number of nodes may
> > be
> > > > > risky.
> > > > > E.g. if user starts up with data stored on disk and unexpected node
> > > joins
> > > > > the topology.
> > > > > Cluster will get activated with N-1 nodes where all the data is
> > > presented
> > > > > and one node completely empty. Data loss may happen in such
> scenario.
> > > > >
> > > > > Thanks,
> > > > > Sergey.
> > > > >
> > > > >
> > > > > On Wed, Aug 30, 2017 at 4:23 PM, Nick Pordash <
> [hidden email]
> > >
> > > > > wrote:
> > > > >
> > > > > > How is a user expected to produce a collection of ClusterNode
> prior
> > > to
> > > > > all
> > > > > > of the expected nodes joining? Users don't create instances of
> > this,
> > > so
> > > > > as
> > > > > > far as I can tell it would have to be retrieved from
> IgniteCluster.
> > > > > > However, would doing that and calling the proposed method be
> really
> > > any
> > > > > > different than calling Ignite.activate and using the current set
> of
> > > > > server
> > > > > > nodes as that collection?
> > > > > >
> > > > > > From a user's perspective is it really necessary that specific
> > nodes
> > > > need
> > > > > > to be identified vs saying that they expect N server nodes to be
> in
> > > the
> > > > > > cluster for auto activation?
> > > > > >
> > > > > > -Nick
> > > > > >
> > > > > > On Wed, Aug 30, 2017, 1:23 AM Sergey Chugunov <
> > > > [hidden email]
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Dmitriy,
> > > > > > >
> > > > > > > Now I see your point and I think you're right.
> > > > > > >
> > > > > > > We can give end-user a simple setter like
> > > > > > > Ignite::activation::setInitialActivationNodes(
> > > > Collection<ClusterNode>
> > > > > > > nodes) to provide collection of nodes that grid must reach to
> > > > activate
> > > > > > > automatically.
> > > > > > >
> > > > > > > And then using the collection we'll create BaselineTopology
> > > > abstraction
> > > > > > > internally.
> > > > > > >
> > > > > > > As a result user won't be exposed to our internal abstractions
> > and
> > > > will
> > > > > > > work with intuitive concept of collection of nodes.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Sergey.
> > > > > > >
> > > > > > >
> > > > > > > On Mon, Aug 28, 2017 at 4:39 PM, Dmitriy Setrakyan <
> > > > > > [hidden email]>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Sergey, the interface you are suggesting is internal, not
> > > external.
> > > > > Why
> > > > > > > > should user ever see it or care about it?
> > > > > > > >
> > > > > > > > D.
> > > > > > > >
> > > > > > > > On Mon, Aug 28, 2017 at 3:18 PM, Sergey Chugunov <
> > > > > > > > [hidden email]>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Dmitriy,
> > > > > > > > >
> > > > > > > > > It was my misunderstanding, I believe that setter is not
> > enough
> > > > and
> > > > > > we
> > > > > > > > need
> > > > > > > > > a full-fledged entity.
> > > > > > > > >
> > > > > > > > > We should also be able to check if BLTs are compatible.
> > > Interface
> > > > > > looks
> > > > > > > > > like this and use case for this functionality is described
> > > below.
> > > > > > > > >
> > > > > > > > > interface BaselineTopology {
> > > > > > > > >    Collection<ClusterNode> nodes();
> > > > > > > > >    boolean isCompatibleWith(BaselineTopology blt);
> > > > > > > > > }
> > > > > > > > >
> > > > > > > > > Let's consider the following scenario:
> > > > > > > > >
> > > > > > > > >    1. We have a grid with N nodes: it is up, active and has
> > > data
> > > > in
> > > > > > it.
> > > > > > > > ->
> > > > > > > > >    BLT #1 created.
> > > > > > > > >    2. We shutdown the grid. Then divide it into two parts:
> > > > > Part1_grid
> > > > > > > and
> > > > > > > > >    Part2_grid.
> > > > > > > > >    3. We start and activate Part1_grid . Topology has
> changed
> > > ->
> > > > > > BLT#2
> > > > > > > > >    created.
> > > > > > > > >    After that we shutdown that Part1_grid.
> > > > > > > > >    4. We start and activate Part2_grid. Topology also has
> > > changed
> > > > > ->
> > > > > > > > BLT#3
> > > > > > > > >    created.
> > > > > > > > >    5. Then we start Part1_grid and it's nodes try to join
> > > > > Part2_grid.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > If join is successful we have an undefined state of the
> > > resulting
> > > > > > grid:
> > > > > > > > > values for the same key may (and will) differ between grid
> > > parts.
> > > > > > > > >
> > > > > > > > > So to prevent this we should keep nodes with BLT#2 from
> > joining
> > > > the
> > > > > > > grid
> > > > > > > > > with BLT#3. And we should fail nodes with an error message.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Sergey.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Wed, Aug 23, 2017 at 5:47 AM, Dmitriy Setrakyan <
> > > > > > > > [hidden email]>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Sergey, I am still confused. What is the BaselineTopology
> > > > > interface
> > > > > > > in
> > > > > > > > > your
> > > > > > > > > > example? I thought that you agreed with me that we simply
> > > need
> > > > a
> > > > > > > setter
> > > > > > > > > for
> > > > > > > > > > activation nodes, no?
> > > > > > > > > >
> > > > > > > > > > D.
> > > > > > > > > >
> > > > > > > > > > On Tue, Aug 22, 2017 at 4:47 AM, Sergey Chugunov <
> > > > > > > > > > [hidden email]>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Dmitriy,
> > > > > > > > > > >
> > > > > > > > > > > As I understand you use the term
> "minimalActivationNodes"
> > > as
> > > > a
> > > > > > > > synonym
> > > > > > > > > > for
> > > > > > > > > > > BaselineTopology concept.
> > > > > > > > > > > In that case I agree with you that we can replace both
> > > > > > "establish*"
> > > > > > > > > > methods
> > > > > > > > > > > with a simple setter method (see below in summary).
> > > > > > > > > > >
> > > > > > > > > > > Summing up the whole discussion I see the functionality
> > as
> > > > > > > following:
> > > > > > > > > > >
> > > > > > > > > > > New concept BaselineTopology is introduced. The main
> > > features
> > > > > it
> > > > > > > > > enables
> > > > > > > > > > > are:
> > > > > > > > > > >
> > > > > > > > > > >    1. automatic activation of cluster;
> > > > > > > > > > >
> > > > > > > > > > >    2. easy management of cluster topology changes
> > (planned
> > > > > nodes
> > > > > > > > > > >    maintenance, adding new nodes etc);
> > > > > > > > > > >
> > > > > > > > > > >    3. eliminating of rebalancing traffic on short-term
> > node
> > > > > > > failures.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Use cases to create BLT:
> > > > > > > > > > >
> > > > > > > > > > >    1. user starts up new cluster of desired number of
> > nodes
> > > > and
> > > > > > > > > activates
> > > > > > > > > > >    it using existing API. BLT is created with all nodes
> > > > > presented
> > > > > > > in
> > > > > > > > > the
> > > > > > > > > > >    cluster at the moment of activation, no API is
> needed;
> > > > > > > > > > >
> > > > > > > > > > >    2. user prepares BLT using web-console or visor CMD
> > > tools
> > > > > and
> > > > > > > sets
> > > > > > > > > it
> > > > > > > > > > to
> > > > > > > > > > >    the cluster. New API setter is needed:
> > > > > > > > > > >    Ignite.activation().setBaselin
> > > eTopology(BaselineTopology
> > > > > > blt);
> > > > > > > > > > >
> > > > > > > > > > >    3. user provides via static configuration a list of
> > > nodes
> > > > > that
> > > > > > > are
> > > > > > > > > > >    expected to be in the cluster.
> > > > > > > > > > >    User starts nodes one by one; when all preconfigured
> > > nodes
> > > > > are
> > > > > > > > > started
> > > > > > > > > > >    cluster is activated and BLT is created.
> > > > > > > > > > >    As list of nodes may be huge it is provided via
> > separate
> > > > > file
> > > > > > to
> > > > > > > > > avoid
> > > > > > > > > > >    flooding main configuration.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Igniters, does this description match with your
> > > understanding
> > > > > of
> > > > > > > > > > > functionality? If it does I'll create a set of tickets
> > and
> > > > > start
> > > > > > > > > working
> > > > > > > > > > on
> > > > > > > > > > > implementation.
> > > > > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > > > > > Sergey.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On Sat, Aug 19, 2017 at 5:41 PM, Dmitriy Setrakyan <
> > > > > > > > > > [hidden email]>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > I still do not see why anyone would explicitly call
> > > these 2
> > > > > > > > methods:
> > > > > > > > > > > >
> > > > > > > > > > > > *Ignite::activation::establishBaselineTopology();*
> > > > > > > > > > > > *Ignite::activation::establishBaselineTopology(
> > > > > > BaselineTopology
> > > > > > > > > > > bltTop);*
> > > > > > > > > > > >
> > > > > > > > > > > > For example, if a web console, or some other admin
> > > process,
> > > > > > want
> > > > > > > to
> > > > > > > > > > > > automatically set currently started nodes as the
> > baseline
> > > > > > > topology,
> > > > > > > > > > > > shouldn't they just call a setter for
> > > > minimalActivationNodes?
> > > > > > > > > > > >
> > > > > > > > > > > > D.
> > > > > > > > > > > >
> > > > > > > > > > > > On Fri, Aug 18, 2017 at 10:18 AM, Alexey Dmitriev <
> > > > > > > > > > > [hidden email]>
> > > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > API is proposed in the head of the thread by
> Sergey,
> > > as I
> > > > > > > > > understood:
> > > > > > > > > > > > > ______________________________
> > ________________________
> > > > > > > > > > > > >
> > > > > > > > > > > > > API for BaselineTopology manipulation may look like
> > > this:
> > > > > > > > > > > > >
> > > > > > > > > > > > > *Ignite::activation::establishBaselineTopology();*
> > > > > > > > > > > > > *Ignite::activation::establishBaselineTopology(
> > > > > > BaselineTopology
> > > > > > > > > > > > bltTop);*
> > > > > > > > > > > > >
> > > > > > > > > > > > > Both methods will establish BT and activate cluster
> > > once
> > > > it
> > > > > > is
> > > > > > > > > > > > established.
> > > > > > > > > > > > >
> > > > > > > > > > > > > The first one allows user to establish BT using
> > current
> > > > > > > topology.
> > > > > > > > > If
> > > > > > > > > > > any
> > > > > > > > > > > > > changes happen to the topology during establishing
> > > > process,
> > > > > > > user
> > > > > > > > > will
> > > > > > > > > > > be
> > > > > > > > > > > > > notified and allowed to proceed or abort the
> > procedure.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Second method allows to use some
> > > monitoring'n'management
> > > > > > tools
> > > > > > > > like
> > > > > > > > > > > > > WebConsole where user can prepare a list of nodes,
> > > using
> > > > > them
> > > > > > > > > create
> > > > > > > > > > a
> > > > > > > > > > > BT
> > > > > > > > > > > > > and send to the cluster a command to finally
> > establish
> > > > it.
> > > > > > > > > > > > >
> > > > > > > > > > > > > From high level BaselineTopology entity contains
> only
> > > > > > > collection
> > > > > > > > of
> > > > > > > > > > > > nodes:
> > > > > > > > > > > > >
> > > > > > > > > > > > > *BaselineTopology {*
> > > > > > > > > > > > > *  Collection<TopologyNode> nodes;*
> > > > > > > > > > > > > *}*
> > > > > > > > > > > > >
> > > > > > > > > > > > > *TopologyNode* here contains information about
> node -
> > > its
> > > > > > > > > consistent
> > > > > > > > > > id
> > > > > > > > > > > > and
> > > > > > > > > > > > > set of user attributes used to calculate affinity
> > > > function.
> > > > > > > > > > > > > ____________________________________________
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > --
> > > > > > > > > > > > > View this message in context:
> http://apache-ignite-
> > > > > > > > > > > > > developers.2346864.n4.nabble.
> > > > com/Cluster-auto-activation-
> > > > > > > > > > > > > design-proposal-tp20295p21066.html
> > > > > > > > > > > > > Sent from the Apache Ignite Developers mailing list
> > > > archive
> > > > > > at
> > > > > > > > > > > > Nabble.com.
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

[Alexey Goncharuk]

Guys,

We are getting closer to the baseline topology finalization. As a first
step, I would like to request a review of the baseline topology management
API. The changes are summarized in [1]. In my opinion, changes are quite
simple and concise. Also, as a side note, I suggest moving cluster
activation methods to the IgniteCluter facade as well because the facade
itself looks like a good place for management API. Looks like the original
decision to place it on Ignite was wrong.

Thanks!

[1] https://issues.apache.org/jira/browse/IGNITE-5850

2017-09-04 17:46 GMT+03:00 Dmitriy Setrakyan <[hidden email]>:

> On Mon, Sep 4, 2017 at 6:13 AM, Sergey Chugunov <[hidden email]
> >
> wrote:
>
> > Dmitriy,
> >
> > I like the idea of ClusterActivator interface.
> >
> > From user perspective it provides the same functionality as the setter
> but
> > in more clear and intuitive way.
> >
>
> BTW, I made a naming mistake in the original email. The setter name should
> be "setClusterActivator(...).
>
>
> >
> > Also it gives us a good place to put all the documentation about the
> > feature.
> >
>
> Agree. Another advantage is that users can now provide custom logic for the
> initial cluster activation.
>
>
> >
> > Any other opinions?
> >
>
> Alexey Goncharuk, given that you provided the initial implementation of the
> cluster activation, can you please take a look at this design and provide
> comments?
>
>
> >
> > On Fri, Sep 1, 2017 at 2:35 PM, Dmitriy Setrakyan <[hidden email]
> >
> > wrote:
> >
> > > How about this:
> > >
> > >
> > > > *interface ClusterActivator {*
> > > > *    boolean activate(Collection<IgniteNode> nodes);**}*
> > >
> > >
> > > Out of the box, we can provide this implementation of the activation
> > > filter:
> > >
> > >
> > > >
> > > > *ClusterInitialActiveSet implements ClusterActivator { *
> > > > *    InigeInitialActiveSet(String... addresses);**}*
> > >
> > >
> > > Then user configuration can look as follows:
> > >
> > > *IgniteConfiguration.setActivationFilter(new
> > > > ClusterInitialActiveSet("1.2.3.4", "4.3.2.1", etc));*
> > >
> > >
> > > Thoughts?
> > >
> > > D.
> > >
> > > On Fri, Sep 1, 2017 at 1:47 PM, Sergey Chugunov <
> > [hidden email]
> > > >
> > > wrote:
> > >
> > > > Dmitriy,
> > > >
> > > > The idea is interesting however I cannot come up with a clear use
> case
> > > > which can be widely adopted.
> > > > I would give users a simple API at first to cover 80% of their needs
> > and
> > > > then collect some feedback and start thinking about adding new
> > > > functionality.
> > > >
> > > > Makes sense?
> > > >
> > > > Sergey.
> > > >
> > > > On Thu, Aug 31, 2017 at 3:55 AM, Dmitriy Setrakyan <
> > > [hidden email]>
> > > > wrote:
> > > >
> > > > > Hm... Can we also ask user to optionally provide a predicate which
> > will
> > > > > receive a collection of nodes started so far and return true if the
> > > > > activation should happen? Will it be useful?
> > > > >
> > > > > On Wed, Aug 30, 2017 at 6:28 PM, Sergey Chugunov <
> > > > > [hidden email]>
> > > > > wrote:
> > > > >
> > > > > > Nick,
> > > > > >
> > > > > > As I summed up in this thread above, calling setter for initial
> > > > > activation
> > > > > > nodes is not the only option:
> > > > > >
> > > > > >    1. user starts up new cluster of desired number of nodes and
> > > > activates
> > > > > >    it using existing API.
> > > > > >    BLT is created with all nodes presented in the cluster at the
> > > moment
> > > > > of
> > > > > >    activation, no API is needed;
> > > > > >
> > > > > >    2. user prepares BLT using web-console or visor CMD tools and
> > sets
> > > > it
> > > > > to
> > > > > >    the cluster. New API setter is needed:
> > > > > >    Ignite.activation().setInitialActivationNodes(Collection<
> > > > ClusterNode>
> > > > > >    nodes);
> > > > > >
> > > > > >    3. user provides via static configuration a list of nodes that
> > are
> > > > > >    expected to be in the cluster.
> > > > > >    User starts nodes one by one; when all preconfigured nodes are
> > > > started
> > > > > >    cluster is activated and BLT is created.
> > > > > >    As list of nodes may be huge it is provided via separate file
> to
> > > > avoid
> > > > > >    flooding main configuration.
> > > > > >
> > > > > > So the option you proposed is already in the list.
> > > > > >
> > > > > > As for idea of activating cluster based only on number of nodes
> may
> > > be
> > > > > > risky.
> > > > > > E.g. if user starts up with data stored on disk and unexpected
> node
> > > > joins
> > > > > > the topology.
> > > > > > Cluster will get activated with N-1 nodes where all the data is
> > > > presented
> > > > > > and one node completely empty. Data loss may happen in such
> > scenario.
> > > > > >
> > > > > > Thanks,
> > > > > > Sergey.
> > > > > >
> > > > > >
> > > > > > On Wed, Aug 30, 2017 at 4:23 PM, Nick Pordash <
> > [hidden email]
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > How is a user expected to produce a collection of ClusterNode
> > prior
> > > > to
> > > > > > all
> > > > > > > of the expected nodes joining? Users don't create instances of
> > > this,
> > > > so
> > > > > > as
> > > > > > > far as I can tell it would have to be retrieved from
> > IgniteCluster.
> > > > > > > However, would doing that and calling the proposed method be
> > really
> > > > any
> > > > > > > different than calling Ignite.activate and using the current
> set
> > of
> > > > > > server
> > > > > > > nodes as that collection?
> > > > > > >
> > > > > > > From a user's perspective is it really necessary that specific
> > > nodes
> > > > > need
> > > > > > > to be identified vs saying that they expect N server nodes to
> be
> > in
> > > > the
> > > > > > > cluster for auto activation?
> > > > > > >
> > > > > > > -Nick
> > > > > > >
> > > > > > > On Wed, Aug 30, 2017, 1:23 AM Sergey Chugunov <
> > > > > [hidden email]
> > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Dmitriy,
> > > > > > > >
> > > > > > > > Now I see your point and I think you're right.
> > > > > > > >
> > > > > > > > We can give end-user a simple setter like
> > > > > > > > Ignite::activation::setInitialActivationNodes(
> > > > > Collection<ClusterNode>
> > > > > > > > nodes) to provide collection of nodes that grid must reach to
> > > > > activate
> > > > > > > > automatically.
> > > > > > > >
> > > > > > > > And then using the collection we'll create BaselineTopology
> > > > > abstraction
> > > > > > > > internally.
> > > > > > > >
> > > > > > > > As a result user won't be exposed to our internal
> abstractions
> > > and
> > > > > will
> > > > > > > > work with intuitive concept of collection of nodes.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Sergey.
> > > > > > > >
> > > > > > > >
> > > > > > > > On Mon, Aug 28, 2017 at 4:39 PM, Dmitriy Setrakyan <
> > > > > > > [hidden email]>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Sergey, the interface you are suggesting is internal, not
> > > > external.
> > > > > > Why
> > > > > > > > > should user ever see it or care about it?
> > > > > > > > >
> > > > > > > > > D.
> > > > > > > > >
> > > > > > > > > On Mon, Aug 28, 2017 at 3:18 PM, Sergey Chugunov <
> > > > > > > > > [hidden email]>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Dmitriy,
> > > > > > > > > >
> > > > > > > > > > It was my misunderstanding, I believe that setter is not
> > > enough
> > > > > and
> > > > > > > we
> > > > > > > > > need
> > > > > > > > > > a full-fledged entity.
> > > > > > > > > >
> > > > > > > > > > We should also be able to check if BLTs are compatible.
> > > > Interface
> > > > > > > looks
> > > > > > > > > > like this and use case for this functionality is
> described
> > > > below.
> > > > > > > > > >
> > > > > > > > > > interface BaselineTopology {
> > > > > > > > > >    Collection<ClusterNode> nodes();
> > > > > > > > > >    boolean isCompatibleWith(BaselineTopology blt);
> > > > > > > > > > }
> > > > > > > > > >
> > > > > > > > > > Let's consider the following scenario:
> > > > > > > > > >
> > > > > > > > > >    1. We have a grid with N nodes: it is up, active and
> has
> > > > data
> > > > > in
> > > > > > > it.
> > > > > > > > > ->
> > > > > > > > > >    BLT #1 created.
> > > > > > > > > >    2. We shutdown the grid. Then divide it into two
> parts:
> > > > > > Part1_grid
> > > > > > > > and
> > > > > > > > > >    Part2_grid.
> > > > > > > > > >    3. We start and activate Part1_grid . Topology has
> > changed
> > > > ->
> > > > > > > BLT#2
> > > > > > > > > >    created.
> > > > > > > > > >    After that we shutdown that Part1_grid.
> > > > > > > > > >    4. We start and activate Part2_grid. Topology also has
> > > > changed
> > > > > > ->
> > > > > > > > > BLT#3
> > > > > > > > > >    created.
> > > > > > > > > >    5. Then we start Part1_grid and it's nodes try to join
> > > > > > Part2_grid.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > If join is successful we have an undefined state of the
> > > > resulting
> > > > > > > grid:
> > > > > > > > > > values for the same key may (and will) differ between
> grid
> > > > parts.
> > > > > > > > > >
> > > > > > > > > > So to prevent this we should keep nodes with BLT#2 from
> > > joining
> > > > > the
> > > > > > > > grid
> > > > > > > > > > with BLT#3. And we should fail nodes with an error
> message.
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > Sergey.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Wed, Aug 23, 2017 at 5:47 AM, Dmitriy Setrakyan <
> > > > > > > > > [hidden email]>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Sergey, I am still confused. What is the
> BaselineTopology
> > > > > > interface
> > > > > > > > in
> > > > > > > > > > your
> > > > > > > > > > > example? I thought that you agreed with me that we
> simply
> > > > need
> > > > > a
> > > > > > > > setter
> > > > > > > > > > for
> > > > > > > > > > > activation nodes, no?
> > > > > > > > > > >
> > > > > > > > > > > D.
> > > > > > > > > > >
> > > > > > > > > > > On Tue, Aug 22, 2017 at 4:47 AM, Sergey Chugunov <
> > > > > > > > > > > [hidden email]>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Dmitriy,
> > > > > > > > > > > >
> > > > > > > > > > > > As I understand you use the term
> > "minimalActivationNodes"
> > > > as
> > > > > a
> > > > > > > > > synonym
> > > > > > > > > > > for
> > > > > > > > > > > > BaselineTopology concept.
> > > > > > > > > > > > In that case I agree with you that we can replace
> both
> > > > > > > "establish*"
> > > > > > > > > > > methods
> > > > > > > > > > > > with a simple setter method (see below in summary).
> > > > > > > > > > > >
> > > > > > > > > > > > Summing up the whole discussion I see the
> functionality
> > > as
> > > > > > > > following:
> > > > > > > > > > > >
> > > > > > > > > > > > New concept BaselineTopology is introduced. The main
> > > > features
> > > > > > it
> > > > > > > > > > enables
> > > > > > > > > > > > are:
> > > > > > > > > > > >
> > > > > > > > > > > >    1. automatic activation of cluster;
> > > > > > > > > > > >
> > > > > > > > > > > >    2. easy management of cluster topology changes
> > > (planned
> > > > > > nodes
> > > > > > > > > > > >    maintenance, adding new nodes etc);
> > > > > > > > > > > >
> > > > > > > > > > > >    3. eliminating of rebalancing traffic on
> short-term
> > > node
> > > > > > > > failures.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > Use cases to create BLT:
> > > > > > > > > > > >
> > > > > > > > > > > >    1. user starts up new cluster of desired number of
> > > nodes
> > > > > and
> > > > > > > > > > activates
> > > > > > > > > > > >    it using existing API. BLT is created with all
> nodes
> > > > > > presented
> > > > > > > > in
> > > > > > > > > > the
> > > > > > > > > > > >    cluster at the moment of activation, no API is
> > needed;
> > > > > > > > > > > >
> > > > > > > > > > > >    2. user prepares BLT using web-console or visor
> CMD
> > > > tools
> > > > > > and
> > > > > > > > sets
> > > > > > > > > > it
> > > > > > > > > > > to
> > > > > > > > > > > >    the cluster. New API setter is needed:
> > > > > > > > > > > >    Ignite.activation().setBaselin
> > > > eTopology(BaselineTopology
> > > > > > > blt);
> > > > > > > > > > > >
> > > > > > > > > > > >    3. user provides via static configuration a list
> of
> > > > nodes
> > > > > > that
> > > > > > > > are
> > > > > > > > > > > >    expected to be in the cluster.
> > > > > > > > > > > >    User starts nodes one by one; when all
> preconfigured
> > > > nodes
> > > > > > are
> > > > > > > > > > started
> > > > > > > > > > > >    cluster is activated and BLT is created.
> > > > > > > > > > > >    As list of nodes may be huge it is provided via
> > > separate
> > > > > > file
> > > > > > > to
> > > > > > > > > > avoid
> > > > > > > > > > > >    flooding main configuration.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > Igniters, does this description match with your
> > > > understanding
> > > > > > of
> > > > > > > > > > > > functionality? If it does I'll create a set of
> tickets
> > > and
> > > > > > start
> > > > > > > > > > working
> > > > > > > > > > > on
> > > > > > > > > > > > implementation.
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks,
> > > > > > > > > > > > Sergey.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > On Sat, Aug 19, 2017 at 5:41 PM, Dmitriy Setrakyan <
> > > > > > > > > > > [hidden email]>
> > > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > I still do not see why anyone would explicitly call
> > > > these 2
> > > > > > > > > methods:
> > > > > > > > > > > > >
> > > > > > > > > > > > > *Ignite::activation::establishBaselineTopology();*
> > > > > > > > > > > > > *Ignite::activation::establishBaselineTopology(
> > > > > > > BaselineTopology
> > > > > > > > > > > > bltTop);*
> > > > > > > > > > > > >
> > > > > > > > > > > > > For example, if a web console, or some other admin
> > > > process,
> > > > > > > want
> > > > > > > > to
> > > > > > > > > > > > > automatically set currently started nodes as the
> > > baseline
> > > > > > > > topology,
> > > > > > > > > > > > > shouldn't they just call a setter for
> > > > > minimalActivationNodes?
> > > > > > > > > > > > >
> > > > > > > > > > > > > D.
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Fri, Aug 18, 2017 at 10:18 AM, Alexey Dmitriev <
> > > > > > > > > > > > [hidden email]>
> > > > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > API is proposed in the head of the thread by
> > Sergey,
> > > > as I
> > > > > > > > > > understood:
> > > > > > > > > > > > > > ______________________________
> > > ________________________
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > API for BaselineTopology manipulation may look
> like
> > > > this:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > *Ignite::activation::
> establishBaselineTopology();*
> > > > > > > > > > > > > > *Ignite::activation::establishBaselineTopology(
> > > > > > > BaselineTopology
> > > > > > > > > > > > > bltTop);*
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Both methods will establish BT and activate
> cluster
> > > > once
> > > > > it
> > > > > > > is
> > > > > > > > > > > > > established.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > The first one allows user to establish BT using
> > > current
> > > > > > > > topology.
> > > > > > > > > > If
> > > > > > > > > > > > any
> > > > > > > > > > > > > > changes happen to the topology during
> establishing
> > > > > process,
> > > > > > > > user
> > > > > > > > > > will
> > > > > > > > > > > > be
> > > > > > > > > > > > > > notified and allowed to proceed or abort the
> > > procedure.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Second method allows to use some
> > > > monitoring'n'management
> > > > > > > tools
> > > > > > > > > like
> > > > > > > > > > > > > > WebConsole where user can prepare a list of
> nodes,
> > > > using
> > > > > > them
> > > > > > > > > > create
> > > > > > > > > > > a
> > > > > > > > > > > > BT
> > > > > > > > > > > > > > and send to the cluster a command to finally
> > > establish
> > > > > it.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > From high level BaselineTopology entity contains
> > only
> > > > > > > > collection
> > > > > > > > > of
> > > > > > > > > > > > > nodes:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > *BaselineTopology {*
> > > > > > > > > > > > > > *  Collection<TopologyNode> nodes;*
> > > > > > > > > > > > > > *}*
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > *TopologyNode* here contains information about
> > node -
> > > > its
> > > > > > > > > > consistent
> > > > > > > > > > > id
> > > > > > > > > > > > > and
> > > > > > > > > > > > > > set of user attributes used to calculate affinity
> > > > > function.
> > > > > > > > > > > > > > ____________________________________________
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > --
> > > > > > > > > > > > > > View this message in context:
> > http://apache-ignite-
> > > > > > > > > > > > > > developers.2346864.n4.nabble.
> > > > > com/Cluster-auto-activation-
> > > > > > > > > > > > > > design-proposal-tp20295p21066.html
> > > > > > > > > > > > > > Sent from the Apache Ignite Developers mailing
> list
> > > > > archive
> > > > > > > at
> > > > > > > > > > > > > Nabble.com.
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>