Consistent ID specification from previous random UUID

[Dmitry Pavlov]

Hi Igniters,

I would like to fix the issue
https://issues.apache.org/jira/browse/IGNITE-11432 about specifying some
previous randomly generated UUID as a new consistent ID. Folder generation
algorithm here (
https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood)
allows two options
-node00+random UUID
- consistendId

I would like to add to Ignite configuration new property nodeIndex in
addition to consistent Id. New Property will be named as storageNodeIndex,
int, zero-based.
This will add the third option of subfolders processing:
node{storageNodeIndex}+consistentID

Please share your vision. I'm going to apply this change by lazy consensus
in 3 days.

Sincerely,
Dmitriy Pavlov

[Ivan Pavlukhin]

Dmitiy,

Could please clarify one thing:
1. Will it be enough to use only storageNodeIndex in order to reuse
the same persistence folders when consistentId is auto-generated? E.g.
I have a configuration with storageNodeIndex=1 and without explicitly
specified consistentId, will the node after restart use the same
persistence folder as before restart?

Also a side note:
> Please share your vision. I'm going to apply this change by lazy consensus
in 3 days.
What do you mean by "apply"? I have not seen any PR yet.

ср, 27 февр. 2019 г. в 21:12, Dmitriy Pavlov <[hidden email]>:

>
> Hi Igniters,
>
> I would like to fix the issue
> https://issues.apache.org/jira/browse/IGNITE-11432 about specifying some
> previous randomly generated UUID as a new consistent ID. Folder generation
> algorithm here (
> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood)
> allows two options
> -node00+random UUID
> - consistendId
>
> I would like to add to Ignite configuration new property nodeIndex in
> addition to consistent Id. New Property will be named as storageNodeIndex,
> int, zero-based.
> This will add the third option of subfolders processing:
> node{storageNodeIndex}+consistentID
>
> Please share your vision. I'm going to apply this change by lazy consensus
> in 3 days.
>
> Sincerely,
> Dmitriy Pavlov

--
Best regards,
Ivan Pavlukhin

[Stanislav Lukyanov]

Dmitry,

I seems to me that the algorithm of using different schemes
for persistence folder naming is flawe.
Why don’t we just name the folder same as consistent ID all the time?

How about the following algorithm of db folder and consistenID generation
1.. If the consistent ID is explicitly set to ABCD then both the ID and the folder name are ABCD
2. If there is no consistentID
2a. If there is an existing, not locked db folder
2aa. If the folder is named nodeXX-UUID then that folder is used and the consistentID is UUID
2ab. If the folder is named ABCD then both the ID and the folder name are ABCD
2b. If there is no existing db folder then both the ID and the folder name are UUID

This behavior makes sure that
- Going forward, folder name and consistentID are always the same
- Existing storages that are used without a consistentID can continue to be used
- New storages created without an explicit consistentID can use an explicit consistentID (so, the problem described in IGNITE-11432 will no affect new users)

Existing users with existing storages named nodeXX-UUID will continue to be affected by IGNITE-11432,
i.e. they will not be able to easily specify their generated consistentID in the configuration.
I don’t think that’s critical though – the problem can be resolved by manually renaming directories.
Let’s not complicate our API for the sake of resolving an existing issue with a workaround – better to simplify the API and make sure that such problem will not happen again.

Also, I assume that the reason behind using a prefix “nodeXX” is to distinguish storage directories from some random trash being in the same folder,
but that can be achieved in a different way. E.g. to scan all folders and check their structure, look for a marker file, etc.

Thanks,
Stan

From: Павлухин Иван
Sent: 28 февраля 2019 г. 8:03
To: [hidden email]
Subject: Re: Consistent ID specification from previous random UUID

Dmitiy,

Could please clarify one thing:
1. Will it be enough to use only storageNodeIndex in order to reuse
the same persistence folders when consistentId is auto-generated? E.g.
I have a configuration with storageNodeIndex=1 and without explicitly
specified consistentId, will the node after restart use the same
persistence folder as before restart?

Also a side note:
> Please share your vision. I'm going to apply this change by lazy consensus
in 3 days.
What do you mean by "apply"? I have not seen any PR yet.

ср, 27 февр. 2019 г. в 21:12, Dmitriy Pavlov <[hidden email]>:

>
> Hi Igniters,
>
> I would like to fix the issue
> https://issues.apache.org/jira/browse/IGNITE-11432 about specifying some
> previous randomly generated UUID as a new consistent ID. Folder generation
> algorithm here (
> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood)
> allows two options
> -node00+random UUID
> - consistendId
>
> I would like to add to Ignite configuration new property nodeIndex in
> addition to consistent Id. New Property will be named as storageNodeIndex,
> int, zero-based.
> This will add the third option of subfolders processing:
> node{storageNodeIndex}+consistentID
>
> Please share your vision. I'm going to apply this change by lazy consensus
> in 3 days.
>
> Sincerely,
> Dmitriy Pavlov

--
Best regards,
Ivan Pavlukhin

[Dmitry Pavlov]

Hi Ivan,

Yes, you catch me, I'm a little bit cheating with lazy consensus on code
modification without providing a PR because I was expecting that nobody
comes to discussion. I will prepare PR shortly. And since we anyway have a
discussion, I will not apply anything by lazy approval.

- storageNodeIndex without consistent ID will not work.
 cfg.getDataStorageConfiguration().setNodeIdx() will be required only for
case we have consistent ID.

Hi Stanislav,

We can't use only consistent ID because

1) we need an order to scan and lock random-UUID based folders.  Node index
provides the order of scan. I can find the corresponding discussion, but I
guess it is not needed.
2) we need to separate backward compatible folders from new random-UUID
based folders. Using UUID as folder will not allow us to scan only new name
format folders.

I guess specifying node index is a quite rare case and good JavaDoc will
always help.

DataStorageConfiguration().setNodeIdx()  JavaDoc may include following
notes:
Node index used for persistent store folders in case several nodes reuse
one persistent store root folder.

Sincerely,
Dmitriy Pavlov

чт, 28 февр. 2019 г. в 08:03, Павлухин Иван <[hidden email]>:

> Dmitiy,
>
> Could please clarify one thing:
> 1. Will it be enough to use only storageNodeIndex in order to reuse
> the same persistence folders when consistentId is auto-generated? E.g.
> I have a configuration with storageNodeIndex=1 and without explicitly
> specified consistentId, will the node after restart use the same
> persistence folder as before restart?
>
> Also a side note:
> > Please share your vision. I'm going to apply this change by lazy
> consensus
> in 3 days.
> What do you mean by "apply"? I have not seen any PR yet.
>
> ср, 27 февр. 2019 г. в 21:12, Dmitriy Pavlov <[hidden email]>:
> >
> > Hi Igniters,
> >
> > I would like to fix the issue
> > https://issues.apache.org/jira/browse/IGNITE-11432 about specifying some
> > previous randomly generated UUID as a new consistent ID. Folder
> generation
> > algorithm here (
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood
> )
> > allows two options
> > -node00+random UUID
> > - consistendId
> >
> > I would like to add to Ignite configuration new property nodeIndex in
> > addition to consistent Id. New Property will be named as
> storageNodeIndex,
> > int, zero-based.
> > This will add the third option of subfolders processing:
> > node{storageNodeIndex}+consistentID
> >
> > Please share your vision. I'm going to apply this change by lazy
> consensus
> > in 3 days.
> >
> > Sincerely,
> > Dmitriy Pavlov
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>

[Ivan Pavlukhin]

Dmitriy,

You wrote
> we need an order to scan and lock random-UUID based folders.

It would be great if you provide a discussion about that order to
complete the picture. Currently I cannot understand why the order is
important.

Also, couple of raw thoughts:
1. Can we extend a directory lookup procedure when consistent id is
specified to check nodeXX-consistentId directories as well?
2. Can we introduce a property like nodeStorageName to specify exact
name of a directory to use? It looks like a straightforward and
universal workaround. Or is node index better in some sense? Why?

Please, share your thoughts.

чт, 28 февр. 2019 г. в 17:43, Dmitriy Pavlov <[hidden email]>:

>
> Hi Ivan,
>
> Yes, you catch me, I'm a little bit cheating with lazy consensus on code
> modification without providing a PR because I was expecting that nobody
> comes to discussion. I will prepare PR shortly. And since we anyway have a
> discussion, I will not apply anything by lazy approval.
>
> - storageNodeIndex without consistent ID will not work.
>  cfg.getDataStorageConfiguration().setNodeIdx() will be required only for
> case we have consistent ID.
>
> Hi Stanislav,
>
> We can't use only consistent ID because
>
> 1) we need an order to scan and lock random-UUID based folders.  Node index
> provides the order of scan. I can find the corresponding discussion, but I
> guess it is not needed.
> 2) we need to separate backward compatible folders from new random-UUID
> based folders. Using UUID as folder will not allow us to scan only new name
> format folders.
>
> I guess specifying node index is a quite rare case and good JavaDoc will
> always help.
>
> DataStorageConfiguration().setNodeIdx()  JavaDoc may include following
> notes:
> Node index used for persistent store folders in case several nodes reuse
> one persistent store root folder.
>
> Sincerely,
> Dmitriy Pavlov
>
> чт, 28 февр. 2019 г. в 08:03, Павлухин Иван <[hidden email]>:
>
> > Dmitiy,
> >
> > Could please clarify one thing:
> > 1. Will it be enough to use only storageNodeIndex in order to reuse
> > the same persistence folders when consistentId is auto-generated? E.g.
> > I have a configuration with storageNodeIndex=1 and without explicitly
> > specified consistentId, will the node after restart use the same
> > persistence folder as before restart?
> >
> > Also a side note:
> > > Please share your vision. I'm going to apply this change by lazy
> > consensus
> > in 3 days.
> > What do you mean by "apply"? I have not seen any PR yet.
> >
> > ср, 27 февр. 2019 г. в 21:12, Dmitriy Pavlov <[hidden email]>:
> > >
> > > Hi Igniters,
> > >
> > > I would like to fix the issue
> > > https://issues.apache.org/jira/browse/IGNITE-11432 about specifying some
> > > previous randomly generated UUID as a new consistent ID. Folder
> > generation
> > > algorithm here (
> > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood
> > )
> > > allows two options
> > > -node00+random UUID
> > > - consistendId
> > >
> > > I would like to add to Ignite configuration new property nodeIndex in
> > > addition to consistent Id. New Property will be named as
> > storageNodeIndex,
> > > int, zero-based.
> > > This will add the third option of subfolders processing:
> > > node{storageNodeIndex}+consistentID
> > >
> > > Please share your vision. I'm going to apply this change by lazy
> > consensus
> > > in 3 days.
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> >
> >
> >
> > --
> > Best regards,
> > Ivan Pavlukhin
> >

--
Best regards,
Ivan Pavlukhin

[Dmitry Pavlov]

Hi Igniters,

A full description can be found at wiki page
https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood#IgnitePersistentStore-underthehood-SubfoldersGeneration

I like the idea of introducing a property like nodeStorageName to specify
exactly name of a directory to use. For example, it could have higher
priority then IgniteConfiguration.getConsistentId(). It will impact the
mentioned algorithm, but it seems to be easier to understand by users.

If nobody else minds, I will try to implement this idea.

Sincerely,
Dmitriy Pavlov


сб, 2 мар. 2019 г. в 08:33, Павлухин Иван <[hidden email]>:

> Dmitriy,
>
> You wrote
> > we need an order to scan and lock random-UUID based folders.
>
> It would be great if you provide a discussion about that order to
> complete the picture. Currently I cannot understand why the order is
> important.
>
> Also, couple of raw thoughts:
> 1. Can we extend a directory lookup procedure when consistent id is
> specified to check nodeXX-consistentId directories as well?
> 2. Can we introduce a property like nodeStorageName to specify exact
> name of a directory to use? It looks like a straightforward and
> universal workaround. Or is node index better in some sense? Why?
>
> Please, share your thoughts.
>
> чт, 28 февр. 2019 г. в 17:43, Dmitriy Pavlov <[hidden email]>:
> >
> > Hi Ivan,
> >
> > Yes, you catch me, I'm a little bit cheating with lazy consensus on code
> > modification without providing a PR because I was expecting that nobody
> > comes to discussion. I will prepare PR shortly. And since we anyway have
> a
> > discussion, I will not apply anything by lazy approval.
> >
> > - storageNodeIndex without consistent ID will not work.
> >  cfg.getDataStorageConfiguration().setNodeIdx() will be required only for
> > case we have consistent ID.
> >
> > Hi Stanislav,
> >
> > We can't use only consistent ID because
> >
> > 1) we need an order to scan and lock random-UUID based folders.  Node
> index
> > provides the order of scan. I can find the corresponding discussion, but
> I
> > guess it is not needed.
> > 2) we need to separate backward compatible folders from new random-UUID
> > based folders. Using UUID as folder will not allow us to scan only new
> name
> > format folders.
> >
> > I guess specifying node index is a quite rare case and good JavaDoc will
> > always help.
> >
> > DataStorageConfiguration().setNodeIdx()  JavaDoc may include following
> > notes:
> > Node index used for persistent store folders in case several nodes reuse
> > one persistent store root folder.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > чт, 28 февр. 2019 г. в 08:03, Павлухин Иван <[hidden email]>:
> >
> > > Dmitiy,
> > >
> > > Could please clarify one thing:
> > > 1. Will it be enough to use only storageNodeIndex in order to reuse
> > > the same persistence folders when consistentId is auto-generated? E.g.
> > > I have a configuration with storageNodeIndex=1 and without explicitly
> > > specified consistentId, will the node after restart use the same
> > > persistence folder as before restart?
> > >
> > > Also a side note:
> > > > Please share your vision. I'm going to apply this change by lazy
> > > consensus
> > > in 3 days.
> > > What do you mean by "apply"? I have not seen any PR yet.
> > >
> > > ср, 27 февр. 2019 г. в 21:12, Dmitriy Pavlov <[hidden email]>:
> > > >
> > > > Hi Igniters,
> > > >
> > > > I would like to fix the issue
> > > > https://issues.apache.org/jira/browse/IGNITE-11432 about specifying
> some
> > > > previous randomly generated UUID as a new consistent ID. Folder
> > > generation
> > > > algorithm here (
> > > >
> > >
> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood
> > > )
> > > > allows two options
> > > > -node00+random UUID
> > > > - consistendId
> > > >
> > > > I would like to add to Ignite configuration new property nodeIndex in
> > > > addition to consistent Id. New Property will be named as
> > > storageNodeIndex,
> > > > int, zero-based.
> > > > This will add the third option of subfolders processing:
> > > > node{storageNodeIndex}+consistentID
> > > >
> > > > Please share your vision. I'm going to apply this change by lazy
> > > consensus
> > > > in 3 days.
> > > >
> > > > Sincerely,
> > > > Dmitriy Pavlov
> > >
> > >
> > >
> > > --
> > > Best regards,
> > > Ivan Pavlukhin
> > >
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>

[Alexey Goncharuk]

Igniters,

I came across the same issue during development and found no sane
workaround for this issue. I believe the solution should be as simple as
possible because we are already adding a warning to let users know that it
is good to specify a consistent ID in production deployments.

As for the current solution, I do not like adding a new configuration
property like 'storageFolder' because it's another way to shoot yourself in
the leg (e.g. different nodes have different consistent IDs but configured
to have the same storage folder).
Why can't we:
 - either check if consistent ID is an instance of UUID and then take the
appropriate folder. This approach is straightforward, but may affect
current users
 - check if consistent ID is set to a string 'nodeXX-UUID'. In this case
the consistent ID is set to UUID, and the storage folder is chosen
according to the proper rules. This change has a minimal chance to affect
current users because it's unlikely that somebody is using auto-generated
folder naming scheme as consistent ID.

Thoughts?

вт, 12 мар. 2019 г. в 16:51, Dmitriy Pavlov <[hidden email]>:

> Hi Igniters,
>
> A full description can be found at wiki page
>
> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood#IgnitePersistentStore-underthehood-SubfoldersGeneration
>
> I like the idea of introducing a property like nodeStorageName to specify
> exactly name of a directory to use. For example, it could have higher
> priority then IgniteConfiguration.getConsistentId(). It will impact the
> mentioned algorithm, but it seems to be easier to understand by users.
>
> If nobody else minds, I will try to implement this idea.
>
> Sincerely,
> Dmitriy Pavlov
>
>
> сб, 2 мар. 2019 г. в 08:33, Павлухин Иван <[hidden email]>:
>
> > Dmitriy,
> >
> > You wrote
> > > we need an order to scan and lock random-UUID based folders.
> >
> > It would be great if you provide a discussion about that order to
> > complete the picture. Currently I cannot understand why the order is
> > important.
> >
> > Also, couple of raw thoughts:
> > 1. Can we extend a directory lookup procedure when consistent id is
> > specified to check nodeXX-consistentId directories as well?
> > 2. Can we introduce a property like nodeStorageName to specify exact
> > name of a directory to use? It looks like a straightforward and
> > universal workaround. Or is node index better in some sense? Why?
> >
> > Please, share your thoughts.
> >
> > чт, 28 февр. 2019 г. в 17:43, Dmitriy Pavlov <[hidden email]>:
> > >
> > > Hi Ivan,
> > >
> > > Yes, you catch me, I'm a little bit cheating with lazy consensus on
> code
> > > modification without providing a PR because I was expecting that nobody
> > > comes to discussion. I will prepare PR shortly. And since we anyway
> have
> > a
> > > discussion, I will not apply anything by lazy approval.
> > >
> > > - storageNodeIndex without consistent ID will not work.
> > >  cfg.getDataStorageConfiguration().setNodeIdx() will be required only
> for
> > > case we have consistent ID.
> > >
> > > Hi Stanislav,
> > >
> > > We can't use only consistent ID because
> > >
> > > 1) we need an order to scan and lock random-UUID based folders.  Node
> > index
> > > provides the order of scan. I can find the corresponding discussion,
> but
> > I
> > > guess it is not needed.
> > > 2) we need to separate backward compatible folders from new random-UUID
> > > based folders. Using UUID as folder will not allow us to scan only new
> > name
> > > format folders.
> > >
> > > I guess specifying node index is a quite rare case and good JavaDoc
> will
> > > always help.
> > >
> > > DataStorageConfiguration().setNodeIdx()  JavaDoc may include following
> > > notes:
> > > Node index used for persistent store folders in case several nodes
> reuse
> > > one persistent store root folder.
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > чт, 28 февр. 2019 г. в 08:03, Павлухин Иван <[hidden email]>:
> > >
> > > > Dmitiy,
> > > >
> > > > Could please clarify one thing:
> > > > 1. Will it be enough to use only storageNodeIndex in order to reuse
> > > > the same persistence folders when consistentId is auto-generated?
> E.g.
> > > > I have a configuration with storageNodeIndex=1 and without explicitly
> > > > specified consistentId, will the node after restart use the same
> > > > persistence folder as before restart?
> > > >
> > > > Also a side note:
> > > > > Please share your vision. I'm going to apply this change by lazy
> > > > consensus
> > > > in 3 days.
> > > > What do you mean by "apply"? I have not seen any PR yet.
> > > >
> > > > ср, 27 февр. 2019 г. в 21:12, Dmitriy Pavlov <[hidden email]>:
> > > > >
> > > > > Hi Igniters,
> > > > >
> > > > > I would like to fix the issue
> > > > > https://issues.apache.org/jira/browse/IGNITE-11432 about
> specifying
> > some
> > > > > previous randomly generated UUID as a new consistent ID. Folder
> > > > generation
> > > > > algorithm here (
> > > > >
> > > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood
> > > > )
> > > > > allows two options
> > > > > -node00+random UUID
> > > > > - consistendId
> > > > >
> > > > > I would like to add to Ignite configuration new property nodeIndex
> in
> > > > > addition to consistent Id. New Property will be named as
> > > > storageNodeIndex,
> > > > > int, zero-based.
> > > > > This will add the third option of subfolders processing:
> > > > > node{storageNodeIndex}+consistentID
> > > > >
> > > > > Please share your vision. I'm going to apply this change by lazy
> > > > consensus
> > > > > in 3 days.
> > > > >
> > > > > Sincerely,
> > > > > Dmitriy Pavlov
> > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Ivan Pavlukhin
> > > >
> >
> >
> >
> > --
> > Best regards,
> > Ivan Pavlukhin
> >
>

[Ivan Pavlukhin]

Alex,

Why do we need
>  - check if consistent ID is set to a string 'nodeXX-UUID'. In this case
> the consistent ID is set to UUID, and the storage folder is chosen
> according to the proper rules. This change has a minimal chance to affect
> current users because it's unlikely that somebody is using auto-generated
> folder naming scheme as consistent ID.
?
It looks hacky as well. The thing I do not like here is that a
consistentId specified in configuration is not a consistentId used by
a node sometimes.

Can we go just with
> - either check if consistent ID is an instance of UUID and then take the
> appropriate folder. This approach is straightforward, but may affect
> current users
?

And as a last chance a user will have a possibility to rename a directory.


вт, 12 мар. 2019 г. в 19:33, Alexey Goncharuk <[hidden email]>:

>
> Igniters,
>
> I came across the same issue during development and found no sane
> workaround for this issue. I believe the solution should be as simple as
> possible because we are already adding a warning to let users know that it
> is good to specify a consistent ID in production deployments.
>
> As for the current solution, I do not like adding a new configuration
> property like 'storageFolder' because it's another way to shoot yourself in
> the leg (e.g. different nodes have different consistent IDs but configured
> to have the same storage folder).
> Why can't we:
>  - either check if consistent ID is an instance of UUID and then take the
> appropriate folder. This approach is straightforward, but may affect
> current users
>  - check if consistent ID is set to a string 'nodeXX-UUID'. In this case
> the consistent ID is set to UUID, and the storage folder is chosen
> according to the proper rules. This change has a minimal chance to affect
> current users because it's unlikely that somebody is using auto-generated
> folder naming scheme as consistent ID.
>
> Thoughts?
>
> вт, 12 мар. 2019 г. в 16:51, Dmitriy Pavlov <[hidden email]>:
>
> > Hi Igniters,
> >
> > A full description can be found at wiki page
> >
> > https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood#IgnitePersistentStore-underthehood-SubfoldersGeneration
> >
> > I like the idea of introducing a property like nodeStorageName to specify
> > exactly name of a directory to use. For example, it could have higher
> > priority then IgniteConfiguration.getConsistentId(). It will impact the
> > mentioned algorithm, but it seems to be easier to understand by users.
> >
> > If nobody else minds, I will try to implement this idea.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> >
> > сб, 2 мар. 2019 г. в 08:33, Павлухин Иван <[hidden email]>:
> >
> > > Dmitriy,
> > >
> > > You wrote
> > > > we need an order to scan and lock random-UUID based folders.
> > >
> > > It would be great if you provide a discussion about that order to
> > > complete the picture. Currently I cannot understand why the order is
> > > important.
> > >
> > > Also, couple of raw thoughts:
> > > 1. Can we extend a directory lookup procedure when consistent id is
> > > specified to check nodeXX-consistentId directories as well?
> > > 2. Can we introduce a property like nodeStorageName to specify exact
> > > name of a directory to use? It looks like a straightforward and
> > > universal workaround. Or is node index better in some sense? Why?
> > >
> > > Please, share your thoughts.
> > >
> > > чт, 28 февр. 2019 г. в 17:43, Dmitriy Pavlov <[hidden email]>:
> > > >
> > > > Hi Ivan,
> > > >
> > > > Yes, you catch me, I'm a little bit cheating with lazy consensus on
> > code
> > > > modification without providing a PR because I was expecting that nobody
> > > > comes to discussion. I will prepare PR shortly. And since we anyway
> > have
> > > a
> > > > discussion, I will not apply anything by lazy approval.
> > > >
> > > > - storageNodeIndex without consistent ID will not work.
> > > >  cfg.getDataStorageConfiguration().setNodeIdx() will be required only
> > for
> > > > case we have consistent ID.
> > > >
> > > > Hi Stanislav,
> > > >
> > > > We can't use only consistent ID because
> > > >
> > > > 1) we need an order to scan and lock random-UUID based folders.  Node
> > > index
> > > > provides the order of scan. I can find the corresponding discussion,
> > but
> > > I
> > > > guess it is not needed.
> > > > 2) we need to separate backward compatible folders from new random-UUID
> > > > based folders. Using UUID as folder will not allow us to scan only new
> > > name
> > > > format folders.
> > > >
> > > > I guess specifying node index is a quite rare case and good JavaDoc
> > will
> > > > always help.
> > > >
> > > > DataStorageConfiguration().setNodeIdx()  JavaDoc may include following
> > > > notes:
> > > > Node index used for persistent store folders in case several nodes
> > reuse
> > > > one persistent store root folder.
> > > >
> > > > Sincerely,
> > > > Dmitriy Pavlov
> > > >
> > > > чт, 28 февр. 2019 г. в 08:03, Павлухин Иван <[hidden email]>:
> > > >
> > > > > Dmitiy,
> > > > >
> > > > > Could please clarify one thing:
> > > > > 1. Will it be enough to use only storageNodeIndex in order to reuse
> > > > > the same persistence folders when consistentId is auto-generated?
> > E.g.
> > > > > I have a configuration with storageNodeIndex=1 and without explicitly
> > > > > specified consistentId, will the node after restart use the same
> > > > > persistence folder as before restart?
> > > > >
> > > > > Also a side note:
> > > > > > Please share your vision. I'm going to apply this change by lazy
> > > > > consensus
> > > > > in 3 days.
> > > > > What do you mean by "apply"? I have not seen any PR yet.
> > > > >
> > > > > ср, 27 февр. 2019 г. в 21:12, Dmitriy Pavlov <[hidden email]>:
> > > > > >
> > > > > > Hi Igniters,
> > > > > >
> > > > > > I would like to fix the issue
> > > > > > https://issues.apache.org/jira/browse/IGNITE-11432 about
> > specifying
> > > some
> > > > > > previous randomly generated UUID as a new consistent ID. Folder
> > > > > generation
> > > > > > algorithm here (
> > > > > >
> > > > >
> > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood
> > > > > )
> > > > > > allows two options
> > > > > > -node00+random UUID
> > > > > > - consistendId
> > > > > >
> > > > > > I would like to add to Ignite configuration new property nodeIndex
> > in
> > > > > > addition to consistent Id. New Property will be named as
> > > > > storageNodeIndex,
> > > > > > int, zero-based.
> > > > > > This will add the third option of subfolders processing:
> > > > > > node{storageNodeIndex}+consistentID
> > > > > >
> > > > > > Please share your vision. I'm going to apply this change by lazy
> > > > > consensus
> > > > > > in 3 days.
> > > > > >
> > > > > > Sincerely,
> > > > > > Dmitriy Pavlov
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Ivan Pavlukhin
> > > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > > Ivan Pavlukhin
> > >
> >

--
Best regards,
Ivan Pavlukhin

[Dmitry Pavlov]

Hi, I've prepared demo PR for approach 1, just checking UUID and trying to
use node00-UUID. https://github.com/apache/ignite/pull/6266/ Actually for
this approach it is really simple, no API change is required.

I guess the case is not very often because automatic folders assignment
works well in most cases.

WDYT?

ср, 13 мар. 2019 г. в 09:42, Павлухин Иван <[hidden email]>:

> Alex,
>
> Why do we need
> >  - check if consistent ID is set to a string 'nodeXX-UUID'. In this case
> > the consistent ID is set to UUID, and the storage folder is chosen
> > according to the proper rules. This change has a minimal chance to affect
> > current users because it's unlikely that somebody is using auto-generated
> > folder naming scheme as consistent ID.
> ?
> It looks hacky as well. The thing I do not like here is that a
> consistentId specified in configuration is not a consistentId used by
> a node sometimes.
>
> Can we go just with
> > - either check if consistent ID is an instance of UUID and then take the
> > appropriate folder. This approach is straightforward, but may affect
> > current users
> ?
>
> And as a last chance a user will have a possibility to rename a directory.
>
>
> вт, 12 мар. 2019 г. в 19:33, Alexey Goncharuk <[hidden email]
> >:
> >
> > Igniters,
> >
> > I came across the same issue during development and found no sane
> > workaround for this issue. I believe the solution should be as simple as
> > possible because we are already adding a warning to let users know that
> it
> > is good to specify a consistent ID in production deployments.
> >
> > As for the current solution, I do not like adding a new configuration
> > property like 'storageFolder' because it's another way to shoot yourself
> in
> > the leg (e.g. different nodes have different consistent IDs but
> configured
> > to have the same storage folder).
> > Why can't we:
> >  - either check if consistent ID is an instance of UUID and then take the
> > appropriate folder. This approach is straightforward, but may affect
> > current users
> >  - check if consistent ID is set to a string 'nodeXX-UUID'. In this case
> > the consistent ID is set to UUID, and the storage folder is chosen
> > according to the proper rules. This change has a minimal chance to affect
> > current users because it's unlikely that somebody is using auto-generated
> > folder naming scheme as consistent ID.
> >
> > Thoughts?
> >
> > вт, 12 мар. 2019 г. в 16:51, Dmitriy Pavlov <[hidden email]>:
> >
> > > Hi Igniters,
> > >
> > > A full description can be found at wiki page
> > >
> > >
> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood#IgnitePersistentStore-underthehood-SubfoldersGeneration
> > >
> > > I like the idea of introducing a property like nodeStorageName to
> specify
> > > exactly name of a directory to use. For example, it could have higher
> > > priority then IgniteConfiguration.getConsistentId(). It will impact the
> > > mentioned algorithm, but it seems to be easier to understand by users.
> > >
> > > If nobody else minds, I will try to implement this idea.
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > >
> > > сб, 2 мар. 2019 г. в 08:33, Павлухин Иван <[hidden email]>:
> > >
> > > > Dmitriy,
> > > >
> > > > You wrote
> > > > > we need an order to scan and lock random-UUID based folders.
> > > >
> > > > It would be great if you provide a discussion about that order to
> > > > complete the picture. Currently I cannot understand why the order is
> > > > important.
> > > >
> > > > Also, couple of raw thoughts:
> > > > 1. Can we extend a directory lookup procedure when consistent id is
> > > > specified to check nodeXX-consistentId directories as well?
> > > > 2. Can we introduce a property like nodeStorageName to specify exact
> > > > name of a directory to use? It looks like a straightforward and
> > > > universal workaround. Or is node index better in some sense? Why?
> > > >
> > > > Please, share your thoughts.
> > > >
> > > > чт, 28 февр. 2019 г. в 17:43, Dmitriy Pavlov <[hidden email]>:
> > > > >
> > > > > Hi Ivan,
> > > > >
> > > > > Yes, you catch me, I'm a little bit cheating with lazy consensus on
> > > code
> > > > > modification without providing a PR because I was expecting that
> nobody
> > > > > comes to discussion. I will prepare PR shortly. And since we anyway
> > > have
> > > > a
> > > > > discussion, I will not apply anything by lazy approval.
> > > > >
> > > > > - storageNodeIndex without consistent ID will not work.
> > > > >  cfg.getDataStorageConfiguration().setNodeIdx() will be required
> only
> > > for
> > > > > case we have consistent ID.
> > > > >
> > > > > Hi Stanislav,
> > > > >
> > > > > We can't use only consistent ID because
> > > > >
> > > > > 1) we need an order to scan and lock random-UUID based folders.
> Node
> > > > index
> > > > > provides the order of scan. I can find the corresponding
> discussion,
> > > but
> > > > I
> > > > > guess it is not needed.
> > > > > 2) we need to separate backward compatible folders from new
> random-UUID
> > > > > based folders. Using UUID as folder will not allow us to scan only
> new
> > > > name
> > > > > format folders.
> > > > >
> > > > > I guess specifying node index is a quite rare case and good JavaDoc
> > > will
> > > > > always help.
> > > > >
> > > > > DataStorageConfiguration().setNodeIdx()  JavaDoc may include
> following
> > > > > notes:
> > > > > Node index used for persistent store folders in case several nodes
> > > reuse
> > > > > one persistent store root folder.
> > > > >
> > > > > Sincerely,
> > > > > Dmitriy Pavlov
> > > > >
> > > > > чт, 28 февр. 2019 г. в 08:03, Павлухин Иван <[hidden email]>:
> > > > >
> > > > > > Dmitiy,
> > > > > >
> > > > > > Could please clarify one thing:
> > > > > > 1. Will it be enough to use only storageNodeIndex in order to
> reuse
> > > > > > the same persistence folders when consistentId is auto-generated?
> > > E.g.
> > > > > > I have a configuration with storageNodeIndex=1 and without
> explicitly
> > > > > > specified consistentId, will the node after restart use the same
> > > > > > persistence folder as before restart?
> > > > > >
> > > > > > Also a side note:
> > > > > > > Please share your vision. I'm going to apply this change by
> lazy
> > > > > > consensus
> > > > > > in 3 days.
> > > > > > What do you mean by "apply"? I have not seen any PR yet.
> > > > > >
> > > > > > ср, 27 февр. 2019 г. в 21:12, Dmitriy Pavlov <[hidden email]
> >:
> > > > > > >
> > > > > > > Hi Igniters,
> > > > > > >
> > > > > > > I would like to fix the issue
> > > > > > > https://issues.apache.org/jira/browse/IGNITE-11432 about
> > > specifying
> > > > some
> > > > > > > previous randomly generated UUID as a new consistent ID. Folder
> > > > > > generation
> > > > > > > algorithm here (
> > > > > > >
> > > > > >
> > > >
> > >
> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood
> > > > > > )
> > > > > > > allows two options
> > > > > > > -node00+random UUID
> > > > > > > - consistendId
> > > > > > >
> > > > > > > I would like to add to Ignite configuration new property
> nodeIndex
> > > in
> > > > > > > addition to consistent Id. New Property will be named as
> > > > > > storageNodeIndex,
> > > > > > > int, zero-based.
> > > > > > > This will add the third option of subfolders processing:
> > > > > > > node{storageNodeIndex}+consistentID
> > > > > > >
> > > > > > > Please share your vision. I'm going to apply this change by
> lazy
> > > > > > consensus
> > > > > > > in 3 days.
> > > > > > >
> > > > > > > Sincerely,
> > > > > > > Dmitriy Pavlov
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Best regards,
> > > > > > Ivan Pavlukhin
> > > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Ivan Pavlukhin
> > > >
> > >
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>

[Ivan Pavlukhin]

Hi Dmitriy,

It looks like that case with only one node in working directory is
covered. But what is problem is solved? As I understand before fix in
such case node after restart uses proper folder and proper
consistentId. After fix we can specify automatically generated UUID in
configuration and everything will go smooth. Is it the main problem
case? If yes then such simplicity is very attractive.

But honestly I do not fully understand why do we need to return a node
to node00-UUID directory after that directory was deleted?

чт, 14 мар. 2019 г. в 17:21, Dmitriy Pavlov <[hidden email]>:

>
> Hi, I've prepared demo PR for approach 1, just checking UUID and trying to
> use node00-UUID. https://github.com/apache/ignite/pull/6266/ Actually for
> this approach it is really simple, no API change is required.
>
> I guess the case is not very often because automatic folders assignment
> works well in most cases.
>
> WDYT?
>
> ср, 13 мар. 2019 г. в 09:42, Павлухин Иван <[hidden email]>:
>
> > Alex,
> >
> > Why do we need
> > >  - check if consistent ID is set to a string 'nodeXX-UUID'. In this case
> > > the consistent ID is set to UUID, and the storage folder is chosen
> > > according to the proper rules. This change has a minimal chance to affect
> > > current users because it's unlikely that somebody is using auto-generated
> > > folder naming scheme as consistent ID.
> > ?
> > It looks hacky as well. The thing I do not like here is that a
> > consistentId specified in configuration is not a consistentId used by
> > a node sometimes.
> >
> > Can we go just with
> > > - either check if consistent ID is an instance of UUID and then take the
> > > appropriate folder. This approach is straightforward, but may affect
> > > current users
> > ?
> >
> > And as a last chance a user will have a possibility to rename a directory.
> >
> >
> > вт, 12 мар. 2019 г. в 19:33, Alexey Goncharuk <[hidden email]
> > >:
> > >
> > > Igniters,
> > >
> > > I came across the same issue during development and found no sane
> > > workaround for this issue. I believe the solution should be as simple as
> > > possible because we are already adding a warning to let users know that
> > it
> > > is good to specify a consistent ID in production deployments.
> > >
> > > As for the current solution, I do not like adding a new configuration
> > > property like 'storageFolder' because it's another way to shoot yourself
> > in
> > > the leg (e.g. different nodes have different consistent IDs but
> > configured
> > > to have the same storage folder).
> > > Why can't we:
> > >  - either check if consistent ID is an instance of UUID and then take the
> > > appropriate folder. This approach is straightforward, but may affect
> > > current users
> > >  - check if consistent ID is set to a string 'nodeXX-UUID'. In this case
> > > the consistent ID is set to UUID, and the storage folder is chosen
> > > according to the proper rules. This change has a minimal chance to affect
> > > current users because it's unlikely that somebody is using auto-generated
> > > folder naming scheme as consistent ID.
> > >
> > > Thoughts?
> > >
> > > вт, 12 мар. 2019 г. в 16:51, Dmitriy Pavlov <[hidden email]>:
> > >
> > > > Hi Igniters,
> > > >
> > > > A full description can be found at wiki page
> > > >
> > > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood#IgnitePersistentStore-underthehood-SubfoldersGeneration
> > > >
> > > > I like the idea of introducing a property like nodeStorageName to
> > specify
> > > > exactly name of a directory to use. For example, it could have higher
> > > > priority then IgniteConfiguration.getConsistentId(). It will impact the
> > > > mentioned algorithm, but it seems to be easier to understand by users.
> > > >
> > > > If nobody else minds, I will try to implement this idea.
> > > >
> > > > Sincerely,
> > > > Dmitriy Pavlov
> > > >
> > > >
> > > > сб, 2 мар. 2019 г. в 08:33, Павлухин Иван <[hidden email]>:
> > > >
> > > > > Dmitriy,
> > > > >
> > > > > You wrote
> > > > > > we need an order to scan and lock random-UUID based folders.
> > > > >
> > > > > It would be great if you provide a discussion about that order to
> > > > > complete the picture. Currently I cannot understand why the order is
> > > > > important.
> > > > >
> > > > > Also, couple of raw thoughts:
> > > > > 1. Can we extend a directory lookup procedure when consistent id is
> > > > > specified to check nodeXX-consistentId directories as well?
> > > > > 2. Can we introduce a property like nodeStorageName to specify exact
> > > > > name of a directory to use? It looks like a straightforward and
> > > > > universal workaround. Or is node index better in some sense? Why?
> > > > >
> > > > > Please, share your thoughts.
> > > > >
> > > > > чт, 28 февр. 2019 г. в 17:43, Dmitriy Pavlov <[hidden email]>:
> > > > > >
> > > > > > Hi Ivan,
> > > > > >
> > > > > > Yes, you catch me, I'm a little bit cheating with lazy consensus on
> > > > code
> > > > > > modification without providing a PR because I was expecting that
> > nobody
> > > > > > comes to discussion. I will prepare PR shortly. And since we anyway
> > > > have
> > > > > a
> > > > > > discussion, I will not apply anything by lazy approval.
> > > > > >
> > > > > > - storageNodeIndex without consistent ID will not work.
> > > > > >  cfg.getDataStorageConfiguration().setNodeIdx() will be required
> > only
> > > > for
> > > > > > case we have consistent ID.
> > > > > >
> > > > > > Hi Stanislav,
> > > > > >
> > > > > > We can't use only consistent ID because
> > > > > >
> > > > > > 1) we need an order to scan and lock random-UUID based folders.
> > Node
> > > > > index
> > > > > > provides the order of scan. I can find the corresponding
> > discussion,
> > > > but
> > > > > I
> > > > > > guess it is not needed.
> > > > > > 2) we need to separate backward compatible folders from new
> > random-UUID
> > > > > > based folders. Using UUID as folder will not allow us to scan only
> > new
> > > > > name
> > > > > > format folders.
> > > > > >
> > > > > > I guess specifying node index is a quite rare case and good JavaDoc
> > > > will
> > > > > > always help.
> > > > > >
> > > > > > DataStorageConfiguration().setNodeIdx()  JavaDoc may include
> > following
> > > > > > notes:
> > > > > > Node index used for persistent store folders in case several nodes
> > > > reuse
> > > > > > one persistent store root folder.
> > > > > >
> > > > > > Sincerely,
> > > > > > Dmitriy Pavlov
> > > > > >
> > > > > > чт, 28 февр. 2019 г. в 08:03, Павлухин Иван <[hidden email]>:
> > > > > >
> > > > > > > Dmitiy,
> > > > > > >
> > > > > > > Could please clarify one thing:
> > > > > > > 1. Will it be enough to use only storageNodeIndex in order to
> > reuse
> > > > > > > the same persistence folders when consistentId is auto-generated?
> > > > E.g.
> > > > > > > I have a configuration with storageNodeIndex=1 and without
> > explicitly
> > > > > > > specified consistentId, will the node after restart use the same
> > > > > > > persistence folder as before restart?
> > > > > > >
> > > > > > > Also a side note:
> > > > > > > > Please share your vision. I'm going to apply this change by
> > lazy
> > > > > > > consensus
> > > > > > > in 3 days.
> > > > > > > What do you mean by "apply"? I have not seen any PR yet.
> > > > > > >
> > > > > > > ср, 27 февр. 2019 г. в 21:12, Dmitriy Pavlov <[hidden email]
> > >:
> > > > > > > >
> > > > > > > > Hi Igniters,
> > > > > > > >
> > > > > > > > I would like to fix the issue
> > > > > > > > https://issues.apache.org/jira/browse/IGNITE-11432 about
> > > > specifying
> > > > > some
> > > > > > > > previous randomly generated UUID as a new consistent ID. Folder
> > > > > > > generation
> > > > > > > > algorithm here (
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood
> > > > > > > )
> > > > > > > > allows two options
> > > > > > > > -node00+random UUID
> > > > > > > > - consistendId
> > > > > > > >
> > > > > > > > I would like to add to Ignite configuration new property
> > nodeIndex
> > > > in
> > > > > > > > addition to consistent Id. New Property will be named as
> > > > > > > storageNodeIndex,
> > > > > > > > int, zero-based.
> > > > > > > > This will add the third option of subfolders processing:
> > > > > > > > node{storageNodeIndex}+consistentID
> > > > > > > >
> > > > > > > > Please share your vision. I'm going to apply this change by
> > lazy
> > > > > > > consensus
> > > > > > > > in 3 days.
> > > > > > > >
> > > > > > > > Sincerely,
> > > > > > > > Dmitriy Pavlov
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Best regards,
> > > > > > > Ivan Pavlukhin
> > > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Ivan Pavlukhin
> > > > >
> > > >
> >
> >
> >
> > --
> > Best regards,
> > Ivan Pavlukhin
> >

--
Best regards,
Ivan Pavlukhin

[Dmitry Pavlov]

Hi Ivan, yes, after the restart we need to re-use both folder name and UUID
as consistent ID.

About returning exact directory name: I'm just guessing here, but the
persistent store is not the only folder, there are several more folders,
e.g. binary-meta, etc.

About several nodes reusing same root folder, yes, the proposed solution
will not works because 00 is hardcoded here. But I expect it will be too
rare case.

Sincerely,
Dmitriy Pavlov

пт, 15 мар. 2019 г. в 11:29, Павлухин Иван <[hidden email]>:

> Hi Dmitriy,
>
> It looks like that case with only one node in working directory is
> covered. But what is problem is solved? As I understand before fix in
> such case node after restart uses proper folder and proper
> consistentId. After fix we can specify automatically generated UUID in
> configuration and everything will go smooth. Is it the main problem
> case? If yes then such simplicity is very attractive.
>
> But honestly I do not fully understand why do we need to return a node
> to node00-UUID directory after that directory was deleted?
>
> чт, 14 мар. 2019 г. в 17:21, Dmitriy Pavlov <[hidden email]>:
> >
> > Hi, I've prepared demo PR for approach 1, just checking UUID and trying
> to
> > use node00-UUID. https://github.com/apache/ignite/pull/6266/ Actually
> for
> > this approach it is really simple, no API change is required.
> >
> > I guess the case is not very often because automatic folders assignment
> > works well in most cases.
> >
> > WDYT?
> >
> > ср, 13 мар. 2019 г. в 09:42, Павлухин Иван <[hidden email]>:
> >
> > > Alex,
> > >
> > > Why do we need
> > > >  - check if consistent ID is set to a string 'nodeXX-UUID'. In this
> case
> > > > the consistent ID is set to UUID, and the storage folder is chosen
> > > > according to the proper rules. This change has a minimal chance to
> affect
> > > > current users because it's unlikely that somebody is using
> auto-generated
> > > > folder naming scheme as consistent ID.
> > > ?
> > > It looks hacky as well. The thing I do not like here is that a
> > > consistentId specified in configuration is not a consistentId used by
> > > a node sometimes.
> > >
> > > Can we go just with
> > > > - either check if consistent ID is an instance of UUID and then take
> the
> > > > appropriate folder. This approach is straightforward, but may affect
> > > > current users
> > > ?
> > >
> > > And as a last chance a user will have a possibility to rename a
> directory.
> > >
> > >
> > > вт, 12 мар. 2019 г. в 19:33, Alexey Goncharuk <
> [hidden email]
> > > >:
> > > >
> > > > Igniters,
> > > >
> > > > I came across the same issue during development and found no sane
> > > > workaround for this issue. I believe the solution should be as
> simple as
> > > > possible because we are already adding a warning to let users know
> that
> > > it
> > > > is good to specify a consistent ID in production deployments.
> > > >
> > > > As for the current solution, I do not like adding a new configuration
> > > > property like 'storageFolder' because it's another way to shoot
> yourself
> > > in
> > > > the leg (e.g. different nodes have different consistent IDs but
> > > configured
> > > > to have the same storage folder).
> > > > Why can't we:
> > > >  - either check if consistent ID is an instance of UUID and then
> take the
> > > > appropriate folder. This approach is straightforward, but may affect
> > > > current users
> > > >  - check if consistent ID is set to a string 'nodeXX-UUID'. In this
> case
> > > > the consistent ID is set to UUID, and the storage folder is chosen
> > > > according to the proper rules. This change has a minimal chance to
> affect
> > > > current users because it's unlikely that somebody is using
> auto-generated
> > > > folder naming scheme as consistent ID.
> > > >
> > > > Thoughts?
> > > >
> > > > вт, 12 мар. 2019 г. в 16:51, Dmitriy Pavlov <[hidden email]>:
> > > >
> > > > > Hi Igniters,
> > > > >
> > > > > A full description can be found at wiki page
> > > > >
> > > > >
> > >
> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood#IgnitePersistentStore-underthehood-SubfoldersGeneration
> > > > >
> > > > > I like the idea of introducing a property like nodeStorageName to
> > > specify
> > > > > exactly name of a directory to use. For example, it could have
> higher
> > > > > priority then IgniteConfiguration.getConsistentId(). It will
> impact the
> > > > > mentioned algorithm, but it seems to be easier to understand by
> users.
> > > > >
> > > > > If nobody else minds, I will try to implement this idea.
> > > > >
> > > > > Sincerely,
> > > > > Dmitriy Pavlov
> > > > >
> > > > >
> > > > > сб, 2 мар. 2019 г. в 08:33, Павлухин Иван <[hidden email]>:
> > > > >
> > > > > > Dmitriy,
> > > > > >
> > > > > > You wrote
> > > > > > > we need an order to scan and lock random-UUID based folders.
> > > > > >
> > > > > > It would be great if you provide a discussion about that order to
> > > > > > complete the picture. Currently I cannot understand why the
> order is
> > > > > > important.
> > > > > >
> > > > > > Also, couple of raw thoughts:
> > > > > > 1. Can we extend a directory lookup procedure when consistent id
> is
> > > > > > specified to check nodeXX-consistentId directories as well?
> > > > > > 2. Can we introduce a property like nodeStorageName to specify
> exact
> > > > > > name of a directory to use? It looks like a straightforward and
> > > > > > universal workaround. Or is node index better in some sense? Why?
> > > > > >
> > > > > > Please, share your thoughts.
> > > > > >
> > > > > > чт, 28 февр. 2019 г. в 17:43, Dmitriy Pavlov <[hidden email]
> >:
> > > > > > >
> > > > > > > Hi Ivan,
> > > > > > >
> > > > > > > Yes, you catch me, I'm a little bit cheating with lazy
> consensus on
> > > > > code
> > > > > > > modification without providing a PR because I was expecting
> that
> > > nobody
> > > > > > > comes to discussion. I will prepare PR shortly. And since we
> anyway
> > > > > have
> > > > > > a
> > > > > > > discussion, I will not apply anything by lazy approval.
> > > > > > >
> > > > > > > - storageNodeIndex without consistent ID will not work.
> > > > > > >  cfg.getDataStorageConfiguration().setNodeIdx() will be
> required
> > > only
> > > > > for
> > > > > > > case we have consistent ID.
> > > > > > >
> > > > > > > Hi Stanislav,
> > > > > > >
> > > > > > > We can't use only consistent ID because
> > > > > > >
> > > > > > > 1) we need an order to scan and lock random-UUID based folders.
> > > Node
> > > > > > index
> > > > > > > provides the order of scan. I can find the corresponding
> > > discussion,
> > > > > but
> > > > > > I
> > > > > > > guess it is not needed.
> > > > > > > 2) we need to separate backward compatible folders from new
> > > random-UUID
> > > > > > > based folders. Using UUID as folder will not allow us to scan
> only
> > > new
> > > > > > name
> > > > > > > format folders.
> > > > > > >
> > > > > > > I guess specifying node index is a quite rare case and good
> JavaDoc
> > > > > will
> > > > > > > always help.
> > > > > > >
> > > > > > > DataStorageConfiguration().setNodeIdx()  JavaDoc may include
> > > following
> > > > > > > notes:
> > > > > > > Node index used for persistent store folders in case several
> nodes
> > > > > reuse
> > > > > > > one persistent store root folder.
> > > > > > >
> > > > > > > Sincerely,
> > > > > > > Dmitriy Pavlov
> > > > > > >
> > > > > > > чт, 28 февр. 2019 г. в 08:03, Павлухин Иван <
> [hidden email]>:
> > > > > > >
> > > > > > > > Dmitiy,
> > > > > > > >
> > > > > > > > Could please clarify one thing:
> > > > > > > > 1. Will it be enough to use only storageNodeIndex in order to
> > > reuse
> > > > > > > > the same persistence folders when consistentId is
> auto-generated?
> > > > > E.g.
> > > > > > > > I have a configuration with storageNodeIndex=1 and without
> > > explicitly
> > > > > > > > specified consistentId, will the node after restart use the
> same
> > > > > > > > persistence folder as before restart?
> > > > > > > >
> > > > > > > > Also a side note:
> > > > > > > > > Please share your vision. I'm going to apply this change by
> > > lazy
> > > > > > > > consensus
> > > > > > > > in 3 days.
> > > > > > > > What do you mean by "apply"? I have not seen any PR yet.
> > > > > > > >
> > > > > > > > ср, 27 февр. 2019 г. в 21:12, Dmitriy Pavlov <
> [hidden email]
> > > >:
> > > > > > > > >
> > > > > > > > > Hi Igniters,
> > > > > > > > >
> > > > > > > > > I would like to fix the issue
> > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-11432 about
> > > > > specifying
> > > > > > some
> > > > > > > > > previous randomly generated UUID as a new consistent ID.
> Folder
> > > > > > > > generation
> > > > > > > > > algorithm here (
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > >
> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood
> > > > > > > > )
> > > > > > > > > allows two options
> > > > > > > > > -node00+random UUID
> > > > > > > > > - consistendId
> > > > > > > > >
> > > > > > > > > I would like to add to Ignite configuration new property
> > > nodeIndex
> > > > > in
> > > > > > > > > addition to consistent Id. New Property will be named as
> > > > > > > > storageNodeIndex,
> > > > > > > > > int, zero-based.
> > > > > > > > > This will add the third option of subfolders processing:
> > > > > > > > > node{storageNodeIndex}+consistentID
> > > > > > > > >
> > > > > > > > > Please share your vision. I'm going to apply this change by
> > > lazy
> > > > > > > > consensus
> > > > > > > > > in 3 days.
> > > > > > > > >
> > > > > > > > > Sincerely,
> > > > > > > > > Dmitriy Pavlov
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Best regards,
> > > > > > > > Ivan Pavlukhin
> > > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Best regards,
> > > > > > Ivan Pavlukhin
> > > > > >
> > > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > > Ivan Pavlukhin
> > >
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>

[Ivan Pavlukhin]

Hi Dmitriy,

It is a pitty that we have to guess here. Some comments:
1. The usage scenario is not very complex and erything might work well
with aforementioned patch. Unfortunately some tradeoff and extra
transformations are involved and consequently it is hard to predict
how many users will be troubled. But on a bright side the change is
seemless and perhaps nobody will be really troubled.
2. From "honest" code perspective introducing nodeStorageName seems
better to me because no (surprising) transformations are involved.
3. Also, it seems to me that original logic for choosing storage was
not designed well. From the first glance it looks strange that need
folders for a particular node in different places. Is not it much
clear to have all data of the node in a single directory isolated from
other nodes? AFAIK Hazelcast follows this approach. They generate an
error when someone is trying to launch second instance in the same
working directory. Perhaps, we at least should think about changing
our layout and consider including such change in some future release.

Thank you!

пт, 15 мар. 2019 г. в 17:55, Dmitriy Pavlov <[hidden email]>:

>
> Hi Ivan, yes, after the restart we need to re-use both folder name and UUID
> as consistent ID.
>
> About returning exact directory name: I'm just guessing here, but the
> persistent store is not the only folder, there are several more folders,
> e.g. binary-meta, etc.
>
> About several nodes reusing same root folder, yes, the proposed solution
> will not works because 00 is hardcoded here. But I expect it will be too
> rare case.
>
> Sincerely,
> Dmitriy Pavlov
>
> пт, 15 мар. 2019 г. в 11:29, Павлухин Иван <[hidden email]>:
>
> > Hi Dmitriy,
> >
> > It looks like that case with only one node in working directory is
> > covered. But what is problem is solved? As I understand before fix in
> > such case node after restart uses proper folder and proper
> > consistentId. After fix we can specify automatically generated UUID in
> > configuration and everything will go smooth. Is it the main problem
> > case? If yes then such simplicity is very attractive.
> >
> > But honestly I do not fully understand why do we need to return a node
> > to node00-UUID directory after that directory was deleted?
> >
> > чт, 14 мар. 2019 г. в 17:21, Dmitriy Pavlov <[hidden email]>:
> > >
> > > Hi, I've prepared demo PR for approach 1, just checking UUID and trying
> > to
> > > use node00-UUID. https://github.com/apache/ignite/pull/6266/ Actually
> > for
> > > this approach it is really simple, no API change is required.
> > >
> > > I guess the case is not very often because automatic folders assignment
> > > works well in most cases.
> > >
> > > WDYT?
> > >
> > > ср, 13 мар. 2019 г. в 09:42, Павлухин Иван <[hidden email]>:
> > >
> > > > Alex,
> > > >
> > > > Why do we need
> > > > >  - check if consistent ID is set to a string 'nodeXX-UUID'. In this
> > case
> > > > > the consistent ID is set to UUID, and the storage folder is chosen
> > > > > according to the proper rules. This change has a minimal chance to
> > affect
> > > > > current users because it's unlikely that somebody is using
> > auto-generated
> > > > > folder naming scheme as consistent ID.
> > > > ?
> > > > It looks hacky as well. The thing I do not like here is that a
> > > > consistentId specified in configuration is not a consistentId used by
> > > > a node sometimes.
> > > >
> > > > Can we go just with
> > > > > - either check if consistent ID is an instance of UUID and then take
> > the
> > > > > appropriate folder. This approach is straightforward, but may affect
> > > > > current users
> > > > ?
> > > >
> > > > And as a last chance a user will have a possibility to rename a
> > directory.
> > > >
> > > >
> > > > вт, 12 мар. 2019 г. в 19:33, Alexey Goncharuk <
> > [hidden email]
> > > > >:
> > > > >
> > > > > Igniters,
> > > > >
> > > > > I came across the same issue during development and found no sane
> > > > > workaround for this issue. I believe the solution should be as
> > simple as
> > > > > possible because we are already adding a warning to let users know
> > that
> > > > it
> > > > > is good to specify a consistent ID in production deployments.
> > > > >
> > > > > As for the current solution, I do not like adding a new configuration
> > > > > property like 'storageFolder' because it's another way to shoot
> > yourself
> > > > in
> > > > > the leg (e.g. different nodes have different consistent IDs but
> > > > configured
> > > > > to have the same storage folder).
> > > > > Why can't we:
> > > > >  - either check if consistent ID is an instance of UUID and then
> > take the
> > > > > appropriate folder. This approach is straightforward, but may affect
> > > > > current users
> > > > >  - check if consistent ID is set to a string 'nodeXX-UUID'. In this
> > case
> > > > > the consistent ID is set to UUID, and the storage folder is chosen
> > > > > according to the proper rules. This change has a minimal chance to
> > affect
> > > > > current users because it's unlikely that somebody is using
> > auto-generated
> > > > > folder naming scheme as consistent ID.
> > > > >
> > > > > Thoughts?
> > > > >
> > > > > вт, 12 мар. 2019 г. в 16:51, Dmitriy Pavlov <[hidden email]>:
> > > > >
> > > > > > Hi Igniters,
> > > > > >
> > > > > > A full description can be found at wiki page
> > > > > >
> > > > > >
> > > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood#IgnitePersistentStore-underthehood-SubfoldersGeneration
> > > > > >
> > > > > > I like the idea of introducing a property like nodeStorageName to
> > > > specify
> > > > > > exactly name of a directory to use. For example, it could have
> > higher
> > > > > > priority then IgniteConfiguration.getConsistentId(). It will
> > impact the
> > > > > > mentioned algorithm, but it seems to be easier to understand by
> > users.
> > > > > >
> > > > > > If nobody else minds, I will try to implement this idea.
> > > > > >
> > > > > > Sincerely,
> > > > > > Dmitriy Pavlov
> > > > > >
> > > > > >
> > > > > > сб, 2 мар. 2019 г. в 08:33, Павлухин Иван <[hidden email]>:
> > > > > >
> > > > > > > Dmitriy,
> > > > > > >
> > > > > > > You wrote
> > > > > > > > we need an order to scan and lock random-UUID based folders.
> > > > > > >
> > > > > > > It would be great if you provide a discussion about that order to
> > > > > > > complete the picture. Currently I cannot understand why the
> > order is
> > > > > > > important.
> > > > > > >
> > > > > > > Also, couple of raw thoughts:
> > > > > > > 1. Can we extend a directory lookup procedure when consistent id
> > is
> > > > > > > specified to check nodeXX-consistentId directories as well?
> > > > > > > 2. Can we introduce a property like nodeStorageName to specify
> > exact
> > > > > > > name of a directory to use? It looks like a straightforward and
> > > > > > > universal workaround. Or is node index better in some sense? Why?
> > > > > > >
> > > > > > > Please, share your thoughts.
> > > > > > >
> > > > > > > чт, 28 февр. 2019 г. в 17:43, Dmitriy Pavlov <[hidden email]
> > >:
> > > > > > > >
> > > > > > > > Hi Ivan,
> > > > > > > >
> > > > > > > > Yes, you catch me, I'm a little bit cheating with lazy
> > consensus on
> > > > > > code
> > > > > > > > modification without providing a PR because I was expecting
> > that
> > > > nobody
> > > > > > > > comes to discussion. I will prepare PR shortly. And since we
> > anyway
> > > > > > have
> > > > > > > a
> > > > > > > > discussion, I will not apply anything by lazy approval.
> > > > > > > >
> > > > > > > > - storageNodeIndex without consistent ID will not work.
> > > > > > > >  cfg.getDataStorageConfiguration().setNodeIdx() will be
> > required
> > > > only
> > > > > > for
> > > > > > > > case we have consistent ID.
> > > > > > > >
> > > > > > > > Hi Stanislav,
> > > > > > > >
> > > > > > > > We can't use only consistent ID because
> > > > > > > >
> > > > > > > > 1) we need an order to scan and lock random-UUID based folders.
> > > > Node
> > > > > > > index
> > > > > > > > provides the order of scan. I can find the corresponding
> > > > discussion,
> > > > > > but
> > > > > > > I
> > > > > > > > guess it is not needed.
> > > > > > > > 2) we need to separate backward compatible folders from new
> > > > random-UUID
> > > > > > > > based folders. Using UUID as folder will not allow us to scan
> > only
> > > > new
> > > > > > > name
> > > > > > > > format folders.
> > > > > > > >
> > > > > > > > I guess specifying node index is a quite rare case and good
> > JavaDoc
> > > > > > will
> > > > > > > > always help.
> > > > > > > >
> > > > > > > > DataStorageConfiguration().setNodeIdx()  JavaDoc may include
> > > > following
> > > > > > > > notes:
> > > > > > > > Node index used for persistent store folders in case several
> > nodes
> > > > > > reuse
> > > > > > > > one persistent store root folder.
> > > > > > > >
> > > > > > > > Sincerely,
> > > > > > > > Dmitriy Pavlov
> > > > > > > >
> > > > > > > > чт, 28 февр. 2019 г. в 08:03, Павлухин Иван <
> > [hidden email]>:
> > > > > > > >
> > > > > > > > > Dmitiy,
> > > > > > > > >
> > > > > > > > > Could please clarify one thing:
> > > > > > > > > 1. Will it be enough to use only storageNodeIndex in order to
> > > > reuse
> > > > > > > > > the same persistence folders when consistentId is
> > auto-generated?
> > > > > > E.g.
> > > > > > > > > I have a configuration with storageNodeIndex=1 and without
> > > > explicitly
> > > > > > > > > specified consistentId, will the node after restart use the
> > same
> > > > > > > > > persistence folder as before restart?
> > > > > > > > >
> > > > > > > > > Also a side note:
> > > > > > > > > > Please share your vision. I'm going to apply this change by
> > > > lazy
> > > > > > > > > consensus
> > > > > > > > > in 3 days.
> > > > > > > > > What do you mean by "apply"? I have not seen any PR yet.
> > > > > > > > >
> > > > > > > > > ср, 27 февр. 2019 г. в 21:12, Dmitriy Pavlov <
> > [hidden email]
> > > > >:
> > > > > > > > > >
> > > > > > > > > > Hi Igniters,
> > > > > > > > > >
> > > > > > > > > > I would like to fix the issue
> > > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-11432 about
> > > > > > specifying
> > > > > > > some
> > > > > > > > > > previous randomly generated UUID as a new consistent ID.
> > Folder
> > > > > > > > > generation
> > > > > > > > > > algorithm here (
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood
> > > > > > > > > )
> > > > > > > > > > allows two options
> > > > > > > > > > -node00+random UUID
> > > > > > > > > > - consistendId
> > > > > > > > > >
> > > > > > > > > > I would like to add to Ignite configuration new property
> > > > nodeIndex
> > > > > > in
> > > > > > > > > > addition to consistent Id. New Property will be named as
> > > > > > > > > storageNodeIndex,
> > > > > > > > > > int, zero-based.
> > > > > > > > > > This will add the third option of subfolders processing:
> > > > > > > > > > node{storageNodeIndex}+consistentID
> > > > > > > > > >
> > > > > > > > > > Please share your vision. I'm going to apply this change by
> > > > lazy
> > > > > > > > > consensus
> > > > > > > > > > in 3 days.
> > > > > > > > > >
> > > > > > > > > > Sincerely,
> > > > > > > > > > Dmitriy Pavlov
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Best regards,
> > > > > > > > > Ivan Pavlukhin
> > > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Best regards,
> > > > > > > Ivan Pavlukhin
> > > > > > >
> > > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Ivan Pavlukhin
> > > >
> >
> >
> >
> > --
> > Best regards,
> > Ivan Pavlukhin
> >

--
Best regards,
Ivan Pavlukhin

[Dmitry Pavlov]

Hi Ivan,

reasonable points, but we had a long and hot discussion about this logic.

Reason for selecting auto-generation was users that just try to use Apache
Ignite with native persistence for the first time. They must be able to
test features without any extensive setup, but before go-live we expect
them to study the product a little bit more. There are not so many reasons
to use the same folder (or even same device) for 2+ nodes.

Sincerely,
Dmitriy Pavlov

вс, 17 мар. 2019 г. в 13:34, Павлухин Иван <[hidden email]>:

> Hi Dmitriy,
>
> It is a pitty that we have to guess here. Some comments:
> 1. The usage scenario is not very complex and erything might work well
> with aforementioned patch. Unfortunately some tradeoff and extra
> transformations are involved and consequently it is hard to predict
> how many users will be troubled. But on a bright side the change is
> seemless and perhaps nobody will be really troubled.
> 2. From "honest" code perspective introducing nodeStorageName seems
> better to me because no (surprising) transformations are involved.
> 3. Also, it seems to me that original logic for choosing storage was
> not designed well. From the first glance it looks strange that need
> folders for a particular node in different places. Is not it much
> clear to have all data of the node in a single directory isolated from
> other nodes? AFAIK Hazelcast follows this approach. They generate an
> error when someone is trying to launch second instance in the same
> working directory. Perhaps, we at least should think about changing
> our layout and consider including such change in some future release.
>
> Thank you!
>
> пт, 15 мар. 2019 г. в 17:55, Dmitriy Pavlov <[hidden email]>:
> >
> > Hi Ivan, yes, after the restart we need to re-use both folder name and
> UUID
> > as consistent ID.
> >
> > About returning exact directory name: I'm just guessing here, but the
> > persistent store is not the only folder, there are several more folders,
> > e.g. binary-meta, etc.
> >
> > About several nodes reusing same root folder, yes, the proposed solution
> > will not works because 00 is hardcoded here. But I expect it will be too
> > rare case.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > пт, 15 мар. 2019 г. в 11:29, Павлухин Иван <[hidden email]>:
> >
> > > Hi Dmitriy,
> > >
> > > It looks like that case with only one node in working directory is
> > > covered. But what is problem is solved? As I understand before fix in
> > > such case node after restart uses proper folder and proper
> > > consistentId. After fix we can specify automatically generated UUID in
> > > configuration and everything will go smooth. Is it the main problem
> > > case? If yes then such simplicity is very attractive.
> > >
> > > But honestly I do not fully understand why do we need to return a node
> > > to node00-UUID directory after that directory was deleted?
> > >
> > > чт, 14 мар. 2019 г. в 17:21, Dmitriy Pavlov <[hidden email]>:
> > > >
> > > > Hi, I've prepared demo PR for approach 1, just checking UUID and
> trying
> > > to
> > > > use node00-UUID. https://github.com/apache/ignite/pull/6266/
> Actually
> > > for
> > > > this approach it is really simple, no API change is required.
> > > >
> > > > I guess the case is not very often because automatic folders
> assignment
> > > > works well in most cases.
> > > >
> > > > WDYT?
> > > >
> > > > ср, 13 мар. 2019 г. в 09:42, Павлухин Иван <[hidden email]>:
> > > >
> > > > > Alex,
> > > > >
> > > > > Why do we need
> > > > > >  - check if consistent ID is set to a string 'nodeXX-UUID'. In
> this
> > > case
> > > > > > the consistent ID is set to UUID, and the storage folder is
> chosen
> > > > > > according to the proper rules. This change has a minimal chance
> to
> > > affect
> > > > > > current users because it's unlikely that somebody is using
> > > auto-generated
> > > > > > folder naming scheme as consistent ID.
> > > > > ?
> > > > > It looks hacky as well. The thing I do not like here is that a
> > > > > consistentId specified in configuration is not a consistentId used
> by
> > > > > a node sometimes.
> > > > >
> > > > > Can we go just with
> > > > > > - either check if consistent ID is an instance of UUID and then
> take
> > > the
> > > > > > appropriate folder. This approach is straightforward, but may
> affect
> > > > > > current users
> > > > > ?
> > > > >
> > > > > And as a last chance a user will have a possibility to rename a
> > > directory.
> > > > >
> > > > >
> > > > > вт, 12 мар. 2019 г. в 19:33, Alexey Goncharuk <
> > > [hidden email]
> > > > > >:
> > > > > >
> > > > > > Igniters,
> > > > > >
> > > > > > I came across the same issue during development and found no sane
> > > > > > workaround for this issue. I believe the solution should be as
> > > simple as
> > > > > > possible because we are already adding a warning to let users
> know
> > > that
> > > > > it
> > > > > > is good to specify a consistent ID in production deployments.
> > > > > >
> > > > > > As for the current solution, I do not like adding a new
> configuration
> > > > > > property like 'storageFolder' because it's another way to shoot
> > > yourself
> > > > > in
> > > > > > the leg (e.g. different nodes have different consistent IDs but
> > > > > configured
> > > > > > to have the same storage folder).
> > > > > > Why can't we:
> > > > > >  - either check if consistent ID is an instance of UUID and then
> > > take the
> > > > > > appropriate folder. This approach is straightforward, but may
> affect
> > > > > > current users
> > > > > >  - check if consistent ID is set to a string 'nodeXX-UUID'. In
> this
> > > case
> > > > > > the consistent ID is set to UUID, and the storage folder is
> chosen
> > > > > > according to the proper rules. This change has a minimal chance
> to
> > > affect
> > > > > > current users because it's unlikely that somebody is using
> > > auto-generated
> > > > > > folder naming scheme as consistent ID.
> > > > > >
> > > > > > Thoughts?
> > > > > >
> > > > > > вт, 12 мар. 2019 г. в 16:51, Dmitriy Pavlov <[hidden email]
> >:
> > > > > >
> > > > > > > Hi Igniters,
> > > > > > >
> > > > > > > A full description can be found at wiki page
> > > > > > >
> > > > > > >
> > > > >
> > >
> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood#IgnitePersistentStore-underthehood-SubfoldersGeneration
> > > > > > >
> > > > > > > I like the idea of introducing a property like nodeStorageName
> to
> > > > > specify
> > > > > > > exactly name of a directory to use. For example, it could have
> > > higher
> > > > > > > priority then IgniteConfiguration.getConsistentId(). It will
> > > impact the
> > > > > > > mentioned algorithm, but it seems to be easier to understand by
> > > users.
> > > > > > >
> > > > > > > If nobody else minds, I will try to implement this idea.
> > > > > > >
> > > > > > > Sincerely,
> > > > > > > Dmitriy Pavlov
> > > > > > >
> > > > > > >
> > > > > > > сб, 2 мар. 2019 г. в 08:33, Павлухин Иван <[hidden email]
> >:
> > > > > > >
> > > > > > > > Dmitriy,
> > > > > > > >
> > > > > > > > You wrote
> > > > > > > > > we need an order to scan and lock random-UUID based
> folders.
> > > > > > > >
> > > > > > > > It would be great if you provide a discussion about that
> order to
> > > > > > > > complete the picture. Currently I cannot understand why the
> > > order is
> > > > > > > > important.
> > > > > > > >
> > > > > > > > Also, couple of raw thoughts:
> > > > > > > > 1. Can we extend a directory lookup procedure when
> consistent id
> > > is
> > > > > > > > specified to check nodeXX-consistentId directories as well?
> > > > > > > > 2. Can we introduce a property like nodeStorageName to
> specify
> > > exact
> > > > > > > > name of a directory to use? It looks like a straightforward
> and
> > > > > > > > universal workaround. Or is node index better in some sense?
> Why?
> > > > > > > >
> > > > > > > > Please, share your thoughts.
> > > > > > > >
> > > > > > > > чт, 28 февр. 2019 г. в 17:43, Dmitriy Pavlov <
> [hidden email]
> > > >:
> > > > > > > > >
> > > > > > > > > Hi Ivan,
> > > > > > > > >
> > > > > > > > > Yes, you catch me, I'm a little bit cheating with lazy
> > > consensus on
> > > > > > > code
> > > > > > > > > modification without providing a PR because I was expecting
> > > that
> > > > > nobody
> > > > > > > > > comes to discussion. I will prepare PR shortly. And since
> we
> > > anyway
> > > > > > > have
> > > > > > > > a
> > > > > > > > > discussion, I will not apply anything by lazy approval.
> > > > > > > > >
> > > > > > > > > - storageNodeIndex without consistent ID will not work.
> > > > > > > > >  cfg.getDataStorageConfiguration().setNodeIdx() will be
> > > required
> > > > > only
> > > > > > > for
> > > > > > > > > case we have consistent ID.
> > > > > > > > >
> > > > > > > > > Hi Stanislav,
> > > > > > > > >
> > > > > > > > > We can't use only consistent ID because
> > > > > > > > >
> > > > > > > > > 1) we need an order to scan and lock random-UUID based
> folders.
> > > > > Node
> > > > > > > > index
> > > > > > > > > provides the order of scan. I can find the corresponding
> > > > > discussion,
> > > > > > > but
> > > > > > > > I
> > > > > > > > > guess it is not needed.
> > > > > > > > > 2) we need to separate backward compatible folders from new
> > > > > random-UUID
> > > > > > > > > based folders. Using UUID as folder will not allow us to
> scan
> > > only
> > > > > new
> > > > > > > > name
> > > > > > > > > format folders.
> > > > > > > > >
> > > > > > > > > I guess specifying node index is a quite rare case and good
> > > JavaDoc
> > > > > > > will
> > > > > > > > > always help.
> > > > > > > > >
> > > > > > > > > DataStorageConfiguration().setNodeIdx()  JavaDoc may
> include
> > > > > following
> > > > > > > > > notes:
> > > > > > > > > Node index used for persistent store folders in case
> several
> > > nodes
> > > > > > > reuse
> > > > > > > > > one persistent store root folder.
> > > > > > > > >
> > > > > > > > > Sincerely,
> > > > > > > > > Dmitriy Pavlov
> > > > > > > > >
> > > > > > > > > чт, 28 февр. 2019 г. в 08:03, Павлухин Иван <
> > > [hidden email]>:
> > > > > > > > >
> > > > > > > > > > Dmitiy,
> > > > > > > > > >
> > > > > > > > > > Could please clarify one thing:
> > > > > > > > > > 1. Will it be enough to use only storageNodeIndex in
> order to
> > > > > reuse
> > > > > > > > > > the same persistence folders when consistentId is
> > > auto-generated?
> > > > > > > E.g.
> > > > > > > > > > I have a configuration with storageNodeIndex=1 and
> without
> > > > > explicitly
> > > > > > > > > > specified consistentId, will the node after restart use
> the
> > > same
> > > > > > > > > > persistence folder as before restart?
> > > > > > > > > >
> > > > > > > > > > Also a side note:
> > > > > > > > > > > Please share your vision. I'm going to apply this
> change by
> > > > > lazy
> > > > > > > > > > consensus
> > > > > > > > > > in 3 days.
> > > > > > > > > > What do you mean by "apply"? I have not seen any PR yet.
> > > > > > > > > >
> > > > > > > > > > ср, 27 февр. 2019 г. в 21:12, Dmitriy Pavlov <
> > > [hidden email]
> > > > > >:
> > > > > > > > > > >
> > > > > > > > > > > Hi Igniters,
> > > > > > > > > > >
> > > > > > > > > > > I would like to fix the issue
> > > > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-11432
> about
> > > > > > > specifying
> > > > > > > > some
> > > > > > > > > > > previous randomly generated UUID as a new consistent
> ID.
> > > Folder
> > > > > > > > > > generation
> > > > > > > > > > > algorithm here (
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > >
> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood
> > > > > > > > > > )
> > > > > > > > > > > allows two options
> > > > > > > > > > > -node00+random UUID
> > > > > > > > > > > - consistendId
> > > > > > > > > > >
> > > > > > > > > > > I would like to add to Ignite configuration new
> property
> > > > > nodeIndex
> > > > > > > in
> > > > > > > > > > > addition to consistent Id. New Property will be named
> as
> > > > > > > > > > storageNodeIndex,
> > > > > > > > > > > int, zero-based.
> > > > > > > > > > > This will add the third option of subfolders
> processing:
> > > > > > > > > > > node{storageNodeIndex}+consistentID
> > > > > > > > > > >
> > > > > > > > > > > Please share your vision. I'm going to apply this
> change by
> > > > > lazy
> > > > > > > > > > consensus
> > > > > > > > > > > in 3 days.
> > > > > > > > > > >
> > > > > > > > > > > Sincerely,
> > > > > > > > > > > Dmitriy Pavlov
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > Best regards,
> > > > > > > > > > Ivan Pavlukhin
> > > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Best regards,
> > > > > > > > Ivan Pavlukhin
> > > > > > > >
> > > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Ivan Pavlukhin
> > > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > > Ivan Pavlukhin
> > >
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>

[Ivan Pavlukhin]

Hi Dmitriy,

Quick start for a new user is a perfect concern. I thought about it as
well. I and would like to outline that I am completely ok with an
approach in your patch.

пн, 18 мар. 2019 г. в 16:16, Dmitriy Pavlov <[hidden email]>:

>
> Hi Ivan,
>
> reasonable points, but we had a long and hot discussion about this logic.
>
> Reason for selecting auto-generation was users that just try to use Apache
> Ignite with native persistence for the first time. They must be able to
> test features without any extensive setup, but before go-live we expect
> them to study the product a little bit more. There are not so many reasons
> to use the same folder (or even same device) for 2+ nodes.
>
> Sincerely,
> Dmitriy Pavlov
>
> вс, 17 мар. 2019 г. в 13:34, Павлухин Иван <[hidden email]>:
>
> > Hi Dmitriy,
> >
> > It is a pitty that we have to guess here. Some comments:
> > 1. The usage scenario is not very complex and erything might work well
> > with aforementioned patch. Unfortunately some tradeoff and extra
> > transformations are involved and consequently it is hard to predict
> > how many users will be troubled. But on a bright side the change is
> > seemless and perhaps nobody will be really troubled.
> > 2. From "honest" code perspective introducing nodeStorageName seems
> > better to me because no (surprising) transformations are involved.
> > 3. Also, it seems to me that original logic for choosing storage was
> > not designed well. From the first glance it looks strange that need
> > folders for a particular node in different places. Is not it much
> > clear to have all data of the node in a single directory isolated from
> > other nodes? AFAIK Hazelcast follows this approach. They generate an
> > error when someone is trying to launch second instance in the same
> > working directory. Perhaps, we at least should think about changing
> > our layout and consider including such change in some future release.
> >
> > Thank you!
> >
> > пт, 15 мар. 2019 г. в 17:55, Dmitriy Pavlov <[hidden email]>:
> > >
> > > Hi Ivan, yes, after the restart we need to re-use both folder name and
> > UUID
> > > as consistent ID.
> > >
> > > About returning exact directory name: I'm just guessing here, but the
> > > persistent store is not the only folder, there are several more folders,
> > > e.g. binary-meta, etc.
> > >
> > > About several nodes reusing same root folder, yes, the proposed solution
> > > will not works because 00 is hardcoded here. But I expect it will be too
> > > rare case.
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > пт, 15 мар. 2019 г. в 11:29, Павлухин Иван <[hidden email]>:
> > >
> > > > Hi Dmitriy,
> > > >
> > > > It looks like that case with only one node in working directory is
> > > > covered. But what is problem is solved? As I understand before fix in
> > > > such case node after restart uses proper folder and proper
> > > > consistentId. After fix we can specify automatically generated UUID in
> > > > configuration and everything will go smooth. Is it the main problem
> > > > case? If yes then such simplicity is very attractive.
> > > >
> > > > But honestly I do not fully understand why do we need to return a node
> > > > to node00-UUID directory after that directory was deleted?
> > > >
> > > > чт, 14 мар. 2019 г. в 17:21, Dmitriy Pavlov <[hidden email]>:
> > > > >
> > > > > Hi, I've prepared demo PR for approach 1, just checking UUID and
> > trying
> > > > to
> > > > > use node00-UUID. https://github.com/apache/ignite/pull/6266/
> > Actually
> > > > for
> > > > > this approach it is really simple, no API change is required.
> > > > >
> > > > > I guess the case is not very often because automatic folders
> > assignment
> > > > > works well in most cases.
> > > > >
> > > > > WDYT?
> > > > >
> > > > > ср, 13 мар. 2019 г. в 09:42, Павлухин Иван <[hidden email]>:
> > > > >
> > > > > > Alex,
> > > > > >
> > > > > > Why do we need
> > > > > > >  - check if consistent ID is set to a string 'nodeXX-UUID'. In
> > this
> > > > case
> > > > > > > the consistent ID is set to UUID, and the storage folder is
> > chosen
> > > > > > > according to the proper rules. This change has a minimal chance
> > to
> > > > affect
> > > > > > > current users because it's unlikely that somebody is using
> > > > auto-generated
> > > > > > > folder naming scheme as consistent ID.
> > > > > > ?
> > > > > > It looks hacky as well. The thing I do not like here is that a
> > > > > > consistentId specified in configuration is not a consistentId used
> > by
> > > > > > a node sometimes.
> > > > > >
> > > > > > Can we go just with
> > > > > > > - either check if consistent ID is an instance of UUID and then
> > take
> > > > the
> > > > > > > appropriate folder. This approach is straightforward, but may
> > affect
> > > > > > > current users
> > > > > > ?
> > > > > >
> > > > > > And as a last chance a user will have a possibility to rename a
> > > > directory.
> > > > > >
> > > > > >
> > > > > > вт, 12 мар. 2019 г. в 19:33, Alexey Goncharuk <
> > > > [hidden email]
> > > > > > >:
> > > > > > >
> > > > > > > Igniters,
> > > > > > >
> > > > > > > I came across the same issue during development and found no sane
> > > > > > > workaround for this issue. I believe the solution should be as
> > > > simple as
> > > > > > > possible because we are already adding a warning to let users
> > know
> > > > that
> > > > > > it
> > > > > > > is good to specify a consistent ID in production deployments.
> > > > > > >
> > > > > > > As for the current solution, I do not like adding a new
> > configuration
> > > > > > > property like 'storageFolder' because it's another way to shoot
> > > > yourself
> > > > > > in
> > > > > > > the leg (e.g. different nodes have different consistent IDs but
> > > > > > configured
> > > > > > > to have the same storage folder).
> > > > > > > Why can't we:
> > > > > > >  - either check if consistent ID is an instance of UUID and then
> > > > take the
> > > > > > > appropriate folder. This approach is straightforward, but may
> > affect
> > > > > > > current users
> > > > > > >  - check if consistent ID is set to a string 'nodeXX-UUID'. In
> > this
> > > > case
> > > > > > > the consistent ID is set to UUID, and the storage folder is
> > chosen
> > > > > > > according to the proper rules. This change has a minimal chance
> > to
> > > > affect
> > > > > > > current users because it's unlikely that somebody is using
> > > > auto-generated
> > > > > > > folder naming scheme as consistent ID.
> > > > > > >
> > > > > > > Thoughts?
> > > > > > >
> > > > > > > вт, 12 мар. 2019 г. в 16:51, Dmitriy Pavlov <[hidden email]
> > >:
> > > > > > >
> > > > > > > > Hi Igniters,
> > > > > > > >
> > > > > > > > A full description can be found at wiki page
> > > > > > > >
> > > > > > > >
> > > > > >
> > > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood#IgnitePersistentStore-underthehood-SubfoldersGeneration
> > > > > > > >
> > > > > > > > I like the idea of introducing a property like nodeStorageName
> > to
> > > > > > specify
> > > > > > > > exactly name of a directory to use. For example, it could have
> > > > higher
> > > > > > > > priority then IgniteConfiguration.getConsistentId(). It will
> > > > impact the
> > > > > > > > mentioned algorithm, but it seems to be easier to understand by
> > > > users.
> > > > > > > >
> > > > > > > > If nobody else minds, I will try to implement this idea.
> > > > > > > >
> > > > > > > > Sincerely,
> > > > > > > > Dmitriy Pavlov
> > > > > > > >
> > > > > > > >
> > > > > > > > сб, 2 мар. 2019 г. в 08:33, Павлухин Иван <[hidden email]
> > >:
> > > > > > > >
> > > > > > > > > Dmitriy,
> > > > > > > > >
> > > > > > > > > You wrote
> > > > > > > > > > we need an order to scan and lock random-UUID based
> > folders.
> > > > > > > > >
> > > > > > > > > It would be great if you provide a discussion about that
> > order to
> > > > > > > > > complete the picture. Currently I cannot understand why the
> > > > order is
> > > > > > > > > important.
> > > > > > > > >
> > > > > > > > > Also, couple of raw thoughts:
> > > > > > > > > 1. Can we extend a directory lookup procedure when
> > consistent id
> > > > is
> > > > > > > > > specified to check nodeXX-consistentId directories as well?
> > > > > > > > > 2. Can we introduce a property like nodeStorageName to
> > specify
> > > > exact
> > > > > > > > > name of a directory to use? It looks like a straightforward
> > and
> > > > > > > > > universal workaround. Or is node index better in some sense?
> > Why?
> > > > > > > > >
> > > > > > > > > Please, share your thoughts.
> > > > > > > > >
> > > > > > > > > чт, 28 февр. 2019 г. в 17:43, Dmitriy Pavlov <
> > [hidden email]
> > > > >:
> > > > > > > > > >
> > > > > > > > > > Hi Ivan,
> > > > > > > > > >
> > > > > > > > > > Yes, you catch me, I'm a little bit cheating with lazy
> > > > consensus on
> > > > > > > > code
> > > > > > > > > > modification without providing a PR because I was expecting
> > > > that
> > > > > > nobody
> > > > > > > > > > comes to discussion. I will prepare PR shortly. And since
> > we
> > > > anyway
> > > > > > > > have
> > > > > > > > > a
> > > > > > > > > > discussion, I will not apply anything by lazy approval.
> > > > > > > > > >
> > > > > > > > > > - storageNodeIndex without consistent ID will not work.
> > > > > > > > > >  cfg.getDataStorageConfiguration().setNodeIdx() will be
> > > > required
> > > > > > only
> > > > > > > > for
> > > > > > > > > > case we have consistent ID.
> > > > > > > > > >
> > > > > > > > > > Hi Stanislav,
> > > > > > > > > >
> > > > > > > > > > We can't use only consistent ID because
> > > > > > > > > >
> > > > > > > > > > 1) we need an order to scan and lock random-UUID based
> > folders.
> > > > > > Node
> > > > > > > > > index
> > > > > > > > > > provides the order of scan. I can find the corresponding
> > > > > > discussion,
> > > > > > > > but
> > > > > > > > > I
> > > > > > > > > > guess it is not needed.
> > > > > > > > > > 2) we need to separate backward compatible folders from new
> > > > > > random-UUID
> > > > > > > > > > based folders. Using UUID as folder will not allow us to
> > scan
> > > > only
> > > > > > new
> > > > > > > > > name
> > > > > > > > > > format folders.
> > > > > > > > > >
> > > > > > > > > > I guess specifying node index is a quite rare case and good
> > > > JavaDoc
> > > > > > > > will
> > > > > > > > > > always help.
> > > > > > > > > >
> > > > > > > > > > DataStorageConfiguration().setNodeIdx()  JavaDoc may
> > include
> > > > > > following
> > > > > > > > > > notes:
> > > > > > > > > > Node index used for persistent store folders in case
> > several
> > > > nodes
> > > > > > > > reuse
> > > > > > > > > > one persistent store root folder.
> > > > > > > > > >
> > > > > > > > > > Sincerely,
> > > > > > > > > > Dmitriy Pavlov
> > > > > > > > > >
> > > > > > > > > > чт, 28 февр. 2019 г. в 08:03, Павлухин Иван <
> > > > [hidden email]>:
> > > > > > > > > >
> > > > > > > > > > > Dmitiy,
> > > > > > > > > > >
> > > > > > > > > > > Could please clarify one thing:
> > > > > > > > > > > 1. Will it be enough to use only storageNodeIndex in
> > order to
> > > > > > reuse
> > > > > > > > > > > the same persistence folders when consistentId is
> > > > auto-generated?
> > > > > > > > E.g.
> > > > > > > > > > > I have a configuration with storageNodeIndex=1 and
> > without
> > > > > > explicitly
> > > > > > > > > > > specified consistentId, will the node after restart use
> > the
> > > > same
> > > > > > > > > > > persistence folder as before restart?
> > > > > > > > > > >
> > > > > > > > > > > Also a side note:
> > > > > > > > > > > > Please share your vision. I'm going to apply this
> > change by
> > > > > > lazy
> > > > > > > > > > > consensus
> > > > > > > > > > > in 3 days.
> > > > > > > > > > > What do you mean by "apply"? I have not seen any PR yet.
> > > > > > > > > > >
> > > > > > > > > > > ср, 27 февр. 2019 г. в 21:12, Dmitriy Pavlov <
> > > > [hidden email]
> > > > > > >:
> > > > > > > > > > > >
> > > > > > > > > > > > Hi Igniters,
> > > > > > > > > > > >
> > > > > > > > > > > > I would like to fix the issue
> > > > > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-11432
> > about
> > > > > > > > specifying
> > > > > > > > > some
> > > > > > > > > > > > previous randomly generated UUID as a new consistent
> > ID.
> > > > Folder
> > > > > > > > > > > generation
> > > > > > > > > > > > algorithm here (
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood
> > > > > > > > > > > )
> > > > > > > > > > > > allows two options
> > > > > > > > > > > > -node00+random UUID
> > > > > > > > > > > > - consistendId
> > > > > > > > > > > >
> > > > > > > > > > > > I would like to add to Ignite configuration new
> > property
> > > > > > nodeIndex
> > > > > > > > in
> > > > > > > > > > > > addition to consistent Id. New Property will be named
> > as
> > > > > > > > > > > storageNodeIndex,
> > > > > > > > > > > > int, zero-based.
> > > > > > > > > > > > This will add the third option of subfolders
> > processing:
> > > > > > > > > > > > node{storageNodeIndex}+consistentID
> > > > > > > > > > > >
> > > > > > > > > > > > Please share your vision. I'm going to apply this
> > change by
> > > > > > lazy
> > > > > > > > > > > consensus
> > > > > > > > > > > > in 3 days.
> > > > > > > > > > > >
> > > > > > > > > > > > Sincerely,
> > > > > > > > > > > > Dmitriy Pavlov
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > --
> > > > > > > > > > > Best regards,
> > > > > > > > > > > Ivan Pavlukhin
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Best regards,
> > > > > > > > > Ivan Pavlukhin
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Best regards,
> > > > > > Ivan Pavlukhin
> > > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Ivan Pavlukhin
> > > >
> >
> >
> >
> > --
> > Best regards,
> > Ivan Pavlukhin
> >

--
Best regards,
Ivan Pavlukhin