Persistence per memory policy configuration

[dsetrakyan]

On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <[hidden email]> wrote:

> From what I see after running an example they are under the same root
> folder and in different subdirectories. The root folder should be defined
> by setPersistencePath as I guess.
>

If that is the case, then you are right. Then we should not have
storagePath or WalPath, and store them both under "persistencePath" root.
However, I would need Alexey Goncharuk or Ivan Rakov to confirm this.

[Ivan Rakov]

Denis,

Data/index storage and WAL are located under the same root by default.
However, this is not mandatory: *storagePath* and *walPath* properties
can contain both absolute and relative paths. If paths are absolute,
storage and WAL can reside on different devices, like this:

> storagePath: /storage1/NMVe_drive/storage
> walPath: /storage2/Big_SSD_drive/wal
We even recommend this in tuning guide:
https://apacheignite.readme.io/docs/durable-memory-tuning

That's why I think *persistencePath* is misleading.

Best Regards,
Ivan Rakov

On 13.10.2017 5:03, Dmitriy Setrakyan wrote:

> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <[hidden email]> wrote:
>
>>  From what I see after running an example they are under the same root
>> folder and in different subdirectories. The root folder should be defined
>> by setPersistencePath as I guess.
>>
> If that is the case, then you are right. Then we should not have
> storagePath or WalPath, and store them both under "persistencePath" root.
> However, I would need Alexey Goncharuk or Ivan Rakov to confirm this.
>

[dmagda]

Seems I've got what you’re talking about.

I’ve tried to change the root directory (*persistencePath*) and saw that only data/indexes were placed to it while wal stayed somewhere in my work dir. It works counterintuitive and causes non productive discussions like we are in arguing about *persistencePath* or *storagePath*. Neither name fits this behavior.

My suggestion will be the following:
- *persistencePath* refers to the path of all storage files (data/indexes, wal, archive). If the path is changed *all the files* will be under the new directory unless *setWalPath* and *setWalArchivePath* are set *explicitly*.
- *setWalPath* overrides the default location of WAL (which is setPersistencePath)
- *setWalArchivePath* overrides the default location of the archive (which is again has to be setPersistencePath).

If we follow this approach the configuration and behavior becomes vivid. Thoughts?

—
Denis

> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <[hidden email]> wrote:
>
> Denis,
>
> Data/index storage and WAL are located under the same root by default.
> However, this is not mandatory: *storagePath* and *walPath* properties can contain both absolute and relative paths. If paths are absolute, storage and WAL can reside on different devices, like this:
>
>> storagePath: /storage1/NMVe_drive/storage
>> walPath: /storage2/Big_SSD_drive/wal
> We even recommend this in tuning guide: https://apacheignite.readme.io/docs/durable-memory-tuning
>
> That's why I think *persistencePath* is misleading.
>
> Best Regards,
> Ivan Rakov
>
> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <[hidden email]> wrote:
>>
>>> From what I see after running an example they are under the same root
>>> folder and in different subdirectories. The root folder should be defined
>>> by setPersistencePath as I guess.
>>>
>> If that is the case, then you are right. Then we should not have
>> storagePath or WalPath, and store them both under "persistencePath" root.
>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm this.
>>
>

[Pavel Tupitsyn]

Igniters, another thing to consider:

DataRegionConfiguration.SwapFilePath should be SwapPath,
since this is actually not a single file, but a directory path.

On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <[hidden email]> wrote:

> Seems I've got what you’re talking about.
>
> I’ve tried to change the root directory (*persistencePath*) and saw that
> only data/indexes were placed to it while wal stayed somewhere in my work
> dir. It works counterintuitive and causes non productive discussions like
> we are in arguing about *persistencePath* or *storagePath*. Neither name
> fits this behavior.
>
> My suggestion will be the following:
> - *persistencePath* refers to the path of all storage files (data/indexes,
> wal, archive). If the path is changed *all the files* will be under the new
> directory unless *setWalPath* and *setWalArchivePath* are set *explicitly*.
> - *setWalPath* overrides the default location of WAL (which is
> setPersistencePath)
> - *setWalArchivePath* overrides the default location of the archive (which
> is again has to be setPersistencePath).
>
> If we follow this approach the configuration and behavior becomes vivid.
> Thoughts?
>
> —
> Denis
>
> > On Oct 13, 2017, at 1:21 AM, Ivan Rakov <[hidden email]> wrote:
> >
> > Denis,
> >
> > Data/index storage and WAL are located under the same root by default.
> > However, this is not mandatory: *storagePath* and *walPath* properties
> can contain both absolute and relative paths. If paths are absolute,
> storage and WAL can reside on different devices, like this:
> >
> >> storagePath: /storage1/NMVe_drive/storage
> >> walPath: /storage2/Big_SSD_drive/wal
> > We even recommend this in tuning guide: https://apacheignite.readme.
> io/docs/durable-memory-tuning
> >
> > That's why I think *persistencePath* is misleading.
> >
> > Best Regards,
> > Ivan Rakov
> >
> > On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
> >> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <[hidden email]>
> wrote:
> >>
> >>> From what I see after running an example they are under the same root
> >>> folder and in different subdirectories. The root folder should be
> defined
> >>> by setPersistencePath as I guess.
> >>>
> >> If that is the case, then you are right. Then we should not have
> >> storagePath or WalPath, and store them both under "persistencePath"
> root.
> >> However, I would need Alexey Goncharuk or Ivan Rakov to confirm this.
> >>
> >
>
>

[Ivan Rakov]

*swapPath* is ok for me. It is also consistent with *walPath* and
*walArchivePath*.

Regarding persistencePath/storagePath, I don't like the idea when path
for WAL is implicitly changed, especially when we have separate option
for it.
WAL and storage files are already located under same $IGNITE_HOME root.
 From user perspective, there's no need to change root for all
persistence-related directories as long as $IGNITE_HOME points to the
correct disk.
 From developer perspective, this change breaks backwards compatibility.
Maintaining backwards compatibility in fail-safe way (checking both
old-style and new-style paths) is complex and hard to maintain in the
codebase.

Best Regards,
Ivan Rakov

My vote is for *storagePath* and keeping behavior as is.

On 16.10.2017 16:53, Pavel Tupitsyn wrote:

> Igniters, another thing to consider:
>
> DataRegionConfiguration.SwapFilePath should be SwapPath,
> since this is actually not a single file, but a directory path.
>
> On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <[hidden email]> wrote:
>
>> Seems I've got what you’re talking about.
>>
>> I’ve tried to change the root directory (*persistencePath*) and saw that
>> only data/indexes were placed to it while wal stayed somewhere in my work
>> dir. It works counterintuitive and causes non productive discussions like
>> we are in arguing about *persistencePath* or *storagePath*. Neither name
>> fits this behavior.
>>
>> My suggestion will be the following:
>> - *persistencePath* refers to the path of all storage files (data/indexes,
>> wal, archive). If the path is changed *all the files* will be under the new
>> directory unless *setWalPath* and *setWalArchivePath* are set *explicitly*.
>> - *setWalPath* overrides the default location of WAL (which is
>> setPersistencePath)
>> - *setWalArchivePath* overrides the default location of the archive (which
>> is again has to be setPersistencePath).
>>
>> If we follow this approach the configuration and behavior becomes vivid.
>> Thoughts?
>>
>> —
>> Denis
>>
>>> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <[hidden email]> wrote:
>>>
>>> Denis,
>>>
>>> Data/index storage and WAL are located under the same root by default.
>>> However, this is not mandatory: *storagePath* and *walPath* properties
>> can contain both absolute and relative paths. If paths are absolute,
>> storage and WAL can reside on different devices, like this:
>>>> storagePath: /storage1/NMVe_drive/storage
>>>> walPath: /storage2/Big_SSD_drive/wal
>>> We even recommend this in tuning guide: https://apacheignite.readme.
>> io/docs/durable-memory-tuning
>>> That's why I think *persistencePath* is misleading.
>>>
>>> Best Regards,
>>> Ivan Rakov
>>>
>>> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
>>>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <[hidden email]>
>> wrote:
>>>>>  From what I see after running an example they are under the same root
>>>>> folder and in different subdirectories. The root folder should be
>> defined
>>>>> by setPersistencePath as I guess.
>>>>>
>>>> If that is the case, then you are right. Then we should not have
>>>> storagePath or WalPath, and store them both under "persistencePath"
>> root.
>>>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm this.
>>>>
>>

[Vladimir Ozerov]

+1 to Ivan's suggestion.

пн, 16 окт. 2017 г. в 21:00, Ivan Rakov <[hidden email]>:

> *swapPath* is ok for me. It is also consistent with *walPath* and
> *walArchivePath*.
>
> Regarding persistencePath/storagePath, I don't like the idea when path
> for WAL is implicitly changed, especially when we have separate option
> for it.
> WAL and storage files are already located under same $IGNITE_HOME root.
>  From user perspective, there's no need to change root for all
> persistence-related directories as long as $IGNITE_HOME points to the
> correct disk.
>  From developer perspective, this change breaks backwards compatibility.
> Maintaining backwards compatibility in fail-safe way (checking both
> old-style and new-style paths) is complex and hard to maintain in the
> codebase.
>
> Best Regards,
> Ivan Rakov
>
> My vote is for *storagePath* and keeping behavior as is.
>
> On 16.10.2017 16:53, Pavel Tupitsyn wrote:
> > Igniters, another thing to consider:
> >
> > DataRegionConfiguration.SwapFilePath should be SwapPath,
> > since this is actually not a single file, but a directory path.
> >
> > On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <[hidden email]> wrote:
> >
> >> Seems I've got what you’re talking about.
> >>
> >> I’ve tried to change the root directory (*persistencePath*) and saw that
> >> only data/indexes were placed to it while wal stayed somewhere in my
> work
> >> dir. It works counterintuitive and causes non productive discussions
> like
> >> we are in arguing about *persistencePath* or *storagePath*. Neither name
> >> fits this behavior.
> >>
> >> My suggestion will be the following:
> >> - *persistencePath* refers to the path of all storage files
> (data/indexes,
> >> wal, archive). If the path is changed *all the files* will be under the
> new
> >> directory unless *setWalPath* and *setWalArchivePath* are set
> *explicitly*.
> >> - *setWalPath* overrides the default location of WAL (which is
> >> setPersistencePath)
> >> - *setWalArchivePath* overrides the default location of the archive
> (which
> >> is again has to be setPersistencePath).
> >>
> >> If we follow this approach the configuration and behavior becomes vivid.
> >> Thoughts?
> >>
> >> —
> >> Denis
> >>
> >>> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <[hidden email]> wrote:
> >>>
> >>> Denis,
> >>>
> >>> Data/index storage and WAL are located under the same root by default.
> >>> However, this is not mandatory: *storagePath* and *walPath* properties
> >> can contain both absolute and relative paths. If paths are absolute,
> >> storage and WAL can reside on different devices, like this:
> >>>> storagePath: /storage1/NMVe_drive/storage
> >>>> walPath: /storage2/Big_SSD_drive/wal
> >>> We even recommend this in tuning guide: https://apacheignite.readme.
> >> io/docs/durable-memory-tuning
> >>> That's why I think *persistencePath* is misleading.
> >>>
> >>> Best Regards,
> >>> Ivan Rakov
> >>>
> >>> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
> >>>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <[hidden email]>
> >> wrote:
> >>>>>  From what I see after running an example they are under the same
> root
> >>>>> folder and in different subdirectories. The root folder should be
> >> defined
> >>>>> by setPersistencePath as I guess.
> >>>>>
> >>>> If that is the case, then you are right. Then we should not have
> >>>> storagePath or WalPath, and store them both under "persistencePath"
> >> root.
> >>>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm this.
> >>>>
> >>
>
>

[dsetrakyan]

Igniters, we should not have Swap in the property names. Swap is something
that behaves completely differently from Ignite persistence, and I don't
think it should be present within Ignite altogether.

D.

On Mon, Oct 16, 2017 at 11:17 AM, Vladimir Ozerov <[hidden email]>
wrote:

> +1 to Ivan's suggestion.
>
> пн, 16 окт. 2017 г. в 21:00, Ivan Rakov <[hidden email]>:
>
> > *swapPath* is ok for me. It is also consistent with *walPath* and
> > *walArchivePath*.
> >
> > Regarding persistencePath/storagePath, I don't like the idea when path
> > for WAL is implicitly changed, especially when we have separate option
> > for it.
> > WAL and storage files are already located under same $IGNITE_HOME root.
> >  From user perspective, there's no need to change root for all
> > persistence-related directories as long as $IGNITE_HOME points to the
> > correct disk.
> >  From developer perspective, this change breaks backwards compatibility.
> > Maintaining backwards compatibility in fail-safe way (checking both
> > old-style and new-style paths) is complex and hard to maintain in the
> > codebase.
> >
> > Best Regards,
> > Ivan Rakov
> >
> > My vote is for *storagePath* and keeping behavior as is.
> >
> > On 16.10.2017 16:53, Pavel Tupitsyn wrote:
> > > Igniters, another thing to consider:
> > >
> > > DataRegionConfiguration.SwapFilePath should be SwapPath,
> > > since this is actually not a single file, but a directory path.
> > >
> > > On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <[hidden email]>
> wrote:
> > >
> > >> Seems I've got what you’re talking about.
> > >>
> > >> I’ve tried to change the root directory (*persistencePath*) and saw
> that
> > >> only data/indexes were placed to it while wal stayed somewhere in my
> > work
> > >> dir. It works counterintuitive and causes non productive discussions
> > like
> > >> we are in arguing about *persistencePath* or *storagePath*. Neither
> name
> > >> fits this behavior.
> > >>
> > >> My suggestion will be the following:
> > >> - *persistencePath* refers to the path of all storage files
> > (data/indexes,
> > >> wal, archive). If the path is changed *all the files* will be under
> the
> > new
> > >> directory unless *setWalPath* and *setWalArchivePath* are set
> > *explicitly*.
> > >> - *setWalPath* overrides the default location of WAL (which is
> > >> setPersistencePath)
> > >> - *setWalArchivePath* overrides the default location of the archive
> > (which
> > >> is again has to be setPersistencePath).
> > >>
> > >> If we follow this approach the configuration and behavior becomes
> vivid.
> > >> Thoughts?
> > >>
> > >> —
> > >> Denis
> > >>
> > >>> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <[hidden email]>
> wrote:
> > >>>
> > >>> Denis,
> > >>>
> > >>> Data/index storage and WAL are located under the same root by
> default.
> > >>> However, this is not mandatory: *storagePath* and *walPath*
> properties
> > >> can contain both absolute and relative paths. If paths are absolute,
> > >> storage and WAL can reside on different devices, like this:
> > >>>> storagePath: /storage1/NMVe_drive/storage
> > >>>> walPath: /storage2/Big_SSD_drive/wal
> > >>> We even recommend this in tuning guide: https://apacheignite.readme.
> > >> io/docs/durable-memory-tuning
> > >>> That's why I think *persistencePath* is misleading.
> > >>>
> > >>> Best Regards,
> > >>> Ivan Rakov
> > >>>
> > >>> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
> > >>>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <[hidden email]>
> > >> wrote:
> > >>>>>  From what I see after running an example they are under the same
> > root
> > >>>>> folder and in different subdirectories. The root folder should be
> > >> defined
> > >>>>> by setPersistencePath as I guess.
> > >>>>>
> > >>>> If that is the case, then you are right. Then we should not have
> > >>>> storagePath or WalPath, and store them both under "persistencePath"
> > >> root.
> > >>>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm
> this.
> > >>>>
> > >>
> >
> >
>

[Alexey Goncharuk]

Agree with Ivan. If we implemented backward compatibility, this would be
completely counterintuitive behavior, so +1 to keep the behavior as is.

As for the swap path, I see nothing wrong with having it for in-memory
caches. This is a simple overflow mechanism that works fine if you do not
need persistence guarantees.

2017-10-16 21:00 GMT+03:00 Ivan Rakov <[hidden email]>:

> *swapPath* is ok for me. It is also consistent with *walPath* and
> *walArchivePath*.
>
> Regarding persistencePath/storagePath, I don't like the idea when path for
> WAL is implicitly changed, especially when we have separate option for it.
> WAL and storage files are already located under same $IGNITE_HOME root.
> From user perspective, there's no need to change root for all
> persistence-related directories as long as $IGNITE_HOME points to the
> correct disk.
> From developer perspective, this change breaks backwards compatibility.
> Maintaining backwards compatibility in fail-safe way (checking both
> old-style and new-style paths) is complex and hard to maintain in the
> codebase.
>
> Best Regards,
> Ivan Rakov
>
> My vote is for *storagePath* and keeping behavior as is.
>
>
> On 16.10.2017 16:53, Pavel Tupitsyn wrote:
>
>> Igniters, another thing to consider:
>>
>> DataRegionConfiguration.SwapFilePath should be SwapPath,
>> since this is actually not a single file, but a directory path.
>>
>> On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <[hidden email]> wrote:
>>
>> Seems I've got what you’re talking about.
>>>
>>> I’ve tried to change the root directory (*persistencePath*) and saw that
>>> only data/indexes were placed to it while wal stayed somewhere in my work
>>> dir. It works counterintuitive and causes non productive discussions like
>>> we are in arguing about *persistencePath* or *storagePath*. Neither name
>>> fits this behavior.
>>>
>>> My suggestion will be the following:
>>> - *persistencePath* refers to the path of all storage files
>>> (data/indexes,
>>> wal, archive). If the path is changed *all the files* will be under the
>>> new
>>> directory unless *setWalPath* and *setWalArchivePath* are set
>>> *explicitly*.
>>> - *setWalPath* overrides the default location of WAL (which is
>>> setPersistencePath)
>>> - *setWalArchivePath* overrides the default location of the archive
>>> (which
>>> is again has to be setPersistencePath).
>>>
>>> If we follow this approach the configuration and behavior becomes vivid.
>>> Thoughts?
>>>
>>> —
>>> Denis
>>>
>>> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <[hidden email]> wrote:
>>>>
>>>> Denis,
>>>>
>>>> Data/index storage and WAL are located under the same root by default.
>>>> However, this is not mandatory: *storagePath* and *walPath* properties
>>>>
>>> can contain both absolute and relative paths. If paths are absolute,
>>> storage and WAL can reside on different devices, like this:
>>>
>>>> storagePath: /storage1/NMVe_drive/storage
>>>>> walPath: /storage2/Big_SSD_drive/wal
>>>>>
>>>> We even recommend this in tuning guide: https://apacheignite.readme.
>>>>
>>> io/docs/durable-memory-tuning
>>>
>>>> That's why I think *persistencePath* is misleading.
>>>>
>>>> Best Regards,
>>>> Ivan Rakov
>>>>
>>>> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
>>>>
>>>>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <[hidden email]>
>>>>>
>>>> wrote:
>>>
>>>>  From what I see after running an example they are under the same root
>>>>>> folder and in different subdirectories. The root folder should be
>>>>>>
>>>>> defined
>>>
>>>> by setPersistencePath as I guess.
>>>>>>
>>>>>> If that is the case, then you are right. Then we should not have
>>>>> storagePath or WalPath, and store them both under "persistencePath"
>>>>>
>>>> root.
>>>
>>>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm this.
>>>>>
>>>>>
>>>
>

[dsetrakyan]

I am now confused. Can I please ask for the final configuration again? What
will it look like?

On Tue, Oct 17, 2017 at 1:16 AM, Alexey Goncharuk <
[hidden email]> wrote:

> Agree with Ivan. If we implemented backward compatibility, this would be
> completely counterintuitive behavior, so +1 to keep the behavior as is.
>
> As for the swap path, I see nothing wrong with having it for in-memory
> caches. This is a simple overflow mechanism that works fine if you do not
> need persistence guarantees.
>
> 2017-10-16 21:00 GMT+03:00 Ivan Rakov <[hidden email]>:
>
> > *swapPath* is ok for me. It is also consistent with *walPath* and
> > *walArchivePath*.
> >
> > Regarding persistencePath/storagePath, I don't like the idea when path
> for
> > WAL is implicitly changed, especially when we have separate option for
> it.
> > WAL and storage files are already located under same $IGNITE_HOME root.
> > From user perspective, there's no need to change root for all
> > persistence-related directories as long as $IGNITE_HOME points to the
> > correct disk.
> > From developer perspective, this change breaks backwards compatibility.
> > Maintaining backwards compatibility in fail-safe way (checking both
> > old-style and new-style paths) is complex and hard to maintain in the
> > codebase.
> >
> > Best Regards,
> > Ivan Rakov
> >
> > My vote is for *storagePath* and keeping behavior as is.
> >
> >
> > On 16.10.2017 16:53, Pavel Tupitsyn wrote:
> >
> >> Igniters, another thing to consider:
> >>
> >> DataRegionConfiguration.SwapFilePath should be SwapPath,
> >> since this is actually not a single file, but a directory path.
> >>
> >> On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <[hidden email]> wrote:
> >>
> >> Seems I've got what you’re talking about.
> >>>
> >>> I’ve tried to change the root directory (*persistencePath*) and saw
> that
> >>> only data/indexes were placed to it while wal stayed somewhere in my
> work
> >>> dir. It works counterintuitive and causes non productive discussions
> like
> >>> we are in arguing about *persistencePath* or *storagePath*. Neither
> name
> >>> fits this behavior.
> >>>
> >>> My suggestion will be the following:
> >>> - *persistencePath* refers to the path of all storage files
> >>> (data/indexes,
> >>> wal, archive). If the path is changed *all the files* will be under the
> >>> new
> >>> directory unless *setWalPath* and *setWalArchivePath* are set
> >>> *explicitly*.
> >>> - *setWalPath* overrides the default location of WAL (which is
> >>> setPersistencePath)
> >>> - *setWalArchivePath* overrides the default location of the archive
> >>> (which
> >>> is again has to be setPersistencePath).
> >>>
> >>> If we follow this approach the configuration and behavior becomes
> vivid.
> >>> Thoughts?
> >>>
> >>> —
> >>> Denis
> >>>
> >>> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <[hidden email]> wrote:
> >>>>
> >>>> Denis,
> >>>>
> >>>> Data/index storage and WAL are located under the same root by default.
> >>>> However, this is not mandatory: *storagePath* and *walPath* properties
> >>>>
> >>> can contain both absolute and relative paths. If paths are absolute,
> >>> storage and WAL can reside on different devices, like this:
> >>>
> >>>> storagePath: /storage1/NMVe_drive/storage
> >>>>> walPath: /storage2/Big_SSD_drive/wal
> >>>>>
> >>>> We even recommend this in tuning guide: https://apacheignite.readme.
> >>>>
> >>> io/docs/durable-memory-tuning
> >>>
> >>>> That's why I think *persistencePath* is misleading.
> >>>>
> >>>> Best Regards,
> >>>> Ivan Rakov
> >>>>
> >>>> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
> >>>>
> >>>>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <[hidden email]>
> >>>>>
> >>>> wrote:
> >>>
> >>>>  From what I see after running an example they are under the same root
> >>>>>> folder and in different subdirectories. The root folder should be
> >>>>>>
> >>>>> defined
> >>>
> >>>> by setPersistencePath as I guess.
> >>>>>>
> >>>>>> If that is the case, then you are right. Then we should not have
> >>>>> storagePath or WalPath, and store them both under "persistencePath"
> >>>>>
> >>>> root.
> >>>
> >>>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm this.
> >>>>>
> >>>>>
> >>>
> >
>

[Ivan Rakov]

Dmitriy,

Please check description of
https://issues.apache.org/jira/browse/IGNITE-6030, I've updated it with
actual list of properties.

Best Regards,
Ivan Rakov

On 17.10.2017 21:46, Dmitriy Setrakyan wrote:

> I am now confused. Can I please ask for the final configuration again? What
> will it look like?
>
> On Tue, Oct 17, 2017 at 1:16 AM, Alexey Goncharuk <
> [hidden email]> wrote:
>
>> Agree with Ivan. If we implemented backward compatibility, this would be
>> completely counterintuitive behavior, so +1 to keep the behavior as is.
>>
>> As for the swap path, I see nothing wrong with having it for in-memory
>> caches. This is a simple overflow mechanism that works fine if you do not
>> need persistence guarantees.
>>
>> 2017-10-16 21:00 GMT+03:00 Ivan Rakov <[hidden email]>:
>>
>>> *swapPath* is ok for me. It is also consistent with *walPath* and
>>> *walArchivePath*.
>>>
>>> Regarding persistencePath/storagePath, I don't like the idea when path
>> for
>>> WAL is implicitly changed, especially when we have separate option for
>> it.
>>> WAL and storage files are already located under same $IGNITE_HOME root.
>>>  From user perspective, there's no need to change root for all
>>> persistence-related directories as long as $IGNITE_HOME points to the
>>> correct disk.
>>>  From developer perspective, this change breaks backwards compatibility.
>>> Maintaining backwards compatibility in fail-safe way (checking both
>>> old-style and new-style paths) is complex and hard to maintain in the
>>> codebase.
>>>
>>> Best Regards,
>>> Ivan Rakov
>>>
>>> My vote is for *storagePath* and keeping behavior as is.
>>>
>>>
>>> On 16.10.2017 16:53, Pavel Tupitsyn wrote:
>>>
>>>> Igniters, another thing to consider:
>>>>
>>>> DataRegionConfiguration.SwapFilePath should be SwapPath,
>>>> since this is actually not a single file, but a directory path.
>>>>
>>>> On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <[hidden email]> wrote:
>>>>
>>>> Seems I've got what you’re talking about.
>>>>> I’ve tried to change the root directory (*persistencePath*) and saw
>> that
>>>>> only data/indexes were placed to it while wal stayed somewhere in my
>> work
>>>>> dir. It works counterintuitive and causes non productive discussions
>> like
>>>>> we are in arguing about *persistencePath* or *storagePath*. Neither
>> name
>>>>> fits this behavior.
>>>>>
>>>>> My suggestion will be the following:
>>>>> - *persistencePath* refers to the path of all storage files
>>>>> (data/indexes,
>>>>> wal, archive). If the path is changed *all the files* will be under the
>>>>> new
>>>>> directory unless *setWalPath* and *setWalArchivePath* are set
>>>>> *explicitly*.
>>>>> - *setWalPath* overrides the default location of WAL (which is
>>>>> setPersistencePath)
>>>>> - *setWalArchivePath* overrides the default location of the archive
>>>>> (which
>>>>> is again has to be setPersistencePath).
>>>>>
>>>>> If we follow this approach the configuration and behavior becomes
>> vivid.
>>>>> Thoughts?
>>>>>
>>>>> —
>>>>> Denis
>>>>>
>>>>> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <[hidden email]> wrote:
>>>>>> Denis,
>>>>>>
>>>>>> Data/index storage and WAL are located under the same root by default.
>>>>>> However, this is not mandatory: *storagePath* and *walPath* properties
>>>>>>
>>>>> can contain both absolute and relative paths. If paths are absolute,
>>>>> storage and WAL can reside on different devices, like this:
>>>>>
>>>>>> storagePath: /storage1/NMVe_drive/storage
>>>>>>> walPath: /storage2/Big_SSD_drive/wal
>>>>>>>
>>>>>> We even recommend this in tuning guide: https://apacheignite.readme.
>>>>>>
>>>>> io/docs/durable-memory-tuning
>>>>>
>>>>>> That's why I think *persistencePath* is misleading.
>>>>>>
>>>>>> Best Regards,
>>>>>> Ivan Rakov
>>>>>>
>>>>>> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
>>>>>>
>>>>>>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <[hidden email]>
>>>>>>>
>>>>>> wrote:
>>>>>>   From what I see after running an example they are under the same root
>>>>>>>> folder and in different subdirectories. The root folder should be
>>>>>>>>
>>>>>>> defined
>>>>>> by setPersistencePath as I guess.
>>>>>>>> If that is the case, then you are right. Then we should not have
>>>>>>> storagePath or WalPath, and store them both under "persistencePath"
>>>>>>>
>>>>>> root.
>>>>>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm this.
>>>>>>>

[dsetrakyan]

Thanks Ivan! Let's make sure that every property gets sufficient javadoc
for our users to understand. We should also document this configuration on
readme.

On Tue, Oct 17, 2017 at 3:06 PM, Ivan Rakov <[hidden email]> wrote:

> Dmitriy,
>
> Please check description of https://issues.apache.org/jira
> /browse/IGNITE-6030, I've updated it with actual list of properties.
>
> Best Regards,
> Ivan Rakov
>
>
> On 17.10.2017 21:46, Dmitriy Setrakyan wrote:
>
>> I am now confused. Can I please ask for the final configuration again?
>> What
>> will it look like?
>>
>> On Tue, Oct 17, 2017 at 1:16 AM, Alexey Goncharuk <
>> [hidden email]> wrote:
>>
>> Agree with Ivan. If we implemented backward compatibility, this would be
>>> completely counterintuitive behavior, so +1 to keep the behavior as is.
>>>
>>> As for the swap path, I see nothing wrong with having it for in-memory
>>> caches. This is a simple overflow mechanism that works fine if you do not
>>> need persistence guarantees.
>>>
>>> 2017-10-16 21:00 GMT+03:00 Ivan Rakov <[hidden email]>:
>>>
>>> *swapPath* is ok for me. It is also consistent with *walPath* and
>>>> *walArchivePath*.
>>>>
>>>> Regarding persistencePath/storagePath, I don't like the idea when path
>>>>
>>> for
>>>
>>>> WAL is implicitly changed, especially when we have separate option for
>>>>
>>> it.
>>>
>>>> WAL and storage files are already located under same $IGNITE_HOME root.
>>>>  From user perspective, there's no need to change root for all
>>>> persistence-related directories as long as $IGNITE_HOME points to the
>>>> correct disk.
>>>>  From developer perspective, this change breaks backwards compatibility.
>>>> Maintaining backwards compatibility in fail-safe way (checking both
>>>> old-style and new-style paths) is complex and hard to maintain in the
>>>> codebase.
>>>>
>>>> Best Regards,
>>>> Ivan Rakov
>>>>
>>>> My vote is for *storagePath* and keeping behavior as is.
>>>>
>>>>
>>>> On 16.10.2017 16:53, Pavel Tupitsyn wrote:
>>>>
>>>> Igniters, another thing to consider:
>>>>>
>>>>> DataRegionConfiguration.SwapFilePath should be SwapPath,
>>>>> since this is actually not a single file, but a directory path.
>>>>>
>>>>> On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <[hidden email]>
>>>>> wrote:
>>>>>
>>>>> Seems I've got what you’re talking about.
>>>>>
>>>>>> I’ve tried to change the root directory (*persistencePath*) and saw
>>>>>>
>>>>> that
>>>
>>>> only data/indexes were placed to it while wal stayed somewhere in my
>>>>>>
>>>>> work
>>>
>>>> dir. It works counterintuitive and causes non productive discussions
>>>>>>
>>>>> like
>>>
>>>> we are in arguing about *persistencePath* or *storagePath*. Neither
>>>>>>
>>>>> name
>>>
>>>> fits this behavior.
>>>>>>
>>>>>> My suggestion will be the following:
>>>>>> - *persistencePath* refers to the path of all storage files
>>>>>> (data/indexes,
>>>>>> wal, archive). If the path is changed *all the files* will be under
>>>>>> the
>>>>>> new
>>>>>> directory unless *setWalPath* and *setWalArchivePath* are set
>>>>>> *explicitly*.
>>>>>> - *setWalPath* overrides the default location of WAL (which is
>>>>>> setPersistencePath)
>>>>>> - *setWalArchivePath* overrides the default location of the archive
>>>>>> (which
>>>>>> is again has to be setPersistencePath).
>>>>>>
>>>>>> If we follow this approach the configuration and behavior becomes
>>>>>>
>>>>> vivid.
>>>
>>>> Thoughts?
>>>>>>
>>>>>> —
>>>>>> Denis
>>>>>>
>>>>>> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <[hidden email]>
>>>>>> wrote:
>>>>>>
>>>>>>> Denis,
>>>>>>>
>>>>>>> Data/index storage and WAL are located under the same root by
>>>>>>> default.
>>>>>>> However, this is not mandatory: *storagePath* and *walPath*
>>>>>>> properties
>>>>>>>
>>>>>>> can contain both absolute and relative paths. If paths are absolute,
>>>>>> storage and WAL can reside on different devices, like this:
>>>>>>
>>>>>> storagePath: /storage1/NMVe_drive/storage
>>>>>>>
>>>>>>>> walPath: /storage2/Big_SSD_drive/wal
>>>>>>>>
>>>>>>>> We even recommend this in tuning guide: https://apacheignite.readme
>>>>>>> .
>>>>>>>
>>>>>>> io/docs/durable-memory-tuning
>>>>>>
>>>>>> That's why I think *persistencePath* is misleading.
>>>>>>>
>>>>>>> Best Regards,
>>>>>>> Ivan Rakov
>>>>>>>
>>>>>>> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
>>>>>>>
>>>>>>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <[hidden email]>
>>>>>>>>
>>>>>>>> wrote:
>>>>>>>   From what I see after running an example they are under the same
>>>>>>> root
>>>>>>>
>>>>>>>> folder and in different subdirectories. The root folder should be
>>>>>>>>>
>>>>>>>>> defined
>>>>>>>>
>>>>>>> by setPersistencePath as I guess.
>>>>>>>
>>>>>>>> If that is the case, then you are right. Then we should not have
>>>>>>>>>
>>>>>>>> storagePath or WalPath, and store them both under "persistencePath"
>>>>>>>>
>>>>>>>> root.
>>>>>>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm this.
>>>>>>>
>>>>>>>>
>>>>>>>>
>

[Denis Magda]

Ivan,

Please don’t forget to update all the persistence and memory pools related
examples to the new configuration format. Let’s make sure non of our
example prints out the class deprecated warning.

Denis

On Tuesday, October 17, 2017, Dmitriy Setrakyan <[hidden email]>
wrote:

> Thanks Ivan! Let's make sure that every property gets sufficient javadoc
> for our users to understand. We should also document this configuration on
> readme.
>
> On Tue, Oct 17, 2017 at 3:06 PM, Ivan Rakov <[hidden email]
> <javascript:;>> wrote:
>
> > Dmitriy,
> >
> > Please check description of https://issues.apache.org/jira
> > /browse/IGNITE-6030, I've updated it with actual list of properties.
> >
> > Best Regards,
> > Ivan Rakov
> >
> >
> > On 17.10.2017 21:46, Dmitriy Setrakyan wrote:
> >
> >> I am now confused. Can I please ask for the final configuration again?
> >> What
> >> will it look like?
> >>
> >> On Tue, Oct 17, 2017 at 1:16 AM, Alexey Goncharuk <
> >> [hidden email] <javascript:;>> wrote:
> >>
> >> Agree with Ivan. If we implemented backward compatibility, this would be
> >>> completely counterintuitive behavior, so +1 to keep the behavior as is.
> >>>
> >>> As for the swap path, I see nothing wrong with having it for in-memory
> >>> caches. This is a simple overflow mechanism that works fine if you do
> not
> >>> need persistence guarantees.
> >>>
> >>> 2017-10-16 21:00 GMT+03:00 Ivan Rakov <[hidden email]
> <javascript:;>>:
> >>>
> >>> *swapPath* is ok for me. It is also consistent with *walPath* and
> >>>> *walArchivePath*.
> >>>>
> >>>> Regarding persistencePath/storagePath, I don't like the idea when path
> >>>>
> >>> for
> >>>
> >>>> WAL is implicitly changed, especially when we have separate option for
> >>>>
> >>> it.
> >>>
> >>>> WAL and storage files are already located under same $IGNITE_HOME
> root.
> >>>>  From user perspective, there's no need to change root for all
> >>>> persistence-related directories as long as $IGNITE_HOME points to the
> >>>> correct disk.
> >>>>  From developer perspective, this change breaks backwards
> compatibility.
> >>>> Maintaining backwards compatibility in fail-safe way (checking both
> >>>> old-style and new-style paths) is complex and hard to maintain in the
> >>>> codebase.
> >>>>
> >>>> Best Regards,
> >>>> Ivan Rakov
> >>>>
> >>>> My vote is for *storagePath* and keeping behavior as is.
> >>>>
> >>>>
> >>>> On 16.10.2017 16:53, Pavel Tupitsyn wrote:
> >>>>
> >>>> Igniters, another thing to consider:
> >>>>>
> >>>>> DataRegionConfiguration.SwapFilePath should be SwapPath,
> >>>>> since this is actually not a single file, but a directory path.
> >>>>>
> >>>>> On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <[hidden email]
> <javascript:;>>
> >>>>> wrote:
> >>>>>
> >>>>> Seems I've got what you’re talking about.
> >>>>>
> >>>>>> I’ve tried to change the root directory (*persistencePath*) and saw
> >>>>>>
> >>>>> that
> >>>
> >>>> only data/indexes were placed to it while wal stayed somewhere in my
> >>>>>>
> >>>>> work
> >>>
> >>>> dir. It works counterintuitive and causes non productive discussions
> >>>>>>
> >>>>> like
> >>>
> >>>> we are in arguing about *persistencePath* or *storagePath*. Neither
> >>>>>>
> >>>>> name
> >>>
> >>>> fits this behavior.
> >>>>>>
> >>>>>> My suggestion will be the following:
> >>>>>> - *persistencePath* refers to the path of all storage files
> >>>>>> (data/indexes,
> >>>>>> wal, archive). If the path is changed *all the files* will be under
> >>>>>> the
> >>>>>> new
> >>>>>> directory unless *setWalPath* and *setWalArchivePath* are set
> >>>>>> *explicitly*.
> >>>>>> - *setWalPath* overrides the default location of WAL (which is
> >>>>>> setPersistencePath)
> >>>>>> - *setWalArchivePath* overrides the default location of the archive
> >>>>>> (which
> >>>>>> is again has to be setPersistencePath).
> >>>>>>
> >>>>>> If we follow this approach the configuration and behavior becomes
> >>>>>>
> >>>>> vivid.
> >>>
> >>>> Thoughts?
> >>>>>>
> >>>>>> —
> >>>>>> Denis
> >>>>>>
> >>>>>> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <[hidden email]
> <javascript:;>>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Denis,
> >>>>>>>
> >>>>>>> Data/index storage and WAL are located under the same root by
> >>>>>>> default.
> >>>>>>> However, this is not mandatory: *storagePath* and *walPath*
> >>>>>>> properties
> >>>>>>>
> >>>>>>> can contain both absolute and relative paths. If paths are
> absolute,
> >>>>>> storage and WAL can reside on different devices, like this:
> >>>>>>
> >>>>>> storagePath: /storage1/NMVe_drive/storage
> >>>>>>>
> >>>>>>>> walPath: /storage2/Big_SSD_drive/wal
> >>>>>>>>
> >>>>>>>> We even recommend this in tuning guide:
> https://apacheignite.readme
> >>>>>>> .
> >>>>>>>
> >>>>>>> io/docs/durable-memory-tuning
> >>>>>>
> >>>>>> That's why I think *persistencePath* is misleading.
> >>>>>>>
> >>>>>>> Best Regards,
> >>>>>>> Ivan Rakov
> >>>>>>>
> >>>>>>> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
> >>>>>>>
> >>>>>>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <[hidden email]
> <javascript:;>>
> >>>>>>>>
> >>>>>>>> wrote:
> >>>>>>>   From what I see after running an example they are under the same
> >>>>>>> root
> >>>>>>>
> >>>>>>>> folder and in different subdirectories. The root folder should be
> >>>>>>>>>
> >>>>>>>>> defined
> >>>>>>>>
> >>>>>>> by setPersistencePath as I guess.
> >>>>>>>
> >>>>>>>> If that is the case, then you are right. Then we should not have
> >>>>>>>>>
> >>>>>>>> storagePath or WalPath, and store them both under
> "persistencePath"
> >>>>>>>>
> >>>>>>>> root.
> >>>>>>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm
> this.
> >>>>>>>
> >>>>>>>>
> >>>>>>>>
> >
>

[Alexey Goncharuk]

Igniters,

I've found one more shortcoming which we missed during the review. The
checkpoint page buffer is attributed to a data region, so originally it was
a mistake to place the checkpoint page buffer size to the global
configuration. This property should go to the data region configuration.

I will prepare a PR shortly so we can review the change.

2017-10-18 4:52 GMT+03:00 Denis Magda <[hidden email]>:

> Ivan,
>
> Please don’t forget to update all the persistence and memory pools related
> examples to the new configuration format. Let’s make sure non of our
> example prints out the class deprecated warning.
>
> Denis
>
> On Tuesday, October 17, 2017, Dmitriy Setrakyan <[hidden email]>
> wrote:
>
> > Thanks Ivan! Let's make sure that every property gets sufficient javadoc
> > for our users to understand. We should also document this configuration
> on
> > readme.
> >
> > On Tue, Oct 17, 2017 at 3:06 PM, Ivan Rakov <[hidden email]
> > <javascript:;>> wrote:
> >
> > > Dmitriy,
> > >
> > > Please check description of https://issues.apache.org/jira
> > > /browse/IGNITE-6030, I've updated it with actual list of properties.
> > >
> > > Best Regards,
> > > Ivan Rakov
> > >
> > >
> > > On 17.10.2017 21:46, Dmitriy Setrakyan wrote:
> > >
> > >> I am now confused. Can I please ask for the final configuration again?
> > >> What
> > >> will it look like?
> > >>
> > >> On Tue, Oct 17, 2017 at 1:16 AM, Alexey Goncharuk <
> > >> [hidden email] <javascript:;>> wrote:
> > >>
> > >> Agree with Ivan. If we implemented backward compatibility, this would
> be
> > >>> completely counterintuitive behavior, so +1 to keep the behavior as
> is.
> > >>>
> > >>> As for the swap path, I see nothing wrong with having it for
> in-memory
> > >>> caches. This is a simple overflow mechanism that works fine if you do
> > not
> > >>> need persistence guarantees.
> > >>>
> > >>> 2017-10-16 21:00 GMT+03:00 Ivan Rakov <[hidden email]
> > <javascript:;>>:
> > >>>
> > >>> *swapPath* is ok for me. It is also consistent with *walPath* and
> > >>>> *walArchivePath*.
> > >>>>
> > >>>> Regarding persistencePath/storagePath, I don't like the idea when
> path
> > >>>>
> > >>> for
> > >>>
> > >>>> WAL is implicitly changed, especially when we have separate option
> for
> > >>>>
> > >>> it.
> > >>>
> > >>>> WAL and storage files are already located under same $IGNITE_HOME
> > root.
> > >>>>  From user perspective, there's no need to change root for all
> > >>>> persistence-related directories as long as $IGNITE_HOME points to
> the
> > >>>> correct disk.
> > >>>>  From developer perspective, this change breaks backwards
> > compatibility.
> > >>>> Maintaining backwards compatibility in fail-safe way (checking both
> > >>>> old-style and new-style paths) is complex and hard to maintain in
> the
> > >>>> codebase.
> > >>>>
> > >>>> Best Regards,
> > >>>> Ivan Rakov
> > >>>>
> > >>>> My vote is for *storagePath* and keeping behavior as is.
> > >>>>
> > >>>>
> > >>>> On 16.10.2017 16:53, Pavel Tupitsyn wrote:
> > >>>>
> > >>>> Igniters, another thing to consider:
> > >>>>>
> > >>>>> DataRegionConfiguration.SwapFilePath should be SwapPath,
> > >>>>> since this is actually not a single file, but a directory path.
> > >>>>>
> > >>>>> On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <[hidden email]
> > <javascript:;>>
> > >>>>> wrote:
> > >>>>>
> > >>>>> Seems I've got what you’re talking about.
> > >>>>>
> > >>>>>> I’ve tried to change the root directory (*persistencePath*) and
> saw
> > >>>>>>
> > >>>>> that
> > >>>
> > >>>> only data/indexes were placed to it while wal stayed somewhere in my
> > >>>>>>
> > >>>>> work
> > >>>
> > >>>> dir. It works counterintuitive and causes non productive discussions
> > >>>>>>
> > >>>>> like
> > >>>
> > >>>> we are in arguing about *persistencePath* or *storagePath*. Neither
> > >>>>>>
> > >>>>> name
> > >>>
> > >>>> fits this behavior.
> > >>>>>>
> > >>>>>> My suggestion will be the following:
> > >>>>>> - *persistencePath* refers to the path of all storage files
> > >>>>>> (data/indexes,
> > >>>>>> wal, archive). If the path is changed *all the files* will be
> under
> > >>>>>> the
> > >>>>>> new
> > >>>>>> directory unless *setWalPath* and *setWalArchivePath* are set
> > >>>>>> *explicitly*.
> > >>>>>> - *setWalPath* overrides the default location of WAL (which is
> > >>>>>> setPersistencePath)
> > >>>>>> - *setWalArchivePath* overrides the default location of the
> archive
> > >>>>>> (which
> > >>>>>> is again has to be setPersistencePath).
> > >>>>>>
> > >>>>>> If we follow this approach the configuration and behavior becomes
> > >>>>>>
> > >>>>> vivid.
> > >>>
> > >>>> Thoughts?
> > >>>>>>
> > >>>>>> —
> > >>>>>> Denis
> > >>>>>>
> > >>>>>> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <[hidden email]
> > <javascript:;>>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> Denis,
> > >>>>>>>
> > >>>>>>> Data/index storage and WAL are located under the same root by
> > >>>>>>> default.
> > >>>>>>> However, this is not mandatory: *storagePath* and *walPath*
> > >>>>>>> properties
> > >>>>>>>
> > >>>>>>> can contain both absolute and relative paths. If paths are
> > absolute,
> > >>>>>> storage and WAL can reside on different devices, like this:
> > >>>>>>
> > >>>>>> storagePath: /storage1/NMVe_drive/storage
> > >>>>>>>
> > >>>>>>>> walPath: /storage2/Big_SSD_drive/wal
> > >>>>>>>>
> > >>>>>>>> We even recommend this in tuning guide:
> > https://apacheignite.readme
> > >>>>>>> .
> > >>>>>>>
> > >>>>>>> io/docs/durable-memory-tuning
> > >>>>>>
> > >>>>>> That's why I think *persistencePath* is misleading.
> > >>>>>>>
> > >>>>>>> Best Regards,
> > >>>>>>> Ivan Rakov
> > >>>>>>>
> > >>>>>>> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
> > >>>>>>>
> > >>>>>>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <
> [hidden email]
> > <javascript:;>>
> > >>>>>>>>
> > >>>>>>>> wrote:
> > >>>>>>>   From what I see after running an example they are under the
> same
> > >>>>>>> root
> > >>>>>>>
> > >>>>>>>> folder and in different subdirectories. The root folder should
> be
> > >>>>>>>>>
> > >>>>>>>>> defined
> > >>>>>>>>
> > >>>>>>> by setPersistencePath as I guess.
> > >>>>>>>
> > >>>>>>>> If that is the case, then you are right. Then we should not have
> > >>>>>>>>>
> > >>>>>>>> storagePath or WalPath, and store them both under
> > "persistencePath"
> > >>>>>>>>
> > >>>>>>>> root.
> > >>>>>>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm
> > this.
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >
> >
>

[Alexey Goncharuk]

The changes are implemented in ignite-6748 branch. TC run is pending.

2017-10-25 11:26 GMT+03:00 Alexey Goncharuk <[hidden email]>:

> Igniters,
>
> I've found one more shortcoming which we missed during the review. The
> checkpoint page buffer is attributed to a data region, so originally it was
> a mistake to place the checkpoint page buffer size to the global
> configuration. This property should go to the data region configuration.
>
> I will prepare a PR shortly so we can review the change.
>
> 2017-10-18 4:52 GMT+03:00 Denis Magda <[hidden email]>:
>
>> Ivan,
>>
>> Please don’t forget to update all the persistence and memory pools related
>> examples to the new configuration format. Let’s make sure non of our
>> example prints out the class deprecated warning.
>>
>> Denis
>>
>> On Tuesday, October 17, 2017, Dmitriy Setrakyan <[hidden email]>
>> wrote:
>>
>> > Thanks Ivan! Let's make sure that every property gets sufficient javadoc
>> > for our users to understand. We should also document this configuration
>> on
>> > readme.
>> >
>> > On Tue, Oct 17, 2017 at 3:06 PM, Ivan Rakov <[hidden email]
>> > <javascript:;>> wrote:
>> >
>> > > Dmitriy,
>> > >
>> > > Please check description of https://issues.apache.org/jira
>> > > /browse/IGNITE-6030, I've updated it with actual list of properties.
>> > >
>> > > Best Regards,
>> > > Ivan Rakov
>> > >
>> > >
>> > > On 17.10.2017 21:46, Dmitriy Setrakyan wrote:
>> > >
>> > >> I am now confused. Can I please ask for the final configuration
>> again?
>> > >> What
>> > >> will it look like?
>> > >>
>> > >> On Tue, Oct 17, 2017 at 1:16 AM, Alexey Goncharuk <
>> > >> [hidden email] <javascript:;>> wrote:
>> > >>
>> > >> Agree with Ivan. If we implemented backward compatibility, this
>> would be
>> > >>> completely counterintuitive behavior, so +1 to keep the behavior as
>> is.
>> > >>>
>> > >>> As for the swap path, I see nothing wrong with having it for
>> in-memory
>> > >>> caches. This is a simple overflow mechanism that works fine if you
>> do
>> > not
>> > >>> need persistence guarantees.
>> > >>>
>> > >>> 2017-10-16 21:00 GMT+03:00 Ivan Rakov <[hidden email]
>> > <javascript:;>>:
>> > >>>
>> > >>> *swapPath* is ok for me. It is also consistent with *walPath* and
>> > >>>> *walArchivePath*.
>> > >>>>
>> > >>>> Regarding persistencePath/storagePath, I don't like the idea when
>> path
>> > >>>>
>> > >>> for
>> > >>>
>> > >>>> WAL is implicitly changed, especially when we have separate option
>> for
>> > >>>>
>> > >>> it.
>> > >>>
>> > >>>> WAL and storage files are already located under same $IGNITE_HOME
>> > root.
>> > >>>>  From user perspective, there's no need to change root for all
>> > >>>> persistence-related directories as long as $IGNITE_HOME points to
>> the
>> > >>>> correct disk.
>> > >>>>  From developer perspective, this change breaks backwards
>> > compatibility.
>> > >>>> Maintaining backwards compatibility in fail-safe way (checking both
>> > >>>> old-style and new-style paths) is complex and hard to maintain in
>> the
>> > >>>> codebase.
>> > >>>>
>> > >>>> Best Regards,
>> > >>>> Ivan Rakov
>> > >>>>
>> > >>>> My vote is for *storagePath* and keeping behavior as is.
>> > >>>>
>> > >>>>
>> > >>>> On 16.10.2017 16:53, Pavel Tupitsyn wrote:
>> > >>>>
>> > >>>> Igniters, another thing to consider:
>> > >>>>>
>> > >>>>> DataRegionConfiguration.SwapFilePath should be SwapPath,
>> > >>>>> since this is actually not a single file, but a directory path.
>> > >>>>>
>> > >>>>> On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <[hidden email]
>> > <javascript:;>>
>> > >>>>> wrote:
>> > >>>>>
>> > >>>>> Seems I've got what you’re talking about.
>> > >>>>>
>> > >>>>>> I’ve tried to change the root directory (*persistencePath*) and
>> saw
>> > >>>>>>
>> > >>>>> that
>> > >>>
>> > >>>> only data/indexes were placed to it while wal stayed somewhere in
>> my
>> > >>>>>>
>> > >>>>> work
>> > >>>
>> > >>>> dir. It works counterintuitive and causes non productive
>> discussions
>> > >>>>>>
>> > >>>>> like
>> > >>>
>> > >>>> we are in arguing about *persistencePath* or *storagePath*. Neither
>> > >>>>>>
>> > >>>>> name
>> > >>>
>> > >>>> fits this behavior.
>> > >>>>>>
>> > >>>>>> My suggestion will be the following:
>> > >>>>>> - *persistencePath* refers to the path of all storage files
>> > >>>>>> (data/indexes,
>> > >>>>>> wal, archive). If the path is changed *all the files* will be
>> under
>> > >>>>>> the
>> > >>>>>> new
>> > >>>>>> directory unless *setWalPath* and *setWalArchivePath* are set
>> > >>>>>> *explicitly*.
>> > >>>>>> - *setWalPath* overrides the default location of WAL (which is
>> > >>>>>> setPersistencePath)
>> > >>>>>> - *setWalArchivePath* overrides the default location of the
>> archive
>> > >>>>>> (which
>> > >>>>>> is again has to be setPersistencePath).
>> > >>>>>>
>> > >>>>>> If we follow this approach the configuration and behavior becomes
>> > >>>>>>
>> > >>>>> vivid.
>> > >>>
>> > >>>> Thoughts?
>> > >>>>>>
>> > >>>>>> —
>> > >>>>>> Denis
>> > >>>>>>
>> > >>>>>> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <[hidden email]
>> > <javascript:;>>
>> > >>>>>> wrote:
>> > >>>>>>
>> > >>>>>>> Denis,
>> > >>>>>>>
>> > >>>>>>> Data/index storage and WAL are located under the same root by
>> > >>>>>>> default.
>> > >>>>>>> However, this is not mandatory: *storagePath* and *walPath*
>> > >>>>>>> properties
>> > >>>>>>>
>> > >>>>>>> can contain both absolute and relative paths. If paths are
>> > absolute,
>> > >>>>>> storage and WAL can reside on different devices, like this:
>> > >>>>>>
>> > >>>>>> storagePath: /storage1/NMVe_drive/storage
>> > >>>>>>>
>> > >>>>>>>> walPath: /storage2/Big_SSD_drive/wal
>> > >>>>>>>>
>> > >>>>>>>> We even recommend this in tuning guide:
>> > https://apacheignite.readme
>> > >>>>>>> .
>> > >>>>>>>
>> > >>>>>>> io/docs/durable-memory-tuning
>> > >>>>>>
>> > >>>>>> That's why I think *persistencePath* is misleading.
>> > >>>>>>>
>> > >>>>>>> Best Regards,
>> > >>>>>>> Ivan Rakov
>> > >>>>>>>
>> > >>>>>>> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
>> > >>>>>>>
>> > >>>>>>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <
>> [hidden email]
>> > <javascript:;>>
>> > >>>>>>>>
>> > >>>>>>>> wrote:
>> > >>>>>>>   From what I see after running an example they are under the
>> same
>> > >>>>>>> root
>> > >>>>>>>
>> > >>>>>>>> folder and in different subdirectories. The root folder should
>> be
>> > >>>>>>>>>
>> > >>>>>>>>> defined
>> > >>>>>>>>
>> > >>>>>>> by setPersistencePath as I guess.
>> > >>>>>>>
>> > >>>>>>>> If that is the case, then you are right. Then we should not
>> have
>> > >>>>>>>>>
>> > >>>>>>>> storagePath or WalPath, and store them both under
>> > "persistencePath"
>> > >>>>>>>>
>> > >>>>>>>> root.
>> > >>>>>>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm
>> > this.
>> > >>>>>>>
>> > >>>>>>>>
>> > >>>>>>>>
>> > >
>> >
>>
>
>