Disable WAL for several cache groups within one exchange

[Ivan Rakov]

Igniters,

API method for disabling WAL in IgniteCluster accepts only one cache
name. Every call triggers exchange and checkpoints cluster-wide - it
takes plenty of time to disable/enable WAL for multiple caches.
I think, we should add option to disable/enable WAL for several caches
with single command.

Thoughts?

--
Best Regards,
Ivan Rakov

[Dmitriy Govorukhin]

Ivan,

Agree, if we have the batch method for cache create, we should have the
ability to enable/disable WAL in the batch too.

On Fri, May 11, 2018 at 5:17 PM, Ivan Rakov <[hidden email]> wrote:

> Igniters,
>
> API method for disabling WAL in IgniteCluster accepts only one cache name.
> Every call triggers exchange and checkpoints cluster-wide - it takes plenty
> of time to disable/enable WAL for multiple caches.
> I think, we should add option to disable/enable WAL for several caches
> with single command.
>
> Thoughts?
>
> --
> Best Regards,
> Ivan Rakov
>
>

[Vladimir Ozerov]

Ivan,

WAL disable for multiple caches was not implemented in the first place for
a reason. We striven to maintain API consistency,
"IgniteCluster.disableWal(cacheName)" maps exactly to "ALTER TABLE
tableName NOLOGGING" command.  There is no SQL equivalent for multiple
caches. WAL disable for multiple tables is typically handled through
tablespace which is an arbitrary logical group of objects sharing the same
physical files. We have only cache group notion which is a set of tables
with the same backup and affinity properties sharing the same files. As I
hope we would replace groups with tablespaces my preference is not to
expose any SQL commands for cache groups. Neither I would expose cache
groups to public API unless absolutely needed.

I would suggest to extend native API with a command accepting multiple
cache names, not cache groups:

IgniteCluster.disableWal(Collection<String> cacheNames)
IgniteCluster.enableWal(Collection<String> cacheNames)

Makes sense?

Vladimir.

On Fri, May 11, 2018 at 5:25 PM, Dmitriy Govorukhin <
[hidden email]> wrote:

> Ivan,
>
> Agree, if we have the batch method for cache create, we should have the
> ability to enable/disable WAL in the batch too.
>
> On Fri, May 11, 2018 at 5:17 PM, Ivan Rakov <[hidden email]> wrote:
>
> > Igniters,
> >
> > API method for disabling WAL in IgniteCluster accepts only one cache
> name.
> > Every call triggers exchange and checkpoints cluster-wide - it takes
> plenty
> > of time to disable/enable WAL for multiple caches.
> > I think, we should add option to disable/enable WAL for several caches
> > with single command.
> >
> > Thoughts?
> >
> > --
> > Best Regards,
> > Ivan Rakov
> >
> >
>

[Andrew Mashenkov]

Ivan,

Huge +1 for this improvement.

I think we can have 2 overloaded method enableWal() with no args to enable
WAL for all caches
and enableWAL(String... caches) for one or multiple caches. (and same for
disable wal)



On Fri, May 11, 2018 at 5:25 PM, Dmitriy Govorukhin <
[hidden email]> wrote:

> Ivan,
>
> Agree, if we have the batch method for cache create, we should have the
> ability to enable/disable WAL in the batch too.
>
> On Fri, May 11, 2018 at 5:17 PM, Ivan Rakov <[hidden email]> wrote:
>
> > Igniters,
> >
> > API method for disabling WAL in IgniteCluster accepts only one cache
> name.
> > Every call triggers exchange and checkpoints cluster-wide - it takes
> plenty
> > of time to disable/enable WAL for multiple caches.
> > I think, we should add option to disable/enable WAL for several caches
> > with single command.
> >
> > Thoughts?
> >
> > --
> > Best Regards,
> > Ivan Rakov
> >
> >
>

--
Best regards,
Andrey V. Mashenkov

[Ivan Rakov]

Vladimir,

I fully agree. We don't expose groups to public API (except for cache
configuration), so we shouldn't start doing it here.

Ignite issue for this improvement:
https://issues.apache.org/jira/browse/IGNITE-8473

Best Regards,
Ivan Rakov

On 11.05.2018 17:49, Vladimir Ozerov wrote:

> Ivan,
>
> WAL disable for multiple caches was not implemented in the first place for
> a reason. We striven to maintain API consistency,
> "IgniteCluster.disableWal(cacheName)" maps exactly to "ALTER TABLE
> tableName NOLOGGING" command.  There is no SQL equivalent for multiple
> caches. WAL disable for multiple tables is typically handled through
> tablespace which is an arbitrary logical group of objects sharing the same
> physical files. We have only cache group notion which is a set of tables
> with the same backup and affinity properties sharing the same files. As I
> hope we would replace groups with tablespaces my preference is not to
> expose any SQL commands for cache groups. Neither I would expose cache
> groups to public API unless absolutely needed.
>
> I would suggest to extend native API with a command accepting multiple
> cache names, not cache groups:
>
> IgniteCluster.disableWal(Collection<String> cacheNames)
> IgniteCluster.enableWal(Collection<String> cacheNames)
>
> Makes sense?
>
> Vladimir.
>
> On Fri, May 11, 2018 at 5:25 PM, Dmitriy Govorukhin <
> [hidden email]> wrote:
>
>> Ivan,
>>
>> Agree, if we have the batch method for cache create, we should have the
>> ability to enable/disable WAL in the batch too.
>>
>> On Fri, May 11, 2018 at 5:17 PM, Ivan Rakov <[hidden email]> wrote:
>>
>>> Igniters,
>>>
>>> API method for disabling WAL in IgniteCluster accepts only one cache
>> name.
>>> Every call triggers exchange and checkpoints cluster-wide - it takes
>> plenty
>>> of time to disable/enable WAL for multiple caches.
>>> I think, we should add option to disable/enable WAL for several caches
>>> with single command.
>>>
>>> Thoughts?
>>>
>>> --
>>> Best Regards,
>>> Ivan Rakov
>>>
>>>

[Ivan Rakov]

It would be six methods in total (3 for enabling, 3 for disabling).
What about accepting null in *enableWAL(String... caches)* as wildcard?

Best Regards,
Ivan Rakov

On 11.05.2018 17:52, Andrey Mashenkov wrote:

> Ivan,
>
> Huge +1 for this improvement.
>
> I think we can have 2 overloaded method enableWal() with no args to enable
> WAL for all caches
> and enableWAL(String... caches) for one or multiple caches. (and same for
> disable wal)
>
>
>
> On Fri, May 11, 2018 at 5:25 PM, Dmitriy Govorukhin <
> [hidden email]> wrote:
>
>> Ivan,
>>
>> Agree, if we have the batch method for cache create, we should have the
>> ability to enable/disable WAL in the batch too.
>>
>> On Fri, May 11, 2018 at 5:17 PM, Ivan Rakov <[hidden email]> wrote:
>>
>>> Igniters,
>>>
>>> API method for disabling WAL in IgniteCluster accepts only one cache
>> name.
>>> Every call triggers exchange and checkpoints cluster-wide - it takes
>> plenty
>>> of time to disable/enable WAL for multiple caches.
>>> I think, we should add option to disable/enable WAL for several caches
>>> with single command.
>>>
>>> Thoughts?
>>>
>>> --
>>> Best Regards,
>>> Ivan Rakov
>>>
>>>
>
>

[Vladimir Ozerov]

Ivan,

This proven to be too hard to understand.  It is better to have a lot small
methods with clear and compact semantics. Also arrays are harder to manage
than collections, users typically prefer the latest.
Also we need to think on what would be a result of this operation. Current
methods with a single cache return true/false based on whether they changed
something or not. Should we continue returning a single boolean for batch
operations as well?

On Fri, May 11, 2018 at 6:13 PM, Ivan Rakov <[hidden email]> wrote:

> It would be six methods in total (3 for enabling, 3 for disabling).
> What about accepting null in *enableWAL(String... caches)* as wildcard?
>
> Best Regards,
> Ivan Rakov
>
>
> On 11.05.2018 17:52, Andrey Mashenkov wrote:
>
>> Ivan,
>>
>> Huge +1 for this improvement.
>>
>> I think we can have 2 overloaded method enableWal() with no args to enable
>> WAL for all caches
>> and enableWAL(String... caches) for one or multiple caches. (and same for
>> disable wal)
>>
>>
>>
>> On Fri, May 11, 2018 at 5:25 PM, Dmitriy Govorukhin <
>> [hidden email]> wrote:
>>
>> Ivan,
>>>
>>> Agree, if we have the batch method for cache create, we should have the
>>> ability to enable/disable WAL in the batch too.
>>>
>>> On Fri, May 11, 2018 at 5:17 PM, Ivan Rakov <[hidden email]>
>>> wrote:
>>>
>>> Igniters,
>>>>
>>>> API method for disabling WAL in IgniteCluster accepts only one cache
>>>>
>>> name.
>>>
>>>> Every call triggers exchange and checkpoints cluster-wide - it takes
>>>>
>>> plenty
>>>
>>>> of time to disable/enable WAL for multiple caches.
>>>> I think, we should add option to disable/enable WAL for several caches
>>>> with single command.
>>>>
>>>> Thoughts?
>>>>
>>>> --
>>>> Best Regards,
>>>> Ivan Rakov
>>>>
>>>>
>>>>
>>
>>
>

[Ivan Rakov]

Agree about collections.

Regarding return type: it's a tricky question. Maybe author of this
feature may help.
Anton V., in which case enableWal/disableWal can return false?

Best Regards,
Ivan Rakov

On 11.05.2018 18:19, Vladimir Ozerov wrote:

> Ivan,
>
> This proven to be too hard to understand.  It is better to have a lot small
> methods with clear and compact semantics. Also arrays are harder to manage
> than collections, users typically prefer the latest.
> Also we need to think on what would be a result of this operation. Current
> methods with a single cache return true/false based on whether they changed
> something or not. Should we continue returning a single boolean for batch
> operations as well?
>
> On Fri, May 11, 2018 at 6:13 PM, Ivan Rakov <[hidden email]> wrote:
>
>> It would be six methods in total (3 for enabling, 3 for disabling).
>> What about accepting null in *enableWAL(String... caches)* as wildcard?
>>
>> Best Regards,
>> Ivan Rakov
>>
>>
>> On 11.05.2018 17:52, Andrey Mashenkov wrote:
>>
>>> Ivan,
>>>
>>> Huge +1 for this improvement.
>>>
>>> I think we can have 2 overloaded method enableWal() with no args to enable
>>> WAL for all caches
>>> and enableWAL(String... caches) for one or multiple caches. (and same for
>>> disable wal)
>>>
>>>
>>>
>>> On Fri, May 11, 2018 at 5:25 PM, Dmitriy Govorukhin <
>>> [hidden email]> wrote:
>>>
>>> Ivan,
>>>> Agree, if we have the batch method for cache create, we should have the
>>>> ability to enable/disable WAL in the batch too.
>>>>
>>>> On Fri, May 11, 2018 at 5:17 PM, Ivan Rakov <[hidden email]>
>>>> wrote:
>>>>
>>>> Igniters,
>>>>> API method for disabling WAL in IgniteCluster accepts only one cache
>>>>>
>>>> name.
>>>>
>>>>> Every call triggers exchange and checkpoints cluster-wide - it takes
>>>>>
>>>> plenty
>>>>
>>>>> of time to disable/enable WAL for multiple caches.
>>>>> I think, we should add option to disable/enable WAL for several caches
>>>>> with single command.
>>>>>
>>>>> Thoughts?
>>>>>
>>>>> --
>>>>> Best Regards,
>>>>> Ivan Rakov
>>>>>
>>>>>
>>>>>
>>>

[Anton Vinogradov-2]

Ivan,

enableWal/disableWal will return false in case enabling/disabling was
caused not by this call.
For example it will return false in case wal already enabled/disabled.

Example:

boolean res1 = srv.cluster().enableWal(CACHE_NAME);
boolean res2 = srv.cluster().enableWal(CACHE_NAME);

assert res1;
assert !res2;


Vova,

Since you made final tuning for this solution, is there any special cases
when false can be returned?

пт, 11 мая 2018 г. в 18:39, Ivan Rakov <[hidden email]>:

> Agree about collections.
>
> Regarding return type: it's a tricky question. Maybe author of this
> feature may help.
> Anton V., in which case enableWal/disableWal can return false?
>
> Best Regards,
> Ivan Rakov
>
> On 11.05.2018 18:19, Vladimir Ozerov wrote:
> > Ivan,
> >
> > This proven to be too hard to understand.  It is better to have a lot
> small
> > methods with clear and compact semantics. Also arrays are harder to
> manage
> > than collections, users typically prefer the latest.
> > Also we need to think on what would be a result of this operation.
> Current
> > methods with a single cache return true/false based on whether they
> changed
> > something or not. Should we continue returning a single boolean for batch
> > operations as well?
> >
> > On Fri, May 11, 2018 at 6:13 PM, Ivan Rakov <[hidden email]>
> wrote:
> >
> >> It would be six methods in total (3 for enabling, 3 for disabling).
> >> What about accepting null in *enableWAL(String... caches)* as wildcard?
> >>
> >> Best Regards,
> >> Ivan Rakov
> >>
> >>
> >> On 11.05.2018 17:52, Andrey Mashenkov wrote:
> >>
> >>> Ivan,
> >>>
> >>> Huge +1 for this improvement.
> >>>
> >>> I think we can have 2 overloaded method enableWal() with no args to
> enable
> >>> WAL for all caches
> >>> and enableWAL(String... caches) for one or multiple caches. (and same
> for
> >>> disable wal)
> >>>
> >>>
> >>>
> >>> On Fri, May 11, 2018 at 5:25 PM, Dmitriy Govorukhin <
> >>> [hidden email]> wrote:
> >>>
> >>> Ivan,
> >>>> Agree, if we have the batch method for cache create, we should have
> the
> >>>> ability to enable/disable WAL in the batch too.
> >>>>
> >>>> On Fri, May 11, 2018 at 5:17 PM, Ivan Rakov <[hidden email]>
> >>>> wrote:
> >>>>
> >>>> Igniters,
> >>>>> API method for disabling WAL in IgniteCluster accepts only one cache
> >>>>>
> >>>> name.
> >>>>
> >>>>> Every call triggers exchange and checkpoints cluster-wide - it takes
> >>>>>
> >>>> plenty
> >>>>
> >>>>> of time to disable/enable WAL for multiple caches.
> >>>>> I think, we should add option to disable/enable WAL for several
> caches
> >>>>> with single command.
> >>>>>
> >>>>> Thoughts?
> >>>>>
> >>>>> --
> >>>>> Best Regards,
> >>>>> Ivan Rakov
> >>>>>
> >>>>>
> >>>>>
> >>>
>
>

[Ivan Rakov]

I got your point. Since WAL state changes are linearized with exchange
worker queue, I think we can keep boolean return type even for new
multiple caches API.
Returned "true" will mean that WAL state of *at least one cache* was
actually changed by this call. Returned "false" will mean that requested
WAL state is exact match for existing WAL state (for the corresponding
topology version).
Method will keep its current semantics "If the same WAL state change is
requested more than once, only one call will return true".

Best Regards,
Ivan Rakov

On 14.05.2018 19:12, Anton Vinogradov wrote:

> Ivan,
>
> enableWal/disableWal will return false in case enabling/disabling was
> caused not by this call.
> For example it will return false in case wal already enabled/disabled.
>
> Example:
>
> boolean res1 = srv.cluster().enableWal(CACHE_NAME);
> boolean res2 = srv.cluster().enableWal(CACHE_NAME);
>
> assert res1;
> assert !res2;
>
>
> Vova,
>
> Since you made final tuning for this solution, is there any special cases
> when false can be returned?
>
> пт, 11 мая 2018 г. в 18:39, Ivan Rakov <[hidden email]>:
>
>> Agree about collections.
>>
>> Regarding return type: it's a tricky question. Maybe author of this
>> feature may help.
>> Anton V., in which case enableWal/disableWal can return false?
>>
>> Best Regards,
>> Ivan Rakov
>>
>> On 11.05.2018 18:19, Vladimir Ozerov wrote:
>>> Ivan,
>>>
>>> This proven to be too hard to understand.  It is better to have a lot
>> small
>>> methods with clear and compact semantics. Also arrays are harder to
>> manage
>>> than collections, users typically prefer the latest.
>>> Also we need to think on what would be a result of this operation.
>> Current
>>> methods with a single cache return true/false based on whether they
>> changed
>>> something or not. Should we continue returning a single boolean for batch
>>> operations as well?
>>>
>>> On Fri, May 11, 2018 at 6:13 PM, Ivan Rakov <[hidden email]>
>> wrote:
>>>> It would be six methods in total (3 for enabling, 3 for disabling).
>>>> What about accepting null in *enableWAL(String... caches)* as wildcard?
>>>>
>>>> Best Regards,
>>>> Ivan Rakov
>>>>
>>>>
>>>> On 11.05.2018 17:52, Andrey Mashenkov wrote:
>>>>
>>>>> Ivan,
>>>>>
>>>>> Huge +1 for this improvement.
>>>>>
>>>>> I think we can have 2 overloaded method enableWal() with no args to
>> enable
>>>>> WAL for all caches
>>>>> and enableWAL(String... caches) for one or multiple caches. (and same
>> for
>>>>> disable wal)
>>>>>
>>>>>
>>>>>
>>>>> On Fri, May 11, 2018 at 5:25 PM, Dmitriy Govorukhin <
>>>>> [hidden email]> wrote:
>>>>>
>>>>> Ivan,
>>>>>> Agree, if we have the batch method for cache create, we should have
>> the
>>>>>> ability to enable/disable WAL in the batch too.
>>>>>>
>>>>>> On Fri, May 11, 2018 at 5:17 PM, Ivan Rakov <[hidden email]>
>>>>>> wrote:
>>>>>>
>>>>>> Igniters,
>>>>>>> API method for disabling WAL in IgniteCluster accepts only one cache
>>>>>>>
>>>>>> name.
>>>>>>
>>>>>>> Every call triggers exchange and checkpoints cluster-wide - it takes
>>>>>>>
>>>>>> plenty
>>>>>>
>>>>>>> of time to disable/enable WAL for multiple caches.
>>>>>>> I think, we should add option to disable/enable WAL for several
>> caches
>>>>>>> with single command.
>>>>>>>
>>>>>>> Thoughts?
>>>>>>>
>>>>>>> --
>>>>>>> Best Regards,
>>>>>>> Ivan Rakov
>>>>>>>
>>>>>>>
>>>>>>>
>>