Should we enable write throttling by default?

[dmagda]

Igniters,

I finished documenting pages write throttling and checkpointing buffers adjustment sections under the performance notes:
https://apacheignite.readme.io/docs/durable-memory-tuning#section-pages-writes-throttling
https://apacheignite.readme.io/docs/durable-memory-tuning#section-checkpointing-buffer-size

Is there any reason why the write throttling shouldn’t be enabled by default? I couldn’t come with any negative effect.

—
Denis

[dsetrakyan]

Thanks Denis, looks very nice!

Igniters, let's stop asking users to enable flags in order for the system
to behave properly. All such flags should be enabled by default.

D.

On Wed, Nov 15, 2017 at 11:26 AM, Denis Magda <[hidden email]> wrote:

> Igniters,
>
> I finished documenting pages write throttling and checkpointing buffers
> adjustment sections under the performance notes:
> https://apacheignite.readme.io/docs/durable-memory-tuning#
> section-pages-writes-throttling
> https://apacheignite.readme.io/docs/durable-memory-tuning#
> section-checkpointing-buffer-size
>
> Is there any reason why the write throttling shouldn’t be enabled by
> default? I couldn’t come with any negative effect.
>
> —
> Denis
>
>

[Vladimir Ozerov]

AFAIK throttling may decrease peak throughput.

Alex, Ivan,
Is it correct?

On Thu, Nov 16, 2017 at 1:12 AM, Dmitriy Setrakyan <[hidden email]>
wrote:

> Thanks Denis, looks very nice!
>
> Igniters, let's stop asking users to enable flags in order for the system
> to behave properly. All such flags should be enabled by default.
>
> D.
>
> On Wed, Nov 15, 2017 at 11:26 AM, Denis Magda <[hidden email]> wrote:
>
> > Igniters,
> >
> > I finished documenting pages write throttling and checkpointing buffers
> > adjustment sections under the performance notes:
> > https://apacheignite.readme.io/docs/durable-memory-tuning#
> > section-pages-writes-throttling
> > https://apacheignite.readme.io/docs/durable-memory-tuning#
> > section-checkpointing-buffer-size
> >
> > Is there any reason why the write throttling shouldn’t be enabled by
> > default? I couldn’t come with any negative effect.
> >
> > —
> > Denis
> >
> >
>

[Dmitriy Pavlov]

Yes, it is correct.

But still, right after implementation I was for enabling this mode by
detault.
- It is not simple to explain need to enable this feature to each user.
- Moreover write freezing after peak load may be considered by some Ignite
users as bug. Not all of them will write to user list.

чт, 16 нояб. 2017 г. в 10:47, Vladimir Ozerov <[hidden email]>:

> AFAIK throttling may decrease peak throughput.
>
> Alex, Ivan,
> Is it correct?
>
> On Thu, Nov 16, 2017 at 1:12 AM, Dmitriy Setrakyan <[hidden email]>
> wrote:
>
> > Thanks Denis, looks very nice!
> >
> > Igniters, let's stop asking users to enable flags in order for the system
> > to behave properly. All such flags should be enabled by default.
> >
> > D.
> >
> > On Wed, Nov 15, 2017 at 11:26 AM, Denis Magda <[hidden email]> wrote:
> >
> > > Igniters,
> > >
> > > I finished documenting pages write throttling and checkpointing buffers
> > > adjustment sections under the performance notes:
> > > https://apacheignite.readme.io/docs/durable-memory-tuning#
> > > section-pages-writes-throttling
> > > https://apacheignite.readme.io/docs/durable-memory-tuning#
> > > section-checkpointing-buffer-size
> > >
> > > Is there any reason why the write throttling shouldn’t be enabled by
> > > default? I couldn’t come with any negative effect.
> > >
> > > —
> > > Denis
> > >
> > >
> >
>

[Ivan Rakov]

I agree.

Moreover, peak throughput after long idle period won't be decreased.
Throttling works only during ongoing checkpoint (and probability of
checkpoint running after idle period is very low).

Best Regards,
Ivan Rakov

On 16.11.2017 10:52, Dmitry Pavlov wrote:

> Yes, it is correct.
>
> But still, right after implementation I was for enabling this mode by
> detault.
> - It is not simple to explain need to enable this feature to each user.
> - Moreover write freezing after peak load may be considered by some
> Ignite users as bug. Not all of them will write to user list.
>
> чт, 16 нояб. 2017 г. в 10:47, Vladimir Ozerov <[hidden email]
> <mailto:[hidden email]>>:
>
>     AFAIK throttling may decrease peak throughput.
>
>     Alex, Ivan,
>     Is it correct?
>
>     On Thu, Nov 16, 2017 at 1:12 AM, Dmitriy Setrakyan
>     <[hidden email] <mailto:[hidden email]>>
>     wrote:
>
>     > Thanks Denis, looks very nice!
>     >
>     > Igniters, let's stop asking users to enable flags in order for
>     the system
>     > to behave properly. All such flags should be enabled by default.
>     >
>     > D.
>     >
>     > On Wed, Nov 15, 2017 at 11:26 AM, Denis Magda <[hidden email]
>     <mailto:[hidden email]>> wrote:
>     >
>     > > Igniters,
>     > >
>     > > I finished documenting pages write throttling and
>     checkpointing buffers
>     > > adjustment sections under the performance notes:
>     > > https://apacheignite.readme.io/docs/durable-memory-tuning#
>     > > section-pages-writes-throttling
>     > > https://apacheignite.readme.io/docs/durable-memory-tuning#
>     > > section-checkpointing-buffer-size
>     > >
>     > > Is there any reason why the write throttling shouldn’t be
>     enabled by
>     > > default? I couldn’t come with any negative effect.
>     > >
>     > > —
>     > > Denis
>     > >
>     > >
>     >
>

[dsetrakyan]

Has anyone benchmarked the write-throttling? Would be nice to see the CPU
and memory graphs.

On Thu, Nov 16, 2017 at 12:03 AM, Ivan Rakov <[hidden email]> wrote:

> I agree.
>
> Moreover, peak throughput after long idle period won't be decreased.
> Throttling works only during ongoing checkpoint (and probability of
> checkpoint running after idle period is very low).
>
> Best Regards,
> Ivan Rakov
>
> On 16.11.2017 10:52, Dmitry Pavlov wrote:
>
>> Yes, it is correct.
>>
>> But still, right after implementation I was for enabling this mode by
>> detault.
>> - It is not simple to explain need to enable this feature to each user.
>> - Moreover write freezing after peak load may be considered by some
>> Ignite users as bug. Not all of them will write to user list.
>>
>> чт, 16 нояб. 2017 г. в 10:47, Vladimir Ozerov <[hidden email]
>> <mailto:[hidden email]>>:
>>
>>     AFAIK throttling may decrease peak throughput.
>>
>>     Alex, Ivan,
>>     Is it correct?
>>
>>     On Thu, Nov 16, 2017 at 1:12 AM, Dmitriy Setrakyan
>>     <[hidden email] <mailto:[hidden email]>>
>>     wrote:
>>
>>     > Thanks Denis, looks very nice!
>>     >
>>     > Igniters, let's stop asking users to enable flags in order for
>>     the system
>>     > to behave properly. All such flags should be enabled by default.
>>     >
>>     > D.
>>     >
>>     > On Wed, Nov 15, 2017 at 11:26 AM, Denis Magda <[hidden email]
>>     <mailto:[hidden email]>> wrote:
>>     >
>>     > > Igniters,
>>     > >
>>     > > I finished documenting pages write throttling and
>>     checkpointing buffers
>>     > > adjustment sections under the performance notes:
>>     > > https://apacheignite.readme.io/docs/durable-memory-tuning#
>>     > > section-pages-writes-throttling
>>     > > https://apacheignite.readme.io/docs/durable-memory-tuning#
>>     > > section-checkpointing-buffer-size
>>     > >
>>     > > Is there any reason why the write throttling shouldn’t be
>>     enabled by
>>     > > default? I couldn’t come with any negative effect.
>>     > >
>>     > > —
>>     > > Denis
>>     > >
>>     > >
>>     >
>>
>>
>