IGNITE-8111: Add extra validation for WAL segment size

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

IGNITE-8111: Add extra validation for WAL segment size

Ivan Rakov
Igniters,

We've bumped into situation when using extremely small (<10 pages) WAL
segment size caused unobvious errors and assertions:
https://issues.apache.org/jira/browse/IGNITE-8111
Let's reach a consensus on which value of WAL segment size should be
considered as minimum. I tried tests with 128kb WAL segment and they
didn't fail.
Keeping a little reserve, I propose 512kb as minimum allowable WAL
segment size. I don't know why anyone may need a lesser value.

Please share your thoughts and let know if you have objections on 512kb.

--
Best Regards,
Ivan Rakov

Reply | Threaded
Open this post in threaded view
|

Re: IGNITE-8111: Add extra validation for WAL segment size

Dmitriy Pavlov
I have no objections on 512kb

пн, 2 апр. 2018 г. в 19:17, Ivan Rakov <[hidden email]>:

> Igniters,
>
> We've bumped into situation when using extremely small (<10 pages) WAL
> segment size caused unobvious errors and assertions:
> https://issues.apache.org/jira/browse/IGNITE-8111
> Let's reach a consensus on which value of WAL segment size should be
> considered as minimum. I tried tests with 128kb WAL segment and they
> didn't fail.
> Keeping a little reserve, I propose 512kb as minimum allowable WAL
> segment size. I don't know why anyone may need a lesser value.
>
> Please share your thoughts and let know if you have objections on 512kb.
>
> --
> Best Regards,
> Ivan Rakov
>
>
Reply | Threaded
Open this post in threaded view
|

Re: IGNITE-8111: Add extra validation for WAL segment size

dmagda
In reply to this post by Ivan Rakov
Ivan,

Will his value be used as a default one as well or only is a possible
minimum? As for the defaults, I see that Postgres and Cassandra set 16mb as
a default segment size:
https://www.postgresql.org/docs/9.1/static/wal-configuration.html
https://www.enterprisedb.com/docs/en/10.0/EPAS_Guide_v10/EDB_Postgres_Advanced_Server_Guide.1.45.html

Should we have the same defaults?

--
Denis


On Mon, Apr 2, 2018 at 9:16 AM, Ivan Rakov <[hidden email]> wrote:

> Igniters,
>
> We've bumped into situation when using extremely small (<10 pages) WAL
> segment size caused unobvious errors and assertions:
> https://issues.apache.org/jira/browse/IGNITE-8111
> Let's reach a consensus on which value of WAL segment size should be
> considered as minimum. I tried tests with 128kb WAL segment and they didn't
> fail.
> Keeping a little reserve, I propose 512kb as minimum allowable WAL segment
> size. I don't know why anyone may need a lesser value.
>
> Please share your thoughts and let know if you have objections on 512kb.
>
> --
> Best Regards,
> Ivan Rakov
>
>
Reply | Threaded
Open this post in threaded view
|

Re: IGNITE-8111: Add extra validation for WAL segment size

Ivan Rakov
Denis,

The topic is about a possible minimum only.
Our default for WAL segment size is 64mb and we are not going to change
it (at least under this ticket).

Best Regards,
Ivan Rakov

On 02.04.2018 19:51, Denis Magda wrote:

> Ivan,
>
> Will his value be used as a default one as well or only is a possible
> minimum? As for the defaults, I see that Postgres and Cassandra set 16mb as
> a default segment size:
> https://www.postgresql.org/docs/9.1/static/wal-configuration.html
> https://www.enterprisedb.com/docs/en/10.0/EPAS_Guide_v10/EDB_Postgres_Advanced_Server_Guide.1.45.html
>
> Should we have the same defaults?
>
> --
> Denis
>
>
> On Mon, Apr 2, 2018 at 9:16 AM, Ivan Rakov <[hidden email]> wrote:
>
>> Igniters,
>>
>> We've bumped into situation when using extremely small (<10 pages) WAL
>> segment size caused unobvious errors and assertions:
>> https://issues.apache.org/jira/browse/IGNITE-8111
>> Let's reach a consensus on which value of WAL segment size should be
>> considered as minimum. I tried tests with 128kb WAL segment and they didn't
>> fail.
>> Keeping a little reserve, I propose 512kb as minimum allowable WAL segment
>> size. I don't know why anyone may need a lesser value.
>>
>> Please share your thoughts and let know if you have objections on 512kb.
>>
>> --
>> Best Regards,
>> Ivan Rakov
>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: IGNITE-8111: Add extra validation for WAL segment size

Lucas Beeler
In reply to this post by dmagda
+1 for this from an Ignite professional services consultant. The more we can do to protect users from non-sensical configuration settings, the better. I’ve seen a lot of them in the field (and not just related to WAL segment size). As Scott McNealy of Sun fame once said, “there’s nothing dumber than an end user.”

Take care,
Lucas

--
Lucas BEELER
Technical Consultant, Professional Services
GridGain Systems
www.gridgain.com

> On Apr 2, 2018, at 9:51 AM, Denis Magda <[hidden email]> wrote:
>
> Ivan,
>
> Will his value be used as a default one as well or only is a possible
> minimum? As for the defaults, I see that Postgres and Cassandra set 16mb as
> a default segment size:
> https://www.postgresql.org/docs/9.1/static/wal-configuration.html
> https://www.enterprisedb.com/docs/en/10.0/EPAS_Guide_v10/EDB_Postgres_Advanced_Server_Guide.1.45.html
>
> Should we have the same defaults?
>
> --
> Denis
>
>
> On Mon, Apr 2, 2018 at 9:16 AM, Ivan Rakov <[hidden email]> wrote:
>
>> Igniters,
>>
>> We've bumped into situation when using extremely small (<10 pages) WAL
>> segment size caused unobvious errors and assertions:
>> https://issues.apache.org/jira/browse/IGNITE-8111
>> Let's reach a consensus on which value of WAL segment size should be
>> considered as minimum. I tried tests with 128kb WAL segment and they didn't
>> fail.
>> Keeping a little reserve, I propose 512kb as minimum allowable WAL segment
>> size. I don't know why anyone may need a lesser value.
>>
>> Please share your thoughts and let know if you have objections on 512kb.
>>
>> --
>> Best Regards,
>> Ivan Rakov
>>
>>