MVCC configuration

[Semyon Boikov]

Folks, thank you for feedback, I want to summarize some decisions:

1. Mvcc is disabled by default. We'll add two flags to enable mvcc:
per-cache flag - CacheConfiguration.isMvccEnabled, default value for all
caches - IgniteConfiguration.isMvccEnabled.
2. For initial implementation mvcc for ATOMIC cache is out of scope, it can
be enabled only for TRANSACTIONAL caches.
3. Mvcc coordinator can be any server node (oldest server node is selected
automatically). Also I believe we need possibility to have *dedicated* mvcc
coordinator nodes which will process only internal mvcc messages. Node can
be marked as dedicated coordinator via new flag
IgniteConfiguration.isMvccCoordinator or we can add separate
MvccConfiguration bean. But let's skip this decision for now before we have
benchmarks numbers.
4. Need add some metrics to monitor mvcc coordinator performance.


Thanks

On Tue, Sep 19, 2017 at 10:47 AM, Vladimir Ozerov <[hidden email]>
wrote:

> This could be something like "preferredMvccCoordinator".
>
> On Tue, Sep 19, 2017 at 10:40 AM, Alexey Goncharuk <
> [hidden email]> wrote:
>
> > >
> > > I agree that we need coordinator nodes, but I do not understand why
> can't
> > > we reuse some cache nodes for it? Why do we need to ask user to start
> up
> > > yet another type of node?
> > >
> >
> > Dmitriy,
> >
> > My understanding is that Semyon does not deny a cache node to be used as
> a
> > coordinator. This property will allow to optionally have a *dedicated*
> node
> > serving as a coordinator to improve cluster throughput under heavy load.
> >
>

[Vladimir Ozerov]

I would say that per-cache configuration should be out of scope as well for
the first iteration. Because we do not know whether it will be valid for
DML.

On Tue, Sep 19, 2017 at 12:15 PM, Semyon Boikov <[hidden email]>
wrote:

> Folks, thank you for feedback, I want to summarize some decisions:
>
> 1. Mvcc is disabled by default. We'll add two flags to enable mvcc:
> per-cache flag - CacheConfiguration.isMvccEnabled, default value for all
> caches - IgniteConfiguration.isMvccEnabled.
> 2. For initial implementation mvcc for ATOMIC cache is out of scope, it can
> be enabled only for TRANSACTIONAL caches.
> 3. Mvcc coordinator can be any server node (oldest server node is selected
> automatically). Also I believe we need possibility to have *dedicated* mvcc
> coordinator nodes which will process only internal mvcc messages. Node can
> be marked as dedicated coordinator via new flag
> IgniteConfiguration.isMvccCoordinator or we can add separate
> MvccConfiguration bean. But let's skip this decision for now before we have
> benchmarks numbers.
> 4. Need add some metrics to monitor mvcc coordinator performance.
>
>
> Thanks
>
> On Tue, Sep 19, 2017 at 10:47 AM, Vladimir Ozerov <[hidden email]>
> wrote:
>
> > This could be something like "preferredMvccCoordinator".
> >
> > On Tue, Sep 19, 2017 at 10:40 AM, Alexey Goncharuk <
> > [hidden email]> wrote:
> >
> > > >
> > > > I agree that we need coordinator nodes, but I do not understand why
> > can't
> > > > we reuse some cache nodes for it? Why do we need to ask user to start
> > up
> > > > yet another type of node?
> > > >
> > >
> > > Dmitriy,
> > >
> > > My understanding is that Semyon does not deny a cache node to be used
> as
> > a
> > > coordinator. This property will allow to optionally have a *dedicated*
> > node
> > > serving as a coordinator to improve cluster throughput under heavy
> load.
> > >
> >
>

[Semyon Boikov]

If it is not valid for DML then we can easily detect this situation and
throw exception, but if I do not use DML why non make it configurable
per-cache?

On Tue, Sep 19, 2017 at 12:22 PM, Vladimir Ozerov <[hidden email]>
wrote:

> I would say that per-cache configuration should be out of scope as well for
> the first iteration. Because we do not know whether it will be valid for
> DML.
>
> On Tue, Sep 19, 2017 at 12:15 PM, Semyon Boikov <[hidden email]>
> wrote:
>
> > Folks, thank you for feedback, I want to summarize some decisions:
> >
> > 1. Mvcc is disabled by default. We'll add two flags to enable mvcc:
> > per-cache flag - CacheConfiguration.isMvccEnabled, default value for all
> > caches - IgniteConfiguration.isMvccEnabled.
> > 2. For initial implementation mvcc for ATOMIC cache is out of scope, it
> can
> > be enabled only for TRANSACTIONAL caches.
> > 3. Mvcc coordinator can be any server node (oldest server node is
> selected
> > automatically). Also I believe we need possibility to have *dedicated*
> mvcc
> > coordinator nodes which will process only internal mvcc messages. Node
> can
> > be marked as dedicated coordinator via new flag
> > IgniteConfiguration.isMvccCoordinator or we can add separate
> > MvccConfiguration bean. But let's skip this decision for now before we
> have
> > benchmarks numbers.
> > 4. Need add some metrics to monitor mvcc coordinator performance.
> >
> >
> > Thanks
> >
> > On Tue, Sep 19, 2017 at 10:47 AM, Vladimir Ozerov <[hidden email]>
> > wrote:
> >
> > > This could be something like "preferredMvccCoordinator".
> > >
> > > On Tue, Sep 19, 2017 at 10:40 AM, Alexey Goncharuk <
> > > [hidden email]> wrote:
> > >
> > > > >
> > > > > I agree that we need coordinator nodes, but I do not understand why
> > > can't
> > > > > we reuse some cache nodes for it? Why do we need to ask user to
> start
> > > up
> > > > > yet another type of node?
> > > > >
> > > >
> > > > Dmitriy,
> > > >
> > > > My understanding is that Semyon does not deny a cache node to be used
> > as
> > > a
> > > > coordinator. This property will allow to optionally have a
> *dedicated*
> > > node
> > > > serving as a coordinator to improve cluster throughput under heavy
> > load.
> > > >
> > >
> >
>

[Vladimir Ozerov]

What I mean is that it might be not applicable for DML by design. E.g. may
be we will have to fallback to per-memory-policy approach, or to
per-cache-group. As we do not know it at the moment and there is no clear
demand from users, I would simply put it aside to avoid in mess in public
API in future.

Moreover, per-cache flag raises additional questions which can be put out
of scope otherwise. E.g. is it legal to mix MVCC and non-MVCC caches in a
single transaction? If yes, what is the contract? Without fine-grained
per-cache approach in the first iteration we can avoid answering it.

On Tue, Sep 19, 2017 at 12:25 PM, Semyon Boikov <[hidden email]>
wrote:

> If it is not valid for DML then we can easily detect this situation and
> throw exception, but if I do not use DML why non make it configurable
> per-cache?
>
> On Tue, Sep 19, 2017 at 12:22 PM, Vladimir Ozerov <[hidden email]>
> wrote:
>
> > I would say that per-cache configuration should be out of scope as well
> for
> > the first iteration. Because we do not know whether it will be valid for
> > DML.
> >
> > On Tue, Sep 19, 2017 at 12:15 PM, Semyon Boikov <[hidden email]>
> > wrote:
> >
> > > Folks, thank you for feedback, I want to summarize some decisions:
> > >
> > > 1. Mvcc is disabled by default. We'll add two flags to enable mvcc:
> > > per-cache flag - CacheConfiguration.isMvccEnabled, default value for
> all
> > > caches - IgniteConfiguration.isMvccEnabled.
> > > 2. For initial implementation mvcc for ATOMIC cache is out of scope, it
> > can
> > > be enabled only for TRANSACTIONAL caches.
> > > 3. Mvcc coordinator can be any server node (oldest server node is
> > selected
> > > automatically). Also I believe we need possibility to have *dedicated*
> > mvcc
> > > coordinator nodes which will process only internal mvcc messages. Node
> > can
> > > be marked as dedicated coordinator via new flag
> > > IgniteConfiguration.isMvccCoordinator or we can add separate
> > > MvccConfiguration bean. But let's skip this decision for now before we
> > have
> > > benchmarks numbers.
> > > 4. Need add some metrics to monitor mvcc coordinator performance.
> > >
> > >
> > > Thanks
> > >
> > > On Tue, Sep 19, 2017 at 10:47 AM, Vladimir Ozerov <
> [hidden email]>
> > > wrote:
> > >
> > > > This could be something like "preferredMvccCoordinator".
> > > >
> > > > On Tue, Sep 19, 2017 at 10:40 AM, Alexey Goncharuk <
> > > > [hidden email]> wrote:
> > > >
> > > > > >
> > > > > > I agree that we need coordinator nodes, but I do not understand
> why
> > > > can't
> > > > > > we reuse some cache nodes for it? Why do we need to ask user to
> > start
> > > > up
> > > > > > yet another type of node?
> > > > > >
> > > > >
> > > > > Dmitriy,
> > > > >
> > > > > My understanding is that Semyon does not deny a cache node to be
> used
> > > as
> > > > a
> > > > > coordinator. This property will allow to optionally have a
> > *dedicated*
> > > > node
> > > > > serving as a coordinator to improve cluster throughput under heavy
> > > load.
> > > > >
> > > >
> > >
> >
>

[yzhdanov]

Guys, I we should additionally provide ability to manually switch mvcc
coordinator via MBean passing order or ID of a new one. We already have all
the machinery for this.

--Yakov

[dsetrakyan]

Can caches within the same group have different MVCC configuration?

On Tue, Sep 19, 2017 at 2:34 AM, Vladimir Ozerov <[hidden email]>
wrote:

> What I mean is that it might be not applicable for DML by design. E.g. may
> be we will have to fallback to per-memory-policy approach, or to
> per-cache-group. As we do not know it at the moment and there is no clear
> demand from users, I would simply put it aside to avoid in mess in public
> API in future.
>
> Moreover, per-cache flag raises additional questions which can be put out
> of scope otherwise. E.g. is it legal to mix MVCC and non-MVCC caches in a
> single transaction? If yes, what is the contract? Without fine-grained
> per-cache approach in the first iteration we can avoid answering it.
>
> On Tue, Sep 19, 2017 at 12:25 PM, Semyon Boikov <[hidden email]>
> wrote:
>
> > If it is not valid for DML then we can easily detect this situation and
> > throw exception, but if I do not use DML why non make it configurable
> > per-cache?
> >
> > On Tue, Sep 19, 2017 at 12:22 PM, Vladimir Ozerov <[hidden email]>
> > wrote:
> >
> > > I would say that per-cache configuration should be out of scope as well
> > for
> > > the first iteration. Because we do not know whether it will be valid
> for
> > > DML.
> > >
> > > On Tue, Sep 19, 2017 at 12:15 PM, Semyon Boikov <[hidden email]>
> > > wrote:
> > >
> > > > Folks, thank you for feedback, I want to summarize some decisions:
> > > >
> > > > 1. Mvcc is disabled by default. We'll add two flags to enable mvcc:
> > > > per-cache flag - CacheConfiguration.isMvccEnabled, default value for
> > all
> > > > caches - IgniteConfiguration.isMvccEnabled.
> > > > 2. For initial implementation mvcc for ATOMIC cache is out of scope,
> it
> > > can
> > > > be enabled only for TRANSACTIONAL caches.
> > > > 3. Mvcc coordinator can be any server node (oldest server node is
> > > selected
> > > > automatically). Also I believe we need possibility to have
> *dedicated*
> > > mvcc
> > > > coordinator nodes which will process only internal mvcc messages.
> Node
> > > can
> > > > be marked as dedicated coordinator via new flag
> > > > IgniteConfiguration.isMvccCoordinator or we can add separate
> > > > MvccConfiguration bean. But let's skip this decision for now before
> we
> > > have
> > > > benchmarks numbers.
> > > > 4. Need add some metrics to monitor mvcc coordinator performance.
> > > >
> > > >
> > > > Thanks
> > > >
> > > > On Tue, Sep 19, 2017 at 10:47 AM, Vladimir Ozerov <
> > [hidden email]>
> > > > wrote:
> > > >
> > > > > This could be something like "preferredMvccCoordinator".
> > > > >
> > > > > On Tue, Sep 19, 2017 at 10:40 AM, Alexey Goncharuk <
> > > > > [hidden email]> wrote:
> > > > >
> > > > > > >
> > > > > > > I agree that we need coordinator nodes, but I do not understand
> > why
> > > > > can't
> > > > > > > we reuse some cache nodes for it? Why do we need to ask user to
> > > start
> > > > > up
> > > > > > > yet another type of node?
> > > > > > >
> > > > > >
> > > > > > Dmitriy,
> > > > > >
> > > > > > My understanding is that Semyon does not deny a cache node to be
> > used
> > > > as
> > > > > a
> > > > > > coordinator. This property will allow to optionally have a
> > > *dedicated*
> > > > > node
> > > > > > serving as a coordinator to improve cluster throughput under
> heavy
> > > > load.
> > > > > >
> > > > >
> > > >
> > >
> >
>

[Semyon Boikov]

> Can caches within the same group have different MVCC configuration?

I think it is possible to implement, but there are some issues:
- for mvcc we need store mvcc version in hash index item (for now it is 16
bytes), since index items have fixed size then if we store in this index
data for caches with disabled mvcc, then it will have unnecessary 16 bytes
overhead
- for mvcc caches we need create correct hash index in advance, so if group
was created with mvcc disabled, then later it is not possible to add in
this group mvcc enabled cache

Do you think we really need have in the same group caches with different
mvcc configuration? for simplicity I would do not allow this.

Thanks

On Wed, Sep 20, 2017 at 7:30 AM, Dmitriy Setrakyan <[hidden email]>
wrote:

> Can caches within the same group have different MVCC configuration?
>
> On Tue, Sep 19, 2017 at 2:34 AM, Vladimir Ozerov <[hidden email]>
> wrote:
>
> > What I mean is that it might be not applicable for DML by design. E.g.
> may
> > be we will have to fallback to per-memory-policy approach, or to
> > per-cache-group. As we do not know it at the moment and there is no clear
> > demand from users, I would simply put it aside to avoid in mess in public
> > API in future.
> >
> > Moreover, per-cache flag raises additional questions which can be put out
> > of scope otherwise. E.g. is it legal to mix MVCC and non-MVCC caches in a
> > single transaction? If yes, what is the contract? Without fine-grained
> > per-cache approach in the first iteration we can avoid answering it.
> >
> > On Tue, Sep 19, 2017 at 12:25 PM, Semyon Boikov <[hidden email]>
> > wrote:
> >
> > > If it is not valid for DML then we can easily detect this situation and
> > > throw exception, but if I do not use DML why non make it configurable
> > > per-cache?
> > >
> > > On Tue, Sep 19, 2017 at 12:22 PM, Vladimir Ozerov <
> [hidden email]>
> > > wrote:
> > >
> > > > I would say that per-cache configuration should be out of scope as
> well
> > > for
> > > > the first iteration. Because we do not know whether it will be valid
> > for
> > > > DML.
> > > >
> > > > On Tue, Sep 19, 2017 at 12:15 PM, Semyon Boikov <
> [hidden email]>
> > > > wrote:
> > > >
> > > > > Folks, thank you for feedback, I want to summarize some decisions:
> > > > >
> > > > > 1. Mvcc is disabled by default. We'll add two flags to enable mvcc:
> > > > > per-cache flag - CacheConfiguration.isMvccEnabled, default value
> for
> > > all
> > > > > caches - IgniteConfiguration.isMvccEnabled.
> > > > > 2. For initial implementation mvcc for ATOMIC cache is out of
> scope,
> > it
> > > > can
> > > > > be enabled only for TRANSACTIONAL caches.
> > > > > 3. Mvcc coordinator can be any server node (oldest server node is
> > > > selected
> > > > > automatically). Also I believe we need possibility to have
> > *dedicated*
> > > > mvcc
> > > > > coordinator nodes which will process only internal mvcc messages.
> > Node
> > > > can
> > > > > be marked as dedicated coordinator via new flag
> > > > > IgniteConfiguration.isMvccCoordinator or we can add separate
> > > > > MvccConfiguration bean. But let's skip this decision for now before
> > we
> > > > have
> > > > > benchmarks numbers.
> > > > > 4. Need add some metrics to monitor mvcc coordinator performance.
> > > > >
> > > > >
> > > > > Thanks
> > > > >
> > > > > On Tue, Sep 19, 2017 at 10:47 AM, Vladimir Ozerov <
> > > [hidden email]>
> > > > > wrote:
> > > > >
> > > > > > This could be something like "preferredMvccCoordinator".
> > > > > >
> > > > > > On Tue, Sep 19, 2017 at 10:40 AM, Alexey Goncharuk <
> > > > > > [hidden email]> wrote:
> > > > > >
> > > > > > > >
> > > > > > > > I agree that we need coordinator nodes, but I do not
> understand
> > > why
> > > > > > can't
> > > > > > > > we reuse some cache nodes for it? Why do we need to ask user
> to
> > > > start
> > > > > > up
> > > > > > > > yet another type of node?
> > > > > > > >
> > > > > > >
> > > > > > > Dmitriy,
> > > > > > >
> > > > > > > My understanding is that Semyon does not deny a cache node to
> be
> > > used
> > > > > as
> > > > > > a
> > > > > > > coordinator. This property will allow to optionally have a
> > > > *dedicated*
> > > > > > node
> > > > > > > serving as a coordinator to improve cluster throughput under
> > heavy
> > > > > load.
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

[dsetrakyan]

On Tue, Sep 19, 2017 at 11:31 PM, Semyon Boikov <[hidden email]>
wrote:

> > Can caches within the same group have different MVCC configuration?
>
> Do you think we really need have in the same group caches with different
> mvcc configuration? for simplicity I would do not allow this.
>

I agree, let's not allow it. In that case, are you going to have extra
validation on startup that caches in the same group must have identical
MVCC config?

[Semyon Boikov]

Yes, we'll add this validation.

On Wed, Sep 20, 2017 at 10:09 AM, Dmitriy Setrakyan <[hidden email]>
wrote:

> On Tue, Sep 19, 2017 at 11:31 PM, Semyon Boikov <[hidden email]>
> wrote:
>
> > > Can caches within the same group have different MVCC configuration?
> >
> > Do you think we really need have in the same group caches with different
> > mvcc configuration? for simplicity I would do not allow this.
> >
>
> I agree, let's not allow it. In that case, are you going to have extra
> validation on startup that caches in the same group must have identical
> MVCC config?
>