Atomic caches

[akurbanov]

Hello Igniters,

I want to address a question on AtomicConfiguration validation. I've tested
in ignite-1.8 branch that it is impossible to start two nodes with different
AtomicConfiguration parameters e.g. different cache modes or numbers of
backups are provided.
JIRA link: https://issues.apache.org/jira/browse/IGNITE-2096

In ignite-2.4 AtomicConfiguration validation is completely skipped on node
startup and this issue is non-reproducible. Node with alternative
configuration successfully joins, but this configuration is being completely
ignored, all created atomics will reference the same initial configuration
and belong to the same cache "ignite-sys-atomic-cache@default-ds-group",
even if configuration is provided in constructor.

Do we need any kind of validation for this configuration?
Would it be correct to use the same approach for atomic types instances
caching as used for IgniteQueue/IgniteSet, cache for each unique
configuration?

Best regards,
Anton Kurbanov




--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

[dsetrakyan]

I would say absolutely YES - we need to have configuration validation.

Igniters, why was the validation skipped in atomic caches?

D.

On Fri, Apr 6, 2018 at 1:43 PM, akurbanov <[hidden email]> wrote:

> Hello Igniters,
>
> I want to address a question on AtomicConfiguration validation. I've tested
> in ignite-1.8 branch that it is impossible to start two nodes with
> different
> AtomicConfiguration parameters e.g. different cache modes or numbers of
> backups are provided.
> JIRA link: https://issues.apache.org/jira/browse/IGNITE-2096
>
> In ignite-2.4 AtomicConfiguration validation is completely skipped on node
> startup and this issue is non-reproducible. Node with alternative
> configuration successfully joins, but this configuration is being
> completely
> ignored, all created atomics will reference the same initial configuration
> and belong to the same cache "ignite-sys-atomic-cache@default-ds-group",
> even if configuration is provided in constructor.
>
> Do we need any kind of validation for this configuration?
> Would it be correct to use the same approach for atomic types instances
> caching as used for IgniteQueue/IgniteSet, cache for each unique
> configuration?
>
> Best regards,
> Anton Kurbanov
>
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>

[yzhdanov]

Dmitry, I think Anton meant AtomicsConfiguration, not atomic caches.
However, I would make sure we validate all conf parameters.

Anton, can you please share junit test that shows the problem?

Yakov Zhdanov

сб, 7 апр. 2018 г., 6:12 Dmitriy Setrakyan <[hidden email]>:

> I would say absolutely YES - we need to have configuration validation.
>
> Igniters, why was the validation skipped in atomic caches?
>
> D.
>
> On Fri, Apr 6, 2018 at 1:43 PM, akurbanov <[hidden email]> wrote:
>
> > Hello Igniters,
> >
> > I want to address a question on AtomicConfiguration validation. I've
> tested
> > in ignite-1.8 branch that it is impossible to start two nodes with
> > different
> > AtomicConfiguration parameters e.g. different cache modes or numbers of
> > backups are provided.
> > JIRA link: https://issues.apache.org/jira/browse/IGNITE-2096
> >
> > In ignite-2.4 AtomicConfiguration validation is completely skipped on
> node
> > startup and this issue is non-reproducible. Node with alternative
> > configuration successfully joins, but this configuration is being
> > completely
> > ignored, all created atomics will reference the same initial
> configuration
> > and belong to the same cache "ignite-sys-atomic-cache@default-ds-group",
> > even if configuration is provided in constructor.
> >
> > Do we need any kind of validation for this configuration?
> > Would it be correct to use the same approach for atomic types instances
> > caching as used for IgniteQueue/IgniteSet, cache for each unique
> > configuration?
> >
> > Best regards,
> > Anton Kurbanov
> >
> >
> >
> >
> > --
> > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> >
>

[akurbanov]

I've attached .java file to https://issues.apache.org/jira/browse/IGNITE-2096

Atomic references created with cacheMode=REPLICATED configuration are being
lost when first references are created with cacheMode=PARTITIONED, no
validation errors.




--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

[Ilya Lantukh]

Anton,

Please do not use term "atomic cache" for system caches that hold internal
data for atomic data structures. This is very confusing.

You are right, currently there is no logic that will validate cache
configuration. It definitely should be fixed. And having configuration
parameters encoded in cache name, like it is currently implemented for
collections, is one of the most straightforward approaches.



On Mon, Apr 9, 2018 at 4:19 PM, akurbanov <[hidden email]> wrote:

> I've attached .java file to https://issues.apache.org/
> jira/browse/IGNITE-2096
>
> Atomic references created with cacheMode=REPLICATED configuration are being
> lost when first references are created with cacheMode=PARTITIONED, no
> validation errors.
>
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>

--
Best regards,
Ilya

[dsetrakyan]

On Mon, Apr 9, 2018 at 6:29 AM, Ilya Lantukh <[hidden email]> wrote:

> Anton,
>
> Please do not use term "atomic cache" for system caches that hold internal
> data for atomic data structures. This is very confusing.
>
> You are right, currently there is no logic that will validate cache
> configuration. It definitely should be fixed. And having configuration
> parameters encoded in cache name, like it is currently implemented for
> collections, is one of the most straightforward approaches.
>

Anton, I am completely confused now. Does Ignite validate the configuration
for the atomic caches (not atomic data structures)?

[akurbanov]

Dmitry,

Sorry for confusing topic. I I'm pretty sure that configuration for atomic
caches is validated, will double-check this. I was referring only atomic
data structures cache.



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

[dsetrakyan]

On Tue, Apr 10, 2018 at 2:03 PM, akurbanov <[hidden email]> wrote:

> Dmitry,
>
> Sorry for confusing topic. I I'm pretty sure that configuration for atomic
> caches is validated, will double-check this. I was referring only atomic
> data structures cache.
>

Got it. We should definitely add validation for the atomic data structures
configuration as well.

[Valentin Kulichenko]

Guys,

I also not sure I understand the purpose of methods like [1] that accept
instance of AtomicConfiguration to create new atomic structure. Per my
knowledge, all atomics are stored in a single cache which is configured by
AtomicConfiguration provided on startup as part of IgniteConfiguration. If
that's the case, providing another configuration for a particular atomic
doesn't make sense because it will never be used.

Any thoughts on this? Unless I'm missing something, I think we can
deprecate these methods.

[1]
https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/Ignite.html#atomicLong-java.lang.String-org.apache.ignite.configuration.AtomicConfiguration-long-boolean-

-Val

On Tue, Apr 10, 2018 at 3:28 PM, Dmitriy Setrakyan <[hidden email]>
wrote:

> On Tue, Apr 10, 2018 at 2:03 PM, akurbanov <[hidden email]> wrote:
>
> > Dmitry,
> >
> > Sorry for confusing topic. I I'm pretty sure that configuration for
> atomic
> > caches is validated, will double-check this. I was referring only atomic
> > data structures cache.
> >
>
> Got it. We should definitely add validation for the atomic data structures
> configuration as well.
>

[Ilya Lantukh]

Val,

The logic you describe changed a while ago, with introduction of persistent
storage and cache group mechanics. Currently there might exist multiple
special caches where data structures are stored. The main reason for it was
to give users ability to put different structures into different cache
groups, and configure which ones should be persistent and which shouldn't.

On Wed, Apr 11, 2018 at 2:33 AM, Valentin Kulichenko <
[hidden email]> wrote:

> Guys,
>
> I also not sure I understand the purpose of methods like [1] that accept
> instance of AtomicConfiguration to create new atomic structure. Per my
> knowledge, all atomics are stored in a single cache which is configured by
> AtomicConfiguration provided on startup as part of IgniteConfiguration. If
> that's the case, providing another configuration for a particular atomic
> doesn't make sense because it will never be used.
>
> Any thoughts on this? Unless I'm missing something, I think we can
> deprecate these methods.
>
> [1]
> https://ignite.apache.org/releases/latest/javadoc/org/
> apache/ignite/Ignite.html#atomicLong-java.lang.String-org.apache.ignite.
> configuration.AtomicConfiguration-long-boolean-
>
> -Val
>
> On Tue, Apr 10, 2018 at 3:28 PM, Dmitriy Setrakyan <[hidden email]>
> wrote:
>
> > On Tue, Apr 10, 2018 at 2:03 PM, akurbanov <[hidden email]> wrote:
> >
> > > Dmitry,
> > >
> > > Sorry for confusing topic. I I'm pretty sure that configuration for
> > atomic
> > > caches is validated, will double-check this. I was referring only
> atomic
> > > data structures cache.
> > >
> >
> > Got it. We should definitely add validation for the atomic data
> structures
> > configuration as well.
> >
>

--
Best regards,
Ilya