Apache Ignite Developers - Legacy Mail Archive

.NET Near Cache - new flag in NearCacheConfiguration.java?

Classic

List

Threaded

14 messages Options

Pavel Tupitsyn

.NET Near Cache - new flag in NearCacheConfiguration.java?

Igniters,

I'm working on .NET Near Cache feature [1]
(storing deserialized cache entries in CLR memory to improve performance).

Implementation is based on Java near cache, with some callbacks to .NET
side
for updating and invalidating cached entries.

However, I'd like to make this feature optional: enabling Java near cache
should not
always enable .NET near cache - some users may have mixed clusters, etc.

Therefore I'm adding NearCacheConfiguration#platformNearCacheEnabled
boolean flag.
Are there any objections or better ideas to configure this behavior?

Thanks,
Pavel

[1] https://issues.apache.org/jira/browse/IGNITE-12691

Igor Sapego-2

Re: .NET Near Cache - new flag in NearCacheConfiguration.java?

Do you suggest to introduce it in general configuration? Why not introduce
it only on platform side? Is there any .NET-specific configuration?

Best Regards,
Igor

On Tue, Feb 18, 2020 at 1:10 AM Pavel Tupitsyn <[hidden email]> wrote:

> Igniters,
>
> I'm working on .NET Near Cache feature [1]
> (storing deserialized cache entries in CLR memory to improve performance).
>
> Implementation is based on Java near cache, with some callbacks to .NET
> side
> for updating and invalidating cached entries.
>
> However, I'd like to make this feature optional: enabling Java near cache
> should not
> always enable .NET near cache - some users may have mixed clusters, etc.
>
> Therefore I'm adding NearCacheConfiguration#platformNearCacheEnabled
> boolean flag.
> Are there any objections or better ideas to configure this behavior?
>
> Thanks,
> Pavel
>
> [1] https://issues.apache.org/jira/browse/IGNITE-12691
>

Pavel Tupitsyn

Re: .NET Near Cache - new flag in NearCacheConfiguration.java?

Igor,

The problem is - we need to pass this flag around the cluster for Server
Near Caches,
so that .NET near caches are started accordingly.

There are 2 kinds of Near Caches:
- On client nodes: created on every client node separately by calling
ignite.CreateNearCache
- On server nodes: created on all server nodes if
CacheConfiguration.NearCacheConfiguration is set

When user says ignite.CreateCache(new CacheConfiguration
{NearCacheConfiguration = ...}),
the whole config is sent to all server nodes, and .NET-specific flag has to
be included somehow.

On Tue, Feb 18, 2020 at 2:59 PM Igor Sapego <[hidden email]> wrote:

> Do you suggest to introduce it in general configuration? Why not introduce
> it only on platform side? Is there any .NET-specific configuration?
>
> Best Regards,
> Igor
>
>
> On Tue, Feb 18, 2020 at 1:10 AM Pavel Tupitsyn <[hidden email]>
> wrote:
>
> > Igniters,
> >
> > I'm working on .NET Near Cache feature [1]
> > (storing deserialized cache entries in CLR memory to improve
> performance).
> >
> > Implementation is based on Java near cache, with some callbacks to .NET
> > side
> > for updating and invalidating cached entries.
> >
> > However, I'd like to make this feature optional: enabling Java near cache
> > should not
> > always enable .NET near cache - some users may have mixed clusters, etc.
> >
> > Therefore I'm adding NearCacheConfiguration#platformNearCacheEnabled
> > boolean flag.
> > Are there any objections or better ideas to configure this behavior?
> >
> > Thanks,
> > Pavel
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-12691
> >
>

Ilya Kasnacheev

Re: .NET Near Cache - new flag in NearCacheConfiguration.java?

Hello!

I suggest parametrizing it with platformId, to make it possible to
implement in e.g. C++ later without creating yet another parameter. Maybe
we can migrate Java near caches to same mechanism in the future.

Why would enabling it affect mixed clusters?

Regards,
--
Ilya Kasnacheev

вт, 18 февр. 2020 г. в 15:05, Pavel Tupitsyn <[hidden email]>:

> Igor,
>
> The problem is - we need to pass this flag around the cluster for Server
> Near Caches,
> so that .NET near caches are started accordingly.
>
> There are 2 kinds of Near Caches:
> - On client nodes: created on every client node separately by calling
> ignite.CreateNearCache
> - On server nodes: created on all server nodes if
> CacheConfiguration.NearCacheConfiguration is set
>
> When user says ignite.CreateCache(new CacheConfiguration
> {NearCacheConfiguration = ...}),
> the whole config is sent to all server nodes, and .NET-specific flag has to
> be included somehow.
>
> On Tue, Feb 18, 2020 at 2:59 PM Igor Sapego <[hidden email]> wrote:
>
> > Do you suggest to introduce it in general configuration? Why not
> introduce
> > it only on platform side? Is there any .NET-specific configuration?
> >
> > Best Regards,
> > Igor
> >
> >
> > On Tue, Feb 18, 2020 at 1:10 AM Pavel Tupitsyn <[hidden email]>
> > wrote:
> >
> > > Igniters,
> > >
> > > I'm working on .NET Near Cache feature [1]
> > > (storing deserialized cache entries in CLR memory to improve
> > performance).
> > >
> > > Implementation is based on Java near cache, with some callbacks to .NET
> > > side
> > > for updating and invalidating cached entries.
> > >
> > > However, I'd like to make this feature optional: enabling Java near
> cache
> > > should not
> > > always enable .NET near cache - some users may have mixed clusters,
> etc.
> > >
> > > Therefore I'm adding NearCacheConfiguration#platformNearCacheEnabled
> > > boolean flag.
> > > Are there any objections or better ideas to configure this behavior?
> > >
> > > Thanks,
> > > Pavel
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-12691
> > >
> >
>

Aleksandr Shapkin

Re: .NET Near Cache - new flag in NearCacheConfiguration.java?

In reply to this post by Pavel Tupitsyn

Pavel,

I think it’s ok to add a new flag.

Though we may change a name to something like #usePlatformCacheIfAvailable

But it may vary depending on the implementation, i.e.

should we throw an error if there is no native cache

available for a platform or just ignore the configuration.

вт, 18 февр. 2020 г. в 15:05, Pavel Tupitsyn <[hidden email]>:

--
Alex.

Pavel Tupitsyn

Re: .NET Near Cache - new flag in NearCacheConfiguration.java?

Ilya, Aleksandr,

The flag is called platformNearCacheEnabled, my idea is to have just one
flag for all platforms.

If some platform is present on the given node (.NET, C++) and it supports
native near caching,
then the flag is honored, otherwise it is ignored. We should not throw
exceptions,
because mixed clusters are possible (.NET nodes along with Java nodes).

> Why would enabling it affect mixed clusters?
Initially the idea was to enable .NET near cache whenever
NearCacheConfiguration is present.
If we assume .NET-only clusters, it makes sense: existing code will work
faster automatically.

Mixed cluster is just one of the possible use cases when this may be a bad
idea,
e.g. users have enabled near caching to speed up their Java-based
computations,
and .NET nodes are used for something else, wasting memory on near caching
unnecessarily.

On Tue, Feb 18, 2020 at 4:30 PM Aleksandr Shapkin <[hidden email]> wrote:

> Pavel,
>
>
>
> I think it’s ok to add a new flag.
>
> Though we may change a name to something like #usePlatformCacheIfAvailable
>
>
>
> But it may vary depending on the implementation, i.e.
>
> should we throw an error if there is no native cache
>
> available for a platform or just ignore the configuration.
>
> вт, 18 февр. 2020 г. в 15:05, Pavel Tupitsyn <[hidden email]>:
>
> > Igor,
> >
> > The problem is - we need to pass this flag around the cluster for Server
> > Near Caches,
> > so that .NET near caches are started accordingly.
> >
> > There are 2 kinds of Near Caches:
> > - On client nodes: created on every client node separately by calling
> > ignite.CreateNearCache
> > - On server nodes: created on all server nodes if
> > CacheConfiguration.NearCacheConfiguration is set
> >
> > When user says ignite.CreateCache(new CacheConfiguration
> > {NearCacheConfiguration = ...}),
> > the whole config is sent to all server nodes, and .NET-specific flag has
> to
> > be included somehow.
> >
> > On Tue, Feb 18, 2020 at 2:59 PM Igor Sapego <[hidden email]> wrote:
> >
> > > Do you suggest to introduce it in general configuration? Why not
> > introduce
> > > it only on platform side? Is there any .NET-specific configuration?
> > >
> > > Best Regards,
> > > Igor
> > >
> > >
> > > On Tue, Feb 18, 2020 at 1:10 AM Pavel Tupitsyn <[hidden email]>
> > > wrote:
> > >
> > > > Igniters,
> > > >
> > > > I'm working on .NET Near Cache feature [1]
> > > > (storing deserialized cache entries in CLR memory to improve
> > > performance).
> > > >
> > > > Implementation is based on Java near cache, with some callbacks to
> .NET
> > > > side
> > > > for updating and invalidating cached entries.
> > > >
> > > > However, I'd like to make this feature optional: enabling Java near
> > cache
> > > > should not
> > > > always enable .NET near cache - some users may have mixed clusters,
> > etc.
> > > >
> > > > Therefore I'm adding NearCacheConfiguration#platformNearCacheEnabled
> > > > boolean flag.
> > > > Are there any objections or better ideas to configure this behavior?
> > > >
> > > > Thanks,
> > > > Pavel
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IGNITE-12691
> > > >
> > >
> >
>
>
> --
> Alex.
>

Ilya Kasnacheev

Re: .NET Near Cache - new flag in NearCacheConfiguration.java?

Hello!

Why would it waste additional memory? All nodes are also Java nodes so they
will start near caches all right.

Don't we have near cache start-up on per-node basis for clients, anyway?

I'm not convinced why we need this flag. I have to admit my understanding
of near caches is limited.

Regards,
--
Ilya Kasnacheev

вт, 18 февр. 2020 г. в 16:56, Pavel Tupitsyn <[hidden email]>:

> Ilya, Aleksandr,
>
> The flag is called platformNearCacheEnabled, my idea is to have just one
> flag for all platforms.
>
> If some platform is present on the given node (.NET, C++) and it supports
> native near caching,
> then the flag is honored, otherwise it is ignored. We should not throw
> exceptions,
> because mixed clusters are possible (.NET nodes along with Java nodes).
>
> > Why would enabling it affect mixed clusters?
> Initially the idea was to enable .NET near cache whenever
> NearCacheConfiguration is present.
> If we assume .NET-only clusters, it makes sense: existing code will work
> faster automatically.
>
> Mixed cluster is just one of the possible use cases when this may be a bad
> idea,
> e.g. users have enabled near caching to speed up their Java-based
> computations,
> and .NET nodes are used for something else, wasting memory on near caching
> unnecessarily.
>
> On Tue, Feb 18, 2020 at 4:30 PM Aleksandr Shapkin <[hidden email]>
> wrote:
>
> > Pavel,
> >
> >
> >
> > I think it’s ok to add a new flag.
> >
> > Though we may change a name to something like
> #usePlatformCacheIfAvailable
> >
> >
> >
> > But it may vary depending on the implementation, i.e.
> >
> > should we throw an error if there is no native cache
> >
> > available for a platform or just ignore the configuration.
> >
> > вт, 18 февр. 2020 г. в 15:05, Pavel Tupitsyn <[hidden email]>:
> >
> > > Igor,
> > >
> > > The problem is - we need to pass this flag around the cluster for
> Server
> > > Near Caches,
> > > so that .NET near caches are started accordingly.
> > >
> > > There are 2 kinds of Near Caches:
> > > - On client nodes: created on every client node separately by calling
> > > ignite.CreateNearCache
> > > - On server nodes: created on all server nodes if
> > > CacheConfiguration.NearCacheConfiguration is set
> > >
> > > When user says ignite.CreateCache(new CacheConfiguration
> > > {NearCacheConfiguration = ...}),
> > > the whole config is sent to all server nodes, and .NET-specific flag
> has
> > to
> > > be included somehow.
> > >
> > > On Tue, Feb 18, 2020 at 2:59 PM Igor Sapego <[hidden email]>
> wrote:
> > >
> > > > Do you suggest to introduce it in general configuration? Why not
> > > introduce
> > > > it only on platform side? Is there any .NET-specific configuration?
> > > >
> > > > Best Regards,
> > > > Igor
> > > >
> > > >
> > > > On Tue, Feb 18, 2020 at 1:10 AM Pavel Tupitsyn <[hidden email]
> >
> > > > wrote:
> > > >
> > > > > Igniters,
> > > > >
> > > > > I'm working on .NET Near Cache feature [1]
> > > > > (storing deserialized cache entries in CLR memory to improve
> > > > performance).
> > > > >
> > > > > Implementation is based on Java near cache, with some callbacks to
> > .NET
> > > > > side
> > > > > for updating and invalidating cached entries.
> > > > >
> > > > > However, I'd like to make this feature optional: enabling Java near
> > > cache
> > > > > should not
> > > > > always enable .NET near cache - some users may have mixed clusters,
> > > etc.
> > > > >
> > > > > Therefore I'm adding
> NearCacheConfiguration#platformNearCacheEnabled
> > > > > boolean flag.
> > > > > Are there any objections or better ideas to configure this
> behavior?
> > > > >
> > > > > Thanks,
> > > > > Pavel
> > > > >
> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12691
> > > > >
> > > >
> > >
> >
> >
> > --
> > Alex.
> >
>

Pavel Tupitsyn

Re: .NET Near Cache - new flag in NearCacheConfiguration.java?

Ilya, as noted above:

> There are 2 kinds of Near Caches:
> - On client nodes: created per-node by calling ignite.CreateNearCache
> - On server nodes: created on all server nodes if
CacheConfiguration.NearCacheConfiguration is not null
>
> When user says ignite.CreateCache(new CacheConfiguration
{NearCacheConfiguration = ...}),
> the whole config is sent to all server nodes, and .NET-specific flag has
to be included somehow.

On Tue, Feb 18, 2020 at 5:10 PM Ilya Kasnacheev <[hidden email]>
wrote:

> Hello!
>
> Why would it waste additional memory? All nodes are also Java nodes so they
> will start near caches all right.
>
> Don't we have near cache start-up on per-node basis for clients, anyway?
>
> I'm not convinced why we need this flag. I have to admit my understanding
> of near caches is limited.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> вт, 18 февр. 2020 г. в 16:56, Pavel Tupitsyn <[hidden email]>:
>
> > Ilya, Aleksandr,
> >
> > The flag is called platformNearCacheEnabled, my idea is to have just one
> > flag for all platforms.
> >
> > If some platform is present on the given node (.NET, C++) and it supports
> > native near caching,
> > then the flag is honored, otherwise it is ignored. We should not throw
> > exceptions,
> > because mixed clusters are possible (.NET nodes along with Java nodes).
> >
> > > Why would enabling it affect mixed clusters?
> > Initially the idea was to enable .NET near cache whenever
> > NearCacheConfiguration is present.
> > If we assume .NET-only clusters, it makes sense: existing code will work
> > faster automatically.
> >
> > Mixed cluster is just one of the possible use cases when this may be a
> bad
> > idea,
> > e.g. users have enabled near caching to speed up their Java-based
> > computations,
> > and .NET nodes are used for something else, wasting memory on near
> caching
> > unnecessarily.
> >
> > On Tue, Feb 18, 2020 at 4:30 PM Aleksandr Shapkin <[hidden email]>
> > wrote:
> >
> > > Pavel,
> > >
> > >
> > >
> > > I think it’s ok to add a new flag.
> > >
> > > Though we may change a name to something like
> > #usePlatformCacheIfAvailable
> > >
> > >
> > >
> > > But it may vary depending on the implementation, i.e.
> > >
> > > should we throw an error if there is no native cache
> > >
> > > available for a platform or just ignore the configuration.
> > >
> > > вт, 18 февр. 2020 г. в 15:05, Pavel Tupitsyn <[hidden email]>:
> > >
> > > > Igor,
> > > >
> > > > The problem is - we need to pass this flag around the cluster for
> > Server
> > > > Near Caches,
> > > > so that .NET near caches are started accordingly.
> > > >
> > > > There are 2 kinds of Near Caches:
> > > > - On client nodes: created on every client node separately by calling
> > > > ignite.CreateNearCache
> > > > - On server nodes: created on all server nodes if
> > > > CacheConfiguration.NearCacheConfiguration is set
> > > >
> > > > When user says ignite.CreateCache(new CacheConfiguration
> > > > {NearCacheConfiguration = ...}),
> > > > the whole config is sent to all server nodes, and .NET-specific flag
> > has
> > > to
> > > > be included somehow.
> > > >
> > > > On Tue, Feb 18, 2020 at 2:59 PM Igor Sapego <[hidden email]>
> > wrote:
> > > >
> > > > > Do you suggest to introduce it in general configuration? Why not
> > > > introduce
> > > > > it only on platform side? Is there any .NET-specific configuration?
> > > > >
> > > > > Best Regards,
> > > > > Igor
> > > > >
> > > > >
> > > > > On Tue, Feb 18, 2020 at 1:10 AM Pavel Tupitsyn <
> [hidden email]
> > >
> > > > > wrote:
> > > > >
> > > > > > Igniters,
> > > > > >
> > > > > > I'm working on .NET Near Cache feature [1]
> > > > > > (storing deserialized cache entries in CLR memory to improve
> > > > > performance).
> > > > > >
> > > > > > Implementation is based on Java near cache, with some callbacks
> to
> > > .NET
> > > > > > side
> > > > > > for updating and invalidating cached entries.
> > > > > >
> > > > > > However, I'd like to make this feature optional: enabling Java
> near
> > > > cache
> > > > > > should not
> > > > > > always enable .NET near cache - some users may have mixed
> clusters,
> > > > etc.
> > > > > >
> > > > > > Therefore I'm adding
> > NearCacheConfiguration#platformNearCacheEnabled
> > > > > > boolean flag.
> > > > > > Are there any objections or better ideas to configure this
> > behavior?
> > > > > >
> > > > > > Thanks,
> > > > > > Pavel
> > > > > >
> > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12691
> > > > > >
> > > > >
> > > >
> > >
> > >
> > > --
> > > Alex.
> > >
> >
>

Ilya Kasnacheev

Re: .NET Near Cache - new flag in NearCacheConfiguration.java?

Hello!

I'm not sure that we can/want to create near cache on a subset of server
nodes.

If this is indeed possible, I would decouple that from .Net and introduce
as a separate feature.

Ideally we should be able to start or stop near cache on any node, server
or client, and this should be the same cache for all platforms (including
Java). WDYT?

Regards,
--
Ilya Kasnacheev

вт, 18 февр. 2020 г. в 17:31, Pavel Tupitsyn <[hidden email]>:

> Ilya, as noted above:
>
> > There are 2 kinds of Near Caches:
> > - On client nodes: created per-node by calling ignite.CreateNearCache
> > - On server nodes: created on all server nodes if
> CacheConfiguration.NearCacheConfiguration is not null
> >
> > When user says ignite.CreateCache(new CacheConfiguration
> {NearCacheConfiguration = ...}),
> > the whole config is sent to all server nodes, and .NET-specific flag has
> to be included somehow.
>
>
>
> On Tue, Feb 18, 2020 at 5:10 PM Ilya Kasnacheev <[hidden email]
> >
> wrote:
>
> > Hello!
> >
> > Why would it waste additional memory? All nodes are also Java nodes so
> they
> > will start near caches all right.
> >
> > Don't we have near cache start-up on per-node basis for clients, anyway?
> >
> > I'm not convinced why we need this flag. I have to admit my understanding
> > of near caches is limited.
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > вт, 18 февр. 2020 г. в 16:56, Pavel Tupitsyn <[hidden email]>:
> >
> > > Ilya, Aleksandr,
> > >
> > > The flag is called platformNearCacheEnabled, my idea is to have just
> one
> > > flag for all platforms.
> > >
> > > If some platform is present on the given node (.NET, C++) and it
> supports
> > > native near caching,
> > > then the flag is honored, otherwise it is ignored. We should not throw
> > > exceptions,
> > > because mixed clusters are possible (.NET nodes along with Java nodes).
> > >
> > > > Why would enabling it affect mixed`clusters?
> > > Initially the idea was to enable .NET near cache whenever
> > > NearCacheConfiguration is present.
> > > If we assume .NET-only clusters, it makes sense: existing code will
> work
> > > faster automatically.
> > >
> > > Mixed cluster is just one of the possible use cases when this may be a
> > bad
> > > idea,
> > > e.g. users have enabled near caching to speed up their Java-based
> > > computations,
> > > and .NET nodes are used for something else, wasting memory on near
> > caching
> > > unnecessarily.
> > >
> > > On ue, Feb 18, 2020 at 4:30 PM Aleksandr Shapkin <[hidden email]>
> > > wrote:
> > >
> > > > Pavel,
> > > >
> > > >
> > > >
> > > > I think it’s ok to add a new flag.
> > > >
> > > > Though we may change a name to something like
> > > #usePlatformCacheIfAvailable
> > > >
> > > >
> > > >
> > > > But it may vary depending on the implementation, i.e.
> > > >
> > > > should we throw an error if there is no native cache
> > > >
> > > > available for a platform or just ignore the configuration.
> > > >
> > > > вт, 18 февр. 2020 г. в 15:05, Pavel Tupitsyn <[hidden email]>:
> > > >
> > > > > Igor,
> > > > >
> > > > > The problem is - we need to pass this flag around the cluster`for
> > > Server
> > > > > Near Caches,
> > > > > so that .NET near caches are started accordingly.
> > > > >
> > > > > There are 2 kinds of Near Caches:
> > > > > - On client nodes: created on every client node separately by
> calling
> > > > > ignite.CreateNearCache
> > > > > - On server nodes: created on all server nodes if
> > > > > CacheConfiguration.NearCacheConfiguration is set
> > > > >
> > > > > When user says ignite.CreateCache(new CacheConfiguration
> > > > > {NearCacheConfiguration = ...}),
> > > > > the whole config is sent 4o all server nodes, and .NET-specific
> flag
> > > has
> > > > to
> > > > > be included somehow.
> > > > >
> > > > > On Tue, Feb 18, 2020 at 2:59 PM Igor Sapego <[hidden email]>
> > > wrote:
> > > > >
> > > > > > Do you suggest to introduce it in general configuration? Why not
> > > > > introduce
> > > > > > it only on platform side? Is there any .NET-specific
> configuration?
> > > > > >
> > > > > > Best Regards,
> > > > > > Igo2
> > > > > >
> > > > > >
> > > > > > On Tue, Feb 18, 2020 at 1:10 AM Pavel Tupitsyn <
> > [hidden email]
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Igniters,
> > > > > > >
> > > > > > > I'm working on .NET Near Cache feature [1]
> > > > > > > (storing deserialized cache entries in CLR memory to improve
> > > > > > performance).
> > > > > > >
> > > > > > > Implementation is based on Java near cache, with some callbacks
> > to
> > > > .NET
> > > > > > > side
> > > > > > > for updating and invalidating cached entries.

~ > > > > > >

> > > > > > > However, I'd like to make this feature optional: enabling Java
> > near
> > > > > cache
> > > > > > > should not
> > > > > > > always enable .NET near cache - some users may have mixed
> > clusters,
> > > > > etc.
> > > > > > >
> > > > > > > Therefore I'm adding
> > > NearCacheConfiguration#platformNearCacheEnabled
> > > > > > > boolean flag.
> > > > > > > Are there any objections or better ideas to configure this
> > > behavior?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Pavel
> > > > > > >
> > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12691
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Alex.
> > > >
> > >
> >
>

Pavel Tupitsyn

Re: .NET Near Cache - new flag in NearCacheConfiguration.java?

Ilya, looks like there is a misunderstanding.

We don't start near cache on a subset of server nodes. Let me explain again.
Consider the following code, executed on any node (client or server - does
not matter):

ignite.createCache(new CacheConfiguration("c").setNearConfiguration(new
NearCacheConfiguration()));

As a result, cache "c" is started on all server nodes, and corresponding
near cache is started on all server nodes,
with the same config, eviction policy, and so on.

Now, what do we do with .NET near cache in this situation?
1. Start .NET near cache as well on server nodes, always.
2. Introduce a config flag, and start .NET near cache only when the flag is
set

The problem with option (1) is that it changes the behavior of existing
applications,
increasing overhead for cache operations and increasing memory usage.

As a user, I don't want some new features to be force-enabled on me
silently in a new product version.
That is why an opt-in config flag is required.

Does this make sense?

On Tue, Feb 18, 2020 at 6:42 PM Ilya Kasnacheev <[hidden email]>
wrote:

> Hello!
>
> I'm not sure that we can/want to create near cache on a subset of server
> nodes.
>
> If this is indeed possible, I would decouple that from .Net and introduce
> as a separate feature.
>
> Ideally we should be able to start or stop near cache on any node, server
> or client, and this should be the same cache for all platforms (including
> Java). WDYT?
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> вт, 18 февр. 2020 г. в 17:31, Pavel Tupitsyn <[hidden email]>:
>
> > Ilya, as noted above:
> >
> > > There are 2 kinds of Near Caches:
> > > - On client nodes: created per-node by calling ignite.CreateNearCache
> > > - On server nodes: created on all server nodes if
> > CacheConfiguration.NearCacheConfiguration is not null
> > >
> > > When user says ignite.CreateCache(new CacheConfiguration
> > {NearCacheConfiguration = ...}),
> > > the whole config is sent to all server nodes, and .NET-specific flag
> has
> > to be included somehow.
> >
> >
> >
> > On Tue, Feb 18, 2020 at 5:10 PM Ilya Kasnacheev <
> [hidden email]
> > >
> > wrote:
> >
> > > Hello!
> > >
> > > Why would it waste additional memory? All nodes are also Java nodes so
> > they
> > > will start near caches all right.
> > >
> > > Don't we have near cache start-up on per-node basis for clients,
> anyway?
> > >
> > > I'm not convinced why we need this flag. I have to admit my
> understanding
> > > of near caches is limited.
> > >
> > > Regards,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > вт, 18 февр. 2020 г. в 16:56, Pavel Tupitsyn <[hidden email]>:
> > >
> > > > Ilya, Aleksandr,
> > > >
> > > > The flag is called platformNearCacheEnabled, my idea is to have just
> > one
> > > > flag for all platforms.
> > > >
> > > > If some platform is present on the given node (.NET, C++) and it
> > supports
> > > > native near caching,
> > > > then the flag is honored, otherwise it is ignored. We should not
> throw
> > > > exceptions,
> > > > because mixed clusters are possible (.NET nodes along with Java
> nodes).
> > > >
> > > > > Why would enabling it affect mixed`clusters?
> > > > Initially the idea was to enable .NET near cache whenever
> > > > NearCacheConfiguration is present.
> > > > If we assume .NET-only clusters, it makes sense: existing code will
> > work
> > > > faster automatically.
> > > >
> > > > Mixed cluster is just one of the possible use cases when this may be
> a
> > > bad
> > > > idea,
> > > > e.g. users have enabled near caching to speed up their Java-based
> > > > computations,
> > > > and .NET nodes are used for something else, wasting memory on near
> > > caching
> > > > unnecessarily.
> > > >
> > > > On ue, Feb 18, 2020 at 4:30 PM Aleksandr Shapkin <[hidden email]
> >
> > > > wrote:
> > > >
> > > > > Pavel,
> > > > >
> > > > >
> > > > >
> > > > > I think it’s ok to add a new flag.
> > > > >
> > > > > Though we may change a name to something like
> > > > #usePlatformCacheIfAvailable
> > > > >
> > > > >
> > > > >
> > > > > But it may vary depending on the implementation, i.e.
> > > > >
> > > > > should we throw an error if there is no native cache
> > > > >
> > > > > available for a platform or just ignore the configuration.
> > > > >
> > > > > вт, 18 февр. 2020 г. в 15:05, Pavel Tupitsyn <[hidden email]
> >:
> > > > >
> > > > > > Igor,
> > > > > >
> > > > > > The problem is - we need to pass this flag around the cluster`for
> > > > Server
> > > > > > Near Caches,
> > > > > > so that .NET near caches are started accordingly.
> > > > > >
> > > > > > There are 2 kinds of Near Caches:
> > > > > > - On client nodes: created on every client node separately by
> > calling
> > > > > > ignite.CreateNearCache
> > > > > > - On server nodes: created on all server nodes if
> > > > > > CacheConfiguration.NearCacheConfiguration is set
> > > > > >
> > > > > > When user says ignite.CreateCache(new CacheConfiguration
> > > > > > {NearCacheConfiguration = ...}),
> > > > > > the whole config is sent 4o all server nodes, and .NET-specific
> > flag
> > > > has
> > > > > to
> > > > > > be included somehow.
> > > > > >
> > > > > > On Tue, Feb 18, 2020 at 2:59 PM Igor Sapego <[hidden email]>
> > > > wrote:
> > > > > >
> > > > > > > Do you suggest to introduce it in general configuration? Why
> not
> > > > > > introduce
> > > > > > > it only on platform side? Is there any .NET-specific
> > configuration?
> > > > > > >
> > > > > > > Best Regards,
> > > > > > > Igo2
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Feb 18, 2020 at 1:10 AM Pavel Tupitsyn <
> > > [hidden email]
> > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Igniters,
> > > > > > > >
> > > > > > > > I'm working on .NET Near Cache feature [1]
> > > > > > > > (storing deserialized cache entries in CLR memory to improve
> > > > > > > performance).
> > > > > > > >
> > > > > > > > Implementation is based on Java near cache, with some
> callbacks
> > > to
> > > > > .NET
> > > > > > > > side
> > > > > > > > for updating and invalidating cached entries.
> ~ > > > > > >
> > > > > > > > However, I'd like to make this feature optional: enabling
> Java
> > > near
> > > > > > cache
> > > > > > > > should not
> > > > > > > > always enable .NET near cache - some users may have mixed
> > > clusters,
> > > > > > etc.
> > > > > > > >
> > > > > > > > Therefore I'm adding
> > > > NearCacheConfiguration#platformNearCacheEnabled
> > > > > > > > boolean flag.
> > > > > > > > Are there any objections or better ideas to configure this
> > > > behavior?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Pavel
> > > > > > > >
> > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12691
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Alex.
> > > > >
> > > >
> > >
> >
>

Sergey Stronchinskiy

Re: .NET Near Cache - new flag in NearCacheConfiguration.java?

Hi, Pavel

This feature looks very interesting, can you add me as a reviewer please.
Or maybe I can contribute somehow?

Thanks,

Sergey.

On Tue, 18 Feb 2020 at 20:46, Pavel Tupitsyn <[hidden email]> wrote:

> Ilya, looks like there is a misunderstanding.
>
> We don't start near cache on a subset of server nodes. Let me explain
> again.
> Consider the following code, executed on any node (client or server - does
> not matter):
>
> ignite.createCache(new CacheConfiguration("c").setNearConfiguration(new
> NearCacheConfiguration()));
>
> As a result, cache "c" is started on all server nodes, and corresponding
> near cache is started on all server nodes,
> with the same config, eviction policy, and so on.
>
> Now, what do we do with .NET near cache in this situation?
> 1. Start .NET near cache as well on server nodes, always.
> 2. Introduce a config flag, and start .NET near cache only when the flag is
> set
>
> The problem with option (1) is that it changes the behavior of existing
> applications,
> increasing overhead for cache operations and increasing memory usage.
>
> As a user, I don't want some new features to be force-enabled on me
> silently in a new product version.
> That is why an opt-in config flag is required.
>
> Does this make sense?
>
> On Tue, Feb 18, 2020 at 6:42 PM Ilya Kasnacheev <[hidden email]
> >
> wrote:
>
> > Hello!
> >
> > I'm not sure that we can/want to create near cache on a subset of server
> > nodes.
> >
> > If this is indeed possible, I would decouple that from .Net and introduce
> > as a separate feature.
> >
> > Ideally we should be able to start or stop near cache on any node, server
> > or client, and this should be the same cache for all platforms (including
> > Java). WDYT?
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > вт, 18 февр. 2020 г. в 17:31, Pavel Tupitsyn <[hidden email]>:
> >
> > > Ilya, as noted above:
> > >
> > > > There are 2 kinds of Near Caches:
> > > > - On client nodes: created per-node by calling ignite.CreateNearCache
> > > > - On server nodes: created on all server nodes if
> > > CacheConfiguration.NearCacheConfiguration is not null
> > > >
> > > > When user says ignite.CreateCache(new CacheConfiguration
> > > {NearCacheConfiguration = ...}),
> > > > the whole config is sent to all server nodes, and .NET-specific flag
> > has
> > > to be included somehow.
> > >
> > >
> > >
> > > On Tue, Feb 18, 2020 at 5:10 PM Ilya Kasnacheev <
> > [hidden email]
> > > >
> > > wrote:
> > >
> > > > Hello!
> > > >
> > > > Why would it waste additional memory? All nodes are also Java nodes
> so
> > > they
> > > > will start near caches all right.
> > > >
> > > > Don't we have near cache start-up on per-node basis for clients,
> > anyway?
> > > >
> > > > I'm not convinced why we need this flag. I have to admit my
> > understanding
> > > > of near caches is limited.
> > > >
> > > > Regards,
> > > > --
> > > > Ilya Kasnacheev
> > > >
> > > >
> > > > вт, 18 февр. 2020 г. в 16:56, Pavel Tupitsyn <[hidden email]>:
> > > >
> > > > > Ilya, Aleksandr,
> > > > >
> > > > > The flag is called platformNearCacheEnabled, my idea is to have
> just
> > > one
> > > > > flag for all platforms.
> > > > >
> > > > > If some platform is present on the given node (.NET, C++) and it
> > > supports
> > > > > native near caching,
> > > > > then the flag is honored, otherwise it is ignored. We should not
> > throw
> > > > > exceptions,
> > > > > because mixed clusters are possible (.NET nodes along with Java
> > nodes).
> > > > >
> > > > > > Why would enabling it affect mixed`clusters?
> > > > > Initially the idea was to enable .NET near cache whenever
> > > > > NearCacheConfiguration is present.
> > > > > If we assume .NET-only clusters, it makes sense: existing code will
> > > work
> > > > > faster automatically.
> > > > >
> > > > > Mixed cluster is just one of the possible use cases when this may
> be
> > a
> > > > bad
> > > > > idea,
> > > > > e.g. users have enabled near caching to speed up their Java-based
> > > > > computations,
> > > > > and .NET nodes are used for something else, wasting memory on near
> > > > caching
> > > > > unnecessarily.
> > > > >
> > > > > On ue, Feb 18, 2020 at 4:30 PM Aleksandr Shapkin <
> [hidden email]
> > >
> > > > > wrote:
> > > > >
> > > > > > Pavel,
> > > > > >
> > > > > >
> > > > > >
> > > > > > I think it’s ok to add a new flag.
> > > > > >
> > > > > > Though we may change a name to something like
> > > > > #usePlatformCacheIfAvailable
> > > > > >
> > > > > >
> > > > > >
> > > > > > But it may vary depending on the implementation, i.e.
> > > > > >
> > > > > > should we throw an error if there is no native cache
> > > > > >
> > > > > > available for a platform or just ignore the configuration.
> > > > > >
> > > > > > вт, 18 февр. 2020 г. в 15:05, Pavel Tupitsyn <
> [hidden email]
> > >:
> > > > > >
> > > > > > > Igor,
> > > > > > >
> > > > > > > The problem is - we need to pass this flag around the
> cluster`for
> > > > > Server
> > > > > > > Near Caches,
> > > > > > > so that .NET near caches are started accordingly.
> > > > > > >
> > > > > > > There are 2 kinds of Near Caches:
> > > > > > > - On client nodes: created on every client node separately by
> > > calling
> > > > > > > ignite.CreateNearCache
> > > > > > > - On server nodes: created on all server nodes if
> > > > > > > CacheConfiguration.NearCacheConfiguration is set
> > > > > > >
> > > > > > > When user says ignite.CreateCache(new CacheConfiguration
> > > > > > > {NearCacheConfiguration = ...}),
> > > > > > > the whole config is sent 4o all server nodes, and .NET-specific
> > > flag
> > > > > has
> > > > > > to
> > > > > > > be included somehow.
> > > > > > >
> > > > > > > On Tue, Feb 18, 2020 at 2:59 PM Igor Sapego <
> [hidden email]>
> > > > > wrote:
> > > > > > >
> > > > > > > > Do you suggest to introduce it in general configuration? Why
> > not
> > > > > > > introduce
> > > > > > > > it only on platform side? Is there any .NET-specific
> > > configuration?
> > > > > > > >
> > > > > > > > Best Regards,
> > > > > > > > Igo2
> > > > > > > >
> > > > > > > >
> > > > > > > > On Tue, Feb 18, 2020 at 1:10 AM Pavel Tupitsyn <
> > > > [hidden email]
> > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Igniters,
> > > > > > > > >
> > > > > > > > > I'm working on .NET Near Cache feature [1]
> > > > > > > > > (storing deserialized cache entries in CLR memory to
> improve
> > > > > > > > performance).
> > > > > > > > >
> > > > > > > > > Implementation is based on Java near cache, with some
> > callbacks
> > > > to
> > > > > > .NET
> > > > > > > > > side
> > > > > > > > > for updating and invalidating cached entries.
> > ~ > > > > > >
> > > > > > > > > However, I'd like to make this feature optional: enabling
> > Java
> > > > near
> > > > > > > cache
> > > > > > > > > should not
> > > > > > > > > always enable .NET near cache - some users may have mixed
> > > > clusters,
> > > > > > > etc.
> > > > > > > > >
> > > > > > > > > Therefore I'm adding
> > > > > NearCacheConfiguration#platformNearCacheEnabled
> > > > > > > > > boolean flag.
> > > > > > > > > Are there any objections or better ideas to configure this
> > > > > behavior?
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Pavel
> > > > > > > > >
> > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12691
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Alex.
> > > > > >
> > > > >
> > > >
> > >
> >
>

Ilya Kasnacheev

Re: .NET Near Cache - new flag in NearCacheConfiguration.java?

In reply to this post by Pavel Tupitsyn

Hello!

Can you please describe what is the overhead on server nodes from starting
.Net near cache, as compared to starting "regular" near cache?

Also, why do we start near cache on all server nodes - is it required by
our architecture?

Otherwise, if you add a new feature, which you have to explicitly invoke
from application .Net code, I assume it's OK to consume some resources upon
such invocation. We can't expect new features to consume zero resources.

Regards,
--
Ilya Kasnacheev

вт, 18 февр. 2020 г. в 20:46, Pavel Tupitsyn <[hidden email]>:

Pavel Tupitsyn

Re: .NET Near Cache - new flag in NearCacheConfiguration.java?

Sergey,

Sure, I'll add you as a reviewer.
And a thorough review will be the best contribution. The feature is rather
complex and in the works for quite some time.

On Fri, Feb 21, 2020 at 4:03 PM Ilya Kasnacheev <[hidden email]>
wrote:

> Hello!
>
> Can you please describe what is the overhead on server nodes from starting
> .Net near cache, as compared to starting "regular" near cache?
>
> Also, why do we start near cache on all server nodes - is it required by
> our architecture?
>
> Otherwise, if you add a new feature, which you have to explicitly invoke
> from application .Net code, I assume it's OK to consume some resources upon
> such invocation. We can't expect new features to consume zero resources.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> вт, 18 февр. 2020 г. в 20:46, Pavel Tupitsyn <[hidden email]>:
>
> > Ilya, looks like there is a misunderstanding.
> >
> > We don't start near cache on a subset of server nodes. Let me explain
> > again.
> > Consider the following code, executed on any node (client or server -
> does
> > not matter):
> >
> > ignite.createCache(new CacheConfiguration("c").setNearConfiguration(new
> > NearCacheConfiguration()));
> >
> > As a result, cache "c" is started on all server nodes, and corresponding
> > near cache is started on all server nodes,
> > with the same config, eviction policy, and so on.
> >
> > Now, what do we do with .NET near cache in this situation?
> > 1. Start .NET near cache as well on server nodes, always.
> > 2. Introduce a config flag, and start .NET near cache only when the flag
> is
> > set
> >
> > The problem with option (1) is that it changes the behavior of existing
> > applications,
> > increasing overhead for cache operations and increasing memory usage.
> >
> > As a user, I don't want some new features to be force-enabled on me
> > silently in a new product version.
> > That is why an opt-in config flag is required.
> >
> > Does this make sense?
> >
> > On Tue, Feb 18, 2020 at 6:42 PM Ilya Kasnacheev <
> [hidden email]
> > >
> > wrote:
> >
> > > Hello!
> > >
> > > I'm not sure that we can/want to create near cache on a subset of
> server
> > > nodes.
> > >
> > > If this is indeed possible, I would decouple that from .Net and
> introduce
> > > as a separate feature.
> > >
> > > Ideally we should be able to start or stop near cache on any node,
> server
> > > or client, and this should be the same cache for all platforms
> (including
> > > Java). WDYT?
> > >
> > > Regards,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > вт, 18 февр. 2020 г. в 17:31, Pavel Tupitsyn <[hidden email]>:
> > >
> > > > Ilya, as noted above:
> > > >
> > > > > There are 2 kinds of Near Caches:
> > > > > - On client nodes: created per-node by calling
> ignite.CreateNearCache
> > > > > - On server nodes: created on all server nodes if
> > > > CacheConfiguration.NearCacheConfiguration is not null
> > > > >
> > > > > When user says ignite.CreateCache(new CacheConfiguration
> > > > {NearCacheConfiguration = ...}),
> > > > > the whole config is sent to all server nodes, and .NET-specific
> flag
> > > has
> > > > to be included somehow.
> > > >
> > > >
> > > >
> > > > On Tue, Feb 18, 2020 at 5:10 PM Ilya Kasnacheev <
> > > [hidden email]
> > > > >
> > > > wrote:
> > > >
> > > > > Hello!
> > > > >
> > > > > Why would it waste additional memory? All nodes are also Java nodes
> > so
> > > > they
> > > > > will start near caches all right.
> > > > >
> > > > > Don't we have near cache start-up on per-node basis for clients,
> > > anyway?
> > > > >
> > > > > I'm not convinced why we need this flag. I have to admit my
> > > understanding
> > > > > of near caches is limited.
> > > > >
> > > > > Regards,
> > > > > --
> > > > > Ilya Kasnacheev
> > > > >
> > > > >
> > > > > вт, 18 февр. 2020 г. в 16:56, Pavel Tupitsyn <[hidden email]
> >:
> > > > >
> > > > > > Ilya, Aleksandr,
> > > > > >
> > > > > > The flag is called platformNearCacheEnabled, my idea is to have
> > just
> > > > one
> > > > > > flag for all platforms.
> > > > > >
> > > > > > If some platform is present on the given node (.NET, C++) and it
> > > > supports
> > > > > > native near caching,
> > > > > > then the flag is honored, otherwise it is ignored. We should not
> > > throw
> > > > > > exceptions,
> > > > > > because mixed clusters are possible (.NET nodes along with Java
> > > nodes).
> > > > > >
> > > > > > > Why would enabling it affect mixed`clusters?
> > > > > > Initially the idea was to enable .NET near cache whenever
> > > > > > NearCacheConfiguration is present.
> > > > > > If we assume .NET-only clusters, it makes sense: existing code
> will
> > > > work
> > > > > > faster automatically.
> > > > > >
> > > > > > Mixed cluster is just one of the possible use cases when this may
> > be
> > > a
> > > > > bad
> > > > > > idea,
> > > > > > e.g. users have enabled near caching to speed up their Java-based
> > > > > > computations,
> > > > > > and .NET nodes are used for something else, wasting memory on
> near
> > > > > caching
> > > > > > unnecessarily.
> > > > > >
> > > > > > On ue, Feb 18, 2020 at 4:30 PM Aleksandr Shapkin <
> > [hidden email]
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Pavel,
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > I think it’s ok to add a new flag.
> > > > > > >
> > > > > > > Though we may change a name to something like
> > > > > > #usePlatformCacheIfAvailable
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > But it may vary depending on the implementation, i.e.
> > > > > > >
> > > > > > > should we throw an error if there is no native cache
> > > > > > >
> > > > > > > available for a platform or just ignore the configuration.
> > > > > > >
> > > > > > > вт, 18 февр. 2020 г. в 15:05, Pavel Tupitsyn <
> > [hidden email]
> > > >:
> > > > > > >
> > > > > > > > Igor,
> > > > > > > >
> > > > > > > > The problem is - we need to pass this flag around the
> > cluster`for
> > > > > > Server
> > > > > > > > Near Caches,
> > > > > > > > so that .NET near caches are started accordingly.
> > > > > > > >
> > > > > > > > There are 2 kinds of Near Caches:
> > > > > > > > - On client nodes: created on every client node separately by
> > > > calling
> > > > > > > > ignite.CreateNearCache
> > > > > > > > - On server nodes: created on all server nodes if
> > > > > > > > CacheConfiguration.NearCacheConfiguration is set
> > > > > > > >
> > > > > > > > When user says ignite.CreateCache(new CacheConfiguration
> > > > > > > > {NearCacheConfiguration = ...}),
> > > > > > > > the whole config is sent 4o all server nodes, and
> .NET-specific
> > > > flag
> > > > > > has
> > > > > > > to
> > > > > > > > be included somehow.
> > > > > > > >
> > > > > > > > On Tue, Feb 18, 2020 at 2:59 PM Igor Sapego <
> > [hidden email]>
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > Do you suggest to introduce it in general configuration?
> Why
> > > not
> > > > > > > > introduce
> > > > > > > > > it only on platform side? Is there any .NET-specific
> > > > configuration?
> > > > > > > > >
> > > > > > > > > Best Regards,
> > > > > > > > > Igo2
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Tue, Feb 18, 2020 at 1:10 AM Pavel Tupitsyn <
> > > > > [hidden email]
> > > > > > >
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Igniters,
> > > > > > > > > >
> > > > > > > > > > I'm working on .NET Near Cache feature [1]
> > > > > > > > > > (storing deserialized cache entries in CLR memory to
> > improve
> > > > > > > > > performance).
> > > > > > > > > >
> > > > > > > > > > Implementation is based on Java near cache, with some
> > > callbacks
> > > > > to
> > > > > > > .NET
> > > > > > > > > > side
> > > > > > > > > > for updating and invalidating cached entries.
> > > ~ > > > > > >
> > > > > > > > > > However, I'd like to make this feature optional: enabling
> > > Java
> > > > > near
> > > > > > > > cache
> > > > > > > > > > should not
> > > > > > > > > > always enable .NET near cache - some users may have mixed
> > > > > clusters,
> > > > > > > > etc.
> > > > > > > > > >
> > > > > > > > > > Therefore I'm adding
> > > > > > NearCacheConfiguration#platformNearCacheEnabled
> > > > > > > > > > boolean flag.
> > > > > > > > > > Are there any objections or better ideas to configure
> this
> > > > > > behavior?
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > Pavel
> > > > > > > > > >
> > > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12691
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Alex.
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Pavel Tupitsyn

Re: .NET Near Cache - new flag in NearCacheConfiguration.java?

Ilya, when .NET Near Cache is enabled, Java invokes callbacks to update or
evict near cache data in .NET, hence the overhead.

I've created a PR, everybody is welcome to do a preliminary review (just
ignore the minor things for now).
Core functionality is ready, put/get is supported. I'm working on other
cache operations.

[1] https://github.com/apache/ignite/pull/7468

On Sat, Feb 22, 2020 at 7:07 PM Pavel Tupitsyn <[hidden email]> wrote:

> Sergey,
>
> Sure, I'll add you as a reviewer.
> And a thorough review will be the best contribution. The feature is rather
> complex and in the works for quite some time.
>
>
> On Fri, Feb 21, 2020 at 4:03 PM Ilya Kasnacheev <[hidden email]>
> wrote:
>
>> Hello!
>>
>> Can you please describe what is the overhead on server nodes from starting
>> .Net near cache, as compared to starting "regular" near cache?
>>
>> Also, why do we start near cache on all server nodes - is it required by
>> our architecture?
>>
>> Otherwise, if you add a new feature, which you have to explicitly invoke
>> from application .Net code, I assume it's OK to consume some resources
>> upon
>> such invocation. We can't expect new features to consume zero resources.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> вт, 18 февр. 2020 г. в 20:46, Pavel Tupitsyn <[hidden email]>:
>>
>> > Ilya, looks like there is a misunderstanding.
>> >
>> > We don't start near cache on a subset of server nodes. Let me explain
>> > again.
>> > Consider the following code, executed on any node (client or server -
>> does
>> > not matter):
>> >
>> > ignite.createCache(new CacheConfiguration("c").setNearConfiguration(new
>> > NearCacheConfiguration()));
>> >
>> > As a result, cache "c" is started on all server nodes, and corresponding
>> > near cache is started on all server nodes,
>> > with the same config, eviction policy, and so on.
>> >
>> > Now, what do we do with .NET near cache in this situation?
>> > 1. Start .NET near cache as well on server nodes, always.
>> > 2. Introduce a config flag, and start .NET near cache only when the
>> flag is
>> > set
>> >
>> > The problem with option (1) is that it changes the behavior of existing
>> > applications,
>> > increasing overhead for cache operations and increasing memory usage.
>> >
>> > As a user, I don't want some new features to be force-enabled on me
>> > silently in a new product version.
>> > That is why an opt-in config flag is required.
>> >
>> > Does this make sense?
>> >
>> > On Tue, Feb 18, 2020 at 6:42 PM Ilya Kasnacheev <
>> [hidden email]
>> > >
>> > wrote:
>> >
>> > > Hello!
>> > >
>> > > I'm not sure that we can/want to create near cache on a subset of
>> server
>> > > nodes.
>> > >
>> > > If this is indeed possible, I would decouple that from .Net and
>> introduce
>> > > as a separate feature.
>> > >
>> > > Ideally we should be able to start or stop near cache on any node,
>> server
>> > > or client, and this should be the same cache for all platforms
>> (including
>> > > Java). WDYT?
>> > >
>> > > Regards,
>> > > --
>> > > Ilya Kasnacheev
>> > >
>> > >
>> > > вт, 18 февр. 2020 г. в 17:31, Pavel Tupitsyn <[hidden email]>:
>> > >
>> > > > Ilya, as noted above:
>> > > >
>> > > > > There are 2 kinds of Near Caches:
>> > > > > - On client nodes: created per-node by calling
>> ignite.CreateNearCache
>> > > > > - On server nodes: created on all server nodes if
>> > > > CacheConfiguration.NearCacheConfiguration is not null
>> > > > >
>> > > > > When user says ignite.CreateCache(new CacheConfiguration
>> > > > {NearCacheConfiguration = ...}),
>> > > > > the whole config is sent to all server nodes, and .NET-specific
>> flag
>> > > has
>> > > > to be included somehow.
>> > > >
>> > > >
>> > > >
>> > > > On Tue, Feb 18, 2020 at 5:10 PM Ilya Kasnacheev <
>> > > [hidden email]
>> > > > >
>> > > > wrote:
>> > > >
>> > > > > Hello!
>> > > > >
>> > > > > Why would it waste additional memory? All nodes are also Java
>> nodes
>> > so
>> > > > they
>> > > > > will start near caches all right.
>> > > > >
>> > > > > Don't we have near cache start-up on per-node basis for clients,
>> > > anyway?
>> > > > >
>> > > > > I'm not convinced why we need this flag. I have to admit my
>> > > understanding
>> > > > > of near caches is limited.
>> > > > >
>> > > > > Regards,
>> > > > > --
>> > > > > Ilya Kasnacheev
>> > > > >
>> > > > >
>> > > > > вт, 18 февр. 2020 г. в 16:56, Pavel Tupitsyn <
>> [hidden email]>:
>> > > > >
>> > > > > > Ilya, Aleksandr,
>> > > > > >
>> > > > > > The flag is called platformNearCacheEnabled, my idea is to have
>> > just
>> > > > one
>> > > > > > flag for all platforms.
>> > > > > >
>> > > > > > If some platform is present on the given node (.NET, C++) and it
>> > > > supports
>> > > > > > native near caching,
>> > > > > > then the flag is honored, otherwise it is ignored. We should not
>> > > throw
>> > > > > > exceptions,
>> > > > > > because mixed clusters are possible (.NET nodes along with Java
>> > > nodes).
>> > > > > >
>> > > > > > > Why would enabling it affect mixed`clusters?
>> > > > > > Initially the idea was to enable .NET near cache whenever
>> > > > > > NearCacheConfiguration is present.
>> > > > > > If we assume .NET-only clusters, it makes sense: existing code
>> will
>> > > > work
>> > > > > > faster automatically.
>> > > > > >
>> > > > > > Mixed cluster is just one of the possible use cases when this
>> may
>> > be
>> > > a
>> > > > > bad
>> > > > > > idea,
>> > > > > > e.g. users have enabled near caching to speed up their
>> Java-based
>> > > > > > computations,
>> > > > > > and .NET nodes are used for something else, wasting memory on
>> near
>> > > > > caching
>> > > > > > unnecessarily.
>> > > > > >
>> > > > > > On ue, Feb 18, 2020 at 4:30 PM Aleksandr Shapkin <
>> > [hidden email]
>> > > >
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Pavel,
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > I think it’s ok to add a new flag.
>> > > > > > >
>> > > > > > > Though we may change a name to something like
>> > > > > > #usePlatformCacheIfAvailable
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > But it may vary depending on the implementation, i.e.
>> > > > > > >
>> > > > > > > should we throw an error if there is no native cache
>> > > > > > >
>> > > > > > > available for a platform or just ignore the configuration.
>> > > > > > >
>> > > > > > > вт, 18 февр. 2020 г. в 15:05, Pavel Tupitsyn <
>> > [hidden email]
>> > > >:
>> > > > > > >
>> > > > > > > > Igor,
>> > > > > > > >
>> > > > > > > > The problem is - we need to pass this flag around the
>> > cluster`for
>> > > > > > Server
>> > > > > > > > Near Caches,
>> > > > > > > > so that .NET near caches are started accordingly.
>> > > > > > > >
>> > > > > > > > There are 2 kinds of Near Caches:
>> > > > > > > > - On client nodes: created on every client node separately
>> by
>> > > > calling
>> > > > > > > > ignite.CreateNearCache
>> > > > > > > > - On server nodes: created on all server nodes if
>> > > > > > > > CacheConfiguration.NearCacheConfiguration is set
>> > > > > > > >
>> > > > > > > > When user says ignite.CreateCache(new CacheConfiguration
>> > > > > > > > {NearCacheConfiguration = ...}),
>> > > > > > > > the whole config is sent 4o all server nodes, and
>> .NET-specific
>> > > > flag
>> > > > > > has
>> > > > > > > to
>> > > > > > > > be included somehow.
>> > > > > > > >
>> > > > > > > > On Tue, Feb 18, 2020 at 2:59 PM Igor Sapego <
>> > [hidden email]>
>> > > > > > wrote:
>> > > > > > > >
>> > > > > > > > > Do you suggest to introduce it in general configuration?
>> Why
>> > > not
>> > > > > > > > introduce
>> > > > > > > > > it only on platform side? Is there any .NET-specific
>> > > > configuration?
>> > > > > > > > >
>> > > > > > > > > Best Regards,
>> > > > > > > > > Igo2
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > On Tue, Feb 18, 2020 at 1:10 AM Pavel Tupitsyn <
>> > > > > [hidden email]
>> > > > > > >
>> > > > > > > > > wrote:
>> > > > > > > > >
>> > > > > > > > > > Igniters,
>> > > > > > > > > >
>> > > > > > > > > > I'm working on .NET Near Cache feature [1]
>> > > > > > > > > > (storing deserialized cache entries in CLR memory to
>> > improve
>> > > > > > > > > performance).
>> > > > > > > > > >
>> > > > > > > > > > Implementation is based on Java near cache, with some
>> > > callbacks
>> > > > > to
>> > > > > > > .NET
>> > > > > > > > > > side
>> > > > > > > > > > for updating and invalidating cached entries.
>> > > ~ > > > > > >
>> > > > > > > > > > However, I'd like to make this feature optional:
>> enabling
>> > > Java
>> > > > > near
>> > > > > > > > cache
>> > > > > > > > > > should not
>> > > > > > > > > > always enable .NET near cache - some users may have
>> mixed
>> > > > > clusters,
>> > > > > > > > etc.
>> > > > > > > > > >
>> > > > > > > > > > Therefore I'm adding
>> > > > > > NearCacheConfiguration#platformNearCacheEnabled
>> > > > > > > > > > boolean flag.
>> > > > > > > > > > Are there any objections or better ideas to configure
>> this
>> > > > > > behavior?
>> > > > > > > > > >
>> > > > > > > > > > Thanks,
>> > > > > > > > > > Pavel
>> > > > > > > > > >
>> > > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12691
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > --
>> > > > > > > Alex.
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>