Question: local cache on client nodes

[Sergey Chugunov]

Hello Igniters,

Participating in big effort of reworking cache storage structures
(IGNITE-3477 [1]) I came across a test that looks strange to me:
*CacheStopAndDestroySelfTest::testLocalClose*.

It is very simple: it starts two server nodes and one client node (with
forceServerMode flag set to true), then creates local cache on one node,
then adds some values to it on ALL nodes including client.

It looks very confusing for me as I thought that client nodes aren't
supposed to consume a lot of resources as they may be started on small
machines, offheap PageMemory structures aren't allocated on clients at all.
Which leads to test failure, obviously.

Could you please clarify what the local cache is, how it is supposed to be
used?
Should it be started on client nodes as well as on server ones or it is
something wrong with the test itself?


[1] https://issues.apache.org/jira/browse/IGNITE-3477


Thanks,
Sergey Chugunov

[Denis Magda]

It's abdolutely fine to have local caches on client nodes if an application
needs to cache data locally in hashtable like data structure and talk to it
using Ignite APIs.

The upshot is that this kind of cache can be started on any node and we
should keep supporting this capability in 2.0.

--
Denis

On Thursday, March 30, 2017, Sergey Chugunov <[hidden email]>
wrote:

> Hello Igniters,
>
> Participating in big effort of reworking cache storage structures
> (IGNITE-3477 [1]) I came across a test that looks strange to me:
> *CacheStopAndDestroySelfTest::testLocalClose*.
>
> It is very simple: it starts two server nodes and one client node (with
> forceServerMode flag set to true), then creates local cache on one node,
> then adds some values to it on ALL nodes including client.
>
> It looks very confusing for me as I thought that client nodes aren't
> supposed to consume a lot of resources as they may be started on small
> machines, offheap PageMemory structures aren't allocated on clients at all.
> Which leads to test failure, obviously.
>
> Could you please clarify what the local cache is, how it is supposed to be
> used?
> Should it be started on client nodes as well as on server ones or it is
> something wrong with the test itself?
>
>
> [1] https://issues.apache.org/jira/browse/IGNITE-3477
>
>
> Thanks,
> Sergey Chugunov
>

[Sergey Chugunov]

In that case I suggest the following solution based on the same model of
allocating memory on node startup.

So, on client nodes if user provides configuration for MemoryPolicies, we
allocate all memory on node startup.
But if no MemoryPolicy configuration is provided on client node startup, no
default MemoryPolicy is allocated and any attempts to create a local cache
on client node will result in exception.

This solution allows users to have local caches on client nodes (with all
appropriate capacity planning; so we are talking about experienced users);
at the same time it allows starting lightweight client nodes with small
memory footprint.

Very simple yet powerful and easy to implement.

Thoughts?


On Thu, Mar 30, 2017 at 3:00 PM, Denis Magda <[hidden email]> wrote:

> It's abdolutely fine to have local caches on client nodes if an application
> needs to cache data locally in hashtable like data structure and talk to it
> using Ignite APIs.
>
> The upshot is that this kind of cache can be started on any node and we
> should keep supporting this capability in 2.0.
>
> --
> Denis
>
> On Thursday, March 30, 2017, Sergey Chugunov <[hidden email]>
> wrote:
>
> > Hello Igniters,
> >
> > Participating in big effort of reworking cache storage structures
> > (IGNITE-3477 [1]) I came across a test that looks strange to me:
> > *CacheStopAndDestroySelfTest::testLocalClose*.
> >
> > It is very simple: it starts two server nodes and one client node (with
> > forceServerMode flag set to true), then creates local cache on one node,
> > then adds some values to it on ALL nodes including client.
> >
> > It looks very confusing for me as I thought that client nodes aren't
> > supposed to consume a lot of resources as they may be started on small
> > machines, offheap PageMemory structures aren't allocated on clients at
> all.
> > Which leads to test failure, obviously.
> >
> > Could you please clarify what the local cache is, how it is supposed to
> be
> > used?
> > Should it be started on client nodes as well as on server ones or it is
> > something wrong with the test itself?
> >
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-3477
> >
> >
> > Thanks,
> > Sergey Chugunov
> >
>

[dsetrakyan]

Sergey,

I think your suggestion makes sense. Please make sure that the exception
thrown states that client side caches can only be configured on system
startup.


On Wed, Apr 5, 2017 at 5:43 AM, Sergey Chugunov <[hidden email]>
wrote:

> In that case I suggest the following solution based on the same model of
> allocating memory on node startup.
>
> So, on client nodes if user provides configuration for MemoryPolicies, we
> allocate all memory on node startup.
> But if no MemoryPolicy configuration is provided on client node startup, no
> default MemoryPolicy is allocated and any attempts to create a local cache
> on client node will result in exception.
>
> This solution allows users to have local caches on client nodes (with all
> appropriate capacity planning; so we are talking about experienced users);
> at the same time it allows starting lightweight client nodes with small
> memory footprint.
>
> Very simple yet powerful and easy to implement.
>
> Thoughts?
>
>
> On Thu, Mar 30, 2017 at 3:00 PM, Denis Magda <[hidden email]> wrote:
>
> > It's abdolutely fine to have local caches on client nodes if an
> application
> > needs to cache data locally in hashtable like data structure and talk to
> it
> > using Ignite APIs.
> >
> > The upshot is that this kind of cache can be started on any node and we
> > should keep supporting this capability in 2.0.
> >
> > --
> > Denis
> >
> > On Thursday, March 30, 2017, Sergey Chugunov <[hidden email]>
> > wrote:
> >
> > > Hello Igniters,
> > >
> > > Participating in big effort of reworking cache storage structures
> > > (IGNITE-3477 [1]) I came across a test that looks strange to me:
> > > *CacheStopAndDestroySelfTest::testLocalClose*.
> > >
> > > It is very simple: it starts two server nodes and one client node (with
> > > forceServerMode flag set to true), then creates local cache on one
> node,
> > > then adds some values to it on ALL nodes including client.
> > >
> > > It looks very confusing for me as I thought that client nodes aren't
> > > supposed to consume a lot of resources as they may be started on small
> > > machines, offheap PageMemory structures aren't allocated on clients at
> > all.
> > > Which leads to test failure, obviously.
> > >
> > > Could you please clarify what the local cache is, how it is supposed to
> > be
> > > used?
> > > Should it be started on client nodes as well as on server ones or it is
> > > something wrong with the test itself?
> > >
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-3477
> > >
> > >
> > > Thanks,
> > > Sergey Chugunov
> > >
> >
>