Missing ways to get access to Ignite CacheEntry
Locked
12
12
Missing ways to get access to Ignite CacheEntry
 
|
Hi Igniters,
I'm looking for a way to get access to the Ignite CacheEntry. For now this is the ways I found: - Through the queris - Through jsr 107 Cache Iterable - Through jsr 107 Cache itterator - Through IgniteCache::randomEntry() If I remember correctly it was possible to get the CacheEntry from a given key in old version of gridgain community version: GridCacheProjection::entry(K key) GridCacheEntry<K,V> I think it could be a good to introduce this feature at IgniteCache level. Or maybe there is an other way to do it. Thanks, Romain. |
Re: Missing ways to get access to Ignite CacheEntry
|
|
Hi Romain,
As the current documentation of org.apache.ignite.cache.CacheEntry states it's possible to get a reference to CacheEntry and its version only for methods that are executed over local data set stored on a node. Among such methods are invoke & invokeAll and randomEntry. You probably can get a CacheEntry and its non null version when a cache iterator is in use but the version will be 'null', as far as I remember, for those entries that are loaded from remote nodes. Presently Ignite doesn't transfer the version from remote nodes as a part of response by perform reasons. Please elaborate more on your particular use case and what API you would like to add in order to support it. -- Denis On 12/16/2015 12:58 PM, Romain Gilles wrote: > Hi Igniters, > I'm looking for a way to get access to the Ignite CacheEntry. For now this > is the ways I found: > > - Through the queris > - Through jsr 107 Cache Iterable > - Through jsr 107 Cache itterator > - Through IgniteCache::randomEntry() > > If I remember correctly it was possible to get the CacheEntry from a given > key in old version of gridgain community > version: GridCacheProjection::entry(K key) GridCacheEntry<K,V> > I think it could be a good to introduce this feature at IgniteCache level. > Or maybe there is an other way to do it. > > Thanks, > > Romain. > |
Re: Missing ways to get access to Ignite CacheEntry
|
|
Hi Denis,
Thanks for you replay. And sorry to not double check it before. I see that if I want to work with CacheEntry, I need to use the invoke method in order to return the CacheEntry. This is the way I should do it. It sounds like complicated for "just" getting an entry. But if you say this is the way I will do it like that. I was just think that it could be a common use case and therefore provide it as a shortcut. Thanks, Romain. Le mer. 16 déc. 2015 à 11:34, Denis Magda <[hidden email]> a écrit : > Hi Romain, > > As the current documentation of org.apache.ignite.cache.CacheEntry > states it's possible to get a reference to CacheEntry and its version > only for methods that are executed over local data set stored on a node. > Among such methods are invoke & invokeAll and randomEntry. > > You probably can get a CacheEntry and its non null version when a cache > iterator is in use but the version will be 'null', as far as I remember, > for those entries that are loaded from remote nodes. > Presently Ignite doesn't transfer the version from remote nodes as a > part of response by perform reasons. > > Please elaborate more on your particular use case and what API you would > like to add in order to support it. > > -- > Denis > > On 12/16/2015 12:58 PM, Romain Gilles wrote: > > Hi Igniters, > > I'm looking for a way to get access to the Ignite CacheEntry. For now > this > > is the ways I found: > > > > - Through the queris > > - Through jsr 107 Cache Iterable > > - Through jsr 107 Cache itterator > > - Through IgniteCache::randomEntry() > > > > If I remember correctly it was possible to get the CacheEntry from a > given > > key in old version of gridgain community > > version: GridCacheProjection::entry(K key) GridCacheEntry<K,V> > > I think it could be a good to introduce this feature at IgniteCache > level. > > Or maybe there is an other way to do it. > > > > Thanks, > > > > Romain. > > > > |
Re: Missing ways to get access to Ignite CacheEntry
|
|
Romain,
If it's implemented this way for now it doesn't mean that we can't enhance the API for other use cases ;) As I said presently it's supported for limited number of methods mostly by performance reasons. Please go ahead and try invoke/invokeAll and if it doesn't work fine for you we can keep discussing what to do next. Regards, Denis On 12/16/2015 1:53 PM, Romain Gilles wrote: > Hi Denis, > Thanks for you replay. And sorry to not double check it before. I see that > if I want to work with CacheEntry, I need to use the invoke method in order > to return the CacheEntry. This is the way I should do it. It sounds like > complicated for "just" getting an entry. But if you say this is the way I > will do it like that. I was just think that it could be a common use case > and therefore provide it as a shortcut. > > Thanks, > > Romain. > > Le mer. 16 déc. 2015 à 11:34, Denis Magda <[hidden email]> a écrit : > >> Hi Romain, >> >> As the current documentation of org.apache.ignite.cache.CacheEntry >> states it's possible to get a reference to CacheEntry and its version >> only for methods that are executed over local data set stored on a node. >> Among such methods are invoke & invokeAll and randomEntry. >> >> You probably can get a CacheEntry and its non null version when a cache >> iterator is in use but the version will be 'null', as far as I remember, >> for those entries that are loaded from remote nodes. >> Presently Ignite doesn't transfer the version from remote nodes as a >> part of response by perform reasons. >> >> Please elaborate more on your particular use case and what API you would >> like to add in order to support it. >> >> -- >> Denis >> >> On 12/16/2015 12:58 PM, Romain Gilles wrote: >>> Hi Igniters, >>> I'm looking for a way to get access to the Ignite CacheEntry. For now >> this >>> is the ways I found: >>> >>> - Through the queris >>> - Through jsr 107 Cache Iterable >>> - Through jsr 107 Cache itterator >>> - Through IgniteCache::randomEntry() >>> >>> If I remember correctly it was possible to get the CacheEntry from a >> given >>> key in old version of gridgain community >>> version: GridCacheProjection::entry(K key) GridCacheEntry<K,V> >>> I think it could be a good to introduce this feature at IgniteCache >> level. >>> Or maybe there is an other way to do it. >>> >>> Thanks, >>> >>> Romain. >>> >> |
RE: Missing ways to get access to Ignite CacheEntry
|
|
Romain
I suggest you be very careful using the invoke() functionality. As explained in this posting (and the follow-ups) http://mail-archives.apache.org/mod_mbox/ignite-dev/201511.mbox/%3CSNT147-W14D2736E1374450B685721CF000%40phx.gbl%3E the correct usage of the invoke() may be difficult, to put it politely. @Denis Very briefly, the use case is "conditional re-compute". There is a pretty expensive computation that needs to be applied to each entry on demand. The value is computed on first access and it is then cached along with the current version of the cache entry. The value must be recomputed on next access only if the entry has been modified. If anyone has a better idea how to achieve something like that, please do share! One constraint is that the computed value is not serializable. Cheers Andrey > Subject: Re: Missing ways to get access to Ignite CacheEntry > To: [hidden email] > From: [hidden email] > Date: Wed, 16 Dec 2015 16:11:48 +0300 > > Romain, > > If it's implemented this way for now it doesn't mean that we can't > enhance the API for other use cases ;) > As I said presently it's supported for limited number of methods mostly > by performance reasons. > > Please go ahead and try invoke/invokeAll and if it doesn't work fine for > you we can keep discussing what to do next. > > Regards, > Denis > > On 12/16/2015 1:53 PM, Romain Gilles wrote: > > Hi Denis, > > Thanks for you replay. And sorry to not double check it before. I see that > > if I want to work with CacheEntry, I need to use the invoke method in order > > to return the CacheEntry. This is the way I should do it. It sounds like > > complicated for "just" getting an entry. But if you say this is the way I > > will do it like that. I was just think that it could be a common use case > > and therefore provide it as a shortcut. > > > > Thanks, > > > > Romain. > > > > Le mer. 16 déc. 2015 à 11:34, Denis Magda <[hidden email]> a écrit : > > > >> Hi Romain, > >> > >> As the current documentation of org.apache.ignite.cache.CacheEntry > >> states it's possible to get a reference to CacheEntry and its version > >> only for methods that are executed over local data set stored on a node. > >> Among such methods are invoke & invokeAll and randomEntry. > >> > >> You probably can get a CacheEntry and its non null version when a cache > >> iterator is in use but the version will be 'null', as far as I remember, > >> for those entries that are loaded from remote nodes. > >> Presently Ignite doesn't transfer the version from remote nodes as a > >> part of response by perform reasons. > >> > >> Please elaborate more on your particular use case and what API you would > >> like to add in order to support it. > >> > >> -- > >> Denis > >> > >> On 12/16/2015 12:58 PM, Romain Gilles wrote: > >>> Hi Igniters, > >>> I'm looking for a way to get access to the Ignite CacheEntry. For now > >> this > >>> is the ways I found: > >>> > >>> - Through the queris > >>> - Through jsr 107 Cache Iterable > >>> - Through jsr 107 Cache itterator > >>> - Through IgniteCache::randomEntry() > >>> > >>> If I remember correctly it was possible to get the CacheEntry from a > >> given > >>> key in old version of gridgain community > >>> version: GridCacheProjection::entry(K key) GridCacheEntry<K,V> > >>> I think it could be a good to introduce this feature at IgniteCache > >> level. > >>> Or maybe there is an other way to do it. > >>> > >>> Thanks, > >>> > >>> Romain. > >>> > >> > |
Re: Missing ways to get access to Ignite CacheEntry
|
|
Hi Romain,
I'm a bit confused cause initially you saying that /The value is computed on first access and it is then cached /and then you add that /One constraint is that the computed value is not serializable. // / Ignite won't be able to store a value of cache if serialization is not supported for it by some reason. Please clarify here. Probably I'm missing something from your description. -- Denis On 12/16/2015 6:21 PM, Andrey Kornev wrote: > Romain > > I suggest you be very careful using the invoke() functionality. As explained in this posting (and the follow-ups) > http://mail-archives.apache.org/mod_mbox/ignite-dev/201511.mbox/%3CSNT147-W14D2736E1374450B685721CF000%40phx.gbl%3E > the correct usage of the invoke() may be difficult, to put it politely. > > @Denis Very briefly, the use case is "conditional re-compute". There is a pretty expensive computation that needs to be applied to each entry on demand. The value is computed on first access and it is then cached along with the current version of the cache entry. The value must be recomputed on next access only if the entry has been modified. > > If anyone has a better idea how to achieve something like that, please do share! One constraint is that the computed value is not serializable. > > Cheers > Andrey > >> Subject: Re: Missing ways to get access to Ignite CacheEntry >> To: [hidden email] >> From: [hidden email] >> Date: Wed, 16 Dec 2015 16:11:48 +0300 >> >> Romain, >> >> If it's implemented this way for now it doesn't mean that we can't >> enhance the API for other use cases ;) >> As I said presently it's supported for limited number of methods mostly >> by performance reasons. >> >> Please go ahead and try invoke/invokeAll and if it doesn't work fine for >> you we can keep discussing what to do next. >> >> Regards, >> Denis >> >> On 12/16/2015 1:53 PM, Romain Gilles wrote: >>> Hi Denis, >>> Thanks for you replay. And sorry to not double check it before. I see that >>> if I want to work with CacheEntry, I need to use the invoke method in order >>> to return the CacheEntry. This is the way I should do it. It sounds like >>> complicated for "just" getting an entry. But if you say this is the way I >>> will do it like that. I was just think that it could be a common use case >>> and therefore provide it as a shortcut. >>> >>> Thanks, >>> >>> Romain. >>> >>> Le mer. 16 déc. 2015 à 11:34, Denis Magda <[hidden email]> a écrit : >>> >>>> Hi Romain, >>>> >>>> As the current documentation of org.apache.ignite.cache.CacheEntry >>>> states it's possible to get a reference to CacheEntry and its version >>>> only for methods that are executed over local data set stored on a node. >>>> Among such methods are invoke & invokeAll and randomEntry. >>>> >>>> You probably can get a CacheEntry and its non null version when a cache >>>> iterator is in use but the version will be 'null', as far as I remember, >>>> for those entries that are loaded from remote nodes. >>>> Presently Ignite doesn't transfer the version from remote nodes as a >>>> part of response by perform reasons. >>>> >>>> Please elaborate more on your particular use case and what API you would >>>> like to add in order to support it. >>>> >>>> -- >>>> Denis >>>> >>>> On 12/16/2015 12:58 PM, Romain Gilles wrote: >>>>> Hi Igniters, >>>>> I'm looking for a way to get access to the Ignite CacheEntry. For now >>>> this >>>>> is the ways I found: >>>>> >>>>> - Through the queris >>>>> - Through jsr 107 Cache Iterable >>>>> - Through jsr 107 Cache itterator >>>>> - Through IgniteCache::randomEntry() >>>>> >>>>> If I remember correctly it was possible to get the CacheEntry from a >>>> given >>>>> key in old version of gridgain community >>>>> version: GridCacheProjection::entry(K key) GridCacheEntry<K,V> >>>>> I think it could be a good to introduce this feature at IgniteCache >>>> level. >>>>> Or maybe there is an other way to do it. >>>>> >>>>> Thanks, >>>>> >>>>> Romain. >>>>> > |
Re: Missing ways to get access to Ignite CacheEntry
|
|
Denis,In this case I used the term "caching" in general sense, meaning saving the computed value for later. I didn't mean the Ignite cache.Sorry about the confusion.Andrey
_____________________________ From: Denis Magda <[hidden email]> Sent: Wednesday, December 16, 2015 9:48 PM Subject: Re: Missing ways to get access to Ignite CacheEntry To: <[hidden email]> Hi Romain, I'm a bit confused cause initially you saying that /The value is computed on first access and it is then cached /and then you add that /One constraint is that the computed value is not serializable. // / Ignite won't be able to store a value of cache if serialization is not supported for it by some reason. Please clarify here. Probably I'm missing something from your description. -- Denis On 12/16/2015 6:21 PM, Andrey Kornev wrote: > Romain > > I suggest you be very careful using the invoke() functionality. As explained in this posting (and the follow-ups) > http://mail-archives.apache.org/mod_mbox/ignite-dev/201511.mbox/%3CSNT147-W14D2736E1374450B685721CF000%40phx.gbl%3E > the correct usage of the invoke() may be difficult, to put it politely. > > @Denis Very briefly, the use case is "conditional re-compute". There is a pretty expensive computation that needs to be applied to each entry on demand. The value is computed on first access and it is then cached along with the current version of the cache entry. The value must be recomputed on next access only if the entry has been modified. > > If anyone has a better idea how to achieve something like that, please do share! One constraint is that the computed value is not serializable. > > Cheers > Andrey > >> Subject: Re: Missing ways to get access to Ignite CacheEntry >> To: [hidden email] >> From: [hidden email] >> Date: Wed, 16 Dec 2015 16:11:48 +0300 >> >> Romain, >> >> If it's implemented this way for now it doesn't mean that we can't >> enhance the API for other use cases ;) >> As I said presently it's supported for limited number of methods mostly >> by performance reasons. >> >> Please go ahead and try invoke/invokeAll and if it doesn't work fine for >> you we can keep discussing what to do next. >> >> Regards, >> Denis >> >> On 12/16/2015 1:53 PM, Romain Gilles wrote: >>> Hi Denis, >>> Thanks for you replay. And sorry to not double check it before. I see that >>> if I want to work with CacheEntry, I need to use the invoke method in order >>> to return the CacheEntry. This is the way I should do it. It sounds like >>> complicated for "just" getting an entry. But if you say this is the way I >>> will do it like that. I was just think that it could be a common use case >>> and therefore provide it as a shortcut. >>> >>> Thanks, >>> >>> Romain. >>> >>> Le mer. 16 déc. 2015 à 11:34, Denis Magda <[hidden email]> a écrit : >>> >>>> Hi Romain, >>>> >>>> As the current documentation of org.apache.ignite.cache.CacheEntry >>>> states it's possible to get a reference to CacheEntry and its version >>>> only for methods that are executed over local data set stored on a node. >>>> Among such methods are invoke & invokeAll and randomEntry. >>>> >>>> You probably can get a CacheEntry and its non null version when a cache >>>> iterator is in use but the version will be 'null', as far as I remember, >>>> for those entries that are loaded from remote nodes. >>>> Presently Ignite doesn't transfer the version from remote nodes as a >>>> part of response by perform reasons. >>>> >>>> Please elaborate more on your particular use case and what API you would >>>> like to add in order to support it. >>>> >>>> -- >>>> Denis >>>> >>>> On 12/16/2015 12:58 PM, Romain Gilles wrote: >>>>> Hi Igniters, >>>>> I'm looking for a way to get access to the Ignite CacheEntry. For now >>>> this >>>>> is the ways I found: >>>>> >>>>> - Through the queris >>>>> - Through jsr 107 Cache Iterable >>>>> - Through jsr 107 Cache itterator >>>>> - Through IgniteCache::randomEntry() >>>>> >>>>> If I remember correctly it was possible to get the CacheEntry from a >>>> given >>>>> key in old version of gridgain community >>>>> version: GridCacheProjection::entry(K key) GridCacheEntry<K,V> >>>>> I think it could be a good to introduce this feature at IgniteCache >>>> level. >>>>> Or maybe there is an other way to do it. >>>>> >>>>> Thanks, >>>>> >>>>> Romain. >>>>> > |
Re: Missing ways to get access to Ignite CacheEntry
|
|
Hi Andrey,
I got you, thanks for the clarification. So since you're storing a computed value in some local data structure what is stored in the Ignite cache as a value in such a case? There should be something. Why don't you (or can't you) store a version identifier in that value that is located in the Ignite cache? This should work perfectly well for you use-case. Personally I'm against adding CacheEntry to a response of every get like operation that happens over a cache (get, getAll, SQL & Scan queries). This will add extra size to every response and only in rare cases someone will benefit from it. However, what if we add a special configuration parameter to CacheConfiguration that will manage whether CacheEntry has to be passed as a part of a response or not? Igniters, what do you think on this? Regards, Denis On 12/17/2015 9:09 AM, Andrey Kornev wrote: > Denis,In this case I used the term "caching" in general sense, meaning saving the computed value for later. I didn't mean the Ignite cache.Sorry about the confusion.Andrey > _____________________________ > From: Denis Magda <[hidden email]> > Sent: Wednesday, December 16, 2015 9:48 PM > Subject: Re: Missing ways to get access to Ignite CacheEntry > To: <[hidden email]> > > > Hi Romain, > > I'm a bit confused cause initially you saying that > > /The value is computed on first access and it is then cached > > /and then you add that > > /One constraint is that the computed value is not serializable. // > / > > > Ignite won't be able to store a value of cache if serialization is not > supported for it by some reason. > > Please clarify here. Probably I'm missing something from your description. > > -- > Denis > > On 12/16/2015 6:21 PM, Andrey Kornev wrote: > > Romain > > > > I suggest you be very careful using the invoke() functionality. As explained in this posting (and the follow-ups) > > http://mail-archives.apache.org/mod_mbox/ignite-dev/201511.mbox/%3CSNT147-W14D2736E1374450B685721CF000%40phx.gbl%3E > > the correct usage of the invoke() may be difficult, to put it politely. > > > > @Denis Very briefly, the use case is "conditional re-compute". There is a pretty expensive computation that needs to be applied to each entry on demand. The value is computed on first access and it is then cached along with the current version of the cache entry. The value must be recomputed on next access only if the entry has been modified. > > > > If anyone has a better idea how to achieve something like that, please do share! One constraint is that the computed value is not serializable. > > > > Cheers > > Andrey > > > >> Subject: Re: Missing ways to get access to Ignite CacheEntry > >> To: [hidden email] > >> From: [hidden email] > >> Date: Wed, 16 Dec 2015 16:11:48 +0300 > >> > >> Romain, > >> > >> If it's implemented this way for now it doesn't mean that we can't > >> enhance the API for other use cases ;) > >> As I said presently it's supported for limited number of methods mostly > >> by performance reasons. > >> > >> Please go ahead and try invoke/invokeAll and if it doesn't work fine for > >> you we can keep discussing what to do next. > >> > >> Regards, > >> Denis > >> > >> On 12/16/2015 1:53 PM, Romain Gilles wrote: > >>> Hi Denis, > >>> Thanks for you replay. And sorry to not double check it before. I see that > >>> if I want to work with CacheEntry, I need to use the invoke method in order > >>> to return the CacheEntry. This is the way I should do it. It sounds like > >>> complicated for "just" getting an entry. But if you say this is the way I > >>> will do it like that. I was just think that it could be a common use case > >>> and therefore provide it as a shortcut. > >>> > >>> Thanks, > >>> > >>> Romain. > >>> > >>> Le mer. 16 déc. 2015 à 11:34, Denis Magda <[hidden email]> a écrit : > >>> > >>>> Hi Romain, > >>>> > >>>> As the current documentation of org.apache.ignite.cache.CacheEntry > >>>> states it's possible to get a reference to CacheEntry and its version > >>>> only for methods that are executed over local data set stored on a node. > >>>> Among such methods are invoke & invokeAll and randomEntry. > >>>> > >>>> You probably can get a CacheEntry and its non null version when a cache > >>>> iterator is in use but the version will be 'null', as far as I remember, > >>>> for those entries that are loaded from remote nodes. > >>>> Presently Ignite doesn't transfer the version from remote nodes as a > >>>> part of response by perform reasons. > >>>> > >>>> Please elaborate more on your particular use case and what API you would > >>>> like to add in order to support it. > >>>> > >>>> -- > >>>> Denis > >>>> > >>>> On 12/16/2015 12:58 PM, Romain Gilles wrote: > >>>>> Hi Igniters, > >>>>> I'm looking for a way to get access to the Ignite CacheEntry. For now > >>>> this > >>>>> is the ways I found: > >>>>> > >>>>> - Through the queris > >>>>> - Through jsr 107 Cache Iterable > >>>>> - Through jsr 107 Cache itterator > >>>>> - Through IgniteCache::randomEntry() > >>>>> > >>>>> If I remember correctly it was possible to get the CacheEntry from a > >>>> given > >>>>> key in old version of gridgain community > >>>>> version: GridCacheProjection::entry(K key) GridCacheEntry<K,V> > >>>>> I think it could be a good to introduce this feature at IgniteCache > >>>> level. > >>>>> Or maybe there is an other way to do it. > >>>>> > >>>>> Thanks, > >>>>> > >>>>> Romain. > >>>>> > > > > > > > |
RE: Missing ways to get access to Ignite CacheEntry
|
|
Denis,
I'm not suggesting to have every cache read operation return a CacheEntry. The "classical" GridGain used to have a getEntry(key) method that was very useful. The current version of Ignite provides virtually no way of obtaining a cache entry except thru an iterator, or using an EntryProcessor invocation which seems more like a hack. As for the question of what's stored in Ignite cache and the pre-compute, it is really an off-topic. It just came about as we were looking for a way to work around the issue with Ignite query not returning cache entries' version numbers in the result iterator. (There is now a ticket that addresses this issue: https://issues.apache.org/jira/browse/IGNITE-2202 )This is when we realized that there is no longer any way in Ignite to retrieve the CacheEntry by its key. But for the sake of completeness I quickly go over the entire use case, because it seems I wasn't clear enough in the previous post. Ignite cache holds the data entries that are used as the input into some expensive computation. The data in the cache gets updated from time to time, the computation only needs to be re-run - on-demand -- when a user asks for the computed value and - the pre-computed value is obsolete -- the source cache data has been updated since the last time the value was computed. Otherwise, we return the pre-computed value. Note that we don't want to rerun the computation for each cache update! Only when the users asks for it. I hope it make it more clear now. Thanks Andrey > Subject: Re: Missing ways to get access to Ignite CacheEntry > To: [hidden email] > From: [hidden email] > Date: Fri, 18 Dec 2015 13:44:16 +0300 > > Hi Andrey, > > I got you, thanks for the clarification. > > So since you're storing a computed value in some local data structure > what is stored in the Ignite cache as a value in such a case? There > should be something. > Why don't you (or can't you) store a version identifier in that value > that is located in the Ignite cache? This should work perfectly well for > you use-case. > > Personally I'm against adding CacheEntry to a response of every get like > operation that happens over a cache (get, getAll, SQL & Scan queries). > This will add extra size to every response and only in rare cases > someone will benefit from it. > However, what if we add a special configuration parameter to > CacheConfiguration that will manage whether CacheEntry has to be passed > as a part of a response or not? Igniters, what do you think on this? > > > Regards, > Denis > > > On 12/17/2015 9:09 AM, Andrey Kornev wrote: > > Denis,In this case I used the term "caching" in general sense, meaning saving the computed value for later. I didn't mean the Ignite cache.Sorry about the confusion.Andrey > > _____________________________ > > From: Denis Magda <[hidden email]> > > Sent: Wednesday, December 16, 2015 9:48 PM > > Subject: Re: Missing ways to get access to Ignite CacheEntry > > To: <[hidden email]> > > > > > > Hi Romain, > > > > I'm a bit confused cause initially you saying that > > > > /The value is computed on first access and it is then cached > > > > /and then you add that > > > > /One constraint is that the computed value is not serializable. // > > / > > > > > > Ignite won't be able to store a value of cache if serialization is not > > supported for it by some reason. > > > > Please clarify here. Probably I'm missing something from your description. > > > > -- > > Denis > > > > On 12/16/2015 6:21 PM, Andrey Kornev wrote: > > > Romain > > > > > > I suggest you be very careful using the invoke() functionality. As explained in this posting (and the follow-ups) > > > http://mail-archives.apache.org/mod_mbox/ignite-dev/201511.mbox/%3CSNT147-W14D2736E1374450B685721CF000%40phx.gbl%3E > > > the correct usage of the invoke() may be difficult, to put it politely. > > > > > > @Denis Very briefly, the use case is "conditional re-compute". There is a pretty expensive computation that needs to be applied to each entry on demand. The value is computed on first access and it is then cached along with the current version of the cache entry. The value must be recomputed on next access only if the entry has been modified. > > > > > > If anyone has a better idea how to achieve something like that, please do share! One constraint is that the computed value is not serializable. > > > > > > Cheers > > > Andrey > > > > > >> Subject: Re: Missing ways to get access to Ignite CacheEntry > > >> To: [hidden email] > > >> From: [hidden email] > > >> Date: Wed, 16 Dec 2015 16:11:48 +0300 > > >> > > >> Romain, > > >> > > >> If it's implemented this way for now it doesn't mean that we can't > > >> enhance the API for other use cases ;) > > >> As I said presently it's supported for limited number of methods mostly > > >> by performance reasons. > > >> > > >> Please go ahead and try invoke/invokeAll and if it doesn't work fine for > > >> you we can keep discussing what to do next. > > >> > > >> Regards, > > >> Denis > > >> > > >> On 12/16/2015 1:53 PM, Romain Gilles wrote: > > >>> Hi Denis, > > >>> Thanks for you replay. And sorry to not double check it before. I see that > > >>> if I want to work with CacheEntry, I need to use the invoke method in order > > >>> to return the CacheEntry. This is the way I should do it. It sounds like > > >>> complicated for "just" getting an entry. But if you say this is the way I > > >>> will do it like that. I was just think that it could be a common use case > > >>> and therefore provide it as a shortcut. > > >>> > > >>> Thanks, > > >>> > > >>> Romain. > > >>> > > >>> Le mer. 16 déc. 2015 à 11:34, Denis Magda <[hidden email]> a écrit : > > >>> > > >>>> Hi Romain, > > >>>> > > >>>> As the current documentation of org.apache.ignite.cache.CacheEntry > > >>>> states it's possible to get a reference to CacheEntry and its version > > >>>> only for methods that are executed over local data set stored on a node. > > >>>> Among such methods are invoke & invokeAll and randomEntry. > > >>>> > > >>>> You probably can get a CacheEntry and its non null version when a cache > > >>>> iterator is in use but the version will be 'null', as far as I remember, > > >>>> for those entries that are loaded from remote nodes. > > >>>> Presently Ignite doesn't transfer the version from remote nodes as a > > >>>> part of response by perform reasons. > > >>>> > > >>>> Please elaborate more on your particular use case and what API you would > > >>>> like to add in order to support it. > > >>>> > > >>>> -- > > >>>> Denis > > >>>> > > >>>> On 12/16/2015 12:58 PM, Romain Gilles wrote: > > >>>>> Hi Igniters, > > >>>>> I'm looking for a way to get access to the Ignite CacheEntry. For now > > >>>> this > > >>>>> is the ways I found: > > >>>>> > > >>>>> - Through the queris > > >>>>> - Through jsr 107 Cache Iterable > > >>>>> - Through jsr 107 Cache itterator > > >>>>> - Through IgniteCache::randomEntry() > > >>>>> > > >>>>> If I remember correctly it was possible to get the CacheEntry from a > > >>>> given > > >>>>> key in old version of gridgain community > > >>>>> version: GridCacheProjection::entry(K key) GridCacheEntry<K,V> > > >>>>> I think it could be a good to introduce this feature at IgniteCache > > >>>> level. > > >>>>> Or maybe there is an other way to do it. > > >>>>> > > >>>>> Thanks, > > >>>>> > > >>>>> Romain. > > >>>>> > > > > > > > > > > > > > > |
Re: Missing ways to get access to Ignite CacheEntry
|
Andrey,
The main issue with returning version is sending redundant information over the wire. What if we added a method IgniteCache.localEntry(K) which would return a local entry instance with a version. You would only be able to invoke this method on the server node (not client). Would this work for you? D. On Fri, Dec 18, 2015 at 8:23 AM, Andrey Kornev <[hidden email]> wrote: > Denis, > > I'm not suggesting to have every cache read operation return a CacheEntry. > The "classical" GridGain used to have a getEntry(key) method that was very > useful. The current version of Ignite provides virtually no way of > obtaining a cache entry except thru an iterator, or using an EntryProcessor > invocation which seems more like a hack. > > As for the question of what's stored in Ignite cache and the pre-compute, > it is really an off-topic. It just came about as we were looking for a way > to work around the issue with Ignite query not returning cache entries' > version numbers in the result iterator. (There is now a ticket that > addresses this issue: https://issues.apache.org/jira/browse/IGNITE-2202 > )This is when we realized that there is no longer any way in Ignite to > retrieve the CacheEntry by its key. > > But for the sake of completeness I quickly go over the entire use case, > because it seems I wasn't clear enough in the previous post. > > Ignite cache holds the data entries that are used as the input into some > expensive computation. The data in the cache gets updated from time to > time, the computation only needs to be re-run > - on-demand -- when a user asks for the computed value and > - the pre-computed value is obsolete -- the source cache data has been > updated since the last time the value was computed. > > Otherwise, we return the pre-computed value. Note that we don't want to > rerun the computation for each cache update! Only when the users asks for > it. > > I hope it make it more clear now. > > Thanks > Andrey > > > Subject: Re: Missing ways to get access to Ignite CacheEntry > > To: [hidden email] > > From: [hidden email] > > Date: Fri, 18 Dec 2015 13:44:16 +0300 > > > > Hi Andrey, > > > > I got you, thanks for the clarification. > > > > So since you're storing a computed value in some local data structure > > what is stored in the Ignite cache as a value in such a case? There > > should be something. > > Why don't you (or can't you) store a version identifier in that value > > that is located in the Ignite cache? This should work perfectly well for > > you use-case. > > > > Personally I'm against adding CacheEntry to a response of every get like > > operation that happens over a cache (get, getAll, SQL & Scan queries). > > This will add extra size to every response and only in rare cases > > someone will benefit from it. > > However, what if we add a special configuration parameter to > > CacheConfiguration that will manage whether CacheEntry has to be passed > > as a part of a response or not? Igniters, what do you think on this? > > > > > > Regards, > > Denis > > > > > > On 12/17/2015 9:09 AM, Andrey Kornev wrote: > > > Denis,In this case I used the term "caching" in general sense, meaning > saving the computed value for later. I didn't mean the Ignite cache.Sorry > about the confusion.Andrey > > > _____________________________ > > > From: Denis Magda <[hidden email]> > > > Sent: Wednesday, December 16, 2015 9:48 PM > > > Subject: Re: Missing ways to get access to Ignite CacheEntry > > > To: <[hidden email]> > > > > > > > > > Hi Romain, > > > > > > I'm a bit confused cause initially you saying that > > > > > > /The value is computed on first access and it is then cached > > > > > > /and then you add that > > > > > > /One constraint is that the computed value is not serializable. // > > > / > > > > > > > > > Ignite won't be able to store a value of cache if serialization is > not > > > supported for it by some reason. > > > > > > Please clarify here. Probably I'm missing something from your > description. > > > > > > -- > > > Denis > > > > > > On 12/16/2015 6:21 PM, Andrey Kornev wrote: > > > > Romain > > > > > > > > I suggest you be very careful using the invoke() functionality. As > explained in this posting (and the follow-ups) > > > > > http://mail-archives.apache.org/mod_mbox/ignite-dev/201511.mbox/%3CSNT147-W14D2736E1374450B685721CF000%40phx.gbl%3E > > > > the correct usage of the invoke() may be difficult, to put it > politely. > > > > > > > > @Denis Very briefly, the use case is "conditional re-compute". > There is a pretty expensive computation that needs to be applied to each > entry on demand. The value is computed on first access and it is then > cached along with the current version of the cache entry. The value must > be recomputed on next access only if the entry has been modified. > > > > > > > > If anyone has a better idea how to achieve something like that, > please do share! One constraint is that the computed value is not > serializable. > > > > > > > > Cheers > > > > Andrey > > > > > > > >> Subject: Re: Missing ways to get access to Ignite CacheEntry > > > >> To: [hidden email] > > > >> From: [hidden email] > > > >> Date: Wed, 16 Dec 2015 16:11:48 +0300 > > > >> > > > >> Romain, > > > >> > > > >> If it's implemented this way for now it doesn't mean that we can't > > > >> enhance the API for other use cases ;) > > > >> As I said presently it's supported for limited number of methods > mostly > > > >> by performance reasons. > > > >> > > > >> Please go ahead and try invoke/invokeAll and if it doesn't work > fine for > > > >> you we can keep discussing what to do next. > > > >> > > > >> Regards, > > > >> Denis > > > >> > > > >> On 12/16/2015 1:53 PM, Romain Gilles wrote: > > > >>> Hi Denis, > > > >>> Thanks for you replay. And sorry to not double check it before. > I see that > > > >>> if I want to work with CacheEntry, I need to use the invoke > method in order > > > >>> to return the CacheEntry. This is the way I should do it. It > sounds like > > > >>> complicated for "just" getting an entry. But if you say this is > the way I > > > >>> will do it like that. I was just think that it could be a common > use case > > > >>> and therefore provide it as a shortcut. > > > >>> > > > >>> Thanks, > > > >>> > > > >>> Romain. > > > >>> > > > >>> Le mer. 16 déc. 2015 à 11:34, Denis Magda <[hidden email]> > a écrit : > > > >>> > > > >>>> Hi Romain, > > > >>>> > > > >>>> As the current documentation of > org.apache.ignite.cache.CacheEntry > > > >>>> states it's possible to get a reference to CacheEntry and its > version > > > >>>> only for methods that are executed over local data set stored > on a node. > > > >>>> Among such methods are invoke & invokeAll and randomEntry. > > > >>>> > > > >>>> You probably can get a CacheEntry and its non null version when > a cache > > > >>>> iterator is in use but the version will be 'null', as far as I > remember, > > > >>>> for those entries that are loaded from remote nodes. > > > >>>> Presently Ignite doesn't transfer the version from remote nodes > as a > > > >>>> part of response by perform reasons. > > > >>>> > > > >>>> Please elaborate more on your particular use case and what API > you would > > > >>>> like to add in order to support it. > > > >>>> > > > >>>> -- > > > >>>> Denis > > > >>>> > > > >>>> On 12/16/2015 12:58 PM, Romain Gilles wrote: > > > >>>>> Hi Igniters, > > > >>>>> I'm looking for a way to get access to the Ignite CacheEntry. > For now > > > >>>> this > > > >>>>> is the ways I found: > > > >>>>> > > > >>>>> - Through the queris > > > >>>>> - Through jsr 107 Cache Iterable > > > >>>>> - Through jsr 107 Cache itterator > > > >>>>> - Through IgniteCache::randomEntry() > > > >>>>> > > > >>>>> If I remember correctly it was possible to get the CacheEntry > from a > > > >>>> given > > > >>>>> key in old version of gridgain community > > > >>>>> version: GridCacheProjection::entry(K key) GridCacheEntry<K,V> > > > >>>>> I think it could be a good to introduce this feature at > IgniteCache > > > >>>> level. > > > >>>>> Or maybe there is an other way to do it. > > > >>>>> > > > >>>>> Thanks, > > > >>>>> > > > >>>>> Romain. > > > >>>>> > > > > > > > > > > > > > > > > > > > > > > > |
RE: Missing ways to get access to Ignite CacheEntry
|
|
Dmitriy,
Based on what criteria does one determine which information is redundant and which is not? To me, if an API declares a method getSomeRedundantInfo(), then the implementer has no choice, but to comply with the contract and provide such info. One can't just return a null or ignore it some other way, simply because it appears to have some performance impact. Besides, the overhead of the version number may turn out to be negligible when benchmarked. Most likely there will be near zero overhead (due to the way the network stack packs the bytes into the ethernet frames). The workaround for the broken API however may be much more expensive in terms of performance and user experience. Please let's keep the API consistent and predictable. Regards Andrey > From: [hidden email] > Date: Fri, 18 Dec 2015 08:41:00 -0800 > Subject: Re: Missing ways to get access to Ignite CacheEntry > To: [hidden email] > > Andrey, > > The main issue with returning version is sending redundant information over > the wire. What if we added a method IgniteCache.localEntry(K) which would > return a local entry instance with a version. You would only be able to > invoke this method on the server node (not client). > > Would this work for you? > > D. > > On Fri, Dec 18, 2015 at 8:23 AM, Andrey Kornev <[hidden email]> > wrote: > > > Denis, > > > > I'm not suggesting to have every cache read operation return a CacheEntry. > > The "classical" GridGain used to have a getEntry(key) method that was very > > useful. The current version of Ignite provides virtually no way of > > obtaining a cache entry except thru an iterator, or using an EntryProcessor > > invocation which seems more like a hack. > > > > As for the question of what's stored in Ignite cache and the pre-compute, > > it is really an off-topic. It just came about as we were looking for a way > > to work around the issue with Ignite query not returning cache entries' > > version numbers in the result iterator. (There is now a ticket that > > addresses this issue: https://issues.apache.org/jira/browse/IGNITE-2202 > > )This is when we realized that there is no longer any way in Ignite to > > retrieve the CacheEntry by its key. > > > > But for the sake of completeness I quickly go over the entire use case, > > because it seems I wasn't clear enough in the previous post. > > > > Ignite cache holds the data entries that are used as the input into some > > expensive computation. The data in the cache gets updated from time to > > time, the computation only needs to be re-run > > - on-demand -- when a user asks for the computed value and > > - the pre-computed value is obsolete -- the source cache data has been > > updated since the last time the value was computed. > > > > Otherwise, we return the pre-computed value. Note that we don't want to > > rerun the computation for each cache update! Only when the users asks for > > it. > > > > I hope it make it more clear now. > > > > Thanks > > Andrey > > > > > Subject: Re: Missing ways to get access to Ignite CacheEntry > > > To: [hidden email] > > > From: [hidden email] > > > Date: Fri, 18 Dec 2015 13:44:16 +0300 > > > > > > Hi Andrey, > > > > > > I got you, thanks for the clarification. > > > > > > So since you're storing a computed value in some local data structure > > > what is stored in the Ignite cache as a value in such a case? There > > > should be something. > > > Why don't you (or can't you) store a version identifier in that value > > > that is located in the Ignite cache? This should work perfectly well for > > > you use-case. > > > > > > Personally I'm against adding CacheEntry to a response of every get like > > > operation that happens over a cache (get, getAll, SQL & Scan queries). > > > This will add extra size to every response and only in rare cases > > > someone will benefit from it. > > > However, what if we add a special configuration parameter to > > > CacheConfiguration that will manage whether CacheEntry has to be passed > > > as a part of a response or not? Igniters, what do you think on this? > > > > > > > > > Regards, > > > Denis > > > > > > > > > On 12/17/2015 9:09 AM, Andrey Kornev wrote: > > > > Denis,In this case I used the term "caching" in general sense, meaning > > saving the computed value for later. I didn't mean the Ignite cache.Sorry > > about the confusion.Andrey > > > > _____________________________ > > > > From: Denis Magda <[hidden email]> > > > > Sent: Wednesday, December 16, 2015 9:48 PM > > > > Subject: Re: Missing ways to get access to Ignite CacheEntry > > > > To: <[hidden email]> > > > > > > > > > > > > Hi Romain, > > > > > > > > I'm a bit confused cause initially you saying that > > > > > > > > /The value is computed on first access and it is then cached > > > > > > > > /and then you add that > > > > > > > > /One constraint is that the computed value is not serializable. // > > > > / > > > > > > > > > > > > Ignite won't be able to store a value of cache if serialization is > > not > > > > supported for it by some reason. > > > > > > > > Please clarify here. Probably I'm missing something from your > > description. > > > > > > > > -- > > > > Denis > > > > > > > > On 12/16/2015 6:21 PM, Andrey Kornev wrote: > > > > > Romain > > > > > > > > > > I suggest you be very careful using the invoke() functionality. As > > explained in this posting (and the follow-ups) > > > > > > > http://mail-archives.apache.org/mod_mbox/ignite-dev/201511.mbox/%3CSNT147-W14D2736E1374450B685721CF000%40phx.gbl%3E > > > > > the correct usage of the invoke() may be difficult, to put it > > politely. > > > > > > > > > > @Denis Very briefly, the use case is "conditional re-compute". > > There is a pretty expensive computation that needs to be applied to each > > entry on demand. The value is computed on first access and it is then > > cached along with the current version of the cache entry. The value must > > be recomputed on next access only if the entry has been modified. > > > > > > > > > > If anyone has a better idea how to achieve something like that, > > please do share! One constraint is that the computed value is not > > serializable. > > > > > > > > > > Cheers > > > > > Andrey > > > > > > > > > >> Subject: Re: Missing ways to get access to Ignite CacheEntry > > > > >> To: [hidden email] > > > > >> From: [hidden email] > > > > >> Date: Wed, 16 Dec 2015 16:11:48 +0300 > > > > >> > > > > >> Romain, > > > > >> > > > > >> If it's implemented this way for now it doesn't mean that we can't > > > > >> enhance the API for other use cases ;) > > > > >> As I said presently it's supported for limited number of methods > > mostly > > > > >> by performance reasons. > > > > >> > > > > >> Please go ahead and try invoke/invokeAll and if it doesn't work > > fine for > > > > >> you we can keep discussing what to do next. > > > > >> > > > > >> Regards, > > > > >> Denis > > > > >> > > > > >> On 12/16/2015 1:53 PM, Romain Gilles wrote: > > > > >>> Hi Denis, > > > > >>> Thanks for you replay. And sorry to not double check it before. > > I see that > > > > >>> if I want to work with CacheEntry, I need to use the invoke > > method in order > > > > >>> to return the CacheEntry. This is the way I should do it. It > > sounds like > > > > >>> complicated for "just" getting an entry. But if you say this is > > the way I > > > > >>> will do it like that. I was just think that it could be a common > > use case > > > > >>> and therefore provide it as a shortcut. > > > > >>> > > > > >>> Thanks, > > > > >>> > > > > >>> Romain. > > > > >>> > > > > >>> Le mer. 16 déc. 2015 à 11:34, Denis Magda <[hidden email]> > > a écrit : > > > > >>> > > > > >>>> Hi Romain, > > > > >>>> > > > > >>>> As the current documentation of > > org.apache.ignite.cache.CacheEntry > > > > >>>> states it's possible to get a reference to CacheEntry and its > > version > > > > >>>> only for methods that are executed over local data set stored > > on a node. > > > > >>>> Among such methods are invoke & invokeAll and randomEntry. > > > > >>>> > > > > >>>> You probably can get a CacheEntry and its non null version when > > a cache > > > > >>>> iterator is in use but the version will be 'null', as far as I > > remember, > > > > >>>> for those entries that are loaded from remote nodes. > > > > >>>> Presently Ignite doesn't transfer the version from remote nodes > > as a > > > > >>>> part of response by perform reasons. > > > > >>>> > > > > >>>> Please elaborate more on your particular use case and what API > > you would > > > > >>>> like to add in order to support it. > > > > >>>> > > > > >>>> -- > > > > >>>> Denis > > > > >>>> > > > > >>>> On 12/16/2015 12:58 PM, Romain Gilles wrote: > > > > >>>>> Hi Igniters, > > > > >>>>> I'm looking for a way to get access to the Ignite CacheEntry. > > For now > > > > >>>> this > > > > >>>>> is the ways I found: > > > > >>>>> > > > > >>>>> - Through the queris > > > > >>>>> - Through jsr 107 Cache Iterable > > > > >>>>> - Through jsr 107 Cache itterator > > > > >>>>> - Through IgniteCache::randomEntry() > > > > >>>>> > > > > >>>>> If I remember correctly it was possible to get the CacheEntry > > from a > > > > >>>> given > > > > >>>>> key in old version of gridgain community > > > > >>>>> version: GridCacheProjection::entry(K key) GridCacheEntry<K,V> > > > > >>>>> I think it could be a good to introduce this feature at > > IgniteCache > > > > >>>> level. > > > > >>>>> Or maybe there is an other way to do it. > > > > >>>>> > > > > >>>>> Thanks, > > > > >>>>> > > > > >>>>> Romain. > > > > >>>>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > |
Re: Missing ways to get access to Ignite CacheEntry
|
|
On Fri, Dec 18, 2015 at 9:47 AM, Andrey Kornev <[hidden email]>
wrote: > Dmitriy, > > Based on what criteria does one determine which information is redundant > and which is not? > > To me, if an API declares a method getSomeRedundantInfo(), then the > implementer has no choice, but to comply with the contract and provide such > info. One can't just return a null or ignore it some other way, simply > because it appears to have some performance impact. Andrey, the CacheEntry is not the JCache entry, but rather the internal Ignite entry that you can get when unwrapping the JCache entry. The unwrapping, obviously, can only happen on the server side. Therefore, I am suggesting that we also add a localEntry(K) method which will provide the entry API on the server side directly (not from iterator or EP). Now I understand that the version() method currently does not work properly on the server side, but this is something we can easily fix. Thoughts? > Besides, the overhead of the version number may turn out to be negligible > when benchmarked. Most likely there will be near zero overhead (due to the > way the network stack packs the bytes into the ethernet frames). The > workaround for the broken API however may be much more expensive in terms > of performance and user experience. > > Please let's keep the API consistent and predictable. > > Regards > Andrey > > > From: [hidden email] > > Date: Fri, 18 Dec 2015 08:41:00 -0800 > > Subject: Re: Missing ways to get access to Ignite CacheEntry > > To: [hidden email] > > > > Andrey, > > > > The main issue with returning version is sending redundant information > over > > the wire. What if we added a method IgniteCache.localEntry(K) which would > > return a local entry instance with a version. You would only be able to > > invoke this method on the server node (not client). > > > > Would this work for you? > > > > D. > > > > On Fri, Dec 18, 2015 at 8:23 AM, Andrey Kornev <[hidden email] > > > > wrote: > > > > > Denis, > > > > > > I'm not suggesting to have every cache read operation return a > CacheEntry. > > > The "classical" GridGain used to have a getEntry(key) method that was > very > > > useful. The current version of Ignite provides virtually no way of > > > obtaining a cache entry except thru an iterator, or using an > EntryProcessor > > > invocation which seems more like a hack. > > > > > > As for the question of what's stored in Ignite cache and the > pre-compute, > > > it is really an off-topic. It just came about as we were looking for a > way > > > to work around the issue with Ignite query not returning cache entries' > > > version numbers in the result iterator. (There is now a ticket that > > > addresses this issue: > https://issues.apache.org/jira/browse/IGNITE-2202 > > > )This is when we realized that there is no longer any way in Ignite to > > > retrieve the CacheEntry by its key. > > > > > > But for the sake of completeness I quickly go over the entire use case, > > > because it seems I wasn't clear enough in the previous post. > > > > > > Ignite cache holds the data entries that are used as the input into > some > > > expensive computation. The data in the cache gets updated from time to > > > time, the computation only needs to be re-run > > > - on-demand -- when a user asks for the computed value and > > > - the pre-computed value is obsolete -- the source cache data has been > > > updated since the last time the value was computed. > > > > > > Otherwise, we return the pre-computed value. Note that we don't want to > > > rerun the computation for each cache update! Only when the users asks > for > > > it. > > > > > > I hope it make it more clear now. > > > > > > Thanks > > > Andrey > > > > > > > Subject: Re: Missing ways to get access to Ignite CacheEntry > > > > To: [hidden email] > > > > From: [hidden email] > > > > Date: Fri, 18 Dec 2015 13:44:16 +0300 > > > > > > > > Hi Andrey, > > > > > > > > I got you, thanks for the clarification. > > > > > > > > So since you're storing a computed value in some local data structure > > > > what is stored in the Ignite cache as a value in such a case? There > > > > should be something. > > > > Why don't you (or can't you) store a version identifier in that value > > > > that is located in the Ignite cache? This should work perfectly well > for > > > > you use-case. > > > > > > > > Personally I'm against adding CacheEntry to a response of every get > like > > > > operation that happens over a cache (get, getAll, SQL & Scan > queries). > > > > This will add extra size to every response and only in rare cases > > > > someone will benefit from it. > > > > However, what if we add a special configuration parameter to > > > > CacheConfiguration that will manage whether CacheEntry has to be > passed > > > > as a part of a response or not? Igniters, what do you think on this? > > > > > > > > > > > > Regards, > > > > Denis > > > > > > > > > > > > On 12/17/2015 9:09 AM, Andrey Kornev wrote: > > > > > Denis,In this case I used the term "caching" in general sense, > meaning > > > saving the computed value for later. I didn't mean the Ignite > cache.Sorry > > > about the confusion.Andrey > > > > > _____________________________ > > > > > From: Denis Magda <[hidden email]> > > > > > Sent: Wednesday, December 16, 2015 9:48 PM > > > > > Subject: Re: Missing ways to get access to Ignite CacheEntry > > > > > To: <[hidden email]> > > > > > > > > > > > > > > > Hi Romain, > > > > > > > > > > I'm a bit confused cause initially you saying that > > > > > > > > > > /The value is computed on first access and it is then cached > > > > > > > > > > /and then you add that > > > > > > > > > > /One constraint is that the computed value is not serializable. > // > > > > > / > > > > > > > > > > > > > > > Ignite won't be able to store a value of cache if serialization > is > > > not > > > > > supported for it by some reason. > > > > > > > > > > Please clarify here. Probably I'm missing something from your > > > description. > > > > > > > > > > -- > > > > > Denis > > > > > > > > > > On 12/16/2015 6:21 PM, Andrey Kornev wrote: > > > > > > Romain > > > > > > > > > > > > I suggest you be very careful using the invoke() > functionality. As > > > explained in this posting (and the follow-ups) > > > > > > > > > > http://mail-archives.apache.org/mod_mbox/ignite-dev/201511.mbox/%3CSNT147-W14D2736E1374450B685721CF000%40phx.gbl%3E > > > > > > the correct usage of the invoke() may be difficult, to put it > > > politely. > > > > > > > > > > > > @Denis Very briefly, the use case is "conditional re-compute". > > > There is a pretty expensive computation that needs to be applied to > each > > > entry on demand. The value is computed on first access and it is then > > > cached along with the current version of the cache entry. The value > must > > > be recomputed on next access only if the entry has been modified. > > > > > > > > > > > > If anyone has a better idea how to achieve something like that, > > > please do share! One constraint is that the computed value is not > > > serializable. > > > > > > > > > > > > Cheers > > > > > > Andrey > > > > > > > > > > > >> Subject: Re: Missing ways to get access to Ignite CacheEntry > > > > > >> To: [hidden email] > > > > > >> From: [hidden email] > > > > > >> Date: Wed, 16 Dec 2015 16:11:48 +0300 > > > > > >> > > > > > >> Romain, > > > > > >> > > > > > >> If it's implemented this way for now it doesn't mean that we > can't > > > > > >> enhance the API for other use cases ;) > > > > > >> As I said presently it's supported for limited number of > methods > > > mostly > > > > > >> by performance reasons. > > > > > >> > > > > > >> Please go ahead and try invoke/invokeAll and if it doesn't > work > > > fine for > > > > > >> you we can keep discussing what to do next. > > > > > >> > > > > > >> Regards, > > > > > >> Denis > > > > > >> > > > > > >> On 12/16/2015 1:53 PM, Romain Gilles wrote: > > > > > >>> Hi Denis, > > > > > >>> Thanks for you replay. And sorry to not double check it > before. > > > I see that > > > > > >>> if I want to work with CacheEntry, I need to use the invoke > > > method in order > > > > > >>> to return the CacheEntry. This is the way I should do it. It > > > sounds like > > > > > >>> complicated for "just" getting an entry. But if you say this > is > > > the way I > > > > > >>> will do it like that. I was just think that it could be a > common > > > use case > > > > > >>> and therefore provide it as a shortcut. > > > > > >>> > > > > > >>> Thanks, > > > > > >>> > > > > > >>> Romain. > > > > > >>> > > > > > >>> Le mer. 16 déc. 2015 à 11:34, Denis Magda < > [hidden email]> > > > a écrit : > > > > > >>> > > > > > >>>> Hi Romain, > > > > > >>>> > > > > > >>>> As the current documentation of > > > org.apache.ignite.cache.CacheEntry > > > > > >>>> states it's possible to get a reference to CacheEntry and > its > > > version > > > > > >>>> only for methods that are executed over local data set > stored > > > on a node. > > > > > >>>> Among such methods are invoke & invokeAll and randomEntry. > > > > > >>>> > > > > > >>>> You probably can get a CacheEntry and its non null version > when > > > a cache > > > > > >>>> iterator is in use but the version will be 'null', as far > as I > > > remember, > > > > > >>>> for those entries that are loaded from remote nodes. > > > > > >>>> Presently Ignite doesn't transfer the version from remote > nodes > > > as a > > > > > >>>> part of response by perform reasons. > > > > > >>>> > > > > > >>>> Please elaborate more on your particular use case and what > API > > > you would > > > > > >>>> like to add in order to support it. > > > > > >>>> > > > > > >>>> -- > > > > > >>>> Denis > > > > > >>>> > > > > > >>>> On 12/16/2015 12:58 PM, Romain Gilles wrote: > > > > > >>>>> Hi Igniters, > > > > > >>>>> I'm looking for a way to get access to the Ignite > CacheEntry. > > > For now > > > > > >>>> this > > > > > >>>>> is the ways I found: > > > > > >>>>> > > > > > >>>>> - Through the queris > > > > > >>>>> - Through jsr 107 Cache Iterable > > > > > >>>>> - Through jsr 107 Cache itterator > > > > > >>>>> - Through IgniteCache::randomEntry() > > > > > >>>>> > > > > > >>>>> If I remember correctly it was possible to get the > CacheEntry > > > from a > > > > > >>>> given > > > > > >>>>> key in old version of gridgain community > > > > > >>>>> version: GridCacheProjection::entry(K key) > GridCacheEntry<K,V> > > > > > >>>>> I think it could be a good to introduce this feature at > > > IgniteCache > > > > > >>>> level. > > > > > >>>>> Or maybe there is an other way to do it. > > > > > >>>>> > > > > > >>>>> Thanks, > > > > > >>>>> > > > > > >>>>> Romain. > > > > > >>>>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > |
RE: Missing ways to get access to Ignite CacheEntry
|
|
Dmitriy,
I understand that we're talking about Ignite CacheEntry and not JCache's CacheEntry that has no version property to speak of. But I don't understand why the method is named localEntry()? What does "local" mean in this context? Does it mean that I can only use it on the primary node for a given key? What would happen if I call this method on a non-primary node? And what would I do if I needed the entry (along with the valid version it contains) on a non-primary node? Thanks Andrey > From: [hidden email] > Date: Fri, 18 Dec 2015 12:19:01 -0800 > Subject: Re: Missing ways to get access to Ignite CacheEntry > To: [hidden email] > > On Fri, Dec 18, 2015 at 9:47 AM, Andrey Kornev <[hidden email]> > wrote: > > > Dmitriy, > > > > Based on what criteria does one determine which information is redundant > > and which is not? > > > > To me, if an API declares a method getSomeRedundantInfo(), then the > > implementer has no choice, but to comply with the contract and provide such > > info. One can't just return a null or ignore it some other way, simply > > because it appears to have some performance impact. > > > Andrey, the CacheEntry is not the JCache entry, but rather the internal > Ignite entry that you can get when unwrapping the JCache entry. The > unwrapping, obviously, can only happen on the server side. Therefore, I am > suggesting that we also add a localEntry(K) method which will provide the > entry API on the server side directly (not from iterator or EP). > > Now I understand that the version() method currently does not work properly > on the server side, but this is something we can easily fix. > > Thoughts? > > > > Besides, the overhead of the version number may turn out to be negligible > > when benchmarked. Most likely there will be near zero overhead (due to the > > way the network stack packs the bytes into the ethernet frames). The > > workaround for the broken API however may be much more expensive in terms > > of performance and user experience. > > > > Please let's keep the API consistent and predictable. > > > > Regards > > Andrey > > > > > From: [hidden email] > > > Date: Fri, 18 Dec 2015 08:41:00 -0800 > > > Subject: Re: Missing ways to get access to Ignite CacheEntry > > > To: [hidden email] > > > > > > Andrey, > > > > > > The main issue with returning version is sending redundant information > > over > > > the wire. What if we added a method IgniteCache.localEntry(K) which would > > > return a local entry instance with a version. You would only be able to > > > invoke this method on the server node (not client). > > > > > > Would this work for you? > > > > > > D. > > > > > > On Fri, Dec 18, 2015 at 8:23 AM, Andrey Kornev <[hidden email] > > > > > > wrote: > > > > > > > Denis, > > > > > > > > I'm not suggesting to have every cache read operation return a > > CacheEntry. > > > > The "classical" GridGain used to have a getEntry(key) method that was > > very > > > > useful. The current version of Ignite provides virtually no way of > > > > obtaining a cache entry except thru an iterator, or using an > > EntryProcessor > > > > invocation which seems more like a hack. > > > > > > > > As for the question of what's stored in Ignite cache and the > > pre-compute, > > > > it is really an off-topic. It just came about as we were looking for a > > way > > > > to work around the issue with Ignite query not returning cache entries' > > > > version numbers in the result iterator. (There is now a ticket that > > > > addresses this issue: > > https://issues.apache.org/jira/browse/IGNITE-2202 > > > > )This is when we realized that there is no longer any way in Ignite to > > > > retrieve the CacheEntry by its key. > > > > > > > > But for the sake of completeness I quickly go over the entire use case, > > > > because it seems I wasn't clear enough in the previous post. > > > > > > > > Ignite cache holds the data entries that are used as the input into > > some > > > > expensive computation. The data in the cache gets updated from time to > > > > time, the computation only needs to be re-run > > > > - on-demand -- when a user asks for the computed value and > > > > - the pre-computed value is obsolete -- the source cache data has been > > > > updated since the last time the value was computed. > > > > > > > > Otherwise, we return the pre-computed value. Note that we don't want to > > > > rerun the computation for each cache update! Only when the users asks > > for > > > > it. > > > > > > > > I hope it make it more clear now. > > > > > > > > Thanks > > > > Andrey > > > > > > > > > Subject: Re: Missing ways to get access to Ignite CacheEntry > > > > > To: [hidden email] > > > > > From: [hidden email] > > > > > Date: Fri, 18 Dec 2015 13:44:16 +0300 > > > > > > > > > > Hi Andrey, > > > > > > > > > > I got you, thanks for the clarification. > > > > > > > > > > So since you're storing a computed value in some local data structure > > > > > what is stored in the Ignite cache as a value in such a case? There > > > > > should be something. > > > > > Why don't you (or can't you) store a version identifier in that value > > > > > that is located in the Ignite cache? This should work perfectly well > > for > > > > > you use-case. > > > > > > > > > > Personally I'm against adding CacheEntry to a response of every get > > like > > > > > operation that happens over a cache (get, getAll, SQL & Scan > > queries). > > > > > This will add extra size to every response and only in rare cases > > > > > someone will benefit from it. > > > > > However, what if we add a special configuration parameter to > > > > > CacheConfiguration that will manage whether CacheEntry has to be > > passed > > > > > as a part of a response or not? Igniters, what do you think on this? > > > > > > > > > > > > > > > Regards, > > > > > Denis > > > > > > > > > > > > > > > On 12/17/2015 9:09 AM, Andrey Kornev wrote: > > > > > > Denis,In this case I used the term "caching" in general sense, > > meaning > > > > saving the computed value for later. I didn't mean the Ignite > > cache.Sorry > > > > about the confusion.Andrey > > > > > > _____________________________ > > > > > > From: Denis Magda <[hidden email]> > > > > > > Sent: Wednesday, December 16, 2015 9:48 PM > > > > > > Subject: Re: Missing ways to get access to Ignite CacheEntry > > > > > > To: <[hidden email]> > > > > > > > > > > > > > > > > > > Hi Romain, > > > > > > > > > > > > I'm a bit confused cause initially you saying that > > > > > > > > > > > > /The value is computed on first access and it is then cached > > > > > > > > > > > > /and then you add that > > > > > > > > > > > > /One constraint is that the computed value is not serializable. > > // > > > > > > / > > > > > > > > > > > > > > > > > > Ignite won't be able to store a value of cache if serialization > > is > > > > not > > > > > > supported for it by some reason. > > > > > > > > > > > > Please clarify here. Probably I'm missing something from your > > > > description. > > > > > > > > > > > > -- > > > > > > Denis > > > > > > > > > > > > On 12/16/2015 6:21 PM, Andrey Kornev wrote: > > > > > > > Romain > > > > > > > > > > > > > > I suggest you be very careful using the invoke() > > functionality. As > > > > explained in this posting (and the follow-ups) > > > > > > > > > > > > > http://mail-archives.apache.org/mod_mbox/ignite-dev/201511.mbox/%3CSNT147-W14D2736E1374450B685721CF000%40phx.gbl%3E > > > > > > > the correct usage of the invoke() may be difficult, to put it > > > > politely. > > > > > > > > > > > > > > @Denis Very briefly, the use case is "conditional re-compute". > > > > There is a pretty expensive computation that needs to be applied to > > each > > > > entry on demand. The value is computed on first access and it is then > > > > cached along with the current version of the cache entry. The value > > must > > > > be recomputed on next access only if the entry has been modified. > > > > > > > > > > > > > > If anyone has a better idea how to achieve something like that, > > > > please do share! One constraint is that the computed value is not > > > > serializable. > > > > > > > > > > > > > > Cheers > > > > > > > Andrey > > > > > > > > > > > > > >> Subject: Re: Missing ways to get access to Ignite CacheEntry > > > > > > >> To: [hidden email] > > > > > > >> From: [hidden email] > > > > > > >> Date: Wed, 16 Dec 2015 16:11:48 +0300 > > > > > > >> > > > > > > >> Romain, > > > > > > >> > > > > > > >> If it's implemented this way for now it doesn't mean that we > > can't > > > > > > >> enhance the API for other use cases ;) > > > > > > >> As I said presently it's supported for limited number of > > methods > > > > mostly > > > > > > >> by performance reasons. > > > > > > >> > > > > > > >> Please go ahead and try invoke/invokeAll and if it doesn't > > work > > > > fine for > > > > > > >> you we can keep discussing what to do next. > > > > > > >> > > > > > > >> Regards, > > > > > > >> Denis > > > > > > >> > > > > > > >> On 12/16/2015 1:53 PM, Romain Gilles wrote: > > > > > > >>> Hi Denis, > > > > > > >>> Thanks for you replay. And sorry to not double check it > > before. > > > > I see that > > > > > > >>> if I want to work with CacheEntry, I need to use the invoke > > > > method in order > > > > > > >>> to return the CacheEntry. This is the way I should do it. It > > > > sounds like > > > > > > >>> complicated for "just" getting an entry. But if you say this > > is > > > > the way I > > > > > > >>> will do it like that. I was just think that it could be a > > common > > > > use case > > > > > > >>> and therefore provide it as a shortcut. > > > > > > >>> > > > > > > >>> Thanks, > > > > > > >>> > > > > > > >>> Romain. > > > > > > >>> > > > > > > >>> Le mer. 16 déc. 2015 à 11:34, Denis Magda < > > [hidden email]> > > > > a écrit : > > > > > > >>> > > > > > > >>>> Hi Romain, > > > > > > >>>> > > > > > > >>>> As the current documentation of > > > > org.apache.ignite.cache.CacheEntry > > > > > > >>>> states it's possible to get a reference to CacheEntry and > > its > > > > version > > > > > > >>>> only for methods that are executed over local data set > > stored > > > > on a node. > > > > > > >>>> Among such methods are invoke & invokeAll and randomEntry. > > > > > > >>>> > > > > > > >>>> You probably can get a CacheEntry and its non null version > > when > > > > a cache > > > > > > >>>> iterator is in use but the version will be 'null', as far > > as I > > > > remember, > > > > > > >>>> for those entries that are loaded from remote nodes. > > > > > > >>>> Presently Ignite doesn't transfer the version from remote > > nodes > > > > as a > > > > > > >>>> part of response by perform reasons. > > > > > > >>>> > > > > > > >>>> Please elaborate more on your particular use case and what > > API > > > > you would > > > > > > >>>> like to add in order to support it. > > > > > > >>>> > > > > > > >>>> -- > > > > > > >>>> Denis > > > > > > >>>> > > > > > > >>>> On 12/16/2015 12:58 PM, Romain Gilles wrote: > > > > > > >>>>> Hi Igniters, > > > > > > >>>>> I'm looking for a way to get access to the Ignite > > CacheEntry. > > > > For now > > > > > > >>>> this > > > > > > >>>>> is the ways I found: > > > > > > >>>>> > > > > > > >>>>> - Through the queris > > > > > > >>>>> - Through jsr 107 Cache Iterable > > > > > > >>>>> - Through jsr 107 Cache itterator > > > > > > >>>>> - Through IgniteCache::randomEntry() > > > > > > >>>>> > > > > > > >>>>> If I remember correctly it was possible to get the > > CacheEntry > > > > from a > > > > > > >>>> given > > > > > > >>>>> key in old version of gridgain community > > > > > > >>>>> version: GridCacheProjection::entry(K key) > > GridCacheEntry<K,V> > > > > > > >>>>> I think it could be a good to introduce this feature at > > > > IgniteCache > > > > > > >>>> level. > > > > > > >>>>> Or maybe there is an other way to do it. > > > > > > >>>>> > > > > > > >>>>> Thanks, > > > > > > >>>>> > > > > > > >>>>> Romain. > > > > > > >>>>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > |
Re: Missing ways to get access to Ignite CacheEntry
|
|
On Fri, Dec 18, 2015 at 12:40 PM, Andrey Kornev <[hidden email]>
wrote: > Dmitriy, > > I understand that we're talking about Ignite CacheEntry and not JCache's > CacheEntry that has no version property to speak of. > > But I don't understand why the method is named localEntry()? What does > "local" mean in this context? Does it mean that I can only use it on the > primary node for a given key? What would happen if I call this method on a > non-primary node? And what would I do if I needed the entry (along with the > valid version it contains) on a non-primary node? > If the node is either primary or backup, then you will get the entry back. Otherwise, you will get null. > > From: [hidden email] > > Date: Fri, 18 Dec 2015 12:19:01 -0800 > > Subject: Re: Missing ways to get access to Ignite CacheEntry > > To: [hidden email] > > > > On Fri, Dec 18, 2015 at 9:47 AM, Andrey Kornev <[hidden email] > > > > wrote: > > > > > Dmitriy, > > > > > > Based on what criteria does one determine which information is > redundant > > > and which is not? > > > > > > To me, if an API declares a method getSomeRedundantInfo(), then the > > > implementer has no choice, but to comply with the contract and provide > such > > > info. One can't just return a null or ignore it some other way, simply > > > because it appears to have some performance impact. > > > > > > Andrey, the CacheEntry is not the JCache entry, but rather the internal > > Ignite entry that you can get when unwrapping the JCache entry. The > > unwrapping, obviously, can only happen on the server side. Therefore, I > am > > suggesting that we also add a localEntry(K) method which will provide the > > entry API on the server side directly (not from iterator or EP). > > > > Now I understand that the version() method currently does not work > properly > > on the server side, but this is something we can easily fix. > > > > Thoughts? > > > > > > > Besides, the overhead of the version number may turn out to be > negligible > > > when benchmarked. Most likely there will be near zero overhead (due to > the > > > way the network stack packs the bytes into the ethernet frames). The > > > workaround for the broken API however may be much more expensive in > terms > > > of performance and user experience. > > > > > > Please let's keep the API consistent and predictable. > > > > > > Regards > > > Andrey > > > > > > > From: [hidden email] > > > > Date: Fri, 18 Dec 2015 08:41:00 -0800 > > > > Subject: Re: Missing ways to get access to Ignite CacheEntry > > > > To: [hidden email] > > > > > > > > Andrey, > > > > > > > > The main issue with returning version is sending redundant > information > > > over > > > > the wire. What if we added a method IgniteCache.localEntry(K) which > would > > > > return a local entry instance with a version. You would only be able > to > > > > invoke this method on the server node (not client). > > > > > > > > Would this work for you? > > > > > > > > D. > > > > > > > > On Fri, Dec 18, 2015 at 8:23 AM, Andrey Kornev < > [hidden email] > > > > > > > > wrote: > > > > > > > > > Denis, > > > > > > > > > > I'm not suggesting to have every cache read operation return a > > > CacheEntry. > > > > > The "classical" GridGain used to have a getEntry(key) method that > was > > > very > > > > > useful. The current version of Ignite provides virtually no way of > > > > > obtaining a cache entry except thru an iterator, or using an > > > EntryProcessor > > > > > invocation which seems more like a hack. > > > > > > > > > > As for the question of what's stored in Ignite cache and the > > > pre-compute, > > > > > it is really an off-topic. It just came about as we were looking > for a > > > way > > > > > to work around the issue with Ignite query not returning cache > entries' > > > > > version numbers in the result iterator. (There is now a ticket that > > > > > addresses this issue: > > > https://issues.apache.org/jira/browse/IGNITE-2202 > > > > > )This is when we realized that there is no longer any way in > Ignite to > > > > > retrieve the CacheEntry by its key. > > > > > > > > > > But for the sake of completeness I quickly go over the entire use > case, > > > > > because it seems I wasn't clear enough in the previous post. > > > > > > > > > > Ignite cache holds the data entries that are used as the input into > > > some > > > > > expensive computation. The data in the cache gets updated from > time to > > > > > time, the computation only needs to be re-run > > > > > - on-demand -- when a user asks for the computed value and > > > > > - the pre-computed value is obsolete -- the source cache data has > been > > > > > updated since the last time the value was computed. > > > > > > > > > > Otherwise, we return the pre-computed value. Note that we don't > want to > > > > > rerun the computation for each cache update! Only when the users > asks > > > for > > > > > it. > > > > > > > > > > I hope it make it more clear now. > > > > > > > > > > Thanks > > > > > Andrey > > > > > > > > > > > Subject: Re: Missing ways to get access to Ignite CacheEntry > > > > > > To: [hidden email] > > > > > > From: [hidden email] > > > > > > Date: Fri, 18 Dec 2015 13:44:16 +0300 > > > > > > > > > > > > Hi Andrey, > > > > > > > > > > > > I got you, thanks for the clarification. > > > > > > > > > > > > So since you're storing a computed value in some local data > structure > > > > > > what is stored in the Ignite cache as a value in such a case? > There > > > > > > should be something. > > > > > > Why don't you (or can't you) store a version identifier in that > value > > > > > > that is located in the Ignite cache? This should work perfectly > well > > > for > > > > > > you use-case. > > > > > > > > > > > > Personally I'm against adding CacheEntry to a response of every > get > > > like > > > > > > operation that happens over a cache (get, getAll, SQL & Scan > > > queries). > > > > > > This will add extra size to every response and only in rare cases > > > > > > someone will benefit from it. > > > > > > However, what if we add a special configuration parameter to > > > > > > CacheConfiguration that will manage whether CacheEntry has to be > > > passed > > > > > > as a part of a response or not? Igniters, what do you think on > this? > > > > > > > > > > > > > > > > > > Regards, > > > > > > Denis > > > > > > > > > > > > > > > > > > On 12/17/2015 9:09 AM, Andrey Kornev wrote: > > > > > > > Denis,In this case I used the term "caching" in general sense, > > > meaning > > > > > saving the computed value for later. I didn't mean the Ignite > > > cache.Sorry > > > > > about the confusion.Andrey > > > > > > > _____________________________ > > > > > > > From: Denis Magda <[hidden email]> > > > > > > > Sent: Wednesday, December 16, 2015 9:48 PM > > > > > > > Subject: Re: Missing ways to get access to Ignite CacheEntry > > > > > > > To: <[hidden email]> > > > > > > > > > > > > > > > > > > > > > Hi Romain, > > > > > > > > > > > > > > I'm a bit confused cause initially you saying that > > > > > > > > > > > > > > /The value is computed on first access and it is then cached > > > > > > > > > > > > > > /and then you add that > > > > > > > > > > > > > > /One constraint is that the computed value is not > serializable. > > > // > > > > > > > / > > > > > > > > > > > > > > > > > > > > > Ignite won't be able to store a value of cache if > serialization > > > is > > > > > not > > > > > > > supported for it by some reason. > > > > > > > > > > > > > > Please clarify here. Probably I'm missing something from your > > > > > description. > > > > > > > > > > > > > > -- > > > > > > > Denis > > > > > > > > > > > > > > On 12/16/2015 6:21 PM, Andrey Kornev wrote: > > > > > > > > Romain > > > > > > > > > > > > > > > > I suggest you be very careful using the invoke() > > > functionality. As > > > > > explained in this posting (and the follow-ups) > > > > > > > > > > > > > > > > > http://mail-archives.apache.org/mod_mbox/ignite-dev/201511.mbox/%3CSNT147-W14D2736E1374450B685721CF000%40phx.gbl%3E > > > > > > > > the correct usage of the invoke() may be difficult, to put > it > > > > > politely. > > > > > > > > > > > > > > > > @Denis Very briefly, the use case is "conditional > re-compute". > > > > > There is a pretty expensive computation that needs to be applied to > > > each > > > > > entry on demand. The value is computed on first access and it is > then > > > > > cached along with the current version of the cache entry. The > value > > > must > > > > > be recomputed on next access only if the entry has been modified. > > > > > > > > > > > > > > > > If anyone has a better idea how to achieve something like > that, > > > > > please do share! One constraint is that the computed value is not > > > > > serializable. > > > > > > > > > > > > > > > > Cheers > > > > > > > > Andrey > > > > > > > > > > > > > > > >> Subject: Re: Missing ways to get access to Ignite > CacheEntry > > > > > > > >> To: [hidden email] > > > > > > > >> From: [hidden email] > > > > > > > >> Date: Wed, 16 Dec 2015 16:11:48 +0300 > > > > > > > >> > > > > > > > >> Romain, > > > > > > > >> > > > > > > > >> If it's implemented this way for now it doesn't mean that > we > > > can't > > > > > > > >> enhance the API for other use cases ;) > > > > > > > >> As I said presently it's supported for limited number of > > > methods > > > > > mostly > > > > > > > >> by performance reasons. > > > > > > > >> > > > > > > > >> Please go ahead and try invoke/invokeAll and if it doesn't > > > work > > > > > fine for > > > > > > > >> you we can keep discussing what to do next. > > > > > > > >> > > > > > > > >> Regards, > > > > > > > >> Denis > > > > > > > >> > > > > > > > >> On 12/16/2015 1:53 PM, Romain Gilles wrote: > > > > > > > >>> Hi Denis, > > > > > > > >>> Thanks for you replay. And sorry to not double check it > > > before. > > > > > I see that > > > > > > > >>> if I want to work with CacheEntry, I need to use the > invoke > > > > > method in order > > > > > > > >>> to return the CacheEntry. This is the way I should do > it. It > > > > > sounds like > > > > > > > >>> complicated for "just" getting an entry. But if you say > this > > > is > > > > > the way I > > > > > > > >>> will do it like that. I was just think that it could be a > > > common > > > > > use case > > > > > > > >>> and therefore provide it as a shortcut. > > > > > > > >>> > > > > > > > >>> Thanks, > > > > > > > >>> > > > > > > > >>> Romain. > > > > > > > >>> > > > > > > > >>> Le mer. 16 déc. 2015 à 11:34, Denis Magda < > > > [hidden email]> > > > > > a écrit : > > > > > > > >>> > > > > > > > >>>> Hi Romain, > > > > > > > >>>> > > > > > > > >>>> As the current documentation of > > > > > org.apache.ignite.cache.CacheEntry > > > > > > > >>>> states it's possible to get a reference to CacheEntry > and > > > its > > > > > version > > > > > > > >>>> only for methods that are executed over local data set > > > stored > > > > > on a node. > > > > > > > >>>> Among such methods are invoke & invokeAll and > randomEntry. > > > > > > > >>>> > > > > > > > >>>> You probably can get a CacheEntry and its non null > version > > > when > > > > > a cache > > > > > > > >>>> iterator is in use but the version will be 'null', as > far > > > as I > > > > > remember, > > > > > > > >>>> for those entries that are loaded from remote nodes. > > > > > > > >>>> Presently Ignite doesn't transfer the version from > remote > > > nodes > > > > > as a > > > > > > > >>>> part of response by perform reasons. > > > > > > > >>>> > > > > > > > >>>> Please elaborate more on your particular use case and > what > > > API > > > > > you would > > > > > > > >>>> like to add in order to support it. > > > > > > > >>>> > > > > > > > >>>> -- > > > > > > > >>>> Denis > > > > > > > >>>> > > > > > > > >>>> On 12/16/2015 12:58 PM, Romain Gilles wrote: > > > > > > > >>>>> Hi Igniters, > > > > > > > >>>>> I'm looking for a way to get access to the Ignite > > > CacheEntry. > > > > > For now > > > > > > > >>>> this > > > > > > > >>>>> is the ways I found: > > > > > > > >>>>> > > > > > > > >>>>> - Through the queris > > > > > > > >>>>> - Through jsr 107 Cache Iterable > > > > > > > >>>>> - Through jsr 107 Cache itterator > > > > > > > >>>>> - Through IgniteCache::randomEntry() > > > > > > > >>>>> > > > > > > > >>>>> If I remember correctly it was possible to get the > > > CacheEntry > > > > > from a > > > > > > > >>>> given > > > > > > > >>>>> key in old version of gridgain community > > > > > > > >>>>> version: GridCacheProjection::entry(K key) > > > GridCacheEntry<K,V> > > > > > > > >>>>> I think it could be a good to introduce this feature at > > > > > IgniteCache > > > > > > > >>>> level. > > > > > > > >>>>> Or maybe there is an other way to do it. > > > > > > > >>>>> > > > > > > > >>>>> Thanks, > > > > > > > >>>>> > > > > > > > >>>>> Romain. > > > > > > > >>>>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > |
RE: Missing ways to get access to Ignite CacheEntry
|
|
How would I know how to interpret the returned null?
Can we just have a simple getEntry(K) method that has semantics of the regular get(K)? Why invent something? Cheers Andrey > From: [hidden email] > Date: Fri, 18 Dec 2015 12:55:47 -0800 > Subject: Re: Missing ways to get access to Ignite CacheEntry > To: [hidden email] > > On Fri, Dec 18, 2015 at 12:40 PM, Andrey Kornev <[hidden email]> > wrote: > > > Dmitriy, > > > > I understand that we're talking about Ignite CacheEntry and not JCache's > > CacheEntry that has no version property to speak of. > > > > But I don't understand why the method is named localEntry()? What does > > "local" mean in this context? Does it mean that I can only use it on the > > primary node for a given key? What would happen if I call this method on a > > non-primary node? And what would I do if I needed the entry (along with the > > valid version it contains) on a non-primary node? > > > > If the node is either primary or backup, then you will get the entry back. > Otherwise, you will get null. > > > > > From: [hidden email] > > > Date: Fri, 18 Dec 2015 12:19:01 -0800 > > > Subject: Re: Missing ways to get access to Ignite CacheEntry > > > To: [hidden email] > > > > > > On Fri, Dec 18, 2015 at 9:47 AM, Andrey Kornev <[hidden email] > > > > > > wrote: > > > > > > > Dmitriy, > > > > > > > > Based on what criteria does one determine which information is > > redundant > > > > and which is not? > > > > > > > > To me, if an API declares a method getSomeRedundantInfo(), then the > > > > implementer has no choice, but to comply with the contract and provide > > such > > > > info. One can't just return a null or ignore it some other way, simply > > > > because it appears to have some performance impact. > > > > > > > > > Andrey, the CacheEntry is not the JCache entry, but rather the internal > > > Ignite entry that you can get when unwrapping the JCache entry. The > > > unwrapping, obviously, can only happen on the server side. Therefore, I > > am > > > suggesting that we also add a localEntry(K) method which will provide the > > > entry API on the server side directly (not from iterator or EP). > > > > > > Now I understand that the version() method currently does not work > > properly > > > on the server side, but this is something we can easily fix. > > > > > > Thoughts? > > > > > > > > > > Besides, the overhead of the version number may turn out to be > > negligible > > > > when benchmarked. Most likely there will be near zero overhead (due to > > the > > > > way the network stack packs the bytes into the ethernet frames). The > > > > workaround for the broken API however may be much more expensive in > > terms > > > > of performance and user experience. > > > > > > > > Please let's keep the API consistent and predictable. > > > > > > > > Regards > > > > Andrey > > > > > > > > > From: [hidden email] > > > > > Date: Fri, 18 Dec 2015 08:41:00 -0800 > > > > > Subject: Re: Missing ways to get access to Ignite CacheEntry > > > > > To: [hidden email] > > > > > > > > > > Andrey, > > > > > > > > > > The main issue with returning version is sending redundant > > information > > > > over > > > > > the wire. What if we added a method IgniteCache.localEntry(K) which > > would > > > > > return a local entry instance with a version. You would only be able > > to > > > > > invoke this method on the server node (not client). > > > > > > > > > > Would this work for you? > > > > > > > > > > D. > > > > > > > > > > On Fri, Dec 18, 2015 at 8:23 AM, Andrey Kornev < > > [hidden email] > > > > > > > > > > wrote: > > > > > > > > > > > Denis, > > > > > > > > > > > > I'm not suggesting to have every cache read operation return a > > > > CacheEntry. > > > > > > The "classical" GridGain used to have a getEntry(key) method that > > was > > > > very > > > > > > useful. The current version of Ignite provides virtually no way of > > > > > > obtaining a cache entry except thru an iterator, or using an > > > > EntryProcessor > > > > > > invocation which seems more like a hack. > > > > > > > > > > > > As for the question of what's stored in Ignite cache and the > > > > pre-compute, > > > > > > it is really an off-topic. It just came about as we were looking > > for a > > > > way > > > > > > to work around the issue with Ignite query not returning cache > > entries' > > > > > > version numbers in the result iterator. (There is now a ticket that > > > > > > addresses this issue: > > > > https://issues.apache.org/jira/browse/IGNITE-2202 > > > > > > )This is when we realized that there is no longer any way in > > Ignite to > > > > > > retrieve the CacheEntry by its key. > > > > > > > > > > > > But for the sake of completeness I quickly go over the entire use > > case, > > > > > > because it seems I wasn't clear enough in the previous post. > > > > > > > > > > > > Ignite cache holds the data entries that are used as the input into > > > > some > > > > > > expensive computation. The data in the cache gets updated from > > time to > > > > > > time, the computation only needs to be re-run > > > > > > - on-demand -- when a user asks for the computed value and > > > > > > - the pre-computed value is obsolete -- the source cache data has > > been > > > > > > updated since the last time the value was computed. > > > > > > > > > > > > Otherwise, we return the pre-computed value. Note that we don't > > want to > > > > > > rerun the computation for each cache update! Only when the users > > asks > > > > for > > > > > > it. > > > > > > > > > > > > I hope it make it more clear now. > > > > > > > > > > > > Thanks > > > > > > Andrey > > > > > > > > > > > > > Subject: Re: Missing ways to get access to Ignite CacheEntry > > > > > > > To: [hidden email] > > > > > > > From: [hidden email] > > > > > > > Date: Fri, 18 Dec 2015 13:44:16 +0300 > > > > > > > > > > > > > > Hi Andrey, > > > > > > > > > > > > > > I got you, thanks for the clarification. > > > > > > > > > > > > > > So since you're storing a computed value in some local data > > structure > > > > > > > what is stored in the Ignite cache as a value in such a case? > > There > > > > > > > should be something. > > > > > > > Why don't you (or can't you) store a version identifier in that > > value > > > > > > > that is located in the Ignite cache? This should work perfectly > > well > > > > for > > > > > > > you use-case. > > > > > > > > > > > > > > Personally I'm against adding CacheEntry to a response of every > > get > > > > like > > > > > > > operation that happens over a cache (get, getAll, SQL & Scan > > > > queries). > > > > > > > This will add extra size to every response and only in rare cases > > > > > > > someone will benefit from it. > > > > > > > However, what if we add a special configuration parameter to > > > > > > > CacheConfiguration that will manage whether CacheEntry has to be > > > > passed > > > > > > > as a part of a response or not? Igniters, what do you think on > > this? > > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > Denis > > > > > > > > > > > > > > > > > > > > > On 12/17/2015 9:09 AM, Andrey Kornev wrote: > > > > > > > > Denis,In this case I used the term "caching" in general sense, > > > > meaning > > > > > > saving the computed value for later. I didn't mean the Ignite > > > > cache.Sorry > > > > > > about the confusion.Andrey > > > > > > > > _____________________________ > > > > > > > > From: Denis Magda <[hidden email]> > > > > > > > > Sent: Wednesday, December 16, 2015 9:48 PM > > > > > > > > Subject: Re: Missing ways to get access to Ignite CacheEntry > > > > > > > > To: <[hidden email]> > > > > > > > > > > > > > > > > > > > > > > > > Hi Romain, > > > > > > > > > > > > > > > > I'm a bit confused cause initially you saying that > > > > > > > > > > > > > > > > /The value is computed on first access and it is then cached > > > > > > > > > > > > > > > > /and then you add that > > > > > > > > > > > > > > > > /One constraint is that the computed value is not > > serializable. > > > > // > > > > > > > > / > > > > > > > > > > > > > > > > > > > > > > > > Ignite won't be able to store a value of cache if > > serialization > > > > is > > > > > > not > > > > > > > > supported for it by some reason. > > > > > > > > > > > > > > > > Please clarify here. Probably I'm missing something from your > > > > > > description. > > > > > > > > > > > > > > > > -- > > > > > > > > Denis > > > > > > > > > > > > > > > > On 12/16/2015 6:21 PM, Andrey Kornev wrote: > > > > > > > > > Romain > > > > > > > > > > > > > > > > > > I suggest you be very careful using the invoke() > > > > functionality. As > > > > > > explained in this posting (and the follow-ups) > > > > > > > > > > > > > > > > > > > > > http://mail-archives.apache.org/mod_mbox/ignite-dev/201511.mbox/%3CSNT147-W14D2736E1374450B685721CF000%40phx.gbl%3E > > > > > > > > > the correct usage of the invoke() may be difficult, to put > > it > > > > > > politely. > > > > > > > > > > > > > > > > > > @Denis Very briefly, the use case is "conditional > > re-compute". > > > > > > There is a pretty expensive computation that needs to be applied to > > > > each > > > > > > entry on demand. The value is computed on first access and it is > > then > > > > > > cached along with the current version of the cache entry. The > > value > > > > must > > > > > > be recomputed on next access only if the entry has been modified. > > > > > > > > > > > > > > > > > > If anyone has a better idea how to achieve something like > > that, > > > > > > please do share! One constraint is that the computed value is not > > > > > > serializable. > > > > > > > > > > > > > > > > > > Cheers > > > > > > > > > Andrey > > > > > > > > > > > > > > > > > >> Subject: Re: Missing ways to get access to Ignite > > CacheEntry > > > > > > > > >> To: [hidden email] > > > > > > > > >> From: [hidden email] > > > > > > > > >> Date: Wed, 16 Dec 2015 16:11:48 +0300 > > > > > > > > >> > > > > > > > > >> Romain, > > > > > > > > >> > > > > > > > > >> If it's implemented this way for now it doesn't mean that > > we > > > > can't > > > > > > > > >> enhance the API for other use cases ;) > > > > > > > > >> As I said presently it's supported for limited number of > > > > methods > > > > > > mostly > > > > > > > > >> by performance reasons. > > > > > > > > >> > > > > > > > > >> Please go ahead and try invoke/invokeAll and if it doesn't > > > > work > > > > > > fine for > > > > > > > > >> you we can keep discussing what to do next. > > > > > > > > >> > > > > > > > > >> Regards, > > > > > > > > >> Denis > > > > > > > > >> > > > > > > > > >> On 12/16/2015 1:53 PM, Romain Gilles wrote: > > > > > > > > >>> Hi Denis, > > > > > > > > >>> Thanks for you replay. And sorry to not double check it > > > > before. > > > > > > I see that > > > > > > > > >>> if I want to work with CacheEntry, I need to use the > > invoke > > > > > > method in order > > > > > > > > >>> to return the CacheEntry. This is the way I should do > > it. It > > > > > > sounds like > > > > > > > > >>> complicated for "just" getting an entry. But if you say > > this > > > > is > > > > > > the way I > > > > > > > > >>> will do it like that. I was just think that it could be a > > > > common > > > > > > use case > > > > > > > > >>> and therefore provide it as a shortcut. > > > > > > > > >>> > > > > > > > > >>> Thanks, > > > > > > > > >>> > > > > > > > > >>> Romain. > > > > > > > > >>> > > > > > > > > >>> Le mer. 16 déc. 2015 à 11:34, Denis Magda < > > > > [hidden email]> > > > > > > a écrit : > > > > > > > > >>> > > > > > > > > >>>> Hi Romain, > > > > > > > > >>>> > > > > > > > > >>>> As the current documentation of > > > > > > org.apache.ignite.cache.CacheEntry > > > > > > > > >>>> states it's possible to get a reference to CacheEntry > > and > > > > its > > > > > > version > > > > > > > > >>>> only for methods that are executed over local data set > > > > stored > > > > > > on a node. > > > > > > > > >>>> Among such methods are invoke & invokeAll and > > randomEntry. > > > > > > > > >>>> > > > > > > > > >>>> You probably can get a CacheEntry and its non null > > version > > > > when > > > > > > a cache > > > > > > > > >>>> iterator is in use but the version will be 'null', as > > far > > > > as I > > > > > > remember, > > > > > > > > >>>> for those entries that are loaded from remote nodes. > > > > > > > > >>>> Presently Ignite doesn't transfer the version from > > remote > > > > nodes > > > > > > as a > > > > > > > > >>>> part of response by perform reasons. > > > > > > > > >>>> > > > > > > > > >>>> Please elaborate more on your particular use case and > > what > > > > API > > > > > > you would > > > > > > > > >>>> like to add in order to support it. > > > > > > > > >>>> > > > > > > > > >>>> -- > > > > > > > > >>>> Denis > > > > > > > > >>>> > > > > > > > > >>>> On 12/16/2015 12:58 PM, Romain Gilles wrote: > > > > > > > > >>>>> Hi Igniters, > > > > > > > > >>>>> I'm looking for a way to get access to the Ignite > > > > CacheEntry. > > > > > > For now > > > > > > > > >>>> this > > > > > > > > >>>>> is the ways I found: > > > > > > > > >>>>> > > > > > > > > >>>>> - Through the queris > > > > > > > > >>>>> - Through jsr 107 Cache Iterable > > > > > > > > >>>>> - Through jsr 107 Cache itterator > > > > > > > > >>>>> - Through IgniteCache::randomEntry() > > > > > > > > >>>>> > > > > > > > > >>>>> If I remember correctly it was possible to get the > > > > CacheEntry > > > > > > from a > > > > > > > > >>>> given > > > > > > > > >>>>> key in old version of gridgain community > > > > > > > > >>>>> version: GridCacheProjection::entry(K key) > > > > GridCacheEntry<K,V> > > > > > > > > >>>>> I think it could be a good to introduce this feature at > > > > > > IgniteCache > > > > > > > > >>>> level. > > > > > > > > >>>>> Or maybe there is an other way to do it. > > > > > > > > >>>>> > > > > > > > > >>>>> Thanks, > > > > > > > > >>>>> > > > > > > > > >>>>> Romain. > > > > > > > > >>>>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > |
Re: Missing ways to get access to Ignite CacheEntry
|
|
On Fri, Dec 18, 2015 at 1:23 PM, Andrey Kornev <[hidden email]>
wrote: > How would I know how to interpret the returned null? > > Can we just have a simple getEntry(K) method that has semantics of the > regular get(K)? Why invent something? > If we do this, how would you unwrap such entry on the client side? > > Cheers > Andrey > > > From: [hidden email] > > Date: Fri, 18 Dec 2015 12:55:47 -0800 > > Subject: Re: Missing ways to get access to Ignite CacheEntry > > To: [hidden email] > > > > On Fri, Dec 18, 2015 at 12:40 PM, Andrey Kornev < > [hidden email]> > > wrote: > > > > > Dmitriy, > > > > > > I understand that we're talking about Ignite CacheEntry and not > JCache's > > > CacheEntry that has no version property to speak of. > > > > > > But I don't understand why the method is named localEntry()? What does > > > "local" mean in this context? Does it mean that I can only use it on > the > > > primary node for a given key? What would happen if I call this method > on a > > > non-primary node? And what would I do if I needed the entry (along > with the > > > valid version it contains) on a non-primary node? > > > > > > > If the node is either primary or backup, then you will get the entry > back. > > Otherwise, you will get null. > > > > > > > > From: [hidden email] > > > > Date: Fri, 18 Dec 2015 12:19:01 -0800 > > > > Subject: Re: Missing ways to get access to Ignite CacheEntry > > > > To: [hidden email] > > > > > > > > On Fri, Dec 18, 2015 at 9:47 AM, Andrey Kornev < > [hidden email] > > > > > > > > wrote: > > > > > > > > > Dmitriy, > > > > > > > > > > Based on what criteria does one determine which information is > > > redundant > > > > > and which is not? > > > > > > > > > > To me, if an API declares a method getSomeRedundantInfo(), then the > > > > > implementer has no choice, but to comply with the contract and > provide > > > such > > > > > info. One can't just return a null or ignore it some other way, > simply > > > > > because it appears to have some performance impact. > > > > > > > > > > > > Andrey, the CacheEntry is not the JCache entry, but rather the > internal > > > > Ignite entry that you can get when unwrapping the JCache entry. The > > > > unwrapping, obviously, can only happen on the server side. > Therefore, I > > > am > > > > suggesting that we also add a localEntry(K) method which will > provide the > > > > entry API on the server side directly (not from iterator or EP). > > > > > > > > Now I understand that the version() method currently does not work > > > properly > > > > on the server side, but this is something we can easily fix. > > > > > > > > Thoughts? > > > > > > > > > > > > > Besides, the overhead of the version number may turn out to be > > > negligible > > > > > when benchmarked. Most likely there will be near zero overhead > (due to > > > the > > > > > way the network stack packs the bytes into the ethernet frames). > The > > > > > workaround for the broken API however may be much more expensive in > > > terms > > > > > of performance and user experience. > > > > > > > > > > Please let's keep the API consistent and predictable. > > > > > > > > > > Regards > > > > > Andrey > > > > > > > > > > > From: [hidden email] > > > > > > Date: Fri, 18 Dec 2015 08:41:00 -0800 > > > > > > Subject: Re: Missing ways to get access to Ignite CacheEntry > > > > > > To: [hidden email] > > > > > > > > > > > > Andrey, > > > > > > > > > > > > The main issue with returning version is sending redundant > > > information > > > > > over > > > > > > the wire. What if we added a method IgniteCache.localEntry(K) > which > > > would > > > > > > return a local entry instance with a version. You would only be > able > > > to > > > > > > invoke this method on the server node (not client). > > > > > > > > > > > > Would this work for you? > > > > > > > > > > > > D. > > > > > > > > > > > > On Fri, Dec 18, 2015 at 8:23 AM, Andrey Kornev < > > > [hidden email] > > > > > > > > > > > > wrote: > > > > > > > > > > > > > Denis, > > > > > > > > > > > > > > I'm not suggesting to have every cache read operation return a > > > > > CacheEntry. > > > > > > > The "classical" GridGain used to have a getEntry(key) method > that > > > was > > > > > very > > > > > > > useful. The current version of Ignite provides virtually no > way of > > > > > > > obtaining a cache entry except thru an iterator, or using an > > > > > EntryProcessor > > > > > > > invocation which seems more like a hack. > > > > > > > > > > > > > > As for the question of what's stored in Ignite cache and the > > > > > pre-compute, > > > > > > > it is really an off-topic. It just came about as we were > looking > > > for a > > > > > way > > > > > > > to work around the issue with Ignite query not returning cache > > > entries' > > > > > > > version numbers in the result iterator. (There is now a ticket > that > > > > > > > addresses this issue: > > > > > https://issues.apache.org/jira/browse/IGNITE-2202 > > > > > > > )This is when we realized that there is no longer any way in > > > Ignite to > > > > > > > retrieve the CacheEntry by its key. > > > > > > > > > > > > > > But for the sake of completeness I quickly go over the entire > use > > > case, > > > > > > > because it seems I wasn't clear enough in the previous post. > > > > > > > > > > > > > > Ignite cache holds the data entries that are used as the input > into > > > > > some > > > > > > > expensive computation. The data in the cache gets updated from > > > time to > > > > > > > time, the computation only needs to be re-run > > > > > > > - on-demand -- when a user asks for the computed value and > > > > > > > - the pre-computed value is obsolete -- the source cache data > has > > > been > > > > > > > updated since the last time the value was computed. > > > > > > > > > > > > > > Otherwise, we return the pre-computed value. Note that we don't > > > want to > > > > > > > rerun the computation for each cache update! Only when the > users > > > asks > > > > > for > > > > > > > it. > > > > > > > > > > > > > > I hope it make it more clear now. > > > > > > > > > > > > > > Thanks > > > > > > > Andrey > > > > > > > > > > > > > > > Subject: Re: Missing ways to get access to Ignite CacheEntry > > > > > > > > To: [hidden email] > > > > > > > > From: [hidden email] > > > > > > > > Date: Fri, 18 Dec 2015 13:44:16 +0300 > > > > > > > > > > > > > > > > Hi Andrey, > > > > > > > > > > > > > > > > I got you, thanks for the clarification. > > > > > > > > > > > > > > > > So since you're storing a computed value in some local data > > > structure > > > > > > > > what is stored in the Ignite cache as a value in such a case? > > > There > > > > > > > > should be something. > > > > > > > > Why don't you (or can't you) store a version identifier in > that > > > value > > > > > > > > that is located in the Ignite cache? This should work > perfectly > > > well > > > > > for > > > > > > > > you use-case. > > > > > > > > > > > > > > > > Personally I'm against adding CacheEntry to a response of > every > > > get > > > > > like > > > > > > > > operation that happens over a cache (get, getAll, SQL & Scan > > > > > queries). > > > > > > > > This will add extra size to every response and only in rare > cases > > > > > > > > someone will benefit from it. > > > > > > > > However, what if we add a special configuration parameter to > > > > > > > > CacheConfiguration that will manage whether CacheEntry has > to be > > > > > passed > > > > > > > > as a part of a response or not? Igniters, what do you think > on > > > this? > > > > > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > Denis > > > > > > > > > > > > > > > > > > > > > > > > On 12/17/2015 9:09 AM, Andrey Kornev wrote: > > > > > > > > > Denis,In this case I used the term "caching" in general > sense, > > > > > meaning > > > > > > > saving the computed value for later. I didn't mean the Ignite > > > > > cache.Sorry > > > > > > > about the confusion.Andrey > > > > > > > > > _____________________________ > > > > > > > > > From: Denis Magda <[hidden email]> > > > > > > > > > Sent: Wednesday, December 16, 2015 9:48 PM > > > > > > > > > Subject: Re: Missing ways to get access to Ignite > CacheEntry > > > > > > > > > To: <[hidden email]> > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Romain, > > > > > > > > > > > > > > > > > > I'm a bit confused cause initially you saying that > > > > > > > > > > > > > > > > > > /The value is computed on first access and it is then > cached > > > > > > > > > > > > > > > > > > /and then you add that > > > > > > > > > > > > > > > > > > /One constraint is that the computed value is not > > > serializable. > > > > > // > > > > > > > > > / > > > > > > > > > > > > > > > > > > > > > > > > > > > Ignite won't be able to store a value of cache if > > > serialization > > > > > is > > > > > > > not > > > > > > > > > supported for it by some reason. > > > > > > > > > > > > > > > > > > Please clarify here. Probably I'm missing something from > your > > > > > > > description. > > > > > > > > > > > > > > > > > > -- > > > > > > > > > Denis > > > > > > > > > > > > > > > > > > On 12/16/2015 6:21 PM, Andrey Kornev wrote: > > > > > > > > > > Romain > > > > > > > > > > > > > > > > > > > > I suggest you be very careful using the invoke() > > > > > functionality. As > > > > > > > explained in this posting (and the follow-ups) > > > > > > > > > > > > > > > > > > > > > > > > > > http://mail-archives.apache.org/mod_mbox/ignite-dev/201511.mbox/%3CSNT147-W14D2736E1374450B685721CF000%40phx.gbl%3E > > > > > > > > > > the correct usage of the invoke() may be difficult, to > put > > > it > > > > > > > politely. > > > > > > > > > > > > > > > > > > > > @Denis Very briefly, the use case is "conditional > > > re-compute". > > > > > > > There is a pretty expensive computation that needs to be > applied to > > > > > each > > > > > > > entry on demand. The value is computed on first access and it > is > > > then > > > > > > > cached along with the current version of the cache entry. The > > > value > > > > > must > > > > > > > be recomputed on next access only if the entry has been > modified. > > > > > > > > > > > > > > > > > > > > If anyone has a better idea how to achieve something > like > > > that, > > > > > > > please do share! One constraint is that the computed value is > not > > > > > > > serializable. > > > > > > > > > > > > > > > > > > > > Cheers > > > > > > > > > > Andrey > > > > > > > > > > > > > > > > > > > >> Subject: Re: Missing ways to get access to Ignite > > > CacheEntry > > > > > > > > > >> To: [hidden email] > > > > > > > > > >> From: [hidden email] > > > > > > > > > >> Date: Wed, 16 Dec 2015 16:11:48 +0300 > > > > > > > > > >> > > > > > > > > > >> Romain, > > > > > > > > > >> > > > > > > > > > >> If it's implemented this way for now it doesn't mean > that > > > we > > > > > can't > > > > > > > > > >> enhance the API for other use cases ;) > > > > > > > > > >> As I said presently it's supported for limited number > of > > > > > methods > > > > > > > mostly > > > > > > > > > >> by performance reasons. > > > > > > > > > >> > > > > > > > > > >> Please go ahead and try invoke/invokeAll and if it > doesn't > > > > > work > > > > > > > fine for > > > > > > > > > >> you we can keep discussing what to do next. > > > > > > > > > >> > > > > > > > > > >> Regards, > > > > > > > > > >> Denis > > > > > > > > > >> > > > > > > > > > >> On 12/16/2015 1:53 PM, Romain Gilles wrote: > > > > > > > > > >>> Hi Denis, > > > > > > > > > >>> Thanks for you replay. And sorry to not double check > it > > > > > before. > > > > > > > I see that > > > > > > > > > >>> if I want to work with CacheEntry, I need to use the > > > invoke > > > > > > > method in order > > > > > > > > > >>> to return the CacheEntry. This is the way I should do > > > it. It > > > > > > > sounds like > > > > > > > > > >>> complicated for "just" getting an entry. But if you > say > > > this > > > > > is > > > > > > > the way I > > > > > > > > > >>> will do it like that. I was just think that it could > be a > > > > > common > > > > > > > use case > > > > > > > > > >>> and therefore provide it as a shortcut. > > > > > > > > > >>> > > > > > > > > > >>> Thanks, > > > > > > > > > >>> > > > > > > > > > >>> Romain. > > > > > > > > > >>> > > > > > > > > > >>> Le mer. 16 déc. 2015 à 11:34, Denis Magda < > > > > > [hidden email]> > > > > > > > a écrit : > > > > > > > > > >>> > > > > > > > > > >>>> Hi Romain, > > > > > > > > > >>>> > > > > > > > > > >>>> As the current documentation of > > > > > > > org.apache.ignite.cache.CacheEntry > > > > > > > > > >>>> states it's possible to get a reference to > CacheEntry > > > and > > > > > its > > > > > > > version > > > > > > > > > >>>> only for methods that are executed over local data > set > > > > > stored > > > > > > > on a node. > > > > > > > > > >>>> Among such methods are invoke & invokeAll and > > > randomEntry. > > > > > > > > > >>>> > > > > > > > > > >>>> You probably can get a CacheEntry and its non null > > > version > > > > > when > > > > > > > a cache > > > > > > > > > >>>> iterator is in use but the version will be 'null', > as > > > far > > > > > as I > > > > > > > remember, > > > > > > > > > >>>> for those entries that are loaded from remote nodes. > > > > > > > > > >>>> Presently Ignite doesn't transfer the version from > > > remote > > > > > nodes > > > > > > > as a > > > > > > > > > >>>> part of response by perform reasons. > > > > > > > > > >>>> > > > > > > > > > >>>> Please elaborate more on your particular use case > and > > > what > > > > > API > > > > > > > you would > > > > > > > > > >>>> like to add in order to support it. > > > > > > > > > >>>> > > > > > > > > > >>>> -- > > > > > > > > > >>>> Denis > > > > > > > > > >>>> > > > > > > > > > >>>> On 12/16/2015 12:58 PM, Romain Gilles wrote: > > > > > > > > > >>>>> Hi Igniters, > > > > > > > > > >>>>> I'm looking for a way to get access to the Ignite > > > > > CacheEntry. > > > > > > > For now > > > > > > > > > >>>> this > > > > > > > > > >>>>> is the ways I found: > > > > > > > > > >>>>> > > > > > > > > > >>>>> - Through the queris > > > > > > > > > >>>>> - Through jsr 107 Cache Iterable > > > > > > > > > >>>>> - Through jsr 107 Cache itterator > > > > > > > > > >>>>> - Through IgniteCache::randomEntry() > > > > > > > > > >>>>> > > > > > > > > > >>>>> If I remember correctly it was possible to get the > > > > > CacheEntry > > > > > > > from a > > > > > > > > > >>>> given > > > > > > > > > >>>>> key in old version of gridgain community > > > > > > > > > >>>>> version: GridCacheProjection::entry(K key) > > > > > GridCacheEntry<K,V> > > > > > > > > > >>>>> I think it could be a good to introduce this > feature at > > > > > > > IgniteCache > > > > > > > > > >>>> level. > > > > > > > > > >>>>> Or maybe there is an other way to do it. > > > > > > > > > >>>>> > > > > > > > > > >>>>> Thanks, > > > > > > > > > >>>>> > > > > > > > > > >>>>> Romain. > > > > > > > > > >>>>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > |
Re: Missing ways to get access to Ignite CacheEntry
|
|
Actually I think that Andrey's proposal on addition of
IgniteCache.getEntry(k) sounds natural. Is there any platform limitation that will prevent us from loading an entry with its version from a remote node when such a particular method is used? I think that it's feasible to support it and in addition I would add on more method to the API - IgniteCache.getAllEntries(keysSet). -- Denis On 12/19/2015 2:05 AM, Dmitriy Setrakyan wrote: > On Fri, Dec 18, 2015 at 1:23 PM, Andrey Kornev <[hidden email]> > wrote: > >> How would I know how to interpret the returned null? >> >> Can we just have a simple getEntry(K) method that has semantics of the >> regular get(K)? Why invent something? >> > If we do this, how would you unwrap such entry on the client side? > > >> Cheers >> Andrey >> >>> From: [hidden email] >>> Date: Fri, 18 Dec 2015 12:55:47 -0800 >>> Subject: Re: Missing ways to get access to Ignite CacheEntry >>> To: [hidden email] >>> >>> On Fri, Dec 18, 2015 at 12:40 PM, Andrey Kornev < >> [hidden email]> >>> wrote: >>> >>>> Dmitriy, >>>> >>>> I understand that we're talking about Ignite CacheEntry and not >> JCache's >>>> CacheEntry that has no version property to speak of. >>>> >>>> But I don't understand why the method is named localEntry()? What does >>>> "local" mean in this context? Does it mean that I can only use it on >> the >>>> primary node for a given key? What would happen if I call this method >> on a >>>> non-primary node? And what would I do if I needed the entry (along >> with the >>>> valid version it contains) on a non-primary node? >>>> >>> If the node is either primary or backup, then you will get the entry >> back. >>> Otherwise, you will get null. >>> >>> >>>>> From: [hidden email] >>>>> Date: Fri, 18 Dec 2015 12:19:01 -0800 >>>>> Subject: Re: Missing ways to get access to Ignite CacheEntry >>>>> To: [hidden email] >>>>> >>>>> On Fri, Dec 18, 2015 at 9:47 AM, Andrey Kornev < >> [hidden email] >>>>> wrote: >>>>> >>>>>> Dmitriy, >>>>>> >>>>>> Based on what criteria does one determine which information is >>>> redundant >>>>>> and which is not? >>>>>> >>>>>> To me, if an API declares a method getSomeRedundantInfo(), then the >>>>>> implementer has no choice, but to comply with the contract and >> provide >>>> such >>>>>> info. One can't just return a null or ignore it some other way, >> simply >>>>>> because it appears to have some performance impact. >>>>> >>>>> Andrey, the CacheEntry is not the JCache entry, but rather the >> internal >>>>> Ignite entry that you can get when unwrapping the JCache entry. The >>>>> unwrapping, obviously, can only happen on the server side. >> Therefore, I >>>> am >>>>> suggesting that we also add a localEntry(K) method which will >> provide the >>>>> entry API on the server side directly (not from iterator or EP). >>>>> >>>>> Now I understand that the version() method currently does not work >>>> properly >>>>> on the server side, but this is something we can easily fix. >>>>> >>>>> Thoughts? >>>>> >>>>> >>>>>> Besides, the overhead of the version number may turn out to be >>>> negligible >>>>>> when benchmarked. Most likely there will be near zero overhead >> (due to >>>> the >>>>>> way the network stack packs the bytes into the ethernet frames). >> The >>>>>> workaround for the broken API however may be much more expensive in >>>> terms >>>>>> of performance and user experience. >>>>>> >>>>>> Please let's keep the API consistent and predictable. >>>>>> >>>>>> Regards >>>>>> Andrey >>>>>> >>>>>>> From: [hidden email] >>>>>>> Date: Fri, 18 Dec 2015 08:41:00 -0800 >>>>>>> Subject: Re: Missing ways to get access to Ignite CacheEntry >>>>>>> To: [hidden email] >>>>>>> >>>>>>> Andrey, >>>>>>> >>>>>>> The main issue with returning version is sending redundant >>>> information >>>>>> over >>>>>>> the wire. What if we added a method IgniteCache.localEntry(K) >> which >>>> would >>>>>>> return a local entry instance with a version. You would only be >> able >>>> to >>>>>>> invoke this method on the server node (not client). >>>>>>> >>>>>>> Would this work for you? >>>>>>> >>>>>>> D. >>>>>>> >>>>>>> On Fri, Dec 18, 2015 at 8:23 AM, Andrey Kornev < >>>> [hidden email] >>>>>>> wrote: >>>>>>> >>>>>>>> Denis, >>>>>>>> >>>>>>>> I'm not suggesting to have every cache read operation return a >>>>>> CacheEntry. >>>>>>>> The "classical" GridGain used to have a getEntry(key) method >> that >>>> was >>>>>> very >>>>>>>> useful. The current version of Ignite provides virtually no >> way of >>>>>>>> obtaining a cache entry except thru an iterator, or using an >>>>>> EntryProcessor >>>>>>>> invocation which seems more like a hack. >>>>>>>> >>>>>>>> As for the question of what's stored in Ignite cache and the >>>>>> pre-compute, >>>>>>>> it is really an off-topic. It just came about as we were >> looking >>>> for a >>>>>> way >>>>>>>> to work around the issue with Ignite query not returning cache >>>> entries' >>>>>>>> version numbers in the result iterator. (There is now a ticket >> that >>>>>>>> addresses this issue: >>>>>> https://issues.apache.org/jira/browse/IGNITE-2202 >>>>>>>> )This is when we realized that there is no longer any way in >>>> Ignite to >>>>>>>> retrieve the CacheEntry by its key. >>>>>>>> >>>>>>>> But for the sake of completeness I quickly go over the entire >> use >>>> case, >>>>>>>> because it seems I wasn't clear enough in the previous post. >>>>>>>> >>>>>>>> Ignite cache holds the data entries that are used as the input >> into >>>>>> some >>>>>>>> expensive computation. The data in the cache gets updated from >>>> time to >>>>>>>> time, the computation only needs to be re-run >>>>>>>> - on-demand -- when a user asks for the computed value and >>>>>>>> - the pre-computed value is obsolete -- the source cache data >> has >>>> been >>>>>>>> updated since the last time the value was computed. >>>>>>>> >>>>>>>> Otherwise, we return the pre-computed value. Note that we don't >>>> want to >>>>>>>> rerun the computation for each cache update! Only when the >> users >>>> asks >>>>>> for >>>>>>>> it. >>>>>>>> >>>>>>>> I hope it make it more clear now. >>>>>>>> >>>>>>>> Thanks >>>>>>>> Andrey >>>>>>>> >>>>>>>>> Subject: Re: Missing ways to get access to Ignite CacheEntry >>>>>>>>> To: [hidden email] >>>>>>>>> From: [hidden email] >>>>>>>>> Date: Fri, 18 Dec 2015 13:44:16 +0300 >>>>>>>>> >>>>>>>>> Hi Andrey, >>>>>>>>> >>>>>>>>> I got you, thanks for the clarification. >>>>>>>>> >>>>>>>>> So since you're storing a computed value in some local data >>>> structure >>>>>>>>> what is stored in the Ignite cache as a value in such a case? >>>> There >>>>>>>>> should be something. >>>>>>>>> Why don't you (or can't you) store a version identifier in >> that >>>> value >>>>>>>>> that is located in the Ignite cache? This should work >> perfectly >>>> well >>>>>> for >>>>>>>>> you use-case. >>>>>>>>> >>>>>>>>> Personally I'm against adding CacheEntry to a response of >> every >>>> get >>>>>> like >>>>>>>>> operation that happens over a cache (get, getAll, SQL & Scan >>>>>> queries). >>>>>>>>> This will add extra size to every response and only in rare >> cases >>>>>>>>> someone will benefit from it. >>>>>>>>> However, what if we add a special configuration parameter to >>>>>>>>> CacheConfiguration that will manage whether CacheEntry has >> to be >>>>>> passed >>>>>>>>> as a part of a response or not? Igniters, what do you think >> on >>>> this? >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Denis >>>>>>>>> >>>>>>>>> >>>>>>>>> On 12/17/2015 9:09 AM, Andrey Kornev wrote: >>>>>>>>>> Denis,In this case I used the term "caching" in general >> sense, >>>>>> meaning >>>>>>>> saving the computed value for later. I didn't mean the Ignite >>>>>> cache.Sorry >>>>>>>> about the confusion.Andrey >>>>>>>>>> _____________________________ >>>>>>>>>> From: Denis Magda <[hidden email]> >>>>>>>>>> Sent: Wednesday, December 16, 2015 9:48 PM >>>>>>>>>> Subject: Re: Missing ways to get access to Ignite >> CacheEntry >>>>>>>>>> To: <[hidden email]> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hi Romain, >>>>>>>>>> >>>>>>>>>> I'm a bit confused cause initially you saying that >>>>>>>>>> >>>>>>>>>> /The value is computed on first access and it is then >> cached >>>>>>>>>> /and then you add that >>>>>>>>>> >>>>>>>>>> /One constraint is that the computed value is not >>>> serializable. >>>>>> // >>>>>>>>>> / >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Ignite won't be able to store a value of cache if >>>> serialization >>>>>> is >>>>>>>> not >>>>>>>>>> supported for it by some reason. >>>>>>>>>> >>>>>>>>>> Please clarify here. Probably I'm missing something from >> your >>>>>>>> description. >>>>>>>>>> -- >>>>>>>>>> Denis >>>>>>>>>> >>>>>>>>>> On 12/16/2015 6:21 PM, Andrey Kornev wrote: >>>>>>>>>> > Romain >>>>>>>>>> > >>>>>>>>>> > I suggest you be very careful using the invoke() >>>>>> functionality. As >>>>>>>> explained in this posting (and the follow-ups) >>>>>>>>>> > >> http://mail-archives.apache.org/mod_mbox/ignite-dev/201511.mbox/%3CSNT147-W14D2736E1374450B685721CF000%40phx.gbl%3E >>>>>>>>>> > the correct usage of the invoke() may be difficult, to >> put >>>> it >>>>>>>> politely. >>>>>>>>>> > >>>>>>>>>> > @Denis Very briefly, the use case is "conditional >>>> re-compute". >>>>>>>> There is a pretty expensive computation that needs to be >> applied to >>>>>> each >>>>>>>> entry on demand. The value is computed on first access and it >> is >>>> then >>>>>>>> cached along with the current version of the cache entry. The >>>> value >>>>>> must >>>>>>>> be recomputed on next access only if the entry has been >> modified. >>>>>>>>>> > >>>>>>>>>> > If anyone has a better idea how to achieve something >> like >>>> that, >>>>>>>> please do share! One constraint is that the computed value is >> not >>>>>>>> serializable. >>>>>>>>>> > >>>>>>>>>> > Cheers >>>>>>>>>> > Andrey >>>>>>>>>> > >>>>>>>>>> >> Subject: Re: Missing ways to get access to Ignite >>>> CacheEntry >>>>>>>>>> >> To: [hidden email] >>>>>>>>>> >> From: [hidden email] >>>>>>>>>> >> Date: Wed, 16 Dec 2015 16:11:48 +0300 >>>>>>>>>> >> >>>>>>>>>> >> Romain, >>>>>>>>>> >> >>>>>>>>>> >> If it's implemented this way for now it doesn't mean >> that >>>> we >>>>>> can't >>>>>>>>>> >> enhance the API for other use cases ;) >>>>>>>>>> >> As I said presently it's supported for limited number >> of >>>>>> methods >>>>>>>> mostly >>>>>>>>>> >> by performance reasons. >>>>>>>>>> >> >>>>>>>>>> >> Please go ahead and try invoke/invokeAll and if it >> doesn't >>>>>> work >>>>>>>> fine for >>>>>>>>>> >> you we can keep discussing what to do next. >>>>>>>>>> >> >>>>>>>>>> >> Regards, >>>>>>>>>> >> Denis >>>>>>>>>> >> >>>>>>>>>> >> On 12/16/2015 1:53 PM, Romain Gilles wrote: >>>>>>>>>> >>> Hi Denis, >>>>>>>>>> >>> Thanks for you replay. And sorry to not double check >> it >>>>>> before. >>>>>>>> I see that >>>>>>>>>> >>> if I want to work with CacheEntry, I need to use the >>>> invoke >>>>>>>> method in order >>>>>>>>>> >>> to return the CacheEntry. This is the way I should do >>>> it. It >>>>>>>> sounds like >>>>>>>>>> >>> complicated for "just" getting an entry. But if you >> say >>>> this >>>>>> is >>>>>>>> the way I >>>>>>>>>> >>> will do it like that. I was just think that it could >> be a >>>>>> common >>>>>>>> use case >>>>>>>>>> >>> and therefore provide it as a shortcut. >>>>>>>>>> >>> >>>>>>>>>> >>> Thanks, >>>>>>>>>> >>> >>>>>>>>>> >>> Romain. >>>>>>>>>> >>> >>>>>>>>>> >>> Le mer. 16 déc. 2015 à 11:34, Denis Magda < >>>>>> [hidden email]> >>>>>>>> a écrit : >>>>>>>>>> >>> >>>>>>>>>> >>>> Hi Romain, >>>>>>>>>> >>>> >>>>>>>>>> >>>> As the current documentation of >>>>>>>> org.apache.ignite.cache.CacheEntry >>>>>>>>>> >>>> states it's possible to get a reference to >> CacheEntry >>>> and >>>>>> its >>>>>>>> version >>>>>>>>>> >>>> only for methods that are executed over local data >> set >>>>>> stored >>>>>>>> on a node. >>>>>>>>>> >>>> Among such methods are invoke & invokeAll and >>>> randomEntry. >>>>>>>>>> >>>> >>>>>>>>>> >>>> You probably can get a CacheEntry and its non null >>>> version >>>>>> when >>>>>>>> a cache >>>>>>>>>> >>>> iterator is in use but the version will be 'null', >> as >>>> far >>>>>> as I >>>>>>>> remember, >>>>>>>>>> >>>> for those entries that are loaded from remote nodes. >>>>>>>>>> >>>> Presently Ignite doesn't transfer the version from >>>> remote >>>>>> nodes >>>>>>>> as a >>>>>>>>>> >>>> part of response by perform reasons. >>>>>>>>>> >>>> >>>>>>>>>> >>>> Please elaborate more on your particular use case >> and >>>> what >>>>>> API >>>>>>>> you would >>>>>>>>>> >>>> like to add in order to support it. >>>>>>>>>> >>>> >>>>>>>>>> >>>> -- >>>>>>>>>> >>>> Denis >>>>>>>>>> >>>> >>>>>>>>>> >>>> On 12/16/2015 12:58 PM, Romain Gilles wrote: >>>>>>>>>> >>>>> Hi Igniters, >>>>>>>>>> >>>>> I'm looking for a way to get access to the Ignite >>>>>> CacheEntry. >>>>>>>> For now >>>>>>>>>> >>>> this >>>>>>>>>> >>>>> is the ways I found: >>>>>>>>>> >>>>> >>>>>>>>>> >>>>> - Through the queris >>>>>>>>>> >>>>> - Through jsr 107 Cache Iterable >>>>>>>>>> >>>>> - Through jsr 107 Cache itterator >>>>>>>>>> >>>>> - Through IgniteCache::randomEntry() >>>>>>>>>> >>>>> >>>>>>>>>> >>>>> If I remember correctly it was possible to get the >>>>>> CacheEntry >>>>>>>> from a >>>>>>>>>> >>>> given >>>>>>>>>> >>>>> key in old version of gridgain community >>>>>>>>>> >>>>> version: GridCacheProjection::entry(K key) >>>>>> GridCacheEntry<K,V> >>>>>>>>>> >>>>> I think it could be a good to introduce this >> feature at >>>>>>>> IgniteCache >>>>>>>>>> >>>> level. >>>>>>>>>> >>>>> Or maybe there is an other way to do it. >>>>>>>>>> >>>>> >>>>>>>>>> >>>>> Thanks, >>>>>>>>>> >>>>> >>>>>>>>>> >>>>> Romain. >>>>>>>>>> >>>>> >>>>>>>>>> > >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>> >>>> >> |
Re: Missing ways to get access to Ignite CacheEntry
|
|
Again, my issue is with unwrap method. Are we all in agreement that
unwrap(…) will not work on the client side? D. On Sun, Dec 20, 2015 at 11:30 PM, Denis Magda <[hidden email]> wrote: > Actually I think that Andrey's proposal on addition of > IgniteCache.getEntry(k) sounds natural. > > Is there any platform limitation that will prevent us from loading an > entry with its version from a remote node when such a particular method is > used? > I think that it's feasible to support it and in addition I would add on > more method to the API - IgniteCache.getAllEntries(keysSet). > > -- > Denis > > > On 12/19/2015 2:05 AM, Dmitriy Setrakyan wrote: > >> On Fri, Dec 18, 2015 at 1:23 PM, Andrey Kornev <[hidden email]> >> wrote: >> >> How would I know how to interpret the returned null? >>> >>> Can we just have a simple getEntry(K) method that has semantics of the >>> regular get(K)? Why invent something? >>> >>> If we do this, how would you unwrap such entry on the client side? >> >> >> Cheers >>> Andrey >>> >>> From: [hidden email] >>>> Date: Fri, 18 Dec 2015 12:55:47 -0800 >>>> Subject: Re: Missing ways to get access to Ignite CacheEntry >>>> To: [hidden email] >>>> >>>> On Fri, Dec 18, 2015 at 12:40 PM, Andrey Kornev < >>>> >>> [hidden email]> >>> >>>> wrote: >>>> >>>> Dmitriy, >>>>> >>>>> I understand that we're talking about Ignite CacheEntry and not >>>>> >>>> JCache's >>> >>>> CacheEntry that has no version property to speak of. >>>>> >>>>> But I don't understand why the method is named localEntry()? What does >>>>> "local" mean in this context? Does it mean that I can only use it on >>>>> >>>> the >>> >>>> primary node for a given key? What would happen if I call this method >>>>> >>>> on a >>> >>>> non-primary node? And what would I do if I needed the entry (along >>>>> >>>> with the >>> >>>> valid version it contains) on a non-primary node? >>>>> >>>>> If the node is either primary or backup, then you will get the entry >>>> >>> back. >>> >>>> Otherwise, you will get null. >>>> >>>> >>>> From: [hidden email] >>>>>> Date: Fri, 18 Dec 2015 12:19:01 -0800 >>>>>> Subject: Re: Missing ways to get access to Ignite CacheEntry >>>>>> To: [hidden email] >>>>>> >>>>>> On Fri, Dec 18, 2015 at 9:47 AM, Andrey Kornev < >>>>>> >>>>> [hidden email] >>> >>>> wrote: >>>>>> >>>>>> Dmitriy, >>>>>>> >>>>>>> Based on what criteria does one determine which information is >>>>>>> >>>>>> redundant >>>>> >>>>>> and which is not? >>>>>>> >>>>>>> To me, if an API declares a method getSomeRedundantInfo(), then the >>>>>>> implementer has no choice, but to comply with the contract and >>>>>>> >>>>>> provide >>> >>>> such >>>>> >>>>>> info. One can't just return a null or ignore it some other way, >>>>>>> >>>>>> simply >>> >>>> because it appears to have some performance impact. >>>>>>> >>>>>> >>>>>> Andrey, the CacheEntry is not the JCache entry, but rather the >>>>>> >>>>> internal >>> >>>> Ignite entry that you can get when unwrapping the JCache entry. The >>>>>> unwrapping, obviously, can only happen on the server side. >>>>>> >>>>> Therefore, I >>> >>>> am >>>>> >>>>>> suggesting that we also add a localEntry(K) method which will >>>>>> >>>>> provide the >>> >>>> entry API on the server side directly (not from iterator or EP). >>>>>> >>>>>> Now I understand that the version() method currently does not work >>>>>> >>>>> properly >>>>> >>>>>> on the server side, but this is something we can easily fix. >>>>>> >>>>>> Thoughts? >>>>>> >>>>>> >>>>>> Besides, the overhead of the version number may turn out to be >>>>>>> >>>>>> negligible >>>>> >>>>>> when benchmarked. Most likely there will be near zero overhead >>>>>>> >>>>>> (due to >>> >>>> the >>>>> >>>>>> way the network stack packs the bytes into the ethernet frames). >>>>>>> >>>>>> The >>> >>>> workaround for the broken API however may be much more expensive in >>>>>>> >>>>>> terms >>>>> >>>>>> of performance and user experience. >>>>>>> >>>>>>> Please let's keep the API consistent and predictable. >>>>>>> >>>>>>> Regards >>>>>>> Andrey >>>>>>> >>>>>>> From: [hidden email] >>>>>>>> Date: Fri, 18 Dec 2015 08:41:00 -0800 >>>>>>>> Subject: Re: Missing ways to get access to Ignite CacheEntry >>>>>>>> To: [hidden email] >>>>>>>> >>>>>>>> Andrey, >>>>>>>> >>>>>>>> The main issue with returning version is sending redundant >>>>>>>> >>>>>>> information >>>>> >>>>>> over >>>>>>> >>>>>>>> the wire. What if we added a method IgniteCache.localEntry(K) >>>>>>>> >>>>>>> which >>> >>>> would >>>>> >>>>>> return a local entry instance with a version. You would only be >>>>>>>> >>>>>>> able >>> >>>> to >>>>> >>>>>> invoke this method on the server node (not client). >>>>>>>> >>>>>>>> Would this work for you? >>>>>>>> >>>>>>>> D. >>>>>>>> >>>>>>>> On Fri, Dec 18, 2015 at 8:23 AM, Andrey Kornev < >>>>>>>> >>>>>>> [hidden email] >>>>> >>>>>> wrote: >>>>>>>> >>>>>>>> Denis, >>>>>>>>> >>>>>>>>> I'm not suggesting to have every cache read operation return a >>>>>>>>> >>>>>>>> CacheEntry. >>>>>>> >>>>>>>> The "classical" GridGain used to have a getEntry(key) method >>>>>>>>> >>>>>>>> that >>> >>>> was >>>>> >>>>>> very >>>>>>> >>>>>>>> useful. The current version of Ignite provides virtually no >>>>>>>>> >>>>>>>> way of >>> >>>> obtaining a cache entry except thru an iterator, or using an >>>>>>>>> >>>>>>>> EntryProcessor >>>>>>> >>>>>>>> invocation which seems more like a hack. >>>>>>>>> >>>>>>>>> As for the question of what's stored in Ignite cache and the >>>>>>>>> >>>>>>>> pre-compute, >>>>>>> >>>>>>>> it is really an off-topic. It just came about as we were >>>>>>>>> >>>>>>>> looking >>> >>>> for a >>>>> >>>>>> way >>>>>>> >>>>>>>> to work around the issue with Ignite query not returning cache >>>>>>>>> >>>>>>>> entries' >>>>> >>>>>> version numbers in the result iterator. (There is now a ticket >>>>>>>>> >>>>>>>> that >>> >>>> addresses this issue: >>>>>>>>> >>>>>>>> https://issues.apache.org/jira/browse/IGNITE-2202 >>>>>>> >>>>>>>> )This is when we realized that there is no longer any way in >>>>>>>>> >>>>>>>> Ignite to >>>>> >>>>>> retrieve the CacheEntry by its key. >>>>>>>>> >>>>>>>>> But for the sake of completeness I quickly go over the entire >>>>>>>>> >>>>>>>> use >>> >>>> case, >>>>> >>>>>> because it seems I wasn't clear enough in the previous post. >>>>>>>>> >>>>>>>>> Ignite cache holds the data entries that are used as the input >>>>>>>>> >>>>>>>> into >>> >>>> some >>>>>>> >>>>>>>> expensive computation. The data in the cache gets updated from >>>>>>>>> >>>>>>>> time to >>>>> >>>>>> time, the computation only needs to be re-run >>>>>>>>> - on-demand -- when a user asks for the computed value and >>>>>>>>> - the pre-computed value is obsolete -- the source cache data >>>>>>>>> >>>>>>>> has >>> >>>> been >>>>> >>>>>> updated since the last time the value was computed. >>>>>>>>> >>>>>>>>> Otherwise, we return the pre-computed value. Note that we don't >>>>>>>>> >>>>>>>> want to >>>>> >>>>>> rerun the computation for each cache update! Only when the >>>>>>>>> >>>>>>>> users >>> >>>> asks >>>>> >>>>>> for >>>>>>> >>>>>>>> it. >>>>>>>>> >>>>>>>>> I hope it make it more clear now. >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> Andrey >>>>>>>>> >>>>>>>>> Subject: Re: Missing ways to get access to Ignite CacheEntry >>>>>>>>>> To: [hidden email] >>>>>>>>>> From: [hidden email] >>>>>>>>>> Date: Fri, 18 Dec 2015 13:44:16 +0300 >>>>>>>>>> >>>>>>>>>> Hi Andrey, >>>>>>>>>> >>>>>>>>>> I got you, thanks for the clarification. >>>>>>>>>> >>>>>>>>>> So since you're storing a computed value in some local data >>>>>>>>>> >>>>>>>>> structure >>>>> >>>>>> what is stored in the Ignite cache as a value in such a case? >>>>>>>>>> >>>>>>>>> There >>>>> >>>>>> should be something. >>>>>>>>>> Why don't you (or can't you) store a version identifier in >>>>>>>>>> >>>>>>>>> that >>> >>>> value >>>>> >>>>>> that is located in the Ignite cache? This should work >>>>>>>>>> >>>>>>>>> perfectly >>> >>>> well >>>>> >>>>>> for >>>>>>> >>>>>>>> you use-case. >>>>>>>>>> >>>>>>>>>> Personally I'm against adding CacheEntry to a response of >>>>>>>>>> >>>>>>>>> every >>> >>>> get >>>>> >>>>>> like >>>>>>> >>>>>>>> operation that happens over a cache (get, getAll, SQL & Scan >>>>>>>>>> >>>>>>>>> queries). >>>>>>> >>>>>>>> This will add extra size to every response and only in rare >>>>>>>>>> >>>>>>>>> cases >>> >>>> someone will benefit from it. >>>>>>>>>> However, what if we add a special configuration parameter to >>>>>>>>>> CacheConfiguration that will manage whether CacheEntry has >>>>>>>>>> >>>>>>>>> to be >>> >>>> passed >>>>>>> >>>>>>>> as a part of a response or not? Igniters, what do you think >>>>>>>>>> >>>>>>>>> on >>> >>>> this? >>>>> >>>>>> >>>>>>>>>> Regards, >>>>>>>>>> Denis >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 12/17/2015 9:09 AM, Andrey Kornev wrote: >>>>>>>>>> >>>>>>>>>>> Denis,In this case I used the term "caching" in general >>>>>>>>>>> >>>>>>>>>> sense, >>> >>>> meaning >>>>>>> >>>>>>>> saving the computed value for later. I didn't mean the Ignite >>>>>>>>> >>>>>>>> cache.Sorry >>>>>>> >>>>>>>> about the confusion.Andrey >>>>>>>>> >>>>>>>>>> _____________________________ >>>>>>>>>>> From: Denis Magda <[hidden email]> >>>>>>>>>>> Sent: Wednesday, December 16, 2015 9:48 PM >>>>>>>>>>> Subject: Re: Missing ways to get access to Ignite >>>>>>>>>>> >>>>>>>>>> CacheEntry >>> >>>> To: <[hidden email]> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hi Romain, >>>>>>>>>>> >>>>>>>>>>> I'm a bit confused cause initially you saying that >>>>>>>>>>> >>>>>>>>>>> /The value is computed on first access and it is then >>>>>>>>>>> >>>>>>>>>> cached >>> >>>> /and then you add that >>>>>>>>>>> >>>>>>>>>>> /One constraint is that the computed value is not >>>>>>>>>>> >>>>>>>>>> serializable. >>>>> >>>>>> // >>>>>>> >>>>>>>> / >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Ignite won't be able to store a value of cache if >>>>>>>>>>> >>>>>>>>>> serialization >>>>> >>>>>> is >>>>>>> >>>>>>>> not >>>>>>>>> >>>>>>>>>> supported for it by some reason. >>>>>>>>>>> >>>>>>>>>>> Please clarify here. Probably I'm missing something from >>>>>>>>>>> >>>>>>>>>> your >>> >>>> description. >>>>>>>>> >>>>>>>>>> -- >>>>>>>>>>> Denis >>>>>>>>>>> >>>>>>>>>>> On 12/16/2015 6:21 PM, Andrey Kornev wrote: >>>>>>>>>>> > Romain >>>>>>>>>>> > >>>>>>>>>>> > I suggest you be very careful using the invoke() >>>>>>>>>>> >>>>>>>>>> functionality. As >>>>>>> >>>>>>>> explained in this posting (and the follow-ups) >>>>>>>>> >>>>>>>>>> > >>>>>>>>>>> >>>>>>>>>> >>> http://mail-archives.apache.org/mod_mbox/ignite-dev/201511.mbox/%3CSNT147-W14D2736E1374450B685721CF000%40phx.gbl%3E >>> >>>> > the correct usage of the invoke() may be difficult, to >>>>>>>>>>> >>>>>>>>>> put >>> >>>> it >>>>> >>>>>> politely. >>>>>>>>> >>>>>>>>>> > >>>>>>>>>>> > @Denis Very briefly, the use case is "conditional >>>>>>>>>>> >>>>>>>>>> re-compute". >>>>> >>>>>> There is a pretty expensive computation that needs to be >>>>>>>>> >>>>>>>> applied to >>> >>>> each >>>>>>> >>>>>>>> entry on demand. The value is computed on first access and it >>>>>>>>> >>>>>>>> is >>> >>>> then >>>>> >>>>>> cached along with the current version of the cache entry. The >>>>>>>>> >>>>>>>> value >>>>> >>>>>> must >>>>>>> >>>>>>>> be recomputed on next access only if the entry has been >>>>>>>>> >>>>>>>> modified. >>> >>>> > >>>>>>>>>>> > If anyone has a better idea how to achieve something >>>>>>>>>>> >>>>>>>>>> like >>> >>>> that, >>>>> >>>>>> please do share! One constraint is that the computed value is >>>>>>>>> >>>>>>>> not >>> >>>> serializable. >>>>>>>>> >>>>>>>>>> > >>>>>>>>>>> > Cheers >>>>>>>>>>> > Andrey >>>>>>>>>>> > >>>>>>>>>>> >> Subject: Re: Missing ways to get access to Ignite >>>>>>>>>>> >>>>>>>>>> CacheEntry >>>>> >>>>>> >> To: [hidden email] >>>>>>>>>>> >> From: [hidden email] >>>>>>>>>>> >> Date: Wed, 16 Dec 2015 16:11:48 +0300 >>>>>>>>>>> >> >>>>>>>>>>> >> Romain, >>>>>>>>>>> >> >>>>>>>>>>> >> If it's implemented this way for now it doesn't mean >>>>>>>>>>> >>>>>>>>>> that >>> >>>> we >>>>> >>>>>> can't >>>>>>> >>>>>>>> >> enhance the API for other use cases ;) >>>>>>>>>>> >> As I said presently it's supported for limited number >>>>>>>>>>> >>>>>>>>>> of >>> >>>> methods >>>>>>> >>>>>>>> mostly >>>>>>>>> >>>>>>>>>> >> by performance reasons. >>>>>>>>>>> >> >>>>>>>>>>> >> Please go ahead and try invoke/invokeAll and if it >>>>>>>>>>> >>>>>>>>>> doesn't >>> >>>> work >>>>>>> >>>>>>>> fine for >>>>>>>>> >>>>>>>>>> >> you we can keep discussing what to do next. >>>>>>>>>>> >> >>>>>>>>>>> >> Regards, >>>>>>>>>>> >> Denis >>>>>>>>>>> >> >>>>>>>>>>> >> On 12/16/2015 1:53 PM, Romain Gilles wrote: >>>>>>>>>>> >>> Hi Denis, >>>>>>>>>>> >>> Thanks for you replay. And sorry to not double check >>>>>>>>>>> >>>>>>>>>> it >>> >>>> before. >>>>>>> >>>>>>>> I see that >>>>>>>>> >>>>>>>>>> >>> if I want to work with CacheEntry, I need to use the >>>>>>>>>>> >>>>>>>>>> invoke >>>>> >>>>>> method in order >>>>>>>>> >>>>>>>>>> >>> to return the CacheEntry. This is the way I should do >>>>>>>>>>> >>>>>>>>>> it. It >>>>> >>>>>> sounds like >>>>>>>>> >>>>>>>>>> >>> complicated for "just" getting an entry. But if you >>>>>>>>>>> >>>>>>>>>> say >>> >>>> this >>>>> >>>>>> is >>>>>>> >>>>>>>> the way I >>>>>>>>> >>>>>>>>>> >>> will do it like that. I was just think that it could >>>>>>>>>>> >>>>>>>>>> be a >>> >>>> common >>>>>>> >>>>>>>> use case >>>>>>>>> >>>>>>>>>> >>> and therefore provide it as a shortcut. >>>>>>>>>>> >>> >>>>>>>>>>> >>> Thanks, >>>>>>>>>>> >>> >>>>>>>>>>> >>> Romain. >>>>>>>>>>> >>> >>>>>>>>>>> >>> Le mer. 16 déc. 2015 à 11:34, Denis Magda < >>>>>>>>>>> >>>>>>>>>> [hidden email]> >>>>>>> >>>>>>>> a écrit : >>>>>>>>> >>>>>>>>>> >>> >>>>>>>>>>> >>>> Hi Romain, >>>>>>>>>>> >>>> >>>>>>>>>>> >>>> As the current documentation of >>>>>>>>>>> >>>>>>>>>> org.apache.ignite.cache.CacheEntry >>>>>>>>> >>>>>>>>>> >>>> states it's possible to get a reference to >>>>>>>>>>> >>>>>>>>>> CacheEntry >>> >>>> and >>>>> >>>>>> its >>>>>>> >>>>>>>> version >>>>>>>>> >>>>>>>>>> >>>> only for methods that are executed over local data >>>>>>>>>>> >>>>>>>>>> set >>> >>>> stored >>>>>>> >>>>>>>> on a node. >>>>>>>>> >>>>>>>>>> >>>> Among such methods are invoke & invokeAll and >>>>>>>>>>> >>>>>>>>>> randomEntry. >>>>> >>>>>> >>>> >>>>>>>>>>> >>>> You probably can get a CacheEntry and its non null >>>>>>>>>>> >>>>>>>>>> version >>>>> >>>>>> when >>>>>>> >>>>>>>> a cache >>>>>>>>> >>>>>>>>>> >>>> iterator is in use but the version will be 'null', >>>>>>>>>>> >>>>>>>>>> as >>> >>>> far >>>>> >>>>>> as I >>>>>>> >>>>>>>> remember, >>>>>>>>> >>>>>>>>>> >>>> for those entries that are loaded from remote nodes. >>>>>>>>>>> >>>> Presently Ignite doesn't transfer the version from >>>>>>>>>>> >>>>>>>>>> remote >>>>> >>>>>> nodes >>>>>>> >>>>>>>> as a >>>>>>>>> >>>>>>>>>> >>>> part of response by perform reasons. >>>>>>>>>>> >>>> >>>>>>>>>>> >>>> Please elaborate more on your particular use case >>>>>>>>>>> >>>>>>>>>> and >>> >>>> what >>>>> >>>>>> API >>>>>>> >>>>>>>> you would >>>>>>>>> >>>>>>>>>> >>>> like to add in order to support it. >>>>>>>>>>> >>>> >>>>>>>>>>> >>>> -- >>>>>>>>>>> >>>> Denis >>>>>>>>>>> >>>> >>>>>>>>>>> >>>> On 12/16/2015 12:58 PM, Romain Gilles wrote: >>>>>>>>>>> >>>>> Hi Igniters, >>>>>>>>>>> >>>>> I'm looking for a way to get access to the Ignite >>>>>>>>>>> >>>>>>>>>> CacheEntry. >>>>>>> >>>>>>>> For now >>>>>>>>> >>>>>>>>>> >>>> this >>>>>>>>>>> >>>>> is the ways I found: >>>>>>>>>>> >>>>> >>>>>>>>>>> >>>>> - Through the queris >>>>>>>>>>> >>>>> - Through jsr 107 Cache Iterable >>>>>>>>>>> >>>>> - Through jsr 107 Cache itterator >>>>>>>>>>> >>>>> - Through IgniteCache::randomEntry() >>>>>>>>>>> >>>>> >>>>>>>>>>> >>>>> If I remember correctly it was possible to get the >>>>>>>>>>> >>>>>>>>>> CacheEntry >>>>>>> >>>>>>>> from a >>>>>>>>> >>>>>>>>>> >>>> given >>>>>>>>>>> >>>>> key in old version of gridgain community >>>>>>>>>>> >>>>> version: GridCacheProjection::entry(K key) >>>>>>>>>>> >>>>>>>>>> GridCacheEntry<K,V> >>>>>>> >>>>>>>> >>>>> I think it could be a good to introduce this >>>>>>>>>>> >>>>>>>>>> feature at >>> >>>> IgniteCache >>>>>>>>> >>>>>>>>>> >>>> level. >>>>>>>>>>> >>>>> Or maybe there is an other way to do it. >>>>>>>>>>> >>>>> >>>>>>>>>>> >>>>> Thanks, >>>>>>>>>>> >>>>> >>>>>>>>>>> >>>>> Romain. >>>>>>>>>>> >>>>> >>>>>>>>>>> > >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>> >>>>> >>> > |
Re: Missing ways to get access to Ignite CacheEntry
|
|
Why it shouldn't work?
If client receives an entry that has a reference to the entry with version, transferred from a server as well as a part of the response, then unwrap(...) should work. Isn't this feasible to implement? -- Denis On 12/21/2015 10:59 AM, Dmitriy Setrakyan wrote: > Again, my issue is with unwrap method. Are we all in agreement that > unwrap(…) will not work on the client side? > > D. > > On Sun, Dec 20, 2015 at 11:30 PM, Denis Magda <[hidden email]> wrote: > >> Actually I think that Andrey's proposal on addition of >> IgniteCache.getEntry(k) sounds natural. >> >> Is there any platform limitation that will prevent us from loading an >> entry with its version from a remote node when such a particular method is >> used? >> I think that it's feasible to support it and in addition I would add on >> more method to the API - IgniteCache.getAllEntries(keysSet). >> >> -- >> Denis >> >> >> On 12/19/2015 2:05 AM, Dmitriy Setrakyan wrote: >> >>> On Fri, Dec 18, 2015 at 1:23 PM, Andrey Kornev <[hidden email]> >>> wrote: >>> >>> How would I know how to interpret the returned null? >>>> Can we just have a simple getEntry(K) method that has semantics of the >>>> regular get(K)? Why invent something? >>>> >>>> If we do this, how would you unwrap such entry on the client side? >>> >>> Cheers >>>> Andrey >>>> >>>> From: [hidden email] >>>>> Date: Fri, 18 Dec 2015 12:55:47 -0800 >>>>> Subject: Re: Missing ways to get access to Ignite CacheEntry >>>>> To: [hidden email] >>>>> >>>>> On Fri, Dec 18, 2015 at 12:40 PM, Andrey Kornev < >>>>> >>>> [hidden email]> >>>> >>>>> wrote: >>>>> >>>>> Dmitriy, >>>>>> I understand that we're talking about Ignite CacheEntry and not >>>>>> >>>>> JCache's >>>>> CacheEntry that has no version property to speak of. >>>>>> But I don't understand why the method is named localEntry()? What does >>>>>> "local" mean in this context? Does it mean that I can only use it on >>>>>> >>>>> the >>>>> primary node for a given key? What would happen if I call this method >>>>> on a >>>>> non-primary node? And what would I do if I needed the entry (along >>>>> with the >>>>> valid version it contains) on a non-primary node? >>>>>> If the node is either primary or backup, then you will get the entry >>>> back. >>>> >>>>> Otherwise, you will get null. >>>>> >>>>> >>>>> From: [hidden email] >>>>>>> Date: Fri, 18 Dec 2015 12:19:01 -0800 >>>>>>> Subject: Re: Missing ways to get access to Ignite CacheEntry >>>>>>> To: [hidden email] >>>>>>> >>>>>>> On Fri, Dec 18, 2015 at 9:47 AM, Andrey Kornev < >>>>>>> >>>>>> [hidden email] >>>>> wrote: >>>>>>> Dmitriy, >>>>>>>> Based on what criteria does one determine which information is >>>>>>>> >>>>>>> redundant >>>>>>> and which is not? >>>>>>>> To me, if an API declares a method getSomeRedundantInfo(), then the >>>>>>>> implementer has no choice, but to comply with the contract and >>>>>>>> >>>>>>> provide >>>>> such >>>>>>> info. One can't just return a null or ignore it some other way, >>>>>>> simply >>>>> because it appears to have some performance impact. >>>>>>> Andrey, the CacheEntry is not the JCache entry, but rather the >>>>>>> >>>>>> internal >>>>> Ignite entry that you can get when unwrapping the JCache entry. The >>>>>>> unwrapping, obviously, can only happen on the server side. >>>>>>> >>>>>> Therefore, I >>>>> am >>>>>>> suggesting that we also add a localEntry(K) method which will >>>>>>> >>>>>> provide the >>>>> entry API on the server side directly (not from iterator or EP). >>>>>>> Now I understand that the version() method currently does not work >>>>>>> >>>>>> properly >>>>>> >>>>>>> on the server side, but this is something we can easily fix. >>>>>>> >>>>>>> Thoughts? >>>>>>> >>>>>>> >>>>>>> Besides, the overhead of the version number may turn out to be >>>>>>> negligible >>>>>>> when benchmarked. Most likely there will be near zero overhead >>>>>>> (due to >>>>> the >>>>>>> way the network stack packs the bytes into the ethernet frames). >>>>>>> The >>>>> workaround for the broken API however may be much more expensive in >>>>>>> terms >>>>>>> of performance and user experience. >>>>>>>> Please let's keep the API consistent and predictable. >>>>>>>> >>>>>>>> Regards >>>>>>>> Andrey >>>>>>>> >>>>>>>> From: [hidden email] >>>>>>>>> Date: Fri, 18 Dec 2015 08:41:00 -0800 >>>>>>>>> Subject: Re: Missing ways to get access to Ignite CacheEntry >>>>>>>>> To: [hidden email] >>>>>>>>> >>>>>>>>> Andrey, >>>>>>>>> >>>>>>>>> The main issue with returning version is sending redundant >>>>>>>>> >>>>>>>> information >>>>>>> over >>>>>>>>> the wire. What if we added a method IgniteCache.localEntry(K) >>>>>>>>> >>>>>>>> which >>>>> would >>>>>>> return a local entry instance with a version. You would only be >>>>>>>> able >>>>> to >>>>>>> invoke this method on the server node (not client). >>>>>>>>> Would this work for you? >>>>>>>>> >>>>>>>>> D. >>>>>>>>> >>>>>>>>> On Fri, Dec 18, 2015 at 8:23 AM, Andrey Kornev < >>>>>>>>> >>>>>>>> [hidden email] >>>>>>> wrote: >>>>>>>>> Denis, >>>>>>>>>> I'm not suggesting to have every cache read operation return a >>>>>>>>>> >>>>>>>>> CacheEntry. >>>>>>>>> The "classical" GridGain used to have a getEntry(key) method >>>>>>>>> that >>>>> was >>>>>>> very >>>>>>>>> useful. The current version of Ignite provides virtually no >>>>>>>>> way of >>>>> obtaining a cache entry except thru an iterator, or using an >>>>>>>>> EntryProcessor >>>>>>>>> invocation which seems more like a hack. >>>>>>>>>> As for the question of what's stored in Ignite cache and the >>>>>>>>>> >>>>>>>>> pre-compute, >>>>>>>>> it is really an off-topic. It just came about as we were >>>>>>>>> looking >>>>> for a >>>>>>> way >>>>>>>>> to work around the issue with Ignite query not returning cache >>>>>>>>> entries' >>>>>>> version numbers in the result iterator. (There is now a ticket >>>>>>>>> that >>>>> addresses this issue: >>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-2202 >>>>>>>>> )This is when we realized that there is no longer any way in >>>>>>>>> Ignite to >>>>>>> retrieve the CacheEntry by its key. >>>>>>>>>> But for the sake of completeness I quickly go over the entire >>>>>>>>>> >>>>>>>>> use >>>>> case, >>>>>>> because it seems I wasn't clear enough in the previous post. >>>>>>>>>> Ignite cache holds the data entries that are used as the input >>>>>>>>>> >>>>>>>>> into >>>>> some >>>>>>>>> expensive computation. The data in the cache gets updated from >>>>>>>>> time to >>>>>>> time, the computation only needs to be re-run >>>>>>>>>> - on-demand -- when a user asks for the computed value and >>>>>>>>>> - the pre-computed value is obsolete -- the source cache data >>>>>>>>>> >>>>>>>>> has >>>>> been >>>>>>> updated since the last time the value was computed. >>>>>>>>>> Otherwise, we return the pre-computed value. Note that we don't >>>>>>>>>> >>>>>>>>> want to >>>>>>> rerun the computation for each cache update! Only when the >>>>>>>>> users >>>>> asks >>>>>>> for >>>>>>>>> it. >>>>>>>>>> I hope it make it more clear now. >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> Andrey >>>>>>>>>> >>>>>>>>>> Subject: Re: Missing ways to get access to Ignite CacheEntry >>>>>>>>>>> To: [hidden email] >>>>>>>>>>> From: [hidden email] >>>>>>>>>>> Date: Fri, 18 Dec 2015 13:44:16 +0300 >>>>>>>>>>> >>>>>>>>>>> Hi Andrey, >>>>>>>>>>> >>>>>>>>>>> I got you, thanks for the clarification. >>>>>>>>>>> >>>>>>>>>>> So since you're storing a computed value in some local data >>>>>>>>>>> >>>>>>>>>> structure >>>>>>> what is stored in the Ignite cache as a value in such a case? >>>>>>>>>> There >>>>>>> should be something. >>>>>>>>>>> Why don't you (or can't you) store a version identifier in >>>>>>>>>>> >>>>>>>>>> that >>>>> value >>>>>>> that is located in the Ignite cache? This should work >>>>>>>>>> perfectly >>>>> well >>>>>>> for >>>>>>>>> you use-case. >>>>>>>>>>> Personally I'm against adding CacheEntry to a response of >>>>>>>>>>> >>>>>>>>>> every >>>>> get >>>>>>> like >>>>>>>>> operation that happens over a cache (get, getAll, SQL & Scan >>>>>>>>>> queries). >>>>>>>>> This will add extra size to every response and only in rare >>>>>>>>>> cases >>>>> someone will benefit from it. >>>>>>>>>>> However, what if we add a special configuration parameter to >>>>>>>>>>> CacheConfiguration that will manage whether CacheEntry has >>>>>>>>>>> >>>>>>>>>> to be >>>>> passed >>>>>>>>> as a part of a response or not? Igniters, what do you think >>>>>>>>>> on >>>>> this? >>>>>>>>>>> Regards, >>>>>>>>>>> Denis >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 12/17/2015 9:09 AM, Andrey Kornev wrote: >>>>>>>>>>> >>>>>>>>>>>> Denis,In this case I used the term "caching" in general >>>>>>>>>>>> >>>>>>>>>>> sense, >>>>> meaning >>>>>>>>> saving the computed value for later. I didn't mean the Ignite >>>>>>>>> cache.Sorry >>>>>>>>> about the confusion.Andrey >>>>>>>>>>> _____________________________ >>>>>>>>>>>> From: Denis Magda <[hidden email]> >>>>>>>>>>>> Sent: Wednesday, December 16, 2015 9:48 PM >>>>>>>>>>>> Subject: Re: Missing ways to get access to Ignite >>>>>>>>>>>> >>>>>>>>>>> CacheEntry >>>>> To: <[hidden email]> >>>>>>>>>>>> >>>>>>>>>>>> Hi Romain, >>>>>>>>>>>> >>>>>>>>>>>> I'm a bit confused cause initially you saying that >>>>>>>>>>>> >>>>>>>>>>>> /The value is computed on first access and it is then >>>>>>>>>>>> >>>>>>>>>>> cached >>>>> /and then you add that >>>>>>>>>>>> /One constraint is that the computed value is not >>>>>>>>>>>> >>>>>>>>>>> serializable. >>>>>>> // >>>>>>>>> / >>>>>>>>>>>> >>>>>>>>>>>> Ignite won't be able to store a value of cache if >>>>>>>>>>>> >>>>>>>>>>> serialization >>>>>>> is >>>>>>>>> not >>>>>>>>>>> supported for it by some reason. >>>>>>>>>>>> Please clarify here. Probably I'm missing something from >>>>>>>>>>>> >>>>>>>>>>> your >>>>> description. >>>>>>>>>>> -- >>>>>>>>>>>> Denis >>>>>>>>>>>> >>>>>>>>>>>> On 12/16/2015 6:21 PM, Andrey Kornev wrote: >>>>>>>>>>>> > Romain >>>>>>>>>>>> > >>>>>>>>>>>> > I suggest you be very careful using the invoke() >>>>>>>>>>>> >>>>>>>>>>> functionality. As >>>>>>>>> explained in this posting (and the follow-ups) >>>>>>>>>>> > >>>> http://mail-archives.apache.org/mod_mbox/ignite-dev/201511.mbox/%3CSNT147-W14D2736E1374450B685721CF000%40phx.gbl%3E >>>> >>>>> > the correct usage of the invoke() may be difficult, to >>>>>>>>>>> put >>>>> it >>>>>>> politely. >>>>>>>>>>> > >>>>>>>>>>>> > @Denis Very briefly, the use case is "conditional >>>>>>>>>>>> >>>>>>>>>>> re-compute". >>>>>>> There is a pretty expensive computation that needs to be >>>>>>>>> applied to >>>>> each >>>>>>>>> entry on demand. The value is computed on first access and it >>>>>>>>> is >>>>> then >>>>>>> cached along with the current version of the cache entry. The >>>>>>>>> value >>>>>>> must >>>>>>>>> be recomputed on next access only if the entry has been >>>>>>>>> modified. >>>>> > >>>>>>>>>>>> > If anyone has a better idea how to achieve something >>>>>>>>>>>> >>>>>>>>>>> like >>>>> that, >>>>>>> please do share! One constraint is that the computed value is >>>>>>>>> not >>>>> serializable. >>>>>>>>>>> > >>>>>>>>>>>> > Cheers >>>>>>>>>>>> > Andrey >>>>>>>>>>>> > >>>>>>>>>>>> >> Subject: Re: Missing ways to get access to Ignite >>>>>>>>>>>> >>>>>>>>>>> CacheEntry >>>>>>> >> To: [hidden email] >>>>>>>>>>>> >> From: [hidden email] >>>>>>>>>>>> >> Date: Wed, 16 Dec 2015 16:11:48 +0300 >>>>>>>>>>>> >> >>>>>>>>>>>> >> Romain, >>>>>>>>>>>> >> >>>>>>>>>>>> >> If it's implemented this way for now it doesn't mean >>>>>>>>>>>> >>>>>>>>>>> that >>>>> we >>>>>>> can't >>>>>>>>> >> enhance the API for other use cases ;) >>>>>>>>>>>> >> As I said presently it's supported for limited number >>>>>>>>>>>> >>>>>>>>>>> of >>>>> methods >>>>>>>>> mostly >>>>>>>>>>> >> by performance reasons. >>>>>>>>>>>> >> >>>>>>>>>>>> >> Please go ahead and try invoke/invokeAll and if it >>>>>>>>>>>> >>>>>>>>>>> doesn't >>>>> work >>>>>>>>> fine for >>>>>>>>>>> >> you we can keep discussing what to do next. >>>>>>>>>>>> >> >>>>>>>>>>>> >> Regards, >>>>>>>>>>>> >> Denis >>>>>>>>>>>> >> >>>>>>>>>>>> >> On 12/16/2015 1:53 PM, Romain Gilles wrote: >>>>>>>>>>>> >>> Hi Denis, >>>>>>>>>>>> >>> Thanks for you replay. And sorry to not double check >>>>>>>>>>>> >>>>>>>>>>> it >>>>> before. >>>>>>>>> I see that >>>>>>>>>>> >>> if I want to work with CacheEntry, I need to use the >>>>>>>>>>> invoke >>>>>>> method in order >>>>>>>>>>> >>> to return the CacheEntry. This is the way I should do >>>>>>>>>>> it. It >>>>>>> sounds like >>>>>>>>>>> >>> complicated for "just" getting an entry. But if you >>>>>>>>>>> say >>>>> this >>>>>>> is >>>>>>>>> the way I >>>>>>>>>>> >>> will do it like that. I was just think that it could >>>>>>>>>>> be a >>>>> common >>>>>>>>> use case >>>>>>>>>>> >>> and therefore provide it as a shortcut. >>>>>>>>>>>> >>> >>>>>>>>>>>> >>> Thanks, >>>>>>>>>>>> >>> >>>>>>>>>>>> >>> Romain. >>>>>>>>>>>> >>> >>>>>>>>>>>> >>> Le mer. 16 déc. 2015 à 11:34, Denis Magda < >>>>>>>>>>>> >>>>>>>>>>> [hidden email]> >>>>>>>>> a écrit : >>>>>>>>>>> >>> >>>>>>>>>>>> >>>> Hi Romain, >>>>>>>>>>>> >>>> >>>>>>>>>>>> >>>> As the current documentation of >>>>>>>>>>>> >>>>>>>>>>> org.apache.ignite.cache.CacheEntry >>>>>>>>>>> >>>> states it's possible to get a reference to >>>>>>>>>>> CacheEntry >>>>> and >>>>>>> its >>>>>>>>> version >>>>>>>>>>> >>>> only for methods that are executed over local data >>>>>>>>>>> set >>>>> stored >>>>>>>>> on a node. >>>>>>>>>>> >>>> Among such methods are invoke & invokeAll and >>>>>>>>>>> randomEntry. >>>>>>> >>>> >>>>>>>>>>>> >>>> You probably can get a CacheEntry and its non null >>>>>>>>>>>> >>>>>>>>>>> version >>>>>>> when >>>>>>>>> a cache >>>>>>>>>>> >>>> iterator is in use but the version will be 'null', >>>>>>>>>>> as >>>>> far >>>>>>> as I >>>>>>>>> remember, >>>>>>>>>>> >>>> for those entries that are loaded from remote nodes. >>>>>>>>>>>> >>>> Presently Ignite doesn't transfer the version from >>>>>>>>>>>> >>>>>>>>>>> remote >>>>>>> nodes >>>>>>>>> as a >>>>>>>>>>> >>>> part of response by perform reasons. >>>>>>>>>>>> >>>> >>>>>>>>>>>> >>>> Please elaborate more on your particular use case >>>>>>>>>>>> >>>>>>>>>>> and >>>>> what >>>>>>> API >>>>>>>>> you would >>>>>>>>>>> >>>> like to add in order to support it. >>>>>>>>>>>> >>>> >>>>>>>>>>>> >>>> -- >>>>>>>>>>>> >>>> Denis >>>>>>>>>>>> >>>> >>>>>>>>>>>> >>>> On 12/16/2015 12:58 PM, Romain Gilles wrote: >>>>>>>>>>>> >>>>> Hi Igniters, >>>>>>>>>>>> >>>>> I'm looking for a way to get access to the Ignite >>>>>>>>>>>> >>>>>>>>>>> CacheEntry. >>>>>>>>> For now >>>>>>>>>>> >>>> this >>>>>>>>>>>> >>>>> is the ways I found: >>>>>>>>>>>> >>>>> >>>>>>>>>>>> >>>>> - Through the queris >>>>>>>>>>>> >>>>> - Through jsr 107 Cache Iterable >>>>>>>>>>>> >>>>> - Through jsr 107 Cache itterator >>>>>>>>>>>> >>>>> - Through IgniteCache::randomEntry() >>>>>>>>>>>> >>>>> >>>>>>>>>>>> >>>>> If I remember correctly it was possible to get the >>>>>>>>>>>> >>>>>>>>>>> CacheEntry >>>>>>>>> from a >>>>>>>>>>> >>>> given >>>>>>>>>>>> >>>>> key in old version of gridgain community >>>>>>>>>>>> >>>>> version: GridCacheProjection::entry(K key) >>>>>>>>>>>> >>>>>>>>>>> GridCacheEntry<K,V> >>>>>>>>> >>>>> I think it could be a good to introduce this >>>>>>>>>>> feature at >>>>> IgniteCache >>>>>>>>>>> >>>> level. >>>>>>>>>>>> >>>>> Or maybe there is an other way to do it. >>>>>>>>>>>> >>>>> >>>>>>>>>>>> >>>>> Thanks, >>>>>>>>>>>> >>>>> >>>>>>>>>>>> >>>>> Romain. >>>>>>>>>>>> >>>>> >>>>>>>>>>>> > >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> |
Re: Missing ways to get access to Ignite CacheEntry
|
|
On Mon, Dec 21, 2015 at 12:11 AM, Denis Magda <[hidden email]> wrote:
> Why it shouldn't work? > If client receives an entry that has a reference to the entry with > version, transferred from a server as well as a part of the response, then > unwrap(...) should work. > Isn't this feasible to implement? > The reason is consistency. If getEntry(…) can be unwrapped on the client side, then all the query results should be un-wrappable as well, which means that we have to always carry the cache version, even if user does not need it. My preference is that unwrap(..) on the client side should result in UnsupportedOperationException(…). > > -- > Denis > > > On 12/21/2015 10:59 AM, Dmitriy Setrakyan wrote: > >> Again, my issue is with unwrap method. Are we all in agreement that >> unwrap(…) will not work on the client side? >> >> D. >> >> On Sun, Dec 20, 2015 at 11:30 PM, Denis Magda <[hidden email]> >> wrote: >> >> Actually I think that Andrey's proposal on addition of >>> IgniteCache.getEntry(k) sounds natural. >>> >>> Is there any platform limitation that will prevent us from loading an >>> entry with its version from a remote node when such a particular method >>> is >>> used? >>> I think that it's feasible to support it and in addition I would add on >>> more method to the API - IgniteCache.getAllEntries(keysSet). >>> >>> -- >>> Denis >>> >>> >>> On 12/19/2015 2:05 AM, Dmitriy Setrakyan wrote: >>> >>> On Fri, Dec 18, 2015 at 1:23 PM, Andrey Kornev <[hidden email] >>>> > >>>> wrote: >>>> >>>> How would I know how to interpret the returned null? >>>> >>>>> Can we just have a simple getEntry(K) method that has semantics of the >>>>> regular get(K)? Why invent something? >>>>> >>>>> If we do this, how would you unwrap such entry on the client side? >>>>> >>>> >>>> Cheers >>>> >>>>> Andrey >>>>> >>>>> From: [hidden email] >>>>> >>>>>> Date: Fri, 18 Dec 2015 12:55:47 -0800 >>>>>> Subject: Re: Missing ways to get access to Ignite CacheEntry >>>>>> To: [hidden email] >>>>>> >>>>>> On Fri, Dec 18, 2015 at 12:40 PM, Andrey Kornev < >>>>>> >>>>>> [hidden email]> >>>>> >>>>> wrote: >>>>>> >>>>>> Dmitriy, >>>>>> >>>>>>> I understand that we're talking about Ignite CacheEntry and not >>>>>>> >>>>>>> JCache's >>>>>> CacheEntry that has no version property to speak of. >>>>>> >>>>>>> But I don't understand why the method is named localEntry()? What >>>>>>> does >>>>>>> "local" mean in this context? Does it mean that I can only use it on >>>>>>> >>>>>>> the >>>>>> primary node for a given key? What would happen if I call this method >>>>>> on a >>>>>> non-primary node? And what would I do if I needed the entry (along >>>>>> with the >>>>>> valid version it contains) on a non-primary node? >>>>>> >>>>>>> If the node is either primary or backup, then you will get the entry >>>>>>> >>>>>> back. >>>>> >>>>> Otherwise, you will get null. >>>>>> >>>>>> >>>>>> From: [hidden email] >>>>>> >>>>>>> Date: Fri, 18 Dec 2015 12:19:01 -0800 >>>>>>>> Subject: Re: Missing ways to get access to Ignite CacheEntry >>>>>>>> To: [hidden email] >>>>>>>> >>>>>>>> On Fri, Dec 18, 2015 at 9:47 AM, Andrey Kornev < >>>>>>>> >>>>>>>> [hidden email] >>>>>>> >>>>>> wrote: >>>>>> >>>>>>> Dmitriy, >>>>>>>> >>>>>>>>> Based on what criteria does one determine which information is >>>>>>>>> >>>>>>>>> redundant >>>>>>>> and which is not? >>>>>>>> >>>>>>>>> To me, if an API declares a method getSomeRedundantInfo(), then the >>>>>>>>> implementer has no choice, but to comply with the contract and >>>>>>>>> >>>>>>>>> provide >>>>>>>> >>>>>>> such >>>>>> >>>>>>> info. One can't just return a null or ignore it some other way, >>>>>>>> simply >>>>>>>> >>>>>>> because it appears to have some performance impact. >>>>>> >>>>>>> Andrey, the CacheEntry is not the JCache entry, but rather the >>>>>>>> >>>>>>>> internal >>>>>>> >>>>>> Ignite entry that you can get when unwrapping the JCache entry. The >>>>>> >>>>>>> unwrapping, obviously, can only happen on the server side. >>>>>>>> >>>>>>>> Therefore, I >>>>>>> >>>>>> am >>>>>> >>>>>>> suggesting that we also add a localEntry(K) method which will >>>>>>>> >>>>>>>> provide the >>>>>>> >>>>>> entry API on the server side directly (not from iterator or EP). >>>>>> >>>>>>> Now I understand that the version() method currently does not work >>>>>>>> >>>>>>>> properly >>>>>>> >>>>>>> on the server side, but this is something we can easily fix. >>>>>>>> >>>>>>>> Thoughts? >>>>>>>> >>>>>>>> >>>>>>>> Besides, the overhead of the version number may turn out to be >>>>>>>> negligible >>>>>>>> when benchmarked. Most likely there will be near zero overhead >>>>>>>> (due to >>>>>>>> >>>>>>> the >>>>>> >>>>>>> way the network stack packs the bytes into the ethernet frames). >>>>>>>> The >>>>>>>> >>>>>>> workaround for the broken API however may be much more expensive in >>>>>> >>>>>>> terms >>>>>>>> of performance and user experience. >>>>>>>> >>>>>>>>> Please let's keep the API consistent and predictable. >>>>>>>>> >>>>>>>>> Regards >>>>>>>>> Andrey >>>>>>>>> >>>>>>>>> From: [hidden email] >>>>>>>>> >>>>>>>>>> Date: Fri, 18 Dec 2015 08:41:00 -0800 >>>>>>>>>> Subject: Re: Missing ways to get access to Ignite CacheEntry >>>>>>>>>> To: [hidden email] >>>>>>>>>> >>>>>>>>>> Andrey, >>>>>>>>>> >>>>>>>>>> The main issue with returning version is sending redundant >>>>>>>>>> >>>>>>>>>> information >>>>>>>>> >>>>>>>> over >>>>>>>> >>>>>>>>> the wire. What if we added a method IgniteCache.localEntry(K) >>>>>>>>>> >>>>>>>>>> which >>>>>>>>> >>>>>>>> would >>>>>> >>>>>>> return a local entry instance with a version. You would only be >>>>>>>> >>>>>>>>> able >>>>>>>>> >>>>>>>> to >>>>>> >>>>>>> invoke this method on the server node (not client). >>>>>>>> >>>>>>>>> Would this work for you? >>>>>>>>>> >>>>>>>>>> D. >>>>>>>>>> >>>>>>>>>> On Fri, Dec 18, 2015 at 8:23 AM, Andrey Kornev < >>>>>>>>>> >>>>>>>>>> [hidden email] >>>>>>>>> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Denis, >>>>>>>>>> >>>>>>>>>>> I'm not suggesting to have every cache read operation return a >>>>>>>>>>> >>>>>>>>>>> CacheEntry. >>>>>>>>>> The "classical" GridGain used to have a getEntry(key) method >>>>>>>>>> that >>>>>>>>>> >>>>>>>>> was >>>>>> >>>>>>> very >>>>>>>> >>>>>>>>> useful. The current version of Ignite provides virtually no >>>>>>>>>> way of >>>>>>>>>> >>>>>>>>> obtaining a cache entry except thru an iterator, or using an >>>>>> >>>>>>> EntryProcessor >>>>>>>>>> invocation which seems more like a hack. >>>>>>>>>> >>>>>>>>>>> As for the question of what's stored in Ignite cache and the >>>>>>>>>>> >>>>>>>>>>> pre-compute, >>>>>>>>>> it is really an off-topic. It just came about as we were >>>>>>>>>> looking >>>>>>>>>> >>>>>>>>> for a >>>>>> >>>>>>> way >>>>>>>> >>>>>>>>> to work around the issue with Ignite query not returning cache >>>>>>>>>> entries' >>>>>>>>>> >>>>>>>>> version numbers in the result iterator. (There is now a ticket >>>>>>>> >>>>>>>>> that >>>>>>>>>> >>>>>>>>> addresses this issue: >>>>>> >>>>>>> https://issues.apache.org/jira/browse/IGNITE-2202 >>>>>>>>>> )This is when we realized that there is no longer any way in >>>>>>>>>> Ignite to >>>>>>>>>> >>>>>>>>> retrieve the CacheEntry by its key. >>>>>>>> >>>>>>>>> But for the sake of completeness I quickly go over the entire >>>>>>>>>>> >>>>>>>>>>> use >>>>>>>>>> >>>>>>>>> case, >>>>>> >>>>>>> because it seems I wasn't clear enough in the previous post. >>>>>>>> >>>>>>>>> Ignite cache holds the data entries that are used as the input >>>>>>>>>>> >>>>>>>>>>> into >>>>>>>>>> >>>>>>>>> some >>>>>> >>>>>>> expensive computation. The data in the cache gets updated from >>>>>>>>>> time to >>>>>>>>>> >>>>>>>>> time, the computation only needs to be re-run >>>>>>>> >>>>>>>>> - on-demand -- when a user asks for the computed value and >>>>>>>>>>> - the pre-computed value is obsolete -- the source cache data >>>>>>>>>>> >>>>>>>>>>> has >>>>>>>>>> >>>>>>>>> been >>>>>> >>>>>>> updated since the last time the value was computed. >>>>>>>> >>>>>>>>> Otherwise, we return the pre-computed value. Note that we don't >>>>>>>>>>> >>>>>>>>>>> want to >>>>>>>>>> >>>>>>>>> rerun the computation for each cache update! Only when the >>>>>>>> >>>>>>>>> users >>>>>>>>>> >>>>>>>>> asks >>>>>> >>>>>>> for >>>>>>>> >>>>>>>>> it. >>>>>>>>>> >>>>>>>>>>> I hope it make it more clear now. >>>>>>>>>>> >>>>>>>>>>> Thanks >>>>>>>>>>> Andrey >>>>>>>>>>> >>>>>>>>>>> Subject: Re: Missing ways to get access to Ignite CacheEntry >>>>>>>>>>> >>>>>>>>>>>> To: [hidden email] >>>>>>>>>>>> From: [hidden email] >>>>>>>>>>>> Date: Fri, 18 Dec 2015 13:44:16 +0300 >>>>>>>>>>>> >>>>>>>>>>>> Hi Andrey, >>>>>>>>>>>> >>>>>>>>>>>> I got you, thanks for the clarification. >>>>>>>>>>>> >>>>>>>>>>>> So since you're storing a computed value in some local data >>>>>>>>>>>> >>>>>>>>>>>> structure >>>>>>>>>>> >>>>>>>>>> what is stored in the Ignite cache as a value in such a case? >>>>>>>> >>>>>>>>> There >>>>>>>>>>> >>>>>>>>>> should be something. >>>>>>>> >>>>>>>>> Why don't you (or can't you) store a version identifier in >>>>>>>>>>>> >>>>>>>>>>>> that >>>>>>>>>>> >>>>>>>>>> value >>>>>> >>>>>>> that is located in the Ignite cache? This should work >>>>>>>> >>>>>>>>> perfectly >>>>>>>>>>> >>>>>>>>>> well >>>>>> >>>>>>> for >>>>>>>> >>>>>>>>> you use-case. >>>>>>>>>> >>>>>>>>>>> Personally I'm against adding CacheEntry to a response of >>>>>>>>>>>> >>>>>>>>>>>> every >>>>>>>>>>> >>>>>>>>>> get >>>>>> >>>>>>> like >>>>>>>> >>>>>>>>> operation that happens over a cache (get, getAll, SQL & Scan >>>>>>>>>> >>>>>>>>>>> queries). >>>>>>>>>>> >>>>>>>>>> This will add extra size to every response and only in rare >>>>>>>>>> >>>>>>>>>>> cases >>>>>>>>>>> >>>>>>>>>> someone will benefit from it. >>>>>> >>>>>>> However, what if we add a special configuration parameter to >>>>>>>>>>>> CacheConfiguration that will manage whether CacheEntry has >>>>>>>>>>>> >>>>>>>>>>>> to be >>>>>>>>>>> >>>>>>>>>> passed >>>>>> >>>>>>> as a part of a response or not? Igniters, what do you think >>>>>>>>>> >>>>>>>>>>> on >>>>>>>>>>> >>>>>>>>>> this? >>>>>> >>>>>>> Regards, >>>>>>>>>>>> Denis >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 12/17/2015 9:09 AM, Andrey Kornev wrote: >>>>>>>>>>>> >>>>>>>>>>>> Denis,In this case I used the term "caching" in general >>>>>>>>>>>>> >>>>>>>>>>>>> sense, >>>>>>>>>>>> >>>>>>>>>>> meaning >>>>>> >>>>>>> saving the computed value for later. I didn't mean the Ignite >>>>>>>>>> cache.Sorry >>>>>>>>>> about the confusion.Andrey >>>>>>>>>> >>>>>>>>>>> _____________________________ >>>>>>>>>>>> >>>>>>>>>>>>> From: Denis Magda <[hidden email]> >>>>>>>>>>>>> Sent: Wednesday, December 16, 2015 9:48 PM >>>>>>>>>>>>> Subject: Re: Missing ways to get access to Ignite >>>>>>>>>>>>> >>>>>>>>>>>>> CacheEntry >>>>>>>>>>>> >>>>>>>>>>> To: <[hidden email]> >>>>>> >>>>>>> >>>>>>>>>>>>> Hi Romain, >>>>>>>>>>>>> >>>>>>>>>>>>> I'm a bit confused cause initially you saying that >>>>>>>>>>>>> >>>>>>>>>>>>> /The value is computed on first access and it is then >>>>>>>>>>>>> >>>>>>>>>>>>> cached >>>>>>>>>>>> >>>>>>>>>>> /and then you add that >>>>>> >>>>>>> /One constraint is that the computed value is not >>>>>>>>>>>>> >>>>>>>>>>>>> serializable. >>>>>>>>>>>> >>>>>>>>>>> // >>>>>>>> >>>>>>>>> / >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>>> Ignite won't be able to store a value of cache if >>>>>>>>>>>>> >>>>>>>>>>>>> serialization >>>>>>>>>>>> >>>>>>>>>>> is >>>>>>>> >>>>>>>>> not >>>>>>>>>> >>>>>>>>>>> supported for it by some reason. >>>>>>>>>>>> >>>>>>>>>>>>> Please clarify here. Probably I'm missing something from >>>>>>>>>>>>> >>>>>>>>>>>>> your >>>>>>>>>>>> >>>>>>>>>>> description. >>>>>> >>>>>>> -- >>>>>>>>>>>> >>>>>>>>>>>>> Denis >>>>>>>>>>>>> >>>>>>>>>>>>> On 12/16/2015 6:21 PM, Andrey Kornev wrote: >>>>>>>>>>>>> > Romain >>>>>>>>>>>>> > >>>>>>>>>>>>> > I suggest you be very careful using the invoke() >>>>>>>>>>>>> >>>>>>>>>>>>> functionality. As >>>>>>>>>>>> >>>>>>>>>>> explained in this posting (and the follow-ups) >>>>>>>>>> >>>>>>>>>>> > >>>>>>>>>>>> >>>>>>>>>>> >>>>> http://mail-archives.apache.org/mod_mbox/ignite-dev/201511.mbox/%3CSNT147-W14D2736E1374450B685721CF000%40phx.gbl%3E >>>>> >>>>> > the correct usage of the invoke() may be difficult, to >>>>>> >>>>>>> put >>>>>>>>>>>> >>>>>>>>>>> it >>>>>> >>>>>>> politely. >>>>>>>> >>>>>>>>> > >>>>>>>>>>>> >>>>>>>>>>>>> > @Denis Very briefly, the use case is "conditional >>>>>>>>>>>>> >>>>>>>>>>>>> re-compute". >>>>>>>>>>>> >>>>>>>>>>> There is a pretty expensive computation that needs to be >>>>>>>> >>>>>>>>> applied to >>>>>>>>>> >>>>>>>>> each >>>>>> >>>>>>> entry on demand. The value is computed on first access and it >>>>>>>>>> is >>>>>>>>>> >>>>>>>>> then >>>>>> >>>>>>> cached along with the current version of the cache entry. The >>>>>>>> >>>>>>>>> value >>>>>>>>>> >>>>>>>>> must >>>>>>>> >>>>>>>>> be recomputed on next access only if the entry has been >>>>>>>>>> modified. >>>>>>>>>> >>>>>>>>> > >>>>>> >>>>>>> > If anyone has a better idea how to achieve something >>>>>>>>>>>>> >>>>>>>>>>>>> like >>>>>>>>>>>> >>>>>>>>>>> that, >>>>>> >>>>>>> please do share! One constraint is that the computed value is >>>>>>>> >>>>>>>>> not >>>>>>>>>> >>>>>>>>> serializable. >>>>>> >>>>>>> > >>>>>>>>>>>> >>>>>>>>>>>>> > Cheers >>>>>>>>>>>>> > Andrey >>>>>>>>>>>>> > >>>>>>>>>>>>> >> Subject: Re: Missing ways to get access to Ignite >>>>>>>>>>>>> >>>>>>>>>>>>> CacheEntry >>>>>>>>>>>> >>>>>>>>>>> >> To: [hidden email] >>>>>>>> >>>>>>>>> >> From: [hidden email] >>>>>>>>>>>>> >> Date: Wed, 16 Dec 2015 16:11:48 +0300 >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> Romain, >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> If it's implemented this way for now it doesn't mean >>>>>>>>>>>>> >>>>>>>>>>>>> that >>>>>>>>>>>> >>>>>>>>>>> we >>>>>> >>>>>>> can't >>>>>>>> >>>>>>>>> >> enhance the API for other use cases ;) >>>>>>>>>> >>>>>>>>>>> >> As I said presently it's supported for limited number >>>>>>>>>>>>> >>>>>>>>>>>>> of >>>>>>>>>>>> >>>>>>>>>>> methods >>>>>> >>>>>>> mostly >>>>>>>>>> >>>>>>>>>>> >> by performance reasons. >>>>>>>>>>>> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> Please go ahead and try invoke/invokeAll and if it >>>>>>>>>>>>> >>>>>>>>>>>>> doesn't >>>>>>>>>>>> >>>>>>>>>>> work >>>>>> >>>>>>> fine for >>>>>>>>>> >>>>>>>>>>> >> you we can keep discussing what to do next. >>>>>>>>>>>> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> Regards, >>>>>>>>>>>>> >> Denis >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> On 12/16/2015 1:53 PM, Romain Gilles wrote: >>>>>>>>>>>>> >>> Hi Denis, >>>>>>>>>>>>> >>> Thanks for you replay. And sorry to not double check >>>>>>>>>>>>> >>>>>>>>>>>>> it >>>>>>>>>>>> >>>>>>>>>>> before. >>>>>> >>>>>>> I see that >>>>>>>>>> >>>>>>>>>>> >>> if I want to work with CacheEntry, I need to use the >>>>>>>>>>>> invoke >>>>>>>>>>>> >>>>>>>>>>> method in order >>>>>>>> >>>>>>>>> >>> to return the CacheEntry. This is the way I should do >>>>>>>>>>>> it. It >>>>>>>>>>>> >>>>>>>>>>> sounds like >>>>>>>> >>>>>>>>> >>> complicated for "just" getting an entry. But if you >>>>>>>>>>>> say >>>>>>>>>>>> >>>>>>>>>>> this >>>>>> >>>>>>> is >>>>>>>> >>>>>>>>> the way I >>>>>>>>>> >>>>>>>>>>> >>> will do it like that. I was just think that it could >>>>>>>>>>>> be a >>>>>>>>>>>> >>>>>>>>>>> common >>>>>> >>>>>>> use case >>>>>>>>>> >>>>>>>>>>> >>> and therefore provide it as a shortcut. >>>>>>>>>>>> >>>>>>>>>>>>> >>> >>>>>>>>>>>>> >>> Thanks, >>>>>>>>>>>>> >>> >>>>>>>>>>>>> >>> Romain. >>>>>>>>>>>>> >>> >>>>>>>>>>>>> >>> Le mer. 16 déc. 2015 à 11:34, Denis Magda < >>>>>>>>>>>>> >>>>>>>>>>>>> [hidden email]> >>>>>>>>>>>> >>>>>>>>>>> a écrit : >>>>>>>>>> >>>>>>>>>>> >>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>> Hi Romain, >>>>>>>>>>>>> >>>> >>>>>>>>>>>>> >>>> As the current documentation of >>>>>>>>>>>>> >>>>>>>>>>>>> org.apache.ignite.cache.CacheEntry >>>>>>>>>>>> >>>> states it's possible to get a reference to >>>>>>>>>>>> CacheEntry >>>>>>>>>>>> >>>>>>>>>>> and >>>>>> >>>>>>> its >>>>>>>> >>>>>>>>> version >>>>>>>>>> >>>>>>>>>>> >>>> only for methods that are executed over local data >>>>>>>>>>>> set >>>>>>>>>>>> >>>>>>>>>>> stored >>>>>> >>>>>>> on a node. >>>>>>>>>> >>>>>>>>>>> >>>> Among such methods are invoke & invokeAll and >>>>>>>>>>>> randomEntry. >>>>>>>>>>>> >>>>>>>>>>> >>>> >>>>>>>> >>>>>>>>> >>>> You probably can get a CacheEntry and its non null >>>>>>>>>>>>> >>>>>>>>>>>>> version >>>>>>>>>>>> >>>>>>>>>>> when >>>>>>>> >>>>>>>>> a cache >>>>>>>>>> >>>>>>>>>>> >>>> iterator is in use but the version will be 'null', >>>>>>>>>>>> as >>>>>>>>>>>> >>>>>>>>>>> far >>>>>> >>>>>>> as I >>>>>>>> >>>>>>>>> remember, >>>>>>>>>> >>>>>>>>>>> >>>> for those entries that are loaded from remote nodes. >>>>>>>>>>>> >>>>>>>>>>>>> >>>> Presently Ignite doesn't transfer the version from >>>>>>>>>>>>> >>>>>>>>>>>>> remote >>>>>>>>>>>> >>>>>>>>>>> nodes >>>>>>>> >>>>>>>>> as a >>>>>>>>>> >>>>>>>>>>> >>>> part of response by perform reasons. >>>>>>>>>>>> >>>>>>>>>>>>> >>>> >>>>>>>>>>>>> >>>> Please elaborate more on your particular use case >>>>>>>>>>>>> >>>>>>>>>>>>> and >>>>>>>>>>>> >>>>>>>>>>> what >>>>>> >>>>>>> API >>>>>>>> >>>>>>>>> you would >>>>>>>>>> >>>>>>>>>>> >>>> like to add in order to support it. >>>>>>>>>>>> >>>>>>>>>>>>> >>>> >>>>>>>>>>>>> >>>> -- >>>>>>>>>>>>> >>>> Denis >>>>>>>>>>>>> >>>> >>>>>>>>>>>>> >>>> On 12/16/2015 12:58 PM, Romain Gilles wrote: >>>>>>>>>>>>> >>>>> Hi Igniters, >>>>>>>>>>>>> >>>>> I'm looking for a way to get access to the Ignite >>>>>>>>>>>>> >>>>>>>>>>>>> CacheEntry. >>>>>>>>>>>> >>>>>>>>>>> For now >>>>>>>>>> >>>>>>>>>>> >>>> this >>>>>>>>>>>> >>>>>>>>>>>>> >>>>> is the ways I found: >>>>>>>>>>>>> >>>>> >>>>>>>>>>>>> >>>>> - Through the queris >>>>>>>>>>>>> >>>>> - Through jsr 107 Cache Iterable >>>>>>>>>>>>> >>>>> - Through jsr 107 Cache itterator >>>>>>>>>>>>> >>>>> - Through IgniteCache::randomEntry() >>>>>>>>>>>>> >>>>> >>>>>>>>>>>>> >>>>> If I remember correctly it was possible to get the >>>>>>>>>>>>> >>>>>>>>>>>>> CacheEntry >>>>>>>>>>>> >>>>>>>>>>> from a >>>>>>>>>> >>>>>>>>>>> >>>> given >>>>>>>>>>>> >>>>>>>>>>>>> >>>>> key in old version of gridgain community >>>>>>>>>>>>> >>>>> version: GridCacheProjection::entry(K key) >>>>>>>>>>>>> >>>>>>>>>>>>> GridCacheEntry<K,V> >>>>>>>>>>>> >>>>>>>>>>> >>>>> I think it could be a good to introduce this >>>>>>>>>> >>>>>>>>>>> feature at >>>>>>>>>>>> >>>>>>>>>>> IgniteCache >>>>>> >>>>>>> >>>> level. >>>>>>>>>>>> >>>>>>>>>>>>> >>>>> Or maybe there is an other way to do it. >>>>>>>>>>>>> >>>>> >>>>>>>>>>>>> >>>>> Thanks, >>>>>>>>>>>>> >>>>> >>>>>>>>>>>>> >>>>> Romain. >>>>>>>>>>>>> >>>>> >>>>>>>>>>>>> > >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> > |
«
Return to Apache Ignite Developers - Legacy Mail Archive
|
1 view|%1 views
| Free forum by Nabble | Edit this page |