I try to understand how statistics work and fixe some problem.
I first case: cache.put(46744, "val 46744"); cache.get(46744); In statistic I see: 2016-05-24 14:19:31 INFO ServerNode:78 - Swap put 0 get 0 (0, 0) entries count 0 2016-05-24 14:19:31 INFO ServerNode:81 - OffHeap put 1 get 0 (0, 0) entries count 1 2016-05-24 14:19:31 INFO ServerNode:84 - OnHeap put 1 get 1 (1, 0) In brackets Hit and Miss values. But I asume OffHeap get must to be one, because cache configured as OFFHEAP_TIERED and swapEnabled - false. My investigation has lead to method org.apache.ignite.internal.processors.cache.GridCacheSwapManager#readOffheapPointer. The method read only pointer from heap, but not get bytes of value and not increase any statistic. If each receive pointer increase statistic (OffHeap get I mean), then each OffHeap put will increased OffHeap get, because readOffheapPointer take place on OffHeap put. The thing confuses my: Has any rules metrics works? Where works with metrics value must take place? |
Hi Vlad,
In my understanding this should work or implemented this way for OFFHEAP_TIRED cache. CacheMetrics.getCacheEvictions - incremented on every put & get operation because an entry “goes through” heap memory and evicted from there when it’s no longer needed (usually at the end of get or put operation). CacheMetrics.getOffHeapGets - should be incremented every time the off-heap layer is accessed for a particular key. This can be an ordinary cache.get() call or during a cache.put() that unswaps an entry before the new value is put. In my understanding you can increase this statistics exactly in this method - GridCacheSwapManager#readOffheapPointer. CacheMetrics.getOffHeapPuts - should be incremented every time a put operations happens and an entry is moved to off heap. — Denis > On May 24, 2016, at 2:47 PM, Vladislav Pyatkov <[hidden email]> wrote: > > I try to understand how statistics work and fixe some problem. > I first case: > cache.put(46744, "val 46744"); > cache.get(46744); > In statistic I see: > 2016-05-24 14:19:31 INFO ServerNode:78 - Swap put 0 get 0 (0, 0) entries > count 0 > 2016-05-24 14:19:31 INFO ServerNode:81 - OffHeap put 1 get 0 (0, 0) > entries count 1 > 2016-05-24 14:19:31 INFO ServerNode:84 - OnHeap put 1 get 1 (1, 0) > > In brackets Hit and Miss values. > > But I asume OffHeap get must to be one, because cache configured as > OFFHEAP_TIERED and swapEnabled - false. > > My investigation has lead to method > org.apache.ignite.internal.processors.cache.GridCacheSwapManager#readOffheapPointer. > The method read only pointer from heap, but not get bytes of value and not > increase any statistic. > If each receive pointer increase statistic (OffHeap get I mean), then each > OffHeap put will increased OffHeap get, because readOffheapPointer take > place on OffHeap put. > > The thing confuses my: > Has any rules metrics works? > Where works with metrics value must take place? |
Denis,
I disagree. readOffheapPointer doesn't touch offheap get/put metrics deliberately. User should have exactly one offheap get operation per cache.get call. Vlad, as I can see offheap gets metric increments every time when get, contains, etc operations perform, so it should work. If you have more then one node then cluster metrics should be updated eventually with discovery message and immediately for local node. So if local node isn't primary for your key you can get metrics with some delay. If particular metric doesn't change then we need find method that should be responsible for update of this metric. On Tue, May 24, 2016 at 4:27 PM, Denis Magda <[hidden email]> wrote: > Hi Vlad, > > In my understanding this should work or implemented this way for > OFFHEAP_TIRED cache. > > CacheMetrics.getCacheEvictions - incremented on every put & get operation > because an entry “goes through” heap memory and evicted from there when > it’s no longer needed (usually at the end of get or put operation). > > CacheMetrics.getOffHeapGets - should be incremented every time the > off-heap layer is accessed for a particular key. This can be an ordinary > cache.get() call or during a cache.put() that unswaps an entry before the > new value is put. In my understanding you can increase this statistics > exactly in this method - GridCacheSwapManager#readOffheapPointer. > > CacheMetrics.getOffHeapPuts - should be incremented every time a put > operations happens and an entry is moved to off heap. > > — > Denis > > > On May 24, 2016, at 2:47 PM, Vladislav Pyatkov <[hidden email]> > wrote: > > > > I try to understand how statistics work and fixe some problem. > > I first case: > > cache.put(46744, "val 46744"); > > cache.get(46744); > > In statistic I see: > > 2016-05-24 14:19:31 INFO ServerNode:78 - Swap put 0 get 0 (0, 0) entries > > count 0 > > 2016-05-24 14:19:31 INFO ServerNode:81 - OffHeap put 1 get 0 (0, 0) > > entries count 1 > > 2016-05-24 14:19:31 INFO ServerNode:84 - OnHeap put 1 get 1 (1, 0) > > > > In brackets Hit and Miss values. > > > > But I asume OffHeap get must to be one, because cache configured as > > OFFHEAP_TIERED and swapEnabled - false. > > > > My investigation has lead to method > > > org.apache.ignite.internal.processors.cache.GridCacheSwapManager#readOffheapPointer. > > The method read only pointer from heap, but not get bytes of value and > not > > increase any statistic. > > If each receive pointer increase statistic (OffHeap get I mean), then > each > > OffHeap put will increased OffHeap get, because readOffheapPointer take > > place on OffHeap put. > > > > The thing confuses my: > > Has any rules metrics works? > > Where works with metrics value must take place? > > -- Andrey Gura GridGain Systems, Inc. www.gridgain.com |
Andrey,
I can see offheap gets metric increments every time when get Unfortunately not. When cache configured as OFFHEAP_TIERED it does not work. About increment Get when Put takes place: org.apache.ignite.internal.processors.cache.local.CacheLocalOffHeapAndSwapMetricsSelfTest#testOffHeapMetrics The logic existed is a long time and were covered tests. for (int i = 0; i < KEYS_CNT; i++) cache.put(i, i); assertEquals(KEYS_CNT, cache.localMetrics().getOffHeapGets()); We execute only put, but get counter also incremented. Is anyone has another opinion? On Wed, May 25, 2016 at 2:51 PM, Andrey Gura <[hidden email]> wrote: > Denis, > > I disagree. readOffheapPointer doesn't touch offheap get/put metrics > deliberately. User should have exactly one offheap get operation per > cache.get call. > > Vlad, > > as I can see offheap gets metric increments every time when get, contains, > etc operations perform, so it should work. If you have more then one node > then cluster metrics should be updated eventually with discovery message > and immediately for local node. So if local node isn't primary for your key > you can get metrics with some delay. > > If particular metric doesn't change then we need find method that should be > responsible for update of this metric. > > > On Tue, May 24, 2016 at 4:27 PM, Denis Magda <[hidden email]> wrote: > > > Hi Vlad, > > > > In my understanding this should work or implemented this way for > > OFFHEAP_TIRED cache. > > > > CacheMetrics.getCacheEvictions - incremented on every put & get operation > > because an entry “goes through” heap memory and evicted from there when > > it’s no longer needed (usually at the end of get or put operation). > > > > CacheMetrics.getOffHeapGets - should be incremented every time the > > off-heap layer is accessed for a particular key. This can be an ordinary > > cache.get() call or during a cache.put() that unswaps an entry before the > > new value is put. In my understanding you can increase this statistics > > exactly in this method - GridCacheSwapManager#readOffheapPointer. > > > > CacheMetrics.getOffHeapPuts - should be incremented every time a put > > operations happens and an entry is moved to off heap. > > > > — > > Denis > > > > > On May 24, 2016, at 2:47 PM, Vladislav Pyatkov <[hidden email]> > > wrote: > > > > > > I try to understand how statistics work and fixe some problem. > > > I first case: > > > cache.put(46744, "val 46744"); > > > cache.get(46744); > > > In statistic I see: > > > 2016-05-24 14:19:31 INFO ServerNode:78 - Swap put 0 get 0 (0, 0) > entries > > > count 0 > > > 2016-05-24 14:19:31 INFO ServerNode:81 - OffHeap put 1 get 0 (0, 0) > > > entries count 1 > > > 2016-05-24 14:19:31 INFO ServerNode:84 - OnHeap put 1 get 1 (1, 0) > > > > > > In brackets Hit and Miss values. > > > > > > But I asume OffHeap get must to be one, because cache configured as > > > OFFHEAP_TIERED and swapEnabled - false. > > > > > > My investigation has lead to method > > > > > > org.apache.ignite.internal.processors.cache.GridCacheSwapManager#readOffheapPointer. > > > The method read only pointer from heap, but not get bytes of value and > > not > > > increase any statistic. > > > If each receive pointer increase statistic (OffHeap get I mean), then > > each > > > OffHeap put will increased OffHeap get, because readOffheapPointer take > > > place on OffHeap put. > > > > > > The thing confuses my: > > > Has any rules metrics works? > > > Where works with metrics value must take place? > > > > > > > -- > Andrey Gura > GridGain Systems, Inc. > www.gridgain.com > |
Hmmm...
Ok, every time when we perform put we go to offheap, becuase it can already contain this key. So my statement about one offheap get pere one cache.get is wrong. Anyway, get operation should update offheap gets metric. See usages of CacheMetricsImpl#onOffheapRead. On Wed, May 25, 2016 at 4:06 PM, Vladislav Pyatkov <[hidden email]> wrote: > Andrey, > > I can see offheap gets metric increments every time when get > > > Unfortunately not. When cache configured as OFFHEAP_TIERED it does not > work. > About increment Get when Put takes place: > > org.apache.ignite.internal.processors.cache.local.CacheLocalOffHeapAndSwapMetricsSelfTest#testOffHeapMetrics > The logic existed is a long time and were covered tests. > > for (int i = 0; i < KEYS_CNT; i++) > cache.put(i, i); > > assertEquals(KEYS_CNT, cache.localMetrics().getOffHeapGets()); > > We execute only put, but get counter also incremented. > > Is anyone has another opinion? > > > > On Wed, May 25, 2016 at 2:51 PM, Andrey Gura <[hidden email]> wrote: > > > Denis, > > > > I disagree. readOffheapPointer doesn't touch offheap get/put metrics > > deliberately. User should have exactly one offheap get operation per > > cache.get call. > > > > Vlad, > > > > as I can see offheap gets metric increments every time when get, > contains, > > etc operations perform, so it should work. If you have more then one node > > then cluster metrics should be updated eventually with discovery message > > and immediately for local node. So if local node isn't primary for your > key > > you can get metrics with some delay. > > > > If particular metric doesn't change then we need find method that should > be > > responsible for update of this metric. > > > > > > On Tue, May 24, 2016 at 4:27 PM, Denis Magda <[hidden email]> > wrote: > > > > > Hi Vlad, > > > > > > In my understanding this should work or implemented this way for > > > OFFHEAP_TIRED cache. > > > > > > CacheMetrics.getCacheEvictions - incremented on every put & get > operation > > > because an entry “goes through” heap memory and evicted from there when > > > it’s no longer needed (usually at the end of get or put operation). > > > > > > CacheMetrics.getOffHeapGets - should be incremented every time the > > > off-heap layer is accessed for a particular key. This can be an > ordinary > > > cache.get() call or during a cache.put() that unswaps an entry before > the > > > new value is put. In my understanding you can increase this statistics > > > exactly in this method - GridCacheSwapManager#readOffheapPointer. > > > > > > CacheMetrics.getOffHeapPuts - should be incremented every time a put > > > operations happens and an entry is moved to off heap. > > > > > > — > > > Denis > > > > > > > On May 24, 2016, at 2:47 PM, Vladislav Pyatkov < > [hidden email]> > > > wrote: > > > > > > > > I try to understand how statistics work and fixe some problem. > > > > I first case: > > > > cache.put(46744, "val 46744"); > > > > cache.get(46744); > > > > In statistic I see: > > > > 2016-05-24 14:19:31 INFO ServerNode:78 - Swap put 0 get 0 (0, 0) > > entries > > > > count 0 > > > > 2016-05-24 14:19:31 INFO ServerNode:81 - OffHeap put 1 get 0 (0, 0) > > > > entries count 1 > > > > 2016-05-24 14:19:31 INFO ServerNode:84 - OnHeap put 1 get 1 (1, 0) > > > > > > > > In brackets Hit and Miss values. > > > > > > > > But I asume OffHeap get must to be one, because cache configured as > > > > OFFHEAP_TIERED and swapEnabled - false. > > > > > > > > My investigation has lead to method > > > > > > > > > > org.apache.ignite.internal.processors.cache.GridCacheSwapManager#readOffheapPointer. > > > > The method read only pointer from heap, but not get bytes of value > and > > > not > > > > increase any statistic. > > > > If each receive pointer increase statistic (OffHeap get I mean), then > > > each > > > > OffHeap put will increased OffHeap get, because readOffheapPointer > take > > > > place on OffHeap put. > > > > > > > > The thing confuses my: > > > > Has any rules metrics works? > > > > Where works with metrics value must take place? > > > > > > > > > > > > -- > > Andrey Gura > > GridGain Systems, Inc. > > www.gridgain.com > > > -- Andrey Gura GridGain Systems, Inc. www.gridgain.com |
Free forum by Nabble | Edit this page |