Wrong off-heap size is reported for a node

[dmagda]

Igniters,

Was troubleshooting an Ignite deployment today and couldn't find out from
the logs what was the actual off-heap space used.

Those were the given memory resoures (Ignite 2.6):

[2018-08-16 15:07:49,961][INFO ][main][GridDiscoveryManager] Topology
snapshot [ver=1, servers=1, clients=0, CPUs=64, *offheap=30.0GB*,
heap=24.0GB]

And that weird stuff was reported by the node (pay attention to the last
line):

[2018-08-16 15:45:50,211][INFO
][grid-timeout-worker-#135%cluster_31-Dec-2017%][IgniteKernal%cluster_31-Dec-2017]
Metrics for local node (to disable set 'metricsLogFrequency' to 0)
    ^-- Node [id=c033026e, name=cluster_31-Dec-2017, uptime=00:38:00.257]
    ^-- H/N/C [hosts=1, nodes=1, CPUs=64]
    ^-- CPU [cur=0.03%, avg=5.54%, GC=0%]
    ^-- PageMemory [pages=6997377]
    ^-- Heap [used=9706MB, free=61.18%, comm=22384MB]
   * ^-- Non heap [used=144MB, free=-1%, comm=148MB] - this line is always
the same!*

Had to change the code by using dataRegion.getPhysicalMemoryPages() to find
out that actual off-heap usage size was
>>> Physical Memory Size: 28651614208 => 27324 MB, 26 GB

Let's fix this issue in 2.7, I proposed a new format. Please review and
share your thoughts:
https://issues.apache.org/jira/browse/IGNITE-9305

--
Denis

[dsetrakyan]

Is there a blocker ticket for 2.7?

On Thu, Aug 16, 2018, 19:59 Denis Magda <[hidden email]> wrote:

> Igniters,
>
> Was troubleshooting an Ignite deployment today and couldn't find out from
> the logs what was the actual off-heap space used.
>
> Those were the given memory resoures (Ignite 2.6):
>
> [2018-08-16 15:07:49,961][INFO ][main][GridDiscoveryManager] Topology
> snapshot [ver=1, servers=1, clients=0, CPUs=64, *offheap=30.0GB*,
> heap=24.0GB]
>
> And that weird stuff was reported by the node (pay attention to the last
> line):
>
> [2018-08-16 15:45:50,211][INFO
>
> ][grid-timeout-worker-#135%cluster_31-Dec-2017%][IgniteKernal%cluster_31-Dec-2017]
> Metrics for local node (to disable set 'metricsLogFrequency' to 0)
>     ^-- Node [id=c033026e, name=cluster_31-Dec-2017, uptime=00:38:00.257]
>     ^-- H/N/C [hosts=1, nodes=1, CPUs=64]
>     ^-- CPU [cur=0.03%, avg=5.54%, GC=0%]
>     ^-- PageMemory [pages=6997377]
>     ^-- Heap [used=9706MB, free=61.18%, comm=22384MB]
>    * ^-- Non heap [used=144MB, free=-1%, comm=148MB] - this line is always
> the same!*
>
> Had to change the code by using dataRegion.getPhysicalMemoryPages() to find
> out that actual off-heap usage size was
> >>> Physical Memory Size: 28651614208 => 27324 MB, 26 GB
>
> Let's fix this issue in 2.7, I proposed a new format. Please review and
> share your thoughts:
> https://issues.apache.org/jira/browse/IGNITE-9305
>
> --
> Denis
>

[dmagda]

Yes, it was at the end of my wordy email :)
https://issues.apache.org/jira/browse/IGNITE-9305

--
Denis

On Thu, Aug 16, 2018 at 11:03 PM Dmitriy Setrakyan <[hidden email]>
wrote:

> Is there a blocker ticket for 2.7?
>
> On Thu, Aug 16, 2018, 19:59 Denis Magda <[hidden email]> wrote:
>
> > Igniters,
> >
> > Was troubleshooting an Ignite deployment today and couldn't find out from
> > the logs what was the actual off-heap space used.
> >
> > Those were the given memory resoures (Ignite 2.6):
> >
> > [2018-08-16 15:07:49,961][INFO ][main][GridDiscoveryManager] Topology
> > snapshot [ver=1, servers=1, clients=0, CPUs=64, *offheap=30.0GB*,
> > heap=24.0GB]
> >
> > And that weird stuff was reported by the node (pay attention to the last
> > line):
> >
> > [2018-08-16 15:45:50,211][INFO
> >
> >
> ][grid-timeout-worker-#135%cluster_31-Dec-2017%][IgniteKernal%cluster_31-Dec-2017]
> > Metrics for local node (to disable set 'metricsLogFrequency' to 0)
> >     ^-- Node [id=c033026e, name=cluster_31-Dec-2017, uptime=00:38:00.257]
> >     ^-- H/N/C [hosts=1, nodes=1, CPUs=64]
> >     ^-- CPU [cur=0.03%, avg=5.54%, GC=0%]
> >     ^-- PageMemory [pages=6997377]
> >     ^-- Heap [used=9706MB, free=61.18%, comm=22384MB]
> >    * ^-- Non heap [used=144MB, free=-1%, comm=148MB] - this line is
> always
> > the same!*
> >
> > Had to change the code by using dataRegion.getPhysicalMemoryPages() to
> find
> > out that actual off-heap usage size was
> > >>> Physical Memory Size: 28651614208 => 27324 MB, 26 GB
> >
> > Let's fix this issue in 2.7, I proposed a new format. Please review and
> > share your thoughts:
> > https://issues.apache.org/jira/browse/IGNITE-9305
> >
> > --
> > Denis
> >
>

[Dmitriy Pavlov]

Hi Igniters,

It is not an easy fix, so I'm not sure it is possible to do in 2.7.

Offheap size is not reported by VM (it returns -1). To implement it we need
totally migrate off-heap memory metrics to durable memory data.

I think this issue was reported and I'll find the duplicate.

Sincerely,
Dmitriy Pavlov

пт, 17 авг. 2018 г. в 6:10, Denis Magda <[hidden email]>:

> Yes, it was at the end of my wordy email :)
> https://issues.apache.org/jira/browse/IGNITE-9305
>
> --
> Denis
>
> On Thu, Aug 16, 2018 at 11:03 PM Dmitriy Setrakyan <[hidden email]>
> wrote:
>
> > Is there a blocker ticket for 2.7?
> >
> > On Thu, Aug 16, 2018, 19:59 Denis Magda <[hidden email]> wrote:
> >
> > > Igniters,
> > >
> > > Was troubleshooting an Ignite deployment today and couldn't find out
> from
> > > the logs what was the actual off-heap space used.
> > >
> > > Those were the given memory resoures (Ignite 2.6):
> > >
> > > [2018-08-16 15:07:49,961][INFO ][main][GridDiscoveryManager] Topology
> > > snapshot [ver=1, servers=1, clients=0, CPUs=64, *offheap=30.0GB*,
> > > heap=24.0GB]
> > >
> > > And that weird stuff was reported by the node (pay attention to the
> last
> > > line):
> > >
> > > [2018-08-16 15:45:50,211][INFO
> > >
> > >
> >
> ][grid-timeout-worker-#135%cluster_31-Dec-2017%][IgniteKernal%cluster_31-Dec-2017]
> > > Metrics for local node (to disable set 'metricsLogFrequency' to 0)
> > >     ^-- Node [id=c033026e, name=cluster_31-Dec-2017,
> uptime=00:38:00.257]
> > >     ^-- H/N/C [hosts=1, nodes=1, CPUs=64]
> > >     ^-- CPU [cur=0.03%, avg=5.54%, GC=0%]
> > >     ^-- PageMemory [pages=6997377]
> > >     ^-- Heap [used=9706MB, free=61.18%, comm=22384MB]
> > >    * ^-- Non heap [used=144MB, free=-1%, comm=148MB] - this line is
> > always
> > > the same!*
> > >
> > > Had to change the code by using dataRegion.getPhysicalMemoryPages() to
> > find
> > > out that actual off-heap usage size was
> > > >>> Physical Memory Size: 28651614208 => 27324 MB, 26 GB
> > >
> > > Let's fix this issue in 2.7, I proposed a new format. Please review and
> > > share your thoughts:
> > > https://issues.apache.org/jira/browse/IGNITE-9305
> > >
> > > --
> > > Denis
> > >
> >
>

[Mmuzaf]

Igniters,

Suppose, Dmitry is talking about IGNITE-5583 [1] - `Switch non-heap memory
metrics
to new page memory semantics` and related previous disscustions to it [4].

Also we have some additional improvements to CacheMetrics:
IGNITE-5490 [2] - `Implement replacement for obsolete
CacheMetrics#getOffHeapAllocatedSize`
IGNITE-5765 [3] - `CacheMetrics interface cleanup, documentation and fixes`


[1] https://issues.apache.org/jira/browse/IGNITE-5583
[2] https://issues.apache.org/jira/browse/IGNITE-5490
[3] https://issues.apache.org/jira/browse/IGNITE-5765
[4]
http://apache-ignite-developers.2346864.n4.nabble.com/Negative-non-heap-memory-maximum-td17990.html

On Fri, 17 Aug 2018 at 10:14 Dmitriy Pavlov <[hidden email]> wrote:

> Hi Igniters,
>
> It is not an easy fix, so I'm not sure it is possible to do in 2.7.
>
> Offheap size is not reported by VM (it returns -1). To implement it we need
> totally migrate off-heap memory metrics to durable memory data.
>
> I think this issue was reported and I'll find the duplicate.
>
> Sincerely,
> Dmitriy Pavlov
>
> пт, 17 авг. 2018 г. в 6:10, Denis Magda <[hidden email]>:
>
> > Yes, it was at the end of my wordy email :)
> > https://issues.apache.org/jira/browse/IGNITE-9305
> >
> > --
> > Denis
> >
> > On Thu, Aug 16, 2018 at 11:03 PM Dmitriy Setrakyan <
> [hidden email]>
> > wrote:
> >
> > > Is there a blocker ticket for 2.7?
> > >
> > > On Thu, Aug 16, 2018, 19:59 Denis Magda <[hidden email]> wrote:
> > >
> > > > Igniters,
> > > >
> > > > Was troubleshooting an Ignite deployment today and couldn't find out
> > from
> > > > the logs what was the actual off-heap space used.
> > > >
> > > > Those were the given memory resoures (Ignite 2.6):
> > > >
> > > > [2018-08-16 15:07:49,961][INFO ][main][GridDiscoveryManager] Topology
> > > > snapshot [ver=1, servers=1, clients=0, CPUs=64, *offheap=30.0GB*,
> > > > heap=24.0GB]
> > > >
> > > > And that weird stuff was reported by the node (pay attention to the
> > last
> > > > line):
> > > >
> > > > [2018-08-16 15:45:50,211][INFO
> > > >
> > > >
> > >
> >
> ][grid-timeout-worker-#135%cluster_31-Dec-2017%][IgniteKernal%cluster_31-Dec-2017]
> > > > Metrics for local node (to disable set 'metricsLogFrequency' to 0)
> > > >     ^-- Node [id=c033026e, name=cluster_31-Dec-2017,
> > uptime=00:38:00.257]
> > > >     ^-- H/N/C [hosts=1, nodes=1, CPUs=64]
> > > >     ^-- CPU [cur=0.03%, avg=5.54%, GC=0%]
> > > >     ^-- PageMemory [pages=6997377]
> > > >     ^-- Heap [used=9706MB, free=61.18%, comm=22384MB]
> > > >    * ^-- Non heap [used=144MB, free=-1%, comm=148MB] - this line is
> > > always
> > > > the same!*
> > > >
> > > > Had to change the code by using dataRegion.getPhysicalMemoryPages()
> to
> > > find
> > > > out that actual off-heap usage size was
> > > > >>> Physical Memory Size: 28651614208 => 27324 MB, 26 GB
> > > >
> > > > Let's fix this issue in 2.7, I proposed a new format. Please review
> and
> > > > share your thoughts:
> > > > https://issues.apache.org/jira/browse/IGNITE-9305
> > > >
> > > > --
> > > > Denis
> > > >
> > >
> >
>

--
--
Maxim Muzafarov

[Dmitriy Pavlov]

Hi Maxim,

thank you for stepping in and for finding these issues. Yes, these tickets
are correct.

I can move https://issues.apache.org/jira/browse/IGNITE-5583 to unassigned
if someone would like to implement this change. I will not have enough time
to complete it in 1 month (before 2.7 release).

Sincerely,
Dmitriy Pavlov

пт, 17 авг. 2018 г. в 11:04, Maxim Muzafarov <[hidden email]>:

> Igniters,
>
> Suppose, Dmitry is talking about IGNITE-5583 [1] - `Switch non-heap memory
> metrics
> to new page memory semantics` and related previous disscustions to it [4].
>
> Also we have some additional improvements to CacheMetrics:
> IGNITE-5490 [2] - `Implement replacement for obsolete
> CacheMetrics#getOffHeapAllocatedSize`
> IGNITE-5765 [3] - `CacheMetrics interface cleanup, documentation and fixes`
>
>
> [1] https://issues.apache.org/jira/browse/IGNITE-5583
> [2] https://issues.apache.org/jira/browse/IGNITE-5490
> [3] https://issues.apache.org/jira/browse/IGNITE-5765
> [4]
>
> http://apache-ignite-developers.2346864.n4.nabble.com/Negative-non-heap-memory-maximum-td17990.html
>
> On Fri, 17 Aug 2018 at 10:14 Dmitriy Pavlov <[hidden email]> wrote:
>
> > Hi Igniters,
> >
> > It is not an easy fix, so I'm not sure it is possible to do in 2.7.
> >
> > Offheap size is not reported by VM (it returns -1). To implement it we
> need
> > totally migrate off-heap memory metrics to durable memory data.
> >
> > I think this issue was reported and I'll find the duplicate.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > пт, 17 авг. 2018 г. в 6:10, Denis Magda <[hidden email]>:
> >
> > > Yes, it was at the end of my wordy email :)
> > > https://issues.apache.org/jira/browse/IGNITE-9305
> > >
> > > --
> > > Denis
> > >
> > > On Thu, Aug 16, 2018 at 11:03 PM Dmitriy Setrakyan <
> > [hidden email]>
> > > wrote:
> > >
> > > > Is there a blocker ticket for 2.7?
> > > >
> > > > On Thu, Aug 16, 2018, 19:59 Denis Magda <[hidden email]> wrote:
> > > >
> > > > > Igniters,
> > > > >
> > > > > Was troubleshooting an Ignite deployment today and couldn't find
> out
> > > from
> > > > > the logs what was the actual off-heap space used.
> > > > >
> > > > > Those were the given memory resoures (Ignite 2.6):
> > > > >
> > > > > [2018-08-16 15:07:49,961][INFO ][main][GridDiscoveryManager]
> Topology
> > > > > snapshot [ver=1, servers=1, clients=0, CPUs=64, *offheap=30.0GB*,
> > > > > heap=24.0GB]
> > > > >
> > > > > And that weird stuff was reported by the node (pay attention to the
> > > last
> > > > > line):
> > > > >
> > > > > [2018-08-16 15:45:50,211][INFO
> > > > >
> > > > >
> > > >
> > >
> >
> ][grid-timeout-worker-#135%cluster_31-Dec-2017%][IgniteKernal%cluster_31-Dec-2017]
> > > > > Metrics for local node (to disable set 'metricsLogFrequency' to 0)
> > > > >     ^-- Node [id=c033026e, name=cluster_31-Dec-2017,
> > > uptime=00:38:00.257]
> > > > >     ^-- H/N/C [hosts=1, nodes=1, CPUs=64]
> > > > >     ^-- CPU [cur=0.03%, avg=5.54%, GC=0%]
> > > > >     ^-- PageMemory [pages=6997377]
> > > > >     ^-- Heap [used=9706MB, free=61.18%, comm=22384MB]
> > > > >    * ^-- Non heap [used=144MB, free=-1%, comm=148MB] - this line is
> > > > always
> > > > > the same!*
> > > > >
> > > > > Had to change the code by using dataRegion.getPhysicalMemoryPages()
> > to
> > > > find
> > > > > out that actual off-heap usage size was
> > > > > >>> Physical Memory Size: 28651614208 => 27324 MB, 26 GB
> > > > >
> > > > > Let's fix this issue in 2.7, I proposed a new format. Please review
> > and
> > > > > share your thoughts:
> > > > > https://issues.apache.org/jira/browse/IGNITE-9305
> > > > >
> > > > > --
> > > > > Denis
> > > > >
> > > >
> > >
> >
> --
> --
> Maxim Muzafarov
>

[Vladimir Ozerov]

Folks,

We already have this:
>>> PageMemory [pages=6997377]

Then we can multiply it by page size and get occupied memory. Am I wrong?

On Fri, Aug 17, 2018 at 12:56 PM Dmitriy Pavlov <[hidden email]>
wrote:

> Hi Maxim,
>
> thank you for stepping in and for finding these issues. Yes, these tickets
> are correct.
>
> I can move https://issues.apache.org/jira/browse/IGNITE-5583 to unassigned
> if someone would like to implement this change. I will not have enough time
> to complete it in 1 month (before 2.7 release).
>
> Sincerely,
> Dmitriy Pavlov
>
> пт, 17 авг. 2018 г. в 11:04, Maxim Muzafarov <[hidden email]>:
>
> > Igniters,
> >
> > Suppose, Dmitry is talking about IGNITE-5583 [1] - `Switch non-heap
> memory
> > metrics
> > to new page memory semantics` and related previous disscustions to it
> [4].
> >
> > Also we have some additional improvements to CacheMetrics:
> > IGNITE-5490 [2] - `Implement replacement for obsolete
> > CacheMetrics#getOffHeapAllocatedSize`
> > IGNITE-5765 [3] - `CacheMetrics interface cleanup, documentation and
> fixes`
> >
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-5583
> > [2] https://issues.apache.org/jira/browse/IGNITE-5490
> > [3] https://issues.apache.org/jira/browse/IGNITE-5765
> > [4]
> >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/Negative-non-heap-memory-maximum-td17990.html
> >
> > On Fri, 17 Aug 2018 at 10:14 Dmitriy Pavlov <[hidden email]>
> wrote:
> >
> > > Hi Igniters,
> > >
> > > It is not an easy fix, so I'm not sure it is possible to do in 2.7.
> > >
> > > Offheap size is not reported by VM (it returns -1). To implement it we
> > need
> > > totally migrate off-heap memory metrics to durable memory data.
> > >
> > > I think this issue was reported and I'll find the duplicate.
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > пт, 17 авг. 2018 г. в 6:10, Denis Magda <[hidden email]>:
> > >
> > > > Yes, it was at the end of my wordy email :)
> > > > https://issues.apache.org/jira/browse/IGNITE-9305
> > > >
> > > > --
> > > > Denis
> > > >
> > > > On Thu, Aug 16, 2018 at 11:03 PM Dmitriy Setrakyan <
> > > [hidden email]>
> > > > wrote:
> > > >
> > > > > Is there a blocker ticket for 2.7?
> > > > >
> > > > > On Thu, Aug 16, 2018, 19:59 Denis Magda <[hidden email]> wrote:
> > > > >
> > > > > > Igniters,
> > > > > >
> > > > > > Was troubleshooting an Ignite deployment today and couldn't find
> > out
> > > > from
> > > > > > the logs what was the actual off-heap space used.
> > > > > >
> > > > > > Those were the given memory resoures (Ignite 2.6):
> > > > > >
> > > > > > [2018-08-16 15:07:49,961][INFO ][main][GridDiscoveryManager]
> > Topology
> > > > > > snapshot [ver=1, servers=1, clients=0, CPUs=64, *offheap=30.0GB*,
> > > > > > heap=24.0GB]
> > > > > >
> > > > > > And that weird stuff was reported by the node (pay attention to
> the
> > > > last
> > > > > > line):
> > > > > >
> > > > > > [2018-08-16 15:45:50,211][INFO
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> ][grid-timeout-worker-#135%cluster_31-Dec-2017%][IgniteKernal%cluster_31-Dec-2017]
> > > > > > Metrics for local node (to disable set 'metricsLogFrequency' to
> 0)
> > > > > >     ^-- Node [id=c033026e, name=cluster_31-Dec-2017,
> > > > uptime=00:38:00.257]
> > > > > >     ^-- H/N/C [hosts=1, nodes=1, CPUs=64]
> > > > > >     ^-- CPU [cur=0.03%, avg=5.54%, GC=0%]
> > > > > >     ^-- PageMemory [pages=6997377]
> > > > > >     ^-- Heap [used=9706MB, free=61.18%, comm=22384MB]
> > > > > >    * ^-- Non heap [used=144MB, free=-1%, comm=148MB] - this line
> is
> > > > > always
> > > > > > the same!*
> > > > > >
> > > > > > Had to change the code by using
> dataRegion.getPhysicalMemoryPages()
> > > to
> > > > > find
> > > > > > out that actual off-heap usage size was
> > > > > > >>> Physical Memory Size: 28651614208 => 27324 MB, 26 GB
> > > > > >
> > > > > > Let's fix this issue in 2.7, I proposed a new format. Please
> review
> > > and
> > > > > > share your thoughts:
> > > > > > https://issues.apache.org/jira/browse/IGNITE-9305
> > > > > >
> > > > > > --
> > > > > > Denis
> > > > > >
> > > > >
> > > >
> > >
> > --
> > --
> > Maxim Muzafarov
> >
>

[Alexey Plekhanov]

To be more precise we need to multiply it by page size with system overhead
(systemPageSize). If we want to print only used offheap memory, this will
solve the problem. If we want to print, for example, currently allocated
(commited) offheap memory (if persistence is disabled for data region then
offheap is allocated by chunks) we need to do more complex calculations and
there is no public API for this now.

2018-08-17 17:19 GMT+03:00 Vladimir Ozerov <[hidden email]>:

> Folks,
>
> We already have this:
> >>> PageMemory [pages=6997377]
>
> Then we can multiply it by page size and get occupied memory. Am I wrong?
>
> On Fri, Aug 17, 2018 at 12:56 PM Dmitriy Pavlov <[hidden email]>
> wrote:
>
> > Hi Maxim,
> >
> > thank you for stepping in and for finding these issues. Yes, these
> tickets
> > are correct.
> >
> > I can move https://issues.apache.org/jira/browse/IGNITE-5583 to
> unassigned
> > if someone would like to implement this change. I will not have enough
> time
> > to complete it in 1 month (before 2.7 release).
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > пт, 17 авг. 2018 г. в 11:04, Maxim Muzafarov <[hidden email]>:
> >
> > > Igniters,
> > >
> > > Suppose, Dmitry is talking about IGNITE-5583 [1] - `Switch non-heap
> > memory
> > > metrics
> > > to new page memory semantics` and related previous disscustions to it
> > [4].
> > >
> > > Also we have some additional improvements to CacheMetrics:
> > > IGNITE-5490 [2] - `Implement replacement for obsolete
> > > CacheMetrics#getOffHeapAllocatedSize`
> > > IGNITE-5765 [3] - `CacheMetrics interface cleanup, documentation and
> > fixes`
> > >
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-5583
> > > [2] https://issues.apache.org/jira/browse/IGNITE-5490
> > > [3] https://issues.apache.org/jira/browse/IGNITE-5765
> > > [4]
> > >
> > >
> > http://apache-ignite-developers.2346864.n4.nabble.
> com/Negative-non-heap-memory-maximum-td17990.html
> > >
> > > On Fri, 17 Aug 2018 at 10:14 Dmitriy Pavlov <[hidden email]>
> > wrote:
> > >
> > > > Hi Igniters,
> > > >
> > > > It is not an easy fix, so I'm not sure it is possible to do in 2.7.
> > > >
> > > > Offheap size is not reported by VM (it returns -1). To implement it
> we
> > > need
> > > > totally migrate off-heap memory metrics to durable memory data.
> > > >
> > > > I think this issue was reported and I'll find the duplicate.
> > > >
> > > > Sincerely,
> > > > Dmitriy Pavlov
> > > >
> > > > пт, 17 авг. 2018 г. в 6:10, Denis Magda <[hidden email]>:
> > > >
> > > > > Yes, it was at the end of my wordy email :)
> > > > > https://issues.apache.org/jira/browse/IGNITE-9305
> > > > >
> > > > > --
> > > > > Denis
> > > > >
> > > > > On Thu, Aug 16, 2018 at 11:03 PM Dmitriy Setrakyan <
> > > > [hidden email]>
> > > > > wrote:
> > > > >
> > > > > > Is there a blocker ticket for 2.7?
> > > > > >
> > > > > > On Thu, Aug 16, 2018, 19:59 Denis Magda <[hidden email]>
> wrote:
> > > > > >
> > > > > > > Igniters,
> > > > > > >
> > > > > > > Was troubleshooting an Ignite deployment today and couldn't
> find
> > > out
> > > > > from
> > > > > > > the logs what was the actual off-heap space used.
> > > > > > >
> > > > > > > Those were the given memory resoures (Ignite 2.6):
> > > > > > >
> > > > > > > [2018-08-16 15:07:49,961][INFO ][main][GridDiscoveryManager]
> > > Topology
> > > > > > > snapshot [ver=1, servers=1, clients=0, CPUs=64,
> *offheap=30.0GB*,
> > > > > > > heap=24.0GB]
> > > > > > >
> > > > > > > And that weird stuff was reported by the node (pay attention to
> > the
> > > > > last
> > > > > > > line):
> > > > > > >
> > > > > > > [2018-08-16 15:45:50,211][INFO
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > ][grid-timeout-worker-#135%cluster_31-Dec-2017%][
> IgniteKernal%cluster_31-Dec-2017]
> > > > > > > Metrics for local node (to disable set 'metricsLogFrequency' to
> > 0)
> > > > > > >     ^-- Node [id=c033026e, name=cluster_31-Dec-2017,
> > > > > uptime=00:38:00.257]
> > > > > > >     ^-- H/N/C [hosts=1, nodes=1, CPUs=64]
> > > > > > >     ^-- CPU [cur=0.03%, avg=5.54%, GC=0%]
> > > > > > >     ^-- PageMemory [pages=6997377]
> > > > > > >     ^-- Heap [used=9706MB, free=61.18%, comm=22384MB]
> > > > > > >    * ^-- Non heap [used=144MB, free=-1%, comm=148MB] - this
> line
> > is
> > > > > > always
> > > > > > > the same!*
> > > > > > >
> > > > > > > Had to change the code by using
> > dataRegion.getPhysicalMemoryPages()
> > > > to
> > > > > > find
> > > > > > > out that actual off-heap usage size was
> > > > > > > >>> Physical Memory Size: 28651614208 => 27324 MB, 26 GB
> > > > > > >
> > > > > > > Let's fix this issue in 2.7, I proposed a new format. Please
> > review
> > > > and
> > > > > > > share your thoughts:
> > > > > > > https://issues.apache.org/jira/browse/IGNITE-9305
> > > > > > >
> > > > > > > --
> > > > > > > Denis
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > --
> > > --
> > > Maxim Muzafarov
> > >
> >
>

[Dmitriy Pavlov]

Yes, I agree.

To calculate free (currently -1) we need to know total & used.
Used=sysPageSize*usedPages, but overall should be obtained from all
segments and chunks. So it would be a remarkable part of all mem.metrics
migration (required by IGNITE-5583).

пт, 17 авг. 2018 г. в 23:55, Alex Plehanov <[hidden email]>:

> To be more precise we need to multiply it by page size with system overhead
> (systemPageSize). If we want to print only used offheap memory, this will
> solve the problem. If we want to print, for example, currently allocated
> (commited) offheap memory (if persistence is disabled for data region then
> offheap is allocated by chunks) we need to do more complex calculations and
> there is no public API for this now.
>
> 2018-08-17 17:19 GMT+03:00 Vladimir Ozerov <[hidden email]>:
>
> > Folks,
> >
> > We already have this:
> > >>> PageMemory [pages=6997377]
> >
> > Then we can multiply it by page size and get occupied memory. Am I wrong?
> >
> > On Fri, Aug 17, 2018 at 12:56 PM Dmitriy Pavlov <[hidden email]>
> > wrote:
> >
> > > Hi Maxim,
> > >
> > > thank you for stepping in and for finding these issues. Yes, these
> > tickets
> > > are correct.
> > >
> > > I can move https://issues.apache.org/jira/browse/IGNITE-5583 to
> > unassigned
> > > if someone would like to implement this change. I will not have enough
> > time
> > > to complete it in 1 month (before 2.7 release).
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > пт, 17 авг. 2018 г. в 11:04, Maxim Muzafarov <[hidden email]>:
> > >
> > > > Igniters,
> > > >
> > > > Suppose, Dmitry is talking about IGNITE-5583 [1] - `Switch non-heap
> > > memory
> > > > metrics
> > > > to new page memory semantics` and related previous disscustions to it
> > > [4].
> > > >
> > > > Also we have some additional improvements to CacheMetrics:
> > > > IGNITE-5490 [2] - `Implement replacement for obsolete
> > > > CacheMetrics#getOffHeapAllocatedSize`
> > > > IGNITE-5765 [3] - `CacheMetrics interface cleanup, documentation and
> > > fixes`
> > > >
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IGNITE-5583
> > > > [2] https://issues.apache.org/jira/browse/IGNITE-5490
> > > > [3] https://issues.apache.org/jira/browse/IGNITE-5765
> > > > [4]
> > > >
> > > >
> > > http://apache-ignite-developers.2346864.n4.nabble.
> > com/Negative-non-heap-memory-maximum-td17990.html
> > > >
> > > > On Fri, 17 Aug 2018 at 10:14 Dmitriy Pavlov <[hidden email]>
> > > wrote:
> > > >
> > > > > Hi Igniters,
> > > > >
> > > > > It is not an easy fix, so I'm not sure it is possible to do in 2.7.
> > > > >
> > > > > Offheap size is not reported by VM (it returns -1). To implement it
> > we
> > > > need
> > > > > totally migrate off-heap memory metrics to durable memory data.
> > > > >
> > > > > I think this issue was reported and I'll find the duplicate.
> > > > >
> > > > > Sincerely,
> > > > > Dmitriy Pavlov
> > > > >
> > > > > пт, 17 авг. 2018 г. в 6:10, Denis Magda <[hidden email]>:
> > > > >
> > > > > > Yes, it was at the end of my wordy email :)
> > > > > > https://issues.apache.org/jira/browse/IGNITE-9305
> > > > > >
> > > > > > --
> > > > > > Denis
> > > > > >
> > > > > > On Thu, Aug 16, 2018 at 11:03 PM Dmitriy Setrakyan <
> > > > > [hidden email]>
> > > > > > wrote:
> > > > > >
> > > > > > > Is there a blocker ticket for 2.7?
> > > > > > >
> > > > > > > On Thu, Aug 16, 2018, 19:59 Denis Magda <[hidden email]>
> > wrote:
> > > > > > >
> > > > > > > > Igniters,
> > > > > > > >
> > > > > > > > Was troubleshooting an Ignite deployment today and couldn't
> > find
> > > > out
> > > > > > from
> > > > > > > > the logs what was the actual off-heap space used.
> > > > > > > >
> > > > > > > > Those were the given memory resoures (Ignite 2.6):
> > > > > > > >
> > > > > > > > [2018-08-16 15:07:49,961][INFO ][main][GridDiscoveryManager]
> > > > Topology
> > > > > > > > snapshot [ver=1, servers=1, clients=0, CPUs=64,
> > *offheap=30.0GB*,
> > > > > > > > heap=24.0GB]
> > > > > > > >
> > > > > > > > And that weird stuff was reported by the node (pay attention
> to
> > > the
> > > > > > last
> > > > > > > > line):
> > > > > > > >
> > > > > > > > [2018-08-16 15:45:50,211][INFO
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > ][grid-timeout-worker-#135%cluster_31-Dec-2017%][
> > IgniteKernal%cluster_31-Dec-2017]
> > > > > > > > Metrics for local node (to disable set 'metricsLogFrequency'
> to
> > > 0)
> > > > > > > >     ^-- Node [id=c033026e, name=cluster_31-Dec-2017,
> > > > > > uptime=00:38:00.257]
> > > > > > > >     ^-- H/N/C [hosts=1, nodes=1, CPUs=64]
> > > > > > > >     ^-- CPU [cur=0.03%, avg=5.54%, GC=0%]
> > > > > > > >     ^-- PageMemory [pages=6997377]
> > > > > > > >     ^-- Heap [used=9706MB, free=61.18%, comm=22384MB]
> > > > > > > >    * ^-- Non heap [used=144MB, free=-1%, comm=148MB] - this
> > line
> > > is
> > > > > > > always
> > > > > > > > the same!*
> > > > > > > >
> > > > > > > > Had to change the code by using
> > > dataRegion.getPhysicalMemoryPages()
> > > > > to
> > > > > > > find
> > > > > > > > out that actual off-heap usage size was
> > > > > > > > >>> Physical Memory Size: 28651614208 => 27324 MB, 26 GB
> > > > > > > >
> > > > > > > > Let's fix this issue in 2.7, I proposed a new format. Please
> > > review
> > > > > and
> > > > > > > > share your thoughts:
> > > > > > > > https://issues.apache.org/jira/browse/IGNITE-9305
> > > > > > > >
> > > > > > > > --
> > > > > > > > Denis
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > --
> > > > --
> > > > Maxim Muzafarov
> > > >
> > >
> >
>

[dmagda]

Vova, the things are even simpler - we have this

ignite.dataRegionMetrics().getPhysicalMemorySize() that returns the
number equal/comparabel to pageNumber X pageSize.


Igniters, if you believe that we need to do more work here then let's
do it iteratively. Let's fix the off-heap occupied size the way above
(just print out getPhysicalMemorySize() for every data region). Then
do the rest. This needs to be fixed in 2.7.


--

Denis


On Fri, Aug 17, 2018 at 10:20 AM Vladimir Ozerov <[hidden email]>
wrote:

> Folks,
>
> We already have this:
> >>> PageMemory [pages=6997377]
>
> Then we can multiply it by page size and get occupied memory. Am I wrong?
>
> On Fri, Aug 17, 2018 at 12:56 PM Dmitriy Pavlov <[hidden email]>
> wrote:
>
> > Hi Maxim,
> >
> > thank you for stepping in and for finding these issues. Yes, these
> tickets
> > are correct.
> >
> > I can move https://issues.apache.org/jira/browse/IGNITE-5583 to
> unassigned
> > if someone would like to implement this change. I will not have enough
> time
> > to complete it in 1 month (before 2.7 release).
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > пт, 17 авг. 2018 г. в 11:04, Maxim Muzafarov <[hidden email]>:
> >
> > > Igniters,
> > >
> > > Suppose, Dmitry is talking about IGNITE-5583 [1] - `Switch non-heap
> > memory
> > > metrics
> > > to new page memory semantics` and related previous disscustions to it
> > [4].
> > >
> > > Also we have some additional improvements to CacheMetrics:
> > > IGNITE-5490 [2] - `Implement replacement for obsolete
> > > CacheMetrics#getOffHeapAllocatedSize`
> > > IGNITE-5765 [3] - `CacheMetrics interface cleanup, documentation and
> > fixes`
> > >
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-5583
> > > [2] https://issues.apache.org/jira/browse/IGNITE-5490
> > > [3] https://issues.apache.org/jira/browse/IGNITE-5765
> > > [4]
> > >
> > >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/Negative-non-heap-memory-maximum-td17990.html
> > >
> > > On Fri, 17 Aug 2018 at 10:14 Dmitriy Pavlov <[hidden email]>
> > wrote:
> > >
> > > > Hi Igniters,
> > > >
> > > > It is not an easy fix, so I'm not sure it is possible to do in 2.7.
> > > >
> > > > Offheap size is not reported by VM (it returns -1). To implement it
> we
> > > need
> > > > totally migrate off-heap memory metrics to durable memory data.
> > > >
> > > > I think this issue was reported and I'll find the duplicate.
> > > >
> > > > Sincerely,
> > > > Dmitriy Pavlov
> > > >
> > > > пт, 17 авг. 2018 г. в 6:10, Denis Magda <[hidden email]>:
> > > >
> > > > > Yes, it was at the end of my wordy email :)
> > > > > https://issues.apache.org/jira/browse/IGNITE-9305
> > > > >
> > > > > --
> > > > > Denis
> > > > >
> > > > > On Thu, Aug 16, 2018 at 11:03 PM Dmitriy Setrakyan <
> > > > [hidden email]>
> > > > > wrote:
> > > > >
> > > > > > Is there a blocker ticket for 2.7?
> > > > > >
> > > > > > On Thu, Aug 16, 2018, 19:59 Denis Magda <[hidden email]>
> wrote:
> > > > > >
> > > > > > > Igniters,
> > > > > > >
> > > > > > > Was troubleshooting an Ignite deployment today and couldn't
> find
> > > out
> > > > > from
> > > > > > > the logs what was the actual off-heap space used.
> > > > > > >
> > > > > > > Those were the given memory resoures (Ignite 2.6):
> > > > > > >
> > > > > > > [2018-08-16 15:07:49,961][INFO ][main][GridDiscoveryManager]
> > > Topology
> > > > > > > snapshot [ver=1, servers=1, clients=0, CPUs=64,
> *offheap=30.0GB*,
> > > > > > > heap=24.0GB]
> > > > > > >
> > > > > > > And that weird stuff was reported by the node (pay attention to
> > the
> > > > > last
> > > > > > > line):
> > > > > > >
> > > > > > > [2018-08-16 15:45:50,211][INFO
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> ][grid-timeout-worker-#135%cluster_31-Dec-2017%][IgniteKernal%cluster_31-Dec-2017]
> > > > > > > Metrics for local node (to disable set 'metricsLogFrequency' to
> > 0)
> > > > > > >     ^-- Node [id=c033026e, name=cluster_31-Dec-2017,
> > > > > uptime=00:38:00.257]
> > > > > > >     ^-- H/N/C [hosts=1, nodes=1, CPUs=64]
> > > > > > >     ^-- CPU [cur=0.03%, avg=5.54%, GC=0%]
> > > > > > >     ^-- PageMemory [pages=6997377]
> > > > > > >     ^-- Heap [used=9706MB, free=61.18%, comm=22384MB]
> > > > > > >    * ^-- Non heap [used=144MB, free=-1%, comm=148MB] - this
> line
> > is
> > > > > > always
> > > > > > > the same!*
> > > > > > >
> > > > > > > Had to change the code by using
> > dataRegion.getPhysicalMemoryPages()
> > > > to
> > > > > > find
> > > > > > > out that actual off-heap usage size was
> > > > > > > >>> Physical Memory Size: 28651614208 => 27324 MB, 26 GB
> > > > > > >
> > > > > > > Let's fix this issue in 2.7, I proposed a new format. Please
> > review
> > > > and
> > > > > > > share your thoughts:
> > > > > > > https://issues.apache.org/jira/browse/IGNITE-9305
> > > > > > >
> > > > > > > --
> > > > > > > Denis
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > --
> > > --
> > > Maxim Muzafarov
> > >
> >
>

[Pavel Pereslegin]

Hello, Igniters.

I assigned ticket [1] created by Denis and want to clarify how to log
committed size.
The metric offHeapSize (in DataRegionMetricsImpl) is always
calculated, but getOffHeapSize returns zero if memory metrics are
disabled for this data region.

So I see the following options:
1. Modify method getOffHeapSize so that it always returns actual value
offHeapSize.
2. Add another offHeapSize() method.
3. Output to log max size instead of committed (change "comm" to "max"
in log output).
4. Don't bother about disabling metrics and output to log value
returned by getOffHeapSize.

Any thoughts?

[1] https://issues.apache.org/jira/browse/IGNITE-9305
сб, 18 авг. 2018 г. в 3:17, Denis Magda <[hidden email]>:

>
> Vova, the things are even simpler - we have this
>
> ignite.dataRegionMetrics().getPhysicalMemorySize() that returns the
> number equal/comparabel to pageNumber X pageSize.
>
>
> Igniters, if you believe that we need to do more work here then let's
> do it iteratively. Let's fix the off-heap occupied size the way above
> (just print out getPhysicalMemorySize() for every data region). Then
> do the rest. This needs to be fixed in 2.7.
>
>
> --
>
> Denis
>
>
> On Fri, Aug 17, 2018 at 10:20 AM Vladimir Ozerov <[hidden email]>
> wrote:
>
> > Folks,
> >
> > We already have this:
> > >>> PageMemory [pages=6997377]
> >
> > Then we can multiply it by page size and get occupied memory. Am I wrong?
> >
> > On Fri, Aug 17, 2018 at 12:56 PM Dmitriy Pavlov <[hidden email]>
> > wrote:
> >
> > > Hi Maxim,
> > >
> > > thank you for stepping in and for finding these issues. Yes, these
> > tickets
> > > are correct.
> > >
> > > I can move https://issues.apache.org/jira/browse/IGNITE-5583 to
> > unassigned
> > > if someone would like to implement this change. I will not have enough
> > time
> > > to complete it in 1 month (before 2.7 release).
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > пт, 17 авг. 2018 г. в 11:04, Maxim Muzafarov <[hidden email]>:
> > >
> > > > Igniters,
> > > >
> > > > Suppose, Dmitry is talking about IGNITE-5583 [1] - `Switch non-heap
> > > memory
> > > > metrics
> > > > to new page memory semantics` and related previous disscustions to it
> > > [4].
> > > >
> > > > Also we have some additional improvements to CacheMetrics:
> > > > IGNITE-5490 [2] - `Implement replacement for obsolete
> > > > CacheMetrics#getOffHeapAllocatedSize`
> > > > IGNITE-5765 [3] - `CacheMetrics interface cleanup, documentation and
> > > fixes`
> > > >
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IGNITE-5583
> > > > [2] https://issues.apache.org/jira/browse/IGNITE-5490
> > > > [3] https://issues.apache.org/jira/browse/IGNITE-5765
> > > > [4]
> > > >
> > > >
> > >
> > http://apache-ignite-developers.2346864.n4.nabble.com/Negative-non-heap-memory-maximum-td17990.html
> > > >
> > > > On Fri, 17 Aug 2018 at 10:14 Dmitriy Pavlov <[hidden email]>
> > > wrote:
> > > >
> > > > > Hi Igniters,
> > > > >
> > > > > It is not an easy fix, so I'm not sure it is possible to do in 2.7.
> > > > >
> > > > > Offheap size is not reported by VM (it returns -1). To implement it
> > we
> > > > need
> > > > > totally migrate off-heap memory metrics to durable memory data.
> > > > >
> > > > > I think this issue was reported and I'll find the duplicate.
> > > > >
> > > > > Sincerely,
> > > > > Dmitriy Pavlov
> > > > >
> > > > > пт, 17 авг. 2018 г. в 6:10, Denis Magda <[hidden email]>:
> > > > >
> > > > > > Yes, it was at the end of my wordy email :)
> > > > > > https://issues.apache.org/jira/browse/IGNITE-9305
> > > > > >
> > > > > > --
> > > > > > Denis
> > > > > >
> > > > > > On Thu, Aug 16, 2018 at 11:03 PM Dmitriy Setrakyan <
> > > > > [hidden email]>
> > > > > > wrote:
> > > > > >
> > > > > > > Is there a blocker ticket for 2.7?
> > > > > > >
> > > > > > > On Thu, Aug 16, 2018, 19:59 Denis Magda <[hidden email]>
> > wrote:
> > > > > > >
> > > > > > > > Igniters,
> > > > > > > >
> > > > > > > > Was troubleshooting an Ignite deployment today and couldn't
> > find
> > > > out
> > > > > > from
> > > > > > > > the logs what was the actual off-heap space used.
> > > > > > > >
> > > > > > > > Those were the given memory resoures (Ignite 2.6):
> > > > > > > >
> > > > > > > > [2018-08-16 15:07:49,961][INFO ][main][GridDiscoveryManager]
> > > > Topology
> > > > > > > > snapshot [ver=1, servers=1, clients=0, CPUs=64,
> > *offheap=30.0GB*,
> > > > > > > > heap=24.0GB]
> > > > > > > >
> > > > > > > > And that weird stuff was reported by the node (pay attention to
> > > the
> > > > > > last
> > > > > > > > line):
> > > > > > > >
> > > > > > > > [2018-08-16 15:45:50,211][INFO
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > ][grid-timeout-worker-#135%cluster_31-Dec-2017%][IgniteKernal%cluster_31-Dec-2017]
> > > > > > > > Metrics for local node (to disable set 'metricsLogFrequency' to
> > > 0)
> > > > > > > >     ^-- Node [id=c033026e, name=cluster_31-Dec-2017,
> > > > > > uptime=00:38:00.257]
> > > > > > > >     ^-- H/N/C [hosts=1, nodes=1, CPUs=64]
> > > > > > > >     ^-- CPU [cur=0.03%, avg=5.54%, GC=0%]
> > > > > > > >     ^-- PageMemory [pages=6997377]
> > > > > > > >     ^-- Heap [used=9706MB, free=61.18%, comm=22384MB]
> > > > > > > >    * ^-- Non heap [used=144MB, free=-1%, comm=148MB] - this
> > line
> > > is
> > > > > > > always
> > > > > > > > the same!*
> > > > > > > >
> > > > > > > > Had to change the code by using
> > > dataRegion.getPhysicalMemoryPages()
> > > > > to
> > > > > > > find
> > > > > > > > out that actual off-heap usage size was
> > > > > > > > >>> Physical Memory Size: 28651614208 => 27324 MB, 26 GB
> > > > > > > >
> > > > > > > > Let's fix this issue in 2.7, I proposed a new format. Please
> > > review
> > > > > and
> > > > > > > > share your thoughts:
> > > > > > > > https://issues.apache.org/jira/browse/IGNITE-9305
> > > > > > > >
> > > > > > > > --
> > > > > > > > Denis
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > --
> > > > --
> > > > Maxim Muzafarov
> > > >
> > >
> >

[Nikita Amelchev]

Pavel,

I think that point 1 is the correct way to calculate the committed
size for a log. It is already calculated regardless of the
metricsEnabled flag.

In addition, I suggest more readable log format for data regions in
issue comments [1].

[1] https://issues.apache.org/jira/browse/IGNITE-9305

вт, 21 авг. 2018 г. в 12:15, Pavel Pereslegin <[hidden email]>:

>
> Hello, Igniters.
>
> I assigned ticket [1] created by Denis and want to clarify how to log
> committed size.
> The metric offHeapSize (in DataRegionMetricsImpl) is always
> calculated, but getOffHeapSize returns zero if memory metrics are
> disabled for this data region.
>
> So I see the following options:
> 1. Modify method getOffHeapSize so that it always returns actual value
> offHeapSize.
> 2. Add another offHeapSize() method.
> 3. Output to log max size instead of committed (change "comm" to "max"
> in log output).
> 4. Don't bother about disabling metrics and output to log value
> returned by getOffHeapSize.
>
> Any thoughts?
>
> [1] https://issues.apache.org/jira/browse/IGNITE-9305
> сб, 18 авг. 2018 г. в 3:17, Denis Magda <[hidden email]>:
> >
> > Vova, the things are even simpler - we have this
> >
> > ignite.dataRegionMetrics().getPhysicalMemorySize() that returns the
> > number equal/comparabel to pageNumber X pageSize.
> >
> >
> > Igniters, if you believe that we need to do more work here then let's
> > do it iteratively. Let's fix the off-heap occupied size the way above
> > (just print out getPhysicalMemorySize() for every data region). Then
> > do the rest. This needs to be fixed in 2.7.
> >
> >
> > --
> >
> > Denis
> >
> >
> > On Fri, Aug 17, 2018 at 10:20 AM Vladimir Ozerov <[hidden email]>
> > wrote:
> >
> > > Folks,
> > >
> > > We already have this:
> > > >>> PageMemory [pages=6997377]
> > >
> > > Then we can multiply it by page size and get occupied memory. Am I wrong?
> > >
> > > On Fri, Aug 17, 2018 at 12:56 PM Dmitriy Pavlov <[hidden email]>
> > > wrote:
> > >
> > > > Hi Maxim,
> > > >
> > > > thank you for stepping in and for finding these issues. Yes, these
> > > tickets
> > > > are correct.
> > > >
> > > > I can move https://issues.apache.org/jira/browse/IGNITE-5583 to
> > > unassigned
> > > > if someone would like to implement this change. I will not have enough
> > > time
> > > > to complete it in 1 month (before 2.7 release).
> > > >
> > > > Sincerely,
> > > > Dmitriy Pavlov
> > > >
> > > > пт, 17 авг. 2018 г. в 11:04, Maxim Muzafarov <[hidden email]>:
> > > >
> > > > > Igniters,
> > > > >
> > > > > Suppose, Dmitry is talking about IGNITE-5583 [1] - `Switch non-heap
> > > > memory
> > > > > metrics
> > > > > to new page memory semantics` and related previous disscustions to it
> > > > [4].
> > > > >
> > > > > Also we have some additional improvements to CacheMetrics:
> > > > > IGNITE-5490 [2] - `Implement replacement for obsolete
> > > > > CacheMetrics#getOffHeapAllocatedSize`
> > > > > IGNITE-5765 [3] - `CacheMetrics interface cleanup, documentation and
> > > > fixes`
> > > > >
> > > > >
> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-5583
> > > > > [2] https://issues.apache.org/jira/browse/IGNITE-5490
> > > > > [3] https://issues.apache.org/jira/browse/IGNITE-5765
> > > > > [4]
> > > > >
> > > > >
> > > >
> > > http://apache-ignite-developers.2346864.n4.nabble.com/Negative-non-heap-memory-maximum-td17990.html
> > > > >
> > > > > On Fri, 17 Aug 2018 at 10:14 Dmitriy Pavlov <[hidden email]>
> > > > wrote:
> > > > >
> > > > > > Hi Igniters,
> > > > > >
> > > > > > It is not an easy fix, so I'm not sure it is possible to do in 2.7.
> > > > > >
> > > > > > Offheap size is not reported by VM (it returns -1). To implement it
> > > we
> > > > > need
> > > > > > totally migrate off-heap memory metrics to durable memory data.
> > > > > >
> > > > > > I think this issue was reported and I'll find the duplicate.
> > > > > >
> > > > > > Sincerely,
> > > > > > Dmitriy Pavlov
> > > > > >
> > > > > > пт, 17 авг. 2018 г. в 6:10, Denis Magda <[hidden email]>:
> > > > > >
> > > > > > > Yes, it was at the end of my wordy email :)
> > > > > > > https://issues.apache.org/jira/browse/IGNITE-9305
> > > > > > >
> > > > > > > --
> > > > > > > Denis
> > > > > > >
> > > > > > > On Thu, Aug 16, 2018 at 11:03 PM Dmitriy Setrakyan <
> > > > > > [hidden email]>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Is there a blocker ticket for 2.7?
> > > > > > > >
> > > > > > > > On Thu, Aug 16, 2018, 19:59 Denis Magda <[hidden email]>
> > > wrote:
> > > > > > > >
> > > > > > > > > Igniters,
> > > > > > > > >
> > > > > > > > > Was troubleshooting an Ignite deployment today and couldn't
> > > find
> > > > > out
> > > > > > > from
> > > > > > > > > the logs what was the actual off-heap space used.
> > > > > > > > >
> > > > > > > > > Those were the given memory resoures (Ignite 2.6):
> > > > > > > > >
> > > > > > > > > [2018-08-16 15:07:49,961][INFO ][main][GridDiscoveryManager]
> > > > > Topology
> > > > > > > > > snapshot [ver=1, servers=1, clients=0, CPUs=64,
> > > *offheap=30.0GB*,
> > > > > > > > > heap=24.0GB]
> > > > > > > > >
> > > > > > > > > And that weird stuff was reported by the node (pay attention to
> > > > the
> > > > > > > last
> > > > > > > > > line):
> > > > > > > > >
> > > > > > > > > [2018-08-16 15:45:50,211][INFO
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > ][grid-timeout-worker-#135%cluster_31-Dec-2017%][IgniteKernal%cluster_31-Dec-2017]
> > > > > > > > > Metrics for local node (to disable set 'metricsLogFrequency' to
> > > > 0)
> > > > > > > > >     ^-- Node [id=c033026e, name=cluster_31-Dec-2017,
> > > > > > > uptime=00:38:00.257]
> > > > > > > > >     ^-- H/N/C [hosts=1, nodes=1, CPUs=64]
> > > > > > > > >     ^-- CPU [cur=0.03%, avg=5.54%, GC=0%]
> > > > > > > > >     ^-- PageMemory [pages=6997377]
> > > > > > > > >     ^-- Heap [used=9706MB, free=61.18%, comm=22384MB]
> > > > > > > > >    * ^-- Non heap [used=144MB, free=-1%, comm=148MB] - this
> > > line
> > > > is
> > > > > > > > always
> > > > > > > > > the same!*
> > > > > > > > >
> > > > > > > > > Had to change the code by using
> > > > dataRegion.getPhysicalMemoryPages()
> > > > > > to
> > > > > > > > find
> > > > > > > > > out that actual off-heap usage size was
> > > > > > > > > >>> Physical Memory Size: 28651614208 => 27324 MB, 26 GB
> > > > > > > > >
> > > > > > > > > Let's fix this issue in 2.7, I proposed a new format. Please
> > > > review
> > > > > > and
> > > > > > > > > share your thoughts:
> > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-9305
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Denis
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > --
> > > > > --
> > > > > Maxim Muzafarov
> > > > >
> > > >
> > >

--
Best wishes,
Amelchev Nikita

[Dmitriy Pavlov]

Hi Igniters,

It seems the issue is a blocker of 2.7.

We need a volunteer here to run review.

Alexey P. will you step in?

Sincerely,
Dmitriy Pavlov

ср, 29 авг. 2018 г. в 16:16, Nikita Amelchev <[hidden email]>:

> Pavel,
>
> I think that point 1 is the correct way to calculate the committed
> size for a log. It is already calculated regardless of the
> metricsEnabled flag.
>
> In addition, I suggest more readable log format for data regions in
> issue comments [1].
>
> [1] https://issues.apache.org/jira/browse/IGNITE-9305
>
> вт, 21 авг. 2018 г. в 12:15, Pavel Pereslegin <[hidden email]>:
> >
> > Hello, Igniters.
> >
> > I assigned ticket [1] created by Denis and want to clarify how to log
> > committed size.
> > The metric offHeapSize (in DataRegionMetricsImpl) is always
> > calculated, but getOffHeapSize returns zero if memory metrics are
> > disabled for this data region.
> >
> > So I see the following options:
> > 1. Modify method getOffHeapSize so that it always returns actual value
> > offHeapSize.
> > 2. Add another offHeapSize() method.
> > 3. Output to log max size instead of committed (change "comm" to "max"
> > in log output).
> > 4. Don't bother about disabling metrics and output to log value
> > returned by getOffHeapSize.
> >
> > Any thoughts?
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-9305
> > сб, 18 авг. 2018 г. в 3:17, Denis Magda <[hidden email]>:
> > >
> > > Vova, the things are even simpler - we have this
> > >
> > > ignite.dataRegionMetrics().getPhysicalMemorySize() that returns the
> > > number equal/comparabel to pageNumber X pageSize.
> > >
> > >
> > > Igniters, if you believe that we need to do more work here then let's
> > > do it iteratively. Let's fix the off-heap occupied size the way above
> > > (just print out getPhysicalMemorySize() for every data region). Then
> > > do the rest. This needs to be fixed in 2.7.
> > >
> > >
> > > --
> > >
> > > Denis
> > >
> > >
> > > On Fri, Aug 17, 2018 at 10:20 AM Vladimir Ozerov <[hidden email]
> >
> > > wrote:
> > >
> > > > Folks,
> > > >
> > > > We already have this:
> > > > >>> PageMemory [pages=6997377]
> > > >
> > > > Then we can multiply it by page size and get occupied memory. Am I
> wrong?
> > > >
> > > > On Fri, Aug 17, 2018 at 12:56 PM Dmitriy Pavlov <
> [hidden email]>
> > > > wrote:
> > > >
> > > > > Hi Maxim,
> > > > >
> > > > > thank you for stepping in and for finding these issues. Yes, these
> > > > tickets
> > > > > are correct.
> > > > >
> > > > > I can move https://issues.apache.org/jira/browse/IGNITE-5583 to
> > > > unassigned
> > > > > if someone would like to implement this change. I will not have
> enough
> > > > time
> > > > > to complete it in 1 month (before 2.7 release).
> > > > >
> > > > > Sincerely,
> > > > > Dmitriy Pavlov
> > > > >
> > > > > пт, 17 авг. 2018 г. в 11:04, Maxim Muzafarov <[hidden email]>:
> > > > >
> > > > > > Igniters,
> > > > > >
> > > > > > Suppose, Dmitry is talking about IGNITE-5583 [1] - `Switch
> non-heap
> > > > > memory
> > > > > > metrics
> > > > > > to new page memory semantics` and related previous disscustions
> to it
> > > > > [4].
> > > > > >
> > > > > > Also we have some additional improvements to CacheMetrics:
> > > > > > IGNITE-5490 [2] - `Implement replacement for obsolete
> > > > > > CacheMetrics#getOffHeapAllocatedSize`
> > > > > > IGNITE-5765 [3] - `CacheMetrics interface cleanup, documentation
> and
> > > > > fixes`
> > > > > >
> > > > > >
> > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-5583
> > > > > > [2] https://issues.apache.org/jira/browse/IGNITE-5490
> > > > > > [3] https://issues.apache.org/jira/browse/IGNITE-5765
> > > > > > [4]
> > > > > >
> > > > > >
> > > > >
> > > >
> http://apache-ignite-developers.2346864.n4.nabble.com/Negative-non-heap-memory-maximum-td17990.html
> > > > > >
> > > > > > On Fri, 17 Aug 2018 at 10:14 Dmitriy Pavlov <
> [hidden email]>
> > > > > wrote:
> > > > > >
> > > > > > > Hi Igniters,
> > > > > > >
> > > > > > > It is not an easy fix, so I'm not sure it is possible to do in
> 2.7.
> > > > > > >
> > > > > > > Offheap size is not reported by VM (it returns -1). To
> implement it
> > > > we
> > > > > > need
> > > > > > > totally migrate off-heap memory metrics to durable memory data.
> > > > > > >
> > > > > > > I think this issue was reported and I'll find the duplicate.
> > > > > > >
> > > > > > > Sincerely,
> > > > > > > Dmitriy Pavlov
> > > > > > >
> > > > > > > пт, 17 авг. 2018 г. в 6:10, Denis Magda <[hidden email]>:
> > > > > > >
> > > > > > > > Yes, it was at the end of my wordy email :)
> > > > > > > > https://issues.apache.org/jira/browse/IGNITE-9305
> > > > > > > >
> > > > > > > > --
> > > > > > > > Denis
> > > > > > > >
> > > > > > > > On Thu, Aug 16, 2018 at 11:03 PM Dmitriy Setrakyan <
> > > > > > > [hidden email]>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Is there a blocker ticket for 2.7?
> > > > > > > > >
> > > > > > > > > On Thu, Aug 16, 2018, 19:59 Denis Magda <[hidden email]
> >
> > > > wrote:
> > > > > > > > >
> > > > > > > > > > Igniters,
> > > > > > > > > >
> > > > > > > > > > Was troubleshooting an Ignite deployment today and
> couldn't
> > > > find
> > > > > > out
> > > > > > > > from
> > > > > > > > > > the logs what was the actual off-heap space used.
> > > > > > > > > >
> > > > > > > > > > Those were the given memory resoures (Ignite 2.6):
> > > > > > > > > >
> > > > > > > > > > [2018-08-16 15:07:49,961][INFO
> ][main][GridDiscoveryManager]
> > > > > > Topology
> > > > > > > > > > snapshot [ver=1, servers=1, clients=0, CPUs=64,
> > > > *offheap=30.0GB*,
> > > > > > > > > > heap=24.0GB]
> > > > > > > > > >
> > > > > > > > > > And that weird stuff was reported by the node (pay
> attention to
> > > > > the
> > > > > > > > last
> > > > > > > > > > line):
> > > > > > > > > >
> > > > > > > > > > [2018-08-16 15:45:50,211][INFO
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> ][grid-timeout-worker-#135%cluster_31-Dec-2017%][IgniteKernal%cluster_31-Dec-2017]
> > > > > > > > > > Metrics for local node (to disable set
> 'metricsLogFrequency' to
> > > > > 0)
> > > > > > > > > >     ^-- Node [id=c033026e, name=cluster_31-Dec-2017,
> > > > > > > > uptime=00:38:00.257]
> > > > > > > > > >     ^-- H/N/C [hosts=1, nodes=1, CPUs=64]
> > > > > > > > > >     ^-- CPU [cur=0.03%, avg=5.54%, GC=0%]
> > > > > > > > > >     ^-- PageMemory [pages=6997377]
> > > > > > > > > >     ^-- Heap [used=9706MB, free=61.18%, comm=22384MB]
> > > > > > > > > >    * ^-- Non heap [used=144MB, free=-1%, comm=148MB] -
> this
> > > > line
> > > > > is
> > > > > > > > > always
> > > > > > > > > > the same!*
> > > > > > > > > >
> > > > > > > > > > Had to change the code by using
> > > > > dataRegion.getPhysicalMemoryPages()
> > > > > > > to
> > > > > > > > > find
> > > > > > > > > > out that actual off-heap usage size was
> > > > > > > > > > >>> Physical Memory Size: 28651614208 => 27324 MB, 26 GB
> > > > > > > > > >
> > > > > > > > > > Let's fix this issue in 2.7, I proposed a new format.
> Please
> > > > > review
> > > > > > > and
> > > > > > > > > > share your thoughts:
> > > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-9305
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > Denis
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > --
> > > > > > --
> > > > > > Maxim Muzafarov
> > > > > >
> > > > >
> > > >
>
>
>
> --
> Best wishes,
> Amelchev Nikita
>

[Alexey Plekhanov]

Hi Dmitriy,

Yes, I will finish review tomorrow.

ср, 12 сент. 2018 г. в 20:37, Dmitriy Pavlov <[hidden email]>:

> Hi Igniters,
>
> It seems the issue is a blocker of 2.7.
>
> We need a volunteer here to run review.
>
> Alexey P. will you step in?
>
> Sincerely,
> Dmitriy Pavlov
>
> ср, 29 авг. 2018 г. в 16:16, Nikita Amelchev <[hidden email]>:
>
> > Pavel,
> >
> > I think that point 1 is the correct way to calculate the committed
> > size for a log. It is already calculated regardless of the
> > metricsEnabled flag.
> >
> > In addition, I suggest more readable log format for data regions in
> > issue comments [1].
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-9305
> >
> > вт, 21 авг. 2018 г. в 12:15, Pavel Pereslegin <[hidden email]>:
> > >
> > > Hello, Igniters.
> > >
> > > I assigned ticket [1] created by Denis and want to clarify how to log
> > > committed size.
> > > The metric offHeapSize (in DataRegionMetricsImpl) is always
> > > calculated, but getOffHeapSize returns zero if memory metrics are
> > > disabled for this data region.
> > >
> > > So I see the following options:
> > > 1. Modify method getOffHeapSize so that it always returns actual value
> > > offHeapSize.
> > > 2. Add another offHeapSize() method.
> > > 3. Output to log max size instead of committed (change "comm" to "max"
> > > in log output).
> > > 4. Don't bother about disabling metrics and output to log value
> > > returned by getOffHeapSize.
> > >
> > > Any thoughts?
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-9305
> > > сб, 18 авг. 2018 г. в 3:17, Denis Magda <[hidden email]>:
> > > >
> > > > Vova, the things are even simpler - we have this
> > > >
> > > > ignite.dataRegionMetrics().getPhysicalMemorySize() that returns the
> > > > number equal/comparabel to pageNumber X pageSize.
> > > >
> > > >
> > > > Igniters, if you believe that we need to do more work here then let's
> > > > do it iteratively. Let's fix the off-heap occupied size the way above
> > > > (just print out getPhysicalMemorySize() for every data region). Then
> > > > do the rest. This needs to be fixed in 2.7.
> > > >
> > > >
> > > > --
> > > >
> > > > Denis
> > > >
> > > >
> > > > On Fri, Aug 17, 2018 at 10:20 AM Vladimir Ozerov <
> [hidden email]
> > >
> > > > wrote:
> > > >
> > > > > Folks,
> > > > >
> > > > > We already have this:
> > > > > >>> PageMemory [pages=6997377]
> > > > >
> > > > > Then we can multiply it by page size and get occupied memory. Am I
> > wrong?
> > > > >
> > > > > On Fri, Aug 17, 2018 at 12:56 PM Dmitriy Pavlov <
> > [hidden email]>
> > > > > wrote:
> > > > >
> > > > > > Hi Maxim,
> > > > > >
> > > > > > thank you for stepping in and for finding these issues. Yes,
> these
> > > > > tickets
> > > > > > are correct.
> > > > > >
> > > > > > I can move https://issues.apache.org/jira/browse/IGNITE-5583 to
> > > > > unassigned
> > > > > > if someone would like to implement this change. I will not have
> > enough
> > > > > time
> > > > > > to complete it in 1 month (before 2.7 release).
> > > > > >
> > > > > > Sincerely,
> > > > > > Dmitriy Pavlov
> > > > > >
> > > > > > пт, 17 авг. 2018 г. в 11:04, Maxim Muzafarov <[hidden email]
> >:
> > > > > >
> > > > > > > Igniters,
> > > > > > >
> > > > > > > Suppose, Dmitry is talking about IGNITE-5583 [1] - `Switch
> > non-heap
> > > > > > memory
> > > > > > > metrics
> > > > > > > to new page memory semantics` and related previous disscustions
> > to it
> > > > > > [4].
> > > > > > >
> > > > > > > Also we have some additional improvements to CacheMetrics:
> > > > > > > IGNITE-5490 [2] - `Implement replacement for obsolete
> > > > > > > CacheMetrics#getOffHeapAllocatedSize`
> > > > > > > IGNITE-5765 [3] - `CacheMetrics interface cleanup,
> documentation
> > and
> > > > > > fixes`
> > > > > > >
> > > > > > >
> > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-5583
> > > > > > > [2] https://issues.apache.org/jira/browse/IGNITE-5490
> > > > > > > [3] https://issues.apache.org/jira/browse/IGNITE-5765
> > > > > > > [4]
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/Negative-non-heap-memory-maximum-td17990.html
> > > > > > >
> > > > > > > On Fri, 17 Aug 2018 at 10:14 Dmitriy Pavlov <
> > [hidden email]>
> > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Igniters,
> > > > > > > >
> > > > > > > > It is not an easy fix, so I'm not sure it is possible to do
> in
> > 2.7.
> > > > > > > >
> > > > > > > > Offheap size is not reported by VM (it returns -1). To
> > implement it
> > > > > we
> > > > > > > need
> > > > > > > > totally migrate off-heap memory metrics to durable memory
> data.
> > > > > > > >
> > > > > > > > I think this issue was reported and I'll find the duplicate.
> > > > > > > >
> > > > > > > > Sincerely,
> > > > > > > > Dmitriy Pavlov
> > > > > > > >
> > > > > > > > пт, 17 авг. 2018 г. в 6:10, Denis Magda <[hidden email]>:
> > > > > > > >
> > > > > > > > > Yes, it was at the end of my wordy email :)
> > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-9305
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Denis
> > > > > > > > >
> > > > > > > > > On Thu, Aug 16, 2018 at 11:03 PM Dmitriy Setrakyan <
> > > > > > > > [hidden email]>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Is there a blocker ticket for 2.7?
> > > > > > > > > >
> > > > > > > > > > On Thu, Aug 16, 2018, 19:59 Denis Magda <
> [hidden email]
> > >
> > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Igniters,
> > > > > > > > > > >
> > > > > > > > > > > Was troubleshooting an Ignite deployment today and
> > couldn't
> > > > > find
> > > > > > > out
> > > > > > > > > from
> > > > > > > > > > > the logs what was the actual off-heap space used.
> > > > > > > > > > >
> > > > > > > > > > > Those were the given memory resoures (Ignite 2.6):
> > > > > > > > > > >
> > > > > > > > > > > [2018-08-16 15:07:49,961][INFO
> > ][main][GridDiscoveryManager]
> > > > > > > Topology
> > > > > > > > > > > snapshot [ver=1, servers=1, clients=0, CPUs=64,
> > > > > *offheap=30.0GB*,
> > > > > > > > > > > heap=24.0GB]
> > > > > > > > > > >
> > > > > > > > > > > And that weird stuff was reported by the node (pay
> > attention to
> > > > > > the
> > > > > > > > > last
> > > > > > > > > > > line):
> > > > > > > > > > >
> > > > > > > > > > > [2018-08-16 15:45:50,211][INFO
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> >
> ][grid-timeout-worker-#135%cluster_31-Dec-2017%][IgniteKernal%cluster_31-Dec-2017]
> > > > > > > > > > > Metrics for local node (to disable set
> > 'metricsLogFrequency' to
> > > > > > 0)
> > > > > > > > > > >     ^-- Node [id=c033026e, name=cluster_31-Dec-2017,
> > > > > > > > > uptime=00:38:00.257]
> > > > > > > > > > >     ^-- H/N/C [hosts=1, nodes=1, CPUs=64]
> > > > > > > > > > >     ^-- CPU [cur=0.03%, avg=5.54%, GC=0%]
> > > > > > > > > > >     ^-- PageMemory [pages=6997377]
> > > > > > > > > > >     ^-- Heap [used=9706MB, free=61.18%, comm=22384MB]
> > > > > > > > > > >    * ^-- Non heap [used=144MB, free=-1%, comm=148MB] -
> > this
> > > > > line
> > > > > > is
> > > > > > > > > > always
> > > > > > > > > > > the same!*
> > > > > > > > > > >
> > > > > > > > > > > Had to change the code by using
> > > > > > dataRegion.getPhysicalMemoryPages()
> > > > > > > > to
> > > > > > > > > > find
> > > > > > > > > > > out that actual off-heap usage size was
> > > > > > > > > > > >>> Physical Memory Size: 28651614208 => 27324 MB, 26
> GB
> > > > > > > > > > >
> > > > > > > > > > > Let's fix this issue in 2.7, I proposed a new format.
> > Please
> > > > > > review
> > > > > > > > and
> > > > > > > > > > > share your thoughts:
> > > > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-9305
> > > > > > > > > > >
> > > > > > > > > > > --
> > > > > > > > > > > Denis
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > --
> > > > > > > --
> > > > > > > Maxim Muzafarov
> > > > > > >
> > > > > >
> > > > >
> >
> >
> >
> > --
> > Best wishes,
> > Amelchev Nikita
> >
>

[aealexsandrov]

Hi Igniters,

Small notes according to these fix.

As I see that all logic of calculation off-neap max size at the moment
located in the ackNodeMetrics method in IgniteKernal.java. I think that it
isn't ok because this logic should be added to other functionality too. I
think that will be better to move this logic for example somewhere inside
ClusterMetrics and update existed metrics.

Also in case if you will push this change in current view then, for example,
the user could be confused because he will see different off-heap values in
the log and rest http://127.0.0.1:8080/ignite?cmd=node&mtr=true&ip=127.0.0.1

Log:

Metrics for local node (to disable set 'metricsLogFrequency' to 0)
    ^-- Node [id=*3229aa83*, name=ignite1, uptime=00:00:41.202]
...
    ^-- Off-heap [used=8MB, free=99,82%, comm=1580MB]
...

Rest:

"nodeId":"*3229aa83*-bfcb-45b3-af05-be33da50aa2f"
"nonHeapMemoryMaximum":-1
"nonHeapMemoryUsed":58390208
"nonHeapMemoryCommitted":59572224

I understand that REST could be implemented later but looks like at the
moment there is no way to get these metrics somehow except parsing the log
files.

I suggest creating the way to get it by the user.

BR,
Andrei





--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

[Pavel Pereslegin]

Hello Andrei,

All these metrics available through JMX (see "DataRegionMetrics" group) [1].

[1] https://apacheignite.readme.io/docs/memory-metrics
пн, 24 сент. 2018 г. в 17:58, aealexsandrov <[hidden email]>:

>
> Hi Igniters,
>
> Small notes according to these fix.
>
> As I see that all logic of calculation off-neap max size at the moment
> located in the ackNodeMetrics method in IgniteKernal.java. I think that it
> isn't ok because this logic should be added to other functionality too. I
> think that will be better to move this logic for example somewhere inside
> ClusterMetrics and update existed metrics.
>
> Also in case if you will push this change in current view then, for example,
> the user could be confused because he will see different off-heap values in
> the log and rest http://127.0.0.1:8080/ignite?cmd=node&mtr=true&ip=127.0.0.1
>
> Log:
>
> Metrics for local node (to disable set 'metricsLogFrequency' to 0)
>     ^-- Node [id=*3229aa83*, name=ignite1, uptime=00:00:41.202]
> ...
>     ^-- Off-heap [used=8MB, free=99,82%, comm=1580MB]
> ...
>
> Rest:
>
> "nodeId":"*3229aa83*-bfcb-45b3-af05-be33da50aa2f"
> "nonHeapMemoryMaximum":-1
> "nonHeapMemoryUsed":58390208
> "nonHeapMemoryCommitted":59572224
>
> I understand that REST could be implemented later but looks like at the
> moment there is no way to get these metrics somehow except parsing the log
> files.
>
> I suggest creating the way to get it by the user.
>
> BR,
> Andrei
>
>
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

[aealexsandrov]

Hi,

OK, the user can use it to calculate the off-heap. But I think that the
reason for your changes to fix the calculation of the nonHeap used in Ignite
now. For example now REST return "-1" for nonHeapMemoryMaximum. I think that
it can't be used somehow. So REST possible should be updated as you did for
log metrics and it will require for the same logic.

BR,
Andrei



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

[Pavel Pereslegin]

Andrei,
I totally agree with you and I think that "ClusterMetrics" should also
be fixed, I'm just not sure that we should include this change in the
same ticket.
пн, 24 сент. 2018 г. в 18:43, aealexsandrov <[hidden email]>:

>
> Hi,
>
> OK, the user can use it to calculate the off-heap. But I think that the
> reason for your changes to fix the calculation of the nonHeap used in Ignite
> now. For example now REST return "-1" for nonHeapMemoryMaximum. I think that
> it can't be used somehow. So REST possible should be updated as you did for
> log metrics and it will require for the same logic.
>
> BR,
> Andrei
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

[Dmitriy Pavlov]

Hi Igniters,

Thanks to everyone, who has participated in this discussion and shared
their view and ideas.

I've merged fix of changes related to logging only. Fixing of cluster
metrics can be done in a separate ticket/discussion.

Pavel, thank you for your contribution and for answering my questions.

Sincerely,
Dmitriy Pavlov

пн, 24 сент. 2018 г. в 18:52, Pavel Pereslegin <[hidden email]>:

> Andrei,
> I totally agree with you and I think that "ClusterMetrics" should also
> be fixed, I'm just not sure that we should include this change in the
> same ticket.
> пн, 24 сент. 2018 г. в 18:43, aealexsandrov <[hidden email]>:
> >
> > Hi,
> >
> > OK, the user can use it to calculate the off-heap. But I think that the
> > reason for your changes to fix the calculation of the nonHeap used in
> Ignite
> > now. For example now REST return "-1" for nonHeapMemoryMaximum. I think
> that
> > it can't be used somehow. So REST possible should be updated as you did
> for
> > log metrics and it will require for the same logic.
> >
> > BR,
> > Andrei
> >
> >
> >
> > --
> > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>

[dmagda]

Thanks, Pavel and the rest of the Igniters involved.

That simple usability improvement is a big deal for those who use Ignite in
production.

Are we getting it in 2.7?

--
Denis

On Wed, Sep 26, 2018 at 10:11 AM Dmitriy Pavlov <[hidden email]>
wrote:

> Hi Igniters,
>
> Thanks to everyone, who has participated in this discussion and shared
> their view and ideas.
>
> I've merged fix of changes related to logging only. Fixing of cluster
> metrics can be done in a separate ticket/discussion.
>
> Pavel, thank you for your contribution and for answering my questions.
>
> Sincerely,
> Dmitriy Pavlov
>
> пн, 24 сент. 2018 г. в 18:52, Pavel Pereslegin <[hidden email]>:
>
> > Andrei,
> > I totally agree with you and I think that "ClusterMetrics" should also
> > be fixed, I'm just not sure that we should include this change in the
> > same ticket.
> > пн, 24 сент. 2018 г. в 18:43, aealexsandrov <[hidden email]>:
> > >
> > > Hi,
> > >
> > > OK, the user can use it to calculate the off-heap. But I think that the
> > > reason for your changes to fix the calculation of the nonHeap used in
> > Ignite
> > > now. For example now REST return "-1" for nonHeapMemoryMaximum. I think
> > that
> > > it can't be used somehow. So REST possible should be updated as you did
> > for
> > > log metrics and it will require for the same logic.
> > >
> > > BR,
> > > Andrei
> > >
> > >
> > >
> > > --
> > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> >
>