Impossible memory region created in TC test
Locked
Impossible memory region created in TC test
 
|
Hello, Igniters.
Why test on TC can allocate Data Region bigger than the amount of available RAM? How can I disable this feature in a specific test? "memoryAllocated=1024.0 GiB ... required=1093413MB, available=96675MB" I have a test [1] that try to create Data Region of 1024 Gib size. I want to check OOM exception that should occur while this DataRegion starting. Locally, I have it. But, On TC that DataRegion successfully created [2] I see following log messages: ``` [2019-04-13 21:31:30,426][INFO ][exchange-worker-#91818%test-server%][PageMemoryImpl] Started page memory [memoryAllocated=1024.0 GiB, pages=260204016, tableSize=19.8 GiB, checkpointBuffer=2.0 GiB] .... [2019-04-13 21:31:18,004][WARN ][test-runner-#91737%pagemem.PageMemoryLazyAllocationWithPDSTest%][IgniteKernal%test-server] Nodes started on local machine require more than 80% of physical RAM what can lead to significant slowdown due to swapping (please decrease JVM heap size, data region size or checkpoint buffer size) [required=1093413MB, available=96675MB] ``` [1] https://github.com/apache/ignite/pull/6388/files#diff-754f5ea2f3579b81c50662f3d20c9126R138 [2] https://ci.ignite.apache.org/viewLog.html?buildId=3591587&buildTypeId=IgniteTests24Java8_RunAll#testNameId-7083016202406489951 |
signature.asc (499 bytes)
Re: Impossible memory region created in TC test
|
|
Re: Impossible memory region created in TC test
|
|
Hello, Ilya.
> Ignite does not touch every page of data region, and until you touch a page OS will not allocate any specific physical RAM to the virtual RAM address of that page. This is not true. Take a look into this discussion [1] > Moreover, AFAIK Ignite will not even allocate all the memory permitted by data region until it is needed May be I miss something, but 1. If persistenEnabled = false Ignite will allocate 1 segment on start [2] 2. If persistenEnabled = true Ignite will allocate all segments on start [3] > If you would use Pre-Touch feature which was suggested in this developer list a few months ago, you will see it fail explicitly. Locally, test already work as expected. But on TC it fails and DataRegion of 1024GiB can be created. Seems, there is some flag on TC that enable this behaviour. [1] http://apache-ignite-developers.2346864.n4.nabble.com/Data-regions-on-client-nodes-td32834.html [2] https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/pagemem/impl/PageMemoryNoStoreImpl.java#L245 [3] https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/pagemem/PageMemoryImpl.java#L386 В Пн, 15/04/2019 в 10:18 +0300, Ilya Kasnacheev пишет: > Hello! > > Ignite does not touch every page of data region, and until you touch a page > OS will not allocate any specific physical RAM to the virtual RAM address > of that page. > > Moreover, AFAIK Ignite will not even allocate all the memory permitted by > data region until it is needed. It will allocate memory in chunks, which > means your system will slow to grind trying to find RAM for the next chunk > as you try to load data into such node. > > If you would use Pre-Touch feature which was suggested in this developer > list a few months ago, you will see it fail explicitly. > > Regards, |
Re: Impossible memory region created in TC test
|
|
Hello, Igniters.
> Why test on TC can allocate Data Region bigger than the amount of available RAM? Seems, we are using `echo 1 > /proc/sys/vm/overcommit_memory` on TC. So I should fix my test :). В Пн, 15/04/2019 в 13:41 +0300, Nikolay Izhikov пишет: > Hello, Ilya. > > > Ignite does not touch every page of data region, and until you touch a page OS will not allocate any specific physical RAM to the virtual RAM address of that page. > > This is not true. > Take a look into this discussion [1] > > > Moreover, AFAIK Ignite will not even allocate all the memory permitted by data region until it is needed > > May be I miss something, but > > 1. If persistenEnabled = false Ignite will allocate 1 segment on start [2] > 2. If persistenEnabled = true Ignite will allocate all segments on start [3] > > > If you would use Pre-Touch feature which was suggested in this developer list a few months ago, you will see it fail explicitly. > > Locally, test already work as expected. > But on TC it fails and DataRegion of 1024GiB can be created. > > Seems, there is some flag on TC that enable this behaviour. > > [1] http://apache-ignite-developers.2346864.n4.nabble.com/Data-regions-on-client-nodes-td32834.html > [2] https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/pagemem/impl/PageMemoryNoStoreImpl.java#L245 > [3] https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/pagemem/PageMemoryImpl.java#L386 > > > В Пн, 15/04/2019 в 10:18 +0300, Ilya Kasnacheev пишет: > > Hello! > > > > Ignite does not touch every page of data region, and until you touch a page > > OS will not allocate any specific physical RAM to the virtual RAM address > > of that page. > > > > Moreover, AFAIK Ignite will not even allocate all the memory permitted by > > data region until it is needed. It will allocate memory in chunks, which > > means your system will slow to grind trying to find RAM for the next chunk > > as you try to load data into such node. > > > > If you would use Pre-Touch feature which was suggested in this developer > > list a few months ago, you will see it fail explicitly. > > > > Regards, |
| Free forum by Nabble | Edit this page |