[jira] [Created] (IGNITE-9969) Race on accessing to WAL segment on node startup.
Locked
[jira] [Created] (IGNITE-9969) Race on accessing to WAL segment on node startup.
 
|
Sergey Antonov created IGNITE-9969:
-------------------------------------- Summary: Race on accessing to WAL segment on node startup. Key: IGNITE-9969 URL: https://issues.apache.org/jira/browse/IGNITE-9969 Project: Ignite Issue Type: Bug Reporter: Sergey Antonov We could get race on WAL segment on node startup. In same time, one thread will be archive WAL segment and another thread will be apply last updates after last checkpoint from this WAL segment. {noformat} [2018-10-22 17:08:30,763][INFO ][wal-file-archiver%persistence.PersistenceBasicCompatibilityTest0-#152%persistence.PersistenceBasicCompatibilityTest0%][FileWriteAheadLogManager] Starting to copy WAL segment [absIdx=0, segIdx=0, origFile=C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal, dstFile=C:\IdeaProjects\apache-ignite\work\db\wal\archive\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal] [2018-10-22 17:08:30,800][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridCacheDatabaseSharedManager] Read checkpoint status [startMarker=null, endMarker=null] [2018-10-22 17:08:30,800][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridCacheDatabaseSharedManager] Applying lost cache updates since last checkpoint record [lastMarked=FileWALPointer [idx=0, fileOff=0, len=0], lastCheckpointId=00000000-0000-0000-0000-000000000000] [2018-10-22 17:08:30,807][ERROR][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridDhtPartitionsExchangeFuture] Failed to reinitialize local partitions (rebalancing will be stopped): GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=3f392b21-9ce6-4ce5-811e-c66796800000, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1540217301530, loc=true, ver=2.7.0#19700101-sha1:00000000, isClient=false], topVer=1, nodeId8=3f392b21, msg=null, type=DISCOVERY_CUSTOM_EVT, tstamp=1540217310618]DiscoveryCustomEvent [customMsg=ChangeGlobalStateMessage [id=4bc2a1c9661-142c551e-2962-403a-b864-0281692574ee, reqId=4ca70fa9-7a54-411a-8ad5-f4633e40edd4, initiatingNodeId=3f392b21-9ce6-4ce5-811e-c66796800000, activate=true, baselineTopology=BaselineTopology [id=0, branchingHash=-416176595, branchingType='New BaselineTopology', baselineNodes=[81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332]], forceChangeBaselineTopology=false, timestamp=1540217310624], affTopVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], super=], nodeId=3f392b21, evt=DISCOVERY_CUSTOM_EVT] class org.apache.ignite.IgniteCheckedException: Failed to initialize WAL segment: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.initReadHandle(AbstractWalRecordsIterator.java:412) at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.initReadHandle(FileWriteAheadLogManager.java:3113) at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.advanceSegment(FileWriteAheadLogManager.java:3195) at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.advance(AbstractWalRecordsIterator.java:163) at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.<init>(FileWriteAheadLogManager.java:3087) at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.<init>(FileWriteAheadLogManager.java:3003) at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.replay(FileWriteAheadLogManager.java:925) at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.applyLastUpdates(GridCacheDatabaseSharedManager.java:2289) at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreState(GridCacheDatabaseSharedManager.java:1632) at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.beforeExchange(GridCacheDatabaseSharedManager.java:1298) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.distributedExchange(GridDhtPartitionsExchangeFuture.java:1455) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:808) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:2746) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2618) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) [2018-10-22 17:08:30,926][INFO ][wal-file-archiver%persistence.PersistenceBasicCompatibilityTest0-#152%persistence.PersistenceBasicCompatibilityTest0%][FileWriteAheadLogManager] Copied file [src=C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal, dst=C:\IdeaProjects\apache-ignite\work\db\wal\archive\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal]{noformat} h4. How to reproduce: Run IgniteCompatibilityBasicTestSuite and test PersistenceBasicCompatibilityTest#testNodeStartByOldVersionPersistenceData_2_3() could fail with following exception: {noformat} [2018-10-22 17:08:30,819][ERROR][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridCachePartitionExchangeManager] Failed to wait for completion of partition map exchange (preloading will not start): GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=3f392b21-9ce6-4ce5-811e-c66796800000, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1540217301530, loc=true, ver=2.7.0#19700101-sha1:00000000, isClient=false], topVer=1, nodeId8=3f392b21, msg=null, type=DISCOVERY_CUSTOM_EVT, tstamp=1540217310618]DiscoveryCustomEvent [customMsg=null, affTopVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], super=], crd=TcpDiscoveryNode [id=3f392b21-9ce6-4ce5-811e-c66796800000, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1540217301530, loc=true, ver=2.7.0#19700101-sha1:00000000, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=3f392b21-9ce6-4ce5-811e-c66796800000, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1540217301530, loc=true, ver=2.7.0#19700101-sha1:00000000, isClient=false], topVer=1, nodeId8=3f392b21, msg=null, type=DISCOVERY_CUSTOM_EVT, tstamp=1540217310618]DiscoveryCustomEvent [customMsg=null, affTopVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], super=], nodeId=3f392b21, evt=DISCOVERY_CUSTOM_EVT], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=false, hash=364933128], init=false, lastVer=null, partReleaseFut=AffinityTopologyVersion [topVer=1, minorTopVer=1]AffinityTopologyVersion [topVer=1, minorTopVer=1]AffinityTopologyVersion [topVer=1, minorTopVer=1]AffinityTopologyVersion [topVer=1, minorTopVer=1]AffinityTopologyVersion [topVer=1, minorTopVer=1]AffinityTopologyVersion [topVer=1, minorTopVer=1]AffinityTopologyVersion [topVer=1, minorTopVer=1]PartitionReleaseFuture [topVer=, futures=[ExplicitLockReleaseFuture [topVer=, futures=[]], AtomicUpdateReleaseFuture [topVer=, futures=[]], DataStreamerReleaseFuture [topVer=, futures=[]], LocalTxReleaseFuture [topVer=, futures=[]], AllTxReleaseFuture [topVer=, futures=[RemoteTxReleaseFuture [topVer=, futures=[]]]]]], exchActions=null, affChangeMsg=null, initTs=1540217310618, centralizedAff=false, forceAffReassignment=true, exchangeLocE=null, cacheChangeFailureMsgSent=false, done=true, state=CRD, registerCachesFuture=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=null, hash=1024540699], partitionsSent=false, partitionsReceived=false, delayedLatestMsg=null, afterLsnrCompleteFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=null, hash=1881071890], evtLatch=0, remaining=HashSet [], super=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=class o.a.i.IgniteCheckedException: Failed to initialize WAL segment: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal, hash=1188696354]] class org.apache.ignite.IgniteCheckedException: Failed to initialize WAL segment: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.initReadHandle(AbstractWalRecordsIterator.java:412) at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.initReadHandle(FileWriteAheadLogManager.java:3113) at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.advanceSegment(FileWriteAheadLogManager.java:3195) at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.advance(AbstractWalRecordsIterator.java:163) at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.<init>(FileWriteAheadLogManager.java:3087) at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.<init>(FileWriteAheadLogManager.java:3003) at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.replay(FileWriteAheadLogManager.java:925) at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.applyLastUpdates(GridCacheDatabaseSharedManager.java:2289) at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreState(GridCacheDatabaseSharedManager.java:1632) at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.beforeExchange(GridCacheDatabaseSharedManager.java:1298) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.distributedExchange(GridDhtPartitionsExchangeFuture.java:1455) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:808) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:2746) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2618) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Caused by: java.nio.file.FileSystemException: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal: The process cannot access the file because it is being used by another process. at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86) at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97) at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102) at sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:115) at java.nio.channels.FileChannel.open(FileChannel.java:287) at java.nio.channels.FileChannel.open(FileChannel.java:335) at org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIO.<init>(RandomAccessFileIO.java:43) at org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIOFactory.create(RandomAccessFileIOFactory.java:42) at org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIOFactory.create(RandomAccessFileIOFactory.java:37) at org.apache.ignite.internal.processors.cache.persistence.wal.FileDescriptor.toIO(FileDescriptor.java:142) at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.initReadHandle(AbstractWalRecordsIterator.java:378) ... 15 more{noformat} Below is a full log of test: {noformat} [2018-10-22 17:08:08,194][INFO ][main][root] >>> Starting test: PersistenceBasicCompatibilityTest#testNodeStartByOldVersionPersistenceData_2_3 <<< [2018-10-22 17:08:08,582][INFO ][Thread-16][jvm-fcbe9773#2_3_0] Starting Ignite Node... Args=[C:\Users\santonov\AppData\Local\Temp\\igniteCompatibilityClosure.tmp_507120272, persistence.PersistenceBasicCompatibilityTest1, fcbe9773-6b4b-495d-bd36-f0fe336ff54e, fcbe9773-6b4b-495d-bd36-f0fe336ff54e, C:\Users\santonov\AppData\Local\Temp\\igniteCompatibilityClosure.tmp_348283360] [2018-10-22 17:08:09,079][INFO ][Thread-16][jvm-fcbe9773#2_3_0] Configured log4j from: C:\IdeaProjects\apache-ignite\modules\core\src\test\config\log4j-test.xml [2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,473][INFO ][main][PersistenceBasicCompatibilityTest1] [2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> __________ ________________ [2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> / _/ ___/ |/ / _/_ __/ __/ [2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> _/ // (7 7 // / / / / _/ [2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> /___/\___/_/|_/___/ /_/ /___/ [2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> [2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> ver. 2.3.0#20171028-sha1:8add7fd5 [2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> 2017 Copyright(C) Apache Software Foundation [2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> [2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> Ignite documentation: http://ignite.apache.org [2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,474][INFO ][main][PersistenceBasicCompatibilityTest1] Config URL: n/a [2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,474][INFO ][main][PersistenceBasicCompatibilityTest1] Daemon mode: off [2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,474][INFO ][main][PersistenceBasicCompatibilityTest1] OS: Windows 10 10.0 amd64 [2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,474][INFO ][main][PersistenceBasicCompatibilityTest1] OS user: santonov [2018-10-22 17:08:09,475][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,474][INFO ][main][PersistenceBasicCompatibilityTest1] PID: 15928 [2018-10-22 17:08:09,475][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,475][INFO ][main][PersistenceBasicCompatibilityTest1] Language runtime: Java Platform API Specification ver. 1.8 [2018-10-22 17:08:09,475][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,475][INFO ][main][PersistenceBasicCompatibilityTest1] VM information: Java(TM) SE Runtime Environment 1.8.0_181-b13 Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 25.181-b13 [2018-10-22 17:08:09,476][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,476][INFO ][main][PersistenceBasicCompatibilityTest1] VM total memory: 3.5GB [2018-10-22 17:08:09,476][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,476][INFO ][main][PersistenceBasicCompatibilityTest1] Remote Management [restart: off, REST: off, JMX (remote: off)] [2018-10-22 17:08:09,478][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,478][INFO ][main][PersistenceBasicCompatibilityTest1] IGNITE_HOME=C:\IdeaProjects\apache-ignite [2018-10-22 17:08:09,479][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,479][INFO ][main][PersistenceBasicCompatibilityTest1] VM arguments: [-ea] [2018-10-22 17:08:09,479][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,479][INFO ][main][PersistenceBasicCompatibilityTest1] System cache's DataRegion size is configured to 40 MB. Use DataStorageConfiguration.systemCacheMemorySize property to change the setting. [2018-10-22 17:08:09,487][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,487][INFO ][main][PersistenceBasicCompatibilityTest1] Configured caches [in 'sysMemPlc' dataRegion: ['ignite-sys-cache']] [2018-10-22 17:08:09,490][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,490][WARN ][pub-#19%persistence.PersistenceBasicCompatibilityTest1%][GridDiagnostic] This operating system has been tested less rigorously: Windows 10 10.0 amd64. Our team will appreciate the feedback if you experience any problems running ignite in this environment. [2018-10-22 17:08:09,491][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,491][INFO ][main][PersistenceBasicCompatibilityTest1] 3-rd party licenses can be found at: C:\IdeaProjects\apache-ignite\libs\licenses [2018-10-22 17:08:09,493][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,493][WARN ][pub-#22%persistence.PersistenceBasicCompatibilityTest1%][GridDiagnostic] Initial heap size is 254MB (should be no less than 512MB, use -Xms512m -Xmx512m). [2018-10-22 17:08:09,509][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,509][INFO ][main][IgnitePluginProcessor] Configured plugins: [2018-10-22 17:08:09,509][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,509][INFO ][main][IgnitePluginProcessor] ^-- TestCompatibilityPlugin 1.0 [2018-10-22 17:08:09,509][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,509][INFO ][main][IgnitePluginProcessor] ^-- [2018-10-22 17:08:09,510][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,509][INFO ][main][IgnitePluginProcessor] [2018-10-22 17:08:09,510][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,510][INFO ][main][IgnitePluginProcessor] ^-- TestReconnectPlugin 1.0 [2018-10-22 17:08:09,510][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,510][INFO ][main][IgnitePluginProcessor] ^-- [2018-10-22 17:08:09,510][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,510][INFO ][main][IgnitePluginProcessor] [2018-10-22 17:08:09,510][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,510][INFO ][main][IgnitePluginProcessor] ^-- StanByClusterTestProvider 1.0 [2018-10-22 17:08:09,510][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,510][INFO ][main][IgnitePluginProcessor] ^-- null [2018-10-22 17:08:09,510][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,510][INFO ][main][IgnitePluginProcessor] [2018-10-22 17:08:09,614][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,614][INFO ][main][TcpCommunicationSpi] Successfully bound communication NIO server to TCP port [port=45010, locHost=/127.0.0.1, selectorsCnt=4, selectorSpins=0, pairedConn=false] [2018-10-22 17:08:09,615][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,615][WARN ][main][TcpCommunicationSpi] Message queue limit is set to 0 which may lead to potential OOMEs when running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due to message queues growth on sender and receiver sides. [2018-10-22 17:08:09,748][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,747][WARN ][main][GridCollisionManager] Collision resolution is disabled (all jobs will be activated upon arrival). [2018-10-22 17:08:09,749][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,749][INFO ][main][PersistenceBasicCompatibilityTest1] Security status [authentication=off, tls/ssl=off] [2018-10-22 17:08:09,789][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,789][INFO ][main][TcpDiscoverySpi] Successfully bound to TCP port [port=47500, localHost=127.0.0.1/127.0.0.1, locNodeId=fcbe9773-6b4b-495d-bd36-f0fe336ff54e] [2018-10-22 17:08:09,808][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,808][INFO ][main][PdsFoldersResolver] Successfully created new persistent storage folder [C:\IdeaProjects\apache-ignite\work\db\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332] [2018-10-22 17:08:09,809][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,809][INFO ][main][PdsFoldersResolver] Consistent ID used for local node is [81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332] according to persistence data storage folders [2018-10-22 17:08:09,810][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,810][INFO ][main][CacheObjectBinaryProcessorImpl] Resolved directory for serialized binary metadata: C:\IdeaProjects\apache-ignite\work\binary_meta\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332 [2018-10-22 17:08:09,932][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,931][INFO ][main][FilePageStoreManager] Resolved page store work directory: C:\IdeaProjects\apache-ignite\work\db\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332 [2018-10-22 17:08:09,932][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,932][INFO ][main][FileWriteAheadLogManager] Resolved write ahead log work directory: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332 [2018-10-22 17:08:09,933][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,933][INFO ][main][FileWriteAheadLogManager] Resolved write ahead log archive directory: C:\IdeaProjects\apache-ignite\work\db\wal\archive\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332 [2018-10-22 17:08:10,064][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:10,063][INFO ][main][FileWriteAheadLogManager] Started write-ahead log manager [mode=DEFAULT] [2018-10-22 17:08:10,146][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:10,146][INFO ][main][ClientListenerProcessor] Client connector processor has started on TCP port 10800 [2018-10-22 17:08:10,559][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:10,559][INFO ][main][PersistenceBasicCompatibilityTest1] Non-loopback local IPs: 10.0.0.100, fe80:0:0:0:41c9:b4fa:d846:61ac%eth5, fe80:0:0:0:5422:f228:82d7:f577%eth0, fe80:0:0:0:9c7:33de:6920:739e%wlan2, fe80:0:0:0:a99b:6b5:1b8c:71d6%wlan0, fe80:0:0:0:c1f7:1e90:f607:de9d%wlan1, fe80:0:0:0:cd36:5279:6ba0:90aa%eth2, fe80:0:0:0:f0d5:95b5:b835:21dc%eth1 [2018-10-22 17:08:10,559][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:10,559][INFO ][main][PersistenceBasicCompatibilityTest1] Enabled local MACs: 00A0C6000000, 30243242E663, 30243242E664, 30243242E667, 32243242E663, 482AE30AD0AE, E04F439828DF [2018-10-22 17:08:19,706][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,706][INFO ][main][PersistenceBasicCompatibilityTest1] Performance suggestions for grid 'persistence.PersistenceBasicCompatibilityTest1' (fix if possible) [2018-10-22 17:08:19,706][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,706][INFO ][main][PersistenceBasicCompatibilityTest1] To disable, set -DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true [2018-10-22 17:08:19,706][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,706][INFO ][main][PersistenceBasicCompatibilityTest1] ^-- Disable checkpoints (remove 'checkpointSpi' from configuration) [2018-10-22 17:08:19,706][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,706][INFO ][main][PersistenceBasicCompatibilityTest1] ^-- Disable grid events (remove 'includeEventTypes' from configuration) [2018-10-22 17:08:19,706][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,706][INFO ][main][PersistenceBasicCompatibilityTest1] ^-- Enable G1 Garbage Collector (add '-XX:+UseG1GC' to JVM options) [2018-10-22 17:08:19,706][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,706][INFO ][main][PersistenceBasicCompatibilityTest1] ^-- Specify JVM heap max size (add '-Xmx<size>[g|G|m|M|k|K]' to JVM options) [2018-10-22 17:08:19,706][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,706][INFO ][main][PersistenceBasicCompatibilityTest1] ^-- Set max direct memory size if getting 'OOME: Direct buffer memory' (add '-XX:MaxDirectMemorySize=<size>[g|G|m|M|k|K]' to JVM options) [2018-10-22 17:08:19,706][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,706][INFO ][main][PersistenceBasicCompatibilityTest1] ^-- Disable processing of calls to System.gc() (add '-XX:+DisableExplicitGC' to JVM options) [2018-10-22 17:08:19,706][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,706][INFO ][main][PersistenceBasicCompatibilityTest1] ^-- Disable assertions (remove '-ea' from JVM options) [2018-10-22 17:08:19,706][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,706][INFO ][main][PersistenceBasicCompatibilityTest1] Refer to this page for more performance suggestions: https://apacheignite.readme.io/docs/jvm-and-system-tuning [2018-10-22 17:08:19,706][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,706][INFO ][main][PersistenceBasicCompatibilityTest1] [2018-10-22 17:08:19,707][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,706][INFO ][main][PersistenceBasicCompatibilityTest1] To start Console Management & Monitoring run ignitevisorcmd.{sh|bat} [2018-10-22 17:08:19,707][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,707][INFO ][main][PersistenceBasicCompatibilityTest1] [2018-10-22 17:08:19,708][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,708][INFO ][main][PersistenceBasicCompatibilityTest1] [2018-10-22 17:08:19,708][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,708][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> +----------------------------------------------------------------------+ [2018-10-22 17:08:19,708][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> Ignite ver. 2.3.0#20171028-sha1:8add7fd5b501b40658096cdde48af9e948aa8150 [2018-10-22 17:08:19,708][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> +----------------------------------------------------------------------+ [2018-10-22 17:08:19,708][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> OS name: Windows 10 10.0 amd64 [2018-10-22 17:08:19,708][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> CPU(s): 8 [2018-10-22 17:08:19,708][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> Heap: 3.5GB [2018-10-22 17:08:19,708][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> VM name: 15928@LAPTOP-IGKOMEL9 [2018-10-22 17:08:19,708][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> Ignite instance name: persistence.PersistenceBasicCompatibilityTest1 [2018-10-22 17:08:19,708][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> Local node [ID=FCBE9773-6B4B-495D-BD36-F0FE336FF54E, order=1, clientMode=false] [2018-10-22 17:08:19,708][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> Local node addresses: [127.0.0.1] [2018-10-22 17:08:19,708][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> Local ports: TCP:10800 TCP:45010 TCP:47500 [2018-10-22 17:08:19,708][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,709][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,709][INFO ][main][GridDiscoveryManager] Topology snapshot [ver=1, servers=1, clients=0, CPUs=8, heap=3.5GB] [2018-10-22 17:08:19,721][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,720][INFO ][tcp-disco-msg-worker-#4%persistence.PersistenceBasicCompatibilityTest1%][GridClusterStateProcessor] Started state transition: true [2018-10-22 17:08:19,733][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,733][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][time] Started exchange init [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], crd=true, evt=DISCOVERY_CUSTOM_EVT, evtNode=fcbe9773-6b4b-495d-bd36-f0fe336ff54e, customEvt=ChangeGlobalStateMessage [id=feaba1c9661-d034c9b8-c91f-4b97-91e2-99320879a2f4, reqId=6578bb8f-b31c-45a0-955c-5360ae2589d6, initiatingNodeId=fcbe9773-6b4b-495d-bd36-f0fe336ff54e, activate=true], allowMerge=false] [2018-10-22 17:08:19,733][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,733][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridDhtPartitionsExchangeFuture] Start activation process [nodeId=fcbe9773-6b4b-495d-bd36-f0fe336ff54e, client=false, topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1]] [2018-10-22 17:08:19,733][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,733][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][FilePageStoreManager] Resolved page store work directory: C:\IdeaProjects\apache-ignite\work\db\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332 [2018-10-22 17:08:19,734][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,733][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][FileWriteAheadLogManager] Resolved write ahead log work directory: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332 [2018-10-22 17:08:19,734][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,734][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][FileWriteAheadLogManager] Resolved write ahead log archive directory: C:\IdeaProjects\apache-ignite\work\db\wal\archive\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332 [2018-10-22 17:08:19,735][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,735][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][FileWriteAheadLogManager] Started write-ahead log manager [mode=DEFAULT] [2018-10-22 17:08:19,794][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,793][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][PageMemoryImpl] Started page memory [memoryAllocated=3.2 GiB, pages=808272, tableSize=46.9 MiB, checkpointBuffer=810.3 MiB] [2018-10-22 17:08:19,818][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,818][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridCacheDatabaseSharedManager] Read checkpoint status [startMarker=null, endMarker=null] [2018-10-22 17:08:19,819][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,818][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridCacheDatabaseSharedManager] Checking memory state [lastValidPos=FileWALPointer [idx=0, fileOffset=0, len=0, forceFlush=false], lastMarked=FileWALPointer [idx=0, fileOffset=0, len=0, forceFlush=false], lastCheckpointId=00000000-0000-0000-0000-000000000000] [2018-10-22 17:08:19,829][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,829][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][FileWriteAheadLogManager] Resuming logging to WAL segment [file=C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal, offset=0, ver=1] [2018-10-22 17:08:19,963][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,963][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridCacheProcessor] Started cache [name=ignite-sys-cache, memoryPolicyName=sysMemPlc, mode=REPLICATED, atomicity=TRANSACTIONAL] [2018-10-22 17:08:19,966][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,966][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridDhtPartitionsExchangeFuture] Successfully activated caches [nodeId=fcbe9773-6b4b-495d-bd36-f0fe336ff54e, client=false, topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1]] [2018-10-22 17:08:19,971][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,971][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridDhtPartitionsExchangeFuture] Finished waiting for partition release future [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], waitTime=0ms, futInfo=NA] [2018-10-22 17:08:20,018][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,018][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridCacheDatabaseSharedManager] Read checkpoint status [startMarker=null, endMarker=null] [2018-10-22 17:08:20,018][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,018][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridCacheDatabaseSharedManager] Applying lost cache updates since last checkpoint record [lastMarked=FileWALPointer [idx=0, fileOffset=0, len=0, forceFlush=false], lastCheckpointId=00000000-0000-0000-0000-000000000000] [2018-10-22 17:08:20,086][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,085][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridCacheDatabaseSharedManager] Finished applying WAL changes [updatesApplied=0, time=73ms] [2018-10-22 17:08:20,088][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,088][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridDhtPartitionsExchangeFuture] finishExchangeOnCoordinator [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], resVer=AffinityTopologyVersion [topVer=1, minorTopVer=1]] [2018-10-22 17:08:20,103][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,103][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridDhtPartitionsExchangeFuture] Finish exchange future [startVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], resVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], err=null] [2018-10-22 17:08:20,103][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,103][INFO ][tcp-disco-msg-worker-#4%persistence.PersistenceBasicCompatibilityTest1%][GridClusterStateProcessor] Received state change finish message: true [2018-10-22 17:08:20,113][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,112][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][time] Finished exchange init [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], crd=true] [2018-10-22 17:08:20,117][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,117][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridCachePartitionExchangeManager] Skipping rebalancing (nothing scheduled) [top=AffinityTopologyVersion [topVer=1, minorTopVer=1], evt=DISCOVERY_CUSTOM_EVT, node=fcbe9773-6b4b-495d-bd36-f0fe336ff54e] [2018-10-22 17:08:20,138][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,138][INFO ][sys-#39%persistence.PersistenceBasicCompatibilityTest1%][GridClusterStateProcessor] Successfully performed final activation steps [nodeId=fcbe9773-6b4b-495d-bd36-f0fe336ff54e, client=false, topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1]] [2018-10-22 17:08:20,156][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,156][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][time] Started exchange init [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=2], crd=true, evt=DISCOVERY_CUSTOM_EVT, evtNode=fcbe9773-6b4b-495d-bd36-f0fe336ff54e, customEvt=DynamicCacheChangeBatch [id=9faba1c9661-d034c9b8-c91f-4b97-91e2-99320879a2f4, reqs=[DynamicCacheChangeRequest [cacheName=PersistenceBasicCompatibilityTest, hasCfg=true, nodeId=fcbe9773-6b4b-495d-bd36-f0fe336ff54e, clientStartOnly=false, stop=false, destroy=false]], exchangeActions=ExchangeActions [startCaches=[PersistenceBasicCompatibilityTest], stopCaches=null, startGrps=[PersistenceBasicCompatibilityTest], stopGrps=[], resetParts=null, stateChangeRequest=null], startCaches=false], allowMerge=false] [2018-10-22 17:08:20,242][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,242][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridCacheProcessor] Started cache [name=PersistenceBasicCompatibilityTest, memoryPolicyName=default, mode=PARTITIONED, atomicity=TRANSACTIONAL] [2018-10-22 17:08:20,253][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,252][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridDhtPartitionsExchangeFuture] Finished waiting for partition release future [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=2], waitTime=0ms, futInfo=NA] [2018-10-22 17:08:20,316][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,316][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridDhtPartitionsExchangeFuture] finishExchangeOnCoordinator [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=2], resVer=AffinityTopologyVersion [topVer=1, minorTopVer=2]] [2018-10-22 17:08:20,321][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,321][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridDhtPartitionsExchangeFuture] Finish exchange future [startVer=AffinityTopologyVersion [topVer=1, minorTopVer=2], resVer=AffinityTopologyVersion [topVer=1, minorTopVer=2], err=null] [2018-10-22 17:08:20,322][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,322][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][time] Finished exchange init [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=2], crd=true] [2018-10-22 17:08:20,328][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,328][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridCachePartitionExchangeManager] Skipping rebalancing (nothing scheduled) [top=AffinityTopologyVersion [topVer=1, minorTopVer=2], evt=DISCOVERY_CUSTOM_EVT, node=fcbe9773-6b4b-495d-bd36-f0fe336ff54e] [2018-10-22 17:08:20,633][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [Compatibility] Node has been started, id=fcbe9773-6b4b-495d-bd36-f0fe336ff54e [2018-10-22 17:08:20,815][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] >>> __________ ________________ >>> / _/ ___/ |/ / _/_ __/ __/ >>> _/ // (7 7 // / / / / _/ >>> /___/\___/_/|_/___/ /_/ /___/ >>> >>> ver. 2.7.0-SNAPSHOT#19700101-sha1:DEV >>> 2018 Copyright(C) Apache Software Foundation >>> >>> Ignite documentation: http://ignite.apache.org [2018-10-22 17:08:20,815][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] Config URL: n/a [2018-10-22 17:08:20,817][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] IgniteConfiguration [igniteInstanceName=persistence.PersistenceBasicCompatibilityTest0, pubPoolSize=8, svcPoolSize=8, callbackPoolSize=8, stripedPoolSize=8, sysPoolSize=8, mgmtPoolSize=4, igfsPoolSize=8, dataStreamerPoolSize=8, utilityCachePoolSize=8, utilityCacheKeepAliveTime=60000, p2pPoolSize=2, qryPoolSize=8, igniteHome=C:\IdeaProjects\apache-ignite, igniteWorkDir=C:\IdeaProjects\apache-ignite\work, mbeanSrv=com.sun.jmx.mbeanserver.JmxMBeanServer@38cccef, nodeId=3f392b21-9ce6-4ce5-811e-c66796800000, marsh=BinaryMarshaller [], marshLocJobs=false, daemon=false, p2pEnabled=false, netTimeout=10000, sndRetryDelay=1000, sndRetryCnt=3, metricsHistSize=10000, metricsUpdateFreq=1000, metricsExpTime=9223372036854775807, discoSpi=TcpDiscoverySpi [addrRslvr=null, sockTimeout=0, ackTimeout=0, marsh=null, reconCnt=10, reconDelay=2000, maxAckTimeout=600000, forceSrvMode=false, clientReconnectDisabled=false, internalLsnr=null, skipAddrsRandomization=false], segPlc=STOP, segResolveAttempts=2, waitForSegOnStart=true, allResolversPassReq=true, segChkFreq=10000, commSpi=TcpCommunicationSpi [connectGate=null, connPlc=org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$FirstConnectionPolicy@2eceb355, enableForcibleNodeKill=false, enableTroubleshootingLog=false, locAddr=null, locHost=null, locPort=45010, locPortRange=100, shmemPort=-1, directBuf=true, directSndBuf=false, idleConnTimeout=600000, connTimeout=5000, maxConnTimeout=600000, reconCnt=10, sockSndBuf=32768, sockRcvBuf=32768, msgQueueLimit=0, slowClientQueueLimit=0, nioSrvr=null, shmemSrv=null, usePairedConnections=false, connectionsPerNode=1, tcpNoDelay=true, filterReachableAddresses=false, ackSndThreshold=32, unackedMsgsBufSize=0, sockWriteTimeout=2000, boundTcpPort=-1, boundTcpShmemPort=-1, selectorsCnt=4, selectorSpins=0, addrRslvr=null, ctxInitLatch=java.util.concurrent.CountDownLatch@112888fe[Count = 1], stopping=false], evtSpi=MemoryEventStorageSpi [expireAgeMs=9223372036854775807, expireCnt=10000, filter=null], colSpi=NoopCollisionSpi [], deploySpi=LocalDeploymentSpi [], indexingSpi=org.apache.ignite.spi.indexing.noop.NoopIndexingSpi@6cd8ff3f, addrRslvr=null, encryptionSpi=org.apache.ignite.spi.encryption.noop.NoopEncryptionSpi@7ebddfb2, clientMode=false, rebalanceThreadPoolSize=1, txCfg=TransactionConfiguration [txSerEnabled=false, dfltIsolation=REPEATABLE_READ, dfltConcurrency=PESSIMISTIC, dfltTxTimeout=0, txTimeoutOnPartitionMapExchange=0, pessimisticTxLogSize=0, pessimisticTxLogLinger=10000, tmLookupClsName=null, txManagerFactory=null, useJtaSync=false], cacheSanityCheckEnabled=true, discoStartupDelay=60000, deployMode=SHARED, p2pMissedCacheSize=100, locHost=127.0.0.1, timeSrvPortBase=31100, timeSrvPortRange=100, failureDetectionTimeout=10000, sysWorkerBlockedTimeout=null, clientFailureDetectionTimeout=10000, metricsLogFreq=0, hadoopCfg=null, connectorCfg=null, odbcCfg=null, warmupClos=null, atomicCfg=AtomicConfiguration [seqReserveSize=1000, cacheMode=PARTITIONED, backups=1, aff=null, grpName=null], classLdr=null, sslCtxFactory=null, platformCfg=null, binaryCfg=BinaryConfiguration [idMapper=null, nameMapper=null, serializer=null, compactFooter=true], memCfg=null, pstCfg=null, dsCfg=DataStorageConfiguration [sysRegionInitSize=41943040, sysRegionMaxSize=104857600, pageSize=0, concLvl=0, dfltDataRegConf=DataRegionConfiguration [name=default, maxSize=268435456, initSize=268435456, swapPath=null, pageEvictionMode=DISABLED, evictionThreshold=0.9, emptyPagesPoolSize=100, metricsEnabled=false, metricsSubIntervalCount=5, metricsRateTimeInterval=60000, persistenceEnabled=true, checkpointPageBufSize=0], dataRegions=null, storagePath=null, checkpointFreq=180000, lockWaitTime=10000, checkpointThreads=4, checkpointWriteOrder=SEQUENTIAL, walHistSize=20, maxWalArchiveSize=1073741824, walSegments=10, walSegmentSize=67108864, walPath=db/wal, walArchivePath=db/wal/archive, metricsEnabled=false, walMode=LOG_ONLY, walTlbSize=131072, walBuffSize=0, walFlushFreq=2000, walFsyncDelay=1000, walRecordIterBuffSize=67108864, alwaysWriteFullPages=false, fileIOFactory=org.apache.ignite.internal.processors.cache.persistence.file.AsyncFileIOFactory@5c3972f7, metricsSubIntervalCnt=5, metricsRateTimeInterval=60000, walAutoArchiveAfterInactivity=-1, writeThrottlingEnabled=false, walCompactionEnabled=false, walCompactionLevel=1, checkpointReadLockTimeout=null], activeOnStart=true, autoActivation=true, longQryWarnTimeout=3000, sqlConnCfg=null, cliConnCfg=ClientConnectorConfiguration [host=null, port=10800, portRange=100, sockSndBufSize=0, sockRcvBufSize=0, tcpNoDelay=true, maxOpenCursorsPerConn=128, threadPoolSize=8, idleTimeout=0, jdbcEnabled=true, odbcEnabled=true, thinCliEnabled=true, sslEnabled=false, useIgniteSslCtxFactory=true, sslClientAuth=false, sslCtxFactory=null], mvccVacuumThreadCnt=2, mvccVacuumFreq=5000, authEnabled=false, failureHnd=AbstractFailureHandler [ignoredFailureTypes=EmptySet []]NoOpFailureHandler [super=], commFailureRslvr=null] [2018-10-22 17:08:20,818][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] Daemon mode: off [2018-10-22 17:08:20,818][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] OS: Windows 10 10.0 amd64 [2018-10-22 17:08:20,818][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] OS user: santonov [2018-10-22 17:08:20,819][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] PID: 17528 [2018-10-22 17:08:20,819][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] Language runtime: Java Platform API Specification ver. 1.8 [2018-10-22 17:08:20,819][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] VM information: Java(TM) SE Runtime Environment 1.8.0_181-b13 Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 25.181-b13 [2018-10-22 17:08:20,819][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] VM total memory: 3.5GB [2018-10-22 17:08:20,819][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] Remote Management [restart: off, REST: off, JMX (remote: off)] [2018-10-22 17:08:20,819][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] Logger: GridTestLog4jLogger [quiet=false, config=null] [2018-10-22 17:08:20,819][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] IGNITE_HOME=C:\IdeaProjects\apache-ignite [2018-10-22 17:08:20,819][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] VM arguments: [-ea, -Didea.test.cyclic.buffer.size=67108864, -javaagent:C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2018.2.2\lib\idea_rt.jar=53501:C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2018.2.2\bin, -Dfile.encoding=UTF-8] [2018-10-22 17:08:20,819][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] System cache's DataRegion size is configured to 40 MB. Use DataStorageConfiguration.systemRegionInitialSize property to change the setting. [2018-10-22 17:08:20,819][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] Configured caches [in 'sysMemPlc' dataRegion: ['ignite-sys-cache']] [2018-10-22 17:08:20,820][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] 3-rd party licenses can be found at: C:\IdeaProjects\apache-ignite\libs\licenses [2018-10-22 17:08:20,819][WARN ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] Please set system property '-Djava.net.preferIPv4Stack=true' to avoid possible problems in mixed environments. [2018-10-22 17:08:20,820][WARN ][pub-#132%persistence.PersistenceBasicCompatibilityTest0%][GridDiagnostic] This operating system has been tested less rigorously: Windows 10 10.0 amd64. Our team will appreciate the feedback if you experience any problems running ignite in this environment. [2018-10-22 17:08:20,820][WARN ][pub-#135%persistence.PersistenceBasicCompatibilityTest0%][GridDiagnostic] Initial heap size is 254MB (should be no less than 512MB, use -Xms512m -Xmx512m). [2018-10-22 17:08:20,825][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][IgnitePluginProcessor] Configured plugins: [2018-10-22 17:08:20,825][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][IgnitePluginProcessor] ^-- TestCompatibilityPlugin 1.0 [2018-10-22 17:08:20,825][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][IgnitePluginProcessor] ^-- [2018-10-22 17:08:20,825][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][IgnitePluginProcessor] [2018-10-22 17:08:20,825][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][IgnitePluginProcessor] ^-- TestReconnectPlugin 1.0 [2018-10-22 17:08:20,825][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][IgnitePluginProcessor] ^-- [2018-10-22 17:08:20,825][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][IgnitePluginProcessor] [2018-10-22 17:08:20,825][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][IgnitePluginProcessor] ^-- StanByClusterTestProvider 1.0 [2018-10-22 17:08:20,825][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][IgnitePluginProcessor] ^-- null [2018-10-22 17:08:20,825][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][IgnitePluginProcessor] [2018-10-22 17:08:20,825][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][IgnitePluginProcessor] ^-- PageMemory tracker plugin 1.0 [2018-10-22 17:08:20,825][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][IgnitePluginProcessor] ^-- [2018-10-22 17:08:20,825][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][IgnitePluginProcessor] [2018-10-22 17:08:20,825][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][FailureProcessor] Configured failure handler: [hnd=NoOpFailureHandler [super=AbstractFailureHandler [ignoredFailureTypes=EmptySet []]]] [2018-10-22 17:08:20,830][WARN ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][TcpCommunicationSpi] Message queue limit is set to 0 which may lead to potential OOMEs when running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due to message queues growth on sender and receiver sides. [2018-10-22 17:08:20,829][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][TcpCommunicationSpi] Successfully bound communication NIO server to TCP port [port=45010, locHost=/127.0.0.1, selectorsCnt=4, selectorSpins=0, pairedConn=false] [2018-10-22 17:08:21,019][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] Security status [authentication=off, tls/ssl=off][2018-10-22 17:08:21,019][WARN ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][GridCollisionManager] Collision resolution is disabled (all jobs will be activated upon arrival). [2018-10-22 17:08:21,022][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][TestTcpDiscoverySpi] Successfully bound to TCP port [port=47500, localHost=127.0.0.1/127.0.0.1, locNodeId=3f392b21-9ce6-4ce5-811e-c66796800000] [2018-10-22 17:08:21,029][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PdsFoldersResolver] Successfully locked persistence storage folder [C:\IdeaProjects\apache-ignite\work\db\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332] [2018-10-22 17:08:21,029][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PdsFoldersResolver] Consistent ID used for local node is [81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332] according to persistence data storage folders [2018-10-22 17:08:21,030][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][CacheObjectBinaryProcessorImpl] Resolved directory for serialized binary metadata: C:\IdeaProjects\apache-ignite\work\binary_meta\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332 [2018-10-22 17:08:21,034][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][FilePageStoreManager] Resolved page store work directory: C:\IdeaProjects\apache-ignite\work\db\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332 [2018-10-22 17:08:21,035][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][FileWriteAheadLogManager] Resolved write ahead log work directory: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332 [2018-10-22 17:08:21,035][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][FileWriteAheadLogManager] Resolved write ahead log archive directory: C:\IdeaProjects\apache-ignite\work\db\wal\archive\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332 [2018-10-22 17:08:21,046][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][FileWriteAheadLogManager] Started write-ahead log manager [mode=LOG_ONLY] [2018-10-22 17:08:21,048][WARN ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][GridCacheDatabaseSharedManager] DataRegionConfiguration.maxWalArchiveSize instead DataRegionConfiguration.walHistorySize would be used for removing old archive wal files [2018-10-22 17:08:21,049][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PartitionsEvictManager] Evict partition permits=2 [2018-10-22 17:08:21,053][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][ClientListenerProcessor] Client connector processor has started on TCP port 10800 [2018-10-22 17:08:21,413][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] Non-loopback local IPs: 10.0.0.100, fe80:0:0:0:41c9:b4fa:d846:61ac%eth5, fe80:0:0:0:5422:f228:82d7:f577%eth0, fe80:0:0:0:9c7:33de:6920:739e%wlan2, fe80:0:0:0:a99b:6b5:1b8c:71d6%wlan0, fe80:0:0:0:c1f7:1e90:f607:de9d%wlan1, fe80:0:0:0:cd36:5279:6ba0:90aa%eth2, fe80:0:0:0:f0d5:95b5:b835:21dc%eth1 [2018-10-22 17:08:21,413][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] Enabled local MACs: 00A0C6000000, 30243242E663, 30243242E664, 30243242E667, 32243242E663, 482AE30AD0AE, E04F439828DF [2018-10-22 17:08:21,414][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][GridCacheDatabaseSharedManager] Read checkpoint status [startMarker=null, endMarker=null] [2018-10-22 17:08:21,416][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PageMemoryImpl] Started page memory [memoryAllocated=100.0 MiB, pages=24808, tableSize=1.9 MiB, checkpointBuffer=100.0 MiB] [2018-10-22 17:08:21,416][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][GridCacheDatabaseSharedManager] Checking memory state [lastValidPos=FileWALPointer [idx=0, fileOff=0, len=0], lastMarked=FileWALPointer [idx=0, fileOff=0, len=0], lastCheckpointId=00000000-0000-0000-0000-000000000000] [2018-10-22 17:08:21,470][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][GridCacheDatabaseSharedManager] Applying lost cache updates since last checkpoint record [lastMarked=FileWALPointer [idx=0, fileOff=0, len=0], lastCheckpointId=00000000-0000-0000-0000-000000000000] [2018-10-22 17:08:21,522][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][GridCacheDatabaseSharedManager] Finished applying WAL changes [updatesApplied=0, time=52ms] [2018-10-22 17:08:21,531][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][TestTcpDiscoverySpi] Connection check threshold is calculated: 10000 [2018-10-22 17:08:30,612][INFO ][disco-notifier-worker-#149%persistence.PersistenceBasicCompatibilityTest0%][MvccProcessorImpl] Assigned mvcc coordinator [crd=MvccCoordinator [nodeId=3f392b21-9ce6-4ce5-811e-c66796800000, crdVer=1540217310609, topVer=AffinityTopologyVersion [topVer=1, minorTopVer=0]], crdNode=TcpDiscoveryNode [id=3f392b21-9ce6-4ce5-811e-c66796800000, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1540217301530, loc=true, ver=2.7.0#19700101-sha1:00000000, isClient=false]] [2018-10-22 17:08:30,617][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] Performance suggestions for grid 'persistence.PersistenceBasicCompatibilityTest0' (fix if possible) [2018-10-22 17:08:30,617][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] To disable, set -DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true [2018-10-22 17:08:30,617][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] ^-- Disable checkpoints (remove 'checkpointSpi' from configuration) [2018-10-22 17:08:30,617][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] ^-- Disable grid events (remove 'includeEventTypes' from configuration) [2018-10-22 17:08:30,617][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] ^-- Enable G1 Garbage Collector (add '-XX:+UseG1GC' to JVM options) [2018-10-22 17:08:30,617][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] ^-- Specify JVM heap max size (add '-Xmx<size>[g|G|m|M|k|K]' to JVM options) [2018-10-22 17:08:30,617][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] ^-- Set max direct memory size if getting 'OOME: Direct buffer memory' (add '-XX:MaxDirectMemorySize=<size>[g|G|m|M|k|K]' to JVM options) [2018-10-22 17:08:30,617][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] ^-- Disable processing of calls to System.gc() (add '-XX:+DisableExplicitGC' to JVM options) [2018-10-22 17:08:30,617][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] ^-- Disable assertions (remove '-ea' from JVM options) [2018-10-22 17:08:30,617][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] Refer to this page for more performance suggestions: https://apacheignite.readme.io/docs/jvm-and-system-tuning [2018-10-22 17:08:30,617][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] [2018-10-22 17:08:30,617][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] To start Console Management & Monitoring run ignitevisorcmd.{sh|bat} [2018-10-22 17:08:30,617][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] Data Regions Configured: [2018-10-22 17:08:30,617][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] ^-- default [initSize=256.0 MiB, maxSize=256.0 MiB, persistence=true] [2018-10-22 17:08:30,618][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] >>> +------------------------------------------+ >>> Ignite ver. 2.7.0-SNAPSHOT#19700101-sha1:DEV >>> +------------------------------------------+ >>> OS name: Windows 10 10.0 amd64 >>> CPU(s): 8 >>> Heap: 3.5GB >>> VM name: 17528@LAPTOP-IGKOMEL9 >>> Ignite instance name: persistence.PersistenceBasicCompatibilityTest0 >>> Local node [ID=3F392B21-9CE6-4CE5-811E-C66796800000, order=1, clientMode=false] >>> Local node addresses: [127.0.0.1] >>> Local ports: TCP:10800 TCP:45010 TCP:47500 [2018-10-22 17:08:30,618][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] >>> Ignite cluster is not active (limited functionality available). Use control.(sh|bat) script or IgniteCluster interface to activate. [2018-10-22 17:08:30,618][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][GridDiscoveryManager] Topology snapshot [ver=1, locNode=3f392b21, servers=1, clients=0, state=INACTIVE, CPUs=8, offheap=0.25GB, heap=3.5GB] [2018-10-22 17:08:30,618][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][root] Node started with the following configuration [id=3f392b21-9ce6-4ce5-811e-c66796800000, marshaller=BinaryMarshaller [], discovery=TcpDiscoverySpi [addrRslvr=null, sockTimeout=5000, ackTimeout=5000, marsh=JdkMarshaller [clsFilter=org.apache.ignite.marshaller.MarshallerUtils$1@3c9eddf4], reconCnt=10, reconDelay=2000, maxAckTimeout=600000, forceSrvMode=false, clientReconnectDisabled=false, internalLsnr=null, skipAddrsRandomization=false], binaryCfg=BinaryConfiguration [idMapper=null, nameMapper=null, serializer=null, compactFooter=true], lateAff=true] [2018-10-22 17:08:30,624][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][GridClusterStateProcessor] Sending activate request with BaselineTopology BaselineTopology [id=0, branchingHash=-416176595, branchingType='New BaselineTopology', baselineNodes=[81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332]] [2018-10-22 17:08:30,625][INFO ][disco-notifier-worker-#149%persistence.PersistenceBasicCompatibilityTest0%][GridClusterStateProcessor] Received activate request with BaselineTopology[id=0] [2018-10-22 17:08:30,626][INFO ][disco-notifier-worker-#149%persistence.PersistenceBasicCompatibilityTest0%][GridClusterStateProcessor] Started state transition: true [2018-10-22 17:08:30,627][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][time] Started exchange init [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], mvccCrd=MvccCoordinator [nodeId=3f392b21-9ce6-4ce5-811e-c66796800000, crdVer=1540217310609, topVer=AffinityTopologyVersion [topVer=1, minorTopVer=0]], mvccCrdChange=true, crd=true, evt=DISCOVERY_CUSTOM_EVT, evtNode=3f392b21-9ce6-4ce5-811e-c66796800000, customEvt=ChangeGlobalStateMessage [id=4bc2a1c9661-142c551e-2962-403a-b864-0281692574ee, reqId=4ca70fa9-7a54-411a-8ad5-f4633e40edd4, initiatingNodeId=3f392b21-9ce6-4ce5-811e-c66796800000, activate=true, baselineTopology=BaselineTopology [id=0, branchingHash=-416176595, branchingType='New BaselineTopology', baselineNodes=[81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332]], forceChangeBaselineTopology=false, timestamp=1540217310624], allowMerge=false] [2018-10-22 17:08:30,627][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridDhtPartitionsExchangeFuture] Start activation process [nodeId=3f392b21-9ce6-4ce5-811e-c66796800000, client=false, topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1]] [2018-10-22 17:08:30,627][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][FilePageStoreManager] Resolved page store work directory: C:\IdeaProjects\apache-ignite\work\db\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332 [2018-10-22 17:08:30,628][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][FileWriteAheadLogManager] Resolved write ahead log work directory: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332 [2018-10-22 17:08:30,628][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][FileWriteAheadLogManager] Resolved write ahead log archive directory: C:\IdeaProjects\apache-ignite\work\db\wal\archive\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332 [2018-10-22 17:08:30,630][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][FileWriteAheadLogManager] Started write-ahead log manager [mode=LOG_ONLY] [2018-10-22 17:08:30,633][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][PageMemoryImpl] Started page memory [memoryAllocated=100.0 MiB, pages=24808, tableSize=1.9 MiB, checkpointBuffer=100.0 MiB] [2018-10-22 17:08:30,638][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][PageMemoryImpl] Started page memory [memoryAllocated=256.0 MiB, pages=63520, tableSize=4.9 MiB, checkpointBuffer=256.0 MiB] [2018-10-22 17:08:30,640][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][PageMemoryImpl] Started page memory [memoryAllocated=100.0 MiB, pages=24808, tableSize=1.9 MiB, checkpointBuffer=100.0 MiB] [2018-10-22 17:08:30,641][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][PageMemoryImpl] Started page memory [memoryAllocated=100.0 MiB, pages=24808, tableSize=1.9 MiB, checkpointBuffer=100.0 MiB] [2018-10-22 17:08:30,641][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][msg] Components activation performed in 14 ms. [2018-10-22 17:08:30,691][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridCacheDatabaseSharedManager] Read checkpoint status [startMarker=null, endMarker=null] [2018-10-22 17:08:30,691][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridCacheDatabaseSharedManager] Checking memory state [lastValidPos=FileWALPointer [idx=0, fileOff=0, len=0], lastMarked=FileWALPointer [idx=0, fileOff=0, len=0], lastCheckpointId=00000000-0000-0000-0000-000000000000] [2018-10-22 17:08:30,756][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][FileWriteAheadLogManager] Resuming logging to WAL segment [file=C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal, offset=140187, ver=1] [2018-10-22 17:08:30,756][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][FileWriteAheadLogManager] Record serializer version change detected, will start logging with a new WAL record serializer to a new WAL segment [curFile=org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileWriteHandle@7191bd14, newVer=2, oldVer=1] [2018-10-22 17:08:30,763][INFO ][wal-file-archiver%persistence.PersistenceBasicCompatibilityTest0-#152%persistence.PersistenceBasicCompatibilityTest0%][FileWriteAheadLogManager] Starting to copy WAL segment [absIdx=0, segIdx=0, origFile=C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal, dstFile=C:\IdeaProjects\apache-ignite\work\db\wal\archive\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal] [2018-10-22 17:08:30,777][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridClusterStateProcessor] Writing BaselineTopology[id=0] [2018-10-22 17:08:30,779][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridCacheDatabaseSharedManager] Binary recovery performed in 137 ms. [2018-10-22 17:08:30,781][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][CacheAffinitySharedManager] Updating caches registry performed in 2 ms. [2018-10-22 17:08:30,788][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridCacheProcessor] Started cache [name=ignite-sys-cache, id=-2100569601, memoryPolicyName=sysMemPlc, mode=REPLICATED, atomicity=TRANSACTIONAL, backups=2147483647, mvcc=false], encryptionEnabled=false] [2018-10-22 17:08:30,795][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridCacheProcessor] Started cache [name=PersistenceBasicCompatibilityTest, id=2041049331, memoryPolicyName=default, mode=PARTITIONED, atomicity=TRANSACTIONAL, backups=1, mvcc=false], encryptionEnabled=false] [2018-10-22 17:08:30,795][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][CacheAffinitySharedManager] Caches starting performed in 14 ms. [2018-10-22 17:08:30,796][INFO ][sys-#157%persistence.PersistenceBasicCompatibilityTest0%][GridAffinityAssignmentCache] Local node affinity assignment distribution is not ideal [cache=PersistenceBasicCompatibilityTest, expectedPrimary=1024.00, actualPrimary=1024, expectedBackups=1024.00, actualBackups=0, warningThreshold=50.00%] [2018-10-22 17:08:30,797][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][CacheAffinitySharedManager] Affinity initialization for started caches performed in 2 ms. [2018-10-22 17:08:30,797][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridDhtPartitionsExchangeFuture] Successfully activated caches [nodeId=3f392b21-9ce6-4ce5-811e-c66796800000, client=false, topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1]] [2018-10-22 17:08:30,798][INFO ][sys-#159%persistence.PersistenceBasicCompatibilityTest0%][GridAffinityAssignmentCache] Local node affinity assignment distribution is not ideal [cache=PersistenceBasicCompatibilityTest, expectedPrimary=1024.00, actualPrimary=1024, expectedBackups=1024.00, actualBackups=0, warningThreshold=50.00%] [2018-10-22 17:08:30,799][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridDhtPartitionsExchangeFuture] Finished waiting for partition release future [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], waitTime=0ms, futInfo=NA, mode=LOCAL] [2018-10-22 17:08:30,800][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridCacheDatabaseSharedManager] Read checkpoint status [startMarker=null, endMarker=null] [2018-10-22 17:08:30,800][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridCacheDatabaseSharedManager] Applying lost cache updates since last checkpoint record [lastMarked=FileWALPointer [idx=0, fileOff=0, len=0], lastCheckpointId=00000000-0000-0000-0000-000000000000] [2018-10-22 17:08:30,807][ERROR][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridDhtPartitionsExchangeFuture] Failed to reinitialize local partitions (rebalancing will be stopped): GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=3f392b21-9ce6-4ce5-811e-c66796800000, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1540217301530, loc=true, ver=2.7.0#19700101-sha1:00000000, isClient=false], topVer=1, nodeId8=3f392b21, msg=null, type=DISCOVERY_CUSTOM_EVT, tstamp=1540217310618]DiscoveryCustomEvent [customMsg=ChangeGlobalStateMessage [id=4bc2a1c9661-142c551e-2962-403a-b864-0281692574ee, reqId=4ca70fa9-7a54-411a-8ad5-f4633e40edd4, initiatingNodeId=3f392b21-9ce6-4ce5-811e-c66796800000, activate=true, baselineTopology=BaselineTopology [id=0, branchingHash=-416176595, branchingType='New BaselineTopology', baselineNodes=[81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332]], forceChangeBaselineTopology=false, timestamp=1540217310624], affTopVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], super=], nodeId=3f392b21, evt=DISCOVERY_CUSTOM_EVT] class org.apache.ignite.IgniteCheckedException: Failed to initialize WAL segment: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.initReadHandle(AbstractWalRecordsIterator.java:412) at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.initReadHandle(FileWriteAheadLogManager.java:3113) at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.advanceSegment(FileWriteAheadLogManager.java:3195) at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.advance(AbstractWalRecordsIterator.java:163) at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.<init>(FileWriteAheadLogManager.java:3087) at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.<init>(FileWriteAheadLogManager.java:3003) at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.replay(FileWriteAheadLogManager.java:925) at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.applyLastUpdates(GridCacheDatabaseSharedManager.java:2289) at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreState(GridCacheDatabaseSharedManager.java:1632) at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.beforeExchange(GridCacheDatabaseSharedManager.java:1298) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.distributedExchange(GridDhtPartitionsExchangeFuture.java:1455) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:808) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:2746) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2618) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Caused by: java.nio.file.FileSystemException: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal: The process cannot access the file because it is being used by another process. at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86) at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97) at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102) at sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:115) at java.nio.channels.FileChannel.open(FileChannel.java:287) at java.nio.channels.FileChannel.open(FileChannel.java:335) at org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIO.<init>(RandomAccessFileIO.java:43) at org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIOFactory.create(RandomAccessFileIOFactory.java:42) at org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIOFactory.create(RandomAccessFileIOFactory.java:37) at org.apache.ignite.internal.processors.cache.persistence.wal.FileDescriptor.toIO(FileDescriptor.java:142) at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.initReadHandle(AbstractWalRecordsIterator.java:378) ... 15 more [2018-10-22 17:08:30,814][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridDhtPartitionsExchangeFuture] Finish exchange future [startVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], resVer=null, err=class org.apache.ignite.IgniteCheckedException: Failed to initialize WAL segment: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal] [2018-10-22 17:08:30,815][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridDhtPartitionsExchangeFuture] Completed partition exchange [localNode=3f392b21-9ce6-4ce5-811e-c66796800000, exchange=GridDhtPartitionsExchangeFuture [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], evt=DISCOVERY_CUSTOM_EVT, evtNode=TcpDiscoveryNode [id=3f392b21-9ce6-4ce5-811e-c66796800000, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1540217301530, loc=true, ver=2.7.0#19700101-sha1:00000000, isClient=false], done=true], topVer=null, durationFromInit=187] [2018-10-22 17:08:30,819][ERROR][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridCachePartitionExchangeManager] Failed to wait for completion of partition map exchange (preloading will not start): GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=3f392b21-9ce6-4ce5-811e-c66796800000, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1540217301530, loc=true, ver=2.7.0#19700101-sha1:00000000, isClient=false], topVer=1, nodeId8=3f392b21, msg=null, type=DISCOVERY_CUSTOM_EVT, tstamp=1540217310618]DiscoveryCustomEvent [customMsg=null, affTopVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], super=], crd=TcpDiscoveryNode [id=3f392b21-9ce6-4ce5-811e-c66796800000, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1540217301530, loc=true, ver=2.7.0#19700101-sha1:00000000, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=3f392b21-9ce6-4ce5-811e-c66796800000, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1540217301530, loc=true, ver=2.7.0#19700101-sha1:00000000, isClient=false], topVer=1, nodeId8=3f392b21, msg=null, type=DISCOVERY_CUSTOM_EVT, tstamp=1540217310618]DiscoveryCustomEvent [customMsg=null, affTopVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], super=], nodeId=3f392b21, evt=DISCOVERY_CUSTOM_EVT], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=false, hash=364933128], init=false, lastVer=null, partReleaseFut=AffinityTopologyVersion [topVer=1, minorTopVer=1]AffinityTopologyVersion [topVer=1, minorTopVer=1]AffinityTopologyVersion [topVer=1, minorTopVer=1]AffinityTopologyVersion [topVer=1, minorTopVer=1]AffinityTopologyVersion [topVer=1, minorTopVer=1]AffinityTopologyVersion [topVer=1, minorTopVer=1]AffinityTopologyVersion [topVer=1, minorTopVer=1]PartitionReleaseFuture [topVer=, futures=[ExplicitLockReleaseFuture [topVer=, futures=[]], AtomicUpdateReleaseFuture [topVer=, futures=[]], DataStreamerReleaseFuture [topVer=, futures=[]], LocalTxReleaseFuture [topVer=, futures=[]], AllTxReleaseFuture [topVer=, futures=[RemoteTxReleaseFuture [topVer=, futures=[]]]]]], exchActions=null, affChangeMsg=null, initTs=1540217310618, centralizedAff=false, forceAffReassignment=true, exchangeLocE=null, cacheChangeFailureMsgSent=false, done=true, state=CRD, registerCachesFuture=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=null, hash=1024540699], partitionsSent=false, partitionsReceived=false, delayedLatestMsg=null, afterLsnrCompleteFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=null, hash=1881071890], evtLatch=0, remaining=HashSet [], super=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=class o.a.i.IgniteCheckedException: Failed to initialize WAL segment: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal, hash=1188696354]] class org.apache.ignite.IgniteCheckedException: Failed to initialize WAL segment: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.initReadHandle(AbstractWalRecordsIterator.java:412) at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.initReadHandle(FileWriteAheadLogManager.java:3113) at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.advanceSegment(FileWriteAheadLogManager.java:3195) at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.advance(AbstractWalRecordsIterator.java:163) at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.<init>(FileWriteAheadLogManager.java:3087) at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.<init>(FileWriteAheadLogManager.java:3003) at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.replay(FileWriteAheadLogManager.java:925) at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.applyLastUpdates(GridCacheDatabaseSharedManager.java:2289) at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreState(GridCacheDatabaseSharedManager.java:1632) at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.beforeExchange(GridCacheDatabaseSharedManager.java:1298) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.distributedExchange(GridDhtPartitionsExchangeFuture.java:1455) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:808) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:2746) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2618) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Caused by: java.nio.file.FileSystemException: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal: The process cannot access the file because it is being used by another process. at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86) at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97) at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102) at sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:115) at java.nio.channels.FileChannel.open(FileChannel.java:287) at java.nio.channels.FileChannel.open(FileChannel.java:335) at org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIO.<init>(RandomAccessFileIO.java:43) at org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIOFactory.create(RandomAccessFileIOFactory.java:42) at org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIOFactory.create(RandomAccessFileIOFactory.java:37) at org.apache.ignite.internal.processors.cache.persistence.wal.FileDescriptor.toIO(FileDescriptor.java:142) at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.initReadHandle(AbstractWalRecordsIterator.java:378) ... 15 more [2018-10-22 17:08:30,823][ERROR][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][IgniteTestResources] Critical system error detected. Will be handled accordingly to configured handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler [ignoredFailureTypes=EmptySet []]], failureCtx=FailureContext [type=SYSTEM_WORKER_TERMINATION, err=class o.a.i.IgniteCheckedException: Failed to initialize WAL segment: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal]] class org.apache.ignite.IgniteCheckedException: Failed to initialize WAL segment: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.initReadHandle(AbstractWalRecordsIterator.java:412) at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.initReadHandle(FileWriteAheadLogManager.java:3113) at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.advanceSegment(FileWriteAheadLogManager.java:3195) at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.advance(AbstractWalRecordsIterator.java:163) at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.<init>(FileWriteAheadLogManager.java:3087) at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.<init>(FileWriteAheadLogManager.java:3003) at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.replay(FileWriteAheadLogManager.java:925) at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.applyLastUpdates(GridCacheDatabaseSharedManager.java:2289) at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreState(GridCacheDatabaseSharedManager.java:1632) at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.beforeExchange(GridCacheDatabaseSharedManager.java:1298) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.distributedExchange(GridDhtPartitionsExchangeFuture.java:1455) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:808) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:2746) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2618) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Caused by: java.nio.file.FileSystemException: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal: The process cannot access the file because it is being used by another process. at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86) at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97) at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102) at sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:115) at java.nio.channels.FileChannel.open(FileChannel.java:287) at java.nio.channels.FileChannel.open(FileChannel.java:335) at org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIO.<init>(RandomAccessFileIO.java:43) at org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIOFactory.create(RandomAccessFileIOFactory.java:42) at org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIOFactory.create(RandomAccessFileIOFactory.java:37) at org.apache.ignite.internal.processors.cache.persistence.wal.FileDescriptor.toIO(FileDescriptor.java:142) at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.initReadHandle(AbstractWalRecordsIterator.java:378) ... 15 more [2018-10-22 17:08:30,829][WARN ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][FailureProcessor] No deadlocked threads detected. [2018-10-22 17:08:30,891][WARN ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][FailureProcessor] Thread dump at 2018/10/22 17:08:30 MSK Thread [name="sys-#161%persistence.PersistenceBasicCompatibilityTest0%", id=208, state=TIMED_WAITING, blockCnt=0, waitCnt=2] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6ec7abc, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Thread [name="sys-#160%persistence.PersistenceBasicCompatibilityTest0%", id=207, state=TIMED_WAITING, blockCnt=0, waitCnt=2] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6ec7abc, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Thread [name="sys-#159%persistence.PersistenceBasicCompatibilityTest0%", id=206, state=TIMED_WAITING, blockCnt=0, waitCnt=1] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6ec7abc, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Thread [name="sys-#158%persistence.PersistenceBasicCompatibilityTest0%", id=205, state=TIMED_WAITING, blockCnt=0, waitCnt=2] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6ec7abc, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Thread [name="sys-#157%persistence.PersistenceBasicCompatibilityTest0%", id=204, state=TIMED_WAITING, blockCnt=0, waitCnt=1] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6ec7abc, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Thread [name="sys-#156%persistence.PersistenceBasicCompatibilityTest0%", id=203, state=TIMED_WAITING, blockCnt=0, waitCnt=2] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6ec7abc, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Thread [name="ttl-cleanup-worker-#155%persistence.PersistenceBasicCompatibilityTest0%", id=202, state=TIMED_WAITING, blockCnt=0, waitCnt=1] at java.lang.Thread.sleep(Native Method) at o.a.i.i.util.IgniteUtils.sleep(IgniteUtils.java:7778) at o.a.i.i.processors.cache.GridCacheSharedTtlCleanupManager$CleanupWorker.body(GridCacheSharedTtlCleanupManager.java:149) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="sys-#154%persistence.PersistenceBasicCompatibilityTest0%", id=201, state=TIMED_WAITING, blockCnt=0, waitCnt=2] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6ec7abc, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Thread [name="wal-segment-syncer-#153%persistence.PersistenceBasicCompatibilityTest0%", id=200, state=TIMED_WAITING, blockCnt=0, waitCnt=1] at java.lang.Thread.sleep(Native Method) at o.a.i.i.util.IgniteUtils.sleep(IgniteUtils.java:7778) at o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$WalSegmentSyncer.body(FileWriteAheadLogManager.java:3614) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="wal-file-archiver%persistence.PersistenceBasicCompatibilityTest0-#152%persistence.PersistenceBasicCompatibilityTest0%", id=199, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.FileDispatcherImpl.force0(Native Method) at sun.nio.ch.FileDispatcherImpl.force(FileDispatcherImpl.java:89) at sun.nio.ch.FileChannelImpl.force(FileChannelImpl.java:388) at o.a.i.i.processors.cache.persistence.file.RandomAccessFileIO.force(RandomAccessFileIO.java:88) at o.a.i.i.processors.cache.persistence.file.RandomAccessFileIO.force(RandomAccessFileIO.java:113) at o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileArchiver.archiveSegment(FileWriteAheadLogManager.java:1913) at o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileArchiver.body(FileWriteAheadLogManager.java:1783) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%", id=197, state=RUNNABLE, blockCnt=0, waitCnt=38] at sun.management.ThreadImpl.dumpThreads0(Native Method) at sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:454) at o.a.i.i.util.IgniteUtils.dumpThreads(IgniteUtils.java:1368) at o.a.i.i.processors.failure.FailureProcessor.process(FailureProcessor.java:140) - locked o.a.i.i.processors.failure.FailureProcessor@3792ce1 at o.a.i.i.processors.failure.FailureProcessor.process(FailureProcessor.java:116) at o.a.i.i.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2634) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="disco-event-worker-#150%persistence.PersistenceBasicCompatibilityTest0%", id=196, state=WAITING, blockCnt=0, waitCnt=4] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3a2d0702, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at o.a.i.i.managers.discovery.GridDiscoveryManager$DiscoveryWorker.body0(GridDiscoveryManager.java:2856) at o.a.i.i.managers.discovery.GridDiscoveryManager$DiscoveryWorker.body(GridDiscoveryManager.java:2825) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="tcp-disco-srvr-#12%persistence.PersistenceBasicCompatibilityTest0%", id=195, state=RUNNABLE, blockCnt=0, waitCnt=0] at java.net.DualStackPlainSocketImpl.accept0(Native Method) at java.net.DualStackPlainSocketImpl.socketAccept(DualStackPlainSocketImpl.java:131) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409) at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:199) - locked java.net.SocksSocketImpl@42ba2f76 at java.net.ServerSocket.implAccept(ServerSocket.java:545) at java.net.ServerSocket.accept(ServerSocket.java:513) at o.a.i.spi.discovery.tcp.ServerImpl$TcpServer.body(ServerImpl.java:5845) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at o.a.i.spi.discovery.tcp.ServerImpl$TcpServerThread.body(ServerImpl.java:5763) at o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:62) Thread [name="tcp-disco-msg-worker-#11%persistence.PersistenceBasicCompatibilityTest0%", id=194, state=TIMED_WAITING, blockCnt=0, waitCnt=940] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@27e68407, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522) at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684) at o.a.i.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7183) at o.a.i.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2700) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at o.a.i.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7119) at o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:62) Thread [name="disco-notifier-worker-#149%persistence.PersistenceBasicCompatibilityTest0%", id=193, state=WAITING, blockCnt=0, waitCnt=5] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6db2ba3e, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at o.a.i.i.managers.discovery.GridDiscoveryManager$DiscoveryMessageNotifierWorker.body0(GridDiscoveryManager.java:2660) at o.a.i.i.managers.discovery.GridDiscoveryManager$DiscoveryMessageNotifierWorker.body(GridDiscoveryManager.java:2705) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="grid-data-loader-flusher-#148%persistence.PersistenceBasicCompatibilityTest0%", id=192, state=WAITING, blockCnt=0, waitCnt=2] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3ff5b59d, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.DelayQueue.take(DelayQueue.java:211) at o.a.i.i.processors.datastreamer.DataStreamProcessor$2.body(DataStreamProcessor.java:109) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="grid-nio-worker-client-listener-3-#145%persistence.PersistenceBasicCompatibilityTest0%", id=189, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278) at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked o.a.i.i.util.nio.SelectedSelectionKeySet@2ca7ec8a - locked java.util.Collections$UnmodifiableSet@3f7283a5 - locked sun.nio.ch.WindowsSelectorImpl@47e9bd3f at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="grid-nio-worker-client-listener-2-#144%persistence.PersistenceBasicCompatibilityTest0%", id=188, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278) at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked o.a.i.i.util.nio.SelectedSelectionKeySet@5522b102 - locked java.util.Collections$UnmodifiableSet@4672d490 - locked sun.nio.ch.WindowsSelectorImpl@6d361850 at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="grid-nio-worker-client-listener-1-#143%persistence.PersistenceBasicCompatibilityTest0%", id=187, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278) at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked o.a.i.i.util.nio.SelectedSelectionKeySet@673c508c - locked java.util.Collections$UnmodifiableSet@275657c8 - locked sun.nio.ch.WindowsSelectorImpl@7750c1e6 at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="grid-nio-worker-client-listener-0-#142%persistence.PersistenceBasicCompatibilityTest0%", id=186, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278) at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked o.a.i.i.util.nio.SelectedSelectionKeySet@2a79a3c4 - locked java.util.Collections$UnmodifiableSet@2f443e37 - locked sun.nio.ch.WindowsSelectorImpl@3ab9c400 at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="nio-acceptor-client-listener-#146%persistence.PersistenceBasicCompatibilityTest0%", id=190, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278) at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked sun.nio.ch.Util$3@4b176d40 - locked java.util.Collections$UnmodifiableSet@2c179065 - locked sun.nio.ch.WindowsSelectorImpl@22acb656 at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at o.a.i.i.util.nio.GridNioServer$GridNioAcceptWorker.accept(GridNioServer.java:2929) at o.a.i.i.util.nio.GridNioServer$GridNioAcceptWorker.body(GridNioServer.java:2877) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="grid-sharedfs-timeout-worker-#10%persistence.PersistenceBasicCompatibilityTest0%", id=185, state=TIMED_WAITING, blockCnt=0, waitCnt=2] Lock [object=java.lang.Object@56c1ffc7, ownerName=null, ownerId=-1] at java.lang.Object.wait(Native Method) at o.a.i.spi.checkpoint.sharedfs.SharedFsTimeoutTask.body(SharedFsTimeoutTask.java:99) at o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:62) Thread [name="tcp-comm-worker-#9%persistence.PersistenceBasicCompatibilityTest0%", id=184, state=TIMED_WAITING, blockCnt=0, waitCnt=1] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@26cce532, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at o.a.i.spi.communication.tcp.TcpCommunicationSpi$CommunicationWorker.body(TcpCommunicationSpi.java:4287) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at o.a.i.spi.communication.tcp.TcpCommunicationSpi$5.body(TcpCommunicationSpi.java:2237) at o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:62) Thread [name="grid-nio-worker-tcp-comm-3-#140%persistence.PersistenceBasicCompatibilityTest0%", id=182, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278) at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked o.a.i.i.util.nio.SelectedSelectionKeySet@3648cd36 - locked java.util.Collections$UnmodifiableSet@7191d3af - locked sun.nio.ch.WindowsSelectorImpl@28f7359d at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="grid-nio-worker-tcp-comm-2-#139%persistence.PersistenceBasicCompatibilityTest0%", id=181, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278) at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked o.a.i.i.util.nio.SelectedSelectionKeySet@6953b0c4 - locked java.util.Collections$UnmodifiableSet@1902f1a5 - locked sun.nio.ch.WindowsSelectorImpl@66714581 at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="grid-nio-worker-tcp-comm-1-#138%persistence.PersistenceBasicCompatibilityTest0%", id=180, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278) at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked o.a.i.i.util.nio.SelectedSelectionKeySet@773bdf0f - locked java.util.Collections$UnmodifiableSet@39a84bc - locked sun.nio.ch.WindowsSelectorImpl@d9bc2b3 at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="grid-nio-worker-tcp-comm-0-#137%persistence.PersistenceBasicCompatibilityTest0%", id=179, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278) at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked o.a.i.i.util.nio.SelectedSelectionKeySet@61aad3cc - locked java.util.Collections$UnmodifiableSet@4840bb51 - locked sun.nio.ch.WindowsSelectorImpl@1fa86981 at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="nio-acceptor-tcp-comm-#141%persistence.PersistenceBasicCompatibilityTest0%", id=183, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278) at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked sun.nio.ch.Util$3@4cf5be9d - locked java.util.Collections$UnmodifiableSet@7bf6207c - locked sun.nio.ch.WindowsSelectorImpl@36219660 at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at o.a.i.i.util.nio.GridNioServer$GridNioAcceptWorker.accept(GridNioServer.java:2929) at o.a.i.i.util.nio.GridNioServer$GridNioAcceptWorker.body(GridNioServer.java:2877) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="grid-timeout-worker-#136%persistence.PersistenceBasicCompatibilityTest0%", id=178, state=TIMED_WAITING, blockCnt=1, waitCnt=14] Lock [object=java.lang.Object@6a990544, ownerName=null, ownerId=-1] at java.lang.Object.wait(Native Method) at o.a.i.i.processors.timeout.GridTimeoutProcessor$TimeoutWorker.body(GridTimeoutProcessor.java:269) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="pub-#135%persistence.PersistenceBasicCompatibilityTest0%", id=177, state=TIMED_WAITING, blockCnt=0, waitCnt=2] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@bf94fdc, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Thread [name="pub-#134%persistence.PersistenceBasicCompatibilityTest0%", id=176, state=TIMED_WAITING, blockCnt=0, waitCnt=2] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@bf94fdc, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Thread [name="pub-#133%persistence.PersistenceBasicCompatibilityTest0%", id=175, state=TIMED_WAITING, blockCnt=0, waitCnt=1] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@bf94fdc, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Thread [name="pub-#132%persistence.PersistenceBasicCompatibilityTest0%", id=174, state=TIMED_WAITING, blockCnt=0, waitCnt=1] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@bf94fdc, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Thread [name="pub-#131%persistence.PersistenceBasicCompatibilityTest0%", id=173, state=TIMED_WAITING, blockCnt=0, waitCnt=2] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@bf94fdc, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Thread [name="pub-#130%persistence.PersistenceBasicCompatibilityTest0%", id=172, state=TIMED_WAITING, blockCnt=0, waitCnt=1] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@bf94fdc, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Thread [name="jvm-pause-detector-worker", id=171, state=TIMED_WAITING, blockCnt=0, waitCnt=200] at java.lang.Thread.sleep(Native Method) at o.a.i.i.LongJVMPauseDetector$1.run(LongJVMPauseDetector.java:100) Thread [name="data-streamer-stripe-7-#129%persistence.PersistenceBasicCompatibilityTest0%", id=170, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="data-streamer-stripe-6-#128%persistence.PersistenceBasicCompatibilityTest0%", id=169, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="data-streamer-stripe-5-#127%persistence.PersistenceBasicCompatibilityTest0%", id=168, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="data-streamer-stripe-4-#126%persistence.PersistenceBasicCompatibilityTest0%", id=167, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="data-streamer-stripe-3-#125%persistence.PersistenceBasicCompatibilityTest0%", id=166, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="data-streamer-stripe-2-#124%persistence.PersistenceBasicCompatibilityTest0%", id=165, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="data-streamer-stripe-1-#123%persistence.PersistenceBasicCompatibilityTest0%", id=164, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="data-streamer-stripe-0-#122%persistence.PersistenceBasicCompatibilityTest0%", id=163, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="sys-stripe-7-#121%persistence.PersistenceBasicCompatibilityTest0%", id=162, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="sys-stripe-6-#120%persistence.PersistenceBasicCompatibilityTest0%", id=161, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="sys-stripe-5-#119%persistence.PersistenceBasicCompatibilityTest0%", id=160, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="sys-stripe-4-#118%persistence.PersistenceBasicCompatibilityTest0%", id=159, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="sys-stripe-3-#117%persistence.PersistenceBasicCompatibilityTest0%", id=158, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="sys-stripe-2-#116%persistence.PersistenceBasicCompatibilityTest0%", id=157, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="sys-stripe-1-#115%persistence.PersistenceBasicCompatibilityTest0%", id=156, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="sys-stripe-0-#114%persistence.PersistenceBasicCompatibilityTest0%", id=155, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="test-runner-#113%persistence.PersistenceBasicCompatibilityTest%", id=152, state=WAITING, blockCnt=2, waitCnt=7] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:178) at o.a.i.i.util.future.GridFutureAdapter.get(GridFutureAdapter.java:141) at o.a.i.i.cluster.IgniteClusterImpl.active(IgniteClusterImpl.java:312) at o.a.i.i.IgniteKernal.active(IgniteKernal.java:3597) at o.a.i.compatibility.persistence.PersistenceBasicCompatibilityTest.doTestStartupWithOldVersion(PersistenceBasicCompatibilityTest.java:121) at o.a.i.compatibility.persistence.PersistenceBasicCompatibilityTest.doTestStartupWithOldVersion(PersistenceBasicCompatibilityTest.java:139) at o.a.i.compatibility.persistence.PersistenceBasicCompatibilityTest.testNodeStartByOldVersionPersistenceData_2_3(PersistenceBasicCompatibilityTest.java:98) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at junit.framework.TestCase.runTest(TestCase.java:176) at o.a.i.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2176) at o.a.i.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:142) at o.a.i.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:2091) at java.lang.Thread.run(Thread.java:748) Thread [name="Thread-11", id=57, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method) at sun.nio.ch.Iocp.access$300(Iocp.java:46) at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333) at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Locked synchronizers: java.util.concurrent.ThreadPoolExecutor$Worker@4fa859e3 Thread [name="Thread-10", id=56, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method) at sun.nio.ch.Iocp.access$300(Iocp.java:46) at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333) at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Locked synchronizers: java.util.concurrent.ThreadPoolExecutor$Worker@7ffb57da Thread [name="Thread-9", id=55, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method) at sun.nio.ch.Iocp.access$300(Iocp.java:46) at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333) at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Locked synchronizers: java.util.concurrent.ThreadPoolExecutor$Worker@5c2099e1 Thread [name="Thread-8", id=54, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method) at sun.nio.ch.Iocp.access$300(Iocp.java:46) at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333) at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Locked synchronizers: java.util.concurrent.ThreadPoolExecutor$Worker@45efd3fb Thread [name="Thread-7", id=53, state=RUNNABLE, blockCnt=1, waitCnt=0] at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method) at sun.nio.ch.Iocp.access$300(Iocp.java:46) at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333) at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Locked synchronizers: java.util.concurrent.ThreadPoolExecutor$Worker@27642a4 Thread [name="Thread-6", id=52, state=RUNNABLE, blockCnt=2, waitCnt=0] at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method) at sun.nio.ch.Iocp.access$300(Iocp.java:46) at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333) at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Locked synchronizers: java.util.concurrent.ThreadPoolExecutor$Worker@2942d82 Thread [name="Thread-5", id=51, state=RUNNABLE, blockCnt=1, waitCnt=0] at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method) at sun.nio.ch.Iocp.access$300(Iocp.java:46) at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333) at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Locked synchronizers: java.util.concurrent.ThreadPoolExecutor$Worker@1b31ca3 Thread [name="Thread-4", id=50, state=RUNNABLE, blockCnt=1, waitCnt=0] at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method) at sun.nio.ch.Iocp.access$300(Iocp.java:46) at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333) at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Locked synchronizers: java.util.concurrent.ThreadPoolExecutor$Worker@66a42f4e Thread [name="Thread-3", id=49, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method) at sun.nio.ch.Iocp.access$300(Iocp.java:46) at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333) at java.lang.Thread.run(Thread.java:748) Thread [name="ignite-clock-for-tests", id=12, state=TIMED_WAITING, blockCnt=0, waitCnt=7288] at java.lang.Thread.sleep(Native Method) at o.a.i.i.util.GridTestClockTimer.run(GridTestClockTimer.java:50) at java.lang.Thread.run(Thread.java:748) Thread [name="Monitor Ctrl-Break", id=6, state=RUNNABLE, blockCnt=1, waitCnt=0] at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284) at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326) at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178) - locked java.io.InputStreamReader@41466b1f at java.io.InputStreamReader.read(InputStreamReader.java:184) at java.io.BufferedReader.fill(BufferedReader.java:161) at java.io.BufferedReader.readLine(BufferedReader.java:324) - locked java.io.InputStreamReader@41466b1f at java.io.BufferedReader.readLine(BufferedReader.java:389) at com.intellij.rt.execution.application.AppMainV2$1.run(AppMainV2.java:64) Thread [name="Attach Listener", id=5, state=RUNNABLE, blockCnt=0, waitCnt=0] Thread [name="Signal Dispatcher", id=4, state=RUNNABLE, blockCnt=0, waitCnt=0] Thread [name="Finalizer", id=3, state=WAITING, blockCnt=60, waitCnt=5] Lock [object=java.lang.ref.ReferenceQueue$Lock@7b6e77c1, ownerName=null, ownerId=-1] at java.lang.Object.wait(Native Method) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:216) Thread [name="Reference Handler", id=2, state=WAITING, blockCnt=5, waitCnt=4] Lock [object=java.lang.ref.Reference$Lock@59ffeecd, ownerName=null, ownerId=-1] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at java.lang.ref.Reference.tryHandlePending(Reference.java:191) at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153) Thread [name="main", id=1, state=TIMED_WAITING, blockCnt=2, waitCnt=3] Lock [object=o.a.i.thread.IgniteThread@1839cd81, ownerName=null, ownerId=-1] at java.lang.Object.wait(Native Method) at java.lang.Thread.join(Thread.java:1260) at o.a.i.testframework.junits.GridAbstractTest.runTest(GridAbstractTest.java:2103) at junit.framework.TestCase.runBare(TestCase.java:141) at junit.framework.TestResult$1.protect(TestResult.java:122) at junit.framework.TestResult.runProtected(TestResult.java:142) at junit.framework.TestResult.run(TestResult.java:125) at junit.framework.TestCase.run(TestCase.java:129) at junit.framework.TestSuite.runTest(TestSuite.java:255) at junit.framework.TestSuite.run(TestSuite.java:250) at junit.framework.TestSuite.runTest(TestSuite.java:255) at junit.framework.TestSuite.run(TestSuite.java:250) at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84) at org.junit.runner.JUnitCore.run(JUnitCore.java:160) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68) at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47) at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242) at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70) [2018-10-22 17:08:30,902][ERROR][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][IgniteTestResources] Critical system error detected. Will be handled accordingly to configured handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler [ignoredFailureTypes=EmptySet []]], failureCtx=FailureContext [type=SYSTEM_WORKER_TERMINATION, err=class o.a.i.IgniteException: GridWorker [name=partition-exchanger, igniteInstanceName=persistence.PersistenceBasicCompatibilityTest0, finished=true, heartbeatTs=1540217310805]]] class org.apache.ignite.IgniteException: GridWorker [name=partition-exchanger, igniteInstanceName=persistence.PersistenceBasicCompatibilityTest0, finished=true, heartbeatTs=1540217310805] at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1831) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1826) at org.apache.ignite.internal.worker.WorkersRegistry.onStopped(WorkersRegistry.java:169) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:153) at java.lang.Thread.run(Thread.java:748) [2018-10-22 17:08:30,903][WARN ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][FailureProcessor] No deadlocked threads detected. [2018-10-22 17:08:30,916][WARN ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][FailureProcessor] Thread dump at 2018/10/22 17:08:30 MSK Thread [name="sys-#161%persistence.PersistenceBasicCompatibilityTest0%", id=208, state=TIMED_WAITING, blockCnt=0, waitCnt=2] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6ec7abc, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Thread [name="sys-#160%persistence.PersistenceBasicCompatibilityTest0%", id=207, state=TIMED_WAITING, blockCnt=0, waitCnt=2] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6ec7abc, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Thread [name="sys-#159%persistence.PersistenceBasicCompatibilityTest0%", id=206, state=TIMED_WAITING, blockCnt=0, waitCnt=1] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6ec7abc, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Thread [name="sys-#158%persistence.PersistenceBasicCompatibilityTest0%", id=205, state=TIMED_WAITING, blockCnt=0, waitCnt=2] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6ec7abc, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Thread [name="sys-#157%persistence.PersistenceBasicCompatibilityTest0%", id=204, state=TIMED_WAITING, blockCnt=0, waitCnt=1] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6ec7abc, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Thread [name="sys-#156%persistence.PersistenceBasicCompatibilityTest0%", id=203, state=TIMED_WAITING, blockCnt=0, waitCnt=2] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6ec7abc, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Thread [name="ttl-cleanup-worker-#155%persistence.PersistenceBasicCompatibilityTest0%", id=202, state=TIMED_WAITING, blockCnt=0, waitCnt=1] at java.lang.Thread.sleep(Native Method) at o.a.i.i.util.IgniteUtils.sleep(IgniteUtils.java:7778) at o.a.i.i.processors.cache.GridCacheSharedTtlCleanupManager$CleanupWorker.body(GridCacheSharedTtlCleanupManager.java:149) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="sys-#154%persistence.PersistenceBasicCompatibilityTest0%", id=201, state=TIMED_WAITING, blockCnt=0, waitCnt=2] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6ec7abc, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Thread [name="wal-segment-syncer-#153%persistence.PersistenceBasicCompatibilityTest0%", id=200, state=TIMED_WAITING, blockCnt=0, waitCnt=1] at java.lang.Thread.sleep(Native Method) at o.a.i.i.util.IgniteUtils.sleep(IgniteUtils.java:7778) at o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$WalSegmentSyncer.body(FileWriteAheadLogManager.java:3614) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="wal-file-archiver%persistence.PersistenceBasicCompatibilityTest0-#152%persistence.PersistenceBasicCompatibilityTest0%", id=199, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.FileDispatcherImpl.force0(Native Method) at sun.nio.ch.FileDispatcherImpl.force(FileDispatcherImpl.java:89) at sun.nio.ch.FileChannelImpl.force(FileChannelImpl.java:388) at o.a.i.i.processors.cache.persistence.file.RandomAccessFileIO.force(RandomAccessFileIO.java:88) at o.a.i.i.processors.cache.persistence.file.RandomAccessFileIO.force(RandomAccessFileIO.java:113) at o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileArchiver.archiveSegment(FileWriteAheadLogManager.java:1913) at o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileArchiver.body(FileWriteAheadLogManager.java:1783) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%", id=197, state=RUNNABLE, blockCnt=0, waitCnt=38] at sun.management.ThreadImpl.dumpThreads0(Native Method) at sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:454) at o.a.i.i.util.IgniteUtils.dumpThreads(IgniteUtils.java:1368) at o.a.i.i.processors.failure.FailureProcessor.process(FailureProcessor.java:140) - locked o.a.i.i.processors.failure.FailureProcessor@3792ce1 at o.a.i.i.processors.failure.FailureProcessor.process(FailureProcessor.java:116) at o.a.i.i.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1829) at o.a.i.i.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1826) at o.a.i.i.worker.WorkersRegistry.onStopped(WorkersRegistry.java:169) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:153) at java.lang.Thread.run(Thread.java:748) Thread [name="disco-event-worker-#150%persistence.PersistenceBasicCompatibilityTest0%", id=196, state=WAITING, blockCnt=0, waitCnt=4] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3a2d0702, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at o.a.i.i.managers.discovery.GridDiscoveryManager$DiscoveryWorker.body0(GridDiscoveryManager.java:2856) at o.a.i.i.managers.discovery.GridDiscoveryManager$DiscoveryWorker.body(GridDiscoveryManager.java:2825) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="tcp-disco-srvr-#12%persistence.PersistenceBasicCompatibilityTest0%", id=195, state=RUNNABLE, blockCnt=0, waitCnt=0] at java.net.DualStackPlainSocketImpl.accept0(Native Method) at java.net.DualStackPlainSocketImpl.socketAccept(DualStackPlainSocketImpl.java:131) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409) at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:199) - locked java.net.SocksSocketImpl@42ba2f76 at java.net.ServerSocket.implAccept(ServerSocket.java:545) at java.net.ServerSocket.accept(ServerSocket.java:513) at o.a.i.spi.discovery.tcp.ServerImpl$TcpServer.body(ServerImpl.java:5845) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at o.a.i.spi.discovery.tcp.ServerImpl$TcpServerThread.body(ServerImpl.java:5763) at o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:62) Thread [name="tcp-disco-msg-worker-#11%persistence.PersistenceBasicCompatibilityTest0%", id=194, state=TIMED_WAITING, blockCnt=0, waitCnt=944] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@27e68407, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522) at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684) at o.a.i.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7183) at o.a.i.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2700) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at o.a.i.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7119) at o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:62) Thread [name="disco-notifier-worker-#149%persistence.PersistenceBasicCompatibilityTest0%", id=193, state=WAITING, blockCnt=0, waitCnt=5] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6db2ba3e, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at o.a.i.i.managers.discovery.GridDiscoveryManager$DiscoveryMessageNotifierWorker.body0(GridDiscoveryManager.java:2660) at o.a.i.i.managers.discovery.GridDiscoveryManager$DiscoveryMessageNotifierWorker.body(GridDiscoveryManager.java:2705) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="grid-data-loader-flusher-#148%persistence.PersistenceBasicCompatibilityTest0%", id=192, state=WAITING, blockCnt=0, waitCnt=2] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3ff5b59d, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.DelayQueue.take(DelayQueue.java:211) at o.a.i.i.processors.datastreamer.DataStreamProcessor$2.body(DataStreamProcessor.java:109) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="grid-nio-worker-client-listener-3-#145%persistence.PersistenceBasicCompatibilityTest0%", id=189, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278) at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked o.a.i.i.util.nio.SelectedSelectionKeySet@2ca7ec8a - locked java.util.Collections$UnmodifiableSet@3f7283a5 - locked sun.nio.ch.WindowsSelectorImpl@47e9bd3f at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="grid-nio-worker-client-listener-2-#144%persistence.PersistenceBasicCompatibilityTest0%", id=188, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278) at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked o.a.i.i.util.nio.SelectedSelectionKeySet@5522b102 - locked java.util.Collections$UnmodifiableSet@4672d490 - locked sun.nio.ch.WindowsSelectorImpl@6d361850 at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="grid-nio-worker-client-listener-1-#143%persistence.PersistenceBasicCompatibilityTest0%", id=187, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278) at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked o.a.i.i.util.nio.SelectedSelectionKeySet@673c508c - locked java.util.Collections$UnmodifiableSet@275657c8 - locked sun.nio.ch.WindowsSelectorImpl@7750c1e6 at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="grid-nio-worker-client-listener-0-#142%persistence.PersistenceBasicCompatibilityTest0%", id=186, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278) at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked o.a.i.i.util.nio.SelectedSelectionKeySet@2a79a3c4 - locked java.util.Collections$UnmodifiableSet@2f443e37 - locked sun.nio.ch.WindowsSelectorImpl@3ab9c400 at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="nio-acceptor-client-listener-#146%persistence.PersistenceBasicCompatibilityTest0%", id=190, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278) at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked sun.nio.ch.Util$3@4b176d40 - locked java.util.Collections$UnmodifiableSet@2c179065 - locked sun.nio.ch.WindowsSelectorImpl@22acb656 at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at o.a.i.i.util.nio.GridNioServer$GridNioAcceptWorker.accept(GridNioServer.java:2929) at o.a.i.i.util.nio.GridNioServer$GridNioAcceptWorker.body(GridNioServer.java:2877) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="grid-sharedfs-timeout-worker-#10%persistence.PersistenceBasicCompatibilityTest0%", id=185, state=TIMED_WAITING, blockCnt=0, waitCnt=3] Lock [object=java.lang.Object@56c1ffc7, ownerName=null, ownerId=-1] at java.lang.Object.wait(Native Method) at o.a.i.spi.checkpoint.sharedfs.SharedFsTimeoutTask.body(SharedFsTimeoutTask.java:99) at o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:62) Thread [name="tcp-comm-worker-#9%persistence.PersistenceBasicCompatibilityTest0%", id=184, state=TIMED_WAITING, blockCnt=0, waitCnt=1] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@26cce532, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at o.a.i.spi.communication.tcp.TcpCommunicationSpi$CommunicationWorker.body(TcpCommunicationSpi.java:4287) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at o.a.i.spi.communication.tcp.TcpCommunicationSpi$5.body(TcpCommunicationSpi.java:2237) at o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:62) Thread [name="grid-nio-worker-tcp-comm-3-#140%persistence.PersistenceBasicCompatibilityTest0%", id=182, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278) at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked o.a.i.i.util.nio.SelectedSelectionKeySet@3648cd36 - locked java.util.Collections$UnmodifiableSet@7191d3af - locked sun.nio.ch.WindowsSelectorImpl@28f7359d at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="grid-nio-worker-tcp-comm-2-#139%persistence.PersistenceBasicCompatibilityTest0%", id=181, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278) at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked o.a.i.i.util.nio.SelectedSelectionKeySet@6953b0c4 - locked java.util.Collections$UnmodifiableSet@1902f1a5 - locked sun.nio.ch.WindowsSelectorImpl@66714581 at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="grid-nio-worker-tcp-comm-1-#138%persistence.PersistenceBasicCompatibilityTest0%", id=180, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278) at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked o.a.i.i.util.nio.SelectedSelectionKeySet@773bdf0f - locked java.util.Collections$UnmodifiableSet@39a84bc - locked sun.nio.ch.WindowsSelectorImpl@d9bc2b3 at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="grid-nio-worker-tcp-comm-0-#137%persistence.PersistenceBasicCompatibilityTest0%", id=179, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278) at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked o.a.i.i.util.nio.SelectedSelectionKeySet@61aad3cc - locked java.util.Collections$UnmodifiableSet@4840bb51 - locked sun.nio.ch.WindowsSelectorImpl@1fa86981 at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151) at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="nio-acceptor-tcp-comm-#141%persistence.PersistenceBasicCompatibilityTest0%", id=183, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278) at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked sun.nio.ch.Util$3@4cf5be9d - locked java.util.Collections$UnmodifiableSet@7bf6207c - locked sun.nio.ch.WindowsSelectorImpl@36219660 at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at o.a.i.i.util.nio.GridNioServer$GridNioAcceptWorker.accept(GridNioServer.java:2929) at o.a.i.i.util.nio.GridNioServer$GridNioAcceptWorker.body(GridNioServer.java:2877) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="grid-timeout-worker-#136%persistence.PersistenceBasicCompatibilityTest0%", id=178, state=TIMED_WAITING, blockCnt=1, waitCnt=14] Lock [object=java.lang.Object@6a990544, ownerName=null, ownerId=-1] at java.lang.Object.wait(Native Method) at o.a.i.i.processors.timeout.GridTimeoutProcessor$TimeoutWorker.body(GridTimeoutProcessor.java:269) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="pub-#135%persistence.PersistenceBasicCompatibilityTest0%", id=177, state=TIMED_WAITING, blockCnt=0, waitCnt=2] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@bf94fdc, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Thread [name="pub-#134%persistence.PersistenceBasicCompatibilityTest0%", id=176, state=TIMED_WAITING, blockCnt=0, waitCnt=2] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@bf94fdc, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Thread [name="pub-#133%persistence.PersistenceBasicCompatibilityTest0%", id=175, state=TIMED_WAITING, blockCnt=0, waitCnt=1] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@bf94fdc, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Thread [name="pub-#132%persistence.PersistenceBasicCompatibilityTest0%", id=174, state=TIMED_WAITING, blockCnt=0, waitCnt=1] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@bf94fdc, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Thread [name="pub-#131%persistence.PersistenceBasicCompatibilityTest0%", id=173, state=TIMED_WAITING, blockCnt=0, waitCnt=2] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@bf94fdc, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Thread [name="pub-#130%persistence.PersistenceBasicCompatibilityTest0%", id=172, state=TIMED_WAITING, blockCnt=0, waitCnt=1] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@bf94fdc, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Thread [name="jvm-pause-detector-worker", id=171, state=TIMED_WAITING, blockCnt=0, waitCnt=201] at java.lang.Thread.sleep(Native Method) at o.a.i.i.LongJVMPauseDetector$1.run(LongJVMPauseDetector.java:100) Thread [name="data-streamer-stripe-7-#129%persistence.PersistenceBasicCompatibilityTest0%", id=170, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="data-streamer-stripe-6-#128%persistence.PersistenceBasicCompatibilityTest0%", id=169, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="data-streamer-stripe-5-#127%persistence.PersistenceBasicCompatibilityTest0%", id=168, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="data-streamer-stripe-4-#126%persistence.PersistenceBasicCompatibilityTest0%", id=167, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="data-streamer-stripe-3-#125%persistence.PersistenceBasicCompatibilityTest0%", id=166, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="data-streamer-stripe-2-#124%persistence.PersistenceBasicCompatibilityTest0%", id=165, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="data-streamer-stripe-1-#123%persistence.PersistenceBasicCompatibilityTest0%", id=164, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="data-streamer-stripe-0-#122%persistence.PersistenceBasicCompatibilityTest0%", id=163, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="sys-stripe-7-#121%persistence.PersistenceBasicCompatibilityTest0%", id=162, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="sys-stripe-6-#120%persistence.PersistenceBasicCompatibilityTest0%", id=161, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="sys-stripe-5-#119%persistence.PersistenceBasicCompatibilityTest0%", id=160, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="sys-stripe-4-#118%persistence.PersistenceBasicCompatibilityTest0%", id=159, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="sys-stripe-3-#117%persistence.PersistenceBasicCompatibilityTest0%", id=158, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="sys-stripe-2-#116%persistence.PersistenceBasicCompatibilityTest0%", id=157, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="sys-stripe-1-#115%persistence.PersistenceBasicCompatibilityTest0%", id=156, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="sys-stripe-0-#114%persistence.PersistenceBasicCompatibilityTest0%", id=155, state=WAITING, blockCnt=0, waitCnt=2] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669) at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Thread [name="test-runner-#113%persistence.PersistenceBasicCompatibilityTest%", id=152, state=WAITING, blockCnt=2, waitCnt=7] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:178) at o.a.i.i.util.future.GridFutureAdapter.get(GridFutureAdapter.java:141) at o.a.i.i.cluster.IgniteClusterImpl.active(IgniteClusterImpl.java:312) at o.a.i.i.IgniteKernal.active(IgniteKernal.java:3597) at o.a.i.compatibility.persistence.PersistenceBasicCompatibilityTest.doTestStartupWithOldVersion(PersistenceBasicCompatibilityTest.java:121) at o.a.i.compatibility.persistence.PersistenceBasicCompatibilityTest.doTestStartupWithOldVersion(PersistenceBasicCompatibilityTest.java:139) at o.a.i.compatibility.persistence.PersistenceBasicCompatibilityTest.testNodeStartByOldVersionPersistenceData_2_3(PersistenceBasicCompatibilityTest.java:98) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at junit.framework.TestCase.runTest(TestCase.java:176) at o.a.i.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2176) at o.a.i.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:142) at o.a.i.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:2091) at java.lang.Thread.run(Thread.java:748) Thread [name="Thread-11", id=57, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method) at sun.nio.ch.Iocp.access$300(Iocp.java:46) at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333) at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Locked synchronizers: java.util.concurrent.ThreadPoolExecutor$Worker@4fa859e3 Thread [name="Thread-10", id=56, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method) at sun.nio.ch.Iocp.access$300(Iocp.java:46) at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333) at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Locked synchronizers: java.util.concurrent.ThreadPoolExecutor$Worker@7ffb57da Thread [name="Thread-9", id=55, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method) at sun.nio.ch.Iocp.access$300(Iocp.java:46) at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333) at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Locked synchronizers: java.util.concurrent.ThreadPoolExecutor$Worker@5c2099e1 Thread [name="Thread-8", id=54, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method) at sun.nio.ch.Iocp.access$300(Iocp.java:46) at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333) at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Locked synchronizers: java.util.concurrent.ThreadPoolExecutor$Worker@45efd3fb Thread [name="Thread-7", id=53, state=RUNNABLE, blockCnt=1, waitCnt=0] at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method) at sun.nio.ch.Iocp.access$300(Iocp.java:46) at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333) at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Locked synchronizers: java.util.concurrent.ThreadPoolExecutor$Worker@27642a4 Thread [name="Thread-6", id=52, state=RUNNABLE, blockCnt=2, waitCnt=0] at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method) at sun.nio.ch.Iocp.access$300(Iocp.java:46) at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333) at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Locked synchronizers: java.util.concurrent.ThreadPoolExecutor$Worker@2942d82 Thread [name="Thread-5", id=51, state=RUNNABLE, blockCnt=1, waitCnt=0] at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method) at sun.nio.ch.Iocp.access$300(Iocp.java:46) at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333) at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Locked synchronizers: java.util.concurrent.ThreadPoolExecutor$Worker@1b31ca3 Thread [name="Thread-4", id=50, state=RUNNABLE, blockCnt=1, waitCnt=0] at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method) at sun.nio.ch.Iocp.access$300(Iocp.java:46) at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333) at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Locked synchronizers: java.util.concurrent.ThreadPoolExecutor$Worker@66a42f4e Thread [name="Thread-3", id=49, state=RUNNABLE, blockCnt=0, waitCnt=0] at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method) at sun.nio.ch.Iocp.access$300(Iocp.java:46) at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333) at java.lang.Thread.run(Thread.java:748) Thread [name="ignite-clock-for-tests", id=12, state=TIMED_WAITING, blockCnt=0, waitCnt=7292] at java.lang.Thread.sleep(Native Method) at o.a.i.i.util.GridTestClockTimer.run(GridTestClockTimer.java:50) at java.lang.Thread.run(Thread.java:748) Thread [name="Monitor Ctrl-Break", id=6, state=RUNNABLE, blockCnt=1, waitCnt=0] at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284) at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326) at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178) - locked java.io.InputStreamReader@41466b1f at java.io.InputStreamReader.read(InputStreamReader.java:184) at java.io.BufferedReader.fill(BufferedReader.java:161) at java.io.BufferedReader.readLine(BufferedReader.java:324) - locked java.io.InputStreamReader@41466b1f at java.io.BufferedReader.readLine(BufferedReader.java:389) at com.intellij.rt.execution.application.AppMainV2$1.run(AppMainV2.java:64) Thread [name="Attach Listener", id=5, state=RUNNABLE, blockCnt=0, waitCnt=0] Thread [name="Signal Dispatcher", id=4, state=RUNNABLE, blockCnt=0, waitCnt=0] Thread [name="Finalizer", id=3, state=WAITING, blockCnt=62, waitCnt=6] Lock [object=java.lang.ref.ReferenceQueue$Lock@7b6e77c1, ownerName=null, ownerId=-1] at java.lang.Object.wait(Native Method) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:216) Thread [name="Reference Handler", id=2, state=WAITING, blockCnt=6, waitCnt=5] Lock [object=java.lang.ref.Reference$Lock@59ffeecd, ownerName=null, ownerId=-1] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at java.lang.ref.Reference.tryHandlePending(Reference.java:191) at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153) Thread [name="main", id=1, state=TIMED_WAITING, blockCnt=2, waitCnt=3] Lock [object=o.a.i.thread.IgniteThread@1839cd81, ownerName=null, ownerId=-1] at java.lang.Object.wait(Native Method) at java.lang.Thread.join(Thread.java:1260) at o.a.i.testframework.junits.GridAbstractTest.runTest(GridAbstractTest.java:2103) at junit.framework.TestCase.runBare(TestCase.java:141) at junit.framework.TestResult$1.protect(TestResult.java:122) at junit.framework.TestResult.runProtected(TestResult.java:142) at junit.framework.TestResult.run(TestResult.java:125) at junit.framework.TestCase.run(TestCase.java:129) at junit.framework.TestSuite.runTest(TestSuite.java:255) at junit.framework.TestSuite.run(TestSuite.java:250) at junit.framework.TestSuite.runTest(TestSuite.java:255) at junit.framework.TestSuite.run(TestSuite.java:250) at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84) at org.junit.runner.JUnitCore.run(JUnitCore.java:160) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68) at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47) at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242) at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70) [2018-10-22 17:08:30,926][INFO ][wal-file-archiver%persistence.PersistenceBasicCompatibilityTest0-#152%persistence.PersistenceBasicCompatibilityTest0%][FileWriteAheadLogManager] Copied file [src=C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal, dst=C:\IdeaProjects\apache-ignite\work\db\wal\archive\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal] Process finished with exit code -1{noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005) |
«
Return to Apache Ignite Developers - Legacy Mail Archive
|
1 view|%1 views
| Free forum by Nabble | Edit this page |