Ignite-1.5 Release
Locked
123456
123456
 
|
Would be nice to get an up to date picture. At this point this thread has
gotten unwieldy. Yakov, would it be worthwhile start a new thread will all the tasks listed again? D. On Mon, Nov 16, 2015 at 1:55 PM, Alexey Goncharuk < [hidden email]> wrote: > Igniters, > > I am finalizing my optimizations in tx cache (currently there are a couple > of failing tests on CI), I am expecting these changes to be merged in > ignite-1.5 branch tomorrow. > > 2015-11-16 19:23 GMT+03:00 Yakov Zhdanov <[hidden email]>: > > > Guys, > > > > Can everyone provide updates on ongoing issues status? > > > > Vlad, I will review Semaphore tomorrow (Russian time). > > > > Thanks! > > > > --Yakov > > > > 2015-11-14 15:38 GMT+03:00 Denis Magda <[hidden email]>: > > > > > Hi Vladislav, > > > > > > Most likely there is a minor issue in your test. > > > > > > I think Yakov would be able to run and check the test as a part of the > > > ongoing review. > > > > > > Thanks, > > > > > > Denis > > > > > > On Friday, November 13, 2015, Vladisav Jelisavcic <[hidden email] > > > > > wrote: > > > > > > > Hi Denis, > > > > > > > > Thanks a lot, it looks like my test setup was wrong, > > > > I added semaphore tests to > > > GridCacheAbstractDataStructuresFailoverSelfTest > > > > suite. > > > > Now I have following problem: > > > > when I run tests with TOP_CHANGE_THREAD_CNT = 3 > > > > tests fail when they reach stop() with the following exception: > > > > > > > > class org.apache.ignite.internal.IgniteInterruptedCheckedException: > > Node > > > is > > > > stopping: 09c5e8b8-8998-468e-960d-223220354fd3 > > > > at > > > > > > > > > > > > > > org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.onKernalStop0(GridCachePartitionExchangeManager.java:382) > > > > at > > > > > > > > > > > > > > org.apache.ignite.internal.processors.cache.GridCacheSharedManagerAdapter.onKernalStop(GridCacheSharedManagerAdapter.java:113) > > > > at > > > > > > > > > > > > > > org.apache.ignite.internal.processors.cache.GridCacheProcessor.onKernalStop(GridCacheProcessor.java:946) > > > > at > > org.apache.ignite.internal.IgniteKernal.stop0(IgniteKernal.java:1823) > > > > at > org.apache.ignite.internal.IgniteKernal.stop(IgniteKernal.java:1769) > > > > at > > > > > > > > > > > > > > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2133) > > > > at > > > > > > > > > > > > > > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2096) > > > > at org.apache.ignite.internal.IgnitionEx.stop(IgnitionEx.java:314) > > > > at org.apache.ignite.Ignition.stop(Ignition.java:223) > > > > at > > > > > > > > > > > > > > org.apache.ignite.testframework.junits.GridAbstractTest.stopGrid(GridAbstractTest.java:802) > > > > at > > > > > > > > > > > > > > org.apache.ignite.testframework.junits.GridAbstractTest.stopGrid(GridAbstractTest.java:784) > > > > at > > > > > > > > > > > > > > org.apache.ignite.internal.processors.cache.datastructures.GridCacheAbstractDataStructuresFailoverSelfTest.access$500(GridCacheAbstractDataStructuresFailoverSelfTest.java:54) > > > > at > > > > > > > > > > > > > > org.apache.ignite.internal.processors.cache.datastructures.GridCacheAbstractDataStructuresFailoverSelfTest$5.apply(GridCacheAbstractDataStructuresFailoverSelfTest.java:459) > > > > > > > > When I run tests with TOP_CHANGE_THEAD_CNT = 1 > > > > everything is running ok; > > > > > > > > > > > > @Yakov > > > > I made a new commit to my IGNITE-638 branch, > > > > can you please take a look? > > > > > > > > > > > > > > > > Best regards, > > > > Vladisav > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 11, 2015 at 3:48 PM, Denis Magda <[hidden email] > > > > <javascript:;>> wrote: > > > > > > > > > > > Hi Vladislav, > > > > > > > > > > > > Please see below.. > > > > > > > > > > > > > > > > > > On 11/11/2015 12:33 PM, Vladisav Jelisavcic wrote: > > > > > > > > > > > >> Yakov, > > > > > >> > > > > > >> sorry for running a bit late. > > > > > >> > > > > > >> Vladislav, do you have any updates for > > > > > >>> https://issues.apache.org/jira/browse/IGNITE-638? Or any > > > questions? > > > > > >>> > > > > > >>> --Yakov > > > > > >>> > > > > > >> I have problems with some fail-over scenarios; > > > > > >> It seems that if the two nodes are in the middle of acquiring or > > > > > releasing > > > > > >> the semaphore, > > > > > >> and one of them fails, all nodes get: > > > > > >> > > > > > >> > > [09:36:38,509][ERROR][ignite-#13%pub-null%][GridCacheSemaphoreImpl] > > > > > >> <ignite-atomics-sys-cache> Failed to compare and set: > > > > > >> > > > > > > o.a.i.i.processors.datastructures.GridCacheSemaphoreImpl$Sync$1@5528b728 > > > > > >> class > > > > > org.apache.ignite.internal.cluster.ClusterTopologyCheckedException: > > > > > >> Failed to acquire lock for keys (primary node left grid, retry > > > > > transaction > > > > > >> if possible) [keys=[UserKeyCacheObjectImpl > > > > [val=GridCacheInternalKeyImpl > > > > > >> [name=ac83b8cb-3052-49a6-9301-81b20b0ecf3a], hasValBytes=true]], > > > > > >> node=c321fcc4-5db5-4b03-9811-6a5587f2c253] > > > > > >> ... > > > > > >> Caused by: class > > > > > >> > > org.apache.ignite.internal.cluster.ClusterTopologyCheckedException: > > > > > Failed > > > > > >> to acquire lock for keys (primary node left grid, retry > > transaction > > > if > > > > > >> possible) [keys=[UserKeyCacheObjectImpl > > > [val=GridCacheInternalKeyImpl > > > > > >> [name=ac83b8cb-3052-49a6-9301-81b20b0ecf3a], hasValBytes=true]], > > > > > >> node=c321fcc4-5db5-4b03-9811-6a5587f2c253] > > > > > >> at > > > > > >> > > > > > >> > > > > > > > > > > > > > > > org.apache.ignite.internal.processors.cache.distributed.dht.colocated.GridDhtColocatedLockFuture.newTopologyException(GridDhtColocatedLockFuture.java:1199) > > > > > >> ... 10 more > > > > > >> > > > > > > You have to process this exception manually at your > implementation > > > > layer > > > > > > since your data structure uses a transactional cache. > > > > > > Below is a kind of template I used when it was required to > process > > > this > > > > > > and some other exeptions. You can use it as-is. > > > > > > > > > > > > int retries = GridCacheAdapter.MAX_RETRIES; > > > > > > > > > > > > IgniteCheckedException err =null; > > > > > > > > > > > > for (int i =0; i < retries; i++) { > > > > > > try { > > > > > > //Your transactional code that may fail > > > > > > } > > > > > > catch (IgniteCheckedException e) { > > > > > > if (i == retries) > > > > > > throw e; > > > > > > > > > > > > if (X.hasCause(e, > ClusterTopologyCheckedException.class)) { > > > > > > ClusterTopologyCheckedException topErr = > > > > > > e.getCause(ClusterTopologyCheckedException.class); > > > > > > > > > > > > topErr.retryReadyFuture().get(); > > > > > > } > > > > > > else if (X.hasCause(e, > > > IgniteTxRollbackCheckedException.class)) > > > > > > U.sleep(1); > > > > > > else throw e; > > > > > > } > > > > > > } > > > > > > > > > > > > > > > > > > > > > > > >> I'm still trying to find out how to exactly reproduce this > > behavior, > > > > > >> I'll send you more details once I try few more things. > > > > > >> > > > > > > There is the test suite called > > > > > > GridCacheAbstractDataStructuresFailoverSelfTest that checks > Ignite > > > > > atomics > > > > > > and data structures with fail-over scenario. > > > > > > The suite will let you reproduce ClusterTopologyCheckedException > > > > easily. > > > > > > Just add your tests there referring to the tests of other data > > > > > structures. > > > > > > > > > > > > Presently I'm improving this test suite under my work on > IGNITE-801 > > > and > > > > > > IGNITE-803. If you finish your task earlier then I'll adopt your > > > tests > > > > > to a > > > > > > new test approach. > > > > > > > > > > > > > > > > > >> I am still using partitioned cache, does it make sense to use > > > > replicated > > > > > >> cache instead? > > > > > >> > > > > > >> Yeah, you should support this as well. Cache mode for the data > > > > > structures > > > > > > is changed using CollectionConfigurations while for atomics using > > > > > > AtomicsConfiguration. > > > > > > > > > > > > -- > > > > > > Denis > > > > > > > > > > > > > > > > > > Other than that, I'm done with everything else. > > > > > >> > > > > > >> Thanks, > > > > > >> Vladisav > > > > > >> > > > > > >> > > > > > >> > > > > > >> On Tue, Nov 10, 2015 at 7:19 PM, Raul Kripalani < > [hidden email] > > > > <javascript:;>> > > > > > wrote: > > > > > >> > > > > > >> Sorry I haven't made an appearance in this thread yet. > > > > > >>> > > > > > >>> 6. MQTT streamer > > > > > >>>> https://issues.apache.org/jira/browse/IGNITE-535 > > > > > >>>> > > > > > >>> Yes, it was merged to master before the ignite-1.5 was created. > > > > > >>> > > > > > >>> I'd like to add: > > > > > >>> > > > > > >>> Camel Streamer => > > > https://issues.apache.org/jira/browse/IGNITE-1790 > > > > > >>> -- I'll merge this as soon as I finished with the OSGi tickets > > with > > > > > >>> demand. > > > > > >>> > > > > > >>> OSGi Manifests, Karaf features and possible ClassLoaderCodec > SPI > > > (or > > > > > >>> whatever agreement we arrive to in mailing lists and Wiki) > > > > > >>> -- https://issues.apache.org/jira/browse/IGNITE-1527 > > > > > >>> -- https://issues.apache.org/jira/browse/IGNITE-1877 > > > > > >>> -- I'm working actively on these two features. > > > > > >>> > > > > > >>> *Raúl Kripalani* > > > > > >>> PMC & Committer @ Apache Ignite, Apache Camel | Integration, > Big > > > Data > > > > > and > > > > > >>> Messaging Engineer > > > > > >>> http://about.me/raulkripalani | > > > > > http://www.linkedin.com/in/raulkripalani > > > > > >>> http://blog.raulkr.net | twitter: @raulvk > > > > > >>> > > > > > >>> On Mon, Nov 2, 2015 at 1:35 PM, Yakov Zhdanov < > > [hidden email] > > > > <javascript:;>> > > > > > >>> wrote: > > > > > >>> > > > > > >>> Guys, > > > > > >>>> > > > > > >>>> I think we can start preparation to Ignite-1.5 release which > > will > > > > > >>>> include > > > > > >>>> many interesting features: > > > > > >>>> > > > > > >>>> 1. Portable object API > > > > > >>>> https://issues.apache.org/jira/browse/IGNITE-1486 > > > > > >>>> > > > > > >>>> 2. Ignite.NET and Ignite C++ > > > > > >>>> https://issues.apache.org/jira/browse/IGNITE-1282 > > > > > >>>> > > > > > >>>> 3. Optimistic serializable transactions > > > > > >>>> https://issues.apache.org/jira/browse/IGNITE-1607 > > > > > >>>> > > > > > >>>> 4. Distributed SQL joins - we will be able to query > > non-collocated > > > > > data > > > > > >>>> > > > > > >>> as > > > > > >>> > > > > > >>>> well > > > > > >>>> https://issues.apache.org/jira/browse/IGNITE-1232 > > > > > >>>> > > > > > >>>> 5. Enhanced Oracle and IBM JDK interoperability > > > > > >>>> https://issues.apache.org/jira/browse/IGNITE-1526 > > > > > >>>> > > > > > >>>> 6. MQTT streamer > > > > > >>>> https://issues.apache.org/jira/browse/IGNITE-535 > > > > > >>>> > > > > > >>>> 7. Continuous query failover > > > > > >>>> https://issues.apache.org/jira/browse/IGNITE-426 > > > > > >>>> > > > > > >>>> 8. Significant transactional cache performance optimizations > - I > > > > will > > > > > >>>> > > > > > >>> merge > > > > > >>> > > > > > >>>> these changes from 'ignite-1.4-slow-server-debug' today or > > > tomorrow. > > > > > >>>> > > > > > >>>> 9. Many stability and fault-tolerance fixes. > > > > > >>>> > > > > > >>>> 10. I would also like to include distributed Semaphore. > > Vladislav, > > > > any > > > > > >>>> chance you can finish with it this week? > > > > > >>>> https://issues.apache.org/jira/browse/IGNITE- > > > > > >>>> <https://issues.apache.org/jira/browse/IGNITE-426>638 > > > > > >>>> > > > > > >>>> Thanks to everyone involved! Guys, esp. assignees of mentioned > > > > issues, > > > > > >>>> please respond to this email and let us know when can we > expect > > > your > > > > > >>>> changes being merged to master and release branch? > > > > > >>>> > > > > > >>>> Can someone create ignite-1.5 release branch? > > > > > >>>> > > > > > >>>> --Yakov > > > > > >>>> > > > > > >>>> > > > > > > > > > > > > > > > > > > > > > |
|
Folks,
I made several enhancements in our direct marshalling protocol that will reduce the amount of data transferred over the network. There are two main changes: - Optimized encoding of integers and longs. Now small numbers (0-127) consume only one byte instead of four, greater number can consume 2, 3 or more bytes (as many as needed only). - Null objects are always written as one byte. Additionally there are several fixes that should reduce the number of created object during marshalling/unmarshalling. I ran a simple test with tx puts and amount of bytes sent over the network is around 1.5 times less than in current ignite-1.5. Yakov, Alex, can you please review my changes in ignite-direct-marsh-opt branch and check the performance? -Val On Mon, Nov 16, 2015 at 2:04 PM, Dmitriy Setrakyan <[hidden email]> wrote: > Would be nice to get an up to date picture. At this point this thread has > gotten unwieldy. > > Yakov, would it be worthwhile start a new thread will all the tasks listed > again? > > D. > > On Mon, Nov 16, 2015 at 1:55 PM, Alexey Goncharuk < > [hidden email]> wrote: > > > Igniters, > > > > I am finalizing my optimizations in tx cache (currently there are a > couple > > of failing tests on CI), I am expecting these changes to be merged in > > ignite-1.5 branch tomorrow. > > > > 2015-11-16 19:23 GMT+03:00 Yakov Zhdanov <[hidden email]>: > > > > > Guys, > > > > > > Can everyone provide updates on ongoing issues status? > > > > > > Vlad, I will review Semaphore tomorrow (Russian time). > > > > > > Thanks! > > > > > > --Yakov > > > > > > 2015-11-14 15:38 GMT+03:00 Denis Magda <[hidden email]>: > > > > > > > Hi Vladislav, > > > > > > > > Most likely there is a minor issue in your test. > > > > > > > > I think Yakov would be able to run and check the test as a part of > the > > > > ongoing review. > > > > > > > > Thanks, > > > > > > > > Denis > > > > > > > > On Friday, November 13, 2015, Vladisav Jelisavcic < > [hidden email] > > > > > > > wrote: > > > > > > > > > Hi Denis, > > > > > > > > > > Thanks a lot, it looks like my test setup was wrong, > > > > > I added semaphore tests to > > > > GridCacheAbstractDataStructuresFailoverSelfTest > > > > > suite. > > > > > Now I have following problem: > > > > > when I run tests with TOP_CHANGE_THREAD_CNT = 3 > > > > > tests fail when they reach stop() with the following exception: > > > > > > > > > > class org.apache.ignite.internal.IgniteInterruptedCheckedException: > > > Node > > > > is > > > > > stopping: 09c5e8b8-8998-468e-960d-223220354fd3 > > > > > at > > > > > > > > > > > > > > > > > > > > org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.onKernalStop0(GridCachePartitionExchangeManager.java:382) > > > > > at > > > > > > > > > > > > > > > > > > > > org.apache.ignite.internal.processors.cache.GridCacheSharedManagerAdapter.onKernalStop(GridCacheSharedManagerAdapter.java:113) > > > > > at > > > > > > > > > > > > > > > > > > > > org.apache.ignite.internal.processors.cache.GridCacheProcessor.onKernalStop(GridCacheProcessor.java:946) > > > > > at > > > org.apache.ignite.internal.IgniteKernal.stop0(IgniteKernal.java:1823) > > > > > at > > org.apache.ignite.internal.IgniteKernal.stop(IgniteKernal.java:1769) > > > > > at > > > > > > > > > > > > > > > > > > > > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2133) > > > > > at > > > > > > > > > > > > > > > > > > > > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2096) > > > > > at org.apache.ignite.internal.IgnitionEx.stop(IgnitionEx.java:314) > > > > > at org.apache.ignite.Ignition.stop(Ignition.java:223) > > > > > at > > > > > > > > > > > > > > > > > > > > org.apache.ignite.testframework.junits.GridAbstractTest.stopGrid(GridAbstractTest.java:802) > > > > > at > > > > > > > > > > > > > > > > > > > > org.apache.ignite.testframework.junits.GridAbstractTest.stopGrid(GridAbstractTest.java:784) > > > > > at > > > > > > > > > > > > > > > > > > > > org.apache.ignite.internal.processors.cache.datastructures.GridCacheAbstractDataStructuresFailoverSelfTest.access$500(GridCacheAbstractDataStructuresFailoverSelfTest.java:54) > > > > > at > > > > > > > > > > > > > > > > > > > > org.apache.ignite.internal.processors.cache.datastructures.GridCacheAbstractDataStructuresFailoverSelfTest$5.apply(GridCacheAbstractDataStructuresFailoverSelfTest.java:459) > > > > > > > > > > When I run tests with TOP_CHANGE_THEAD_CNT = 1 > > > > > everything is running ok; > > > > > > > > > > > > > > > @Yakov > > > > > I made a new commit to my IGNITE-638 branch, > > > > > can you please take a look? > > > > > > > > > > > > > > > > > > > > Best regards, > > > > > Vladisav > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 11, 2015 at 3:48 PM, Denis Magda < > [hidden email] > > > > > <javascript:;>> wrote: > > > > > > > > > > > > > Hi Vladislav, > > > > > > > > > > > > > > Please see below.. > > > > > > > > > > > > > > > > > > > > > On 11/11/2015 12:33 PM, Vladisav Jelisavcic wrote: > > > > > > > > > > > > > >> Yakov, > > > > > > >> > > > > > > >> sorry for running a bit late. > > > > > > >> > > > > > > >> Vladislav, do you have any updates for > > > > > > >>> https://issues.apache.org/jira/browse/IGNITE-638? Or any > > > > questions? > > > > > > >>> > > > > > > >>> --Yakov > > > > > > >>> > > > > > > >> I have problems with some fail-over scenarios; > > > > > > >> It seems that if the two nodes are in the middle of acquiring > or > > > > > > releasing > > > > > > >> the semaphore, > > > > > > >> and one of them fails, all nodes get: > > > > > > >> > > > > > > >> > > > [09:36:38,509][ERROR][ignite-#13%pub-null%][GridCacheSemaphoreImpl] > > > > > > >> <ignite-atomics-sys-cache> Failed to compare and set: > > > > > > >> > > > > > > > > > o.a.i.i.processors.datastructures.GridCacheSemaphoreImpl$Sync$1@5528b728 > > > > > > >> class > > > > > > > org.apache.ignite.internal.cluster.ClusterTopologyCheckedException: > > > > > > >> Failed to acquire lock for keys (primary node left grid, retry > > > > > > transaction > > > > > > >> if possible) [keys=[UserKeyCacheObjectImpl > > > > > [val=GridCacheInternalKeyImpl > > > > > > >> [name=ac83b8cb-3052-49a6-9301-81b20b0ecf3a], > hasValBytes=true]], > > > > > > >> node=c321fcc4-5db5-4b03-9811-6a5587f2c253] > > > > > > >> ... > > > > > > >> Caused by: class > > > > > > >> > > > org.apache.ignite.internal.cluster.ClusterTopologyCheckedException: > > > > > > Failed > > > > > > >> to acquire lock for keys (primary node left grid, retry > > > transaction > > > > if > > > > > > >> possible) [keys=[UserKeyCacheObjectImpl > > > > [val=GridCacheInternalKeyImpl > > > > > > >> [name=ac83b8cb-3052-49a6-9301-81b20b0ecf3a], > hasValBytes=true]], > > > > > > >> node=c321fcc4-5db5-4b03-9811-6a5587f2c253] > > > > > > >> at > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > > > > > > > org.apache.ignite.internal.processors.cache.distributed.dht.colocated.GridDhtColocatedLockFuture.newTopologyException(GridDhtColocatedLockFuture.java:1199) > > > > > > >> ... 10 more > > > > > > >> > > > > > > > You have to process this exception manually at your > > implementation > > > > > layer > > > > > > > since your data structure uses a transactional cache. > > > > > > > Below is a kind of template I used when it was required to > > process > > > > this > > > > > > > and some other exeptions. You can use it as-is. > > > > > > > > > > > > > > int retries = GridCacheAdapter.MAX_RETRIES; > > > > > > > > > > > > > > IgniteCheckedException err =null; > > > > > > > > > > > > > > for (int i =0; i < retries; i++) { > > > > > > > try { > > > > > > > //Your transactional code that may fail > > > > > > > } > > > > > > > catch (IgniteCheckedException e) { > > > > > > > if (i == retries) > > > > > > > throw e; > > > > > > > > > > > > > > if (X.hasCause(e, > > ClusterTopologyCheckedException.class)) { > > > > > > > ClusterTopologyCheckedException topErr = > > > > > > > e.getCause(ClusterTopologyCheckedException.class); > > > > > > > > > > > > > > topErr.retryReadyFuture().get(); > > > > > > > } > > > > > > > else if (X.hasCause(e, > > > > IgniteTxRollbackCheckedException.class)) > > > > > > > U.sleep(1); > > > > > > > else throw e; > > > > > > > } > > > > > > > } > > > > > > > > > > > > > > > > > > > > > > > > > > > >> I'm still trying to find out how to exactly reproduce this > > > behavior, > > > > > > >> I'll send you more details once I try few more things. > > > > > > >> > > > > > > > There is the test suite called > > > > > > > GridCacheAbstractDataStructuresFailoverSelfTest that checks > > Ignite > > > > > > atomics > > > > > > > and data structures with fail-over scenario. > > > > > > > The suite will let you reproduce > ClusterTopologyCheckedException > > > > > easily. > > > > > > > Just add your tests there referring to the tests of other data > > > > > > structures. > > > > > > > > > > > > > > Presently I'm improving this test suite under my work on > > IGNITE-801 > > > > and > > > > > > > IGNITE-803. If you finish your task earlier then I'll adopt > your > > > > tests > > > > > > to a > > > > > > > new test approach. > > > > > > > > > > > > > > > > > > > > >> I am still using partitioned cache, does it make sense to use > > > > > replicated > > > > > > >> cache instead? > > > > > > >> > > > > > > >> Yeah, you should support this as well. Cache mode for the data > > > > > > structures > > > > > > > is changed using CollectionConfigurations while for atomics > using > > > > > > > AtomicsConfiguration. > > > > > > > > > > > > > > -- > > > > > > > Denis > > > > > > > > > > > > > > > > > > > > > Other than that, I'm done with everything else. > > > > > > >> > > > > > > >> Thanks, > > > > > > >> Vladisav > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> On Tue, Nov 10, 2015 at 7:19 PM, Raul Kripalani < > > [hidden email] > > > > > <javascript:;>> > > > > > > wrote: > > > > > > >> > > > > > > >> Sorry I haven't made an appearance in this thread yet. > > > > > > >>> > > > > > > >>> 6. MQTT streamer > > > > > > >>>> https://issues.apache.org/jira/browse/IGNITE-535 > > > > > > >>>> > > > > > > >>> Yes, it was merged to master before the ignite-1.5 was > created. > > > > > > >>> > > > > > > >>> I'd like to add: > > > > > > >>> > > > > > > >>> Camel Streamer => > > > > https://issues.apache.org/jira/browse/IGNITE-1790 > > > > > > >>> -- I'll merge this as soon as I finished with the OSGi > tickets > > > with > > > > > > >>> demand. > > > > > > >>> > > > > > > >>> OSGi Manifests, Karaf features and possible ClassLoaderCodec > > SPI > > > > (or > > > > > > >>> whatever agreement we arrive to in mailing lists and Wiki) > > > > > > >>> -- https://issues.apache.org/jira/browse/IGNITE-1527 > > > > > > >>> -- https://issues.apache.org/jira/browse/IGNITE-1877 > > > > > > >>> -- I'm working actively on these two features. > > > > > > >>> > > > > > > >>> *Raúl Kripalani* > > > > > > >>> PMC & Committer @ Apache Ignite, Apache Camel | Integration, > > Big > > > > Data > > > > > > and > > > > > > >>> Messaging Engineer > > > > > > >>> http://about.me/raulkripalani | > > > > > > http://www.linkedin.com/in/raulkripalani > > > > > > >>> http://blog.raulkr.net | twitter: @raulvk > > > > > > >>> > > > > > > >>> On Mon, Nov 2, 2015 at 1:35 PM, Yakov Zhdanov < > > > [hidden email] > > > > > <javascript:;>> > > > > > > >>> wrote: > > > > > > >>> > > > > > > >>> Guys, > > > > > > >>>> > > > > > > >>>> I think we can start preparation to Ignite-1.5 release which > > > will > > > > > > >>>> include > > > > > > >>>> many interesting features: > > > > > > >>>> > > > > > > >>>> 1. Portable object API > > > > > > >>>> https://issues.apache.org/jira/browse/IGNITE-1486 > > > > > > >>>> > > > > > > >>>> 2. Ignite.NET and Ignite C++ > > > > > > >>>> https://issues.apache.org/jira/browse/IGNITE-1282 > > > > > > >>>> > > > > > > >>>> 3. Optimistic serializable transactions > > > > > > >>>> https://issues.apache.org/jira/browse/IGNITE-1607 > > > > > > >>>> > > > > > > >>>> 4. Distributed SQL joins - we will be able to query > > > non-collocated > > > > > > data > > > > > > >>>> > > > > > > >>> as > > > > > > >>> > > > > > > >>>> well > > > > > > >>>> https://issues.apache.org/jira/browse/IGNITE-1232 > > > > > > >>>> > > > > > > >>>> 5. Enhanced Oracle and IBM JDK interoperability > > > > > > >>>> https://issues.apache.org/jira/browse/IGNITE-1526 > > > > > > >>>> > > > > > > >>>> 6. MQTT streamer > > > > > > >>>> https://issues.apache.org/jira/browse/IGNITE-535 > > > > > > >>>> > > > > > > >>>> 7. Continuous query failover > > > > > > >>>> https://issues.apache.org/jira/browse/IGNITE-426 > > > > > > >>>> > > > > > > >>>> 8. Significant transactional cache performance optimizations > > - I > > > > > will > > > > > > >>>> > > > > > > >>> merge > > > > > > >>> > > > > > > >>>> these changes from 'ignite-1.4-slow-server-debug' today or > > > > tomorrow. > > > > > > >>>> > > > > > > >>>> 9. Many stability and fault-tolerance fixes. > > > > > > >>>> > > > > > > >>>> 10. I would also like to include distributed Semaphore. > > > Vladislav, > > > > > any > > > > > > >>>> chance you can finish with it this week? > > > > > > >>>> https://issues.apache.org/jira/browse/IGNITE- > > > > > > >>>> <https://issues.apache.org/jira/browse/IGNITE-426>638 > > > > > > >>>> > > > > > > >>>> Thanks to everyone involved! Guys, esp. assignees of > mentioned > > > > > issues, > > > > > > >>>> please respond to this email and let us know when can we > > expect > > > > your > > > > > > >>>> changes being merged to master and release branch? > > > > > > >>>> > > > > > > >>>> Can someone create ignite-1.5 release branch? > > > > > > >>>> > > > > > > >>>> --Yakov > > > > > > >>>> > > > > > > >>>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > |
|
|
In reply to this post by Vladisav Jelisavcic
Vladislav,
I started to review the latest changes and have couple of questions: 1. latest changes are here - https://github.com/apache/ignite/pull/120? Is that correct? 2. org.apache.ignite.internal.processors.datastructures.GridCacheSemaphoreImpl.Sync#nodeMap is accessed in both sync and unsync context. Are you sure this is fine. 3. As far as failing test - can you please isolate it into separate junit or point out existing one? --Yakov 2015-11-11 12:33 GMT+03:00 Vladisav Jelisavcic <[hidden email]>: > Yakov, > > sorry for running a bit late. > > > Vladislav, do you have any updates for > > https://issues.apache.org/jira/browse/IGNITE-638? Or any questions? > > > > --Yakov > > I have problems with some fail-over scenarios; > It seems that if the two nodes are in the middle of acquiring or releasing > the semaphore, > and one of them fails, all nodes get: > > [09:36:38,509][ERROR][ignite-#13%pub-null%][GridCacheSemaphoreImpl] > <ignite-atomics-sys-cache> Failed to compare and set: > o.a.i.i.processors.datastructures.GridCacheSemaphoreImpl$Sync$1@5528b728 > class org.apache.ignite.internal.cluster.ClusterTopologyCheckedException: > Failed to acquire lock for keys (primary node left grid, retry transaction > if possible) [keys=[UserKeyCacheObjectImpl [val=GridCacheInternalKeyImpl > [name=ac83b8cb-3052-49a6-9301-81b20b0ecf3a], hasValBytes=true]], > node=c321fcc4-5db5-4b03-9811-6a5587f2c253] > ... > Caused by: class > org.apache.ignite.internal.cluster.ClusterTopologyCheckedException: Failed > to acquire lock for keys (primary node left grid, retry transaction if > possible) [keys=[UserKeyCacheObjectImpl [val=GridCacheInternalKeyImpl > [name=ac83b8cb-3052-49a6-9301-81b20b0ecf3a], hasValBytes=true]], > node=c321fcc4-5db5-4b03-9811-6a5587f2c253] > at > > org.apache.ignite.internal.processors.cache.distributed.dht.colocated.GridDhtColocatedLockFuture.newTopologyException(GridDhtColocatedLockFuture.java:1199) > ... 10 more > > > I'm still trying to find out how to exactly reproduce this behavior, > I'll send you more details once I try few more things. > > I am still using partitioned cache, does it make sense to use replicated > cache instead? > > > Other than that, I'm done with everything else. > > Thanks, > Vladisav > > |
|
|
Hi Yakov,
1. Yes 2. if you mean that nodeMap is accessed in onNodeRemoved(UUID nodeID) method of the GridCacheSemaphoreImpl class, it shouldn't be a problem, but it can be changed easily not to do so; 3. org.apache.ignite.internal.processors.cache.datastructures.GridCacheAbstractDataStructuresFailoverSelfTest#testSemaphoreConstantTopologyChangeFailoverSafe() org.apache.ignite.internal.processors.cache.datastructures.GridCacheAbstractDataStructuresFailoverSelfTest#testSemaphoreConstantMultipleTopologyChangeFailoverSafe() I think the problem is with the atomicity of the simulated grid failure; once stopGrid() is called for a node, other threads on this same node start throwing interrupted exceptions, which are in turn not handled properly in the GridCacheAbstractDataStructuresFailoverSelfTest; Those exceptions shouldn't be dealt with inside the GridCacheSemaphoreImpl itself. In a realworld node failure scenario, all those threads would fail at the same time (none of them would influence the rest of the grid anymore); I think fixing the issue Denis is working on can fix this (IGNITE-801 and IGNITE-803) Am i right? Does it makes sense? Best regards, Vladisav On Tue, Nov 17, 2015 at 5:40 PM, Yakov Zhdanov <[hidden email]> wrote: > Vladislav, > > I started to review the latest changes and have couple of questions: > > 1. latest changes are here - https://github.com/apache/ignite/pull/120? Is > that correct? > 2. > org.apache.ignite.internal.processors.datastructures.GridCacheSemaphoreImpl.Sync#nodeMap > is accessed in both sync and unsync context. Are you sure this is fine. > 3. As far as failing test - can you please isolate it into separate junit > or point out existing one? > > --Yakov > > 2015-11-11 12:33 GMT+03:00 Vladisav Jelisavcic <[hidden email]>: > > > Yakov, > > > > sorry for running a bit late. > > > > > Vladislav, do you have any updates for > > > https://issues.apache.org/jira/browse/IGNITE-638? Or any questions? > > > > > > --Yakov > > > > I have problems with some fail-over scenarios; > > It seems that if the two nodes are in the middle of acquiring or > releasing > > the semaphore, > > and one of them fails, all nodes get: > > > > [09:36:38,509][ERROR][ignite-#13%pub-null%][GridCacheSemaphoreImpl] > > <ignite-atomics-sys-cache> Failed to compare and set: > > o.a.i.i.processors.datastructures.GridCacheSemaphoreImpl$Sync$1@5528b728 > > class org.apache.ignite.internal.cluster.ClusterTopologyCheckedException: > > Failed to acquire lock for keys (primary node left grid, retry > transaction > > if possible) [keys=[UserKeyCacheObjectImpl [val=GridCacheInternalKeyImpl > > [name=ac83b8cb-3052-49a6-9301-81b20b0ecf3a], hasValBytes=true]], > > node=c321fcc4-5db5-4b03-9811-6a5587f2c253] > > ... > > Caused by: class > > org.apache.ignite.internal.cluster.ClusterTopologyCheckedException: > Failed > > to acquire lock for keys (primary node left grid, retry transaction if > > possible) [keys=[UserKeyCacheObjectImpl [val=GridCacheInternalKeyImpl > > [name=ac83b8cb-3052-49a6-9301-81b20b0ecf3a], hasValBytes=true]], > > node=c321fcc4-5db5-4b03-9811-6a5587f2c253] > > at > > > > > org.apache.ignite.internal.processors.cache.distributed.dht.colocated.GridDhtColocatedLockFuture.newTopologyException(GridDhtColocatedLockFuture.java:1199) > > ... 10 more > > > > > > I'm still trying to find out how to exactly reproduce this behavior, > > I'll send you more details once I try few more things. > > > > I am still using partitioned cache, does it make sense to use replicated > > cache instead? > > > > > > Other than that, I'm done with everything else. > > > > Thanks, > > Vladisav > > > > > |
|
|
Folks,
I continue working on IGNITE-1917 - marshalling microoptimizations. I did a bunch of things today which increased performance a bit, so that unmarshalling with PortableMarshaller is now a bit faster than OptimizedMarshaller when object has several fields, but still slower when there are lots of small fields. I'm going to apply the last easy optimization in the nearest time and then will focus on merging all pending tickets to master. Once all important things are merged, I'm going to spend some more efforts on performance again. On Tue, Nov 17, 2015 at 8:30 PM, Vladisav Jelisavcic <[hidden email]> wrote: > Hi Yakov, > 1. Yes > > 2. if you mean that nodeMap is accessed in onNodeRemoved(UUID nodeID) > method of the GridCacheSemaphoreImpl class, > it shouldn't be a problem, but it can be changed easily not to do so; > > 3. > > org.apache.ignite.internal.processors.cache.datastructures.GridCacheAbstractDataStructuresFailoverSelfTest#testSemaphoreConstantTopologyChangeFailoverSafe() > > org.apache.ignite.internal.processors.cache.datastructures.GridCacheAbstractDataStructuresFailoverSelfTest#testSemaphoreConstantMultipleTopologyChangeFailoverSafe() > > I think the problem is with the atomicity of the simulated grid failure; > once stopGrid() is called for a node, other threads on this same node start > throwing interrupted exceptions, > which are in turn not handled properly in the > GridCacheAbstractDataStructuresFailoverSelfTest; > Those exceptions shouldn't be dealt with inside the GridCacheSemaphoreImpl > itself. > In a realworld node failure scenario, all those threads would fail at the > same time > (none of them would influence the rest of the grid anymore); > > I think fixing the issue Denis is working on can fix this (IGNITE-801 and > IGNITE-803) > Am i right? Does it makes sense? > > > Best regards, > Vladisav > > > > > On Tue, Nov 17, 2015 at 5:40 PM, Yakov Zhdanov <[hidden email]> > wrote: > > > Vladislav, > > > > I started to review the latest changes and have couple of questions: > > > > 1. latest changes are here - https://github.com/apache/ignite/pull/120? > Is > > that correct? > > 2. > > > org.apache.ignite.internal.processors.datastructures.GridCacheSemaphoreImpl.Sync#nodeMap > > is accessed in both sync and unsync context. Are you sure this is fine. > > 3. As far as failing test - can you please isolate it into separate junit > > or point out existing one? > > > > --Yakov > > > > 2015-11-11 12:33 GMT+03:00 Vladisav Jelisavcic <[hidden email]>: > > > > > Yakov, > > > > > > sorry for running a bit late. > > > > > > > Vladislav, do you have any updates for > > > > https://issues.apache.org/jira/browse/IGNITE-638? Or any questions? > > > > > > > > --Yakov > > > > > > I have problems with some fail-over scenarios; > > > It seems that if the two nodes are in the middle of acquiring or > > releasing > > > the semaphore, > > > and one of them fails, all nodes get: > > > > > > [09:36:38,509][ERROR][ignite-#13%pub-null%][GridCacheSemaphoreImpl] > > > <ignite-atomics-sys-cache> Failed to compare and set: > > > > o.a.i.i.processors.datastructures.GridCacheSemaphoreImpl$Sync$1@5528b728 > > > class > org.apache.ignite.internal.cluster.ClusterTopologyCheckedException: > > > Failed to acquire lock for keys (primary node left grid, retry > > transaction > > > if possible) [keys=[UserKeyCacheObjectImpl > [val=GridCacheInternalKeyImpl > > > [name=ac83b8cb-3052-49a6-9301-81b20b0ecf3a], hasValBytes=true]], > > > node=c321fcc4-5db5-4b03-9811-6a5587f2c253] > > > ... > > > Caused by: class > > > org.apache.ignite.internal.cluster.ClusterTopologyCheckedException: > > Failed > > > to acquire lock for keys (primary node left grid, retry transaction if > > > possible) [keys=[UserKeyCacheObjectImpl [val=GridCacheInternalKeyImpl > > > [name=ac83b8cb-3052-49a6-9301-81b20b0ecf3a], hasValBytes=true]], > > > node=c321fcc4-5db5-4b03-9811-6a5587f2c253] > > > at > > > > > > > > > org.apache.ignite.internal.processors.cache.distributed.dht.colocated.GridDhtColocatedLockFuture.newTopologyException(GridDhtColocatedLockFuture.java:1199) > > > ... 10 more > > > > > > > > > I'm still trying to find out how to exactly reproduce this behavior, > > > I'll send you more details once I try few more things. > > > > > > I am still using partitioned cache, does it make sense to use > replicated > > > cache instead? > > > > > > > > > Other than that, I'm done with everything else. > > > > > > Thanks, > > > Vladisav > > > > > > > > > |
|
|
I meant "release branch", not "master".
On Tue, Nov 17, 2015 at 9:25 PM, Vladimir Ozerov <[hidden email]> wrote: > Folks, > > I continue working on IGNITE-1917 - marshalling microoptimizations. I did > a bunch of things today which increased performance a bit, so that > unmarshalling with PortableMarshaller is now a bit faster than > OptimizedMarshaller when object has several fields, but still slower when > there are lots of small fields. > > I'm going to apply the last easy optimization in the nearest time and then > will focus on merging all pending tickets to master. Once all important > things are merged, I'm going to spend some more efforts on performance > again. > > On Tue, Nov 17, 2015 at 8:30 PM, Vladisav Jelisavcic <[hidden email]> > wrote: > >> Hi Yakov, >> 1. Yes >> >> 2. if you mean that nodeMap is accessed in onNodeRemoved(UUID nodeID) >> method of the GridCacheSemaphoreImpl class, >> it shouldn't be a problem, but it can be changed easily not to do so; >> >> 3. >> >> org.apache.ignite.internal.processors.cache.datastructures.GridCacheAbstractDataStructuresFailoverSelfTest#testSemaphoreConstantTopologyChangeFailoverSafe() >> >> org.apache.ignite.internal.processors.cache.datastructures.GridCacheAbstractDataStructuresFailoverSelfTest#testSemaphoreConstantMultipleTopologyChangeFailoverSafe() >> >> I think the problem is with the atomicity of the simulated grid failure; >> once stopGrid() is called for a node, other threads on this same node >> start >> throwing interrupted exceptions, >> which are in turn not handled properly in the >> GridCacheAbstractDataStructuresFailoverSelfTest; >> Those exceptions shouldn't be dealt with inside the GridCacheSemaphoreImpl >> itself. >> In a realworld node failure scenario, all those threads would fail at the >> same time >> (none of them would influence the rest of the grid anymore); >> >> I think fixing the issue Denis is working on can fix this (IGNITE-801 and >> IGNITE-803) >> Am i right? Does it makes sense? >> >> >> Best regards, >> Vladisav >> >> >> >> >> On Tue, Nov 17, 2015 at 5:40 PM, Yakov Zhdanov <[hidden email]> >> wrote: >> >> > Vladislav, >> > >> > I started to review the latest changes and have couple of questions: >> > >> > 1. latest changes are here - https://github.com/apache/ignite/pull/120? >> Is >> > that correct? >> > 2. >> > >> org.apache.ignite.internal.processors.datastructures.GridCacheSemaphoreImpl.Sync#nodeMap >> > is accessed in both sync and unsync context. Are you sure this is fine. >> > 3. As far as failing test - can you please isolate it into separate >> junit >> > or point out existing one? >> > >> > --Yakov >> > >> > 2015-11-11 12:33 GMT+03:00 Vladisav Jelisavcic <[hidden email]>: >> > >> > > Yakov, >> > > >> > > sorry for running a bit late. >> > > >> > > > Vladislav, do you have any updates for >> > > > https://issues.apache.org/jira/browse/IGNITE-638? Or any questions? >> > > > >> > > > --Yakov >> > > >> > > I have problems with some fail-over scenarios; >> > > It seems that if the two nodes are in the middle of acquiring or >> > releasing >> > > the semaphore, >> > > and one of them fails, all nodes get: >> > > >> > > [09:36:38,509][ERROR][ignite-#13%pub-null%][GridCacheSemaphoreImpl] >> > > <ignite-atomics-sys-cache> Failed to compare and set: >> > > >> o.a.i.i.processors.datastructures.GridCacheSemaphoreImpl$Sync$1@5528b728 >> > > class >> org.apache.ignite.internal.cluster.ClusterTopologyCheckedException: >> > > Failed to acquire lock for keys (primary node left grid, retry >> > transaction >> > > if possible) [keys=[UserKeyCacheObjectImpl >> [val=GridCacheInternalKeyImpl >> > > [name=ac83b8cb-3052-49a6-9301-81b20b0ecf3a], hasValBytes=true]], >> > > node=c321fcc4-5db5-4b03-9811-6a5587f2c253] >> > > ... >> > > Caused by: class >> > > org.apache.ignite.internal.cluster.ClusterTopologyCheckedException: >> > Failed >> > > to acquire lock for keys (primary node left grid, retry transaction if >> > > possible) [keys=[UserKeyCacheObjectImpl [val=GridCacheInternalKeyImpl >> > > [name=ac83b8cb-3052-49a6-9301-81b20b0ecf3a], hasValBytes=true]], >> > > node=c321fcc4-5db5-4b03-9811-6a5587f2c253] >> > > at >> > > >> > > >> > >> org.apache.ignite.internal.processors.cache.distributed.dht.colocated.GridDhtColocatedLockFuture.newTopologyException(GridDhtColocatedLockFuture.java:1199) >> > > ... 10 more >> > > >> > > >> > > I'm still trying to find out how to exactly reproduce this behavior, >> > > I'll send you more details once I try few more things. >> > > >> > > I am still using partitioned cache, does it make sense to use >> replicated >> > > cache instead? >> > > >> > > >> > > Other than that, I'm done with everything else. >> > > >> > > Thanks, >> > > Vladisav >> > > >> > > >> > >> > > |
|
|
In reply to this post by Vladisav Jelisavcic
1. Understood
2. Well, I understand that object gets deserialized on get from cache and this most probably should not cause any issue. However, code does not look healthy and someone may think that there is an issue - field is not volatile, initialized in constructor, has sync setter, but is accessed several times in code outside of sync. Can you please refactor this? 3. Yes, Denis is looking into this. And I think your issue will be fixed in the scope of Denis one. Vlad, can you please refactor [2] and I will pick it up and merge tomorrow. Thank you for your interest and contribution! --Yakov 2015-11-17 21:30 GMT+04:00 Vladisav Jelisavcic <[hidden email]>: > Hi Yakov, > 1. Yes > > 2. if you mean that nodeMap is accessed in onNodeRemoved(UUID nodeID) > method of the GridCacheSemaphoreImpl class, > it shouldn't be a problem, but it can be changed easily not to do so; > > 3. > > org.apache.ignite.internal.processors.cache.datastructures.GridCacheAbstractDataStructuresFailoverSelfTest#testSemaphoreConstantTopologyChangeFailoverSafe() > > org.apache.ignite.internal.processors.cache.datastructures.GridCacheAbstractDataStructuresFailoverSelfTest#testSemaphoreConstantMultipleTopologyChangeFailoverSafe() > > I think the problem is with the atomicity of the simulated grid failure; > once stopGrid() is called for a node, other threads on this same node start > throwing interrupted exceptions, > which are in turn not handled properly in the > GridCacheAbstractDataStructuresFailoverSelfTest; > Those exceptions shouldn't be dealt with inside the GridCacheSemaphoreImpl > itself. > In a realworld node failure scenario, all those threads would fail at the > same time > (none of them would influence the rest of the grid anymore); > > I think fixing the issue Denis is working on can fix this (IGNITE-801 and > IGNITE-803) > Am i right? Does it makes sense? > > > Best regards, > Vladisav > > |
|
|
In reply to this post by yzhdanov
11. Flume intergration
https://issues.apache.org/jira/browse/IGNITE-529 is done too. Thanks Anton for reviews! Roman On Monday, November 2, 2015 10:35 PM, Yakov Zhdanov <[hidden email]> wrote: Guys, I think we can start preparation to Ignite-1.5 release which will include many interesting features: 1. Portable object API https://issues.apache.org/jira/browse/IGNITE-1486 2. Ignite.NET and Ignite C++ https://issues.apache.org/jira/browse/IGNITE-1282 3. Optimistic serializable transactions https://issues.apache.org/jira/browse/IGNITE-1607 4. Distributed SQL joins - we will be able to query non-collocated data as well https://issues.apache.org/jira/browse/IGNITE-1232 5. Enhanced Oracle and IBM JDK interoperability https://issues.apache.org/jira/browse/IGNITE-1526 6. MQTT streamer https://issues.apache.org/jira/browse/IGNITE-535 7. Continuous query failover https://issues.apache.org/jira/browse/IGNITE-426 8. Significant transactional cache performance optimizations - I will merge these changes from 'ignite-1.4-slow-server-debug' today or tomorrow. 9. Many stability and fault-tolerance fixes. 10. I would also like to include distributed Semaphore. Vladislav, any chance you can finish with it this week? https://issues.apache.org/jira/browse/IGNITE- <https://issues.apache.org/jira/browse/IGNITE-426>638 Thanks to everyone involved! Guys, esp. assignees of mentioned issues, please respond to this email and let us know when can we expect your changes being merged to master and release branch? Can someone create ignite-1.5 release branch? --Yakov |
|
|
Folks,
I'm currently working on finalizing optimizations in communication and direct marshalling, as well as supporting backward compatibility. It turned out that this requires some refactoring, so it will take one more day at least. I hope to merge tomorrow. On Tue, Nov 17, 2015 at 2:43 PM, Roman Shtykh <[hidden email]> wrote: > 11. Flume intergration > https://issues.apache.org/jira/browse/IGNITE-529 > > > is done too. Thanks Anton for reviews! > > Roman > > > > > On Monday, November 2, 2015 10:35 PM, Yakov Zhdanov <[hidden email]> > wrote: > > > > Guys, > > I think we can start preparation to Ignite-1.5 release which will include > many interesting features: > > 1. Portable object API > https://issues.apache.org/jira/browse/IGNITE-1486 > > 2. Ignite.NET and Ignite C++ > https://issues.apache.org/jira/browse/IGNITE-1282 > > 3. Optimistic serializable transactions > https://issues.apache.org/jira/browse/IGNITE-1607 > > 4. Distributed SQL joins - we will be able to query non-collocated data as > well > https://issues.apache.org/jira/browse/IGNITE-1232 > > 5. Enhanced Oracle and IBM JDK interoperability > https://issues.apache.org/jira/browse/IGNITE-1526 > > 6. MQTT streamer > https://issues.apache.org/jira/browse/IGNITE-535 > > 7. Continuous query failover > https://issues.apache.org/jira/browse/IGNITE-426 > > 8. Significant transactional cache performance optimizations - I will merge > these changes from 'ignite-1.4-slow-server-debug' today or tomorrow. > > 9. Many stability and fault-tolerance fixes. > > 10. I would also like to include distributed Semaphore. Vladislav, any > chance you can finish with it this week? > https://issues.apache.org/jira/browse/IGNITE- > <https://issues.apache.org/jira/browse/IGNITE-426>638 > > Thanks to everyone involved! Guys, esp. assignees of mentioned issues, > please respond to this email and let us know when can we expect your > changes being merged to master and release branch? > > Can someone create ignite-1.5 release branch? > > --Yakov > |
|
|
In reply to this post by yzhdanov
On 11/17/2015 9:37 PM, Yakov Zhdanov wrote: > 1. Understood > 2. Well, I understand that object gets deserialized on get from cache and > this most probably should not cause any issue. However, code does not look > healthy and someone may think that there is an issue - field is not > volatile, initialized in constructor, has sync setter, but is accessed > several times in code outside of sync. Can you please refactor this? > 3. Yes, Denis is looking into this. And I think your issue will be fixed in > the scope of Denis one. I'm afraid that my changes won't fix the issue in Vladislav's implementation. However, when my changes are merged we will be able to rework these semaphore failover tests and fix any issue in the semaphore impl related to failover. I can take care of this. -- Denis > > Vlad, can you please refactor [2] and I will pick it up and merge tomorrow. > > Thank you for your interest and contribution! > > --Yakov > > 2015-11-17 21:30 GMT+04:00 Vladisav Jelisavcic <[hidden email]>: > >> Hi Yakov, >> 1. Yes >> >> 2. if you mean that nodeMap is accessed in onNodeRemoved(UUID nodeID) >> method of the GridCacheSemaphoreImpl class, >> it shouldn't be a problem, but it can be changed easily not to do so; >> >> 3. >> >> org.apache.ignite.internal.processors.cache.datastructures.GridCacheAbstractDataStructuresFailoverSelfTest#testSemaphoreConstantTopologyChangeFailoverSafe() >> >> org.apache.ignite.internal.processors.cache.datastructures.GridCacheAbstractDataStructuresFailoverSelfTest#testSemaphoreConstantMultipleTopologyChangeFailoverSafe() >> >> I think the problem is with the atomicity of the simulated grid failure; >> once stopGrid() is called for a node, other threads on this same node start >> throwing interrupted exceptions, >> which are in turn not handled properly in the >> GridCacheAbstractDataStructuresFailoverSelfTest; >> Those exceptions shouldn't be dealt with inside the GridCacheSemaphoreImpl >> itself. >> In a realworld node failure scenario, all those threads would fail at the >> same time >> (none of them would influence the rest of the grid anymore); >> >> I think fixing the issue Denis is working on can fix this (IGNITE-801 and >> IGNITE-803) >> Am i right? Does it makes sense? >> >> >> Best regards, >> Vladisav >> >> |
|
|
In reply to this post by yzhdanov
Hi,
Yesterday I merged optimizations for tx update operations working single key, got ~7% improvement in tx-put benchmark. And today I finished to implement optimization for cache 'get' operation with single key. Got another benchmark results improvements: ~10% with atomic-put-get, ~5% with tx-put-get. Need to verify TC and hopefully will merge these changes tomorrow. |
|
|
Hi folks,
I merged performance optimizations for tx mode which gave up to 5% improvement for tx-put benchmark to ignite-1.5. I also pushed finalizaing changes to Binary configuration in ignite-1945 branch and currently waiting for TC. If all is ok, will cooperate with Vladimir Ozerov and merge changes tomorrow. 2015-11-18 17:58 GMT+03:00 Semyon Boikov <[hidden email]>: > Hi, > > Yesterday I merged optimizations for tx update operations working single > key, got ~7% improvement in tx-put benchmark. > > And today I finished to implement optimization for cache 'get' operation > with single key. Got another benchmark results improvements: ~10% with > atomic-put-get, ~5% with tx-put-get. Need to verify TC and hopefully will > merge these changes tomorrow. > > > |
|
|
Hi Alex,
Today I marged several big things into 1282 - reworked metadat and compacted footers optimization. I have on big ticket left - IGNITE-1917 - with marshalling microoptimizations. They should not conflict with anything and I expect them to be finalized tomorrow. All optimizations are ready, I just need to perform some cleanup and refactoring. On Wed, Nov 18, 2015 at 7:22 PM, Alexey Goncharuk < [hidden email]> wrote: > Hi folks, > > I merged performance optimizations for tx mode which gave up to 5% > improvement for tx-put benchmark to ignite-1.5. > > I also pushed finalizaing changes to Binary configuration in ignite-1945 > branch and currently waiting for TC. If all is ok, will cooperate with > Vladimir Ozerov and merge changes tomorrow. > > 2015-11-18 17:58 GMT+03:00 Semyon Boikov <[hidden email]>: > > > Hi, > > > > Yesterday I merged optimizations for tx update operations working single > > key, got ~7% improvement in tx-put benchmark. > > > > And today I finished to implement optimization for cache 'get' operation > > with single key. Got another benchmark results improvements: ~10% with > > atomic-put-get, ~5% with tx-put-get. Need to verify TC and hopefully will > > merge these changes tomorrow. > > > > > > > |
|
|
I merged the Camel Streamer today and I hope to finish the first phase of
OSGi tomorrow. I'll need someone to review the latter quickly if we want it merged for 1.5... No code changes in existing codebase. Just a new osgi module and many POM changes mainly to generate the manifests. Should be a quick task. Regards, Raúl. On 18 Nov 2015 20:38, "Vladimir Ozerov" <[hidden email]> wrote: > Hi Alex, > > Today I marged several big things into 1282 - reworked metadat and > compacted footers optimization. I have on big ticket left - IGNITE-1917 - > with marshalling microoptimizations. They should not conflict with anything > and I expect them to be finalized tomorrow. All optimizations are ready, I > just need to perform some cleanup and refactoring. > > On Wed, Nov 18, 2015 at 7:22 PM, Alexey Goncharuk < > [hidden email]> wrote: > > > Hi folks, > > > > I merged performance optimizations for tx mode which gave up to 5% > > improvement for tx-put benchmark to ignite-1.5. > > > > I also pushed finalizaing changes to Binary configuration in ignite-1945 > > branch and currently waiting for TC. If all is ok, will cooperate with > > Vladimir Ozerov and merge changes tomorrow. > > > > 2015-11-18 17:58 GMT+03:00 Semyon Boikov <[hidden email]>: > > > > > Hi, > > > > > > Yesterday I merged optimizations for tx update operations working > single > > > key, got ~7% improvement in tx-put benchmark. > > > > > > And today I finished to implement optimization for cache 'get' > operation > > > with single key. Got another benchmark results improvements: ~10% with > > > atomic-put-get, ~5% with tx-put-get. Need to verify TC and hopefully > will > > > merge these changes tomorrow. > > > > > > > > > > > > |
|
|
On Wed, Nov 18, 2015 at 2:17 PM, Raul Kripalani <[hidden email]> wrote:
> I merged the Camel Streamer today and I hope to finish the first phase of > OSGi tomorrow. > > I'll need someone to review the latter quickly if we want it merged for > 1.5... No code changes in existing codebase. Just a new osgi module and > many POM changes mainly to generate the manifests. > Raul, did you create a patch or a PR? I can’t find it anywhere. > > Should be a quick task. > > Regards, > Raúl. > On 18 Nov 2015 20:38, "Vladimir Ozerov" <[hidden email]> wrote: > > > Hi Alex, > > > > Today I marged several big things into 1282 - reworked metadat and > > compacted footers optimization. I have on big ticket left - IGNITE-1917 - > > with marshalling microoptimizations. They should not conflict with > anything > > and I expect them to be finalized tomorrow. All optimizations are ready, > I > > just need to perform some cleanup and refactoring. > > > > On Wed, Nov 18, 2015 at 7:22 PM, Alexey Goncharuk < > > [hidden email]> wrote: > > > > > Hi folks, > > > > > > I merged performance optimizations for tx mode which gave up to 5% > > > improvement for tx-put benchmark to ignite-1.5. > > > > > > I also pushed finalizaing changes to Binary configuration in > ignite-1945 > > > branch and currently waiting for TC. If all is ok, will cooperate with > > > Vladimir Ozerov and merge changes tomorrow. > > > > > > 2015-11-18 17:58 GMT+03:00 Semyon Boikov <[hidden email]>: > > > > > > > Hi, > > > > > > > > Yesterday I merged optimizations for tx update operations working > > single > > > > key, got ~7% improvement in tx-put benchmark. > > > > > > > > And today I finished to implement optimization for cache 'get' > > operation > > > > with single key. Got another benchmark results improvements: ~10% > with > > > > atomic-put-get, ~5% with tx-put-get. Need to verify TC and hopefully > > will > > > > merge these changes tomorrow. > > > > > > > > > > > > > > > > > > |
|
|
Hi Raul,
Could you please give the pull request or the branch or the ignite ticket number? I will be please to have a look to your pull request. Thanks, Romain Le jeu. 19 nov. 2015 à 04:01, Dmitriy Setrakyan <[hidden email]> a écrit : > On Wed, Nov 18, 2015 at 2:17 PM, Raul Kripalani <[hidden email]> wrote: > > > I merged the Camel Streamer today and I hope to finish the first phase of > > OSGi tomorrow. > > > > I'll need someone to review the latter quickly if we want it merged for > > 1.5... No code changes in existing codebase. Just a new osgi module and > > many POM changes mainly to generate the manifests. > > > > Raul, did you create a patch or a PR? I can’t find it anywhere. > > > > > > Should be a quick task. > > > > Regards, > > Raúl. > > On 18 Nov 2015 20:38, "Vladimir Ozerov" <[hidden email]> wrote: > > > > > Hi Alex, > > > > > > Today I marged several big things into 1282 - reworked metadat and > > > compacted footers optimization. I have on big ticket left - > IGNITE-1917 - > > > with marshalling microoptimizations. They should not conflict with > > anything > > > and I expect them to be finalized tomorrow. All optimizations are > ready, > > I > > > just need to perform some cleanup and refactoring. > > > > > > On Wed, Nov 18, 2015 at 7:22 PM, Alexey Goncharuk < > > > [hidden email]> wrote: > > > > > > > Hi folks, > > > > > > > > I merged performance optimizations for tx mode which gave up to 5% > > > > improvement for tx-put benchmark to ignite-1.5. > > > > > > > > I also pushed finalizaing changes to Binary configuration in > > ignite-1945 > > > > branch and currently waiting for TC. If all is ok, will cooperate > with > > > > Vladimir Ozerov and merge changes tomorrow. > > > > > > > > 2015-11-18 17:58 GMT+03:00 Semyon Boikov <[hidden email]>: > > > > > > > > > Hi, > > > > > > > > > > Yesterday I merged optimizations for tx update operations working > > > single > > > > > key, got ~7% improvement in tx-put benchmark. > > > > > > > > > > And today I finished to implement optimization for cache 'get' > > > operation > > > > > with single key. Got another benchmark results improvements: ~10% > > with > > > > > atomic-put-get, ~5% with tx-put-get. Need to verify TC and > hopefully > > > will > > > > > merge these changes tomorrow. > > > > > > > > > > > > > > > > > > > > > > > > > |
|
|
In reply to this post by dsetrakyan
Branch ignite-1790 merged into ignite-1.5 with --squash. Like it was
requested previous on previous occasions. On 19 Nov 2015 03:01, "Dmitriy Setrakyan" <[hidden email]> wrote: > On Wed, Nov 18, 2015 at 2:17 PM, Raul Kripalani <[hidden email]> wrote: > > > I merged the Camel Streamer today and I hope to finish the first phase of > > OSGi tomorrow. > > > > I'll need someone to review the latter quickly if we want it merged for > > 1.5... No code changes in existing codebase. Just a new osgi module and > > many POM changes mainly to generate the manifests. > > > > Raul, did you create a patch or a PR? I can’t find it anywhere. > > > > > > Should be a quick task. > > > > Regards, > > Raúl. > > On 18 Nov 2015 20:38, "Vladimir Ozerov" <[hidden email]> wrote: > > > > > Hi Alex, > > > > > > Today I marged several big things into 1282 - reworked metadat and > > > compacted footers optimization. I have on big ticket left - > IGNITE-1917 - > > > with marshalling microoptimizations. They should not conflict with > > anything > > > and I expect them to be finalized tomorrow. All optimizations are > ready, > > I > > > just need to perform some cleanup and refactoring. > > > > > > On Wed, Nov 18, 2015 at 7:22 PM, Alexey Goncharuk < > > > [hidden email]> wrote: > > > > > > > Hi folks, > > > > > > > > I merged performance optimizations for tx mode which gave up to 5% > > > > improvement for tx-put benchmark to ignite-1.5. > > > > > > > > I also pushed finalizaing changes to Binary configuration in > > ignite-1945 > > > > branch and currently waiting for TC. If all is ok, will cooperate > with > > > > Vladimir Ozerov and merge changes tomorrow. > > > > > > > > 2015-11-18 17:58 GMT+03:00 Semyon Boikov <[hidden email]>: > > > > > > > > > Hi, > > > > > > > > > > Yesterday I merged optimizations for tx update operations working > > > single > > > > > key, got ~7% improvement in tx-put benchmark. > > > > > > > > > > And today I finished to implement optimization for cache 'get' > > > operation > > > > > with single key. Got another benchmark results improvements: ~10% > > with > > > > > atomic-put-get, ~5% with tx-put-get. Need to verify TC and > hopefully > > > will > > > > > merge these changes tomorrow. > > > > > > > > > > > > > > > > > > > > > > > > > |
|
|
In reply to this post by Romain Gilles-2
The code was already reviewed and merged with squash. IGNITE-1790.
On 19 Nov 2015 08:30, "Romain Gilles" <[hidden email]> wrote: > Hi Raul, > Could you please give the pull request or the branch or the ignite ticket > number? > I will be please to have a look to your pull request. > > Thanks, > > Romain > > Le jeu. 19 nov. 2015 à 04:01, Dmitriy Setrakyan <[hidden email]> a > écrit : > > > On Wed, Nov 18, 2015 at 2:17 PM, Raul Kripalani <[hidden email]> > wrote: > > > > > I merged the Camel Streamer today and I hope to finish the first phase > of > > > OSGi tomorrow. > > > > > > I'll need someone to review the latter quickly if we want it merged for > > > 1.5... No code changes in existing codebase. Just a new osgi module and > > > many POM changes mainly to generate the manifests. > > > > > > > Raul, did you create a patch or a PR? I can’t find it anywhere. > > > > > > > > > > Should be a quick task. > > > > > > Regards, > > > Raúl. > > > On 18 Nov 2015 20:38, "Vladimir Ozerov" <[hidden email]> wrote: > > > > > > > Hi Alex, > > > > > > > > Today I marged several big things into 1282 - reworked metadat and > > > > compacted footers optimization. I have on big ticket left - > > IGNITE-1917 - > > > > with marshalling microoptimizations. They should not conflict with > > > anything > > > > and I expect them to be finalized tomorrow. All optimizations are > > ready, > > > I > > > > just need to perform some cleanup and refactoring. > > > > > > > > On Wed, Nov 18, 2015 at 7:22 PM, Alexey Goncharuk < > > > > [hidden email]> wrote: > > > > > > > > > Hi folks, > > > > > > > > > > I merged performance optimizations for tx mode which gave up to 5% > > > > > improvement for tx-put benchmark to ignite-1.5. > > > > > > > > > > I also pushed finalizaing changes to Binary configuration in > > > ignite-1945 > > > > > branch and currently waiting for TC. If all is ok, will cooperate > > with > > > > > Vladimir Ozerov and merge changes tomorrow. > > > > > > > > > > 2015-11-18 17:58 GMT+03:00 Semyon Boikov <[hidden email]>: > > > > > > > > > > > Hi, > > > > > > > > > > > > Yesterday I merged optimizations for tx update operations working > > > > single > > > > > > key, got ~7% improvement in tx-put benchmark. > > > > > > > > > > > > And today I finished to implement optimization for cache 'get' > > > > operation > > > > > > with single key. Got another benchmark results improvements: ~10% > > > with > > > > > > atomic-put-get, ~5% with tx-put-get. Need to verify TC and > > hopefully > > > > will > > > > > > merge these changes tomorrow. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > |
|
|
Hey guys!
We are getting closer to the release! Thanks to everyone making this closer! Let's update status info. My understanding is that the following major parts are still not merged. Can everyone mentioned provide ticket number(s) and estimates on when the functionality will be merged. 1. Data grid performance optimizations: single get/put - Semyon 2. Direct marshalling compactions: communication optimization - Valentin 3. Introduce new binary format - Alexey Goncharuk and Vladimir Ozerov 4. Data grid performance optimizations: SQL queries - Sergi and Semyon Thanks! --Yakov |
|
|
Just merged single get optimizations (improvements in the last benchmarks
run: ~10% for atomic-put-get, ~5% for tx-put-get). |
«
Return to Apache Ignite Developers - Legacy Mail Archive
|
1 view|%1 views
| Free forum by Nabble | Edit this page |