Partition data lost event
Locked
Partition data lost event
 
|
Folks,
Does anyone know how the EVT_CACHE_REBALANCE_PART_DATA_LOST event is fired? I just create a small test and the result confused me. This is what I do: 1. Start several server nodes, all listen to EVT_CACHE_REBALANCE_PART_DATA_LOST event. 2. Create a cache with no backups. 3. Put some values. 4. Stop one of the nodes (lose some partitions). What I noticed is that the event is fired on all nodes that are still in topology, but for different partitions. For example, if there are 4 partitions lost (1,2,3,4), I can get notifications for 1,3 on first node and 2,4 on second node. This seems weird and it's not clear to me how to use this event. Can anyone explain this behavior? Is this expected? Here is the test code: public class PartitionTest { public static void main(String[] args) throws InterruptedException { for (int i = 0; i < 4; i++) Ignition.start(config("server-" + i)); Ignition.setClientMode(true); Ignite ignite = Ignition.start(); CacheConfiguration<Integer, Integer> cfg = new CacheConfiguration<>("test"); cfg.setAffinity(new RendezvousAffinityFunction(20, null)); IgniteCache<Integer, Integer> cache = ignite.createCache(cfg); for (int i = 0; i < 100; i++) cache.put(i, i); System.out.println("Populated."); Thread.sleep(2000); Ignition.stop("server-3", true); } private static IgniteConfiguration config(String name) { IgniteConfiguration cfg = new IgniteConfiguration(); cfg.setGridName(name); cfg.setIncludeEventTypes(EventType.EVT_CACHE_REBALANCE_PART_DATA_LOST); cfg.setLocalEventListeners(Collections.<IgnitePredicate<? extends Event>, int[]>singletonMap( new IgnitePredicate<Event>() { @Override public boolean apply(Event event) { U.debug("EVENT: " + event); return true; } }, new int[] {EventType.EVT_CACHE_REBALANCE_PART_DATA_LOST} )); return cfg; } } -Val |
|
|
Alex G. and Yakov should be able to clarify this since they took part in the creation of the following tickets related to partitions lost consistency
https://issues.apache.org/jira/browse/IGNITE-1605 <https://issues.apache.org/jira/browse/IGNITE-1605> https://issues.apache.org/jira/browse/IGNITE-2378 <https://issues.apache.org/jira/browse/IGNITE-2378> — Denis > On Jan 18, 2017, at 12:16 PM, Valentin Kulichenko <[hidden email]> wrote: > > Folks, > > Does anyone know how the EVT_CACHE_REBALANCE_PART_DATA_LOST event is fired? > I just create a small test and the result confused me. This is what I do: > > 1. Start several server nodes, all listen to > EVT_CACHE_REBALANCE_PART_DATA_LOST event. > 2. Create a cache with no backups. > 3. Put some values. > 4. Stop one of the nodes (lose some partitions). > > What I noticed is that the event is fired on all nodes that are still in > topology, but for different partitions. For example, if there are 4 > partitions lost (1,2,3,4), I can get notifications for 1,3 on first node > and 2,4 on second node. This seems weird and it's not clear to me how to > use this event. > > Can anyone explain this behavior? Is this expected? > > Here is the test code: > > public class PartitionTest { > public static void main(String[] args) throws InterruptedException { > for (int i = 0; i < 4; i++) > Ignition.start(config("server-" + i)); > > Ignition.setClientMode(true); > > Ignite ignite = Ignition.start(); > > CacheConfiguration<Integer, Integer> cfg = new > CacheConfiguration<>("test"); > > cfg.setAffinity(new RendezvousAffinityFunction(20, null)); > > IgniteCache<Integer, Integer> cache = ignite.createCache(cfg); > > for (int i = 0; i < 100; i++) > cache.put(i, i); > > System.out.println("Populated."); > > Thread.sleep(2000); > > Ignition.stop("server-3", true); > } > > private static IgniteConfiguration config(String name) { > IgniteConfiguration cfg = new IgniteConfiguration(); > > cfg.setGridName(name); > > cfg.setIncludeEventTypes(EventType.EVT_CACHE_REBALANCE_PART_DATA_LOST); > cfg.setLocalEventListeners(Collections.<IgnitePredicate<? extends > Event>, int[]>singletonMap( > new IgnitePredicate<Event>() { > @Override public boolean apply(Event event) { > U.debug("EVENT: " + event); > > return true; > } > }, > new int[] {EventType.EVT_CACHE_REBALANCE_PART_DATA_LOST} > )); > > return cfg; > } > } > > -Val |
Re: Partition data lost event
|
|
Val et al,
Currently partition lost event is fired on nodes which should become partition owners, but did not find other partition owners to rebalance data from. In your example one of the node should have became an owner for partitions 1, 3, the other - for partitions 2, 4 (according to the affinity function assignment), thus this set of events is fired. In Ignite 2.0 lost partitions will be handled on coordinator and events will be fired for all partitions on all cache nodes. --AG 2017-01-18 23:39 GMT+03:00 Denis Magda <[hidden email]>: > Alex G. and Yakov should be able to clarify this since they took part in > the creation of the following tickets related to partitions lost consistency > https://issues.apache.org/jira/browse/IGNITE-1605 < > https://issues.apache.org/jira/browse/IGNITE-1605> > https://issues.apache.org/jira/browse/IGNITE-2378 < > https://issues.apache.org/jira/browse/IGNITE-2378> > > — > Denis > > > On Jan 18, 2017, at 12:16 PM, Valentin Kulichenko < > [hidden email]> wrote: > > > > Folks, > > > > Does anyone know how the EVT_CACHE_REBALANCE_PART_DATA_LOST event is > fired? > > I just create a small test and the result confused me. This is what I do: > > > > 1. Start several server nodes, all listen to > > EVT_CACHE_REBALANCE_PART_DATA_LOST event. > > 2. Create a cache with no backups. > > 3. Put some values. > > 4. Stop one of the nodes (lose some partitions). > > > > What I noticed is that the event is fired on all nodes that are still in > > topology, but for different partitions. For example, if there are 4 > > partitions lost (1,2,3,4), I can get notifications for 1,3 on first node > > and 2,4 on second node. This seems weird and it's not clear to me how to > > use this event. > > > > Can anyone explain this behavior? Is this expected? > > > > Here is the test code: > > > > public class PartitionTest { > > public static void main(String[] args) throws InterruptedException { > > for (int i = 0; i < 4; i++) > > Ignition.start(config("server-" + i)); > > > > Ignition.setClientMode(true); > > > > Ignite ignite = Ignition.start(); > > > > CacheConfiguration<Integer, Integer> cfg = new > > CacheConfiguration<>("test"); > > > > cfg.setAffinity(new RendezvousAffinityFunction(20, null)); > > > > IgniteCache<Integer, Integer> cache = ignite.createCache(cfg); > > > > for (int i = 0; i < 100; i++) > > cache.put(i, i); > > > > System.out.println("Populated."); > > > > Thread.sleep(2000); > > > > Ignition.stop("server-3", true); > > } > > > > private static IgniteConfiguration config(String name) { > > IgniteConfiguration cfg = new IgniteConfiguration(); > > > > cfg.setGridName(name); > > > > cfg.setIncludeEventTypes(EventType.EVT_CACHE_REBALANCE_PART_DATA_LOST); > > cfg.setLocalEventListeners(Collections.<IgnitePredicate<? extends > > Event>, int[]>singletonMap( > > new IgnitePredicate<Event>() { > > @Override public boolean apply(Event event) { > > U.debug("EVENT: " + event); > > > > return true; > > } > > }, > > new int[] {EventType.EVT_CACHE_REBALANCE_PART_DATA_LOST} > > )); > > > > return cfg; > > } > > } > > > > -Val > > |
Re: Partition data lost event
|
|
Alex,
Thanks for response! -Val On Wed, Jan 18, 2017 at 10:17 PM, Alexey Goncharuk < [hidden email]> wrote: > Val et al, > > Currently partition lost event is fired on nodes which should become > partition owners, but did not find other partition owners to rebalance data > from. In your example one of the node should have became an owner for > partitions 1, 3, the other - for partitions 2, 4 (according to the affinity > function assignment), thus this set of events is fired. > > In Ignite 2.0 lost partitions will be handled on coordinator and events > will be fired for all partitions on all cache nodes. > > --AG > > 2017-01-18 23:39 GMT+03:00 Denis Magda <[hidden email]>: > > > Alex G. and Yakov should be able to clarify this since they took part in > > the creation of the following tickets related to partitions lost > consistency > > https://issues.apache.org/jira/browse/IGNITE-1605 < > > https://issues.apache.org/jira/browse/IGNITE-1605> > > https://issues.apache.org/jira/browse/IGNITE-2378 < > > https://issues.apache.org/jira/browse/IGNITE-2378> > > > > — > > Denis > > > > > On Jan 18, 2017, at 12:16 PM, Valentin Kulichenko < > > [hidden email]> wrote: > > > > > > Folks, > > > > > > Does anyone know how the EVT_CACHE_REBALANCE_PART_DATA_LOST event is > > fired? > > > I just create a small test and the result confused me. This is what I > do: > > > > > > 1. Start several server nodes, all listen to > > > EVT_CACHE_REBALANCE_PART_DATA_LOST event. > > > 2. Create a cache with no backups. > > > 3. Put some values. > > > 4. Stop one of the nodes (lose some partitions). > > > > > > What I noticed is that the event is fired on all nodes that are still > in > > > topology, but for different partitions. For example, if there are 4 > > > partitions lost (1,2,3,4), I can get notifications for 1,3 on first > node > > > and 2,4 on second node. This seems weird and it's not clear to me how > to > > > use this event. > > > > > > Can anyone explain this behavior? Is this expected? > > > > > > Here is the test code: > > > > > > public class PartitionTest { > > > public static void main(String[] args) throws InterruptedException { > > > for (int i = 0; i < 4; i++) > > > Ignition.start(config("server-" + i)); > > > > > > Ignition.setClientMode(true); > > > > > > Ignite ignite = Ignition.start(); > > > > > > CacheConfiguration<Integer, Integer> cfg = new > > > CacheConfiguration<>("test"); > > > > > > cfg.setAffinity(new RendezvousAffinityFunction(20, null)); > > > > > > IgniteCache<Integer, Integer> cache = ignite.createCache(cfg); > > > > > > for (int i = 0; i < 100; i++) > > > cache.put(i, i); > > > > > > System.out.println("Populated."); > > > > > > Thread.sleep(2000); > > > > > > Ignition.stop("server-3", true); > > > } > > > > > > private static IgniteConfiguration config(String name) { > > > IgniteConfiguration cfg = new IgniteConfiguration(); > > > > > > cfg.setGridName(name); > > > > > > cfg.setIncludeEventTypes(EventType.EVT_CACHE_REBALANCE_ > PART_DATA_LOST); > > > cfg.setLocalEventListeners(Collections.<IgnitePredicate<? > extends > > > Event>, int[]>singletonMap( > > > new IgnitePredicate<Event>() { > > > @Override public boolean apply(Event event) { > > > U.debug("EVENT: " + event); > > > > > > return true; > > > } > > > }, > > > new int[] {EventType.EVT_CACHE_REBALANCE_PART_DATA_LOST} > > > )); > > > > > > return cfg; > > > } > > > } > > > > > > -Val > > > > > |
|
|
Val, Alex, could anyone of you document this on readme.io <http://readme.io/>?
— Denis > On Jan 19, 2017, at 9:49 AM, Valentin Kulichenko <[hidden email]> wrote: > > Alex, > > Thanks for response! > > -Val > > On Wed, Jan 18, 2017 at 10:17 PM, Alexey Goncharuk < > [hidden email]> wrote: > >> Val et al, >> >> Currently partition lost event is fired on nodes which should become >> partition owners, but did not find other partition owners to rebalance data >> from. In your example one of the node should have became an owner for >> partitions 1, 3, the other - for partitions 2, 4 (according to the affinity >> function assignment), thus this set of events is fired. >> >> In Ignite 2.0 lost partitions will be handled on coordinator and events >> will be fired for all partitions on all cache nodes. >> >> --AG >> >> 2017-01-18 23:39 GMT+03:00 Denis Magda <[hidden email]>: >> >>> Alex G. and Yakov should be able to clarify this since they took part in >>> the creation of the following tickets related to partitions lost >> consistency >>> https://issues.apache.org/jira/browse/IGNITE-1605 < >>> https://issues.apache.org/jira/browse/IGNITE-1605> >>> https://issues.apache.org/jira/browse/IGNITE-2378 < >>> https://issues.apache.org/jira/browse/IGNITE-2378> >>> >>> — >>> Denis >>> >>>> On Jan 18, 2017, at 12:16 PM, Valentin Kulichenko < >>> [hidden email]> wrote: >>>> >>>> Folks, >>>> >>>> Does anyone know how the EVT_CACHE_REBALANCE_PART_DATA_LOST event is >>> fired? >>>> I just create a small test and the result confused me. This is what I >> do: >>>> >>>> 1. Start several server nodes, all listen to >>>> EVT_CACHE_REBALANCE_PART_DATA_LOST event. >>>> 2. Create a cache with no backups. >>>> 3. Put some values. >>>> 4. Stop one of the nodes (lose some partitions). >>>> >>>> What I noticed is that the event is fired on all nodes that are still >> in >>>> topology, but for different partitions. For example, if there are 4 >>>> partitions lost (1,2,3,4), I can get notifications for 1,3 on first >> node >>>> and 2,4 on second node. This seems weird and it's not clear to me how >> to >>>> use this event. >>>> >>>> Can anyone explain this behavior? Is this expected? >>>> >>>> Here is the test code: >>>> >>>> public class PartitionTest { >>>> public static void main(String[] args) throws InterruptedException { >>>> for (int i = 0; i < 4; i++) >>>> Ignition.start(config("server-" + i)); >>>> >>>> Ignition.setClientMode(true); >>>> >>>> Ignite ignite = Ignition.start(); >>>> >>>> CacheConfiguration<Integer, Integer> cfg = new >>>> CacheConfiguration<>("test"); >>>> >>>> cfg.setAffinity(new RendezvousAffinityFunction(20, null)); >>>> >>>> IgniteCache<Integer, Integer> cache = ignite.createCache(cfg); >>>> >>>> for (int i = 0; i < 100; i++) >>>> cache.put(i, i); >>>> >>>> System.out.println("Populated."); >>>> >>>> Thread.sleep(2000); >>>> >>>> Ignition.stop("server-3", true); >>>> } >>>> >>>> private static IgniteConfiguration config(String name) { >>>> IgniteConfiguration cfg = new IgniteConfiguration(); >>>> >>>> cfg.setGridName(name); >>>> >>>> cfg.setIncludeEventTypes(EventType.EVT_CACHE_REBALANCE_ >> PART_DATA_LOST); >>>> cfg.setLocalEventListeners(Collections.<IgnitePredicate<? >> extends >>>> Event>, int[]>singletonMap( >>>> new IgnitePredicate<Event>() { >>>> @Override public boolean apply(Event event) { >>>> U.debug("EVENT: " + event); >>>> >>>> return true; >>>> } >>>> }, >>>> new int[] {EventType.EVT_CACHE_REBALANCE_PART_DATA_LOST} >>>> )); >>>> >>>> return cfg; >>>> } >>>> } >>>> >>>> -Val >>> >>> >> |
«
Return to Apache Ignite Developers - Legacy Mail Archive
|
1 view|%1 views
| Free forum by Nabble | Edit this page |