Which thread is used to run IgniteFuture continuations?

[Vladimir Ozerov]

Igniters,

We are missing an ability to specify which thread should run continuation
routine when future is completed.

Currently we either run routine in the callee thread or in completion
thread (usually system pool thread). It means user cannot have any blocking
or cache operations inside the continuation. The code below could surprise
user with either a deadlock or system pool starvation:

final CountDownLatch latch = new CountDownLatch();

Cache cache = ignite.cache().withAsync();
cache.invoke(...);

cache.future().listen(() => {
    latch.await();
    /** Do something useful. */
});

/** Do something else. */
latch.countDown();

Java 8 and Hazelcast already support custom thread pools for continuations.
E.g.:
Hazelcast.CompletableFutureTask.andThen(ExecutionCallback<V> callback,
Executor executor);

Looks like we should allow users to specify optional thread pool in futures
likewise.

Thoughts?

Vladimir.

[Andrey Kornev]

Excellent idea! +10
PS. Got burnt by this a few times already
    _____________________________
From: Vladimir Ozerov <[hidden email]>
Sent: Friday, October 9, 2015 4:22 PM
Subject: Which thread is used to run IgniteFuture continuations?
To:  <[hidden email]>


                   Igniters,  
   
 We are missing an ability to specify which thread should run continuation  
 routine when future is completed.  
   
 Currently we either run routine in the callee thread or in completion  
 thread (usually system pool thread). It means user cannot have any blocking  
 or cache operations inside the continuation. The code below could surprise  
 user with either a deadlock or system pool starvation:  
   
 final CountDownLatch latch = new CountDownLatch();  
   
 Cache cache = ignite.cache().withAsync();  
 cache.invoke(...);  
   
 cache.future().listen(() => {  
     latch.await();  
     /** Do something useful. */  
 });  
   
 /** Do something else. */  
 latch.countDown();  
   
 Java 8 and Hazelcast already support custom thread pools for continuations.  
 E.g.:  
 Hazelcast.CompletableFutureTask.andThen(ExecutionCallback<V> callback,  
 Executor executor);  
   
 Looks like we should allow users to specify optional thread pool in futures  
 likewise.  
   
 Thoughts?  
   
 Vladimir.

[Dmitriy Setrakyan]

I will add +1 on this, which together with Andrey's +10, brings the total
to +11 :)

D.

On Fri, Oct 9, 2015 at 7:48 AM, Andrey Kornev <[hidden email]>
wrote:

> Excellent idea! +10
> PS. Got burnt by this a few times already
>     _____________________________
> From: Vladimir Ozerov <[hidden email]>
> Sent: Friday, October 9, 2015 4:22 PM
> Subject: Which thread is used to run IgniteFuture continuations?
> To:  <[hidden email]>
>
>
>                    Igniters,
>
>  We are missing an ability to specify which thread should run continuation
>  routine when future is completed.
>
>  Currently we either run routine in the callee thread or in completion
>  thread (usually system pool thread). It means user cannot have any
> blocking
>  or cache operations inside the continuation. The code below could surprise
>  user with either a deadlock or system pool starvation:
>
>  final CountDownLatch latch = new CountDownLatch();
>
>  Cache cache = ignite.cache().withAsync();
>  cache.invoke(...);
>
>  cache.future().listen(() => {
>      latch.await();
>      /** Do something useful. */
>  });
>
>  /** Do something else. */
>  latch.countDown();
>
>  Java 8 and Hazelcast already support custom thread pools for
> continuations.
>  E.g.:
>  Hazelcast.CompletableFutureTask.andThen(ExecutionCallback<V> callback,
>  Executor executor);
>
>  Looks like we should allow users to specify optional thread pool in
> futures
>  likewise.
>
>  Thoughts?
>
>  Vladimir.
>

[yzhdanov]

Not sure if I get the point. You can do exactly the same inside your
listener.

final Executor executor = ...;

// Bla-bla

cache.future().listen(() => {
    executor.execute(new Runnable() {
        latch.await();
        /** Do something useful. */
    }
});

--Yakov

2015-10-09 17:22 GMT+03:00 Vladimir Ozerov <[hidden email]>:

> Igniters,
>
> We are missing an ability to specify which thread should run continuation
> routine when future is completed.
>
> Currently we either run routine in the callee thread or in completion
> thread (usually system pool thread). It means user cannot have any blocking
> or cache operations inside the continuation. The code below could surprise
> user with either a deadlock or system pool starvation:
>
> final CountDownLatch latch = new CountDownLatch();
>
> Cache cache = ignite.cache().withAsync();
> cache.invoke(...);
>
> cache.future().listen(() => {
>     latch.await();
>     /** Do something useful. */
> });
>
> /** Do something else. */
> latch.countDown();
>
> Java 8 and Hazelcast already support custom thread pools for continuations.
> E.g.:
> Hazelcast.CompletableFutureTask.andThen(ExecutionCallback<V> callback,
> Executor executor);
>
> Looks like we should allow users to specify optional thread pool in futures
> likewise.
>
> Thoughts?
>
> Vladimir.
>

[Vladimir Ozerov]

Yakov, two points there:

1) This is a matter of convenience. Current mainstream solution for similar
tasks is Java's CompletableFuture. Here is what it offers:
https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html#runAsync-java.lang.Runnable-
https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html#runAsync-java.lang.Runnable-java.util.concurrent.Executor-

2) Probably we should go event further and do not execute continuations in
user thread or our system threads by default because this is error prone
and is subject to deadlocks and unpredictable slowdowns in normal
compute/cache operations. By default we can either delegate to FJP, provide
configurable pool for continuations, or use Ignite public pool.

On Mon, Oct 12, 2015 at 4:25 PM, Yakov Zhdanov <[hidden email]> wrote:

> Not sure if I get the point. You can do exactly the same inside your
> listener.
>
> final Executor executor = ...;
>
> // Bla-bla
>
> cache.future().listen(() => {
>     executor.execute(new Runnable() {
>         latch.await();
>         /** Do something useful. */
>     }
> });
>
> --Yakov
>
> 2015-10-09 17:22 GMT+03:00 Vladimir Ozerov <[hidden email]>:
>
> > Igniters,
> >
> > We are missing an ability to specify which thread should run continuation
> > routine when future is completed.
> >
> > Currently we either run routine in the callee thread or in completion
> > thread (usually system pool thread). It means user cannot have any
> blocking
> > or cache operations inside the continuation. The code below could
> surprise
> > user with either a deadlock or system pool starvation:
> >
> > final CountDownLatch latch = new CountDownLatch();
> >
> > Cache cache = ignite.cache().withAsync();
> > cache.invoke(...);
> >
> > cache.future().listen(() => {
> >     latch.await();
> >     /** Do something useful. */
> > });
> >
> > /** Do something else. */
> > latch.countDown();
> >
> > Java 8 and Hazelcast already support custom thread pools for
> continuations.
> > E.g.:
> > Hazelcast.CompletableFutureTask.andThen(ExecutionCallback<V> callback,
> > Executor executor);
> >
> > Looks like we should allow users to specify optional thread pool in
> futures
> > likewise.
> >
> > Thoughts?
> >
> > Vladimir.
> >
>

[dsetrakyan]

Would it be possible for us to support CompletableFuture from Java instead
of adding one-off methods?

On Mon, Oct 12, 2015 at 6:53 AM, Vladimir Ozerov <[hidden email]>
wrote:

> Yakov, two points there:
>
> 1) This is a matter of convenience. Current mainstream solution for similar
> tasks is Java's CompletableFuture. Here is what it offers:
>
> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html#runAsync-java.lang.Runnable-
>
> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html#runAsync-java.lang.Runnable-java.util.concurrent.Executor-
>
> 2) Probably we should go event further and do not execute continuations in
> user thread or our system threads by default because this is error prone
> and is subject to deadlocks and unpredictable slowdowns in normal
> compute/cache operations. By default we can either delegate to FJP, provide
> configurable pool for continuations, or use Ignite public pool.
>
> On Mon, Oct 12, 2015 at 4:25 PM, Yakov Zhdanov <[hidden email]>
> wrote:
>
> > Not sure if I get the point. You can do exactly the same inside your
> > listener.
> >
> > final Executor executor = ...;
> >
> > // Bla-bla
> >
> > cache.future().listen(() => {
> >     executor.execute(new Runnable() {
> >         latch.await();
> >         /** Do something useful. */
> >     }
> > });
> >
> > --Yakov
> >
> > 2015-10-09 17:22 GMT+03:00 Vladimir Ozerov <[hidden email]>:
> >
> > > Igniters,
> > >
> > > We are missing an ability to specify which thread should run
> continuation
> > > routine when future is completed.
> > >
> > > Currently we either run routine in the callee thread or in completion
> > > thread (usually system pool thread). It means user cannot have any
> > blocking
> > > or cache operations inside the continuation. The code below could
> > surprise
> > > user with either a deadlock or system pool starvation:
> > >
> > > final CountDownLatch latch = new CountDownLatch();
> > >
> > > Cache cache = ignite.cache().withAsync();
> > > cache.invoke(...);
> > >
> > > cache.future().listen(() => {
> > >     latch.await();
> > >     /** Do something useful. */
> > > });
> > >
> > > /** Do something else. */
> > > latch.countDown();
> > >
> > > Java 8 and Hazelcast already support custom thread pools for
> > continuations.
> > > E.g.:
> > > Hazelcast.CompletableFutureTask.andThen(ExecutionCallback<V> callback,
> > > Executor executor);
> > >
> > > Looks like we should allow users to specify optional thread pool in
> > futures
> > > likewise.
> > >
> > > Thoughts?
> > >
> > > Vladimir.
> > >
> >
>

[Vladimir Ozerov]

It would be cool, but CompletableFuture is available since Java 8.

On Mon, Oct 12, 2015 at 7:25 PM, Dmitriy Setrakyan <[hidden email]>
wrote:

> Would it be possible for us to support CompletableFuture from Java instead
> of adding one-off methods?
>
> On Mon, Oct 12, 2015 at 6:53 AM, Vladimir Ozerov <[hidden email]>
> wrote:
>
> > Yakov, two points there:
> >
> > 1) This is a matter of convenience. Current mainstream solution for
> similar
> > tasks is Java's CompletableFuture. Here is what it offers:
> >
> >
> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html#runAsync-java.lang.Runnable-
> >
> >
> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html#runAsync-java.lang.Runnable-java.util.concurrent.Executor-
> >
> > 2) Probably we should go event further and do not execute continuations
> in
> > user thread or our system threads by default because this is error prone
> > and is subject to deadlocks and unpredictable slowdowns in normal
> > compute/cache operations. By default we can either delegate to FJP,
> provide
> > configurable pool for continuations, or use Ignite public pool.
> >
> > On Mon, Oct 12, 2015 at 4:25 PM, Yakov Zhdanov <[hidden email]>
> > wrote:
> >
> > > Not sure if I get the point. You can do exactly the same inside your
> > > listener.
> > >
> > > final Executor executor = ...;
> > >
> > > // Bla-bla
> > >
> > > cache.future().listen(() => {
> > >     executor.execute(new Runnable() {
> > >         latch.await();
> > >         /** Do something useful. */
> > >     }
> > > });
> > >
> > > --Yakov
> > >
> > > 2015-10-09 17:22 GMT+03:00 Vladimir Ozerov <[hidden email]>:
> > >
> > > > Igniters,
> > > >
> > > > We are missing an ability to specify which thread should run
> > continuation
> > > > routine when future is completed.
> > > >
> > > > Currently we either run routine in the callee thread or in completion
> > > > thread (usually system pool thread). It means user cannot have any
> > > blocking
> > > > or cache operations inside the continuation. The code below could
> > > surprise
> > > > user with either a deadlock or system pool starvation:
> > > >
> > > > final CountDownLatch latch = new CountDownLatch();
> > > >
> > > > Cache cache = ignite.cache().withAsync();
> > > > cache.invoke(...);
> > > >
> > > > cache.future().listen(() => {
> > > >     latch.await();
> > > >     /** Do something useful. */
> > > > });
> > > >
> > > > /** Do something else. */
> > > > latch.countDown();
> > > >
> > > > Java 8 and Hazelcast already support custom thread pools for
> > > continuations.
> > > > E.g.:
> > > > Hazelcast.CompletableFutureTask.andThen(ExecutionCallback<V>
> callback,
> > > > Executor executor);
> > > >
> > > > Looks like we should allow users to specify optional thread pool in
> > > futures
> > > > likewise.
> > > >
> > > > Thoughts?
> > > >
> > > > Vladimir.
> > > >
> > >
> >
>

[dsetrakyan]

On Mon, Oct 12, 2015 at 9:39 AM, Vladimir Ozerov <[hidden email]>
wrote:

> It would be cool, but CompletableFuture is available since Java 8.
>

Well, we could add some methods from CompletableFuture to IgniteFuture
then. I will take a look and suggest a few.

>
> On Mon, Oct 12, 2015 at 7:25 PM, Dmitriy Setrakyan <[hidden email]>
> wrote:
>
> > Would it be possible for us to support CompletableFuture from Java
> instead
> > of adding one-off methods?
> >
> > On Mon, Oct 12, 2015 at 6:53 AM, Vladimir Ozerov <[hidden email]>
> > wrote:
> >
> > > Yakov, two points there:
> > >
> > > 1) This is a matter of convenience. Current mainstream solution for
> > similar
> > > tasks is Java's CompletableFuture. Here is what it offers:
> > >
> > >
> >
> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html#runAsync-java.lang.Runnable-
> > >
> > >
> >
> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html#runAsync-java.lang.Runnable-java.util.concurrent.Executor-
> > >
> > > 2) Probably we should go event further and do not execute continuations
> > in
> > > user thread or our system threads by default because this is error
> prone
> > > and is subject to deadlocks and unpredictable slowdowns in normal
> > > compute/cache operations. By default we can either delegate to FJP,
> > provide
> > > configurable pool for continuations, or use Ignite public pool.
> > >
> > > On Mon, Oct 12, 2015 at 4:25 PM, Yakov Zhdanov <[hidden email]>
> > > wrote:
> > >
> > > > Not sure if I get the point. You can do exactly the same inside your
> > > > listener.
> > > >
> > > > final Executor executor = ...;
> > > >
> > > > // Bla-bla
> > > >
> > > > cache.future().listen(() => {
> > > >     executor.execute(new Runnable() {
> > > >         latch.await();
> > > >         /** Do something useful. */
> > > >     }
> > > > });
> > > >
> > > > --Yakov
> > > >
> > > > 2015-10-09 17:22 GMT+03:00 Vladimir Ozerov <[hidden email]>:
> > > >
> > > > > Igniters,
> > > > >
> > > > > We are missing an ability to specify which thread should run
> > > continuation
> > > > > routine when future is completed.
> > > > >
> > > > > Currently we either run routine in the callee thread or in
> completion
> > > > > thread (usually system pool thread). It means user cannot have any
> > > > blocking
> > > > > or cache operations inside the continuation. The code below could
> > > > surprise
> > > > > user with either a deadlock or system pool starvation:
> > > > >
> > > > > final CountDownLatch latch = new CountDownLatch();
> > > > >
> > > > > Cache cache = ignite.cache().withAsync();
> > > > > cache.invoke(...);
> > > > >
> > > > > cache.future().listen(() => {
> > > > >     latch.await();
> > > > >     /** Do something useful. */
> > > > > });
> > > > >
> > > > > /** Do something else. */
> > > > > latch.countDown();
> > > > >
> > > > > Java 8 and Hazelcast already support custom thread pools for
> > > > continuations.
> > > > > E.g.:
> > > > > Hazelcast.CompletableFutureTask.andThen(ExecutionCallback<V>
> > callback,
> > > > > Executor executor);
> > > > >
> > > > > Looks like we should allow users to specify optional thread pool in
> > > > futures
> > > > > likewise.
> > > > >
> > > > > Thoughts?
> > > > >
> > > > > Vladimir.
> > > > >
> > > >
> > >
> >
>

[Vladimir Ozerov]

Folks,

It looks like the problem is deeper and is not limited to futures. It
relates to any user code. For example:

1) Register either local or remote message listener:
Ignite.message().localListen("myTopic", myLsnr);

2) Send a message to the topic from the same node:
Ignite.message().send("myTopic", "Hello world!");

As a result, listener will be invoked synchronously from the same thread.
It means we cannot utilize resources efficiently in case of local message.

On Mon, Oct 12, 2015 at 7:51 PM, Dmitriy Setrakyan <[hidden email]>
wrote:

> On Mon, Oct 12, 2015 at 9:39 AM, Vladimir Ozerov <[hidden email]>
> wrote:
>
> > It would be cool, but CompletableFuture is available since Java 8.
> >
>
> Well, we could add some methods from CompletableFuture to IgniteFuture
> then. I will take a look and suggest a few.
>
>
> >
> > On Mon, Oct 12, 2015 at 7:25 PM, Dmitriy Setrakyan <
> [hidden email]>
> > wrote:
> >
> > > Would it be possible for us to support CompletableFuture from Java
> > instead
> > > of adding one-off methods?
> > >
> > > On Mon, Oct 12, 2015 at 6:53 AM, Vladimir Ozerov <[hidden email]
> >
> > > wrote:
> > >
> > > > Yakov, two points there:
> > > >
> > > > 1) This is a matter of convenience. Current mainstream solution for
> > > similar
> > > > tasks is Java's CompletableFuture. Here is what it offers:
> > > >
> > > >
> > >
> >
> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html#runAsync-java.lang.Runnable-
> > > >
> > > >
> > >
> >
> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html#runAsync-java.lang.Runnable-java.util.concurrent.Executor-
> > > >
> > > > 2) Probably we should go event further and do not execute
> continuations
> > > in
> > > > user thread or our system threads by default because this is error
> > prone
> > > > and is subject to deadlocks and unpredictable slowdowns in normal
> > > > compute/cache operations. By default we can either delegate to FJP,
> > > provide
> > > > configurable pool for continuations, or use Ignite public pool.
> > > >
> > > > On Mon, Oct 12, 2015 at 4:25 PM, Yakov Zhdanov <[hidden email]>
> > > > wrote:
> > > >
> > > > > Not sure if I get the point. You can do exactly the same inside
> your
> > > > > listener.
> > > > >
> > > > > final Executor executor = ...;
> > > > >
> > > > > // Bla-bla
> > > > >
> > > > > cache.future().listen(() => {
> > > > >     executor.execute(new Runnable() {
> > > > >         latch.await();
> > > > >         /** Do something useful. */
> > > > >     }
> > > > > });
> > > > >
> > > > > --Yakov
> > > > >
> > > > > 2015-10-09 17:22 GMT+03:00 Vladimir Ozerov <[hidden email]>:
> > > > >
> > > > > > Igniters,
> > > > > >
> > > > > > We are missing an ability to specify which thread should run
> > > > continuation
> > > > > > routine when future is completed.
> > > > > >
> > > > > > Currently we either run routine in the callee thread or in
> > completion
> > > > > > thread (usually system pool thread). It means user cannot have
> any
> > > > > blocking
> > > > > > or cache operations inside the continuation. The code below could
> > > > > surprise
> > > > > > user with either a deadlock or system pool starvation:
> > > > > >
> > > > > > final CountDownLatch latch = new CountDownLatch();
> > > > > >
> > > > > > Cache cache = ignite.cache().withAsync();
> > > > > > cache.invoke(...);
> > > > > >
> > > > > > cache.future().listen(() => {
> > > > > >     latch.await();
> > > > > >     /** Do something useful. */
> > > > > > });
> > > > > >
> > > > > > /** Do something else. */
> > > > > > latch.countDown();
> > > > > >
> > > > > > Java 8 and Hazelcast already support custom thread pools for
> > > > > continuations.
> > > > > > E.g.:
> > > > > > Hazelcast.CompletableFutureTask.andThen(ExecutionCallback<V>
> > > callback,
> > > > > > Executor executor);
> > > > > >
> > > > > > Looks like we should allow users to specify optional thread pool
> in
> > > > > futures
> > > > > > likewise.
> > > > > >
> > > > > > Thoughts?
> > > > > >
> > > > > > Vladimir.
> > > > > >
> > > > >
> > > >
> > >
> >
>

[dsetrakyan]

On Thu, Oct 15, 2015 at 9:59 AM, Vladimir Ozerov <[hidden email]>
wrote:

> Folks,
>
> It looks like the problem is deeper and is not limited to futures. It
> relates to any user code. For example:
>
> 1) Register either local or remote message listener:
> Ignite.message().localListen("myTopic", myLsnr);
>
> 2) Send a message to the topic from the same node:
> Ignite.message().send("myTopic", "Hello world!");
>
> As a result, listener will be invoked synchronously from the same thread.
> It means we cannot utilize resources efficiently in case of local message.
>

I am a bit confused. Which thread is the "same" thread? Do you mean the
thread that does the listener notification? In that case, I can say that I
have performed various benchmarks in the past, and this is the most
performant way, assuming that the listener logic can execute relatively
fast and does not block the calling thread.

[Vladimir Ozerov]

Sorry, not very clear. "same" means the same thread that sent a message.
This way if we have a listener and a single thread generating messages
locally, only one CPU core will be utilizied.

On Fri, Oct 16, 2015 at 3:49 AM, Dmitriy Setrakyan <[hidden email]>
wrote:

> On Thu, Oct 15, 2015 at 9:59 AM, Vladimir Ozerov <[hidden email]>
> wrote:
>
> > Folks,
> >
> > It looks like the problem is deeper and is not limited to futures. It
> > relates to any user code. For example:
> >
> > 1) Register either local or remote message listener:
> > Ignite.message().localListen("myTopic", myLsnr);
> >
> > 2) Send a message to the topic from the same node:
> > Ignite.message().send("myTopic", "Hello world!");
> >
> > As a result, listener will be invoked synchronously from the same thread.
> > It means we cannot utilize resources efficiently in case of local
> message.
> >
>
> I am a bit confused. Which thread is the "same" thread? Do you mean the
> thread that does the listener notification? In that case, I can say that I
> have performed various benchmarks in the past, and this is the most
> performant way, assuming that the listener logic can execute relatively
> fast and does not block the calling thread.
>

[dsetrakyan]

On Thu, Oct 15, 2015 at 11:00 PM, Vladimir Ozerov <[hidden email]>
wrote:

> Sorry, not very clear. "same" means the same thread that sent a message.
> This way if we have a listener and a single thread generating messages
> locally, only one CPU core will be utilizied.
>

Still confused. How can the thread that send a message be notified in a
listener about anything? Are you talking about synchronous
request-response? In this case it is probably done on purpose.

I am still not sure what the problem is.

>
> On Fri, Oct 16, 2015 at 3:49 AM, Dmitriy Setrakyan <[hidden email]>
> wrote:
>
> > On Thu, Oct 15, 2015 at 9:59 AM, Vladimir Ozerov <[hidden email]>
> > wrote:
> >
> > > Folks,
> > >
> > > It looks like the problem is deeper and is not limited to futures. It
> > > relates to any user code. For example:
> > >
> > > 1) Register either local or remote message listener:
> > > Ignite.message().localListen("myTopic", myLsnr);
> > >
> > > 2) Send a message to the topic from the same node:
> > > Ignite.message().send("myTopic", "Hello world!");
> > >
> > > As a result, listener will be invoked synchronously from the same
> thread.
> > > It means we cannot utilize resources efficiently in case of local
> > message.
> > >
> >
> > I am a bit confused. Which thread is the "same" thread? Do you mean the
> > thread that does the listener notification? In that case, I can say that
> I
> > have performed various benchmarks in the past, and this is the most
> > performant way, assuming that the listener logic can execute relatively
> > fast and does not block the calling thread.
> >
>

[Vladimir Ozerov]

Dima,

If user call IgniteMessaging.send() in the thread T1, then local message
listeners on this topic will be notified in the same thread before
returning from send() method. This our pseudo-code:

IgniteMessaging.send(Object topic, Object msg) {
    ...
    sendToRemoteNodes(topic, msg);
    ...
    for (Listener lsnr : localTopicListeners)
        lsnr.notify(msg);
}

If user produces a burst of messages in a single thread, he will be very
surprised that remote node process them concurrently and utilize all CPUs,
while local node use only 1 core for this.

Vladimir.

On Fri, Oct 16, 2015 at 11:01 AM, Dmitriy Setrakyan <[hidden email]>
wrote:

> On Thu, Oct 15, 2015 at 11:00 PM, Vladimir Ozerov <[hidden email]>
> wrote:
>
> > Sorry, not very clear. "same" means the same thread that sent a message.
> > This way if we have a listener and a single thread generating messages
> > locally, only one CPU core will be utilizied.
> >
>
> Still confused. How can the thread that send a message be notified in a
> listener about anything? Are you talking about synchronous
> request-response? In this case it is probably done on purpose.
>
> I am still not sure what the problem is.
>
>
> >
> > On Fri, Oct 16, 2015 at 3:49 AM, Dmitriy Setrakyan <
> [hidden email]>
> > wrote:
> >
> > > On Thu, Oct 15, 2015 at 9:59 AM, Vladimir Ozerov <[hidden email]
> >
> > > wrote:
> > >
> > > > Folks,
> > > >
> > > > It looks like the problem is deeper and is not limited to futures. It
> > > > relates to any user code. For example:
> > > >
> > > > 1) Register either local or remote message listener:
> > > > Ignite.message().localListen("myTopic", myLsnr);
> > > >
> > > > 2) Send a message to the topic from the same node:
> > > > Ignite.message().send("myTopic", "Hello world!");
> > > >
> > > > As a result, listener will be invoked synchronously from the same
> > thread.
> > > > It means we cannot utilize resources efficiently in case of local
> > > message.
> > > >
> > >
> > > I am a bit confused. Which thread is the "same" thread? Do you mean the
> > > thread that does the listener notification? In that case, I can say
> that
> > I
> > > have performed various benchmarks in the past, and this is the most
> > > performant way, assuming that the listener logic can execute relatively
> > > fast and does not block the calling thread.
> > >
> >
>

[dsetrakyan]

Vova,

Are we notifying listeners that a message has been sent? I thought that you
were talking about reply notifications.

D.

On Fri, Oct 16, 2015 at 2:13 AM, Vladimir Ozerov <[hidden email]>
wrote:

> Dima,
>
> If user call IgniteMessaging.send() in the thread T1, then local message
> listeners on this topic will be notified in the same thread before
> returning from send() method. This our pseudo-code:
>
> IgniteMessaging.send(Object topic, Object msg) {
>     ...
>     sendToRemoteNodes(topic, msg);
>     ...
>     for (Listener lsnr : localTopicListeners)
>         lsnr.notify(msg);
> }
>
> If user produces a burst of messages in a single thread, he will be very
> surprised that remote node process them concurrently and utilize all CPUs,
> while local node use only 1 core for this.
>
> Vladimir.
>
> On Fri, Oct 16, 2015 at 11:01 AM, Dmitriy Setrakyan <[hidden email]
> >
> wrote:
>
> > On Thu, Oct 15, 2015 at 11:00 PM, Vladimir Ozerov <[hidden email]>
> > wrote:
> >
> > > Sorry, not very clear. "same" means the same thread that sent a
> message.
> > > This way if we have a listener and a single thread generating messages
> > > locally, only one CPU core will be utilizied.
> > >
> >
> > Still confused. How can the thread that send a message be notified in a
> > listener about anything? Are you talking about synchronous
> > request-response? In this case it is probably done on purpose.
> >
> > I am still not sure what the problem is.
> >
> >
> > >
> > > On Fri, Oct 16, 2015 at 3:49 AM, Dmitriy Setrakyan <
> > [hidden email]>
> > > wrote:
> > >
> > > > On Thu, Oct 15, 2015 at 9:59 AM, Vladimir Ozerov <
> [hidden email]
> > >
> > > > wrote:
> > > >
> > > > > Folks,
> > > > >
> > > > > It looks like the problem is deeper and is not limited to futures.
> It
> > > > > relates to any user code. For example:
> > > > >
> > > > > 1) Register either local or remote message listener:
> > > > > Ignite.message().localListen("myTopic", myLsnr);
> > > > >
> > > > > 2) Send a message to the topic from the same node:
> > > > > Ignite.message().send("myTopic", "Hello world!");
> > > > >
> > > > > As a result, listener will be invoked synchronously from the same
> > > thread.
> > > > > It means we cannot utilize resources efficiently in case of local
> > > > message.
> > > > >
> > > >
> > > > I am a bit confused. Which thread is the "same" thread? Do you mean
> the
> > > > thread that does the listener notification? In that case, I can say
> > that
> > > I
> > > > have performed various benchmarks in the past, and this is the most
> > > > performant way, assuming that the listener logic can execute
> relatively
> > > > fast and does not block the calling thread.
> > > >
> > >
> >
>