We cannot compensate blocked threads with new ones forever. At some point
we will have to limit new threads which will bring us to the same problem,
no?
I do like the idea, however, of users creating their own thread pools and
utilizing them for execution, as proposed by Vladimir.
D.
On Mon, Feb 13, 2017 at 6:37 AM, Alexey Goncharuk <
[hidden email]> wrote:
> Vladimir,
>
> I like the idea of compensating threads with new ones. However, I do not
> see why users should notify Ignite when a new distributed operation is
> started because Ignite already is the task executor. We can intercept
> (virtually) all such calls and adjust pool size automatically. No need to
> introduce ManagedBlocker.
>
> --AG
>
> 2017-02-11 23:43 GMT+03:00 Vladimir Ozerov <
[hidden email]>:
>
> > Igniters,
> >
> > We have one well-known limitation: one distributed opeartion cannot be
> > executed from within another if they both end up in the same pool,
> because
> > it will lead to starvation and hang. Most notable example - running data
> > streamer from compute task.
> >
> > Users with complex logic will have the same issue if they want to execute
> > one job from another. I think it could be resolved with introduction of
> > custom executors and routing computing jobs to them.
> >
> > Config:
> > ExecutorConfiguration execCfg = new
> > ExecutorConfiguration().setName("myExecutor").setThreads(8);
> >
> > Starting MR:
> > ignite.compute().withExecutor("myExecutor").execute(...);
> >
> > Another improvement we can apply here is managed blockers, similar to
> > ForkJoinPool.ManagedBlocker [1]. If user is to execute distributed
> > operation, he can infrom us about it, and we will compensate blocked
> thread
> > with a new one.
> >
> > Thoughts?
> >
> > [1]
> >
https://docs.oracle.com/javase/8/docs/api/java/util/> > concurrent/ForkJoinPool.ManagedBlocker.html
> >
>
Locked
