Guys,
We had a problem with thread starvation in PUBLIC pool because of queries as described in https://issues.apache.org/jira/browse/IGNITE-1174 I switched it to SYSTEM pool but this can cause delays for other cache operations like put/get if we have many long running queries. I propose adding one more thread pool for queries to make them independent. Thoughts? Sergi |
Were the benchmarks run on the patch attached in the JIRA?
On Fri, Jul 31, 2015 at 12:36 PM, Sergi Vladykin <[hidden email]> wrote: > Guys, > > We had a problem with thread starvation in PUBLIC pool because of queries > as described in https://issues.apache.org/jira/browse/IGNITE-1174 > > I switched it to SYSTEM pool but this can cause delays for other cache > operations like put/get if we have many long running queries. > > I propose adding one more thread pool for queries to make them independent. > > Thoughts? > > Sergi > -- Regards, Atri *l'apprenant* |
In reply to this post by Sergi
On Fri, Jul 31, 2015 at 12:06 AM, Sergi Vladykin <[hidden email]>
wrote: > Guys, > > We had a problem with thread starvation in PUBLIC pool because of queries > as described in https://issues.apache.org/jira/browse/IGNITE-1174 > > I switched it to SYSTEM pool but this can cause delays for other cache > operations like put/get if we have many long running queries. > > I propose adding one more thread pool for queries to make them independent. > I like the idea, however, I think the design should be more generic. I think Ignite should be able to have N thread pools. Then for each set of operations, e.g. task-execution, message-replies, query-processing, etc. user should be able to assign any of the available N thread-pools. This can be done in the configuration. I believe this approach leaves the door open to have more thread-pools down the road. > Thoughts? > > Sergi > |
In reply to this post by Atri Sharma
Sergi,
What is the difference between system and public pool? It is possible to get thread starvation in this case also? In general, separate thread pool looks good for me, or we need to somehow to balance queries and cache operations. -- Alexey Kuznetsov GridGain Systems www.gridgain.com |
In reply to this post by dsetrakyan
On Fri, Jul 31, 2015 at 12:30 AM, Dmitriy Setrakyan <[hidden email]>
wrote: > On Fri, Jul 31, 2015 at 12:06 AM, Sergi Vladykin <[hidden email] > > > wrote: > > > Guys, > > > > We had a problem with thread starvation in PUBLIC pool because of queries > > as described in https://issues.apache.org/jira/browse/IGNITE-1174 > > > > I switched it to SYSTEM pool but this can cause delays for other cache > > operations like put/get if we have many long running queries. > > > > I propose adding one more thread pool for queries to make them > independent. > > > > I like the idea, however, I think the design should be more generic. I > think Ignite should be able to have N thread pools. Then for each set of > operations, e.g. task-execution, message-replies, query-processing, etc. > user should be able to assign any of the available N thread-pools. This can > be done in the configuration. > We actually already have similar design for plugins (see IoPool extension). I think we can reuse it for all pools in the system. > > > Thoughts? > > > > Sergi > > > |
In reply to this post by Atri Sharma
Atri,
I don't think that we'll see any meaningful changes in benchmark results. There can be little throughput drop but I always can fix this by changing number of threads in system pool so that it will work the same way as it did with public pool. Sergi On Jul 31, 2015 10:12 AM, "Atri Sharma" <[hidden email]> wrote: > Were the benchmarks run on the patch attached in the JIRA? > > On Fri, Jul 31, 2015 at 12:36 PM, Sergi Vladykin <[hidden email] > > > wrote: > > > Guys, > > > > We had a problem with thread starvation in PUBLIC pool because of queries > > as described in https://issues.apache.org/jira/browse/IGNITE-1174 > > > > I switched it to SYSTEM pool but this can cause delays for other cache > > operations like put/get if we have many long running queries. > > > > I propose adding one more thread pool for queries to make them > independent. > > > > Thoughts? > > > > Sergi > > > > > > -- > Regards, > > Atri > *l'apprenant* > |
In reply to this post by Valentin Kulichenko
Agree with Val, we already have multiple pools support, the only thing we
should take care about is backward compatibility, when new node sends message to an old one which does not have a needed pool yet. We can handle it manually each time, but I prefer just to fallback to some default pool in old nodes to avoid complexity. Sergi On Jul 31, 2015 10:50 AM, "Valentin Kulichenko" < [hidden email]> wrote: > On Fri, Jul 31, 2015 at 12:30 AM, Dmitriy Setrakyan <[hidden email] > > > wrote: > > > On Fri, Jul 31, 2015 at 12:06 AM, Sergi Vladykin < > [hidden email] > > > > > wrote: > > > > > Guys, > > > > > > We had a problem with thread starvation in PUBLIC pool because of > queries > > > as described in https://issues.apache.org/jira/browse/IGNITE-1174 > > > > > > I switched it to SYSTEM pool but this can cause delays for other cache > > > operations like put/get if we have many long running queries. > > > > > > I propose adding one more thread pool for queries to make them > > independent. > > > > > > > I like the idea, however, I think the design should be more generic. I > > think Ignite should be able to have N thread pools. Then for each set of > > operations, e.g. task-execution, message-replies, query-processing, etc. > > user should be able to assign any of the available N thread-pools. This > can > > be done in the configuration. > > > > We actually already have similar design for plugins (see IoPool extension). > I think we can reuse it for all pools in the system. > > > > > > > Thoughts? > > > > > > Sergi > > > > > > |
Is there a ticket about supporting multiple thread pools? I wanted to leave
some design thoughts on this subject, but can't see to find the appropriate ticket for it. On Fri, Jul 31, 2015 at 6:49 AM, Sergi Vladykin <[hidden email]> wrote: > Agree with Val, we already have multiple pools support, the only thing we > should take care about is backward compatibility, when new node sends > message to an old one which does not have a needed pool yet. We can handle > it manually each time, but I prefer just to fallback to some default pool > in old nodes to avoid complexity. > > Sergi > On Jul 31, 2015 10:50 AM, "Valentin Kulichenko" < > [hidden email]> wrote: > > > On Fri, Jul 31, 2015 at 12:30 AM, Dmitriy Setrakyan < > [hidden email] > > > > > wrote: > > > > > On Fri, Jul 31, 2015 at 12:06 AM, Sergi Vladykin < > > [hidden email] > > > > > > > wrote: > > > > > > > Guys, > > > > > > > > We had a problem with thread starvation in PUBLIC pool because of > > queries > > > > as described in https://issues.apache.org/jira/browse/IGNITE-1174 > > > > > > > > I switched it to SYSTEM pool but this can cause delays for other > cache > > > > operations like put/get if we have many long running queries. > > > > > > > > I propose adding one more thread pool for queries to make them > > > independent. > > > > > > > > > > I like the idea, however, I think the design should be more generic. I > > > think Ignite should be able to have N thread pools. Then for each set > of > > > operations, e.g. task-execution, message-replies, query-processing, > etc. > > > user should be able to assign any of the available N thread-pools. This > > can > > > be done in the configuration. > > > > > > > We actually already have similar design for plugins (see IoPool > extension). > > I think we can reuse it for all pools in the system. > > > > > > > > > > > Thoughts? > > > > > > > > Sergi > > > > > > > > > > |
I created one here:
https://issues.apache.org/jira/browse/IGNITE-1193 Sergi 2015-08-04 3:21 GMT+03:00 Dmitriy Setrakyan <[hidden email]>: > Is there a ticket about supporting multiple thread pools? I wanted to leave > some design thoughts on this subject, but can't see to find the appropriate > ticket for it. > > On Fri, Jul 31, 2015 at 6:49 AM, Sergi Vladykin <[hidden email]> > wrote: > > > Agree with Val, we already have multiple pools support, the only thing we > > should take care about is backward compatibility, when new node sends > > message to an old one which does not have a needed pool yet. We can > handle > > it manually each time, but I prefer just to fallback to some default pool > > in old nodes to avoid complexity. > > > > Sergi > > On Jul 31, 2015 10:50 AM, "Valentin Kulichenko" < > > [hidden email]> wrote: > > > > > On Fri, Jul 31, 2015 at 12:30 AM, Dmitriy Setrakyan < > > [hidden email] > > > > > > > wrote: > > > > > > > On Fri, Jul 31, 2015 at 12:06 AM, Sergi Vladykin < > > > [hidden email] > > > > > > > > > wrote: > > > > > > > > > Guys, > > > > > > > > > > We had a problem with thread starvation in PUBLIC pool because of > > > queries > > > > > as described in https://issues.apache.org/jira/browse/IGNITE-1174 > > > > > > > > > > I switched it to SYSTEM pool but this can cause delays for other > > cache > > > > > operations like put/get if we have many long running queries. > > > > > > > > > > I propose adding one more thread pool for queries to make them > > > > independent. > > > > > > > > > > > > > I like the idea, however, I think the design should be more generic. > I > > > > think Ignite should be able to have N thread pools. Then for each set > > of > > > > operations, e.g. task-execution, message-replies, query-processing, > > etc. > > > > user should be able to assign any of the available N thread-pools. > This > > > can > > > > be done in the configuration. > > > > > > > > > > We actually already have similar design for plugins (see IoPool > > extension). > > > I think we can reuse it for all pools in the system. > > > > > > > > > > > > > > > Thoughts? > > > > > > > > > > Sergi > > > > > > > > > > > > > > > |
We already worked on the problem a bit. Now IO pool is not an enum, but a
byte. We have 0..31 bytes reserved for Ignite pools (and some of them are already used for existing pools), and the rest values are for plugins which can register their own pools and send messages to them. It seems to me that only minor changes to support backward compatibility are required here. On Tue, Aug 4, 2015 at 11:25 AM, Sergi Vladykin <[hidden email]> wrote: > I created one here: > https://issues.apache.org/jira/browse/IGNITE-1193 > > Sergi > > 2015-08-04 3:21 GMT+03:00 Dmitriy Setrakyan <[hidden email]>: > > > Is there a ticket about supporting multiple thread pools? I wanted to > leave > > some design thoughts on this subject, but can't see to find the > appropriate > > ticket for it. > > > > On Fri, Jul 31, 2015 at 6:49 AM, Sergi Vladykin < > [hidden email]> > > wrote: > > > > > Agree with Val, we already have multiple pools support, the only thing > we > > > should take care about is backward compatibility, when new node sends > > > message to an old one which does not have a needed pool yet. We can > > handle > > > it manually each time, but I prefer just to fallback to some default > pool > > > in old nodes to avoid complexity. > > > > > > Sergi > > > On Jul 31, 2015 10:50 AM, "Valentin Kulichenko" < > > > [hidden email]> wrote: > > > > > > > On Fri, Jul 31, 2015 at 12:30 AM, Dmitriy Setrakyan < > > > [hidden email] > > > > > > > > > wrote: > > > > > > > > > On Fri, Jul 31, 2015 at 12:06 AM, Sergi Vladykin < > > > > [hidden email] > > > > > > > > > > > wrote: > > > > > > > > > > > Guys, > > > > > > > > > > > > We had a problem with thread starvation in PUBLIC pool because of > > > > queries > > > > > > as described in > https://issues.apache.org/jira/browse/IGNITE-1174 > > > > > > > > > > > > I switched it to SYSTEM pool but this can cause delays for other > > > cache > > > > > > operations like put/get if we have many long running queries. > > > > > > > > > > > > I propose adding one more thread pool for queries to make them > > > > > independent. > > > > > > > > > > > > > > > > I like the idea, however, I think the design should be more > generic. > > I > > > > > think Ignite should be able to have N thread pools. Then for each > set > > > of > > > > > operations, e.g. task-execution, message-replies, query-processing, > > > etc. > > > > > user should be able to assign any of the available N thread-pools. > > This > > > > can > > > > > be done in the configuration. > > > > > > > > > > > > > We actually already have similar design for plugins (see IoPool > > > extension). > > > > I think we can reuse it for all pools in the system. > > > > > > > > > > > > > > > > > > > Thoughts? > > > > > > > > > > > > Sergi > > > > > > > > > > > > > > > > > > > > > |
Free forum by Nabble | Edit this page |