Hi there,
Working with SQL queries with setLocal(true) with partitioned cache, it is very easy for someone to run SQL queries without affinityRun or affinityCall computations which are the preferred ways of running queries on partition cache, as described in [1]. Now what I was thinking whenever a SQL is about to execute against partitioned caches it should check for a check whether the call for this SQL is made through an affinityRun or affinityCall function. If the call to SQL is not part of affinityRun or affinityCall then by default it should log a WARNING message or throw an exception which should be configurable in CacheConfiguration. The advantage would be it won't break others code instantly and allow them some time to fix it. This can be achieved when the affinityCall or affinityRun method is called we can set something specifically for SQL queries in the context which can be read before executing the queries. If the SQL processor cannot find the value in the given context for partitioned caches we can either log the warning or throw an exception based on the cache configuration. Let me know if it makes sense? Thanks, Luqman [1]. https://apacheignite-sql.readme.io/docs/local-queries <https://apacheignite-sql.readme.io/docs/local-queries> -- Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ |
Sounds like a good idea. Vladimir, would be nice to hear your thoughts.
D. On Mon, Nov 20, 2017 at 7:45 AM, luqmanahmad <[hidden email]> wrote: > Hi there, > > Working with SQL queries with setLocal(true) with partitioned cache, it is > very easy for someone to run SQL queries without affinityRun or > affinityCall > computations which are the preferred ways of running queries on partition > cache, as described in [1]. > > Now what I was thinking whenever a SQL is about to execute against > partitioned caches it should check for a check whether the call for this > SQL > is made through an affinityRun or affinityCall function. If the call to SQL > is not part of affinityRun or affinityCall then by default it should log a > WARNING message or throw an exception which should be configurable in > CacheConfiguration. The advantage would be it won't break others code > instantly and allow them some time to fix it. > > This can be achieved when the affinityCall or affinityRun method is called > we can set something specifically for SQL queries in the context which can > be read before executing the queries. If the SQL processor cannot find the > value in the given context for partitioned caches we can either log the > warning or throw an exception based on the cache configuration. > > Let me know if it makes sense? > > Thanks, > Luqman > > [1]. https://apacheignite-sql.readme.io/docs/local-queries > <https://apacheignite-sql.readme.io/docs/local-queries> > > > > -- > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ > |
Thanks dsetrakyan,
I would like to add a few more things over here which should be applicable to partitioned caches. This context variable which is set through affinityCall or affinityRun should be available through either a helper class or cache configuration. There could be other advantages as well for example: 1. We can check the context variable in all the partitioned cache operations. In department and employee example if an employee is accessed without an affinityRun or affinityCall computation it should also log a WARNING message or through an exception based on the cache configuration. 2. The user would be able to implement their own custom checks using it. For example, if we want to have some abstract level checks to restrict developers to use specific functionality related to partitioned caches. Luqman -- Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ |
Hi Luqman,
I do not think SQL and compute should be coupled in the product. Instead, we should fix local query execution and pin partitions in the same way it is done for affinityCall/Run and distributed SQL. On Tue, Nov 21, 2017 at 6:25 PM, luqmanahmad <[hidden email]> wrote: > Thanks dsetrakyan, > > I would like to add a few more things over here which should be applicable > to partitioned caches. > > This context variable which is set through affinityCall or affinityRun > should be available through either a helper class or cache configuration. > There could be other advantages as well for example: > > 1. We can check the context variable in all the partitioned cache > operations. In department and employee example if an employee is accessed > without an affinityRun or affinityCall computation it should also log a > WARNING message or through an exception based on the cache configuration. > > 2. The user would be able to implement their own custom checks using it. > For > example, if we want to have some abstract level checks to restrict > developers to use specific functionality related to partitioned caches. > > Luqman > > > > -- > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ > |
Hi Vladmir,
Agree - they shouldnt be coupled togethor but what if we can set something in affinity api which can be read in sql api. Please correct me if I am wrong but in the affinityCall/Run we have to provide all the cache names and rebalancing will skip if there is already an operation in process. If we go with your approach not sure whether we can calculate all the related partitioned caches to be locked dynamically. Ofcourse you would be in a better position to comment on it but cant we introduce something in affinity api which can be set/read through each affinityCall/Run, and the affinity api can be used inside SQL api - just like the same way calculating partition id for a specific key or an finding an atomic reference. Thanks, Luqman On 21 Nov 2017 20:17, "Vladimir Ozerov" <[hidden email]> wrote: Hi Luqman, I do not think SQL and compute should be coupled in the product. Instead, we should fix local query execution and pin partitions in the same way it is done for affinityCall/Run and distributed SQL. On Tue, Nov 21, 2017 at 6:25 PM, luqmanahmad <[hidden email]> wrote: > Thanks dsetrakyan, > > I would like to add a few more things over here which should be applicable > to partitioned caches. > > This context variable which is set through affinityCall or affinityRun > should be available through either a helper class or cache configuration. > There could be other advantages as well for example: > > 1. We can check the context variable in all the partitioned cache > operations. In department and employee example if an employee is accessed > without an affinityRun or affinityCall computation it should also log a > WARNING message or through an exception based on the cache configuration. > > 2. The user would be able to implement their own custom checks using it. > For > example, if we want to have some abstract level checks to restrict > developers to use specific functionality related to partitioned caches. > > Luqman > > > > -- > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ > |
Hi Luqman,
Required caches are already derived from SQL query through Ignite SQL internals. We should just re-use this code for local queries. I filed a ticket to fix this [1]. [1] https://issues.apache.org/jira/browse/IGNITE-7039 On Wed, Nov 22, 2017 at 12:05 AM, Luqman Ahmad <[hidden email]> wrote: > Hi Vladmir, > > Agree - they shouldnt be coupled togethor but what if we can set something > in affinity api which can be read in sql api. > > Please correct me if I am wrong but in the affinityCall/Run we have to > provide all the cache names and rebalancing will skip if there is already > an operation in process. If we go with your approach not sure whether we > can calculate all the related partitioned caches to be locked dynamically. > > Ofcourse you would be in a better position to comment on it but cant we > introduce something in affinity api which can be set/read through each > affinityCall/Run, and the affinity api can be used inside SQL api - just > like the same way calculating partition id for a specific key or an finding > an atomic reference. > > Thanks, > Luqman > > > > On 21 Nov 2017 20:17, "Vladimir Ozerov" <[hidden email]> wrote: > > Hi Luqman, > > I do not think SQL and compute should be coupled in the product. Instead, > we should fix local query execution and pin partitions in the same way it > is done for affinityCall/Run and distributed SQL. > > On Tue, Nov 21, 2017 at 6:25 PM, luqmanahmad <[hidden email]> > wrote: > > > Thanks dsetrakyan, > > > > I would like to add a few more things over here which should be > applicable > > to partitioned caches. > > > > This context variable which is set through affinityCall or affinityRun > > should be available through either a helper class or cache configuration. > > There could be other advantages as well for example: > > > > 1. We can check the context variable in all the partitioned cache > > operations. In department and employee example if an employee is accessed > > without an affinityRun or affinityCall computation it should also log a > > WARNING message or through an exception based on the cache configuration. > > > > 2. The user would be able to implement their own custom checks using it. > > For > > example, if we want to have some abstract level checks to restrict > > developers to use specific functionality related to partitioned caches. > > > > Luqman > > > > > > > > -- > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ > > > |
Free forum by Nabble | Edit this page |