Implement collecting what SQL statements executed on cluster and their metrics.

[Alexey Kuznetsov-2]

Hi, All!

I'm working on issue "Implement collecting what SQL statements executed on
cluster and their metrics" [1]

For now I'm investigating code and have following questions:

1) We have different types of queries:  GridCacheQueryType: SPI, SCAN, SQL,
SQL_FIELDS, TEXT, SET
Should I implement collecting metrics for ALL of them?
Or for example I could collect:
   SCAN: cache name
   SQL: type name + SQL clauseю
   SQL_FIELDS: SQL statement text
   TEXT: type + searched string

What I could collect for SPI and SET query types? Or I may ignore them?

2) What metrics I could collect? For now I could collect per query:
  Number of executions.
  Number of failures.
  Duration.
Is there something else I could collect?


3) Configuration.
  Should we have this feature configured per cache or globally on
IgniteConfiguration?
  Should this feature be ON or OFF by default?
  How many items I could collect by default? 100? 1000?

Any comments are welcome.

------
https://issues.apache.org/jira/browse/IGNITE-3443

--
Alexey Kuznetsov

[Alexey Kuznetsov-2]

Hi All!

Just kindly remind for feedback.

Thanks!

On Thu, Jul 7, 2016 at 8:41 PM, Alexey Kuznetsov <[hidden email]>
wrote:

> Hi, All!
>
> I'm working on issue "Implement collecting what SQL statements executed on
> cluster and their metrics" [1]
>
> For now I'm investigating code and have following questions:
>
> 1) We have different types of queries:  GridCacheQueryType: SPI, SCAN,
> SQL, SQL_FIELDS, TEXT, SET
> Should I implement collecting metrics for ALL of them?
> Or for example I could collect:
>    SCAN: cache name
>    SQL: type name + SQL clauseю
>    SQL_FIELDS: SQL statement text
>    TEXT: type + searched string
>
> What I could collect for SPI and SET query types? Or I may ignore them?
>
> 2) What metrics I could collect? For now I could collect per query:
>   Number of executions.
>   Number of failures.
>   Duration.
> Is there something else I could collect?
>
>
> 3) Configuration.
>   Should we have this feature configured per cache or globally on
> IgniteConfiguration?
>   Should this feature be ON or OFF by default?
>   How many items I could collect by default? 100? 1000?
>
> Any comments are welcome.
>
> ------
> https://issues.apache.org/jira/browse/IGNITE-3443
>
> --
> Alexey Kuznetsov
>
>

--
Alexey Kuznetsov
GridGain Systems
www.gridgain.com

[dsetrakyan]

My comments are below...

On Thu, Jul 7, 2016 at 9:41 AM, Alexey Kuznetsov <[hidden email]>
wrote:

> Hi, All!
>
> I'm working on issue "Implement collecting what SQL statements executed on
> cluster and their metrics" [1]
>
> For now I'm investigating code and have following questions:
>
> 1) We have different types of queries:  GridCacheQueryType: SPI, SCAN, SQL,
> SQL_FIELDS, TEXT, SET
> Should I implement collecting metrics for ALL of them?
> Or for example I could collect:
>    SCAN: cache name
>    SQL: type name + SQL clauseю
>    SQL_FIELDS: SQL statement text
>    TEXT: type + searched string
>
> What I could collect for SPI and SET query types? Or I may ignore them?
>

I would implement stat gathering for all the known ones. SPI is user
defined type, so  I am not sure if we can provide useful stats there.

Can anyone chime in and describe what SET is for? Sergi, do you know?


> 2) What metrics I could collect? For now I could collect per query:
>   Number of executions.
>   Number of failures.
>   Duration.
> Is there something else I could collect?
>

The metrics you selected look like a good starting point. I would also add
execution plan to it, if possible.


>
>
> 3) Configuration.
>   Should we have this feature configured per cache or globally on
> IgniteConfiguration?
>   Should this feature be ON or OFF by default?
>   How many items I could collect by default? 100? 1000?
>

How about both? There should be a global setting which can be overridden on
per-cache basis.


>
> Any comments are welcome.
>
> ------
> https://issues.apache.org/jira/browse/IGNITE-3443
>
> --
> Alexey Kuznetsov
>

[Valentin Kulichenko]

SET query is for IgniteSet and it's internal only, not exposed to the
public API.

-Val

On Mon, Jul 25, 2016 at 3:37 PM, Dmitriy Setrakyan <[hidden email]>
wrote:

> My comments are below...
>
> On Thu, Jul 7, 2016 at 9:41 AM, Alexey Kuznetsov <[hidden email]>
> wrote:
>
> > Hi, All!
> >
> > I'm working on issue "Implement collecting what SQL statements executed
> on
> > cluster and their metrics" [1]
> >
> > For now I'm investigating code and have following questions:
> >
> > 1) We have different types of queries:  GridCacheQueryType: SPI, SCAN,
> SQL,
> > SQL_FIELDS, TEXT, SET
> > Should I implement collecting metrics for ALL of them?
> > Or for example I could collect:
> >    SCAN: cache name
> >    SQL: type name + SQL clauseю
> >    SQL_FIELDS: SQL statement text
> >    TEXT: type + searched string
> >
> > What I could collect for SPI and SET query types? Or I may ignore them?
> >
>
> I would implement stat gathering for all the known ones. SPI is user
> defined type, so  I am not sure if we can provide useful stats there.
>
> Can anyone chime in and describe what SET is for? Sergi, do you know?
>
>
> > 2) What metrics I could collect? For now I could collect per query:
> >   Number of executions.
> >   Number of failures.
> >   Duration.
> > Is there something else I could collect?
> >
>
> The metrics you selected look like a good starting point. I would also add
> execution plan to it, if possible.
>
>
> >
> >
> > 3) Configuration.
> >   Should we have this feature configured per cache or globally on
> > IgniteConfiguration?
> >   Should this feature be ON or OFF by default?
> >   How many items I could collect by default? 100? 1000?
> >
>
> How about both? There should be a global setting which can be overridden on
> per-cache basis.
>
>
> >
> > Any comments are welcome.
> >
> > ------
> > https://issues.apache.org/jira/browse/IGNITE-3443
> >
> > --
> > Alexey Kuznetsov
> >
>