Prepared statement concept in indexing

[Vladimir Ozerov]

Igniters,

Currently I'm profiling query execution on a single node on a realtively
small data set (100K entries). What I see is that query parsing consumes
about 20% of all execution time.

Do we have something like JDBC PreparedStatement for our queries to
minimize this overhead? If no, does it make sense to have it?

Vladimir.

[Sergi]

Yes, it is a problem and I believe after distributed joins feature will be
merged, it will become even worse.

Though I'd prefer to avoid changing public API for now. I believe it must
be enough to cache
and reuse parsed query information, generated queries and other stuff.

Sergi

2015-11-04 16:43 GMT+03:00 Vladimir Ozerov <[hidden email]>:

> Igniters,
>
> Currently I'm profiling query execution on a single node on a realtively
> small data set (100K entries). What I see is that query parsing consumes
> about 20% of all execution time.
>
> Do we have something like JDBC PreparedStatement for our queries to
> minimize this overhead? If no, does it make sense to have it?
>
> Vladimir.
>

[dsetrakyan]

Sergi,

I really want those 20% of performance :)

Can we just cache the parsed query based on the query string that is passed
in? If not, can you think of another way to do it, without changing the
API? If not, can we change the API and still keep its
backward-compatibility, like we do with QueryEntity configuration?

D.

On Wed, Nov 4, 2015 at 7:00 AM, Sergi Vladykin <[hidden email]>
wrote:

> Yes, it is a problem and I believe after distributed joins feature will be
> merged, it will become even worse.
>
> Though I'd prefer to avoid changing public API for now. I believe it must
> be enough to cache
> and reuse parsed query information, generated queries and other stuff.
>
> Sergi
>
> 2015-11-04 16:43 GMT+03:00 Vladimir Ozerov <[hidden email]>:
>
> > Igniters,
> >
> > Currently I'm profiling query execution on a single node on a realtively
> > small data set (100K entries). What I see is that query parsing consumes
> > about 20% of all execution time.
> >
> > Do we have something like JDBC PreparedStatement for our queries to
> > minimize this overhead? If no, does it make sense to have it?
> >
> > Vladimir.
> >
>

[Vladimir Ozerov]

Agree, it would be cool if we cache parsed queries without any changes to
public API.

But Dima, note that I saw this "20%" on local benchmark with only 1 node. I
expect that in distributed environment it will be amortized by network.

On Wed, Nov 4, 2015 at 6:00 PM, Sergi Vladykin <[hidden email]>
wrote:

> Yes, it is a problem and I believe after distributed joins feature will be
> merged, it will become even worse.
>
> Though I'd prefer to avoid changing public API for now. I believe it must
> be enough to cache
> and reuse parsed query information, generated queries and other stuff.
>
> Sergi
>
> 2015-11-04 16:43 GMT+03:00 Vladimir Ozerov <[hidden email]>:
>
> > Igniters,
> >
> > Currently I'm profiling query execution on a single node on a realtively
> > small data set (100K entries). What I see is that query parsing consumes
> > about 20% of all execution time.
> >
> > Do we have something like JDBC PreparedStatement for our queries to
> > minimize this overhead? If no, does it make sense to have it?
> >
> > Vladimir.
> >
>

[Sergi]

Ok, I've created jira issue and will try to put this into 1.5. release.

https://issues.apache.org/jira/browse/IGNITE-1856

Sergi

2015-11-04 21:17 GMT+03:00 Vladimir Ozerov <[hidden email]>:

> Agree, it would be cool if we cache parsed queries without any changes to
> public API.
>
> But Dima, note that I saw this "20%" on local benchmark with only 1 node. I
> expect that in distributed environment it will be amortized by network.
>
> On Wed, Nov 4, 2015 at 6:00 PM, Sergi Vladykin <[hidden email]>
> wrote:
>
> > Yes, it is a problem and I believe after distributed joins feature will
> be
> > merged, it will become even worse.
> >
> > Though I'd prefer to avoid changing public API for now. I believe it must
> > be enough to cache
> > and reuse parsed query information, generated queries and other stuff.
> >
> > Sergi
> >
> > 2015-11-04 16:43 GMT+03:00 Vladimir Ozerov <[hidden email]>:
> >
> > > Igniters,
> > >
> > > Currently I'm profiling query execution on a single node on a
> realtively
> > > small data set (100K entries). What I see is that query parsing
> consumes
> > > about 20% of all execution time.
> > >
> > > Do we have something like JDBC PreparedStatement for our queries to
> > > minimize this overhead? If no, does it make sense to have it?
> > >
> > > Vladimir.
> > >
> >
>