Apache Ignite Developers - Legacy Mail Archive

Criteria query to web console

Classic

List

Threaded

11 messages Options

Alexey Kuznetsov

Criteria query to web console

Hi, All.

Recently, Yakov opened issue: Criteria query to web console [1]

We can create criteria based filter that could be passed to ScanQuery over
BinaryObjecs.

I think it make sens to implement this filter as first-class citizen
of org.apache.ignite.cache.query package and also support it from Web
Console. But it could be re-used directly from code also.

I think that we should implement a set of predicates to support AND, OR and
NOT logical operations to group several predicates.

We should support following operations:
Numbers: ==, <, >, <=, >=, !=
Strings: equals, startsWith, endsWith, contains, matchRegExp (and also with
IgnoreCase mode).
Dates: ==, <, >, <=, >=, !=, between
All: isNull, isDefined*

*isDefined - Here I mean a check that some property is present in
BinaryObject

What do you think?
If I missed some operations, please advice what could be added.

Also, I think that a separate issue should be created for such "rules
engine for BinaryObjects".
Make sense?

[1] https://issues.apache.org/jira/browse/IGNITE-6132

--
Alexey Kuznetsov

Vladimir Ozerov

Re: Criteria query to web console

My strong recommendation here is to think carefully of all potential use
cases first, so that we end up with consistent and extendable API. This
filters could be useful for both ScanQuery, platforms and 3-rd party
clients. Hence, "query" package is definitely not the right place. I would
said that this is our "expression language" or so, and put these filters to
"org.apache.ignite.el" package.

On Tue, Aug 22, 2017 at 6:30 PM, Alexey Kuznetsov <[hidden email]>
wrote:

> Hi, All.
>
> Recently, Yakov opened issue: Criteria query to web console [1]
>
> We can create criteria based filter that could be passed to ScanQuery over
> BinaryObjecs.
>
> I think it make sens to implement this filter as first-class citizen
> of org.apache.ignite.cache.query package and also support it from Web
> Console. But it could be re-used directly from code also.
>
> I think that we should implement a set of predicates to support AND, OR and
> NOT logical operations to group several predicates.
>
> We should support following operations:
> Numbers: ==, <, >, <=, >=, !=
> Strings: equals, startsWith, endsWith, contains, matchRegExp (and also with
> IgnoreCase mode).
> Dates: ==, <, >, <=, >=, !=, between
> All: isNull, isDefined*
>
> *isDefined - Here I mean a check that some property is present in
> BinaryObject
>
> What do you think?
> If I missed some operations, please advice what could be added.
>
> Also, I think that a separate issue should be created for such "rules
> engine for BinaryObjects".
> Make sense?
>
>
> [1] https://issues.apache.org/jira/browse/IGNITE-6132
>
>
> --
> Alexey Kuznetsov
>

dsetrakyan

Re: Criteria query to web console

I am a bit confused. How about "select * from SomeTable where myCriteria >
1"?

On Tue, Aug 22, 2017 at 9:16 AM, Vladimir Ozerov <[hidden email]>
wrote:

> My strong recommendation here is to think carefully of all potential use
> cases first, so that we end up with consistent and extendable API. This
> filters could be useful for both ScanQuery, platforms and 3-rd party
> clients. Hence, "query" package is definitely not the right place. I would
> said that this is our "expression language" or so, and put these filters to
> "org.apache.ignite.el" package.
>
> On Tue, Aug 22, 2017 at 6:30 PM, Alexey Kuznetsov <[hidden email]>
> wrote:
>
> > Hi, All.
> >
> > Recently, Yakov opened issue: Criteria query to web console [1]
> >
> > We can create criteria based filter that could be passed to ScanQuery
> over
> > BinaryObjecs.
> >
> > I think it make sens to implement this filter as first-class citizen
> > of org.apache.ignite.cache.query package and also support it from Web
> > Console. But it could be re-used directly from code also.
> >
> > I think that we should implement a set of predicates to support AND, OR
> and
> > NOT logical operations to group several predicates.
> >
> > We should support following operations:
> > Numbers: ==, <, >, <=, >=, !=
> > Strings: equals, startsWith, endsWith, contains, matchRegExp (and also
> with
> > IgnoreCase mode).
> > Dates: ==, <, >, <=, >=, !=, between
> > All: isNull, isDefined*
> >
> > *isDefined - Here I mean a check that some property is present in
> > BinaryObject
> >
> > What do you think?
> > If I missed some operations, please advice what could be added.
> >
> > Also, I think that a separate issue should be created for such "rules
> > engine for BinaryObjects".
> > Make sense?
> >
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-6132
> >
> >
> > --
> > Alexey Kuznetsov
> >
>

Alexey Kuznetsov-2

Re: Criteria query to web console

Vladimir,

Thanks for your feedback, I think it make sense.

Dmitriy,

No, critera based filter wull work only for ScanQuery, see ScanQuery
constructor with filter:
ScanQuery(@Nullable IgniteBiPredicate<K, V> filter)

This should be implemented with buider, smth like this:

CriteriaBuilder builder = new CriteriaBuilder();

IgniteBiPredicate filter =
builder.add(TCriteria(...)).and(TCriteria(...)).or(TCriteria(...)).andNot(TCriteria(...)).build();

ScanQuery q = new ScanQuery(filter);

cache.query(q)....

And we could have nice Web UI for this: IgniteBiPredicate filter =
builder.add(TCriteria(...)).and(TCriteria(...)).or(TCriteria(...)).andNot(TCriteria(...)).build();

On Wed, Aug 23, 2017 at 8:46 AM, Dmitriy Setrakyan <[hidden email]>
wrote:

> I am a bit confused. How about "select * from SomeTable where myCriteria >
> 1"?
>
> On Tue, Aug 22, 2017 at 9:16 AM, Vladimir Ozerov <[hidden email]>
> wrote:
>
> > My strong recommendation here is to think carefully of all potential use
> > cases first, so that we end up with consistent and extendable API. This
> > filters could be useful for both ScanQuery, platforms and 3-rd party
> > clients. Hence, "query" package is definitely not the right place. I
> would
> > said that this is our "expression language" or so, and put these filters
> to
> > "org.apache.ignite.el" package.
> >
> > On Tue, Aug 22, 2017 at 6:30 PM, Alexey Kuznetsov <[hidden email]
> >
> > wrote:
> >
> > > Hi, All.
> > >
> > > Recently, Yakov opened issue: Criteria query to web console [1]
> > >
> > > We can create criteria based filter that could be passed to ScanQuery
> > over
> > > BinaryObjecs.
> > >
> > > I think it make sens to implement this filter as first-class citizen
> > > of org.apache.ignite.cache.query package and also support it from Web
> > > Console. But it could be re-used directly from code also.
> > >
> > > I think that we should implement a set of predicates to support AND, OR
> > and
> > > NOT logical operations to group several predicates.
> > >
> > > We should support following operations:
> > > Numbers: ==, <, >, <=, >=, !=
> > > Strings: equals, startsWith, endsWith, contains, matchRegExp (and also
> > with
> > > IgnoreCase mode).
> > > Dates: ==, <, >, <=, >=, !=, between
> > > All: isNull, isDefined*
> > >
> > > *isDefined - Here I mean a check that some property is present in
> > > BinaryObject
> > >
> > > What do you think?
> > > If I missed some operations, please advice what could be added.
> > >
> > > Also, I think that a separate issue should be created for such "rules
> > > engine for BinaryObjects".
> > > Make sense?
> > >
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-6132
> > >
> > >
> > > --
> > > Alexey Kuznetsov
> > >
> >
>

--
Alexey Kuznetsov
GridGain Systems
www.gridgain.com

dsetrakyan

Re: Criteria query to web console

I still don't get it. If I need a filter, then I will just use SQL.

On Tue, Aug 22, 2017 at 8:46 PM, Alexey Kuznetsov <[hidden email]>
wrote:

> Vladimir,
>
> Thanks for your feedback, I think it make sense.
>
> Dmitriy,
>
> No, critera based filter wull work only for ScanQuery, see ScanQuery
> constructor with filter:
> ScanQuery(@Nullable IgniteBiPredicate<K, V> filter)
>
> This should be implemented with buider, smth like this:
>
> CriteriaBuilder builder = new CriteriaBuilder();
>
> IgniteBiPredicate filter =
> builder.add(TCriteria(...)).and(TCriteria(...)).or(TCriteria(...)).andNot(
> TCriteria(...)).build();
>
> ScanQuery q = new ScanQuery(filter);
>
> cache.query(q)....
>
> And we could have nice Web UI for this: IgniteBiPredicate filter =
> builder.add(TCriteria(...)).and(TCriteria(...)).or(TCriteria(...)).andNot(
> TCriteria(...)).build();
>
>
> On Wed, Aug 23, 2017 at 8:46 AM, Dmitriy Setrakyan <[hidden email]>
> wrote:
>
> > I am a bit confused. How about "select * from SomeTable where myCriteria
> >
> > 1"?
> >
> > On Tue, Aug 22, 2017 at 9:16 AM, Vladimir Ozerov <[hidden email]>
> > wrote:
> >
> > > My strong recommendation here is to think carefully of all potential
> use
> > > cases first, so that we end up with consistent and extendable API. This
> > > filters could be useful for both ScanQuery, platforms and 3-rd party
> > > clients. Hence, "query" package is definitely not the right place. I
> > would
> > > said that this is our "expression language" or so, and put these
> filters
> > to
> > > "org.apache.ignite.el" package.
> > >
> > > On Tue, Aug 22, 2017 at 6:30 PM, Alexey Kuznetsov <
> [hidden email]
> > >
> > > wrote:
> > >
> > > > Hi, All.
> > > >
> > > > Recently, Yakov opened issue: Criteria query to web console [1]
> > > >
> > > > We can create criteria based filter that could be passed to ScanQuery
> > > over
> > > > BinaryObjecs.
> > > >
> > > > I think it make sens to implement this filter as first-class citizen
> > > > of org.apache.ignite.cache.query package and also support it from Web
> > > > Console. But it could be re-used directly from code also.
> > > >
> > > > I think that we should implement a set of predicates to support AND,
> OR
> > > and
> > > > NOT logical operations to group several predicates.
> > > >
> > > > We should support following operations:
> > > > Numbers: ==, <, >, <=, >=, !=
> > > > Strings: equals, startsWith, endsWith, contains, matchRegExp (and
> also
> > > with
> > > > IgnoreCase mode).
> > > > Dates: ==, <, >, <=, >=, !=, between
> > > > All: isNull, isDefined*
> > > >
> > > > *isDefined - Here I mean a check that some property is present in
> > > > BinaryObject
> > > >
> > > > What do you think?
> > > > If I missed some operations, please advice what could be added.
> > > >
> > > > Also, I think that a separate issue should be created for such "rules
> > > > engine for BinaryObjects".
> > > > Make sense?
> > > >
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IGNITE-6132
> > > >
> > > >
> > > > --
> > > > Alexey Kuznetsov
> > > >
> > >
> >
>
>
>
> --
> Alexey Kuznetsov
> GridGain Systems
> www.gridgain.com
>

Vladimir Ozerov

Re: Criteria query to web console

Dima,

Personally I do not know why Yakov and Alex needed it on a WebConsole. But
as a whole ability to construct arbitrary queries through API is a
killer-feature. First, it is much more convenient for users - you do not
need to hard-code strings, you have full API support - refactoring, code
completion, etc., you will never meet unsupported features (like on our
SQL), etc. The most widely known example is LINQ in .NET - nobody wants to
use our SQL, they use LINQ because it is super-convenient. Hazelcast goes
in the same direction and this is right decision.

Another major point is performance. With current state of affairs we
already can make simple EL-queries (similar to "SELECT a FROM b WHERE c=?")
times more efficient than our SQL, because we will bypass a lot heavy
places such as parsing, dumb H2 optimizer, inefficient H2 API producing
garbage, etc..

Vladimir.

On Wed, Aug 23, 2017 at 8:25 AM, Dmitriy Setrakyan <[hidden email]>
wrote:

> I still don't get it. If I need a filter, then I will just use SQL.
>
> On Tue, Aug 22, 2017 at 8:46 PM, Alexey Kuznetsov <[hidden email]
> >
> wrote:
>
> > Vladimir,
> >
> > Thanks for your feedback, I think it make sense.
> >
> > Dmitriy,
> >
> > No, critera based filter wull work only for ScanQuery, see ScanQuery
> > constructor with filter:
> > ScanQuery(@Nullable IgniteBiPredicate<K, V> filter)
> >
> > This should be implemented with buider, smth like this:
> >
> > CriteriaBuilder builder = new CriteriaBuilder();
> >
> > IgniteBiPredicate filter =
> > builder.add(TCriteria(...)).and(TCriteria(...)).or(
> TCriteria(...)).andNot(
> > TCriteria(...)).build();
> >
> > ScanQuery q = new ScanQuery(filter);
> >
> > cache.query(q)....
> >
> > And we could have nice Web UI for this: IgniteBiPredicate filter =
> > builder.add(TCriteria(...)).and(TCriteria(...)).or(
> TCriteria(...)).andNot(
> > TCriteria(...)).build();
> >
> >
> > On Wed, Aug 23, 2017 at 8:46 AM, Dmitriy Setrakyan <
> [hidden email]>
> > wrote:
> >
> > > I am a bit confused. How about "select * from SomeTable where
> myCriteria
> > >
> > > 1"?
> > >
> > > On Tue, Aug 22, 2017 at 9:16 AM, Vladimir Ozerov <[hidden email]
> >
> > > wrote:
> > >
> > > > My strong recommendation here is to think carefully of all potential
> > use
> > > > cases first, so that we end up with consistent and extendable API.
> This
> > > > filters could be useful for both ScanQuery, platforms and 3-rd party
> > > > clients. Hence, "query" package is definitely not the right place. I
> > > would
> > > > said that this is our "expression language" or so, and put these
> > filters
> > > to
> > > > "org.apache.ignite.el" package.
> > > >
> > > > On Tue, Aug 22, 2017 at 6:30 PM, Alexey Kuznetsov <
> > [hidden email]
> > > >
> > > > wrote:
> > > >
> > > > > Hi, All.
> > > > >
> > > > > Recently, Yakov opened issue: Criteria query to web console [1]
> > > > >
> > > > > We can create criteria based filter that could be passed to
> ScanQuery
> > > > over
> > > > > BinaryObjecs.
> > > > >
> > > > > I think it make sens to implement this filter as first-class
> citizen
> > > > > of org.apache.ignite.cache.query package and also support it from
> Web
> > > > > Console. But it could be re-used directly from code also.
> > > > >
> > > > > I think that we should implement a set of predicates to support
> AND,
> > OR
> > > > and
> > > > > NOT logical operations to group several predicates.
> > > > >
> > > > > We should support following operations:
> > > > > Numbers: ==, <, >, <=, >=, !=
> > > > > Strings: equals, startsWith, endsWith, contains, matchRegExp (and
> > also
> > > > with
> > > > > IgnoreCase mode).
> > > > > Dates: ==, <, >, <=, >=, !=, between
> > > > > All: isNull, isDefined*
> > > > >
> > > > > *isDefined - Here I mean a check that some property is present in
> > > > > BinaryObject
> > > > >
> > > > > What do you think?
> > > > > If I missed some operations, please advice what could be added.
> > > > >
> > > > > Also, I think that a separate issue should be created for such
> "rules
> > > > > engine for BinaryObjects".
> > > > > Make sense?
> > > > >
> > > > >
> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-6132
> > > > >
> > > > >
> > > > > --
> > > > > Alexey Kuznetsov
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Alexey Kuznetsov
> > GridGain Systems
> > www.gridgain.com
> >
>

dsetrakyan

Re: Criteria query to web console

Vova, I think you are suggesting an API-based query language. This seems
like a huge undertaking for a very questionable benefit, given that SQL is
way more flexible.

I would rather fix the H2 GC and optimizer problems.

D.

On Wed, Aug 23, 2017 at 12:01 AM, Vladimir Ozerov <[hidden email]>
wrote:

> Dima,
>
> Personally I do not know why Yakov and Alex needed it on a WebConsole. But
> as a whole ability to construct arbitrary queries through API is a
> killer-feature. First, it is much more convenient for users - you do not
> need to hard-code strings, you have full API support - refactoring, code
> completion, etc., you will never meet unsupported features (like on our
> SQL), etc. The most widely known example is LINQ in .NET - nobody wants to
> use our SQL, they use LINQ because it is super-convenient. Hazelcast goes
> in the same direction and this is right decision.
>
> Another major point is performance. With current state of affairs we
> already can make simple EL-queries (similar to "SELECT a FROM b WHERE c=?")
> times more efficient than our SQL, because we will bypass a lot heavy
> places such as parsing, dumb H2 optimizer, inefficient H2 API producing
> garbage, etc..
>
> Vladimir.
>
> On Wed, Aug 23, 2017 at 8:25 AM, Dmitriy Setrakyan <[hidden email]>
> wrote:
>
> > I still don't get it. If I need a filter, then I will just use SQL.
> >
> > On Tue, Aug 22, 2017 at 8:46 PM, Alexey Kuznetsov <
> [hidden email]
> > >
> > wrote:
> >
> > > Vladimir,
> > >
> > > Thanks for your feedback, I think it make sense.
> > >
> > > Dmitriy,
> > >
> > > No, critera based filter wull work only for ScanQuery, see ScanQuery
> > > constructor with filter:
> > > ScanQuery(@Nullable IgniteBiPredicate<K, V> filter)
> > >
> > > This should be implemented with buider, smth like this:
> > >
> > > CriteriaBuilder builder = new CriteriaBuilder();
> > >
> > > IgniteBiPredicate filter =
> > > builder.add(TCriteria(...)).and(TCriteria(...)).or(
> > TCriteria(...)).andNot(
> > > TCriteria(...)).build();
> > >
> > > ScanQuery q = new ScanQuery(filter);
> > >
> > > cache.query(q)....
> > >
> > > And we could have nice Web UI for this: IgniteBiPredicate filter =
> > > builder.add(TCriteria(...)).and(TCriteria(...)).or(
> > TCriteria(...)).andNot(
> > > TCriteria(...)).build();
> > >
> > >
> > > On Wed, Aug 23, 2017 at 8:46 AM, Dmitriy Setrakyan <
> > [hidden email]>
> > > wrote:
> > >
> > > > I am a bit confused. How about "select * from SomeTable where
> > myCriteria
> > > >
> > > > 1"?
> > > >
> > > > On Tue, Aug 22, 2017 at 9:16 AM, Vladimir Ozerov <
> [hidden email]
> > >
> > > > wrote:
> > > >
> > > > > My strong recommendation here is to think carefully of all
> potential
> > > use
> > > > > cases first, so that we end up with consistent and extendable API.
> > This
> > > > > filters could be useful for both ScanQuery, platforms and 3-rd
> party
> > > > > clients. Hence, "query" package is definitely not the right place.
> I
> > > > would
> > > > > said that this is our "expression language" or so, and put these
> > > filters
> > > > to
> > > > > "org.apache.ignite.el" package.
> > > > >
> > > > > On Tue, Aug 22, 2017 at 6:30 PM, Alexey Kuznetsov <
> > > [hidden email]
> > > > >
> > > > > wrote:
> > > > >
> > > > > > Hi, All.
> > > > > >
> > > > > > Recently, Yakov opened issue: Criteria query to web console [1]
> > > > > >
> > > > > > We can create criteria based filter that could be passed to
> > ScanQuery
> > > > > over
> > > > > > BinaryObjecs.
> > > > > >
> > > > > > I think it make sens to implement this filter as first-class
> > citizen
> > > > > > of org.apache.ignite.cache.query package and also support it from
> > Web
> > > > > > Console. But it could be re-used directly from code also.
> > > > > >
> > > > > > I think that we should implement a set of predicates to support
> > AND,
> > > OR
> > > > > and
> > > > > > NOT logical operations to group several predicates.
> > > > > >
> > > > > > We should support following operations:
> > > > > > Numbers: ==, <, >, <=, >=, !=
> > > > > > Strings: equals, startsWith, endsWith, contains, matchRegExp (and
> > > also
> > > > > with
> > > > > > IgnoreCase mode).
> > > > > > Dates: ==, <, >, <=, >=, !=, between
> > > > > > All: isNull, isDefined*
> > > > > >
> > > > > > *isDefined - Here I mean a check that some property is present in
> > > > > > BinaryObject
> > > > > >
> > > > > > What do you think?
> > > > > > If I missed some operations, please advice what could be added.
> > > > > >
> > > > > > Also, I think that a separate issue should be created for such
> > "rules
> > > > > > engine for BinaryObjects".
> > > > > > Make sense?
> > > > > >
> > > > > >
> > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-6132
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Alexey Kuznetsov
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Alexey Kuznetsov
> > > GridGain Systems
> > > www.gridgain.com
> > >
> >
>

Alexey Kuznetsov

Re: Criteria query to web console

In reply to this post by dsetrakyan

Dima,

It is quite common situation that you have cache, but it is not configured
to run SQL on it.
In this case passing a filter to SCAN query could be an option.

Make sense now?

Vova,

It is cool to write a kind of LINQ for Ignite, but lets start from filter
for SCAN query.
What do you think?

On Wed, Aug 23, 2017 at 12:25 PM, Dmitriy Setrakyan <[hidden email]>
wrote:

--
Alexey Kuznetsov

yzhdanov

Re: Criteria query to web console

Guys, currently it is impossible to do scan query and filter by 2 or more
fields. All I want is to have pretty simple logic on WebConsole that will
create a predicate to filter the data based on user provided field names,
values and operations. This is what we can have really fast and will be
very handy if cluster does not have SQL configured.

Vladimir suggests generic approach which makes sense to me, but will
require much more effort and time.

--Yakov

Alexey Kuznetsov

Re: Criteria query to web console

Yakov,

I got the idea. Let's create something simple and try to use it.
And improve later as per Vladimir's ideas.

On Wed, Aug 23, 2017 at 4:58 PM, Yakov Zhdanov <[hidden email]> wrote:

> Guys, currently it is impossible to do scan query and filter by 2 or more
> fields. All I want is to have pretty simple logic on WebConsole that will
> create a predicate to filter the data based on user provided field names,
> values and operations. This is what we can have really fast and will be
> very handy if cluster does not have SQL configured.
>
> Vladimir suggests generic approach which makes sense to me, but will
> require much more effort and time.
>
> --Yakov
>

--
Alexey Kuznetsov

Vladimir Ozerov

Re: Criteria query to web console

Let's make sure that nothing is exposed to public API then, so that we can
change it in future without any problems with compatibility.

On Wed, Aug 23, 2017 at 1:55 PM, Alexey Kuznetsov <[hidden email]>
wrote:

> Yakov,
>
> I got the idea. Let's create something simple and try to use it.
> And improve later as per Vladimir's ideas.
>
> On Wed, Aug 23, 2017 at 4:58 PM, Yakov Zhdanov <[hidden email]>
> wrote:
>
> > Guys, currently it is impossible to do scan query and filter by 2 or more
> > fields. All I want is to have pretty simple logic on WebConsole that will
> > create a predicate to filter the data based on user provided field names,
> > values and operations. This is what we can have really fast and will be
> > very handy if cluster does not have SQL configured.
> >
> > Vladimir suggests generic approach which makes sense to me, but will
> > require much more effort and time.
> >
> > --Yakov
> >
>
>
>
> --
> Alexey Kuznetsov
>