Apache Ignite Developers - Legacy Mail Archive

IEP-51: Java Thin Client Async API

Classic

List

Threaded

20 messages Options

Pavel Tupitsyn

IEP-51: Java Thin Client Async API

Igniters,

I've prepared an IEP [1], please review and let me know what you think.

In particular, I'd like to discuss the Future interface to be used:
* IgniteFuture is the first candidate - Thin APIs will be consistent with
Thick APIs, probably better for existing Ignite users.
* CompletableFuture is the standard for async Java APIs. Many users may
prefer that instead of a custom IgniteFuture.

[1]
https://cwiki.apache.org/confluence/display/IGNITE/IEP-51%3A+Java+Thin+Client+Async+API

Valentin Kulichenko

Re: IEP-51: Java Thin Client Async API

Hi Pavel,

Are there any benefits of IgniteFuture over CompletableFuture?

IgniteFuture was created long ago, during the time when CompletableFuture
did not exist. There is a big chance that IgniteFuture actually became
redundant at the moment we transitioned to Java8. If that's the case, I
would prefer using CompletableFuture in the thin client and getting rid of
IgniteFuture altogether in 3.0.

What do you think?

-Val

On Thu, Aug 20, 2020 at 7:19 AM Pavel Tupitsyn <[hidden email]> wrote:

> Igniters,
>
> I've prepared an IEP [1], please review and let me know what you think.
>
> In particular, I'd like to discuss the Future interface to be used:
> * IgniteFuture is the first candidate - Thin APIs will be consistent with
> Thick APIs, probably better for existing Ignite users.
> * CompletableFuture is the standard for async Java APIs. Many users may
> prefer that instead of a custom IgniteFuture.
>
> [1]
>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-51%3A+Java+Thin+Client+Async+API
>

Pavel Tupitsyn

Re: IEP-51: Java Thin Client Async API

Val, no objections from my side.
As noted above, the only benefit of IgniteFuture is consistency across
thin/thick APIs,
which is probably not so important.

On Thu, Aug 20, 2020 at 6:28 PM Valentin Kulichenko <
[hidden email]> wrote:

> Hi Pavel,
>
> Are there any benefits of IgniteFuture over CompletableFuture?
>
> IgniteFuture was created long ago, during the time when CompletableFuture
> did not exist. There is a big chance that IgniteFuture actually became
> redundant at the moment we transitioned to Java8. If that's the case, I
> would prefer using CompletableFuture in the thin client and getting rid of
> IgniteFuture altogether in 3.0.
>
> What do you think?
>
> -Val
>
> On Thu, Aug 20, 2020 at 7:19 AM Pavel Tupitsyn <[hidden email]>
> wrote:
>
> > Igniters,
> >
> > I've prepared an IEP [1], please review and let me know what you think.
> >
> > In particular, I'd like to discuss the Future interface to be used:
> > * IgniteFuture is the first candidate - Thin APIs will be consistent with
> > Thick APIs, probably better for existing Ignite users.
> > * CompletableFuture is the standard for async Java APIs. Many users may
> > prefer that instead of a custom IgniteFuture.
> >
> > [1]
> >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-51%3A+Java+Thin+Client+Async+API
> >
>

Valentin Kulichenko

Re: IEP-51: Java Thin Client Async API

Sounds good. I've added this to the 3.0 roadmap:
https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0

Unless there are any objections from others, let's stick with the
CompletableFuture for any future development, including the thin client.

-Val

On Thu, Aug 20, 2020 at 9:30 AM Pavel Tupitsyn <[hidden email]> wrote:

> Val, no objections from my side.
> As noted above, the only benefit of IgniteFuture is consistency across
> thin/thick APIs,
> which is probably not so important.
>
> On Thu, Aug 20, 2020 at 6:28 PM Valentin Kulichenko <
> [hidden email]> wrote:
>
> > Hi Pavel,
> >
> > Are there any benefits of IgniteFuture over CompletableFuture?
> >
> > IgniteFuture was created long ago, during the time when CompletableFuture
> > did not exist. There is a big chance that IgniteFuture actually became
> > redundant at the moment we transitioned to Java8. If that's the case, I
> > would prefer using CompletableFuture in the thin client and getting rid
> of
> > IgniteFuture altogether in 3.0.
> >
> > What do you think?
> >
> > -Val
> >
> > On Thu, Aug 20, 2020 at 7:19 AM Pavel Tupitsyn <[hidden email]>
> > wrote:
> >
> > > Igniters,
> > >
> > > I've prepared an IEP [1], please review and let me know what you think.
> > >
> > > In particular, I'd like to discuss the Future interface to be used:
> > > * IgniteFuture is the first candidate - Thin APIs will be consistent
> with
> > > Thick APIs, probably better for existing Ignite users.
> > > * CompletableFuture is the standard for async Java APIs. Many users may
> > > prefer that instead of a custom IgniteFuture.
> > >
> > > [1]
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-51%3A+Java+Thin+Client+Async+API
> > >
> >
>

Pavel Tupitsyn

Re: IEP-51: Java Thin Client Async API

Ok, I've reflected this in the IEP

On Fri, Aug 21, 2020 at 3:35 AM Valentin Kulichenko <
[hidden email]> wrote:

> Sounds good. I've added this to the 3.0 roadmap:
> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0
>
> Unless there are any objections from others, let's stick with the
> CompletableFuture for any future development, including the thin client.
>
> -Val
>
> On Thu, Aug 20, 2020 at 9:30 AM Pavel Tupitsyn <[hidden email]>
> wrote:
>
> > Val, no objections from my side.
> > As noted above, the only benefit of IgniteFuture is consistency across
> > thin/thick APIs,
> > which is probably not so important.
> >
> > On Thu, Aug 20, 2020 at 6:28 PM Valentin Kulichenko <
> > [hidden email]> wrote:
> >
> > > Hi Pavel,
> > >
> > > Are there any benefits of IgniteFuture over CompletableFuture?
> > >
> > > IgniteFuture was created long ago, during the time when
> CompletableFuture
> > > did not exist. There is a big chance that IgniteFuture actually became
> > > redundant at the moment we transitioned to Java8. If that's the case, I
> > > would prefer using CompletableFuture in the thin client and getting rid
> > of
> > > IgniteFuture altogether in 3.0.
> > >
> > > What do you think?
> > >
> > > -Val
> > >
> > > On Thu, Aug 20, 2020 at 7:19 AM Pavel Tupitsyn <[hidden email]>
> > > wrote:
> > >
> > > > Igniters,
> > > >
> > > > I've prepared an IEP [1], please review and let me know what you
> think.
> > > >
> > > > In particular, I'd like to discuss the Future interface to be used:
> > > > * IgniteFuture is the first candidate - Thin APIs will be consistent
> > with
> > > > Thick APIs, probably better for existing Ignite users.
> > > > * CompletableFuture is the standard for async Java APIs. Many users
> may
> > > > prefer that instead of a custom IgniteFuture.
> > > >
> > > > [1]
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-51%3A+Java+Thin+Client+Async+API
> > > >
> > >
> >
>

Alexey Plekhanov

Re: IEP-51: Java Thin Client Async API

In reply to this post by Valentin Kulichenko

Pavel,

Thanks for the discussion, I've also faced with the necessity of having
async calls while implementing POC for thin client data streamer [1] and
solve it similarly (but in my case it's required only for internal
implementation, so I've only changed the internal API).

I want to note that described in IEP approach (and implemented in POC) is
not fully async, since "send" is still used in the user's thread. To make
it fully async we need additional sending thread (since blocking IO is used
for communication with the server). If partition awareness is enabled there
will be 2 threads per each server connection, perhaps we should think about
moving to NIO and introducing some kind of communication thread pool.

[1]: https://github.com/apache/ignite/pull/8175

пт, 21 авг. 2020 г. в 03:35, Valentin Kulichenko <
[hidden email]>:

Pavel Tupitsyn

Re: IEP-51: Java Thin Client Async API

Val,

The problems with CompletableFuture in public API are:
* It is a class, not an interface
* It is completable - anyone can call .complete(), which is not what we want

There seems to be no clear guidance in Java world on async API design;
however, it is often recommended to return CompletionStage instead of
CompletableFuture
from the public APIs [1] [2], and some products follow this [3].

Other products return their own future interface that extends both Future
and CompletionStage,
which seems to be a better alternative to me [4].

Thoughts?

[1]
https://stackoverflow.com/questions/47571117/what-is-the-difference-between-completionstage-and-completablefuture
[2]
https://stackoverflow.com/questions/34930840/should-i-return-completablefuture-or-future-when-defining-api
<https://stackoverflow.com/questions/34930840/should-i-return-completablefuture-or-future-when-defining-api#:~:text=by%20returning%20a%20CompletableFuture%2C%20you,API%2C%20which%20is%20not%20good.>
[3]
https://docs.hazelcast.org/docs/latest/javadoc/com/hazelcast/cache/ICache.html
[4]
https://lettuce.io/lettuce-4/release/api/com/lambdaworks/redis/RedisFuture.html

On Fri, Aug 21, 2020 at 10:28 AM Alex Plehanov <[hidden email]>
wrote:

> Pavel,
>
> Thanks for the discussion, I've also faced with the necessity of having
> async calls while implementing POC for thin client data streamer [1] and
> solve it similarly (but in my case it's required only for internal
> implementation, so I've only changed the internal API).
>
> I want to note that described in IEP approach (and implemented in POC) is
> not fully async, since "send" is still used in the user's thread. To make
> it fully async we need additional sending thread (since blocking IO is used
> for communication with the server). If partition awareness is enabled there
> will be 2 threads per each server connection, perhaps we should think about
> moving to NIO and introducing some kind of communication thread pool.
>
> [1]: https://github.com/apache/ignite/pull/8175
>
> пт, 21 авг. 2020 г. в 03:35, Valentin Kulichenko <
> [hidden email]>:
>
> > Sounds good. I've added this to the 3.0 roadmap:
> > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0
> >
> > Unless there are any objections from others, let's stick with the
> > CompletableFuture for any future development, including the thin client.
> >
> > -Val
> >
> > On Thu, Aug 20, 2020 at 9:30 AM Pavel Tupitsyn <[hidden email]>
> > wrote:
> >
> > > Val, no objections from my side.
> > > As noted above, the only benefit of IgniteFuture is consistency across
> > > thin/thick APIs,
> > > which is probably not so important.
> > >
> > > On Thu, Aug 20, 2020 at 6:28 PM Valentin Kulichenko <
> > > [hidden email]> wrote:
> > >
> > > > Hi Pavel,
> > > >
> > > > Are there any benefits of IgniteFuture over CompletableFuture?
> > > >
> > > > IgniteFuture was created long ago, during the time when
> > CompletableFuture
> > > > did not exist. There is a big chance that IgniteFuture actually
> became
> > > > redundant at the moment we transitioned to Java8. If that's the
> case, I
> > > > would prefer using CompletableFuture in the thin client and getting
> rid
> > > of
> > > > IgniteFuture altogether in 3.0.
> > > >
> > > > What do you think?
> > > >
> > > > -Val
> > > >
> > > > On Thu, Aug 20, 2020 at 7:19 AM Pavel Tupitsyn <[hidden email]
> >
> > > > wrote:
> > > >
> > > > > Igniters,
> > > > >
> > > > > I've prepared an IEP [1], please review and let me know what you
> > think.
> > > > >
> > > > > In particular, I'd like to discuss the Future interface to be used:
> > > > > * IgniteFuture is the first candidate - Thin APIs will be
> consistent
> > > with
> > > > > Thick APIs, probably better for existing Ignite users.
> > > > > * CompletableFuture is the standard for async Java APIs. Many users
> > may
> > > > > prefer that instead of a custom IgniteFuture.
> > > > >
> > > > > [1]
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-51%3A+Java+Thin+Client+Async+API
> > > > >
> > > >
> > >
> >
>

Pavel Tupitsyn

Re: IEP-51: Java Thin Client Async API

Alex,

> not fully async, since "send" is still used in the user's thread
Correct. I've added those details to the IEP.
I'll investigate the ways to make this fully non-blocking,
but for now let's consider this to be out of scope of this IEP.

> POC for thin client data streamer
Very interesting, is there a ticket or IEP to follow?

On Fri, Aug 21, 2020 at 11:01 AM Pavel Tupitsyn <[hidden email]>
wrote:

> Val,
>
> The problems with CompletableFuture in public API are:
> * It is a class, not an interface
> * It is completable - anyone can call .complete(), which is not what we
> want
>
> There seems to be no clear guidance in Java world on async API design;
> however, it is often recommended to return CompletionStage instead of
> CompletableFuture
> from the public APIs [1] [2], and some products follow this [3].
>
> Other products return their own future interface that extends both Future
> and CompletionStage,
> which seems to be a better alternative to me [4].
>
> Thoughts?
>
> [1]
> https://stackoverflow.com/questions/47571117/what-is-the-difference-between-completionstage-and-completablefuture
> [2]
> https://stackoverflow.com/questions/34930840/should-i-return-completablefuture-or-future-when-defining-api
> <https://stackoverflow.com/questions/34930840/should-i-return-completablefuture-or-future-when-defining-api#:~:text=by%20returning%20a%20CompletableFuture%2C%20you,API%2C%20which%20is%20not%20good.>
> [3]
> https://docs.hazelcast.org/docs/latest/javadoc/com/hazelcast/cache/ICache.html
> [4]
> https://lettuce.io/lettuce-4/release/api/com/lambdaworks/redis/RedisFuture.html
>
> On Fri, Aug 21, 2020 at 10:28 AM Alex Plehanov <[hidden email]>
> wrote:
>
>> Pavel,
>>
>> Thanks for the discussion, I've also faced with the necessity of having
>> async calls while implementing POC for thin client data streamer [1] and
>> solve it similarly (but in my case it's required only for internal
>> implementation, so I've only changed the internal API).
>>
>> I want to note that described in IEP approach (and implemented in POC) is
>> not fully async, since "send" is still used in the user's thread. To make
>> it fully async we need additional sending thread (since blocking IO is
>> used
>> for communication with the server). If partition awareness is enabled
>> there
>> will be 2 threads per each server connection, perhaps we should think
>> about
>> moving to NIO and introducing some kind of communication thread pool.
>>
>> [1]: https://github.com/apache/ignite/pull/8175
>>
>> пт, 21 авг. 2020 г. в 03:35, Valentin Kulichenko <
>> [hidden email]>:
>>
>> > Sounds good. I've added this to the 3.0 roadmap:
>> > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0
>> >
>> > Unless there are any objections from others, let's stick with the
>> > CompletableFuture for any future development, including the thin client.
>> >
>> > -Val
>> >
>> > On Thu, Aug 20, 2020 at 9:30 AM Pavel Tupitsyn <[hidden email]>
>> > wrote:
>> >
>> > > Val, no objections from my side.
>> > > As noted above, the only benefit of IgniteFuture is consistency across
>> > > thin/thick APIs,
>> > > which is probably not so important.
>> > >
>> > > On Thu, Aug 20, 2020 at 6:28 PM Valentin Kulichenko <
>> > > [hidden email]> wrote:
>> > >
>> > > > Hi Pavel,
>> > > >
>> > > > Are there any benefits of IgniteFuture over CompletableFuture?
>> > > >
>> > > > IgniteFuture was created long ago, during the time when
>> > CompletableFuture
>> > > > did not exist. There is a big chance that IgniteFuture actually
>> became
>> > > > redundant at the moment we transitioned to Java8. If that's the
>> case, I
>> > > > would prefer using CompletableFuture in the thin client and getting
>> rid
>> > > of
>> > > > IgniteFuture altogether in 3.0.
>> > > >
>> > > > What do you think?
>> > > >
>> > > > -Val
>> > > >
>> > > > On Thu, Aug 20, 2020 at 7:19 AM Pavel Tupitsyn <
>> [hidden email]>
>> > > > wrote:
>> > > >
>> > > > > Igniters,
>> > > > >
>> > > > > I've prepared an IEP [1], please review and let me know what you
>> > think.
>> > > > >
>> > > > > In particular, I'd like to discuss the Future interface to be
>> used:
>> > > > > * IgniteFuture is the first candidate - Thin APIs will be
>> consistent
>> > > with
>> > > > > Thick APIs, probably better for existing Ignite users.
>> > > > > * CompletableFuture is the standard for async Java APIs. Many
>> users
>> > may
>> > > > > prefer that instead of a custom IgniteFuture.
>> > > > >
>> > > > > [1]
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-51%3A+Java+Thin+Client+Async+API
>> > > > >
>> > > >
>> > >
>> >
>>
>

Alexey Plekhanov

Re: IEP-51: Java Thin Client Async API

Pavel,

> Very interesting, is there a ticket or IEP to follow?
No, there is no ticket or IEP. I will create ticket, IEP and dev-list
discussion after benchmarking of POC implementation.

пт, 21 авг. 2020 г. в 12:17, Pavel Tupitsyn <[hidden email]>:

> Alex,
>
> > not fully async, since "send" is still used in the user's thread
> Correct. I've added those details to the IEP.
> I'll investigate the ways to make this fully non-blocking,
> but for now let's consider this to be out of scope of this IEP.
>
> > POC for thin client data streamer
> Very interesting, is there a ticket or IEP to follow?
>
> On Fri, Aug 21, 2020 at 11:01 AM Pavel Tupitsyn <[hidden email]>
> wrote:
>
> > Val,
> >
> > The problems with CompletableFuture in public API are:
> > * It is a class, not an interface
> > * It is completable - anyone can call .complete(), which is not what we
> > want
> >
> > There seems to be no clear guidance in Java world on async API design;
> > however, it is often recommended to return CompletionStage instead of
> > CompletableFuture
> > from the public APIs [1] [2], and some products follow this [3].
> >
> > Other products return their own future interface that extends both Future
> > and CompletionStage,
> > which seems to be a better alternative to me [4].
> >
> > Thoughts?
> >
> > [1]
> >
> https://stackoverflow.com/questions/47571117/what-is-the-difference-between-completionstage-and-completablefuture
> > [2]
> >
> https://stackoverflow.com/questions/34930840/should-i-return-completablefuture-or-future-when-defining-api
> > <
> https://stackoverflow.com/questions/34930840/should-i-return-completablefuture-or-future-when-defining-api#:~:text=by%20returning%20a%20CompletableFuture%2C%20you,API%2C%20which%20is%20not%20good
> .>
> > [3]
> >
> https://docs.hazelcast.org/docs/latest/javadoc/com/hazelcast/cache/ICache.html
> > [4]
> >
> https://lettuce.io/lettuce-4/release/api/com/lambdaworks/redis/RedisFuture.html
> >
> > On Fri, Aug 21, 2020 at 10:28 AM Alex Plehanov <[hidden email]>
> > wrote:
> >
> >> Pavel,
> >>
> >> Thanks for the discussion, I've also faced with the necessity of having
> >> async calls while implementing POC for thin client data streamer [1] and
> >> solve it similarly (but in my case it's required only for internal
> >> implementation, so I've only changed the internal API).
> >>
> >> I want to note that described in IEP approach (and implemented in POC)
> is
> >> not fully async, since "send" is still used in the user's thread. To
> make
> >> it fully async we need additional sending thread (since blocking IO is
> >> used
> >> for communication with the server). If partition awareness is enabled
> >> there
> >> will be 2 threads per each server connection, perhaps we should think
> >> about
> >> moving to NIO and introducing some kind of communication thread pool.
> >>
> >> [1]: https://github.com/apache/ignite/pull/8175
> >>
> >> пт, 21 авг. 2020 г. в 03:35, Valentin Kulichenko <
> >> [hidden email]>:
> >>
> >> > Sounds good. I've added this to the 3.0 roadmap:
> >> > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0
> >> >
> >> > Unless there are any objections from others, let's stick with the
> >> > CompletableFuture for any future development, including the thin
> client.
> >> >
> >> > -Val
> >> >
> >> > On Thu, Aug 20, 2020 at 9:30 AM Pavel Tupitsyn <[hidden email]>
> >> > wrote:
> >> >
> >> > > Val, no objections from my side.
> >> > > As noted above, the only benefit of IgniteFuture is consistency
> across
> >> > > thin/thick APIs,
> >> > > which is probably not so important.
> >> > >
> >> > > On Thu, Aug 20, 2020 at 6:28 PM Valentin Kulichenko <
> >> > > [hidden email]> wrote:
> >> > >
> >> > > > Hi Pavel,
> >> > > >
> >> > > > Are there any benefits of IgniteFuture over CompletableFuture?
> >> > > >
> >> > > > IgniteFuture was created long ago, during the time when
> >> > CompletableFuture
> >> > > > did not exist. There is a big chance that IgniteFuture actually
> >> became
> >> > > > redundant at the moment we transitioned to Java8. If that's the
> >> case, I
> >> > > > would prefer using CompletableFuture in the thin client and
> getting
> >> rid
> >> > > of
> >> > > > IgniteFuture altogether in 3.0.
> >> > > >
> >> > > > What do you think?
> >> > > >
> >> > > > -Val
> >> > > >
> >> > > > On Thu, Aug 20, 2020 at 7:19 AM Pavel Tupitsyn <
> >> [hidden email]>
> >> > > > wrote:
> >> > > >
> >> > > > > Igniters,
> >> > > > >
> >> > > > > I've prepared an IEP [1], please review and let me know what you
> >> > think.
> >> > > > >
> >> > > > > In particular, I'd like to discuss the Future interface to be
> >> used:
> >> > > > > * IgniteFuture is the first candidate - Thin APIs will be
> >> consistent
> >> > > with
> >> > > > > Thick APIs, probably better for existing Ignite users.
> >> > > > > * CompletableFuture is the standard for async Java APIs. Many
> >> users
> >> > may
> >> > > > > prefer that instead of a custom IgniteFuture.
> >> > > > >
> >> > > > > [1]
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-51%3A+Java+Thin+Client+Async+API
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
>

Valentin Kulichenko

Re: IEP-51: Java Thin Client Async API

In reply to this post by Pavel Tupitsyn

Hi Pavel,

Interesting findings :) Agree that we should not use the CompletableFuture
- it clearly has a different purpose.

I think that the approach taken by Redis makes more sense. I don't like
that it requires a custom interface, but I think we can live with that.

I would be glad to hear other opinions though.

-Val

On Fri, Aug 21, 2020 at 1:02 AM Pavel Tupitsyn <[hidden email]> wrote:

> Val,
>
> The problems with CompletableFuture in public API are:
> * It is a class, not an interface
> * It is completable - anyone can call .complete(), which is not what we
> want
>
> There seems to be no clear guidance in Java world on async API design;
> however, it is often recommended to return CompletionStage instead of
> CompletableFuture
> from the public APIs [1] [2], and some products follow this [3].
>
> Other products return their own future interface that extends both Future
> and CompletionStage,
> which seems to be a better alternative to me [4].
>
> Thoughts?
>
> [1]
>
> https://stackoverflow.com/questions/47571117/what-is-the-difference-between-completionstage-and-completablefuture
> [2]
>
> https://stackoverflow.com/questions/34930840/should-i-return-completablefuture-or-future-when-defining-api
> <
> https://stackoverflow.com/questions/34930840/should-i-return-completablefuture-or-future-when-defining-api#:~:text=by%20returning%20a%20CompletableFuture%2C%20you,API%2C%20which%20is%20not%20good
> .>
> [3]
>
> https://docs.hazelcast.org/docs/latest/javadoc/com/hazelcast/cache/ICache.html
> [4]
>
> https://lettuce.io/lettuce-4/release/api/com/lambdaworks/redis/RedisFuture.html
>
> On Fri, Aug 21, 2020 at 10:28 AM Alex Plehanov <[hidden email]>
> wrote:
>
> > Pavel,
> >
> > Thanks for the discussion, I've also faced with the necessity of having
> > async calls while implementing POC for thin client data streamer [1] and
> > solve it similarly (but in my case it's required only for internal
> > implementation, so I've only changed the internal API).
> >
> > I want to note that described in IEP approach (and implemented in POC) is
> > not fully async, since "send" is still used in the user's thread. To make
> > it fully async we need additional sending thread (since blocking IO is
> used
> > for communication with the server). If partition awareness is enabled
> there
> > will be 2 threads per each server connection, perhaps we should think
> about
> > moving to NIO and introducing some kind of communication thread pool.
> >
> > [1]: https://github.com/apache/ignite/pull/8175
> >
> > пт, 21 авг. 2020 г. в 03:35, Valentin Kulichenko <
> > [hidden email]>:
> >
> > > Sounds good. I've added this to the 3.0 roadmap:
> > > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0
> > >
> > > Unless there are any objections from others, let's stick with the
> > > CompletableFuture for any future development, including the thin
> client.
> > >
> > > -Val
> > >
> > > On Thu, Aug 20, 2020 at 9:30 AM Pavel Tupitsyn <[hidden email]>
> > > wrote:
> > >
> > > > Val, no objections from my side.
> > > > As noted above, the only benefit of IgniteFuture is consistency
> across
> > > > thin/thick APIs,
> > > > which is probably not so important.
> > > >
> > > > On Thu, Aug 20, 2020 at 6:28 PM Valentin Kulichenko <
> > > > [hidden email]> wrote:
> > > >
> > > > > Hi Pavel,
> > > > >
> > > > > Are there any benefits of IgniteFuture over CompletableFuture?
> > > > >
> > > > > IgniteFuture was created long ago, during the time when
> > > CompletableFuture
> > > > > did not exist. There is a big chance that IgniteFuture actually
> > became
> > > > > redundant at the moment we transitioned to Java8. If that's the
> > case, I
> > > > > would prefer using CompletableFuture in the thin client and getting
> > rid
> > > > of
> > > > > IgniteFuture altogether in 3.0.
> > > > >
> > > > > What do you think?
> > > > >
> > > > > -Val
> > > > >
> > > > > On Thu, Aug 20, 2020 at 7:19 AM Pavel Tupitsyn <
> [hidden email]
> > >
> > > > > wrote:
> > > > >
> > > > > > Igniters,
> > > > > >
> > > > > > I've prepared an IEP [1], please review and let me know what you
> > > think.
> > > > > >
> > > > > > In particular, I'd like to discuss the Future interface to be
> used:
> > > > > > * IgniteFuture is the first candidate - Thin APIs will be
> > consistent
> > > > with
> > > > > > Thick APIs, probably better for existing Ignite users.
> > > > > > * CompletableFuture is the standard for async Java APIs. Many
> users
> > > may
> > > > > > prefer that instead of a custom IgniteFuture.
> > > > > >
> > > > > > [1]
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-51%3A+Java+Thin+Client+Async+API
> > > > > >
> > > > >
> > > >
> > >
> >
>

Pavel Tupitsyn

Re: IEP-51: Java Thin Client Async API

I've changed the IEP and added a new future interface to the POC:
interface IgniteClientFuture<T> extends Future<T>, CompletionStage<T>

The implementation simply wraps the CompletableFuture.

On Fri, Aug 21, 2020 at 11:22 PM Valentin Kulichenko <
[hidden email]> wrote:

> Hi Pavel,
>
> Interesting findings :) Agree that we should not use the CompletableFuture
> - it clearly has a different purpose.
>
> I think that the approach taken by Redis makes more sense. I don't like
> that it requires a custom interface, but I think we can live with that.
>
> I would be glad to hear other opinions though.
>
> -Val
>
> On Fri, Aug 21, 2020 at 1:02 AM Pavel Tupitsyn <[hidden email]>
> wrote:
>
> > Val,
> >
> > The problems with CompletableFuture in public API are:
> > * It is a class, not an interface
> > * It is completable - anyone can call .complete(), which is not what we
> > want
> >
> > There seems to be no clear guidance in Java world on async API design;
> > however, it is often recommended to return CompletionStage instead of
> > CompletableFuture
> > from the public APIs [1] [2], and some products follow this [3].
> >
> > Other products return their own future interface that extends both Future
> > and CompletionStage,
> > which seems to be a better alternative to me [4].
> >
> > Thoughts?
> >
> > [1]
> >
> >
> https://stackoverflow.com/questions/47571117/what-is-the-difference-between-completionstage-and-completablefuture
> > [2]
> >
> >
> https://stackoverflow.com/questions/34930840/should-i-return-completablefuture-or-future-when-defining-api
> > <
> >
> https://stackoverflow.com/questions/34930840/should-i-return-completablefuture-or-future-when-defining-api#:~:text=by%20returning%20a%20CompletableFuture%2C%20you,API%2C%20which%20is%20not%20good
> > .>
> > [3]
> >
> >
> https://docs.hazelcast.org/docs/latest/javadoc/com/hazelcast/cache/ICache.html
> > [4]
> >
> >
> https://lettuce.io/lettuce-4/release/api/com/lambdaworks/redis/RedisFuture.html
> >
> > On Fri, Aug 21, 2020 at 10:28 AM Alex Plehanov <[hidden email]>
> > wrote:
> >
> > > Pavel,
> > >
> > > Thanks for the discussion, I've also faced with the necessity of having
> > > async calls while implementing POC for thin client data streamer [1]
> and
> > > solve it similarly (but in my case it's required only for internal
> > > implementation, so I've only changed the internal API).
> > >
> > > I want to note that described in IEP approach (and implemented in POC)
> is
> > > not fully async, since "send" is still used in the user's thread. To
> make
> > > it fully async we need additional sending thread (since blocking IO is
> > used
> > > for communication with the server). If partition awareness is enabled
> > there
> > > will be 2 threads per each server connection, perhaps we should think
> > about
> > > moving to NIO and introducing some kind of communication thread pool.
> > >
> > > [1]: https://github.com/apache/ignite/pull/8175
> > >
> > > пт, 21 авг. 2020 г. в 03:35, Valentin Kulichenko <
> > > [hidden email]>:
> > >
> > > > Sounds good. I've added this to the 3.0 roadmap:
> > > > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0
> > > >
> > > > Unless there are any objections from others, let's stick with the
> > > > CompletableFuture for any future development, including the thin
> > client.
> > > >
> > > > -Val
> > > >
> > > > On Thu, Aug 20, 2020 at 9:30 AM Pavel Tupitsyn <[hidden email]
> >
> > > > wrote:
> > > >
> > > > > Val, no objections from my side.
> > > > > As noted above, the only benefit of IgniteFuture is consistency
> > across
> > > > > thin/thick APIs,
> > > > > which is probably not so important.
> > > > >
> > > > > On Thu, Aug 20, 2020 at 6:28 PM Valentin Kulichenko <
> > > > > [hidden email]> wrote:
> > > > >
> > > > > > Hi Pavel,
> > > > > >
> > > > > > Are there any benefits of IgniteFuture over CompletableFuture?
> > > > > >
> > > > > > IgniteFuture was created long ago, during the time when
> > > > CompletableFuture
> > > > > > did not exist. There is a big chance that IgniteFuture actually
> > > became
> > > > > > redundant at the moment we transitioned to Java8. If that's the
> > > case, I
> > > > > > would prefer using CompletableFuture in the thin client and
> getting
> > > rid
> > > > > of
> > > > > > IgniteFuture altogether in 3.0.
> > > > > >
> > > > > > What do you think?
> > > > > >
> > > > > > -Val
> > > > > >
> > > > > > On Thu, Aug 20, 2020 at 7:19 AM Pavel Tupitsyn <
> > [hidden email]
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Igniters,
> > > > > > >
> > > > > > > I've prepared an IEP [1], please review and let me know what
> you
> > > > think.
> > > > > > >
> > > > > > > In particular, I'd like to discuss the Future interface to be
> > used:
> > > > > > > * IgniteFuture is the first candidate - Thin APIs will be
> > > consistent
> > > > > with
> > > > > > > Thick APIs, probably better for existing Ignite users.
> > > > > > > * CompletableFuture is the standard for async Java APIs. Many
> > users
> > > > may
> > > > > > > prefer that instead of a custom IgniteFuture.
> > > > > > >
> > > > > > > [1]
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-51%3A+Java+Thin+Client+Async+API
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Igor Sapego-2

Re: IEP-51: Java Thin Client Async API

Alexey, what do you think? Which Future should be used here?

Now, about the "not fully sync" interface - I believe this is acceptable as
a first
approach.

Best Regards,
Igor

On Mon, Aug 24, 2020 at 12:37 PM Pavel Tupitsyn <[hidden email]>
wrote:

> I've changed the IEP and added a new future interface to the POC:
> interface IgniteClientFuture<T> extends Future<T>, CompletionStage<T>
>
> The implementation simply wraps the CompletableFuture.
>
> On Fri, Aug 21, 2020 at 11:22 PM Valentin Kulichenko <
> [hidden email]> wrote:
>
> > Hi Pavel,
> >
> > Interesting findings :) Agree that we should not use the
> CompletableFuture
> > - it clearly has a different purpose.
> >
> > I think that the approach taken by Redis makes more sense. I don't like
> > that it requires a custom interface, but I think we can live with that.
> >
> > I would be glad to hear other opinions though.
> >
> > -Val
> >
> > On Fri, Aug 21, 2020 at 1:02 AM Pavel Tupitsyn <[hidden email]>
> > wrote:
> >
> > > Val,
> > >
> > > The problems with CompletableFuture in public API are:
> > > * It is a class, not an interface
> > > * It is completable - anyone can call .complete(), which is not what we
> > > want
> > >
> > > There seems to be no clear guidance in Java world on async API design;
> > > however, it is often recommended to return CompletionStage instead of
> > > CompletableFuture
> > > from the public APIs [1] [2], and some products follow this [3].
> > >
> > > Other products return their own future interface that extends both
> Future
> > > and CompletionStage,
> > > which seems to be a better alternative to me [4].
> > >
> > > Thoughts?
> > >
> > > [1]
> > >
> > >
> >
> https://stackoverflow.com/questions/47571117/what-is-the-difference-between-completionstage-and-completablefuture
> > > [2]
> > >
> > >
> >
> https://stackoverflow.com/questions/34930840/should-i-return-completablefuture-or-future-when-defining-api
> > > <
> > >
> >
> https://stackoverflow.com/questions/34930840/should-i-return-completablefuture-or-future-when-defining-api#:~:text=by%20returning%20a%20CompletableFuture%2C%20you,API%2C%20which%20is%20not%20good
> > > .>
> > > [3]
> > >
> > >
> >
> https://docs.hazelcast.org/docs/latest/javadoc/com/hazelcast/cache/ICache.html
> > > [4]
> > >
> > >
> >
> https://lettuce.io/lettuce-4/release/api/com/lambdaworks/redis/RedisFuture.html
> > >
> > > On Fri, Aug 21, 2020 at 10:28 AM Alex Plehanov <
> [hidden email]>
> > > wrote:
> > >
> > > > Pavel,
> > > >
> > > > Thanks for the discussion, I've also faced with the necessity of
> having
> > > > async calls while implementing POC for thin client data streamer [1]
> > and
> > > > solve it similarly (but in my case it's required only for internal
> > > > implementation, so I've only changed the internal API).
> > > >
> > > > I want to note that described in IEP approach (and implemented in
> POC)
> > is
> > > > not fully async, since "send" is still used in the user's thread. To
> > make
> > > > it fully async we need additional sending thread (since blocking IO
> is
> > > used
> > > > for communication with the server). If partition awareness is enabled
> > > there
> > > > will be 2 threads per each server connection, perhaps we should think
> > > about
> > > > moving to NIO and introducing some kind of communication thread pool.
> > > >
> > > > [1]: https://github.com/apache/ignite/pull/8175
> > > >
> > > > пт, 21 авг. 2020 г. в 03:35, Valentin Kulichenko <
> > > > [hidden email]>:
> > > >
> > > > > Sounds good. I've added this to the 3.0 roadmap:
> > > > >
> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0
> > > > >
> > > > > Unless there are any objections from others, let's stick with the
> > > > > CompletableFuture for any future development, including the thin
> > > client.
> > > > >
> > > > > -Val
> > > > >
> > > > > On Thu, Aug 20, 2020 at 9:30 AM Pavel Tupitsyn <
> [hidden email]
> > >
> > > > > wrote:
> > > > >
> > > > > > Val, no objections from my side.
> > > > > > As noted above, the only benefit of IgniteFuture is consistency
> > > across
> > > > > > thin/thick APIs,
> > > > > > which is probably not so important.
> > > > > >
> > > > > > On Thu, Aug 20, 2020 at 6:28 PM Valentin Kulichenko <
> > > > > > [hidden email]> wrote:
> > > > > >
> > > > > > > Hi Pavel,
> > > > > > >
> > > > > > > Are there any benefits of IgniteFuture over CompletableFuture?
> > > > > > >
> > > > > > > IgniteFuture was created long ago, during the time when
> > > > > CompletableFuture
> > > > > > > did not exist. There is a big chance that IgniteFuture actually
> > > > became
> > > > > > > redundant at the moment we transitioned to Java8. If that's the
> > > > case, I
> > > > > > > would prefer using CompletableFuture in the thin client and
> > getting
> > > > rid
> > > > > > of
> > > > > > > IgniteFuture altogether in 3.0.
> > > > > > >
> > > > > > > What do you think?
> > > > > > >
> > > > > > > -Val
> > > > > > >
> > > > > > > On Thu, Aug 20, 2020 at 7:19 AM Pavel Tupitsyn <
> > > [hidden email]
> > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Igniters,
> > > > > > > >
> > > > > > > > I've prepared an IEP [1], please review and let me know what
> > you
> > > > > think.
> > > > > > > >
> > > > > > > > In particular, I'd like to discuss the Future interface to be
> > > used:
> > > > > > > > * IgniteFuture is the first candidate - Thin APIs will be
> > > > consistent
> > > > > > with
> > > > > > > > Thick APIs, probably better for existing Ignite users.
> > > > > > > > * CompletableFuture is the standard for async Java APIs. Many
> > > users
> > > > > may
> > > > > > > > prefer that instead of a custom IgniteFuture.
> > > > > > > >
> > > > > > > > [1]
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-51%3A+Java+Thin+Client+Async+API
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Vladimir Pligin

Re: IEP-51: Java Thin Client Async API

Hi guys,

It would be really cool to have async operations.
My point is that the worst thing that we have at the moment is that we need
to write data to the socket under the sndLock.
It makes it nearly impossible for users to implement async operations on
their side.
Of course they can scale up by adding new instances of IgniteClient but it
leads to the growing number of connections.
Sometimes it's not an option. My feeling is that we should focus on getting
rid of that stuff.
In general proposed API looks really great for me.
Thanks for the design!

--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Alexey Goncharuk

Re: IEP-51: Java Thin Client Async API

Hello folks,

I agree with Val: IgniteFuture was created long before we could use the
Java futures and is being kept for compatibility reasons. While keeping API
consistent between thin/thick clients is a good reason, I think moving to
Java futures benefits more to the project and end-users.

Agree with the current state of the IEP.

mnk

Re: IEP-51: Java Thin Client Async API

So one question I have here (and this maybe out of scope) - but what about
the cases where the result of a remote execution is a CompletionStage? In
this case, I would want to have a local CompletionStage which "represents"
the remote one. In other words when the remote one is completed, the value
should be sent back to the local node for completion the local stage. This
could even be "doubly asynchronous" - in other words, I executeAsync and the
result of that is a CompletionStage. Note that this differs from what is
supported today in that the result (the CompletionStage) is immediately
serialized back to the local node, which of course isn't what's needed. More
advanced would be the case where the result of the remote execution is a
CompleteableFuture in which case I may want the ability to cancel it
remotely as well.

--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

mnk

Re: IEP-51: Java Thin Client Async API

In reply to this post by Alexey Goncharuk

Pavel Tupitsyn

Re: IEP-51: Java Thin Client Async API

> result of a remote execution is a CompletionStage

Can you give an example? What is a remote execution? Is this about Compute
and/or Services?

On Mon, Sep 21, 2020 at 5:11 PM mnk <[hidden email]> wrote:

> So one question I have here (and this maybe out of scope) - but what about
> the cases where the result of a remote execution is a CompletionStage? In
> this case, I would want to have a local CompletionStage which "represents"
> the remote one. In other words when the remote one is completed, the value
> should be sent back to the local node for completion the local stage. This
> could even be "doubly asynchronous" - in other words, I executeAsync and
> the
> result of that is a CompletionStage. Note that this differs from what is
> supported today in that the result (the CompletionStage) is immediately
> serialized back to the local node, which of course isn't what's needed.
> More
> advanced would be the case where the result of the remote execution is a
> CompleteableFuture in which case I may want the ability to cancel it
> remotely as well.
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>

mnk

Re: IEP-51: Java Thin Client Async API

Pavel Tupitsyn wrote
>> result of a remote execution is a CompletionStage
>
> Can you give an example? What is a remote execution? Is this about Compute
> and/or Services?

This is about Compute.

Let's say I'm doing affinityCallAsync for IgniteCallable<R> where R
implements CompletableStage<T> or R is a CompletableFuture<T>. Then I
wouldn't want to have the CompleteableStage or CompleteableFuture come back
to me actually, rather something that will complete with the T when it's
ready (and it would be nice if I could also cancel it for the
CompleteableFuture case). It could be done as well for affinityCall
(non-async) too, although in my mind the former case is the more important
one. Of course, affinityCall is just an example, it should apply to the
other compute methods as appropriate.

--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Pavel Tupitsyn

Re: IEP-51: Java Thin Client Async API

Yes, this makes a lot of sense (and can be applied to Services, too).

I've filed the ticket: https://issues.apache.org/jira/browse/IGNITE-13471
This requires a separate IEP, of course.

On Mon, Sep 21, 2020 at 6:33 PM mnk <[hidden email]> wrote:

> Pavel Tupitsyn wrote
> >> result of a remote execution is a CompletionStage
> >
> > Can you give an example? What is a remote execution? Is this about
> Compute
> > and/or Services?
>
> This is about Compute.
>
> Let's say I'm doing affinityCallAsync for IgniteCallable<R> where R
> implements CompletableStage<T> or R is a CompletableFuture<T>. Then I
> wouldn't want to have the CompleteableStage or CompleteableFuture come back
> to me actually, rather something that will complete with the T when it's
> ready (and it would be nice if I could also cancel it for the
> CompleteableFuture case). It could be done as well for affinityCall
> (non-async) too, although in my mind the former case is the more important
> one. Of course, affinityCall is just an example, it should apply to the
> other compute methods as appropriate.
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>

Pavel Tupitsyn

Re: IEP-51: Java Thin Client Async API

Igniters,

The PR is ready for review
https://github.com/apache/ignite/pull/8174

On Tue, Sep 22, 2020 at 11:51 AM Pavel Tupitsyn <[hidden email]>
wrote:

> Yes, this makes a lot of sense (and can be applied to Services, too).
>
> I've filed the ticket: https://issues.apache.org/jira/browse/IGNITE-13471
> This requires a separate IEP, of course.
>
> On Mon, Sep 21, 2020 at 6:33 PM mnk <[hidden email]> wrote:
>
>> Pavel Tupitsyn wrote
>> >> result of a remote execution is a CompletionStage
>> >
>> > Can you give an example? What is a remote execution? Is this about
>> Compute
>> > and/or Services?
>>
>> This is about Compute.
>>
>> Let's say I'm doing affinityCallAsync for IgniteCallable<R> where R
>> implements CompletableStage<T> or R is a CompletableFuture<T>. Then I
>> wouldn't want to have the CompleteableStage or CompleteableFuture come
>> back
>> to me actually, rather something that will complete with the T when it's
>> ready (and it would be nice if I could also cancel it for the
>> CompleteableFuture case). It could be done as well for affinityCall
>> (non-async) too, although in my mind the former case is the more important
>> one. Of course, affinityCall is just an example, it should apply to the
>> other compute methods as appropriate.
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>>
>