IEP-44 Thin Client Discovery

[Pavel Tupitsyn]

Igniters,

I've prepared an IEP [1] and a POC [2] for Thin Client Discovery feature.
Let's discuss it here.

In particular, I'd like to address the following points:

1. Value: do you think this would be a good feature to have?
2. Public API changes: is a boolean property enough? Should we have
something more complex, so users can plug in custom logic to filter and/or
translate IPs and host names?
3. Server-side implementation details: should we use Compute, Node
Attributes, or something else to retrieve client endpoints from all nodes
in cluster?

[1]
https://cwiki.apache.org/confluence/display/IGNITE/IEP-44%3A+Thin+client+cluster+discovery
[2] https://github.com/apache/ignite/pull/7744
[3] https://issues.apache.org/jira/browse/IGNITE-12932

[Igor Sapego-2]

Great, this feature is long awaited.

1. I believe so. Since I've proposed Partition Awareness feature I was
thinking
about a way for clients to discover cluster nodes.

2. In my opinion a simple boolean flag is enough for the beginning. In
future
maybe we can add a node filter. This can be useful in cases when part of a
cluster is not available from client network or if user wants to avoid
connection
to a node for some other reason.

3. There is already a list of addresses of every node kept by every node,
isn't it?

Best Regards,
Igor


On Mon, Apr 27, 2020 at 12:58 PM Pavel Tupitsyn <[hidden email]>
wrote:

> Igniters,
>
> I've prepared an IEP [1] and a POC [2] for Thin Client Discovery feature.
> Let's discuss it here.
>
> In particular, I'd like to address the following points:
>
> 1. Value: do you think this would be a good feature to have?
> 2. Public API changes: is a boolean property enough? Should we have
> something more complex, so users can plug in custom logic to filter and/or
> translate IPs and host names?
> 3. Server-side implementation details: should we use Compute, Node
> Attributes, or something else to retrieve client endpoints from all nodes
> in cluster?
>
> [1]
>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-44%3A+Thin+client+cluster+discovery
> [2] https://github.com/apache/ignite/pull/7744
> [3] https://issues.apache.org/jira/browse/IGNITE-12932
>

[Pavel Tupitsyn]

> There is already a list of addresses of every node kept by every node,
isn't it?

Can you please elaborate, what exactly do you mean here?
Sure, we can get addresses of other nodes from Discovery/Communication SPIs,
but those addresses may be not what we want. Nodes can belong to multiple
subnets.

For example, server nodes may communicate over 192.168.0.0/24, but expose
client endpoints over 10.0.0.0/24 as well.

We have to conform with IgniteConfiguration.LocalHost setting.
If it is not set, or set to 0.0.0.0, we should gather IPs from all
interfaces.
But if it is set to something, we should gather only matching IPs.

On Mon, Apr 27, 2020 at 5:43 PM Igor Sapego <[hidden email]> wrote:

> Great, this feature is long awaited.
>
> 1. I believe so. Since I've proposed Partition Awareness feature I was
> thinking
> about a way for clients to discover cluster nodes.
>
> 2. In my opinion a simple boolean flag is enough for the beginning. In
> future
> maybe we can add a node filter. This can be useful in cases when part of a
> cluster is not available from client network or if user wants to avoid
> connection
> to a node for some other reason.
>
> 3. There is already a list of addresses of every node kept by every node,
> isn't it?
>
> Best Regards,
> Igor
>
>
> On Mon, Apr 27, 2020 at 12:58 PM Pavel Tupitsyn <[hidden email]>
> wrote:
>
> > Igniters,
> >
> > I've prepared an IEP [1] and a POC [2] for Thin Client Discovery feature.
> > Let's discuss it here.
> >
> > In particular, I'd like to address the following points:
> >
> > 1. Value: do you think this would be a good feature to have?
> > 2. Public API changes: is a boolean property enough? Should we have
> > something more complex, so users can plug in custom logic to filter
> and/or
> > translate IPs and host names?
> > 3. Server-side implementation details: should we use Compute, Node
> > Attributes, or something else to retrieve client endpoints from all nodes
> > in cluster?
> >
> > [1]
> >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-44%3A+Thin+client+cluster+discovery
> > [2] https://github.com/apache/ignite/pull/7744
> > [3] https://issues.apache.org/jira/browse/IGNITE-12932
> >
>

[Pavel Tupitsyn]

Igor, actually you are right.

We already have org.apache.ignite.ips node attribute, which contains all
the IPs.
No need to add anything else, just filter the value according to
IgniteConfiguration.LocalHost.
I'll update the IEP and POC.

Thanks



On Mon, Apr 27, 2020 at 8:17 PM Pavel Tupitsyn <[hidden email]> wrote:

> > There is already a list of addresses of every node kept by every node,
> isn't it?
>
> Can you please elaborate, what exactly do you mean here?
> Sure, we can get addresses of other nodes from Discovery/Communication
> SPIs,
> but those addresses may be not what we want. Nodes can belong to multiple
> subnets.
>
> For example, server nodes may communicate over 192.168.0.0/24, but expose
> client endpoints over 10.0.0.0/24 as well.
>
> We have to conform with IgniteConfiguration.LocalHost setting.
> If it is not set, or set to 0.0.0.0, we should gather IPs from all
> interfaces.
> But if it is set to something, we should gather only matching IPs.
>
> On Mon, Apr 27, 2020 at 5:43 PM Igor Sapego <[hidden email]> wrote:
>
>> Great, this feature is long awaited.
>>
>> 1. I believe so. Since I've proposed Partition Awareness feature I was
>> thinking
>> about a way for clients to discover cluster nodes.
>>
>> 2. In my opinion a simple boolean flag is enough for the beginning. In
>> future
>> maybe we can add a node filter. This can be useful in cases when part of a
>> cluster is not available from client network or if user wants to avoid
>> connection
>> to a node for some other reason.
>>
>> 3. There is already a list of addresses of every node kept by every node,
>> isn't it?
>>
>> Best Regards,
>> Igor
>>
>>
>> On Mon, Apr 27, 2020 at 12:58 PM Pavel Tupitsyn <[hidden email]>
>> wrote:
>>
>> > Igniters,
>> >
>> > I've prepared an IEP [1] and a POC [2] for Thin Client Discovery
>> feature.
>> > Let's discuss it here.
>> >
>> > In particular, I'd like to address the following points:
>> >
>> > 1. Value: do you think this would be a good feature to have?
>> > 2. Public API changes: is a boolean property enough? Should we have
>> > something more complex, so users can plug in custom logic to filter
>> and/or
>> > translate IPs and host names?
>> > 3. Server-side implementation details: should we use Compute, Node
>> > Attributes, or something else to retrieve client endpoints from all
>> nodes
>> > in cluster?
>> >
>> > [1]
>> >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-44%3A+Thin+client+cluster+discovery
>> > [2] https://github.com/apache/ignite/pull/7744
>> > [3] https://issues.apache.org/jira/browse/IGNITE-12932
>> >
>>
>

[Igor Sapego-2]

Yeah, I remember that because I used this approach when I implemented
first experimental version of partition awareness for the C++ thin client.
Only thing we need to add is the "ThinClientPort" node attribute, or
something like that.

Best Regards,
Igor


On Mon, Apr 27, 2020 at 8:41 PM Pavel Tupitsyn <[hidden email]> wrote:

> Igor, actually you are right.
>
> We already have org.apache.ignite.ips node attribute, which contains all
> the IPs.
> No need to add anything else, just filter the value according to
> IgniteConfiguration.LocalHost.
> I'll update the IEP and POC.
>
> Thanks
>
>
>
> On Mon, Apr 27, 2020 at 8:17 PM Pavel Tupitsyn <[hidden email]>
> wrote:
>
> > > There is already a list of addresses of every node kept by every node,
> > isn't it?
> >
> > Can you please elaborate, what exactly do you mean here?
> > Sure, we can get addresses of other nodes from Discovery/Communication
> > SPIs,
> > but those addresses may be not what we want. Nodes can belong to multiple
> > subnets.
> >
> > For example, server nodes may communicate over 192.168.0.0/24, but
> expose
> > client endpoints over 10.0.0.0/24 as well.
> >
> > We have to conform with IgniteConfiguration.LocalHost setting.
> > If it is not set, or set to 0.0.0.0, we should gather IPs from all
> > interfaces.
> > But if it is set to something, we should gather only matching IPs.
> >
> > On Mon, Apr 27, 2020 at 5:43 PM Igor Sapego <[hidden email]> wrote:
> >
> >> Great, this feature is long awaited.
> >>
> >> 1. I believe so. Since I've proposed Partition Awareness feature I was
> >> thinking
> >> about a way for clients to discover cluster nodes.
> >>
> >> 2. In my opinion a simple boolean flag is enough for the beginning. In
> >> future
> >> maybe we can add a node filter. This can be useful in cases when part
> of a
> >> cluster is not available from client network or if user wants to avoid
> >> connection
> >> to a node for some other reason.
> >>
> >> 3. There is already a list of addresses of every node kept by every
> node,
> >> isn't it?
> >>
> >> Best Regards,
> >> Igor
> >>
> >>
> >> On Mon, Apr 27, 2020 at 12:58 PM Pavel Tupitsyn <[hidden email]>
> >> wrote:
> >>
> >> > Igniters,
> >> >
> >> > I've prepared an IEP [1] and a POC [2] for Thin Client Discovery
> >> feature.
> >> > Let's discuss it here.
> >> >
> >> > In particular, I'd like to address the following points:
> >> >
> >> > 1. Value: do you think this would be a good feature to have?
> >> > 2. Public API changes: is a boolean property enough? Should we have
> >> > something more complex, so users can plug in custom logic to filter
> >> and/or
> >> > translate IPs and host names?
> >> > 3. Server-side implementation details: should we use Compute, Node
> >> > Attributes, or something else to retrieve client endpoints from all
> >> nodes
> >> > in cluster?
> >> >
> >> > [1]
> >> >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-44%3A+Thin+client+cluster+discovery
> >> > [2] https://github.com/apache/ignite/pull/7744
> >> > [3] https://issues.apache.org/jira/browse/IGNITE-12932
> >> >
> >>
> >
>

[dmagda]

Pavel,

That would be a tremendous improvement for the recently release best effort
affinity feature. Without this capability, we force application developers
to reopen thin client connections every type a cluster is scaled out. I
believe that once the folks start using the best effort affinity, we'll be
hearing more of a feature request for what you're proposing in this thread.
So, thanks for taking care of this proactively!

As for the public API changes, do we really need any extra flag? I would
enable the capability if the best effort affinity is on. For me, it's just
a natural improvement of the latter and it sounds reasonable to reuse the
best effort affinity's flag.

-
Denis


On Mon, Apr 27, 2020 at 2:58 AM Pavel Tupitsyn <[hidden email]> wrote:

> Igniters,
>
> I've prepared an IEP [1] and a POC [2] for Thin Client Discovery feature.
> Let's discuss it here.
>
> In particular, I'd like to address the following points:
>
> 1. Value: do you think this would be a good feature to have?
> 2. Public API changes: is a boolean property enough? Should we have
> something more complex, so users can plug in custom logic to filter and/or
> translate IPs and host names?
> 3. Server-side implementation details: should we use Compute, Node
> Attributes, or something else to retrieve client endpoints from all nodes
> in cluster?
>
> [1]
>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-44%3A+Thin+client+cluster+discovery
> [2] https://github.com/apache/ignite/pull/7744
> [3] https://issues.apache.org/jira/browse/IGNITE-12932
>

[Pavel Tupitsyn]

> enable the capability if the best effort affinity is on
I agree, makes sense.

Igor, what do you think?

On Tue, Apr 28, 2020 at 8:25 AM Denis Magda <[hidden email]> wrote:

> Pavel,
>
> That would be a tremendous improvement for the recently release best effort
> affinity feature. Without this capability, we force application developers
> to reopen thin client connections every type a cluster is scaled out. I
> believe that once the folks start using the best effort affinity, we'll be
> hearing more of a feature request for what you're proposing in this thread.
> So, thanks for taking care of this proactively!
>
> As for the public API changes, do we really need any extra flag? I would
> enable the capability if the best effort affinity is on. For me, it's just
> a natural improvement of the latter and it sounds reasonable to reuse the
> best effort affinity's flag.
>
> -
> Denis
>
>
> On Mon, Apr 27, 2020 at 2:58 AM Pavel Tupitsyn <[hidden email]>
> wrote:
>
> > Igniters,
> >
> > I've prepared an IEP [1] and a POC [2] for Thin Client Discovery feature.
> > Let's discuss it here.
> >
> > In particular, I'd like to address the following points:
> >
> > 1. Value: do you think this would be a good feature to have?
> > 2. Public API changes: is a boolean property enough? Should we have
> > something more complex, so users can plug in custom logic to filter
> and/or
> > translate IPs and host names?
> > 3. Server-side implementation details: should we use Compute, Node
> > Attributes, or something else to retrieve client endpoints from all nodes
> > in cluster?
> >
> > [1]
> >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-44%3A+Thin+client+cluster+discovery
> > [2] https://github.com/apache/ignite/pull/7744
> > [3] https://issues.apache.org/jira/browse/IGNITE-12932
> >
>

[Igor Sapego-2]

I guess it makes sense. If anyone needs more control over connection
we would need to implement a new feature anyway (like node filter we
discussed earlier)

Best Regards,
Igor


On Tue, Apr 28, 2020 at 12:29 PM Pavel Tupitsyn <[hidden email]>
wrote:

> > enable the capability if the best effort affinity is on
> I agree, makes sense.
>
> Igor, what do you think?
>
> On Tue, Apr 28, 2020 at 8:25 AM Denis Magda <[hidden email]> wrote:
>
> > Pavel,
> >
> > That would be a tremendous improvement for the recently release best
> effort
> > affinity feature. Without this capability, we force application
> developers
> > to reopen thin client connections every type a cluster is scaled out. I
> > believe that once the folks start using the best effort affinity, we'll
> be
> > hearing more of a feature request for what you're proposing in this
> thread.
> > So, thanks for taking care of this proactively!
> >
> > As for the public API changes, do we really need any extra flag? I would
> > enable the capability if the best effort affinity is on. For me, it's
> just
> > a natural improvement of the latter and it sounds reasonable to reuse the
> > best effort affinity's flag.
> >
> > -
> > Denis
> >
> >
> > On Mon, Apr 27, 2020 at 2:58 AM Pavel Tupitsyn <[hidden email]>
> > wrote:
> >
> > > Igniters,
> > >
> > > I've prepared an IEP [1] and a POC [2] for Thin Client Discovery
> feature.
> > > Let's discuss it here.
> > >
> > > In particular, I'd like to address the following points:
> > >
> > > 1. Value: do you think this would be a good feature to have?
> > > 2. Public API changes: is a boolean property enough? Should we have
> > > something more complex, so users can plug in custom logic to filter
> > and/or
> > > translate IPs and host names?
> > > 3. Server-side implementation details: should we use Compute, Node
> > > Attributes, or something else to retrieve client endpoints from all
> nodes
> > > in cluster?
> > >
> > > [1]
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-44%3A+Thin+client+cluster+discovery
> > > [2] https://github.com/apache/ignite/pull/7744
> > > [3] https://issues.apache.org/jira/browse/IGNITE-12932
> > >
> >
>

[Pavel Tupitsyn]

Ok, I've updated IEP and POC accordingly:
* Config flag removed
* IPs and host names retrieval simplified - use existing node properties
and attributes instead of Compute

On Tue, Apr 28, 2020 at 7:57 PM Igor Sapego <[hidden email]> wrote:

> I guess it makes sense. If anyone needs more control over connection
> we would need to implement a new feature anyway (like node filter we
> discussed earlier)
>
> Best Regards,
> Igor
>
>
> On Tue, Apr 28, 2020 at 12:29 PM Pavel Tupitsyn <[hidden email]>
> wrote:
>
> > > enable the capability if the best effort affinity is on
> > I agree, makes sense.
> >
> > Igor, what do you think?
> >
> > On Tue, Apr 28, 2020 at 8:25 AM Denis Magda <[hidden email]> wrote:
> >
> > > Pavel,
> > >
> > > That would be a tremendous improvement for the recently release best
> > effort
> > > affinity feature. Without this capability, we force application
> > developers
> > > to reopen thin client connections every type a cluster is scaled out. I
> > > believe that once the folks start using the best effort affinity, we'll
> > be
> > > hearing more of a feature request for what you're proposing in this
> > thread.
> > > So, thanks for taking care of this proactively!
> > >
> > > As for the public API changes, do we really need any extra flag? I
> would
> > > enable the capability if the best effort affinity is on. For me, it's
> > just
> > > a natural improvement of the latter and it sounds reasonable to reuse
> the
> > > best effort affinity's flag.
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Mon, Apr 27, 2020 at 2:58 AM Pavel Tupitsyn <[hidden email]>
> > > wrote:
> > >
> > > > Igniters,
> > > >
> > > > I've prepared an IEP [1] and a POC [2] for Thin Client Discovery
> > feature.
> > > > Let's discuss it here.
> > > >
> > > > In particular, I'd like to address the following points:
> > > >
> > > > 1. Value: do you think this would be a good feature to have?
> > > > 2. Public API changes: is a boolean property enough? Should we have
> > > > something more complex, so users can plug in custom logic to filter
> > > and/or
> > > > translate IPs and host names?
> > > > 3. Server-side implementation details: should we use Compute, Node
> > > > Attributes, or something else to retrieve client endpoints from all
> > nodes
> > > > in cluster?
> > > >
> > > > [1]
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-44%3A+Thin+client+cluster+discovery
> > > > [2] https://github.com/apache/ignite/pull/7744
> > > > [3] https://issues.apache.org/jira/browse/IGNITE-12932
> > > >
> > >
> >
>

[Pavel Tupitsyn]

Igniters, let's discuss the following issue:

For partition awareness, and now for cluster discovery, we use a response
flag to detect topology changes.
The problem is - if the client does not do anything (user code does not
perform operations),
then we'll never know about topology changes and may even lose the cluster
(all known nodes leave).

Should we introduce some keep-alive mechanism, so that thin clients send
periodic ping requests?
Maybe do this as a separate feature.

Thoughts?

On Tue, Apr 28, 2020 at 8:14 PM Pavel Tupitsyn <[hidden email]> wrote:

> Ok, I've updated IEP and POC accordingly:
> * Config flag removed
> * IPs and host names retrieval simplified - use existing node properties
> and attributes instead of Compute
>
> On Tue, Apr 28, 2020 at 7:57 PM Igor Sapego <[hidden email]> wrote:
>
>> I guess it makes sense. If anyone needs more control over connection
>> we would need to implement a new feature anyway (like node filter we
>> discussed earlier)
>>
>> Best Regards,
>> Igor
>>
>>
>> On Tue, Apr 28, 2020 at 12:29 PM Pavel Tupitsyn <[hidden email]>
>> wrote:
>>
>> > > enable the capability if the best effort affinity is on
>> > I agree, makes sense.
>> >
>> > Igor, what do you think?
>> >
>> > On Tue, Apr 28, 2020 at 8:25 AM Denis Magda <[hidden email]> wrote:
>> >
>> > > Pavel,
>> > >
>> > > That would be a tremendous improvement for the recently release best
>> > effort
>> > > affinity feature. Without this capability, we force application
>> > developers
>> > > to reopen thin client connections every type a cluster is scaled out.
>> I
>> > > believe that once the folks start using the best effort affinity,
>> we'll
>> > be
>> > > hearing more of a feature request for what you're proposing in this
>> > thread.
>> > > So, thanks for taking care of this proactively!
>> > >
>> > > As for the public API changes, do we really need any extra flag? I
>> would
>> > > enable the capability if the best effort affinity is on. For me, it's
>> > just
>> > > a natural improvement of the latter and it sounds reasonable to reuse
>> the
>> > > best effort affinity's flag.
>> > >
>> > > -
>> > > Denis
>> > >
>> > >
>> > > On Mon, Apr 27, 2020 at 2:58 AM Pavel Tupitsyn <[hidden email]>
>> > > wrote:
>> > >
>> > > > Igniters,
>> > > >
>> > > > I've prepared an IEP [1] and a POC [2] for Thin Client Discovery
>> > feature.
>> > > > Let's discuss it here.
>> > > >
>> > > > In particular, I'd like to address the following points:
>> > > >
>> > > > 1. Value: do you think this would be a good feature to have?
>> > > > 2. Public API changes: is a boolean property enough? Should we have
>> > > > something more complex, so users can plug in custom logic to filter
>> > > and/or
>> > > > translate IPs and host names?
>> > > > 3. Server-side implementation details: should we use Compute, Node
>> > > > Attributes, or something else to retrieve client endpoints from all
>> > nodes
>> > > > in cluster?
>> > > >
>> > > > [1]
>> > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-44%3A+Thin+client+cluster+discovery
>> > > > [2] https://github.com/apache/ignite/pull/7744
>> > > > [3] https://issues.apache.org/jira/browse/IGNITE-12932
>> > > >
>> > >
>> >
>>
>

[Alexey Plekhanov]

Pavel,

Since we have a notification mechanism for thin clients now, we can
implement a subscription to some types of events and this can be used
to inform a client about topology change as well. I think it's a
more appropriate way to detect topology changes than ping requests. But
approach with ping requests has another advantage: the client can detect
that connection was lost earlier. With subscriptions approach client will
detect problems with a connection only after the next request to the server.


ср, 6 мая 2020 г. в 17:31, Pavel Tupitsyn <[hidden email]>:

> Igniters, let's discuss the following issue:
>
> For partition awareness, and now for cluster discovery, we use a response
> flag to detect topology changes.
> The problem is - if the client does not do anything (user code does not
> perform operations),
> then we'll never know about topology changes and may even lose the cluster
> (all known nodes leave).
>
> Should we introduce some keep-alive mechanism, so that thin clients send
> periodic ping requests?
> Maybe do this as a separate feature.
>
> Thoughts?
>
> On Tue, Apr 28, 2020 at 8:14 PM Pavel Tupitsyn <[hidden email]>
> wrote:
>
> > Ok, I've updated IEP and POC accordingly:
> > * Config flag removed
> > * IPs and host names retrieval simplified - use existing node properties
> > and attributes instead of Compute
> >
> > On Tue, Apr 28, 2020 at 7:57 PM Igor Sapego <[hidden email]> wrote:
> >
> >> I guess it makes sense. If anyone needs more control over connection
> >> we would need to implement a new feature anyway (like node filter we
> >> discussed earlier)
> >>
> >> Best Regards,
> >> Igor
> >>
> >>
> >> On Tue, Apr 28, 2020 at 12:29 PM Pavel Tupitsyn <[hidden email]>
> >> wrote:
> >>
> >> > > enable the capability if the best effort affinity is on
> >> > I agree, makes sense.
> >> >
> >> > Igor, what do you think?
> >> >
> >> > On Tue, Apr 28, 2020 at 8:25 AM Denis Magda <[hidden email]>
> wrote:
> >> >
> >> > > Pavel,
> >> > >
> >> > > That would be a tremendous improvement for the recently release best
> >> > effort
> >> > > affinity feature. Without this capability, we force application
> >> > developers
> >> > > to reopen thin client connections every type a cluster is scaled
> out.
> >> I
> >> > > believe that once the folks start using the best effort affinity,
> >> we'll
> >> > be
> >> > > hearing more of a feature request for what you're proposing in this
> >> > thread.
> >> > > So, thanks for taking care of this proactively!
> >> > >
> >> > > As for the public API changes, do we really need any extra flag? I
> >> would
> >> > > enable the capability if the best effort affinity is on. For me,
> it's
> >> > just
> >> > > a natural improvement of the latter and it sounds reasonable to
> reuse
> >> the
> >> > > best effort affinity's flag.
> >> > >
> >> > > -
> >> > > Denis
> >> > >
> >> > >
> >> > > On Mon, Apr 27, 2020 at 2:58 AM Pavel Tupitsyn <
> [hidden email]>
> >> > > wrote:
> >> > >
> >> > > > Igniters,
> >> > > >
> >> > > > I've prepared an IEP [1] and a POC [2] for Thin Client Discovery
> >> > feature.
> >> > > > Let's discuss it here.
> >> > > >
> >> > > > In particular, I'd like to address the following points:
> >> > > >
> >> > > > 1. Value: do you think this would be a good feature to have?
> >> > > > 2. Public API changes: is a boolean property enough? Should we
> have
> >> > > > something more complex, so users can plug in custom logic to
> filter
> >> > > and/or
> >> > > > translate IPs and host names?
> >> > > > 3. Server-side implementation details: should we use Compute, Node
> >> > > > Attributes, or something else to retrieve client endpoints from
> all
> >> > nodes
> >> > > > in cluster?
> >> > > >
> >> > > > [1]
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-44%3A+Thin+client+cluster+discovery
> >> > > > [2] https://github.com/apache/ignite/pull/7744
> >> > > > [3] https://issues.apache.org/jira/browse/IGNITE-12932
> >> > > >
> >> > >
> >> >
> >>
> >
>

[Ilya Kasnacheev]

Hello!

I think it is OK if client loses cluster, in this case it should only check
the addresses with which it was configured to start.


Regards,
--
Ilya Kasnacheev


ср, 6 мая 2020 г. в 17:31, Pavel Tupitsyn <[hidden email]>:

> Igniters, let's discuss the following issue:
>
> For partition awareness, and now for cluster discovery, we use a response
> flag to detect topology changes.
> The problem is - if the client does not do anything (user code does not
> perform operations),
> then we'll never know about topology changes and may even lose the cluster
> (all known nodes leave).
>
> Should we introduce some keep-alive mechanism, so that thin clients send
> periodic ping requests?
> Maybe do this as a separate feature.
>
> Thoughts?
>
> On Tue, Apr 28, 2020 at 8:14 PM Pavel Tupitsyn <[hidden email]>
> wrote:
>
> > Ok, I've updated IEP and POC accordingly:
> > * Config flag removed
> > * IPs and host names retrieval simplified - use existing node properties
> > and attributes instead of Compute
> >
> > On Tue, Apr 28, 2020 at 7:57 PM Igor Sapego <[hidden email]> wrote:
> >
> >> I guess it makes sense. If anyone needs more control over connection
> >> we would need to implement a new feature anyway (like node filter we
> >> discussed earlier)
> >>
> >> Best Regards,
> >> Igor
> >>
> >>
> >> On Tue, Apr 28, 2020 at 12:29 PM Pavel Tupitsyn <[hidden email]>
> >> wrote:
> >>
> >> > > enable the capability if the best effort affinity is on
> >> > I agree, makes sense.
> >> >
> >> > Igor, what do you think?
> >> >
> >> > On Tue, Apr 28, 2020 at 8:25 AM Denis Magda <[hidden email]>
> wrote:
> >> >
> >> > > Pavel,
> >> > >
> >> > > That would be a tremendous improvement for the recently release best
> >> > effort
> >> > > affinity feature. Without this capability, we force application
> >> > developers
> >> > > to reopen thin client connections every type a cluster is scaled
> out.
> >> I
> >> > > believe that once the folks start using the best effort affinity,
> >> we'll
> >> > be
> >> > > hearing more of a feature request for what you're proposing in this
> >> > thread.
> >> > > So, thanks for taking care of this proactively!
> >> > >
> >> > > As for the public API changes, do we really need any extra flag? I
> >> would
> >> > > enable the capability if the best effort affinity is on. For me,
> it's
> >> > just
> >> > > a natural improvement of the latter and it sounds reasonable to
> reuse
> >> the
> >> > > best effort affinity's flag.
> >> > >
> >> > > -
> >> > > Denis
> >> > >
> >> > >
> >> > > On Mon, Apr 27, 2020 at 2:58 AM Pavel Tupitsyn <
> [hidden email]>
> >> > > wrote:
> >> > >
> >> > > > Igniters,
> >> > > >
> >> > > > I've prepared an IEP [1] and a POC [2] for Thin Client Discovery
> >> > feature.
> >> > > > Let's discuss it here.
> >> > > >
> >> > > > In particular, I'd like to address the following points:
> >> > > >
> >> > > > 1. Value: do you think this would be a good feature to have?
> >> > > > 2. Public API changes: is a boolean property enough? Should we
> have
> >> > > > something more complex, so users can plug in custom logic to
> filter
> >> > > and/or
> >> > > > translate IPs and host names?
> >> > > > 3. Server-side implementation details: should we use Compute, Node
> >> > > > Attributes, or something else to retrieve client endpoints from
> all
> >> > nodes
> >> > > > in cluster?
> >> > > >
> >> > > > [1]
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-44%3A+Thin+client+cluster+discovery
> >> > > > [2] https://github.com/apache/ignite/pull/7744
> >> > > > [3] https://issues.apache.org/jira/browse/IGNITE-12932
> >> > > >
> >> > >
> >> >
> >>
> >
>

[Igor Sapego-2]

Pavel,

First of all, I think we need to move it out of scope of this feature.

Also, why do we need to keep connection alive? I think, if user do not
use connection for a long time and connection is lost we could just
detect this and re-establish connection on the next user action which
requires network interaction. Any issues with this approach?

Best Regards,
Igor


On Thu, May 7, 2020 at 1:18 AM Alex Plehanov <[hidden email]>
wrote:

> Pavel,
>
> Since we have a notification mechanism for thin clients now, we can
> implement a subscription to some types of events and this can be used
> to inform a client about topology change as well. I think it's a
> more appropriate way to detect topology changes than ping requests. But
> approach with ping requests has another advantage: the client can detect
> that connection was lost earlier. With subscriptions approach client will
> detect problems with a connection only after the next request to the
> server.
>
>
> ср, 6 мая 2020 г. в 17:31, Pavel Tupitsyn <[hidden email]>:
>
> > Igniters, let's discuss the following issue:
> >
> > For partition awareness, and now for cluster discovery, we use a response
> > flag to detect topology changes.
> > The problem is - if the client does not do anything (user code does not
> > perform operations),
> > then we'll never know about topology changes and may even lose the
> cluster
> > (all known nodes leave).
> >
> > Should we introduce some keep-alive mechanism, so that thin clients send
> > periodic ping requests?
> > Maybe do this as a separate feature.
> >
> > Thoughts?
> >
> > On Tue, Apr 28, 2020 at 8:14 PM Pavel Tupitsyn <[hidden email]>
> > wrote:
> >
> > > Ok, I've updated IEP and POC accordingly:
> > > * Config flag removed
> > > * IPs and host names retrieval simplified - use existing node
> properties
> > > and attributes instead of Compute
> > >
> > > On Tue, Apr 28, 2020 at 7:57 PM Igor Sapego <[hidden email]>
> wrote:
> > >
> > >> I guess it makes sense. If anyone needs more control over connection
> > >> we would need to implement a new feature anyway (like node filter we
> > >> discussed earlier)
> > >>
> > >> Best Regards,
> > >> Igor
> > >>
> > >>
> > >> On Tue, Apr 28, 2020 at 12:29 PM Pavel Tupitsyn <[hidden email]
> >
> > >> wrote:
> > >>
> > >> > > enable the capability if the best effort affinity is on
> > >> > I agree, makes sense.
> > >> >
> > >> > Igor, what do you think?
> > >> >
> > >> > On Tue, Apr 28, 2020 at 8:25 AM Denis Magda <[hidden email]>
> > wrote:
> > >> >
> > >> > > Pavel,
> > >> > >
> > >> > > That would be a tremendous improvement for the recently release
> best
> > >> > effort
> > >> > > affinity feature. Without this capability, we force application
> > >> > developers
> > >> > > to reopen thin client connections every type a cluster is scaled
> > out.
> > >> I
> > >> > > believe that once the folks start using the best effort affinity,
> > >> we'll
> > >> > be
> > >> > > hearing more of a feature request for what you're proposing in
> this
> > >> > thread.
> > >> > > So, thanks for taking care of this proactively!
> > >> > >
> > >> > > As for the public API changes, do we really need any extra flag? I
> > >> would
> > >> > > enable the capability if the best effort affinity is on. For me,
> > it's
> > >> > just
> > >> > > a natural improvement of the latter and it sounds reasonable to
> > reuse
> > >> the
> > >> > > best effort affinity's flag.
> > >> > >
> > >> > > -
> > >> > > Denis
> > >> > >
> > >> > >
> > >> > > On Mon, Apr 27, 2020 at 2:58 AM Pavel Tupitsyn <
> > [hidden email]>
> > >> > > wrote:
> > >> > >
> > >> > > > Igniters,
> > >> > > >
> > >> > > > I've prepared an IEP [1] and a POC [2] for Thin Client Discovery
> > >> > feature.
> > >> > > > Let's discuss it here.
> > >> > > >
> > >> > > > In particular, I'd like to address the following points:
> > >> > > >
> > >> > > > 1. Value: do you think this would be a good feature to have?
> > >> > > > 2. Public API changes: is a boolean property enough? Should we
> > have
> > >> > > > something more complex, so users can plug in custom logic to
> > filter
> > >> > > and/or
> > >> > > > translate IPs and host names?
> > >> > > > 3. Server-side implementation details: should we use Compute,
> Node
> > >> > > > Attributes, or something else to retrieve client endpoints from
> > all
> > >> > nodes
> > >> > > > in cluster?
> > >> > > >
> > >> > > > [1]
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-44%3A+Thin+client+cluster+discovery
> > >> > > > [2] https://github.com/apache/ignite/pull/7744
> > >> > > > [3] https://issues.apache.org/jira/browse/IGNITE-12932
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> >
>

[Pavel Tupitsyn]

Alex,

Event subscription is a good idea. We should certainly add this in future.
However, it might be non-trivial with multiple connections: should we
subscribe on every server?
Then we'll get an event from every server, which seems redundant.

Igor,

Agreed. Let's keep existing behavior.

On Thu, May 7, 2020 at 5:29 PM Igor Sapego <[hidden email]> wrote:

> Pavel,
>
> First of all, I think we need to move it out of scope of this feature.
>
> Also, why do we need to keep connection alive? I think, if user do not
> use connection for a long time and connection is lost we could just
> detect this and re-establish connection on the next user action which
> requires network interaction. Any issues with this approach?
>
> Best Regards,
> Igor
>
>
> On Thu, May 7, 2020 at 1:18 AM Alex Plehanov <[hidden email]>
> wrote:
>
> > Pavel,
> >
> > Since we have a notification mechanism for thin clients now, we can
> > implement a subscription to some types of events and this can be used
> > to inform a client about topology change as well. I think it's a
> > more appropriate way to detect topology changes than ping requests. But
> > approach with ping requests has another advantage: the client can detect
> > that connection was lost earlier. With subscriptions approach client will
> > detect problems with a connection only after the next request to the
> > server.
> >
> >
> > ср, 6 мая 2020 г. в 17:31, Pavel Tupitsyn <[hidden email]>:
> >
> > > Igniters, let's discuss the following issue:
> > >
> > > For partition awareness, and now for cluster discovery, we use a
> response
> > > flag to detect topology changes.
> > > The problem is - if the client does not do anything (user code does not
> > > perform operations),
> > > then we'll never know about topology changes and may even lose the
> > cluster
> > > (all known nodes leave).
> > >
> > > Should we introduce some keep-alive mechanism, so that thin clients
> send
> > > periodic ping requests?
> > > Maybe do this as a separate feature.
> > >
> > > Thoughts?
> > >
> > > On Tue, Apr 28, 2020 at 8:14 PM Pavel Tupitsyn <[hidden email]>
> > > wrote:
> > >
> > > > Ok, I've updated IEP and POC accordingly:
> > > > * Config flag removed
> > > > * IPs and host names retrieval simplified - use existing node
> > properties
> > > > and attributes instead of Compute
> > > >
> > > > On Tue, Apr 28, 2020 at 7:57 PM Igor Sapego <[hidden email]>
> > wrote:
> > > >
> > > >> I guess it makes sense. If anyone needs more control over connection
> > > >> we would need to implement a new feature anyway (like node filter we
> > > >> discussed earlier)
> > > >>
> > > >> Best Regards,
> > > >> Igor
> > > >>
> > > >>
> > > >> On Tue, Apr 28, 2020 at 12:29 PM Pavel Tupitsyn <
> [hidden email]
> > >
> > > >> wrote:
> > > >>
> > > >> > > enable the capability if the best effort affinity is on
> > > >> > I agree, makes sense.
> > > >> >
> > > >> > Igor, what do you think?
> > > >> >
> > > >> > On Tue, Apr 28, 2020 at 8:25 AM Denis Magda <[hidden email]>
> > > wrote:
> > > >> >
> > > >> > > Pavel,
> > > >> > >
> > > >> > > That would be a tremendous improvement for the recently release
> > best
> > > >> > effort
> > > >> > > affinity feature. Without this capability, we force application
> > > >> > developers
> > > >> > > to reopen thin client connections every type a cluster is scaled
> > > out.
> > > >> I
> > > >> > > believe that once the folks start using the best effort
> affinity,
> > > >> we'll
> > > >> > be
> > > >> > > hearing more of a feature request for what you're proposing in
> > this
> > > >> > thread.
> > > >> > > So, thanks for taking care of this proactively!
> > > >> > >
> > > >> > > As for the public API changes, do we really need any extra
> flag? I
> > > >> would
> > > >> > > enable the capability if the best effort affinity is on. For me,
> > > it's
> > > >> > just
> > > >> > > a natural improvement of the latter and it sounds reasonable to
> > > reuse
> > > >> the
> > > >> > > best effort affinity's flag.
> > > >> > >
> > > >> > > -
> > > >> > > Denis
> > > >> > >
> > > >> > >
> > > >> > > On Mon, Apr 27, 2020 at 2:58 AM Pavel Tupitsyn <
> > > [hidden email]>
> > > >> > > wrote:
> > > >> > >
> > > >> > > > Igniters,
> > > >> > > >
> > > >> > > > I've prepared an IEP [1] and a POC [2] for Thin Client
> Discovery
> > > >> > feature.
> > > >> > > > Let's discuss it here.
> > > >> > > >
> > > >> > > > In particular, I'd like to address the following points:
> > > >> > > >
> > > >> > > > 1. Value: do you think this would be a good feature to have?
> > > >> > > > 2. Public API changes: is a boolean property enough? Should we
> > > have
> > > >> > > > something more complex, so users can plug in custom logic to
> > > filter
> > > >> > > and/or
> > > >> > > > translate IPs and host names?
> > > >> > > > 3. Server-side implementation details: should we use Compute,
> > Node
> > > >> > > > Attributes, or something else to retrieve client endpoints
> from
> > > all
> > > >> > nodes
> > > >> > > > in cluster?
> > > >> > > >
> > > >> > > > [1]
> > > >> > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-44%3A+Thin+client+cluster+discovery
> > > >> > > > [2] https://github.com/apache/ignite/pull/7744
> > > >> > > > [3] https://issues.apache.org/jira/browse/IGNITE-12932
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > >
> >
>

[Pavel Tupitsyn]

Igniters, the feature is ready for review:
https://issues.apache.org/jira/browse/IGNITE-12932
https://github.com/apache/ignite/pull/7744

On Thu, May 7, 2020 at 6:23 PM Pavel Tupitsyn <[hidden email]> wrote:

> Alex,
>
> Event subscription is a good idea. We should certainly add this in future.
> However, it might be non-trivial with multiple connections: should we
> subscribe on every server?
> Then we'll get an event from every server, which seems redundant.
>
> Igor,
>
> Agreed. Let's keep existing behavior.
>
> On Thu, May 7, 2020 at 5:29 PM Igor Sapego <[hidden email]> wrote:
>
>> Pavel,
>>
>> First of all, I think we need to move it out of scope of this feature.
>>
>> Also, why do we need to keep connection alive? I think, if user do not
>> use connection for a long time and connection is lost we could just
>> detect this and re-establish connection on the next user action which
>> requires network interaction. Any issues with this approach?
>>
>> Best Regards,
>> Igor
>>
>>
>> On Thu, May 7, 2020 at 1:18 AM Alex Plehanov <[hidden email]>
>> wrote:
>>
>> > Pavel,
>> >
>> > Since we have a notification mechanism for thin clients now, we can
>> > implement a subscription to some types of events and this can be used
>> > to inform a client about topology change as well. I think it's a
>> > more appropriate way to detect topology changes than ping requests. But
>> > approach with ping requests has another advantage: the client can detect
>> > that connection was lost earlier. With subscriptions approach client
>> will
>> > detect problems with a connection only after the next request to the
>> > server.
>> >
>> >
>> > ср, 6 мая 2020 г. в 17:31, Pavel Tupitsyn <[hidden email]>:
>> >
>> > > Igniters, let's discuss the following issue:
>> > >
>> > > For partition awareness, and now for cluster discovery, we use a
>> response
>> > > flag to detect topology changes.
>> > > The problem is - if the client does not do anything (user code does
>> not
>> > > perform operations),
>> > > then we'll never know about topology changes and may even lose the
>> > cluster
>> > > (all known nodes leave).
>> > >
>> > > Should we introduce some keep-alive mechanism, so that thin clients
>> send
>> > > periodic ping requests?
>> > > Maybe do this as a separate feature.
>> > >
>> > > Thoughts?
>> > >
>> > > On Tue, Apr 28, 2020 at 8:14 PM Pavel Tupitsyn <[hidden email]>
>> > > wrote:
>> > >
>> > > > Ok, I've updated IEP and POC accordingly:
>> > > > * Config flag removed
>> > > > * IPs and host names retrieval simplified - use existing node
>> > properties
>> > > > and attributes instead of Compute
>> > > >
>> > > > On Tue, Apr 28, 2020 at 7:57 PM Igor Sapego <[hidden email]>
>> > wrote:
>> > > >
>> > > >> I guess it makes sense. If anyone needs more control over
>> connection
>> > > >> we would need to implement a new feature anyway (like node filter
>> we
>> > > >> discussed earlier)
>> > > >>
>> > > >> Best Regards,
>> > > >> Igor
>> > > >>
>> > > >>
>> > > >> On Tue, Apr 28, 2020 at 12:29 PM Pavel Tupitsyn <
>> [hidden email]
>> > >
>> > > >> wrote:
>> > > >>
>> > > >> > > enable the capability if the best effort affinity is on
>> > > >> > I agree, makes sense.
>> > > >> >
>> > > >> > Igor, what do you think?
>> > > >> >
>> > > >> > On Tue, Apr 28, 2020 at 8:25 AM Denis Magda <[hidden email]>
>> > > wrote:
>> > > >> >
>> > > >> > > Pavel,
>> > > >> > >
>> > > >> > > That would be a tremendous improvement for the recently release
>> > best
>> > > >> > effort
>> > > >> > > affinity feature. Without this capability, we force application
>> > > >> > developers
>> > > >> > > to reopen thin client connections every type a cluster is
>> scaled
>> > > out.
>> > > >> I
>> > > >> > > believe that once the folks start using the best effort
>> affinity,
>> > > >> we'll
>> > > >> > be
>> > > >> > > hearing more of a feature request for what you're proposing in
>> > this
>> > > >> > thread.
>> > > >> > > So, thanks for taking care of this proactively!
>> > > >> > >
>> > > >> > > As for the public API changes, do we really need any extra
>> flag? I
>> > > >> would
>> > > >> > > enable the capability if the best effort affinity is on. For
>> me,
>> > > it's
>> > > >> > just
>> > > >> > > a natural improvement of the latter and it sounds reasonable to
>> > > reuse
>> > > >> the
>> > > >> > > best effort affinity's flag.
>> > > >> > >
>> > > >> > > -
>> > > >> > > Denis
>> > > >> > >
>> > > >> > >
>> > > >> > > On Mon, Apr 27, 2020 at 2:58 AM Pavel Tupitsyn <
>> > > [hidden email]>
>> > > >> > > wrote:
>> > > >> > >
>> > > >> > > > Igniters,
>> > > >> > > >
>> > > >> > > > I've prepared an IEP [1] and a POC [2] for Thin Client
>> Discovery
>> > > >> > feature.
>> > > >> > > > Let's discuss it here.
>> > > >> > > >
>> > > >> > > > In particular, I'd like to address the following points:
>> > > >> > > >
>> > > >> > > > 1. Value: do you think this would be a good feature to have?
>> > > >> > > > 2. Public API changes: is a boolean property enough? Should
>> we
>> > > have
>> > > >> > > > something more complex, so users can plug in custom logic to
>> > > filter
>> > > >> > > and/or
>> > > >> > > > translate IPs and host names?
>> > > >> > > > 3. Server-side implementation details: should we use Compute,
>> > Node
>> > > >> > > > Attributes, or something else to retrieve client endpoints
>> from
>> > > all
>> > > >> > nodes
>> > > >> > > > in cluster?
>> > > >> > > >
>> > > >> > > > [1]
>> > > >> > > >
>> > > >> > > >
>> > > >> > >
>> > > >> >
>> > > >>
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-44%3A+Thin+client+cluster+discovery
>> > > >> > > > [2] https://github.com/apache/ignite/pull/7744
>> > > >> > > > [3] https://issues.apache.org/jira/browse/IGNITE-12932
>> > > >> > > >
>> > > >> > >
>> > > >> >
>> > > >>
>> > > >
>> > >
>> >
>>
>