Binary clients: fallback to the previous versions of the protocol

[Dmitry Melnichuk]

Hello fellow igniters!

I recently started reconnaissance before the
implementation of IEP-23 [1] in Python thin client. I think it is an
interesting and much promising feature. As I understand, the changes
associated with this feature will result in a new version of the binary
protocol.

What bothers me is not the changes required by the
implementation itself, but the upcoming changes in the way that the
fallback to the previous versions of the protocol is going to be
handled.

I have not found a open discussion on this topic neither on
this mailing list, nor in Jira. If such a discussion took place, please
inform me with a link. Based on what I have heard so far, I made the
following conclusions:

1) the client is now expected to handle multiple
binary protocol versions;
2) since the protocol is implemented the way
the client first reports its version to the server on handshake, and
not the other way, the client now must try to connect to the server
using its preferred version of the protocol, and in case of an error,
reconnect with another version.

These changes are raised some concerns
for me, for example:

1) the version negotiation mechanism implemented on
client can not be made transparent to the end user. We must give the
user an option, so that they could choose from the entire set of
versions supported by the client, a subset of the versions their
application is designed to work with. That will complicate the user
API;

2) the complexity of the client will also be increased, especially
considering unit testing. Certain tests will require a certain version
of Ignite server, so they can no longer be implemented environment-
agnostically, leading to a weird mix on unit and integration levels;

3)
I do not see, or not aware of, the preconditions to such changes. I
think the number of potential applications that can benefit from multi-
protocol client is quite low. Usually all the hassle of version
matching lie on the deployment level, above the app level. Please prove
me wrong on this subject.

Instead of supporting multiple binary
protocols inside one client package, I would strongly prefer to support
each version of the protocol separately, in the corresponding branch of
the package repository. It would still be possible for the end user to
use multiple protocols in one app, but with certain code-level and
deployment-level efforts combined.

Again, I most probably lack awareness
on the subject, and therefore my conclusions may be premature, but
anyway, let us discuss. I will appreciate any bit of knowledge from the
community.

[1] https://cwiki.apache.org/confluence/display/IGNITE/IEP-
23:+Best+Effort+Affinity+for+thin+clients

[Igor Sapego-2]

Hi,

Well, this approach was used for quite some time in ODBC and JDBC,
so thin client has just inherited it, as we have the same port and the
same handshake message header. Maybe that's why you could not find
any mention of versioning in thin client context.

3. Let's start from this point as this is the root of the issue.
Preconditions
are simple - the backward compatibility. We should guarantee that user
can use any version of thin client with any version of server while they
share the same major version. This can be important, if someone have
user applications built with thin client and they don't want to break their
user applications, when they just want to update servers. What is
appropriate here is to give user some kind of warning that they use
outdated client or server.

Can you explain how your approach is going to work in the example
above? I'm not quite sure I understand you correctly.

1. The API for this can be as simple as a set of allowed version in
a configuration for both server and client. Maybe we should discuss such
kind of API as it can be useful for example in terms of security.

2. Complexity is the price developers have to pay for better user
experience.
Nothing new here. Software is not designed for developers, it's designed
for a user.

Best Regards,
Igor


On Tue, Feb 12, 2019 at 11:56 AM Dmitry Melnichuk <
[hidden email]> wrote:

> Hello fellow igniters!
>
> I recently started reconnaissance before the
> implementation of IEP-23 [1] in Python thin client. I think it is an
> interesting and much promising feature. As I understand, the changes
> associated with this feature will result in a new version of the binary
> protocol.
>
> What bothers me is not the changes required by the
> implementation itself, but the upcoming changes in the way that the
> fallback to the previous versions of the protocol is going to be
> handled.
>
> I have not found a open discussion on this topic neither on
> this mailing list, nor in Jira. If such a discussion took place, please
> inform me with a link. Based on what I have heard so far, I made the
> following conclusions:
>
> 1) the client is now expected to handle multiple
> binary protocol versions;
> 2) since the protocol is implemented the way
> the client first reports its version to the server on handshake, and
> not the other way, the client now must try to connect to the server
> using its preferred version of the protocol, and in case of an error,
> reconnect with another version.
>
> These changes are raised some concerns
> for me, for example:
>
> 1) the version negotiation mechanism implemented on
> client can not be made transparent to the end user. We must give the
> user an option, so that they could choose from the entire set of
> versions supported by the client, a subset of the versions their
> application is designed to work with. That will complicate the user
> API;
>
> 2) the complexity of the client will also be increased, especially
> considering unit testing. Certain tests will require a certain version
> of Ignite server, so they can no longer be implemented environment-
> agnostically, leading to a weird mix on unit and integration levels;
>
> 3)
> I do not see, or not aware of, the preconditions to such changes. I
> think the number of potential applications that can benefit from multi-
> protocol client is quite low. Usually all the hassle of version
> matching lie on the deployment level, above the app level. Please prove
> me wrong on this subject.
>
> Instead of supporting multiple binary
> protocols inside one client package, I would strongly prefer to support
> each version of the protocol separately, in the corresponding branch of
> the package repository. It would still be possible for the end user to
> use multiple protocols in one app, but with certain code-level and
> deployment-level efforts combined.
>
> Again, I most probably lack awareness
> on the subject, and therefore my conclusions may be premature, but
> anyway, let us discuss. I will appreciate any bit of knowledge from the
> community.
>
> [1] https://cwiki.apache.org/confluence/display/IGNITE/IEP-
> 23:+Best+Effort+Affinity+for+thin+clients
>
>

[Dmitry Melnichuk]

Igor,

Thank you for your explanation. I think the matter begins to clear up
for me now.

The backward compatibility issue you described can not be applied to
Python, because Python applications, unlike Java ones, do not have to
be built. They rely on package manager (pip, conda, et c.) to run
anywhere, including production.

At the stage of deployment, the package manager collects dependencies
using a specially crafted response file, often called
`requirements.txt`.

For example, to ensure that their application will work with the
current _and_ future minor versions of pyignite, the user may include a
line in their `requirements.txt` file:

pyignite < x

where x is a next major version number. In compliance with semantic
versioning, the line is basically says: “Use the latest available
version, that is earlier than x”.

When upgrading Ignite server, system administrator or devops engineer
must also update or recreate the app's environment, or update OS-level
packages, or redeploy the app using Docker − the exact procedure may
vary, but in any case it should be completely standard − to deliver the
latest suitable dependencies.

And then the same app connects to a latest Ignite server.

Here is more about how pip understands versions:

https://pip.pypa.io/en/stable/reference/pip_install/#requirement-specifiers

What we really need to do for this to work seamlessly, is to establish
the clear relation between products' versions. Regretfully, I have not
done this before; just did not expect for this issue to come up. I
think it would be best for pyignite major and minor to be set according
to the Ignite binary protocol versions, i.e. pyignite 1.2.z handles
Ignite binary protocol v1.2, and so on. But that is another matter.

On Tue, 2019-02-12 at 13:39 +0300, Igor Sapego wrote:

> Hi,
>
> Well, this approach was used for quite some time in ODBC and JDBC,
> so thin client has just inherited it, as we have the same port and
> the
> same handshake message header. Maybe that's why you could not find
> any mention of versioning in thin client context.
>
> 3. Let's start from this point as this is the root of the issue.
> Preconditions
> are simple - the backward compatibility. We should guarantee that
> user
> can use any version of thin client with any version of server while
> they
> share the same major version. This can be important, if someone have
> user applications built with thin client and they don't want to break
> their
> user applications, when they just want to update servers. What is
> appropriate here is to give user some kind of warning that they use
> outdated client or server.
>
> Can you explain how your approach is going to work in the example
> above? I'm not quite sure I understand you correctly.
>
> 1. The API for this can be as simple as a set of allowed version in
> a configuration for both server and client. Maybe we should discuss
> such
> kind of API as it can be useful for example in terms of security.
>
> 2. Complexity is the price developers have to pay for better user
> experience.
> Nothing new here. Software is not designed for developers, it's
> designed
> for a user.
>
> Best Regards,
> Igor
>
>
> On Tue, Feb 12, 2019 at 11:56 AM Dmitry Melnichuk <
> [hidden email]> wrote:
>
> > Hello fellow igniters!
> >
> > I recently started reconnaissance before the
> > implementation of IEP-23 [1] in Python thin client. I think it is
> > an
> > interesting and much promising feature. As I understand, the
> > changes
> > associated with this feature will result in a new version of the
> > binary
> > protocol.
> >
> > What bothers me is not the changes required by the
> > implementation itself, but the upcoming changes in the way that the
> > fallback to the previous versions of the protocol is going to be
> > handled.
> >
> > I have not found a open discussion on this topic neither on
> > this mailing list, nor in Jira. If such a discussion took place,
> > please
> > inform me with a link. Based on what I have heard so far, I made
> > the
> > following conclusions:
> >
> > 1) the client is now expected to handle multiple
> > binary protocol versions;
> > 2) since the protocol is implemented the way
> > the client first reports its version to the server on handshake,
> > and
> > not the other way, the client now must try to connect to the server
> > using its preferred version of the protocol, and in case of an
> > error,
> > reconnect with another version.
> >
> > These changes are raised some concerns
> > for me, for example:
> >
> > 1) the version negotiation mechanism implemented on
> > client can not be made transparent to the end user. We must give
> > the
> > user an option, so that they could choose from the entire set of
> > versions supported by the client, a subset of the versions their
> > application is designed to work with. That will complicate the user
> > API;
> >
> > 2) the complexity of the client will also be increased, especially
> > considering unit testing. Certain tests will require a certain
> > version
> > of Ignite server, so they can no longer be implemented environment-
> > agnostically, leading to a weird mix on unit and integration
> > levels;
> >
> > 3)
> > I do not see, or not aware of, the preconditions to such changes. I
> > think the number of potential applications that can benefit from
> > multi-
> > protocol client is quite low. Usually all the hassle of version
> > matching lie on the deployment level, above the app level. Please
> > prove
> > me wrong on this subject.
> >
> > Instead of supporting multiple binary
> > protocols inside one client package, I would strongly prefer to
> > support
> > each version of the protocol separately, in the corresponding
> > branch of
> > the package repository. It would still be possible for the end user
> > to
> > use multiple protocols in one app, but with certain code-level and
> > deployment-level efforts combined.
> >
> > Again, I most probably lack awareness
> > on the subject, and therefore my conclusions may be premature, but
> > anyway, let us discuss. I will appreciate any bit of knowledge from
> > the
> > community.
> >
> > [1] https://cwiki.apache.org/confluence/display/IGNITE/IEP-
> > 23:+Best+Effort+Affinity+for+thin+clients
> >
> >

[Igor Sapego-2]

The approach you suggest looks to me pretty much the same as installing
a new version of client software in C++ or Java. The issue here that we
break
existing installed software and require for user to update software in order
to have ability to connect to a server. Just imagine that application which
made
with thin client is not used by a developer that knows how to use pip and
all the
stuff, but someone with another background. Imagine, you have thousands of
such users. And now imagine, you want to update your servers.

Best Regards,
Igor


On Tue, Feb 12, 2019 at 8:51 PM Dmitry Melnichuk <
[hidden email]> wrote:

> Igor,
>
> Thank you for your explanation. I think the matter begins to clear up
> for me now.
>
> The backward compatibility issue you described can not be applied to
> Python, because Python applications, unlike Java ones, do not have to
> be built. They rely on package manager (pip, conda, et c.) to run
> anywhere, including production.
>
> At the stage of deployment, the package manager collects dependencies
> using a specially crafted response file, often called
> `requirements.txt`.
>
> For example, to ensure that their application will work with the
> current _and_ future minor versions of pyignite, the user may include a
> line in their `requirements.txt` file:
>
> pyignite < x
>
> where x is a next major version number. In compliance with semantic
> versioning, the line is basically says: “Use the latest available
> version, that is earlier than x”.
>
> When upgrading Ignite server, system administrator or devops engineer
> must also update or recreate the app's environment, or update OS-level
> packages, or redeploy the app using Docker − the exact procedure may
> vary, but in any case it should be completely standard − to deliver the
> latest suitable dependencies.
>
> And then the same app connects to a latest Ignite server.
>
> Here is more about how pip understands versions:
>
> https://pip.pypa.io/en/stable/reference/pip_install/#requirement-specifiers
>
> What we really need to do for this to work seamlessly, is to establish
> the clear relation between products' versions. Regretfully, I have not
> done this before; just did not expect for this issue to come up. I
> think it would be best for pyignite major and minor to be set according
> to the Ignite binary protocol versions, i.e. pyignite 1.2.z handles
> Ignite binary protocol v1.2, and so on. But that is another matter.
>
> On Tue, 2019-02-12 at 13:39 +0300, Igor Sapego wrote:
> > Hi,
> >
> > Well, this approach was used for quite some time in ODBC and JDBC,
> > so thin client has just inherited it, as we have the same port and
> > the
> > same handshake message header. Maybe that's why you could not find
> > any mention of versioning in thin client context.
> >
> > 3. Let's start from this point as this is the root of the issue.
> > Preconditions
> > are simple - the backward compatibility. We should guarantee that
> > user
> > can use any version of thin client with any version of server while
> > they
> > share the same major version. This can be important, if someone have
> > user applications built with thin client and they don't want to break
> > their
> > user applications, when they just want to update servers. What is
> > appropriate here is to give user some kind of warning that they use
> > outdated client or server.
> >
> > Can you explain how your approach is going to work in the example
> > above? I'm not quite sure I understand you correctly.
> >
> > 1. The API for this can be as simple as a set of allowed version in
> > a configuration for both server and client. Maybe we should discuss
> > such
> > kind of API as it can be useful for example in terms of security.
> >
> > 2. Complexity is the price developers have to pay for better user
> > experience.
> > Nothing new here. Software is not designed for developers, it's
> > designed
> > for a user.
> >
> > Best Regards,
> > Igor
> >
> >
> > On Tue, Feb 12, 2019 at 11:56 AM Dmitry Melnichuk <
> > [hidden email]> wrote:
> >
> > > Hello fellow igniters!
> > >
> > > I recently started reconnaissance before the
> > > implementation of IEP-23 [1] in Python thin client. I think it is
> > > an
> > > interesting and much promising feature. As I understand, the
> > > changes
> > > associated with this feature will result in a new version of the
> > > binary
> > > protocol.
> > >
> > > What bothers me is not the changes required by the
> > > implementation itself, but the upcoming changes in the way that the
> > > fallback to the previous versions of the protocol is going to be
> > > handled.
> > >
> > > I have not found a open discussion on this topic neither on
> > > this mailing list, nor in Jira. If such a discussion took place,
> > > please
> > > inform me with a link. Based on what I have heard so far, I made
> > > the
> > > following conclusions:
> > >
> > > 1) the client is now expected to handle multiple
> > > binary protocol versions;
> > > 2) since the protocol is implemented the way
> > > the client first reports its version to the server on handshake,
> > > and
> > > not the other way, the client now must try to connect to the server
> > > using its preferred version of the protocol, and in case of an
> > > error,
> > > reconnect with another version.
> > >
> > > These changes are raised some concerns
> > > for me, for example:
> > >
> > > 1) the version negotiation mechanism implemented on
> > > client can not be made transparent to the end user. We must give
> > > the
> > > user an option, so that they could choose from the entire set of
> > > versions supported by the client, a subset of the versions their
> > > application is designed to work with. That will complicate the user
> > > API;
> > >
> > > 2) the complexity of the client will also be increased, especially
> > > considering unit testing. Certain tests will require a certain
> > > version
> > > of Ignite server, so they can no longer be implemented environment-
> > > agnostically, leading to a weird mix on unit and integration
> > > levels;
> > >
> > > 3)
> > > I do not see, or not aware of, the preconditions to such changes. I
> > > think the number of potential applications that can benefit from
> > > multi-
> > > protocol client is quite low. Usually all the hassle of version
> > > matching lie on the deployment level, above the app level. Please
> > > prove
> > > me wrong on this subject.
> > >
> > > Instead of supporting multiple binary
> > > protocols inside one client package, I would strongly prefer to
> > > support
> > > each version of the protocol separately, in the corresponding
> > > branch of
> > > the package repository. It would still be possible for the end user
> > > to
> > > use multiple protocols in one app, but with certain code-level and
> > > deployment-level efforts combined.
> > >
> > > Again, I most probably lack awareness
> > > on the subject, and therefore my conclusions may be premature, but
> > > anyway, let us discuss. I will appreciate any bit of knowledge from
> > > the
> > > community.
> > >
> > > [1] https://cwiki.apache.org/confluence/display/IGNITE/IEP-
> > > 23:+Best+Effort+Affinity+for+thin+clients
> > >
> > >
>
>

[Dmitry Melnichuk]

Igor,

I am sorry it took me a while to fully understand your reasoning.

“Update user software first, then update the server” approach still
looks somewhat weird to me (I think of Let's Encrypt client as an
example of “normal” approach in Python world), but since this approach
is vivid, I just have to take it into account, so I must agree with
you.

I just want to reiterate on one downside of such multi-protocol client,
that was not yet addressed (not in Jira tasks or in docs, at least).

Imagine a coder wrote a program with the latest client, using a feature
available only in latest binary protocol. When the coder tests his
program against the latest Ignite cluster, the program works perfectly.

But then the end user runs the program against the previous version of
the server, which client is still backwards-compatible with, the
program runs, but at some point it tries to use the latest feature of
the binary protocol and fails with some cryptic message. The end user
is clueless, so as the coder.

To avoid such a case, we must include an explicit parameter in our
client's initialization method, that would set the desired protocol
version(s) the user application is designed to work with. This
parameter should be explicit, i.e. not have a default value, since it
just will be useless the other way. And yes, this parameter renders all
the software built with previous client versions incompatible with the
new client.

I think this problem concerns not only the Python client, but all the
thin clients. What do you think?

On Wed, 2019-02-13 at 13:45 +0300, Igor Sapego wrote:

> The approach you suggest looks to me pretty much the same as
> installing a new version of client software in C++ or Java. The issue
> here that we break existing installed software and require for user
> to update software in order to have ability to connect to a server.
> Just imagine that application which made with thin client is not used
> by a developer that knows how to use pip and all the stuff, but
> someone with another background. Imagine, you have thousands of such
> users. And now imagine, you want to update your servers.
>
> Best Regards,
> Igor
>
>
> On Tue, Feb 12, 2019 at 8:51 PM Dmitry Melnichuk <
> [hidden email]> wrote:
>
> > Igor,
> >
> > Thank you for your explanation. I think the matter begins to clear
> > up
> > for me now.
> >
> > The backward compatibility issue you described can not be applied
> > to
> > Python, because Python applications, unlike Java ones, do not have
> > to
> > be built. They rely on package manager (pip, conda, et c.) to run
> > anywhere, including production.
> >
> > At the stage of deployment, the package manager collects
> > dependencies
> > using a specially crafted response file, often called
> > `requirements.txt`.
> >
> > For example, to ensure that their application will work with the
> > current _and_ future minor versions of pyignite, the user may
> > include a
> > line in their `requirements.txt` file:
> >
> > pyignite < x
> >
> > where x is a next major version number. In compliance with semantic
> > versioning, the line is basically says: “Use the latest available
> > version, that is earlier than x”.
> >
> > When upgrading Ignite server, system administrator or devops
> > engineer
> > must also update or recreate the app's environment, or update OS-
> > level
> > packages, or redeploy the app using Docker − the exact procedure
> > may
> > vary, but in any case it should be completely standard − to deliver
> > the
> > latest suitable dependencies.
> >
> > And then the same app connects to a latest Ignite server.
> >
> > Here is more about how pip understands versions:
> >
> > https://pip.pypa.io/en/stable/reference/pip_install/#requirement-specifiers
> >
> > What we really need to do for this to work seamlessly, is to
> > establish
> > the clear relation between products' versions. Regretfully, I have
> > not
> > done this before; just did not expect for this issue to come up. I
> > think it would be best for pyignite major and minor to be set
> > according
> > to the Ignite binary protocol versions, i.e. pyignite 1.2.z handles
> > Ignite binary protocol v1.2, and so on. But that is another matter.
> >
> > On Tue, 2019-02-12 at 13:39 +0300, Igor Sapego wrote:
> > > Hi,
> > >
> > > Well, this approach was used for quite some time in ODBC and
> > > JDBC,
> > > so thin client has just inherited it, as we have the same port
> > > and
> > > the
> > > same handshake message header. Maybe that's why you could not
> > > find
> > > any mention of versioning in thin client context.
> > >
> > > 3. Let's start from this point as this is the root of the issue.
> > > Preconditions
> > > are simple - the backward compatibility. We should guarantee that
> > > user
> > > can use any version of thin client with any version of server
> > > while
> > > they
> > > share the same major version. This can be important, if someone
> > > have
> > > user applications built with thin client and they don't want to
> > > break
> > > their
> > > user applications, when they just want to update servers. What is
> > > appropriate here is to give user some kind of warning that they
> > > use
> > > outdated client or server.
> > >
> > > Can you explain how your approach is going to work in the example
> > > above? I'm not quite sure I understand you correctly.
> > >
> > > 1. The API for this can be as simple as a set of allowed version
> > > in
> > > a configuration for both server and client. Maybe we should
> > > discuss
> > > such
> > > kind of API as it can be useful for example in terms of security.
> > >
> > > 2. Complexity is the price developers have to pay for better user
> > > experience.
> > > Nothing new here. Software is not designed for developers, it's
> > > designed
> > > for a user.
> > >
> > > Best Regards,
> > > Igor
> > >
> > >
> > > On Tue, Feb 12, 2019 at 11:56 AM Dmitry Melnichuk <
> > > [hidden email]> wrote:
> > >
> > > > Hello fellow igniters!
> > > >
> > > > I recently started reconnaissance before the
> > > > implementation of IEP-23 [1] in Python thin client. I think it
> > > > is
> > > > an
> > > > interesting and much promising feature. As I understand, the
> > > > changes
> > > > associated with this feature will result in a new version of
> > > > the
> > > > binary
> > > > protocol.
> > > >
> > > > What bothers me is not the changes required by the
> > > > implementation itself, but the upcoming changes in the way that
> > > > the
> > > > fallback to the previous versions of the protocol is going to
> > > > be
> > > > handled.
> > > >
> > > > I have not found a open discussion on this topic neither on
> > > > this mailing list, nor in Jira. If such a discussion took
> > > > place,
> > > > please
> > > > inform me with a link. Based on what I have heard so far, I
> > > > made
> > > > the
> > > > following conclusions:
> > > >
> > > > 1) the client is now expected to handle multiple
> > > > binary protocol versions;
> > > > 2) since the protocol is implemented the way
> > > > the client first reports its version to the server on
> > > > handshake,
> > > > and
> > > > not the other way, the client now must try to connect to the
> > > > server
> > > > using its preferred version of the protocol, and in case of an
> > > > error,
> > > > reconnect with another version.
> > > >
> > > > These changes are raised some concerns
> > > > for me, for example:
> > > >
> > > > 1) the version negotiation mechanism implemented on
> > > > client can not be made transparent to the end user. We must
> > > > give
> > > > the
> > > > user an option, so that they could choose from the entire set
> > > > of
> > > > versions supported by the client, a subset of the versions
> > > > their
> > > > application is designed to work with. That will complicate the
> > > > user
> > > > API;
> > > >
> > > > 2) the complexity of the client will also be increased,
> > > > especially
> > > > considering unit testing. Certain tests will require a certain
> > > > version
> > > > of Ignite server, so they can no longer be implemented
> > > > environment-
> > > > agnostically, leading to a weird mix on unit and integration
> > > > levels;
> > > >
> > > > 3)
> > > > I do not see, or not aware of, the preconditions to such
> > > > changes. I
> > > > think the number of potential applications that can benefit
> > > > from
> > > > multi-
> > > > protocol client is quite low. Usually all the hassle of version
> > > > matching lie on the deployment level, above the app level.
> > > > Please
> > > > prove
> > > > me wrong on this subject.
> > > >
> > > > Instead of supporting multiple binary
> > > > protocols inside one client package, I would strongly prefer to
> > > > support
> > > > each version of the protocol separately, in the corresponding
> > > > branch of
> > > > the package repository. It would still be possible for the end
> > > > user
> > > > to
> > > > use multiple protocols in one app, but with certain code-level
> > > > and
> > > > deployment-level efforts combined.
> > > >
> > > > Again, I most probably lack awareness
> > > > on the subject, and therefore my conclusions may be premature,
> > > > but
> > > > anyway, let us discuss. I will appreciate any bit of knowledge
> > > > from
> > > > the
> > > > community.
> > > >
> > > > [1] https://cwiki.apache.org/confluence/display/IGNITE/IEP-
> > > > 23:+Best+Effort+Affinity+for+thin+clients
> > > >
> > > >

[Vladimir Ozerov]

Hi Dmitriy,

It is very common practice to keep client protocol compatible with multiple
versions of the server. We constantly face this in practice. I do not see
any reason to drop or complicate this functional: user just connects to the
server and we automatically negotiate on the best feature set possible. No
need to expose it somehow to users. As far as develpoment and testing, we
are not afraid of challenges and difficulties. Yes, it takes more time, but
it worth it.

Vladimir.

On Thu, Feb 14, 2019 at 6:28 AM Dmitry Melnichuk <
[hidden email]> wrote:

> Igor,
>
> I am sorry it took me a while to fully understand your reasoning.
>
> “Update user software first, then update the server” approach still
> looks somewhat weird to me (I think of Let's Encrypt client as an
> example of “normal” approach in Python world), but since this approach
> is vivid, I just have to take it into account, so I must agree with
> you.
>
> I just want to reiterate on one downside of such multi-protocol client,
> that was not yet addressed (not in Jira tasks or in docs, at least).
>
> Imagine a coder wrote a program with the latest client, using a feature
> available only in latest binary protocol. When the coder tests his
> program against the latest Ignite cluster, the program works perfectly.
>
> But then the end user runs the program against the previous version of
> the server, which client is still backwards-compatible with, the
> program runs, but at some point it tries to use the latest feature of
> the binary protocol and fails with some cryptic message. The end user
> is clueless, so as the coder.
>
> To avoid such a case, we must include an explicit parameter in our
> client's initialization method, that would set the desired protocol
> version(s) the user application is designed to work with. This
> parameter should be explicit, i.e. not have a default value, since it
> just will be useless the other way. And yes, this parameter renders all
> the software built with previous client versions incompatible with the
> new client.
>
> I think this problem concerns not only the Python client, but all the
> thin clients. What do you think?
>
> On Wed, 2019-02-13 at 13:45 +0300, Igor Sapego wrote:
> > The approach you suggest looks to me pretty much the same as
> > installing a new version of client software in C++ or Java. The issue
> > here that we break existing installed software and require for user
> > to update software in order to have ability to connect to a server.
> > Just imagine that application which made with thin client is not used
> > by a developer that knows how to use pip and all the stuff, but
> > someone with another background. Imagine, you have thousands of such
> > users. And now imagine, you want to update your servers.
> >
> > Best Regards,
> > Igor
> >
> >
> > On Tue, Feb 12, 2019 at 8:51 PM Dmitry Melnichuk <
> > [hidden email]> wrote:
> >
> > > Igor,
> > >
> > > Thank you for your explanation. I think the matter begins to clear
> > > up
> > > for me now.
> > >
> > > The backward compatibility issue you described can not be applied
> > > to
> > > Python, because Python applications, unlike Java ones, do not have
> > > to
> > > be built. They rely on package manager (pip, conda, et c.) to run
> > > anywhere, including production.
> > >
> > > At the stage of deployment, the package manager collects
> > > dependencies
> > > using a specially crafted response file, often called
> > > `requirements.txt`.
> > >
> > > For example, to ensure that their application will work with the
> > > current _and_ future minor versions of pyignite, the user may
> > > include a
> > > line in their `requirements.txt` file:
> > >
> > > pyignite < x
> > >
> > > where x is a next major version number. In compliance with semantic
> > > versioning, the line is basically says: “Use the latest available
> > > version, that is earlier than x”.
> > >
> > > When upgrading Ignite server, system administrator or devops
> > > engineer
> > > must also update or recreate the app's environment, or update OS-
> > > level
> > > packages, or redeploy the app using Docker − the exact procedure
> > > may
> > > vary, but in any case it should be completely standard − to deliver
> > > the
> > > latest suitable dependencies.
> > >
> > > And then the same app connects to a latest Ignite server.
> > >
> > > Here is more about how pip understands versions:
> > >
> > >
> https://pip.pypa.io/en/stable/reference/pip_install/#requirement-specifiers
> > >
> > > What we really need to do for this to work seamlessly, is to
> > > establish
> > > the clear relation between products' versions. Regretfully, I have
> > > not
> > > done this before; just did not expect for this issue to come up. I
> > > think it would be best for pyignite major and minor to be set
> > > according
> > > to the Ignite binary protocol versions, i.e. pyignite 1.2.z handles
> > > Ignite binary protocol v1.2, and so on. But that is another matter.
> > >
> > > On Tue, 2019-02-12 at 13:39 +0300, Igor Sapego wrote:
> > > > Hi,
> > > >
> > > > Well, this approach was used for quite some time in ODBC and
> > > > JDBC,
> > > > so thin client has just inherited it, as we have the same port
> > > > and
> > > > the
> > > > same handshake message header. Maybe that's why you could not
> > > > find
> > > > any mention of versioning in thin client context.
> > > >
> > > > 3. Let's start from this point as this is the root of the issue.
> > > > Preconditions
> > > > are simple - the backward compatibility. We should guarantee that
> > > > user
> > > > can use any version of thin client with any version of server
> > > > while
> > > > they
> > > > share the same major version. This can be important, if someone
> > > > have
> > > > user applications built with thin client and they don't want to
> > > > break
> > > > their
> > > > user applications, when they just want to update servers. What is
> > > > appropriate here is to give user some kind of warning that they
> > > > use
> > > > outdated client or server.
> > > >
> > > > Can you explain how your approach is going to work in the example
> > > > above? I'm not quite sure I understand you correctly.
> > > >
> > > > 1. The API for this can be as simple as a set of allowed version
> > > > in
> > > > a configuration for both server and client. Maybe we should
> > > > discuss
> > > > such
> > > > kind of API as it can be useful for example in terms of security.
> > > >
> > > > 2. Complexity is the price developers have to pay for better user
> > > > experience.
> > > > Nothing new here. Software is not designed for developers, it's
> > > > designed
> > > > for a user.
> > > >
> > > > Best Regards,
> > > > Igor
> > > >
> > > >
> > > > On Tue, Feb 12, 2019 at 11:56 AM Dmitry Melnichuk <
> > > > [hidden email]> wrote:
> > > >
> > > > > Hello fellow igniters!
> > > > >
> > > > > I recently started reconnaissance before the
> > > > > implementation of IEP-23 [1] in Python thin client. I think it
> > > > > is
> > > > > an
> > > > > interesting and much promising feature. As I understand, the
> > > > > changes
> > > > > associated with this feature will result in a new version of
> > > > > the
> > > > > binary
> > > > > protocol.
> > > > >
> > > > > What bothers me is not the changes required by the
> > > > > implementation itself, but the upcoming changes in the way that
> > > > > the
> > > > > fallback to the previous versions of the protocol is going to
> > > > > be
> > > > > handled.
> > > > >
> > > > > I have not found a open discussion on this topic neither on
> > > > > this mailing list, nor in Jira. If such a discussion took
> > > > > place,
> > > > > please
> > > > > inform me with a link. Based on what I have heard so far, I
> > > > > made
> > > > > the
> > > > > following conclusions:
> > > > >
> > > > > 1) the client is now expected to handle multiple
> > > > > binary protocol versions;
> > > > > 2) since the protocol is implemented the way
> > > > > the client first reports its version to the server on
> > > > > handshake,
> > > > > and
> > > > > not the other way, the client now must try to connect to the
> > > > > server
> > > > > using its preferred version of the protocol, and in case of an
> > > > > error,
> > > > > reconnect with another version.
> > > > >
> > > > > These changes are raised some concerns
> > > > > for me, for example:
> > > > >
> > > > > 1) the version negotiation mechanism implemented on
> > > > > client can not be made transparent to the end user. We must
> > > > > give
> > > > > the
> > > > > user an option, so that they could choose from the entire set
> > > > > of
> > > > > versions supported by the client, a subset of the versions
> > > > > their
> > > > > application is designed to work with. That will complicate the
> > > > > user
> > > > > API;
> > > > >
> > > > > 2) the complexity of the client will also be increased,
> > > > > especially
> > > > > considering unit testing. Certain tests will require a certain
> > > > > version
> > > > > of Ignite server, so they can no longer be implemented
> > > > > environment-
> > > > > agnostically, leading to a weird mix on unit and integration
> > > > > levels;
> > > > >
> > > > > 3)
> > > > > I do not see, or not aware of, the preconditions to such
> > > > > changes. I
> > > > > think the number of potential applications that can benefit
> > > > > from
> > > > > multi-
> > > > > protocol client is quite low. Usually all the hassle of version
> > > > > matching lie on the deployment level, above the app level.
> > > > > Please
> > > > > prove
> > > > > me wrong on this subject.
> > > > >
> > > > > Instead of supporting multiple binary
> > > > > protocols inside one client package, I would strongly prefer to
> > > > > support
> > > > > each version of the protocol separately, in the corresponding
> > > > > branch of
> > > > > the package repository. It would still be possible for the end
> > > > > user
> > > > > to
> > > > > use multiple protocols in one app, but with certain code-level
> > > > > and
> > > > > deployment-level efforts combined.
> > > > >
> > > > > Again, I most probably lack awareness
> > > > > on the subject, and therefore my conclusions may be premature,
> > > > > but
> > > > > anyway, let us discuss. I will appreciate any bit of knowledge
> > > > > from
> > > > > the
> > > > > community.
> > > > >
> > > > > [1] https://cwiki.apache.org/confluence/display/IGNITE/IEP-
> > > > > 23:+Best+Effort+Affinity+for+thin+clients
> > > > >
> > > > >
>
>