Thin client / Binary protocol: questions

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Thin client / Binary protocol: questions

Alexey Kosenchuk
Folks,

Designing node.js thin lib (IGNITE-7777) we have the following questions.
Could you please have a look and clarify.

Thanks,
-Alexey

- Supported standard types and their type codes.
   Are defined in the spec:
https://apacheignite.readme.io/docs/binary-client-protocol#data-format
   But, as we see, java and .net libs (common parts which are used by
thin clients as well) support additional types and type codes, not
described in the spec. Eg. defined here:
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/binary/GridBinaryMarshaller.java
   Just to double-check - we should support the types from the spec
only. Right?

- Key-Value and SQL and Scan Queries.
   The most of operations has "Flag" field in Request: "Pass 0 for
default, or 1 to keep the value in binary form."
   What is it for?
   Java and .net libs always pass 0. Why?

- OP_QUERY_SCAN: Filter platform.
   Why is it required?
   Why an app working remotely via the binary protocol should know the
exact platform where the node is running?

- Binary Type operations (register/get type name, put/get type).
   What are they for?
   What is a use case?
   It seems possible to put data of any binary object type into a cache
w/o registering type name and structure in advance.

- OP_REGISTER_BINARY_TYPE_NAME, OP_GET_BINARY_TYPE_NAME: Platform id.
   Is it a type of the node platform?
   Why a remote app should know it?
   Why non-identical values are used here and in OP_QUERY_SCAN: Filter
platform?

- OP_REGISTER_BINARY_TYPE_NAME, OP_PUT_BINARY_TYPE
   Why the both - Type name and Java-style hash code of the type name -
are in the request? (The same for Field name and Java-style hash code of
the field name.)
   Seems redundant.

- Just interesting - why it was decided that hash code should be also
calculated on a client side? Why it could not be returned by a server side?
   Eg. OP_CACHE_CREATE_WITH_NAME Response can return hash code for the
provided Cache name.
   That would simplify thin client implementations, guarantee
consistency/correctness of hash code calculations...

- Name/password authentication.
   When will it be available in the spec?
   Is there any draft we can look at?

- SSL/TLS communication.
   When will it be available in the spec?
   Is there any draft we can look at?

- As Denis said, we should implement a node failover algorithm
(switching to another predefined node).
   Should the algorithm be the same in all thin libs?
   Is it described somewhere?
   We have not found it in java and .net libs. Will it be supported by
them? When?

- Is there any recommendation/suggestion how to test a node failover
case on a thin client side?

- We are going to use jsdoc comments for the API methods:
http://usejsdoc.org/
   This tool, for example, may be used to generate the API spec from the
comments: https://www.npmjs.com/package/jsdoc
   Any objections?

- We are going to write the lib readme/guide in the markdown format and
place it at the repo (.md file).
   Is it OK? Or should we use dash.readme.io ?

- May we easily use 3rd-party components with the following licenses?
     -- Apache 2.0
     -- MIT
     -- GNU LGPL v3
     -- GNU GPL v2
   Should we pre-approve somehow / notify somebody about usage of
concrete 3rd-party components?
Reply | Threaded
Open this post in threaded view
|

Re: Thin client / Binary protocol: questions

Vladimir Ozerov
Hi Alex,

>>> Just to double-check - we should support the types from the spec only.
Right?
Please provide the list of types you are in doubt of

>>> - Key-Value and SQL and Scan Queries.
>>>  The most of operations has "Flag" field in Request: "Pass 0 for
default, or 1 to keep the value in binary form."
>>>  What is it for?
Please see IgniteCache.withKeepBinary method. For SQL this flag has no
effect. For complex SCAN queries with filters this defines whether with
pass real objects or binary objects to the filter.

>>>  Java and .net libs always pass 0. Why?
There is no Java client at the moment. As far as .NET, it doesn't support
binary mode at the moment.

>>>  Why an app working remotely via the binary protocol should know the
exact platform where the node is running?
Because if node is started from .NET, you may execute both Java filters and
.NET fitlers on it. This flag defines what kind of filter is passed.

>>> - Binary Type operations (register/get type name, put/get type).
>>>  What are they for?
>>>  What is a use case?
Please get familiar with binary type concepts, especially binary metadata:
https://apacheignite.readme.io/docs#binaryobject-type-metadata

>>>  - Just interesting - why it was decided that hash code should be also
calculated on a client side? Why it could not be returned by a server side?
Because this is inefficient - we would have to rebuild binary objects on
the server side. Instead, it is better to implement pretty simple routine
for hashcode calculation inside every client.

>>> - Name/password authentication.
>>> When will it be available in the spec?
>>> Is there any draft we can look at?
It is still under development. Will be available in several weeks. It is
better to skip this part for now.

>>> - SSL/TLS communication.
>>> When will it be available in the spec?
This is not part of protocol specification. Please refer to .NET client
where it is already implemented. *Pavel*, do we have docs for this?

>>> - As Denis said, we should implement a node failover algorithm
(switching to another predefined node).
>>> Should the algorithm be the same in all thin libs?
>>> Is it described somewhere?
>>> We have not found it in java and .net libs. Will it be supported by
them? When?
It is still under development, will be ready in 1-2 weeks. Please skip for
now. Yes, it is better to have the same algorithm on all clients for the
sake of consistency.

>>> - We are going to write the lib readme/guide in the markdown format and
place it at the repo (.md file).
>>> Is it OK? Or should we use dash.readme.io ?
All our documentation is hosted on readme.io. There should be no
exclusions. *Denis*, could you make sure that necessary spaces and
permissions are set up?

>>>  - May we easily use 3rd-party components with the following licenses?
As a general rule of thumb it is not recommended to use any external
libraries. The whole Ignite core is built with almost no dependencies, not
to say about much less complex thin clients ans JDBC/ODBC drivers. IMO the
only possible place where external dependency might be required is SSL
support (e.g. we use OpenSSL for ODBC).

Vladimir.

On Wed, Feb 28, 2018 at 9:41 AM, Alexey Kosenchuk <
[hidden email]> wrote:

> Folks,
>
> Designing node.js thin lib (IGNITE-7777) we have the following questions.
> Could you please have a look and clarify.
>
> Thanks,
> -Alexey
>
> - Supported standard types and their type codes.
>   Are defined in the spec: https://apacheignite.readme.io
> /docs/binary-client-protocol#data-format
>   But, as we see, java and .net libs (common parts which are used by thin
> clients as well) support additional types and type codes, not described in
> the spec. Eg. defined here: https://github.com/apache/igni
> te/blob/master/modules/core/src/main/java/org/apache/ignite/
> internal/binary/GridBinaryMarshaller.java
>   Just to double-check - we should support the types from the spec only.
> Right?
>
> - Key-Value and SQL and Scan Queries.
>   The most of operations has "Flag" field in Request: "Pass 0 for default,
> or 1 to keep the value in binary form."
>   What is it for?
>   Java and .net libs always pass 0. Why?
>
> - OP_QUERY_SCAN: Filter platform.
>   Why is it required?
>   Why an app working remotely via the binary protocol should know the
> exact platform where the node is running?
>
> - Binary Type operations (register/get type name, put/get type).
>   What are they for?
>   What is a use case?
>   It seems possible to put data of any binary object type into a cache w/o
> registering type name and structure in advance.
>
> - OP_REGISTER_BINARY_TYPE_NAME, OP_GET_BINARY_TYPE_NAME: Platform id.
>   Is it a type of the node platform?
>   Why a remote app should know it?
>   Why non-identical values are used here and in OP_QUERY_SCAN: Filter
> platform?
>
> - OP_REGISTER_BINARY_TYPE_NAME, OP_PUT_BINARY_TYPE
>   Why the both - Type name and Java-style hash code of the type name - are
> in the request? (The same for Field name and Java-style hash code of the
> field name.)
>   Seems redundant.
>
> - Just interesting - why it was decided that hash code should be also
> calculated on a client side? Why it could not be returned by a server side?
>   Eg. OP_CACHE_CREATE_WITH_NAME Response can return hash code for the
> provided Cache name.
>   That would simplify thin client implementations, guarantee
> consistency/correctness of hash code calculations...
>
> - Name/password authentication.
>   When will it be available in the spec?
>   Is there any draft we can look at?
>
> - SSL/TLS communication.
>   When will it be available in the spec?
>   Is there any draft we can look at?
>
> - As Denis said, we should implement a node failover algorithm (switching
> to another predefined node).
>   Should the algorithm be the same in all thin libs?
>   Is it described somewhere?
>   We have not found it in java and .net libs. Will it be supported by
> them? When?
>
> - Is there any recommendation/suggestion how to test a node failover case
> on a thin client side?
>
> - We are going to use jsdoc comments for the API methods:
> http://usejsdoc.org/
>   This tool, for example, may be used to generate the API spec from the
> comments: https://www.npmjs.com/package/jsdoc
>   Any objections?
>
> - We are going to write the lib readme/guide in the markdown format and
> place it at the repo (.md file).
>   Is it OK? Or should we use dash.readme.io ?
>
> - May we easily use 3rd-party components with the following licenses?
>     -- Apache 2.0
>     -- MIT
>     -- GNU LGPL v3
>     -- GNU GPL v2
>   Should we pre-approve somehow / notify somebody about usage of concrete
> 3rd-party components?
>
Reply | Threaded
Open this post in threaded view
|

Re: Thin client / Binary protocol: questions

Pavel Tupitsyn
> This is not part of protocol specification. Please refer to .NET client
> where it is already implemented. *Pavel*, do we have docs for this?

I don't think we have docs for SLL.
These docs should be common for any type of client (ODBC/JDBC/...),
and this is mostly about configuring certificates on server.

Otherwise it is just standard SSL/TLS, not much to document.
For .NET I've used standard SslStream, most languages/frameworks have this
in one form or another.


> node failover algorithm
See the following:
* https://issues.apache.org/jira/browse/IGNITE-7282
*
http://apache-ignite-developers.2346864.n4.nabble.com/Thin-client-failover-mechanism-ODBC-JDBC-td26553.html


Thanks,
Pavel


On Wed, Feb 28, 2018 at 1:32 PM, Vladimir Ozerov <[hidden email]>
wrote:

> Hi Alex,
>
> >>> Just to double-check - we should support the types from the spec only.
> Right?
> Please provide the list of types you are in doubt of
>
> >>> - Key-Value and SQL and Scan Queries.
> >>>  The most of operations has "Flag" field in Request: "Pass 0 for
> default, or 1 to keep the value in binary form."
> >>>  What is it for?
> Please see IgniteCache.withKeepBinary method. For SQL this flag has no
> effect. For complex SCAN queries with filters this defines whether with
> pass real objects or binary objects to the filter.
>
> >>>  Java and .net libs always pass 0. Why?
> There is no Java client at the moment. As far as .NET, it doesn't support
> binary mode at the moment.
>
> >>>  Why an app working remotely via the binary protocol should know the
> exact platform where the node is running?
> Because if node is started from .NET, you may execute both Java filters and
> .NET fitlers on it. This flag defines what kind of filter is passed.
>
> >>> - Binary Type operations (register/get type name, put/get type).
> >>>  What are they for?
> >>>  What is a use case?
> Please get familiar with binary type concepts, especially binary metadata:
> https://apacheignite.readme.io/docs#binaryobject-type-metadata
>
> >>>  - Just interesting - why it was decided that hash code should be also
> calculated on a client side? Why it could not be returned by a server side?
> Because this is inefficient - we would have to rebuild binary objects on
> the server side. Instead, it is better to implement pretty simple routine
> for hashcode calculation inside every client.
>
> >>> - Name/password authentication.
> >>> When will it be available in the spec?
> >>> Is there any draft we can look at?
> It is still under development. Will be available in several weeks. It is
> better to skip this part for now.
>
> >>> - SSL/TLS communication.
> >>> When will it be available in the spec?
> This is not part of protocol specification. Please refer to .NET client
> where it is already implemented. *Pavel*, do we have docs for this?
>
> >>> - As Denis said, we should implement a node failover algorithm
> (switching to another predefined node).
> >>> Should the algorithm be the same in all thin libs?
> >>> Is it described somewhere?
> >>> We have not found it in java and .net libs. Will it be supported by
> them? When?
> It is still under development, will be ready in 1-2 weeks. Please skip for
> now. Yes, it is better to have the same algorithm on all clients for the
> sake of consistency.
>
> >>> - We are going to write the lib readme/guide in the markdown format and
> place it at the repo (.md file).
> >>> Is it OK? Or should we use dash.readme.io ?
> All our documentation is hosted on readme.io. There should be no
> exclusions. *Denis*, could you make sure that necessary spaces and
> permissions are set up?
>
> >>>  - May we easily use 3rd-party components with the following licenses?
> As a general rule of thumb it is not recommended to use any external
> libraries. The whole Ignite core is built with almost no dependencies, not
> to say about much less complex thin clients ans JDBC/ODBC drivers. IMO the
> only possible place where external dependency might be required is SSL
> support (e.g. we use OpenSSL for ODBC).
>
> Vladimir.
>
> On Wed, Feb 28, 2018 at 9:41 AM, Alexey Kosenchuk <
> [hidden email]> wrote:
>
> > Folks,
> >
> > Designing node.js thin lib (IGNITE-7777) we have the following questions.
> > Could you please have a look and clarify.
> >
> > Thanks,
> > -Alexey
> >
> > - Supported standard types and their type codes.
> >   Are defined in the spec: https://apacheignite.readme.io
> > /docs/binary-client-protocol#data-format
> >   But, as we see, java and .net libs (common parts which are used by thin
> > clients as well) support additional types and type codes, not described
> in
> > the spec. Eg. defined here: https://github.com/apache/igni
> > te/blob/master/modules/core/src/main/java/org/apache/ignite/
> > internal/binary/GridBinaryMarshaller.java
> >   Just to double-check - we should support the types from the spec only.
> > Right?
> >
> > - Key-Value and SQL and Scan Queries.
> >   The most of operations has "Flag" field in Request: "Pass 0 for
> default,
> > or 1 to keep the value in binary form."
> >   What is it for?
> >   Java and .net libs always pass 0. Why?
> >
> > - OP_QUERY_SCAN: Filter platform.
> >   Why is it required?
> >   Why an app working remotely via the binary protocol should know the
> > exact platform where the node is running?
> >
> > - Binary Type operations (register/get type name, put/get type).
> >   What are they for?
> >   What is a use case?
> >   It seems possible to put data of any binary object type into a cache
> w/o
> > registering type name and structure in advance.
> >
> > - OP_REGISTER_BINARY_TYPE_NAME, OP_GET_BINARY_TYPE_NAME: Platform id.
> >   Is it a type of the node platform?
> >   Why a remote app should know it?
> >   Why non-identical values are used here and in OP_QUERY_SCAN: Filter
> > platform?
> >
> > - OP_REGISTER_BINARY_TYPE_NAME, OP_PUT_BINARY_TYPE
> >   Why the both - Type name and Java-style hash code of the type name -
> are
> > in the request? (The same for Field name and Java-style hash code of the
> > field name.)
> >   Seems redundant.
> >
> > - Just interesting - why it was decided that hash code should be also
> > calculated on a client side? Why it could not be returned by a server
> side?
> >   Eg. OP_CACHE_CREATE_WITH_NAME Response can return hash code for the
> > provided Cache name.
> >   That would simplify thin client implementations, guarantee
> > consistency/correctness of hash code calculations...
> >
> > - Name/password authentication.
> >   When will it be available in the spec?
> >   Is there any draft we can look at?
> >
> > - SSL/TLS communication.
> >   When will it be available in the spec?
> >   Is there any draft we can look at?
> >
> > - As Denis said, we should implement a node failover algorithm (switching
> > to another predefined node).
> >   Should the algorithm be the same in all thin libs?
> >   Is it described somewhere?
> >   We have not found it in java and .net libs. Will it be supported by
> > them? When?
> >
> > - Is there any recommendation/suggestion how to test a node failover case
> > on a thin client side?
> >
> > - We are going to use jsdoc comments for the API methods:
> > http://usejsdoc.org/
> >   This tool, for example, may be used to generate the API spec from the
> > comments: https://www.npmjs.com/package/jsdoc
> >   Any objections?
> >
> > - We are going to write the lib readme/guide in the markdown format and
> > place it at the repo (.md file).
> >   Is it OK? Or should we use dash.readme.io ?
> >
> > - May we easily use 3rd-party components with the following licenses?
> >     -- Apache 2.0
> >     -- MIT
> >     -- GNU LGPL v3
> >     -- GNU GPL v2
> >   Should we pre-approve somehow / notify somebody about usage of concrete
> > 3rd-party components?
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Thin client / Binary protocol: questions

dmagda
In reply to this post by Vladimir Ozerov
Guys, please consider my five cents

>>> - Name/password authentication.
> >>> When will it be available in the spec?
> >>> Is there any draft we can look at?
> It is still under development. Will be available in several weeks. It is
> better to skip this part for now.


Let's give a reference to the branch where the guys can see those changes
at the protocol level. My guess they want to know what to expect and how it
can affect the design they've been working on. *Alexey Kukushkin*, please
point out to your Java thin client branch were you already have this
functionality embedded.

>>> - SSL/TLS communication.
> >>> When will it be available in the spec?


This doc explains how SSL is enabled on the server side:
https://apacheignite.readme.io/docs/ssltls

As always, the thin client needs to establish the SSL tunneling first and
then starts sending the protocol messages. Hope this is good for the
beginning. Will be happy to answer more specific questions.

>>>  - May we easily use 3rd-party components with the following licenses?
> As a general rule of thumb it is not recommended to use any external
> libraries. The whole Ignite core is built with almost no dependencies, not
> to say about much less complex thin clients ans JDBC/ODBC drivers. IMO the
> only possible place where external dependency might be required is SSL
> support (e.g. we use OpenSSL for ODBC).


I wouldn't discourage Alexey from using 3rd party libs taking Ignite core
as an example. Ignite optional libs use 3rd party libs a lot. Take our REST
protocol for instance that uses Jetty and not based on its own HTTP server
implementation. So, I would alter the rule of thumb a little bit - if it
takes weeks to develop a functionality already available in a 3rd party lib
then let's go for the 3rd party.

Here is a list of licenses that can be included in Ignite without any extra
permissions or licensing concerns:
https://www.apache.org/legal/resolved.html

According to the doc you're free to use Apache 2.0 and MIT, but LGPL code
can't be delivered within Ignite.

--
Denis

On Wed, Feb 28, 2018 at 2:32 AM, Vladimir Ozerov <[hidden email]>
wrote:

> Hi Alex,
>
> >>> Just to double-check - we should support the types from the spec only.
> Right?
> Please provide the list of types you are in doubt of
>
> >>> - Key-Value and SQL and Scan Queries.
> >>>  The most of operations has "Flag" field in Request: "Pass 0 for
> default, or 1 to keep the value in binary form."
> >>>  What is it for?
> Please see IgniteCache.withKeepBinary method. For SQL this flag has no
> effect. For complex SCAN queries with filters this defines whether with
> pass real objects or binary objects to the filter.
>
> >>>  Java and .net libs always pass 0. Why?
> There is no Java client at the moment. As far as .NET, it doesn't support
> binary mode at the moment.
>
> >>>  Why an app working remotely via the binary protocol should know the
> exact platform where the node is running?
> Because if node is started from .NET, you may execute both Java filters and
> .NET fitlers on it. This flag defines what kind of filter is passed.
>
> >>> - Binary Type operations (register/get type name, put/get type).
> >>>  What are they for?
> >>>  What is a use case?
> Please get familiar with binary type concepts, especially binary metadata:
> https://apacheignite.readme.io/docs#binaryobject-type-metadata
>
> >>>  - Just interesting - why it was decided that hash code should be also
> calculated on a client side? Why it could not be returned by a server side?
> Because this is inefficient - we would have to rebuild binary objects on
> the server side. Instead, it is better to implement pretty simple routine
> for hashcode calculation inside every client.
>
> >>> - Name/password authentication.
> >>> When will it be available in the spec?
> >>> Is there any draft we can look at?
> It is still under development. Will be available in several weeks. It is
> better to skip this part for now.
>
> >>> - SSL/TLS communication.
> >>> When will it be available in the spec?
> This is not part of protocol specification. Please refer to .NET client
> where it is already implemented. *Pavel*, do we have docs for this?
>
> >>> - As Denis said, we should implement a node failover algorithm
> (switching to another predefined node).
> >>> Should the algorithm be the same in all thin libs?
> >>> Is it described somewhere?
> >>> We have not found it in java and .net libs. Will it be supported by
> them? When?
> It is still under development, will be ready in 1-2 weeks. Please skip for
> now. Yes, it is better to have the same algorithm on all clients for the
> sake of consistency.
>
> >>> - We are going to write the lib readme/guide in the markdown format and
> place it at the repo (.md file).
> >>> Is it OK? Or should we use dash.readme.io ?
> All our documentation is hosted on readme.io. There should be no
> exclusions. *Denis*, could you make sure that necessary spaces and
> permissions are set up?
>
> >>>  - May we easily use 3rd-party components with the following licenses?
> As a general rule of thumb it is not recommended to use any external
> libraries. The whole Ignite core is built with almost no dependencies, not
> to say about much less complex thin clients ans JDBC/ODBC drivers. IMO the
> only possible place where external dependency might be required is SSL
> support (e.g. we use OpenSSL for ODBC).
>
> Vladimir.
>
> On Wed, Feb 28, 2018 at 9:41 AM, Alexey Kosenchuk <
> [hidden email]> wrote:
>
> > Folks,
> >
> > Designing node.js thin lib (IGNITE-7777) we have the following questions.
> > Could you please have a look and clarify.
> >
> > Thanks,
> > -Alexey
> >
> > - Supported standard types and their type codes.
> >   Are defined in the spec: https://apacheignite.readme.io
> > /docs/binary-client-protocol#data-format
> >   But, as we see, java and .net libs (common parts which are used by thin
> > clients as well) support additional types and type codes, not described
> in
> > the spec. Eg. defined here: https://github.com/apache/igni
> > te/blob/master/modules/core/src/main/java/org/apache/ignite/
> > internal/binary/GridBinaryMarshaller.java
> >   Just to double-check - we should support the types from the spec only.
> > Right?
> >
> > - Key-Value and SQL and Scan Queries.
> >   The most of operations has "Flag" field in Request: "Pass 0 for
> default,
> > or 1 to keep the value in binary form."
> >   What is it for?
> >   Java and .net libs always pass 0. Why?
> >
> > - OP_QUERY_SCAN: Filter platform.
> >   Why is it required?
> >   Why an app working remotely via the binary protocol should know the
> > exact platform where the node is running?
> >
> > - Binary Type operations (register/get type name, put/get type).
> >   What are they for?
> >   What is a use case?
> >   It seems possible to put data of any binary object type into a cache
> w/o
> > registering type name and structure in advance.
> >
> > - OP_REGISTER_BINARY_TYPE_NAME, OP_GET_BINARY_TYPE_NAME: Platform id.
> >   Is it a type of the node platform?
> >   Why a remote app should know it?
> >   Why non-identical values are used here and in OP_QUERY_SCAN: Filter
> > platform?
> >
> > - OP_REGISTER_BINARY_TYPE_NAME, OP_PUT_BINARY_TYPE
> >   Why the both - Type name and Java-style hash code of the type name -
> are
> > in the request? (The same for Field name and Java-style hash code of the
> > field name.)
> >   Seems redundant.
> >
> > - Just interesting - why it was decided that hash code should be also
> > calculated on a client side? Why it could not be returned by a server
> side?
> >   Eg. OP_CACHE_CREATE_WITH_NAME Response can return hash code for the
> > provided Cache name.
> >   That would simplify thin client implementations, guarantee
> > consistency/correctness of hash code calculations...
> >
> > - Name/password authentication.
> >   When will it be available in the spec?
> >   Is there any draft we can look at?
> >
> > - SSL/TLS communication.
> >   When will it be available in the spec?
> >   Is there any draft we can look at?
> >
> > - As Denis said, we should implement a node failover algorithm (switching
> > to another predefined node).
> >   Should the algorithm be the same in all thin libs?
> >   Is it described somewhere?
> >   We have not found it in java and .net libs. Will it be supported by
> > them? When?
> >
> > - Is there any recommendation/suggestion how to test a node failover case
> > on a thin client side?
> >
> > - We are going to use jsdoc comments for the API methods:
> > http://usejsdoc.org/
> >   This tool, for example, may be used to generate the API spec from the
> > comments: https://www.npmjs.com/package/jsdoc
> >   Any objections?
> >
> > - We are going to write the lib readme/guide in the markdown format and
> > place it at the repo (.md file).
> >   Is it OK? Or should we use dash.readme.io ?
> >
> > - May we easily use 3rd-party components with the following licenses?
> >     -- Apache 2.0
> >     -- MIT
> >     -- GNU LGPL v3
> >     -- GNU GPL v2
> >   Should we pre-approve somehow / notify somebody about usage of concrete
> > 3rd-party components?
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Thin client / Binary protocol: questions

Alexey Kukushkin-3
Denis and All,

The links you asked for:

   - Java thin client development branch
   <https://github.com/gridgain/apache-ignite/tree/ignite-7421>
   - Code review in Upsource
   <https://reviews.ignite.apache.org/apache-ignite/review/IG-CR-23>
   - Thin client documentation
   <https://apacheignite.readme.io/v2.3/docs/java-thin-client>
      - Quick Start
      <https://apacheignite.readme.io/v2.3/docs/java-thin-client-quick-start>
      - API <https://apacheignite.readme.io/v2.3/docs/java-thin-client-api>
      - Security
      <https://apacheignite.readme.io/v2.3/docs/java-thin-client-security>
      - High Availability
      <https://apacheignite.readme.io/v2.3/docs/java-thin-client-high-availability>
      - Monitoring
      <https://apacheignite.readme.io/v2.3/docs/java-thin-client-monitoring>


On Wed, Feb 28, 2018 at 10:44 PM, Denis Magda <[hidden email]> wrote:

> Guys, please consider my five cents
>
> >>> - Name/password authentication.
>> >>> When will it be available in the spec?
>> >>> Is there any draft we can look at?
>> It is still under development. Will be available in several weeks. It is
>> better to skip this part for now.
>
>
> Let's give a reference to the branch where the guys can see those changes
> at the protocol level. My guess they want to know what to expect and how it
> can affect the design they've been working on. *Alexey Kukushkin*, please
> point out to your Java thin client branch were you already have this
> functionality embedded.
>
> >>> - SSL/TLS communication.
>> >>> When will it be available in the spec?
>
>
> This doc explains how SSL is enabled on the server side:
> https://apacheignite.readme.io/docs/ssltls
>
> As always, the thin client needs to establish the SSL tunneling first and
> then starts sending the protocol messages. Hope this is good for the
> beginning. Will be happy to answer more specific questions.
>
> >>>  - May we easily use 3rd-party components with the following licenses?
>> As a general rule of thumb it is not recommended to use any external
>> libraries. The whole Ignite core is built with almost no dependencies, not
>> to say about much less complex thin clients ans JDBC/ODBC drivers. IMO the
>> only possible place where external dependency might be required is SSL
>> support (e.g. we use OpenSSL for ODBC).
>
>
> I wouldn't discourage Alexey from using 3rd party libs taking Ignite core
> as an example. Ignite optional libs use 3rd party libs a lot. Take our REST
> protocol for instance that uses Jetty and not based on its own HTTP server
> implementation. So, I would alter the rule of thumb a little bit - if it
> takes weeks to develop a functionality already available in a 3rd party lib
> then let's go for the 3rd party.
>
> Here is a list of licenses that can be included in Ignite without any
> extra permissions or licensing concerns: https://www.apache.
> org/legal/resolved.html
>
> According to the doc you're free to use Apache 2.0 and MIT, but LGPL code
> can't be delivered within Ignite.
>
> --
> Denis
>
> On Wed, Feb 28, 2018 at 2:32 AM, Vladimir Ozerov <[hidden email]>
> wrote:
>
>> Hi Alex,
>>
>> >>> Just to double-check - we should support the types from the spec only.
>> Right?
>> Please provide the list of types you are in doubt of
>>
>> >>> - Key-Value and SQL and Scan Queries.
>> >>>  The most of operations has "Flag" field in Request: "Pass 0 for
>> default, or 1 to keep the value in binary form."
>> >>>  What is it for?
>> Please see IgniteCache.withKeepBinary method. For SQL this flag has no
>> effect. For complex SCAN queries with filters this defines whether with
>> pass real objects or binary objects to the filter.
>>
>> >>>  Java and .net libs always pass 0. Why?
>> There is no Java client at the moment. As far as .NET, it doesn't support
>> binary mode at the moment.
>>
>> >>>  Why an app working remotely via the binary protocol should know the
>> exact platform where the node is running?
>> Because if node is started from .NET, you may execute both Java filters
>> and
>> .NET fitlers on it. This flag defines what kind of filter is passed.
>>
>> >>> - Binary Type operations (register/get type name, put/get type).
>> >>>  What are they for?
>> >>>  What is a use case?
>> Please get familiar with binary type concepts, especially binary metadata:
>> https://apacheignite.readme.io/docs#binaryobject-type-metadata
>>
>> >>>  - Just interesting - why it was decided that hash code should be also
>> calculated on a client side? Why it could not be returned by a server
>> side?
>> Because this is inefficient - we would have to rebuild binary objects on
>> the server side. Instead, it is better to implement pretty simple routine
>> for hashcode calculation inside every client.
>>
>> >>> - Name/password authentication.
>> >>> When will it be available in the spec?
>> >>> Is there any draft we can look at?
>> It is still under development. Will be available in several weeks. It is
>> better to skip this part for now.
>>
>> >>> - SSL/TLS communication.
>> >>> When will it be available in the spec?
>> This is not part of protocol specification. Please refer to .NET client
>> where it is already implemented. *Pavel*, do we have docs for this?
>>
>> >>> - As Denis said, we should implement a node failover algorithm
>> (switching to another predefined node).
>> >>> Should the algorithm be the same in all thin libs?
>> >>> Is it described somewhere?
>> >>> We have not found it in java and .net libs. Will it be supported by
>> them? When?
>> It is still under development, will be ready in 1-2 weeks. Please skip for
>> now. Yes, it is better to have the same algorithm on all clients for the
>> sake of consistency.
>>
>> >>> - We are going to write the lib readme/guide in the markdown format
>> and
>> place it at the repo (.md file).
>> >>> Is it OK? Or should we use dash.readme.io ?
>> All our documentation is hosted on readme.io. There should be no
>> exclusions. *Denis*, could you make sure that necessary spaces and
>> permissions are set up?
>>
>> >>>  - May we easily use 3rd-party components with the following licenses?
>> As a general rule of thumb it is not recommended to use any external
>> libraries. The whole Ignite core is built with almost no dependencies, not
>> to say about much less complex thin clients ans JDBC/ODBC drivers. IMO the
>> only possible place where external dependency might be required is SSL
>> support (e.g. we use OpenSSL for ODBC).
>>
>> Vladimir.
>>
>> On Wed, Feb 28, 2018 at 9:41 AM, Alexey Kosenchuk <
>> [hidden email]> wrote:
>>
>> > Folks,
>> >
>> > Designing node.js thin lib (IGNITE-7777) we have the following
>> questions.
>> > Could you please have a look and clarify.
>> >
>> > Thanks,
>> > -Alexey
>> >
>> > - Supported standard types and their type codes.
>> >   Are defined in the spec: https://apacheignite.readme.io
>> > /docs/binary-client-protocol#data-format
>> >   But, as we see, java and .net libs (common parts which are used by
>> thin
>> > clients as well) support additional types and type codes, not described
>> in
>> > the spec. Eg. defined here: https://github.com/apache/igni
>> > te/blob/master/modules/core/src/main/java/org/apache/ignite/
>> > internal/binary/GridBinaryMarshaller.java
>> >   Just to double-check - we should support the types from the spec only.
>> > Right?
>> >
>> > - Key-Value and SQL and Scan Queries.
>> >   The most of operations has "Flag" field in Request: "Pass 0 for
>> default,
>> > or 1 to keep the value in binary form."
>> >   What is it for?
>> >   Java and .net libs always pass 0. Why?
>> >
>> > - OP_QUERY_SCAN: Filter platform.
>> >   Why is it required?
>> >   Why an app working remotely via the binary protocol should know the
>> > exact platform where the node is running?
>> >
>> > - Binary Type operations (register/get type name, put/get type).
>> >   What are they for?
>> >   What is a use case?
>> >   It seems possible to put data of any binary object type into a cache
>> w/o
>> > registering type name and structure in advance.
>> >
>> > - OP_REGISTER_BINARY_TYPE_NAME, OP_GET_BINARY_TYPE_NAME: Platform id.
>> >   Is it a type of the node platform?
>> >   Why a remote app should know it?
>> >   Why non-identical values are used here and in OP_QUERY_SCAN: Filter
>> > platform?
>> >
>> > - OP_REGISTER_BINARY_TYPE_NAME, OP_PUT_BINARY_TYPE
>> >   Why the both - Type name and Java-style hash code of the type name -
>> are
>> > in the request? (The same for Field name and Java-style hash code of the
>> > field name.)
>> >   Seems redundant.
>> >
>> > - Just interesting - why it was decided that hash code should be also
>> > calculated on a client side? Why it could not be returned by a server
>> side?
>> >   Eg. OP_CACHE_CREATE_WITH_NAME Response can return hash code for the
>> > provided Cache name.
>> >   That would simplify thin client implementations, guarantee
>> > consistency/correctness of hash code calculations...
>> >
>> > - Name/password authentication.
>> >   When will it be available in the spec?
>> >   Is there any draft we can look at?
>> >
>> > - SSL/TLS communication.
>> >   When will it be available in the spec?
>> >   Is there any draft we can look at?
>> >
>> > - As Denis said, we should implement a node failover algorithm
>> (switching
>> > to another predefined node).
>> >   Should the algorithm be the same in all thin libs?
>> >   Is it described somewhere?
>> >   We have not found it in java and .net libs. Will it be supported by
>> > them? When?
>> >
>> > - Is there any recommendation/suggestion how to test a node failover
>> case
>> > on a thin client side?
>> >
>> > - We are going to use jsdoc comments for the API methods:
>> > http://usejsdoc.org/
>> >   This tool, for example, may be used to generate the API spec from the
>> > comments: https://www.npmjs.com/package/jsdoc
>> >   Any objections?
>> >
>> > - We are going to write the lib readme/guide in the markdown format and
>> > place it at the repo (.md file).
>> >   Is it OK? Or should we use dash.readme.io ?
>> >
>> > - May we easily use 3rd-party components with the following licenses?
>> >     -- Apache 2.0
>> >     -- MIT
>> >     -- GNU LGPL v3
>> >     -- GNU GPL v2
>> >   Should we pre-approve somehow / notify somebody about usage of
>> concrete
>> > 3rd-party components?
>> >
>>
>
>


--
Best regards,
Alexey
+7 981 156 56 47