Thin Client Protocol documentation

classic Classic list List threaded Threaded
27 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: Thin Client Protocol documentation

Prachi Garg
Hi Pavel,

I am trying to use the OP_CACHE_CREATE_WITH_NAME operation, but can't get
it to work.  Digging deeper into the source code, it seems like I have to
provide a flag, string length, and position, in addition to the type code
and the actual string. Is that correct?

Here is the request I am sending to the server-

DataOutputStream out = new DataOutputStream(socket.getOutputStream());

// Message length
writeIntLittleEndian(24, out);

// Op code = OP_CACHE_CREATE_WITH_NAME
writeShortLittleEndian(1051, out);

// Request id
long reqId = 1;
writeLongLittleEndian(reqId, out);

// String (cache name)
writeByteLittleEndian(9, out); // Type code
writeByteLittleEndian(0, out); // Flag
writeIntLittleEndian(20, out); // String length
writeIntLittleEndian(0, out); // Position
out.writeUTF("myNewCache"); // Cache name

// Send request
out.flush();

But I get the following error on the server side.

[2017-12-04
17:27:39,421][ERROR][client-connector-#53][ClientListenerNioListener]
Failed to parse client request.
java.lang.StringIndexOutOfBoundsException: String index out of range: 2575
at java.lang.String.checkBounds(String.java:385)
at java.lang.String.<init>(String.java:462)
at
org.apache.ignite.internal.binary.BinaryUtils.doReadString(BinaryUtils.java:1314)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.readString(BinaryReaderExImpl.java:1055)
at
org.apache.ignite.internal.processors.platform.client.cache.ClientCacheCreateWithNameRequest.<init>(ClientCacheCreateWithNameRequest.java:43)
at
org.apache.ignite.internal.processors.platform.client.ClientMessageParser.decode(ClientMessageParser.java:318)
at
org.apache.ignite.internal.processors.platform.client.ClientMessageParser.decode(ClientMessageParser.java:220)
at
org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:119)
at
org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:40)
at
org.apache.ignite.internal.util.nio.GridNioFilterChain$TailFilter.onMessageReceived(GridNioFilterChain.java:279)
at
org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109)
at
org.apache.ignite.internal.util.nio.GridNioAsyncNotifyFilter$3.body(GridNioAsyncNotifyFilter.java:97)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at
org.apache.ignite.internal.util.worker.GridWorkerPool$1.run(GridWorkerPool.java:70)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

What am I missing here? Can you provide an example of how to send a
'String' type request?

-Prachi

On Mon, Dec 4, 2017 at 1:06 PM, Andrey Kornev <[hidden email]>
wrote:

> Pavel,
>
> Thanks! While we're at it, are there any plans to add cluster-related
> operations? For example, I think it'd be nice to allow the thin clients to
> obtain a current topology snapshot. This would make it possible the clients
> to send requests directly to the affinity host for colocated computation.
> To make it even more useful, all server responses could optionally include
> the topology version the operation has been executed against. This would
> effectively give us a kind out-of-band topology change notification
> mechanism. This way the clients can detect a topology change and refresh
> the topology snapshot next time they need to compute affinity.
>
> Regards
> Andrey
> ________________________________
> From: Pavel Tupitsyn <[hidden email]>
> Sent: Sunday, December 3, 2017 9:23 AM
> To: [hidden email]
> Subject: Re: Thin Client Protocol documentation
>
> Hi Andrey,
>
> Compute and other APIs are certainly planned, cache is just a start.
> We intentionally limit the scope to actually release something in 2.4 and
> not delay it further.
>
> Adding operations to existing protocol is relatively easy.
> Current focus is to make sure that the protocol itself is solid and
> future-proof.
>
> Thanks,
> Pavel
>
Reply | Threaded
Open this post in threaded view
|

Re: Thin Client Protocol documentation

Pavel Tupitsyn-3
Andrey,

All of this is to come :)


Prachi,

1) There are no flags, see the doc
2) String is simply 4 byte length (n) + n bytes
3) Op codes have changed

writeByteLittleEndian(1051, out); // Type code
writeIntLittleEndian(20, out); // String length
out.writeUTF("myNewCache"); // Cache name

On Tue, Dec 5, 2017 at 4:39 AM, Prachi Garg <[hidden email]> wrote:

> Hi Pavel,
>
> I am trying to use the OP_CACHE_CREATE_WITH_NAME operation, but can't get
> it to work.  Digging deeper into the source code, it seems like I have to
> provide a flag, string length, and position, in addition to the type code
> and the actual string. Is that correct?
>
> Here is the request I am sending to the server-
>
> DataOutputStream out = new DataOutputStream(socket.getOutputStream());
>
> // Message length
> writeIntLittleEndian(24, out);
>
> // Op code = OP_CACHE_CREATE_WITH_NAME
> writeShortLittleEndian(1051, out);
>
> // Request id
> long reqId = 1;
> writeLongLittleEndian(reqId, out);
>
> // String (cache name)
> writeByteLittleEndian(9, out); // Type code
> writeByteLittleEndian(0, out); // Flag
> writeIntLittleEndian(20, out); // String length
> writeIntLittleEndian(0, out); // Position
> out.writeUTF("myNewCache"); // Cache name
>
> // Send request
> out.flush();
>
> But I get the following error on the server side.
>
> [2017-12-04 17:27:39,421][ERROR][client-connector-#53][ClientListenerNioListener]
> Failed to parse client request.
> java.lang.StringIndexOutOfBoundsException: String index out of range: 2575
> at java.lang.String.checkBounds(String.java:385)
> at java.lang.String.<init>(String.java:462)
> at org.apache.ignite.internal.binary.BinaryUtils.
> doReadString(BinaryUtils.java:1314)
> at org.apache.ignite.internal.binary.BinaryReaderExImpl.
> readString(BinaryReaderExImpl.java:1055)
> at org.apache.ignite.internal.processors.platform.client.cache.
> ClientCacheCreateWithNameRequest.<init>(ClientCacheCreateWithNameReque
> st.java:43)
> at org.apache.ignite.internal.processors.platform.client.
> ClientMessageParser.decode(ClientMessageParser.java:318)
> at org.apache.ignite.internal.processors.platform.client.
> ClientMessageParser.decode(ClientMessageParser.java:220)
> at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.
> onMessage(ClientListenerNioListener.java:119)
> at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.
> onMessage(ClientListenerNioListener.java:40)
> at org.apache.ignite.internal.util.nio.GridNioFilterChain$
> TailFilter.onMessageReceived(GridNioFilterChain.java:279)
> at org.apache.ignite.internal.util.nio.GridNioFilterAdapter.
> proceedMessageReceived(GridNioFilterAdapter.java:109)
> at org.apache.ignite.internal.util.nio.GridNioAsyncNotifyFilter$3.
> body(GridNioAsyncNotifyFilter.java:97)
> at org.apache.ignite.internal.util.worker.GridWorker.run(
> GridWorker.java:110)
> at org.apache.ignite.internal.util.worker.GridWorkerPool$1.
> run(GridWorkerPool.java:70)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>
> What am I missing here? Can you provide an example of how to send a
> 'String' type request?
>
> -Prachi
>
> On Mon, Dec 4, 2017 at 1:06 PM, Andrey Kornev <[hidden email]>
> wrote:
>
>> Pavel,
>>
>> Thanks! While we're at it, are there any plans to add cluster-related
>> operations? For example, I think it'd be nice to allow the thin clients to
>> obtain a current topology snapshot. This would make it possible the clients
>> to send requests directly to the affinity host for colocated computation.
>> To make it even more useful, all server responses could optionally include
>> the topology version the operation has been executed against. This would
>> effectively give us a kind out-of-band topology change notification
>> mechanism. This way the clients can detect a topology change and refresh
>> the topology snapshot next time they need to compute affinity.
>>
>> Regards
>> Andrey
>> ________________________________
>> From: Pavel Tupitsyn <[hidden email]>
>> Sent: Sunday, December 3, 2017 9:23 AM
>> To: [hidden email]
>> Subject: Re: Thin Client Protocol documentation
>>
>> Hi Andrey,
>>
>> Compute and other APIs are certainly planned, cache is just a start.
>> We intentionally limit the scope to actually release something in 2.4 and
>> not delay it further.
>>
>> Adding operations to existing protocol is relatively easy.
>> Current focus is to make sure that the protocol itself is solid and
>> future-proof.
>>
>> Thanks,
>> Pavel
>>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Thin Client Protocol documentation

Andrey Kornev
Pavel,

I have absolutely no doubts that support for all those features will be added eventually. And if so, wouldn't it be the right thing to do to document the operations and their semantics right now without necessarily implementing them? It should really help to ensure that the protocol can accommodate all those use cases before it gets released to the public.

Thanks
Andrey
________________________________
From: Pavel Tupitsyn <[hidden email]>
Sent: Tuesday, December 5, 2017 12:07 AM
Cc: [hidden email]
Subject: Re: Thin Client Protocol documentation

Andrey,

All of this is to come :)


Prachi,

1) There are no flags, see the doc
2) String is simply 4 byte length (n) + n bytes
3) Op codes have changed

writeByteLittleEndian(1051, out); // Type code
writeIntLittleEndian(20, out); // String length
out.writeUTF("myNewCache"); // Cache name

On Tue, Dec 5, 2017 at 4:39 AM, Prachi Garg <[hidden email]> wrote:

> Hi Pavel,
>
> I am trying to use the OP_CACHE_CREATE_WITH_NAME operation, but can't get
> it to work.  Digging deeper into the source code, it seems like I have to
> provide a flag, string length, and position, in addition to the type code
> and the actual string. Is that correct?
>
> Here is the request I am sending to the server-
>
> DataOutputStream out = new DataOutputStream(socket.getOutputStream());
>
> // Message length
> writeIntLittleEndian(24, out);
>
> // Op code = OP_CACHE_CREATE_WITH_NAME
> writeShortLittleEndian(1051, out);
>
> // Request id
> long reqId = 1;
> writeLongLittleEndian(reqId, out);
>
> // String (cache name)
> writeByteLittleEndian(9, out); // Type code
> writeByteLittleEndian(0, out); // Flag
> writeIntLittleEndian(20, out); // String length
> writeIntLittleEndian(0, out); // Position
> out.writeUTF("myNewCache"); // Cache name
>
> // Send request
> out.flush();
>
> But I get the following error on the server side.
>
> [2017-12-04 17:27:39,421][ERROR][client-connector-#53][ClientListenerNioListener]
> Failed to parse client request.
> java.lang.StringIndexOutOfBoundsException: String index out of range: 2575
> at java.lang.String.checkBounds(String.java:385)
> at java.lang.String.<init>(String.java:462)
> at org.apache.ignite.internal.binary.BinaryUtils.
> doReadString(BinaryUtils.java:1314)
> at org.apache.ignite.internal.binary.BinaryReaderExImpl.
> readString(BinaryReaderExImpl.java:1055)
> at org.apache.ignite.internal.processors.platform.client.cache.
> ClientCacheCreateWithNameRequest.<init>(ClientCacheCreateWithNameReque
> st.java:43)
> at org.apache.ignite.internal.processors.platform.client.
> ClientMessageParser.decode(ClientMessageParser.java:318)
> at org.apache.ignite.internal.processors.platform.client.
> ClientMessageParser.decode(ClientMessageParser.java:220)
> at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.
> onMessage(ClientListenerNioListener.java:119)
> at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.
> onMessage(ClientListenerNioListener.java:40)
> at org.apache.ignite.internal.util.nio.GridNioFilterChain$
> TailFilter.onMessageReceived(GridNioFilterChain.java:279)
> at org.apache.ignite.internal.util.nio.GridNioFilterAdapter.
> proceedMessageReceived(GridNioFilterAdapter.java:109)
> at org.apache.ignite.internal.util.nio.GridNioAsyncNotifyFilter$3.
> body(GridNioAsyncNotifyFilter.java:97)
> at org.apache.ignite.internal.util.worker.GridWorker.run(
> GridWorker.java:110)
> at org.apache.ignite.internal.util.worker.GridWorkerPool$1.
> run(GridWorkerPool.java:70)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>
> What am I missing here? Can you provide an example of how to send a
> 'String' type request?
>
> -Prachi
>
> On Mon, Dec 4, 2017 at 1:06 PM, Andrey Kornev <[hidden email]>
> wrote:
>
>> Pavel,
>>
>> Thanks! While we're at it, are there any plans to add cluster-related
>> operations? For example, I think it'd be nice to allow the thin clients to
>> obtain a current topology snapshot. This would make it possible the clients
>> to send requests directly to the affinity host for colocated computation.
>> To make it even more useful, all server responses could optionally include
>> the topology version the operation has been executed against. This would
>> effectively give us a kind out-of-band topology change notification
>> mechanism. This way the clients can detect a topology change and refresh
>> the topology snapshot next time they need to compute affinity.
>>
>> Regards
>> Andrey
>> ________________________________
>> From: Pavel Tupitsyn <[hidden email]>
>> Sent: Sunday, December 3, 2017 9:23 AM
>> To: [hidden email]
>> Subject: Re: Thin Client Protocol documentation
>>
>> Hi Andrey,
>>
>> Compute and other APIs are certainly planned, cache is just a start.
>> We intentionally limit the scope to actually release something in 2.4 and
>> not delay it further.
>>
>> Adding operations to existing protocol is relatively easy.
>> Current focus is to make sure that the protocol itself is solid and
>> future-proof.
>>
>> Thanks,
>> Pavel
>>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Thin Client Protocol documentation

Pavel Tupitsyn-3
Andrey,

As I understand, you suggest to document every prospective feature right
now.
That would be (at least) compute, clustering, transactions, services,
messages, events, failover, data structures, metrics.

I don't think it makes sense to do that. Another problem is that it will
take weeks :)
But, of course, you are welcome to take the initiative.

Thanks,
Pavel

On Tue, Dec 5, 2017 at 10:20 PM, Andrey Kornev <[hidden email]>
wrote:

> Pavel,
>
> I have absolutely no doubts that support for all those features will be
> added eventually. And if so, wouldn't it be the right thing to do to
> document the operations and their semantics right now without
> necessarily implementing them? It should really help to ensure that the
> protocol can accommodate all those use cases before it gets released to the
> public.
>
> Thanks
> Andrey
> ------------------------------
> *From:* Pavel Tupitsyn <[hidden email]>
> *Sent:* Tuesday, December 5, 2017 12:07 AM
> *Cc:* [hidden email]
> *Subject:* Re: Thin Client Protocol documentation
>
> Andrey,
>
> All of this is to come :)
>
>
> Prachi,
>
> 1) There are no flags, see the doc
> 2) String is simply 4 byte length (n) + n bytes
> 3) Op codes have changed
>
> writeByteLittleEndian(1051, out); // Type code
> writeIntLittleEndian(20, out); // String length
> out.writeUTF("myNewCache"); // Cache name
>
> On Tue, Dec 5, 2017 at 4:39 AM, Prachi Garg <[hidden email]> wrote:
>
> > Hi Pavel,
> >
> > I am trying to use the OP_CACHE_CREATE_WITH_NAME operation, but can't get
> > it to work.  Digging deeper into the source code, it seems like I have to
> > provide a flag, string length, and position, in addition to the type code
> > and the actual string. Is that correct?
> >
> > Here is the request I am sending to the server-
> >
> > DataOutputStream out = new DataOutputStream(socket.getOutputStream());
> >
> > // Message length
> > writeIntLittleEndian(24, out);
> >
> > // Op code = OP_CACHE_CREATE_WITH_NAME
> > writeShortLittleEndian(1051, out);
> >
> > // Request id
> > long reqId = 1;
> > writeLongLittleEndian(reqId, out);
> >
> > // String (cache name)
> > writeByteLittleEndian(9, out); // Type code
> > writeByteLittleEndian(0, out); // Flag
> > writeIntLittleEndian(20, out); // String length
> > writeIntLittleEndian(0, out); // Position
> > out.writeUTF("myNewCache"); // Cache name
> >
> > // Send request
> > out.flush();
> >
> > But I get the following error on the server side.
> >
> > [2017-12-04 17:27:39,421][ERROR][client-connector-#53][
> ClientListenerNioListener]
> > Failed to parse client request.
> > java.lang.StringIndexOutOfBoundsException: String index out of range:
> 2575
> > at java.lang.String.checkBounds(String.java:385)
> > at java.lang.String.<init>(String.java:462)
> > at org.apache.ignite.internal.binary.BinaryUtils.
> > doReadString(BinaryUtils.java:1314)
> > at org.apache.ignite.internal.binary.BinaryReaderExImpl.
> > readString(BinaryReaderExImpl.java:1055)
> > at org.apache.ignite.internal.processors.platform.client.cache.
> > ClientCacheCreateWithNameRequest.<init>(ClientCacheCreateWithNameReque
> > st.java:43)
> > at org.apache.ignite.internal.processors.platform.client.
> > ClientMessageParser.decode(ClientMessageParser.java:318)
> > at org.apache.ignite.internal.processors.platform.client.
> > ClientMessageParser.decode(ClientMessageParser.java:220)
> > at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.
> > onMessage(ClientListenerNioListener.java:119)
> > at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.
> > onMessage(ClientListenerNioListener.java:40)
> > at org.apache.ignite.internal.util.nio.GridNioFilterChain$
> > TailFilter.onMessageReceived(GridNioFilterChain.java:279)
> > at org.apache.ignite.internal.util.nio.GridNioFilterAdapter.
> > proceedMessageReceived(GridNioFilterAdapter.java:109)
> > at org.apache.ignite.internal.util.nio.GridNioAsyncNotifyFilter$3.
> > body(GridNioAsyncNotifyFilter.java:97)
> > at org.apache.ignite.internal.util.worker.GridWorker.run(
> > GridWorker.java:110)
> > at org.apache.ignite.internal.util.worker.GridWorkerPool$1.
> > run(GridWorkerPool.java:70)
> > at java.util.concurrent.ThreadPoolExecutor.runWorker(
> > ThreadPoolExecutor.java:1142)
> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> > ThreadPoolExecutor.java:617)
> > at java.lang.Thread.run(Thread.java:745)
> >
> > What am I missing here? Can you provide an example of how to send a
> > 'String' type request?
> >
> > -Prachi
> >
> > On Mon, Dec 4, 2017 at 1:06 PM, Andrey Kornev <[hidden email]>
> > wrote:
> >
> >> Pavel,
> >>
> >> Thanks! While we're at it, are there any plans to add cluster-related
> >> operations? For example, I think it'd be nice to allow the thin clients
> to
> >> obtain a current topology snapshot. This would make it possible the
> clients
> >> to send requests directly to the affinity host for colocated
> computation.
> >> To make it even more useful, all server responses could optionally
> include
> >> the topology version the operation has been executed against. This would
> >> effectively give us a kind out-of-band topology change notification
> >> mechanism. This way the clients can detect a topology change and refresh
> >> the topology snapshot next time they need to compute affinity.
> >>
> >> Regards
> >> Andrey
> >> ________________________________
> >> From: Pavel Tupitsyn <[hidden email]>
> >> Sent: Sunday, December 3, 2017 9:23 AM
> >> To: [hidden email]
> >> Subject: Re: Thin Client Protocol documentation
> >>
> >> Hi Andrey,
> >>
> >> Compute and other APIs are certainly planned, cache is just a start.
> >> We intentionally limit the scope to actually release something in 2.4
> and
> >> not delay it further.
> >>
> >> Adding operations to existing protocol is relatively easy.
> >> Current focus is to make sure that the protocol itself is solid and
> >> future-proof.
> >>
> >> Thanks,
> >> Pavel
> >>
> >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Thin Client Protocol documentation

Andrey Kornev
>>> I don't think it makes sense to do that. Another problem is that it will take weeks :)

I'm sorry, Pavel. None of what you've said so far makes it obvious why it doesn't make sense to produce the complete spec. Would you care to elaborate, please?

Also, what's the problem with the "another problem" of the effort taking weeks? If based on your best estimate the effort requires a few weeks of your time, why is this a problem then?

>>> But, of course, you are welcome to take the initiative.

Saying something like that is not very helpful, Pavel. But let me just say, I believe that as the current spec lead and an employee of the company behind Apache Ignite project you're in a much better position to continue to lead the effort (you've been doing great so far!). From my side, I'd be more than happy to review the spec and provide my feedback. Also, I'm very interested in becoming a contributor/early adopter of the Java-based implementation of the protocol.

Regards
Andrey

________________________________
From: Pavel Tupitsyn <[hidden email]>
Sent: Tuesday, December 5, 2017 11:15 PM
To: [hidden email]
Subject: Re: Thin Client Protocol documentation

Andrey,

As I understand, you suggest to document every prospective feature right
now.
That would be (at least) compute, clustering, transactions, services,
messages, events, failover, data structures, metrics.

I don't think it makes sense to do that. Another problem is that it will
take weeks :)
But, of course, you are welcome to take the initiative.

Thanks,
Pavel

On Tue, Dec 5, 2017 at 10:20 PM, Andrey Kornev <[hidden email]>
wrote:

> Pavel,
>
> I have absolutely no doubts that support for all those features will be
> added eventually. And if so, wouldn't it be the right thing to do to
> document the operations and their semantics right now without
> necessarily implementing them? It should really help to ensure that the
> protocol can accommodate all those use cases before it gets released to the
> public.
>
> Thanks
> Andrey
> ------------------------------
> *From:* Pavel Tupitsyn <[hidden email]>
> *Sent:* Tuesday, December 5, 2017 12:07 AM
> *Cc:* [hidden email]
> *Subject:* Re: Thin Client Protocol documentation
>
> Andrey,
>
> All of this is to come :)
>
>
> Prachi,
>
> 1) There are no flags, see the doc
> 2) String is simply 4 byte length (n) + n bytes
> 3) Op codes have changed
>
> writeByteLittleEndian(1051, out); // Type code
> writeIntLittleEndian(20, out); // String length
> out.writeUTF("myNewCache"); // Cache name
>
> On Tue, Dec 5, 2017 at 4:39 AM, Prachi Garg <[hidden email]> wrote:
>
> > Hi Pavel,
> >
> > I am trying to use the OP_CACHE_CREATE_WITH_NAME operation, but can't get
> > it to work.  Digging deeper into the source code, it seems like I have to
> > provide a flag, string length, and position, in addition to the type code
> > and the actual string. Is that correct?
> >
> > Here is the request I am sending to the server-
> >
> > DataOutputStream out = new DataOutputStream(socket.getOutputStream());
> >
> > // Message length
> > writeIntLittleEndian(24, out);
> >
> > // Op code = OP_CACHE_CREATE_WITH_NAME
> > writeShortLittleEndian(1051, out);
> >
> > // Request id
> > long reqId = 1;
> > writeLongLittleEndian(reqId, out);
> >
> > // String (cache name)
> > writeByteLittleEndian(9, out); // Type code
> > writeByteLittleEndian(0, out); // Flag
> > writeIntLittleEndian(20, out); // String length
> > writeIntLittleEndian(0, out); // Position
> > out.writeUTF("myNewCache"); // Cache name
> >
> > // Send request
> > out.flush();
> >
> > But I get the following error on the server side.
> >
> > [2017-12-04 17:27:39,421][ERROR][client-connector-#53][
> ClientListenerNioListener]
> > Failed to parse client request.
> > java.lang.StringIndexOutOfBoundsException: String index out of range:
> 2575
> > at java.lang.String.checkBounds(String.java:385)
> > at java.lang.String.<init>(String.java:462)
> > at org.apache.ignite.internal.binary.BinaryUtils.
> > doReadString(BinaryUtils.java:1314)
> > at org.apache.ignite.internal.binary.BinaryReaderExImpl.
> > readString(BinaryReaderExImpl.java:1055)
> > at org.apache.ignite.internal.processors.platform.client.cache.
> > ClientCacheCreateWithNameRequest.<init>(ClientCacheCreateWithNameReque
> > st.java:43)
> > at org.apache.ignite.internal.processors.platform.client.
> > ClientMessageParser.decode(ClientMessageParser.java:318)
> > at org.apache.ignite.internal.processors.platform.client.
> > ClientMessageParser.decode(ClientMessageParser.java:220)
> > at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.
> > onMessage(ClientListenerNioListener.java:119)
> > at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.
> > onMessage(ClientListenerNioListener.java:40)
> > at org.apache.ignite.internal.util.nio.GridNioFilterChain$
> > TailFilter.onMessageReceived(GridNioFilterChain.java:279)
> > at org.apache.ignite.internal.util.nio.GridNioFilterAdapter.
> > proceedMessageReceived(GridNioFilterAdapter.java:109)
> > at org.apache.ignite.internal.util.nio.GridNioAsyncNotifyFilter$3.
> > body(GridNioAsyncNotifyFilter.java:97)
> > at org.apache.ignite.internal.util.worker.GridWorker.run(
> > GridWorker.java:110)
> > at org.apache.ignite.internal.util.worker.GridWorkerPool$1.
> > run(GridWorkerPool.java:70)
> > at java.util.concurrent.ThreadPoolExecutor.runWorker(
> > ThreadPoolExecutor.java:1142)
> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> > ThreadPoolExecutor.java:617)
> > at java.lang.Thread.run(Thread.java:745)
> >
> > What am I missing here? Can you provide an example of how to send a
> > 'String' type request?
> >
> > -Prachi
> >
> > On Mon, Dec 4, 2017 at 1:06 PM, Andrey Kornev <[hidden email]>
> > wrote:
> >
> >> Pavel,
> >>
> >> Thanks! While we're at it, are there any plans to add cluster-related
> >> operations? For example, I think it'd be nice to allow the thin clients
> to
> >> obtain a current topology snapshot. This would make it possible the
> clients
> >> to send requests directly to the affinity host for colocated
> computation.
> >> To make it even more useful, all server responses could optionally
> include
> >> the topology version the operation has been executed against. This would
> >> effectively give us a kind out-of-band topology change notification
> >> mechanism. This way the clients can detect a topology change and refresh
> >> the topology snapshot next time they need to compute affinity.
> >>
> >> Regards
> >> Andrey
> >> ________________________________
> >> From: Pavel Tupitsyn <[hidden email]>
> >> Sent: Sunday, December 3, 2017 9:23 AM
> >> To: [hidden email]
> >> Subject: Re: Thin Client Protocol documentation
> >>
> >> Hi Andrey,
> >>
> >> Compute and other APIs are certainly planned, cache is just a start.
> >> We intentionally limit the scope to actually release something in 2.4
> and
> >> not delay it further.
> >>
> >> Adding operations to existing protocol is relatively easy.
> >> Current focus is to make sure that the protocol itself is solid and
> >> future-proof.
> >>
> >> Thanks,
> >> Pavel
> >>
> >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Thin Client Protocol documentation

Andrey Kornev
In reply to this post by Pavel Tupitsyn-3
Pavel, could you please at least share your time frame wrt adding compute, clustering and services features to the spec?

Thanks
Andrey
________________________________
From: Pavel Tupitsyn <[hidden email]>
Sent: Tuesday, December 5, 2017 11:15 PM
To: [hidden email]
Subject: Re: Thin Client Protocol documentation

Andrey,

As I understand, you suggest to document every prospective feature right
now.
That would be (at least) compute, clustering, transactions, services,
messages, events, failover, data structures, metrics.

I don't think it makes sense to do that. Another problem is that it will
take weeks :)
But, of course, you are welcome to take the initiative.

Thanks,
Pavel

On Tue, Dec 5, 2017 at 10:20 PM, Andrey Kornev <[hidden email]>
wrote:

> Pavel,
>
> I have absolutely no doubts that support for all those features will be
> added eventually. And if so, wouldn't it be the right thing to do to
> document the operations and their semantics right now without
> necessarily implementing them? It should really help to ensure that the
> protocol can accommodate all those use cases before it gets released to the
> public.
>
> Thanks
> Andrey
> ------------------------------
> *From:* Pavel Tupitsyn <[hidden email]>
> *Sent:* Tuesday, December 5, 2017 12:07 AM
> *Cc:* [hidden email]
> *Subject:* Re: Thin Client Protocol documentation
>
> Andrey,
>
> All of this is to come :)
>
>
> Prachi,
>
> 1) There are no flags, see the doc
> 2) String is simply 4 byte length (n) + n bytes
> 3) Op codes have changed
>
> writeByteLittleEndian(1051, out); // Type code
> writeIntLittleEndian(20, out); // String length
> out.writeUTF("myNewCache"); // Cache name
>
> On Tue, Dec 5, 2017 at 4:39 AM, Prachi Garg <[hidden email]> wrote:
>
> > Hi Pavel,
> >
> > I am trying to use the OP_CACHE_CREATE_WITH_NAME operation, but can't get
> > it to work.  Digging deeper into the source code, it seems like I have to
> > provide a flag, string length, and position, in addition to the type code
> > and the actual string. Is that correct?
> >
> > Here is the request I am sending to the server-
> >
> > DataOutputStream out = new DataOutputStream(socket.getOutputStream());
> >
> > // Message length
> > writeIntLittleEndian(24, out);
> >
> > // Op code = OP_CACHE_CREATE_WITH_NAME
> > writeShortLittleEndian(1051, out);
> >
> > // Request id
> > long reqId = 1;
> > writeLongLittleEndian(reqId, out);
> >
> > // String (cache name)
> > writeByteLittleEndian(9, out); // Type code
> > writeByteLittleEndian(0, out); // Flag
> > writeIntLittleEndian(20, out); // String length
> > writeIntLittleEndian(0, out); // Position
> > out.writeUTF("myNewCache"); // Cache name
> >
> > // Send request
> > out.flush();
> >
> > But I get the following error on the server side.
> >
> > [2017-12-04 17:27:39,421][ERROR][client-connector-#53][
> ClientListenerNioListener]
> > Failed to parse client request.
> > java.lang.StringIndexOutOfBoundsException: String index out of range:
> 2575
> > at java.lang.String.checkBounds(String.java:385)
> > at java.lang.String.<init>(String.java:462)
> > at org.apache.ignite.internal.binary.BinaryUtils.
> > doReadString(BinaryUtils.java:1314)
> > at org.apache.ignite.internal.binary.BinaryReaderExImpl.
> > readString(BinaryReaderExImpl.java:1055)
> > at org.apache.ignite.internal.processors.platform.client.cache.
> > ClientCacheCreateWithNameRequest.<init>(ClientCacheCreateWithNameReque
> > st.java:43)
> > at org.apache.ignite.internal.processors.platform.client.
> > ClientMessageParser.decode(ClientMessageParser.java:318)
> > at org.apache.ignite.internal.processors.platform.client.
> > ClientMessageParser.decode(ClientMessageParser.java:220)
> > at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.
> > onMessage(ClientListenerNioListener.java:119)
> > at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.
> > onMessage(ClientListenerNioListener.java:40)
> > at org.apache.ignite.internal.util.nio.GridNioFilterChain$
> > TailFilter.onMessageReceived(GridNioFilterChain.java:279)
> > at org.apache.ignite.internal.util.nio.GridNioFilterAdapter.
> > proceedMessageReceived(GridNioFilterAdapter.java:109)
> > at org.apache.ignite.internal.util.nio.GridNioAsyncNotifyFilter$3.
> > body(GridNioAsyncNotifyFilter.java:97)
> > at org.apache.ignite.internal.util.worker.GridWorker.run(
> > GridWorker.java:110)
> > at org.apache.ignite.internal.util.worker.GridWorkerPool$1.
> > run(GridWorkerPool.java:70)
> > at java.util.concurrent.ThreadPoolExecutor.runWorker(
> > ThreadPoolExecutor.java:1142)
> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> > ThreadPoolExecutor.java:617)
> > at java.lang.Thread.run(Thread.java:745)
> >
> > What am I missing here? Can you provide an example of how to send a
> > 'String' type request?
> >
> > -Prachi
> >
> > On Mon, Dec 4, 2017 at 1:06 PM, Andrey Kornev <[hidden email]>
> > wrote:
> >
> >> Pavel,
> >>
> >> Thanks! While we're at it, are there any plans to add cluster-related
> >> operations? For example, I think it'd be nice to allow the thin clients
> to
> >> obtain a current topology snapshot. This would make it possible the
> clients
> >> to send requests directly to the affinity host for colocated
> computation.
> >> To make it even more useful, all server responses could optionally
> include
> >> the topology version the operation has been executed against. This would
> >> effectively give us a kind out-of-band topology change notification
> >> mechanism. This way the clients can detect a topology change and refresh
> >> the topology snapshot next time they need to compute affinity.
> >>
> >> Regards
> >> Andrey
> >> ________________________________
> >> From: Pavel Tupitsyn <[hidden email]>
> >> Sent: Sunday, December 3, 2017 9:23 AM
> >> To: [hidden email]
> >> Subject: Re: Thin Client Protocol documentation
> >>
> >> Hi Andrey,
> >>
> >> Compute and other APIs are certainly planned, cache is just a start.
> >> We intentionally limit the scope to actually release something in 2.4
> and
> >> not delay it further.
> >>
> >> Adding operations to existing protocol is relatively easy.
> >> Current focus is to make sure that the protocol itself is solid and
> >> future-proof.
> >>
> >> Thanks,
> >> Pavel
> >>
> >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Thin Client Protocol documentation

dmagda
Hi Andrey,

I’m afraid there are no plans for this features yet (at least I didn’t spot any relevant discussion on @dev).

Feel free to start a separate discussion proposing the protocol expansion. It’s better not to intermix this with the documentation thread.


Denis

> On Dec 15, 2017, at 11:12 AM, Andrey Kornev <[hidden email]> wrote:
>
> Pavel, could you please at least share your time frame wrt adding compute, clustering and services features to the spec?
>
> Thanks
> Andrey
> ________________________________
> From: Pavel Tupitsyn <[hidden email]>
> Sent: Tuesday, December 5, 2017 11:15 PM
> To: [hidden email]
> Subject: Re: Thin Client Protocol documentation
>
> Andrey,
>
> As I understand, you suggest to document every prospective feature right
> now.
> That would be (at least) compute, clustering, transactions, services,
> messages, events, failover, data structures, metrics.
>
> I don't think it makes sense to do that. Another problem is that it will
> take weeks :)
> But, of course, you are welcome to take the initiative.
>
> Thanks,
> Pavel
>
> On Tue, Dec 5, 2017 at 10:20 PM, Andrey Kornev <[hidden email]>
> wrote:
>
>> Pavel,
>>
>> I have absolutely no doubts that support for all those features will be
>> added eventually. And if so, wouldn't it be the right thing to do to
>> document the operations and their semantics right now without
>> necessarily implementing them? It should really help to ensure that the
>> protocol can accommodate all those use cases before it gets released to the
>> public.
>>
>> Thanks
>> Andrey
>> ------------------------------
>> *From:* Pavel Tupitsyn <[hidden email]>
>> *Sent:* Tuesday, December 5, 2017 12:07 AM
>> *Cc:* [hidden email]
>> *Subject:* Re: Thin Client Protocol documentation
>>
>> Andrey,
>>
>> All of this is to come :)
>>
>>
>> Prachi,
>>
>> 1) There are no flags, see the doc
>> 2) String is simply 4 byte length (n) + n bytes
>> 3) Op codes have changed
>>
>> writeByteLittleEndian(1051, out); // Type code
>> writeIntLittleEndian(20, out); // String length
>> out.writeUTF("myNewCache"); // Cache name
>>
>> On Tue, Dec 5, 2017 at 4:39 AM, Prachi Garg <[hidden email]> wrote:
>>
>>> Hi Pavel,
>>>
>>> I am trying to use the OP_CACHE_CREATE_WITH_NAME operation, but can't get
>>> it to work.  Digging deeper into the source code, it seems like I have to
>>> provide a flag, string length, and position, in addition to the type code
>>> and the actual string. Is that correct?
>>>
>>> Here is the request I am sending to the server-
>>>
>>> DataOutputStream out = new DataOutputStream(socket.getOutputStream());
>>>
>>> // Message length
>>> writeIntLittleEndian(24, out);
>>>
>>> // Op code = OP_CACHE_CREATE_WITH_NAME
>>> writeShortLittleEndian(1051, out);
>>>
>>> // Request id
>>> long reqId = 1;
>>> writeLongLittleEndian(reqId, out);
>>>
>>> // String (cache name)
>>> writeByteLittleEndian(9, out); // Type code
>>> writeByteLittleEndian(0, out); // Flag
>>> writeIntLittleEndian(20, out); // String length
>>> writeIntLittleEndian(0, out); // Position
>>> out.writeUTF("myNewCache"); // Cache name
>>>
>>> // Send request
>>> out.flush();
>>>
>>> But I get the following error on the server side.
>>>
>>> [2017-12-04 17:27:39,421][ERROR][client-connector-#53][
>> ClientListenerNioListener]
>>> Failed to parse client request.
>>> java.lang.StringIndexOutOfBoundsException: String index out of range:
>> 2575
>>> at java.lang.String.checkBounds(String.java:385)
>>> at java.lang.String.<init>(String.java:462)
>>> at org.apache.ignite.internal.binary.BinaryUtils.
>>> doReadString(BinaryUtils.java:1314)
>>> at org.apache.ignite.internal.binary.BinaryReaderExImpl.
>>> readString(BinaryReaderExImpl.java:1055)
>>> at org.apache.ignite.internal.processors.platform.client.cache.
>>> ClientCacheCreateWithNameRequest.<init>(ClientCacheCreateWithNameReque
>>> st.java:43)
>>> at org.apache.ignite.internal.processors.platform.client.
>>> ClientMessageParser.decode(ClientMessageParser.java:318)
>>> at org.apache.ignite.internal.processors.platform.client.
>>> ClientMessageParser.decode(ClientMessageParser.java:220)
>>> at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.
>>> onMessage(ClientListenerNioListener.java:119)
>>> at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.
>>> onMessage(ClientListenerNioListener.java:40)
>>> at org.apache.ignite.internal.util.nio.GridNioFilterChain$
>>> TailFilter.onMessageReceived(GridNioFilterChain.java:279)
>>> at org.apache.ignite.internal.util.nio.GridNioFilterAdapter.
>>> proceedMessageReceived(GridNioFilterAdapter.java:109)
>>> at org.apache.ignite.internal.util.nio.GridNioAsyncNotifyFilter$3.
>>> body(GridNioAsyncNotifyFilter.java:97)
>>> at org.apache.ignite.internal.util.worker.GridWorker.run(
>>> GridWorker.java:110)
>>> at org.apache.ignite.internal.util.worker.GridWorkerPool$1.
>>> run(GridWorkerPool.java:70)
>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(
>>> ThreadPoolExecutor.java:1142)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
>>> ThreadPoolExecutor.java:617)
>>> at java.lang.Thread.run(Thread.java:745)
>>>
>>> What am I missing here? Can you provide an example of how to send a
>>> 'String' type request?
>>>
>>> -Prachi
>>>
>>> On Mon, Dec 4, 2017 at 1:06 PM, Andrey Kornev <[hidden email]>
>>> wrote:
>>>
>>>> Pavel,
>>>>
>>>> Thanks! While we're at it, are there any plans to add cluster-related
>>>> operations? For example, I think it'd be nice to allow the thin clients
>> to
>>>> obtain a current topology snapshot. This would make it possible the
>> clients
>>>> to send requests directly to the affinity host for colocated
>> computation.
>>>> To make it even more useful, all server responses could optionally
>> include
>>>> the topology version the operation has been executed against. This would
>>>> effectively give us a kind out-of-band topology change notification
>>>> mechanism. This way the clients can detect a topology change and refresh
>>>> the topology snapshot next time they need to compute affinity.
>>>>
>>>> Regards
>>>> Andrey
>>>> ________________________________
>>>> From: Pavel Tupitsyn <[hidden email]>
>>>> Sent: Sunday, December 3, 2017 9:23 AM
>>>> To: [hidden email]
>>>> Subject: Re: Thin Client Protocol documentation
>>>>
>>>> Hi Andrey,
>>>>
>>>> Compute and other APIs are certainly planned, cache is just a start.
>>>> We intentionally limit the scope to actually release something in 2.4
>> and
>>>> not delay it further.
>>>>
>>>> Adding operations to existing protocol is relatively easy.
>>>> Current focus is to make sure that the protocol itself is solid and
>>>> future-proof.
>>>>
>>>> Thanks,
>>>> Pavel
>>>>
>>>
>>>
>>

12