Igniters,
Can someone explain to me how Ignite executes SQL from JDBC and ODBC drivers? Do we start an Ignite client node on the driver side? Or do we use some other protocol to send commands to one of the Ignite nodes? Thanks, D. |
Dmitriy,
JDBC drivers spawns an Ignite client node and uses it for cluster connectivity and queries execution. Queries issued over the JDBC are turned into SqlFieldsQueries and sent to the cluster in this form. ODBC driver works in a bit different way. It connects to the cluster via ODBC processor that needs to be running on one of the nodes: https://apacheignite.readme.io/docs/odbc-driver#cluster-configuration <https://apacheignite.readme.io/docs/odbc-driver#cluster-configuration> — Denis > On Dec 16, 2016, at 2:41 PM, Dmitriy Setrakyan <[hidden email]> wrote: > > Igniters, > > Can someone explain to me how Ignite executes SQL from JDBC and ODBC > drivers? Do we start an Ignite client node on the driver side? Or do we use > some other protocol to send commands to one of the Ignite nodes? > > Thanks, > D. |
To my understanding, we are implementing JDBC batches by sending a callable
to another node. If we already have a client node on the JDBC driver side, why not just issue a putAll(...) call from the client? D. On Fri, Dec 16, 2016 at 3:02 PM, Denis Magda <[hidden email]> wrote: > Dmitriy, > > JDBC drivers spawns an Ignite client node and uses it for cluster > connectivity and queries execution. Queries issued over the JDBC are turned > into SqlFieldsQueries and sent to the cluster in this form. > > ODBC driver works in a bit different way. It connects to the cluster via > ODBC processor that needs to be running on one of the nodes: > https://apacheignite.readme.io/docs/odbc-driver#cluster-configuration < > https://apacheignite.readme.io/docs/odbc-driver#cluster-configuration> > > — > Denis > > > On Dec 16, 2016, at 2:41 PM, Dmitriy Setrakyan <[hidden email]> > wrote: > > > > Igniters, > > > > Can someone explain to me how Ignite executes SQL from JDBC and ODBC > > drivers? Do we start an Ignite client node on the driver side? Or do we > use > > some other protocol to send commands to one of the Ignite nodes? > > > > Thanks, > > D. > > |
Frankly speaking, even single (non batched) updates or queries are sent as callables. This is what I see in the code.
No idea what was the reason behind this design. Andrey G., Alex P. could you shed a light on this? — Denis > On Dec 16, 2016, at 3:08 PM, Dmitriy Setrakyan <[hidden email]> wrote: > > To my understanding, we are implementing JDBC batches by sending a callable > to another node. If we already have a client node on the JDBC driver side, > why not just issue a putAll(...) call from the client? > > D. > > On Fri, Dec 16, 2016 at 3:02 PM, Denis Magda <[hidden email]> wrote: > >> Dmitriy, >> >> JDBC drivers spawns an Ignite client node and uses it for cluster >> connectivity and queries execution. Queries issued over the JDBC are turned >> into SqlFieldsQueries and sent to the cluster in this form. >> >> ODBC driver works in a bit different way. It connects to the cluster via >> ODBC processor that needs to be running on one of the nodes: >> https://apacheignite.readme.io/docs/odbc-driver#cluster-configuration < >> https://apacheignite.readme.io/docs/odbc-driver#cluster-configuration> >> >> — >> Denis >> >>> On Dec 16, 2016, at 2:41 PM, Dmitriy Setrakyan <[hidden email]> >> wrote: >>> >>> Igniters, >>> >>> Can someone explain to me how Ignite executes SQL from JDBC and ODBC >>> drivers? Do we start an Ignite client node on the driver side? Or do we >> use >>> some other protocol to send commands to one of the Ignite nodes? >>> >>> Thanks, >>> D. >> >> |
I'm not sure about updates, but can tell about how selects are implemented
there. Basically, there is an option to execute the query on a particular node specified by ignite.jdbc.nodeId property. Not sure why we need this though, probably it's just leftover from the legacy version of the driver based on thin client. If the property is set, the callable is sent to a remote node. But if it is not, the same callable is created, but it is invoked directly on the embedded client which is the behavior that you expect. And it's the default one. -Val On Fri, Dec 16, 2016 at 7:51 PM, Denis Magda <[hidden email]> wrote: > Frankly speaking, even single (non batched) updates or queries are sent as > callables. This is what I see in the code. > No idea what was the reason behind this design. > > Andrey G., Alex P. could you shed a light on this? > > — > Denis > > > On Dec 16, 2016, at 3:08 PM, Dmitriy Setrakyan <[hidden email]> > wrote: > > > > To my understanding, we are implementing JDBC batches by sending a > callable > > to another node. If we already have a client node on the JDBC driver > side, > > why not just issue a putAll(...) call from the client? > > > > D. > > > > On Fri, Dec 16, 2016 at 3:02 PM, Denis Magda <[hidden email]> wrote: > > > >> Dmitriy, > >> > >> JDBC drivers spawns an Ignite client node and uses it for cluster > >> connectivity and queries execution. Queries issued over the JDBC are > turned > >> into SqlFieldsQueries and sent to the cluster in this form. > >> > >> ODBC driver works in a bit different way. It connects to the cluster via > >> ODBC processor that needs to be running on one of the nodes: > >> https://apacheignite.readme.io/docs/odbc-driver#cluster-configuration < > >> https://apacheignite.readme.io/docs/odbc-driver#cluster-configuration> > >> > >> — > >> Denis > >> > >>> On Dec 16, 2016, at 2:41 PM, Dmitriy Setrakyan <[hidden email]> > >> wrote: > >>> > >>> Igniters, > >>> > >>> Can someone explain to me how Ignite executes SQL from JDBC and ODBC > >>> drivers? Do we start an Ignite client node on the driver side? Or do we > >> use > >>> some other protocol to send commands to one of the Ignite nodes? > >>> > >>> Thanks, > >>> D. > >> > >> > > |
On Fri, Dec 16, 2016 at 9:53 PM, Valentin Kulichenko <
[hidden email]> wrote: > I'm not sure about updates, but can tell about how selects are implemented > there. Basically, there is an option to execute the query on a particular > node specified by ignite.jdbc.nodeId property. Not sure why we need this > though, probably it's just leftover from the legacy version of the driver > based on thin client. > > If the property is set, the callable is sent to a remote node. But if it is > not, the same callable is created, but it is invoked directly on the > embedded client which is the behavior that you expect. And it's the default > one. > > think it makes sense and it significantly slows down the implementation. > -Val > > On Fri, Dec 16, 2016 at 7:51 PM, Denis Magda <[hidden email]> wrote: > > > Frankly speaking, even single (non batched) updates or queries are sent > as > > callables. This is what I see in the code. > > No idea what was the reason behind this design. > > > > Andrey G., Alex P. could you shed a light on this? > > > > — > > Denis > > > > > On Dec 16, 2016, at 3:08 PM, Dmitriy Setrakyan <[hidden email]> > > wrote: > > > > > > To my understanding, we are implementing JDBC batches by sending a > > callable > > > to another node. If we already have a client node on the JDBC driver > > side, > > > why not just issue a putAll(...) call from the client? > > > > > > D. > > > > > > On Fri, Dec 16, 2016 at 3:02 PM, Denis Magda <[hidden email]> > wrote: > > > > > >> Dmitriy, > > >> > > >> JDBC drivers spawns an Ignite client node and uses it for cluster > > >> connectivity and queries execution. Queries issued over the JDBC are > > turned > > >> into SqlFieldsQueries and sent to the cluster in this form. > > >> > > >> ODBC driver works in a bit different way. It connects to the cluster > via > > >> ODBC processor that needs to be running on one of the nodes: > > >> https://apacheignite.readme.io/docs/odbc-driver#cluster-configuration > < > > >> https://apacheignite.readme.io/docs/odbc-driver#cluster-configuration > > > > >> > > >> — > > >> Denis > > >> > > >>> On Dec 16, 2016, at 2:41 PM, Dmitriy Setrakyan < > [hidden email]> > > >> wrote: > > >>> > > >>> Igniters, > > >>> > > >>> Can someone explain to me how Ignite executes SQL from JDBC and ODBC > > >>> drivers? Do we start an Ignite client node on the driver side? Or do > we > > >> use > > >>> some other protocol to send commands to one of the Ignite nodes? > > >>> > > >>> Thanks, > > >>> D. > > >> > > >> > > > > > |
Dima, Val,
Introduction of updates has not changed public API at all (only configuration, in some cases), so they work in this respect exactly like SELECTs - by default they run on client node started by the driver itself, but can be sent via the same callables mechanism to any remote node by its id. So Dima, you're right, currently it's possible to send query to any given node. And, at the same time, currently by default everything works exactly like you want it to - that is, any MERGE, batched or not, boils down to putAll, and by default this call happens locally. - Alex 2016-12-17 18:47 GMT+03:00 Dmitriy Setrakyan <[hidden email]>: > On Fri, Dec 16, 2016 at 9:53 PM, Valentin Kulichenko < > [hidden email]> wrote: > >> I'm not sure about updates, but can tell about how selects are implemented >> there. Basically, there is an option to execute the query on a particular >> node specified by ignite.jdbc.nodeId property. Not sure why we need this >> though, probably it's just leftover from the legacy version of the driver >> based on thin client. >> >> If the property is set, the callable is sent to a remote node. But if it is >> not, the same callable is created, but it is invoked directly on the >> embedded client which is the behavior that you expect. And it's the default >> one. >> >> > Ouch. If this is the reason, I would drop the nodeId property. I don't > think it makes sense and it significantly slows down the implementation. > > >> -Val >> >> On Fri, Dec 16, 2016 at 7:51 PM, Denis Magda <[hidden email]> wrote: >> >> > Frankly speaking, even single (non batched) updates or queries are sent >> as >> > callables. This is what I see in the code. >> > No idea what was the reason behind this design. >> > >> > Andrey G., Alex P. could you shed a light on this? >> > >> > — >> > Denis >> > >> > > On Dec 16, 2016, at 3:08 PM, Dmitriy Setrakyan <[hidden email]> >> > wrote: >> > > >> > > To my understanding, we are implementing JDBC batches by sending a >> > callable >> > > to another node. If we already have a client node on the JDBC driver >> > side, >> > > why not just issue a putAll(...) call from the client? >> > > >> > > D. >> > > >> > > On Fri, Dec 16, 2016 at 3:02 PM, Denis Magda <[hidden email]> >> wrote: >> > > >> > >> Dmitriy, >> > >> >> > >> JDBC drivers spawns an Ignite client node and uses it for cluster >> > >> connectivity and queries execution. Queries issued over the JDBC are >> > turned >> > >> into SqlFieldsQueries and sent to the cluster in this form. >> > >> >> > >> ODBC driver works in a bit different way. It connects to the cluster >> via >> > >> ODBC processor that needs to be running on one of the nodes: >> > >> https://apacheignite.readme.io/docs/odbc-driver#cluster-configuration >> < >> > >> https://apacheignite.readme.io/docs/odbc-driver#cluster-configuration >> > >> > >> >> > >> — >> > >> Denis >> > >> >> > >>> On Dec 16, 2016, at 2:41 PM, Dmitriy Setrakyan < >> [hidden email]> >> > >> wrote: >> > >>> >> > >>> Igniters, >> > >>> >> > >>> Can someone explain to me how Ignite executes SQL from JDBC and ODBC >> > >>> drivers? Do we start an Ignite client node on the driver side? Or do >> we >> > >> use >> > >>> some other protocol to send commands to one of the Ignite nodes? >> > >>> >> > >>> Thanks, >> > >>> D. >> > >> >> > >> >> > >> > >> |
Thanks, Alex.
Can you clarify if the putAll(...) call is made directly, or from locally executed callable? According to Denis, "even single (non batched) updates or queries are sent as callables", which should be fixed in my view. D. On Mon, Dec 19, 2016 at 3:32 AM, Alexander Paschenko < [hidden email]> wrote: > Dima, Val, > > Introduction of updates has not changed public API at all (only > configuration, in some cases), so they work in this respect exactly > like SELECTs - by default they run on client node started by the > driver itself, but can be sent via the same callables mechanism to any > remote node by its id. > > So Dima, you're right, currently it's possible to send query to any > given node. And, at the same time, currently by default everything > works exactly like you want it to - that is, any MERGE, batched or > not, boils down to putAll, and by default this call happens locally. > > - Alex > > 2016-12-17 18:47 GMT+03:00 Dmitriy Setrakyan <[hidden email]>: > > On Fri, Dec 16, 2016 at 9:53 PM, Valentin Kulichenko < > > [hidden email]> wrote: > > > >> I'm not sure about updates, but can tell about how selects are > implemented > >> there. Basically, there is an option to execute the query on a > particular > >> node specified by ignite.jdbc.nodeId property. Not sure why we need this > >> though, probably it's just leftover from the legacy version of the > driver > >> based on thin client. > >> > >> If the property is set, the callable is sent to a remote node. But if > it is > >> not, the same callable is created, but it is invoked directly on the > >> embedded client which is the behavior that you expect. And it's the > default > >> one. > >> > >> > > Ouch. If this is the reason, I would drop the nodeId property. I don't > > think it makes sense and it significantly slows down the implementation. > > > > > >> -Val > >> > >> On Fri, Dec 16, 2016 at 7:51 PM, Denis Magda <[hidden email]> wrote: > >> > >> > Frankly speaking, even single (non batched) updates or queries are > sent > >> as > >> > callables. This is what I see in the code. > >> > No idea what was the reason behind this design. > >> > > >> > Andrey G., Alex P. could you shed a light on this? > >> > > >> > — > >> > Denis > >> > > >> > > On Dec 16, 2016, at 3:08 PM, Dmitriy Setrakyan < > [hidden email]> > >> > wrote: > >> > > > >> > > To my understanding, we are implementing JDBC batches by sending a > >> > callable > >> > > to another node. If we already have a client node on the JDBC driver > >> > side, > >> > > why not just issue a putAll(...) call from the client? > >> > > > >> > > D. > >> > > > >> > > On Fri, Dec 16, 2016 at 3:02 PM, Denis Magda <[hidden email]> > >> wrote: > >> > > > >> > >> Dmitriy, > >> > >> > >> > >> JDBC drivers spawns an Ignite client node and uses it for cluster > >> > >> connectivity and queries execution. Queries issued over the JDBC > are > >> > turned > >> > >> into SqlFieldsQueries and sent to the cluster in this form. > >> > >> > >> > >> ODBC driver works in a bit different way. It connects to the > cluster > >> via > >> > >> ODBC processor that needs to be running on one of the nodes: > >> > >> https://apacheignite.readme.io/docs/odbc-driver#cluster- > configuration > >> < > >> > >> https://apacheignite.readme.io/docs/odbc-driver#cluster- > configuration > >> > > >> > >> > >> > >> — > >> > >> Denis > >> > >> > >> > >>> On Dec 16, 2016, at 2:41 PM, Dmitriy Setrakyan < > >> [hidden email]> > >> > >> wrote: > >> > >>> > >> > >>> Igniters, > >> > >>> > >> > >>> Can someone explain to me how Ignite executes SQL from JDBC and > ODBC > >> > >>> drivers? Do we start an Ignite client node on the driver side? Or > do > >> we > >> > >> use > >> > >>> some other protocol to send commands to one of the Ignite nodes? > >> > >>> > >> > >>> Thanks, > >> > >>> D. > >> > >> > >> > >> > >> > > >> > > >> > |
Dmitriy,
According to Val and Alex P. explanations this happens when a specific node id is set. I got confused by the code flow initially. — Denis > On Dec 19, 2016, at 9:13 AM, Dmitriy Setrakyan <[hidden email]> wrote: > > Thanks, Alex. > > Can you clarify if the putAll(...) call is made directly, or from locally > executed callable? According to Denis, "even single (non batched) updates > or queries are sent as callables", which should be fixed in my view. > > D. > > On Mon, Dec 19, 2016 at 3:32 AM, Alexander Paschenko < > [hidden email]> wrote: > >> Dima, Val, >> >> Introduction of updates has not changed public API at all (only >> configuration, in some cases), so they work in this respect exactly >> like SELECTs - by default they run on client node started by the >> driver itself, but can be sent via the same callables mechanism to any >> remote node by its id. >> >> So Dima, you're right, currently it's possible to send query to any >> given node. And, at the same time, currently by default everything >> works exactly like you want it to - that is, any MERGE, batched or >> not, boils down to putAll, and by default this call happens locally. >> >> - Alex >> >> 2016-12-17 18:47 GMT+03:00 Dmitriy Setrakyan <[hidden email]>: >>> On Fri, Dec 16, 2016 at 9:53 PM, Valentin Kulichenko < >>> [hidden email]> wrote: >>> >>>> I'm not sure about updates, but can tell about how selects are >> implemented >>>> there. Basically, there is an option to execute the query on a >> particular >>>> node specified by ignite.jdbc.nodeId property. Not sure why we need this >>>> though, probably it's just leftover from the legacy version of the >> driver >>>> based on thin client. >>>> >>>> If the property is set, the callable is sent to a remote node. But if >> it is >>>> not, the same callable is created, but it is invoked directly on the >>>> embedded client which is the behavior that you expect. And it's the >> default >>>> one. >>>> >>>> >>> Ouch. If this is the reason, I would drop the nodeId property. I don't >>> think it makes sense and it significantly slows down the implementation. >>> >>> >>>> -Val >>>> >>>> On Fri, Dec 16, 2016 at 7:51 PM, Denis Magda <[hidden email]> wrote: >>>> >>>>> Frankly speaking, even single (non batched) updates or queries are >> sent >>>> as >>>>> callables. This is what I see in the code. >>>>> No idea what was the reason behind this design. >>>>> >>>>> Andrey G., Alex P. could you shed a light on this? >>>>> >>>>> — >>>>> Denis >>>>> >>>>>> On Dec 16, 2016, at 3:08 PM, Dmitriy Setrakyan < >> [hidden email]> >>>>> wrote: >>>>>> >>>>>> To my understanding, we are implementing JDBC batches by sending a >>>>> callable >>>>>> to another node. If we already have a client node on the JDBC driver >>>>> side, >>>>>> why not just issue a putAll(...) call from the client? >>>>>> >>>>>> D. >>>>>> >>>>>> On Fri, Dec 16, 2016 at 3:02 PM, Denis Magda <[hidden email]> >>>> wrote: >>>>>> >>>>>>> Dmitriy, >>>>>>> >>>>>>> JDBC drivers spawns an Ignite client node and uses it for cluster >>>>>>> connectivity and queries execution. Queries issued over the JDBC >> are >>>>> turned >>>>>>> into SqlFieldsQueries and sent to the cluster in this form. >>>>>>> >>>>>>> ODBC driver works in a bit different way. It connects to the >> cluster >>>> via >>>>>>> ODBC processor that needs to be running on one of the nodes: >>>>>>> https://apacheignite.readme.io/docs/odbc-driver#cluster- >> configuration >>>> < >>>>>>> https://apacheignite.readme.io/docs/odbc-driver#cluster- >> configuration >>>>> >>>>>>> >>>>>>> — >>>>>>> Denis >>>>>>> >>>>>>>> On Dec 16, 2016, at 2:41 PM, Dmitriy Setrakyan < >>>> [hidden email]> >>>>>>> wrote: >>>>>>>> >>>>>>>> Igniters, >>>>>>>> >>>>>>>> Can someone explain to me how Ignite executes SQL from JDBC and >> ODBC >>>>>>>> drivers? Do we start an Ignite client node on the driver side? Or >> do >>>> we >>>>>>> use >>>>>>>> some other protocol to send commands to one of the Ignite nodes? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> D. >>>>>>> >>>>>>> >>>>> >>>>> >>>> >> |
When node is local, no network interaction occurs on query send. Would be
shame otherwise :) — Alex 19 дек. 2016 г. 8:36 PM пользователь "Denis Magda" <[hidden email]> написал: > Dmitriy, > > According to Val and Alex P. explanations this happens when a specific > node id is set. I got confused by the code flow initially. > > — > Denis > > > On Dec 19, 2016, at 9:13 AM, Dmitriy Setrakyan <[hidden email]> > wrote: > > > > Thanks, Alex. > > > > Can you clarify if the putAll(...) call is made directly, or from locally > > executed callable? According to Denis, "even single (non batched) updates > > or queries are sent as callables", which should be fixed in my view. > > > > D. > > > > On Mon, Dec 19, 2016 at 3:32 AM, Alexander Paschenko < > > [hidden email]> wrote: > > > >> Dima, Val, > >> > >> Introduction of updates has not changed public API at all (only > >> configuration, in some cases), so they work in this respect exactly > >> like SELECTs - by default they run on client node started by the > >> driver itself, but can be sent via the same callables mechanism to any > >> remote node by its id. > >> > >> So Dima, you're right, currently it's possible to send query to any > >> given node. And, at the same time, currently by default everything > >> works exactly like you want it to - that is, any MERGE, batched or > >> not, boils down to putAll, and by default this call happens locally. > >> > >> - Alex > >> > >> 2016-12-17 18:47 GMT+03:00 Dmitriy Setrakyan <[hidden email]>: > >>> On Fri, Dec 16, 2016 at 9:53 PM, Valentin Kulichenko < > >>> [hidden email]> wrote: > >>> > >>>> I'm not sure about updates, but can tell about how selects are > >> implemented > >>>> there. Basically, there is an option to execute the query on a > >> particular > >>>> node specified by ignite.jdbc.nodeId property. Not sure why we need > this > >>>> though, probably it's just leftover from the legacy version of the > >> driver > >>>> based on thin client. > >>>> > >>>> If the property is set, the callable is sent to a remote node. But if > >> it is > >>>> not, the same callable is created, but it is invoked directly on the > >>>> embedded client which is the behavior that you expect. And it's the > >> default > >>>> one. > >>>> > >>>> > >>> Ouch. If this is the reason, I would drop the nodeId property. I don't > >>> think it makes sense and it significantly slows down the > implementation. > >>> > >>> > >>>> -Val > >>>> > >>>> On Fri, Dec 16, 2016 at 7:51 PM, Denis Magda <[hidden email]> > wrote: > >>>> > >>>>> Frankly speaking, even single (non batched) updates or queries are > >> sent > >>>> as > >>>>> callables. This is what I see in the code. > >>>>> No idea what was the reason behind this design. > >>>>> > >>>>> Andrey G., Alex P. could you shed a light on this? > >>>>> > >>>>> — > >>>>> Denis > >>>>> > >>>>>> On Dec 16, 2016, at 3:08 PM, Dmitriy Setrakyan < > >> [hidden email]> > >>>>> wrote: > >>>>>> > >>>>>> To my understanding, we are implementing JDBC batches by sending a > >>>>> callable > >>>>>> to another node. If we already have a client node on the JDBC driver > >>>>> side, > >>>>>> why not just issue a putAll(...) call from the client? > >>>>>> > >>>>>> D. > >>>>>> > >>>>>> On Fri, Dec 16, 2016 at 3:02 PM, Denis Magda <[hidden email]> > >>>> wrote: > >>>>>> > >>>>>>> Dmitriy, > >>>>>>> > >>>>>>> JDBC drivers spawns an Ignite client node and uses it for cluster > >>>>>>> connectivity and queries execution. Queries issued over the JDBC > >> are > >>>>> turned > >>>>>>> into SqlFieldsQueries and sent to the cluster in this form. > >>>>>>> > >>>>>>> ODBC driver works in a bit different way. It connects to the > >> cluster > >>>> via > >>>>>>> ODBC processor that needs to be running on one of the nodes: > >>>>>>> https://apacheignite.readme.io/docs/odbc-driver#cluster- > >> configuration > >>>> < > >>>>>>> https://apacheignite.readme.io/docs/odbc-driver#cluster- > >> configuration > >>>>> > >>>>>>> > >>>>>>> — > >>>>>>> Denis > >>>>>>> > >>>>>>>> On Dec 16, 2016, at 2:41 PM, Dmitriy Setrakyan < > >>>> [hidden email]> > >>>>>>> wrote: > >>>>>>>> > >>>>>>>> Igniters, > >>>>>>>> > >>>>>>>> Can someone explain to me how Ignite executes SQL from JDBC and > >> ODBC > >>>>>>>> drivers? Do we start an Ignite client node on the driver side? Or > >> do > >>>> we > >>>>>>> use > >>>>>>>> some other protocol to send commands to one of the Ignite nodes? > >>>>>>>> > >>>>>>>> Thanks, > >>>>>>>> D. > >>>>>>> > >>>>>>> > >>>>> > >>>>> > >>>> > >> > > |
By "locally executed callable" I meant something like this:
new JdbcCallable(...).call(); So it's not a network call and it's not even going through IgniteCompute. It's just the logic is incapsulated in the callable to support remote execution of the same logic. But anyway, +1 to removing nodeId property. It doesn't make much sense with the new implementation of JDBC. -Val On Mon, Dec 19, 2016 at 10:15 AM, Alexander Paschenko < [hidden email]> wrote: > When node is local, no network interaction occurs on query send. Would be > shame otherwise :) > > — Alex > 19 дек. 2016 г. 8:36 PM пользователь "Denis Magda" <[hidden email]> > написал: > > > Dmitriy, > > > > According to Val and Alex P. explanations this happens when a specific > > node id is set. I got confused by the code flow initially. > > > > — > > Denis > > > > > On Dec 19, 2016, at 9:13 AM, Dmitriy Setrakyan <[hidden email]> > > wrote: > > > > > > Thanks, Alex. > > > > > > Can you clarify if the putAll(...) call is made directly, or from > locally > > > executed callable? According to Denis, "even single (non batched) > updates > > > or queries are sent as callables", which should be fixed in my view. > > > > > > D. > > > > > > On Mon, Dec 19, 2016 at 3:32 AM, Alexander Paschenko < > > > [hidden email]> wrote: > > > > > >> Dima, Val, > > >> > > >> Introduction of updates has not changed public API at all (only > > >> configuration, in some cases), so they work in this respect exactly > > >> like SELECTs - by default they run on client node started by the > > >> driver itself, but can be sent via the same callables mechanism to any > > >> remote node by its id. > > >> > > >> So Dima, you're right, currently it's possible to send query to any > > >> given node. And, at the same time, currently by default everything > > >> works exactly like you want it to - that is, any MERGE, batched or > > >> not, boils down to putAll, and by default this call happens locally. > > >> > > >> - Alex > > >> > > >> 2016-12-17 18:47 GMT+03:00 Dmitriy Setrakyan <[hidden email]>: > > >>> On Fri, Dec 16, 2016 at 9:53 PM, Valentin Kulichenko < > > >>> [hidden email]> wrote: > > >>> > > >>>> I'm not sure about updates, but can tell about how selects are > > >> implemented > > >>>> there. Basically, there is an option to execute the query on a > > >> particular > > >>>> node specified by ignite.jdbc.nodeId property. Not sure why we need > > this > > >>>> though, probably it's just leftover from the legacy version of the > > >> driver > > >>>> based on thin client. > > >>>> > > >>>> If the property is set, the callable is sent to a remote node. But > if > > >> it is > > >>>> not, the same callable is created, but it is invoked directly on the > > >>>> embedded client which is the behavior that you expect. And it's the > > >> default > > >>>> one. > > >>>> > > >>>> > > >>> Ouch. If this is the reason, I would drop the nodeId property. I > don't > > >>> think it makes sense and it significantly slows down the > > implementation. > > >>> > > >>> > > >>>> -Val > > >>>> > > >>>> On Fri, Dec 16, 2016 at 7:51 PM, Denis Magda <[hidden email]> > > wrote: > > >>>> > > >>>>> Frankly speaking, even single (non batched) updates or queries are > > >> sent > > >>>> as > > >>>>> callables. This is what I see in the code. > > >>>>> No idea what was the reason behind this design. > > >>>>> > > >>>>> Andrey G., Alex P. could you shed a light on this? > > >>>>> > > >>>>> — > > >>>>> Denis > > >>>>> > > >>>>>> On Dec 16, 2016, at 3:08 PM, Dmitriy Setrakyan < > > >> [hidden email]> > > >>>>> wrote: > > >>>>>> > > >>>>>> To my understanding, we are implementing JDBC batches by sending a > > >>>>> callable > > >>>>>> to another node. If we already have a client node on the JDBC > driver > > >>>>> side, > > >>>>>> why not just issue a putAll(...) call from the client? > > >>>>>> > > >>>>>> D. > > >>>>>> > > >>>>>> On Fri, Dec 16, 2016 at 3:02 PM, Denis Magda <[hidden email]> > > >>>> wrote: > > >>>>>> > > >>>>>>> Dmitriy, > > >>>>>>> > > >>>>>>> JDBC drivers spawns an Ignite client node and uses it for cluster > > >>>>>>> connectivity and queries execution. Queries issued over the JDBC > > >> are > > >>>>> turned > > >>>>>>> into SqlFieldsQueries and sent to the cluster in this form. > > >>>>>>> > > >>>>>>> ODBC driver works in a bit different way. It connects to the > > >> cluster > > >>>> via > > >>>>>>> ODBC processor that needs to be running on one of the nodes: > > >>>>>>> https://apacheignite.readme.io/docs/odbc-driver#cluster- > > >> configuration > > >>>> < > > >>>>>>> https://apacheignite.readme.io/docs/odbc-driver#cluster- > > >> configuration > > >>>>> > > >>>>>>> > > >>>>>>> — > > >>>>>>> Denis > > >>>>>>> > > >>>>>>>> On Dec 16, 2016, at 2:41 PM, Dmitriy Setrakyan < > > >>>> [hidden email]> > > >>>>>>> wrote: > > >>>>>>>> > > >>>>>>>> Igniters, > > >>>>>>>> > > >>>>>>>> Can someone explain to me how Ignite executes SQL from JDBC and > > >> ODBC > > >>>>>>>> drivers? Do we start an Ignite client node on the driver side? > Or > > >> do > > >>>> we > > >>>>>>> use > > >>>>>>>> some other protocol to send commands to one of the Ignite nodes? > > >>>>>>>> > > >>>>>>>> Thanks, > > >>>>>>>> D. > > >>>>>>> > > >>>>>>> > > >>>>> > > >>>>> > > >>>> > > >> > > > > > |
Alex,
What Val and I are suggesting is that we get rid of any JdbcCallable calls and invoke putAll(...) directly. Wrapping it into JdbcCallable will slow us down and introduce unnecessary thread context switch. Do you agree? D. On Mon, Dec 19, 2016 at 11:01 AM, Valentin Kulichenko < [hidden email]> wrote: > By "locally executed callable" I meant something like this: > > new JdbcCallable(...).call(); > > So it's not a network call and it's not even going through IgniteCompute. > It's just the logic is incapsulated in the callable to support remote > execution of the same logic. > > But anyway, +1 to removing nodeId property. It doesn't make much sense with > the new implementation of JDBC. > > -Val > > On Mon, Dec 19, 2016 at 10:15 AM, Alexander Paschenko < > [hidden email]> wrote: > > > When node is local, no network interaction occurs on query send. Would be > > shame otherwise :) > > > > — Alex > > 19 дек. 2016 г. 8:36 PM пользователь "Denis Magda" <[hidden email]> > > написал: > > > > > Dmitriy, > > > > > > According to Val and Alex P. explanations this happens when a specific > > > node id is set. I got confused by the code flow initially. > > > > > > — > > > Denis > > > > > > > On Dec 19, 2016, at 9:13 AM, Dmitriy Setrakyan < > [hidden email]> > > > wrote: > > > > > > > > Thanks, Alex. > > > > > > > > Can you clarify if the putAll(...) call is made directly, or from > > locally > > > > executed callable? According to Denis, "even single (non batched) > > updates > > > > or queries are sent as callables", which should be fixed in my view. > > > > > > > > D. > > > > > > > > On Mon, Dec 19, 2016 at 3:32 AM, Alexander Paschenko < > > > > [hidden email]> wrote: > > > > > > > >> Dima, Val, > > > >> > > > >> Introduction of updates has not changed public API at all (only > > > >> configuration, in some cases), so they work in this respect exactly > > > >> like SELECTs - by default they run on client node started by the > > > >> driver itself, but can be sent via the same callables mechanism to > any > > > >> remote node by its id. > > > >> > > > >> So Dima, you're right, currently it's possible to send query to any > > > >> given node. And, at the same time, currently by default everything > > > >> works exactly like you want it to - that is, any MERGE, batched or > > > >> not, boils down to putAll, and by default this call happens locally. > > > >> > > > >> - Alex > > > >> > > > >> 2016-12-17 18:47 GMT+03:00 Dmitriy Setrakyan <[hidden email] > >: > > > >>> On Fri, Dec 16, 2016 at 9:53 PM, Valentin Kulichenko < > > > >>> [hidden email]> wrote: > > > >>> > > > >>>> I'm not sure about updates, but can tell about how selects are > > > >> implemented > > > >>>> there. Basically, there is an option to execute the query on a > > > >> particular > > > >>>> node specified by ignite.jdbc.nodeId property. Not sure why we > need > > > this > > > >>>> though, probably it's just leftover from the legacy version of the > > > >> driver > > > >>>> based on thin client. > > > >>>> > > > >>>> If the property is set, the callable is sent to a remote node. But > > if > > > >> it is > > > >>>> not, the same callable is created, but it is invoked directly on > the > > > >>>> embedded client which is the behavior that you expect. And it's > the > > > >> default > > > >>>> one. > > > >>>> > > > >>>> > > > >>> Ouch. If this is the reason, I would drop the nodeId property. I > > don't > > > >>> think it makes sense and it significantly slows down the > > > implementation. > > > >>> > > > >>> > > > >>>> -Val > > > >>>> > > > >>>> On Fri, Dec 16, 2016 at 7:51 PM, Denis Magda <[hidden email]> > > > wrote: > > > >>>> > > > >>>>> Frankly speaking, even single (non batched) updates or queries > are > > > >> sent > > > >>>> as > > > >>>>> callables. This is what I see in the code. > > > >>>>> No idea what was the reason behind this design. > > > >>>>> > > > >>>>> Andrey G., Alex P. could you shed a light on this? > > > >>>>> > > > >>>>> — > > > >>>>> Denis > > > >>>>> > > > >>>>>> On Dec 16, 2016, at 3:08 PM, Dmitriy Setrakyan < > > > >> [hidden email]> > > > >>>>> wrote: > > > >>>>>> > > > >>>>>> To my understanding, we are implementing JDBC batches by > sending a > > > >>>>> callable > > > >>>>>> to another node. If we already have a client node on the JDBC > > driver > > > >>>>> side, > > > >>>>>> why not just issue a putAll(...) call from the client? > > > >>>>>> > > > >>>>>> D. > > > >>>>>> > > > >>>>>> On Fri, Dec 16, 2016 at 3:02 PM, Denis Magda <[hidden email] > > > > > >>>> wrote: > > > >>>>>> > > > >>>>>>> Dmitriy, > > > >>>>>>> > > > >>>>>>> JDBC drivers spawns an Ignite client node and uses it for > cluster > > > >>>>>>> connectivity and queries execution. Queries issued over the > JDBC > > > >> are > > > >>>>> turned > > > >>>>>>> into SqlFieldsQueries and sent to the cluster in this form. > > > >>>>>>> > > > >>>>>>> ODBC driver works in a bit different way. It connects to the > > > >> cluster > > > >>>> via > > > >>>>>>> ODBC processor that needs to be running on one of the nodes: > > > >>>>>>> https://apacheignite.readme.io/docs/odbc-driver#cluster- > > > >> configuration > > > >>>> < > > > >>>>>>> https://apacheignite.readme.io/docs/odbc-driver#cluster- > > > >> configuration > > > >>>>> > > > >>>>>>> > > > >>>>>>> — > > > >>>>>>> Denis > > > >>>>>>> > > > >>>>>>>> On Dec 16, 2016, at 2:41 PM, Dmitriy Setrakyan < > > > >>>> [hidden email]> > > > >>>>>>> wrote: > > > >>>>>>>> > > > >>>>>>>> Igniters, > > > >>>>>>>> > > > >>>>>>>> Can someone explain to me how Ignite executes SQL from JDBC > and > > > >> ODBC > > > >>>>>>>> drivers? Do we start an Ignite client node on the driver side? > > Or > > > >> do > > > >>>> we > > > >>>>>>> use > > > >>>>>>>> some other protocol to send commands to one of the Ignite > nodes? > > > >>>>>>>> > > > >>>>>>>> Thanks, > > > >>>>>>>> D. > > > >>>>>>> > > > >>>>>>> > > > >>>>> > > > >>>>> > > > >>>> > > > >> > > > > > > > > > |
I see what you are suggesting and will remove this feature from the driver
if there are no objections. (Anyone?) Still, for clarity and your better understanding I'd like to again point out that there's no thread context switching or any Ignite imposed overhead by default as even ignite compute ia not used — closure's call() method is called directly in the same thread. Wrapped task isn't going anywhere in local case. And it's the pattern that is used in Ignite code quite extensively — if the task is local anyway, let's just run it "on the spot", directly, without any unnecessary overhead. — Alex 19 дек. 2016 г. 11:44 PM пользователь "Dmitriy Setrakyan" < [hidden email]> написал: > Alex, > > What Val and I are suggesting is that we get rid of any JdbcCallable calls > and invoke putAll(...) directly. Wrapping it into JdbcCallable will slow us > down and introduce unnecessary thread context switch. > > Do you agree? > > D. > > On Mon, Dec 19, 2016 at 11:01 AM, Valentin Kulichenko < > [hidden email]> wrote: > > > By "locally executed callable" I meant something like this: > > > > new JdbcCallable(...).call(); > > > > So it's not a network call and it's not even going through IgniteCompute. > > It's just the logic is incapsulated in the callable to support remote > > execution of the same logic. > > > > But anyway, +1 to removing nodeId property. It doesn't make much sense > with > > the new implementation of JDBC. > > > > -Val > > > > On Mon, Dec 19, 2016 at 10:15 AM, Alexander Paschenko < > > [hidden email]> wrote: > > > > > When node is local, no network interaction occurs on query send. Would > be > > > shame otherwise :) > > > > > > — Alex > > > 19 дек. 2016 г. 8:36 PM пользователь "Denis Magda" <[hidden email]> > > > написал: > > > > > > > Dmitriy, > > > > > > > > According to Val and Alex P. explanations this happens when a > specific > > > > node id is set. I got confused by the code flow initially. > > > > > > > > — > > > > Denis > > > > > > > > > On Dec 19, 2016, at 9:13 AM, Dmitriy Setrakyan < > > [hidden email]> > > > > wrote: > > > > > > > > > > Thanks, Alex. > > > > > > > > > > Can you clarify if the putAll(...) call is made directly, or from > > > locally > > > > > executed callable? According to Denis, "even single (non batched) > > > updates > > > > > or queries are sent as callables", which should be fixed in my > view. > > > > > > > > > > D. > > > > > > > > > > On Mon, Dec 19, 2016 at 3:32 AM, Alexander Paschenko < > > > > > [hidden email]> wrote: > > > > > > > > > >> Dima, Val, > > > > >> > > > > >> Introduction of updates has not changed public API at all (only > > > > >> configuration, in some cases), so they work in this respect > exactly > > > > >> like SELECTs - by default they run on client node started by the > > > > >> driver itself, but can be sent via the same callables mechanism to > > any > > > > >> remote node by its id. > > > > >> > > > > >> So Dima, you're right, currently it's possible to send query to > any > > > > >> given node. And, at the same time, currently by default everything > > > > >> works exactly like you want it to - that is, any MERGE, batched or > > > > >> not, boils down to putAll, and by default this call happens > locally. > > > > >> > > > > >> - Alex > > > > >> > > > > >> 2016-12-17 18:47 GMT+03:00 Dmitriy Setrakyan < > [hidden email] > > >: > > > > >>> On Fri, Dec 16, 2016 at 9:53 PM, Valentin Kulichenko < > > > > >>> [hidden email]> wrote: > > > > >>> > > > > >>>> I'm not sure about updates, but can tell about how selects are > > > > >> implemented > > > > >>>> there. Basically, there is an option to execute the query on a > > > > >> particular > > > > >>>> node specified by ignite.jdbc.nodeId property. Not sure why we > > need > > > > this > > > > >>>> though, probably it's just leftover from the legacy version of > the > > > > >> driver > > > > >>>> based on thin client. > > > > >>>> > > > > >>>> If the property is set, the callable is sent to a remote node. > But > > > if > > > > >> it is > > > > >>>> not, the same callable is created, but it is invoked directly on > > the > > > > >>>> embedded client which is the behavior that you expect. And it's > > the > > > > >> default > > > > >>>> one. > > > > >>>> > > > > >>>> > > > > >>> Ouch. If this is the reason, I would drop the nodeId property. I > > > don't > > > > >>> think it makes sense and it significantly slows down the > > > > implementation. > > > > >>> > > > > >>> > > > > >>>> -Val > > > > >>>> > > > > >>>> On Fri, Dec 16, 2016 at 7:51 PM, Denis Magda <[hidden email] > > > > > > wrote: > > > > >>>> > > > > >>>>> Frankly speaking, even single (non batched) updates or queries > > are > > > > >> sent > > > > >>>> as > > > > >>>>> callables. This is what I see in the code. > > > > >>>>> No idea what was the reason behind this design. > > > > >>>>> > > > > >>>>> Andrey G., Alex P. could you shed a light on this? > > > > >>>>> > > > > >>>>> — > > > > >>>>> Denis > > > > >>>>> > > > > >>>>>> On Dec 16, 2016, at 3:08 PM, Dmitriy Setrakyan < > > > > >> [hidden email]> > > > > >>>>> wrote: > > > > >>>>>> > > > > >>>>>> To my understanding, we are implementing JDBC batches by > > sending a > > > > >>>>> callable > > > > >>>>>> to another node. If we already have a client node on the JDBC > > > driver > > > > >>>>> side, > > > > >>>>>> why not just issue a putAll(...) call from the client? > > > > >>>>>> > > > > >>>>>> D. > > > > >>>>>> > > > > >>>>>> On Fri, Dec 16, 2016 at 3:02 PM, Denis Magda < > [hidden email] > > > > > > > >>>> wrote: > > > > >>>>>> > > > > >>>>>>> Dmitriy, > > > > >>>>>>> > > > > >>>>>>> JDBC drivers spawns an Ignite client node and uses it for > > cluster > > > > >>>>>>> connectivity and queries execution. Queries issued over the > > JDBC > > > > >> are > > > > >>>>> turned > > > > >>>>>>> into SqlFieldsQueries and sent to the cluster in this form. > > > > >>>>>>> > > > > >>>>>>> ODBC driver works in a bit different way. It connects to the > > > > >> cluster > > > > >>>> via > > > > >>>>>>> ODBC processor that needs to be running on one of the nodes: > > > > >>>>>>> https://apacheignite.readme.io/docs/odbc-driver#cluster- > > > > >> configuration > > > > >>>> < > > > > >>>>>>> https://apacheignite.readme.io/docs/odbc-driver#cluster- > > > > >> configuration > > > > >>>>> > > > > >>>>>>> > > > > >>>>>>> — > > > > >>>>>>> Denis > > > > >>>>>>> > > > > >>>>>>>> On Dec 16, 2016, at 2:41 PM, Dmitriy Setrakyan < > > > > >>>> [hidden email]> > > > > >>>>>>> wrote: > > > > >>>>>>>> > > > > >>>>>>>> Igniters, > > > > >>>>>>>> > > > > >>>>>>>> Can someone explain to me how Ignite executes SQL from JDBC > > and > > > > >> ODBC > > > > >>>>>>>> drivers? Do we start an Ignite client node on the driver > side? > > > Or > > > > >> do > > > > >>>> we > > > > >>>>>>> use > > > > >>>>>>>> some other protocol to send commands to one of the Ignite > > nodes? > > > > >>>>>>>> > > > > >>>>>>>> Thanks, > > > > >>>>>>>> D. > > > > >>>>>>> > > > > >>>>>>> > > > > >>>>> > > > > >>>>> > > > > >>>> > > > > >> > > > > > > > > > > > > > > |
On Tue, Dec 20, 2016 at 1:23 AM, Alexander Paschenko <
[hidden email]> wrote: > I see what you are suggesting and will remove this feature from the driver > if there are no objections. (Anyone?) > Still, for clarity and your better understanding I'd like to again point > out that there's no thread context switching or any Ignite imposed overhead > by default as even ignite compute ia not used — closure's call() method is > called directly in the same thread. Wrapped task isn't going anywhere in > local case. And it's the pattern that is used in Ignite code quite > extensively — if the task is local anyway, let's just run it "on the spot", > directly, without any unnecessary overhead. > > Why do you keep insisting on having this flawed design? Why not fix it properly by calling putAll(...) directly? > — Alex > 19 дек. 2016 г. 11:44 PM пользователь "Dmitriy Setrakyan" < > [hidden email]> написал: > > > Alex, > > > > What Val and I are suggesting is that we get rid of any JdbcCallable > calls > > and invoke putAll(...) directly. Wrapping it into JdbcCallable will slow > us > > down and introduce unnecessary thread context switch. > > > > Do you agree? > > > > D. > > > > On Mon, Dec 19, 2016 at 11:01 AM, Valentin Kulichenko < > > [hidden email]> wrote: > > > > > By "locally executed callable" I meant something like this: > > > > > > new JdbcCallable(...).call(); > > > > > > So it's not a network call and it's not even going through > IgniteCompute. > > > It's just the logic is incapsulated in the callable to support remote > > > execution of the same logic. > > > > > > But anyway, +1 to removing nodeId property. It doesn't make much sense > > with > > > the new implementation of JDBC. > > > > > > -Val > > > > > > On Mon, Dec 19, 2016 at 10:15 AM, Alexander Paschenko < > > > [hidden email]> wrote: > > > > > > > When node is local, no network interaction occurs on query send. > Would > > be > > > > shame otherwise :) > > > > > > > > — Alex > > > > 19 дек. 2016 г. 8:36 PM пользователь "Denis Magda" < > [hidden email]> > > > > написал: > > > > > > > > > Dmitriy, > > > > > > > > > > According to Val and Alex P. explanations this happens when a > > specific > > > > > node id is set. I got confused by the code flow initially. > > > > > > > > > > — > > > > > Denis > > > > > > > > > > > On Dec 19, 2016, at 9:13 AM, Dmitriy Setrakyan < > > > [hidden email]> > > > > > wrote: > > > > > > > > > > > > Thanks, Alex. > > > > > > > > > > > > Can you clarify if the putAll(...) call is made directly, or from > > > > locally > > > > > > executed callable? According to Denis, "even single (non batched) > > > > updates > > > > > > or queries are sent as callables", which should be fixed in my > > view. > > > > > > > > > > > > D. > > > > > > > > > > > > On Mon, Dec 19, 2016 at 3:32 AM, Alexander Paschenko < > > > > > > [hidden email]> wrote: > > > > > > > > > > > >> Dima, Val, > > > > > >> > > > > > >> Introduction of updates has not changed public API at all (only > > > > > >> configuration, in some cases), so they work in this respect > > exactly > > > > > >> like SELECTs - by default they run on client node started by the > > > > > >> driver itself, but can be sent via the same callables mechanism > to > > > any > > > > > >> remote node by its id. > > > > > >> > > > > > >> So Dima, you're right, currently it's possible to send query to > > any > > > > > >> given node. And, at the same time, currently by default > everything > > > > > >> works exactly like you want it to - that is, any MERGE, batched > or > > > > > >> not, boils down to putAll, and by default this call happens > > locally. > > > > > >> > > > > > >> - Alex > > > > > >> > > > > > >> 2016-12-17 18:47 GMT+03:00 Dmitriy Setrakyan < > > [hidden email] > > > >: > > > > > >>> On Fri, Dec 16, 2016 at 9:53 PM, Valentin Kulichenko < > > > > > >>> [hidden email]> wrote: > > > > > >>> > > > > > >>>> I'm not sure about updates, but can tell about how selects are > > > > > >> implemented > > > > > >>>> there. Basically, there is an option to execute the query on a > > > > > >> particular > > > > > >>>> node specified by ignite.jdbc.nodeId property. Not sure why we > > > need > > > > > this > > > > > >>>> though, probably it's just leftover from the legacy version of > > the > > > > > >> driver > > > > > >>>> based on thin client. > > > > > >>>> > > > > > >>>> If the property is set, the callable is sent to a remote node. > > But > > > > if > > > > > >> it is > > > > > >>>> not, the same callable is created, but it is invoked directly > on > > > the > > > > > >>>> embedded client which is the behavior that you expect. And > it's > > > the > > > > > >> default > > > > > >>>> one. > > > > > >>>> > > > > > >>>> > > > > > >>> Ouch. If this is the reason, I would drop the nodeId property. > I > > > > don't > > > > > >>> think it makes sense and it significantly slows down the > > > > > implementation. > > > > > >>> > > > > > >>> > > > > > >>>> -Val > > > > > >>>> > > > > > >>>> On Fri, Dec 16, 2016 at 7:51 PM, Denis Magda < > [hidden email] > > > > > > > > wrote: > > > > > >>>> > > > > > >>>>> Frankly speaking, even single (non batched) updates or > queries > > > are > > > > > >> sent > > > > > >>>> as > > > > > >>>>> callables. This is what I see in the code. > > > > > >>>>> No idea what was the reason behind this design. > > > > > >>>>> > > > > > >>>>> Andrey G., Alex P. could you shed a light on this? > > > > > >>>>> > > > > > >>>>> — > > > > > >>>>> Denis > > > > > >>>>> > > > > > >>>>>> On Dec 16, 2016, at 3:08 PM, Dmitriy Setrakyan < > > > > > >> [hidden email]> > > > > > >>>>> wrote: > > > > > >>>>>> > > > > > >>>>>> To my understanding, we are implementing JDBC batches by > > > sending a > > > > > >>>>> callable > > > > > >>>>>> to another node. If we already have a client node on the > JDBC > > > > driver > > > > > >>>>> side, > > > > > >>>>>> why not just issue a putAll(...) call from the client? > > > > > >>>>>> > > > > > >>>>>> D. > > > > > >>>>>> > > > > > >>>>>> On Fri, Dec 16, 2016 at 3:02 PM, Denis Magda < > > [hidden email] > > > > > > > > > >>>> wrote: > > > > > >>>>>> > > > > > >>>>>>> Dmitriy, > > > > > >>>>>>> > > > > > >>>>>>> JDBC drivers spawns an Ignite client node and uses it for > > > cluster > > > > > >>>>>>> connectivity and queries execution. Queries issued over the > > > JDBC > > > > > >> are > > > > > >>>>> turned > > > > > >>>>>>> into SqlFieldsQueries and sent to the cluster in this form. > > > > > >>>>>>> > > > > > >>>>>>> ODBC driver works in a bit different way. It connects to > the > > > > > >> cluster > > > > > >>>> via > > > > > >>>>>>> ODBC processor that needs to be running on one of the > nodes: > > > > > >>>>>>> https://apacheignite.readme.io/docs/odbc-driver#cluster- > > > > > >> configuration > > > > > >>>> < > > > > > >>>>>>> https://apacheignite.readme.io/docs/odbc-driver#cluster- > > > > > >> configuration > > > > > >>>>> > > > > > >>>>>>> > > > > > >>>>>>> — > > > > > >>>>>>> Denis > > > > > >>>>>>> > > > > > >>>>>>>> On Dec 16, 2016, at 2:41 PM, Dmitriy Setrakyan < > > > > > >>>> [hidden email]> > > > > > >>>>>>> wrote: > > > > > >>>>>>>> > > > > > >>>>>>>> Igniters, > > > > > >>>>>>>> > > > > > >>>>>>>> Can someone explain to me how Ignite executes SQL from > JDBC > > > and > > > > > >> ODBC > > > > > >>>>>>>> drivers? Do we start an Ignite client node on the driver > > side? > > > > Or > > > > > >> do > > > > > >>>> we > > > > > >>>>>>> use > > > > > >>>>>>>> some other protocol to send commands to one of the Ignite > > > nodes? > > > > > >>>>>>>> > > > > > >>>>>>>> Thanks, > > > > > >>>>>>>> D. > > > > > >>>>>>> > > > > > >>>>>>> > > > > > >>>>> > > > > > >>>>> > > > > > >>>> > > > > > >> > > > > > > > > > > > > > > > > > > > > |
OK, will remove redundant query tasks creation for the case of DML
statements - they at least incur some GC overhead even if not context switching. - Alex 2016-12-20 12:49 GMT+03:00 Dmitriy Setrakyan <[hidden email]>: > On Tue, Dec 20, 2016 at 1:23 AM, Alexander Paschenko < > [hidden email]> wrote: > >> I see what you are suggesting and will remove this feature from the driver >> if there are no objections. (Anyone?) >> Still, for clarity and your better understanding I'd like to again point >> out that there's no thread context switching or any Ignite imposed overhead >> by default as even ignite compute ia not used — closure's call() method is >> called directly in the same thread. Wrapped task isn't going anywhere in >> local case. And it's the pattern that is used in Ignite code quite >> extensively — if the task is local anyway, let's just run it "on the spot", >> directly, without any unnecessary overhead. >> >> > I disagree, there is an overhead, especially for relatively small batches. > Why do you keep insisting on having this flawed design? Why not fix it > properly by calling putAll(...) directly? > > >> — Alex >> 19 дек. 2016 г. 11:44 PM пользователь "Dmitriy Setrakyan" < >> [hidden email]> написал: >> >> > Alex, >> > >> > What Val and I are suggesting is that we get rid of any JdbcCallable >> calls >> > and invoke putAll(...) directly. Wrapping it into JdbcCallable will slow >> us >> > down and introduce unnecessary thread context switch. >> > >> > Do you agree? >> > >> > D. >> > >> > On Mon, Dec 19, 2016 at 11:01 AM, Valentin Kulichenko < >> > [hidden email]> wrote: >> > >> > > By "locally executed callable" I meant something like this: >> > > >> > > new JdbcCallable(...).call(); >> > > >> > > So it's not a network call and it's not even going through >> IgniteCompute. >> > > It's just the logic is incapsulated in the callable to support remote >> > > execution of the same logic. >> > > >> > > But anyway, +1 to removing nodeId property. It doesn't make much sense >> > with >> > > the new implementation of JDBC. >> > > >> > > -Val >> > > >> > > On Mon, Dec 19, 2016 at 10:15 AM, Alexander Paschenko < >> > > [hidden email]> wrote: >> > > >> > > > When node is local, no network interaction occurs on query send. >> Would >> > be >> > > > shame otherwise :) >> > > > >> > > > — Alex >> > > > 19 дек. 2016 г. 8:36 PM пользователь "Denis Magda" < >> [hidden email]> >> > > > написал: >> > > > >> > > > > Dmitriy, >> > > > > >> > > > > According to Val and Alex P. explanations this happens when a >> > specific >> > > > > node id is set. I got confused by the code flow initially. >> > > > > >> > > > > — >> > > > > Denis >> > > > > >> > > > > > On Dec 19, 2016, at 9:13 AM, Dmitriy Setrakyan < >> > > [hidden email]> >> > > > > wrote: >> > > > > > >> > > > > > Thanks, Alex. >> > > > > > >> > > > > > Can you clarify if the putAll(...) call is made directly, or from >> > > > locally >> > > > > > executed callable? According to Denis, "even single (non batched) >> > > > updates >> > > > > > or queries are sent as callables", which should be fixed in my >> > view. >> > > > > > >> > > > > > D. >> > > > > > >> > > > > > On Mon, Dec 19, 2016 at 3:32 AM, Alexander Paschenko < >> > > > > > [hidden email]> wrote: >> > > > > > >> > > > > >> Dima, Val, >> > > > > >> >> > > > > >> Introduction of updates has not changed public API at all (only >> > > > > >> configuration, in some cases), so they work in this respect >> > exactly >> > > > > >> like SELECTs - by default they run on client node started by the >> > > > > >> driver itself, but can be sent via the same callables mechanism >> to >> > > any >> > > > > >> remote node by its id. >> > > > > >> >> > > > > >> So Dima, you're right, currently it's possible to send query to >> > any >> > > > > >> given node. And, at the same time, currently by default >> everything >> > > > > >> works exactly like you want it to - that is, any MERGE, batched >> or >> > > > > >> not, boils down to putAll, and by default this call happens >> > locally. >> > > > > >> >> > > > > >> - Alex >> > > > > >> >> > > > > >> 2016-12-17 18:47 GMT+03:00 Dmitriy Setrakyan < >> > [hidden email] >> > > >: >> > > > > >>> On Fri, Dec 16, 2016 at 9:53 PM, Valentin Kulichenko < >> > > > > >>> [hidden email]> wrote: >> > > > > >>> >> > > > > >>>> I'm not sure about updates, but can tell about how selects are >> > > > > >> implemented >> > > > > >>>> there. Basically, there is an option to execute the query on a >> > > > > >> particular >> > > > > >>>> node specified by ignite.jdbc.nodeId property. Not sure why we >> > > need >> > > > > this >> > > > > >>>> though, probably it's just leftover from the legacy version of >> > the >> > > > > >> driver >> > > > > >>>> based on thin client. >> > > > > >>>> >> > > > > >>>> If the property is set, the callable is sent to a remote node. >> > But >> > > > if >> > > > > >> it is >> > > > > >>>> not, the same callable is created, but it is invoked directly >> on >> > > the >> > > > > >>>> embedded client which is the behavior that you expect. And >> it's >> > > the >> > > > > >> default >> > > > > >>>> one. >> > > > > >>>> >> > > > > >>>> >> > > > > >>> Ouch. If this is the reason, I would drop the nodeId property. >> I >> > > > don't >> > > > > >>> think it makes sense and it significantly slows down the >> > > > > implementation. >> > > > > >>> >> > > > > >>> >> > > > > >>>> -Val >> > > > > >>>> >> > > > > >>>> On Fri, Dec 16, 2016 at 7:51 PM, Denis Magda < >> [hidden email] >> > > >> > > > > wrote: >> > > > > >>>> >> > > > > >>>>> Frankly speaking, even single (non batched) updates or >> queries >> > > are >> > > > > >> sent >> > > > > >>>> as >> > > > > >>>>> callables. This is what I see in the code. >> > > > > >>>>> No idea what was the reason behind this design. >> > > > > >>>>> >> > > > > >>>>> Andrey G., Alex P. could you shed a light on this? >> > > > > >>>>> >> > > > > >>>>> — >> > > > > >>>>> Denis >> > > > > >>>>> >> > > > > >>>>>> On Dec 16, 2016, at 3:08 PM, Dmitriy Setrakyan < >> > > > > >> [hidden email]> >> > > > > >>>>> wrote: >> > > > > >>>>>> >> > > > > >>>>>> To my understanding, we are implementing JDBC batches by >> > > sending a >> > > > > >>>>> callable >> > > > > >>>>>> to another node. If we already have a client node on the >> JDBC >> > > > driver >> > > > > >>>>> side, >> > > > > >>>>>> why not just issue a putAll(...) call from the client? >> > > > > >>>>>> >> > > > > >>>>>> D. >> > > > > >>>>>> >> > > > > >>>>>> On Fri, Dec 16, 2016 at 3:02 PM, Denis Magda < >> > [hidden email] >> > > > >> > > > > >>>> wrote: >> > > > > >>>>>> >> > > > > >>>>>>> Dmitriy, >> > > > > >>>>>>> >> > > > > >>>>>>> JDBC drivers spawns an Ignite client node and uses it for >> > > cluster >> > > > > >>>>>>> connectivity and queries execution. Queries issued over the >> > > JDBC >> > > > > >> are >> > > > > >>>>> turned >> > > > > >>>>>>> into SqlFieldsQueries and sent to the cluster in this form. >> > > > > >>>>>>> >> > > > > >>>>>>> ODBC driver works in a bit different way. It connects to >> the >> > > > > >> cluster >> > > > > >>>> via >> > > > > >>>>>>> ODBC processor that needs to be running on one of the >> nodes: >> > > > > >>>>>>> https://apacheignite.readme.io/docs/odbc-driver#cluster- >> > > > > >> configuration >> > > > > >>>> < >> > > > > >>>>>>> https://apacheignite.readme.io/docs/odbc-driver#cluster- >> > > > > >> configuration >> > > > > >>>>> >> > > > > >>>>>>> >> > > > > >>>>>>> — >> > > > > >>>>>>> Denis >> > > > > >>>>>>> >> > > > > >>>>>>>> On Dec 16, 2016, at 2:41 PM, Dmitriy Setrakyan < >> > > > > >>>> [hidden email]> >> > > > > >>>>>>> wrote: >> > > > > >>>>>>>> >> > > > > >>>>>>>> Igniters, >> > > > > >>>>>>>> >> > > > > >>>>>>>> Can someone explain to me how Ignite executes SQL from >> JDBC >> > > and >> > > > > >> ODBC >> > > > > >>>>>>>> drivers? Do we start an Ignite client node on the driver >> > side? >> > > > Or >> > > > > >> do >> > > > > >>>> we >> > > > > >>>>>>> use >> > > > > >>>>>>>> some other protocol to send commands to one of the Ignite >> > > nodes? >> > > > > >>>>>>>> >> > > > > >>>>>>>> Thanks, >> > > > > >>>>>>>> D. >> > > > > >>>>>>> >> > > > > >>>>>>> >> > > > > >>>>> >> > > > > >>>>> >> > > > > >>>> >> > > > > >> >> > > > > >> > > > > >> > > > >> > > >> > >> |
Guys,
Doesn’t it make sense to wait for 2.0 release where we’re free to break the compatibility by removing deprecated or bad designed APIs? There is a chance that this JDBC’s “nodeId” parameter is being used by someone. — Denis > On Dec 20, 2016, at 10:18 AM, Alexander Paschenko <[hidden email]> wrote: > > OK, will remove redundant query tasks creation for the case of DML > statements - they at least incur some GC overhead even if not context > switching. > > - Alex > > 2016-12-20 12:49 GMT+03:00 Dmitriy Setrakyan <[hidden email]>: >> On Tue, Dec 20, 2016 at 1:23 AM, Alexander Paschenko < >> [hidden email]> wrote: >> >>> I see what you are suggesting and will remove this feature from the driver >>> if there are no objections. (Anyone?) >>> Still, for clarity and your better understanding I'd like to again point >>> out that there's no thread context switching or any Ignite imposed overhead >>> by default as even ignite compute ia not used — closure's call() method is >>> called directly in the same thread. Wrapped task isn't going anywhere in >>> local case. And it's the pattern that is used in Ignite code quite >>> extensively — if the task is local anyway, let's just run it "on the spot", >>> directly, without any unnecessary overhead. >>> >>> >> I disagree, there is an overhead, especially for relatively small batches. >> Why do you keep insisting on having this flawed design? Why not fix it >> properly by calling putAll(...) directly? >> >> >>> — Alex >>> 19 дек. 2016 г. 11:44 PM пользователь "Dmitriy Setrakyan" < >>> [hidden email]> написал: >>> >>>> Alex, >>>> >>>> What Val and I are suggesting is that we get rid of any JdbcCallable >>> calls >>>> and invoke putAll(...) directly. Wrapping it into JdbcCallable will slow >>> us >>>> down and introduce unnecessary thread context switch. >>>> >>>> Do you agree? >>>> >>>> D. >>>> >>>> On Mon, Dec 19, 2016 at 11:01 AM, Valentin Kulichenko < >>>> [hidden email]> wrote: >>>> >>>>> By "locally executed callable" I meant something like this: >>>>> >>>>> new JdbcCallable(...).call(); >>>>> >>>>> So it's not a network call and it's not even going through >>> IgniteCompute. >>>>> It's just the logic is incapsulated in the callable to support remote >>>>> execution of the same logic. >>>>> >>>>> But anyway, +1 to removing nodeId property. It doesn't make much sense >>>> with >>>>> the new implementation of JDBC. >>>>> >>>>> -Val >>>>> >>>>> On Mon, Dec 19, 2016 at 10:15 AM, Alexander Paschenko < >>>>> [hidden email]> wrote: >>>>> >>>>>> When node is local, no network interaction occurs on query send. >>> Would >>>> be >>>>>> shame otherwise :) >>>>>> >>>>>> — Alex >>>>>> 19 дек. 2016 г. 8:36 PM пользователь "Denis Magda" < >>> [hidden email]> >>>>>> написал: >>>>>> >>>>>>> Dmitriy, >>>>>>> >>>>>>> According to Val and Alex P. explanations this happens when a >>>> specific >>>>>>> node id is set. I got confused by the code flow initially. >>>>>>> >>>>>>> — >>>>>>> Denis >>>>>>> >>>>>>>> On Dec 19, 2016, at 9:13 AM, Dmitriy Setrakyan < >>>>> [hidden email]> >>>>>>> wrote: >>>>>>>> >>>>>>>> Thanks, Alex. >>>>>>>> >>>>>>>> Can you clarify if the putAll(...) call is made directly, or from >>>>>> locally >>>>>>>> executed callable? According to Denis, "even single (non batched) >>>>>> updates >>>>>>>> or queries are sent as callables", which should be fixed in my >>>> view. >>>>>>>> >>>>>>>> D. >>>>>>>> >>>>>>>> On Mon, Dec 19, 2016 at 3:32 AM, Alexander Paschenko < >>>>>>>> [hidden email]> wrote: >>>>>>>> >>>>>>>>> Dima, Val, >>>>>>>>> >>>>>>>>> Introduction of updates has not changed public API at all (only >>>>>>>>> configuration, in some cases), so they work in this respect >>>> exactly >>>>>>>>> like SELECTs - by default they run on client node started by the >>>>>>>>> driver itself, but can be sent via the same callables mechanism >>> to >>>>> any >>>>>>>>> remote node by its id. >>>>>>>>> >>>>>>>>> So Dima, you're right, currently it's possible to send query to >>>> any >>>>>>>>> given node. And, at the same time, currently by default >>> everything >>>>>>>>> works exactly like you want it to - that is, any MERGE, batched >>> or >>>>>>>>> not, boils down to putAll, and by default this call happens >>>> locally. >>>>>>>>> >>>>>>>>> - Alex >>>>>>>>> >>>>>>>>> 2016-12-17 18:47 GMT+03:00 Dmitriy Setrakyan < >>>> [hidden email] >>>>>> : >>>>>>>>>> On Fri, Dec 16, 2016 at 9:53 PM, Valentin Kulichenko < >>>>>>>>>> [hidden email]> wrote: >>>>>>>>>> >>>>>>>>>>> I'm not sure about updates, but can tell about how selects are >>>>>>>>> implemented >>>>>>>>>>> there. Basically, there is an option to execute the query on a >>>>>>>>> particular >>>>>>>>>>> node specified by ignite.jdbc.nodeId property. Not sure why we >>>>> need >>>>>>> this >>>>>>>>>>> though, probably it's just leftover from the legacy version of >>>> the >>>>>>>>> driver >>>>>>>>>>> based on thin client. >>>>>>>>>>> >>>>>>>>>>> If the property is set, the callable is sent to a remote node. >>>> But >>>>>> if >>>>>>>>> it is >>>>>>>>>>> not, the same callable is created, but it is invoked directly >>> on >>>>> the >>>>>>>>>>> embedded client which is the behavior that you expect. And >>> it's >>>>> the >>>>>>>>> default >>>>>>>>>>> one. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> Ouch. If this is the reason, I would drop the nodeId property. >>> I >>>>>> don't >>>>>>>>>> think it makes sense and it significantly slows down the >>>>>>> implementation. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> -Val >>>>>>>>>>> >>>>>>>>>>> On Fri, Dec 16, 2016 at 7:51 PM, Denis Magda < >>> [hidden email] >>>>> >>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Frankly speaking, even single (non batched) updates or >>> queries >>>>> are >>>>>>>>> sent >>>>>>>>>>> as >>>>>>>>>>>> callables. This is what I see in the code. >>>>>>>>>>>> No idea what was the reason behind this design. >>>>>>>>>>>> >>>>>>>>>>>> Andrey G., Alex P. could you shed a light on this? >>>>>>>>>>>> >>>>>>>>>>>> — >>>>>>>>>>>> Denis >>>>>>>>>>>> >>>>>>>>>>>>> On Dec 16, 2016, at 3:08 PM, Dmitriy Setrakyan < >>>>>>>>> [hidden email]> >>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> To my understanding, we are implementing JDBC batches by >>>>> sending a >>>>>>>>>>>> callable >>>>>>>>>>>>> to another node. If we already have a client node on the >>> JDBC >>>>>> driver >>>>>>>>>>>> side, >>>>>>>>>>>>> why not just issue a putAll(...) call from the client? >>>>>>>>>>>>> >>>>>>>>>>>>> D. >>>>>>>>>>>>> >>>>>>>>>>>>> On Fri, Dec 16, 2016 at 3:02 PM, Denis Magda < >>>> [hidden email] >>>>>> >>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Dmitriy, >>>>>>>>>>>>>> >>>>>>>>>>>>>> JDBC drivers spawns an Ignite client node and uses it for >>>>> cluster >>>>>>>>>>>>>> connectivity and queries execution. Queries issued over the >>>>> JDBC >>>>>>>>> are >>>>>>>>>>>> turned >>>>>>>>>>>>>> into SqlFieldsQueries and sent to the cluster in this form. >>>>>>>>>>>>>> >>>>>>>>>>>>>> ODBC driver works in a bit different way. It connects to >>> the >>>>>>>>> cluster >>>>>>>>>>> via >>>>>>>>>>>>>> ODBC processor that needs to be running on one of the >>> nodes: >>>>>>>>>>>>>> https://apacheignite.readme.io/docs/odbc-driver#cluster- >>>>>>>>> configuration >>>>>>>>>>> < >>>>>>>>>>>>>> https://apacheignite.readme.io/docs/odbc-driver#cluster- >>>>>>>>> configuration >>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> — >>>>>>>>>>>>>> Denis >>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Dec 16, 2016, at 2:41 PM, Dmitriy Setrakyan < >>>>>>>>>>> [hidden email]> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Igniters, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Can someone explain to me how Ignite executes SQL from >>> JDBC >>>>> and >>>>>>>>> ODBC >>>>>>>>>>>>>>> drivers? Do we start an Ignite client node on the driver >>>> side? >>>>>> Or >>>>>>>>> do >>>>>>>>>>> we >>>>>>>>>>>>>> use >>>>>>>>>>>>>>> some other protocol to send commands to one of the Ignite >>>>> nodes? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> D. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> |
Free forum by Nabble | Edit this page |