Data loading with thin clients

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Data loading with thin clients

dmagda
Igor, Igniters,

I've come across a couple of posts recently where users attempted to
preload initial data with a thin client like Python:
https://stackoverflow.com/questions/56778778/apache-ignite-inserts-extremely-slow/56791088#56791088

Our JDBC/ODBC drivers have a special streaming mode that boosts the loading
for such use cases.

Will there be any benefits from the initial data loading standpoint when
partition-awareness is released or do we need to enhance our thin clients'
protocol? Users use SQL for preloading.


Denis Magda
Reply | Threaded
Open this post in threaded view
|

Re: Data loading with thin clients

Igor Sapego-2
If they use python client, why wont they just use put_all()?

To me it just looks like they are using wrong tool here.

As for your questing, Affinity Awareness for thin clients is only for Cache
API now, it does not affect SQL.

Best Regards,
Igor


On Thu, Jun 27, 2019 at 7:33 PM Denis Magda <[hidden email]> wrote:

> Igor, Igniters,
>
> I've come across a couple of posts recently where users attempted to
> preload initial data with a thin client like Python:
> https://stackoverflow.com/questions/56778778/apache-ignite-inserts-extremely-slow/56791088#56791088
>
> Our JDBC/ODBC drivers have a special streaming mode that boosts the
> loading for such use cases.
>
> Will there be any benefits from the initial data loading standpoint when
> partition-awareness is released or do we need to enhance our thin clients'
> protocol? Users use SQL for preloading.
>
>
> Denis Magda
>
Reply | Threaded
Open this post in threaded view
|

Re: Data loading with thin clients

dmagda
Igor,

The user prefers using SQL syntax which is widespread. For sure, we'll
recommend alternate solutions as long as INSERT is not suited for loading
if a thin client is used.

However, how have we supported the streaming mode for INSERTS for ODBC?
Isn't it already based on the thin client protocol?

-
Denis


On Fri, Jun 28, 2019 at 8:02 AM Igor Sapego <[hidden email]> wrote:

> If they use python client, why wont they just use put_all()?
>
> To me it just looks like they are using wrong tool here.
>
> As for your questing, Affinity Awareness for thin clients is only for Cache
> API now, it does not affect SQL.
>
> Best Regards,
> Igor
>
>
> On Thu, Jun 27, 2019 at 7:33 PM Denis Magda <[hidden email]> wrote:
>
> > Igor, Igniters,
> >
> > I've come across a couple of posts recently where users attempted to
> > preload initial data with a thin client like Python:
> >
> https://stackoverflow.com/questions/56778778/apache-ignite-inserts-extremely-slow/56791088#56791088
> >
> > Our JDBC/ODBC drivers have a special streaming mode that boosts the
> > loading for such use cases.
> >
> > Will there be any benefits from the initial data loading standpoint when
> > partition-awareness is released or do we need to enhance our thin
> clients'
> > protocol? Users use SQL for preloading.
> >
> >
> > Denis Magda
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Data loading with thin clients

Igor Sapego-2
Denis,

We have streaming support for ODBC. If they only need SQL maybe
they should use ODBC instead of thin client.

Best Regards,
Igor


On Sat, Jun 29, 2019 at 1:01 AM Denis Magda <[hidden email]> wrote:

> Igor,
>
> The user prefers using SQL syntax which is widespread. For sure, we'll
> recommend alternate solutions as long as INSERT is not suited for loading
> if a thin client is used.
>
> However, how have we supported the streaming mode for INSERTS for ODBC?
> Isn't it already based on the thin client protocol?
>
> -
> Denis
>
>
> On Fri, Jun 28, 2019 at 8:02 AM Igor Sapego <[hidden email]> wrote:
>
> > If they use python client, why wont they just use put_all()?
> >
> > To me it just looks like they are using wrong tool here.
> >
> > As for your questing, Affinity Awareness for thin clients is only for
> Cache
> > API now, it does not affect SQL.
> >
> > Best Regards,
> > Igor
> >
> >
> > On Thu, Jun 27, 2019 at 7:33 PM Denis Magda <[hidden email]> wrote:
> >
> > > Igor, Igniters,
> > >
> > > I've come across a couple of posts recently where users attempted to
> > > preload initial data with a thin client like Python:
> > >
> >
> https://stackoverflow.com/questions/56778778/apache-ignite-inserts-extremely-slow/56791088#56791088
> > >
> > > Our JDBC/ODBC drivers have a special streaming mode that boosts the
> > > loading for such use cases.
> > >
> > > Will there be any benefits from the initial data loading standpoint
> when
> > > partition-awareness is released or do we need to enhance our thin
> > clients'
> > > protocol? Users use SQL for preloading.
> > >
> > >
> > > Denis Magda
> > >
> >
>