Contribution wanted: Spark Shared RDD examples

[dmagda]

Igniters,

As many of you know Ignite enhances Spark by implementing a concept of Ignite Shared RDD. This kind of RDD allows sharing the state between Spark workers and execute blazingly fast Ignite SQL queries over it.
https://apacheignite-fs.readme.io/docs/ignite-for-spark

However, by some reason there is no any ready to run example available. Ignite and Spark users have to learn basics making up their own examples all the time. Let’s fill this gap. Here is the ticket.
https://issues.apache.org/jira/browse/IGNITE-4526 <https://issues.apache.org/jira/browse/IGNITE-4526>

Is there anyone who is willing to learn both Spark and Ignite Shared RDDs better taking over this task?

—
Denis

[dmagda]

Cross-posting to the user list.

—
Denis

> On Jan 5, 2017, at 9:43 AM, Denis Magda <[hidden email]> wrote:
>
> Igniters,
>
> As many of you know Ignite enhances Spark by implementing a concept of Ignite Shared RDD. This kind of RDD allows sharing the state between Spark workers and execute blazingly fast Ignite SQL queries over it.
> https://apacheignite-fs.readme.io/docs/ignite-for-spark
>
> However, by some reason there is no any ready to run example available. Ignite and Spark users have to learn basics making up their own examples all the time. Let’s fill this gap. Here is the ticket.
> https://issues.apache.org/jira/browse/IGNITE-4526 <https://issues.apache.org/jira/browse/IGNITE-4526>
>
> Is there anyone who is willing to learn both Spark and Ignite Shared RDDs better taking over this task?
>
> —
> Denis

[Manish Mishra]

Hey, I would like to contribute. I have an already a running example use
case for it. Let me know how we want to proceed for this.
https://github.com/knoldus/spark-ignite

On Thu, Jan 5, 2017 at 11:25 PM, Denis Magda <[hidden email]> wrote:

> Cross-posting to the user list.
>
> —
> Denis
>
> > On Jan 5, 2017, at 9:43 AM, Denis Magda <[hidden email]> wrote:
> >
> > Igniters,
> >
> > As many of you know Ignite enhances Spark by implementing a concept of
> Ignite Shared RDD. This kind of RDD allows sharing the state between Spark
> workers and execute blazingly fast Ignite SQL queries over it.
> > https://apacheignite-fs.readme.io/docs/ignite-for-spark
> >
> > However, by some reason there is no any ready to run example available.
> Ignite and Spark users have to learn basics making up their own examples
> all the time. Let’s fill this gap. Here is the ticket.
> > https://issues.apache.org/jira/browse/IGNITE-4526 <
> https://issues.apache.org/jira/browse/IGNITE-4526>
> >
> > Is there anyone who is willing to learn both Spark and Ignite Shared
> RDDs better taking over this task?
> >
> > —
> > Denis
>
>

--
Thanks & Regard

Manish Mishra
Software Consultant,
Knoldus Software, LLP

[Holden Karau]

I know the Spark side pretty well if people are willing to answer my silly
ignite questions I'd be happy to give this a shot.

On Thu, Jan 5, 2017 at 9:55 AM Denis Magda <[hidden email]> wrote:

> Cross-posting to the user list.
>
>
>
> —
>
> Denis
>
>
>
> > On Jan 5, 2017, at 9:43 AM, Denis Magda <[hidden email]> wrote:
>
> >
>
> > Igniters,
>
> >
>
> > As many of you know Ignite enhances Spark by implementing a concept of
> Ignite Shared RDD. This kind of RDD allows sharing the state between Spark
> workers and execute blazingly fast Ignite SQL queries over it.
>
> > https://apacheignite-fs.readme.io/docs/ignite-for-spark
>
> >
>
> > However, by some reason there is no any ready to run example available.
> Ignite and Spark users have to learn basics making up their own examples
> all the time. Let’s fill this gap. Here is the ticket.
>
> > https://issues.apache.org/jira/browse/IGNITE-4526 <
> https://issues.apache.org/jira/browse/IGNITE-4526>
>
> >
>
> > Is there anyone who is willing to learn both Spark and Ignite Shared
> RDDs better taking over this task?
>
> >
>
> > —
>
> > Denis
>
>
>
>

[dmagda]

Hi Manish, perfect thanks!

Replied you in the ticket.

—
Denis

> On Jan 5, 2017, at 10:04 AM, Manish Mishra <[hidden email]> wrote:
>
> Hey, I would like to contribute. I have an already a running example use case for it. Let me know how we want to proceed for this. https://github.com/knoldus/spark-ignite <https://github.com/knoldus/spark-ignite>
>
> On Thu, Jan 5, 2017 at 11:25 PM, Denis Magda <[hidden email] <mailto:[hidden email]>> wrote:
> Cross-posting to the user list.
>
> —
> Denis
>
> > On Jan 5, 2017, at 9:43 AM, Denis Magda <[hidden email] <mailto:[hidden email]>> wrote:
> >
> > Igniters,
> >
> > As many of you know Ignite enhances Spark by implementing a concept of Ignite Shared RDD. This kind of RDD allows sharing the state between Spark workers and execute blazingly fast Ignite SQL queries over it.
> > https://apacheignite-fs.readme.io/docs/ignite-for-spark <https://apacheignite-fs.readme.io/docs/ignite-for-spark>
> >
> > However, by some reason there is no any ready to run example available. Ignite and Spark users have to learn basics making up their own examples all the time. Let’s fill this gap. Here is the ticket.
> > https://issues.apache.org/jira/browse/IGNITE-4526 <https://issues.apache.org/jira/browse/IGNITE-4526> <https://issues.apache.org/jira/browse/IGNITE-4526 <https://issues.apache.org/jira/browse/IGNITE-4526>>
> >
> > Is there anyone who is willing to learn both Spark and Ignite Shared RDDs better taking over this task?
> >
> > —
> > Denis
>
>
>
>
> --
> Thanks & Regard
>
> Manish Mishra
> Software Consultant,
> Knoldus Software, LLP
>

[dmagda]

Hi Holden,

Thanks for showing an interest in this! However, Manish has already took over this.

In any case, feel free to participate in the discussion of this ticket and, in general, we’re not limited to a number of examples. We can add more than one Spark related examples if you have anything in mind.

In addition, there is one bigger Spark related task sitting in the queue
https://issues.apache.org/jira/browse/IGNITE-949 <https://issues.apache.org/jira/browse/IGNITE-949>

Do you have any Python experience? Probably I would be interested of IGNITE-949 which will let you to become an Ignite expert if you finish it.

—
Denis

> On Jan 5, 2017, at 10:06 AM, Holden Karau <[hidden email]> wrote:
>
> I know the Spark side pretty well if people are willing to answer my silly ignite questions I'd be happy to give this a shot.
>
> On Thu, Jan 5, 2017 at 9:55 AM Denis Magda <[hidden email] <mailto:[hidden email]>> wrote:
> Cross-posting to the user list.
>
>
>
> —
>
> Denis
>
>
>
> > On Jan 5, 2017, at 9:43 AM, Denis Magda <[hidden email] <mailto:[hidden email]>> wrote:
>
> >
>
> > Igniters,
>
> >
>
> > As many of you know Ignite enhances Spark by implementing a concept of Ignite Shared RDD. This kind of RDD allows sharing the state between Spark workers and execute blazingly fast Ignite SQL queries over it.
>
> > https://apacheignite-fs.readme.io/docs/ignite-for-spark <https://apacheignite-fs.readme.io/docs/ignite-for-spark>
>
> >
>
> > However, by some reason there is no any ready to run example available. Ignite and Spark users have to learn basics making up their own examples all the time. Let’s fill this gap. Here is the ticket.
>
> > https://issues.apache.org/jira/browse/IGNITE-4526 <https://issues.apache.org/jira/browse/IGNITE-4526> <https://issues.apache.org/jira/browse/IGNITE-4526 <https://issues.apache.org/jira/browse/IGNITE-4526>>
>
> >
>
> > Is there anyone who is willing to learn both Spark and Ignite Shared RDDs better taking over this task?
>
> >
>
> > —
>
> > Denis
>
>
>