Integration with Apache Apex

[Thomas Weise]

Dear Ignite community,

Apache Apex is a data in-motion processing platform. It was developed since
2012 and recently became an ASF top level project:

http://apex.apache.org/docs.html

The Apex engine can process large scale, high throughput streams with very
low latency. It is a stateful system with strong processing guarantees. A
unique feature of Apex is the ability to recover from failures without
resetting the entire topology and also the ability to scale or change the
application dynamically. Apex also has a large library of connectors,
supporting integration with many well known projects in the wider big data
ecosystem out of the box.

It would be great to add Ignite to the list. I think Apex as processing
engine and Ignite as storage layer would be a very powerful combination.
Here are some use cases for such integration:

   - Connectors to read from Ignite or land data in Ignite (
   https://issues.apache.org/jira/browse/IGNITE-3131)
   - Checkpointing backend (alternative implementation for Apex
   StorageAgent)
   - Backend for operator specific state management

It would be great if someone in Ignite community wants to take this up and
collaborate. There is also a ticket in the Apex JIRA to track this:
https://issues.apache.org/jira/browse/APEXMALHAR-2091

Thanks,
Thomas

[Vladisav Jelisavcic]

Hi,

it looks really interesting to me,
I'll be glad to pick this one.


Best regards,
Vladisav


On Fri, May 13, 2016 at 7:41 AM, Thomas Weise <[hidden email]> wrote:

> Dear Ignite community,
>
> Apache Apex is a data in-motion processing platform. It was developed since
> 2012 and recently became an ASF top level project:
>
> http://apex.apache.org/docs.html
>
> The Apex engine can process large scale, high throughput streams with very
> low latency. It is a stateful system with strong processing guarantees. A
> unique feature of Apex is the ability to recover from failures without
> resetting the entire topology and also the ability to scale or change the
> application dynamically. Apex also has a large library of connectors,
> supporting integration with many well known projects in the wider big data
> ecosystem out of the box.
>
> It would be great to add Ignite to the list. I think Apex as processing
> engine and Ignite as storage layer would be a very powerful combination.
> Here are some use cases for such integration:
>
>    - Connectors to read from Ignite or land data in Ignite (
>    https://issues.apache.org/jira/browse/IGNITE-3131)
>    - Checkpointing backend (alternative implementation for Apex
>    StorageAgent)
>    - Backend for operator specific state management
>
> It would be great if someone in Ignite community wants to take this up and
> collaborate. There is also a ticket in the Apex JIRA to track this:
> https://issues.apache.org/jira/browse/APEXMALHAR-2091
>
> Thanks,
> Thomas
>

[dsetrakyan]

This will be great. Feel free to assign the ticket to yourself.

On Fri, May 13, 2016 at 12:35 AM, Vladisav Jelisavcic <[hidden email]>
wrote:

> Hi,
>
> it looks really interesting to me,
> I'll be glad to pick this one.
>
>
> Best regards,
> Vladisav
>
>
> On Fri, May 13, 2016 at 7:41 AM, Thomas Weise <[hidden email]> wrote:
>
> > Dear Ignite community,
> >
> > Apache Apex is a data in-motion processing platform. It was developed
> since
> > 2012 and recently became an ASF top level project:
> >
> > http://apex.apache.org/docs.html
> >
> > The Apex engine can process large scale, high throughput streams with
> very
> > low latency. It is a stateful system with strong processing guarantees. A
> > unique feature of Apex is the ability to recover from failures without
> > resetting the entire topology and also the ability to scale or change the
> > application dynamically. Apex also has a large library of connectors,
> > supporting integration with many well known projects in the wider big
> data
> > ecosystem out of the box.
> >
> > It would be great to add Ignite to the list. I think Apex as processing
> > engine and Ignite as storage layer would be a very powerful combination.
> > Here are some use cases for such integration:
> >
> >    - Connectors to read from Ignite or land data in Ignite (
> >    https://issues.apache.org/jira/browse/IGNITE-3131)
> >    - Checkpointing backend (alternative implementation for Apex
> >    StorageAgent)
> >    - Backend for operator specific state management
> >
> > It would be great if someone in Ignite community wants to take this up
> and
> > collaborate. There is also a ticket in the Apex JIRA to track this:
> > https://issues.apache.org/jira/browse/APEXMALHAR-2091
> >
> > Thanks,
> > Thomas
> >
>

[Thomas Weise]

Hi Vladisav,

Great, thanks for taking this up. Please contact us on the Apex users or dev list if you have any questions regarding the Apex side of things:

http://apex.apache.org/community.html

You can also create a subtask here:

https://issues.apache.org/jira/browse/APEXMALHAR-2091

Thanks,
Thomas

On 2016-05-13 00:35, Vladisav Jelisavcic <[hidden email]> wrote:

> Hi,
>
> it looks really interesting to me,
> I'll be glad to pick this one.
>
>
> Best regards,
> Vladisav
>
>
> On Fri, May 13, 2016 at 7:41 AM, Thomas Weise <[hidden email]> wrote:
>
> > Dear Ignite community,
> >
> > Apache Apex is a data in-motion processing platform. It was developed since
> > 2012 and recently became an ASF top level project:
> >
> > http://apex.apache.org/docs.html
> >
> > The Apex engine can process large scale, high throughput streams with very
> > low latency. It is a stateful system with strong processing guarantees. A
> > unique feature of Apex is the ability to recover from failures without
> > resetting the entire topology and also the ability to scale or change the
> > application dynamically. Apex also has a large library of connectors,
> > supporting integration with many well known projects in the wider big data
> > ecosystem out of the box.
> >
> > It would be great to add Ignite to the list. I think Apex as processing
> > engine and Ignite as storage layer would be a very powerful combination.
> > Here are some use cases for such integration:
> >
> >    - Connectors to read from Ignite or land data in Ignite (
> >    https://issues.apache.org/jira/browse/IGNITE-3131)
> >    - Checkpointing backend (alternative implementation for Apex
> >    StorageAgent)
> >    - Backend for operator specific state management
> >
> > It would be great if someone in Ignite community wants to take this up and
> > collaborate. There is also a ticket in the Apex JIRA to track this:
> > https://issues.apache.org/jira/browse/APEXMALHAR-2091
> >
> > Thanks,
> > Thomas
> >
>