[DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra]

Hi Emmanouil,

The master branch has been initialized. I have raised a PR to migrate Flink
module to ignite-extensions.

PR https://github.com/apache/ignite-extensions/pull/1
Jira : https://issues.apache.org/jira/browse/IGNITE-12356

Regards,
Saikat


On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <[hidden email]>
wrote:

> Hi all!
>
> This is an update. The integration is tested with mocks based on the
> original Ignite repo [1].
> I am currently waiting for the first commits on the new integration project
> to take place in order to create my pull request accordingly to the initial
> commits.
> In order to simulate the Pub/Sub server in a test I used a test server
> implementation [2]. Unfortunately they don't have this util uploaded in a
> maven repo, thus it needs to be hosted on a maven repo (Apache License
> 2.0). Is there a repo for utilities that Ignite uses for the test/build?
> I have also created a ticket [3]
>
> Kind regards
>
> [1] https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> [2] https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> [3] https://issues.apache.org/jira/browse/IGNITE-12262
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <[hidden email]>
> wrote:
>
> > Hello Ilya,
> >
> > Thank you for your email. Yes, before we remove any integration we will
> > move them to another repository and release.
> >
> > We are thinking we will start with Pub/Sub Streamer implementation and
> > continue to move other integration as mentioned in the following doc.
> >
> >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >
> > Regards,
> > Saikat
> >
> >
> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
> > [hidden email]>
> > wrote:
> >
> > > Hello!
> > >
> > > My take on this: we want to offload some of integrations, but please
> > commit
> > > and release them to another project first, and only then we will decide
> > to
> > > remove them from our repository.
> > >
> > > Please also make sure they are available with same artifact name and
> API.
> > > If this has to be changed, let's wait for AI 3.0
> > >
> > > Regards,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <[hidden email]>:
> > >
> > > > Hello Emmanouil,
> > > >
> > > > As discussed earlier, I discussed with Apache Bahir community and
> they
> > > are
> > > > interested to have Apache Ignite extensions as part of Apache Bahir
> > > > project.
> > > >
> > > > Can you please share Pub/Sub streamer implementation details with
> > Apache
> > > > Bahir community and request for feedback.
> > > >
> > > > https://bahir.apache.org/
> > > >
> > > > I have also requested for contributor access in Jira for Apache Bahir
> > > > project so that I can create issues and assign to myself.
> > > >
> > > > I can help with code reviews as well.
> > > >
> > > > Here is the email thread for reference
> > > > https://www.mail-archive.com/dev@.../msg02703.html
> > > >
> > > > Regards,
> > > > Saikat
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
> [hidden email]
> > >
> > > > wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > As discussed I have created following discussion threads on our
> > > migration
> > > > > proposals for Apache Ignite extensions:
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> > > > >
> > > > > https://www.mail-archive.com/dev@.../msg02703.html
> > > > >
> > > > > I will share update as I hear more information.
> > > > >
> > > > > Regards,
> > > > > Saikat
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
> > [hidden email]
> > > >
> > > > > wrote:
> > > > >
> > > > >> Hello Denis,
> > > > >>
> > > > >> Sure, sounds good. I will create a separate discussion thread to
> get
> > > > >> community feedback on whether we
> > > > >> should join the Bahir or kick-off "Ignite Extensions" project.
> > > > >>
> > > > >> Regards,
> > > > >> Saikat
> > > > >>
> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <[hidden email]>
> > > wrote:
> > > > >>
> > > > >>> Folks,
> > > > >>>
> > > > >>> The concept of Apache Bahir is precisely what we need! Saikat,
> > thanks
> > > > for
> > > > >>> doing the research. Why I believe the * idea* fits us well:
> > > > >>>
> > > > >>>    - All integrations can be stored in separate Github
> repositories
> > > and
> > > > >>>    have their dev lifecycles. We've not obliged to couple all the
> > > > >>> integrations
> > > > >>>    in a single repo. For instance, Spark can be located in a
> > > dedicated
> > > > >>> repo
> > > > >>>    while streaming integrations might be in a single one. It's up
> > to
> > > > us.
> > > > >>>    - All the repositories and integrations belong to ASF. We're
> not
> > > > >>> dumping
> > > > >>>    them on Github but rather keep supporting and developing in
> > > > >>> accordance with
> > > > >>>    ASF vision and practices.
> > > > >>>    - There is a way to reward contributors via committership and
> > PMC
> > > > >>>    membership. You won't get this if a project is just one of the
> > > > >>> millions of
> > > > >>>    Github projects.
> > > > >>>
> > > > >>> Saikat, as Ignite PMC member, could you please kick-off a
> separate
> > > > >>> dev-list
> > > > >>> discussion to involve more community members and referring to
> this
> > > > >>> thread?
> > > > >>> I think the primary question the community needs to answer
> whether
> > we
> > > > >>> should join the Bahir or kick-off "Ignite Extensions" project?
> > > > >>>
> > > > >>> -
> > > > >>> Denis
> > > > >>>
> > > > >>>
> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev <
> > > > [hidden email]>
> > > > >>> wrote:
> > > > >>>
> > > > >>> > Maybe we could move all our Streaming Integrations there, but
> > what
> > > is
> > > > >>> about
> > > > >>> > maintaining and committer permissions to the new repositories?
> > > > >>> > I see the list of the committers and PMC members there
> > > > >>> > https://bahir.apache.org/community-members/
> > > > >>> >
> > > > >>> > Could somebody from Ignite community be added to this list as a
> > > > >>> > PMC/committer to maintain Ignite integrations?
> > > > >>> > If yes, I'd happy to join this project and collaborate with
> these
> > > > guys
> > > > >>> > together
> > > > >>> > If no, and we should start from the zero level with external
> PRs
> > -
> > > > >>> hmmm,
> > > > >>> > it's better to have our own external repository with
> ApacheBahirr
> > > > >>> ideology.
> > > > >>> >
> > > > >>> > Also, I agree, that all connectors could be moved there (in
> > > > >>> ApacheBahirr
> > > > >>> > like repository), but not all modules from the page
> > > > >>> >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > >>> > because
> > > > >>> > they use different parts of core modules and could be developed
> > > with
> > > > >>> > different velocity.
> > > > >>> >
> > > > >>> > In reality, before creation of new repositories we need wide
> > > > discussion
> > > > >>> > based on the document mentioned above.
> > > > >>> >
> > > > >>> > P.S. Of course, we could experiment in the next release 2.8
> with
> > > one
> > > > >>> or two
> > > > >>> > integrations like twitter or ZeroMQ
> > > > >>> > Also, I'd like Denis Magda idea for the separate repositories
> in
> > > > Apache
> > > > >>> > Github to maintain them by anybody.
> > > > >>> >
> > > > >>> >
> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra <
> > > [hidden email]
> > > > >:
> > > > >>> >
> > > > >>> > > Hi Denis, Emmanouil
> > > > >>> > >
> > > > >>> > > Thank you for your email. I think creating a separate
> > integration
> > > > >>> project
> > > > >>> > > in github and also proposing it as an apache incubator
> project
> > > will
> > > > >>> be a
> > > > >>> > > good move. The other separate integration modules
> > > > >>> > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > >>> > > can
> > > > >>> > > be moved to this new apache incubator project.
> > > > >>> > >
> > > > >>> > > There are similar integration available for Flink and Spark
> in
> > > > Apache
> > > > >>> > Bahir
> > > > >>> > > https://bahir.apache.org/
> > > > >>> > >
> > > > >>> > > Do you think we should reach out to Apache Bahir community
> > with a
> > > > >>> > proposal
> > > > >>> > > or we should plan to create a separate Apache Incubator
> > project?
> > > > >>> > >
> > > > >>> > > Regards,
> > > > >>> > > Saikat
> > > > >>> > >
> > > > >>> > >
> > > > >>> > >
> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil Gkatziouras <
> > > > >>> > > [hidden email]>
> > > > >>> > > wrote:
> > > > >>> > >
> > > > >>> > > > Hi all!
> > > > >>> > > >
> > > > >>> > > > Based on the discussions most probably this is going to be
> on
> > > > >>> another
> > > > >>> > > > GitHub repo. Therefore I will prepare a standalone project
> > with
> > > > the
> > > > >>> > > Pub/Sub
> > > > >>> > > > module and once the repository is created a pull request
> > shall
> > > be
> > > > >>> > issued.
> > > > >>> > > > If there is anything else I can do in order to facilitate
> > this
> > > > >>> please
> > > > >>> > let
> > > > >>> > > > me know.
> > > > >>> > > >
> > > > >>> > > > If a ticket is created for this issue, I shall be able to
> > > create
> > > > a
> > > > >>> pull
> > > > >>> > > > request and thus a build will take place on ignite's CI.
> > > > >>> > > > If everyone is aligned with adding a Pub/Sub implementation
> > > may I
> > > > >>> > create
> > > > >>> > > a
> > > > >>> > > > ticket on JIRA?
> > > > >>> > > >
> > > > >>> > > > My next question has to do on how Ignite configures the
> build
> > > > jobs
> > > > >>> in
> > > > >>> > > Team
> > > > >>> > > > City. The unit tests for my implementation need a local
> > Pub/Sub
> > > > >>> server
> > > > >>> > up
> > > > >>> > > > and running (they provide one for testing purposes). Will
> > this
> > > > take
> > > > >>> > > effect
> > > > >>> > > > in a form of a build script inside my implementation or is
> it
> > > > >>> something
> > > > >>> > > > that should be configured on Team City?
> > > > >>> > > >
> > > > >>> > > > Kind regards,
> > > > >>> > > > Emmanouil
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > > > *Emmanouil Gkatziouras*
> > > > >>> > > > https://egkatzioura.com/ |
> > > > >>> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >>> > > > https://github.com/gkatzioura
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis Magda <
> [hidden email]
> > >
> > > > >>> wrote:
> > > > >>> > > >
> > > > >>> > > > > Emmanouil, Saikat,
> > > > >>> > > > >
> > > > >>> > > > > After contemplating on this, let me propose another way
> > with
> > > > this
> > > > >>> > > > > particular integration that is aligned with our
> > > modularization
> > > > >>> > > endeavor.
> > > > >>> > > > > The modularization [1] defines the Ignite core, Ignite
> > > modules,
> > > > >>> and
> > > > >>> > > > > independent integrations. The first two are to be located
> > in
> > > > >>> Ignite
> > > > >>> > > > > repositories and to be tested/updated by the community
> for
> > > > every
> > > > >>> > > release.
> > > > >>> > > > > The modules are a subset of existing integrations that
> are
> > > > >>> adopted
> > > > >>> > > widely
> > > > >>> > > > > and require community attention.
> > > > >>> > > > >
> > > > >>> > > > > As for the independent integrations [1], those are
> supposed
> > > to
> > > > be
> > > > >>> > > located
> > > > >>> > > > > in a separate GitHub repo. I would advise us to start
> > > > >>> implementing
> > > > >>> > this
> > > > >>> > > > > concept with the Pub/Sub integration. Even though it will
> > be
> > > > >>> located
> > > > >>> > > in a
> > > > >>> > > > > separate Github repo, it will be listed among officially
> > > > >>> available
> > > > >>> > > > > integrations on the Ignite website featuring Emmanouil
> as a
> > > > >>> > > contributor.
> > > > >>> > > > >
> > > > >>> > > > > What do you think?
> > > > >>> > > > >
> > > > >>> > > > >
> > > > >>> > > > > [1]
> > > > >>> > > > >
> > > > >>> > > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > >>> > > > >
> > > > >>> > > > >
> > > > >>> > > > >
> > > > >>> > > > > -
> > > > >>> > > > > Denis
> > > > >>> > > > >
> > > > >>> > > > >
> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat Maitra <
> > > > >>> > [hidden email]
> > > > >>> > > >
> > > > >>> > > > > wrote:
> > > > >>> > > > >
> > > > >>> > > > > > Hi Emmanouil,
> > > > >>> > > > > >
> > > > >>> > > > > > The changes looks good to me. I wanted to check if you
> > are
> > > > also
> > > > >>> > able
> > > > >>> > > to
> > > > >>> > > > > > validate that the once data is added to Ignite cluster
> > you
> > > > are
> > > > >>> also
> > > > >>> > > > able
> > > > >>> > > > > to
> > > > >>> > > > > > access it using another Ignite client or using rest
> > > > endpoints.
> > > > >>> > > > > >
> > > > >>> > > > > > We use Teamcity for CI process, the details are as
> > follows
> > > > >>> > > > > >
> > > > >>> > > > > > Trigger validation of those test suites that have been
> > > > >>> affected by
> > > > >>> > > your
> > > > >>> > > > > > changes on TeamCity <http://ci.ignite.apache.org/>: <
> > > > >>> > > > > http://204.14.53.153/
> > > > >>> > > > > > >
> > > > >>> > > > > >
> > > > >>> > > > > >    - Locate a test suite you have to check, press
> button
> > > > named
> > > > >>> > "..."
> > > > >>> > > > that
> > > > >>> > > > > >    is located on the left of "Run" button. "Run custom
> > > build"
> > > > >>> > dialog
> > > > >>> > > > will
> > > > >>> > > > > >    appear;
> > > > >>> > > > > >    - Go to "Changes" tab and choose
> > > > >>> > "pull/<pull-request-number>/head"
> > > > >>> > > > in
> > > > >>> > > > > >    "Build branch" dropdown list;
> > > > >>> > > > > >    - Press "Run build" button and monitor tests
> results.
> > > > >>> > > > > >
> > > > >>> > > > > >
> > > > >>> > > > > >
> > > > >>> > > > > > @Roman [hidden email] - can you also please take a
> > > look?
> > > > >>> > > > > >
> > > > >>> > > > > > Regards,
> > > > >>> > > > > > Saikat
> > > > >>> > > > > >
> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM Emmanouil Gkatziouras <
> > > > >>> > > > > > [hidden email]>
> > > > >>> > > > > > wrote:
> > > > >>> > > > > >
> > > > >>> > > > > > > Hi @Saikat Maitra
> > > > >>> > > > > > >
> > > > >>> > > > > > > I have my implementation on an Ignite fork on my
> > personal
> > > > >>> GitHub
> > > > >>> > > > > account
> > > > >>> > > > > > > [1]
> > > > >>> > > > > > > It is based on the Kafka implementation which is of a
> > > > polling
> > > > >>> > > nature.
> > > > >>> > > > > > > GCP does not provide a service like Kafka. Instead it
> > > > >>> provides us
> > > > >>> > > > with
> > > > >>> > > > > > > Pub/Sub which is just like Kafka.
> > > > >>> > > > > > > Therefore using the Kafka streamer as a guideline
> made
> > > more
> > > > >>> sense
> > > > >>> > > to
> > > > >>> > > > > me.
> > > > >>> > > > > > >
> > > > >>> > > > > > > Apart from the implementation I have also added a
> unit
> > > > test.
> > > > >>> > > > > > > It definitely needs some polishing and a double check
> > on
> > > my
> > > > >>> side
> > > > >>> > to
> > > > >>> > > > > make
> > > > >>> > > > > > > the review smooth.
> > > > >>> > > > > > > For the tests a local Pub/Sub [2] server has to be
> > setup
> > > > >>> however
> > > > >>> > I
> > > > >>> > > am
> > > > >>> > > > > not
> > > > >>> > > > > > > sure which is the best way to do so with regards to
> > your
> > > > >>> CI/CD.
> > > > >>> > > > > > >
> > > > >>> > > > > > > [1]
> > > > >>> > >
> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > > > >>> > > > > > > [2]
> > > > >>> https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
> > > > >>> > > > > > >
> > > > >>> > > > > > > *Emmanouil Gkatziouras*
> > > > >>> > > > > > > https://egkatzioura.com/ |
> > > > >>> > > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >>> > > > > > > https://github.com/gkatzioura
> > > > >>> > > > > > >
> > > > >>> > > > > > >
> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11, Saikat Maitra <
> > > > >>> > > [hidden email]
> > > > >>> > > > >
> > > > >>> > > > > > > wrote:
> > > > >>> > > > > > >
> > > > >>> > > > > > > > Hello Emmanouil,
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > Can you please share more info about the
> > > implementation?
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > We have similar implementation for data streamers
> > with
> > > > >>> Kafka
> > > > >>> > and
> > > > >>> > > > > > RocketMQ
> > > > >>> > > > > > > >
> > > > >>> > > > > > > >
> > https://apacheignite-mix.readme.io/docs/kafka-streamer
> > > > >>> > > > > > > >
> > > > >>> > > > > > > >
> > > > https://apacheignite-mix.readme.io/docs/rocketmq-streamer
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > Please review and share if you have any questions.
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > Regards,
> > > > >>> > > > > > > > Saikat
> > > > >>> > > > > > > >
> > > > >>> > > > > > > >
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56 PM Emmanouil
> > Gkatziouras <
> > > > >>> > > > > > > > [hidden email]>
> > > > >>> > > > > > > > wrote:
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > > Hi all,
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > By mentioning Pub/Sub I mean Google Cloud's
> Pub/Sub
> > > [1]
> > > > >>> > > > > > > > > Yes it is indeed cloud specific. It is the
> > messaging
> > > > >>> service
> > > > >>> > > > > provided
> > > > >>> > > > > > > by
> > > > >>> > > > > > > > > GCP.
> > > > >>> > > > > > > > > I have made the integration with it, but I
> haven't
> > > > >>> published
> > > > >>> > it
> > > > >>> > > > yet
> > > > >>> > > > > > (I
> > > > >>> > > > > > > am
> > > > >>> > > > > > > > > waiting for the discussion/ticket and proceed
> with
> > > the
> > > > >>> > > > > contribution).
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > Kind regards,
> > > > >>> > > > > > > > > Emmanouil
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > [1]
> https://cloud.google.com/pubsub/docs/overview
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
> > > > >>> > > > > > > > > https://egkatzioura.com/ |
> > > > >>> > > > > > > > >
> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >>> > > > > > > > > https://github.com/gkatzioura
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at 21:45, Denis Magda <
> > > > >>> [hidden email]
> > > > >>> > >
> > > > >>> > > > > wrote:
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > > Hello Emmanouil,
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > Are you referring to any specific
> implementation
> > > > >>> tailored
> > > > >>> > > for a
> > > > >>> > > > > > > cloud?
> > > > >>> > > > > > > > If
> > > > >>> > > > > > > > > > I'm not mistaken you were mentioning AWS/GCE.
> Is
> > > > there
> > > > >>> any
> > > > >>> > > > Github
> > > > >>> > > > > > > > source
> > > > >>> > > > > > > > > > code?
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > Here is a list of existing integrations with
> > > > streaming
> > > > >>> and
> > > > >>> > > > > pub/sub
> > > > >>> > > > > > > > > > frameworks:
> > > > >>> > > > > > > > > >
> https://apacheignite-mix.readme.io/docs/overview
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > with the source code in Ignite master:
> > > > >>> > > > > > > > > >
> > > https://github.com/apache/ignite/tree/master/modules
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > Check out those resources to see what are the
> > most
> > > > >>> suitable
> > > > >>> > > > > Ignite
> > > > >>> > > > > > > APIs
> > > > >>> > > > > > > > > for
> > > > >>> > > > > > > > > > such integrations.
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > @Saikat Maitra <[hidden email]> ,
> > @Roman
> > > <
> > > > >>> > > > > > [hidden email]
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > ,
> > > > >>> > > > > > > > > as
> > > > >>> > > > > > > > > > streaming technologies
> contributors/maintainers,
> > > > could
> > > > >>> you
> > > > >>> > > > please
> > > > >>> > > > > > > join
> > > > >>> > > > > > > > > this
> > > > >>> > > > > > > > > > thread?
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > -
> > > > >>> > > > > > > > > > Denis
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > On Thu, Oct 10, 2019 at 7:58 AM Emmanouil
> > > > Gkatziouras <
> > > > >>> > > > > > > > > > [hidden email]>
> > > > >>> > > > > > > > > > wrote:
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > > Hello everyone,
> > > > >>> > > > > > > > > > >
> > > > >>> > > > > > > > > > > I started using Ignite lately. Part of my
> work
> > > > >>> involves
> > > > >>> > > > > Pub/Sub,
> > > > >>> > > > > > > > thus I
> > > > >>> > > > > > > > > > > created a pub/sub streamer.
> > > > >>> > > > > > > > > > > Pub/Sub is pretty close to Kafka.
> > > > >>> > > > > > > > > > > Here's how the streamer works. Every node
> > creates
> > > > an
> > > > >>> > > instance
> > > > >>> > > > > of
> > > > >>> > > > > > a
> > > > >>> > > > > > > > > > Pub/Sub
> > > > >>> > > > > > > > > > > client.
> > > > >>> > > > > > > > > > > The clients subscribe to the same topic using
> > the
> > > > >>> same
> > > > >>> > > > > > subscription
> > > > >>> > > > > > > > > name.
> > > > >>> > > > > > > > > > > A message is being sent and one of the nodes
> > will
> > > > >>> receive
> > > > >>> > > the
> > > > >>> > > > > > > > message,
> > > > >>> > > > > > > > > > > process it and then forward it to Ignite
> using
> > > the
> > > > >>> > > extractor
> > > > >>> > > > > > > > provided.
> > > > >>> > > > > > > > > > >
> > > > >>> > > > > > > > > > > Any thoughts?
> > > > >>> > > > > > > > > > >
> > > > >>> > > > > > > > > > > *Emmanouil Gkatziouras*
> > > > >>> > > > > > > > > > > https://egkatzioura.com/ |
> > > > >>> > > > > > > > > > >
> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >>> > > > > > > > > > > https://github.com/gkatzioura
> > > > >>> > > > > > > > > > >
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > >
> > > > >>> > > > > > >
> > > > >>> > > > > >
> > > > >>> > > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > > >>
> > > >
> > >
> >
>

[Saikat Maitra]

Hi Emmanouil,

Can you please take a looks at dev utils, if this is something you are
looking for https://github.com/apache/ignite/tree/master/modules/dev-utils?

IMO, if you can release Pub/Sub server in maven and then use it as
dependency, that would be great.

Regards,
Saikat

On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <[hidden email]>
wrote:

> Hi Emmanouil,
>
> The master branch has been initialized. I have raised a PR to migrate
> Flink module to ignite-extensions.
>
> PR https://github.com/apache/ignite-extensions/pull/1
> Jira : https://issues.apache.org/jira/browse/IGNITE-12356
>
> Regards,
> Saikat
>
>
> On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
> [hidden email]> wrote:
>
>> Hi all!
>>
>> This is an update. The integration is tested with mocks based on the
>> original Ignite repo [1].
>> I am currently waiting for the first commits on the new integration
>> project
>> to take place in order to create my pull request accordingly to the
>> initial
>> commits.
>> In order to simulate the Pub/Sub server in a test I used a test server
>> implementation [2]. Unfortunately they don't have this util uploaded in a
>> maven repo, thus it needs to be hosted on a maven repo (Apache License
>> 2.0). Is there a repo for utilities that Ignite uses for the test/build?
>> I have also created a ticket [3]
>>
>> Kind regards
>>
>> [1] https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
>> [2] https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
>> [3] https://issues.apache.org/jira/browse/IGNITE-12262
>>
>> *Emmanouil Gkatziouras*
>> https://egkatzioura.com/ |
>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> https://github.com/gkatzioura
>>
>>
>> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <[hidden email]>
>> wrote:
>>
>> > Hello Ilya,
>> >
>> > Thank you for your email. Yes, before we remove any integration we will
>> > move them to another repository and release.
>> >
>> > We are thinking we will start with Pub/Sub Streamer implementation and
>> > continue to move other integration as mentioned in the following doc.
>> >
>> >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> >
>> > Regards,
>> > Saikat
>> >
>> >
>> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
>> > [hidden email]>
>> > wrote:
>> >
>> > > Hello!
>> > >
>> > > My take on this: we want to offload some of integrations, but please
>> > commit
>> > > and release them to another project first, and only then we will
>> decide
>> > to
>> > > remove them from our repository.
>> > >
>> > > Please also make sure they are available with same artifact name and
>> API.
>> > > If this has to be changed, let's wait for AI 3.0
>> > >
>> > > Regards,
>> > > --
>> > > Ilya Kasnacheev
>> > >
>> > >
>> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <[hidden email]>:
>> > >
>> > > > Hello Emmanouil,
>> > > >
>> > > > As discussed earlier, I discussed with Apache Bahir community and
>> they
>> > > are
>> > > > interested to have Apache Ignite extensions as part of Apache Bahir
>> > > > project.
>> > > >
>> > > > Can you please share Pub/Sub streamer implementation details with
>> > Apache
>> > > > Bahir community and request for feedback.
>> > > >
>> > > > https://bahir.apache.org/
>> > > >
>> > > > I have also requested for contributor access in Jira for Apache
>> Bahir
>> > > > project so that I can create issues and assign to myself.
>> > > >
>> > > > I can help with code reviews as well.
>> > > >
>> > > > Here is the email thread for reference
>> > > > https://www.mail-archive.com/dev@.../msg02703.html
>> > > >
>> > > > Regards,
>> > > > Saikat
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
>> [hidden email]
>> > >
>> > > > wrote:
>> > > >
>> > > > > Hello,
>> > > > >
>> > > > > As discussed I have created following discussion threads on our
>> > > migration
>> > > > > proposals for Apache Ignite extensions:
>> > > > >
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
>> > > > >
>> > > > > https://www.mail-archive.com/dev@.../msg02703.html
>> > > > >
>> > > > > I will share update as I hear more information.
>> > > > >
>> > > > > Regards,
>> > > > > Saikat
>> > > > >
>> > > > >
>> > > > >
>> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
>> > [hidden email]
>> > > >
>> > > > > wrote:
>> > > > >
>> > > > >> Hello Denis,
>> > > > >>
>> > > > >> Sure, sounds good. I will create a separate discussion thread to
>> get
>> > > > >> community feedback on whether we
>> > > > >> should join the Bahir or kick-off "Ignite Extensions" project.
>> > > > >>
>> > > > >> Regards,
>> > > > >> Saikat
>> > > > >>
>> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <[hidden email]>
>> > > wrote:
>> > > > >>
>> > > > >>> Folks,
>> > > > >>>
>> > > > >>> The concept of Apache Bahir is precisely what we need! Saikat,
>> > thanks
>> > > > for
>> > > > >>> doing the research. Why I believe the * idea* fits us well:
>> > > > >>>
>> > > > >>>    - All integrations can be stored in separate Github
>> repositories
>> > > and
>> > > > >>>    have their dev lifecycles. We've not obliged to couple all
>> the
>> > > > >>> integrations
>> > > > >>>    in a single repo. For instance, Spark can be located in a
>> > > dedicated
>> > > > >>> repo
>> > > > >>>    while streaming integrations might be in a single one. It's
>> up
>> > to
>> > > > us.
>> > > > >>>    - All the repositories and integrations belong to ASF. We're
>> not
>> > > > >>> dumping
>> > > > >>>    them on Github but rather keep supporting and developing in
>> > > > >>> accordance with
>> > > > >>>    ASF vision and practices.
>> > > > >>>    - There is a way to reward contributors via committership and
>> > PMC
>> > > > >>>    membership. You won't get this if a project is just one of
>> the
>> > > > >>> millions of
>> > > > >>>    Github projects.
>> > > > >>>
>> > > > >>> Saikat, as Ignite PMC member, could you please kick-off a
>> separate
>> > > > >>> dev-list
>> > > > >>> discussion to involve more community members and referring to
>> this
>> > > > >>> thread?
>> > > > >>> I think the primary question the community needs to answer
>> whether
>> > we
>> > > > >>> should join the Bahir or kick-off "Ignite Extensions" project?
>> > > > >>>
>> > > > >>> -
>> > > > >>> Denis
>> > > > >>>
>> > > > >>>
>> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev <
>> > > > [hidden email]>
>> > > > >>> wrote:
>> > > > >>>
>> > > > >>> > Maybe we could move all our Streaming Integrations there, but
>> > what
>> > > is
>> > > > >>> about
>> > > > >>> > maintaining and committer permissions to the new repositories?
>> > > > >>> > I see the list of the committers and PMC members there
>> > > > >>> > https://bahir.apache.org/community-members/
>> > > > >>> >
>> > > > >>> > Could somebody from Ignite community be added to this list as
>> a
>> > > > >>> > PMC/committer to maintain Ignite integrations?
>> > > > >>> > If yes, I'd happy to join this project and collaborate with
>> these
>> > > > guys
>> > > > >>> > together
>> > > > >>> > If no, and we should start from the zero level with external
>> PRs
>> > -
>> > > > >>> hmmm,
>> > > > >>> > it's better to have our own external repository with
>> ApacheBahirr
>> > > > >>> ideology.
>> > > > >>> >
>> > > > >>> > Also, I agree, that all connectors could be moved there (in
>> > > > >>> ApacheBahirr
>> > > > >>> > like repository), but not all modules from the page
>> > > > >>> >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> > > > >>> > because
>> > > > >>> > they use different parts of core modules and could be
>> developed
>> > > with
>> > > > >>> > different velocity.
>> > > > >>> >
>> > > > >>> > In reality, before creation of new repositories we need wide
>> > > > discussion
>> > > > >>> > based on the document mentioned above.
>> > > > >>> >
>> > > > >>> > P.S. Of course, we could experiment in the next release 2.8
>> with
>> > > one
>> > > > >>> or two
>> > > > >>> > integrations like twitter or ZeroMQ
>> > > > >>> > Also, I'd like Denis Magda idea for the separate repositories
>> in
>> > > > Apache
>> > > > >>> > Github to maintain them by anybody.
>> > > > >>> >
>> > > > >>> >
>> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra <
>> > > [hidden email]
>> > > > >:
>> > > > >>> >
>> > > > >>> > > Hi Denis, Emmanouil
>> > > > >>> > >
>> > > > >>> > > Thank you for your email. I think creating a separate
>> > integration
>> > > > >>> project
>> > > > >>> > > in github and also proposing it as an apache incubator
>> project
>> > > will
>> > > > >>> be a
>> > > > >>> > > good move. The other separate integration modules
>> > > > >>> > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> > > > >>> > > can
>> > > > >>> > > be moved to this new apache incubator project.
>> > > > >>> > >
>> > > > >>> > > There are similar integration available for Flink and Spark
>> in
>> > > > Apache
>> > > > >>> > Bahir
>> > > > >>> > > https://bahir.apache.org/
>> > > > >>> > >
>> > > > >>> > > Do you think we should reach out to Apache Bahir community
>> > with a
>> > > > >>> > proposal
>> > > > >>> > > or we should plan to create a separate Apache Incubator
>> > project?
>> > > > >>> > >
>> > > > >>> > > Regards,
>> > > > >>> > > Saikat
>> > > > >>> > >
>> > > > >>> > >
>> > > > >>> > >
>> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil Gkatziouras <
>> > > > >>> > > [hidden email]>
>> > > > >>> > > wrote:
>> > > > >>> > >
>> > > > >>> > > > Hi all!
>> > > > >>> > > >
>> > > > >>> > > > Based on the discussions most probably this is going to
>> be on
>> > > > >>> another
>> > > > >>> > > > GitHub repo. Therefore I will prepare a standalone project
>> > with
>> > > > the
>> > > > >>> > > Pub/Sub
>> > > > >>> > > > module and once the repository is created a pull request
>> > shall
>> > > be
>> > > > >>> > issued.
>> > > > >>> > > > If there is anything else I can do in order to facilitate
>> > this
>> > > > >>> please
>> > > > >>> > let
>> > > > >>> > > > me know.
>> > > > >>> > > >
>> > > > >>> > > > If a ticket is created for this issue, I shall be able to
>> > > create
>> > > > a
>> > > > >>> pull
>> > > > >>> > > > request and thus a build will take place on ignite's CI.
>> > > > >>> > > > If everyone is aligned with adding a Pub/Sub
>> implementation
>> > > may I
>> > > > >>> > create
>> > > > >>> > > a
>> > > > >>> > > > ticket on JIRA?
>> > > > >>> > > >
>> > > > >>> > > > My next question has to do on how Ignite configures the
>> build
>> > > > jobs
>> > > > >>> in
>> > > > >>> > > Team
>> > > > >>> > > > City. The unit tests for my implementation need a local
>> > Pub/Sub
>> > > > >>> server
>> > > > >>> > up
>> > > > >>> > > > and running (they provide one for testing purposes). Will
>> > this
>> > > > take
>> > > > >>> > > effect
>> > > > >>> > > > in a form of a build script inside my implementation or
>> is it
>> > > > >>> something
>> > > > >>> > > > that should be configured on Team City?
>> > > > >>> > > >
>> > > > >>> > > > Kind regards,
>> > > > >>> > > > Emmanouil
>> > > > >>> > > >
>> > > > >>> > > >
>> > > > >>> > > > *Emmanouil Gkatziouras*
>> > > > >>> > > > https://egkatzioura.com/ |
>> > > > >>> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > >>> > > > https://github.com/gkatzioura
>> > > > >>> > > >
>> > > > >>> > > >
>> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis Magda <
>> [hidden email]
>> > >
>> > > > >>> wrote:
>> > > > >>> > > >
>> > > > >>> > > > > Emmanouil, Saikat,
>> > > > >>> > > > >
>> > > > >>> > > > > After contemplating on this, let me propose another way
>> > with
>> > > > this
>> > > > >>> > > > > particular integration that is aligned with our
>> > > modularization
>> > > > >>> > > endeavor.
>> > > > >>> > > > > The modularization [1] defines the Ignite core, Ignite
>> > > modules,
>> > > > >>> and
>> > > > >>> > > > > independent integrations. The first two are to be
>> located
>> > in
>> > > > >>> Ignite
>> > > > >>> > > > > repositories and to be tested/updated by the community
>> for
>> > > > every
>> > > > >>> > > release.
>> > > > >>> > > > > The modules are a subset of existing integrations that
>> are
>> > > > >>> adopted
>> > > > >>> > > widely
>> > > > >>> > > > > and require community attention.
>> > > > >>> > > > >
>> > > > >>> > > > > As for the independent integrations [1], those are
>> supposed
>> > > to
>> > > > be
>> > > > >>> > > located
>> > > > >>> > > > > in a separate GitHub repo. I would advise us to start
>> > > > >>> implementing
>> > > > >>> > this
>> > > > >>> > > > > concept with the Pub/Sub integration. Even though it
>> will
>> > be
>> > > > >>> located
>> > > > >>> > > in a
>> > > > >>> > > > > separate Github repo, it will be listed among officially
>> > > > >>> available
>> > > > >>> > > > > integrations on the Ignite website featuring Emmanouil
>> as a
>> > > > >>> > > contributor.
>> > > > >>> > > > >
>> > > > >>> > > > > What do you think?
>> > > > >>> > > > >
>> > > > >>> > > > >
>> > > > >>> > > > > [1]
>> > > > >>> > > > >
>> > > > >>> > > > >
>> > > > >>> > > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> > > > >>> > > > >
>> > > > >>> > > > >
>> > > > >>> > > > >
>> > > > >>> > > > > -
>> > > > >>> > > > > Denis
>> > > > >>> > > > >
>> > > > >>> > > > >
>> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat Maitra <
>> > > > >>> > [hidden email]
>> > > > >>> > > >
>> > > > >>> > > > > wrote:
>> > > > >>> > > > >
>> > > > >>> > > > > > Hi Emmanouil,
>> > > > >>> > > > > >
>> > > > >>> > > > > > The changes looks good to me. I wanted to check if you
>> > are
>> > > > also
>> > > > >>> > able
>> > > > >>> > > to
>> > > > >>> > > > > > validate that the once data is added to Ignite cluster
>> > you
>> > > > are
>> > > > >>> also
>> > > > >>> > > > able
>> > > > >>> > > > > to
>> > > > >>> > > > > > access it using another Ignite client or using rest
>> > > > endpoints.
>> > > > >>> > > > > >
>> > > > >>> > > > > > We use Teamcity for CI process, the details are as
>> > follows
>> > > > >>> > > > > >
>> > > > >>> > > > > > Trigger validation of those test suites that have been
>> > > > >>> affected by
>> > > > >>> > > your
>> > > > >>> > > > > > changes on TeamCity <http://ci.ignite.apache.org/>: <
>> > > > >>> > > > > http://204.14.53.153/
>> > > > >>> > > > > > >
>> > > > >>> > > > > >
>> > > > >>> > > > > >    - Locate a test suite you have to check, press
>> button
>> > > > named
>> > > > >>> > "..."
>> > > > >>> > > > that
>> > > > >>> > > > > >    is located on the left of "Run" button. "Run custom
>> > > build"
>> > > > >>> > dialog
>> > > > >>> > > > will
>> > > > >>> > > > > >    appear;
>> > > > >>> > > > > >    - Go to "Changes" tab and choose
>> > > > >>> > "pull/<pull-request-number>/head"
>> > > > >>> > > > in
>> > > > >>> > > > > >    "Build branch" dropdown list;
>> > > > >>> > > > > >    - Press "Run build" button and monitor tests
>> results.
>> > > > >>> > > > > >
>> > > > >>> > > > > >
>> > > > >>> > > > > >
>> > > > >>> > > > > > @Roman [hidden email] - can you also please take a
>> > > look?
>> > > > >>> > > > > >
>> > > > >>> > > > > > Regards,
>> > > > >>> > > > > > Saikat
>> > > > >>> > > > > >
>> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM Emmanouil Gkatziouras
>> <
>> > > > >>> > > > > > [hidden email]>
>> > > > >>> > > > > > wrote:
>> > > > >>> > > > > >
>> > > > >>> > > > > > > Hi @Saikat Maitra
>> > > > >>> > > > > > >
>> > > > >>> > > > > > > I have my implementation on an Ignite fork on my
>> > personal
>> > > > >>> GitHub
>> > > > >>> > > > > account
>> > > > >>> > > > > > > [1]
>> > > > >>> > > > > > > It is based on the Kafka implementation which is of
>> a
>> > > > polling
>> > > > >>> > > nature.
>> > > > >>> > > > > > > GCP does not provide a service like Kafka. Instead
>> it
>> > > > >>> provides us
>> > > > >>> > > > with
>> > > > >>> > > > > > > Pub/Sub which is just like Kafka.
>> > > > >>> > > > > > > Therefore using the Kafka streamer as a guideline
>> made
>> > > more
>> > > > >>> sense
>> > > > >>> > > to
>> > > > >>> > > > > me.
>> > > > >>> > > > > > >
>> > > > >>> > > > > > > Apart from the implementation I have also added a
>> unit
>> > > > test.
>> > > > >>> > > > > > > It definitely needs some polishing and a double
>> check
>> > on
>> > > my
>> > > > >>> side
>> > > > >>> > to
>> > > > >>> > > > > make
>> > > > >>> > > > > > > the review smooth.
>> > > > >>> > > > > > > For the tests a local Pub/Sub [2] server has to be
>> > setup
>> > > > >>> however
>> > > > >>> > I
>> > > > >>> > > am
>> > > > >>> > > > > not
>> > > > >>> > > > > > > sure which is the best way to do so with regards to
>> > your
>> > > > >>> CI/CD.
>> > > > >>> > > > > > >
>> > > > >>> > > > > > > [1]
>> > > > >>> > >
>> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
>> > > > >>> > > > > > > [2]
>> > > > >>> https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
>> > > > >>> > > > > > >
>> > > > >>> > > > > > > *Emmanouil Gkatziouras*
>> > > > >>> > > > > > > https://egkatzioura.com/ |
>> > > > >>> > > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > >>> > > > > > > https://github.com/gkatzioura
>> > > > >>> > > > > > >
>> > > > >>> > > > > > >
>> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11, Saikat Maitra <
>> > > > >>> > > [hidden email]
>> > > > >>> > > > >
>> > > > >>> > > > > > > wrote:
>> > > > >>> > > > > > >
>> > > > >>> > > > > > > > Hello Emmanouil,
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > Can you please share more info about the
>> > > implementation?
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > We have similar implementation for data streamers
>> > with
>> > > > >>> Kafka
>> > > > >>> > and
>> > > > >>> > > > > > RocketMQ
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > >
>> > https://apacheignite-mix.readme.io/docs/kafka-streamer
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > >
>> > > > https://apacheignite-mix.readme.io/docs/rocketmq-streamer
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > Please review and share if you have any questions.
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > Regards,
>> > > > >>> > > > > > > > Saikat
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56 PM Emmanouil
>> > Gkatziouras <
>> > > > >>> > > > > > > > [hidden email]>
>> > > > >>> > > > > > > > wrote:
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > > Hi all,
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > By mentioning Pub/Sub I mean Google Cloud's
>> Pub/Sub
>> > > [1]
>> > > > >>> > > > > > > > > Yes it is indeed cloud specific. It is the
>> > messaging
>> > > > >>> service
>> > > > >>> > > > > provided
>> > > > >>> > > > > > > by
>> > > > >>> > > > > > > > > GCP.
>> > > > >>> > > > > > > > > I have made the integration with it, but I
>> haven't
>> > > > >>> published
>> > > > >>> > it
>> > > > >>> > > > yet
>> > > > >>> > > > > > (I
>> > > > >>> > > > > > > am
>> > > > >>> > > > > > > > > waiting for the discussion/ticket and proceed
>> with
>> > > the
>> > > > >>> > > > > contribution).
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > Kind regards,
>> > > > >>> > > > > > > > > Emmanouil
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > [1]
>> https://cloud.google.com/pubsub/docs/overview
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
>> > > > >>> > > > > > > > > https://egkatzioura.com/ |
>> > > > >>> > > > > > > > >
>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > >>> > > > > > > > > https://github.com/gkatzioura
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at 21:45, Denis Magda <
>> > > > >>> [hidden email]
>> > > > >>> > >
>> > > > >>> > > > > wrote:
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > > Hello Emmanouil,
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > Are you referring to any specific
>> implementation
>> > > > >>> tailored
>> > > > >>> > > for a
>> > > > >>> > > > > > > cloud?
>> > > > >>> > > > > > > > If
>> > > > >>> > > > > > > > > > I'm not mistaken you were mentioning AWS/GCE.
>> Is
>> > > > there
>> > > > >>> any
>> > > > >>> > > > Github
>> > > > >>> > > > > > > > source
>> > > > >>> > > > > > > > > > code?
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > Here is a list of existing integrations with
>> > > > streaming
>> > > > >>> and
>> > > > >>> > > > > pub/sub
>> > > > >>> > > > > > > > > > frameworks:
>> > > > >>> > > > > > > > > >
>> https://apacheignite-mix.readme.io/docs/overview
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > with the source code in Ignite master:
>> > > > >>> > > > > > > > > >
>> > > https://github.com/apache/ignite/tree/master/modules
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > Check out those resources to see what are the
>> > most
>> > > > >>> suitable
>> > > > >>> > > > > Ignite
>> > > > >>> > > > > > > APIs
>> > > > >>> > > > > > > > > for
>> > > > >>> > > > > > > > > > such integrations.
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > @Saikat Maitra <[hidden email]> ,
>> > @Roman
>> > > <
>> > > > >>> > > > > > [hidden email]
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > ,
>> > > > >>> > > > > > > > > as
>> > > > >>> > > > > > > > > > streaming technologies
>> contributors/maintainers,
>> > > > could
>> > > > >>> you
>> > > > >>> > > > please
>> > > > >>> > > > > > > join
>> > > > >>> > > > > > > > > this
>> > > > >>> > > > > > > > > > thread?
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > -
>> > > > >>> > > > > > > > > > Denis
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > On Thu, Oct 10, 2019 at 7:58 AM Emmanouil
>> > > > Gkatziouras <
>> > > > >>> > > > > > > > > > [hidden email]>
>> > > > >>> > > > > > > > > > wrote:
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > > Hello everyone,
>> > > > >>> > > > > > > > > > >
>> > > > >>> > > > > > > > > > > I started using Ignite lately. Part of my
>> work
>> > > > >>> involves
>> > > > >>> > > > > Pub/Sub,
>> > > > >>> > > > > > > > thus I
>> > > > >>> > > > > > > > > > > created a pub/sub streamer.
>> > > > >>> > > > > > > > > > > Pub/Sub is pretty close to Kafka.
>> > > > >>> > > > > > > > > > > Here's how the streamer works. Every node
>> > creates
>> > > > an
>> > > > >>> > > instance
>> > > > >>> > > > > of
>> > > > >>> > > > > > a
>> > > > >>> > > > > > > > > > Pub/Sub
>> > > > >>> > > > > > > > > > > client.
>> > > > >>> > > > > > > > > > > The clients subscribe to the same topic
>> using
>> > the
>> > > > >>> same
>> > > > >>> > > > > > subscription
>> > > > >>> > > > > > > > > name.
>> > > > >>> > > > > > > > > > > A message is being sent and one of the nodes
>> > will
>> > > > >>> receive
>> > > > >>> > > the
>> > > > >>> > > > > > > > message,
>> > > > >>> > > > > > > > > > > process it and then forward it to Ignite
>> using
>> > > the
>> > > > >>> > > extractor
>> > > > >>> > > > > > > > provided.
>> > > > >>> > > > > > > > > > >
>> > > > >>> > > > > > > > > > > Any thoughts?
>> > > > >>> > > > > > > > > > >
>> > > > >>> > > > > > > > > > > *Emmanouil Gkatziouras*
>> > > > >>> > > > > > > > > > > https://egkatzioura.com/ |
>> > > > >>> > > > > > > > > > >
>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > >>> > > > > > > > > > > https://github.com/gkatzioura
>> > > > >>> > > > > > > > > > >
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > >
>> > > > >>> > > > > >
>> > > > >>> > > > >
>> > > > >>> > > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > > >>
>> > > >
>> > >
>> >
>>
>

[dmagda]

Hi Emmanouil,

Do you have any questions or need any support from the community?

-
Denis


On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <[hidden email]>
wrote:

> Hi Emmanouil,
>
> Can you please take a looks at dev utils, if this is something you are
> looking for https://github.com/apache/ignite/tree/master/modules/dev-utils
> ?
>
> IMO, if you can release Pub/Sub server in maven and then use it as
> dependency, that would be great.
>
> Regards,
> Saikat
>
> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <[hidden email]>
> wrote:
>
> > Hi Emmanouil,
> >
> > The master branch has been initialized. I have raised a PR to migrate
> > Flink module to ignite-extensions.
> >
> > PR https://github.com/apache/ignite-extensions/pull/1
> > Jira : https://issues.apache.org/jira/browse/IGNITE-12356
> >
> > Regards,
> > Saikat
> >
> >
> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
> > [hidden email]> wrote:
> >
> >> Hi all!
> >>
> >> This is an update. The integration is tested with mocks based on the
> >> original Ignite repo [1].
> >> I am currently waiting for the first commits on the new integration
> >> project
> >> to take place in order to create my pull request accordingly to the
> >> initial
> >> commits.
> >> In order to simulate the Pub/Sub server in a test I used a test server
> >> implementation [2]. Unfortunately they don't have this util uploaded in
> a
> >> maven repo, thus it needs to be hosted on a maven repo (Apache License
> >> 2.0). Is there a repo for utilities that Ignite uses for the test/build?
> >> I have also created a ticket [3]
> >>
> >> Kind regards
> >>
> >> [1] https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> >> [2] https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> >> [3] https://issues.apache.org/jira/browse/IGNITE-12262
> >>
> >> *Emmanouil Gkatziouras*
> >> https://egkatzioura.com/ |
> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> https://github.com/gkatzioura
> >>
> >>
> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <[hidden email]>
> >> wrote:
> >>
> >> > Hello Ilya,
> >> >
> >> > Thank you for your email. Yes, before we remove any integration we
> will
> >> > move them to another repository and release.
> >> >
> >> > We are thinking we will start with Pub/Sub Streamer implementation and
> >> > continue to move other integration as mentioned in the following doc.
> >> >
> >> >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> >
> >> > Regards,
> >> > Saikat
> >> >
> >> >
> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
> >> > [hidden email]>
> >> > wrote:
> >> >
> >> > > Hello!
> >> > >
> >> > > My take on this: we want to offload some of integrations, but please
> >> > commit
> >> > > and release them to another project first, and only then we will
> >> decide
> >> > to
> >> > > remove them from our repository.
> >> > >
> >> > > Please also make sure they are available with same artifact name and
> >> API.
> >> > > If this has to be changed, let's wait for AI 3.0
> >> > >
> >> > > Regards,
> >> > > --
> >> > > Ilya Kasnacheev
> >> > >
> >> > >
> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <[hidden email]
> >:
> >> > >
> >> > > > Hello Emmanouil,
> >> > > >
> >> > > > As discussed earlier, I discussed with Apache Bahir community and
> >> they
> >> > > are
> >> > > > interested to have Apache Ignite extensions as part of Apache
> Bahir
> >> > > > project.
> >> > > >
> >> > > > Can you please share Pub/Sub streamer implementation details with
> >> > Apache
> >> > > > Bahir community and request for feedback.
> >> > > >
> >> > > > https://bahir.apache.org/
> >> > > >
> >> > > > I have also requested for contributor access in Jira for Apache
> >> Bahir
> >> > > > project so that I can create issues and assign to myself.
> >> > > >
> >> > > > I can help with code reviews as well.
> >> > > >
> >> > > > Here is the email thread for reference
> >> > > > https://www.mail-archive.com/dev@.../msg02703.html
> >> > > >
> >> > > > Regards,
> >> > > > Saikat
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
> >> [hidden email]
> >> > >
> >> > > > wrote:
> >> > > >
> >> > > > > Hello,
> >> > > > >
> >> > > > > As discussed I have created following discussion threads on our
> >> > > migration
> >> > > > > proposals for Apache Ignite extensions:
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> >> > > > >
> >> > > > > https://www.mail-archive.com/dev@.../msg02703.html
> >> > > > >
> >> > > > > I will share update as I hear more information.
> >> > > > >
> >> > > > > Regards,
> >> > > > > Saikat
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
> >> > [hidden email]
> >> > > >
> >> > > > > wrote:
> >> > > > >
> >> > > > >> Hello Denis,
> >> > > > >>
> >> > > > >> Sure, sounds good. I will create a separate discussion thread
> to
> >> get
> >> > > > >> community feedback on whether we
> >> > > > >> should join the Bahir or kick-off "Ignite Extensions" project.
> >> > > > >>
> >> > > > >> Regards,
> >> > > > >> Saikat
> >> > > > >>
> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <[hidden email]
> >
> >> > > wrote:
> >> > > > >>
> >> > > > >>> Folks,
> >> > > > >>>
> >> > > > >>> The concept of Apache Bahir is precisely what we need! Saikat,
> >> > thanks
> >> > > > for
> >> > > > >>> doing the research. Why I believe the * idea* fits us well:
> >> > > > >>>
> >> > > > >>>    - All integrations can be stored in separate Github
> >> repositories
> >> > > and
> >> > > > >>>    have their dev lifecycles. We've not obliged to couple all
> >> the
> >> > > > >>> integrations
> >> > > > >>>    in a single repo. For instance, Spark can be located in a
> >> > > dedicated
> >> > > > >>> repo
> >> > > > >>>    while streaming integrations might be in a single one. It's
> >> up
> >> > to
> >> > > > us.
> >> > > > >>>    - All the repositories and integrations belong to ASF.
> We're
> >> not
> >> > > > >>> dumping
> >> > > > >>>    them on Github but rather keep supporting and developing in
> >> > > > >>> accordance with
> >> > > > >>>    ASF vision and practices.
> >> > > > >>>    - There is a way to reward contributors via committership
> and
> >> > PMC
> >> > > > >>>    membership. You won't get this if a project is just one of
> >> the
> >> > > > >>> millions of
> >> > > > >>>    Github projects.
> >> > > > >>>
> >> > > > >>> Saikat, as Ignite PMC member, could you please kick-off a
> >> separate
> >> > > > >>> dev-list
> >> > > > >>> discussion to involve more community members and referring to
> >> this
> >> > > > >>> thread?
> >> > > > >>> I think the primary question the community needs to answer
> >> whether
> >> > we
> >> > > > >>> should join the Bahir or kick-off "Ignite Extensions" project?
> >> > > > >>>
> >> > > > >>> -
> >> > > > >>> Denis
> >> > > > >>>
> >> > > > >>>
> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev <
> >> > > > [hidden email]>
> >> > > > >>> wrote:
> >> > > > >>>
> >> > > > >>> > Maybe we could move all our Streaming Integrations there,
> but
> >> > what
> >> > > is
> >> > > > >>> about
> >> > > > >>> > maintaining and committer permissions to the new
> repositories?
> >> > > > >>> > I see the list of the committers and PMC members there
> >> > > > >>> > https://bahir.apache.org/community-members/
> >> > > > >>> >
> >> > > > >>> > Could somebody from Ignite community be added to this list
> as
> >> a
> >> > > > >>> > PMC/committer to maintain Ignite integrations?
> >> > > > >>> > If yes, I'd happy to join this project and collaborate with
> >> these
> >> > > > guys
> >> > > > >>> > together
> >> > > > >>> > If no, and we should start from the zero level with external
> >> PRs
> >> > -
> >> > > > >>> hmmm,
> >> > > > >>> > it's better to have our own external repository with
> >> ApacheBahirr
> >> > > > >>> ideology.
> >> > > > >>> >
> >> > > > >>> > Also, I agree, that all connectors could be moved there (in
> >> > > > >>> ApacheBahirr
> >> > > > >>> > like repository), but not all modules from the page
> >> > > > >>> >
> >> > > > >>> >
> >> > > > >>>
> >> > > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> > > > >>> > because
> >> > > > >>> > they use different parts of core modules and could be
> >> developed
> >> > > with
> >> > > > >>> > different velocity.
> >> > > > >>> >
> >> > > > >>> > In reality, before creation of new repositories we need wide
> >> > > > discussion
> >> > > > >>> > based on the document mentioned above.
> >> > > > >>> >
> >> > > > >>> > P.S. Of course, we could experiment in the next release 2.8
> >> with
> >> > > one
> >> > > > >>> or two
> >> > > > >>> > integrations like twitter or ZeroMQ
> >> > > > >>> > Also, I'd like Denis Magda idea for the separate
> repositories
> >> in
> >> > > > Apache
> >> > > > >>> > Github to maintain them by anybody.
> >> > > > >>> >
> >> > > > >>> >
> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra <
> >> > > [hidden email]
> >> > > > >:
> >> > > > >>> >
> >> > > > >>> > > Hi Denis, Emmanouil
> >> > > > >>> > >
> >> > > > >>> > > Thank you for your email. I think creating a separate
> >> > integration
> >> > > > >>> project
> >> > > > >>> > > in github and also proposing it as an apache incubator
> >> project
> >> > > will
> >> > > > >>> be a
> >> > > > >>> > > good move. The other separate integration modules
> >> > > > >>> > >
> >> > > > >>> > >
> >> > > > >>> >
> >> > > > >>>
> >> > > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> > > > >>> > > can
> >> > > > >>> > > be moved to this new apache incubator project.
> >> > > > >>> > >
> >> > > > >>> > > There are similar integration available for Flink and
> Spark
> >> in
> >> > > > Apache
> >> > > > >>> > Bahir
> >> > > > >>> > > https://bahir.apache.org/
> >> > > > >>> > >
> >> > > > >>> > > Do you think we should reach out to Apache Bahir community
> >> > with a
> >> > > > >>> > proposal
> >> > > > >>> > > or we should plan to create a separate Apache Incubator
> >> > project?
> >> > > > >>> > >
> >> > > > >>> > > Regards,
> >> > > > >>> > > Saikat
> >> > > > >>> > >
> >> > > > >>> > >
> >> > > > >>> > >
> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil Gkatziouras <
> >> > > > >>> > > [hidden email]>
> >> > > > >>> > > wrote:
> >> > > > >>> > >
> >> > > > >>> > > > Hi all!
> >> > > > >>> > > >
> >> > > > >>> > > > Based on the discussions most probably this is going to
> >> be on
> >> > > > >>> another
> >> > > > >>> > > > GitHub repo. Therefore I will prepare a standalone
> project
> >> > with
> >> > > > the
> >> > > > >>> > > Pub/Sub
> >> > > > >>> > > > module and once the repository is created a pull request
> >> > shall
> >> > > be
> >> > > > >>> > issued.
> >> > > > >>> > > > If there is anything else I can do in order to
> facilitate
> >> > this
> >> > > > >>> please
> >> > > > >>> > let
> >> > > > >>> > > > me know.
> >> > > > >>> > > >
> >> > > > >>> > > > If a ticket is created for this issue, I shall be able
> to
> >> > > create
> >> > > > a
> >> > > > >>> pull
> >> > > > >>> > > > request and thus a build will take place on ignite's CI.
> >> > > > >>> > > > If everyone is aligned with adding a Pub/Sub
> >> implementation
> >> > > may I
> >> > > > >>> > create
> >> > > > >>> > > a
> >> > > > >>> > > > ticket on JIRA?
> >> > > > >>> > > >
> >> > > > >>> > > > My next question has to do on how Ignite configures the
> >> build
> >> > > > jobs
> >> > > > >>> in
> >> > > > >>> > > Team
> >> > > > >>> > > > City. The unit tests for my implementation need a local
> >> > Pub/Sub
> >> > > > >>> server
> >> > > > >>> > up
> >> > > > >>> > > > and running (they provide one for testing purposes).
> Will
> >> > this
> >> > > > take
> >> > > > >>> > > effect
> >> > > > >>> > > > in a form of a build script inside my implementation or
> >> is it
> >> > > > >>> something
> >> > > > >>> > > > that should be configured on Team City?
> >> > > > >>> > > >
> >> > > > >>> > > > Kind regards,
> >> > > > >>> > > > Emmanouil
> >> > > > >>> > > >
> >> > > > >>> > > >
> >> > > > >>> > > > *Emmanouil Gkatziouras*
> >> > > > >>> > > > https://egkatzioura.com/ |
> >> > > > >>> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > > >>> > > > https://github.com/gkatzioura
> >> > > > >>> > > >
> >> > > > >>> > > >
> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis Magda <
> >> [hidden email]
> >> > >
> >> > > > >>> wrote:
> >> > > > >>> > > >
> >> > > > >>> > > > > Emmanouil, Saikat,
> >> > > > >>> > > > >
> >> > > > >>> > > > > After contemplating on this, let me propose another
> way
> >> > with
> >> > > > this
> >> > > > >>> > > > > particular integration that is aligned with our
> >> > > modularization
> >> > > > >>> > > endeavor.
> >> > > > >>> > > > > The modularization [1] defines the Ignite core, Ignite
> >> > > modules,
> >> > > > >>> and
> >> > > > >>> > > > > independent integrations. The first two are to be
> >> located
> >> > in
> >> > > > >>> Ignite
> >> > > > >>> > > > > repositories and to be tested/updated by the community
> >> for
> >> > > > every
> >> > > > >>> > > release.
> >> > > > >>> > > > > The modules are a subset of existing integrations that
> >> are
> >> > > > >>> adopted
> >> > > > >>> > > widely
> >> > > > >>> > > > > and require community attention.
> >> > > > >>> > > > >
> >> > > > >>> > > > > As for the independent integrations [1], those are
> >> supposed
> >> > > to
> >> > > > be
> >> > > > >>> > > located
> >> > > > >>> > > > > in a separate GitHub repo. I would advise us to start
> >> > > > >>> implementing
> >> > > > >>> > this
> >> > > > >>> > > > > concept with the Pub/Sub integration. Even though it
> >> will
> >> > be
> >> > > > >>> located
> >> > > > >>> > > in a
> >> > > > >>> > > > > separate Github repo, it will be listed among
> officially
> >> > > > >>> available
> >> > > > >>> > > > > integrations on the Ignite website featuring Emmanouil
> >> as a
> >> > > > >>> > > contributor.
> >> > > > >>> > > > >
> >> > > > >>> > > > > What do you think?
> >> > > > >>> > > > >
> >> > > > >>> > > > >
> >> > > > >>> > > > > [1]
> >> > > > >>> > > > >
> >> > > > >>> > > > >
> >> > > > >>> > > >
> >> > > > >>> > >
> >> > > > >>> >
> >> > > > >>>
> >> > > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> > > > >>> > > > >
> >> > > > >>> > > > >
> >> > > > >>> > > > >
> >> > > > >>> > > > > -
> >> > > > >>> > > > > Denis
> >> > > > >>> > > > >
> >> > > > >>> > > > >
> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat Maitra <
> >> > > > >>> > [hidden email]
> >> > > > >>> > > >
> >> > > > >>> > > > > wrote:
> >> > > > >>> > > > >
> >> > > > >>> > > > > > Hi Emmanouil,
> >> > > > >>> > > > > >
> >> > > > >>> > > > > > The changes looks good to me. I wanted to check if
> you
> >> > are
> >> > > > also
> >> > > > >>> > able
> >> > > > >>> > > to
> >> > > > >>> > > > > > validate that the once data is added to Ignite
> cluster
> >> > you
> >> > > > are
> >> > > > >>> also
> >> > > > >>> > > > able
> >> > > > >>> > > > > to
> >> > > > >>> > > > > > access it using another Ignite client or using rest
> >> > > > endpoints.
> >> > > > >>> > > > > >
> >> > > > >>> > > > > > We use Teamcity for CI process, the details are as
> >> > follows
> >> > > > >>> > > > > >
> >> > > > >>> > > > > > Trigger validation of those test suites that have
> been
> >> > > > >>> affected by
> >> > > > >>> > > your
> >> > > > >>> > > > > > changes on TeamCity <http://ci.ignite.apache.org/>:
> <
> >> > > > >>> > > > > http://204.14.53.153/
> >> > > > >>> > > > > > >
> >> > > > >>> > > > > >
> >> > > > >>> > > > > >    - Locate a test suite you have to check, press
> >> button
> >> > > > named
> >> > > > >>> > "..."
> >> > > > >>> > > > that
> >> > > > >>> > > > > >    is located on the left of "Run" button. "Run
> custom
> >> > > build"
> >> > > > >>> > dialog
> >> > > > >>> > > > will
> >> > > > >>> > > > > >    appear;
> >> > > > >>> > > > > >    - Go to "Changes" tab and choose
> >> > > > >>> > "pull/<pull-request-number>/head"
> >> > > > >>> > > > in
> >> > > > >>> > > > > >    "Build branch" dropdown list;
> >> > > > >>> > > > > >    - Press "Run build" button and monitor tests
> >> results.
> >> > > > >>> > > > > >
> >> > > > >>> > > > > >
> >> > > > >>> > > > > >
> >> > > > >>> > > > > > @Roman [hidden email] - can you also please
> take a
> >> > > look?
> >> > > > >>> > > > > >
> >> > > > >>> > > > > > Regards,
> >> > > > >>> > > > > > Saikat
> >> > > > >>> > > > > >
> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM Emmanouil
> Gkatziouras
> >> <
> >> > > > >>> > > > > > [hidden email]>
> >> > > > >>> > > > > > wrote:
> >> > > > >>> > > > > >
> >> > > > >>> > > > > > > Hi @Saikat Maitra
> >> > > > >>> > > > > > >
> >> > > > >>> > > > > > > I have my implementation on an Ignite fork on my
> >> > personal
> >> > > > >>> GitHub
> >> > > > >>> > > > > account
> >> > > > >>> > > > > > > [1]
> >> > > > >>> > > > > > > It is based on the Kafka implementation which is
> of
> >> a
> >> > > > polling
> >> > > > >>> > > nature.
> >> > > > >>> > > > > > > GCP does not provide a service like Kafka. Instead
> >> it
> >> > > > >>> provides us
> >> > > > >>> > > > with
> >> > > > >>> > > > > > > Pub/Sub which is just like Kafka.
> >> > > > >>> > > > > > > Therefore using the Kafka streamer as a guideline
> >> made
> >> > > more
> >> > > > >>> sense
> >> > > > >>> > > to
> >> > > > >>> > > > > me.
> >> > > > >>> > > > > > >
> >> > > > >>> > > > > > > Apart from the implementation I have also added a
> >> unit
> >> > > > test.
> >> > > > >>> > > > > > > It definitely needs some polishing and a double
> >> check
> >> > on
> >> > > my
> >> > > > >>> side
> >> > > > >>> > to
> >> > > > >>> > > > > make
> >> > > > >>> > > > > > > the review smooth.
> >> > > > >>> > > > > > > For the tests a local Pub/Sub [2] server has to be
> >> > setup
> >> > > > >>> however
> >> > > > >>> > I
> >> > > > >>> > > am
> >> > > > >>> > > > > not
> >> > > > >>> > > > > > > sure which is the best way to do so with regards
> to
> >> > your
> >> > > > >>> CI/CD.
> >> > > > >>> > > > > > >
> >> > > > >>> > > > > > > [1]
> >> > > > >>> > >
> >> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> >> > > > >>> > > > > > > [2]
> >> > > > >>> https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
> >> > > > >>> > > > > > >
> >> > > > >>> > > > > > > *Emmanouil Gkatziouras*
> >> > > > >>> > > > > > > https://egkatzioura.com/ |
> >> > > > >>> > > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > > >>> > > > > > > https://github.com/gkatzioura
> >> > > > >>> > > > > > >
> >> > > > >>> > > > > > >
> >> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11, Saikat Maitra <
> >> > > > >>> > > [hidden email]
> >> > > > >>> > > > >
> >> > > > >>> > > > > > > wrote:
> >> > > > >>> > > > > > >
> >> > > > >>> > > > > > > > Hello Emmanouil,
> >> > > > >>> > > > > > > >
> >> > > > >>> > > > > > > > Can you please share more info about the
> >> > > implementation?
> >> > > > >>> > > > > > > >
> >> > > > >>> > > > > > > > We have similar implementation for data
> streamers
> >> > with
> >> > > > >>> Kafka
> >> > > > >>> > and
> >> > > > >>> > > > > > RocketMQ
> >> > > > >>> > > > > > > >
> >> > > > >>> > > > > > > >
> >> > https://apacheignite-mix.readme.io/docs/kafka-streamer
> >> > > > >>> > > > > > > >
> >> > > > >>> > > > > > > >
> >> > > > https://apacheignite-mix.readme.io/docs/rocketmq-streamer
> >> > > > >>> > > > > > > >
> >> > > > >>> > > > > > > > Please review and share if you have any
> questions.
> >> > > > >>> > > > > > > >
> >> > > > >>> > > > > > > > Regards,
> >> > > > >>> > > > > > > > Saikat
> >> > > > >>> > > > > > > >
> >> > > > >>> > > > > > > >
> >> > > > >>> > > > > > > >
> >> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56 PM Emmanouil
> >> > Gkatziouras <
> >> > > > >>> > > > > > > > [hidden email]>
> >> > > > >>> > > > > > > > wrote:
> >> > > > >>> > > > > > > >
> >> > > > >>> > > > > > > > > Hi all,
> >> > > > >>> > > > > > > > >
> >> > > > >>> > > > > > > > > By mentioning Pub/Sub I mean Google Cloud's
> >> Pub/Sub
> >> > > [1]
> >> > > > >>> > > > > > > > > Yes it is indeed cloud specific. It is the
> >> > messaging
> >> > > > >>> service
> >> > > > >>> > > > > provided
> >> > > > >>> > > > > > > by
> >> > > > >>> > > > > > > > > GCP.
> >> > > > >>> > > > > > > > > I have made the integration with it, but I
> >> haven't
> >> > > > >>> published
> >> > > > >>> > it
> >> > > > >>> > > > yet
> >> > > > >>> > > > > > (I
> >> > > > >>> > > > > > > am
> >> > > > >>> > > > > > > > > waiting for the discussion/ticket and proceed
> >> with
> >> > > the
> >> > > > >>> > > > > contribution).
> >> > > > >>> > > > > > > > >
> >> > > > >>> > > > > > > > > Kind regards,
> >> > > > >>> > > > > > > > > Emmanouil
> >> > > > >>> > > > > > > > >
> >> > > > >>> > > > > > > > > [1]
> >> https://cloud.google.com/pubsub/docs/overview
> >> > > > >>> > > > > > > > >
> >> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
> >> > > > >>> > > > > > > > > https://egkatzioura.com/ |
> >> > > > >>> > > > > > > > >
> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > > >>> > > > > > > > > https://github.com/gkatzioura
> >> > > > >>> > > > > > > > >
> >> > > > >>> > > > > > > > >
> >> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at 21:45, Denis Magda <
> >> > > > >>> [hidden email]
> >> > > > >>> > >
> >> > > > >>> > > > > wrote:
> >> > > > >>> > > > > > > > >
> >> > > > >>> > > > > > > > > > Hello Emmanouil,
> >> > > > >>> > > > > > > > > >
> >> > > > >>> > > > > > > > > > Are you referring to any specific
> >> implementation
> >> > > > >>> tailored
> >> > > > >>> > > for a
> >> > > > >>> > > > > > > cloud?
> >> > > > >>> > > > > > > > If
> >> > > > >>> > > > > > > > > > I'm not mistaken you were mentioning
> AWS/GCE.
> >> Is
> >> > > > there
> >> > > > >>> any
> >> > > > >>> > > > Github
> >> > > > >>> > > > > > > > source
> >> > > > >>> > > > > > > > > > code?
> >> > > > >>> > > > > > > > > >
> >> > > > >>> > > > > > > > > > Here is a list of existing integrations with
> >> > > > streaming
> >> > > > >>> and
> >> > > > >>> > > > > pub/sub
> >> > > > >>> > > > > > > > > > frameworks:
> >> > > > >>> > > > > > > > > >
> >> https://apacheignite-mix.readme.io/docs/overview
> >> > > > >>> > > > > > > > > >
> >> > > > >>> > > > > > > > > > with the source code in Ignite master:
> >> > > > >>> > > > > > > > > >
> >> > > https://github.com/apache/ignite/tree/master/modules
> >> > > > >>> > > > > > > > > >
> >> > > > >>> > > > > > > > > > Check out those resources to see what are
> the
> >> > most
> >> > > > >>> suitable
> >> > > > >>> > > > > Ignite
> >> > > > >>> > > > > > > APIs
> >> > > > >>> > > > > > > > > for
> >> > > > >>> > > > > > > > > > such integrations.
> >> > > > >>> > > > > > > > > >
> >> > > > >>> > > > > > > > > > @Saikat Maitra <[hidden email]> ,
> >> > @Roman
> >> > > <
> >> > > > >>> > > > > > [hidden email]
> >> > > > >>> > > > > > > >
> >> > > > >>> > > > > > > > ,
> >> > > > >>> > > > > > > > > as
> >> > > > >>> > > > > > > > > > streaming technologies
> >> contributors/maintainers,
> >> > > > could
> >> > > > >>> you
> >> > > > >>> > > > please
> >> > > > >>> > > > > > > join
> >> > > > >>> > > > > > > > > this
> >> > > > >>> > > > > > > > > > thread?
> >> > > > >>> > > > > > > > > >
> >> > > > >>> > > > > > > > > > -
> >> > > > >>> > > > > > > > > > Denis
> >> > > > >>> > > > > > > > > >
> >> > > > >>> > > > > > > > > >
> >> > > > >>> > > > > > > > > > On Thu, Oct 10, 2019 at 7:58 AM Emmanouil
> >> > > > Gkatziouras <
> >> > > > >>> > > > > > > > > > [hidden email]>
> >> > > > >>> > > > > > > > > > wrote:
> >> > > > >>> > > > > > > > > >
> >> > > > >>> > > > > > > > > > > Hello everyone,
> >> > > > >>> > > > > > > > > > >
> >> > > > >>> > > > > > > > > > > I started using Ignite lately. Part of my
> >> work
> >> > > > >>> involves
> >> > > > >>> > > > > Pub/Sub,
> >> > > > >>> > > > > > > > thus I
> >> > > > >>> > > > > > > > > > > created a pub/sub streamer.
> >> > > > >>> > > > > > > > > > > Pub/Sub is pretty close to Kafka.
> >> > > > >>> > > > > > > > > > > Here's how the streamer works. Every node
> >> > creates
> >> > > > an
> >> > > > >>> > > instance
> >> > > > >>> > > > > of
> >> > > > >>> > > > > > a
> >> > > > >>> > > > > > > > > > Pub/Sub
> >> > > > >>> > > > > > > > > > > client.
> >> > > > >>> > > > > > > > > > > The clients subscribe to the same topic
> >> using
> >> > the
> >> > > > >>> same
> >> > > > >>> > > > > > subscription
> >> > > > >>> > > > > > > > > name.
> >> > > > >>> > > > > > > > > > > A message is being sent and one of the
> nodes
> >> > will
> >> > > > >>> receive
> >> > > > >>> > > the
> >> > > > >>> > > > > > > > message,
> >> > > > >>> > > > > > > > > > > process it and then forward it to Ignite
> >> using
> >> > > the
> >> > > > >>> > > extractor
> >> > > > >>> > > > > > > > provided.
> >> > > > >>> > > > > > > > > > >
> >> > > > >>> > > > > > > > > > > Any thoughts?
> >> > > > >>> > > > > > > > > > >
> >> > > > >>> > > > > > > > > > > *Emmanouil Gkatziouras*
> >> > > > >>> > > > > > > > > > > https://egkatzioura.com/ |
> >> > > > >>> > > > > > > > > > >
> >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > > >>> > > > > > > > > > > https://github.com/gkatzioura
> >> > > > >>> > > > > > > > > > >
> >> > > > >>> > > > > > > > > >
> >> > > > >>> > > > > > > > >
> >> > > > >>> > > > > > > >
> >> > > > >>> > > > > > >
> >> > > > >>> > > > > >
> >> > > > >>> > > > >
> >> > > > >>> > > >
> >> > > > >>> > >
> >> > > > >>> >
> >> > > > >>>
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
> >
>

[Emmanouil Gkatziouras]

Hi all!

I am sorry for being late on that. I was trying to refactor the test in
order not to be in need of any external tools or spinning up a server.
I did forked the new repo and indeed my changes there, so a pull request is
a matter of time!

Kind regards

*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Fri, 22 Nov 2019 at 20:45, Denis Magda <[hidden email]> wrote:

> Hi Emmanouil,
>
> Do you have any questions or need any support from the community?
>
> -
> Denis
>
>
> On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <[hidden email]>
> wrote:
>
>> Hi Emmanouil,
>>
>> Can you please take a looks at dev utils, if this is something you are
>> looking for
>> https://github.com/apache/ignite/tree/master/modules/dev-utils?
>>
>> IMO, if you can release Pub/Sub server in maven and then use it as
>> dependency, that would be great.
>>
>> Regards,
>> Saikat
>>
>> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <[hidden email]>
>> wrote:
>>
>> > Hi Emmanouil,
>> >
>> > The master branch has been initialized. I have raised a PR to migrate
>> > Flink module to ignite-extensions.
>> >
>> > PR https://github.com/apache/ignite-extensions/pull/1
>> > Jira : https://issues.apache.org/jira/browse/IGNITE-12356
>> >
>> > Regards,
>> > Saikat
>> >
>> >
>> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
>> > [hidden email]> wrote:
>> >
>> >> Hi all!
>> >>
>> >> This is an update. The integration is tested with mocks based on the
>> >> original Ignite repo [1].
>> >> I am currently waiting for the first commits on the new integration
>> >> project
>> >> to take place in order to create my pull request accordingly to the
>> >> initial
>> >> commits.
>> >> In order to simulate the Pub/Sub server in a test I used a test server
>> >> implementation [2]. Unfortunately they don't have this util uploaded
>> in a
>> >> maven repo, thus it needs to be hosted on a maven repo (Apache License
>> >> 2.0). Is there a repo for utilities that Ignite uses for the
>> test/build?
>> >> I have also created a ticket [3]
>> >>
>> >> Kind regards
>> >>
>> >> [1] https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
>> >> [2] https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
>> >> [3] https://issues.apache.org/jira/browse/IGNITE-12262
>> >>
>> >> *Emmanouil Gkatziouras*
>> >> https://egkatzioura.com/ |
>> >> https://www.linkedin.com/in/gkatziourasemmanouil/
>> >> https://github.com/gkatzioura
>> >>
>> >>
>> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <[hidden email]>
>> >> wrote:
>> >>
>> >> > Hello Ilya,
>> >> >
>> >> > Thank you for your email. Yes, before we remove any integration we
>> will
>> >> > move them to another repository and release.
>> >> >
>> >> > We are thinking we will start with Pub/Sub Streamer implementation
>> and
>> >> > continue to move other integration as mentioned in the following doc.
>> >> >
>> >> >
>> >> >
>> >>
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> >> >
>> >> > Regards,
>> >> > Saikat
>> >> >
>> >> >
>> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
>> >> > [hidden email]>
>> >> > wrote:
>> >> >
>> >> > > Hello!
>> >> > >
>> >> > > My take on this: we want to offload some of integrations, but
>> please
>> >> > commit
>> >> > > and release them to another project first, and only then we will
>> >> decide
>> >> > to
>> >> > > remove them from our repository.
>> >> > >
>> >> > > Please also make sure they are available with same artifact name
>> and
>> >> API.
>> >> > > If this has to be changed, let's wait for AI 3.0
>> >> > >
>> >> > > Regards,
>> >> > > --
>> >> > > Ilya Kasnacheev
>> >> > >
>> >> > >
>> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <
>> [hidden email]>:
>> >> > >
>> >> > > > Hello Emmanouil,
>> >> > > >
>> >> > > > As discussed earlier, I discussed with Apache Bahir community and
>> >> they
>> >> > > are
>> >> > > > interested to have Apache Ignite extensions as part of Apache
>> Bahir
>> >> > > > project.
>> >> > > >
>> >> > > > Can you please share Pub/Sub streamer implementation details with
>> >> > Apache
>> >> > > > Bahir community and request for feedback.
>> >> > > >
>> >> > > > https://bahir.apache.org/
>> >> > > >
>> >> > > > I have also requested for contributor access in Jira for Apache
>> >> Bahir
>> >> > > > project so that I can create issues and assign to myself.
>> >> > > >
>> >> > > > I can help with code reviews as well.
>> >> > > >
>> >> > > > Here is the email thread for reference
>> >> > > > https://www.mail-archive.com/dev@.../msg02703.html
>> >> > > >
>> >> > > > Regards,
>> >> > > > Saikat
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
>> >> [hidden email]
>> >> > >
>> >> > > > wrote:
>> >> > > >
>> >> > > > > Hello,
>> >> > > > >
>> >> > > > > As discussed I have created following discussion threads on our
>> >> > > migration
>> >> > > > > proposals for Apache Ignite extensions:
>> >> > > > >
>> >> > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
>> >> > > > >
>> >> > > > >
>> https://www.mail-archive.com/dev@.../msg02703.html
>> >> > > > >
>> >> > > > > I will share update as I hear more information.
>> >> > > > >
>> >> > > > > Regards,
>> >> > > > > Saikat
>> >> > > > >
>> >> > > > >
>> >> > > > >
>> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
>> >> > [hidden email]
>> >> > > >
>> >> > > > > wrote:
>> >> > > > >
>> >> > > > >> Hello Denis,
>> >> > > > >>
>> >> > > > >> Sure, sounds good. I will create a separate discussion thread
>> to
>> >> get
>> >> > > > >> community feedback on whether we
>> >> > > > >> should join the Bahir or kick-off "Ignite Extensions" project.
>> >> > > > >>
>> >> > > > >> Regards,
>> >> > > > >> Saikat
>> >> > > > >>
>> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <
>> [hidden email]>
>> >> > > wrote:
>> >> > > > >>
>> >> > > > >>> Folks,
>> >> > > > >>>
>> >> > > > >>> The concept of Apache Bahir is precisely what we need!
>> Saikat,
>> >> > thanks
>> >> > > > for
>> >> > > > >>> doing the research. Why I believe the * idea* fits us well:
>> >> > > > >>>
>> >> > > > >>>    - All integrations can be stored in separate Github
>> >> repositories
>> >> > > and
>> >> > > > >>>    have their dev lifecycles. We've not obliged to couple all
>> >> the
>> >> > > > >>> integrations
>> >> > > > >>>    in a single repo. For instance, Spark can be located in a
>> >> > > dedicated
>> >> > > > >>> repo
>> >> > > > >>>    while streaming integrations might be in a single one.
>> It's
>> >> up
>> >> > to
>> >> > > > us.
>> >> > > > >>>    - All the repositories and integrations belong to ASF.
>> We're
>> >> not
>> >> > > > >>> dumping
>> >> > > > >>>    them on Github but rather keep supporting and developing
>> in
>> >> > > > >>> accordance with
>> >> > > > >>>    ASF vision and practices.
>> >> > > > >>>    - There is a way to reward contributors via committership
>> and
>> >> > PMC
>> >> > > > >>>    membership. You won't get this if a project is just one of
>> >> the
>> >> > > > >>> millions of
>> >> > > > >>>    Github projects.
>> >> > > > >>>
>> >> > > > >>> Saikat, as Ignite PMC member, could you please kick-off a
>> >> separate
>> >> > > > >>> dev-list
>> >> > > > >>> discussion to involve more community members and referring to
>> >> this
>> >> > > > >>> thread?
>> >> > > > >>> I think the primary question the community needs to answer
>> >> whether
>> >> > we
>> >> > > > >>> should join the Bahir or kick-off "Ignite Extensions"
>> project?
>> >> > > > >>>
>> >> > > > >>> -
>> >> > > > >>> Denis
>> >> > > > >>>
>> >> > > > >>>
>> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev <
>> >> > > > [hidden email]>
>> >> > > > >>> wrote:
>> >> > > > >>>
>> >> > > > >>> > Maybe we could move all our Streaming Integrations there,
>> but
>> >> > what
>> >> > > is
>> >> > > > >>> about
>> >> > > > >>> > maintaining and committer permissions to the new
>> repositories?
>> >> > > > >>> > I see the list of the committers and PMC members there
>> >> > > > >>> > https://bahir.apache.org/community-members/
>> >> > > > >>> >
>> >> > > > >>> > Could somebody from Ignite community be added to this list
>> as
>> >> a
>> >> > > > >>> > PMC/committer to maintain Ignite integrations?
>> >> > > > >>> > If yes, I'd happy to join this project and collaborate with
>> >> these
>> >> > > > guys
>> >> > > > >>> > together
>> >> > > > >>> > If no, and we should start from the zero level with
>> external
>> >> PRs
>> >> > -
>> >> > > > >>> hmmm,
>> >> > > > >>> > it's better to have our own external repository with
>> >> ApacheBahirr
>> >> > > > >>> ideology.
>> >> > > > >>> >
>> >> > > > >>> > Also, I agree, that all connectors could be moved there (in
>> >> > > > >>> ApacheBahirr
>> >> > > > >>> > like repository), but not all modules from the page
>> >> > > > >>> >
>> >> > > > >>> >
>> >> > > > >>>
>> >> > > >
>> >> > >
>> >> >
>> >>
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> >> > > > >>> > because
>> >> > > > >>> > they use different parts of core modules and could be
>> >> developed
>> >> > > with
>> >> > > > >>> > different velocity.
>> >> > > > >>> >
>> >> > > > >>> > In reality, before creation of new repositories we need
>> wide
>> >> > > > discussion
>> >> > > > >>> > based on the document mentioned above.
>> >> > > > >>> >
>> >> > > > >>> > P.S. Of course, we could experiment in the next release 2.8
>> >> with
>> >> > > one
>> >> > > > >>> or two
>> >> > > > >>> > integrations like twitter or ZeroMQ
>> >> > > > >>> > Also, I'd like Denis Magda idea for the separate
>> repositories
>> >> in
>> >> > > > Apache
>> >> > > > >>> > Github to maintain them by anybody.
>> >> > > > >>> >
>> >> > > > >>> >
>> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra <
>> >> > > [hidden email]
>> >> > > > >:
>> >> > > > >>> >
>> >> > > > >>> > > Hi Denis, Emmanouil
>> >> > > > >>> > >
>> >> > > > >>> > > Thank you for your email. I think creating a separate
>> >> > integration
>> >> > > > >>> project
>> >> > > > >>> > > in github and also proposing it as an apache incubator
>> >> project
>> >> > > will
>> >> > > > >>> be a
>> >> > > > >>> > > good move. The other separate integration modules
>> >> > > > >>> > >
>> >> > > > >>> > >
>> >> > > > >>> >
>> >> > > > >>>
>> >> > > >
>> >> > >
>> >> >
>> >>
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> >> > > > >>> > > can
>> >> > > > >>> > > be moved to this new apache incubator project.
>> >> > > > >>> > >
>> >> > > > >>> > > There are similar integration available for Flink and
>> Spark
>> >> in
>> >> > > > Apache
>> >> > > > >>> > Bahir
>> >> > > > >>> > > https://bahir.apache.org/
>> >> > > > >>> > >
>> >> > > > >>> > > Do you think we should reach out to Apache Bahir
>> community
>> >> > with a
>> >> > > > >>> > proposal
>> >> > > > >>> > > or we should plan to create a separate Apache Incubator
>> >> > project?
>> >> > > > >>> > >
>> >> > > > >>> > > Regards,
>> >> > > > >>> > > Saikat
>> >> > > > >>> > >
>> >> > > > >>> > >
>> >> > > > >>> > >
>> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil Gkatziouras <
>> >> > > > >>> > > [hidden email]>
>> >> > > > >>> > > wrote:
>> >> > > > >>> > >
>> >> > > > >>> > > > Hi all!
>> >> > > > >>> > > >
>> >> > > > >>> > > > Based on the discussions most probably this is going to
>> >> be on
>> >> > > > >>> another
>> >> > > > >>> > > > GitHub repo. Therefore I will prepare a standalone
>> project
>> >> > with
>> >> > > > the
>> >> > > > >>> > > Pub/Sub
>> >> > > > >>> > > > module and once the repository is created a pull
>> request
>> >> > shall
>> >> > > be
>> >> > > > >>> > issued.
>> >> > > > >>> > > > If there is anything else I can do in order to
>> facilitate
>> >> > this
>> >> > > > >>> please
>> >> > > > >>> > let
>> >> > > > >>> > > > me know.
>> >> > > > >>> > > >
>> >> > > > >>> > > > If a ticket is created for this issue, I shall be able
>> to
>> >> > > create
>> >> > > > a
>> >> > > > >>> pull
>> >> > > > >>> > > > request and thus a build will take place on ignite's
>> CI.
>> >> > > > >>> > > > If everyone is aligned with adding a Pub/Sub
>> >> implementation
>> >> > > may I
>> >> > > > >>> > create
>> >> > > > >>> > > a
>> >> > > > >>> > > > ticket on JIRA?
>> >> > > > >>> > > >
>> >> > > > >>> > > > My next question has to do on how Ignite configures the
>> >> build
>> >> > > > jobs
>> >> > > > >>> in
>> >> > > > >>> > > Team
>> >> > > > >>> > > > City. The unit tests for my implementation need a local
>> >> > Pub/Sub
>> >> > > > >>> server
>> >> > > > >>> > up
>> >> > > > >>> > > > and running (they provide one for testing purposes).
>> Will
>> >> > this
>> >> > > > take
>> >> > > > >>> > > effect
>> >> > > > >>> > > > in a form of a build script inside my implementation or
>> >> is it
>> >> > > > >>> something
>> >> > > > >>> > > > that should be configured on Team City?
>> >> > > > >>> > > >
>> >> > > > >>> > > > Kind regards,
>> >> > > > >>> > > > Emmanouil
>> >> > > > >>> > > >
>> >> > > > >>> > > >
>> >> > > > >>> > > > *Emmanouil Gkatziouras*
>> >> > > > >>> > > > https://egkatzioura.com/ |
>> >> > > > >>> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> >> > > > >>> > > > https://github.com/gkatzioura
>> >> > > > >>> > > >
>> >> > > > >>> > > >
>> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis Magda <
>> >> [hidden email]
>> >> > >
>> >> > > > >>> wrote:
>> >> > > > >>> > > >
>> >> > > > >>> > > > > Emmanouil, Saikat,
>> >> > > > >>> > > > >
>> >> > > > >>> > > > > After contemplating on this, let me propose another
>> way
>> >> > with
>> >> > > > this
>> >> > > > >>> > > > > particular integration that is aligned with our
>> >> > > modularization
>> >> > > > >>> > > endeavor.
>> >> > > > >>> > > > > The modularization [1] defines the Ignite core,
>> Ignite
>> >> > > modules,
>> >> > > > >>> and
>> >> > > > >>> > > > > independent integrations. The first two are to be
>> >> located
>> >> > in
>> >> > > > >>> Ignite
>> >> > > > >>> > > > > repositories and to be tested/updated by the
>> community
>> >> for
>> >> > > > every
>> >> > > > >>> > > release.
>> >> > > > >>> > > > > The modules are a subset of existing integrations
>> that
>> >> are
>> >> > > > >>> adopted
>> >> > > > >>> > > widely
>> >> > > > >>> > > > > and require community attention.
>> >> > > > >>> > > > >
>> >> > > > >>> > > > > As for the independent integrations [1], those are
>> >> supposed
>> >> > > to
>> >> > > > be
>> >> > > > >>> > > located
>> >> > > > >>> > > > > in a separate GitHub repo. I would advise us to start
>> >> > > > >>> implementing
>> >> > > > >>> > this
>> >> > > > >>> > > > > concept with the Pub/Sub integration. Even though it
>> >> will
>> >> > be
>> >> > > > >>> located
>> >> > > > >>> > > in a
>> >> > > > >>> > > > > separate Github repo, it will be listed among
>> officially
>> >> > > > >>> available
>> >> > > > >>> > > > > integrations on the Ignite website featuring
>> Emmanouil
>> >> as a
>> >> > > > >>> > > contributor.
>> >> > > > >>> > > > >
>> >> > > > >>> > > > > What do you think?
>> >> > > > >>> > > > >
>> >> > > > >>> > > > >
>> >> > > > >>> > > > > [1]
>> >> > > > >>> > > > >
>> >> > > > >>> > > > >
>> >> > > > >>> > > >
>> >> > > > >>> > >
>> >> > > > >>> >
>> >> > > > >>>
>> >> > > >
>> >> > >
>> >> >
>> >>
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> >> > > > >>> > > > >
>> >> > > > >>> > > > >
>> >> > > > >>> > > > >
>> >> > > > >>> > > > > -
>> >> > > > >>> > > > > Denis
>> >> > > > >>> > > > >
>> >> > > > >>> > > > >
>> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat Maitra <
>> >> > > > >>> > [hidden email]
>> >> > > > >>> > > >
>> >> > > > >>> > > > > wrote:
>> >> > > > >>> > > > >
>> >> > > > >>> > > > > > Hi Emmanouil,
>> >> > > > >>> > > > > >
>> >> > > > >>> > > > > > The changes looks good to me. I wanted to check if
>> you
>> >> > are
>> >> > > > also
>> >> > > > >>> > able
>> >> > > > >>> > > to
>> >> > > > >>> > > > > > validate that the once data is added to Ignite
>> cluster
>> >> > you
>> >> > > > are
>> >> > > > >>> also
>> >> > > > >>> > > > able
>> >> > > > >>> > > > > to
>> >> > > > >>> > > > > > access it using another Ignite client or using rest
>> >> > > > endpoints.
>> >> > > > >>> > > > > >
>> >> > > > >>> > > > > > We use Teamcity for CI process, the details are as
>> >> > follows
>> >> > > > >>> > > > > >
>> >> > > > >>> > > > > > Trigger validation of those test suites that have
>> been
>> >> > > > >>> affected by
>> >> > > > >>> > > your
>> >> > > > >>> > > > > > changes on TeamCity <http://ci.ignite.apache.org/>:
>> <
>> >> > > > >>> > > > > http://204.14.53.153/
>> >> > > > >>> > > > > > >
>> >> > > > >>> > > > > >
>> >> > > > >>> > > > > >    - Locate a test suite you have to check, press
>> >> button
>> >> > > > named
>> >> > > > >>> > "..."
>> >> > > > >>> > > > that
>> >> > > > >>> > > > > >    is located on the left of "Run" button. "Run
>> custom
>> >> > > build"
>> >> > > > >>> > dialog
>> >> > > > >>> > > > will
>> >> > > > >>> > > > > >    appear;
>> >> > > > >>> > > > > >    - Go to "Changes" tab and choose
>> >> > > > >>> > "pull/<pull-request-number>/head"
>> >> > > > >>> > > > in
>> >> > > > >>> > > > > >    "Build branch" dropdown list;
>> >> > > > >>> > > > > >    - Press "Run build" button and monitor tests
>> >> results.
>> >> > > > >>> > > > > >
>> >> > > > >>> > > > > >
>> >> > > > >>> > > > > >
>> >> > > > >>> > > > > > @Roman [hidden email] - can you also please
>> take a
>> >> > > look?
>> >> > > > >>> > > > > >
>> >> > > > >>> > > > > > Regards,
>> >> > > > >>> > > > > > Saikat
>> >> > > > >>> > > > > >
>> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM Emmanouil
>> Gkatziouras
>> >> <
>> >> > > > >>> > > > > > [hidden email]>
>> >> > > > >>> > > > > > wrote:
>> >> > > > >>> > > > > >
>> >> > > > >>> > > > > > > Hi @Saikat Maitra
>> >> > > > >>> > > > > > >
>> >> > > > >>> > > > > > > I have my implementation on an Ignite fork on my
>> >> > personal
>> >> > > > >>> GitHub
>> >> > > > >>> > > > > account
>> >> > > > >>> > > > > > > [1]
>> >> > > > >>> > > > > > > It is based on the Kafka implementation which is
>> of
>> >> a
>> >> > > > polling
>> >> > > > >>> > > nature.
>> >> > > > >>> > > > > > > GCP does not provide a service like Kafka.
>> Instead
>> >> it
>> >> > > > >>> provides us
>> >> > > > >>> > > > with
>> >> > > > >>> > > > > > > Pub/Sub which is just like Kafka.
>> >> > > > >>> > > > > > > Therefore using the Kafka streamer as a guideline
>> >> made
>> >> > > more
>> >> > > > >>> sense
>> >> > > > >>> > > to
>> >> > > > >>> > > > > me.
>> >> > > > >>> > > > > > >
>> >> > > > >>> > > > > > > Apart from the implementation I have also added a
>> >> unit
>> >> > > > test.
>> >> > > > >>> > > > > > > It definitely needs some polishing and a double
>> >> check
>> >> > on
>> >> > > my
>> >> > > > >>> side
>> >> > > > >>> > to
>> >> > > > >>> > > > > make
>> >> > > > >>> > > > > > > the review smooth.
>> >> > > > >>> > > > > > > For the tests a local Pub/Sub [2] server has to
>> be
>> >> > setup
>> >> > > > >>> however
>> >> > > > >>> > I
>> >> > > > >>> > > am
>> >> > > > >>> > > > > not
>> >> > > > >>> > > > > > > sure which is the best way to do so with regards
>> to
>> >> > your
>> >> > > > >>> CI/CD.
>> >> > > > >>> > > > > > >
>> >> > > > >>> > > > > > > [1]
>> >> > > > >>> > >
>> >> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
>> >> > > > >>> > > > > > > [2]
>> >> > > > >>> https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
>> >> > > > >>> > > > > > >
>> >> > > > >>> > > > > > > *Emmanouil Gkatziouras*
>> >> > > > >>> > > > > > > https://egkatzioura.com/ |
>> >> > > > >>> > > > > > >
>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> >> > > > >>> > > > > > > https://github.com/gkatzioura
>> >> > > > >>> > > > > > >
>> >> > > > >>> > > > > > >
>> >> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11, Saikat Maitra <
>> >> > > > >>> > > [hidden email]
>> >> > > > >>> > > > >
>> >> > > > >>> > > > > > > wrote:
>> >> > > > >>> > > > > > >
>> >> > > > >>> > > > > > > > Hello Emmanouil,
>> >> > > > >>> > > > > > > >
>> >> > > > >>> > > > > > > > Can you please share more info about the
>> >> > > implementation?
>> >> > > > >>> > > > > > > >
>> >> > > > >>> > > > > > > > We have similar implementation for data
>> streamers
>> >> > with
>> >> > > > >>> Kafka
>> >> > > > >>> > and
>> >> > > > >>> > > > > > RocketMQ
>> >> > > > >>> > > > > > > >
>> >> > > > >>> > > > > > > >
>> >> > https://apacheignite-mix.readme.io/docs/kafka-streamer
>> >> > > > >>> > > > > > > >
>> >> > > > >>> > > > > > > >
>> >> > > > https://apacheignite-mix.readme.io/docs/rocketmq-streamer
>> >> > > > >>> > > > > > > >
>> >> > > > >>> > > > > > > > Please review and share if you have any
>> questions.
>> >> > > > >>> > > > > > > >
>> >> > > > >>> > > > > > > > Regards,
>> >> > > > >>> > > > > > > > Saikat
>> >> > > > >>> > > > > > > >
>> >> > > > >>> > > > > > > >
>> >> > > > >>> > > > > > > >
>> >> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56 PM Emmanouil
>> >> > Gkatziouras <
>> >> > > > >>> > > > > > > > [hidden email]>
>> >> > > > >>> > > > > > > > wrote:
>> >> > > > >>> > > > > > > >
>> >> > > > >>> > > > > > > > > Hi all,
>> >> > > > >>> > > > > > > > >
>> >> > > > >>> > > > > > > > > By mentioning Pub/Sub I mean Google Cloud's
>> >> Pub/Sub
>> >> > > [1]
>> >> > > > >>> > > > > > > > > Yes it is indeed cloud specific. It is the
>> >> > messaging
>> >> > > > >>> service
>> >> > > > >>> > > > > provided
>> >> > > > >>> > > > > > > by
>> >> > > > >>> > > > > > > > > GCP.
>> >> > > > >>> > > > > > > > > I have made the integration with it, but I
>> >> haven't
>> >> > > > >>> published
>> >> > > > >>> > it
>> >> > > > >>> > > > yet
>> >> > > > >>> > > > > > (I
>> >> > > > >>> > > > > > > am
>> >> > > > >>> > > > > > > > > waiting for the discussion/ticket and proceed
>> >> with
>> >> > > the
>> >> > > > >>> > > > > contribution).
>> >> > > > >>> > > > > > > > >
>> >> > > > >>> > > > > > > > > Kind regards,
>> >> > > > >>> > > > > > > > > Emmanouil
>> >> > > > >>> > > > > > > > >
>> >> > > > >>> > > > > > > > > [1]
>> >> https://cloud.google.com/pubsub/docs/overview
>> >> > > > >>> > > > > > > > >
>> >> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
>> >> > > > >>> > > > > > > > > https://egkatzioura.com/ |
>> >> > > > >>> > > > > > > > >
>> >> https://www.linkedin.com/in/gkatziourasemmanouil/
>> >> > > > >>> > > > > > > > > https://github.com/gkatzioura
>> >> > > > >>> > > > > > > > >
>> >> > > > >>> > > > > > > > >
>> >> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at 21:45, Denis Magda <
>> >> > > > >>> [hidden email]
>> >> > > > >>> > >
>> >> > > > >>> > > > > wrote:
>> >> > > > >>> > > > > > > > >
>> >> > > > >>> > > > > > > > > > Hello Emmanouil,
>> >> > > > >>> > > > > > > > > >
>> >> > > > >>> > > > > > > > > > Are you referring to any specific
>> >> implementation
>> >> > > > >>> tailored
>> >> > > > >>> > > for a
>> >> > > > >>> > > > > > > cloud?
>> >> > > > >>> > > > > > > > If
>> >> > > > >>> > > > > > > > > > I'm not mistaken you were mentioning
>> AWS/GCE.
>> >> Is
>> >> > > > there
>> >> > > > >>> any
>> >> > > > >>> > > > Github
>> >> > > > >>> > > > > > > > source
>> >> > > > >>> > > > > > > > > > code?
>> >> > > > >>> > > > > > > > > >
>> >> > > > >>> > > > > > > > > > Here is a list of existing integrations
>> with
>> >> > > > streaming
>> >> > > > >>> and
>> >> > > > >>> > > > > pub/sub
>> >> > > > >>> > > > > > > > > > frameworks:
>> >> > > > >>> > > > > > > > > >
>> >> https://apacheignite-mix.readme.io/docs/overview
>> >> > > > >>> > > > > > > > > >
>> >> > > > >>> > > > > > > > > > with the source code in Ignite master:
>> >> > > > >>> > > > > > > > > >
>> >> > > https://github.com/apache/ignite/tree/master/modules
>> >> > > > >>> > > > > > > > > >
>> >> > > > >>> > > > > > > > > > Check out those resources to see what are
>> the
>> >> > most
>> >> > > > >>> suitable
>> >> > > > >>> > > > > Ignite
>> >> > > > >>> > > > > > > APIs
>> >> > > > >>> > > > > > > > > for
>> >> > > > >>> > > > > > > > > > such integrations.
>> >> > > > >>> > > > > > > > > >
>> >> > > > >>> > > > > > > > > > @Saikat Maitra <[hidden email]> ,
>> >> > @Roman
>> >> > > <
>> >> > > > >>> > > > > > [hidden email]
>> >> > > > >>> > > > > > > >
>> >> > > > >>> > > > > > > > ,
>> >> > > > >>> > > > > > > > > as
>> >> > > > >>> > > > > > > > > > streaming technologies
>> >> contributors/maintainers,
>> >> > > > could
>> >> > > > >>> you
>> >> > > > >>> > > > please
>> >> > > > >>> > > > > > > join
>> >> > > > >>> > > > > > > > > this
>> >> > > > >>> > > > > > > > > > thread?
>> >> > > > >>> > > > > > > > > >
>> >> > > > >>> > > > > > > > > > -
>> >> > > > >>> > > > > > > > > > Denis
>> >> > > > >>> > > > > > > > > >
>> >> > > > >>> > > > > > > > > >
>> >> > > > >>> > > > > > > > > > On Thu, Oct 10, 2019 at 7:58 AM Emmanouil
>> >> > > > Gkatziouras <
>> >> > > > >>> > > > > > > > > > [hidden email]>
>> >> > > > >>> > > > > > > > > > wrote:
>> >> > > > >>> > > > > > > > > >
>> >> > > > >>> > > > > > > > > > > Hello everyone,
>> >> > > > >>> > > > > > > > > > >
>> >> > > > >>> > > > > > > > > > > I started using Ignite lately. Part of my
>> >> work
>> >> > > > >>> involves
>> >> > > > >>> > > > > Pub/Sub,
>> >> > > > >>> > > > > > > > thus I
>> >> > > > >>> > > > > > > > > > > created a pub/sub streamer.
>> >> > > > >>> > > > > > > > > > > Pub/Sub is pretty close to Kafka.
>> >> > > > >>> > > > > > > > > > > Here's how the streamer works. Every node
>> >> > creates
>> >> > > > an
>> >> > > > >>> > > instance
>> >> > > > >>> > > > > of
>> >> > > > >>> > > > > > a
>> >> > > > >>> > > > > > > > > > Pub/Sub
>> >> > > > >>> > > > > > > > > > > client.
>> >> > > > >>> > > > > > > > > > > The clients subscribe to the same topic
>> >> using
>> >> > the
>> >> > > > >>> same
>> >> > > > >>> > > > > > subscription
>> >> > > > >>> > > > > > > > > name.
>> >> > > > >>> > > > > > > > > > > A message is being sent and one of the
>> nodes
>> >> > will
>> >> > > > >>> receive
>> >> > > > >>> > > the
>> >> > > > >>> > > > > > > > message,
>> >> > > > >>> > > > > > > > > > > process it and then forward it to Ignite
>> >> using
>> >> > > the
>> >> > > > >>> > > extractor
>> >> > > > >>> > > > > > > > provided.
>> >> > > > >>> > > > > > > > > > >
>> >> > > > >>> > > > > > > > > > > Any thoughts?
>> >> > > > >>> > > > > > > > > > >
>> >> > > > >>> > > > > > > > > > > *Emmanouil Gkatziouras*
>> >> > > > >>> > > > > > > > > > > https://egkatzioura.com/ |
>> >> > > > >>> > > > > > > > > > >
>> >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> >> > > > >>> > > > > > > > > > > https://github.com/gkatzioura
>> >> > > > >>> > > > > > > > > > >
>> >> > > > >>> > > > > > > > > >
>> >> > > > >>> > > > > > > > >
>> >> > > > >>> > > > > > > >
>> >> > > > >>> > > > > > >
>> >> > > > >>> > > > > >
>> >> > > > >>> > > > >
>> >> > > > >>> > > >
>> >> > > > >>> > >
>> >> > > > >>> >
>> >> > > > >>>
>> >> > > > >>
>> >> > > >
>> >> > >
>> >> >
>> >>
>> >
>>
>

[dmagda]

Awesome, ping us whenever you're ready!

-
Denis


On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <[hidden email]>
wrote:

> Hi all!
>
> I am sorry for being late on that. I was trying to refactor the test in
> order not to be in need of any external tools or spinning up a server.
> I did forked the new repo and indeed my changes there, so a pull request is
> a matter of time!
>
> Kind regards
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Fri, 22 Nov 2019 at 20:45, Denis Magda <[hidden email]> wrote:
>
> > Hi Emmanouil,
> >
> > Do you have any questions or need any support from the community?
> >
> > -
> > Denis
> >
> >
> > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <[hidden email]>
> > wrote:
> >
> >> Hi Emmanouil,
> >>
> >> Can you please take a looks at dev utils, if this is something you are
> >> looking for
> >> https://github.com/apache/ignite/tree/master/modules/dev-utils?
> >>
> >> IMO, if you can release Pub/Sub server in maven and then use it as
> >> dependency, that would be great.
> >>
> >> Regards,
> >> Saikat
> >>
> >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <[hidden email]>
> >> wrote:
> >>
> >> > Hi Emmanouil,
> >> >
> >> > The master branch has been initialized. I have raised a PR to migrate
> >> > Flink module to ignite-extensions.
> >> >
> >> > PR https://github.com/apache/ignite-extensions/pull/1
> >> > Jira : https://issues.apache.org/jira/browse/IGNITE-12356
> >> >
> >> > Regards,
> >> > Saikat
> >> >
> >> >
> >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
> >> > [hidden email]> wrote:
> >> >
> >> >> Hi all!
> >> >>
> >> >> This is an update. The integration is tested with mocks based on the
> >> >> original Ignite repo [1].
> >> >> I am currently waiting for the first commits on the new integration
> >> >> project
> >> >> to take place in order to create my pull request accordingly to the
> >> >> initial
> >> >> commits.
> >> >> In order to simulate the Pub/Sub server in a test I used a test
> server
> >> >> implementation [2]. Unfortunately they don't have this util uploaded
> >> in a
> >> >> maven repo, thus it needs to be hosted on a maven repo (Apache
> License
> >> >> 2.0). Is there a repo for utilities that Ignite uses for the
> >> test/build?
> >> >> I have also created a ticket [3]
> >> >>
> >> >> Kind regards
> >> >>
> >> >> [1] https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> >> >> [2] https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> >> >> [3] https://issues.apache.org/jira/browse/IGNITE-12262
> >> >>
> >> >> *Emmanouil Gkatziouras*
> >> >> https://egkatzioura.com/ |
> >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> >> https://github.com/gkatzioura
> >> >>
> >> >>
> >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <[hidden email]
> >
> >> >> wrote:
> >> >>
> >> >> > Hello Ilya,
> >> >> >
> >> >> > Thank you for your email. Yes, before we remove any integration we
> >> will
> >> >> > move them to another repository and release.
> >> >> >
> >> >> > We are thinking we will start with Pub/Sub Streamer implementation
> >> and
> >> >> > continue to move other integration as mentioned in the following
> doc.
> >> >> >
> >> >> >
> >> >> >
> >> >>
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> >> >
> >> >> > Regards,
> >> >> > Saikat
> >> >> >
> >> >> >
> >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
> >> >> > [hidden email]>
> >> >> > wrote:
> >> >> >
> >> >> > > Hello!
> >> >> > >
> >> >> > > My take on this: we want to offload some of integrations, but
> >> please
> >> >> > commit
> >> >> > > and release them to another project first, and only then we will
> >> >> decide
> >> >> > to
> >> >> > > remove them from our repository.
> >> >> > >
> >> >> > > Please also make sure they are available with same artifact name
> >> and
> >> >> API.
> >> >> > > If this has to be changed, let's wait for AI 3.0
> >> >> > >
> >> >> > > Regards,
> >> >> > > --
> >> >> > > Ilya Kasnacheev
> >> >> > >
> >> >> > >
> >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <
> >> [hidden email]>:
> >> >> > >
> >> >> > > > Hello Emmanouil,
> >> >> > > >
> >> >> > > > As discussed earlier, I discussed with Apache Bahir community
> and
> >> >> they
> >> >> > > are
> >> >> > > > interested to have Apache Ignite extensions as part of Apache
> >> Bahir
> >> >> > > > project.
> >> >> > > >
> >> >> > > > Can you please share Pub/Sub streamer implementation details
> with
> >> >> > Apache
> >> >> > > > Bahir community and request for feedback.
> >> >> > > >
> >> >> > > > https://bahir.apache.org/
> >> >> > > >
> >> >> > > > I have also requested for contributor access in Jira for Apache
> >> >> Bahir
> >> >> > > > project so that I can create issues and assign to myself.
> >> >> > > >
> >> >> > > > I can help with code reviews as well.
> >> >> > > >
> >> >> > > > Here is the email thread for reference
> >> >> > > >
> https://www.mail-archive.com/dev@.../msg02703.html
> >> >> > > >
> >> >> > > > Regards,
> >> >> > > > Saikat
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
> >> >> [hidden email]
> >> >> > >
> >> >> > > > wrote:
> >> >> > > >
> >> >> > > > > Hello,
> >> >> > > > >
> >> >> > > > > As discussed I have created following discussion threads on
> our
> >> >> > > migration
> >> >> > > > > proposals for Apache Ignite extensions:
> >> >> > > > >
> >> >> > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> >> >> > > > >
> >> >> > > > >
> >> https://www.mail-archive.com/dev@.../msg02703.html
> >> >> > > > >
> >> >> > > > > I will share update as I hear more information.
> >> >> > > > >
> >> >> > > > > Regards,
> >> >> > > > > Saikat
> >> >> > > > >
> >> >> > > > >
> >> >> > > > >
> >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
> >> >> > [hidden email]
> >> >> > > >
> >> >> > > > > wrote:
> >> >> > > > >
> >> >> > > > >> Hello Denis,
> >> >> > > > >>
> >> >> > > > >> Sure, sounds good. I will create a separate discussion
> thread
> >> to
> >> >> get
> >> >> > > > >> community feedback on whether we
> >> >> > > > >> should join the Bahir or kick-off "Ignite Extensions"
> project.
> >> >> > > > >>
> >> >> > > > >> Regards,
> >> >> > > > >> Saikat
> >> >> > > > >>
> >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <
> >> [hidden email]>
> >> >> > > wrote:
> >> >> > > > >>
> >> >> > > > >>> Folks,
> >> >> > > > >>>
> >> >> > > > >>> The concept of Apache Bahir is precisely what we need!
> >> Saikat,
> >> >> > thanks
> >> >> > > > for
> >> >> > > > >>> doing the research. Why I believe the * idea* fits us well:
> >> >> > > > >>>
> >> >> > > > >>>    - All integrations can be stored in separate Github
> >> >> repositories
> >> >> > > and
> >> >> > > > >>>    have their dev lifecycles. We've not obliged to couple
> all
> >> >> the
> >> >> > > > >>> integrations
> >> >> > > > >>>    in a single repo. For instance, Spark can be located in
> a
> >> >> > > dedicated
> >> >> > > > >>> repo
> >> >> > > > >>>    while streaming integrations might be in a single one.
> >> It's
> >> >> up
> >> >> > to
> >> >> > > > us.
> >> >> > > > >>>    - All the repositories and integrations belong to ASF.
> >> We're
> >> >> not
> >> >> > > > >>> dumping
> >> >> > > > >>>    them on Github but rather keep supporting and developing
> >> in
> >> >> > > > >>> accordance with
> >> >> > > > >>>    ASF vision and practices.
> >> >> > > > >>>    - There is a way to reward contributors via
> committership
> >> and
> >> >> > PMC
> >> >> > > > >>>    membership. You won't get this if a project is just one
> of
> >> >> the
> >> >> > > > >>> millions of
> >> >> > > > >>>    Github projects.
> >> >> > > > >>>
> >> >> > > > >>> Saikat, as Ignite PMC member, could you please kick-off a
> >> >> separate
> >> >> > > > >>> dev-list
> >> >> > > > >>> discussion to involve more community members and referring
> to
> >> >> this
> >> >> > > > >>> thread?
> >> >> > > > >>> I think the primary question the community needs to answer
> >> >> whether
> >> >> > we
> >> >> > > > >>> should join the Bahir or kick-off "Ignite Extensions"
> >> project?
> >> >> > > > >>>
> >> >> > > > >>> -
> >> >> > > > >>> Denis
> >> >> > > > >>>
> >> >> > > > >>>
> >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev <
> >> >> > > > [hidden email]>
> >> >> > > > >>> wrote:
> >> >> > > > >>>
> >> >> > > > >>> > Maybe we could move all our Streaming Integrations there,
> >> but
> >> >> > what
> >> >> > > is
> >> >> > > > >>> about
> >> >> > > > >>> > maintaining and committer permissions to the new
> >> repositories?
> >> >> > > > >>> > I see the list of the committers and PMC members there
> >> >> > > > >>> > https://bahir.apache.org/community-members/
> >> >> > > > >>> >
> >> >> > > > >>> > Could somebody from Ignite community be added to this
> list
> >> as
> >> >> a
> >> >> > > > >>> > PMC/committer to maintain Ignite integrations?
> >> >> > > > >>> > If yes, I'd happy to join this project and collaborate
> with
> >> >> these
> >> >> > > > guys
> >> >> > > > >>> > together
> >> >> > > > >>> > If no, and we should start from the zero level with
> >> external
> >> >> PRs
> >> >> > -
> >> >> > > > >>> hmmm,
> >> >> > > > >>> > it's better to have our own external repository with
> >> >> ApacheBahirr
> >> >> > > > >>> ideology.
> >> >> > > > >>> >
> >> >> > > > >>> > Also, I agree, that all connectors could be moved there
> (in
> >> >> > > > >>> ApacheBahirr
> >> >> > > > >>> > like repository), but not all modules from the page
> >> >> > > > >>> >
> >> >> > > > >>> >
> >> >> > > > >>>
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> >> > > > >>> > because
> >> >> > > > >>> > they use different parts of core modules and could be
> >> >> developed
> >> >> > > with
> >> >> > > > >>> > different velocity.
> >> >> > > > >>> >
> >> >> > > > >>> > In reality, before creation of new repositories we need
> >> wide
> >> >> > > > discussion
> >> >> > > > >>> > based on the document mentioned above.
> >> >> > > > >>> >
> >> >> > > > >>> > P.S. Of course, we could experiment in the next release
> 2.8
> >> >> with
> >> >> > > one
> >> >> > > > >>> or two
> >> >> > > > >>> > integrations like twitter or ZeroMQ
> >> >> > > > >>> > Also, I'd like Denis Magda idea for the separate
> >> repositories
> >> >> in
> >> >> > > > Apache
> >> >> > > > >>> > Github to maintain them by anybody.
> >> >> > > > >>> >
> >> >> > > > >>> >
> >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra <
> >> >> > > [hidden email]
> >> >> > > > >:
> >> >> > > > >>> >
> >> >> > > > >>> > > Hi Denis, Emmanouil
> >> >> > > > >>> > >
> >> >> > > > >>> > > Thank you for your email. I think creating a separate
> >> >> > integration
> >> >> > > > >>> project
> >> >> > > > >>> > > in github and also proposing it as an apache incubator
> >> >> project
> >> >> > > will
> >> >> > > > >>> be a
> >> >> > > > >>> > > good move. The other separate integration modules
> >> >> > > > >>> > >
> >> >> > > > >>> > >
> >> >> > > > >>> >
> >> >> > > > >>>
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> >> > > > >>> > > can
> >> >> > > > >>> > > be moved to this new apache incubator project.
> >> >> > > > >>> > >
> >> >> > > > >>> > > There are similar integration available for Flink and
> >> Spark
> >> >> in
> >> >> > > > Apache
> >> >> > > > >>> > Bahir
> >> >> > > > >>> > > https://bahir.apache.org/
> >> >> > > > >>> > >
> >> >> > > > >>> > > Do you think we should reach out to Apache Bahir
> >> community
> >> >> > with a
> >> >> > > > >>> > proposal
> >> >> > > > >>> > > or we should plan to create a separate Apache Incubator
> >> >> > project?
> >> >> > > > >>> > >
> >> >> > > > >>> > > Regards,
> >> >> > > > >>> > > Saikat
> >> >> > > > >>> > >
> >> >> > > > >>> > >
> >> >> > > > >>> > >
> >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil Gkatziouras <
> >> >> > > > >>> > > [hidden email]>
> >> >> > > > >>> > > wrote:
> >> >> > > > >>> > >
> >> >> > > > >>> > > > Hi all!
> >> >> > > > >>> > > >
> >> >> > > > >>> > > > Based on the discussions most probably this is going
> to
> >> >> be on
> >> >> > > > >>> another
> >> >> > > > >>> > > > GitHub repo. Therefore I will prepare a standalone
> >> project
> >> >> > with
> >> >> > > > the
> >> >> > > > >>> > > Pub/Sub
> >> >> > > > >>> > > > module and once the repository is created a pull
> >> request
> >> >> > shall
> >> >> > > be
> >> >> > > > >>> > issued.
> >> >> > > > >>> > > > If there is anything else I can do in order to
> >> facilitate
> >> >> > this
> >> >> > > > >>> please
> >> >> > > > >>> > let
> >> >> > > > >>> > > > me know.
> >> >> > > > >>> > > >
> >> >> > > > >>> > > > If a ticket is created for this issue, I shall be
> able
> >> to
> >> >> > > create
> >> >> > > > a
> >> >> > > > >>> pull
> >> >> > > > >>> > > > request and thus a build will take place on ignite's
> >> CI.
> >> >> > > > >>> > > > If everyone is aligned with adding a Pub/Sub
> >> >> implementation
> >> >> > > may I
> >> >> > > > >>> > create
> >> >> > > > >>> > > a
> >> >> > > > >>> > > > ticket on JIRA?
> >> >> > > > >>> > > >
> >> >> > > > >>> > > > My next question has to do on how Ignite configures
> the
> >> >> build
> >> >> > > > jobs
> >> >> > > > >>> in
> >> >> > > > >>> > > Team
> >> >> > > > >>> > > > City. The unit tests for my implementation need a
> local
> >> >> > Pub/Sub
> >> >> > > > >>> server
> >> >> > > > >>> > up
> >> >> > > > >>> > > > and running (they provide one for testing purposes).
> >> Will
> >> >> > this
> >> >> > > > take
> >> >> > > > >>> > > effect
> >> >> > > > >>> > > > in a form of a build script inside my implementation
> or
> >> >> is it
> >> >> > > > >>> something
> >> >> > > > >>> > > > that should be configured on Team City?
> >> >> > > > >>> > > >
> >> >> > > > >>> > > > Kind regards,
> >> >> > > > >>> > > > Emmanouil
> >> >> > > > >>> > > >
> >> >> > > > >>> > > >
> >> >> > > > >>> > > > *Emmanouil Gkatziouras*
> >> >> > > > >>> > > > https://egkatzioura.com/ |
> >> >> > > > >>> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> >> > > > >>> > > > https://github.com/gkatzioura
> >> >> > > > >>> > > >
> >> >> > > > >>> > > >
> >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis Magda <
> >> >> [hidden email]
> >> >> > >
> >> >> > > > >>> wrote:
> >> >> > > > >>> > > >
> >> >> > > > >>> > > > > Emmanouil, Saikat,
> >> >> > > > >>> > > > >
> >> >> > > > >>> > > > > After contemplating on this, let me propose another
> >> way
> >> >> > with
> >> >> > > > this
> >> >> > > > >>> > > > > particular integration that is aligned with our
> >> >> > > modularization
> >> >> > > > >>> > > endeavor.
> >> >> > > > >>> > > > > The modularization [1] defines the Ignite core,
> >> Ignite
> >> >> > > modules,
> >> >> > > > >>> and
> >> >> > > > >>> > > > > independent integrations. The first two are to be
> >> >> located
> >> >> > in
> >> >> > > > >>> Ignite
> >> >> > > > >>> > > > > repositories and to be tested/updated by the
> >> community
> >> >> for
> >> >> > > > every
> >> >> > > > >>> > > release.
> >> >> > > > >>> > > > > The modules are a subset of existing integrations
> >> that
> >> >> are
> >> >> > > > >>> adopted
> >> >> > > > >>> > > widely
> >> >> > > > >>> > > > > and require community attention.
> >> >> > > > >>> > > > >
> >> >> > > > >>> > > > > As for the independent integrations [1], those are
> >> >> supposed
> >> >> > > to
> >> >> > > > be
> >> >> > > > >>> > > located
> >> >> > > > >>> > > > > in a separate GitHub repo. I would advise us to
> start
> >> >> > > > >>> implementing
> >> >> > > > >>> > this
> >> >> > > > >>> > > > > concept with the Pub/Sub integration. Even though
> it
> >> >> will
> >> >> > be
> >> >> > > > >>> located
> >> >> > > > >>> > > in a
> >> >> > > > >>> > > > > separate Github repo, it will be listed among
> >> officially
> >> >> > > > >>> available
> >> >> > > > >>> > > > > integrations on the Ignite website featuring
> >> Emmanouil
> >> >> as a
> >> >> > > > >>> > > contributor.
> >> >> > > > >>> > > > >
> >> >> > > > >>> > > > > What do you think?
> >> >> > > > >>> > > > >
> >> >> > > > >>> > > > >
> >> >> > > > >>> > > > > [1]
> >> >> > > > >>> > > > >
> >> >> > > > >>> > > > >
> >> >> > > > >>> > > >
> >> >> > > > >>> > >
> >> >> > > > >>> >
> >> >> > > > >>>
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> >> > > > >>> > > > >
> >> >> > > > >>> > > > >
> >> >> > > > >>> > > > >
> >> >> > > > >>> > > > > -
> >> >> > > > >>> > > > > Denis
> >> >> > > > >>> > > > >
> >> >> > > > >>> > > > >
> >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat Maitra <
> >> >> > > > >>> > [hidden email]
> >> >> > > > >>> > > >
> >> >> > > > >>> > > > > wrote:
> >> >> > > > >>> > > > >
> >> >> > > > >>> > > > > > Hi Emmanouil,
> >> >> > > > >>> > > > > >
> >> >> > > > >>> > > > > > The changes looks good to me. I wanted to check
> if
> >> you
> >> >> > are
> >> >> > > > also
> >> >> > > > >>> > able
> >> >> > > > >>> > > to
> >> >> > > > >>> > > > > > validate that the once data is added to Ignite
> >> cluster
> >> >> > you
> >> >> > > > are
> >> >> > > > >>> also
> >> >> > > > >>> > > > able
> >> >> > > > >>> > > > > to
> >> >> > > > >>> > > > > > access it using another Ignite client or using
> rest
> >> >> > > > endpoints.
> >> >> > > > >>> > > > > >
> >> >> > > > >>> > > > > > We use Teamcity for CI process, the details are
> as
> >> >> > follows
> >> >> > > > >>> > > > > >
> >> >> > > > >>> > > > > > Trigger validation of those test suites that have
> >> been
> >> >> > > > >>> affected by
> >> >> > > > >>> > > your
> >> >> > > > >>> > > > > > changes on TeamCity <
> http://ci.ignite.apache.org/>:
> >> <
> >> >> > > > >>> > > > > http://204.14.53.153/
> >> >> > > > >>> > > > > > >
> >> >> > > > >>> > > > > >
> >> >> > > > >>> > > > > >    - Locate a test suite you have to check, press
> >> >> button
> >> >> > > > named
> >> >> > > > >>> > "..."
> >> >> > > > >>> > > > that
> >> >> > > > >>> > > > > >    is located on the left of "Run" button. "Run
> >> custom
> >> >> > > build"
> >> >> > > > >>> > dialog
> >> >> > > > >>> > > > will
> >> >> > > > >>> > > > > >    appear;
> >> >> > > > >>> > > > > >    - Go to "Changes" tab and choose
> >> >> > > > >>> > "pull/<pull-request-number>/head"
> >> >> > > > >>> > > > in
> >> >> > > > >>> > > > > >    "Build branch" dropdown list;
> >> >> > > > >>> > > > > >    - Press "Run build" button and monitor tests
> >> >> results.
> >> >> > > > >>> > > > > >
> >> >> > > > >>> > > > > >
> >> >> > > > >>> > > > > >
> >> >> > > > >>> > > > > > @Roman [hidden email] - can you also please
> >> take a
> >> >> > > look?
> >> >> > > > >>> > > > > >
> >> >> > > > >>> > > > > > Regards,
> >> >> > > > >>> > > > > > Saikat
> >> >> > > > >>> > > > > >
> >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM Emmanouil
> >> Gkatziouras
> >> >> <
> >> >> > > > >>> > > > > > [hidden email]>
> >> >> > > > >>> > > > > > wrote:
> >> >> > > > >>> > > > > >
> >> >> > > > >>> > > > > > > Hi @Saikat Maitra
> >> >> > > > >>> > > > > > >
> >> >> > > > >>> > > > > > > I have my implementation on an Ignite fork on
> my
> >> >> > personal
> >> >> > > > >>> GitHub
> >> >> > > > >>> > > > > account
> >> >> > > > >>> > > > > > > [1]
> >> >> > > > >>> > > > > > > It is based on the Kafka implementation which
> is
> >> of
> >> >> a
> >> >> > > > polling
> >> >> > > > >>> > > nature.
> >> >> > > > >>> > > > > > > GCP does not provide a service like Kafka.
> >> Instead
> >> >> it
> >> >> > > > >>> provides us
> >> >> > > > >>> > > > with
> >> >> > > > >>> > > > > > > Pub/Sub which is just like Kafka.
> >> >> > > > >>> > > > > > > Therefore using the Kafka streamer as a
> guideline
> >> >> made
> >> >> > > more
> >> >> > > > >>> sense
> >> >> > > > >>> > > to
> >> >> > > > >>> > > > > me.
> >> >> > > > >>> > > > > > >
> >> >> > > > >>> > > > > > > Apart from the implementation I have also
> added a
> >> >> unit
> >> >> > > > test.
> >> >> > > > >>> > > > > > > It definitely needs some polishing and a double
> >> >> check
> >> >> > on
> >> >> > > my
> >> >> > > > >>> side
> >> >> > > > >>> > to
> >> >> > > > >>> > > > > make
> >> >> > > > >>> > > > > > > the review smooth.
> >> >> > > > >>> > > > > > > For the tests a local Pub/Sub [2] server has to
> >> be
> >> >> > setup
> >> >> > > > >>> however
> >> >> > > > >>> > I
> >> >> > > > >>> > > am
> >> >> > > > >>> > > > > not
> >> >> > > > >>> > > > > > > sure which is the best way to do so with
> regards
> >> to
> >> >> > your
> >> >> > > > >>> CI/CD.
> >> >> > > > >>> > > > > > >
> >> >> > > > >>> > > > > > > [1]
> >> >> > > > >>> > >
> >> >> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> >> >> > > > >>> > > > > > > [2]
> >> >> > > > >>> https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
> >> >> > > > >>> > > > > > >
> >> >> > > > >>> > > > > > > *Emmanouil Gkatziouras*
> >> >> > > > >>> > > > > > > https://egkatzioura.com/ |
> >> >> > > > >>> > > > > > >
> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> >> > > > >>> > > > > > > https://github.com/gkatzioura
> >> >> > > > >>> > > > > > >
> >> >> > > > >>> > > > > > >
> >> >> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11, Saikat Maitra <
> >> >> > > > >>> > > [hidden email]
> >> >> > > > >>> > > > >
> >> >> > > > >>> > > > > > > wrote:
> >> >> > > > >>> > > > > > >
> >> >> > > > >>> > > > > > > > Hello Emmanouil,
> >> >> > > > >>> > > > > > > >
> >> >> > > > >>> > > > > > > > Can you please share more info about the
> >> >> > > implementation?
> >> >> > > > >>> > > > > > > >
> >> >> > > > >>> > > > > > > > We have similar implementation for data
> >> streamers
> >> >> > with
> >> >> > > > >>> Kafka
> >> >> > > > >>> > and
> >> >> > > > >>> > > > > > RocketMQ
> >> >> > > > >>> > > > > > > >
> >> >> > > > >>> > > > > > > >
> >> >> > https://apacheignite-mix.readme.io/docs/kafka-streamer
> >> >> > > > >>> > > > > > > >
> >> >> > > > >>> > > > > > > >
> >> >> > > > https://apacheignite-mix.readme.io/docs/rocketmq-streamer
> >> >> > > > >>> > > > > > > >
> >> >> > > > >>> > > > > > > > Please review and share if you have any
> >> questions.
> >> >> > > > >>> > > > > > > >
> >> >> > > > >>> > > > > > > > Regards,
> >> >> > > > >>> > > > > > > > Saikat
> >> >> > > > >>> > > > > > > >
> >> >> > > > >>> > > > > > > >
> >> >> > > > >>> > > > > > > >
> >> >> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56 PM Emmanouil
> >> >> > Gkatziouras <
> >> >> > > > >>> > > > > > > > [hidden email]>
> >> >> > > > >>> > > > > > > > wrote:
> >> >> > > > >>> > > > > > > >
> >> >> > > > >>> > > > > > > > > Hi all,
> >> >> > > > >>> > > > > > > > >
> >> >> > > > >>> > > > > > > > > By mentioning Pub/Sub I mean Google Cloud's
> >> >> Pub/Sub
> >> >> > > [1]
> >> >> > > > >>> > > > > > > > > Yes it is indeed cloud specific. It is the
> >> >> > messaging
> >> >> > > > >>> service
> >> >> > > > >>> > > > > provided
> >> >> > > > >>> > > > > > > by
> >> >> > > > >>> > > > > > > > > GCP.
> >> >> > > > >>> > > > > > > > > I have made the integration with it, but I
> >> >> haven't
> >> >> > > > >>> published
> >> >> > > > >>> > it
> >> >> > > > >>> > > > yet
> >> >> > > > >>> > > > > > (I
> >> >> > > > >>> > > > > > > am
> >> >> > > > >>> > > > > > > > > waiting for the discussion/ticket and
> proceed
> >> >> with
> >> >> > > the
> >> >> > > > >>> > > > > contribution).
> >> >> > > > >>> > > > > > > > >
> >> >> > > > >>> > > > > > > > > Kind regards,
> >> >> > > > >>> > > > > > > > > Emmanouil
> >> >> > > > >>> > > > > > > > >
> >> >> > > > >>> > > > > > > > > [1]
> >> >> https://cloud.google.com/pubsub/docs/overview
> >> >> > > > >>> > > > > > > > >
> >> >> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
> >> >> > > > >>> > > > > > > > > https://egkatzioura.com/ |
> >> >> > > > >>> > > > > > > > >
> >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> >> > > > >>> > > > > > > > > https://github.com/gkatzioura
> >> >> > > > >>> > > > > > > > >
> >> >> > > > >>> > > > > > > > >
> >> >> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at 21:45, Denis Magda <
> >> >> > > > >>> [hidden email]
> >> >> > > > >>> > >
> >> >> > > > >>> > > > > wrote:
> >> >> > > > >>> > > > > > > > >
> >> >> > > > >>> > > > > > > > > > Hello Emmanouil,
> >> >> > > > >>> > > > > > > > > >
> >> >> > > > >>> > > > > > > > > > Are you referring to any specific
> >> >> implementation
> >> >> > > > >>> tailored
> >> >> > > > >>> > > for a
> >> >> > > > >>> > > > > > > cloud?
> >> >> > > > >>> > > > > > > > If
> >> >> > > > >>> > > > > > > > > > I'm not mistaken you were mentioning
> >> AWS/GCE.
> >> >> Is
> >> >> > > > there
> >> >> > > > >>> any
> >> >> > > > >>> > > > Github
> >> >> > > > >>> > > > > > > > source
> >> >> > > > >>> > > > > > > > > > code?
> >> >> > > > >>> > > > > > > > > >
> >> >> > > > >>> > > > > > > > > > Here is a list of existing integrations
> >> with
> >> >> > > > streaming
> >> >> > > > >>> and
> >> >> > > > >>> > > > > pub/sub
> >> >> > > > >>> > > > > > > > > > frameworks:
> >> >> > > > >>> > > > > > > > > >
> >> >> https://apacheignite-mix.readme.io/docs/overview
> >> >> > > > >>> > > > > > > > > >
> >> >> > > > >>> > > > > > > > > > with the source code in Ignite master:
> >> >> > > > >>> > > > > > > > > >
> >> >> > > https://github.com/apache/ignite/tree/master/modules
> >> >> > > > >>> > > > > > > > > >
> >> >> > > > >>> > > > > > > > > > Check out those resources to see what are
> >> the
> >> >> > most
> >> >> > > > >>> suitable
> >> >> > > > >>> > > > > Ignite
> >> >> > > > >>> > > > > > > APIs
> >> >> > > > >>> > > > > > > > > for
> >> >> > > > >>> > > > > > > > > > such integrations.
> >> >> > > > >>> > > > > > > > > >
> >> >> > > > >>> > > > > > > > > > @Saikat Maitra <[hidden email]>
> ,
> >> >> > @Roman
> >> >> > > <
> >> >> > > > >>> > > > > > [hidden email]
> >> >> > > > >>> > > > > > > >
> >> >> > > > >>> > > > > > > > ,
> >> >> > > > >>> > > > > > > > > as
> >> >> > > > >>> > > > > > > > > > streaming technologies
> >> >> contributors/maintainers,
> >> >> > > > could
> >> >> > > > >>> you
> >> >> > > > >>> > > > please
> >> >> > > > >>> > > > > > > join
> >> >> > > > >>> > > > > > > > > this
> >> >> > > > >>> > > > > > > > > > thread?
> >> >> > > > >>> > > > > > > > > >
> >> >> > > > >>> > > > > > > > > > -
> >> >> > > > >>> > > > > > > > > > Denis
> >> >> > > > >>> > > > > > > > > >
> >> >> > > > >>> > > > > > > > > >
> >> >> > > > >>> > > > > > > > > > On Thu, Oct 10, 2019 at 7:58 AM Emmanouil
> >> >> > > > Gkatziouras <
> >> >> > > > >>> > > > > > > > > > [hidden email]>
> >> >> > > > >>> > > > > > > > > > wrote:
> >> >> > > > >>> > > > > > > > > >
> >> >> > > > >>> > > > > > > > > > > Hello everyone,
> >> >> > > > >>> > > > > > > > > > >
> >> >> > > > >>> > > > > > > > > > > I started using Ignite lately. Part of
> my
> >> >> work
> >> >> > > > >>> involves
> >> >> > > > >>> > > > > Pub/Sub,
> >> >> > > > >>> > > > > > > > thus I
> >> >> > > > >>> > > > > > > > > > > created a pub/sub streamer.
> >> >> > > > >>> > > > > > > > > > > Pub/Sub is pretty close to Kafka.
> >> >> > > > >>> > > > > > > > > > > Here's how the streamer works. Every
> node
> >> >> > creates
> >> >> > > > an
> >> >> > > > >>> > > instance
> >> >> > > > >>> > > > > of
> >> >> > > > >>> > > > > > a
> >> >> > > > >>> > > > > > > > > > Pub/Sub
> >> >> > > > >>> > > > > > > > > > > client.
> >> >> > > > >>> > > > > > > > > > > The clients subscribe to the same topic
> >> >> using
> >> >> > the
> >> >> > > > >>> same
> >> >> > > > >>> > > > > > subscription
> >> >> > > > >>> > > > > > > > > name.
> >> >> > > > >>> > > > > > > > > > > A message is being sent and one of the
> >> nodes
> >> >> > will
> >> >> > > > >>> receive
> >> >> > > > >>> > > the
> >> >> > > > >>> > > > > > > > message,
> >> >> > > > >>> > > > > > > > > > > process it and then forward it to
> Ignite
> >> >> using
> >> >> > > the
> >> >> > > > >>> > > extractor
> >> >> > > > >>> > > > > > > > provided.
> >> >> > > > >>> > > > > > > > > > >
> >> >> > > > >>> > > > > > > > > > > Any thoughts?
> >> >> > > > >>> > > > > > > > > > >
> >> >> > > > >>> > > > > > > > > > > *Emmanouil Gkatziouras*
> >> >> > > > >>> > > > > > > > > > > https://egkatzioura.com/ |
> >> >> > > > >>> > > > > > > > > > >
> >> >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> >> > > > >>> > > > > > > > > > > https://github.com/gkatzioura
> >> >> > > > >>> > > > > > > > > > >
> >> >> > > > >>> > > > > > > > > >
> >> >> > > > >>> > > > > > > > >
> >> >> > > > >>> > > > > > > >
> >> >> > > > >>> > > > > > >
> >> >> > > > >>> > > > > >
> >> >> > > > >>> > > > >
> >> >> > > > >>> > > >
> >> >> > > > >>> > >
> >> >> > > > >>> >
> >> >> > > > >>>
> >> >> > > > >>
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >> >
> >>
> >
>

[Emmanouil Gkatziouras]

Hi all,

I made my first pull request [1]. Since this project is brand new (no
parent poms, licensing), I reckoned it was best to use Saikat's branch on
Flink.
I suppose the Flink branch will be merged first. If not please give me
guidelines on how I should proceed next.

Kind regards
Emmanouil

[1] https://github.com/apache/ignite-extensions/pull/2

*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Fri, 22 Nov 2019 at 20:55, Denis Magda <[hidden email]> wrote:

> Awesome, ping us whenever you're ready!
>
> -
> Denis
>
>
> On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
> [hidden email]>
> wrote:
>
> > Hi all!
> >
> > I am sorry for being late on that. I was trying to refactor the test in
> > order not to be in need of any external tools or spinning up a server.
> > I did forked the new repo and indeed my changes there, so a pull request
> is
> > a matter of time!
> >
> > Kind regards
> >
> > *Emmanouil Gkatziouras*
> > https://egkatzioura.com/ |
> > https://www.linkedin.com/in/gkatziourasemmanouil/
> > https://github.com/gkatzioura
> >
> >
> > On Fri, 22 Nov 2019 at 20:45, Denis Magda <[hidden email]> wrote:
> >
> > > Hi Emmanouil,
> > >
> > > Do you have any questions or need any support from the community?
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <[hidden email]
> >
> > > wrote:
> > >
> > >> Hi Emmanouil,
> > >>
> > >> Can you please take a looks at dev utils, if this is something you are
> > >> looking for
> > >> https://github.com/apache/ignite/tree/master/modules/dev-utils?
> > >>
> > >> IMO, if you can release Pub/Sub server in maven and then use it as
> > >> dependency, that would be great.
> > >>
> > >> Regards,
> > >> Saikat
> > >>
> > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
> [hidden email]>
> > >> wrote:
> > >>
> > >> > Hi Emmanouil,
> > >> >
> > >> > The master branch has been initialized. I have raised a PR to
> migrate
> > >> > Flink module to ignite-extensions.
> > >> >
> > >> > PR https://github.com/apache/ignite-extensions/pull/1
> > >> > Jira : https://issues.apache.org/jira/browse/IGNITE-12356
> > >> >
> > >> > Regards,
> > >> > Saikat
> > >> >
> > >> >
> > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
> > >> > [hidden email]> wrote:
> > >> >
> > >> >> Hi all!
> > >> >>
> > >> >> This is an update. The integration is tested with mocks based on
> the
> > >> >> original Ignite repo [1].
> > >> >> I am currently waiting for the first commits on the new integration
> > >> >> project
> > >> >> to take place in order to create my pull request accordingly to the
> > >> >> initial
> > >> >> commits.
> > >> >> In order to simulate the Pub/Sub server in a test I used a test
> > server
> > >> >> implementation [2]. Unfortunately they don't have this util
> uploaded
> > >> in a
> > >> >> maven repo, thus it needs to be hosted on a maven repo (Apache
> > License
> > >> >> 2.0). Is there a repo for utilities that Ignite uses for the
> > >> test/build?
> > >> >> I have also created a ticket [3]
> > >> >>
> > >> >> Kind regards
> > >> >>
> > >> >> [1]
> https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > >> >> [2] https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> > >> >> [3] https://issues.apache.org/jira/browse/IGNITE-12262
> > >> >>
> > >> >> *Emmanouil Gkatziouras*
> > >> >> https://egkatzioura.com/ |
> > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> >> https://github.com/gkatzioura
> > >> >>
> > >> >>
> > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
> [hidden email]
> > >
> > >> >> wrote:
> > >> >>
> > >> >> > Hello Ilya,
> > >> >> >
> > >> >> > Thank you for your email. Yes, before we remove any integration
> we
> > >> will
> > >> >> > move them to another repository and release.
> > >> >> >
> > >> >> > We are thinking we will start with Pub/Sub Streamer
> implementation
> > >> and
> > >> >> > continue to move other integration as mentioned in the following
> > doc.
> > >> >> >
> > >> >> >
> > >> >> >
> > >> >>
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > >> >> >
> > >> >> > Regards,
> > >> >> > Saikat
> > >> >> >
> > >> >> >
> > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
> > >> >> > [hidden email]>
> > >> >> > wrote:
> > >> >> >
> > >> >> > > Hello!
> > >> >> > >
> > >> >> > > My take on this: we want to offload some of integrations, but
> > >> please
> > >> >> > commit
> > >> >> > > and release them to another project first, and only then we
> will
> > >> >> decide
> > >> >> > to
> > >> >> > > remove them from our repository.
> > >> >> > >
> > >> >> > > Please also make sure they are available with same artifact
> name
> > >> and
> > >> >> API.
> > >> >> > > If this has to be changed, let's wait for AI 3.0
> > >> >> > >
> > >> >> > > Regards,
> > >> >> > > --
> > >> >> > > Ilya Kasnacheev
> > >> >> > >
> > >> >> > >
> > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <
> > >> [hidden email]>:
> > >> >> > >
> > >> >> > > > Hello Emmanouil,
> > >> >> > > >
> > >> >> > > > As discussed earlier, I discussed with Apache Bahir community
> > and
> > >> >> they
> > >> >> > > are
> > >> >> > > > interested to have Apache Ignite extensions as part of Apache
> > >> Bahir
> > >> >> > > > project.
> > >> >> > > >
> > >> >> > > > Can you please share Pub/Sub streamer implementation details
> > with
> > >> >> > Apache
> > >> >> > > > Bahir community and request for feedback.
> > >> >> > > >
> > >> >> > > > https://bahir.apache.org/
> > >> >> > > >
> > >> >> > > > I have also requested for contributor access in Jira for
> Apache
> > >> >> Bahir
> > >> >> > > > project so that I can create issues and assign to myself.
> > >> >> > > >
> > >> >> > > > I can help with code reviews as well.
> > >> >> > > >
> > >> >> > > > Here is the email thread for reference
> > >> >> > > >
> > https://www.mail-archive.com/dev@.../msg02703.html
> > >> >> > > >
> > >> >> > > > Regards,
> > >> >> > > > Saikat
> > >> >> > > >
> > >> >> > > >
> > >> >> > > >
> > >> >> > > >
> > >> >> > > >
> > >> >> > > >
> > >> >> > > >
> > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
> > >> >> [hidden email]
> > >> >> > >
> > >> >> > > > wrote:
> > >> >> > > >
> > >> >> > > > > Hello,
> > >> >> > > > >
> > >> >> > > > > As discussed I have created following discussion threads on
> > our
> > >> >> > > migration
> > >> >> > > > > proposals for Apache Ignite extensions:
> > >> >> > > > >
> > >> >> > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> > >> >> > > > >
> > >> >> > > > >
> > >> https://www.mail-archive.com/dev@.../msg02703.html
> > >> >> > > > >
> > >> >> > > > > I will share update as I hear more information.
> > >> >> > > > >
> > >> >> > > > > Regards,
> > >> >> > > > > Saikat
> > >> >> > > > >
> > >> >> > > > >
> > >> >> > > > >
> > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
> > >> >> > [hidden email]
> > >> >> > > >
> > >> >> > > > > wrote:
> > >> >> > > > >
> > >> >> > > > >> Hello Denis,
> > >> >> > > > >>
> > >> >> > > > >> Sure, sounds good. I will create a separate discussion
> > thread
> > >> to
> > >> >> get
> > >> >> > > > >> community feedback on whether we
> > >> >> > > > >> should join the Bahir or kick-off "Ignite Extensions"
> > project.
> > >> >> > > > >>
> > >> >> > > > >> Regards,
> > >> >> > > > >> Saikat
> > >> >> > > > >>
> > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <
> > >> [hidden email]>
> > >> >> > > wrote:
> > >> >> > > > >>
> > >> >> > > > >>> Folks,
> > >> >> > > > >>>
> > >> >> > > > >>> The concept of Apache Bahir is precisely what we need!
> > >> Saikat,
> > >> >> > thanks
> > >> >> > > > for
> > >> >> > > > >>> doing the research. Why I believe the * idea* fits us
> well:
> > >> >> > > > >>>
> > >> >> > > > >>>    - All integrations can be stored in separate Github
> > >> >> repositories
> > >> >> > > and
> > >> >> > > > >>>    have their dev lifecycles. We've not obliged to couple
> > all
> > >> >> the
> > >> >> > > > >>> integrations
> > >> >> > > > >>>    in a single repo. For instance, Spark can be located
> in
> > a
> > >> >> > > dedicated
> > >> >> > > > >>> repo
> > >> >> > > > >>>    while streaming integrations might be in a single one.
> > >> It's
> > >> >> up
> > >> >> > to
> > >> >> > > > us.
> > >> >> > > > >>>    - All the repositories and integrations belong to ASF.
> > >> We're
> > >> >> not
> > >> >> > > > >>> dumping
> > >> >> > > > >>>    them on Github but rather keep supporting and
> developing
> > >> in
> > >> >> > > > >>> accordance with
> > >> >> > > > >>>    ASF vision and practices.
> > >> >> > > > >>>    - There is a way to reward contributors via
> > committership
> > >> and
> > >> >> > PMC
> > >> >> > > > >>>    membership. You won't get this if a project is just
> one
> > of
> > >> >> the
> > >> >> > > > >>> millions of
> > >> >> > > > >>>    Github projects.
> > >> >> > > > >>>
> > >> >> > > > >>> Saikat, as Ignite PMC member, could you please kick-off a
> > >> >> separate
> > >> >> > > > >>> dev-list
> > >> >> > > > >>> discussion to involve more community members and
> referring
> > to
> > >> >> this
> > >> >> > > > >>> thread?
> > >> >> > > > >>> I think the primary question the community needs to
> answer
> > >> >> whether
> > >> >> > we
> > >> >> > > > >>> should join the Bahir or kick-off "Ignite Extensions"
> > >> project?
> > >> >> > > > >>>
> > >> >> > > > >>> -
> > >> >> > > > >>> Denis
> > >> >> > > > >>>
> > >> >> > > > >>>
> > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev <
> > >> >> > > > [hidden email]>
> > >> >> > > > >>> wrote:
> > >> >> > > > >>>
> > >> >> > > > >>> > Maybe we could move all our Streaming Integrations
> there,
> > >> but
> > >> >> > what
> > >> >> > > is
> > >> >> > > > >>> about
> > >> >> > > > >>> > maintaining and committer permissions to the new
> > >> repositories?
> > >> >> > > > >>> > I see the list of the committers and PMC members there
> > >> >> > > > >>> > https://bahir.apache.org/community-members/
> > >> >> > > > >>> >
> > >> >> > > > >>> > Could somebody from Ignite community be added to this
> > list
> > >> as
> > >> >> a
> > >> >> > > > >>> > PMC/committer to maintain Ignite integrations?
> > >> >> > > > >>> > If yes, I'd happy to join this project and collaborate
> > with
> > >> >> these
> > >> >> > > > guys
> > >> >> > > > >>> > together
> > >> >> > > > >>> > If no, and we should start from the zero level with
> > >> external
> > >> >> PRs
> > >> >> > -
> > >> >> > > > >>> hmmm,
> > >> >> > > > >>> > it's better to have our own external repository with
> > >> >> ApacheBahirr
> > >> >> > > > >>> ideology.
> > >> >> > > > >>> >
> > >> >> > > > >>> > Also, I agree, that all connectors could be moved there
> > (in
> > >> >> > > > >>> ApacheBahirr
> > >> >> > > > >>> > like repository), but not all modules from the page
> > >> >> > > > >>> >
> > >> >> > > > >>> >
> > >> >> > > > >>>
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > >> >> > > > >>> > because
> > >> >> > > > >>> > they use different parts of core modules and could be
> > >> >> developed
> > >> >> > > with
> > >> >> > > > >>> > different velocity.
> > >> >> > > > >>> >
> > >> >> > > > >>> > In reality, before creation of new repositories we need
> > >> wide
> > >> >> > > > discussion
> > >> >> > > > >>> > based on the document mentioned above.
> > >> >> > > > >>> >
> > >> >> > > > >>> > P.S. Of course, we could experiment in the next release
> > 2.8
> > >> >> with
> > >> >> > > one
> > >> >> > > > >>> or two
> > >> >> > > > >>> > integrations like twitter or ZeroMQ
> > >> >> > > > >>> > Also, I'd like Denis Magda idea for the separate
> > >> repositories
> > >> >> in
> > >> >> > > > Apache
> > >> >> > > > >>> > Github to maintain them by anybody.
> > >> >> > > > >>> >
> > >> >> > > > >>> >
> > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra <
> > >> >> > > [hidden email]
> > >> >> > > > >:
> > >> >> > > > >>> >
> > >> >> > > > >>> > > Hi Denis, Emmanouil
> > >> >> > > > >>> > >
> > >> >> > > > >>> > > Thank you for your email. I think creating a separate
> > >> >> > integration
> > >> >> > > > >>> project
> > >> >> > > > >>> > > in github and also proposing it as an apache
> incubator
> > >> >> project
> > >> >> > > will
> > >> >> > > > >>> be a
> > >> >> > > > >>> > > good move. The other separate integration modules
> > >> >> > > > >>> > >
> > >> >> > > > >>> > >
> > >> >> > > > >>> >
> > >> >> > > > >>>
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > >> >> > > > >>> > > can
> > >> >> > > > >>> > > be moved to this new apache incubator project.
> > >> >> > > > >>> > >
> > >> >> > > > >>> > > There are similar integration available for Flink and
> > >> Spark
> > >> >> in
> > >> >> > > > Apache
> > >> >> > > > >>> > Bahir
> > >> >> > > > >>> > > https://bahir.apache.org/
> > >> >> > > > >>> > >
> > >> >> > > > >>> > > Do you think we should reach out to Apache Bahir
> > >> community
> > >> >> > with a
> > >> >> > > > >>> > proposal
> > >> >> > > > >>> > > or we should plan to create a separate Apache
> Incubator
> > >> >> > project?
> > >> >> > > > >>> > >
> > >> >> > > > >>> > > Regards,
> > >> >> > > > >>> > > Saikat
> > >> >> > > > >>> > >
> > >> >> > > > >>> > >
> > >> >> > > > >>> > >
> > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil
> Gkatziouras <
> > >> >> > > > >>> > > [hidden email]>
> > >> >> > > > >>> > > wrote:
> > >> >> > > > >>> > >
> > >> >> > > > >>> > > > Hi all!
> > >> >> > > > >>> > > >
> > >> >> > > > >>> > > > Based on the discussions most probably this is
> going
> > to
> > >> >> be on
> > >> >> > > > >>> another
> > >> >> > > > >>> > > > GitHub repo. Therefore I will prepare a standalone
> > >> project
> > >> >> > with
> > >> >> > > > the
> > >> >> > > > >>> > > Pub/Sub
> > >> >> > > > >>> > > > module and once the repository is created a pull
> > >> request
> > >> >> > shall
> > >> >> > > be
> > >> >> > > > >>> > issued.
> > >> >> > > > >>> > > > If there is anything else I can do in order to
> > >> facilitate
> > >> >> > this
> > >> >> > > > >>> please
> > >> >> > > > >>> > let
> > >> >> > > > >>> > > > me know.
> > >> >> > > > >>> > > >
> > >> >> > > > >>> > > > If a ticket is created for this issue, I shall be
> > able
> > >> to
> > >> >> > > create
> > >> >> > > > a
> > >> >> > > > >>> pull
> > >> >> > > > >>> > > > request and thus a build will take place on
> ignite's
> > >> CI.
> > >> >> > > > >>> > > > If everyone is aligned with adding a Pub/Sub
> > >> >> implementation
> > >> >> > > may I
> > >> >> > > > >>> > create
> > >> >> > > > >>> > > a
> > >> >> > > > >>> > > > ticket on JIRA?
> > >> >> > > > >>> > > >
> > >> >> > > > >>> > > > My next question has to do on how Ignite configures
> > the
> > >> >> build
> > >> >> > > > jobs
> > >> >> > > > >>> in
> > >> >> > > > >>> > > Team
> > >> >> > > > >>> > > > City. The unit tests for my implementation need a
> > local
> > >> >> > Pub/Sub
> > >> >> > > > >>> server
> > >> >> > > > >>> > up
> > >> >> > > > >>> > > > and running (they provide one for testing
> purposes).
> > >> Will
> > >> >> > this
> > >> >> > > > take
> > >> >> > > > >>> > > effect
> > >> >> > > > >>> > > > in a form of a build script inside my
> implementation
> > or
> > >> >> is it
> > >> >> > > > >>> something
> > >> >> > > > >>> > > > that should be configured on Team City?
> > >> >> > > > >>> > > >
> > >> >> > > > >>> > > > Kind regards,
> > >> >> > > > >>> > > > Emmanouil
> > >> >> > > > >>> > > >
> > >> >> > > > >>> > > >
> > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
> > >> >> > > > >>> > > > https://egkatzioura.com/ |
> > >> >> > > > >>> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> >> > > > >>> > > > https://github.com/gkatzioura
> > >> >> > > > >>> > > >
> > >> >> > > > >>> > > >
> > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis Magda <
> > >> >> [hidden email]
> > >> >> > >
> > >> >> > > > >>> wrote:
> > >> >> > > > >>> > > >
> > >> >> > > > >>> > > > > Emmanouil, Saikat,
> > >> >> > > > >>> > > > >
> > >> >> > > > >>> > > > > After contemplating on this, let me propose
> another
> > >> way
> > >> >> > with
> > >> >> > > > this
> > >> >> > > > >>> > > > > particular integration that is aligned with our
> > >> >> > > modularization
> > >> >> > > > >>> > > endeavor.
> > >> >> > > > >>> > > > > The modularization [1] defines the Ignite core,
> > >> Ignite
> > >> >> > > modules,
> > >> >> > > > >>> and
> > >> >> > > > >>> > > > > independent integrations. The first two are to be
> > >> >> located
> > >> >> > in
> > >> >> > > > >>> Ignite
> > >> >> > > > >>> > > > > repositories and to be tested/updated by the
> > >> community
> > >> >> for
> > >> >> > > > every
> > >> >> > > > >>> > > release.
> > >> >> > > > >>> > > > > The modules are a subset of existing integrations
> > >> that
> > >> >> are
> > >> >> > > > >>> adopted
> > >> >> > > > >>> > > widely
> > >> >> > > > >>> > > > > and require community attention.
> > >> >> > > > >>> > > > >
> > >> >> > > > >>> > > > > As for the independent integrations [1], those
> are
> > >> >> supposed
> > >> >> > > to
> > >> >> > > > be
> > >> >> > > > >>> > > located
> > >> >> > > > >>> > > > > in a separate GitHub repo. I would advise us to
> > start
> > >> >> > > > >>> implementing
> > >> >> > > > >>> > this
> > >> >> > > > >>> > > > > concept with the Pub/Sub integration. Even though
> > it
> > >> >> will
> > >> >> > be
> > >> >> > > > >>> located
> > >> >> > > > >>> > > in a
> > >> >> > > > >>> > > > > separate Github repo, it will be listed among
> > >> officially
> > >> >> > > > >>> available
> > >> >> > > > >>> > > > > integrations on the Ignite website featuring
> > >> Emmanouil
> > >> >> as a
> > >> >> > > > >>> > > contributor.
> > >> >> > > > >>> > > > >
> > >> >> > > > >>> > > > > What do you think?
> > >> >> > > > >>> > > > >
> > >> >> > > > >>> > > > >
> > >> >> > > > >>> > > > > [1]
> > >> >> > > > >>> > > > >
> > >> >> > > > >>> > > > >
> > >> >> > > > >>> > > >
> > >> >> > > > >>> > >
> > >> >> > > > >>> >
> > >> >> > > > >>>
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > >> >> > > > >>> > > > >
> > >> >> > > > >>> > > > >
> > >> >> > > > >>> > > > >
> > >> >> > > > >>> > > > > -
> > >> >> > > > >>> > > > > Denis
> > >> >> > > > >>> > > > >
> > >> >> > > > >>> > > > >
> > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat Maitra <
> > >> >> > > > >>> > [hidden email]
> > >> >> > > > >>> > > >
> > >> >> > > > >>> > > > > wrote:
> > >> >> > > > >>> > > > >
> > >> >> > > > >>> > > > > > Hi Emmanouil,
> > >> >> > > > >>> > > > > >
> > >> >> > > > >>> > > > > > The changes looks good to me. I wanted to check
> > if
> > >> you
> > >> >> > are
> > >> >> > > > also
> > >> >> > > > >>> > able
> > >> >> > > > >>> > > to
> > >> >> > > > >>> > > > > > validate that the once data is added to Ignite
> > >> cluster
> > >> >> > you
> > >> >> > > > are
> > >> >> > > > >>> also
> > >> >> > > > >>> > > > able
> > >> >> > > > >>> > > > > to
> > >> >> > > > >>> > > > > > access it using another Ignite client or using
> > rest
> > >> >> > > > endpoints.
> > >> >> > > > >>> > > > > >
> > >> >> > > > >>> > > > > > We use Teamcity for CI process, the details are
> > as
> > >> >> > follows
> > >> >> > > > >>> > > > > >
> > >> >> > > > >>> > > > > > Trigger validation of those test suites that
> have
> > >> been
> > >> >> > > > >>> affected by
> > >> >> > > > >>> > > your
> > >> >> > > > >>> > > > > > changes on TeamCity <
> > http://ci.ignite.apache.org/>:
> > >> <
> > >> >> > > > >>> > > > > http://204.14.53.153/
> > >> >> > > > >>> > > > > > >
> > >> >> > > > >>> > > > > >
> > >> >> > > > >>> > > > > >    - Locate a test suite you have to check,
> press
> > >> >> button
> > >> >> > > > named
> > >> >> > > > >>> > "..."
> > >> >> > > > >>> > > > that
> > >> >> > > > >>> > > > > >    is located on the left of "Run" button. "Run
> > >> custom
> > >> >> > > build"
> > >> >> > > > >>> > dialog
> > >> >> > > > >>> > > > will
> > >> >> > > > >>> > > > > >    appear;
> > >> >> > > > >>> > > > > >    - Go to "Changes" tab and choose
> > >> >> > > > >>> > "pull/<pull-request-number>/head"
> > >> >> > > > >>> > > > in
> > >> >> > > > >>> > > > > >    "Build branch" dropdown list;
> > >> >> > > > >>> > > > > >    - Press "Run build" button and monitor tests
> > >> >> results.
> > >> >> > > > >>> > > > > >
> > >> >> > > > >>> > > > > >
> > >> >> > > > >>> > > > > >
> > >> >> > > > >>> > > > > > @Roman [hidden email] - can you also please
> > >> take a
> > >> >> > > look?
> > >> >> > > > >>> > > > > >
> > >> >> > > > >>> > > > > > Regards,
> > >> >> > > > >>> > > > > > Saikat
> > >> >> > > > >>> > > > > >
> > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM Emmanouil
> > >> Gkatziouras
> > >> >> <
> > >> >> > > > >>> > > > > > [hidden email]>
> > >> >> > > > >>> > > > > > wrote:
> > >> >> > > > >>> > > > > >
> > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
> > >> >> > > > >>> > > > > > >
> > >> >> > > > >>> > > > > > > I have my implementation on an Ignite fork on
> > my
> > >> >> > personal
> > >> >> > > > >>> GitHub
> > >> >> > > > >>> > > > > account
> > >> >> > > > >>> > > > > > > [1]
> > >> >> > > > >>> > > > > > > It is based on the Kafka implementation which
> > is
> > >> of
> > >> >> a
> > >> >> > > > polling
> > >> >> > > > >>> > > nature.
> > >> >> > > > >>> > > > > > > GCP does not provide a service like Kafka.
> > >> Instead
> > >> >> it
> > >> >> > > > >>> provides us
> > >> >> > > > >>> > > > with
> > >> >> > > > >>> > > > > > > Pub/Sub which is just like Kafka.
> > >> >> > > > >>> > > > > > > Therefore using the Kafka streamer as a
> > guideline
> > >> >> made
> > >> >> > > more
> > >> >> > > > >>> sense
> > >> >> > > > >>> > > to
> > >> >> > > > >>> > > > > me.
> > >> >> > > > >>> > > > > > >
> > >> >> > > > >>> > > > > > > Apart from the implementation I have also
> > added a
> > >> >> unit
> > >> >> > > > test.
> > >> >> > > > >>> > > > > > > It definitely needs some polishing and a
> double
> > >> >> check
> > >> >> > on
> > >> >> > > my
> > >> >> > > > >>> side
> > >> >> > > > >>> > to
> > >> >> > > > >>> > > > > make
> > >> >> > > > >>> > > > > > > the review smooth.
> > >> >> > > > >>> > > > > > > For the tests a local Pub/Sub [2] server has
> to
> > >> be
> > >> >> > setup
> > >> >> > > > >>> however
> > >> >> > > > >>> > I
> > >> >> > > > >>> > > am
> > >> >> > > > >>> > > > > not
> > >> >> > > > >>> > > > > > > sure which is the best way to do so with
> > regards
> > >> to
> > >> >> > your
> > >> >> > > > >>> CI/CD.
> > >> >> > > > >>> > > > > > >
> > >> >> > > > >>> > > > > > > [1]
> > >> >> > > > >>> > >
> > >> >> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > >> >> > > > >>> > > > > > > [2]
> > >> >> > > > >>>
> https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
> > >> >> > > > >>> > > > > > >
> > >> >> > > > >>> > > > > > > *Emmanouil Gkatziouras*
> > >> >> > > > >>> > > > > > > https://egkatzioura.com/ |
> > >> >> > > > >>> > > > > > >
> > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> >> > > > >>> > > > > > > https://github.com/gkatzioura
> > >> >> > > > >>> > > > > > >
> > >> >> > > > >>> > > > > > >
> > >> >> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11, Saikat Maitra <
> > >> >> > > > >>> > > [hidden email]
> > >> >> > > > >>> > > > >
> > >> >> > > > >>> > > > > > > wrote:
> > >> >> > > > >>> > > > > > >
> > >> >> > > > >>> > > > > > > > Hello Emmanouil,
> > >> >> > > > >>> > > > > > > >
> > >> >> > > > >>> > > > > > > > Can you please share more info about the
> > >> >> > > implementation?
> > >> >> > > > >>> > > > > > > >
> > >> >> > > > >>> > > > > > > > We have similar implementation for data
> > >> streamers
> > >> >> > with
> > >> >> > > > >>> Kafka
> > >> >> > > > >>> > and
> > >> >> > > > >>> > > > > > RocketMQ
> > >> >> > > > >>> > > > > > > >
> > >> >> > > > >>> > > > > > > >
> > >> >> > https://apacheignite-mix.readme.io/docs/kafka-streamer
> > >> >> > > > >>> > > > > > > >
> > >> >> > > > >>> > > > > > > >
> > >> >> > > > https://apacheignite-mix.readme.io/docs/rocketmq-streamer
> > >> >> > > > >>> > > > > > > >
> > >> >> > > > >>> > > > > > > > Please review and share if you have any
> > >> questions.
> > >> >> > > > >>> > > > > > > >
> > >> >> > > > >>> > > > > > > > Regards,
> > >> >> > > > >>> > > > > > > > Saikat
> > >> >> > > > >>> > > > > > > >
> > >> >> > > > >>> > > > > > > >
> > >> >> > > > >>> > > > > > > >
> > >> >> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56 PM Emmanouil
> > >> >> > Gkatziouras <
> > >> >> > > > >>> > > > > > > > [hidden email]>
> > >> >> > > > >>> > > > > > > > wrote:
> > >> >> > > > >>> > > > > > > >
> > >> >> > > > >>> > > > > > > > > Hi all,
> > >> >> > > > >>> > > > > > > > >
> > >> >> > > > >>> > > > > > > > > By mentioning Pub/Sub I mean Google
> Cloud's
> > >> >> Pub/Sub
> > >> >> > > [1]
> > >> >> > > > >>> > > > > > > > > Yes it is indeed cloud specific. It is
> the
> > >> >> > messaging
> > >> >> > > > >>> service
> > >> >> > > > >>> > > > > provided
> > >> >> > > > >>> > > > > > > by
> > >> >> > > > >>> > > > > > > > > GCP.
> > >> >> > > > >>> > > > > > > > > I have made the integration with it, but
> I
> > >> >> haven't
> > >> >> > > > >>> published
> > >> >> > > > >>> > it
> > >> >> > > > >>> > > > yet
> > >> >> > > > >>> > > > > > (I
> > >> >> > > > >>> > > > > > > am
> > >> >> > > > >>> > > > > > > > > waiting for the discussion/ticket and
> > proceed
> > >> >> with
> > >> >> > > the
> > >> >> > > > >>> > > > > contribution).
> > >> >> > > > >>> > > > > > > > >
> > >> >> > > > >>> > > > > > > > > Kind regards,
> > >> >> > > > >>> > > > > > > > > Emmanouil
> > >> >> > > > >>> > > > > > > > >
> > >> >> > > > >>> > > > > > > > > [1]
> > >> >> https://cloud.google.com/pubsub/docs/overview
> > >> >> > > > >>> > > > > > > > >
> > >> >> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
> > >> >> > > > >>> > > > > > > > > https://egkatzioura.com/ |
> > >> >> > > > >>> > > > > > > > >
> > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> >> > > > >>> > > > > > > > > https://github.com/gkatzioura
> > >> >> > > > >>> > > > > > > > >
> > >> >> > > > >>> > > > > > > > >
> > >> >> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at 21:45, Denis
> Magda <
> > >> >> > > > >>> [hidden email]
> > >> >> > > > >>> > >
> > >> >> > > > >>> > > > > wrote:
> > >> >> > > > >>> > > > > > > > >
> > >> >> > > > >>> > > > > > > > > > Hello Emmanouil,
> > >> >> > > > >>> > > > > > > > > >
> > >> >> > > > >>> > > > > > > > > > Are you referring to any specific
> > >> >> implementation
> > >> >> > > > >>> tailored
> > >> >> > > > >>> > > for a
> > >> >> > > > >>> > > > > > > cloud?
> > >> >> > > > >>> > > > > > > > If
> > >> >> > > > >>> > > > > > > > > > I'm not mistaken you were mentioning
> > >> AWS/GCE.
> > >> >> Is
> > >> >> > > > there
> > >> >> > > > >>> any
> > >> >> > > > >>> > > > Github
> > >> >> > > > >>> > > > > > > > source
> > >> >> > > > >>> > > > > > > > > > code?
> > >> >> > > > >>> > > > > > > > > >
> > >> >> > > > >>> > > > > > > > > > Here is a list of existing integrations
> > >> with
> > >> >> > > > streaming
> > >> >> > > > >>> and
> > >> >> > > > >>> > > > > pub/sub
> > >> >> > > > >>> > > > > > > > > > frameworks:
> > >> >> > > > >>> > > > > > > > > >
> > >> >> https://apacheignite-mix.readme.io/docs/overview
> > >> >> > > > >>> > > > > > > > > >
> > >> >> > > > >>> > > > > > > > > > with the source code in Ignite master:
> > >> >> > > > >>> > > > > > > > > >
> > >> >> > > https://github.com/apache/ignite/tree/master/modules
> > >> >> > > > >>> > > > > > > > > >
> > >> >> > > > >>> > > > > > > > > > Check out those resources to see what
> are
> > >> the
> > >> >> > most
> > >> >> > > > >>> suitable
> > >> >> > > > >>> > > > > Ignite
> > >> >> > > > >>> > > > > > > APIs
> > >> >> > > > >>> > > > > > > > > for
> > >> >> > > > >>> > > > > > > > > > such integrations.
> > >> >> > > > >>> > > > > > > > > >
> > >> >> > > > >>> > > > > > > > > > @Saikat Maitra <
> [hidden email]>
> > ,
> > >> >> > @Roman
> > >> >> > > <
> > >> >> > > > >>> > > > > > [hidden email]
> > >> >> > > > >>> > > > > > > >
> > >> >> > > > >>> > > > > > > > ,
> > >> >> > > > >>> > > > > > > > > as
> > >> >> > > > >>> > > > > > > > > > streaming technologies
> > >> >> contributors/maintainers,
> > >> >> > > > could
> > >> >> > > > >>> you
> > >> >> > > > >>> > > > please
> > >> >> > > > >>> > > > > > > join
> > >> >> > > > >>> > > > > > > > > this
> > >> >> > > > >>> > > > > > > > > > thread?
> > >> >> > > > >>> > > > > > > > > >
> > >> >> > > > >>> > > > > > > > > > -
> > >> >> > > > >>> > > > > > > > > > Denis
> > >> >> > > > >>> > > > > > > > > >
> > >> >> > > > >>> > > > > > > > > >
> > >> >> > > > >>> > > > > > > > > > On Thu, Oct 10, 2019 at 7:58 AM
> Emmanouil
> > >> >> > > > Gkatziouras <
> > >> >> > > > >>> > > > > > > > > > [hidden email]>
> > >> >> > > > >>> > > > > > > > > > wrote:
> > >> >> > > > >>> > > > > > > > > >
> > >> >> > > > >>> > > > > > > > > > > Hello everyone,
> > >> >> > > > >>> > > > > > > > > > >
> > >> >> > > > >>> > > > > > > > > > > I started using Ignite lately. Part
> of
> > my
> > >> >> work
> > >> >> > > > >>> involves
> > >> >> > > > >>> > > > > Pub/Sub,
> > >> >> > > > >>> > > > > > > > thus I
> > >> >> > > > >>> > > > > > > > > > > created a pub/sub streamer.
> > >> >> > > > >>> > > > > > > > > > > Pub/Sub is pretty close to Kafka.
> > >> >> > > > >>> > > > > > > > > > > Here's how the streamer works. Every
> > node
> > >> >> > creates
> > >> >> > > > an
> > >> >> > > > >>> > > instance
> > >> >> > > > >>> > > > > of
> > >> >> > > > >>> > > > > > a
> > >> >> > > > >>> > > > > > > > > > Pub/Sub
> > >> >> > > > >>> > > > > > > > > > > client.
> > >> >> > > > >>> > > > > > > > > > > The clients subscribe to the same
> topic
> > >> >> using
> > >> >> > the
> > >> >> > > > >>> same
> > >> >> > > > >>> > > > > > subscription
> > >> >> > > > >>> > > > > > > > > name.
> > >> >> > > > >>> > > > > > > > > > > A message is being sent and one of
> the
> > >> nodes
> > >> >> > will
> > >> >> > > > >>> receive
> > >> >> > > > >>> > > the
> > >> >> > > > >>> > > > > > > > message,
> > >> >> > > > >>> > > > > > > > > > > process it and then forward it to
> > Ignite
> > >> >> using
> > >> >> > > the
> > >> >> > > > >>> > > extractor
> > >> >> > > > >>> > > > > > > > provided.
> > >> >> > > > >>> > > > > > > > > > >
> > >> >> > > > >>> > > > > > > > > > > Any thoughts?
> > >> >> > > > >>> > > > > > > > > > >
> > >> >> > > > >>> > > > > > > > > > > *Emmanouil Gkatziouras*
> > >> >> > > > >>> > > > > > > > > > > https://egkatzioura.com/ |
> > >> >> > > > >>> > > > > > > > > > >
> > >> >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> >> > > > >>> > > > > > > > > > > https://github.com/gkatzioura
> > >> >> > > > >>> > > > > > > > > > >
> > >> >> > > > >>> > > > > > > > > >
> > >> >> > > > >>> > > > > > > > >
> > >> >> > > > >>> > > > > > > >
> > >> >> > > > >>> > > > > > >
> > >> >> > > > >>> > > > > >
> > >> >> > > > >>> > > > >
> > >> >> > > > >>> > > >
> > >> >> > > > >>> > >
> > >> >> > > > >>> >
> > >> >> > > > >>>
> > >> >> > > > >>
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >> >
> > >>
> > >
> >
>

[Saikat Maitra]

Hi Emmanouil,

Thank you for your email. Yes, the plan is once the Flink PR is merged it
will provide the Licence, parent POMs etc and you can rebase from master
branch and apply the changes on top of it.

I am getting some issues with test failures in local with
GridTestProperties as the test.properties is not present in this new
project but should be available from dependencies.

Once I address this issue, I will go ahead and merge the changes and then
we can take it from there.

Regards,
Saikat

On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <[hidden email]>
wrote:

> Hi all,
>
> I made my first pull request [1]. Since this project is brand new (no
> parent poms, licensing), I reckoned it was best to use Saikat's branch on
> Flink.
> I suppose the Flink branch will be merged first. If not please give me
> guidelines on how I should proceed next.
>
> Kind regards
> Emmanouil
>
> [1] https://github.com/apache/ignite-extensions/pull/2
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Fri, 22 Nov 2019 at 20:55, Denis Magda <[hidden email]> wrote:
>
> > Awesome, ping us whenever you're ready!
> >
> > -
> > Denis
> >
> >
> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
> > [hidden email]>
> > wrote:
> >
> > > Hi all!
> > >
> > > I am sorry for being late on that. I was trying to refactor the test in
> > > order not to be in need of any external tools or spinning up a server.
> > > I did forked the new repo and indeed my changes there, so a pull
> request
> > is
> > > a matter of time!
> > >
> > > Kind regards
> > >
> > > *Emmanouil Gkatziouras*
> > > https://egkatzioura.com/ |
> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > https://github.com/gkatzioura
> > >
> > >
> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <[hidden email]> wrote:
> > >
> > > > Hi Emmanouil,
> > > >
> > > > Do you have any questions or need any support from the community?
> > > >
> > > > -
> > > > Denis
> > > >
> > > >
> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
> [hidden email]
> > >
> > > > wrote:
> > > >
> > > >> Hi Emmanouil,
> > > >>
> > > >> Can you please take a looks at dev utils, if this is something you
> are
> > > >> looking for
> > > >> https://github.com/apache/ignite/tree/master/modules/dev-utils?
> > > >>
> > > >> IMO, if you can release Pub/Sub server in maven and then use it as
> > > >> dependency, that would be great.
> > > >>
> > > >> Regards,
> > > >> Saikat
> > > >>
> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
> > [hidden email]>
> > > >> wrote:
> > > >>
> > > >> > Hi Emmanouil,
> > > >> >
> > > >> > The master branch has been initialized. I have raised a PR to
> > migrate
> > > >> > Flink module to ignite-extensions.
> > > >> >
> > > >> > PR https://github.com/apache/ignite-extensions/pull/1
> > > >> > Jira : https://issues.apache.org/jira/browse/IGNITE-12356
> > > >> >
> > > >> > Regards,
> > > >> > Saikat
> > > >> >
> > > >> >
> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
> > > >> > [hidden email]> wrote:
> > > >> >
> > > >> >> Hi all!
> > > >> >>
> > > >> >> This is an update. The integration is tested with mocks based on
> > the
> > > >> >> original Ignite repo [1].
> > > >> >> I am currently waiting for the first commits on the new
> integration
> > > >> >> project
> > > >> >> to take place in order to create my pull request accordingly to
> the
> > > >> >> initial
> > > >> >> commits.
> > > >> >> In order to simulate the Pub/Sub server in a test I used a test
> > > server
> > > >> >> implementation [2]. Unfortunately they don't have this util
> > uploaded
> > > >> in a
> > > >> >> maven repo, thus it needs to be hosted on a maven repo (Apache
> > > License
> > > >> >> 2.0). Is there a repo for utilities that Ignite uses for the
> > > >> test/build?
> > > >> >> I have also created a ticket [3]
> > > >> >>
> > > >> >> Kind regards
> > > >> >>
> > > >> >> [1]
> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > > >> >> [2] https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> > > >> >> [3] https://issues.apache.org/jira/browse/IGNITE-12262
> > > >> >>
> > > >> >> *Emmanouil Gkatziouras*
> > > >> >> https://egkatzioura.com/ |
> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> >> https://github.com/gkatzioura
> > > >> >>
> > > >> >>
> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
> > [hidden email]
> > > >
> > > >> >> wrote:
> > > >> >>
> > > >> >> > Hello Ilya,
> > > >> >> >
> > > >> >> > Thank you for your email. Yes, before we remove any integration
> > we
> > > >> will
> > > >> >> > move them to another repository and release.
> > > >> >> >
> > > >> >> > We are thinking we will start with Pub/Sub Streamer
> > implementation
> > > >> and
> > > >> >> > continue to move other integration as mentioned in the
> following
> > > doc.
> > > >> >> >
> > > >> >> >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > >> >> >
> > > >> >> > Regards,
> > > >> >> > Saikat
> > > >> >> >
> > > >> >> >
> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
> > > >> >> > [hidden email]>
> > > >> >> > wrote:
> > > >> >> >
> > > >> >> > > Hello!
> > > >> >> > >
> > > >> >> > > My take on this: we want to offload some of integrations, but
> > > >> please
> > > >> >> > commit
> > > >> >> > > and release them to another project first, and only then we
> > will
> > > >> >> decide
> > > >> >> > to
> > > >> >> > > remove them from our repository.
> > > >> >> > >
> > > >> >> > > Please also make sure they are available with same artifact
> > name
> > > >> and
> > > >> >> API.
> > > >> >> > > If this has to be changed, let's wait for AI 3.0
> > > >> >> > >
> > > >> >> > > Regards,
> > > >> >> > > --
> > > >> >> > > Ilya Kasnacheev
> > > >> >> > >
> > > >> >> > >
> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <
> > > >> [hidden email]>:
> > > >> >> > >
> > > >> >> > > > Hello Emmanouil,
> > > >> >> > > >
> > > >> >> > > > As discussed earlier, I discussed with Apache Bahir
> community
> > > and
> > > >> >> they
> > > >> >> > > are
> > > >> >> > > > interested to have Apache Ignite extensions as part of
> Apache
> > > >> Bahir
> > > >> >> > > > project.
> > > >> >> > > >
> > > >> >> > > > Can you please share Pub/Sub streamer implementation
> details
> > > with
> > > >> >> > Apache
> > > >> >> > > > Bahir community and request for feedback.
> > > >> >> > > >
> > > >> >> > > > https://bahir.apache.org/
> > > >> >> > > >
> > > >> >> > > > I have also requested for contributor access in Jira for
> > Apache
> > > >> >> Bahir
> > > >> >> > > > project so that I can create issues and assign to myself.
> > > >> >> > > >
> > > >> >> > > > I can help with code reviews as well.
> > > >> >> > > >
> > > >> >> > > > Here is the email thread for reference
> > > >> >> > > >
> > > https://www.mail-archive.com/dev@.../msg02703.html
> > > >> >> > > >
> > > >> >> > > > Regards,
> > > >> >> > > > Saikat
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
> > > >> >> [hidden email]
> > > >> >> > >
> > > >> >> > > > wrote:
> > > >> >> > > >
> > > >> >> > > > > Hello,
> > > >> >> > > > >
> > > >> >> > > > > As discussed I have created following discussion threads
> on
> > > our
> > > >> >> > > migration
> > > >> >> > > > > proposals for Apache Ignite extensions:
> > > >> >> > > > >
> > > >> >> > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> > > >> >> > > > >
> > > >> >> > > > >
> > > >> https://www.mail-archive.com/dev@.../msg02703.html
> > > >> >> > > > >
> > > >> >> > > > > I will share update as I hear more information.
> > > >> >> > > > >
> > > >> >> > > > > Regards,
> > > >> >> > > > > Saikat
> > > >> >> > > > >
> > > >> >> > > > >
> > > >> >> > > > >
> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
> > > >> >> > [hidden email]
> > > >> >> > > >
> > > >> >> > > > > wrote:
> > > >> >> > > > >
> > > >> >> > > > >> Hello Denis,
> > > >> >> > > > >>
> > > >> >> > > > >> Sure, sounds good. I will create a separate discussion
> > > thread
> > > >> to
> > > >> >> get
> > > >> >> > > > >> community feedback on whether we
> > > >> >> > > > >> should join the Bahir or kick-off "Ignite Extensions"
> > > project.
> > > >> >> > > > >>
> > > >> >> > > > >> Regards,
> > > >> >> > > > >> Saikat
> > > >> >> > > > >>
> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <
> > > >> [hidden email]>
> > > >> >> > > wrote:
> > > >> >> > > > >>
> > > >> >> > > > >>> Folks,
> > > >> >> > > > >>>
> > > >> >> > > > >>> The concept of Apache Bahir is precisely what we need!
> > > >> Saikat,
> > > >> >> > thanks
> > > >> >> > > > for
> > > >> >> > > > >>> doing the research. Why I believe the * idea* fits us
> > well:
> > > >> >> > > > >>>
> > > >> >> > > > >>>    - All integrations can be stored in separate Github
> > > >> >> repositories
> > > >> >> > > and
> > > >> >> > > > >>>    have their dev lifecycles. We've not obliged to
> couple
> > > all
> > > >> >> the
> > > >> >> > > > >>> integrations
> > > >> >> > > > >>>    in a single repo. For instance, Spark can be located
> > in
> > > a
> > > >> >> > > dedicated
> > > >> >> > > > >>> repo
> > > >> >> > > > >>>    while streaming integrations might be in a single
> one.
> > > >> It's
> > > >> >> up
> > > >> >> > to
> > > >> >> > > > us.
> > > >> >> > > > >>>    - All the repositories and integrations belong to
> ASF.
> > > >> We're
> > > >> >> not
> > > >> >> > > > >>> dumping
> > > >> >> > > > >>>    them on Github but rather keep supporting and
> > developing
> > > >> in
> > > >> >> > > > >>> accordance with
> > > >> >> > > > >>>    ASF vision and practices.
> > > >> >> > > > >>>    - There is a way to reward contributors via
> > > committership
> > > >> and
> > > >> >> > PMC
> > > >> >> > > > >>>    membership. You won't get this if a project is just
> > one
> > > of
> > > >> >> the
> > > >> >> > > > >>> millions of
> > > >> >> > > > >>>    Github projects.
> > > >> >> > > > >>>
> > > >> >> > > > >>> Saikat, as Ignite PMC member, could you please
> kick-off a
> > > >> >> separate
> > > >> >> > > > >>> dev-list
> > > >> >> > > > >>> discussion to involve more community members and
> > referring
> > > to
> > > >> >> this
> > > >> >> > > > >>> thread?
> > > >> >> > > > >>> I think the primary question the community needs to
> > answer
> > > >> >> whether
> > > >> >> > we
> > > >> >> > > > >>> should join the Bahir or kick-off "Ignite Extensions"
> > > >> project?
> > > >> >> > > > >>>
> > > >> >> > > > >>> -
> > > >> >> > > > >>> Denis
> > > >> >> > > > >>>
> > > >> >> > > > >>>
> > > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev <
> > > >> >> > > > [hidden email]>
> > > >> >> > > > >>> wrote:
> > > >> >> > > > >>>
> > > >> >> > > > >>> > Maybe we could move all our Streaming Integrations
> > there,
> > > >> but
> > > >> >> > what
> > > >> >> > > is
> > > >> >> > > > >>> about
> > > >> >> > > > >>> > maintaining and committer permissions to the new
> > > >> repositories?
> > > >> >> > > > >>> > I see the list of the committers and PMC members
> there
> > > >> >> > > > >>> > https://bahir.apache.org/community-members/
> > > >> >> > > > >>> >
> > > >> >> > > > >>> > Could somebody from Ignite community be added to this
> > > list
> > > >> as
> > > >> >> a
> > > >> >> > > > >>> > PMC/committer to maintain Ignite integrations?
> > > >> >> > > > >>> > If yes, I'd happy to join this project and
> collaborate
> > > with
> > > >> >> these
> > > >> >> > > > guys
> > > >> >> > > > >>> > together
> > > >> >> > > > >>> > If no, and we should start from the zero level with
> > > >> external
> > > >> >> PRs
> > > >> >> > -
> > > >> >> > > > >>> hmmm,
> > > >> >> > > > >>> > it's better to have our own external repository with
> > > >> >> ApacheBahirr
> > > >> >> > > > >>> ideology.
> > > >> >> > > > >>> >
> > > >> >> > > > >>> > Also, I agree, that all connectors could be moved
> there
> > > (in
> > > >> >> > > > >>> ApacheBahirr
> > > >> >> > > > >>> > like repository), but not all modules from the page
> > > >> >> > > > >>> >
> > > >> >> > > > >>> >
> > > >> >> > > > >>>
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > >> >> > > > >>> > because
> > > >> >> > > > >>> > they use different parts of core modules and could be
> > > >> >> developed
> > > >> >> > > with
> > > >> >> > > > >>> > different velocity.
> > > >> >> > > > >>> >
> > > >> >> > > > >>> > In reality, before creation of new repositories we
> need
> > > >> wide
> > > >> >> > > > discussion
> > > >> >> > > > >>> > based on the document mentioned above.
> > > >> >> > > > >>> >
> > > >> >> > > > >>> > P.S. Of course, we could experiment in the next
> release
> > > 2.8
> > > >> >> with
> > > >> >> > > one
> > > >> >> > > > >>> or two
> > > >> >> > > > >>> > integrations like twitter or ZeroMQ
> > > >> >> > > > >>> > Also, I'd like Denis Magda idea for the separate
> > > >> repositories
> > > >> >> in
> > > >> >> > > > Apache
> > > >> >> > > > >>> > Github to maintain them by anybody.
> > > >> >> > > > >>> >
> > > >> >> > > > >>> >
> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra <
> > > >> >> > > [hidden email]
> > > >> >> > > > >:
> > > >> >> > > > >>> >
> > > >> >> > > > >>> > > Hi Denis, Emmanouil
> > > >> >> > > > >>> > >
> > > >> >> > > > >>> > > Thank you for your email. I think creating a
> separate
> > > >> >> > integration
> > > >> >> > > > >>> project
> > > >> >> > > > >>> > > in github and also proposing it as an apache
> > incubator
> > > >> >> project
> > > >> >> > > will
> > > >> >> > > > >>> be a
> > > >> >> > > > >>> > > good move. The other separate integration modules
> > > >> >> > > > >>> > >
> > > >> >> > > > >>> > >
> > > >> >> > > > >>> >
> > > >> >> > > > >>>
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > >> >> > > > >>> > > can
> > > >> >> > > > >>> > > be moved to this new apache incubator project.
> > > >> >> > > > >>> > >
> > > >> >> > > > >>> > > There are similar integration available for Flink
> and
> > > >> Spark
> > > >> >> in
> > > >> >> > > > Apache
> > > >> >> > > > >>> > Bahir
> > > >> >> > > > >>> > > https://bahir.apache.org/
> > > >> >> > > > >>> > >
> > > >> >> > > > >>> > > Do you think we should reach out to Apache Bahir
> > > >> community
> > > >> >> > with a
> > > >> >> > > > >>> > proposal
> > > >> >> > > > >>> > > or we should plan to create a separate Apache
> > Incubator
> > > >> >> > project?
> > > >> >> > > > >>> > >
> > > >> >> > > > >>> > > Regards,
> > > >> >> > > > >>> > > Saikat
> > > >> >> > > > >>> > >
> > > >> >> > > > >>> > >
> > > >> >> > > > >>> > >
> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil
> > Gkatziouras <
> > > >> >> > > > >>> > > [hidden email]>
> > > >> >> > > > >>> > > wrote:
> > > >> >> > > > >>> > >
> > > >> >> > > > >>> > > > Hi all!
> > > >> >> > > > >>> > > >
> > > >> >> > > > >>> > > > Based on the discussions most probably this is
> > going
> > > to
> > > >> >> be on
> > > >> >> > > > >>> another
> > > >> >> > > > >>> > > > GitHub repo. Therefore I will prepare a
> standalone
> > > >> project
> > > >> >> > with
> > > >> >> > > > the
> > > >> >> > > > >>> > > Pub/Sub
> > > >> >> > > > >>> > > > module and once the repository is created a pull
> > > >> request
> > > >> >> > shall
> > > >> >> > > be
> > > >> >> > > > >>> > issued.
> > > >> >> > > > >>> > > > If there is anything else I can do in order to
> > > >> facilitate
> > > >> >> > this
> > > >> >> > > > >>> please
> > > >> >> > > > >>> > let
> > > >> >> > > > >>> > > > me know.
> > > >> >> > > > >>> > > >
> > > >> >> > > > >>> > > > If a ticket is created for this issue, I shall be
> > > able
> > > >> to
> > > >> >> > > create
> > > >> >> > > > a
> > > >> >> > > > >>> pull
> > > >> >> > > > >>> > > > request and thus a build will take place on
> > ignite's
> > > >> CI.
> > > >> >> > > > >>> > > > If everyone is aligned with adding a Pub/Sub
> > > >> >> implementation
> > > >> >> > > may I
> > > >> >> > > > >>> > create
> > > >> >> > > > >>> > > a
> > > >> >> > > > >>> > > > ticket on JIRA?
> > > >> >> > > > >>> > > >
> > > >> >> > > > >>> > > > My next question has to do on how Ignite
> configures
> > > the
> > > >> >> build
> > > >> >> > > > jobs
> > > >> >> > > > >>> in
> > > >> >> > > > >>> > > Team
> > > >> >> > > > >>> > > > City. The unit tests for my implementation need a
> > > local
> > > >> >> > Pub/Sub
> > > >> >> > > > >>> server
> > > >> >> > > > >>> > up
> > > >> >> > > > >>> > > > and running (they provide one for testing
> > purposes).
> > > >> Will
> > > >> >> > this
> > > >> >> > > > take
> > > >> >> > > > >>> > > effect
> > > >> >> > > > >>> > > > in a form of a build script inside my
> > implementation
> > > or
> > > >> >> is it
> > > >> >> > > > >>> something
> > > >> >> > > > >>> > > > that should be configured on Team City?
> > > >> >> > > > >>> > > >
> > > >> >> > > > >>> > > > Kind regards,
> > > >> >> > > > >>> > > > Emmanouil
> > > >> >> > > > >>> > > >
> > > >> >> > > > >>> > > >
> > > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
> > > >> >> > > > >>> > > > https://egkatzioura.com/ |
> > > >> >> > > > >>> > > >
> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> >> > > > >>> > > > https://github.com/gkatzioura
> > > >> >> > > > >>> > > >
> > > >> >> > > > >>> > > >
> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis Magda <
> > > >> >> [hidden email]
> > > >> >> > >
> > > >> >> > > > >>> wrote:
> > > >> >> > > > >>> > > >
> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
> > > >> >> > > > >>> > > > >
> > > >> >> > > > >>> > > > > After contemplating on this, let me propose
> > another
> > > >> way
> > > >> >> > with
> > > >> >> > > > this
> > > >> >> > > > >>> > > > > particular integration that is aligned with our
> > > >> >> > > modularization
> > > >> >> > > > >>> > > endeavor.
> > > >> >> > > > >>> > > > > The modularization [1] defines the Ignite core,
> > > >> Ignite
> > > >> >> > > modules,
> > > >> >> > > > >>> and
> > > >> >> > > > >>> > > > > independent integrations. The first two are to
> be
> > > >> >> located
> > > >> >> > in
> > > >> >> > > > >>> Ignite
> > > >> >> > > > >>> > > > > repositories and to be tested/updated by the
> > > >> community
> > > >> >> for
> > > >> >> > > > every
> > > >> >> > > > >>> > > release.
> > > >> >> > > > >>> > > > > The modules are a subset of existing
> integrations
> > > >> that
> > > >> >> are
> > > >> >> > > > >>> adopted
> > > >> >> > > > >>> > > widely
> > > >> >> > > > >>> > > > > and require community attention.
> > > >> >> > > > >>> > > > >
> > > >> >> > > > >>> > > > > As for the independent integrations [1], those
> > are
> > > >> >> supposed
> > > >> >> > > to
> > > >> >> > > > be
> > > >> >> > > > >>> > > located
> > > >> >> > > > >>> > > > > in a separate GitHub repo. I would advise us to
> > > start
> > > >> >> > > > >>> implementing
> > > >> >> > > > >>> > this
> > > >> >> > > > >>> > > > > concept with the Pub/Sub integration. Even
> though
> > > it
> > > >> >> will
> > > >> >> > be
> > > >> >> > > > >>> located
> > > >> >> > > > >>> > > in a
> > > >> >> > > > >>> > > > > separate Github repo, it will be listed among
> > > >> officially
> > > >> >> > > > >>> available
> > > >> >> > > > >>> > > > > integrations on the Ignite website featuring
> > > >> Emmanouil
> > > >> >> as a
> > > >> >> > > > >>> > > contributor.
> > > >> >> > > > >>> > > > >
> > > >> >> > > > >>> > > > > What do you think?
> > > >> >> > > > >>> > > > >
> > > >> >> > > > >>> > > > >
> > > >> >> > > > >>> > > > > [1]
> > > >> >> > > > >>> > > > >
> > > >> >> > > > >>> > > > >
> > > >> >> > > > >>> > > >
> > > >> >> > > > >>> > >
> > > >> >> > > > >>> >
> > > >> >> > > > >>>
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > >> >> > > > >>> > > > >
> > > >> >> > > > >>> > > > >
> > > >> >> > > > >>> > > > >
> > > >> >> > > > >>> > > > > -
> > > >> >> > > > >>> > > > > Denis
> > > >> >> > > > >>> > > > >
> > > >> >> > > > >>> > > > >
> > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat Maitra <
> > > >> >> > > > >>> > [hidden email]
> > > >> >> > > > >>> > > >
> > > >> >> > > > >>> > > > > wrote:
> > > >> >> > > > >>> > > > >
> > > >> >> > > > >>> > > > > > Hi Emmanouil,
> > > >> >> > > > >>> > > > > >
> > > >> >> > > > >>> > > > > > The changes looks good to me. I wanted to
> check
> > > if
> > > >> you
> > > >> >> > are
> > > >> >> > > > also
> > > >> >> > > > >>> > able
> > > >> >> > > > >>> > > to
> > > >> >> > > > >>> > > > > > validate that the once data is added to
> Ignite
> > > >> cluster
> > > >> >> > you
> > > >> >> > > > are
> > > >> >> > > > >>> also
> > > >> >> > > > >>> > > > able
> > > >> >> > > > >>> > > > > to
> > > >> >> > > > >>> > > > > > access it using another Ignite client or
> using
> > > rest
> > > >> >> > > > endpoints.
> > > >> >> > > > >>> > > > > >
> > > >> >> > > > >>> > > > > > We use Teamcity for CI process, the details
> are
> > > as
> > > >> >> > follows
> > > >> >> > > > >>> > > > > >
> > > >> >> > > > >>> > > > > > Trigger validation of those test suites that
> > have
> > > >> been
> > > >> >> > > > >>> affected by
> > > >> >> > > > >>> > > your
> > > >> >> > > > >>> > > > > > changes on TeamCity <
> > > http://ci.ignite.apache.org/>:
> > > >> <
> > > >> >> > > > >>> > > > > http://204.14.53.153/
> > > >> >> > > > >>> > > > > > >
> > > >> >> > > > >>> > > > > >
> > > >> >> > > > >>> > > > > >    - Locate a test suite you have to check,
> > press
> > > >> >> button
> > > >> >> > > > named
> > > >> >> > > > >>> > "..."
> > > >> >> > > > >>> > > > that
> > > >> >> > > > >>> > > > > >    is located on the left of "Run" button.
> "Run
> > > >> custom
> > > >> >> > > build"
> > > >> >> > > > >>> > dialog
> > > >> >> > > > >>> > > > will
> > > >> >> > > > >>> > > > > >    appear;
> > > >> >> > > > >>> > > > > >    - Go to "Changes" tab and choose
> > > >> >> > > > >>> > "pull/<pull-request-number>/head"
> > > >> >> > > > >>> > > > in
> > > >> >> > > > >>> > > > > >    "Build branch" dropdown list;
> > > >> >> > > > >>> > > > > >    - Press "Run build" button and monitor
> tests
> > > >> >> results.
> > > >> >> > > > >>> > > > > >
> > > >> >> > > > >>> > > > > >
> > > >> >> > > > >>> > > > > >
> > > >> >> > > > >>> > > > > > @Roman [hidden email] - can you also
> please
> > > >> take a
> > > >> >> > > look?
> > > >> >> > > > >>> > > > > >
> > > >> >> > > > >>> > > > > > Regards,
> > > >> >> > > > >>> > > > > > Saikat
> > > >> >> > > > >>> > > > > >
> > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM Emmanouil
> > > >> Gkatziouras
> > > >> >> <
> > > >> >> > > > >>> > > > > > [hidden email]>
> > > >> >> > > > >>> > > > > > wrote:
> > > >> >> > > > >>> > > > > >
> > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
> > > >> >> > > > >>> > > > > > >
> > > >> >> > > > >>> > > > > > > I have my implementation on an Ignite fork
> on
> > > my
> > > >> >> > personal
> > > >> >> > > > >>> GitHub
> > > >> >> > > > >>> > > > > account
> > > >> >> > > > >>> > > > > > > [1]
> > > >> >> > > > >>> > > > > > > It is based on the Kafka implementation
> which
> > > is
> > > >> of
> > > >> >> a
> > > >> >> > > > polling
> > > >> >> > > > >>> > > nature.
> > > >> >> > > > >>> > > > > > > GCP does not provide a service like Kafka.
> > > >> Instead
> > > >> >> it
> > > >> >> > > > >>> provides us
> > > >> >> > > > >>> > > > with
> > > >> >> > > > >>> > > > > > > Pub/Sub which is just like Kafka.
> > > >> >> > > > >>> > > > > > > Therefore using the Kafka streamer as a
> > > guideline
> > > >> >> made
> > > >> >> > > more
> > > >> >> > > > >>> sense
> > > >> >> > > > >>> > > to
> > > >> >> > > > >>> > > > > me.
> > > >> >> > > > >>> > > > > > >
> > > >> >> > > > >>> > > > > > > Apart from the implementation I have also
> > > added a
> > > >> >> unit
> > > >> >> > > > test.
> > > >> >> > > > >>> > > > > > > It definitely needs some polishing and a
> > double
> > > >> >> check
> > > >> >> > on
> > > >> >> > > my
> > > >> >> > > > >>> side
> > > >> >> > > > >>> > to
> > > >> >> > > > >>> > > > > make
> > > >> >> > > > >>> > > > > > > the review smooth.
> > > >> >> > > > >>> > > > > > > For the tests a local Pub/Sub [2] server
> has
> > to
> > > >> be
> > > >> >> > setup
> > > >> >> > > > >>> however
> > > >> >> > > > >>> > I
> > > >> >> > > > >>> > > am
> > > >> >> > > > >>> > > > > not
> > > >> >> > > > >>> > > > > > > sure which is the best way to do so with
> > > regards
> > > >> to
> > > >> >> > your
> > > >> >> > > > >>> CI/CD.
> > > >> >> > > > >>> > > > > > >
> > > >> >> > > > >>> > > > > > > [1]
> > > >> >> > > > >>> > >
> > > >> >> >
> https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > > >> >> > > > >>> > > > > > > [2]
> > > >> >> > > > >>>
> > https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
> > > >> >> > > > >>> > > > > > >
> > > >> >> > > > >>> > > > > > > *Emmanouil Gkatziouras*
> > > >> >> > > > >>> > > > > > > https://egkatzioura.com/ |
> > > >> >> > > > >>> > > > > > >
> > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> >> > > > >>> > > > > > > https://github.com/gkatzioura
> > > >> >> > > > >>> > > > > > >
> > > >> >> > > > >>> > > > > > >
> > > >> >> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11, Saikat
> Maitra <
> > > >> >> > > > >>> > > [hidden email]
> > > >> >> > > > >>> > > > >
> > > >> >> > > > >>> > > > > > > wrote:
> > > >> >> > > > >>> > > > > > >
> > > >> >> > > > >>> > > > > > > > Hello Emmanouil,
> > > >> >> > > > >>> > > > > > > >
> > > >> >> > > > >>> > > > > > > > Can you please share more info about the
> > > >> >> > > implementation?
> > > >> >> > > > >>> > > > > > > >
> > > >> >> > > > >>> > > > > > > > We have similar implementation for data
> > > >> streamers
> > > >> >> > with
> > > >> >> > > > >>> Kafka
> > > >> >> > > > >>> > and
> > > >> >> > > > >>> > > > > > RocketMQ
> > > >> >> > > > >>> > > > > > > >
> > > >> >> > > > >>> > > > > > > >
> > > >> >> > https://apacheignite-mix.readme.io/docs/kafka-streamer
> > > >> >> > > > >>> > > > > > > >
> > > >> >> > > > >>> > > > > > > >
> > > >> >> > > > https://apacheignite-mix.readme.io/docs/rocketmq-streamer
> > > >> >> > > > >>> > > > > > > >
> > > >> >> > > > >>> > > > > > > > Please review and share if you have any
> > > >> questions.
> > > >> >> > > > >>> > > > > > > >
> > > >> >> > > > >>> > > > > > > > Regards,
> > > >> >> > > > >>> > > > > > > > Saikat
> > > >> >> > > > >>> > > > > > > >
> > > >> >> > > > >>> > > > > > > >
> > > >> >> > > > >>> > > > > > > >
> > > >> >> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56 PM Emmanouil
> > > >> >> > Gkatziouras <
> > > >> >> > > > >>> > > > > > > > [hidden email]>
> > > >> >> > > > >>> > > > > > > > wrote:
> > > >> >> > > > >>> > > > > > > >
> > > >> >> > > > >>> > > > > > > > > Hi all,
> > > >> >> > > > >>> > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > By mentioning Pub/Sub I mean Google
> > Cloud's
> > > >> >> Pub/Sub
> > > >> >> > > [1]
> > > >> >> > > > >>> > > > > > > > > Yes it is indeed cloud specific. It is
> > the
> > > >> >> > messaging
> > > >> >> > > > >>> service
> > > >> >> > > > >>> > > > > provided
> > > >> >> > > > >>> > > > > > > by
> > > >> >> > > > >>> > > > > > > > > GCP.
> > > >> >> > > > >>> > > > > > > > > I have made the integration with it,
> but
> > I
> > > >> >> haven't
> > > >> >> > > > >>> published
> > > >> >> > > > >>> > it
> > > >> >> > > > >>> > > > yet
> > > >> >> > > > >>> > > > > > (I
> > > >> >> > > > >>> > > > > > > am
> > > >> >> > > > >>> > > > > > > > > waiting for the discussion/ticket and
> > > proceed
> > > >> >> with
> > > >> >> > > the
> > > >> >> > > > >>> > > > > contribution).
> > > >> >> > > > >>> > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > Kind regards,
> > > >> >> > > > >>> > > > > > > > > Emmanouil
> > > >> >> > > > >>> > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > [1]
> > > >> >> https://cloud.google.com/pubsub/docs/overview
> > > >> >> > > > >>> > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
> > > >> >> > > > >>> > > > > > > > > https://egkatzioura.com/ |
> > > >> >> > > > >>> > > > > > > > >
> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> >> > > > >>> > > > > > > > > https://github.com/gkatzioura
> > > >> >> > > > >>> > > > > > > > >
> > > >> >> > > > >>> > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at 21:45, Denis
> > Magda <
> > > >> >> > > > >>> [hidden email]
> > > >> >> > > > >>> > >
> > > >> >> > > > >>> > > > > wrote:
> > > >> >> > > > >>> > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > > Hello Emmanouil,
> > > >> >> > > > >>> > > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > > Are you referring to any specific
> > > >> >> implementation
> > > >> >> > > > >>> tailored
> > > >> >> > > > >>> > > for a
> > > >> >> > > > >>> > > > > > > cloud?
> > > >> >> > > > >>> > > > > > > > If
> > > >> >> > > > >>> > > > > > > > > > I'm not mistaken you were mentioning
> > > >> AWS/GCE.
> > > >> >> Is
> > > >> >> > > > there
> > > >> >> > > > >>> any
> > > >> >> > > > >>> > > > Github
> > > >> >> > > > >>> > > > > > > > source
> > > >> >> > > > >>> > > > > > > > > > code?
> > > >> >> > > > >>> > > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > > Here is a list of existing
> integrations
> > > >> with
> > > >> >> > > > streaming
> > > >> >> > > > >>> and
> > > >> >> > > > >>> > > > > pub/sub
> > > >> >> > > > >>> > > > > > > > > > frameworks:
> > > >> >> > > > >>> > > > > > > > > >
> > > >> >> https://apacheignite-mix.readme.io/docs/overview
> > > >> >> > > > >>> > > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > > with the source code in Ignite
> master:
> > > >> >> > > > >>> > > > > > > > > >
> > > >> >> > > https://github.com/apache/ignite/tree/master/modules
> > > >> >> > > > >>> > > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > > Check out those resources to see what
> > are
> > > >> the
> > > >> >> > most
> > > >> >> > > > >>> suitable
> > > >> >> > > > >>> > > > > Ignite
> > > >> >> > > > >>> > > > > > > APIs
> > > >> >> > > > >>> > > > > > > > > for
> > > >> >> > > > >>> > > > > > > > > > such integrations.
> > > >> >> > > > >>> > > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > > @Saikat Maitra <
> > [hidden email]>
> > > ,
> > > >> >> > @Roman
> > > >> >> > > <
> > > >> >> > > > >>> > > > > > [hidden email]
> > > >> >> > > > >>> > > > > > > >
> > > >> >> > > > >>> > > > > > > > ,
> > > >> >> > > > >>> > > > > > > > > as
> > > >> >> > > > >>> > > > > > > > > > streaming technologies
> > > >> >> contributors/maintainers,
> > > >> >> > > > could
> > > >> >> > > > >>> you
> > > >> >> > > > >>> > > > please
> > > >> >> > > > >>> > > > > > > join
> > > >> >> > > > >>> > > > > > > > > this
> > > >> >> > > > >>> > > > > > > > > > thread?
> > > >> >> > > > >>> > > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > > -
> > > >> >> > > > >>> > > > > > > > > > Denis
> > > >> >> > > > >>> > > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > > On Thu, Oct 10, 2019 at 7:58 AM
> > Emmanouil
> > > >> >> > > > Gkatziouras <
> > > >> >> > > > >>> > > > > > > > > > [hidden email]>
> > > >> >> > > > >>> > > > > > > > > > wrote:
> > > >> >> > > > >>> > > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > > > Hello everyone,
> > > >> >> > > > >>> > > > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > > > I started using Ignite lately. Part
> > of
> > > my
> > > >> >> work
> > > >> >> > > > >>> involves
> > > >> >> > > > >>> > > > > Pub/Sub,
> > > >> >> > > > >>> > > > > > > > thus I
> > > >> >> > > > >>> > > > > > > > > > > created a pub/sub streamer.
> > > >> >> > > > >>> > > > > > > > > > > Pub/Sub is pretty close to Kafka.
> > > >> >> > > > >>> > > > > > > > > > > Here's how the streamer works.
> Every
> > > node
> > > >> >> > creates
> > > >> >> > > > an
> > > >> >> > > > >>> > > instance
> > > >> >> > > > >>> > > > > of
> > > >> >> > > > >>> > > > > > a
> > > >> >> > > > >>> > > > > > > > > > Pub/Sub
> > > >> >> > > > >>> > > > > > > > > > > client.
> > > >> >> > > > >>> > > > > > > > > > > The clients subscribe to the same
> > topic
> > > >> >> using
> > > >> >> > the
> > > >> >> > > > >>> same
> > > >> >> > > > >>> > > > > > subscription
> > > >> >> > > > >>> > > > > > > > > name.
> > > >> >> > > > >>> > > > > > > > > > > A message is being sent and one of
> > the
> > > >> nodes
> > > >> >> > will
> > > >> >> > > > >>> receive
> > > >> >> > > > >>> > > the
> > > >> >> > > > >>> > > > > > > > message,
> > > >> >> > > > >>> > > > > > > > > > > process it and then forward it to
> > > Ignite
> > > >> >> using
> > > >> >> > > the
> > > >> >> > > > >>> > > extractor
> > > >> >> > > > >>> > > > > > > > provided.
> > > >> >> > > > >>> > > > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > > > Any thoughts?
> > > >> >> > > > >>> > > > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > > > *Emmanouil Gkatziouras*
> > > >> >> > > > >>> > > > > > > > > > > https://egkatzioura.com/ |
> > > >> >> > > > >>> > > > > > > > > > >
> > > >> >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> >> > > > >>> > > > > > > > > > > https://github.com/gkatzioura
> > > >> >> > > > >>> > > > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > >
> > > >> >> > > > >>> > > > > > > > >
> > > >> >> > > > >>> > > > > > > >
> > > >> >> > > > >>> > > > > > >
> > > >> >> > > > >>> > > > > >
> > > >> >> > > > >>> > > > >
> > > >> >> > > > >>> > > >
> > > >> >> > > > >>> > >
> > > >> >> > > > >>> >
> > > >> >> > > > >>>
> > > >> >> > > > >>
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >> >
> > > >>
> > > >
> > >
> >
>

[Saikat Maitra]

Hi Emmanouil,

I have fixed the Flink module testsuites and have setup new project in
teamcity.

https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview

I am looking into setting up the build.

Regards,
Saikat

On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <[hidden email]>
wrote:

> Hi Emmanouil,
>
> Thank you for your email. Yes, the plan is once the Flink PR is merged it
> will provide the Licence, parent POMs etc and you can rebase from master
> branch and apply the changes on top of it.
>
> I am getting some issues with test failures in local with
> GridTestProperties as the test.properties is not present in this new
> project but should be available from dependencies.
>
> Once I address this issue, I will go ahead and merge the changes and then
> we can take it from there.
>
> Regards,
> Saikat
>
> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
> [hidden email]> wrote:
>
>> Hi all,
>>
>> I made my first pull request [1]. Since this project is brand new (no
>> parent poms, licensing), I reckoned it was best to use Saikat's branch on
>> Flink.
>> I suppose the Flink branch will be merged first. If not please give me
>> guidelines on how I should proceed next.
>>
>> Kind regards
>> Emmanouil
>>
>> [1] https://github.com/apache/ignite-extensions/pull/2
>>
>> *Emmanouil Gkatziouras*
>> https://egkatzioura.com/ |
>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> https://github.com/gkatzioura
>>
>>
>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <[hidden email]> wrote:
>>
>> > Awesome, ping us whenever you're ready!
>> >
>> > -
>> > Denis
>> >
>> >
>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
>> > [hidden email]>
>> > wrote:
>> >
>> > > Hi all!
>> > >
>> > > I am sorry for being late on that. I was trying to refactor the test
>> in
>> > > order not to be in need of any external tools or spinning up a server.
>> > > I did forked the new repo and indeed my changes there, so a pull
>> request
>> > is
>> > > a matter of time!
>> > >
>> > > Kind regards
>> > >
>> > > *Emmanouil Gkatziouras*
>> > > https://egkatzioura.com/ |
>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > https://github.com/gkatzioura
>> > >
>> > >
>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <[hidden email]> wrote:
>> > >
>> > > > Hi Emmanouil,
>> > > >
>> > > > Do you have any questions or need any support from the community?
>> > > >
>> > > > -
>> > > > Denis
>> > > >
>> > > >
>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
>> [hidden email]
>> > >
>> > > > wrote:
>> > > >
>> > > >> Hi Emmanouil,
>> > > >>
>> > > >> Can you please take a looks at dev utils, if this is something you
>> are
>> > > >> looking for
>> > > >> https://github.com/apache/ignite/tree/master/modules/dev-utils?
>> > > >>
>> > > >> IMO, if you can release Pub/Sub server in maven and then use it as
>> > > >> dependency, that would be great.
>> > > >>
>> > > >> Regards,
>> > > >> Saikat
>> > > >>
>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
>> > [hidden email]>
>> > > >> wrote:
>> > > >>
>> > > >> > Hi Emmanouil,
>> > > >> >
>> > > >> > The master branch has been initialized. I have raised a PR to
>> > migrate
>> > > >> > Flink module to ignite-extensions.
>> > > >> >
>> > > >> > PR https://github.com/apache/ignite-extensions/pull/1
>> > > >> > Jira : https://issues.apache.org/jira/browse/IGNITE-12356
>> > > >> >
>> > > >> > Regards,
>> > > >> > Saikat
>> > > >> >
>> > > >> >
>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
>> > > >> > [hidden email]> wrote:
>> > > >> >
>> > > >> >> Hi all!
>> > > >> >>
>> > > >> >> This is an update. The integration is tested with mocks based on
>> > the
>> > > >> >> original Ignite repo [1].
>> > > >> >> I am currently waiting for the first commits on the new
>> integration
>> > > >> >> project
>> > > >> >> to take place in order to create my pull request accordingly to
>> the
>> > > >> >> initial
>> > > >> >> commits.
>> > > >> >> In order to simulate the Pub/Sub server in a test I used a test
>> > > server
>> > > >> >> implementation [2]. Unfortunately they don't have this util
>> > uploaded
>> > > >> in a
>> > > >> >> maven repo, thus it needs to be hosted on a maven repo (Apache
>> > > License
>> > > >> >> 2.0). Is there a repo for utilities that Ignite uses for the
>> > > >> test/build?
>> > > >> >> I have also created a ticket [3]
>> > > >> >>
>> > > >> >> Kind regards
>> > > >> >>
>> > > >> >> [1]
>> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
>> > > >> >> [2]
>> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
>> > > >> >> [3] https://issues.apache.org/jira/browse/IGNITE-12262
>> > > >> >>
>> > > >> >> *Emmanouil Gkatziouras*
>> > > >> >> https://egkatzioura.com/ |
>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > >> >> https://github.com/gkatzioura
>> > > >> >>
>> > > >> >>
>> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
>> > [hidden email]
>> > > >
>> > > >> >> wrote:
>> > > >> >>
>> > > >> >> > Hello Ilya,
>> > > >> >> >
>> > > >> >> > Thank you for your email. Yes, before we remove any
>> integration
>> > we
>> > > >> will
>> > > >> >> > move them to another repository and release.
>> > > >> >> >
>> > > >> >> > We are thinking we will start with Pub/Sub Streamer
>> > implementation
>> > > >> and
>> > > >> >> > continue to move other integration as mentioned in the
>> following
>> > > doc.
>> > > >> >> >
>> > > >> >> >
>> > > >> >> >
>> > > >> >>
>> > > >>
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> > > >> >> >
>> > > >> >> > Regards,
>> > > >> >> > Saikat
>> > > >> >> >
>> > > >> >> >
>> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
>> > > >> >> > [hidden email]>
>> > > >> >> > wrote:
>> > > >> >> >
>> > > >> >> > > Hello!
>> > > >> >> > >
>> > > >> >> > > My take on this: we want to offload some of integrations,
>> but
>> > > >> please
>> > > >> >> > commit
>> > > >> >> > > and release them to another project first, and only then we
>> > will
>> > > >> >> decide
>> > > >> >> > to
>> > > >> >> > > remove them from our repository.
>> > > >> >> > >
>> > > >> >> > > Please also make sure they are available with same artifact
>> > name
>> > > >> and
>> > > >> >> API.
>> > > >> >> > > If this has to be changed, let's wait for AI 3.0
>> > > >> >> > >
>> > > >> >> > > Regards,
>> > > >> >> > > --
>> > > >> >> > > Ilya Kasnacheev
>> > > >> >> > >
>> > > >> >> > >
>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <
>> > > >> [hidden email]>:
>> > > >> >> > >
>> > > >> >> > > > Hello Emmanouil,
>> > > >> >> > > >
>> > > >> >> > > > As discussed earlier, I discussed with Apache Bahir
>> community
>> > > and
>> > > >> >> they
>> > > >> >> > > are
>> > > >> >> > > > interested to have Apache Ignite extensions as part of
>> Apache
>> > > >> Bahir
>> > > >> >> > > > project.
>> > > >> >> > > >
>> > > >> >> > > > Can you please share Pub/Sub streamer implementation
>> details
>> > > with
>> > > >> >> > Apache
>> > > >> >> > > > Bahir community and request for feedback.
>> > > >> >> > > >
>> > > >> >> > > > https://bahir.apache.org/
>> > > >> >> > > >
>> > > >> >> > > > I have also requested for contributor access in Jira for
>> > Apache
>> > > >> >> Bahir
>> > > >> >> > > > project so that I can create issues and assign to myself.
>> > > >> >> > > >
>> > > >> >> > > > I can help with code reviews as well.
>> > > >> >> > > >
>> > > >> >> > > > Here is the email thread for reference
>> > > >> >> > > >
>> > > https://www.mail-archive.com/dev@.../msg02703.html
>> > > >> >> > > >
>> > > >> >> > > > Regards,
>> > > >> >> > > > Saikat
>> > > >> >> > > >
>> > > >> >> > > >
>> > > >> >> > > >
>> > > >> >> > > >
>> > > >> >> > > >
>> > > >> >> > > >
>> > > >> >> > > >
>> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
>> > > >> >> [hidden email]
>> > > >> >> > >
>> > > >> >> > > > wrote:
>> > > >> >> > > >
>> > > >> >> > > > > Hello,
>> > > >> >> > > > >
>> > > >> >> > > > > As discussed I have created following discussion
>> threads on
>> > > our
>> > > >> >> > > migration
>> > > >> >> > > > > proposals for Apache Ignite extensions:
>> > > >> >> > > > >
>> > > >> >> > > > >
>> > > >> >> > > > >
>> > > >> >> > > >
>> > > >> >> > >
>> > > >> >> >
>> > > >> >>
>> > > >>
>> > >
>> >
>> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
>> > > >> >> > > > >
>> > > >> >> > > > >
>> > > >> https://www.mail-archive.com/dev@.../msg02703.html
>> > > >> >> > > > >
>> > > >> >> > > > > I will share update as I hear more information.
>> > > >> >> > > > >
>> > > >> >> > > > > Regards,
>> > > >> >> > > > > Saikat
>> > > >> >> > > > >
>> > > >> >> > > > >
>> > > >> >> > > > >
>> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
>> > > >> >> > [hidden email]
>> > > >> >> > > >
>> > > >> >> > > > > wrote:
>> > > >> >> > > > >
>> > > >> >> > > > >> Hello Denis,
>> > > >> >> > > > >>
>> > > >> >> > > > >> Sure, sounds good. I will create a separate discussion
>> > > thread
>> > > >> to
>> > > >> >> get
>> > > >> >> > > > >> community feedback on whether we
>> > > >> >> > > > >> should join the Bahir or kick-off "Ignite Extensions"
>> > > project.
>> > > >> >> > > > >>
>> > > >> >> > > > >> Regards,
>> > > >> >> > > > >> Saikat
>> > > >> >> > > > >>
>> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <
>> > > >> [hidden email]>
>> > > >> >> > > wrote:
>> > > >> >> > > > >>
>> > > >> >> > > > >>> Folks,
>> > > >> >> > > > >>>
>> > > >> >> > > > >>> The concept of Apache Bahir is precisely what we need!
>> > > >> Saikat,
>> > > >> >> > thanks
>> > > >> >> > > > for
>> > > >> >> > > > >>> doing the research. Why I believe the * idea* fits us
>> > well:
>> > > >> >> > > > >>>
>> > > >> >> > > > >>>    - All integrations can be stored in separate Github
>> > > >> >> repositories
>> > > >> >> > > and
>> > > >> >> > > > >>>    have their dev lifecycles. We've not obliged to
>> couple
>> > > all
>> > > >> >> the
>> > > >> >> > > > >>> integrations
>> > > >> >> > > > >>>    in a single repo. For instance, Spark can be
>> located
>> > in
>> > > a
>> > > >> >> > > dedicated
>> > > >> >> > > > >>> repo
>> > > >> >> > > > >>>    while streaming integrations might be in a single
>> one.
>> > > >> It's
>> > > >> >> up
>> > > >> >> > to
>> > > >> >> > > > us.
>> > > >> >> > > > >>>    - All the repositories and integrations belong to
>> ASF.
>> > > >> We're
>> > > >> >> not
>> > > >> >> > > > >>> dumping
>> > > >> >> > > > >>>    them on Github but rather keep supporting and
>> > developing
>> > > >> in
>> > > >> >> > > > >>> accordance with
>> > > >> >> > > > >>>    ASF vision and practices.
>> > > >> >> > > > >>>    - There is a way to reward contributors via
>> > > committership
>> > > >> and
>> > > >> >> > PMC
>> > > >> >> > > > >>>    membership. You won't get this if a project is just
>> > one
>> > > of
>> > > >> >> the
>> > > >> >> > > > >>> millions of
>> > > >> >> > > > >>>    Github projects.
>> > > >> >> > > > >>>
>> > > >> >> > > > >>> Saikat, as Ignite PMC member, could you please
>> kick-off a
>> > > >> >> separate
>> > > >> >> > > > >>> dev-list
>> > > >> >> > > > >>> discussion to involve more community members and
>> > referring
>> > > to
>> > > >> >> this
>> > > >> >> > > > >>> thread?
>> > > >> >> > > > >>> I think the primary question the community needs to
>> > answer
>> > > >> >> whether
>> > > >> >> > we
>> > > >> >> > > > >>> should join the Bahir or kick-off "Ignite Extensions"
>> > > >> project?
>> > > >> >> > > > >>>
>> > > >> >> > > > >>> -
>> > > >> >> > > > >>> Denis
>> > > >> >> > > > >>>
>> > > >> >> > > > >>>
>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev <
>> > > >> >> > > > [hidden email]>
>> > > >> >> > > > >>> wrote:
>> > > >> >> > > > >>>
>> > > >> >> > > > >>> > Maybe we could move all our Streaming Integrations
>> > there,
>> > > >> but
>> > > >> >> > what
>> > > >> >> > > is
>> > > >> >> > > > >>> about
>> > > >> >> > > > >>> > maintaining and committer permissions to the new
>> > > >> repositories?
>> > > >> >> > > > >>> > I see the list of the committers and PMC members
>> there
>> > > >> >> > > > >>> > https://bahir.apache.org/community-members/
>> > > >> >> > > > >>> >
>> > > >> >> > > > >>> > Could somebody from Ignite community be added to
>> this
>> > > list
>> > > >> as
>> > > >> >> a
>> > > >> >> > > > >>> > PMC/committer to maintain Ignite integrations?
>> > > >> >> > > > >>> > If yes, I'd happy to join this project and
>> collaborate
>> > > with
>> > > >> >> these
>> > > >> >> > > > guys
>> > > >> >> > > > >>> > together
>> > > >> >> > > > >>> > If no, and we should start from the zero level with
>> > > >> external
>> > > >> >> PRs
>> > > >> >> > -
>> > > >> >> > > > >>> hmmm,
>> > > >> >> > > > >>> > it's better to have our own external repository with
>> > > >> >> ApacheBahirr
>> > > >> >> > > > >>> ideology.
>> > > >> >> > > > >>> >
>> > > >> >> > > > >>> > Also, I agree, that all connectors could be moved
>> there
>> > > (in
>> > > >> >> > > > >>> ApacheBahirr
>> > > >> >> > > > >>> > like repository), but not all modules from the page
>> > > >> >> > > > >>> >
>> > > >> >> > > > >>> >
>> > > >> >> > > > >>>
>> > > >> >> > > >
>> > > >> >> > >
>> > > >> >> >
>> > > >> >>
>> > > >>
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> > > >> >> > > > >>> > because
>> > > >> >> > > > >>> > they use different parts of core modules and could
>> be
>> > > >> >> developed
>> > > >> >> > > with
>> > > >> >> > > > >>> > different velocity.
>> > > >> >> > > > >>> >
>> > > >> >> > > > >>> > In reality, before creation of new repositories we
>> need
>> > > >> wide
>> > > >> >> > > > discussion
>> > > >> >> > > > >>> > based on the document mentioned above.
>> > > >> >> > > > >>> >
>> > > >> >> > > > >>> > P.S. Of course, we could experiment in the next
>> release
>> > > 2.8
>> > > >> >> with
>> > > >> >> > > one
>> > > >> >> > > > >>> or two
>> > > >> >> > > > >>> > integrations like twitter or ZeroMQ
>> > > >> >> > > > >>> > Also, I'd like Denis Magda idea for the separate
>> > > >> repositories
>> > > >> >> in
>> > > >> >> > > > Apache
>> > > >> >> > > > >>> > Github to maintain them by anybody.
>> > > >> >> > > > >>> >
>> > > >> >> > > > >>> >
>> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra <
>> > > >> >> > > [hidden email]
>> > > >> >> > > > >:
>> > > >> >> > > > >>> >
>> > > >> >> > > > >>> > > Hi Denis, Emmanouil
>> > > >> >> > > > >>> > >
>> > > >> >> > > > >>> > > Thank you for your email. I think creating a
>> separate
>> > > >> >> > integration
>> > > >> >> > > > >>> project
>> > > >> >> > > > >>> > > in github and also proposing it as an apache
>> > incubator
>> > > >> >> project
>> > > >> >> > > will
>> > > >> >> > > > >>> be a
>> > > >> >> > > > >>> > > good move. The other separate integration modules
>> > > >> >> > > > >>> > >
>> > > >> >> > > > >>> > >
>> > > >> >> > > > >>> >
>> > > >> >> > > > >>>
>> > > >> >> > > >
>> > > >> >> > >
>> > > >> >> >
>> > > >> >>
>> > > >>
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> > > >> >> > > > >>> > > can
>> > > >> >> > > > >>> > > be moved to this new apache incubator project.
>> > > >> >> > > > >>> > >
>> > > >> >> > > > >>> > > There are similar integration available for Flink
>> and
>> > > >> Spark
>> > > >> >> in
>> > > >> >> > > > Apache
>> > > >> >> > > > >>> > Bahir
>> > > >> >> > > > >>> > > https://bahir.apache.org/
>> > > >> >> > > > >>> > >
>> > > >> >> > > > >>> > > Do you think we should reach out to Apache Bahir
>> > > >> community
>> > > >> >> > with a
>> > > >> >> > > > >>> > proposal
>> > > >> >> > > > >>> > > or we should plan to create a separate Apache
>> > Incubator
>> > > >> >> > project?
>> > > >> >> > > > >>> > >
>> > > >> >> > > > >>> > > Regards,
>> > > >> >> > > > >>> > > Saikat
>> > > >> >> > > > >>> > >
>> > > >> >> > > > >>> > >
>> > > >> >> > > > >>> > >
>> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil
>> > Gkatziouras <
>> > > >> >> > > > >>> > > [hidden email]>
>> > > >> >> > > > >>> > > wrote:
>> > > >> >> > > > >>> > >
>> > > >> >> > > > >>> > > > Hi all!
>> > > >> >> > > > >>> > > >
>> > > >> >> > > > >>> > > > Based on the discussions most probably this is
>> > going
>> > > to
>> > > >> >> be on
>> > > >> >> > > > >>> another
>> > > >> >> > > > >>> > > > GitHub repo. Therefore I will prepare a
>> standalone
>> > > >> project
>> > > >> >> > with
>> > > >> >> > > > the
>> > > >> >> > > > >>> > > Pub/Sub
>> > > >> >> > > > >>> > > > module and once the repository is created a pull
>> > > >> request
>> > > >> >> > shall
>> > > >> >> > > be
>> > > >> >> > > > >>> > issued.
>> > > >> >> > > > >>> > > > If there is anything else I can do in order to
>> > > >> facilitate
>> > > >> >> > this
>> > > >> >> > > > >>> please
>> > > >> >> > > > >>> > let
>> > > >> >> > > > >>> > > > me know.
>> > > >> >> > > > >>> > > >
>> > > >> >> > > > >>> > > > If a ticket is created for this issue, I shall
>> be
>> > > able
>> > > >> to
>> > > >> >> > > create
>> > > >> >> > > > a
>> > > >> >> > > > >>> pull
>> > > >> >> > > > >>> > > > request and thus a build will take place on
>> > ignite's
>> > > >> CI.
>> > > >> >> > > > >>> > > > If everyone is aligned with adding a Pub/Sub
>> > > >> >> implementation
>> > > >> >> > > may I
>> > > >> >> > > > >>> > create
>> > > >> >> > > > >>> > > a
>> > > >> >> > > > >>> > > > ticket on JIRA?
>> > > >> >> > > > >>> > > >
>> > > >> >> > > > >>> > > > My next question has to do on how Ignite
>> configures
>> > > the
>> > > >> >> build
>> > > >> >> > > > jobs
>> > > >> >> > > > >>> in
>> > > >> >> > > > >>> > > Team
>> > > >> >> > > > >>> > > > City. The unit tests for my implementation need
>> a
>> > > local
>> > > >> >> > Pub/Sub
>> > > >> >> > > > >>> server
>> > > >> >> > > > >>> > up
>> > > >> >> > > > >>> > > > and running (they provide one for testing
>> > purposes).
>> > > >> Will
>> > > >> >> > this
>> > > >> >> > > > take
>> > > >> >> > > > >>> > > effect
>> > > >> >> > > > >>> > > > in a form of a build script inside my
>> > implementation
>> > > or
>> > > >> >> is it
>> > > >> >> > > > >>> something
>> > > >> >> > > > >>> > > > that should be configured on Team City?
>> > > >> >> > > > >>> > > >
>> > > >> >> > > > >>> > > > Kind regards,
>> > > >> >> > > > >>> > > > Emmanouil
>> > > >> >> > > > >>> > > >
>> > > >> >> > > > >>> > > >
>> > > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
>> > > >> >> > > > >>> > > > https://egkatzioura.com/ |
>> > > >> >> > > > >>> > > >
>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > >> >> > > > >>> > > > https://github.com/gkatzioura
>> > > >> >> > > > >>> > > >
>> > > >> >> > > > >>> > > >
>> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis Magda <
>> > > >> >> [hidden email]
>> > > >> >> > >
>> > > >> >> > > > >>> wrote:
>> > > >> >> > > > >>> > > >
>> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
>> > > >> >> > > > >>> > > > >
>> > > >> >> > > > >>> > > > > After contemplating on this, let me propose
>> > another
>> > > >> way
>> > > >> >> > with
>> > > >> >> > > > this
>> > > >> >> > > > >>> > > > > particular integration that is aligned with
>> our
>> > > >> >> > > modularization
>> > > >> >> > > > >>> > > endeavor.
>> > > >> >> > > > >>> > > > > The modularization [1] defines the Ignite
>> core,
>> > > >> Ignite
>> > > >> >> > > modules,
>> > > >> >> > > > >>> and
>> > > >> >> > > > >>> > > > > independent integrations. The first two are
>> to be
>> > > >> >> located
>> > > >> >> > in
>> > > >> >> > > > >>> Ignite
>> > > >> >> > > > >>> > > > > repositories and to be tested/updated by the
>> > > >> community
>> > > >> >> for
>> > > >> >> > > > every
>> > > >> >> > > > >>> > > release.
>> > > >> >> > > > >>> > > > > The modules are a subset of existing
>> integrations
>> > > >> that
>> > > >> >> are
>> > > >> >> > > > >>> adopted
>> > > >> >> > > > >>> > > widely
>> > > >> >> > > > >>> > > > > and require community attention.
>> > > >> >> > > > >>> > > > >
>> > > >> >> > > > >>> > > > > As for the independent integrations [1], those
>> > are
>> > > >> >> supposed
>> > > >> >> > > to
>> > > >> >> > > > be
>> > > >> >> > > > >>> > > located
>> > > >> >> > > > >>> > > > > in a separate GitHub repo. I would advise us
>> to
>> > > start
>> > > >> >> > > > >>> implementing
>> > > >> >> > > > >>> > this
>> > > >> >> > > > >>> > > > > concept with the Pub/Sub integration. Even
>> though
>> > > it
>> > > >> >> will
>> > > >> >> > be
>> > > >> >> > > > >>> located
>> > > >> >> > > > >>> > > in a
>> > > >> >> > > > >>> > > > > separate Github repo, it will be listed among
>> > > >> officially
>> > > >> >> > > > >>> available
>> > > >> >> > > > >>> > > > > integrations on the Ignite website featuring
>> > > >> Emmanouil
>> > > >> >> as a
>> > > >> >> > > > >>> > > contributor.
>> > > >> >> > > > >>> > > > >
>> > > >> >> > > > >>> > > > > What do you think?
>> > > >> >> > > > >>> > > > >
>> > > >> >> > > > >>> > > > >
>> > > >> >> > > > >>> > > > > [1]
>> > > >> >> > > > >>> > > > >
>> > > >> >> > > > >>> > > > >
>> > > >> >> > > > >>> > > >
>> > > >> >> > > > >>> > >
>> > > >> >> > > > >>> >
>> > > >> >> > > > >>>
>> > > >> >> > > >
>> > > >> >> > >
>> > > >> >> >
>> > > >> >>
>> > > >>
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> > > >> >> > > > >>> > > > >
>> > > >> >> > > > >>> > > > >
>> > > >> >> > > > >>> > > > >
>> > > >> >> > > > >>> > > > > -
>> > > >> >> > > > >>> > > > > Denis
>> > > >> >> > > > >>> > > > >
>> > > >> >> > > > >>> > > > >
>> > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat Maitra
>> <
>> > > >> >> > > > >>> > [hidden email]
>> > > >> >> > > > >>> > > >
>> > > >> >> > > > >>> > > > > wrote:
>> > > >> >> > > > >>> > > > >
>> > > >> >> > > > >>> > > > > > Hi Emmanouil,
>> > > >> >> > > > >>> > > > > >
>> > > >> >> > > > >>> > > > > > The changes looks good to me. I wanted to
>> check
>> > > if
>> > > >> you
>> > > >> >> > are
>> > > >> >> > > > also
>> > > >> >> > > > >>> > able
>> > > >> >> > > > >>> > > to
>> > > >> >> > > > >>> > > > > > validate that the once data is added to
>> Ignite
>> > > >> cluster
>> > > >> >> > you
>> > > >> >> > > > are
>> > > >> >> > > > >>> also
>> > > >> >> > > > >>> > > > able
>> > > >> >> > > > >>> > > > > to
>> > > >> >> > > > >>> > > > > > access it using another Ignite client or
>> using
>> > > rest
>> > > >> >> > > > endpoints.
>> > > >> >> > > > >>> > > > > >
>> > > >> >> > > > >>> > > > > > We use Teamcity for CI process, the details
>> are
>> > > as
>> > > >> >> > follows
>> > > >> >> > > > >>> > > > > >
>> > > >> >> > > > >>> > > > > > Trigger validation of those test suites that
>> > have
>> > > >> been
>> > > >> >> > > > >>> affected by
>> > > >> >> > > > >>> > > your
>> > > >> >> > > > >>> > > > > > changes on TeamCity <
>> > > http://ci.ignite.apache.org/>:
>> > > >> <
>> > > >> >> > > > >>> > > > > http://204.14.53.153/
>> > > >> >> > > > >>> > > > > > >
>> > > >> >> > > > >>> > > > > >
>> > > >> >> > > > >>> > > > > >    - Locate a test suite you have to check,
>> > press
>> > > >> >> button
>> > > >> >> > > > named
>> > > >> >> > > > >>> > "..."
>> > > >> >> > > > >>> > > > that
>> > > >> >> > > > >>> > > > > >    is located on the left of "Run" button.
>> "Run
>> > > >> custom
>> > > >> >> > > build"
>> > > >> >> > > > >>> > dialog
>> > > >> >> > > > >>> > > > will
>> > > >> >> > > > >>> > > > > >    appear;
>> > > >> >> > > > >>> > > > > >    - Go to "Changes" tab and choose
>> > > >> >> > > > >>> > "pull/<pull-request-number>/head"
>> > > >> >> > > > >>> > > > in
>> > > >> >> > > > >>> > > > > >    "Build branch" dropdown list;
>> > > >> >> > > > >>> > > > > >    - Press "Run build" button and monitor
>> tests
>> > > >> >> results.
>> > > >> >> > > > >>> > > > > >
>> > > >> >> > > > >>> > > > > >
>> > > >> >> > > > >>> > > > > >
>> > > >> >> > > > >>> > > > > > @Roman [hidden email] - can you also
>> please
>> > > >> take a
>> > > >> >> > > look?
>> > > >> >> > > > >>> > > > > >
>> > > >> >> > > > >>> > > > > > Regards,
>> > > >> >> > > > >>> > > > > > Saikat
>> > > >> >> > > > >>> > > > > >
>> > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM Emmanouil
>> > > >> Gkatziouras
>> > > >> >> <
>> > > >> >> > > > >>> > > > > > [hidden email]>
>> > > >> >> > > > >>> > > > > > wrote:
>> > > >> >> > > > >>> > > > > >
>> > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
>> > > >> >> > > > >>> > > > > > >
>> > > >> >> > > > >>> > > > > > > I have my implementation on an Ignite
>> fork on
>> > > my
>> > > >> >> > personal
>> > > >> >> > > > >>> GitHub
>> > > >> >> > > > >>> > > > > account
>> > > >> >> > > > >>> > > > > > > [1]
>> > > >> >> > > > >>> > > > > > > It is based on the Kafka implementation
>> which
>> > > is
>> > > >> of
>> > > >> >> a
>> > > >> >> > > > polling
>> > > >> >> > > > >>> > > nature.
>> > > >> >> > > > >>> > > > > > > GCP does not provide a service like Kafka.
>> > > >> Instead
>> > > >> >> it
>> > > >> >> > > > >>> provides us
>> > > >> >> > > > >>> > > > with
>> > > >> >> > > > >>> > > > > > > Pub/Sub which is just like Kafka.
>> > > >> >> > > > >>> > > > > > > Therefore using the Kafka streamer as a
>> > > guideline
>> > > >> >> made
>> > > >> >> > > more
>> > > >> >> > > > >>> sense
>> > > >> >> > > > >>> > > to
>> > > >> >> > > > >>> > > > > me.
>> > > >> >> > > > >>> > > > > > >
>> > > >> >> > > > >>> > > > > > > Apart from the implementation I have also
>> > > added a
>> > > >> >> unit
>> > > >> >> > > > test.
>> > > >> >> > > > >>> > > > > > > It definitely needs some polishing and a
>> > double
>> > > >> >> check
>> > > >> >> > on
>> > > >> >> > > my
>> > > >> >> > > > >>> side
>> > > >> >> > > > >>> > to
>> > > >> >> > > > >>> > > > > make
>> > > >> >> > > > >>> > > > > > > the review smooth.
>> > > >> >> > > > >>> > > > > > > For the tests a local Pub/Sub [2] server
>> has
>> > to
>> > > >> be
>> > > >> >> > setup
>> > > >> >> > > > >>> however
>> > > >> >> > > > >>> > I
>> > > >> >> > > > >>> > > am
>> > > >> >> > > > >>> > > > > not
>> > > >> >> > > > >>> > > > > > > sure which is the best way to do so with
>> > > regards
>> > > >> to
>> > > >> >> > your
>> > > >> >> > > > >>> CI/CD.
>> > > >> >> > > > >>> > > > > > >
>> > > >> >> > > > >>> > > > > > > [1]
>> > > >> >> > > > >>> > >
>> > > >> >> >
>> https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
>> > > >> >> > > > >>> > > > > > > [2]
>> > > >> >> > > > >>>
>> > https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
>> > > >> >> > > > >>> > > > > > >
>> > > >> >> > > > >>> > > > > > > *Emmanouil Gkatziouras*
>> > > >> >> > > > >>> > > > > > > https://egkatzioura.com/ |
>> > > >> >> > > > >>> > > > > > >
>> > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > >> >> > > > >>> > > > > > > https://github.com/gkatzioura
>> > > >> >> > > > >>> > > > > > >
>> > > >> >> > > > >>> > > > > > >
>> > > >> >> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11, Saikat
>> Maitra <
>> > > >> >> > > > >>> > > [hidden email]
>> > > >> >> > > > >>> > > > >
>> > > >> >> > > > >>> > > > > > > wrote:
>> > > >> >> > > > >>> > > > > > >
>> > > >> >> > > > >>> > > > > > > > Hello Emmanouil,
>> > > >> >> > > > >>> > > > > > > >
>> > > >> >> > > > >>> > > > > > > > Can you please share more info about the
>> > > >> >> > > implementation?
>> > > >> >> > > > >>> > > > > > > >
>> > > >> >> > > > >>> > > > > > > > We have similar implementation for data
>> > > >> streamers
>> > > >> >> > with
>> > > >> >> > > > >>> Kafka
>> > > >> >> > > > >>> > and
>> > > >> >> > > > >>> > > > > > RocketMQ
>> > > >> >> > > > >>> > > > > > > >
>> > > >> >> > > > >>> > > > > > > >
>> > > >> >> > https://apacheignite-mix.readme.io/docs/kafka-streamer
>> > > >> >> > > > >>> > > > > > > >
>> > > >> >> > > > >>> > > > > > > >
>> > > >> >> > > > https://apacheignite-mix.readme.io/docs/rocketmq-streamer
>> > > >> >> > > > >>> > > > > > > >
>> > > >> >> > > > >>> > > > > > > > Please review and share if you have any
>> > > >> questions.
>> > > >> >> > > > >>> > > > > > > >
>> > > >> >> > > > >>> > > > > > > > Regards,
>> > > >> >> > > > >>> > > > > > > > Saikat
>> > > >> >> > > > >>> > > > > > > >
>> > > >> >> > > > >>> > > > > > > >
>> > > >> >> > > > >>> > > > > > > >
>> > > >> >> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56 PM
>> Emmanouil
>> > > >> >> > Gkatziouras <
>> > > >> >> > > > >>> > > > > > > > [hidden email]>
>> > > >> >> > > > >>> > > > > > > > wrote:
>> > > >> >> > > > >>> > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > Hi all,
>> > > >> >> > > > >>> > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > By mentioning Pub/Sub I mean Google
>> > Cloud's
>> > > >> >> Pub/Sub
>> > > >> >> > > [1]
>> > > >> >> > > > >>> > > > > > > > > Yes it is indeed cloud specific. It is
>> > the
>> > > >> >> > messaging
>> > > >> >> > > > >>> service
>> > > >> >> > > > >>> > > > > provided
>> > > >> >> > > > >>> > > > > > > by
>> > > >> >> > > > >>> > > > > > > > > GCP.
>> > > >> >> > > > >>> > > > > > > > > I have made the integration with it,
>> but
>> > I
>> > > >> >> haven't
>> > > >> >> > > > >>> published
>> > > >> >> > > > >>> > it
>> > > >> >> > > > >>> > > > yet
>> > > >> >> > > > >>> > > > > > (I
>> > > >> >> > > > >>> > > > > > > am
>> > > >> >> > > > >>> > > > > > > > > waiting for the discussion/ticket and
>> > > proceed
>> > > >> >> with
>> > > >> >> > > the
>> > > >> >> > > > >>> > > > > contribution).
>> > > >> >> > > > >>> > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > Kind regards,
>> > > >> >> > > > >>> > > > > > > > > Emmanouil
>> > > >> >> > > > >>> > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > [1]
>> > > >> >> https://cloud.google.com/pubsub/docs/overview
>> > > >> >> > > > >>> > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
>> > > >> >> > > > >>> > > > > > > > > https://egkatzioura.com/ |
>> > > >> >> > > > >>> > > > > > > > >
>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > >> >> > > > >>> > > > > > > > > https://github.com/gkatzioura
>> > > >> >> > > > >>> > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at 21:45, Denis
>> > Magda <
>> > > >> >> > > > >>> [hidden email]
>> > > >> >> > > > >>> > >
>> > > >> >> > > > >>> > > > > wrote:
>> > > >> >> > > > >>> > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > > Hello Emmanouil,
>> > > >> >> > > > >>> > > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > > Are you referring to any specific
>> > > >> >> implementation
>> > > >> >> > > > >>> tailored
>> > > >> >> > > > >>> > > for a
>> > > >> >> > > > >>> > > > > > > cloud?
>> > > >> >> > > > >>> > > > > > > > If
>> > > >> >> > > > >>> > > > > > > > > > I'm not mistaken you were mentioning
>> > > >> AWS/GCE.
>> > > >> >> Is
>> > > >> >> > > > there
>> > > >> >> > > > >>> any
>> > > >> >> > > > >>> > > > Github
>> > > >> >> > > > >>> > > > > > > > source
>> > > >> >> > > > >>> > > > > > > > > > code?
>> > > >> >> > > > >>> > > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > > Here is a list of existing
>> integrations
>> > > >> with
>> > > >> >> > > > streaming
>> > > >> >> > > > >>> and
>> > > >> >> > > > >>> > > > > pub/sub
>> > > >> >> > > > >>> > > > > > > > > > frameworks:
>> > > >> >> > > > >>> > > > > > > > > >
>> > > >> >> https://apacheignite-mix.readme.io/docs/overview
>> > > >> >> > > > >>> > > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > > with the source code in Ignite
>> master:
>> > > >> >> > > > >>> > > > > > > > > >
>> > > >> >> > > https://github.com/apache/ignite/tree/master/modules
>> > > >> >> > > > >>> > > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > > Check out those resources to see
>> what
>> > are
>> > > >> the
>> > > >> >> > most
>> > > >> >> > > > >>> suitable
>> > > >> >> > > > >>> > > > > Ignite
>> > > >> >> > > > >>> > > > > > > APIs
>> > > >> >> > > > >>> > > > > > > > > for
>> > > >> >> > > > >>> > > > > > > > > > such integrations.
>> > > >> >> > > > >>> > > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > > @Saikat Maitra <
>> > [hidden email]>
>> > > ,
>> > > >> >> > @Roman
>> > > >> >> > > <
>> > > >> >> > > > >>> > > > > > [hidden email]
>> > > >> >> > > > >>> > > > > > > >
>> > > >> >> > > > >>> > > > > > > > ,
>> > > >> >> > > > >>> > > > > > > > > as
>> > > >> >> > > > >>> > > > > > > > > > streaming technologies
>> > > >> >> contributors/maintainers,
>> > > >> >> > > > could
>> > > >> >> > > > >>> you
>> > > >> >> > > > >>> > > > please
>> > > >> >> > > > >>> > > > > > > join
>> > > >> >> > > > >>> > > > > > > > > this
>> > > >> >> > > > >>> > > > > > > > > > thread?
>> > > >> >> > > > >>> > > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > > -
>> > > >> >> > > > >>> > > > > > > > > > Denis
>> > > >> >> > > > >>> > > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > > On Thu, Oct 10, 2019 at 7:58 AM
>> > Emmanouil
>> > > >> >> > > > Gkatziouras <
>> > > >> >> > > > >>> > > > > > > > > > [hidden email]>
>> > > >> >> > > > >>> > > > > > > > > > wrote:
>> > > >> >> > > > >>> > > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > > > Hello everyone,
>> > > >> >> > > > >>> > > > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > > > I started using Ignite lately.
>> Part
>> > of
>> > > my
>> > > >> >> work
>> > > >> >> > > > >>> involves
>> > > >> >> > > > >>> > > > > Pub/Sub,
>> > > >> >> > > > >>> > > > > > > > thus I
>> > > >> >> > > > >>> > > > > > > > > > > created a pub/sub streamer.
>> > > >> >> > > > >>> > > > > > > > > > > Pub/Sub is pretty close to Kafka.
>> > > >> >> > > > >>> > > > > > > > > > > Here's how the streamer works.
>> Every
>> > > node
>> > > >> >> > creates
>> > > >> >> > > > an
>> > > >> >> > > > >>> > > instance
>> > > >> >> > > > >>> > > > > of
>> > > >> >> > > > >>> > > > > > a
>> > > >> >> > > > >>> > > > > > > > > > Pub/Sub
>> > > >> >> > > > >>> > > > > > > > > > > client.
>> > > >> >> > > > >>> > > > > > > > > > > The clients subscribe to the same
>> > topic
>> > > >> >> using
>> > > >> >> > the
>> > > >> >> > > > >>> same
>> > > >> >> > > > >>> > > > > > subscription
>> > > >> >> > > > >>> > > > > > > > > name.
>> > > >> >> > > > >>> > > > > > > > > > > A message is being sent and one of
>> > the
>> > > >> nodes
>> > > >> >> > will
>> > > >> >> > > > >>> receive
>> > > >> >> > > > >>> > > the
>> > > >> >> > > > >>> > > > > > > > message,
>> > > >> >> > > > >>> > > > > > > > > > > process it and then forward it to
>> > > Ignite
>> > > >> >> using
>> > > >> >> > > the
>> > > >> >> > > > >>> > > extractor
>> > > >> >> > > > >>> > > > > > > > provided.
>> > > >> >> > > > >>> > > > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > > > Any thoughts?
>> > > >> >> > > > >>> > > > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > > > *Emmanouil Gkatziouras*
>> > > >> >> > > > >>> > > > > > > > > > > https://egkatzioura.com/ |
>> > > >> >> > > > >>> > > > > > > > > > >
>> > > >> >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > >> >> > > > >>> > > > > > > > > > > https://github.com/gkatzioura
>> > > >> >> > > > >>> > > > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > >
>> > > >> >> > > > >>> > > > > > > >
>> > > >> >> > > > >>> > > > > > >
>> > > >> >> > > > >>> > > > > >
>> > > >> >> > > > >>> > > > >
>> > > >> >> > > > >>> > > >
>> > > >> >> > > > >>> > >
>> > > >> >> > > > >>> >
>> > > >> >> > > > >>>
>> > > >> >> > > > >>
>> > > >> >> > > >
>> > > >> >> > >
>> > > >> >> >
>> > > >> >>
>> > > >> >
>> > > >>
>> > > >
>> > >
>> >
>>
>

[Saikat Maitra]

Hi Emmanouil,

The latest build on teamcity has passed on Flink pull request.

https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead

you should be able to take the changes I made in my PR and run build on
your pull request.

Please let me know if you have any questions.

Regards,
Saikat

On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <[hidden email]>
wrote:

> Hi Emmanouil,
>
> I have fixed the Flink module testsuites and have setup new project in
> teamcity.
>
>
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
>
> I am looking into setting up the build.
>
> Regards,
> Saikat
>
> On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <[hidden email]>
> wrote:
>
>> Hi Emmanouil,
>>
>> Thank you for your email. Yes, the plan is once the Flink PR is merged it
>> will provide the Licence, parent POMs etc and you can rebase from master
>> branch and apply the changes on top of it.
>>
>> I am getting some issues with test failures in local with
>> GridTestProperties as the test.properties is not present in this new
>> project but should be available from dependencies.
>>
>> Once I address this issue, I will go ahead and merge the changes and then
>> we can take it from there.
>>
>> Regards,
>> Saikat
>>
>> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
>> [hidden email]> wrote:
>>
>>> Hi all,
>>>
>>> I made my first pull request [1]. Since this project is brand new (no
>>> parent poms, licensing), I reckoned it was best to use Saikat's branch on
>>> Flink.
>>> I suppose the Flink branch will be merged first. If not please give me
>>> guidelines on how I should proceed next.
>>>
>>> Kind regards
>>> Emmanouil
>>>
>>> [1] https://github.com/apache/ignite-extensions/pull/2
>>>
>>> *Emmanouil Gkatziouras*
>>> https://egkatzioura.com/ |
>>> https://www.linkedin.com/in/gkatziourasemmanouil/
>>> https://github.com/gkatzioura
>>>
>>>
>>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <[hidden email]> wrote:
>>>
>>> > Awesome, ping us whenever you're ready!
>>> >
>>> > -
>>> > Denis
>>> >
>>> >
>>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
>>> > [hidden email]>
>>> > wrote:
>>> >
>>> > > Hi all!
>>> > >
>>> > > I am sorry for being late on that. I was trying to refactor the test
>>> in
>>> > > order not to be in need of any external tools or spinning up a
>>> server.
>>> > > I did forked the new repo and indeed my changes there, so a pull
>>> request
>>> > is
>>> > > a matter of time!
>>> > >
>>> > > Kind regards
>>> > >
>>> > > *Emmanouil Gkatziouras*
>>> > > https://egkatzioura.com/ |
>>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
>>> > > https://github.com/gkatzioura
>>> > >
>>> > >
>>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <[hidden email]> wrote:
>>> > >
>>> > > > Hi Emmanouil,
>>> > > >
>>> > > > Do you have any questions or need any support from the community?
>>> > > >
>>> > > > -
>>> > > > Denis
>>> > > >
>>> > > >
>>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
>>> [hidden email]
>>> > >
>>> > > > wrote:
>>> > > >
>>> > > >> Hi Emmanouil,
>>> > > >>
>>> > > >> Can you please take a looks at dev utils, if this is something
>>> you are
>>> > > >> looking for
>>> > > >> https://github.com/apache/ignite/tree/master/modules/dev-utils?
>>> > > >>
>>> > > >> IMO, if you can release Pub/Sub server in maven and then use it as
>>> > > >> dependency, that would be great.
>>> > > >>
>>> > > >> Regards,
>>> > > >> Saikat
>>> > > >>
>>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
>>> > [hidden email]>
>>> > > >> wrote:
>>> > > >>
>>> > > >> > Hi Emmanouil,
>>> > > >> >
>>> > > >> > The master branch has been initialized. I have raised a PR to
>>> > migrate
>>> > > >> > Flink module to ignite-extensions.
>>> > > >> >
>>> > > >> > PR https://github.com/apache/ignite-extensions/pull/1
>>> > > >> > Jira : https://issues.apache.org/jira/browse/IGNITE-12356
>>> > > >> >
>>> > > >> > Regards,
>>> > > >> > Saikat
>>> > > >> >
>>> > > >> >
>>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
>>> > > >> > [hidden email]> wrote:
>>> > > >> >
>>> > > >> >> Hi all!
>>> > > >> >>
>>> > > >> >> This is an update. The integration is tested with mocks based
>>> on
>>> > the
>>> > > >> >> original Ignite repo [1].
>>> > > >> >> I am currently waiting for the first commits on the new
>>> integration
>>> > > >> >> project
>>> > > >> >> to take place in order to create my pull request accordingly
>>> to the
>>> > > >> >> initial
>>> > > >> >> commits.
>>> > > >> >> In order to simulate the Pub/Sub server in a test I used a test
>>> > > server
>>> > > >> >> implementation [2]. Unfortunately they don't have this util
>>> > uploaded
>>> > > >> in a
>>> > > >> >> maven repo, thus it needs to be hosted on a maven repo (Apache
>>> > > License
>>> > > >> >> 2.0). Is there a repo for utilities that Ignite uses for the
>>> > > >> test/build?
>>> > > >> >> I have also created a ticket [3]
>>> > > >> >>
>>> > > >> >> Kind regards
>>> > > >> >>
>>> > > >> >> [1]
>>> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
>>> > > >> >> [2]
>>> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
>>> > > >> >> [3] https://issues.apache.org/jira/browse/IGNITE-12262
>>> > > >> >>
>>> > > >> >> *Emmanouil Gkatziouras*
>>> > > >> >> https://egkatzioura.com/ |
>>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
>>> > > >> >> https://github.com/gkatzioura
>>> > > >> >>
>>> > > >> >>
>>> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
>>> > [hidden email]
>>> > > >
>>> > > >> >> wrote:
>>> > > >> >>
>>> > > >> >> > Hello Ilya,
>>> > > >> >> >
>>> > > >> >> > Thank you for your email. Yes, before we remove any
>>> integration
>>> > we
>>> > > >> will
>>> > > >> >> > move them to another repository and release.
>>> > > >> >> >
>>> > > >> >> > We are thinking we will start with Pub/Sub Streamer
>>> > implementation
>>> > > >> and
>>> > > >> >> > continue to move other integration as mentioned in the
>>> following
>>> > > doc.
>>> > > >> >> >
>>> > > >> >> >
>>> > > >> >> >
>>> > > >> >>
>>> > > >>
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>>> > > >> >> >
>>> > > >> >> > Regards,
>>> > > >> >> > Saikat
>>> > > >> >> >
>>> > > >> >> >
>>> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
>>> > > >> >> > [hidden email]>
>>> > > >> >> > wrote:
>>> > > >> >> >
>>> > > >> >> > > Hello!
>>> > > >> >> > >
>>> > > >> >> > > My take on this: we want to offload some of integrations,
>>> but
>>> > > >> please
>>> > > >> >> > commit
>>> > > >> >> > > and release them to another project first, and only then we
>>> > will
>>> > > >> >> decide
>>> > > >> >> > to
>>> > > >> >> > > remove them from our repository.
>>> > > >> >> > >
>>> > > >> >> > > Please also make sure they are available with same artifact
>>> > name
>>> > > >> and
>>> > > >> >> API.
>>> > > >> >> > > If this has to be changed, let's wait for AI 3.0
>>> > > >> >> > >
>>> > > >> >> > > Regards,
>>> > > >> >> > > --
>>> > > >> >> > > Ilya Kasnacheev
>>> > > >> >> > >
>>> > > >> >> > >
>>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <
>>> > > >> [hidden email]>:
>>> > > >> >> > >
>>> > > >> >> > > > Hello Emmanouil,
>>> > > >> >> > > >
>>> > > >> >> > > > As discussed earlier, I discussed with Apache Bahir
>>> community
>>> > > and
>>> > > >> >> they
>>> > > >> >> > > are
>>> > > >> >> > > > interested to have Apache Ignite extensions as part of
>>> Apache
>>> > > >> Bahir
>>> > > >> >> > > > project.
>>> > > >> >> > > >
>>> > > >> >> > > > Can you please share Pub/Sub streamer implementation
>>> details
>>> > > with
>>> > > >> >> > Apache
>>> > > >> >> > > > Bahir community and request for feedback.
>>> > > >> >> > > >
>>> > > >> >> > > > https://bahir.apache.org/
>>> > > >> >> > > >
>>> > > >> >> > > > I have also requested for contributor access in Jira for
>>> > Apache
>>> > > >> >> Bahir
>>> > > >> >> > > > project so that I can create issues and assign to myself.
>>> > > >> >> > > >
>>> > > >> >> > > > I can help with code reviews as well.
>>> > > >> >> > > >
>>> > > >> >> > > > Here is the email thread for reference
>>> > > >> >> > > >
>>> > > https://www.mail-archive.com/dev@.../msg02703.html
>>> > > >> >> > > >
>>> > > >> >> > > > Regards,
>>> > > >> >> > > > Saikat
>>> > > >> >> > > >
>>> > > >> >> > > >
>>> > > >> >> > > >
>>> > > >> >> > > >
>>> > > >> >> > > >
>>> > > >> >> > > >
>>> > > >> >> > > >
>>> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
>>> > > >> >> [hidden email]
>>> > > >> >> > >
>>> > > >> >> > > > wrote:
>>> > > >> >> > > >
>>> > > >> >> > > > > Hello,
>>> > > >> >> > > > >
>>> > > >> >> > > > > As discussed I have created following discussion
>>> threads on
>>> > > our
>>> > > >> >> > > migration
>>> > > >> >> > > > > proposals for Apache Ignite extensions:
>>> > > >> >> > > > >
>>> > > >> >> > > > >
>>> > > >> >> > > > >
>>> > > >> >> > > >
>>> > > >> >> > >
>>> > > >> >> >
>>> > > >> >>
>>> > > >>
>>> > >
>>> >
>>> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
>>> > > >> >> > > > >
>>> > > >> >> > > > >
>>> > > >> https://www.mail-archive.com/dev@.../msg02703.html
>>> > > >> >> > > > >
>>> > > >> >> > > > > I will share update as I hear more information.
>>> > > >> >> > > > >
>>> > > >> >> > > > > Regards,
>>> > > >> >> > > > > Saikat
>>> > > >> >> > > > >
>>> > > >> >> > > > >
>>> > > >> >> > > > >
>>> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
>>> > > >> >> > [hidden email]
>>> > > >> >> > > >
>>> > > >> >> > > > > wrote:
>>> > > >> >> > > > >
>>> > > >> >> > > > >> Hello Denis,
>>> > > >> >> > > > >>
>>> > > >> >> > > > >> Sure, sounds good. I will create a separate discussion
>>> > > thread
>>> > > >> to
>>> > > >> >> get
>>> > > >> >> > > > >> community feedback on whether we
>>> > > >> >> > > > >> should join the Bahir or kick-off "Ignite Extensions"
>>> > > project.
>>> > > >> >> > > > >>
>>> > > >> >> > > > >> Regards,
>>> > > >> >> > > > >> Saikat
>>> > > >> >> > > > >>
>>> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <
>>> > > >> [hidden email]>
>>> > > >> >> > > wrote:
>>> > > >> >> > > > >>
>>> > > >> >> > > > >>> Folks,
>>> > > >> >> > > > >>>
>>> > > >> >> > > > >>> The concept of Apache Bahir is precisely what we
>>> need!
>>> > > >> Saikat,
>>> > > >> >> > thanks
>>> > > >> >> > > > for
>>> > > >> >> > > > >>> doing the research. Why I believe the * idea* fits us
>>> > well:
>>> > > >> >> > > > >>>
>>> > > >> >> > > > >>>    - All integrations can be stored in separate
>>> Github
>>> > > >> >> repositories
>>> > > >> >> > > and
>>> > > >> >> > > > >>>    have their dev lifecycles. We've not obliged to
>>> couple
>>> > > all
>>> > > >> >> the
>>> > > >> >> > > > >>> integrations
>>> > > >> >> > > > >>>    in a single repo. For instance, Spark can be
>>> located
>>> > in
>>> > > a
>>> > > >> >> > > dedicated
>>> > > >> >> > > > >>> repo
>>> > > >> >> > > > >>>    while streaming integrations might be in a single
>>> one.
>>> > > >> It's
>>> > > >> >> up
>>> > > >> >> > to
>>> > > >> >> > > > us.
>>> > > >> >> > > > >>>    - All the repositories and integrations belong to
>>> ASF.
>>> > > >> We're
>>> > > >> >> not
>>> > > >> >> > > > >>> dumping
>>> > > >> >> > > > >>>    them on Github but rather keep supporting and
>>> > developing
>>> > > >> in
>>> > > >> >> > > > >>> accordance with
>>> > > >> >> > > > >>>    ASF vision and practices.
>>> > > >> >> > > > >>>    - There is a way to reward contributors via
>>> > > committership
>>> > > >> and
>>> > > >> >> > PMC
>>> > > >> >> > > > >>>    membership. You won't get this if a project is
>>> just
>>> > one
>>> > > of
>>> > > >> >> the
>>> > > >> >> > > > >>> millions of
>>> > > >> >> > > > >>>    Github projects.
>>> > > >> >> > > > >>>
>>> > > >> >> > > > >>> Saikat, as Ignite PMC member, could you please
>>> kick-off a
>>> > > >> >> separate
>>> > > >> >> > > > >>> dev-list
>>> > > >> >> > > > >>> discussion to involve more community members and
>>> > referring
>>> > > to
>>> > > >> >> this
>>> > > >> >> > > > >>> thread?
>>> > > >> >> > > > >>> I think the primary question the community needs to
>>> > answer
>>> > > >> >> whether
>>> > > >> >> > we
>>> > > >> >> > > > >>> should join the Bahir or kick-off "Ignite Extensions"
>>> > > >> project?
>>> > > >> >> > > > >>>
>>> > > >> >> > > > >>> -
>>> > > >> >> > > > >>> Denis
>>> > > >> >> > > > >>>
>>> > > >> >> > > > >>>
>>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev <
>>> > > >> >> > > > [hidden email]>
>>> > > >> >> > > > >>> wrote:
>>> > > >> >> > > > >>>
>>> > > >> >> > > > >>> > Maybe we could move all our Streaming Integrations
>>> > there,
>>> > > >> but
>>> > > >> >> > what
>>> > > >> >> > > is
>>> > > >> >> > > > >>> about
>>> > > >> >> > > > >>> > maintaining and committer permissions to the new
>>> > > >> repositories?
>>> > > >> >> > > > >>> > I see the list of the committers and PMC members
>>> there
>>> > > >> >> > > > >>> > https://bahir.apache.org/community-members/
>>> > > >> >> > > > >>> >
>>> > > >> >> > > > >>> > Could somebody from Ignite community be added to
>>> this
>>> > > list
>>> > > >> as
>>> > > >> >> a
>>> > > >> >> > > > >>> > PMC/committer to maintain Ignite integrations?
>>> > > >> >> > > > >>> > If yes, I'd happy to join this project and
>>> collaborate
>>> > > with
>>> > > >> >> these
>>> > > >> >> > > > guys
>>> > > >> >> > > > >>> > together
>>> > > >> >> > > > >>> > If no, and we should start from the zero level with
>>> > > >> external
>>> > > >> >> PRs
>>> > > >> >> > -
>>> > > >> >> > > > >>> hmmm,
>>> > > >> >> > > > >>> > it's better to have our own external repository
>>> with
>>> > > >> >> ApacheBahirr
>>> > > >> >> > > > >>> ideology.
>>> > > >> >> > > > >>> >
>>> > > >> >> > > > >>> > Also, I agree, that all connectors could be moved
>>> there
>>> > > (in
>>> > > >> >> > > > >>> ApacheBahirr
>>> > > >> >> > > > >>> > like repository), but not all modules from the page
>>> > > >> >> > > > >>> >
>>> > > >> >> > > > >>> >
>>> > > >> >> > > > >>>
>>> > > >> >> > > >
>>> > > >> >> > >
>>> > > >> >> >
>>> > > >> >>
>>> > > >>
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>>> > > >> >> > > > >>> > because
>>> > > >> >> > > > >>> > they use different parts of core modules and could
>>> be
>>> > > >> >> developed
>>> > > >> >> > > with
>>> > > >> >> > > > >>> > different velocity.
>>> > > >> >> > > > >>> >
>>> > > >> >> > > > >>> > In reality, before creation of new repositories we
>>> need
>>> > > >> wide
>>> > > >> >> > > > discussion
>>> > > >> >> > > > >>> > based on the document mentioned above.
>>> > > >> >> > > > >>> >
>>> > > >> >> > > > >>> > P.S. Of course, we could experiment in the next
>>> release
>>> > > 2.8
>>> > > >> >> with
>>> > > >> >> > > one
>>> > > >> >> > > > >>> or two
>>> > > >> >> > > > >>> > integrations like twitter or ZeroMQ
>>> > > >> >> > > > >>> > Also, I'd like Denis Magda idea for the separate
>>> > > >> repositories
>>> > > >> >> in
>>> > > >> >> > > > Apache
>>> > > >> >> > > > >>> > Github to maintain them by anybody.
>>> > > >> >> > > > >>> >
>>> > > >> >> > > > >>> >
>>> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra <
>>> > > >> >> > > [hidden email]
>>> > > >> >> > > > >:
>>> > > >> >> > > > >>> >
>>> > > >> >> > > > >>> > > Hi Denis, Emmanouil
>>> > > >> >> > > > >>> > >
>>> > > >> >> > > > >>> > > Thank you for your email. I think creating a
>>> separate
>>> > > >> >> > integration
>>> > > >> >> > > > >>> project
>>> > > >> >> > > > >>> > > in github and also proposing it as an apache
>>> > incubator
>>> > > >> >> project
>>> > > >> >> > > will
>>> > > >> >> > > > >>> be a
>>> > > >> >> > > > >>> > > good move. The other separate integration modules
>>> > > >> >> > > > >>> > >
>>> > > >> >> > > > >>> > >
>>> > > >> >> > > > >>> >
>>> > > >> >> > > > >>>
>>> > > >> >> > > >
>>> > > >> >> > >
>>> > > >> >> >
>>> > > >> >>
>>> > > >>
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>>> > > >> >> > > > >>> > > can
>>> > > >> >> > > > >>> > > be moved to this new apache incubator project.
>>> > > >> >> > > > >>> > >
>>> > > >> >> > > > >>> > > There are similar integration available for
>>> Flink and
>>> > > >> Spark
>>> > > >> >> in
>>> > > >> >> > > > Apache
>>> > > >> >> > > > >>> > Bahir
>>> > > >> >> > > > >>> > > https://bahir.apache.org/
>>> > > >> >> > > > >>> > >
>>> > > >> >> > > > >>> > > Do you think we should reach out to Apache Bahir
>>> > > >> community
>>> > > >> >> > with a
>>> > > >> >> > > > >>> > proposal
>>> > > >> >> > > > >>> > > or we should plan to create a separate Apache
>>> > Incubator
>>> > > >> >> > project?
>>> > > >> >> > > > >>> > >
>>> > > >> >> > > > >>> > > Regards,
>>> > > >> >> > > > >>> > > Saikat
>>> > > >> >> > > > >>> > >
>>> > > >> >> > > > >>> > >
>>> > > >> >> > > > >>> > >
>>> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil
>>> > Gkatziouras <
>>> > > >> >> > > > >>> > > [hidden email]>
>>> > > >> >> > > > >>> > > wrote:
>>> > > >> >> > > > >>> > >
>>> > > >> >> > > > >>> > > > Hi all!
>>> > > >> >> > > > >>> > > >
>>> > > >> >> > > > >>> > > > Based on the discussions most probably this is
>>> > going
>>> > > to
>>> > > >> >> be on
>>> > > >> >> > > > >>> another
>>> > > >> >> > > > >>> > > > GitHub repo. Therefore I will prepare a
>>> standalone
>>> > > >> project
>>> > > >> >> > with
>>> > > >> >> > > > the
>>> > > >> >> > > > >>> > > Pub/Sub
>>> > > >> >> > > > >>> > > > module and once the repository is created a
>>> pull
>>> > > >> request
>>> > > >> >> > shall
>>> > > >> >> > > be
>>> > > >> >> > > > >>> > issued.
>>> > > >> >> > > > >>> > > > If there is anything else I can do in order to
>>> > > >> facilitate
>>> > > >> >> > this
>>> > > >> >> > > > >>> please
>>> > > >> >> > > > >>> > let
>>> > > >> >> > > > >>> > > > me know.
>>> > > >> >> > > > >>> > > >
>>> > > >> >> > > > >>> > > > If a ticket is created for this issue, I shall
>>> be
>>> > > able
>>> > > >> to
>>> > > >> >> > > create
>>> > > >> >> > > > a
>>> > > >> >> > > > >>> pull
>>> > > >> >> > > > >>> > > > request and thus a build will take place on
>>> > ignite's
>>> > > >> CI.
>>> > > >> >> > > > >>> > > > If everyone is aligned with adding a Pub/Sub
>>> > > >> >> implementation
>>> > > >> >> > > may I
>>> > > >> >> > > > >>> > create
>>> > > >> >> > > > >>> > > a
>>> > > >> >> > > > >>> > > > ticket on JIRA?
>>> > > >> >> > > > >>> > > >
>>> > > >> >> > > > >>> > > > My next question has to do on how Ignite
>>> configures
>>> > > the
>>> > > >> >> build
>>> > > >> >> > > > jobs
>>> > > >> >> > > > >>> in
>>> > > >> >> > > > >>> > > Team
>>> > > >> >> > > > >>> > > > City. The unit tests for my implementation
>>> need a
>>> > > local
>>> > > >> >> > Pub/Sub
>>> > > >> >> > > > >>> server
>>> > > >> >> > > > >>> > up
>>> > > >> >> > > > >>> > > > and running (they provide one for testing
>>> > purposes).
>>> > > >> Will
>>> > > >> >> > this
>>> > > >> >> > > > take
>>> > > >> >> > > > >>> > > effect
>>> > > >> >> > > > >>> > > > in a form of a build script inside my
>>> > implementation
>>> > > or
>>> > > >> >> is it
>>> > > >> >> > > > >>> something
>>> > > >> >> > > > >>> > > > that should be configured on Team City?
>>> > > >> >> > > > >>> > > >
>>> > > >> >> > > > >>> > > > Kind regards,
>>> > > >> >> > > > >>> > > > Emmanouil
>>> > > >> >> > > > >>> > > >
>>> > > >> >> > > > >>> > > >
>>> > > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
>>> > > >> >> > > > >>> > > > https://egkatzioura.com/ |
>>> > > >> >> > > > >>> > > >
>>> https://www.linkedin.com/in/gkatziourasemmanouil/
>>> > > >> >> > > > >>> > > > https://github.com/gkatzioura
>>> > > >> >> > > > >>> > > >
>>> > > >> >> > > > >>> > > >
>>> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis Magda <
>>> > > >> >> [hidden email]
>>> > > >> >> > >
>>> > > >> >> > > > >>> wrote:
>>> > > >> >> > > > >>> > > >
>>> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
>>> > > >> >> > > > >>> > > > >
>>> > > >> >> > > > >>> > > > > After contemplating on this, let me propose
>>> > another
>>> > > >> way
>>> > > >> >> > with
>>> > > >> >> > > > this
>>> > > >> >> > > > >>> > > > > particular integration that is aligned with
>>> our
>>> > > >> >> > > modularization
>>> > > >> >> > > > >>> > > endeavor.
>>> > > >> >> > > > >>> > > > > The modularization [1] defines the Ignite
>>> core,
>>> > > >> Ignite
>>> > > >> >> > > modules,
>>> > > >> >> > > > >>> and
>>> > > >> >> > > > >>> > > > > independent integrations. The first two are
>>> to be
>>> > > >> >> located
>>> > > >> >> > in
>>> > > >> >> > > > >>> Ignite
>>> > > >> >> > > > >>> > > > > repositories and to be tested/updated by the
>>> > > >> community
>>> > > >> >> for
>>> > > >> >> > > > every
>>> > > >> >> > > > >>> > > release.
>>> > > >> >> > > > >>> > > > > The modules are a subset of existing
>>> integrations
>>> > > >> that
>>> > > >> >> are
>>> > > >> >> > > > >>> adopted
>>> > > >> >> > > > >>> > > widely
>>> > > >> >> > > > >>> > > > > and require community attention.
>>> > > >> >> > > > >>> > > > >
>>> > > >> >> > > > >>> > > > > As for the independent integrations [1],
>>> those
>>> > are
>>> > > >> >> supposed
>>> > > >> >> > > to
>>> > > >> >> > > > be
>>> > > >> >> > > > >>> > > located
>>> > > >> >> > > > >>> > > > > in a separate GitHub repo. I would advise us
>>> to
>>> > > start
>>> > > >> >> > > > >>> implementing
>>> > > >> >> > > > >>> > this
>>> > > >> >> > > > >>> > > > > concept with the Pub/Sub integration. Even
>>> though
>>> > > it
>>> > > >> >> will
>>> > > >> >> > be
>>> > > >> >> > > > >>> located
>>> > > >> >> > > > >>> > > in a
>>> > > >> >> > > > >>> > > > > separate Github repo, it will be listed among
>>> > > >> officially
>>> > > >> >> > > > >>> available
>>> > > >> >> > > > >>> > > > > integrations on the Ignite website featuring
>>> > > >> Emmanouil
>>> > > >> >> as a
>>> > > >> >> > > > >>> > > contributor.
>>> > > >> >> > > > >>> > > > >
>>> > > >> >> > > > >>> > > > > What do you think?
>>> > > >> >> > > > >>> > > > >
>>> > > >> >> > > > >>> > > > >
>>> > > >> >> > > > >>> > > > > [1]
>>> > > >> >> > > > >>> > > > >
>>> > > >> >> > > > >>> > > > >
>>> > > >> >> > > > >>> > > >
>>> > > >> >> > > > >>> > >
>>> > > >> >> > > > >>> >
>>> > > >> >> > > > >>>
>>> > > >> >> > > >
>>> > > >> >> > >
>>> > > >> >> >
>>> > > >> >>
>>> > > >>
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>>> > > >> >> > > > >>> > > > >
>>> > > >> >> > > > >>> > > > >
>>> > > >> >> > > > >>> > > > >
>>> > > >> >> > > > >>> > > > > -
>>> > > >> >> > > > >>> > > > > Denis
>>> > > >> >> > > > >>> > > > >
>>> > > >> >> > > > >>> > > > >
>>> > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat
>>> Maitra <
>>> > > >> >> > > > >>> > [hidden email]
>>> > > >> >> > > > >>> > > >
>>> > > >> >> > > > >>> > > > > wrote:
>>> > > >> >> > > > >>> > > > >
>>> > > >> >> > > > >>> > > > > > Hi Emmanouil,
>>> > > >> >> > > > >>> > > > > >
>>> > > >> >> > > > >>> > > > > > The changes looks good to me. I wanted to
>>> check
>>> > > if
>>> > > >> you
>>> > > >> >> > are
>>> > > >> >> > > > also
>>> > > >> >> > > > >>> > able
>>> > > >> >> > > > >>> > > to
>>> > > >> >> > > > >>> > > > > > validate that the once data is added to
>>> Ignite
>>> > > >> cluster
>>> > > >> >> > you
>>> > > >> >> > > > are
>>> > > >> >> > > > >>> also
>>> > > >> >> > > > >>> > > > able
>>> > > >> >> > > > >>> > > > > to
>>> > > >> >> > > > >>> > > > > > access it using another Ignite client or
>>> using
>>> > > rest
>>> > > >> >> > > > endpoints.
>>> > > >> >> > > > >>> > > > > >
>>> > > >> >> > > > >>> > > > > > We use Teamcity for CI process, the
>>> details are
>>> > > as
>>> > > >> >> > follows
>>> > > >> >> > > > >>> > > > > >
>>> > > >> >> > > > >>> > > > > > Trigger validation of those test suites
>>> that
>>> > have
>>> > > >> been
>>> > > >> >> > > > >>> affected by
>>> > > >> >> > > > >>> > > your
>>> > > >> >> > > > >>> > > > > > changes on TeamCity <
>>> > > http://ci.ignite.apache.org/>:
>>> > > >> <
>>> > > >> >> > > > >>> > > > > http://204.14.53.153/
>>> > > >> >> > > > >>> > > > > > >
>>> > > >> >> > > > >>> > > > > >
>>> > > >> >> > > > >>> > > > > >    - Locate a test suite you have to check,
>>> > press
>>> > > >> >> button
>>> > > >> >> > > > named
>>> > > >> >> > > > >>> > "..."
>>> > > >> >> > > > >>> > > > that
>>> > > >> >> > > > >>> > > > > >    is located on the left of "Run" button.
>>> "Run
>>> > > >> custom
>>> > > >> >> > > build"
>>> > > >> >> > > > >>> > dialog
>>> > > >> >> > > > >>> > > > will
>>> > > >> >> > > > >>> > > > > >    appear;
>>> > > >> >> > > > >>> > > > > >    - Go to "Changes" tab and choose
>>> > > >> >> > > > >>> > "pull/<pull-request-number>/head"
>>> > > >> >> > > > >>> > > > in
>>> > > >> >> > > > >>> > > > > >    "Build branch" dropdown list;
>>> > > >> >> > > > >>> > > > > >    - Press "Run build" button and monitor
>>> tests
>>> > > >> >> results.
>>> > > >> >> > > > >>> > > > > >
>>> > > >> >> > > > >>> > > > > >
>>> > > >> >> > > > >>> > > > > >
>>> > > >> >> > > > >>> > > > > > @Roman [hidden email] - can you also
>>> please
>>> > > >> take a
>>> > > >> >> > > look?
>>> > > >> >> > > > >>> > > > > >
>>> > > >> >> > > > >>> > > > > > Regards,
>>> > > >> >> > > > >>> > > > > > Saikat
>>> > > >> >> > > > >>> > > > > >
>>> > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM Emmanouil
>>> > > >> Gkatziouras
>>> > > >> >> <
>>> > > >> >> > > > >>> > > > > > [hidden email]>
>>> > > >> >> > > > >>> > > > > > wrote:
>>> > > >> >> > > > >>> > > > > >
>>> > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
>>> > > >> >> > > > >>> > > > > > >
>>> > > >> >> > > > >>> > > > > > > I have my implementation on an Ignite
>>> fork on
>>> > > my
>>> > > >> >> > personal
>>> > > >> >> > > > >>> GitHub
>>> > > >> >> > > > >>> > > > > account
>>> > > >> >> > > > >>> > > > > > > [1]
>>> > > >> >> > > > >>> > > > > > > It is based on the Kafka implementation
>>> which
>>> > > is
>>> > > >> of
>>> > > >> >> a
>>> > > >> >> > > > polling
>>> > > >> >> > > > >>> > > nature.
>>> > > >> >> > > > >>> > > > > > > GCP does not provide a service like
>>> Kafka.
>>> > > >> Instead
>>> > > >> >> it
>>> > > >> >> > > > >>> provides us
>>> > > >> >> > > > >>> > > > with
>>> > > >> >> > > > >>> > > > > > > Pub/Sub which is just like Kafka.
>>> > > >> >> > > > >>> > > > > > > Therefore using the Kafka streamer as a
>>> > > guideline
>>> > > >> >> made
>>> > > >> >> > > more
>>> > > >> >> > > > >>> sense
>>> > > >> >> > > > >>> > > to
>>> > > >> >> > > > >>> > > > > me.
>>> > > >> >> > > > >>> > > > > > >
>>> > > >> >> > > > >>> > > > > > > Apart from the implementation I have also
>>> > > added a
>>> > > >> >> unit
>>> > > >> >> > > > test.
>>> > > >> >> > > > >>> > > > > > > It definitely needs some polishing and a
>>> > double
>>> > > >> >> check
>>> > > >> >> > on
>>> > > >> >> > > my
>>> > > >> >> > > > >>> side
>>> > > >> >> > > > >>> > to
>>> > > >> >> > > > >>> > > > > make
>>> > > >> >> > > > >>> > > > > > > the review smooth.
>>> > > >> >> > > > >>> > > > > > > For the tests a local Pub/Sub [2] server
>>> has
>>> > to
>>> > > >> be
>>> > > >> >> > setup
>>> > > >> >> > > > >>> however
>>> > > >> >> > > > >>> > I
>>> > > >> >> > > > >>> > > am
>>> > > >> >> > > > >>> > > > > not
>>> > > >> >> > > > >>> > > > > > > sure which is the best way to do so with
>>> > > regards
>>> > > >> to
>>> > > >> >> > your
>>> > > >> >> > > > >>> CI/CD.
>>> > > >> >> > > > >>> > > > > > >
>>> > > >> >> > > > >>> > > > > > > [1]
>>> > > >> >> > > > >>> > >
>>> > > >> >> >
>>> https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
>>> > > >> >> > > > >>> > > > > > > [2]
>>> > > >> >> > > > >>>
>>> > https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
>>> > > >> >> > > > >>> > > > > > >
>>> > > >> >> > > > >>> > > > > > > *Emmanouil Gkatziouras*
>>> > > >> >> > > > >>> > > > > > > https://egkatzioura.com/ |
>>> > > >> >> > > > >>> > > > > > >
>>> > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
>>> > > >> >> > > > >>> > > > > > > https://github.com/gkatzioura
>>> > > >> >> > > > >>> > > > > > >
>>> > > >> >> > > > >>> > > > > > >
>>> > > >> >> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11, Saikat
>>> Maitra <
>>> > > >> >> > > > >>> > > [hidden email]
>>> > > >> >> > > > >>> > > > >
>>> > > >> >> > > > >>> > > > > > > wrote:
>>> > > >> >> > > > >>> > > > > > >
>>> > > >> >> > > > >>> > > > > > > > Hello Emmanouil,
>>> > > >> >> > > > >>> > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > Can you please share more info about
>>> the
>>> > > >> >> > > implementation?
>>> > > >> >> > > > >>> > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > We have similar implementation for data
>>> > > >> streamers
>>> > > >> >> > with
>>> > > >> >> > > > >>> Kafka
>>> > > >> >> > > > >>> > and
>>> > > >> >> > > > >>> > > > > > RocketMQ
>>> > > >> >> > > > >>> > > > > > > >
>>> > > >> >> > > > >>> > > > > > > >
>>> > > >> >> > https://apacheignite-mix.readme.io/docs/kafka-streamer
>>> > > >> >> > > > >>> > > > > > > >
>>> > > >> >> > > > >>> > > > > > > >
>>> > > >> >> > > >
>>> https://apacheignite-mix.readme.io/docs/rocketmq-streamer
>>> > > >> >> > > > >>> > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > Please review and share if you have any
>>> > > >> questions.
>>> > > >> >> > > > >>> > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > Regards,
>>> > > >> >> > > > >>> > > > > > > > Saikat
>>> > > >> >> > > > >>> > > > > > > >
>>> > > >> >> > > > >>> > > > > > > >
>>> > > >> >> > > > >>> > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56 PM
>>> Emmanouil
>>> > > >> >> > Gkatziouras <
>>> > > >> >> > > > >>> > > > > > > > [hidden email]>
>>> > > >> >> > > > >>> > > > > > > > wrote:
>>> > > >> >> > > > >>> > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > Hi all,
>>> > > >> >> > > > >>> > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > By mentioning Pub/Sub I mean Google
>>> > Cloud's
>>> > > >> >> Pub/Sub
>>> > > >> >> > > [1]
>>> > > >> >> > > > >>> > > > > > > > > Yes it is indeed cloud specific. It
>>> is
>>> > the
>>> > > >> >> > messaging
>>> > > >> >> > > > >>> service
>>> > > >> >> > > > >>> > > > > provided
>>> > > >> >> > > > >>> > > > > > > by
>>> > > >> >> > > > >>> > > > > > > > > GCP.
>>> > > >> >> > > > >>> > > > > > > > > I have made the integration with it,
>>> but
>>> > I
>>> > > >> >> haven't
>>> > > >> >> > > > >>> published
>>> > > >> >> > > > >>> > it
>>> > > >> >> > > > >>> > > > yet
>>> > > >> >> > > > >>> > > > > > (I
>>> > > >> >> > > > >>> > > > > > > am
>>> > > >> >> > > > >>> > > > > > > > > waiting for the discussion/ticket and
>>> > > proceed
>>> > > >> >> with
>>> > > >> >> > > the
>>> > > >> >> > > > >>> > > > > contribution).
>>> > > >> >> > > > >>> > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > Kind regards,
>>> > > >> >> > > > >>> > > > > > > > > Emmanouil
>>> > > >> >> > > > >>> > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > [1]
>>> > > >> >> https://cloud.google.com/pubsub/docs/overview
>>> > > >> >> > > > >>> > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
>>> > > >> >> > > > >>> > > > > > > > > https://egkatzioura.com/ |
>>> > > >> >> > > > >>> > > > > > > > >
>>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
>>> > > >> >> > > > >>> > > > > > > > > https://github.com/gkatzioura
>>> > > >> >> > > > >>> > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at 21:45, Denis
>>> > Magda <
>>> > > >> >> > > > >>> [hidden email]
>>> > > >> >> > > > >>> > >
>>> > > >> >> > > > >>> > > > > wrote:
>>> > > >> >> > > > >>> > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > > Hello Emmanouil,
>>> > > >> >> > > > >>> > > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > > Are you referring to any specific
>>> > > >> >> implementation
>>> > > >> >> > > > >>> tailored
>>> > > >> >> > > > >>> > > for a
>>> > > >> >> > > > >>> > > > > > > cloud?
>>> > > >> >> > > > >>> > > > > > > > If
>>> > > >> >> > > > >>> > > > > > > > > > I'm not mistaken you were
>>> mentioning
>>> > > >> AWS/GCE.
>>> > > >> >> Is
>>> > > >> >> > > > there
>>> > > >> >> > > > >>> any
>>> > > >> >> > > > >>> > > > Github
>>> > > >> >> > > > >>> > > > > > > > source
>>> > > >> >> > > > >>> > > > > > > > > > code?
>>> > > >> >> > > > >>> > > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > > Here is a list of existing
>>> integrations
>>> > > >> with
>>> > > >> >> > > > streaming
>>> > > >> >> > > > >>> and
>>> > > >> >> > > > >>> > > > > pub/sub
>>> > > >> >> > > > >>> > > > > > > > > > frameworks:
>>> > > >> >> > > > >>> > > > > > > > > >
>>> > > >> >> https://apacheignite-mix.readme.io/docs/overview
>>> > > >> >> > > > >>> > > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > > with the source code in Ignite
>>> master:
>>> > > >> >> > > > >>> > > > > > > > > >
>>> > > >> >> > > https://github.com/apache/ignite/tree/master/modules
>>> > > >> >> > > > >>> > > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > > Check out those resources to see
>>> what
>>> > are
>>> > > >> the
>>> > > >> >> > most
>>> > > >> >> > > > >>> suitable
>>> > > >> >> > > > >>> > > > > Ignite
>>> > > >> >> > > > >>> > > > > > > APIs
>>> > > >> >> > > > >>> > > > > > > > > for
>>> > > >> >> > > > >>> > > > > > > > > > such integrations.
>>> > > >> >> > > > >>> > > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > > @Saikat Maitra <
>>> > [hidden email]>
>>> > > ,
>>> > > >> >> > @Roman
>>> > > >> >> > > <
>>> > > >> >> > > > >>> > > > > > [hidden email]
>>> > > >> >> > > > >>> > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > ,
>>> > > >> >> > > > >>> > > > > > > > > as
>>> > > >> >> > > > >>> > > > > > > > > > streaming technologies
>>> > > >> >> contributors/maintainers,
>>> > > >> >> > > > could
>>> > > >> >> > > > >>> you
>>> > > >> >> > > > >>> > > > please
>>> > > >> >> > > > >>> > > > > > > join
>>> > > >> >> > > > >>> > > > > > > > > this
>>> > > >> >> > > > >>> > > > > > > > > > thread?
>>> > > >> >> > > > >>> > > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > > -
>>> > > >> >> > > > >>> > > > > > > > > > Denis
>>> > > >> >> > > > >>> > > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > > On Thu, Oct 10, 2019 at 7:58 AM
>>> > Emmanouil
>>> > > >> >> > > > Gkatziouras <
>>> > > >> >> > > > >>> > > > > > > > > > [hidden email]>
>>> > > >> >> > > > >>> > > > > > > > > > wrote:
>>> > > >> >> > > > >>> > > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > > > Hello everyone,
>>> > > >> >> > > > >>> > > > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > > > I started using Ignite lately.
>>> Part
>>> > of
>>> > > my
>>> > > >> >> work
>>> > > >> >> > > > >>> involves
>>> > > >> >> > > > >>> > > > > Pub/Sub,
>>> > > >> >> > > > >>> > > > > > > > thus I
>>> > > >> >> > > > >>> > > > > > > > > > > created a pub/sub streamer.
>>> > > >> >> > > > >>> > > > > > > > > > > Pub/Sub is pretty close to Kafka.
>>> > > >> >> > > > >>> > > > > > > > > > > Here's how the streamer works.
>>> Every
>>> > > node
>>> > > >> >> > creates
>>> > > >> >> > > > an
>>> > > >> >> > > > >>> > > instance
>>> > > >> >> > > > >>> > > > > of
>>> > > >> >> > > > >>> > > > > > a
>>> > > >> >> > > > >>> > > > > > > > > > Pub/Sub
>>> > > >> >> > > > >>> > > > > > > > > > > client.
>>> > > >> >> > > > >>> > > > > > > > > > > The clients subscribe to the same
>>> > topic
>>> > > >> >> using
>>> > > >> >> > the
>>> > > >> >> > > > >>> same
>>> > > >> >> > > > >>> > > > > > subscription
>>> > > >> >> > > > >>> > > > > > > > > name.
>>> > > >> >> > > > >>> > > > > > > > > > > A message is being sent and one
>>> of
>>> > the
>>> > > >> nodes
>>> > > >> >> > will
>>> > > >> >> > > > >>> receive
>>> > > >> >> > > > >>> > > the
>>> > > >> >> > > > >>> > > > > > > > message,
>>> > > >> >> > > > >>> > > > > > > > > > > process it and then forward it to
>>> > > Ignite
>>> > > >> >> using
>>> > > >> >> > > the
>>> > > >> >> > > > >>> > > extractor
>>> > > >> >> > > > >>> > > > > > > > provided.
>>> > > >> >> > > > >>> > > > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > > > Any thoughts?
>>> > > >> >> > > > >>> > > > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > > > *Emmanouil Gkatziouras*
>>> > > >> >> > > > >>> > > > > > > > > > > https://egkatzioura.com/ |
>>> > > >> >> > > > >>> > > > > > > > > > >
>>> > > >> >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
>>> > > >> >> > > > >>> > > > > > > > > > > https://github.com/gkatzioura
>>> > > >> >> > > > >>> > > > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > >
>>> > > >> >> > > > >>> > > > > > >
>>> > > >> >> > > > >>> > > > > >
>>> > > >> >> > > > >>> > > > >
>>> > > >> >> > > > >>> > > >
>>> > > >> >> > > > >>> > >
>>> > > >> >> > > > >>> >
>>> > > >> >> > > > >>>
>>> > > >> >> > > > >>
>>> > > >> >> > > >
>>> > > >> >> > >
>>> > > >> >> >
>>> > > >> >>
>>> > > >> >
>>> > > >>
>>> > > >
>>> > >
>>> >
>>>
>>

[Emmanouil Gkatziouras]

Hi Saikat!

I just rebased with the flink branch. Unfortunately I do not have read
access to the team city link you provided and thus evaluate the tests for
my pull request.
I guess it has to do with ignite extensions being new.
My username is gkatzioura and email the one I am using on this email

Thank you for your time.


*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <[hidden email]> wrote:

> Hi Emmanouil,
>
> The latest build on teamcity has passed on Flink pull request.
>
>
> https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
>
> you should be able to take the changes I made in my PR and run build on
> your pull request.
>
> Please let me know if you have any questions.
>
> Regards,
> Saikat
>
> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <[hidden email]>
> wrote:
>
> > Hi Emmanouil,
> >
> > I have fixed the Flink module testsuites and have setup new project in
> > teamcity.
> >
> >
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
> >
> > I am looking into setting up the build.
> >
> > Regards,
> > Saikat
> >
> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <[hidden email]>
> > wrote:
> >
> >> Hi Emmanouil,
> >>
> >> Thank you for your email. Yes, the plan is once the Flink PR is merged
> it
> >> will provide the Licence, parent POMs etc and you can rebase from master
> >> branch and apply the changes on top of it.
> >>
> >> I am getting some issues with test failures in local with
> >> GridTestProperties as the test.properties is not present in this new
> >> project but should be available from dependencies.
> >>
> >> Once I address this issue, I will go ahead and merge the changes and
> then
> >> we can take it from there.
> >>
> >> Regards,
> >> Saikat
> >>
> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
> >> [hidden email]> wrote:
> >>
> >>> Hi all,
> >>>
> >>> I made my first pull request [1]. Since this project is brand new (no
> >>> parent poms, licensing), I reckoned it was best to use Saikat's branch
> on
> >>> Flink.
> >>> I suppose the Flink branch will be merged first. If not please give me
> >>> guidelines on how I should proceed next.
> >>>
> >>> Kind regards
> >>> Emmanouil
> >>>
> >>> [1] https://github.com/apache/ignite-extensions/pull/2
> >>>
> >>> *Emmanouil Gkatziouras*
> >>> https://egkatzioura.com/ |
> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> >>> https://github.com/gkatzioura
> >>>
> >>>
> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <[hidden email]> wrote:
> >>>
> >>> > Awesome, ping us whenever you're ready!
> >>> >
> >>> > -
> >>> > Denis
> >>> >
> >>> >
> >>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
> >>> > [hidden email]>
> >>> > wrote:
> >>> >
> >>> > > Hi all!
> >>> > >
> >>> > > I am sorry for being late on that. I was trying to refactor the
> test
> >>> in
> >>> > > order not to be in need of any external tools or spinning up a
> >>> server.
> >>> > > I did forked the new repo and indeed my changes there, so a pull
> >>> request
> >>> > is
> >>> > > a matter of time!
> >>> > >
> >>> > > Kind regards
> >>> > >
> >>> > > *Emmanouil Gkatziouras*
> >>> > > https://egkatzioura.com/ |
> >>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >>> > > https://github.com/gkatzioura
> >>> > >
> >>> > >
> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <[hidden email]>
> wrote:
> >>> > >
> >>> > > > Hi Emmanouil,
> >>> > > >
> >>> > > > Do you have any questions or need any support from the community?
> >>> > > >
> >>> > > > -
> >>> > > > Denis
> >>> > > >
> >>> > > >
> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
> >>> [hidden email]
> >>> > >
> >>> > > > wrote:
> >>> > > >
> >>> > > >> Hi Emmanouil,
> >>> > > >>
> >>> > > >> Can you please take a looks at dev utils, if this is something
> >>> you are
> >>> > > >> looking for
> >>> > > >> https://github.com/apache/ignite/tree/master/modules/dev-utils?
> >>> > > >>
> >>> > > >> IMO, if you can release Pub/Sub server in maven and then use it
> as
> >>> > > >> dependency, that would be great.
> >>> > > >>
> >>> > > >> Regards,
> >>> > > >> Saikat
> >>> > > >>
> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
> >>> > [hidden email]>
> >>> > > >> wrote:
> >>> > > >>
> >>> > > >> > Hi Emmanouil,
> >>> > > >> >
> >>> > > >> > The master branch has been initialized. I have raised a PR to
> >>> > migrate
> >>> > > >> > Flink module to ignite-extensions.
> >>> > > >> >
> >>> > > >> > PR https://github.com/apache/ignite-extensions/pull/1
> >>> > > >> > Jira : https://issues.apache.org/jira/browse/IGNITE-12356
> >>> > > >> >
> >>> > > >> > Regards,
> >>> > > >> > Saikat
> >>> > > >> >
> >>> > > >> >
> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
> >>> > > >> > [hidden email]> wrote:
> >>> > > >> >
> >>> > > >> >> Hi all!
> >>> > > >> >>
> >>> > > >> >> This is an update. The integration is tested with mocks based
> >>> on
> >>> > the
> >>> > > >> >> original Ignite repo [1].
> >>> > > >> >> I am currently waiting for the first commits on the new
> >>> integration
> >>> > > >> >> project
> >>> > > >> >> to take place in order to create my pull request accordingly
> >>> to the
> >>> > > >> >> initial
> >>> > > >> >> commits.
> >>> > > >> >> In order to simulate the Pub/Sub server in a test I used a
> test
> >>> > > server
> >>> > > >> >> implementation [2]. Unfortunately they don't have this util
> >>> > uploaded
> >>> > > >> in a
> >>> > > >> >> maven repo, thus it needs to be hosted on a maven repo
> (Apache
> >>> > > License
> >>> > > >> >> 2.0). Is there a repo for utilities that Ignite uses for the
> >>> > > >> test/build?
> >>> > > >> >> I have also created a ticket [3]
> >>> > > >> >>
> >>> > > >> >> Kind regards
> >>> > > >> >>
> >>> > > >> >> [1]
> >>> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> >>> > > >> >> [2]
> >>> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> >>> > > >> >> [3] https://issues.apache.org/jira/browse/IGNITE-12262
> >>> > > >> >>
> >>> > > >> >> *Emmanouil Gkatziouras*
> >>> > > >> >> https://egkatzioura.com/ |
> >>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >>> > > >> >> https://github.com/gkatzioura
> >>> > > >> >>
> >>> > > >> >>
> >>> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
> >>> > [hidden email]
> >>> > > >
> >>> > > >> >> wrote:
> >>> > > >> >>
> >>> > > >> >> > Hello Ilya,
> >>> > > >> >> >
> >>> > > >> >> > Thank you for your email. Yes, before we remove any
> >>> integration
> >>> > we
> >>> > > >> will
> >>> > > >> >> > move them to another repository and release.
> >>> > > >> >> >
> >>> > > >> >> > We are thinking we will start with Pub/Sub Streamer
> >>> > implementation
> >>> > > >> and
> >>> > > >> >> > continue to move other integration as mentioned in the
> >>> following
> >>> > > doc.
> >>> > > >> >> >
> >>> > > >> >> >
> >>> > > >> >> >
> >>> > > >> >>
> >>> > > >>
> >>> > >
> >>> >
> >>>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >>> > > >> >> >
> >>> > > >> >> > Regards,
> >>> > > >> >> > Saikat
> >>> > > >> >> >
> >>> > > >> >> >
> >>> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
> >>> > > >> >> > [hidden email]>
> >>> > > >> >> > wrote:
> >>> > > >> >> >
> >>> > > >> >> > > Hello!
> >>> > > >> >> > >
> >>> > > >> >> > > My take on this: we want to offload some of integrations,
> >>> but
> >>> > > >> please
> >>> > > >> >> > commit
> >>> > > >> >> > > and release them to another project first, and only then
> we
> >>> > will
> >>> > > >> >> decide
> >>> > > >> >> > to
> >>> > > >> >> > > remove them from our repository.
> >>> > > >> >> > >
> >>> > > >> >> > > Please also make sure they are available with same
> artifact
> >>> > name
> >>> > > >> and
> >>> > > >> >> API.
> >>> > > >> >> > > If this has to be changed, let's wait for AI 3.0
> >>> > > >> >> > >
> >>> > > >> >> > > Regards,
> >>> > > >> >> > > --
> >>> > > >> >> > > Ilya Kasnacheev
> >>> > > >> >> > >
> >>> > > >> >> > >
> >>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <
> >>> > > >> [hidden email]>:
> >>> > > >> >> > >
> >>> > > >> >> > > > Hello Emmanouil,
> >>> > > >> >> > > >
> >>> > > >> >> > > > As discussed earlier, I discussed with Apache Bahir
> >>> community
> >>> > > and
> >>> > > >> >> they
> >>> > > >> >> > > are
> >>> > > >> >> > > > interested to have Apache Ignite extensions as part of
> >>> Apache
> >>> > > >> Bahir
> >>> > > >> >> > > > project.
> >>> > > >> >> > > >
> >>> > > >> >> > > > Can you please share Pub/Sub streamer implementation
> >>> details
> >>> > > with
> >>> > > >> >> > Apache
> >>> > > >> >> > > > Bahir community and request for feedback.
> >>> > > >> >> > > >
> >>> > > >> >> > > > https://bahir.apache.org/
> >>> > > >> >> > > >
> >>> > > >> >> > > > I have also requested for contributor access in Jira
> for
> >>> > Apache
> >>> > > >> >> Bahir
> >>> > > >> >> > > > project so that I can create issues and assign to
> myself.
> >>> > > >> >> > > >
> >>> > > >> >> > > > I can help with code reviews as well.
> >>> > > >> >> > > >
> >>> > > >> >> > > > Here is the email thread for reference
> >>> > > >> >> > > >
> >>> > > https://www.mail-archive.com/dev@.../msg02703.html
> >>> > > >> >> > > >
> >>> > > >> >> > > > Regards,
> >>> > > >> >> > > > Saikat
> >>> > > >> >> > > >
> >>> > > >> >> > > >
> >>> > > >> >> > > >
> >>> > > >> >> > > >
> >>> > > >> >> > > >
> >>> > > >> >> > > >
> >>> > > >> >> > > >
> >>> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
> >>> > > >> >> [hidden email]
> >>> > > >> >> > >
> >>> > > >> >> > > > wrote:
> >>> > > >> >> > > >
> >>> > > >> >> > > > > Hello,
> >>> > > >> >> > > > >
> >>> > > >> >> > > > > As discussed I have created following discussion
> >>> threads on
> >>> > > our
> >>> > > >> >> > > migration
> >>> > > >> >> > > > > proposals for Apache Ignite extensions:
> >>> > > >> >> > > > >
> >>> > > >> >> > > > >
> >>> > > >> >> > > > >
> >>> > > >> >> > > >
> >>> > > >> >> > >
> >>> > > >> >> >
> >>> > > >> >>
> >>> > > >>
> >>> > >
> >>> >
> >>>
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> >>> > > >> >> > > > >
> >>> > > >> >> > > > >
> >>> > > >> https://www.mail-archive.com/dev@.../msg02703.html
> >>> > > >> >> > > > >
> >>> > > >> >> > > > > I will share update as I hear more information.
> >>> > > >> >> > > > >
> >>> > > >> >> > > > > Regards,
> >>> > > >> >> > > > > Saikat
> >>> > > >> >> > > > >
> >>> > > >> >> > > > >
> >>> > > >> >> > > > >
> >>> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
> >>> > > >> >> > [hidden email]
> >>> > > >> >> > > >
> >>> > > >> >> > > > > wrote:
> >>> > > >> >> > > > >
> >>> > > >> >> > > > >> Hello Denis,
> >>> > > >> >> > > > >>
> >>> > > >> >> > > > >> Sure, sounds good. I will create a separate
> discussion
> >>> > > thread
> >>> > > >> to
> >>> > > >> >> get
> >>> > > >> >> > > > >> community feedback on whether we
> >>> > > >> >> > > > >> should join the Bahir or kick-off "Ignite
> Extensions"
> >>> > > project.
> >>> > > >> >> > > > >>
> >>> > > >> >> > > > >> Regards,
> >>> > > >> >> > > > >> Saikat
> >>> > > >> >> > > > >>
> >>> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <
> >>> > > >> [hidden email]>
> >>> > > >> >> > > wrote:
> >>> > > >> >> > > > >>
> >>> > > >> >> > > > >>> Folks,
> >>> > > >> >> > > > >>>
> >>> > > >> >> > > > >>> The concept of Apache Bahir is precisely what we
> >>> need!
> >>> > > >> Saikat,
> >>> > > >> >> > thanks
> >>> > > >> >> > > > for
> >>> > > >> >> > > > >>> doing the research. Why I believe the * idea* fits
> us
> >>> > well:
> >>> > > >> >> > > > >>>
> >>> > > >> >> > > > >>>    - All integrations can be stored in separate
> >>> Github
> >>> > > >> >> repositories
> >>> > > >> >> > > and
> >>> > > >> >> > > > >>>    have their dev lifecycles. We've not obliged to
> >>> couple
> >>> > > all
> >>> > > >> >> the
> >>> > > >> >> > > > >>> integrations
> >>> > > >> >> > > > >>>    in a single repo. For instance, Spark can be
> >>> located
> >>> > in
> >>> > > a
> >>> > > >> >> > > dedicated
> >>> > > >> >> > > > >>> repo
> >>> > > >> >> > > > >>>    while streaming integrations might be in a
> single
> >>> one.
> >>> > > >> It's
> >>> > > >> >> up
> >>> > > >> >> > to
> >>> > > >> >> > > > us.
> >>> > > >> >> > > > >>>    - All the repositories and integrations belong
> to
> >>> ASF.
> >>> > > >> We're
> >>> > > >> >> not
> >>> > > >> >> > > > >>> dumping
> >>> > > >> >> > > > >>>    them on Github but rather keep supporting and
> >>> > developing
> >>> > > >> in
> >>> > > >> >> > > > >>> accordance with
> >>> > > >> >> > > > >>>    ASF vision and practices.
> >>> > > >> >> > > > >>>    - There is a way to reward contributors via
> >>> > > committership
> >>> > > >> and
> >>> > > >> >> > PMC
> >>> > > >> >> > > > >>>    membership. You won't get this if a project is
> >>> just
> >>> > one
> >>> > > of
> >>> > > >> >> the
> >>> > > >> >> > > > >>> millions of
> >>> > > >> >> > > > >>>    Github projects.
> >>> > > >> >> > > > >>>
> >>> > > >> >> > > > >>> Saikat, as Ignite PMC member, could you please
> >>> kick-off a
> >>> > > >> >> separate
> >>> > > >> >> > > > >>> dev-list
> >>> > > >> >> > > > >>> discussion to involve more community members and
> >>> > referring
> >>> > > to
> >>> > > >> >> this
> >>> > > >> >> > > > >>> thread?
> >>> > > >> >> > > > >>> I think the primary question the community needs to
> >>> > answer
> >>> > > >> >> whether
> >>> > > >> >> > we
> >>> > > >> >> > > > >>> should join the Bahir or kick-off "Ignite
> Extensions"
> >>> > > >> project?
> >>> > > >> >> > > > >>>
> >>> > > >> >> > > > >>> -
> >>> > > >> >> > > > >>> Denis
> >>> > > >> >> > > > >>>
> >>> > > >> >> > > > >>>
> >>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev <
> >>> > > >> >> > > > [hidden email]>
> >>> > > >> >> > > > >>> wrote:
> >>> > > >> >> > > > >>>
> >>> > > >> >> > > > >>> > Maybe we could move all our Streaming
> Integrations
> >>> > there,
> >>> > > >> but
> >>> > > >> >> > what
> >>> > > >> >> > > is
> >>> > > >> >> > > > >>> about
> >>> > > >> >> > > > >>> > maintaining and committer permissions to the new
> >>> > > >> repositories?
> >>> > > >> >> > > > >>> > I see the list of the committers and PMC members
> >>> there
> >>> > > >> >> > > > >>> > https://bahir.apache.org/community-members/
> >>> > > >> >> > > > >>> >
> >>> > > >> >> > > > >>> > Could somebody from Ignite community be added to
> >>> this
> >>> > > list
> >>> > > >> as
> >>> > > >> >> a
> >>> > > >> >> > > > >>> > PMC/committer to maintain Ignite integrations?
> >>> > > >> >> > > > >>> > If yes, I'd happy to join this project and
> >>> collaborate
> >>> > > with
> >>> > > >> >> these
> >>> > > >> >> > > > guys
> >>> > > >> >> > > > >>> > together
> >>> > > >> >> > > > >>> > If no, and we should start from the zero level
> with
> >>> > > >> external
> >>> > > >> >> PRs
> >>> > > >> >> > -
> >>> > > >> >> > > > >>> hmmm,
> >>> > > >> >> > > > >>> > it's better to have our own external repository
> >>> with
> >>> > > >> >> ApacheBahirr
> >>> > > >> >> > > > >>> ideology.
> >>> > > >> >> > > > >>> >
> >>> > > >> >> > > > >>> > Also, I agree, that all connectors could be moved
> >>> there
> >>> > > (in
> >>> > > >> >> > > > >>> ApacheBahirr
> >>> > > >> >> > > > >>> > like repository), but not all modules from the
> page
> >>> > > >> >> > > > >>> >
> >>> > > >> >> > > > >>> >
> >>> > > >> >> > > > >>>
> >>> > > >> >> > > >
> >>> > > >> >> > >
> >>> > > >> >> >
> >>> > > >> >>
> >>> > > >>
> >>> > >
> >>> >
> >>>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >>> > > >> >> > > > >>> > because
> >>> > > >> >> > > > >>> > they use different parts of core modules and
> could
> >>> be
> >>> > > >> >> developed
> >>> > > >> >> > > with
> >>> > > >> >> > > > >>> > different velocity.
> >>> > > >> >> > > > >>> >
> >>> > > >> >> > > > >>> > In reality, before creation of new repositories
> we
> >>> need
> >>> > > >> wide
> >>> > > >> >> > > > discussion
> >>> > > >> >> > > > >>> > based on the document mentioned above.
> >>> > > >> >> > > > >>> >
> >>> > > >> >> > > > >>> > P.S. Of course, we could experiment in the next
> >>> release
> >>> > > 2.8
> >>> > > >> >> with
> >>> > > >> >> > > one
> >>> > > >> >> > > > >>> or two
> >>> > > >> >> > > > >>> > integrations like twitter or ZeroMQ
> >>> > > >> >> > > > >>> > Also, I'd like Denis Magda idea for the separate
> >>> > > >> repositories
> >>> > > >> >> in
> >>> > > >> >> > > > Apache
> >>> > > >> >> > > > >>> > Github to maintain them by anybody.
> >>> > > >> >> > > > >>> >
> >>> > > >> >> > > > >>> >
> >>> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra <
> >>> > > >> >> > > [hidden email]
> >>> > > >> >> > > > >:
> >>> > > >> >> > > > >>> >
> >>> > > >> >> > > > >>> > > Hi Denis, Emmanouil
> >>> > > >> >> > > > >>> > >
> >>> > > >> >> > > > >>> > > Thank you for your email. I think creating a
> >>> separate
> >>> > > >> >> > integration
> >>> > > >> >> > > > >>> project
> >>> > > >> >> > > > >>> > > in github and also proposing it as an apache
> >>> > incubator
> >>> > > >> >> project
> >>> > > >> >> > > will
> >>> > > >> >> > > > >>> be a
> >>> > > >> >> > > > >>> > > good move. The other separate integration
> modules
> >>> > > >> >> > > > >>> > >
> >>> > > >> >> > > > >>> > >
> >>> > > >> >> > > > >>> >
> >>> > > >> >> > > > >>>
> >>> > > >> >> > > >
> >>> > > >> >> > >
> >>> > > >> >> >
> >>> > > >> >>
> >>> > > >>
> >>> > >
> >>> >
> >>>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >>> > > >> >> > > > >>> > > can
> >>> > > >> >> > > > >>> > > be moved to this new apache incubator project.
> >>> > > >> >> > > > >>> > >
> >>> > > >> >> > > > >>> > > There are similar integration available for
> >>> Flink and
> >>> > > >> Spark
> >>> > > >> >> in
> >>> > > >> >> > > > Apache
> >>> > > >> >> > > > >>> > Bahir
> >>> > > >> >> > > > >>> > > https://bahir.apache.org/
> >>> > > >> >> > > > >>> > >
> >>> > > >> >> > > > >>> > > Do you think we should reach out to Apache
> Bahir
> >>> > > >> community
> >>> > > >> >> > with a
> >>> > > >> >> > > > >>> > proposal
> >>> > > >> >> > > > >>> > > or we should plan to create a separate Apache
> >>> > Incubator
> >>> > > >> >> > project?
> >>> > > >> >> > > > >>> > >
> >>> > > >> >> > > > >>> > > Regards,
> >>> > > >> >> > > > >>> > > Saikat
> >>> > > >> >> > > > >>> > >
> >>> > > >> >> > > > >>> > >
> >>> > > >> >> > > > >>> > >
> >>> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil
> >>> > Gkatziouras <
> >>> > > >> >> > > > >>> > > [hidden email]>
> >>> > > >> >> > > > >>> > > wrote:
> >>> > > >> >> > > > >>> > >
> >>> > > >> >> > > > >>> > > > Hi all!
> >>> > > >> >> > > > >>> > > >
> >>> > > >> >> > > > >>> > > > Based on the discussions most probably this
> is
> >>> > going
> >>> > > to
> >>> > > >> >> be on
> >>> > > >> >> > > > >>> another
> >>> > > >> >> > > > >>> > > > GitHub repo. Therefore I will prepare a
> >>> standalone
> >>> > > >> project
> >>> > > >> >> > with
> >>> > > >> >> > > > the
> >>> > > >> >> > > > >>> > > Pub/Sub
> >>> > > >> >> > > > >>> > > > module and once the repository is created a
> >>> pull
> >>> > > >> request
> >>> > > >> >> > shall
> >>> > > >> >> > > be
> >>> > > >> >> > > > >>> > issued.
> >>> > > >> >> > > > >>> > > > If there is anything else I can do in order
> to
> >>> > > >> facilitate
> >>> > > >> >> > this
> >>> > > >> >> > > > >>> please
> >>> > > >> >> > > > >>> > let
> >>> > > >> >> > > > >>> > > > me know.
> >>> > > >> >> > > > >>> > > >
> >>> > > >> >> > > > >>> > > > If a ticket is created for this issue, I
> shall
> >>> be
> >>> > > able
> >>> > > >> to
> >>> > > >> >> > > create
> >>> > > >> >> > > > a
> >>> > > >> >> > > > >>> pull
> >>> > > >> >> > > > >>> > > > request and thus a build will take place on
> >>> > ignite's
> >>> > > >> CI.
> >>> > > >> >> > > > >>> > > > If everyone is aligned with adding a Pub/Sub
> >>> > > >> >> implementation
> >>> > > >> >> > > may I
> >>> > > >> >> > > > >>> > create
> >>> > > >> >> > > > >>> > > a
> >>> > > >> >> > > > >>> > > > ticket on JIRA?
> >>> > > >> >> > > > >>> > > >
> >>> > > >> >> > > > >>> > > > My next question has to do on how Ignite
> >>> configures
> >>> > > the
> >>> > > >> >> build
> >>> > > >> >> > > > jobs
> >>> > > >> >> > > > >>> in
> >>> > > >> >> > > > >>> > > Team
> >>> > > >> >> > > > >>> > > > City. The unit tests for my implementation
> >>> need a
> >>> > > local
> >>> > > >> >> > Pub/Sub
> >>> > > >> >> > > > >>> server
> >>> > > >> >> > > > >>> > up
> >>> > > >> >> > > > >>> > > > and running (they provide one for testing
> >>> > purposes).
> >>> > > >> Will
> >>> > > >> >> > this
> >>> > > >> >> > > > take
> >>> > > >> >> > > > >>> > > effect
> >>> > > >> >> > > > >>> > > > in a form of a build script inside my
> >>> > implementation
> >>> > > or
> >>> > > >> >> is it
> >>> > > >> >> > > > >>> something
> >>> > > >> >> > > > >>> > > > that should be configured on Team City?
> >>> > > >> >> > > > >>> > > >
> >>> > > >> >> > > > >>> > > > Kind regards,
> >>> > > >> >> > > > >>> > > > Emmanouil
> >>> > > >> >> > > > >>> > > >
> >>> > > >> >> > > > >>> > > >
> >>> > > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
> >>> > > >> >> > > > >>> > > > https://egkatzioura.com/ |
> >>> > > >> >> > > > >>> > > >
> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> >>> > > >> >> > > > >>> > > > https://github.com/gkatzioura
> >>> > > >> >> > > > >>> > > >
> >>> > > >> >> > > > >>> > > >
> >>> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis Magda <
> >>> > > >> >> [hidden email]
> >>> > > >> >> > >
> >>> > > >> >> > > > >>> wrote:
> >>> > > >> >> > > > >>> > > >
> >>> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
> >>> > > >> >> > > > >>> > > > >
> >>> > > >> >> > > > >>> > > > > After contemplating on this, let me propose
> >>> > another
> >>> > > >> way
> >>> > > >> >> > with
> >>> > > >> >> > > > this
> >>> > > >> >> > > > >>> > > > > particular integration that is aligned with
> >>> our
> >>> > > >> >> > > modularization
> >>> > > >> >> > > > >>> > > endeavor.
> >>> > > >> >> > > > >>> > > > > The modularization [1] defines the Ignite
> >>> core,
> >>> > > >> Ignite
> >>> > > >> >> > > modules,
> >>> > > >> >> > > > >>> and
> >>> > > >> >> > > > >>> > > > > independent integrations. The first two are
> >>> to be
> >>> > > >> >> located
> >>> > > >> >> > in
> >>> > > >> >> > > > >>> Ignite
> >>> > > >> >> > > > >>> > > > > repositories and to be tested/updated by
> the
> >>> > > >> community
> >>> > > >> >> for
> >>> > > >> >> > > > every
> >>> > > >> >> > > > >>> > > release.
> >>> > > >> >> > > > >>> > > > > The modules are a subset of existing
> >>> integrations
> >>> > > >> that
> >>> > > >> >> are
> >>> > > >> >> > > > >>> adopted
> >>> > > >> >> > > > >>> > > widely
> >>> > > >> >> > > > >>> > > > > and require community attention.
> >>> > > >> >> > > > >>> > > > >
> >>> > > >> >> > > > >>> > > > > As for the independent integrations [1],
> >>> those
> >>> > are
> >>> > > >> >> supposed
> >>> > > >> >> > > to
> >>> > > >> >> > > > be
> >>> > > >> >> > > > >>> > > located
> >>> > > >> >> > > > >>> > > > > in a separate GitHub repo. I would advise
> us
> >>> to
> >>> > > start
> >>> > > >> >> > > > >>> implementing
> >>> > > >> >> > > > >>> > this
> >>> > > >> >> > > > >>> > > > > concept with the Pub/Sub integration. Even
> >>> though
> >>> > > it
> >>> > > >> >> will
> >>> > > >> >> > be
> >>> > > >> >> > > > >>> located
> >>> > > >> >> > > > >>> > > in a
> >>> > > >> >> > > > >>> > > > > separate Github repo, it will be listed
> among
> >>> > > >> officially
> >>> > > >> >> > > > >>> available
> >>> > > >> >> > > > >>> > > > > integrations on the Ignite website
> featuring
> >>> > > >> Emmanouil
> >>> > > >> >> as a
> >>> > > >> >> > > > >>> > > contributor.
> >>> > > >> >> > > > >>> > > > >
> >>> > > >> >> > > > >>> > > > > What do you think?
> >>> > > >> >> > > > >>> > > > >
> >>> > > >> >> > > > >>> > > > >
> >>> > > >> >> > > > >>> > > > > [1]
> >>> > > >> >> > > > >>> > > > >
> >>> > > >> >> > > > >>> > > > >
> >>> > > >> >> > > > >>> > > >
> >>> > > >> >> > > > >>> > >
> >>> > > >> >> > > > >>> >
> >>> > > >> >> > > > >>>
> >>> > > >> >> > > >
> >>> > > >> >> > >
> >>> > > >> >> >
> >>> > > >> >>
> >>> > > >>
> >>> > >
> >>> >
> >>>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >>> > > >> >> > > > >>> > > > >
> >>> > > >> >> > > > >>> > > > >
> >>> > > >> >> > > > >>> > > > >
> >>> > > >> >> > > > >>> > > > > -
> >>> > > >> >> > > > >>> > > > > Denis
> >>> > > >> >> > > > >>> > > > >
> >>> > > >> >> > > > >>> > > > >
> >>> > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat
> >>> Maitra <
> >>> > > >> >> > > > >>> > [hidden email]
> >>> > > >> >> > > > >>> > > >
> >>> > > >> >> > > > >>> > > > > wrote:
> >>> > > >> >> > > > >>> > > > >
> >>> > > >> >> > > > >>> > > > > > Hi Emmanouil,
> >>> > > >> >> > > > >>> > > > > >
> >>> > > >> >> > > > >>> > > > > > The changes looks good to me. I wanted to
> >>> check
> >>> > > if
> >>> > > >> you
> >>> > > >> >> > are
> >>> > > >> >> > > > also
> >>> > > >> >> > > > >>> > able
> >>> > > >> >> > > > >>> > > to
> >>> > > >> >> > > > >>> > > > > > validate that the once data is added to
> >>> Ignite
> >>> > > >> cluster
> >>> > > >> >> > you
> >>> > > >> >> > > > are
> >>> > > >> >> > > > >>> also
> >>> > > >> >> > > > >>> > > > able
> >>> > > >> >> > > > >>> > > > > to
> >>> > > >> >> > > > >>> > > > > > access it using another Ignite client or
> >>> using
> >>> > > rest
> >>> > > >> >> > > > endpoints.
> >>> > > >> >> > > > >>> > > > > >
> >>> > > >> >> > > > >>> > > > > > We use Teamcity for CI process, the
> >>> details are
> >>> > > as
> >>> > > >> >> > follows
> >>> > > >> >> > > > >>> > > > > >
> >>> > > >> >> > > > >>> > > > > > Trigger validation of those test suites
> >>> that
> >>> > have
> >>> > > >> been
> >>> > > >> >> > > > >>> affected by
> >>> > > >> >> > > > >>> > > your
> >>> > > >> >> > > > >>> > > > > > changes on TeamCity <
> >>> > > http://ci.ignite.apache.org/>:
> >>> > > >> <
> >>> > > >> >> > > > >>> > > > > http://204.14.53.153/
> >>> > > >> >> > > > >>> > > > > > >
> >>> > > >> >> > > > >>> > > > > >
> >>> > > >> >> > > > >>> > > > > >    - Locate a test suite you have to
> check,
> >>> > press
> >>> > > >> >> button
> >>> > > >> >> > > > named
> >>> > > >> >> > > > >>> > "..."
> >>> > > >> >> > > > >>> > > > that
> >>> > > >> >> > > > >>> > > > > >    is located on the left of "Run"
> button.
> >>> "Run
> >>> > > >> custom
> >>> > > >> >> > > build"
> >>> > > >> >> > > > >>> > dialog
> >>> > > >> >> > > > >>> > > > will
> >>> > > >> >> > > > >>> > > > > >    appear;
> >>> > > >> >> > > > >>> > > > > >    - Go to "Changes" tab and choose
> >>> > > >> >> > > > >>> > "pull/<pull-request-number>/head"
> >>> > > >> >> > > > >>> > > > in
> >>> > > >> >> > > > >>> > > > > >    "Build branch" dropdown list;
> >>> > > >> >> > > > >>> > > > > >    - Press "Run build" button and monitor
> >>> tests
> >>> > > >> >> results.
> >>> > > >> >> > > > >>> > > > > >
> >>> > > >> >> > > > >>> > > > > >
> >>> > > >> >> > > > >>> > > > > >
> >>> > > >> >> > > > >>> > > > > > @Roman [hidden email] - can you also
> >>> please
> >>> > > >> take a
> >>> > > >> >> > > look?
> >>> > > >> >> > > > >>> > > > > >
> >>> > > >> >> > > > >>> > > > > > Regards,
> >>> > > >> >> > > > >>> > > > > > Saikat
> >>> > > >> >> > > > >>> > > > > >
> >>> > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM Emmanouil
> >>> > > >> Gkatziouras
> >>> > > >> >> <
> >>> > > >> >> > > > >>> > > > > > [hidden email]>
> >>> > > >> >> > > > >>> > > > > > wrote:
> >>> > > >> >> > > > >>> > > > > >
> >>> > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
> >>> > > >> >> > > > >>> > > > > > >
> >>> > > >> >> > > > >>> > > > > > > I have my implementation on an Ignite
> >>> fork on
> >>> > > my
> >>> > > >> >> > personal
> >>> > > >> >> > > > >>> GitHub
> >>> > > >> >> > > > >>> > > > > account
> >>> > > >> >> > > > >>> > > > > > > [1]
> >>> > > >> >> > > > >>> > > > > > > It is based on the Kafka implementation
> >>> which
> >>> > > is
> >>> > > >> of
> >>> > > >> >> a
> >>> > > >> >> > > > polling
> >>> > > >> >> > > > >>> > > nature.
> >>> > > >> >> > > > >>> > > > > > > GCP does not provide a service like
> >>> Kafka.
> >>> > > >> Instead
> >>> > > >> >> it
> >>> > > >> >> > > > >>> provides us
> >>> > > >> >> > > > >>> > > > with
> >>> > > >> >> > > > >>> > > > > > > Pub/Sub which is just like Kafka.
> >>> > > >> >> > > > >>> > > > > > > Therefore using the Kafka streamer as a
> >>> > > guideline
> >>> > > >> >> made
> >>> > > >> >> > > more
> >>> > > >> >> > > > >>> sense
> >>> > > >> >> > > > >>> > > to
> >>> > > >> >> > > > >>> > > > > me.
> >>> > > >> >> > > > >>> > > > > > >
> >>> > > >> >> > > > >>> > > > > > > Apart from the implementation I have
> also
> >>> > > added a
> >>> > > >> >> unit
> >>> > > >> >> > > > test.
> >>> > > >> >> > > > >>> > > > > > > It definitely needs some polishing and
> a
> >>> > double
> >>> > > >> >> check
> >>> > > >> >> > on
> >>> > > >> >> > > my
> >>> > > >> >> > > > >>> side
> >>> > > >> >> > > > >>> > to
> >>> > > >> >> > > > >>> > > > > make
> >>> > > >> >> > > > >>> > > > > > > the review smooth.
> >>> > > >> >> > > > >>> > > > > > > For the tests a local Pub/Sub [2]
> server
> >>> has
> >>> > to
> >>> > > >> be
> >>> > > >> >> > setup
> >>> > > >> >> > > > >>> however
> >>> > > >> >> > > > >>> > I
> >>> > > >> >> > > > >>> > > am
> >>> > > >> >> > > > >>> > > > > not
> >>> > > >> >> > > > >>> > > > > > > sure which is the best way to do so
> with
> >>> > > regards
> >>> > > >> to
> >>> > > >> >> > your
> >>> > > >> >> > > > >>> CI/CD.
> >>> > > >> >> > > > >>> > > > > > >
> >>> > > >> >> > > > >>> > > > > > > [1]
> >>> > > >> >> > > > >>> > >
> >>> > > >> >> >
> >>> https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> >>> > > >> >> > > > >>> > > > > > > [2]
> >>> > > >> >> > > > >>>
> >>> > https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
> >>> > > >> >> > > > >>> > > > > > >
> >>> > > >> >> > > > >>> > > > > > > *Emmanouil Gkatziouras*
> >>> > > >> >> > > > >>> > > > > > > https://egkatzioura.com/ |
> >>> > > >> >> > > > >>> > > > > > >
> >>> > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >>> > > >> >> > > > >>> > > > > > > https://github.com/gkatzioura
> >>> > > >> >> > > > >>> > > > > > >
> >>> > > >> >> > > > >>> > > > > > >
> >>> > > >> >> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11, Saikat
> >>> Maitra <
> >>> > > >> >> > > > >>> > > [hidden email]
> >>> > > >> >> > > > >>> > > > >
> >>> > > >> >> > > > >>> > > > > > > wrote:
> >>> > > >> >> > > > >>> > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > Hello Emmanouil,
> >>> > > >> >> > > > >>> > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > Can you please share more info about
> >>> the
> >>> > > >> >> > > implementation?
> >>> > > >> >> > > > >>> > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > We have similar implementation for
> data
> >>> > > >> streamers
> >>> > > >> >> > with
> >>> > > >> >> > > > >>> Kafka
> >>> > > >> >> > > > >>> > and
> >>> > > >> >> > > > >>> > > > > > RocketMQ
> >>> > > >> >> > > > >>> > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > >
> >>> > > >> >> > https://apacheignite-mix.readme.io/docs/kafka-streamer
> >>> > > >> >> > > > >>> > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > >
> >>> > > >> >> > > >
> >>> https://apacheignite-mix.readme.io/docs/rocketmq-streamer
> >>> > > >> >> > > > >>> > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > Please review and share if you have
> any
> >>> > > >> questions.
> >>> > > >> >> > > > >>> > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > Regards,
> >>> > > >> >> > > > >>> > > > > > > > Saikat
> >>> > > >> >> > > > >>> > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56 PM
> >>> Emmanouil
> >>> > > >> >> > Gkatziouras <
> >>> > > >> >> > > > >>> > > > > > > > [hidden email]>
> >>> > > >> >> > > > >>> > > > > > > > wrote:
> >>> > > >> >> > > > >>> > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > Hi all,
> >>> > > >> >> > > > >>> > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > By mentioning Pub/Sub I mean Google
> >>> > Cloud's
> >>> > > >> >> Pub/Sub
> >>> > > >> >> > > [1]
> >>> > > >> >> > > > >>> > > > > > > > > Yes it is indeed cloud specific. It
> >>> is
> >>> > the
> >>> > > >> >> > messaging
> >>> > > >> >> > > > >>> service
> >>> > > >> >> > > > >>> > > > > provided
> >>> > > >> >> > > > >>> > > > > > > by
> >>> > > >> >> > > > >>> > > > > > > > > GCP.
> >>> > > >> >> > > > >>> > > > > > > > > I have made the integration with
> it,
> >>> but
> >>> > I
> >>> > > >> >> haven't
> >>> > > >> >> > > > >>> published
> >>> > > >> >> > > > >>> > it
> >>> > > >> >> > > > >>> > > > yet
> >>> > > >> >> > > > >>> > > > > > (I
> >>> > > >> >> > > > >>> > > > > > > am
> >>> > > >> >> > > > >>> > > > > > > > > waiting for the discussion/ticket
> and
> >>> > > proceed
> >>> > > >> >> with
> >>> > > >> >> > > the
> >>> > > >> >> > > > >>> > > > > contribution).
> >>> > > >> >> > > > >>> > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > Kind regards,
> >>> > > >> >> > > > >>> > > > > > > > > Emmanouil
> >>> > > >> >> > > > >>> > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > [1]
> >>> > > >> >> https://cloud.google.com/pubsub/docs/overview
> >>> > > >> >> > > > >>> > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
> >>> > > >> >> > > > >>> > > > > > > > > https://egkatzioura.com/ |
> >>> > > >> >> > > > >>> > > > > > > > >
> >>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >>> > > >> >> > > > >>> > > > > > > > > https://github.com/gkatzioura
> >>> > > >> >> > > > >>> > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at 21:45, Denis
> >>> > Magda <
> >>> > > >> >> > > > >>> [hidden email]
> >>> > > >> >> > > > >>> > >
> >>> > > >> >> > > > >>> > > > > wrote:
> >>> > > >> >> > > > >>> > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > > Hello Emmanouil,
> >>> > > >> >> > > > >>> > > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > > Are you referring to any specific
> >>> > > >> >> implementation
> >>> > > >> >> > > > >>> tailored
> >>> > > >> >> > > > >>> > > for a
> >>> > > >> >> > > > >>> > > > > > > cloud?
> >>> > > >> >> > > > >>> > > > > > > > If
> >>> > > >> >> > > > >>> > > > > > > > > > I'm not mistaken you were
> >>> mentioning
> >>> > > >> AWS/GCE.
> >>> > > >> >> Is
> >>> > > >> >> > > > there
> >>> > > >> >> > > > >>> any
> >>> > > >> >> > > > >>> > > > Github
> >>> > > >> >> > > > >>> > > > > > > > source
> >>> > > >> >> > > > >>> > > > > > > > > > code?
> >>> > > >> >> > > > >>> > > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > > Here is a list of existing
> >>> integrations
> >>> > > >> with
> >>> > > >> >> > > > streaming
> >>> > > >> >> > > > >>> and
> >>> > > >> >> > > > >>> > > > > pub/sub
> >>> > > >> >> > > > >>> > > > > > > > > > frameworks:
> >>> > > >> >> > > > >>> > > > > > > > > >
> >>> > > >> >> https://apacheignite-mix.readme.io/docs/overview
> >>> > > >> >> > > > >>> > > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > > with the source code in Ignite
> >>> master:
> >>> > > >> >> > > > >>> > > > > > > > > >
> >>> > > >> >> > > https://github.com/apache/ignite/tree/master/modules
> >>> > > >> >> > > > >>> > > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > > Check out those resources to see
> >>> what
> >>> > are
> >>> > > >> the
> >>> > > >> >> > most
> >>> > > >> >> > > > >>> suitable
> >>> > > >> >> > > > >>> > > > > Ignite
> >>> > > >> >> > > > >>> > > > > > > APIs
> >>> > > >> >> > > > >>> > > > > > > > > for
> >>> > > >> >> > > > >>> > > > > > > > > > such integrations.
> >>> > > >> >> > > > >>> > > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > > @Saikat Maitra <
> >>> > [hidden email]>
> >>> > > ,
> >>> > > >> >> > @Roman
> >>> > > >> >> > > <
> >>> > > >> >> > > > >>> > > > > > [hidden email]
> >>> > > >> >> > > > >>> > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > ,
> >>> > > >> >> > > > >>> > > > > > > > > as
> >>> > > >> >> > > > >>> > > > > > > > > > streaming technologies
> >>> > > >> >> contributors/maintainers,
> >>> > > >> >> > > > could
> >>> > > >> >> > > > >>> you
> >>> > > >> >> > > > >>> > > > please
> >>> > > >> >> > > > >>> > > > > > > join
> >>> > > >> >> > > > >>> > > > > > > > > this
> >>> > > >> >> > > > >>> > > > > > > > > > thread?
> >>> > > >> >> > > > >>> > > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > > -
> >>> > > >> >> > > > >>> > > > > > > > > > Denis
> >>> > > >> >> > > > >>> > > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > > On Thu, Oct 10, 2019 at 7:58 AM
> >>> > Emmanouil
> >>> > > >> >> > > > Gkatziouras <
> >>> > > >> >> > > > >>> > > > > > > > > > [hidden email]>
> >>> > > >> >> > > > >>> > > > > > > > > > wrote:
> >>> > > >> >> > > > >>> > > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > > > Hello everyone,
> >>> > > >> >> > > > >>> > > > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > > > I started using Ignite lately.
> >>> Part
> >>> > of
> >>> > > my
> >>> > > >> >> work
> >>> > > >> >> > > > >>> involves
> >>> > > >> >> > > > >>> > > > > Pub/Sub,
> >>> > > >> >> > > > >>> > > > > > > > thus I
> >>> > > >> >> > > > >>> > > > > > > > > > > created a pub/sub streamer.
> >>> > > >> >> > > > >>> > > > > > > > > > > Pub/Sub is pretty close to
> Kafka.
> >>> > > >> >> > > > >>> > > > > > > > > > > Here's how the streamer works.
> >>> Every
> >>> > > node
> >>> > > >> >> > creates
> >>> > > >> >> > > > an
> >>> > > >> >> > > > >>> > > instance
> >>> > > >> >> > > > >>> > > > > of
> >>> > > >> >> > > > >>> > > > > > a
> >>> > > >> >> > > > >>> > > > > > > > > > Pub/Sub
> >>> > > >> >> > > > >>> > > > > > > > > > > client.
> >>> > > >> >> > > > >>> > > > > > > > > > > The clients subscribe to the
> same
> >>> > topic
> >>> > > >> >> using
> >>> > > >> >> > the
> >>> > > >> >> > > > >>> same
> >>> > > >> >> > > > >>> > > > > > subscription
> >>> > > >> >> > > > >>> > > > > > > > > name.
> >>> > > >> >> > > > >>> > > > > > > > > > > A message is being sent and one
> >>> of
> >>> > the
> >>> > > >> nodes
> >>> > > >> >> > will
> >>> > > >> >> > > > >>> receive
> >>> > > >> >> > > > >>> > > the
> >>> > > >> >> > > > >>> > > > > > > > message,
> >>> > > >> >> > > > >>> > > > > > > > > > > process it and then forward it
> to
> >>> > > Ignite
> >>> > > >> >> using
> >>> > > >> >> > > the
> >>> > > >> >> > > > >>> > > extractor
> >>> > > >> >> > > > >>> > > > > > > > provided.
> >>> > > >> >> > > > >>> > > > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > > > Any thoughts?
> >>> > > >> >> > > > >>> > > > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > > > *Emmanouil Gkatziouras*
> >>> > > >> >> > > > >>> > > > > > > > > > > https://egkatzioura.com/ |
> >>> > > >> >> > > > >>> > > > > > > > > > >
> >>> > > >> >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >>> > > >> >> > > > >>> > > > > > > > > > > https://github.com/gkatzioura
> >>> > > >> >> > > > >>> > > > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > >
> >>> > > >> >> > > > >>> > > > > > >
> >>> > > >> >> > > > >>> > > > > >
> >>> > > >> >> > > > >>> > > > >
> >>> > > >> >> > > > >>> > > >
> >>> > > >> >> > > > >>> > >
> >>> > > >> >> > > > >>> >
> >>> > > >> >> > > > >>>
> >>> > > >> >> > > > >>
> >>> > > >> >> > > >
> >>> > > >> >> > >
> >>> > > >> >> >
> >>> > > >> >>
> >>> > > >> >
> >>> > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> >>
>

[Emmanouil Gkatziouras]

Hi all!

If someone could give me read access to the Job of Ignite Extensions on
Team City will greatly help me (username gkatzioura).
I suppose the builds get triggered automatically on a pull request.

Kind regards,
Emmanouil

*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras <[hidden email]>
wrote:

> Hi Saikat!
>
> I just rebased with the flink branch. Unfortunately I do not have read
> access to the team city link you provided and thus evaluate the tests for
> my pull request.
> I guess it has to do with ignite extensions being new.
> My username is gkatzioura and email the one I am using on this email
>
> Thank you for your time.
>
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <[hidden email]>
> wrote:
>
>> Hi Emmanouil,
>>
>> The latest build on teamcity has passed on Flink pull request.
>>
>>
>> https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
>>
>> you should be able to take the changes I made in my PR and run build on
>> your pull request.
>>
>> Please let me know if you have any questions.
>>
>> Regards,
>> Saikat
>>
>> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <[hidden email]>
>> wrote:
>>
>> > Hi Emmanouil,
>> >
>> > I have fixed the Flink module testsuites and have setup new project in
>> > teamcity.
>> >
>> >
>> >
>> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
>> >
>> > I am looking into setting up the build.
>> >
>> > Regards,
>> > Saikat
>> >
>> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <[hidden email]>
>> > wrote:
>> >
>> >> Hi Emmanouil,
>> >>
>> >> Thank you for your email. Yes, the plan is once the Flink PR is merged
>> it
>> >> will provide the Licence, parent POMs etc and you can rebase from
>> master
>> >> branch and apply the changes on top of it.
>> >>
>> >> I am getting some issues with test failures in local with
>> >> GridTestProperties as the test.properties is not present in this new
>> >> project but should be available from dependencies.
>> >>
>> >> Once I address this issue, I will go ahead and merge the changes and
>> then
>> >> we can take it from there.
>> >>
>> >> Regards,
>> >> Saikat
>> >>
>> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
>> >> [hidden email]> wrote:
>> >>
>> >>> Hi all,
>> >>>
>> >>> I made my first pull request [1]. Since this project is brand new (no
>> >>> parent poms, licensing), I reckoned it was best to use Saikat's
>> branch on
>> >>> Flink.
>> >>> I suppose the Flink branch will be merged first. If not please give me
>> >>> guidelines on how I should proceed next.
>> >>>
>> >>> Kind regards
>> >>> Emmanouil
>> >>>
>> >>> [1] https://github.com/apache/ignite-extensions/pull/2
>> >>>
>> >>> *Emmanouil Gkatziouras*
>> >>> https://egkatzioura.com/ |
>> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> >>> https://github.com/gkatzioura
>> >>>
>> >>>
>> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <[hidden email]> wrote:
>> >>>
>> >>> > Awesome, ping us whenever you're ready!
>> >>> >
>> >>> > -
>> >>> > Denis
>> >>> >
>> >>> >
>> >>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
>> >>> > [hidden email]>
>> >>> > wrote:
>> >>> >
>> >>> > > Hi all!
>> >>> > >
>> >>> > > I am sorry for being late on that. I was trying to refactor the
>> test
>> >>> in
>> >>> > > order not to be in need of any external tools or spinning up a
>> >>> server.
>> >>> > > I did forked the new repo and indeed my changes there, so a pull
>> >>> request
>> >>> > is
>> >>> > > a matter of time!
>> >>> > >
>> >>> > > Kind regards
>> >>> > >
>> >>> > > *Emmanouil Gkatziouras*
>> >>> > > https://egkatzioura.com/ |
>> >>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> >>> > > https://github.com/gkatzioura
>> >>> > >
>> >>> > >
>> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <[hidden email]>
>> wrote:
>> >>> > >
>> >>> > > > Hi Emmanouil,
>> >>> > > >
>> >>> > > > Do you have any questions or need any support from the
>> community?
>> >>> > > >
>> >>> > > > -
>> >>> > > > Denis
>> >>> > > >
>> >>> > > >
>> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
>> >>> [hidden email]
>> >>> > >
>> >>> > > > wrote:
>> >>> > > >
>> >>> > > >> Hi Emmanouil,
>> >>> > > >>
>> >>> > > >> Can you please take a looks at dev utils, if this is something
>> >>> you are
>> >>> > > >> looking for
>> >>> > > >> https://github.com/apache/ignite/tree/master/modules/dev-utils
>> ?
>> >>> > > >>
>> >>> > > >> IMO, if you can release Pub/Sub server in maven and then use
>> it as
>> >>> > > >> dependency, that would be great.
>> >>> > > >>
>> >>> > > >> Regards,
>> >>> > > >> Saikat
>> >>> > > >>
>> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
>> >>> > [hidden email]>
>> >>> > > >> wrote:
>> >>> > > >>
>> >>> > > >> > Hi Emmanouil,
>> >>> > > >> >
>> >>> > > >> > The master branch has been initialized. I have raised a PR to
>> >>> > migrate
>> >>> > > >> > Flink module to ignite-extensions.
>> >>> > > >> >
>> >>> > > >> > PR https://github.com/apache/ignite-extensions/pull/1
>> >>> > > >> > Jira : https://issues.apache.org/jira/browse/IGNITE-12356
>> >>> > > >> >
>> >>> > > >> > Regards,
>> >>> > > >> > Saikat
>> >>> > > >> >
>> >>> > > >> >
>> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
>> >>> > > >> > [hidden email]> wrote:
>> >>> > > >> >
>> >>> > > >> >> Hi all!
>> >>> > > >> >>
>> >>> > > >> >> This is an update. The integration is tested with mocks
>> based
>> >>> on
>> >>> > the
>> >>> > > >> >> original Ignite repo [1].
>> >>> > > >> >> I am currently waiting for the first commits on the new
>> >>> integration
>> >>> > > >> >> project
>> >>> > > >> >> to take place in order to create my pull request accordingly
>> >>> to the
>> >>> > > >> >> initial
>> >>> > > >> >> commits.
>> >>> > > >> >> In order to simulate the Pub/Sub server in a test I used a
>> test
>> >>> > > server
>> >>> > > >> >> implementation [2]. Unfortunately they don't have this util
>> >>> > uploaded
>> >>> > > >> in a
>> >>> > > >> >> maven repo, thus it needs to be hosted on a maven repo
>> (Apache
>> >>> > > License
>> >>> > > >> >> 2.0). Is there a repo for utilities that Ignite uses for the
>> >>> > > >> test/build?
>> >>> > > >> >> I have also created a ticket [3]
>> >>> > > >> >>
>> >>> > > >> >> Kind regards
>> >>> > > >> >>
>> >>> > > >> >> [1]
>> >>> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
>> >>> > > >> >> [2]
>> >>> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
>> >>> > > >> >> [3] https://issues.apache.org/jira/browse/IGNITE-12262
>> >>> > > >> >>
>> >>> > > >> >> *Emmanouil Gkatziouras*
>> >>> > > >> >> https://egkatzioura.com/ |
>> >>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
>> >>> > > >> >> https://github.com/gkatzioura
>> >>> > > >> >>
>> >>> > > >> >>
>> >>> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
>> >>> > [hidden email]
>> >>> > > >
>> >>> > > >> >> wrote:
>> >>> > > >> >>
>> >>> > > >> >> > Hello Ilya,
>> >>> > > >> >> >
>> >>> > > >> >> > Thank you for your email. Yes, before we remove any
>> >>> integration
>> >>> > we
>> >>> > > >> will
>> >>> > > >> >> > move them to another repository and release.
>> >>> > > >> >> >
>> >>> > > >> >> > We are thinking we will start with Pub/Sub Streamer
>> >>> > implementation
>> >>> > > >> and
>> >>> > > >> >> > continue to move other integration as mentioned in the
>> >>> following
>> >>> > > doc.
>> >>> > > >> >> >
>> >>> > > >> >> >
>> >>> > > >> >> >
>> >>> > > >> >>
>> >>> > > >>
>> >>> > >
>> >>> >
>> >>>
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> >>> > > >> >> >
>> >>> > > >> >> > Regards,
>> >>> > > >> >> > Saikat
>> >>> > > >> >> >
>> >>> > > >> >> >
>> >>> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
>> >>> > > >> >> > [hidden email]>
>> >>> > > >> >> > wrote:
>> >>> > > >> >> >
>> >>> > > >> >> > > Hello!
>> >>> > > >> >> > >
>> >>> > > >> >> > > My take on this: we want to offload some of
>> integrations,
>> >>> but
>> >>> > > >> please
>> >>> > > >> >> > commit
>> >>> > > >> >> > > and release them to another project first, and only
>> then we
>> >>> > will
>> >>> > > >> >> decide
>> >>> > > >> >> > to
>> >>> > > >> >> > > remove them from our repository.
>> >>> > > >> >> > >
>> >>> > > >> >> > > Please also make sure they are available with same
>> artifact
>> >>> > name
>> >>> > > >> and
>> >>> > > >> >> API.
>> >>> > > >> >> > > If this has to be changed, let's wait for AI 3.0
>> >>> > > >> >> > >
>> >>> > > >> >> > > Regards,
>> >>> > > >> >> > > --
>> >>> > > >> >> > > Ilya Kasnacheev
>> >>> > > >> >> > >
>> >>> > > >> >> > >
>> >>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <
>> >>> > > >> [hidden email]>:
>> >>> > > >> >> > >
>> >>> > > >> >> > > > Hello Emmanouil,
>> >>> > > >> >> > > >
>> >>> > > >> >> > > > As discussed earlier, I discussed with Apache Bahir
>> >>> community
>> >>> > > and
>> >>> > > >> >> they
>> >>> > > >> >> > > are
>> >>> > > >> >> > > > interested to have Apache Ignite extensions as part of
>> >>> Apache
>> >>> > > >> Bahir
>> >>> > > >> >> > > > project.
>> >>> > > >> >> > > >
>> >>> > > >> >> > > > Can you please share Pub/Sub streamer implementation
>> >>> details
>> >>> > > with
>> >>> > > >> >> > Apache
>> >>> > > >> >> > > > Bahir community and request for feedback.
>> >>> > > >> >> > > >
>> >>> > > >> >> > > > https://bahir.apache.org/
>> >>> > > >> >> > > >
>> >>> > > >> >> > > > I have also requested for contributor access in Jira
>> for
>> >>> > Apache
>> >>> > > >> >> Bahir
>> >>> > > >> >> > > > project so that I can create issues and assign to
>> myself.
>> >>> > > >> >> > > >
>> >>> > > >> >> > > > I can help with code reviews as well.
>> >>> > > >> >> > > >
>> >>> > > >> >> > > > Here is the email thread for reference
>> >>> > > >> >> > > >
>> >>> > > https://www.mail-archive.com/dev@.../msg02703.html
>> >>> > > >> >> > > >
>> >>> > > >> >> > > > Regards,
>> >>> > > >> >> > > > Saikat
>> >>> > > >> >> > > >
>> >>> > > >> >> > > >
>> >>> > > >> >> > > >
>> >>> > > >> >> > > >
>> >>> > > >> >> > > >
>> >>> > > >> >> > > >
>> >>> > > >> >> > > >
>> >>> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
>> >>> > > >> >> [hidden email]
>> >>> > > >> >> > >
>> >>> > > >> >> > > > wrote:
>> >>> > > >> >> > > >
>> >>> > > >> >> > > > > Hello,
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > > > As discussed I have created following discussion
>> >>> threads on
>> >>> > > our
>> >>> > > >> >> > > migration
>> >>> > > >> >> > > > > proposals for Apache Ignite extensions:
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > >
>> >>> > > >> >> > >
>> >>> > > >> >> >
>> >>> > > >> >>
>> >>> > > >>
>> >>> > >
>> >>> >
>> >>>
>> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > > >
>> >>> > > >>
>> https://www.mail-archive.com/dev@.../msg02703.html
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > > > I will share update as I hear more information.
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > > > Regards,
>> >>> > > >> >> > > > > Saikat
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
>> >>> > > >> >> > [hidden email]
>> >>> > > >> >> > > >
>> >>> > > >> >> > > > > wrote:
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > > >> Hello Denis,
>> >>> > > >> >> > > > >>
>> >>> > > >> >> > > > >> Sure, sounds good. I will create a separate
>> discussion
>> >>> > > thread
>> >>> > > >> to
>> >>> > > >> >> get
>> >>> > > >> >> > > > >> community feedback on whether we
>> >>> > > >> >> > > > >> should join the Bahir or kick-off "Ignite
>> Extensions"
>> >>> > > project.
>> >>> > > >> >> > > > >>
>> >>> > > >> >> > > > >> Regards,
>> >>> > > >> >> > > > >> Saikat
>> >>> > > >> >> > > > >>
>> >>> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <
>> >>> > > >> [hidden email]>
>> >>> > > >> >> > > wrote:
>> >>> > > >> >> > > > >>
>> >>> > > >> >> > > > >>> Folks,
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > > >>> The concept of Apache Bahir is precisely what we
>> >>> need!
>> >>> > > >> Saikat,
>> >>> > > >> >> > thanks
>> >>> > > >> >> > > > for
>> >>> > > >> >> > > > >>> doing the research. Why I believe the * idea*
>> fits us
>> >>> > well:
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > > >>>    - All integrations can be stored in separate
>> >>> Github
>> >>> > > >> >> repositories
>> >>> > > >> >> > > and
>> >>> > > >> >> > > > >>>    have their dev lifecycles. We've not obliged to
>> >>> couple
>> >>> > > all
>> >>> > > >> >> the
>> >>> > > >> >> > > > >>> integrations
>> >>> > > >> >> > > > >>>    in a single repo. For instance, Spark can be
>> >>> located
>> >>> > in
>> >>> > > a
>> >>> > > >> >> > > dedicated
>> >>> > > >> >> > > > >>> repo
>> >>> > > >> >> > > > >>>    while streaming integrations might be in a
>> single
>> >>> one.
>> >>> > > >> It's
>> >>> > > >> >> up
>> >>> > > >> >> > to
>> >>> > > >> >> > > > us.
>> >>> > > >> >> > > > >>>    - All the repositories and integrations belong
>> to
>> >>> ASF.
>> >>> > > >> We're
>> >>> > > >> >> not
>> >>> > > >> >> > > > >>> dumping
>> >>> > > >> >> > > > >>>    them on Github but rather keep supporting and
>> >>> > developing
>> >>> > > >> in
>> >>> > > >> >> > > > >>> accordance with
>> >>> > > >> >> > > > >>>    ASF vision and practices.
>> >>> > > >> >> > > > >>>    - There is a way to reward contributors via
>> >>> > > committership
>> >>> > > >> and
>> >>> > > >> >> > PMC
>> >>> > > >> >> > > > >>>    membership. You won't get this if a project is
>> >>> just
>> >>> > one
>> >>> > > of
>> >>> > > >> >> the
>> >>> > > >> >> > > > >>> millions of
>> >>> > > >> >> > > > >>>    Github projects.
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > > >>> Saikat, as Ignite PMC member, could you please
>> >>> kick-off a
>> >>> > > >> >> separate
>> >>> > > >> >> > > > >>> dev-list
>> >>> > > >> >> > > > >>> discussion to involve more community members and
>> >>> > referring
>> >>> > > to
>> >>> > > >> >> this
>> >>> > > >> >> > > > >>> thread?
>> >>> > > >> >> > > > >>> I think the primary question the community needs
>> to
>> >>> > answer
>> >>> > > >> >> whether
>> >>> > > >> >> > we
>> >>> > > >> >> > > > >>> should join the Bahir or kick-off "Ignite
>> Extensions"
>> >>> > > >> project?
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > > >>> -
>> >>> > > >> >> > > > >>> Denis
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev <
>> >>> > > >> >> > > > [hidden email]>
>> >>> > > >> >> > > > >>> wrote:
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > > >>> > Maybe we could move all our Streaming
>> Integrations
>> >>> > there,
>> >>> > > >> but
>> >>> > > >> >> > what
>> >>> > > >> >> > > is
>> >>> > > >> >> > > > >>> about
>> >>> > > >> >> > > > >>> > maintaining and committer permissions to the new
>> >>> > > >> repositories?
>> >>> > > >> >> > > > >>> > I see the list of the committers and PMC members
>> >>> there
>> >>> > > >> >> > > > >>> > https://bahir.apache.org/community-members/
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>> > Could somebody from Ignite community be added to
>> >>> this
>> >>> > > list
>> >>> > > >> as
>> >>> > > >> >> a
>> >>> > > >> >> > > > >>> > PMC/committer to maintain Ignite integrations?
>> >>> > > >> >> > > > >>> > If yes, I'd happy to join this project and
>> >>> collaborate
>> >>> > > with
>> >>> > > >> >> these
>> >>> > > >> >> > > > guys
>> >>> > > >> >> > > > >>> > together
>> >>> > > >> >> > > > >>> > If no, and we should start from the zero level
>> with
>> >>> > > >> external
>> >>> > > >> >> PRs
>> >>> > > >> >> > -
>> >>> > > >> >> > > > >>> hmmm,
>> >>> > > >> >> > > > >>> > it's better to have our own external repository
>> >>> with
>> >>> > > >> >> ApacheBahirr
>> >>> > > >> >> > > > >>> ideology.
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>> > Also, I agree, that all connectors could be
>> moved
>> >>> there
>> >>> > > (in
>> >>> > > >> >> > > > >>> ApacheBahirr
>> >>> > > >> >> > > > >>> > like repository), but not all modules from the
>> page
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > >
>> >>> > > >> >> > >
>> >>> > > >> >> >
>> >>> > > >> >>
>> >>> > > >>
>> >>> > >
>> >>> >
>> >>>
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> >>> > > >> >> > > > >>> > because
>> >>> > > >> >> > > > >>> > they use different parts of core modules and
>> could
>> >>> be
>> >>> > > >> >> developed
>> >>> > > >> >> > > with
>> >>> > > >> >> > > > >>> > different velocity.
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>> > In reality, before creation of new repositories
>> we
>> >>> need
>> >>> > > >> wide
>> >>> > > >> >> > > > discussion
>> >>> > > >> >> > > > >>> > based on the document mentioned above.
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>> > P.S. Of course, we could experiment in the next
>> >>> release
>> >>> > > 2.8
>> >>> > > >> >> with
>> >>> > > >> >> > > one
>> >>> > > >> >> > > > >>> or two
>> >>> > > >> >> > > > >>> > integrations like twitter or ZeroMQ
>> >>> > > >> >> > > > >>> > Also, I'd like Denis Magda idea for the separate
>> >>> > > >> repositories
>> >>> > > >> >> in
>> >>> > > >> >> > > > Apache
>> >>> > > >> >> > > > >>> > Github to maintain them by anybody.
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra <
>> >>> > > >> >> > > [hidden email]
>> >>> > > >> >> > > > >:
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>> > > Hi Denis, Emmanouil
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> > > Thank you for your email. I think creating a
>> >>> separate
>> >>> > > >> >> > integration
>> >>> > > >> >> > > > >>> project
>> >>> > > >> >> > > > >>> > > in github and also proposing it as an apache
>> >>> > incubator
>> >>> > > >> >> project
>> >>> > > >> >> > > will
>> >>> > > >> >> > > > >>> be a
>> >>> > > >> >> > > > >>> > > good move. The other separate integration
>> modules
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > >
>> >>> > > >> >> > >
>> >>> > > >> >> >
>> >>> > > >> >>
>> >>> > > >>
>> >>> > >
>> >>> >
>> >>>
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> >>> > > >> >> > > > >>> > > can
>> >>> > > >> >> > > > >>> > > be moved to this new apache incubator project.
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> > > There are similar integration available for
>> >>> Flink and
>> >>> > > >> Spark
>> >>> > > >> >> in
>> >>> > > >> >> > > > Apache
>> >>> > > >> >> > > > >>> > Bahir
>> >>> > > >> >> > > > >>> > > https://bahir.apache.org/
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> > > Do you think we should reach out to Apache
>> Bahir
>> >>> > > >> community
>> >>> > > >> >> > with a
>> >>> > > >> >> > > > >>> > proposal
>> >>> > > >> >> > > > >>> > > or we should plan to create a separate Apache
>> >>> > Incubator
>> >>> > > >> >> > project?
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> > > Regards,
>> >>> > > >> >> > > > >>> > > Saikat
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil
>> >>> > Gkatziouras <
>> >>> > > >> >> > > > >>> > > [hidden email]>
>> >>> > > >> >> > > > >>> > > wrote:
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> > > > Hi all!
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > > > Based on the discussions most probably this
>> is
>> >>> > going
>> >>> > > to
>> >>> > > >> >> be on
>> >>> > > >> >> > > > >>> another
>> >>> > > >> >> > > > >>> > > > GitHub repo. Therefore I will prepare a
>> >>> standalone
>> >>> > > >> project
>> >>> > > >> >> > with
>> >>> > > >> >> > > > the
>> >>> > > >> >> > > > >>> > > Pub/Sub
>> >>> > > >> >> > > > >>> > > > module and once the repository is created a
>> >>> pull
>> >>> > > >> request
>> >>> > > >> >> > shall
>> >>> > > >> >> > > be
>> >>> > > >> >> > > > >>> > issued.
>> >>> > > >> >> > > > >>> > > > If there is anything else I can do in order
>> to
>> >>> > > >> facilitate
>> >>> > > >> >> > this
>> >>> > > >> >> > > > >>> please
>> >>> > > >> >> > > > >>> > let
>> >>> > > >> >> > > > >>> > > > me know.
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > > > If a ticket is created for this issue, I
>> shall
>> >>> be
>> >>> > > able
>> >>> > > >> to
>> >>> > > >> >> > > create
>> >>> > > >> >> > > > a
>> >>> > > >> >> > > > >>> pull
>> >>> > > >> >> > > > >>> > > > request and thus a build will take place on
>> >>> > ignite's
>> >>> > > >> CI.
>> >>> > > >> >> > > > >>> > > > If everyone is aligned with adding a Pub/Sub
>> >>> > > >> >> implementation
>> >>> > > >> >> > > may I
>> >>> > > >> >> > > > >>> > create
>> >>> > > >> >> > > > >>> > > a
>> >>> > > >> >> > > > >>> > > > ticket on JIRA?
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > > > My next question has to do on how Ignite
>> >>> configures
>> >>> > > the
>> >>> > > >> >> build
>> >>> > > >> >> > > > jobs
>> >>> > > >> >> > > > >>> in
>> >>> > > >> >> > > > >>> > > Team
>> >>> > > >> >> > > > >>> > > > City. The unit tests for my implementation
>> >>> need a
>> >>> > > local
>> >>> > > >> >> > Pub/Sub
>> >>> > > >> >> > > > >>> server
>> >>> > > >> >> > > > >>> > up
>> >>> > > >> >> > > > >>> > > > and running (they provide one for testing
>> >>> > purposes).
>> >>> > > >> Will
>> >>> > > >> >> > this
>> >>> > > >> >> > > > take
>> >>> > > >> >> > > > >>> > > effect
>> >>> > > >> >> > > > >>> > > > in a form of a build script inside my
>> >>> > implementation
>> >>> > > or
>> >>> > > >> >> is it
>> >>> > > >> >> > > > >>> something
>> >>> > > >> >> > > > >>> > > > that should be configured on Team City?
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > > > Kind regards,
>> >>> > > >> >> > > > >>> > > > Emmanouil
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
>> >>> > > >> >> > > > >>> > > > https://egkatzioura.com/ |
>> >>> > > >> >> > > > >>> > > >
>> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> >>> > > >> >> > > > >>> > > > https://github.com/gkatzioura
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis Magda <
>> >>> > > >> >> [hidden email]
>> >>> > > >> >> > >
>> >>> > > >> >> > > > >>> wrote:
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > > After contemplating on this, let me
>> propose
>> >>> > another
>> >>> > > >> way
>> >>> > > >> >> > with
>> >>> > > >> >> > > > this
>> >>> > > >> >> > > > >>> > > > > particular integration that is aligned
>> with
>> >>> our
>> >>> > > >> >> > > modularization
>> >>> > > >> >> > > > >>> > > endeavor.
>> >>> > > >> >> > > > >>> > > > > The modularization [1] defines the Ignite
>> >>> core,
>> >>> > > >> Ignite
>> >>> > > >> >> > > modules,
>> >>> > > >> >> > > > >>> and
>> >>> > > >> >> > > > >>> > > > > independent integrations. The first two
>> are
>> >>> to be
>> >>> > > >> >> located
>> >>> > > >> >> > in
>> >>> > > >> >> > > > >>> Ignite
>> >>> > > >> >> > > > >>> > > > > repositories and to be tested/updated by
>> the
>> >>> > > >> community
>> >>> > > >> >> for
>> >>> > > >> >> > > > every
>> >>> > > >> >> > > > >>> > > release.
>> >>> > > >> >> > > > >>> > > > > The modules are a subset of existing
>> >>> integrations
>> >>> > > >> that
>> >>> > > >> >> are
>> >>> > > >> >> > > > >>> adopted
>> >>> > > >> >> > > > >>> > > widely
>> >>> > > >> >> > > > >>> > > > > and require community attention.
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > > As for the independent integrations [1],
>> >>> those
>> >>> > are
>> >>> > > >> >> supposed
>> >>> > > >> >> > > to
>> >>> > > >> >> > > > be
>> >>> > > >> >> > > > >>> > > located
>> >>> > > >> >> > > > >>> > > > > in a separate GitHub repo. I would advise
>> us
>> >>> to
>> >>> > > start
>> >>> > > >> >> > > > >>> implementing
>> >>> > > >> >> > > > >>> > this
>> >>> > > >> >> > > > >>> > > > > concept with the Pub/Sub integration. Even
>> >>> though
>> >>> > > it
>> >>> > > >> >> will
>> >>> > > >> >> > be
>> >>> > > >> >> > > > >>> located
>> >>> > > >> >> > > > >>> > > in a
>> >>> > > >> >> > > > >>> > > > > separate Github repo, it will be listed
>> among
>> >>> > > >> officially
>> >>> > > >> >> > > > >>> available
>> >>> > > >> >> > > > >>> > > > > integrations on the Ignite website
>> featuring
>> >>> > > >> Emmanouil
>> >>> > > >> >> as a
>> >>> > > >> >> > > > >>> > > contributor.
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > > What do you think?
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > > [1]
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > >
>> >>> > > >> >> > >
>> >>> > > >> >> >
>> >>> > > >> >>
>> >>> > > >>
>> >>> > >
>> >>> >
>> >>>
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > > -
>> >>> > > >> >> > > > >>> > > > > Denis
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat
>> >>> Maitra <
>> >>> > > >> >> > > > >>> > [hidden email]
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > > > > wrote:
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > > > Hi Emmanouil,
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > > > The changes looks good to me. I wanted
>> to
>> >>> check
>> >>> > > if
>> >>> > > >> you
>> >>> > > >> >> > are
>> >>> > > >> >> > > > also
>> >>> > > >> >> > > > >>> > able
>> >>> > > >> >> > > > >>> > > to
>> >>> > > >> >> > > > >>> > > > > > validate that the once data is added to
>> >>> Ignite
>> >>> > > >> cluster
>> >>> > > >> >> > you
>> >>> > > >> >> > > > are
>> >>> > > >> >> > > > >>> also
>> >>> > > >> >> > > > >>> > > > able
>> >>> > > >> >> > > > >>> > > > > to
>> >>> > > >> >> > > > >>> > > > > > access it using another Ignite client or
>> >>> using
>> >>> > > rest
>> >>> > > >> >> > > > endpoints.
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > > > We use Teamcity for CI process, the
>> >>> details are
>> >>> > > as
>> >>> > > >> >> > follows
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > > > Trigger validation of those test suites
>> >>> that
>> >>> > have
>> >>> > > >> been
>> >>> > > >> >> > > > >>> affected by
>> >>> > > >> >> > > > >>> > > your
>> >>> > > >> >> > > > >>> > > > > > changes on TeamCity <
>> >>> > > http://ci.ignite.apache.org/>:
>> >>> > > >> <
>> >>> > > >> >> > > > >>> > > > > http://204.14.53.153/
>> >>> > > >> >> > > > >>> > > > > > >
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > > >    - Locate a test suite you have to
>> check,
>> >>> > press
>> >>> > > >> >> button
>> >>> > > >> >> > > > named
>> >>> > > >> >> > > > >>> > "..."
>> >>> > > >> >> > > > >>> > > > that
>> >>> > > >> >> > > > >>> > > > > >    is located on the left of "Run"
>> button.
>> >>> "Run
>> >>> > > >> custom
>> >>> > > >> >> > > build"
>> >>> > > >> >> > > > >>> > dialog
>> >>> > > >> >> > > > >>> > > > will
>> >>> > > >> >> > > > >>> > > > > >    appear;
>> >>> > > >> >> > > > >>> > > > > >    - Go to "Changes" tab and choose
>> >>> > > >> >> > > > >>> > "pull/<pull-request-number>/head"
>> >>> > > >> >> > > > >>> > > > in
>> >>> > > >> >> > > > >>> > > > > >    "Build branch" dropdown list;
>> >>> > > >> >> > > > >>> > > > > >    - Press "Run build" button and
>> monitor
>> >>> tests
>> >>> > > >> >> results.
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > > > @Roman [hidden email] - can you also
>> >>> please
>> >>> > > >> take a
>> >>> > > >> >> > > look?
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > > > Regards,
>> >>> > > >> >> > > > >>> > > > > > Saikat
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM
>> Emmanouil
>> >>> > > >> Gkatziouras
>> >>> > > >> >> <
>> >>> > > >> >> > > > >>> > > > > > [hidden email]>
>> >>> > > >> >> > > > >>> > > > > > wrote:
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
>> >>> > > >> >> > > > >>> > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > I have my implementation on an Ignite
>> >>> fork on
>> >>> > > my
>> >>> > > >> >> > personal
>> >>> > > >> >> > > > >>> GitHub
>> >>> > > >> >> > > > >>> > > > > account
>> >>> > > >> >> > > > >>> > > > > > > [1]
>> >>> > > >> >> > > > >>> > > > > > > It is based on the Kafka
>> implementation
>> >>> which
>> >>> > > is
>> >>> > > >> of
>> >>> > > >> >> a
>> >>> > > >> >> > > > polling
>> >>> > > >> >> > > > >>> > > nature.
>> >>> > > >> >> > > > >>> > > > > > > GCP does not provide a service like
>> >>> Kafka.
>> >>> > > >> Instead
>> >>> > > >> >> it
>> >>> > > >> >> > > > >>> provides us
>> >>> > > >> >> > > > >>> > > > with
>> >>> > > >> >> > > > >>> > > > > > > Pub/Sub which is just like Kafka.
>> >>> > > >> >> > > > >>> > > > > > > Therefore using the Kafka streamer as
>> a
>> >>> > > guideline
>> >>> > > >> >> made
>> >>> > > >> >> > > more
>> >>> > > >> >> > > > >>> sense
>> >>> > > >> >> > > > >>> > > to
>> >>> > > >> >> > > > >>> > > > > me.
>> >>> > > >> >> > > > >>> > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > Apart from the implementation I have
>> also
>> >>> > > added a
>> >>> > > >> >> unit
>> >>> > > >> >> > > > test.
>> >>> > > >> >> > > > >>> > > > > > > It definitely needs some polishing
>> and a
>> >>> > double
>> >>> > > >> >> check
>> >>> > > >> >> > on
>> >>> > > >> >> > > my
>> >>> > > >> >> > > > >>> side
>> >>> > > >> >> > > > >>> > to
>> >>> > > >> >> > > > >>> > > > > make
>> >>> > > >> >> > > > >>> > > > > > > the review smooth.
>> >>> > > >> >> > > > >>> > > > > > > For the tests a local Pub/Sub [2]
>> server
>> >>> has
>> >>> > to
>> >>> > > >> be
>> >>> > > >> >> > setup
>> >>> > > >> >> > > > >>> however
>> >>> > > >> >> > > > >>> > I
>> >>> > > >> >> > > > >>> > > am
>> >>> > > >> >> > > > >>> > > > > not
>> >>> > > >> >> > > > >>> > > > > > > sure which is the best way to do so
>> with
>> >>> > > regards
>> >>> > > >> to
>> >>> > > >> >> > your
>> >>> > > >> >> > > > >>> CI/CD.
>> >>> > > >> >> > > > >>> > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > [1]
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> >
>> >>> https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
>> >>> > > >> >> > > > >>> > > > > > > [2]
>> >>> > > >> >> > > > >>>
>> >>> > https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
>> >>> > > >> >> > > > >>> > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > *Emmanouil Gkatziouras*
>> >>> > > >> >> > > > >>> > > > > > > https://egkatzioura.com/ |
>> >>> > > >> >> > > > >>> > > > > > >
>> >>> > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
>> >>> > > >> >> > > > >>> > > > > > > https://github.com/gkatzioura
>> >>> > > >> >> > > > >>> > > > > > >
>> >>> > > >> >> > > > >>> > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11, Saikat
>> >>> Maitra <
>> >>> > > >> >> > > > >>> > > [hidden email]
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > > > > wrote:
>> >>> > > >> >> > > > >>> > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > Hello Emmanouil,
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > Can you please share more info about
>> >>> the
>> >>> > > >> >> > > implementation?
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > We have similar implementation for
>> data
>> >>> > > >> streamers
>> >>> > > >> >> > with
>> >>> > > >> >> > > > >>> Kafka
>> >>> > > >> >> > > > >>> > and
>> >>> > > >> >> > > > >>> > > > > > RocketMQ
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > https://apacheignite-mix.readme.io/docs/kafka-streamer
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > >
>> >>> https://apacheignite-mix.readme.io/docs/rocketmq-streamer
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > Please review and share if you have
>> any
>> >>> > > >> questions.
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > Regards,
>> >>> > > >> >> > > > >>> > > > > > > > Saikat
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56 PM
>> >>> Emmanouil
>> >>> > > >> >> > Gkatziouras <
>> >>> > > >> >> > > > >>> > > > > > > > [hidden email]>
>> >>> > > >> >> > > > >>> > > > > > > > wrote:
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > Hi all,
>> >>> > > >> >> > > > >>> > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > By mentioning Pub/Sub I mean
>> Google
>> >>> > Cloud's
>> >>> > > >> >> Pub/Sub
>> >>> > > >> >> > > [1]
>> >>> > > >> >> > > > >>> > > > > > > > > Yes it is indeed cloud specific.
>> It
>> >>> is
>> >>> > the
>> >>> > > >> >> > messaging
>> >>> > > >> >> > > > >>> service
>> >>> > > >> >> > > > >>> > > > > provided
>> >>> > > >> >> > > > >>> > > > > > > by
>> >>> > > >> >> > > > >>> > > > > > > > > GCP.
>> >>> > > >> >> > > > >>> > > > > > > > > I have made the integration with
>> it,
>> >>> but
>> >>> > I
>> >>> > > >> >> haven't
>> >>> > > >> >> > > > >>> published
>> >>> > > >> >> > > > >>> > it
>> >>> > > >> >> > > > >>> > > > yet
>> >>> > > >> >> > > > >>> > > > > > (I
>> >>> > > >> >> > > > >>> > > > > > > am
>> >>> > > >> >> > > > >>> > > > > > > > > waiting for the discussion/ticket
>> and
>> >>> > > proceed
>> >>> > > >> >> with
>> >>> > > >> >> > > the
>> >>> > > >> >> > > > >>> > > > > contribution).
>> >>> > > >> >> > > > >>> > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > Kind regards,
>> >>> > > >> >> > > > >>> > > > > > > > > Emmanouil
>> >>> > > >> >> > > > >>> > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > [1]
>> >>> > > >> >> https://cloud.google.com/pubsub/docs/overview
>> >>> > > >> >> > > > >>> > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
>> >>> > > >> >> > > > >>> > > > > > > > > https://egkatzioura.com/ |
>> >>> > > >> >> > > > >>> > > > > > > > >
>> >>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
>> >>> > > >> >> > > > >>> > > > > > > > > https://github.com/gkatzioura
>> >>> > > >> >> > > > >>> > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at 21:45,
>> Denis
>> >>> > Magda <
>> >>> > > >> >> > > > >>> [hidden email]
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> > > > > wrote:
>> >>> > > >> >> > > > >>> > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > Hello Emmanouil,
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > Are you referring to any
>> specific
>> >>> > > >> >> implementation
>> >>> > > >> >> > > > >>> tailored
>> >>> > > >> >> > > > >>> > > for a
>> >>> > > >> >> > > > >>> > > > > > > cloud?
>> >>> > > >> >> > > > >>> > > > > > > > If
>> >>> > > >> >> > > > >>> > > > > > > > > > I'm not mistaken you were
>> >>> mentioning
>> >>> > > >> AWS/GCE.
>> >>> > > >> >> Is
>> >>> > > >> >> > > > there
>> >>> > > >> >> > > > >>> any
>> >>> > > >> >> > > > >>> > > > Github
>> >>> > > >> >> > > > >>> > > > > > > > source
>> >>> > > >> >> > > > >>> > > > > > > > > > code?
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > Here is a list of existing
>> >>> integrations
>> >>> > > >> with
>> >>> > > >> >> > > > streaming
>> >>> > > >> >> > > > >>> and
>> >>> > > >> >> > > > >>> > > > > pub/sub
>> >>> > > >> >> > > > >>> > > > > > > > > > frameworks:
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> https://apacheignite-mix.readme.io/docs/overview
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > with the source code in Ignite
>> >>> master:
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > https://github.com/apache/ignite/tree/master/modules
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > Check out those resources to see
>> >>> what
>> >>> > are
>> >>> > > >> the
>> >>> > > >> >> > most
>> >>> > > >> >> > > > >>> suitable
>> >>> > > >> >> > > > >>> > > > > Ignite
>> >>> > > >> >> > > > >>> > > > > > > APIs
>> >>> > > >> >> > > > >>> > > > > > > > > for
>> >>> > > >> >> > > > >>> > > > > > > > > > such integrations.
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > @Saikat Maitra <
>> >>> > [hidden email]>
>> >>> > > ,
>> >>> > > >> >> > @Roman
>> >>> > > >> >> > > <
>> >>> > > >> >> > > > >>> > > > > > [hidden email]
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > ,
>> >>> > > >> >> > > > >>> > > > > > > > > as
>> >>> > > >> >> > > > >>> > > > > > > > > > streaming technologies
>> >>> > > >> >> contributors/maintainers,
>> >>> > > >> >> > > > could
>> >>> > > >> >> > > > >>> you
>> >>> > > >> >> > > > >>> > > > please
>> >>> > > >> >> > > > >>> > > > > > > join
>> >>> > > >> >> > > > >>> > > > > > > > > this
>> >>> > > >> >> > > > >>> > > > > > > > > > thread?
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > -
>> >>> > > >> >> > > > >>> > > > > > > > > > Denis
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > On Thu, Oct 10, 2019 at 7:58 AM
>> >>> > Emmanouil
>> >>> > > >> >> > > > Gkatziouras <
>> >>> > > >> >> > > > >>> > > > > > > > > > [hidden email]>
>> >>> > > >> >> > > > >>> > > > > > > > > > wrote:
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > > Hello everyone,
>> >>> > > >> >> > > > >>> > > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > > I started using Ignite lately.
>> >>> Part
>> >>> > of
>> >>> > > my
>> >>> > > >> >> work
>> >>> > > >> >> > > > >>> involves
>> >>> > > >> >> > > > >>> > > > > Pub/Sub,
>> >>> > > >> >> > > > >>> > > > > > > > thus I
>> >>> > > >> >> > > > >>> > > > > > > > > > > created a pub/sub streamer.
>> >>> > > >> >> > > > >>> > > > > > > > > > > Pub/Sub is pretty close to
>> Kafka.
>> >>> > > >> >> > > > >>> > > > > > > > > > > Here's how the streamer works.
>> >>> Every
>> >>> > > node
>> >>> > > >> >> > creates
>> >>> > > >> >> > > > an
>> >>> > > >> >> > > > >>> > > instance
>> >>> > > >> >> > > > >>> > > > > of
>> >>> > > >> >> > > > >>> > > > > > a
>> >>> > > >> >> > > > >>> > > > > > > > > > Pub/Sub
>> >>> > > >> >> > > > >>> > > > > > > > > > > client.
>> >>> > > >> >> > > > >>> > > > > > > > > > > The clients subscribe to the
>> same
>> >>> > topic
>> >>> > > >> >> using
>> >>> > > >> >> > the
>> >>> > > >> >> > > > >>> same
>> >>> > > >> >> > > > >>> > > > > > subscription
>> >>> > > >> >> > > > >>> > > > > > > > > name.
>> >>> > > >> >> > > > >>> > > > > > > > > > > A message is being sent and
>> one
>> >>> of
>> >>> > the
>> >>> > > >> nodes
>> >>> > > >> >> > will
>> >>> > > >> >> > > > >>> receive
>> >>> > > >> >> > > > >>> > > the
>> >>> > > >> >> > > > >>> > > > > > > > message,
>> >>> > > >> >> > > > >>> > > > > > > > > > > process it and then forward
>> it to
>> >>> > > Ignite
>> >>> > > >> >> using
>> >>> > > >> >> > > the
>> >>> > > >> >> > > > >>> > > extractor
>> >>> > > >> >> > > > >>> > > > > > > > provided.
>> >>> > > >> >> > > > >>> > > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > > Any thoughts?
>> >>> > > >> >> > > > >>> > > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > > *Emmanouil Gkatziouras*
>> >>> > > >> >> > > > >>> > > > > > > > > > > https://egkatzioura.com/ |
>> >>> > > >> >> > > > >>> > > > > > > > > > >
>> >>> > > >> >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> >>> > > >> >> > > > >>> > > > > > > > > > > https://github.com/gkatzioura
>> >>> > > >> >> > > > >>> > > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > >
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > > >>
>> >>> > > >> >> > > >
>> >>> > > >> >> > >
>> >>> > > >> >> >
>> >>> > > >> >>
>> >>> > > >> >
>> >>> > > >>
>> >>> > > >
>> >>> > >
>> >>> >
>> >>>
>> >>
>>
>

[Saikat Maitra]

Hi Emmanouil,

I have added you as contributor in Ignite Extensions project. Can you
please check and confirm if you are able to see the project and execute
build on your pull request.

Regards,
Saikat

On Tue, Nov 26, 2019 at 12:49 PM Emmanouil Gkatziouras <[hidden email]>
wrote:

> Hi all!
>
> If someone could give me read access to the Job of Ignite Extensions on
> Team City will greatly help me (username gkatzioura).
> I suppose the builds get triggered automatically on a pull request.
>
> Kind regards,
> Emmanouil
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras <[hidden email]>
> wrote:
>
> > Hi Saikat!
> >
> > I just rebased with the flink branch. Unfortunately I do not have read
> > access to the team city link you provided and thus evaluate the tests for
> > my pull request.
> > I guess it has to do with ignite extensions being new.
> > My username is gkatzioura and email the one I am using on this email
> >
> > Thank you for your time.
> >
> >
> > *Emmanouil Gkatziouras*
> > https://egkatzioura.com/ |
> > https://www.linkedin.com/in/gkatziourasemmanouil/
> > https://github.com/gkatzioura
> >
> >
> > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <[hidden email]>
> > wrote:
> >
> >> Hi Emmanouil,
> >>
> >> The latest build on teamcity has passed on Flink pull request.
> >>
> >>
> >>
> https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> >>
> >> you should be able to take the changes I made in my PR and run build on
> >> your pull request.
> >>
> >> Please let me know if you have any questions.
> >>
> >> Regards,
> >> Saikat
> >>
> >> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <[hidden email]
> >
> >> wrote:
> >>
> >> > Hi Emmanouil,
> >> >
> >> > I have fixed the Flink module testsuites and have setup new project in
> >> > teamcity.
> >> >
> >> >
> >> >
> >>
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
> >> >
> >> > I am looking into setting up the build.
> >> >
> >> > Regards,
> >> > Saikat
> >> >
> >> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <
> [hidden email]>
> >> > wrote:
> >> >
> >> >> Hi Emmanouil,
> >> >>
> >> >> Thank you for your email. Yes, the plan is once the Flink PR is
> merged
> >> it
> >> >> will provide the Licence, parent POMs etc and you can rebase from
> >> master
> >> >> branch and apply the changes on top of it.
> >> >>
> >> >> I am getting some issues with test failures in local with
> >> >> GridTestProperties as the test.properties is not present in this new
> >> >> project but should be available from dependencies.
> >> >>
> >> >> Once I address this issue, I will go ahead and merge the changes and
> >> then
> >> >> we can take it from there.
> >> >>
> >> >> Regards,
> >> >> Saikat
> >> >>
> >> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
> >> >> [hidden email]> wrote:
> >> >>
> >> >>> Hi all,
> >> >>>
> >> >>> I made my first pull request [1]. Since this project is brand new
> (no
> >> >>> parent poms, licensing), I reckoned it was best to use Saikat's
> >> branch on
> >> >>> Flink.
> >> >>> I suppose the Flink branch will be merged first. If not please give
> me
> >> >>> guidelines on how I should proceed next.
> >> >>>
> >> >>> Kind regards
> >> >>> Emmanouil
> >> >>>
> >> >>> [1] https://github.com/apache/ignite-extensions/pull/2
> >> >>>
> >> >>> *Emmanouil Gkatziouras*
> >> >>> https://egkatzioura.com/ |
> >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> >>> https://github.com/gkatzioura
> >> >>>
> >> >>>
> >> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <[hidden email]>
> wrote:
> >> >>>
> >> >>> > Awesome, ping us whenever you're ready!
> >> >>> >
> >> >>> > -
> >> >>> > Denis
> >> >>> >
> >> >>> >
> >> >>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
> >> >>> > [hidden email]>
> >> >>> > wrote:
> >> >>> >
> >> >>> > > Hi all!
> >> >>> > >
> >> >>> > > I am sorry for being late on that. I was trying to refactor the
> >> test
> >> >>> in
> >> >>> > > order not to be in need of any external tools or spinning up a
> >> >>> server.
> >> >>> > > I did forked the new repo and indeed my changes there, so a pull
> >> >>> request
> >> >>> > is
> >> >>> > > a matter of time!
> >> >>> > >
> >> >>> > > Kind regards
> >> >>> > >
> >> >>> > > *Emmanouil Gkatziouras*
> >> >>> > > https://egkatzioura.com/ |
> >> >>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> >>> > > https://github.com/gkatzioura
> >> >>> > >
> >> >>> > >
> >> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <[hidden email]>
> >> wrote:
> >> >>> > >
> >> >>> > > > Hi Emmanouil,
> >> >>> > > >
> >> >>> > > > Do you have any questions or need any support from the
> >> community?
> >> >>> > > >
> >> >>> > > > -
> >> >>> > > > Denis
> >> >>> > > >
> >> >>> > > >
> >> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
> >> >>> [hidden email]
> >> >>> > >
> >> >>> > > > wrote:
> >> >>> > > >
> >> >>> > > >> Hi Emmanouil,
> >> >>> > > >>
> >> >>> > > >> Can you please take a looks at dev utils, if this is
> something
> >> >>> you are
> >> >>> > > >> looking for
> >> >>> > > >>
> https://github.com/apache/ignite/tree/master/modules/dev-utils
> >> ?
> >> >>> > > >>
> >> >>> > > >> IMO, if you can release Pub/Sub server in maven and then use
> >> it as
> >> >>> > > >> dependency, that would be great.
> >> >>> > > >>
> >> >>> > > >> Regards,
> >> >>> > > >> Saikat
> >> >>> > > >>
> >> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
> >> >>> > [hidden email]>
> >> >>> > > >> wrote:
> >> >>> > > >>
> >> >>> > > >> > Hi Emmanouil,
> >> >>> > > >> >
> >> >>> > > >> > The master branch has been initialized. I have raised a PR
> to
> >> >>> > migrate
> >> >>> > > >> > Flink module to ignite-extensions.
> >> >>> > > >> >
> >> >>> > > >> > PR https://github.com/apache/ignite-extensions/pull/1
> >> >>> > > >> > Jira : https://issues.apache.org/jira/browse/IGNITE-12356
> >> >>> > > >> >
> >> >>> > > >> > Regards,
> >> >>> > > >> > Saikat
> >> >>> > > >> >
> >> >>> > > >> >
> >> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
> >> >>> > > >> > [hidden email]> wrote:
> >> >>> > > >> >
> >> >>> > > >> >> Hi all!
> >> >>> > > >> >>
> >> >>> > > >> >> This is an update. The integration is tested with mocks
> >> based
> >> >>> on
> >> >>> > the
> >> >>> > > >> >> original Ignite repo [1].
> >> >>> > > >> >> I am currently waiting for the first commits on the new
> >> >>> integration
> >> >>> > > >> >> project
> >> >>> > > >> >> to take place in order to create my pull request
> accordingly
> >> >>> to the
> >> >>> > > >> >> initial
> >> >>> > > >> >> commits.
> >> >>> > > >> >> In order to simulate the Pub/Sub server in a test I used a
> >> test
> >> >>> > > server
> >> >>> > > >> >> implementation [2]. Unfortunately they don't have this
> util
> >> >>> > uploaded
> >> >>> > > >> in a
> >> >>> > > >> >> maven repo, thus it needs to be hosted on a maven repo
> >> (Apache
> >> >>> > > License
> >> >>> > > >> >> 2.0). Is there a repo for utilities that Ignite uses for
> the
> >> >>> > > >> test/build?
> >> >>> > > >> >> I have also created a ticket [3]
> >> >>> > > >> >>
> >> >>> > > >> >> Kind regards
> >> >>> > > >> >>
> >> >>> > > >> >> [1]
> >> >>> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> >> >>> > > >> >> [2]
> >> >>> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> >> >>> > > >> >> [3] https://issues.apache.org/jira/browse/IGNITE-12262
> >> >>> > > >> >>
> >> >>> > > >> >> *Emmanouil Gkatziouras*
> >> >>> > > >> >> https://egkatzioura.com/ |
> >> >>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> >>> > > >> >> https://github.com/gkatzioura
> >> >>> > > >> >>
> >> >>> > > >> >>
> >> >>> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
> >> >>> > [hidden email]
> >> >>> > > >
> >> >>> > > >> >> wrote:
> >> >>> > > >> >>
> >> >>> > > >> >> > Hello Ilya,
> >> >>> > > >> >> >
> >> >>> > > >> >> > Thank you for your email. Yes, before we remove any
> >> >>> integration
> >> >>> > we
> >> >>> > > >> will
> >> >>> > > >> >> > move them to another repository and release.
> >> >>> > > >> >> >
> >> >>> > > >> >> > We are thinking we will start with Pub/Sub Streamer
> >> >>> > implementation
> >> >>> > > >> and
> >> >>> > > >> >> > continue to move other integration as mentioned in the
> >> >>> following
> >> >>> > > doc.
> >> >>> > > >> >> >
> >> >>> > > >> >> >
> >> >>> > > >> >> >
> >> >>> > > >> >>
> >> >>> > > >>
> >> >>> > >
> >> >>> >
> >> >>>
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> >>> > > >> >> >
> >> >>> > > >> >> > Regards,
> >> >>> > > >> >> > Saikat
> >> >>> > > >> >> >
> >> >>> > > >> >> >
> >> >>> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
> >> >>> > > >> >> > [hidden email]>
> >> >>> > > >> >> > wrote:
> >> >>> > > >> >> >
> >> >>> > > >> >> > > Hello!
> >> >>> > > >> >> > >
> >> >>> > > >> >> > > My take on this: we want to offload some of
> >> integrations,
> >> >>> but
> >> >>> > > >> please
> >> >>> > > >> >> > commit
> >> >>> > > >> >> > > and release them to another project first, and only
> >> then we
> >> >>> > will
> >> >>> > > >> >> decide
> >> >>> > > >> >> > to
> >> >>> > > >> >> > > remove them from our repository.
> >> >>> > > >> >> > >
> >> >>> > > >> >> > > Please also make sure they are available with same
> >> artifact
> >> >>> > name
> >> >>> > > >> and
> >> >>> > > >> >> API.
> >> >>> > > >> >> > > If this has to be changed, let's wait for AI 3.0
> >> >>> > > >> >> > >
> >> >>> > > >> >> > > Regards,
> >> >>> > > >> >> > > --
> >> >>> > > >> >> > > Ilya Kasnacheev
> >> >>> > > >> >> > >
> >> >>> > > >> >> > >
> >> >>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <
> >> >>> > > >> [hidden email]>:
> >> >>> > > >> >> > >
> >> >>> > > >> >> > > > Hello Emmanouil,
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > > As discussed earlier, I discussed with Apache Bahir
> >> >>> community
> >> >>> > > and
> >> >>> > > >> >> they
> >> >>> > > >> >> > > are
> >> >>> > > >> >> > > > interested to have Apache Ignite extensions as part
> of
> >> >>> Apache
> >> >>> > > >> Bahir
> >> >>> > > >> >> > > > project.
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > > Can you please share Pub/Sub streamer implementation
> >> >>> details
> >> >>> > > with
> >> >>> > > >> >> > Apache
> >> >>> > > >> >> > > > Bahir community and request for feedback.
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > > https://bahir.apache.org/
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > > I have also requested for contributor access in Jira
> >> for
> >> >>> > Apache
> >> >>> > > >> >> Bahir
> >> >>> > > >> >> > > > project so that I can create issues and assign to
> >> myself.
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > > I can help with code reviews as well.
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > > Here is the email thread for reference
> >> >>> > > >> >> > > >
> >> >>> > > https://www.mail-archive.com/dev@.../msg02703.html
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > > Regards,
> >> >>> > > >> >> > > > Saikat
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
> >> >>> > > >> >> [hidden email]
> >> >>> > > >> >> > >
> >> >>> > > >> >> > > > wrote:
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > > > Hello,
> >> >>> > > >> >> > > > >
> >> >>> > > >> >> > > > > As discussed I have created following discussion
> >> >>> threads on
> >> >>> > > our
> >> >>> > > >> >> > > migration
> >> >>> > > >> >> > > > > proposals for Apache Ignite extensions:
> >> >>> > > >> >> > > > >
> >> >>> > > >> >> > > > >
> >> >>> > > >> >> > > > >
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > >
> >> >>> > > >> >> >
> >> >>> > > >> >>
> >> >>> > > >>
> >> >>> > >
> >> >>> >
> >> >>>
> >>
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> >> >>> > > >> >> > > > >
> >> >>> > > >> >> > > > >
> >> >>> > > >>
> >> https://www.mail-archive.com/dev@.../msg02703.html
> >> >>> > > >> >> > > > >
> >> >>> > > >> >> > > > > I will share update as I hear more information.
> >> >>> > > >> >> > > > >
> >> >>> > > >> >> > > > > Regards,
> >> >>> > > >> >> > > > > Saikat
> >> >>> > > >> >> > > > >
> >> >>> > > >> >> > > > >
> >> >>> > > >> >> > > > >
> >> >>> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
> >> >>> > > >> >> > [hidden email]
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > > > wrote:
> >> >>> > > >> >> > > > >
> >> >>> > > >> >> > > > >> Hello Denis,
> >> >>> > > >> >> > > > >>
> >> >>> > > >> >> > > > >> Sure, sounds good. I will create a separate
> >> discussion
> >> >>> > > thread
> >> >>> > > >> to
> >> >>> > > >> >> get
> >> >>> > > >> >> > > > >> community feedback on whether we
> >> >>> > > >> >> > > > >> should join the Bahir or kick-off "Ignite
> >> Extensions"
> >> >>> > > project.
> >> >>> > > >> >> > > > >>
> >> >>> > > >> >> > > > >> Regards,
> >> >>> > > >> >> > > > >> Saikat
> >> >>> > > >> >> > > > >>
> >> >>> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <
> >> >>> > > >> [hidden email]>
> >> >>> > > >> >> > > wrote:
> >> >>> > > >> >> > > > >>
> >> >>> > > >> >> > > > >>> Folks,
> >> >>> > > >> >> > > > >>>
> >> >>> > > >> >> > > > >>> The concept of Apache Bahir is precisely what we
> >> >>> need!
> >> >>> > > >> Saikat,
> >> >>> > > >> >> > thanks
> >> >>> > > >> >> > > > for
> >> >>> > > >> >> > > > >>> doing the research. Why I believe the * idea*
> >> fits us
> >> >>> > well:
> >> >>> > > >> >> > > > >>>
> >> >>> > > >> >> > > > >>>    - All integrations can be stored in separate
> >> >>> Github
> >> >>> > > >> >> repositories
> >> >>> > > >> >> > > and
> >> >>> > > >> >> > > > >>>    have their dev lifecycles. We've not obliged
> to
> >> >>> couple
> >> >>> > > all
> >> >>> > > >> >> the
> >> >>> > > >> >> > > > >>> integrations
> >> >>> > > >> >> > > > >>>    in a single repo. For instance, Spark can be
> >> >>> located
> >> >>> > in
> >> >>> > > a
> >> >>> > > >> >> > > dedicated
> >> >>> > > >> >> > > > >>> repo
> >> >>> > > >> >> > > > >>>    while streaming integrations might be in a
> >> single
> >> >>> one.
> >> >>> > > >> It's
> >> >>> > > >> >> up
> >> >>> > > >> >> > to
> >> >>> > > >> >> > > > us.
> >> >>> > > >> >> > > > >>>    - All the repositories and integrations
> belong
> >> to
> >> >>> ASF.
> >> >>> > > >> We're
> >> >>> > > >> >> not
> >> >>> > > >> >> > > > >>> dumping
> >> >>> > > >> >> > > > >>>    them on Github but rather keep supporting and
> >> >>> > developing
> >> >>> > > >> in
> >> >>> > > >> >> > > > >>> accordance with
> >> >>> > > >> >> > > > >>>    ASF vision and practices.
> >> >>> > > >> >> > > > >>>    - There is a way to reward contributors via
> >> >>> > > committership
> >> >>> > > >> and
> >> >>> > > >> >> > PMC
> >> >>> > > >> >> > > > >>>    membership. You won't get this if a project
> is
> >> >>> just
> >> >>> > one
> >> >>> > > of
> >> >>> > > >> >> the
> >> >>> > > >> >> > > > >>> millions of
> >> >>> > > >> >> > > > >>>    Github projects.
> >> >>> > > >> >> > > > >>>
> >> >>> > > >> >> > > > >>> Saikat, as Ignite PMC member, could you please
> >> >>> kick-off a
> >> >>> > > >> >> separate
> >> >>> > > >> >> > > > >>> dev-list
> >> >>> > > >> >> > > > >>> discussion to involve more community members and
> >> >>> > referring
> >> >>> > > to
> >> >>> > > >> >> this
> >> >>> > > >> >> > > > >>> thread?
> >> >>> > > >> >> > > > >>> I think the primary question the community needs
> >> to
> >> >>> > answer
> >> >>> > > >> >> whether
> >> >>> > > >> >> > we
> >> >>> > > >> >> > > > >>> should join the Bahir or kick-off "Ignite
> >> Extensions"
> >> >>> > > >> project?
> >> >>> > > >> >> > > > >>>
> >> >>> > > >> >> > > > >>> -
> >> >>> > > >> >> > > > >>> Denis
> >> >>> > > >> >> > > > >>>
> >> >>> > > >> >> > > > >>>
> >> >>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev
> <
> >> >>> > > >> >> > > > [hidden email]>
> >> >>> > > >> >> > > > >>> wrote:
> >> >>> > > >> >> > > > >>>
> >> >>> > > >> >> > > > >>> > Maybe we could move all our Streaming
> >> Integrations
> >> >>> > there,
> >> >>> > > >> but
> >> >>> > > >> >> > what
> >> >>> > > >> >> > > is
> >> >>> > > >> >> > > > >>> about
> >> >>> > > >> >> > > > >>> > maintaining and committer permissions to the
> new
> >> >>> > > >> repositories?
> >> >>> > > >> >> > > > >>> > I see the list of the committers and PMC
> members
> >> >>> there
> >> >>> > > >> >> > > > >>> > https://bahir.apache.org/community-members/
> >> >>> > > >> >> > > > >>> >
> >> >>> > > >> >> > > > >>> > Could somebody from Ignite community be added
> to
> >> >>> this
> >> >>> > > list
> >> >>> > > >> as
> >> >>> > > >> >> a
> >> >>> > > >> >> > > > >>> > PMC/committer to maintain Ignite integrations?
> >> >>> > > >> >> > > > >>> > If yes, I'd happy to join this project and
> >> >>> collaborate
> >> >>> > > with
> >> >>> > > >> >> these
> >> >>> > > >> >> > > > guys
> >> >>> > > >> >> > > > >>> > together
> >> >>> > > >> >> > > > >>> > If no, and we should start from the zero level
> >> with
> >> >>> > > >> external
> >> >>> > > >> >> PRs
> >> >>> > > >> >> > -
> >> >>> > > >> >> > > > >>> hmmm,
> >> >>> > > >> >> > > > >>> > it's better to have our own external
> repository
> >> >>> with
> >> >>> > > >> >> ApacheBahirr
> >> >>> > > >> >> > > > >>> ideology.
> >> >>> > > >> >> > > > >>> >
> >> >>> > > >> >> > > > >>> > Also, I agree, that all connectors could be
> >> moved
> >> >>> there
> >> >>> > > (in
> >> >>> > > >> >> > > > >>> ApacheBahirr
> >> >>> > > >> >> > > > >>> > like repository), but not all modules from the
> >> page
> >> >>> > > >> >> > > > >>> >
> >> >>> > > >> >> > > > >>> >
> >> >>> > > >> >> > > > >>>
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > >
> >> >>> > > >> >> >
> >> >>> > > >> >>
> >> >>> > > >>
> >> >>> > >
> >> >>> >
> >> >>>
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> >>> > > >> >> > > > >>> > because
> >> >>> > > >> >> > > > >>> > they use different parts of core modules and
> >> could
> >> >>> be
> >> >>> > > >> >> developed
> >> >>> > > >> >> > > with
> >> >>> > > >> >> > > > >>> > different velocity.
> >> >>> > > >> >> > > > >>> >
> >> >>> > > >> >> > > > >>> > In reality, before creation of new
> repositories
> >> we
> >> >>> need
> >> >>> > > >> wide
> >> >>> > > >> >> > > > discussion
> >> >>> > > >> >> > > > >>> > based on the document mentioned above.
> >> >>> > > >> >> > > > >>> >
> >> >>> > > >> >> > > > >>> > P.S. Of course, we could experiment in the
> next
> >> >>> release
> >> >>> > > 2.8
> >> >>> > > >> >> with
> >> >>> > > >> >> > > one
> >> >>> > > >> >> > > > >>> or two
> >> >>> > > >> >> > > > >>> > integrations like twitter or ZeroMQ
> >> >>> > > >> >> > > > >>> > Also, I'd like Denis Magda idea for the
> separate
> >> >>> > > >> repositories
> >> >>> > > >> >> in
> >> >>> > > >> >> > > > Apache
> >> >>> > > >> >> > > > >>> > Github to maintain them by anybody.
> >> >>> > > >> >> > > > >>> >
> >> >>> > > >> >> > > > >>> >
> >> >>> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra <
> >> >>> > > >> >> > > [hidden email]
> >> >>> > > >> >> > > > >:
> >> >>> > > >> >> > > > >>> >
> >> >>> > > >> >> > > > >>> > > Hi Denis, Emmanouil
> >> >>> > > >> >> > > > >>> > >
> >> >>> > > >> >> > > > >>> > > Thank you for your email. I think creating a
> >> >>> separate
> >> >>> > > >> >> > integration
> >> >>> > > >> >> > > > >>> project
> >> >>> > > >> >> > > > >>> > > in github and also proposing it as an apache
> >> >>> > incubator
> >> >>> > > >> >> project
> >> >>> > > >> >> > > will
> >> >>> > > >> >> > > > >>> be a
> >> >>> > > >> >> > > > >>> > > good move. The other separate integration
> >> modules
> >> >>> > > >> >> > > > >>> > >
> >> >>> > > >> >> > > > >>> > >
> >> >>> > > >> >> > > > >>> >
> >> >>> > > >> >> > > > >>>
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > >
> >> >>> > > >> >> >
> >> >>> > > >> >>
> >> >>> > > >>
> >> >>> > >
> >> >>> >
> >> >>>
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> >>> > > >> >> > > > >>> > > can
> >> >>> > > >> >> > > > >>> > > be moved to this new apache incubator
> project.
> >> >>> > > >> >> > > > >>> > >
> >> >>> > > >> >> > > > >>> > > There are similar integration available for
> >> >>> Flink and
> >> >>> > > >> Spark
> >> >>> > > >> >> in
> >> >>> > > >> >> > > > Apache
> >> >>> > > >> >> > > > >>> > Bahir
> >> >>> > > >> >> > > > >>> > > https://bahir.apache.org/
> >> >>> > > >> >> > > > >>> > >
> >> >>> > > >> >> > > > >>> > > Do you think we should reach out to Apache
> >> Bahir
> >> >>> > > >> community
> >> >>> > > >> >> > with a
> >> >>> > > >> >> > > > >>> > proposal
> >> >>> > > >> >> > > > >>> > > or we should plan to create a separate
> Apache
> >> >>> > Incubator
> >> >>> > > >> >> > project?
> >> >>> > > >> >> > > > >>> > >
> >> >>> > > >> >> > > > >>> > > Regards,
> >> >>> > > >> >> > > > >>> > > Saikat
> >> >>> > > >> >> > > > >>> > >
> >> >>> > > >> >> > > > >>> > >
> >> >>> > > >> >> > > > >>> > >
> >> >>> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil
> >> >>> > Gkatziouras <
> >> >>> > > >> >> > > > >>> > > [hidden email]>
> >> >>> > > >> >> > > > >>> > > wrote:
> >> >>> > > >> >> > > > >>> > >
> >> >>> > > >> >> > > > >>> > > > Hi all!
> >> >>> > > >> >> > > > >>> > > >
> >> >>> > > >> >> > > > >>> > > > Based on the discussions most probably
> this
> >> is
> >> >>> > going
> >> >>> > > to
> >> >>> > > >> >> be on
> >> >>> > > >> >> > > > >>> another
> >> >>> > > >> >> > > > >>> > > > GitHub repo. Therefore I will prepare a
> >> >>> standalone
> >> >>> > > >> project
> >> >>> > > >> >> > with
> >> >>> > > >> >> > > > the
> >> >>> > > >> >> > > > >>> > > Pub/Sub
> >> >>> > > >> >> > > > >>> > > > module and once the repository is created
> a
> >> >>> pull
> >> >>> > > >> request
> >> >>> > > >> >> > shall
> >> >>> > > >> >> > > be
> >> >>> > > >> >> > > > >>> > issued.
> >> >>> > > >> >> > > > >>> > > > If there is anything else I can do in
> order
> >> to
> >> >>> > > >> facilitate
> >> >>> > > >> >> > this
> >> >>> > > >> >> > > > >>> please
> >> >>> > > >> >> > > > >>> > let
> >> >>> > > >> >> > > > >>> > > > me know.
> >> >>> > > >> >> > > > >>> > > >
> >> >>> > > >> >> > > > >>> > > > If a ticket is created for this issue, I
> >> shall
> >> >>> be
> >> >>> > > able
> >> >>> > > >> to
> >> >>> > > >> >> > > create
> >> >>> > > >> >> > > > a
> >> >>> > > >> >> > > > >>> pull
> >> >>> > > >> >> > > > >>> > > > request and thus a build will take place
> on
> >> >>> > ignite's
> >> >>> > > >> CI.
> >> >>> > > >> >> > > > >>> > > > If everyone is aligned with adding a
> Pub/Sub
> >> >>> > > >> >> implementation
> >> >>> > > >> >> > > may I
> >> >>> > > >> >> > > > >>> > create
> >> >>> > > >> >> > > > >>> > > a
> >> >>> > > >> >> > > > >>> > > > ticket on JIRA?
> >> >>> > > >> >> > > > >>> > > >
> >> >>> > > >> >> > > > >>> > > > My next question has to do on how Ignite
> >> >>> configures
> >> >>> > > the
> >> >>> > > >> >> build
> >> >>> > > >> >> > > > jobs
> >> >>> > > >> >> > > > >>> in
> >> >>> > > >> >> > > > >>> > > Team
> >> >>> > > >> >> > > > >>> > > > City. The unit tests for my implementation
> >> >>> need a
> >> >>> > > local
> >> >>> > > >> >> > Pub/Sub
> >> >>> > > >> >> > > > >>> server
> >> >>> > > >> >> > > > >>> > up
> >> >>> > > >> >> > > > >>> > > > and running (they provide one for testing
> >> >>> > purposes).
> >> >>> > > >> Will
> >> >>> > > >> >> > this
> >> >>> > > >> >> > > > take
> >> >>> > > >> >> > > > >>> > > effect
> >> >>> > > >> >> > > > >>> > > > in a form of a build script inside my
> >> >>> > implementation
> >> >>> > > or
> >> >>> > > >> >> is it
> >> >>> > > >> >> > > > >>> something
> >> >>> > > >> >> > > > >>> > > > that should be configured on Team City?
> >> >>> > > >> >> > > > >>> > > >
> >> >>> > > >> >> > > > >>> > > > Kind regards,
> >> >>> > > >> >> > > > >>> > > > Emmanouil
> >> >>> > > >> >> > > > >>> > > >
> >> >>> > > >> >> > > > >>> > > >
> >> >>> > > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
> >> >>> > > >> >> > > > >>> > > > https://egkatzioura.com/ |
> >> >>> > > >> >> > > > >>> > > >
> >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> >>> > > >> >> > > > >>> > > > https://github.com/gkatzioura
> >> >>> > > >> >> > > > >>> > > >
> >> >>> > > >> >> > > > >>> > > >
> >> >>> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis Magda
> <
> >> >>> > > >> >> [hidden email]
> >> >>> > > >> >> > >
> >> >>> > > >> >> > > > >>> wrote:
> >> >>> > > >> >> > > > >>> > > >
> >> >>> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
> >> >>> > > >> >> > > > >>> > > > >
> >> >>> > > >> >> > > > >>> > > > > After contemplating on this, let me
> >> propose
> >> >>> > another
> >> >>> > > >> way
> >> >>> > > >> >> > with
> >> >>> > > >> >> > > > this
> >> >>> > > >> >> > > > >>> > > > > particular integration that is aligned
> >> with
> >> >>> our
> >> >>> > > >> >> > > modularization
> >> >>> > > >> >> > > > >>> > > endeavor.
> >> >>> > > >> >> > > > >>> > > > > The modularization [1] defines the
> Ignite
> >> >>> core,
> >> >>> > > >> Ignite
> >> >>> > > >> >> > > modules,
> >> >>> > > >> >> > > > >>> and
> >> >>> > > >> >> > > > >>> > > > > independent integrations. The first two
> >> are
> >> >>> to be
> >> >>> > > >> >> located
> >> >>> > > >> >> > in
> >> >>> > > >> >> > > > >>> Ignite
> >> >>> > > >> >> > > > >>> > > > > repositories and to be tested/updated by
> >> the
> >> >>> > > >> community
> >> >>> > > >> >> for
> >> >>> > > >> >> > > > every
> >> >>> > > >> >> > > > >>> > > release.
> >> >>> > > >> >> > > > >>> > > > > The modules are a subset of existing
> >> >>> integrations
> >> >>> > > >> that
> >> >>> > > >> >> are
> >> >>> > > >> >> > > > >>> adopted
> >> >>> > > >> >> > > > >>> > > widely
> >> >>> > > >> >> > > > >>> > > > > and require community attention.
> >> >>> > > >> >> > > > >>> > > > >
> >> >>> > > >> >> > > > >>> > > > > As for the independent integrations [1],
> >> >>> those
> >> >>> > are
> >> >>> > > >> >> supposed
> >> >>> > > >> >> > > to
> >> >>> > > >> >> > > > be
> >> >>> > > >> >> > > > >>> > > located
> >> >>> > > >> >> > > > >>> > > > > in a separate GitHub repo. I would
> advise
> >> us
> >> >>> to
> >> >>> > > start
> >> >>> > > >> >> > > > >>> implementing
> >> >>> > > >> >> > > > >>> > this
> >> >>> > > >> >> > > > >>> > > > > concept with the Pub/Sub integration.
> Even
> >> >>> though
> >> >>> > > it
> >> >>> > > >> >> will
> >> >>> > > >> >> > be
> >> >>> > > >> >> > > > >>> located
> >> >>> > > >> >> > > > >>> > > in a
> >> >>> > > >> >> > > > >>> > > > > separate Github repo, it will be listed
> >> among
> >> >>> > > >> officially
> >> >>> > > >> >> > > > >>> available
> >> >>> > > >> >> > > > >>> > > > > integrations on the Ignite website
> >> featuring
> >> >>> > > >> Emmanouil
> >> >>> > > >> >> as a
> >> >>> > > >> >> > > > >>> > > contributor.
> >> >>> > > >> >> > > > >>> > > > >
> >> >>> > > >> >> > > > >>> > > > > What do you think?
> >> >>> > > >> >> > > > >>> > > > >
> >> >>> > > >> >> > > > >>> > > > >
> >> >>> > > >> >> > > > >>> > > > > [1]
> >> >>> > > >> >> > > > >>> > > > >
> >> >>> > > >> >> > > > >>> > > > >
> >> >>> > > >> >> > > > >>> > > >
> >> >>> > > >> >> > > > >>> > >
> >> >>> > > >> >> > > > >>> >
> >> >>> > > >> >> > > > >>>
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > >
> >> >>> > > >> >> >
> >> >>> > > >> >>
> >> >>> > > >>
> >> >>> > >
> >> >>> >
> >> >>>
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> >>> > > >> >> > > > >>> > > > >
> >> >>> > > >> >> > > > >>> > > > >
> >> >>> > > >> >> > > > >>> > > > >
> >> >>> > > >> >> > > > >>> > > > > -
> >> >>> > > >> >> > > > >>> > > > > Denis
> >> >>> > > >> >> > > > >>> > > > >
> >> >>> > > >> >> > > > >>> > > > >
> >> >>> > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat
> >> >>> Maitra <
> >> >>> > > >> >> > > > >>> > [hidden email]
> >> >>> > > >> >> > > > >>> > > >
> >> >>> > > >> >> > > > >>> > > > > wrote:
> >> >>> > > >> >> > > > >>> > > > >
> >> >>> > > >> >> > > > >>> > > > > > Hi Emmanouil,
> >> >>> > > >> >> > > > >>> > > > > >
> >> >>> > > >> >> > > > >>> > > > > > The changes looks good to me. I wanted
> >> to
> >> >>> check
> >> >>> > > if
> >> >>> > > >> you
> >> >>> > > >> >> > are
> >> >>> > > >> >> > > > also
> >> >>> > > >> >> > > > >>> > able
> >> >>> > > >> >> > > > >>> > > to
> >> >>> > > >> >> > > > >>> > > > > > validate that the once data is added
> to
> >> >>> Ignite
> >> >>> > > >> cluster
> >> >>> > > >> >> > you
> >> >>> > > >> >> > > > are
> >> >>> > > >> >> > > > >>> also
> >> >>> > > >> >> > > > >>> > > > able
> >> >>> > > >> >> > > > >>> > > > > to
> >> >>> > > >> >> > > > >>> > > > > > access it using another Ignite client
> or
> >> >>> using
> >> >>> > > rest
> >> >>> > > >> >> > > > endpoints.
> >> >>> > > >> >> > > > >>> > > > > >
> >> >>> > > >> >> > > > >>> > > > > > We use Teamcity for CI process, the
> >> >>> details are
> >> >>> > > as
> >> >>> > > >> >> > follows
> >> >>> > > >> >> > > > >>> > > > > >
> >> >>> > > >> >> > > > >>> > > > > > Trigger validation of those test
> suites
> >> >>> that
> >> >>> > have
> >> >>> > > >> been
> >> >>> > > >> >> > > > >>> affected by
> >> >>> > > >> >> > > > >>> > > your
> >> >>> > > >> >> > > > >>> > > > > > changes on TeamCity <
> >> >>> > > http://ci.ignite.apache.org/>:
> >> >>> > > >> <
> >> >>> > > >> >> > > > >>> > > > > http://204.14.53.153/
> >> >>> > > >> >> > > > >>> > > > > > >
> >> >>> > > >> >> > > > >>> > > > > >
> >> >>> > > >> >> > > > >>> > > > > >    - Locate a test suite you have to
> >> check,
> >> >>> > press
> >> >>> > > >> >> button
> >> >>> > > >> >> > > > named
> >> >>> > > >> >> > > > >>> > "..."
> >> >>> > > >> >> > > > >>> > > > that
> >> >>> > > >> >> > > > >>> > > > > >    is located on the left of "Run"
> >> button.
> >> >>> "Run
> >> >>> > > >> custom
> >> >>> > > >> >> > > build"
> >> >>> > > >> >> > > > >>> > dialog
> >> >>> > > >> >> > > > >>> > > > will
> >> >>> > > >> >> > > > >>> > > > > >    appear;
> >> >>> > > >> >> > > > >>> > > > > >    - Go to "Changes" tab and choose
> >> >>> > > >> >> > > > >>> > "pull/<pull-request-number>/head"
> >> >>> > > >> >> > > > >>> > > > in
> >> >>> > > >> >> > > > >>> > > > > >    "Build branch" dropdown list;
> >> >>> > > >> >> > > > >>> > > > > >    - Press "Run build" button and
> >> monitor
> >> >>> tests
> >> >>> > > >> >> results.
> >> >>> > > >> >> > > > >>> > > > > >
> >> >>> > > >> >> > > > >>> > > > > >
> >> >>> > > >> >> > > > >>> > > > > >
> >> >>> > > >> >> > > > >>> > > > > > @Roman [hidden email] - can you
> also
> >> >>> please
> >> >>> > > >> take a
> >> >>> > > >> >> > > look?
> >> >>> > > >> >> > > > >>> > > > > >
> >> >>> > > >> >> > > > >>> > > > > > Regards,
> >> >>> > > >> >> > > > >>> > > > > > Saikat
> >> >>> > > >> >> > > > >>> > > > > >
> >> >>> > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM
> >> Emmanouil
> >> >>> > > >> Gkatziouras
> >> >>> > > >> >> <
> >> >>> > > >> >> > > > >>> > > > > > [hidden email]>
> >> >>> > > >> >> > > > >>> > > > > > wrote:
> >> >>> > > >> >> > > > >>> > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
> >> >>> > > >> >> > > > >>> > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > I have my implementation on an
> Ignite
> >> >>> fork on
> >> >>> > > my
> >> >>> > > >> >> > personal
> >> >>> > > >> >> > > > >>> GitHub
> >> >>> > > >> >> > > > >>> > > > > account
> >> >>> > > >> >> > > > >>> > > > > > > [1]
> >> >>> > > >> >> > > > >>> > > > > > > It is based on the Kafka
> >> implementation
> >> >>> which
> >> >>> > > is
> >> >>> > > >> of
> >> >>> > > >> >> a
> >> >>> > > >> >> > > > polling
> >> >>> > > >> >> > > > >>> > > nature.
> >> >>> > > >> >> > > > >>> > > > > > > GCP does not provide a service like
> >> >>> Kafka.
> >> >>> > > >> Instead
> >> >>> > > >> >> it
> >> >>> > > >> >> > > > >>> provides us
> >> >>> > > >> >> > > > >>> > > > with
> >> >>> > > >> >> > > > >>> > > > > > > Pub/Sub which is just like Kafka.
> >> >>> > > >> >> > > > >>> > > > > > > Therefore using the Kafka streamer
> as
> >> a
> >> >>> > > guideline
> >> >>> > > >> >> made
> >> >>> > > >> >> > > more
> >> >>> > > >> >> > > > >>> sense
> >> >>> > > >> >> > > > >>> > > to
> >> >>> > > >> >> > > > >>> > > > > me.
> >> >>> > > >> >> > > > >>> > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > Apart from the implementation I have
> >> also
> >> >>> > > added a
> >> >>> > > >> >> unit
> >> >>> > > >> >> > > > test.
> >> >>> > > >> >> > > > >>> > > > > > > It definitely needs some polishing
> >> and a
> >> >>> > double
> >> >>> > > >> >> check
> >> >>> > > >> >> > on
> >> >>> > > >> >> > > my
> >> >>> > > >> >> > > > >>> side
> >> >>> > > >> >> > > > >>> > to
> >> >>> > > >> >> > > > >>> > > > > make
> >> >>> > > >> >> > > > >>> > > > > > > the review smooth.
> >> >>> > > >> >> > > > >>> > > > > > > For the tests a local Pub/Sub [2]
> >> server
> >> >>> has
> >> >>> > to
> >> >>> > > >> be
> >> >>> > > >> >> > setup
> >> >>> > > >> >> > > > >>> however
> >> >>> > > >> >> > > > >>> > I
> >> >>> > > >> >> > > > >>> > > am
> >> >>> > > >> >> > > > >>> > > > > not
> >> >>> > > >> >> > > > >>> > > > > > > sure which is the best way to do so
> >> with
> >> >>> > > regards
> >> >>> > > >> to
> >> >>> > > >> >> > your
> >> >>> > > >> >> > > > >>> CI/CD.
> >> >>> > > >> >> > > > >>> > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > [1]
> >> >>> > > >> >> > > > >>> > >
> >> >>> > > >> >> >
> >> >>> https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> >> >>> > > >> >> > > > >>> > > > > > > [2]
> >> >>> > > >> >> > > > >>>
> >> >>> > https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
> >> >>> > > >> >> > > > >>> > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > *Emmanouil Gkatziouras*
> >> >>> > > >> >> > > > >>> > > > > > > https://egkatzioura.com/ |
> >> >>> > > >> >> > > > >>> > > > > > >
> >> >>> > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> >>> > > >> >> > > > >>> > > > > > > https://github.com/gkatzioura
> >> >>> > > >> >> > > > >>> > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11, Saikat
> >> >>> Maitra <
> >> >>> > > >> >> > > > >>> > > [hidden email]
> >> >>> > > >> >> > > > >>> > > > >
> >> >>> > > >> >> > > > >>> > > > > > > wrote:
> >> >>> > > >> >> > > > >>> > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > Hello Emmanouil,
> >> >>> > > >> >> > > > >>> > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > Can you please share more info
> about
> >> >>> the
> >> >>> > > >> >> > > implementation?
> >> >>> > > >> >> > > > >>> > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > We have similar implementation for
> >> data
> >> >>> > > >> streamers
> >> >>> > > >> >> > with
> >> >>> > > >> >> > > > >>> Kafka
> >> >>> > > >> >> > > > >>> > and
> >> >>> > > >> >> > > > >>> > > > > > RocketMQ
> >> >>> > > >> >> > > > >>> > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > >
> >> >>> > > >> >> > https://apacheignite-mix.readme.io/docs/kafka-streamer
> >> >>> > > >> >> > > > >>> > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > >
> >> >>> > > >> >> > > >
> >> >>> https://apacheignite-mix.readme.io/docs/rocketmq-streamer
> >> >>> > > >> >> > > > >>> > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > Please review and share if you
> have
> >> any
> >> >>> > > >> questions.
> >> >>> > > >> >> > > > >>> > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > Regards,
> >> >>> > > >> >> > > > >>> > > > > > > > Saikat
> >> >>> > > >> >> > > > >>> > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56 PM
> >> >>> Emmanouil
> >> >>> > > >> >> > Gkatziouras <
> >> >>> > > >> >> > > > >>> > > > > > > > [hidden email]>
> >> >>> > > >> >> > > > >>> > > > > > > > wrote:
> >> >>> > > >> >> > > > >>> > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > Hi all,
> >> >>> > > >> >> > > > >>> > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > By mentioning Pub/Sub I mean
> >> Google
> >> >>> > Cloud's
> >> >>> > > >> >> Pub/Sub
> >> >>> > > >> >> > > [1]
> >> >>> > > >> >> > > > >>> > > > > > > > > Yes it is indeed cloud specific.
> >> It
> >> >>> is
> >> >>> > the
> >> >>> > > >> >> > messaging
> >> >>> > > >> >> > > > >>> service
> >> >>> > > >> >> > > > >>> > > > > provided
> >> >>> > > >> >> > > > >>> > > > > > > by
> >> >>> > > >> >> > > > >>> > > > > > > > > GCP.
> >> >>> > > >> >> > > > >>> > > > > > > > > I have made the integration with
> >> it,
> >> >>> but
> >> >>> > I
> >> >>> > > >> >> haven't
> >> >>> > > >> >> > > > >>> published
> >> >>> > > >> >> > > > >>> > it
> >> >>> > > >> >> > > > >>> > > > yet
> >> >>> > > >> >> > > > >>> > > > > > (I
> >> >>> > > >> >> > > > >>> > > > > > > am
> >> >>> > > >> >> > > > >>> > > > > > > > > waiting for the
> discussion/ticket
> >> and
> >> >>> > > proceed
> >> >>> > > >> >> with
> >> >>> > > >> >> > > the
> >> >>> > > >> >> > > > >>> > > > > contribution).
> >> >>> > > >> >> > > > >>> > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > Kind regards,
> >> >>> > > >> >> > > > >>> > > > > > > > > Emmanouil
> >> >>> > > >> >> > > > >>> > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > [1]
> >> >>> > > >> >> https://cloud.google.com/pubsub/docs/overview
> >> >>> > > >> >> > > > >>> > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
> >> >>> > > >> >> > > > >>> > > > > > > > > https://egkatzioura.com/ |
> >> >>> > > >> >> > > > >>> > > > > > > > >
> >> >>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> >>> > > >> >> > > > >>> > > > > > > > > https://github.com/gkatzioura
> >> >>> > > >> >> > > > >>> > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at 21:45,
> >> Denis
> >> >>> > Magda <
> >> >>> > > >> >> > > > >>> [hidden email]
> >> >>> > > >> >> > > > >>> > >
> >> >>> > > >> >> > > > >>> > > > > wrote:
> >> >>> > > >> >> > > > >>> > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > > Hello Emmanouil,
> >> >>> > > >> >> > > > >>> > > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > > Are you referring to any
> >> specific
> >> >>> > > >> >> implementation
> >> >>> > > >> >> > > > >>> tailored
> >> >>> > > >> >> > > > >>> > > for a
> >> >>> > > >> >> > > > >>> > > > > > > cloud?
> >> >>> > > >> >> > > > >>> > > > > > > > If
> >> >>> > > >> >> > > > >>> > > > > > > > > > I'm not mistaken you were
> >> >>> mentioning
> >> >>> > > >> AWS/GCE.
> >> >>> > > >> >> Is
> >> >>> > > >> >> > > > there
> >> >>> > > >> >> > > > >>> any
> >> >>> > > >> >> > > > >>> > > > Github
> >> >>> > > >> >> > > > >>> > > > > > > > source
> >> >>> > > >> >> > > > >>> > > > > > > > > > code?
> >> >>> > > >> >> > > > >>> > > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > > Here is a list of existing
> >> >>> integrations
> >> >>> > > >> with
> >> >>> > > >> >> > > > streaming
> >> >>> > > >> >> > > > >>> and
> >> >>> > > >> >> > > > >>> > > > > pub/sub
> >> >>> > > >> >> > > > >>> > > > > > > > > > frameworks:
> >> >>> > > >> >> > > > >>> > > > > > > > > >
> >> >>> > > >> >> https://apacheignite-mix.readme.io/docs/overview
> >> >>> > > >> >> > > > >>> > > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > > with the source code in Ignite
> >> >>> master:
> >> >>> > > >> >> > > > >>> > > > > > > > > >
> >> >>> > > >> >> > > https://github.com/apache/ignite/tree/master/modules
> >> >>> > > >> >> > > > >>> > > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > > Check out those resources to
> see
> >> >>> what
> >> >>> > are
> >> >>> > > >> the
> >> >>> > > >> >> > most
> >> >>> > > >> >> > > > >>> suitable
> >> >>> > > >> >> > > > >>> > > > > Ignite
> >> >>> > > >> >> > > > >>> > > > > > > APIs
> >> >>> > > >> >> > > > >>> > > > > > > > > for
> >> >>> > > >> >> > > > >>> > > > > > > > > > such integrations.
> >> >>> > > >> >> > > > >>> > > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > > @Saikat Maitra <
> >> >>> > [hidden email]>
> >> >>> > > ,
> >> >>> > > >> >> > @Roman
> >> >>> > > >> >> > > <
> >> >>> > > >> >> > > > >>> > > > > > [hidden email]
> >> >>> > > >> >> > > > >>> > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > ,
> >> >>> > > >> >> > > > >>> > > > > > > > > as
> >> >>> > > >> >> > > > >>> > > > > > > > > > streaming technologies
> >> >>> > > >> >> contributors/maintainers,
> >> >>> > > >> >> > > > could
> >> >>> > > >> >> > > > >>> you
> >> >>> > > >> >> > > > >>> > > > please
> >> >>> > > >> >> > > > >>> > > > > > > join
> >> >>> > > >> >> > > > >>> > > > > > > > > this
> >> >>> > > >> >> > > > >>> > > > > > > > > > thread?
> >> >>> > > >> >> > > > >>> > > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > > -
> >> >>> > > >> >> > > > >>> > > > > > > > > > Denis
> >> >>> > > >> >> > > > >>> > > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > > On Thu, Oct 10, 2019 at 7:58
> AM
> >> >>> > Emmanouil
> >> >>> > > >> >> > > > Gkatziouras <
> >> >>> > > >> >> > > > >>> > > > > > > > > > [hidden email]>
> >> >>> > > >> >> > > > >>> > > > > > > > > > wrote:
> >> >>> > > >> >> > > > >>> > > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > > > Hello everyone,
> >> >>> > > >> >> > > > >>> > > > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > > > I started using Ignite
> lately.
> >> >>> Part
> >> >>> > of
> >> >>> > > my
> >> >>> > > >> >> work
> >> >>> > > >> >> > > > >>> involves
> >> >>> > > >> >> > > > >>> > > > > Pub/Sub,
> >> >>> > > >> >> > > > >>> > > > > > > > thus I
> >> >>> > > >> >> > > > >>> > > > > > > > > > > created a pub/sub streamer.
> >> >>> > > >> >> > > > >>> > > > > > > > > > > Pub/Sub is pretty close to
> >> Kafka.
> >> >>> > > >> >> > > > >>> > > > > > > > > > > Here's how the streamer
> works.
> >> >>> Every
> >> >>> > > node
> >> >>> > > >> >> > creates
> >> >>> > > >> >> > > > an
> >> >>> > > >> >> > > > >>> > > instance
> >> >>> > > >> >> > > > >>> > > > > of
> >> >>> > > >> >> > > > >>> > > > > > a
> >> >>> > > >> >> > > > >>> > > > > > > > > > Pub/Sub
> >> >>> > > >> >> > > > >>> > > > > > > > > > > client.
> >> >>> > > >> >> > > > >>> > > > > > > > > > > The clients subscribe to the
> >> same
> >> >>> > topic
> >> >>> > > >> >> using
> >> >>> > > >> >> > the
> >> >>> > > >> >> > > > >>> same
> >> >>> > > >> >> > > > >>> > > > > > subscription
> >> >>> > > >> >> > > > >>> > > > > > > > > name.
> >> >>> > > >> >> > > > >>> > > > > > > > > > > A message is being sent and
> >> one
> >> >>> of
> >> >>> > the
> >> >>> > > >> nodes
> >> >>> > > >> >> > will
> >> >>> > > >> >> > > > >>> receive
> >> >>> > > >> >> > > > >>> > > the
> >> >>> > > >> >> > > > >>> > > > > > > > message,
> >> >>> > > >> >> > > > >>> > > > > > > > > > > process it and then forward
> >> it to
> >> >>> > > Ignite
> >> >>> > > >> >> using
> >> >>> > > >> >> > > the
> >> >>> > > >> >> > > > >>> > > extractor
> >> >>> > > >> >> > > > >>> > > > > > > > provided.
> >> >>> > > >> >> > > > >>> > > > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > > > Any thoughts?
> >> >>> > > >> >> > > > >>> > > > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > > > *Emmanouil Gkatziouras*
> >> >>> > > >> >> > > > >>> > > > > > > > > > > https://egkatzioura.com/ |
> >> >>> > > >> >> > > > >>> > > > > > > > > > >
> >> >>> > > >> >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> >>> > > >> >> > > > >>> > > > > > > > > > >
> https://github.com/gkatzioura
> >> >>> > > >> >> > > > >>> > > > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > >
> >> >>> > > >> >> > > > >>> > > > > >
> >> >>> > > >> >> > > > >>> > > > >
> >> >>> > > >> >> > > > >>> > > >
> >> >>> > > >> >> > > > >>> > >
> >> >>> > > >> >> > > > >>> >
> >> >>> > > >> >> > > > >>>
> >> >>> > > >> >> > > > >>
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > >
> >> >>> > > >> >> >
> >> >>> > > >> >>
> >> >>> > > >> >
> >> >>> > > >>
> >> >>> > > >
> >> >>> > >
> >> >>> >
> >> >>>
> >> >>
> >>
> >
>

[Emmanouil Gkatziouras]

Hi all!

I was successful building and running my pull requests. It seems that the
tests do not run both for Flink and Pub/Sub [1]
If there is something I cannot do to make them discoverable please let me
know.

[1]
https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4796722&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=283

*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Wed, 27 Nov 2019 at 04:57, Saikat Maitra <[hidden email]> wrote:

> Hi Emmanouil,
>
> I have added you as contributor in Ignite Extensions project. Can you
> please check and confirm if you are able to see the project and execute
> build on your pull request.
>
> Regards,
> Saikat
>
> On Tue, Nov 26, 2019 at 12:49 PM Emmanouil Gkatziouras <
> [hidden email]>
> wrote:
>
> > Hi all!
> >
> > If someone could give me read access to the Job of Ignite Extensions on
> > Team City will greatly help me (username gkatzioura).
> > I suppose the builds get triggered automatically on a pull request.
> >
> > Kind regards,
> > Emmanouil
> >
> > *Emmanouil Gkatziouras*
> > https://egkatzioura.com/ |
> > https://www.linkedin.com/in/gkatziourasemmanouil/
> > https://github.com/gkatzioura
> >
> >
> > On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras <
> [hidden email]>
> > wrote:
> >
> > > Hi Saikat!
> > >
> > > I just rebased with the flink branch. Unfortunately I do not have read
> > > access to the team city link you provided and thus evaluate the tests
> for
> > > my pull request.
> > > I guess it has to do with ignite extensions being new.
> > > My username is gkatzioura and email the one I am using on this email
> > >
> > > Thank you for your time.
> > >
> > >
> > > *Emmanouil Gkatziouras*
> > > https://egkatzioura.com/ |
> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > https://github.com/gkatzioura
> > >
> > >
> > > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <[hidden email]>
> > > wrote:
> > >
> > >> Hi Emmanouil,
> > >>
> > >> The latest build on teamcity has passed on Flink pull request.
> > >>
> > >>
> > >>
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> > >>
> > >> you should be able to take the changes I made in my PR and run build
> on
> > >> your pull request.
> > >>
> > >> Please let me know if you have any questions.
> > >>
> > >> Regards,
> > >> Saikat
> > >>
> > >> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <
> [hidden email]
> > >
> > >> wrote:
> > >>
> > >> > Hi Emmanouil,
> > >> >
> > >> > I have fixed the Flink module testsuites and have setup new project
> in
> > >> > teamcity.
> > >> >
> > >> >
> > >> >
> > >>
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
> > >> >
> > >> > I am looking into setting up the build.
> > >> >
> > >> > Regards,
> > >> > Saikat
> > >> >
> > >> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <
> > [hidden email]>
> > >> > wrote:
> > >> >
> > >> >> Hi Emmanouil,
> > >> >>
> > >> >> Thank you for your email. Yes, the plan is once the Flink PR is
> > merged
> > >> it
> > >> >> will provide the Licence, parent POMs etc and you can rebase from
> > >> master
> > >> >> branch and apply the changes on top of it.
> > >> >>
> > >> >> I am getting some issues with test failures in local with
> > >> >> GridTestProperties as the test.properties is not present in this
> new
> > >> >> project but should be available from dependencies.
> > >> >>
> > >> >> Once I address this issue, I will go ahead and merge the changes
> and
> > >> then
> > >> >> we can take it from there.
> > >> >>
> > >> >> Regards,
> > >> >> Saikat
> > >> >>
> > >> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
> > >> >> [hidden email]> wrote:
> > >> >>
> > >> >>> Hi all,
> > >> >>>
> > >> >>> I made my first pull request [1]. Since this project is brand new
> > (no
> > >> >>> parent poms, licensing), I reckoned it was best to use Saikat's
> > >> branch on
> > >> >>> Flink.
> > >> >>> I suppose the Flink branch will be merged first. If not please
> give
> > me
> > >> >>> guidelines on how I should proceed next.
> > >> >>>
> > >> >>> Kind regards
> > >> >>> Emmanouil
> > >> >>>
> > >> >>> [1] https://github.com/apache/ignite-extensions/pull/2
> > >> >>>
> > >> >>> *Emmanouil Gkatziouras*
> > >> >>> https://egkatzioura.com/ |
> > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> >>> https://github.com/gkatzioura
> > >> >>>
> > >> >>>
> > >> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <[hidden email]>
> > wrote:
> > >> >>>
> > >> >>> > Awesome, ping us whenever you're ready!
> > >> >>> >
> > >> >>> > -
> > >> >>> > Denis
> > >> >>> >
> > >> >>> >
> > >> >>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
> > >> >>> > [hidden email]>
> > >> >>> > wrote:
> > >> >>> >
> > >> >>> > > Hi all!
> > >> >>> > >
> > >> >>> > > I am sorry for being late on that. I was trying to refactor
> the
> > >> test
> > >> >>> in
> > >> >>> > > order not to be in need of any external tools or spinning up a
> > >> >>> server.
> > >> >>> > > I did forked the new repo and indeed my changes there, so a
> pull
> > >> >>> request
> > >> >>> > is
> > >> >>> > > a matter of time!
> > >> >>> > >
> > >> >>> > > Kind regards
> > >> >>> > >
> > >> >>> > > *Emmanouil Gkatziouras*
> > >> >>> > > https://egkatzioura.com/ |
> > >> >>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> >>> > > https://github.com/gkatzioura
> > >> >>> > >
> > >> >>> > >
> > >> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <[hidden email]>
> > >> wrote:
> > >> >>> > >
> > >> >>> > > > Hi Emmanouil,
> > >> >>> > > >
> > >> >>> > > > Do you have any questions or need any support from the
> > >> community?
> > >> >>> > > >
> > >> >>> > > > -
> > >> >>> > > > Denis
> > >> >>> > > >
> > >> >>> > > >
> > >> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
> > >> >>> [hidden email]
> > >> >>> > >
> > >> >>> > > > wrote:
> > >> >>> > > >
> > >> >>> > > >> Hi Emmanouil,
> > >> >>> > > >>
> > >> >>> > > >> Can you please take a looks at dev utils, if this is
> > something
> > >> >>> you are
> > >> >>> > > >> looking for
> > >> >>> > > >>
> > https://github.com/apache/ignite/tree/master/modules/dev-utils
> > >> ?
> > >> >>> > > >>
> > >> >>> > > >> IMO, if you can release Pub/Sub server in maven and then
> use
> > >> it as
> > >> >>> > > >> dependency, that would be great.
> > >> >>> > > >>
> > >> >>> > > >> Regards,
> > >> >>> > > >> Saikat
> > >> >>> > > >>
> > >> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
> > >> >>> > [hidden email]>
> > >> >>> > > >> wrote:
> > >> >>> > > >>
> > >> >>> > > >> > Hi Emmanouil,
> > >> >>> > > >> >
> > >> >>> > > >> > The master branch has been initialized. I have raised a
> PR
> > to
> > >> >>> > migrate
> > >> >>> > > >> > Flink module to ignite-extensions.
> > >> >>> > > >> >
> > >> >>> > > >> > PR https://github.com/apache/ignite-extensions/pull/1
> > >> >>> > > >> > Jira :
> https://issues.apache.org/jira/browse/IGNITE-12356
> > >> >>> > > >> >
> > >> >>> > > >> > Regards,
> > >> >>> > > >> > Saikat
> > >> >>> > > >> >
> > >> >>> > > >> >
> > >> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
> > >> >>> > > >> > [hidden email]> wrote:
> > >> >>> > > >> >
> > >> >>> > > >> >> Hi all!
> > >> >>> > > >> >>
> > >> >>> > > >> >> This is an update. The integration is tested with mocks
> > >> based
> > >> >>> on
> > >> >>> > the
> > >> >>> > > >> >> original Ignite repo [1].
> > >> >>> > > >> >> I am currently waiting for the first commits on the new
> > >> >>> integration
> > >> >>> > > >> >> project
> > >> >>> > > >> >> to take place in order to create my pull request
> > accordingly
> > >> >>> to the
> > >> >>> > > >> >> initial
> > >> >>> > > >> >> commits.
> > >> >>> > > >> >> In order to simulate the Pub/Sub server in a test I
> used a
> > >> test
> > >> >>> > > server
> > >> >>> > > >> >> implementation [2]. Unfortunately they don't have this
> > util
> > >> >>> > uploaded
> > >> >>> > > >> in a
> > >> >>> > > >> >> maven repo, thus it needs to be hosted on a maven repo
> > >> (Apache
> > >> >>> > > License
> > >> >>> > > >> >> 2.0). Is there a repo for utilities that Ignite uses for
> > the
> > >> >>> > > >> test/build?
> > >> >>> > > >> >> I have also created a ticket [3]
> > >> >>> > > >> >>
> > >> >>> > > >> >> Kind regards
> > >> >>> > > >> >>
> > >> >>> > > >> >> [1]
> > >> >>> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > >> >>> > > >> >> [2]
> > >> >>> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> > >> >>> > > >> >> [3] https://issues.apache.org/jira/browse/IGNITE-12262
> > >> >>> > > >> >>
> > >> >>> > > >> >> *Emmanouil Gkatziouras*
> > >> >>> > > >> >> https://egkatzioura.com/ |
> > >> >>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> >>> > > >> >> https://github.com/gkatzioura
> > >> >>> > > >> >>
> > >> >>> > > >> >>
> > >> >>> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
> > >> >>> > [hidden email]
> > >> >>> > > >
> > >> >>> > > >> >> wrote:
> > >> >>> > > >> >>
> > >> >>> > > >> >> > Hello Ilya,
> > >> >>> > > >> >> >
> > >> >>> > > >> >> > Thank you for your email. Yes, before we remove any
> > >> >>> integration
> > >> >>> > we
> > >> >>> > > >> will
> > >> >>> > > >> >> > move them to another repository and release.
> > >> >>> > > >> >> >
> > >> >>> > > >> >> > We are thinking we will start with Pub/Sub Streamer
> > >> >>> > implementation
> > >> >>> > > >> and
> > >> >>> > > >> >> > continue to move other integration as mentioned in the
> > >> >>> following
> > >> >>> > > doc.
> > >> >>> > > >> >> >
> > >> >>> > > >> >> >
> > >> >>> > > >> >> >
> > >> >>> > > >> >>
> > >> >>> > > >>
> > >> >>> > >
> > >> >>> >
> > >> >>>
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > >> >>> > > >> >> >
> > >> >>> > > >> >> > Regards,
> > >> >>> > > >> >> > Saikat
> > >> >>> > > >> >> >
> > >> >>> > > >> >> >
> > >> >>> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
> > >> >>> > > >> >> > [hidden email]>
> > >> >>> > > >> >> > wrote:
> > >> >>> > > >> >> >
> > >> >>> > > >> >> > > Hello!
> > >> >>> > > >> >> > >
> > >> >>> > > >> >> > > My take on this: we want to offload some of
> > >> integrations,
> > >> >>> but
> > >> >>> > > >> please
> > >> >>> > > >> >> > commit
> > >> >>> > > >> >> > > and release them to another project first, and only
> > >> then we
> > >> >>> > will
> > >> >>> > > >> >> decide
> > >> >>> > > >> >> > to
> > >> >>> > > >> >> > > remove them from our repository.
> > >> >>> > > >> >> > >
> > >> >>> > > >> >> > > Please also make sure they are available with same
> > >> artifact
> > >> >>> > name
> > >> >>> > > >> and
> > >> >>> > > >> >> API.
> > >> >>> > > >> >> > > If this has to be changed, let's wait for AI 3.0
> > >> >>> > > >> >> > >
> > >> >>> > > >> >> > > Regards,
> > >> >>> > > >> >> > > --
> > >> >>> > > >> >> > > Ilya Kasnacheev
> > >> >>> > > >> >> > >
> > >> >>> > > >> >> > >
> > >> >>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <
> > >> >>> > > >> [hidden email]>:
> > >> >>> > > >> >> > >
> > >> >>> > > >> >> > > > Hello Emmanouil,
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > > As discussed earlier, I discussed with Apache
> Bahir
> > >> >>> community
> > >> >>> > > and
> > >> >>> > > >> >> they
> > >> >>> > > >> >> > > are
> > >> >>> > > >> >> > > > interested to have Apache Ignite extensions as
> part
> > of
> > >> >>> Apache
> > >> >>> > > >> Bahir
> > >> >>> > > >> >> > > > project.
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > > Can you please share Pub/Sub streamer
> implementation
> > >> >>> details
> > >> >>> > > with
> > >> >>> > > >> >> > Apache
> > >> >>> > > >> >> > > > Bahir community and request for feedback.
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > > https://bahir.apache.org/
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > > I have also requested for contributor access in
> Jira
> > >> for
> > >> >>> > Apache
> > >> >>> > > >> >> Bahir
> > >> >>> > > >> >> > > > project so that I can create issues and assign to
> > >> myself.
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > > I can help with code reviews as well.
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > > Here is the email thread for reference
> > >> >>> > > >> >> > > >
> > >> >>> > >
> https://www.mail-archive.com/dev@.../msg02703.html
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > > Regards,
> > >> >>> > > >> >> > > > Saikat
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
> > >> >>> > > >> >> [hidden email]
> > >> >>> > > >> >> > >
> > >> >>> > > >> >> > > > wrote:
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > > > Hello,
> > >> >>> > > >> >> > > > >
> > >> >>> > > >> >> > > > > As discussed I have created following discussion
> > >> >>> threads on
> > >> >>> > > our
> > >> >>> > > >> >> > > migration
> > >> >>> > > >> >> > > > > proposals for Apache Ignite extensions:
> > >> >>> > > >> >> > > > >
> > >> >>> > > >> >> > > > >
> > >> >>> > > >> >> > > > >
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > >
> > >> >>> > > >> >> >
> > >> >>> > > >> >>
> > >> >>> > > >>
> > >> >>> > >
> > >> >>> >
> > >> >>>
> > >>
> >
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> > >> >>> > > >> >> > > > >
> > >> >>> > > >> >> > > > >
> > >> >>> > > >>
> > >> https://www.mail-archive.com/dev@.../msg02703.html
> > >> >>> > > >> >> > > > >
> > >> >>> > > >> >> > > > > I will share update as I hear more information.
> > >> >>> > > >> >> > > > >
> > >> >>> > > >> >> > > > > Regards,
> > >> >>> > > >> >> > > > > Saikat
> > >> >>> > > >> >> > > > >
> > >> >>> > > >> >> > > > >
> > >> >>> > > >> >> > > > >
> > >> >>> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
> > >> >>> > > >> >> > [hidden email]
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > > > wrote:
> > >> >>> > > >> >> > > > >
> > >> >>> > > >> >> > > > >> Hello Denis,
> > >> >>> > > >> >> > > > >>
> > >> >>> > > >> >> > > > >> Sure, sounds good. I will create a separate
> > >> discussion
> > >> >>> > > thread
> > >> >>> > > >> to
> > >> >>> > > >> >> get
> > >> >>> > > >> >> > > > >> community feedback on whether we
> > >> >>> > > >> >> > > > >> should join the Bahir or kick-off "Ignite
> > >> Extensions"
> > >> >>> > > project.
> > >> >>> > > >> >> > > > >>
> > >> >>> > > >> >> > > > >> Regards,
> > >> >>> > > >> >> > > > >> Saikat
> > >> >>> > > >> >> > > > >>
> > >> >>> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <
> > >> >>> > > >> [hidden email]>
> > >> >>> > > >> >> > > wrote:
> > >> >>> > > >> >> > > > >>
> > >> >>> > > >> >> > > > >>> Folks,
> > >> >>> > > >> >> > > > >>>
> > >> >>> > > >> >> > > > >>> The concept of Apache Bahir is precisely what
> we
> > >> >>> need!
> > >> >>> > > >> Saikat,
> > >> >>> > > >> >> > thanks
> > >> >>> > > >> >> > > > for
> > >> >>> > > >> >> > > > >>> doing the research. Why I believe the * idea*
> > >> fits us
> > >> >>> > well:
> > >> >>> > > >> >> > > > >>>
> > >> >>> > > >> >> > > > >>>    - All integrations can be stored in
> separate
> > >> >>> Github
> > >> >>> > > >> >> repositories
> > >> >>> > > >> >> > > and
> > >> >>> > > >> >> > > > >>>    have their dev lifecycles. We've not
> obliged
> > to
> > >> >>> couple
> > >> >>> > > all
> > >> >>> > > >> >> the
> > >> >>> > > >> >> > > > >>> integrations
> > >> >>> > > >> >> > > > >>>    in a single repo. For instance, Spark can
> be
> > >> >>> located
> > >> >>> > in
> > >> >>> > > a
> > >> >>> > > >> >> > > dedicated
> > >> >>> > > >> >> > > > >>> repo
> > >> >>> > > >> >> > > > >>>    while streaming integrations might be in a
> > >> single
> > >> >>> one.
> > >> >>> > > >> It's
> > >> >>> > > >> >> up
> > >> >>> > > >> >> > to
> > >> >>> > > >> >> > > > us.
> > >> >>> > > >> >> > > > >>>    - All the repositories and integrations
> > belong
> > >> to
> > >> >>> ASF.
> > >> >>> > > >> We're
> > >> >>> > > >> >> not
> > >> >>> > > >> >> > > > >>> dumping
> > >> >>> > > >> >> > > > >>>    them on Github but rather keep supporting
> and
> > >> >>> > developing
> > >> >>> > > >> in
> > >> >>> > > >> >> > > > >>> accordance with
> > >> >>> > > >> >> > > > >>>    ASF vision and practices.
> > >> >>> > > >> >> > > > >>>    - There is a way to reward contributors via
> > >> >>> > > committership
> > >> >>> > > >> and
> > >> >>> > > >> >> > PMC
> > >> >>> > > >> >> > > > >>>    membership. You won't get this if a project
> > is
> > >> >>> just
> > >> >>> > one
> > >> >>> > > of
> > >> >>> > > >> >> the
> > >> >>> > > >> >> > > > >>> millions of
> > >> >>> > > >> >> > > > >>>    Github projects.
> > >> >>> > > >> >> > > > >>>
> > >> >>> > > >> >> > > > >>> Saikat, as Ignite PMC member, could you please
> > >> >>> kick-off a
> > >> >>> > > >> >> separate
> > >> >>> > > >> >> > > > >>> dev-list
> > >> >>> > > >> >> > > > >>> discussion to involve more community members
> and
> > >> >>> > referring
> > >> >>> > > to
> > >> >>> > > >> >> this
> > >> >>> > > >> >> > > > >>> thread?
> > >> >>> > > >> >> > > > >>> I think the primary question the community
> needs
> > >> to
> > >> >>> > answer
> > >> >>> > > >> >> whether
> > >> >>> > > >> >> > we
> > >> >>> > > >> >> > > > >>> should join the Bahir or kick-off "Ignite
> > >> Extensions"
> > >> >>> > > >> project?
> > >> >>> > > >> >> > > > >>>
> > >> >>> > > >> >> > > > >>> -
> > >> >>> > > >> >> > > > >>> Denis
> > >> >>> > > >> >> > > > >>>
> > >> >>> > > >> >> > > > >>>
> > >> >>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey
> Zinoviev
> > <
> > >> >>> > > >> >> > > > [hidden email]>
> > >> >>> > > >> >> > > > >>> wrote:
> > >> >>> > > >> >> > > > >>>
> > >> >>> > > >> >> > > > >>> > Maybe we could move all our Streaming
> > >> Integrations
> > >> >>> > there,
> > >> >>> > > >> but
> > >> >>> > > >> >> > what
> > >> >>> > > >> >> > > is
> > >> >>> > > >> >> > > > >>> about
> > >> >>> > > >> >> > > > >>> > maintaining and committer permissions to the
> > new
> > >> >>> > > >> repositories?
> > >> >>> > > >> >> > > > >>> > I see the list of the committers and PMC
> > members
> > >> >>> there
> > >> >>> > > >> >> > > > >>> > https://bahir.apache.org/community-members/
> > >> >>> > > >> >> > > > >>> >
> > >> >>> > > >> >> > > > >>> > Could somebody from Ignite community be
> added
> > to
> > >> >>> this
> > >> >>> > > list
> > >> >>> > > >> as
> > >> >>> > > >> >> a
> > >> >>> > > >> >> > > > >>> > PMC/committer to maintain Ignite
> integrations?
> > >> >>> > > >> >> > > > >>> > If yes, I'd happy to join this project and
> > >> >>> collaborate
> > >> >>> > > with
> > >> >>> > > >> >> these
> > >> >>> > > >> >> > > > guys
> > >> >>> > > >> >> > > > >>> > together
> > >> >>> > > >> >> > > > >>> > If no, and we should start from the zero
> level
> > >> with
> > >> >>> > > >> external
> > >> >>> > > >> >> PRs
> > >> >>> > > >> >> > -
> > >> >>> > > >> >> > > > >>> hmmm,
> > >> >>> > > >> >> > > > >>> > it's better to have our own external
> > repository
> > >> >>> with
> > >> >>> > > >> >> ApacheBahirr
> > >> >>> > > >> >> > > > >>> ideology.
> > >> >>> > > >> >> > > > >>> >
> > >> >>> > > >> >> > > > >>> > Also, I agree, that all connectors could be
> > >> moved
> > >> >>> there
> > >> >>> > > (in
> > >> >>> > > >> >> > > > >>> ApacheBahirr
> > >> >>> > > >> >> > > > >>> > like repository), but not all modules from
> the
> > >> page
> > >> >>> > > >> >> > > > >>> >
> > >> >>> > > >> >> > > > >>> >
> > >> >>> > > >> >> > > > >>>
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > >
> > >> >>> > > >> >> >
> > >> >>> > > >> >>
> > >> >>> > > >>
> > >> >>> > >
> > >> >>> >
> > >> >>>
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > >> >>> > > >> >> > > > >>> > because
> > >> >>> > > >> >> > > > >>> > they use different parts of core modules and
> > >> could
> > >> >>> be
> > >> >>> > > >> >> developed
> > >> >>> > > >> >> > > with
> > >> >>> > > >> >> > > > >>> > different velocity.
> > >> >>> > > >> >> > > > >>> >
> > >> >>> > > >> >> > > > >>> > In reality, before creation of new
> > repositories
> > >> we
> > >> >>> need
> > >> >>> > > >> wide
> > >> >>> > > >> >> > > > discussion
> > >> >>> > > >> >> > > > >>> > based on the document mentioned above.
> > >> >>> > > >> >> > > > >>> >
> > >> >>> > > >> >> > > > >>> > P.S. Of course, we could experiment in the
> > next
> > >> >>> release
> > >> >>> > > 2.8
> > >> >>> > > >> >> with
> > >> >>> > > >> >> > > one
> > >> >>> > > >> >> > > > >>> or two
> > >> >>> > > >> >> > > > >>> > integrations like twitter or ZeroMQ
> > >> >>> > > >> >> > > > >>> > Also, I'd like Denis Magda idea for the
> > separate
> > >> >>> > > >> repositories
> > >> >>> > > >> >> in
> > >> >>> > > >> >> > > > Apache
> > >> >>> > > >> >> > > > >>> > Github to maintain them by anybody.
> > >> >>> > > >> >> > > > >>> >
> > >> >>> > > >> >> > > > >>> >
> > >> >>> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra <
> > >> >>> > > >> >> > > [hidden email]
> > >> >>> > > >> >> > > > >:
> > >> >>> > > >> >> > > > >>> >
> > >> >>> > > >> >> > > > >>> > > Hi Denis, Emmanouil
> > >> >>> > > >> >> > > > >>> > >
> > >> >>> > > >> >> > > > >>> > > Thank you for your email. I think
> creating a
> > >> >>> separate
> > >> >>> > > >> >> > integration
> > >> >>> > > >> >> > > > >>> project
> > >> >>> > > >> >> > > > >>> > > in github and also proposing it as an
> apache
> > >> >>> > incubator
> > >> >>> > > >> >> project
> > >> >>> > > >> >> > > will
> > >> >>> > > >> >> > > > >>> be a
> > >> >>> > > >> >> > > > >>> > > good move. The other separate integration
> > >> modules
> > >> >>> > > >> >> > > > >>> > >
> > >> >>> > > >> >> > > > >>> > >
> > >> >>> > > >> >> > > > >>> >
> > >> >>> > > >> >> > > > >>>
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > >
> > >> >>> > > >> >> >
> > >> >>> > > >> >>
> > >> >>> > > >>
> > >> >>> > >
> > >> >>> >
> > >> >>>
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > >> >>> > > >> >> > > > >>> > > can
> > >> >>> > > >> >> > > > >>> > > be moved to this new apache incubator
> > project.
> > >> >>> > > >> >> > > > >>> > >
> > >> >>> > > >> >> > > > >>> > > There are similar integration available
> for
> > >> >>> Flink and
> > >> >>> > > >> Spark
> > >> >>> > > >> >> in
> > >> >>> > > >> >> > > > Apache
> > >> >>> > > >> >> > > > >>> > Bahir
> > >> >>> > > >> >> > > > >>> > > https://bahir.apache.org/
> > >> >>> > > >> >> > > > >>> > >
> > >> >>> > > >> >> > > > >>> > > Do you think we should reach out to Apache
> > >> Bahir
> > >> >>> > > >> community
> > >> >>> > > >> >> > with a
> > >> >>> > > >> >> > > > >>> > proposal
> > >> >>> > > >> >> > > > >>> > > or we should plan to create a separate
> > Apache
> > >> >>> > Incubator
> > >> >>> > > >> >> > project?
> > >> >>> > > >> >> > > > >>> > >
> > >> >>> > > >> >> > > > >>> > > Regards,
> > >> >>> > > >> >> > > > >>> > > Saikat
> > >> >>> > > >> >> > > > >>> > >
> > >> >>> > > >> >> > > > >>> > >
> > >> >>> > > >> >> > > > >>> > >
> > >> >>> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil
> > >> >>> > Gkatziouras <
> > >> >>> > > >> >> > > > >>> > > [hidden email]>
> > >> >>> > > >> >> > > > >>> > > wrote:
> > >> >>> > > >> >> > > > >>> > >
> > >> >>> > > >> >> > > > >>> > > > Hi all!
> > >> >>> > > >> >> > > > >>> > > >
> > >> >>> > > >> >> > > > >>> > > > Based on the discussions most probably
> > this
> > >> is
> > >> >>> > going
> > >> >>> > > to
> > >> >>> > > >> >> be on
> > >> >>> > > >> >> > > > >>> another
> > >> >>> > > >> >> > > > >>> > > > GitHub repo. Therefore I will prepare a
> > >> >>> standalone
> > >> >>> > > >> project
> > >> >>> > > >> >> > with
> > >> >>> > > >> >> > > > the
> > >> >>> > > >> >> > > > >>> > > Pub/Sub
> > >> >>> > > >> >> > > > >>> > > > module and once the repository is
> created
> > a
> > >> >>> pull
> > >> >>> > > >> request
> > >> >>> > > >> >> > shall
> > >> >>> > > >> >> > > be
> > >> >>> > > >> >> > > > >>> > issued.
> > >> >>> > > >> >> > > > >>> > > > If there is anything else I can do in
> > order
> > >> to
> > >> >>> > > >> facilitate
> > >> >>> > > >> >> > this
> > >> >>> > > >> >> > > > >>> please
> > >> >>> > > >> >> > > > >>> > let
> > >> >>> > > >> >> > > > >>> > > > me know.
> > >> >>> > > >> >> > > > >>> > > >
> > >> >>> > > >> >> > > > >>> > > > If a ticket is created for this issue, I
> > >> shall
> > >> >>> be
> > >> >>> > > able
> > >> >>> > > >> to
> > >> >>> > > >> >> > > create
> > >> >>> > > >> >> > > > a
> > >> >>> > > >> >> > > > >>> pull
> > >> >>> > > >> >> > > > >>> > > > request and thus a build will take place
> > on
> > >> >>> > ignite's
> > >> >>> > > >> CI.
> > >> >>> > > >> >> > > > >>> > > > If everyone is aligned with adding a
> > Pub/Sub
> > >> >>> > > >> >> implementation
> > >> >>> > > >> >> > > may I
> > >> >>> > > >> >> > > > >>> > create
> > >> >>> > > >> >> > > > >>> > > a
> > >> >>> > > >> >> > > > >>> > > > ticket on JIRA?
> > >> >>> > > >> >> > > > >>> > > >
> > >> >>> > > >> >> > > > >>> > > > My next question has to do on how Ignite
> > >> >>> configures
> > >> >>> > > the
> > >> >>> > > >> >> build
> > >> >>> > > >> >> > > > jobs
> > >> >>> > > >> >> > > > >>> in
> > >> >>> > > >> >> > > > >>> > > Team
> > >> >>> > > >> >> > > > >>> > > > City. The unit tests for my
> implementation
> > >> >>> need a
> > >> >>> > > local
> > >> >>> > > >> >> > Pub/Sub
> > >> >>> > > >> >> > > > >>> server
> > >> >>> > > >> >> > > > >>> > up
> > >> >>> > > >> >> > > > >>> > > > and running (they provide one for
> testing
> > >> >>> > purposes).
> > >> >>> > > >> Will
> > >> >>> > > >> >> > this
> > >> >>> > > >> >> > > > take
> > >> >>> > > >> >> > > > >>> > > effect
> > >> >>> > > >> >> > > > >>> > > > in a form of a build script inside my
> > >> >>> > implementation
> > >> >>> > > or
> > >> >>> > > >> >> is it
> > >> >>> > > >> >> > > > >>> something
> > >> >>> > > >> >> > > > >>> > > > that should be configured on Team City?
> > >> >>> > > >> >> > > > >>> > > >
> > >> >>> > > >> >> > > > >>> > > > Kind regards,
> > >> >>> > > >> >> > > > >>> > > > Emmanouil
> > >> >>> > > >> >> > > > >>> > > >
> > >> >>> > > >> >> > > > >>> > > >
> > >> >>> > > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
> > >> >>> > > >> >> > > > >>> > > > https://egkatzioura.com/ |
> > >> >>> > > >> >> > > > >>> > > >
> > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> >>> > > >> >> > > > >>> > > > https://github.com/gkatzioura
> > >> >>> > > >> >> > > > >>> > > >
> > >> >>> > > >> >> > > > >>> > > >
> > >> >>> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis
> Magda
> > <
> > >> >>> > > >> >> [hidden email]
> > >> >>> > > >> >> > >
> > >> >>> > > >> >> > > > >>> wrote:
> > >> >>> > > >> >> > > > >>> > > >
> > >> >>> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
> > >> >>> > > >> >> > > > >>> > > > >
> > >> >>> > > >> >> > > > >>> > > > > After contemplating on this, let me
> > >> propose
> > >> >>> > another
> > >> >>> > > >> way
> > >> >>> > > >> >> > with
> > >> >>> > > >> >> > > > this
> > >> >>> > > >> >> > > > >>> > > > > particular integration that is aligned
> > >> with
> > >> >>> our
> > >> >>> > > >> >> > > modularization
> > >> >>> > > >> >> > > > >>> > > endeavor.
> > >> >>> > > >> >> > > > >>> > > > > The modularization [1] defines the
> > Ignite
> > >> >>> core,
> > >> >>> > > >> Ignite
> > >> >>> > > >> >> > > modules,
> > >> >>> > > >> >> > > > >>> and
> > >> >>> > > >> >> > > > >>> > > > > independent integrations. The first
> two
> > >> are
> > >> >>> to be
> > >> >>> > > >> >> located
> > >> >>> > > >> >> > in
> > >> >>> > > >> >> > > > >>> Ignite
> > >> >>> > > >> >> > > > >>> > > > > repositories and to be tested/updated
> by
> > >> the
> > >> >>> > > >> community
> > >> >>> > > >> >> for
> > >> >>> > > >> >> > > > every
> > >> >>> > > >> >> > > > >>> > > release.
> > >> >>> > > >> >> > > > >>> > > > > The modules are a subset of existing
> > >> >>> integrations
> > >> >>> > > >> that
> > >> >>> > > >> >> are
> > >> >>> > > >> >> > > > >>> adopted
> > >> >>> > > >> >> > > > >>> > > widely
> > >> >>> > > >> >> > > > >>> > > > > and require community attention.
> > >> >>> > > >> >> > > > >>> > > > >
> > >> >>> > > >> >> > > > >>> > > > > As for the independent integrations
> [1],
> > >> >>> those
> > >> >>> > are
> > >> >>> > > >> >> supposed
> > >> >>> > > >> >> > > to
> > >> >>> > > >> >> > > > be
> > >> >>> > > >> >> > > > >>> > > located
> > >> >>> > > >> >> > > > >>> > > > > in a separate GitHub repo. I would
> > advise
> > >> us
> > >> >>> to
> > >> >>> > > start
> > >> >>> > > >> >> > > > >>> implementing
> > >> >>> > > >> >> > > > >>> > this
> > >> >>> > > >> >> > > > >>> > > > > concept with the Pub/Sub integration.
> > Even
> > >> >>> though
> > >> >>> > > it
> > >> >>> > > >> >> will
> > >> >>> > > >> >> > be
> > >> >>> > > >> >> > > > >>> located
> > >> >>> > > >> >> > > > >>> > > in a
> > >> >>> > > >> >> > > > >>> > > > > separate Github repo, it will be
> listed
> > >> among
> > >> >>> > > >> officially
> > >> >>> > > >> >> > > > >>> available
> > >> >>> > > >> >> > > > >>> > > > > integrations on the Ignite website
> > >> featuring
> > >> >>> > > >> Emmanouil
> > >> >>> > > >> >> as a
> > >> >>> > > >> >> > > > >>> > > contributor.
> > >> >>> > > >> >> > > > >>> > > > >
> > >> >>> > > >> >> > > > >>> > > > > What do you think?
> > >> >>> > > >> >> > > > >>> > > > >
> > >> >>> > > >> >> > > > >>> > > > >
> > >> >>> > > >> >> > > > >>> > > > > [1]
> > >> >>> > > >> >> > > > >>> > > > >
> > >> >>> > > >> >> > > > >>> > > > >
> > >> >>> > > >> >> > > > >>> > > >
> > >> >>> > > >> >> > > > >>> > >
> > >> >>> > > >> >> > > > >>> >
> > >> >>> > > >> >> > > > >>>
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > >
> > >> >>> > > >> >> >
> > >> >>> > > >> >>
> > >> >>> > > >>
> > >> >>> > >
> > >> >>> >
> > >> >>>
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > >> >>> > > >> >> > > > >>> > > > >
> > >> >>> > > >> >> > > > >>> > > > >
> > >> >>> > > >> >> > > > >>> > > > >
> > >> >>> > > >> >> > > > >>> > > > > -
> > >> >>> > > >> >> > > > >>> > > > > Denis
> > >> >>> > > >> >> > > > >>> > > > >
> > >> >>> > > >> >> > > > >>> > > > >
> > >> >>> > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat
> > >> >>> Maitra <
> > >> >>> > > >> >> > > > >>> > [hidden email]
> > >> >>> > > >> >> > > > >>> > > >
> > >> >>> > > >> >> > > > >>> > > > > wrote:
> > >> >>> > > >> >> > > > >>> > > > >
> > >> >>> > > >> >> > > > >>> > > > > > Hi Emmanouil,
> > >> >>> > > >> >> > > > >>> > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > The changes looks good to me. I
> wanted
> > >> to
> > >> >>> check
> > >> >>> > > if
> > >> >>> > > >> you
> > >> >>> > > >> >> > are
> > >> >>> > > >> >> > > > also
> > >> >>> > > >> >> > > > >>> > able
> > >> >>> > > >> >> > > > >>> > > to
> > >> >>> > > >> >> > > > >>> > > > > > validate that the once data is added
> > to
> > >> >>> Ignite
> > >> >>> > > >> cluster
> > >> >>> > > >> >> > you
> > >> >>> > > >> >> > > > are
> > >> >>> > > >> >> > > > >>> also
> > >> >>> > > >> >> > > > >>> > > > able
> > >> >>> > > >> >> > > > >>> > > > > to
> > >> >>> > > >> >> > > > >>> > > > > > access it using another Ignite
> client
> > or
> > >> >>> using
> > >> >>> > > rest
> > >> >>> > > >> >> > > > endpoints.
> > >> >>> > > >> >> > > > >>> > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > We use Teamcity for CI process, the
> > >> >>> details are
> > >> >>> > > as
> > >> >>> > > >> >> > follows
> > >> >>> > > >> >> > > > >>> > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > Trigger validation of those test
> > suites
> > >> >>> that
> > >> >>> > have
> > >> >>> > > >> been
> > >> >>> > > >> >> > > > >>> affected by
> > >> >>> > > >> >> > > > >>> > > your
> > >> >>> > > >> >> > > > >>> > > > > > changes on TeamCity <
> > >> >>> > > http://ci.ignite.apache.org/>:
> > >> >>> > > >> <
> > >> >>> > > >> >> > > > >>> > > > > http://204.14.53.153/
> > >> >>> > > >> >> > > > >>> > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > >
> > >> >>> > > >> >> > > > >>> > > > > >    - Locate a test suite you have to
> > >> check,
> > >> >>> > press
> > >> >>> > > >> >> button
> > >> >>> > > >> >> > > > named
> > >> >>> > > >> >> > > > >>> > "..."
> > >> >>> > > >> >> > > > >>> > > > that
> > >> >>> > > >> >> > > > >>> > > > > >    is located on the left of "Run"
> > >> button.
> > >> >>> "Run
> > >> >>> > > >> custom
> > >> >>> > > >> >> > > build"
> > >> >>> > > >> >> > > > >>> > dialog
> > >> >>> > > >> >> > > > >>> > > > will
> > >> >>> > > >> >> > > > >>> > > > > >    appear;
> > >> >>> > > >> >> > > > >>> > > > > >    - Go to "Changes" tab and choose
> > >> >>> > > >> >> > > > >>> > "pull/<pull-request-number>/head"
> > >> >>> > > >> >> > > > >>> > > > in
> > >> >>> > > >> >> > > > >>> > > > > >    "Build branch" dropdown list;
> > >> >>> > > >> >> > > > >>> > > > > >    - Press "Run build" button and
> > >> monitor
> > >> >>> tests
> > >> >>> > > >> >> results.
> > >> >>> > > >> >> > > > >>> > > > > >
> > >> >>> > > >> >> > > > >>> > > > > >
> > >> >>> > > >> >> > > > >>> > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > @Roman [hidden email] - can you
> > also
> > >> >>> please
> > >> >>> > > >> take a
> > >> >>> > > >> >> > > look?
> > >> >>> > > >> >> > > > >>> > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > Regards,
> > >> >>> > > >> >> > > > >>> > > > > > Saikat
> > >> >>> > > >> >> > > > >>> > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM
> > >> Emmanouil
> > >> >>> > > >> Gkatziouras
> > >> >>> > > >> >> <
> > >> >>> > > >> >> > > > >>> > > > > > [hidden email]>
> > >> >>> > > >> >> > > > >>> > > > > > wrote:
> > >> >>> > > >> >> > > > >>> > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
> > >> >>> > > >> >> > > > >>> > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > I have my implementation on an
> > Ignite
> > >> >>> fork on
> > >> >>> > > my
> > >> >>> > > >> >> > personal
> > >> >>> > > >> >> > > > >>> GitHub
> > >> >>> > > >> >> > > > >>> > > > > account
> > >> >>> > > >> >> > > > >>> > > > > > > [1]
> > >> >>> > > >> >> > > > >>> > > > > > > It is based on the Kafka
> > >> implementation
> > >> >>> which
> > >> >>> > > is
> > >> >>> > > >> of
> > >> >>> > > >> >> a
> > >> >>> > > >> >> > > > polling
> > >> >>> > > >> >> > > > >>> > > nature.
> > >> >>> > > >> >> > > > >>> > > > > > > GCP does not provide a service
> like
> > >> >>> Kafka.
> > >> >>> > > >> Instead
> > >> >>> > > >> >> it
> > >> >>> > > >> >> > > > >>> provides us
> > >> >>> > > >> >> > > > >>> > > > with
> > >> >>> > > >> >> > > > >>> > > > > > > Pub/Sub which is just like Kafka.
> > >> >>> > > >> >> > > > >>> > > > > > > Therefore using the Kafka streamer
> > as
> > >> a
> > >> >>> > > guideline
> > >> >>> > > >> >> made
> > >> >>> > > >> >> > > more
> > >> >>> > > >> >> > > > >>> sense
> > >> >>> > > >> >> > > > >>> > > to
> > >> >>> > > >> >> > > > >>> > > > > me.
> > >> >>> > > >> >> > > > >>> > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > Apart from the implementation I
> have
> > >> also
> > >> >>> > > added a
> > >> >>> > > >> >> unit
> > >> >>> > > >> >> > > > test.
> > >> >>> > > >> >> > > > >>> > > > > > > It definitely needs some polishing
> > >> and a
> > >> >>> > double
> > >> >>> > > >> >> check
> > >> >>> > > >> >> > on
> > >> >>> > > >> >> > > my
> > >> >>> > > >> >> > > > >>> side
> > >> >>> > > >> >> > > > >>> > to
> > >> >>> > > >> >> > > > >>> > > > > make
> > >> >>> > > >> >> > > > >>> > > > > > > the review smooth.
> > >> >>> > > >> >> > > > >>> > > > > > > For the tests a local Pub/Sub [2]
> > >> server
> > >> >>> has
> > >> >>> > to
> > >> >>> > > >> be
> > >> >>> > > >> >> > setup
> > >> >>> > > >> >> > > > >>> however
> > >> >>> > > >> >> > > > >>> > I
> > >> >>> > > >> >> > > > >>> > > am
> > >> >>> > > >> >> > > > >>> > > > > not
> > >> >>> > > >> >> > > > >>> > > > > > > sure which is the best way to do
> so
> > >> with
> > >> >>> > > regards
> > >> >>> > > >> to
> > >> >>> > > >> >> > your
> > >> >>> > > >> >> > > > >>> CI/CD.
> > >> >>> > > >> >> > > > >>> > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > [1]
> > >> >>> > > >> >> > > > >>> > >
> > >> >>> > > >> >> >
> > >> >>> https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > >> >>> > > >> >> > > > >>> > > > > > > [2]
> > >> >>> > > >> >> > > > >>>
> > >> >>> > https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
> > >> >>> > > >> >> > > > >>> > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > *Emmanouil Gkatziouras*
> > >> >>> > > >> >> > > > >>> > > > > > > https://egkatzioura.com/ |
> > >> >>> > > >> >> > > > >>> > > > > > >
> > >> >>> > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> >>> > > >> >> > > > >>> > > > > > > https://github.com/gkatzioura
> > >> >>> > > >> >> > > > >>> > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11,
> Saikat
> > >> >>> Maitra <
> > >> >>> > > >> >> > > > >>> > > [hidden email]
> > >> >>> > > >> >> > > > >>> > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > wrote:
> > >> >>> > > >> >> > > > >>> > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > Hello Emmanouil,
> > >> >>> > > >> >> > > > >>> > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > Can you please share more info
> > about
> > >> >>> the
> > >> >>> > > >> >> > > implementation?
> > >> >>> > > >> >> > > > >>> > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > We have similar implementation
> for
> > >> data
> > >> >>> > > >> streamers
> > >> >>> > > >> >> > with
> > >> >>> > > >> >> > > > >>> Kafka
> > >> >>> > > >> >> > > > >>> > and
> > >> >>> > > >> >> > > > >>> > > > > > RocketMQ
> > >> >>> > > >> >> > > > >>> > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > >
> > >> >>> > > >> >> >
> https://apacheignite-mix.readme.io/docs/kafka-streamer
> > >> >>> > > >> >> > > > >>> > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > >
> > >> >>> > > >> >> > > >
> > >> >>> https://apacheignite-mix.readme.io/docs/rocketmq-streamer
> > >> >>> > > >> >> > > > >>> > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > Please review and share if you
> > have
> > >> any
> > >> >>> > > >> questions.
> > >> >>> > > >> >> > > > >>> > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > Regards,
> > >> >>> > > >> >> > > > >>> > > > > > > > Saikat
> > >> >>> > > >> >> > > > >>> > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56 PM
> > >> >>> Emmanouil
> > >> >>> > > >> >> > Gkatziouras <
> > >> >>> > > >> >> > > > >>> > > > > > > > [hidden email]>
> > >> >>> > > >> >> > > > >>> > > > > > > > wrote:
> > >> >>> > > >> >> > > > >>> > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > Hi all,
> > >> >>> > > >> >> > > > >>> > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > By mentioning Pub/Sub I mean
> > >> Google
> > >> >>> > Cloud's
> > >> >>> > > >> >> Pub/Sub
> > >> >>> > > >> >> > > [1]
> > >> >>> > > >> >> > > > >>> > > > > > > > > Yes it is indeed cloud
> specific.
> > >> It
> > >> >>> is
> > >> >>> > the
> > >> >>> > > >> >> > messaging
> > >> >>> > > >> >> > > > >>> service
> > >> >>> > > >> >> > > > >>> > > > > provided
> > >> >>> > > >> >> > > > >>> > > > > > > by
> > >> >>> > > >> >> > > > >>> > > > > > > > > GCP.
> > >> >>> > > >> >> > > > >>> > > > > > > > > I have made the integration
> with
> > >> it,
> > >> >>> but
> > >> >>> > I
> > >> >>> > > >> >> haven't
> > >> >>> > > >> >> > > > >>> published
> > >> >>> > > >> >> > > > >>> > it
> > >> >>> > > >> >> > > > >>> > > > yet
> > >> >>> > > >> >> > > > >>> > > > > > (I
> > >> >>> > > >> >> > > > >>> > > > > > > am
> > >> >>> > > >> >> > > > >>> > > > > > > > > waiting for the
> > discussion/ticket
> > >> and
> > >> >>> > > proceed
> > >> >>> > > >> >> with
> > >> >>> > > >> >> > > the
> > >> >>> > > >> >> > > > >>> > > > > contribution).
> > >> >>> > > >> >> > > > >>> > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > Kind regards,
> > >> >>> > > >> >> > > > >>> > > > > > > > > Emmanouil
> > >> >>> > > >> >> > > > >>> > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > [1]
> > >> >>> > > >> >> https://cloud.google.com/pubsub/docs/overview
> > >> >>> > > >> >> > > > >>> > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
> > >> >>> > > >> >> > > > >>> > > > > > > > > https://egkatzioura.com/ |
> > >> >>> > > >> >> > > > >>> > > > > > > > >
> > >> >>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> >>> > > >> >> > > > >>> > > > > > > > > https://github.com/gkatzioura
> > >> >>> > > >> >> > > > >>> > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at 21:45,
> > >> Denis
> > >> >>> > Magda <
> > >> >>> > > >> >> > > > >>> [hidden email]
> > >> >>> > > >> >> > > > >>> > >
> > >> >>> > > >> >> > > > >>> > > > > wrote:
> > >> >>> > > >> >> > > > >>> > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > > Hello Emmanouil,
> > >> >>> > > >> >> > > > >>> > > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > > Are you referring to any
> > >> specific
> > >> >>> > > >> >> implementation
> > >> >>> > > >> >> > > > >>> tailored
> > >> >>> > > >> >> > > > >>> > > for a
> > >> >>> > > >> >> > > > >>> > > > > > > cloud?
> > >> >>> > > >> >> > > > >>> > > > > > > > If
> > >> >>> > > >> >> > > > >>> > > > > > > > > > I'm not mistaken you were
> > >> >>> mentioning
> > >> >>> > > >> AWS/GCE.
> > >> >>> > > >> >> Is
> > >> >>> > > >> >> > > > there
> > >> >>> > > >> >> > > > >>> any
> > >> >>> > > >> >> > > > >>> > > > Github
> > >> >>> > > >> >> > > > >>> > > > > > > > source
> > >> >>> > > >> >> > > > >>> > > > > > > > > > code?
> > >> >>> > > >> >> > > > >>> > > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > > Here is a list of existing
> > >> >>> integrations
> > >> >>> > > >> with
> > >> >>> > > >> >> > > > streaming
> > >> >>> > > >> >> > > > >>> and
> > >> >>> > > >> >> > > > >>> > > > > pub/sub
> > >> >>> > > >> >> > > > >>> > > > > > > > > > frameworks:
> > >> >>> > > >> >> > > > >>> > > > > > > > > >
> > >> >>> > > >> >> https://apacheignite-mix.readme.io/docs/overview
> > >> >>> > > >> >> > > > >>> > > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > > with the source code in
> Ignite
> > >> >>> master:
> > >> >>> > > >> >> > > > >>> > > > > > > > > >
> > >> >>> > > >> >> > >
> https://github.com/apache/ignite/tree/master/modules
> > >> >>> > > >> >> > > > >>> > > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > > Check out those resources to
> > see
> > >> >>> what
> > >> >>> > are
> > >> >>> > > >> the
> > >> >>> > > >> >> > most
> > >> >>> > > >> >> > > > >>> suitable
> > >> >>> > > >> >> > > > >>> > > > > Ignite
> > >> >>> > > >> >> > > > >>> > > > > > > APIs
> > >> >>> > > >> >> > > > >>> > > > > > > > > for
> > >> >>> > > >> >> > > > >>> > > > > > > > > > such integrations.
> > >> >>> > > >> >> > > > >>> > > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > > @Saikat Maitra <
> > >> >>> > [hidden email]>
> > >> >>> > > ,
> > >> >>> > > >> >> > @Roman
> > >> >>> > > >> >> > > <
> > >> >>> > > >> >> > > > >>> > > > > > [hidden email]
> > >> >>> > > >> >> > > > >>> > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > ,
> > >> >>> > > >> >> > > > >>> > > > > > > > > as
> > >> >>> > > >> >> > > > >>> > > > > > > > > > streaming technologies
> > >> >>> > > >> >> contributors/maintainers,
> > >> >>> > > >> >> > > > could
> > >> >>> > > >> >> > > > >>> you
> > >> >>> > > >> >> > > > >>> > > > please
> > >> >>> > > >> >> > > > >>> > > > > > > join
> > >> >>> > > >> >> > > > >>> > > > > > > > > this
> > >> >>> > > >> >> > > > >>> > > > > > > > > > thread?
> > >> >>> > > >> >> > > > >>> > > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > > -
> > >> >>> > > >> >> > > > >>> > > > > > > > > > Denis
> > >> >>> > > >> >> > > > >>> > > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > > On Thu, Oct 10, 2019 at 7:58
> > AM
> > >> >>> > Emmanouil
> > >> >>> > > >> >> > > > Gkatziouras <
> > >> >>> > > >> >> > > > >>> > > > > > > > > > [hidden email]>
> > >> >>> > > >> >> > > > >>> > > > > > > > > > wrote:
> > >> >>> > > >> >> > > > >>> > > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > > > Hello everyone,
> > >> >>> > > >> >> > > > >>> > > > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > > > I started using Ignite
> > lately.
> > >> >>> Part
> > >> >>> > of
> > >> >>> > > my
> > >> >>> > > >> >> work
> > >> >>> > > >> >> > > > >>> involves
> > >> >>> > > >> >> > > > >>> > > > > Pub/Sub,
> > >> >>> > > >> >> > > > >>> > > > > > > > thus I
> > >> >>> > > >> >> > > > >>> > > > > > > > > > > created a pub/sub
> streamer.
> > >> >>> > > >> >> > > > >>> > > > > > > > > > > Pub/Sub is pretty close to
> > >> Kafka.
> > >> >>> > > >> >> > > > >>> > > > > > > > > > > Here's how the streamer
> > works.
> > >> >>> Every
> > >> >>> > > node
> > >> >>> > > >> >> > creates
> > >> >>> > > >> >> > > > an
> > >> >>> > > >> >> > > > >>> > > instance
> > >> >>> > > >> >> > > > >>> > > > > of
> > >> >>> > > >> >> > > > >>> > > > > > a
> > >> >>> > > >> >> > > > >>> > > > > > > > > > Pub/Sub
> > >> >>> > > >> >> > > > >>> > > > > > > > > > > client.
> > >> >>> > > >> >> > > > >>> > > > > > > > > > > The clients subscribe to
> the
> > >> same
> > >> >>> > topic
> > >> >>> > > >> >> using
> > >> >>> > > >> >> > the
> > >> >>> > > >> >> > > > >>> same
> > >> >>> > > >> >> > > > >>> > > > > > subscription
> > >> >>> > > >> >> > > > >>> > > > > > > > > name.
> > >> >>> > > >> >> > > > >>> > > > > > > > > > > A message is being sent
> and
> > >> one
> > >> >>> of
> > >> >>> > the
> > >> >>> > > >> nodes
> > >> >>> > > >> >> > will
> > >> >>> > > >> >> > > > >>> receive
> > >> >>> > > >> >> > > > >>> > > the
> > >> >>> > > >> >> > > > >>> > > > > > > > message,
> > >> >>> > > >> >> > > > >>> > > > > > > > > > > process it and then
> forward
> > >> it to
> > >> >>> > > Ignite
> > >> >>> > > >> >> using
> > >> >>> > > >> >> > > the
> > >> >>> > > >> >> > > > >>> > > extractor
> > >> >>> > > >> >> > > > >>> > > > > > > > provided.
> > >> >>> > > >> >> > > > >>> > > > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > > > Any thoughts?
> > >> >>> > > >> >> > > > >>> > > > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > > > *Emmanouil Gkatziouras*
> > >> >>> > > >> >> > > > >>> > > > > > > > > > > https://egkatzioura.com/
> |
> > >> >>> > > >> >> > > > >>> > > > > > > > > > >
> > >> >>> > > >> >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> >>> > > >> >> > > > >>> > > > > > > > > > >
> > https://github.com/gkatzioura
> > >> >>> > > >> >> > > > >>> > > > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > >
> > >> >>> > > >> >> > > > >>> > > > >
> > >> >>> > > >> >> > > > >>> > > >
> > >> >>> > > >> >> > > > >>> > >
> > >> >>> > > >> >> > > > >>> >
> > >> >>> > > >> >> > > > >>>
> > >> >>> > > >> >> > > > >>
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > >
> > >> >>> > > >> >> >
> > >> >>> > > >> >>
> > >> >>> > > >> >
> > >> >>> > > >>
> > >> >>> > > >
> > >> >>> > >
> > >> >>> >
> > >> >>>
> > >> >>
> > >>
> > >
> >
>

[Saikat Maitra]

Hi,

We will need to add test suites for tests to be executed in build. We pass
them as TestSuites param. I had added Flink sink and source testsuites.

Can you please review and confirm.

Regards
Saikat

On Wed, 27 Nov 2019 at 6:00 AM, Emmanouil Gkatziouras <[hidden email]>
wrote:

> Hi all!
>
> I was successful building and running my pull requests. It seems that the
> tests do not run both for Flink and Pub/Sub [1]
> If there is something I cannot do to make them discoverable please let me
> know.
>
> [1]
>
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4796722&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=283
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Wed, 27 Nov 2019 at 04:57, Saikat Maitra <[hidden email]>
> wrote:
>
> > Hi Emmanouil,
> >
> > I have added you as contributor in Ignite Extensions project. Can you
> > please check and confirm if you are able to see the project and execute
> > build on your pull request.
> >
> > Regards,
> > Saikat
> >
> > On Tue, Nov 26, 2019 at 12:49 PM Emmanouil Gkatziouras <
> > [hidden email]>
> > wrote:
> >
> > > Hi all!
> > >
> > > If someone could give me read access to the Job of Ignite Extensions on
> > > Team City will greatly help me (username gkatzioura).
> > > I suppose the builds get triggered automatically on a pull request.
> > >
> > > Kind regards,
> > > Emmanouil
> > >
> > > *Emmanouil Gkatziouras*
> > > https://egkatzioura.com/ |
> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > https://github.com/gkatzioura
> > >
> > >
> > > On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras <
> > [hidden email]>
> > > wrote:
> > >
> > > > Hi Saikat!
> > > >
> > > > I just rebased with the flink branch. Unfortunately I do not have
> read
> > > > access to the team city link you provided and thus evaluate the tests
> > for
> > > > my pull request.
> > > > I guess it has to do with ignite extensions being new.
> > > > My username is gkatzioura and email the one I am using on this email
> > > >
> > > > Thank you for your time.
> > > >
> > > >
> > > > *Emmanouil Gkatziouras*
> > > > https://egkatzioura.com/ |
> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > https://github.com/gkatzioura
> > > >
> > > >
> > > > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <[hidden email]
> >
> > > > wrote:
> > > >
> > > >> Hi Emmanouil,
> > > >>
> > > >> The latest build on teamcity has passed on Flink pull request.
> > > >>
> > > >>
> > > >>
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> > > >>
> > > >> you should be able to take the changes I made in my PR and run build
> > on
> > > >> your pull request.
> > > >>
> > > >> Please let me know if you have any questions.
> > > >>
> > > >> Regards,
> > > >> Saikat
> > > >>
> > > >> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <
> > [hidden email]
> > > >
> > > >> wrote:
> > > >>
> > > >> > Hi Emmanouil,
> > > >> >
> > > >> > I have fixed the Flink module testsuites and have setup new
> project
> > in
> > > >> > teamcity.
> > > >> >
> > > >> >
> > > >> >
> > > >>
> > >
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
> > > >> >
> > > >> > I am looking into setting up the build.
> > > >> >
> > > >> > Regards,
> > > >> > Saikat
> > > >> >
> > > >> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <
> > > [hidden email]>
> > > >> > wrote:
> > > >> >
> > > >> >> Hi Emmanouil,
> > > >> >>
> > > >> >> Thank you for your email. Yes, the plan is once the Flink PR is
> > > merged
> > > >> it
> > > >> >> will provide the Licence, parent POMs etc and you can rebase from
> > > >> master
> > > >> >> branch and apply the changes on top of it.
> > > >> >>
> > > >> >> I am getting some issues with test failures in local with
> > > >> >> GridTestProperties as the test.properties is not present in this
> > new
> > > >> >> project but should be available from dependencies.
> > > >> >>
> > > >> >> Once I address this issue, I will go ahead and merge the changes
> > and
> > > >> then
> > > >> >> we can take it from there.
> > > >> >>
> > > >> >> Regards,
> > > >> >> Saikat
> > > >> >>
> > > >> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
> > > >> >> [hidden email]> wrote:
> > > >> >>
> > > >> >>> Hi all,
> > > >> >>>
> > > >> >>> I made my first pull request [1]. Since this project is brand
> new
> > > (no
> > > >> >>> parent poms, licensing), I reckoned it was best to use Saikat's
> > > >> branch on
> > > >> >>> Flink.
> > > >> >>> I suppose the Flink branch will be merged first. If not please
> > give
> > > me
> > > >> >>> guidelines on how I should proceed next.
> > > >> >>>
> > > >> >>> Kind regards
> > > >> >>> Emmanouil
> > > >> >>>
> > > >> >>> [1] https://github.com/apache/ignite-extensions/pull/2
> > > >> >>>
> > > >> >>> *Emmanouil Gkatziouras*
> > > >> >>> https://egkatzioura.com/ |
> > > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> >>> https://github.com/gkatzioura
> > > >> >>>
> > > >> >>>
> > > >> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <[hidden email]>
> > > wrote:
> > > >> >>>
> > > >> >>> > Awesome, ping us whenever you're ready!
> > > >> >>> >
> > > >> >>> > -
> > > >> >>> > Denis
> > > >> >>> >
> > > >> >>> >
> > > >> >>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
> > > >> >>> > [hidden email]>
> > > >> >>> > wrote:
> > > >> >>> >
> > > >> >>> > > Hi all!
> > > >> >>> > >
> > > >> >>> > > I am sorry for being late on that. I was trying to refactor
> > the
> > > >> test
> > > >> >>> in
> > > >> >>> > > order not to be in need of any external tools or spinning
> up a
> > > >> >>> server.
> > > >> >>> > > I did forked the new repo and indeed my changes there, so a
> > pull
> > > >> >>> request
> > > >> >>> > is
> > > >> >>> > > a matter of time!
> > > >> >>> > >
> > > >> >>> > > Kind regards
> > > >> >>> > >
> > > >> >>> > > *Emmanouil Gkatziouras*
> > > >> >>> > > https://egkatzioura.com/ |
> > > >> >>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> >>> > > https://github.com/gkatzioura
> > > >> >>> > >
> > > >> >>> > >
> > > >> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <
> [hidden email]>
> > > >> wrote:
> > > >> >>> > >
> > > >> >>> > > > Hi Emmanouil,
> > > >> >>> > > >
> > > >> >>> > > > Do you have any questions or need any support from the
> > > >> community?
> > > >> >>> > > >
> > > >> >>> > > > -
> > > >> >>> > > > Denis
> > > >> >>> > > >
> > > >> >>> > > >
> > > >> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
> > > >> >>> [hidden email]
> > > >> >>> > >
> > > >> >>> > > > wrote:
> > > >> >>> > > >
> > > >> >>> > > >> Hi Emmanouil,
> > > >> >>> > > >>
> > > >> >>> > > >> Can you please take a looks at dev utils, if this is
> > > something
> > > >> >>> you are
> > > >> >>> > > >> looking for
> > > >> >>> > > >>
> > > https://github.com/apache/ignite/tree/master/modules/dev-utils
> > > >> ?
> > > >> >>> > > >>
> > > >> >>> > > >> IMO, if you can release Pub/Sub server in maven and then
> > use
> > > >> it as
> > > >> >>> > > >> dependency, that would be great.
> > > >> >>> > > >>
> > > >> >>> > > >> Regards,
> > > >> >>> > > >> Saikat
> > > >> >>> > > >>
> > > >> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
> > > >> >>> > [hidden email]>
> > > >> >>> > > >> wrote:
> > > >> >>> > > >>
> > > >> >>> > > >> > Hi Emmanouil,
> > > >> >>> > > >> >
> > > >> >>> > > >> > The master branch has been initialized. I have raised a
> > PR
> > > to
> > > >> >>> > migrate
> > > >> >>> > > >> > Flink module to ignite-extensions.
> > > >> >>> > > >> >
> > > >> >>> > > >> > PR https://github.com/apache/ignite-extensions/pull/1
> > > >> >>> > > >> > Jira :
> > https://issues.apache.org/jira/browse/IGNITE-12356
> > > >> >>> > > >> >
> > > >> >>> > > >> > Regards,
> > > >> >>> > > >> > Saikat
> > > >> >>> > > >> >
> > > >> >>> > > >> >
> > > >> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
> > > >> >>> > > >> > [hidden email]> wrote:
> > > >> >>> > > >> >
> > > >> >>> > > >> >> Hi all!
> > > >> >>> > > >> >>
> > > >> >>> > > >> >> This is an update. The integration is tested with
> mocks
> > > >> based
> > > >> >>> on
> > > >> >>> > the
> > > >> >>> > > >> >> original Ignite repo [1].
> > > >> >>> > > >> >> I am currently waiting for the first commits on the
> new
> > > >> >>> integration
> > > >> >>> > > >> >> project
> > > >> >>> > > >> >> to take place in order to create my pull request
> > > accordingly
> > > >> >>> to the
> > > >> >>> > > >> >> initial
> > > >> >>> > > >> >> commits.
> > > >> >>> > > >> >> In order to simulate the Pub/Sub server in a test I
> > used a
> > > >> test
> > > >> >>> > > server
> > > >> >>> > > >> >> implementation [2]. Unfortunately they don't have this
> > > util
> > > >> >>> > uploaded
> > > >> >>> > > >> in a
> > > >> >>> > > >> >> maven repo, thus it needs to be hosted on a maven repo
> > > >> (Apache
> > > >> >>> > > License
> > > >> >>> > > >> >> 2.0). Is there a repo for utilities that Ignite uses
> for
> > > the
> > > >> >>> > > >> test/build?
> > > >> >>> > > >> >> I have also created a ticket [3]
> > > >> >>> > > >> >>
> > > >> >>> > > >> >> Kind regards
> > > >> >>> > > >> >>
> > > >> >>> > > >> >> [1]
> > > >> >>> >
> https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > > >> >>> > > >> >> [2]
> > > >> >>> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> > > >> >>> > > >> >> [3]
> https://issues.apache.org/jira/browse/IGNITE-12262
> > > >> >>> > > >> >>
> > > >> >>> > > >> >> *Emmanouil Gkatziouras*
> > > >> >>> > > >> >> https://egkatzioura.com/ |
> > > >> >>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> >>> > > >> >> https://github.com/gkatzioura
> > > >> >>> > > >> >>
> > > >> >>> > > >> >>
> > > >> >>> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
> > > >> >>> > [hidden email]
> > > >> >>> > > >
> > > >> >>> > > >> >> wrote:
> > > >> >>> > > >> >>
> > > >> >>> > > >> >> > Hello Ilya,
> > > >> >>> > > >> >> >
> > > >> >>> > > >> >> > Thank you for your email. Yes, before we remove any
> > > >> >>> integration
> > > >> >>> > we
> > > >> >>> > > >> will
> > > >> >>> > > >> >> > move them to another repository and release.
> > > >> >>> > > >> >> >
> > > >> >>> > > >> >> > We are thinking we will start with Pub/Sub Streamer
> > > >> >>> > implementation
> > > >> >>> > > >> and
> > > >> >>> > > >> >> > continue to move other integration as mentioned in
> the
> > > >> >>> following
> > > >> >>> > > doc.
> > > >> >>> > > >> >> >
> > > >> >>> > > >> >> >
> > > >> >>> > > >> >> >
> > > >> >>> > > >> >>
> > > >> >>> > > >>
> > > >> >>> > >
> > > >> >>> >
> > > >> >>>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > >> >>> > > >> >> >
> > > >> >>> > > >> >> > Regards,
> > > >> >>> > > >> >> > Saikat
> > > >> >>> > > >> >> >
> > > >> >>> > > >> >> >
> > > >> >>> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
> > > >> >>> > > >> >> > [hidden email]>
> > > >> >>> > > >> >> > wrote:
> > > >> >>> > > >> >> >
> > > >> >>> > > >> >> > > Hello!
> > > >> >>> > > >> >> > >
> > > >> >>> > > >> >> > > My take on this: we want to offload some of
> > > >> integrations,
> > > >> >>> but
> > > >> >>> > > >> please
> > > >> >>> > > >> >> > commit
> > > >> >>> > > >> >> > > and release them to another project first, and
> only
> > > >> then we
> > > >> >>> > will
> > > >> >>> > > >> >> decide
> > > >> >>> > > >> >> > to
> > > >> >>> > > >> >> > > remove them from our repository.
> > > >> >>> > > >> >> > >
> > > >> >>> > > >> >> > > Please also make sure they are available with same
> > > >> artifact
> > > >> >>> > name
> > > >> >>> > > >> and
> > > >> >>> > > >> >> API.
> > > >> >>> > > >> >> > > If this has to be changed, let's wait for AI 3.0
> > > >> >>> > > >> >> > >
> > > >> >>> > > >> >> > > Regards,
> > > >> >>> > > >> >> > > --
> > > >> >>> > > >> >> > > Ilya Kasnacheev
> > > >> >>> > > >> >> > >
> > > >> >>> > > >> >> > >
> > > >> >>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <
> > > >> >>> > > >> [hidden email]>:
> > > >> >>> > > >> >> > >
> > > >> >>> > > >> >> > > > Hello Emmanouil,
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > > As discussed earlier, I discussed with Apache
> > Bahir
> > > >> >>> community
> > > >> >>> > > and
> > > >> >>> > > >> >> they
> > > >> >>> > > >> >> > > are
> > > >> >>> > > >> >> > > > interested to have Apache Ignite extensions as
> > part
> > > of
> > > >> >>> Apache
> > > >> >>> > > >> Bahir
> > > >> >>> > > >> >> > > > project.
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > > Can you please share Pub/Sub streamer
> > implementation
> > > >> >>> details
> > > >> >>> > > with
> > > >> >>> > > >> >> > Apache
> > > >> >>> > > >> >> > > > Bahir community and request for feedback.
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > > https://bahir.apache.org/
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > > I have also requested for contributor access in
> > Jira
> > > >> for
> > > >> >>> > Apache
> > > >> >>> > > >> >> Bahir
> > > >> >>> > > >> >> > > > project so that I can create issues and assign
> to
> > > >> myself.
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > > I can help with code reviews as well.
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > > Here is the email thread for reference
> > > >> >>> > > >> >> > > >
> > > >> >>> > >
> > https://www.mail-archive.com/dev@.../msg02703.html
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > > Regards,
> > > >> >>> > > >> >> > > > Saikat
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
> > > >> >>> > > >> >> [hidden email]
> > > >> >>> > > >> >> > >
> > > >> >>> > > >> >> > > > wrote:
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > > > Hello,
> > > >> >>> > > >> >> > > > >
> > > >> >>> > > >> >> > > > > As discussed I have created following
> discussion
> > > >> >>> threads on
> > > >> >>> > > our
> > > >> >>> > > >> >> > > migration
> > > >> >>> > > >> >> > > > > proposals for Apache Ignite extensions:
> > > >> >>> > > >> >> > > > >
> > > >> >>> > > >> >> > > > >
> > > >> >>> > > >> >> > > > >
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > >
> > > >> >>> > > >> >> >
> > > >> >>> > > >> >>
> > > >> >>> > > >>
> > > >> >>> > >
> > > >> >>> >
> > > >> >>>
> > > >>
> > >
> >
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> > > >> >>> > > >> >> > > > >
> > > >> >>> > > >> >> > > > >
> > > >> >>> > > >>
> > > >> https://www.mail-archive.com/dev@.../msg02703.html
> > > >> >>> > > >> >> > > > >
> > > >> >>> > > >> >> > > > > I will share update as I hear more
> information.
> > > >> >>> > > >> >> > > > >
> > > >> >>> > > >> >> > > > > Regards,
> > > >> >>> > > >> >> > > > > Saikat
> > > >> >>> > > >> >> > > > >
> > > >> >>> > > >> >> > > > >
> > > >> >>> > > >> >> > > > >
> > > >> >>> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra
> <
> > > >> >>> > > >> >> > [hidden email]
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > > > wrote:
> > > >> >>> > > >> >> > > > >
> > > >> >>> > > >> >> > > > >> Hello Denis,
> > > >> >>> > > >> >> > > > >>
> > > >> >>> > > >> >> > > > >> Sure, sounds good. I will create a separate
> > > >> discussion
> > > >> >>> > > thread
> > > >> >>> > > >> to
> > > >> >>> > > >> >> get
> > > >> >>> > > >> >> > > > >> community feedback on whether we
> > > >> >>> > > >> >> > > > >> should join the Bahir or kick-off "Ignite
> > > >> Extensions"
> > > >> >>> > > project.
> > > >> >>> > > >> >> > > > >>
> > > >> >>> > > >> >> > > > >> Regards,
> > > >> >>> > > >> >> > > > >> Saikat
> > > >> >>> > > >> >> > > > >>
> > > >> >>> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <
> > > >> >>> > > >> [hidden email]>
> > > >> >>> > > >> >> > > wrote:
> > > >> >>> > > >> >> > > > >>
> > > >> >>> > > >> >> > > > >>> Folks,
> > > >> >>> > > >> >> > > > >>>
> > > >> >>> > > >> >> > > > >>> The concept of Apache Bahir is precisely
> what
> > we
> > > >> >>> need!
> > > >> >>> > > >> Saikat,
> > > >> >>> > > >> >> > thanks
> > > >> >>> > > >> >> > > > for
> > > >> >>> > > >> >> > > > >>> doing the research. Why I believe the *
> idea*
> > > >> fits us
> > > >> >>> > well:
> > > >> >>> > > >> >> > > > >>>
> > > >> >>> > > >> >> > > > >>>    - All integrations can be stored in
> > separate
> > > >> >>> Github
> > > >> >>> > > >> >> repositories
> > > >> >>> > > >> >> > > and
> > > >> >>> > > >> >> > > > >>>    have their dev lifecycles. We've not
> > obliged
> > > to
> > > >> >>> couple
> > > >> >>> > > all
> > > >> >>> > > >> >> the
> > > >> >>> > > >> >> > > > >>> integrations
> > > >> >>> > > >> >> > > > >>>    in a single repo. For instance, Spark can
> > be
> > > >> >>> located
> > > >> >>> > in
> > > >> >>> > > a
> > > >> >>> > > >> >> > > dedicated
> > > >> >>> > > >> >> > > > >>> repo
> > > >> >>> > > >> >> > > > >>>    while streaming integrations might be in
> a
> > > >> single
> > > >> >>> one.
> > > >> >>> > > >> It's
> > > >> >>> > > >> >> up
> > > >> >>> > > >> >> > to
> > > >> >>> > > >> >> > > > us.
> > > >> >>> > > >> >> > > > >>>    - All the repositories and integrations
> > > belong
> > > >> to
> > > >> >>> ASF.
> > > >> >>> > > >> We're
> > > >> >>> > > >> >> not
> > > >> >>> > > >> >> > > > >>> dumping
> > > >> >>> > > >> >> > > > >>>    them on Github but rather keep supporting
> > and
> > > >> >>> > developing
> > > >> >>> > > >> in
> > > >> >>> > > >> >> > > > >>> accordance with
> > > >> >>> > > >> >> > > > >>>    ASF vision and practices.
> > > >> >>> > > >> >> > > > >>>    - There is a way to reward contributors
> via
> > > >> >>> > > committership
> > > >> >>> > > >> and
> > > >> >>> > > >> >> > PMC
> > > >> >>> > > >> >> > > > >>>    membership. You won't get this if a
> project
> > > is
> > > >> >>> just
> > > >> >>> > one
> > > >> >>> > > of
> > > >> >>> > > >> >> the
> > > >> >>> > > >> >> > > > >>> millions of
> > > >> >>> > > >> >> > > > >>>    Github projects.
> > > >> >>> > > >> >> > > > >>>
> > > >> >>> > > >> >> > > > >>> Saikat, as Ignite PMC member, could you
> please
> > > >> >>> kick-off a
> > > >> >>> > > >> >> separate
> > > >> >>> > > >> >> > > > >>> dev-list
> > > >> >>> > > >> >> > > > >>> discussion to involve more community members
> > and
> > > >> >>> > referring
> > > >> >>> > > to
> > > >> >>> > > >> >> this
> > > >> >>> > > >> >> > > > >>> thread?
> > > >> >>> > > >> >> > > > >>> I think the primary question the community
> > needs
> > > >> to
> > > >> >>> > answer
> > > >> >>> > > >> >> whether
> > > >> >>> > > >> >> > we
> > > >> >>> > > >> >> > > > >>> should join the Bahir or kick-off "Ignite
> > > >> Extensions"
> > > >> >>> > > >> project?
> > > >> >>> > > >> >> > > > >>>
> > > >> >>> > > >> >> > > > >>> -
> > > >> >>> > > >> >> > > > >>> Denis
> > > >> >>> > > >> >> > > > >>>
> > > >> >>> > > >> >> > > > >>>
> > > >> >>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey
> > Zinoviev
> > > <
> > > >> >>> > > >> >> > > > [hidden email]>
> > > >> >>> > > >> >> > > > >>> wrote:
> > > >> >>> > > >> >> > > > >>>
> > > >> >>> > > >> >> > > > >>> > Maybe we could move all our Streaming
> > > >> Integrations
> > > >> >>> > there,
> > > >> >>> > > >> but
> > > >> >>> > > >> >> > what
> > > >> >>> > > >> >> > > is
> > > >> >>> > > >> >> > > > >>> about
> > > >> >>> > > >> >> > > > >>> > maintaining and committer permissions to
> the
> > > new
> > > >> >>> > > >> repositories?
> > > >> >>> > > >> >> > > > >>> > I see the list of the committers and PMC
> > > members
> > > >> >>> there
> > > >> >>> > > >> >> > > > >>> >
> https://bahir.apache.org/community-members/
> > > >> >>> > > >> >> > > > >>> >
> > > >> >>> > > >> >> > > > >>> > Could somebody from Ignite community be
> > added
> > > to
> > > >> >>> this
> > > >> >>> > > list
> > > >> >>> > > >> as
> > > >> >>> > > >> >> a
> > > >> >>> > > >> >> > > > >>> > PMC/committer to maintain Ignite
> > integrations?
> > > >> >>> > > >> >> > > > >>> > If yes, I'd happy to join this project and
> > > >> >>> collaborate
> > > >> >>> > > with
> > > >> >>> > > >> >> these
> > > >> >>> > > >> >> > > > guys
> > > >> >>> > > >> >> > > > >>> > together
> > > >> >>> > > >> >> > > > >>> > If no, and we should start from the zero
> > level
> > > >> with
> > > >> >>> > > >> external
> > > >> >>> > > >> >> PRs
> > > >> >>> > > >> >> > -
> > > >> >>> > > >> >> > > > >>> hmmm,
> > > >> >>> > > >> >> > > > >>> > it's better to have our own external
> > > repository
> > > >> >>> with
> > > >> >>> > > >> >> ApacheBahirr
> > > >> >>> > > >> >> > > > >>> ideology.
> > > >> >>> > > >> >> > > > >>> >
> > > >> >>> > > >> >> > > > >>> > Also, I agree, that all connectors could
> be
> > > >> moved
> > > >> >>> there
> > > >> >>> > > (in
> > > >> >>> > > >> >> > > > >>> ApacheBahirr
> > > >> >>> > > >> >> > > > >>> > like repository), but not all modules from
> > the
> > > >> page
> > > >> >>> > > >> >> > > > >>> >
> > > >> >>> > > >> >> > > > >>> >
> > > >> >>> > > >> >> > > > >>>
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > >
> > > >> >>> > > >> >> >
> > > >> >>> > > >> >>
> > > >> >>> > > >>
> > > >> >>> > >
> > > >> >>> >
> > > >> >>>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > >> >>> > > >> >> > > > >>> > because
> > > >> >>> > > >> >> > > > >>> > they use different parts of core modules
> and
> > > >> could
> > > >> >>> be
> > > >> >>> > > >> >> developed
> > > >> >>> > > >> >> > > with
> > > >> >>> > > >> >> > > > >>> > different velocity.
> > > >> >>> > > >> >> > > > >>> >
> > > >> >>> > > >> >> > > > >>> > In reality, before creation of new
> > > repositories
> > > >> we
> > > >> >>> need
> > > >> >>> > > >> wide
> > > >> >>> > > >> >> > > > discussion
> > > >> >>> > > >> >> > > > >>> > based on the document mentioned above.
> > > >> >>> > > >> >> > > > >>> >
> > > >> >>> > > >> >> > > > >>> > P.S. Of course, we could experiment in the
> > > next
> > > >> >>> release
> > > >> >>> > > 2.8
> > > >> >>> > > >> >> with
> > > >> >>> > > >> >> > > one
> > > >> >>> > > >> >> > > > >>> or two
> > > >> >>> > > >> >> > > > >>> > integrations like twitter or ZeroMQ
> > > >> >>> > > >> >> > > > >>> > Also, I'd like Denis Magda idea for the
> > > separate
> > > >> >>> > > >> repositories
> > > >> >>> > > >> >> in
> > > >> >>> > > >> >> > > > Apache
> > > >> >>> > > >> >> > > > >>> > Github to maintain them by anybody.
> > > >> >>> > > >> >> > > > >>> >
> > > >> >>> > > >> >> > > > >>> >
> > > >> >>> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat
> Maitra <
> > > >> >>> > > >> >> > > [hidden email]
> > > >> >>> > > >> >> > > > >:
> > > >> >>> > > >> >> > > > >>> >
> > > >> >>> > > >> >> > > > >>> > > Hi Denis, Emmanouil
> > > >> >>> > > >> >> > > > >>> > >
> > > >> >>> > > >> >> > > > >>> > > Thank you for your email. I think
> > creating a
> > > >> >>> separate
> > > >> >>> > > >> >> > integration
> > > >> >>> > > >> >> > > > >>> project
> > > >> >>> > > >> >> > > > >>> > > in github and also proposing it as an
> > apache
> > > >> >>> > incubator
> > > >> >>> > > >> >> project
> > > >> >>> > > >> >> > > will
> > > >> >>> > > >> >> > > > >>> be a
> > > >> >>> > > >> >> > > > >>> > > good move. The other separate
> integration
> > > >> modules
> > > >> >>> > > >> >> > > > >>> > >
> > > >> >>> > > >> >> > > > >>> > >
> > > >> >>> > > >> >> > > > >>> >
> > > >> >>> > > >> >> > > > >>>
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > >
> > > >> >>> > > >> >> >
> > > >> >>> > > >> >>
> > > >> >>> > > >>
> > > >> >>> > >
> > > >> >>> >
> > > >> >>>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > >> >>> > > >> >> > > > >>> > > can
> > > >> >>> > > >> >> > > > >>> > > be moved to this new apache incubator
> > > project.
> > > >> >>> > > >> >> > > > >>> > >
> > > >> >>> > > >> >> > > > >>> > > There are similar integration available
> > for
> > > >> >>> Flink and
> > > >> >>> > > >> Spark
> > > >> >>> > > >> >> in
> > > >> >>> > > >> >> > > > Apache
> > > >> >>> > > >> >> > > > >>> > Bahir
> > > >> >>> > > >> >> > > > >>> > > https://bahir.apache.org/
> > > >> >>> > > >> >> > > > >>> > >
> > > >> >>> > > >> >> > > > >>> > > Do you think we should reach out to
> Apache
> > > >> Bahir
> > > >> >>> > > >> community
> > > >> >>> > > >> >> > with a
> > > >> >>> > > >> >> > > > >>> > proposal
> > > >> >>> > > >> >> > > > >>> > > or we should plan to create a separate
> > > Apache
> > > >> >>> > Incubator
> > > >> >>> > > >> >> > project?
> > > >> >>> > > >> >> > > > >>> > >
> > > >> >>> > > >> >> > > > >>> > > Regards,
> > > >> >>> > > >> >> > > > >>> > > Saikat
> > > >> >>> > > >> >> > > > >>> > >
> > > >> >>> > > >> >> > > > >>> > >
> > > >> >>> > > >> >> > > > >>> > >
> > > >> >>> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM
> Emmanouil
> > > >> >>> > Gkatziouras <
> > > >> >>> > > >> >> > > > >>> > > [hidden email]>
> > > >> >>> > > >> >> > > > >>> > > wrote:
> > > >> >>> > > >> >> > > > >>> > >
> > > >> >>> > > >> >> > > > >>> > > > Hi all!
> > > >> >>> > > >> >> > > > >>> > > >
> > > >> >>> > > >> >> > > > >>> > > > Based on the discussions most probably
> > > this
> > > >> is
> > > >> >>> > going
> > > >> >>> > > to
> > > >> >>> > > >> >> be on
> > > >> >>> > > >> >> > > > >>> another
> > > >> >>> > > >> >> > > > >>> > > > GitHub repo. Therefore I will prepare
> a
> > > >> >>> standalone
> > > >> >>> > > >> project
> > > >> >>> > > >> >> > with
> > > >> >>> > > >> >> > > > the
> > > >> >>> > > >> >> > > > >>> > > Pub/Sub
> > > >> >>> > > >> >> > > > >>> > > > module and once the repository is
> > created
> > > a
> > > >> >>> pull
> > > >> >>> > > >> request
> > > >> >>> > > >> >> > shall
> > > >> >>> > > >> >> > > be
> > > >> >>> > > >> >> > > > >>> > issued.
> > > >> >>> > > >> >> > > > >>> > > > If there is anything else I can do in
> > > order
> > > >> to
> > > >> >>> > > >> facilitate
> > > >> >>> > > >> >> > this
> > > >> >>> > > >> >> > > > >>> please
> > > >> >>> > > >> >> > > > >>> > let
> > > >> >>> > > >> >> > > > >>> > > > me know.
> > > >> >>> > > >> >> > > > >>> > > >
> > > >> >>> > > >> >> > > > >>> > > > If a ticket is created for this
> issue, I
> > > >> shall
> > > >> >>> be
> > > >> >>> > > able
> > > >> >>> > > >> to
> > > >> >>> > > >> >> > > create
> > > >> >>> > > >> >> > > > a
> > > >> >>> > > >> >> > > > >>> pull
> > > >> >>> > > >> >> > > > >>> > > > request and thus a build will take
> place
> > > on
> > > >> >>> > ignite's
> > > >> >>> > > >> CI.
> > > >> >>> > > >> >> > > > >>> > > > If everyone is aligned with adding a
> > > Pub/Sub
> > > >> >>> > > >> >> implementation
> > > >> >>> > > >> >> > > may I
> > > >> >>> > > >> >> > > > >>> > create
> > > >> >>> > > >> >> > > > >>> > > a
> > > >> >>> > > >> >> > > > >>> > > > ticket on JIRA?
> > > >> >>> > > >> >> > > > >>> > > >
> > > >> >>> > > >> >> > > > >>> > > > My next question has to do on how
> Ignite
> > > >> >>> configures
> > > >> >>> > > the
> > > >> >>> > > >> >> build
> > > >> >>> > > >> >> > > > jobs
> > > >> >>> > > >> >> > > > >>> in
> > > >> >>> > > >> >> > > > >>> > > Team
> > > >> >>> > > >> >> > > > >>> > > > City. The unit tests for my
> > implementation
> > > >> >>> need a
> > > >> >>> > > local
> > > >> >>> > > >> >> > Pub/Sub
> > > >> >>> > > >> >> > > > >>> server
> > > >> >>> > > >> >> > > > >>> > up
> > > >> >>> > > >> >> > > > >>> > > > and running (they provide one for
> > testing
> > > >> >>> > purposes).
> > > >> >>> > > >> Will
> > > >> >>> > > >> >> > this
> > > >> >>> > > >> >> > > > take
> > > >> >>> > > >> >> > > > >>> > > effect
> > > >> >>> > > >> >> > > > >>> > > > in a form of a build script inside my
> > > >> >>> > implementation
> > > >> >>> > > or
> > > >> >>> > > >> >> is it
> > > >> >>> > > >> >> > > > >>> something
> > > >> >>> > > >> >> > > > >>> > > > that should be configured on Team
> City?
> > > >> >>> > > >> >> > > > >>> > > >
> > > >> >>> > > >> >> > > > >>> > > > Kind regards,
> > > >> >>> > > >> >> > > > >>> > > > Emmanouil
> > > >> >>> > > >> >> > > > >>> > > >
> > > >> >>> > > >> >> > > > >>> > > >
> > > >> >>> > > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
> > > >> >>> > > >> >> > > > >>> > > > https://egkatzioura.com/ |
> > > >> >>> > > >> >> > > > >>> > > >
> > > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> >>> > > >> >> > > > >>> > > > https://github.com/gkatzioura
> > > >> >>> > > >> >> > > > >>> > > >
> > > >> >>> > > >> >> > > > >>> > > >
> > > >> >>> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis
> > Magda
> > > <
> > > >> >>> > > >> >> [hidden email]
> > > >> >>> > > >> >> > >
> > > >> >>> > > >> >> > > > >>> wrote:
> > > >> >>> > > >> >> > > > >>> > > >
> > > >> >>> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
> > > >> >>> > > >> >> > > > >>> > > > >
> > > >> >>> > > >> >> > > > >>> > > > > After contemplating on this, let me
> > > >> propose
> > > >> >>> > another
> > > >> >>> > > >> way
> > > >> >>> > > >> >> > with
> > > >> >>> > > >> >> > > > this
> > > >> >>> > > >> >> > > > >>> > > > > particular integration that is
> aligned
> > > >> with
> > > >> >>> our
> > > >> >>> > > >> >> > > modularization
> > > >> >>> > > >> >> > > > >>> > > endeavor.
> > > >> >>> > > >> >> > > > >>> > > > > The modularization [1] defines the
> > > Ignite
> > > >> >>> core,
> > > >> >>> > > >> Ignite
> > > >> >>> > > >> >> > > modules,
> > > >> >>> > > >> >> > > > >>> and
> > > >> >>> > > >> >> > > > >>> > > > > independent integrations. The first
> > two
> > > >> are
> > > >> >>> to be
> > > >> >>> > > >> >> located
> > > >> >>> > > >> >> > in
> > > >> >>> > > >> >> > > > >>> Ignite
> > > >> >>> > > >> >> > > > >>> > > > > repositories and to be
> tested/updated
> > by
> > > >> the
> > > >> >>> > > >> community
> > > >> >>> > > >> >> for
> > > >> >>> > > >> >> > > > every
> > > >> >>> > > >> >> > > > >>> > > release.
> > > >> >>> > > >> >> > > > >>> > > > > The modules are a subset of existing
> > > >> >>> integrations
> > > >> >>> > > >> that
> > > >> >>> > > >> >> are
> > > >> >>> > > >> >> > > > >>> adopted
> > > >> >>> > > >> >> > > > >>> > > widely
> > > >> >>> > > >> >> > > > >>> > > > > and require community attention.
> > > >> >>> > > >> >> > > > >>> > > > >
> > > >> >>> > > >> >> > > > >>> > > > > As for the independent integrations
> > [1],
> > > >> >>> those
> > > >> >>> > are
> > > >> >>> > > >> >> supposed
> > > >> >>> > > >> >> > > to
> > > >> >>> > > >> >> > > > be
> > > >> >>> > > >> >> > > > >>> > > located
> > > >> >>> > > >> >> > > > >>> > > > > in a separate GitHub repo. I would
> > > advise
> > > >> us
> > > >> >>> to
> > > >> >>> > > start
> > > >> >>> > > >> >> > > > >>> implementing
> > > >> >>> > > >> >> > > > >>> > this
> > > >> >>> > > >> >> > > > >>> > > > > concept with the Pub/Sub
> integration.
> > > Even
> > > >> >>> though
> > > >> >>> > > it
> > > >> >>> > > >> >> will
> > > >> >>> > > >> >> > be
> > > >> >>> > > >> >> > > > >>> located
> > > >> >>> > > >> >> > > > >>> > > in a
> > > >> >>> > > >> >> > > > >>> > > > > separate Github repo, it will be
> > listed
> > > >> among
> > > >> >>> > > >> officially
> > > >> >>> > > >> >> > > > >>> available
> > > >> >>> > > >> >> > > > >>> > > > > integrations on the Ignite website
> > > >> featuring
> > > >> >>> > > >> Emmanouil
> > > >> >>> > > >> >> as a
> > > >> >>> > > >> >> > > > >>> > > contributor.
> > > >> >>> > > >> >> > > > >>> > > > >
> > > >> >>> > > >> >> > > > >>> > > > > What do you think?
> > > >> >>> > > >> >> > > > >>> > > > >
> > > >> >>> > > >> >> > > > >>> > > > >
> > > >> >>> > > >> >> > > > >>> > > > > [1]
> > > >> >>> > > >> >> > > > >>> > > > >
> > > >> >>> > > >> >> > > > >>> > > > >
> > > >> >>> > > >> >> > > > >>> > > >
> > > >> >>> > > >> >> > > > >>> > >
> > > >> >>> > > >> >> > > > >>> >
> > > >> >>> > > >> >> > > > >>>
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > >
> > > >> >>> > > >> >> >
> > > >> >>> > > >> >>
> > > >> >>> > > >>
> > > >> >>> > >
> > > >> >>> >
> > > >> >>>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > >> >>> > > >> >> > > > >>> > > > >
> > > >> >>> > > >> >> > > > >>> > > > >
> > > >> >>> > > >> >> > > > >>> > > > >
> > > >> >>> > > >> >> > > > >>> > > > > -
> > > >> >>> > > >> >> > > > >>> > > > > Denis
> > > >> >>> > > >> >> > > > >>> > > > >
> > > >> >>> > > >> >> > > > >>> > > > >
> > > >> >>> > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM
> Saikat
> > > >> >>> Maitra <
> > > >> >>> > > >> >> > > > >>> > [hidden email]
> > > >> >>> > > >> >> > > > >>> > > >
> > > >> >>> > > >> >> > > > >>> > > > > wrote:
> > > >> >>> > > >> >> > > > >>> > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > Hi Emmanouil,
> > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > The changes looks good to me. I
> > wanted
> > > >> to
> > > >> >>> check
> > > >> >>> > > if
> > > >> >>> > > >> you
> > > >> >>> > > >> >> > are
> > > >> >>> > > >> >> > > > also
> > > >> >>> > > >> >> > > > >>> > able
> > > >> >>> > > >> >> > > > >>> > > to
> > > >> >>> > > >> >> > > > >>> > > > > > validate that the once data is
> added
> > > to
> > > >> >>> Ignite
> > > >> >>> > > >> cluster
> > > >> >>> > > >> >> > you
> > > >> >>> > > >> >> > > > are
> > > >> >>> > > >> >> > > > >>> also
> > > >> >>> > > >> >> > > > >>> > > > able
> > > >> >>> > > >> >> > > > >>> > > > > to
> > > >> >>> > > >> >> > > > >>> > > > > > access it using another Ignite
> > client
> > > or
> > > >> >>> using
> > > >> >>> > > rest
> > > >> >>> > > >> >> > > > endpoints.
> > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > We use Teamcity for CI process,
> the
> > > >> >>> details are
> > > >> >>> > > as
> > > >> >>> > > >> >> > follows
> > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > Trigger validation of those test
> > > suites
> > > >> >>> that
> > > >> >>> > have
> > > >> >>> > > >> been
> > > >> >>> > > >> >> > > > >>> affected by
> > > >> >>> > > >> >> > > > >>> > > your
> > > >> >>> > > >> >> > > > >>> > > > > > changes on TeamCity <
> > > >> >>> > > http://ci.ignite.apache.org/>:
> > > >> >>> > > >> <
> > > >> >>> > > >> >> > > > >>> > > > > http://204.14.53.153/
> > > >> >>> > > >> >> > > > >>> > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > >    - Locate a test suite you have
> to
> > > >> check,
> > > >> >>> > press
> > > >> >>> > > >> >> button
> > > >> >>> > > >> >> > > > named
> > > >> >>> > > >> >> > > > >>> > "..."
> > > >> >>> > > >> >> > > > >>> > > > that
> > > >> >>> > > >> >> > > > >>> > > > > >    is located on the left of "Run"
> > > >> button.
> > > >> >>> "Run
> > > >> >>> > > >> custom
> > > >> >>> > > >> >> > > build"
> > > >> >>> > > >> >> > > > >>> > dialog
> > > >> >>> > > >> >> > > > >>> > > > will
> > > >> >>> > > >> >> > > > >>> > > > > >    appear;
> > > >> >>> > > >> >> > > > >>> > > > > >    - Go to "Changes" tab and
> choose
> > > >> >>> > > >> >> > > > >>> > "pull/<pull-request-number>/head"
> > > >> >>> > > >> >> > > > >>> > > > in
> > > >> >>> > > >> >> > > > >>> > > > > >    "Build branch" dropdown list;
> > > >> >>> > > >> >> > > > >>> > > > > >    - Press "Run build" button and
> > > >> monitor
> > > >> >>> tests
> > > >> >>> > > >> >> results.
> > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > @Roman [hidden email] - can
> you
> > > also
> > > >> >>> please
> > > >> >>> > > >> take a
> > > >> >>> > > >> >> > > look?
> > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > Regards,
> > > >> >>> > > >> >> > > > >>> > > > > > Saikat
> > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM
> > > >> Emmanouil
> > > >> >>> > > >> Gkatziouras
> > > >> >>> > > >> >> <
> > > >> >>> > > >> >> > > > >>> > > > > > [hidden email]>
> > > >> >>> > > >> >> > > > >>> > > > > > wrote:
> > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
> > > >> >>> > > >> >> > > > >>> > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > I have my implementation on an
> > > Ignite
> > > >> >>> fork on
> > > >> >>> > > my
> > > >> >>> > > >> >> > personal
> > > >> >>> > > >> >> > > > >>> GitHub
> > > >> >>> > > >> >> > > > >>> > > > > account
> > > >> >>> > > >> >> > > > >>> > > > > > > [1]
> > > >> >>> > > >> >> > > > >>> > > > > > > It is based on the Kafka
> > > >> implementation
> > > >> >>> which
> > > >> >>> > > is
> > > >> >>> > > >> of
> > > >> >>> > > >> >> a
> > > >> >>> > > >> >> > > > polling
> > > >> >>> > > >> >> > > > >>> > > nature.
> > > >> >>> > > >> >> > > > >>> > > > > > > GCP does not provide a service
> > like
> > > >> >>> Kafka.
> > > >> >>> > > >> Instead
> > > >> >>> > > >> >> it
> > > >> >>> > > >> >> > > > >>> provides us
> > > >> >>> > > >> >> > > > >>> > > > with
> > > >> >>> > > >> >> > > > >>> > > > > > > Pub/Sub which is just like
> Kafka.
> > > >> >>> > > >> >> > > > >>> > > > > > > Therefore using the Kafka
> streamer
> > > as
> > > >> a
> > > >> >>> > > guideline
> > > >> >>> > > >> >> made
> > > >> >>> > > >> >> > > more
> > > >> >>> > > >> >> > > > >>> sense
> > > >> >>> > > >> >> > > > >>> > > to
> > > >> >>> > > >> >> > > > >>> > > > > me.
> > > >> >>> > > >> >> > > > >>> > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > Apart from the implementation I
> > have
> > > >> also
> > > >> >>> > > added a
> > > >> >>> > > >> >> unit
> > > >> >>> > > >> >> > > > test.
> > > >> >>> > > >> >> > > > >>> > > > > > > It definitely needs some
> polishing
> > > >> and a
> > > >> >>> > double
> > > >> >>> > > >> >> check
> > > >> >>> > > >> >> > on
> > > >> >>> > > >> >> > > my
> > > >> >>> > > >> >> > > > >>> side
> > > >> >>> > > >> >> > > > >>> > to
> > > >> >>> > > >> >> > > > >>> > > > > make
> > > >> >>> > > >> >> > > > >>> > > > > > > the review smooth.
> > > >> >>> > > >> >> > > > >>> > > > > > > For the tests a local Pub/Sub
> [2]
> > > >> server
> > > >> >>> has
> > > >> >>> > to
> > > >> >>> > > >> be
> > > >> >>> > > >> >> > setup
> > > >> >>> > > >> >> > > > >>> however
> > > >> >>> > > >> >> > > > >>> > I
> > > >> >>> > > >> >> > > > >>> > > am
> > > >> >>> > > >> >> > > > >>> > > > > not
> > > >> >>> > > >> >> > > > >>> > > > > > > sure which is the best way to do
> > so
> > > >> with
> > > >> >>> > > regards
> > > >> >>> > > >> to
> > > >> >>> > > >> >> > your
> > > >> >>> > > >> >> > > > >>> CI/CD.
> > > >> >>> > > >> >> > > > >>> > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > [1]
> > > >> >>> > > >> >> > > > >>> > >
> > > >> >>> > > >> >> >
> > > >> >>> https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > > >> >>> > > >> >> > > > >>> > > > > > > [2]
> > > >> >>> > > >> >> > > > >>>
> > > >> >>> > https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
> > > >> >>> > > >> >> > > > >>> > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > *Emmanouil Gkatziouras*
> > > >> >>> > > >> >> > > > >>> > > > > > > https://egkatzioura.com/ |
> > > >> >>> > > >> >> > > > >>> > > > > > >
> > > >> >>> > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> >>> > > >> >> > > > >>> > > > > > > https://github.com/gkatzioura
> > > >> >>> > > >> >> > > > >>> > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11,
> > Saikat
> > > >> >>> Maitra <
> > > >> >>> > > >> >> > > > >>> > > [hidden email]
> > > >> >>> > > >> >> > > > >>> > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > wrote:
> > > >> >>> > > >> >> > > > >>> > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > > Hello Emmanouil,
> > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > > Can you please share more info
> > > about
> > > >> >>> the
> > > >> >>> > > >> >> > > implementation?
> > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > > We have similar implementation
> > for
> > > >> data
> > > >> >>> > > >> streamers
> > > >> >>> > > >> >> > with
> > > >> >>> > > >> >> > > > >>> Kafka
> > > >> >>> > > >> >> > > > >>> > and
> > > >> >>> > > >> >> > > > >>> > > > > > RocketMQ
> > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > >> >>> > > >> >> >
> > https://apacheignite-mix.readme.io/docs/kafka-streamer
> > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > >> >>> > > >> >> > > >
> > > >> >>> https://apacheignite-mix.readme.io/docs/rocketmq-streamer
> > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > > Please review and share if you
> > > have
> > > >> any
> > > >> >>> > > >> questions.
> > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > > Regards,
> > > >> >>> > > >> >> > > > >>> > > > > > > > Saikat
> > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56
> PM
> > > >> >>> Emmanouil
> > > >> >>> > > >> >> > Gkatziouras <
> > > >> >>> > > >> >> > > > >>> > > > > > > > [hidden email]>
> > > >> >>> > > >> >> > > > >>> > > > > > > > wrote:
> > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > > > Hi all,
> > > >> >>> > > >> >> > > > >>> > > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > > > By mentioning Pub/Sub I mean
> > > >> Google
> > > >> >>> > Cloud's
> > > >> >>> > > >> >> Pub/Sub
> > > >> >>> > > >> >> > > [1]
> > > >> >>> > > >> >> > > > >>> > > > > > > > > Yes it is indeed cloud
> > specific.
> > > >> It
> > > >> >>> is
> > > >> >>> > the
> > > >> >>> > > >> >> > messaging
> > > >> >>> > > >> >> > > > >>> service
> > > >> >>> > > >> >> > > > >>> > > > > provided
> > > >> >>> > > >> >> > > > >>> > > > > > > by
> > > >> >>> > > >> >> > > > >>> > > > > > > > > GCP.
> > > >> >>> > > >> >> > > > >>> > > > > > > > > I have made the integration
> > with
> > > >> it,
> > > >> >>> but
> > > >> >>> > I
> > > >> >>> > > >> >> haven't
> > > >> >>> > > >> >> > > > >>> published
> > > >> >>> > > >> >> > > > >>> > it
> > > >> >>> > > >> >> > > > >>> > > > yet
> > > >> >>> > > >> >> > > > >>> > > > > > (I
> > > >> >>> > > >> >> > > > >>> > > > > > > am
> > > >> >>> > > >> >> > > > >>> > > > > > > > > waiting for the
> > > discussion/ticket
> > > >> and
> > > >> >>> > > proceed
> > > >> >>> > > >> >> with
> > > >> >>> > > >> >> > > the
> > > >> >>> > > >> >> > > > >>> > > > > contribution).
> > > >> >>> > > >> >> > > > >>> > > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > > > Kind regards,
> > > >> >>> > > >> >> > > > >>> > > > > > > > > Emmanouil
> > > >> >>> > > >> >> > > > >>> > > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > > > [1]
> > > >> >>> > > >> >> https://cloud.google.com/pubsub/docs/overview
> > > >> >>> > > >> >> > > > >>> > > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
> > > >> >>> > > >> >> > > > >>> > > > > > > > > https://egkatzioura.com/ |
> > > >> >>> > > >> >> > > > >>> > > > > > > > >
> > > >> >>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> >>> > > >> >> > > > >>> > > > > > > > >
> https://github.com/gkatzioura
> > > >> >>> > > >> >> > > > >>> > > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at
> 21:45,
> > > >> Denis
> > > >> >>> > Magda <
> > > >> >>> > > >> >> > > > >>> [hidden email]
> > > >> >>> > > >> >> > > > >>> > >
> > > >> >>> > > >>

[Emmanouil Gkatziouras]

Hi all,

I did add a test suite and run the build with the corresponding arguments,
however the tests did not run [1].
I checked the Flink's build logs and the same message is displayed there
too `No tests to run` [2].

[1]
https://ci.ignite.apache.org/viewLog.html?buildId=4798203&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=248
[2]
https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4789199&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=1256

Kind regards
*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Wed, 27 Nov 2019 at 18:58, Saikat Maitra <[hidden email]> wrote:

> Hi,
>
> We will need to add test suites for tests to be executed in build. We pass
> them as TestSuites param. I had added Flink sink and source testsuites.
>
> Can you please review and confirm.
>
> Regards
> Saikat
>
> On Wed, 27 Nov 2019 at 6:00 AM, Emmanouil Gkatziouras <
> [hidden email]>
> wrote:
>
> > Hi all!
> >
> > I was successful building and running my pull requests. It seems that the
> > tests do not run both for Flink and Pub/Sub [1]
> > If there is something I cannot do to make them discoverable please let me
> > know.
> >
> > [1]
> >
> >
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4796722&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=283
> >
> > *Emmanouil Gkatziouras*
> > https://egkatzioura.com/ |
> > https://www.linkedin.com/in/gkatziourasemmanouil/
> > https://github.com/gkatzioura
> >
> >
> > On Wed, 27 Nov 2019 at 04:57, Saikat Maitra <[hidden email]>
> > wrote:
> >
> > > Hi Emmanouil,
> > >
> > > I have added you as contributor in Ignite Extensions project. Can you
> > > please check and confirm if you are able to see the project and execute
> > > build on your pull request.
> > >
> > > Regards,
> > > Saikat
> > >
> > > On Tue, Nov 26, 2019 at 12:49 PM Emmanouil Gkatziouras <
> > > [hidden email]>
> > > wrote:
> > >
> > > > Hi all!
> > > >
> > > > If someone could give me read access to the Job of Ignite Extensions
> on
> > > > Team City will greatly help me (username gkatzioura).
> > > > I suppose the builds get triggered automatically on a pull request.
> > > >
> > > > Kind regards,
> > > > Emmanouil
> > > >
> > > > *Emmanouil Gkatziouras*
> > > > https://egkatzioura.com/ |
> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > https://github.com/gkatzioura
> > > >
> > > >
> > > > On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras <
> > > [hidden email]>
> > > > wrote:
> > > >
> > > > > Hi Saikat!
> > > > >
> > > > > I just rebased with the flink branch. Unfortunately I do not have
> > read
> > > > > access to the team city link you provided and thus evaluate the
> tests
> > > for
> > > > > my pull request.
> > > > > I guess it has to do with ignite extensions being new.
> > > > > My username is gkatzioura and email the one I am using on this
> email
> > > > >
> > > > > Thank you for your time.
> > > > >
> > > > >
> > > > > *Emmanouil Gkatziouras*
> > > > > https://egkatzioura.com/ |
> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > https://github.com/gkatzioura
> > > > >
> > > > >
> > > > > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <
> [hidden email]
> > >
> > > > > wrote:
> > > > >
> > > > >> Hi Emmanouil,
> > > > >>
> > > > >> The latest build on teamcity has passed on Flink pull request.
> > > > >>
> > > > >>
> > > > >>
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> > > > >>
> > > > >> you should be able to take the changes I made in my PR and run
> build
> > > on
> > > > >> your pull request.
> > > > >>
> > > > >> Please let me know if you have any questions.
> > > > >>
> > > > >> Regards,
> > > > >> Saikat
> > > > >>
> > > > >> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <
> > > [hidden email]
> > > > >
> > > > >> wrote:
> > > > >>
> > > > >> > Hi Emmanouil,
> > > > >> >
> > > > >> > I have fixed the Flink module testsuites and have setup new
> > project
> > > in
> > > > >> > teamcity.
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
> > > > >> >
> > > > >> > I am looking into setting up the build.
> > > > >> >
> > > > >> > Regards,
> > > > >> > Saikat
> > > > >> >
> > > > >> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <
> > > > [hidden email]>
> > > > >> > wrote:
> > > > >> >
> > > > >> >> Hi Emmanouil,
> > > > >> >>
> > > > >> >> Thank you for your email. Yes, the plan is once the Flink PR is
> > > > merged
> > > > >> it
> > > > >> >> will provide the Licence, parent POMs etc and you can rebase
> from
> > > > >> master
> > > > >> >> branch and apply the changes on top of it.
> > > > >> >>
> > > > >> >> I am getting some issues with test failures in local with
> > > > >> >> GridTestProperties as the test.properties is not present in
> this
> > > new
> > > > >> >> project but should be available from dependencies.
> > > > >> >>
> > > > >> >> Once I address this issue, I will go ahead and merge the
> changes
> > > and
> > > > >> then
> > > > >> >> we can take it from there.
> > > > >> >>
> > > > >> >> Regards,
> > > > >> >> Saikat
> > > > >> >>
> > > > >> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
> > > > >> >> [hidden email]> wrote:
> > > > >> >>
> > > > >> >>> Hi all,
> > > > >> >>>
> > > > >> >>> I made my first pull request [1]. Since this project is brand
> > new
> > > > (no
> > > > >> >>> parent poms, licensing), I reckoned it was best to use
> Saikat's
> > > > >> branch on
> > > > >> >>> Flink.
> > > > >> >>> I suppose the Flink branch will be merged first. If not please
> > > give
> > > > me
> > > > >> >>> guidelines on how I should proceed next.
> > > > >> >>>
> > > > >> >>> Kind regards
> > > > >> >>> Emmanouil
> > > > >> >>>
> > > > >> >>> [1] https://github.com/apache/ignite-extensions/pull/2
> > > > >> >>>
> > > > >> >>> *Emmanouil Gkatziouras*
> > > > >> >>> https://egkatzioura.com/ |
> > > > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >> >>> https://github.com/gkatzioura
> > > > >> >>>
> > > > >> >>>
> > > > >> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <[hidden email]>
> > > > wrote:
> > > > >> >>>
> > > > >> >>> > Awesome, ping us whenever you're ready!
> > > > >> >>> >
> > > > >> >>> > -
> > > > >> >>> > Denis
> > > > >> >>> >
> > > > >> >>> >
> > > > >> >>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
> > > > >> >>> > [hidden email]>
> > > > >> >>> > wrote:
> > > > >> >>> >
> > > > >> >>> > > Hi all!
> > > > >> >>> > >
> > > > >> >>> > > I am sorry for being late on that. I was trying to
> refactor
> > > the
> > > > >> test
> > > > >> >>> in
> > > > >> >>> > > order not to be in need of any external tools or spinning
> > up a
> > > > >> >>> server.
> > > > >> >>> > > I did forked the new repo and indeed my changes there, so
> a
> > > pull
> > > > >> >>> request
> > > > >> >>> > is
> > > > >> >>> > > a matter of time!
> > > > >> >>> > >
> > > > >> >>> > > Kind regards
> > > > >> >>> > >
> > > > >> >>> > > *Emmanouil Gkatziouras*
> > > > >> >>> > > https://egkatzioura.com/ |
> > > > >> >>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >> >>> > > https://github.com/gkatzioura
> > > > >> >>> > >
> > > > >> >>> > >
> > > > >> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <
> > [hidden email]>
> > > > >> wrote:
> > > > >> >>> > >
> > > > >> >>> > > > Hi Emmanouil,
> > > > >> >>> > > >
> > > > >> >>> > > > Do you have any questions or need any support from the
> > > > >> community?
> > > > >> >>> > > >
> > > > >> >>> > > > -
> > > > >> >>> > > > Denis
> > > > >> >>> > > >
> > > > >> >>> > > >
> > > > >> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
> > > > >> >>> [hidden email]
> > > > >> >>> > >
> > > > >> >>> > > > wrote:
> > > > >> >>> > > >
> > > > >> >>> > > >> Hi Emmanouil,
> > > > >> >>> > > >>
> > > > >> >>> > > >> Can you please take a looks at dev utils, if this is
> > > > something
> > > > >> >>> you are
> > > > >> >>> > > >> looking for
> > > > >> >>> > > >>
> > > > https://github.com/apache/ignite/tree/master/modules/dev-utils
> > > > >> ?
> > > > >> >>> > > >>
> > > > >> >>> > > >> IMO, if you can release Pub/Sub server in maven and
> then
> > > use
> > > > >> it as
> > > > >> >>> > > >> dependency, that would be great.
> > > > >> >>> > > >>
> > > > >> >>> > > >> Regards,
> > > > >> >>> > > >> Saikat
> > > > >> >>> > > >>
> > > > >> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
> > > > >> >>> > [hidden email]>
> > > > >> >>> > > >> wrote:
> > > > >> >>> > > >>
> > > > >> >>> > > >> > Hi Emmanouil,
> > > > >> >>> > > >> >
> > > > >> >>> > > >> > The master branch has been initialized. I have
> raised a
> > > PR
> > > > to
> > > > >> >>> > migrate
> > > > >> >>> > > >> > Flink module to ignite-extensions.
> > > > >> >>> > > >> >
> > > > >> >>> > > >> > PR
> https://github.com/apache/ignite-extensions/pull/1
> > > > >> >>> > > >> > Jira :
> > > https://issues.apache.org/jira/browse/IGNITE-12356
> > > > >> >>> > > >> >
> > > > >> >>> > > >> > Regards,
> > > > >> >>> > > >> > Saikat
> > > > >> >>> > > >> >
> > > > >> >>> > > >> >
> > > > >> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil
> Gkatziouras <
> > > > >> >>> > > >> > [hidden email]> wrote:
> > > > >> >>> > > >> >
> > > > >> >>> > > >> >> Hi all!
> > > > >> >>> > > >> >>
> > > > >> >>> > > >> >> This is an update. The integration is tested with
> > mocks
> > > > >> based
> > > > >> >>> on
> > > > >> >>> > the
> > > > >> >>> > > >> >> original Ignite repo [1].
> > > > >> >>> > > >> >> I am currently waiting for the first commits on the
> > new
> > > > >> >>> integration
> > > > >> >>> > > >> >> project
> > > > >> >>> > > >> >> to take place in order to create my pull request
> > > > accordingly
> > > > >> >>> to the
> > > > >> >>> > > >> >> initial
> > > > >> >>> > > >> >> commits.
> > > > >> >>> > > >> >> In order to simulate the Pub/Sub server in a test I
> > > used a
> > > > >> test
> > > > >> >>> > > server
> > > > >> >>> > > >> >> implementation [2]. Unfortunately they don't have
> this
> > > > util
> > > > >> >>> > uploaded
> > > > >> >>> > > >> in a
> > > > >> >>> > > >> >> maven repo, thus it needs to be hosted on a maven
> repo
> > > > >> (Apache
> > > > >> >>> > > License
> > > > >> >>> > > >> >> 2.0). Is there a repo for utilities that Ignite uses
> > for
> > > > the
> > > > >> >>> > > >> test/build?
> > > > >> >>> > > >> >> I have also created a ticket [3]
> > > > >> >>> > > >> >>
> > > > >> >>> > > >> >> Kind regards
> > > > >> >>> > > >> >>
> > > > >> >>> > > >> >> [1]
> > > > >> >>> >
> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > > > >> >>> > > >> >> [2]
> > > > >> >>> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> > > > >> >>> > > >> >> [3]
> > https://issues.apache.org/jira/browse/IGNITE-12262
> > > > >> >>> > > >> >>
> > > > >> >>> > > >> >> *Emmanouil Gkatziouras*
> > > > >> >>> > > >> >> https://egkatzioura.com/ |
> > > > >> >>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >> >>> > > >> >> https://github.com/gkatzioura
> > > > >> >>> > > >> >>
> > > > >> >>> > > >> >>
> > > > >> >>> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
> > > > >> >>> > [hidden email]
> > > > >> >>> > > >
> > > > >> >>> > > >> >> wrote:
> > > > >> >>> > > >> >>
> > > > >> >>> > > >> >> > Hello Ilya,
> > > > >> >>> > > >> >> >
> > > > >> >>> > > >> >> > Thank you for your email. Yes, before we remove
> any
> > > > >> >>> integration
> > > > >> >>> > we
> > > > >> >>> > > >> will
> > > > >> >>> > > >> >> > move them to another repository and release.
> > > > >> >>> > > >> >> >
> > > > >> >>> > > >> >> > We are thinking we will start with Pub/Sub
> Streamer
> > > > >> >>> > implementation
> > > > >> >>> > > >> and
> > > > >> >>> > > >> >> > continue to move other integration as mentioned in
> > the
> > > > >> >>> following
> > > > >> >>> > > doc.
> > > > >> >>> > > >> >> >
> > > > >> >>> > > >> >> >
> > > > >> >>> > > >> >> >
> > > > >> >>> > > >> >>
> > > > >> >>> > > >>
> > > > >> >>> > >
> > > > >> >>> >
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > >> >>> > > >> >> >
> > > > >> >>> > > >> >> > Regards,
> > > > >> >>> > > >> >> > Saikat
> > > > >> >>> > > >> >> >
> > > > >> >>> > > >> >> >
> > > > >> >>> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
> > > > >> >>> > > >> >> > [hidden email]>
> > > > >> >>> > > >> >> > wrote:
> > > > >> >>> > > >> >> >
> > > > >> >>> > > >> >> > > Hello!
> > > > >> >>> > > >> >> > >
> > > > >> >>> > > >> >> > > My take on this: we want to offload some of
> > > > >> integrations,
> > > > >> >>> but
> > > > >> >>> > > >> please
> > > > >> >>> > > >> >> > commit
> > > > >> >>> > > >> >> > > and release them to another project first, and
> > only
> > > > >> then we
> > > > >> >>> > will
> > > > >> >>> > > >> >> decide
> > > > >> >>> > > >> >> > to
> > > > >> >>> > > >> >> > > remove them from our repository.
> > > > >> >>> > > >> >> > >
> > > > >> >>> > > >> >> > > Please also make sure they are available with
> same
> > > > >> artifact
> > > > >> >>> > name
> > > > >> >>> > > >> and
> > > > >> >>> > > >> >> API.
> > > > >> >>> > > >> >> > > If this has to be changed, let's wait for AI 3.0
> > > > >> >>> > > >> >> > >
> > > > >> >>> > > >> >> > > Regards,
> > > > >> >>> > > >> >> > > --
> > > > >> >>> > > >> >> > > Ilya Kasnacheev
> > > > >> >>> > > >> >> > >
> > > > >> >>> > > >> >> > >
> > > > >> >>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <
> > > > >> >>> > > >> [hidden email]>:
> > > > >> >>> > > >> >> > >
> > > > >> >>> > > >> >> > > > Hello Emmanouil,
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > > As discussed earlier, I discussed with Apache
> > > Bahir
> > > > >> >>> community
> > > > >> >>> > > and
> > > > >> >>> > > >> >> they
> > > > >> >>> > > >> >> > > are
> > > > >> >>> > > >> >> > > > interested to have Apache Ignite extensions as
> > > part
> > > > of
> > > > >> >>> Apache
> > > > >> >>> > > >> Bahir
> > > > >> >>> > > >> >> > > > project.
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > > Can you please share Pub/Sub streamer
> > > implementation
> > > > >> >>> details
> > > > >> >>> > > with
> > > > >> >>> > > >> >> > Apache
> > > > >> >>> > > >> >> > > > Bahir community and request for feedback.
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > > https://bahir.apache.org/
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > > I have also requested for contributor access
> in
> > > Jira
> > > > >> for
> > > > >> >>> > Apache
> > > > >> >>> > > >> >> Bahir
> > > > >> >>> > > >> >> > > > project so that I can create issues and assign
> > to
> > > > >> myself.
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > > I can help with code reviews as well.
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > > Here is the email thread for reference
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > >
> > > https://www.mail-archive.com/dev@.../msg02703.html
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > > Regards,
> > > > >> >>> > > >> >> > > > Saikat
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra
> <
> > > > >> >>> > > >> >> [hidden email]
> > > > >> >>> > > >> >> > >
> > > > >> >>> > > >> >> > > > wrote:
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > > > Hello,
> > > > >> >>> > > >> >> > > > >
> > > > >> >>> > > >> >> > > > > As discussed I have created following
> > discussion
> > > > >> >>> threads on
> > > > >> >>> > > our
> > > > >> >>> > > >> >> > > migration
> > > > >> >>> > > >> >> > > > > proposals for Apache Ignite extensions:
> > > > >> >>> > > >> >> > > > >
> > > > >> >>> > > >> >> > > > >
> > > > >> >>> > > >> >> > > > >
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > >
> > > > >> >>> > > >> >> >
> > > > >> >>> > > >> >>
> > > > >> >>> > > >>
> > > > >> >>> > >
> > > > >> >>> >
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> > > > >> >>> > > >> >> > > > >
> > > > >> >>> > > >> >> > > > >
> > > > >> >>> > > >>
> > > > >> https://www.mail-archive.com/dev@.../msg02703.html
> > > > >> >>> > > >> >> > > > >
> > > > >> >>> > > >> >> > > > > I will share update as I hear more
> > information.
> > > > >> >>> > > >> >> > > > >
> > > > >> >>> > > >> >> > > > > Regards,
> > > > >> >>> > > >> >> > > > > Saikat
> > > > >> >>> > > >> >> > > > >
> > > > >> >>> > > >> >> > > > >
> > > > >> >>> > > >> >> > > > >
> > > > >> >>> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat
> Maitra
> > <
> > > > >> >>> > > >> >> > [hidden email]
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > > > wrote:
> > > > >> >>> > > >> >> > > > >
> > > > >> >>> > > >> >> > > > >> Hello Denis,
> > > > >> >>> > > >> >> > > > >>
> > > > >> >>> > > >> >> > > > >> Sure, sounds good. I will create a separate
> > > > >> discussion
> > > > >> >>> > > thread
> > > > >> >>> > > >> to
> > > > >> >>> > > >> >> get
> > > > >> >>> > > >> >> > > > >> community feedback on whether we
> > > > >> >>> > > >> >> > > > >> should join the Bahir or kick-off "Ignite
> > > > >> Extensions"
> > > > >> >>> > > project.
> > > > >> >>> > > >> >> > > > >>
> > > > >> >>> > > >> >> > > > >> Regards,
> > > > >> >>> > > >> >> > > > >> Saikat
> > > > >> >>> > > >> >> > > > >>
> > > > >> >>> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis
> Magda <
> > > > >> >>> > > >> [hidden email]>
> > > > >> >>> > > >> >> > > wrote:
> > > > >> >>> > > >> >> > > > >>
> > > > >> >>> > > >> >> > > > >>> Folks,
> > > > >> >>> > > >> >> > > > >>>
> > > > >> >>> > > >> >> > > > >>> The concept of Apache Bahir is precisely
> > what
> > > we
> > > > >> >>> need!
> > > > >> >>> > > >> Saikat,
> > > > >> >>> > > >> >> > thanks
> > > > >> >>> > > >> >> > > > for
> > > > >> >>> > > >> >> > > > >>> doing the research. Why I believe the *
> > idea*
> > > > >> fits us
> > > > >> >>> > well:
> > > > >> >>> > > >> >> > > > >>>
> > > > >> >>> > > >> >> > > > >>>    - All integrations can be stored in
> > > separate
> > > > >> >>> Github
> > > > >> >>> > > >> >> repositories
> > > > >> >>> > > >> >> > > and
> > > > >> >>> > > >> >> > > > >>>    have their dev lifecycles. We've not
> > > obliged
> > > > to
> > > > >> >>> couple
> > > > >> >>> > > all
> > > > >> >>> > > >> >> the
> > > > >> >>> > > >> >> > > > >>> integrations
> > > > >> >>> > > >> >> > > > >>>    in a single repo. For instance, Spark
> can
> > > be
> > > > >> >>> located
> > > > >> >>> > in
> > > > >> >>> > > a
> > > > >> >>> > > >> >> > > dedicated
> > > > >> >>> > > >> >> > > > >>> repo
> > > > >> >>> > > >> >> > > > >>>    while streaming integrations might be
> in
> > a
> > > > >> single
> > > > >> >>> one.
> > > > >> >>> > > >> It's
> > > > >> >>> > > >> >> up
> > > > >> >>> > > >> >> > to
> > > > >> >>> > > >> >> > > > us.
> > > > >> >>> > > >> >> > > > >>>    - All the repositories and integrations
> > > > belong
> > > > >> to
> > > > >> >>> ASF.
> > > > >> >>> > > >> We're
> > > > >> >>> > > >> >> not
> > > > >> >>> > > >> >> > > > >>> dumping
> > > > >> >>> > > >> >> > > > >>>    them on Github but rather keep
> supporting
> > > and
> > > > >> >>> > developing
> > > > >> >>> > > >> in
> > > > >> >>> > > >> >> > > > >>> accordance with
> > > > >> >>> > > >> >> > > > >>>    ASF vision and practices.
> > > > >> >>> > > >> >> > > > >>>    - There is a way to reward contributors
> > via
> > > > >> >>> > > committership
> > > > >> >>> > > >> and
> > > > >> >>> > > >> >> > PMC
> > > > >> >>> > > >> >> > > > >>>    membership. You won't get this if a
> > project
> > > > is
> > > > >> >>> just
> > > > >> >>> > one
> > > > >> >>> > > of
> > > > >> >>> > > >> >> the
> > > > >> >>> > > >> >> > > > >>> millions of
> > > > >> >>> > > >> >> > > > >>>    Github projects.
> > > > >> >>> > > >> >> > > > >>>
> > > > >> >>> > > >> >> > > > >>> Saikat, as Ignite PMC member, could you
> > please
> > > > >> >>> kick-off a
> > > > >> >>> > > >> >> separate
> > > > >> >>> > > >> >> > > > >>> dev-list
> > > > >> >>> > > >> >> > > > >>> discussion to involve more community
> members
> > > and
> > > > >> >>> > referring
> > > > >> >>> > > to
> > > > >> >>> > > >> >> this
> > > > >> >>> > > >> >> > > > >>> thread?
> > > > >> >>> > > >> >> > > > >>> I think the primary question the community
> > > needs
> > > > >> to
> > > > >> >>> > answer
> > > > >> >>> > > >> >> whether
> > > > >> >>> > > >> >> > we
> > > > >> >>> > > >> >> > > > >>> should join the Bahir or kick-off "Ignite
> > > > >> Extensions"
> > > > >> >>> > > >> project?
> > > > >> >>> > > >> >> > > > >>>
> > > > >> >>> > > >> >> > > > >>> -
> > > > >> >>> > > >> >> > > > >>> Denis
> > > > >> >>> > > >> >> > > > >>>
> > > > >> >>> > > >> >> > > > >>>
> > > > >> >>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey
> > > Zinoviev
> > > > <
> > > > >> >>> > > >> >> > > > [hidden email]>
> > > > >> >>> > > >> >> > > > >>> wrote:
> > > > >> >>> > > >> >> > > > >>>
> > > > >> >>> > > >> >> > > > >>> > Maybe we could move all our Streaming
> > > > >> Integrations
> > > > >> >>> > there,
> > > > >> >>> > > >> but
> > > > >> >>> > > >> >> > what
> > > > >> >>> > > >> >> > > is
> > > > >> >>> > > >> >> > > > >>> about
> > > > >> >>> > > >> >> > > > >>> > maintaining and committer permissions to
> > the
> > > > new
> > > > >> >>> > > >> repositories?
> > > > >> >>> > > >> >> > > > >>> > I see the list of the committers and PMC
> > > > members
> > > > >> >>> there
> > > > >> >>> > > >> >> > > > >>> >
> > https://bahir.apache.org/community-members/
> > > > >> >>> > > >> >> > > > >>> >
> > > > >> >>> > > >> >> > > > >>> > Could somebody from Ignite community be
> > > added
> > > > to
> > > > >> >>> this
> > > > >> >>> > > list
> > > > >> >>> > > >> as
> > > > >> >>> > > >> >> a
> > > > >> >>> > > >> >> > > > >>> > PMC/committer to maintain Ignite
> > > integrations?
> > > > >> >>> > > >> >> > > > >>> > If yes, I'd happy to join this project
> and
> > > > >> >>> collaborate
> > > > >> >>> > > with
> > > > >> >>> > > >> >> these
> > > > >> >>> > > >> >> > > > guys
> > > > >> >>> > > >> >> > > > >>> > together
> > > > >> >>> > > >> >> > > > >>> > If no, and we should start from the zero
> > > level
> > > > >> with
> > > > >> >>> > > >> external
> > > > >> >>> > > >> >> PRs
> > > > >> >>> > > >> >> > -
> > > > >> >>> > > >> >> > > > >>> hmmm,
> > > > >> >>> > > >> >> > > > >>> > it's better to have our own external
> > > > repository
> > > > >> >>> with
> > > > >> >>> > > >> >> ApacheBahirr
> > > > >> >>> > > >> >> > > > >>> ideology.
> > > > >> >>> > > >> >> > > > >>> >
> > > > >> >>> > > >> >> > > > >>> > Also, I agree, that all connectors could
> > be
> > > > >> moved
> > > > >> >>> there
> > > > >> >>> > > (in
> > > > >> >>> > > >> >> > > > >>> ApacheBahirr
> > > > >> >>> > > >> >> > > > >>> > like repository), but not all modules
> from
> > > the
> > > > >> page
> > > > >> >>> > > >> >> > > > >>> >
> > > > >> >>> > > >> >> > > > >>> >
> > > > >> >>> > > >> >> > > > >>>
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > >
> > > > >> >>> > > >> >> >
> > > > >> >>> > > >> >>
> > > > >> >>> > > >>
> > > > >> >>> > >
> > > > >> >>> >
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > >> >>> > > >> >> > > > >>> > because
> > > > >> >>> > > >> >> > > > >>> > they use different parts of core modules
> > and
> > > > >> could
> > > > >> >>> be
> > > > >> >>> > > >> >> developed
> > > > >> >>> > > >> >> > > with
> > > > >> >>> > > >> >> > > > >>> > different velocity.
> > > > >> >>> > > >> >> > > > >>> >
> > > > >> >>> > > >> >> > > > >>> > In reality, before creation of new
> > > > repositories
> > > > >> we
> > > > >> >>> need
> > > > >> >>> > > >> wide
> > > > >> >>> > > >> >> > > > discussion
> > > > >> >>> > > >> >> > > > >>> > based on the document mentioned above.
> > > > >> >>> > > >> >> > > > >>> >
> > > > >> >>> > > >> >> > > > >>> > P.S. Of course, we could experiment in
> the
> > > > next
> > > > >> >>> release
> > > > >> >>> > > 2.8
> > > > >> >>> > > >> >> with
> > > > >> >>> > > >> >> > > one
> > > > >> >>> > > >> >> > > > >>> or two
> > > > >> >>> > > >> >> > > > >>> > integrations like twitter or ZeroMQ
> > > > >> >>> > > >> >> > > > >>> > Also, I'd like Denis Magda idea for the
> > > > separate
> > > > >> >>> > > >> repositories
> > > > >> >>> > > >> >> in
> > > > >> >>> > > >> >> > > > Apache
> > > > >> >>> > > >> >> > > > >>> > Github to maintain them by anybody.
> > > > >> >>> > > >> >> > > > >>> >
> > > > >> >>> > > >> >> > > > >>> >
> > > > >> >>> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat
> > Maitra <
> > > > >> >>> > > >> >> > > [hidden email]
> > > > >> >>> > > >> >> > > > >:
> > > > >> >>> > > >> >> > > > >>> >
> > > > >> >>> > > >> >> > > > >>> > > Hi Denis, Emmanouil
> > > > >> >>> > > >> >> > > > >>> > >
> > > > >> >>> > > >> >> > > > >>> > > Thank you for your email. I think
> > > creating a
> > > > >> >>> separate
> > > > >> >>> > > >> >> > integration
> > > > >> >>> > > >> >> > > > >>> project
> > > > >> >>> > > >> >> > > > >>> > > in github and also proposing it as an
> > > apache
> > > > >> >>> > incubator
> > > > >> >>> > > >> >> project
> > > > >> >>> > > >> >> > > will
> > > > >> >>> > > >> >> > > > >>> be a
> > > > >> >>> > > >> >> > > > >>> > > good move. The other separate
> > integration
> > > > >> modules
> > > > >> >>> > > >> >> > > > >>> > >
> > > > >> >>> > > >> >> > > > >>> > >
> > > > >> >>> > > >> >> > > > >>> >
> > > > >> >>> > > >> >> > > > >>>
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > >
> > > > >> >>> > > >> >> >
> > > > >> >>> > > >> >>
> > > > >> >>> > > >>
> > > > >> >>> > >
> > > > >> >>> >
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > >> >>> > > >> >> > > > >>> > > can
> > > > >> >>> > > >> >> > > > >>> > > be moved to this new apache incubator
> > > > project.
> > > > >> >>> > > >> >> > > > >>> > >
> > > > >> >>> > > >> >> > > > >>> > > There are similar integration
> available
> > > for
> > > > >> >>> Flink and
> > > > >> >>> > > >> Spark
> > > > >> >>> > > >> >> in
> > > > >> >>> > > >> >> > > > Apache
> > > > >> >>> > > >> >> > > > >>> > Bahir
> > > > >> >>> > > >> >> > > > >>> > > https://bahir.apache.org/
> > > > >> >>> > > >> >> > > > >>> > >
> > > > >> >>> > > >> >> > > > >>> > > Do you think we should reach out to
> > Apache
> > > > >> Bahir
> > > > >> >>> > > >> community
> > > > >> >>> > > >> >> > with a
> > > > >> >>> > > >> >> > > > >>> > proposal
> > > > >> >>> > > >> >> > > > >>> > > or we should plan to create a separate
> > > > Apache
> > > > >> >>> > Incubator
> > > > >> >>> > > >> >> > project?
> > > > >> >>> > > >> >> > > > >>> > >
> > > > >> >>> > > >> >> > > > >>> > > Regards,
> > > > >> >>> > > >> >> > > > >>> > > Saikat
> > > > >> >>> > > >> >> > > > >>> > >
> > > > >> >>> > > >> >> > > > >>> > >
> > > > >> >>> > > >> >> > > > >>> > >
> > > > >> >>> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM
> > Emmanouil
> > > > >> >>> > Gkatziouras <
> > > > >> >>> > > >> >> > > > >>> > > [hidden email]>
> > > > >> >>> > > >> >> > > > >>> > > wrote:
> > > > >> >>> > > >> >> > > > >>> > >
> > > > >> >>> > > >> >> > > > >>> > > > Hi all!
> > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> >>> > > >> >> > > > >>> > > > Based on the discussions most
> probably
> > > > this
> > > > >> is
> > > > >> >>> > going
> > > > >> >>> > > to
> > > > >> >>> > > >> >> be on
> > > > >> >>> > > >> >> > > > >>> another
> > > > >> >>> > > >> >> > > > >>> > > > GitHub repo. Therefore I will
> prepare
> > a
> > > > >> >>> standalone
> > > > >> >>> > > >> project
> > > > >> >>> > > >> >> > with
> > > > >> >>> > > >> >> > > > the
> > > > >> >>> > > >> >> > > > >>> > > Pub/Sub
> > > > >> >>> > > >> >> > > > >>> > > > module and once the repository is
> > > created
> > > > a
> > > > >> >>> pull
> > > > >> >>> > > >> request
> > > > >> >>> > > >> >> > shall
> > > > >> >>> > > >> >> > > be
> > > > >> >>> > > >> >> > > > >>> > issued.
> > > > >> >>> > > >> >> > > > >>> > > > If there is anything else I can do
> in
> > > > order
> > > > >> to
> > > > >> >>> > > >> facilitate
> > > > >> >>> > > >> >> > this
> > > > >> >>> > > >> >> > > > >>> please
> > > > >> >>> > > >> >> > > > >>> > let
> > > > >> >>> > > >> >> > > > >>> > > > me know.
> > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> >>> > > >> >> > > > >>> > > > If a ticket is created for this
> > issue, I
> > > > >> shall
> > > > >> >>> be
> > > > >> >>> > > able
> > > > >> >>> > > >> to
> > > > >> >>> > > >> >> > > create
> > > > >> >>> > > >> >> > > > a
> > > > >> >>> > > >> >> > > > >>> pull
> > > > >> >>> > > >> >> > > > >>> > > > request and thus a build will take
> > place
> > > > on
> > > > >> >>> > ignite's
> > > > >> >>> > > >> CI.
> > > > >> >>> > > >> >> > > > >>> > > > If everyone is aligned with adding a
> > > > Pub/Sub
> > > > >> >>> > > >> >> implementation
> > > > >> >>> > > >> >> > > may I
> > > > >> >>> > > >> >> > > > >>> > create
> > > > >> >>> > > >> >> > > > >>> > > a
> > > > >> >>> > > >> >> > > > >>> > > > ticket on JIRA?
> > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> >>> > > >> >> > > > >>> > > > My next question has to do on how
> > Ignite
> > > > >> >>> configures
> > > > >> >>> > > the
> > > > >> >>> > > >> >> build
> > > > >> >>> > > >> >> > > > jobs
> > > > >> >>> > > >> >> > > > >>> in
> > > > >> >>> > > >> >> > > > >>> > > Team
> > > > >> >>> > > >> >> > > > >>> > > > City. The unit tests for my
> > > implementation
> > > > >> >>> need a
> > > > >> >>> > > local
> > > > >> >>> > > >> >> > Pub/Sub
> > > > >> >>> > > >> >> > > > >>> server
> > > > >> >>> > > >> >> > > > >>> > up
> > > > >> >>> > > >> >> > > > >>> > > > and running (they provide one for
> > > testing
> > > > >> >>> > purposes).
> > > > >> >>> > > >> Will
> > > > >> >>> > > >> >> > this
> > > > >> >>> > > >> >> > > > take
> > > > >> >>> > > >> >> > > > >>> > > effect
> > > > >> >>> > > >> >> > > > >>> > > > in a form of a build script inside
> my
> > > > >> >>> > implementation
> > > > >> >>> > > or
> > > > >> >>> > > >> >> is it
> > > > >> >>> > > >> >> > > > >>> something
> > > > >> >>> > > >> >> > > > >>> > > > that should be configured on Team
> > City?
> > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> >>> > > >> >> > > > >>> > > > Kind regards,
> > > > >> >>> > > >> >> > > > >>> > > > Emmanouil
> > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> >>> > > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
> > > > >> >>> > > >> >> > > > >>> > > > https://egkatzioura.com/ |
> > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >> >>> > > >> >> > > > >>> > > > https://github.com/gkatzioura
> > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> >>> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis
> > > Magda
> > > > <
> > > > >> >>> > > >> >> [hidden email]
> > > > >> >>> > > >> >> > >
> > > > >> >>> > > >> >> > > > >>> wrote:
> > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> >>> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
> > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > After contemplating on this, let
> me
> > > > >> propose
> > > > >> >>> > another
> > > > >> >>> > > >> way
> > > > >> >>> > > >> >> > with
> > > > >> >>> > > >> >> > > > this
> > > > >> >>> > > >> >> > > > >>> > > > > particular integration that is
> > aligned
> > > > >> with
> > > > >> >>> our
> > > > >> >>> > > >> >> > > modularization
> > > > >> >>> > > >> >> > > > >>> > > endeavor.
> > > > >> >>> > > >> >> > > > >>> > > > > The modularization [1] defines the
> > > > Ignite
> > > > >> >>> core,
> > > > >> >>> > > >> Ignite
> > > > >> >>> > > >> >> > > modules,
> > > > >> >>> > > >> >> > > > >>> and
> > > > >> >>> > > >> >> > > > >>> > > > > independent integrations. The
> first
> > > two
> > > > >> are
> > > > >> >>> to be
> > > > >> >>> > > >> >> located
> > > > >> >>> > > >> >> > in
> > > > >> >>> > > >> >> > > > >>> Ignite
> > > > >> >>> > > >> >> > > > >>> > > > > repositories and to be
> > tested/updated
> > > by
> > > > >> the
> > > > >> >>> > > >> community
> > > > >> >>> > > >> >> for
> > > > >> >>> > > >> >> > > > every
> > > > >> >>> > > >> >> > > > >>> > > release.
> > > > >> >>> > > >> >> > > > >>> > > > > The modules are a subset of
> existing
> > > > >> >>> integrations
> > > > >> >>> > > >> that
> > > > >> >>> > > >> >> are
> > > > >> >>> > > >> >> > > > >>> adopted
> > > > >> >>> > > >> >> > > > >>> > > widely
> > > > >> >>> > > >> >> > > > >>> > > > > and require community attention.
> > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > As for the independent
> integrations
> > > [1],
> > > > >> >>> those
> > > > >> >>> > are
> > > > >> >>> > > >> >> supposed
> > > > >> >>> > > >> >> > > to
> > > > >> >>> > > >> >> > > > be
> > > > >> >>> > > >> >> > > > >>> > > located
> > > > >> >>> > > >> >> > > > >>> > > > > in a separate GitHub repo. I would
> > > > advise
> > > > >> us
> > > > >> >>> to
> > > > >> >>> > > start
> > > > >> >>> > > >> >> > > > >>> implementing
> > > > >> >>> > > >> >> > > > >>> > this
> > > > >> >>> > > >> >> > > > >>> > > > > concept with the Pub/Sub
> > integration.
> > > > Even
> > > > >> >>> though
> > > > >> >>> > > it
> > > > >> >>> > > >> >> will
> > > > >> >>> > > >> >> > be
> > > > >> >>> > > >> >> > > > >>> located
> > > > >> >>> > > >> >> > > > >>> > > in a
> > > > >> >>> > > >> >> > > > >>> > > > > separate Github repo, it will be
> > > listed
> > > > >> among
> > > > >> >>> > > >> officially
> > > > >> >>> > > >> >> > > > >>> available
> > > > >> >>> > > >> >> > > > >>> > > > > integrations on the Ignite website
> > > > >> featuring
> > > > >> >>> > > >> Emmanouil
> > > > >> >>> > > >> >> as a
> > > > >> >>> > > >> >> > > > >>> > > contributor.
> > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > What do you think?
> > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > [1]
> > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> >>> > > >> >> > > > >>> > >
> > > > >> >>> > > >> >> > > > >>> >
> > > > >> >>> > > >> >> > > > >>>
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > >
> > > > >> >>> > > >> >> >
> > > > >> >>> > > >> >>
> > > > >> >>> > > >>
> > > > >> >>> > >
> > > > >> >>> >
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > -
> > > > >> >>> > > >> >> > > > >>> > > > > Denis
> > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM
> > Saikat
> > > > >> >>> Maitra <
> > > > >> >>> > > >> >> > > > >>> > [hidden email]
> > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> >>> > > >> >> > > > >>> > > > > wrote:
> > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > Hi Emmanouil,
> > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > The changes looks good to me. I
> > > wanted
> > > > >> to
> > > > >> >>> check
> > > > >> >>> > > if
> > > > >> >>> > > >> you
> > > > >> >>> > > >> >> > are
> > > > >> >>> > > >> >> > > > also
> > > > >> >>> > > >> >> > > > >>> > able
> > > > >> >>> > > >> >> > > > >>> > > to
> > > > >> >>> > > >> >> > > > >>> > > > > > validate that the once data is
> > added
> > > > to
> > > > >> >>> Ignite
> > > > >> >>> > > >> cluster
> > > > >> >>> > > >> >> > you
> > > > >> >>> > > >> >> > > > are
> > > > >> >>> > > >> >> > > > >>> also
> > > > >> >>> > > >> >> > > > >>> > > > able
> > > > >> >>> > > >> >> > > > >>> > > > > to
> > > > >> >>> > > >> >> > > > >>> > > > > > access it using another Ignite
> > > client
> > > > or
> > > > >> >>> using
> > > > >> >>> > > rest
> > > > >> >>> > > >> >> > > > endpoints.
> > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > We use Teamcity for CI process,
> > the
> > > > >> >>> details are
> > > > >> >>> > > as
> > > > >> >>> > > >> >> > follows
> > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > Trigger validation of those test
> > > > suites
> > > > >> >>> that
> > > > >> >>> > have
> > > > >> >>> > > >> been
> > > > >> >>> > > >> >> > > > >>> affected by
> > > > >> >>> > > >> >> > > > >>> > > your
> > > > >> >>> > > >> >> > > > >>> > > > > > changes on TeamCity <
> > > > >> >>> > > http://ci.ignite.apache.org/>:
> > > > >> >>> > > >> <
> > > > >> >>> > > >> >> > > > >>> > > > > http://204.14.53.153/
> > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > >    - Locate a test suite you
> have
> > to
> > > > >> check,
> > > > >> >>> > press
> > > > >> >>> > > >> >> button
> > > > >> >>> > > >> >> > > > named
> > > > >> >>> > > >> >> > > > >>> > "..."
> > > > >> >>> > > >> >> > > > >>> > > > that
> > > > >> >>> > > >> >> > > > >>> > > > > >    is located on the left of
> "Run"
> > > > >> button.
> > > > >> >>> "Run
> > > > >> >>> > > >> custom
> > > > >> >>> > > >> >> > > build"
> > > > >> >>> > > >> >> > > > >>> > dialog
> > > > >> >>> > > >> >> > > > >>> > > > will
> > > > >> >>> > > >> >> > > > >>> > > > > >    appear;
> > > > >> >>> > > >> >> > > > >>> > > > > >    - Go to "Changes" tab and
> > choose
> > > > >> >>> > > >> >> > > > >>> > "pull/<pull-request-number>/head"
> > > > >> >>> > > >> >> > > > >>> > > > in
> > > > >> >>> > > >> >> > > > >>> > > > > >    "Build branch" dropdown list;
> > > > >> >>> > > >> >> > > > >>> > > > > >    - Press "Run build" button
> and
> > > > >> monitor
> > > > >> >>> tests
> > > > >> >>> > > >> >> results.
> > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > @Roman [hidden email] - can
> > you
> > > > also
> > > > >> >>> please
> > > > >> >>> > > >> take a
> > > > >> >>> > > >> >> > > look?
> > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > Regards,
> > > > >> >>> > > >> >> > > > >>> > > > > > Saikat
> > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM
> > > > >> Emmanouil
> > > > >> >>> > > >> Gkatziouras
> > > > >> >>> > > >> >> <
> > > > >> >>> > > >> >> > > > >>> > > > > > [hidden email]>
> > > > >> >>> > > >> >> > > > >>> > > > > > wrote:
> > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
> > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > I have my implementation on an
> > > > Ignite
> > > > >> >>> fork on
> > > > >> >>> > > my
> > > > >> >>> > > >> >> > personal
> > > > >> >>> > > >> >> > > > >>> GitHub
> > > > >> >>> > > >> >> > > > >>> > > > > account
> > > > >> >>> > > >> >> > > > >>> > > > > > > [1]
> > > > >> >>> > > >> >> > > > >>> > > > > > > It is based on the Kafka
> > > > >> implementation
> > > > >> >>> which
> > > > >> >>> > > is
> > > > >> >>> > > >> of
> > > > >> >>> > > >> >> a
> > > > >> >>> > > >> >> > > > polling
> > > > >> >>> > > >> >> > > > >>> > > nature.
> > > > >> >>> > > >> >> > > > >>> > > > > > > GCP does not provide a service
> > > like
> > > > >> >>> Kafka.
> > > > >> >>> > > >> Instead
> > > > >> >>> > > >> >> it
> > > > >> >>> > > >> >> > > > >>> provides us
> > > > >> >>> > > >> >> > > > >>> > > > with
> > > > >> >>> > > >> >> > > > >>> > > > > > > Pub/Sub which is just like
> > Kafka.
> > > > >> >>> > > >> >> > > > >>> > > > > > > Therefore using the Kafka
> > streamer
> > > > as
> > > > >> a
> > > > >> >>> > > guideline
> > > > >> >>> > > >> >> made
> > > > >> >>> > > >> >> > > more
> > > > >> >>> > > >> >> > > > >>> sense
> > > > >> >>> > > >> >> > > > >>> > > to
> > > > >> >>> > > >> >> > > > >>> > > > > me.
> > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > Apart from the implementation
> I
> > > have
> > > > >> also
> > > > >> >>> > > added a
> > > > >> >>> > > >> >> unit
> > > > >> >>> > > >> >> > > > test.
> > > > >> >>> > > >> >> > > > >>> > > > > > > It definitely needs some
> > polishing
> > > > >> and a
> > > > >> >>> > double
> > > > >> >>> > > >> >> check
> > > > >> >>> > > >> >> > on
> > > > >> >>> > > >> >> > > my
> > > > >> >>> > > >> >> > > > >>> side
> > > > >> >>> > > >> >> > > > >>> > to
> > > > >> >>> > > >> >> > > > >>> > > > > make
> > > > >> >>> > > >> >> > > > >>> > > > > > > the review smooth.
> > > > >> >>> > > >> >> > > > >>> > > > > > > For the tests a local Pub/Sub
> > [2]
> > > > >> server
> > > > >> >>> has
> > > > >> >>> > to
> > > > >> >>> > > >> be
> > > > >> >>> > > >> >> > setup
> > > > >> >>> > > >> >> > > > >>> however
> > > > >> >>> > > >> >> > > > >>> > I
> > > > >> >>> > > >> >> > > > >>> > > am
> > > > >> >>> > > >> >> > > > >>> > > > > not
> > > > >> >>> > > >> >> > > > >>> > > > > > > sure which is the best way to
> do
> > > so
> > > > >> with
> > > > >> >>> > > regards
> > > > >> >>> > > >> to
> > > > >> >>> > > >> >> > your
> > > > >> >>> > > >> >> > > > >>> CI/CD.
> > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > [1]
> > > > >> >>> > > >> >> > > > >>> > >
> > > > >> >>> > > >> >> >
> > > > >> >>>
> https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > > > >> >>> > > >> >> > > > >>> > > > > > > [2]
> > > > >> >>> > > >> >> > > > >>>
> > > > >> >>> > https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
> > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > *Emmanouil Gkatziouras*
> > > > >> >>> > > >> >> > > > >>> > > > > > > https://egkatzioura.com/ |
> > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > >> >>> > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >> >>> > > >> >> > > > >>> > > > > > > https://github.com/gkatzioura
> > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11,
> > > Saikat
> > > > >> >>> Maitra <
> > > > >> >>> > > >> >> > > > >>> > > [hidden email]
> > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > wrote:
> > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > > Hello Emmanouil,
> > > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > > Can you please share more
> info
> > > > about
> > > > >> >>> the
> > > > >> >>> > > >> >> > > implementation?
> > > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > > We have similar
> implementation
> > > for
> > > > >> data
> > > > >> >>> > > >> streamers
> > > > >> >>> > > >> >> > with
> > > > >> >>> > > >> >> > > > >>> Kafka
> > > > >> >>> > > >> >> > > > >>> > and
> > > > >> >>> > > >> >> > > > >>> > > > > > RocketMQ
> > > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > > >> >>> > > >> >> >
> > > https://apacheignite-mix.readme.io/docs/kafka-streamer
> > > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > > >> >>> > > >> >> > > >
> > > > >> >>> https://apacheignite-mix.readme.io/docs/rocketmq-streamer
> > > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > > Please review and share if
> you
> > > > have
> > > > >> any
> > > > >> >>> > > >> questions.
> > > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > > Regards,
> > > > >> >>> > > >> >> > > > >>> > > > > > > > Saikat
> > > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56
> > PM
> > > > >> >>> Emmanouil
> > > > >> >>> > > >> >> > Gkatziouras <
> > > > >> >>> > > >> >> > > > >>> > > > > > > > [hidden email]>
> > > > >> >>> > > >> >> > > > >>> > > > > > > > wrote:
> > > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > > > Hi all,
> > > > >> >>> > > >> >> > > > >>> > > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > > > By mentioning Pub/Sub I
> mean
> > > > >> Google
> > > > >> >>> > Cloud's
> > > > >> >>> > > >> >> Pub/Sub
> > > > >> >>> > > >> >> > > [1]
> > > > >> >>> > > >> >> > > > >>> > > > > > > > > Yes it is indeed cloud
> > > specific.
> > > > >> It
> > > > >> >>> is
> > > > >> >>> > the
> > > > >> >>> > > >> >> > messaging
> > > > >> >>> > > >> >> > > > >>> service
> > > > >> >>> > > >> >> > > > >>> > > > > provided
> > > > >> >>> > > >> >> > > > >>> > > > > > > by
> > > > >> >>> > > >> >> > > > >>> > > > > > > > > GCP.
> > > > >> >>> > > >> >> > > > >>> > > > > > > > > I have made the
> integration
> > > with
> > > > >> it,
> > > > >> >>> but
> > > > >> >>> > I
> > > > >> >>> > > >> >> haven't
> > > > >> >>> > > >> >> > > > >>> published
> > > > >> >>> > > >> >> > > > >>> > it
> > > > >> >>> > > >> >> > > > >>> > > > yet
> > > > >> >>> > > >> >> > > > >>> > > > > > (I
> > > > >> >>> > > >> >> > > > >>> > > > > > > am
> > > > >> >>> > > >> >> > > > >>> > > > > > > > > waiting for the
> > > > discussion/ticket
> > > > >> and
> > > > >> >>> > > proceed
> > > > >> >>> > > >> >> with
> > > > >> >>> > > >> >> > > the
> > > > >> >>> > > >> >> > > > >>> > > > > contribution).
> > > > >> >>> > > >> >> > > > >>> > > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > > > Kind regards,
> > > > >> >>> > > >> >> > > > >>> > > > > > > > > Emmanouil
> > > > >> >>> > > >> >> > > > >>> > > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > > > [1]
> > > > >> >>> > > >> >> https://cloud.google.com/pubsub/docs/overview
> > > > >> >>> > > >> >> > > > >>> > > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
> > > > >> >>> > > >> >> > > > >>> > > > > > > > > https://egkatzioura.com/
> |
> > > > >> >>> > > >> >> > > > >>> > > > > > > > >
> > > > >> >>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >> >>> > > >> >> > > > >>> > > > > > > > >
> > https://github.com/gkatzioura
> > > > >> >>> > > >> >> > > > >>> > > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at
> > 21:45,
> > > > >> Denis
> > > > >> >>> > Magda <
> > > > >> >>> > > >> >> > > > >>> [hidden email]
> > > > >> >>> > > >> >> > > > >>> > >
> > > > >> >>> > > >>
>

[Saikat Maitra]

Hi,

We do run build with -DskipTests and then configure selective test suites
to run.

I will check why these test suites are not getting executed.

Regards
Saikat

On Wed, 27 Nov 2019 at 3:05 PM, Emmanouil Gkatziouras <[hidden email]>
wrote:

> Hi all,
>
> I did add a test suite and run the build with the corresponding arguments,
> however the tests did not run [1].
> I checked the Flink's build logs and the same message is displayed there
> too `No tests to run` [2].
>
> [1]
>
> https://ci.ignite.apache.org/viewLog.html?buildId=4798203&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=248
> [2]
>
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4789199&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=1256
>
> Kind regards
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Wed, 27 Nov 2019 at 18:58, Saikat Maitra <[hidden email]>
> wrote:
>
> > Hi,
> >
> > We will need to add test suites for tests to be executed in build. We
> pass
> > them as TestSuites param. I had added Flink sink and source testsuites.
> >
> > Can you please review and confirm.
> >
> > Regards
> > Saikat
> >
> > On Wed, 27 Nov 2019 at 6:00 AM, Emmanouil Gkatziouras <
> > [hidden email]>
> > wrote:
> >
> > > Hi all!
> > >
> > > I was successful building and running my pull requests. It seems that
> the
> > > tests do not run both for Flink and Pub/Sub [1]
> > > If there is something I cannot do to make them discoverable please let
> me
> > > know.
> > >
> > > [1]
> > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4796722&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=283
> > >
> > > *Emmanouil Gkatziouras*
> > > https://egkatzioura.com/ |
> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > https://github.com/gkatzioura
> > >
> > >
> > > On Wed, 27 Nov 2019 at 04:57, Saikat Maitra <[hidden email]>
> > > wrote:
> > >
> > > > Hi Emmanouil,
> > > >
> > > > I have added you as contributor in Ignite Extensions project. Can you
> > > > please check and confirm if you are able to see the project and
> execute
> > > > build on your pull request.
> > > >
> > > > Regards,
> > > > Saikat
> > > >
> > > > On Tue, Nov 26, 2019 at 12:49 PM Emmanouil Gkatziouras <
> > > > [hidden email]>
> > > > wrote:
> > > >
> > > > > Hi all!
> > > > >
> > > > > If someone could give me read access to the Job of Ignite
> Extensions
> > on
> > > > > Team City will greatly help me (username gkatzioura).
> > > > > I suppose the builds get triggered automatically on a pull request.
> > > > >
> > > > > Kind regards,
> > > > > Emmanouil
> > > > >
> > > > > *Emmanouil Gkatziouras*
> > > > > https://egkatzioura.com/ |
> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > https://github.com/gkatzioura
> > > > >
> > > > >
> > > > > On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras <
> > > > [hidden email]>
> > > > > wrote:
> > > > >
> > > > > > Hi Saikat!
> > > > > >
> > > > > > I just rebased with the flink branch. Unfortunately I do not have
> > > read
> > > > > > access to the team city link you provided and thus evaluate the
> > tests
> > > > for
> > > > > > my pull request.
> > > > > > I guess it has to do with ignite extensions being new.
> > > > > > My username is gkatzioura and email the one I am using on this
> > email
> > > > > >
> > > > > > Thank you for your time.
> > > > > >
> > > > > >
> > > > > > *Emmanouil Gkatziouras*
> > > > > > https://egkatzioura.com/ |
> > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > > https://github.com/gkatzioura
> > > > > >
> > > > > >
> > > > > > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <
> > [hidden email]
> > > >
> > > > > > wrote:
> > > > > >
> > > > > >> Hi Emmanouil,
> > > > > >>
> > > > > >> The latest build on teamcity has passed on Flink pull request.
> > > > > >>
> > > > > >>
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> > > > > >>
> > > > > >> you should be able to take the changes I made in my PR and run
> > build
> > > > on
> > > > > >> your pull request.
> > > > > >>
> > > > > >> Please let me know if you have any questions.
> > > > > >>
> > > > > >> Regards,
> > > > > >> Saikat
> > > > > >>
> > > > > >> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <
> > > > [hidden email]
> > > > > >
> > > > > >> wrote:
> > > > > >>
> > > > > >> > Hi Emmanouil,
> > > > > >> >
> > > > > >> > I have fixed the Flink module testsuites and have setup new
> > > project
> > > > in
> > > > > >> > teamcity.
> > > > > >> >
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
> > > > > >> >
> > > > > >> > I am looking into setting up the build.
> > > > > >> >
> > > > > >> > Regards,
> > > > > >> > Saikat
> > > > > >> >
> > > > > >> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <
> > > > > [hidden email]>
> > > > > >> > wrote:
> > > > > >> >
> > > > > >> >> Hi Emmanouil,
> > > > > >> >>
> > > > > >> >> Thank you for your email. Yes, the plan is once the Flink PR
> is
> > > > > merged
> > > > > >> it
> > > > > >> >> will provide the Licence, parent POMs etc and you can rebase
> > from
> > > > > >> master
> > > > > >> >> branch and apply the changes on top of it.
> > > > > >> >>
> > > > > >> >> I am getting some issues with test failures in local with
> > > > > >> >> GridTestProperties as the test.properties is not present in
> > this
> > > > new
> > > > > >> >> project but should be available from dependencies.
> > > > > >> >>
> > > > > >> >> Once I address this issue, I will go ahead and merge the
> > changes
> > > > and
> > > > > >> then
> > > > > >> >> we can take it from there.
> > > > > >> >>
> > > > > >> >> Regards,
> > > > > >> >> Saikat
> > > > > >> >>
> > > > > >> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
> > > > > >> >> [hidden email]> wrote:
> > > > > >> >>
> > > > > >> >>> Hi all,
> > > > > >> >>>
> > > > > >> >>> I made my first pull request [1]. Since this project is
> brand
> > > new
> > > > > (no
> > > > > >> >>> parent poms, licensing), I reckoned it was best to use
> > Saikat's
> > > > > >> branch on
> > > > > >> >>> Flink.
> > > > > >> >>> I suppose the Flink branch will be merged first. If not
> please
> > > > give
> > > > > me
> > > > > >> >>> guidelines on how I should proceed next.
> > > > > >> >>>
> > > > > >> >>> Kind regards
> > > > > >> >>> Emmanouil
> > > > > >> >>>
> > > > > >> >>> [1] https://github.com/apache/ignite-extensions/pull/2
> > > > > >> >>>
> > > > > >> >>> *Emmanouil Gkatziouras*
> > > > > >> >>> https://egkatzioura.com/ |
> > > > > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > >> >>> https://github.com/gkatzioura
> > > > > >> >>>
> > > > > >> >>>
> > > > > >> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <
> [hidden email]>
> > > > > wrote:
> > > > > >> >>>
> > > > > >> >>> > Awesome, ping us whenever you're ready!
> > > > > >> >>> >
> > > > > >> >>> > -
> > > > > >> >>> > Denis
> > > > > >> >>> >
> > > > > >> >>> >
> > > > > >> >>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
> > > > > >> >>> > [hidden email]>
> > > > > >> >>> > wrote:
> > > > > >> >>> >
> > > > > >> >>> > > Hi all!
> > > > > >> >>> > >
> > > > > >> >>> > > I am sorry for being late on that. I was trying to
> > refactor
> > > > the
> > > > > >> test
> > > > > >> >>> in
> > > > > >> >>> > > order not to be in need of any external tools or
> spinning
> > > up a
> > > > > >> >>> server.
> > > > > >> >>> > > I did forked the new repo and indeed my changes there,
> so
> > a
> > > > pull
> > > > > >> >>> request
> > > > > >> >>> > is
> > > > > >> >>> > > a matter of time!
> > > > > >> >>> > >
> > > > > >> >>> > > Kind regards
> > > > > >> >>> > >
> > > > > >> >>> > > *Emmanouil Gkatziouras*
> > > > > >> >>> > > https://egkatzioura.com/ |
> > > > > >> >>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > >> >>> > > https://github.com/gkatzioura
> > > > > >> >>> > >
> > > > > >> >>> > >
> > > > > >> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <
> > > [hidden email]>
> > > > > >> wrote:
> > > > > >> >>> > >
> > > > > >> >>> > > > Hi Emmanouil,
> > > > > >> >>> > > >
> > > > > >> >>> > > > Do you have any questions or need any support from the
> > > > > >> community?
> > > > > >> >>> > > >
> > > > > >> >>> > > > -
> > > > > >> >>> > > > Denis
> > > > > >> >>> > > >
> > > > > >> >>> > > >
> > > > > >> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
> > > > > >> >>> [hidden email]
> > > > > >> >>> > >
> > > > > >> >>> > > > wrote:
> > > > > >> >>> > > >
> > > > > >> >>> > > >> Hi Emmanouil,
> > > > > >> >>> > > >>
> > > > > >> >>> > > >> Can you please take a looks at dev utils, if this is
> > > > > something
> > > > > >> >>> you are
> > > > > >> >>> > > >> looking for
> > > > > >> >>> > > >>
> > > > > https://github.com/apache/ignite/tree/master/modules/dev-utils
> > > > > >> ?
> > > > > >> >>> > > >>
> > > > > >> >>> > > >> IMO, if you can release Pub/Sub server in maven and
> > then
> > > > use
> > > > > >> it as
> > > > > >> >>> > > >> dependency, that would be great.
> > > > > >> >>> > > >>
> > > > > >> >>> > > >> Regards,
> > > > > >> >>> > > >> Saikat
> > > > > >> >>> > > >>
> > > > > >> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
> > > > > >> >>> > [hidden email]>
> > > > > >> >>> > > >> wrote:
> > > > > >> >>> > > >>
> > > > > >> >>> > > >> > Hi Emmanouil,
> > > > > >> >>> > > >> >
> > > > > >> >>> > > >> > The master branch has been initialized. I have
> > raised a
> > > > PR
> > > > > to
> > > > > >> >>> > migrate
> > > > > >> >>> > > >> > Flink module to ignite-extensions.
> > > > > >> >>> > > >> >
> > > > > >> >>> > > >> > PR
> > https://github.com/apache/ignite-extensions/pull/1
> > > > > >> >>> > > >> > Jira :
> > > > https://issues.apache.org/jira/browse/IGNITE-12356
> > > > > >> >>> > > >> >
> > > > > >> >>> > > >> > Regards,
> > > > > >> >>> > > >> > Saikat
> > > > > >> >>> > > >> >
> > > > > >> >>> > > >> >
> > > > > >> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil
> > Gkatziouras <
> > > > > >> >>> > > >> > [hidden email]> wrote:
> > > > > >> >>> > > >> >
> > > > > >> >>> > > >> >> Hi all!
> > > > > >> >>> > > >> >>
> > > > > >> >>> > > >> >> This is an update. The integration is tested with
> > > mocks
> > > > > >> based
> > > > > >> >>> on
> > > > > >> >>> > the
> > > > > >> >>> > > >> >> original Ignite repo [1].
> > > > > >> >>> > > >> >> I am currently waiting for the first commits on
> the
> > > new
> > > > > >> >>> integration
> > > > > >> >>> > > >> >> project
> > > > > >> >>> > > >> >> to take place in order to create my pull request
> > > > > accordingly
> > > > > >> >>> to the
> > > > > >> >>> > > >> >> initial
> > > > > >> >>> > > >> >> commits.
> > > > > >> >>> > > >> >> In order to simulate the Pub/Sub server in a test
> I
> > > > used a
> > > > > >> test
> > > > > >> >>> > > server
> > > > > >> >>> > > >> >> implementation [2]. Unfortunately they don't have
> > this
> > > > > util
> > > > > >> >>> > uploaded
> > > > > >> >>> > > >> in a
> > > > > >> >>> > > >> >> maven repo, thus it needs to be hosted on a maven
> > repo
> > > > > >> (Apache
> > > > > >> >>> > > License
> > > > > >> >>> > > >> >> 2.0). Is there a repo for utilities that Ignite
> uses
> > > for
> > > > > the
> > > > > >> >>> > > >> test/build?
> > > > > >> >>> > > >> >> I have also created a ticket [3]
> > > > > >> >>> > > >> >>
> > > > > >> >>> > > >> >> Kind regards
> > > > > >> >>> > > >> >>
> > > > > >> >>> > > >> >> [1]
> > > > > >> >>> >
> > > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > > > > >> >>> > > >> >> [2]
> > > > > >> >>>
> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> > > > > >> >>> > > >> >> [3]
> > > https://issues.apache.org/jira/browse/IGNITE-12262
> > > > > >> >>> > > >> >>
> > > > > >> >>> > > >> >> *Emmanouil Gkatziouras*
> > > > > >> >>> > > >> >> https://egkatzioura.com/ |
> > > > > >> >>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > >> >>> > > >> >> https://github.com/gkatzioura
> > > > > >> >>> > > >> >>
> > > > > >> >>> > > >> >>
> > > > > >> >>> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
> > > > > >> >>> > [hidden email]
> > > > > >> >>> > > >
> > > > > >> >>> > > >> >> wrote:
> > > > > >> >>> > > >> >>
> > > > > >> >>> > > >> >> > Hello Ilya,
> > > > > >> >>> > > >> >> >
> > > > > >> >>> > > >> >> > Thank you for your email. Yes, before we remove
> > any
> > > > > >> >>> integration
> > > > > >> >>> > we
> > > > > >> >>> > > >> will
> > > > > >> >>> > > >> >> > move them to another repository and release.
> > > > > >> >>> > > >> >> >
> > > > > >> >>> > > >> >> > We are thinking we will start with Pub/Sub
> > Streamer
> > > > > >> >>> > implementation
> > > > > >> >>> > > >> and
> > > > > >> >>> > > >> >> > continue to move other integration as mentioned
> in
> > > the
> > > > > >> >>> following
> > > > > >> >>> > > doc.
> > > > > >> >>> > > >> >> >
> > > > > >> >>> > > >> >> >
> > > > > >> >>> > > >> >> >
> > > > > >> >>> > > >> >>
> > > > > >> >>> > > >>
> > > > > >> >>> > >
> > > > > >> >>> >
> > > > > >> >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > > >> >>> > > >> >> >
> > > > > >> >>> > > >> >> > Regards,
> > > > > >> >>> > > >> >> > Saikat
> > > > > >> >>> > > >> >> >
> > > > > >> >>> > > >> >> >
> > > > > >> >>> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya
> Kasnacheev <
> > > > > >> >>> > > >> >> > [hidden email]>
> > > > > >> >>> > > >> >> > wrote:
> > > > > >> >>> > > >> >> >
> > > > > >> >>> > > >> >> > > Hello!
> > > > > >> >>> > > >> >> > >
> > > > > >> >>> > > >> >> > > My take on this: we want to offload some of
> > > > > >> integrations,
> > > > > >> >>> but
> > > > > >> >>> > > >> please
> > > > > >> >>> > > >> >> > commit
> > > > > >> >>> > > >> >> > > and release them to another project first, and
> > > only
> > > > > >> then we
> > > > > >> >>> > will
> > > > > >> >>> > > >> >> decide
> > > > > >> >>> > > >> >> > to
> > > > > >> >>> > > >> >> > > remove them from our repository.
> > > > > >> >>> > > >> >> > >
> > > > > >> >>> > > >> >> > > Please also make sure they are available with
> > same
> > > > > >> artifact
> > > > > >> >>> > name
> > > > > >> >>> > > >> and
> > > > > >> >>> > > >> >> API.
> > > > > >> >>> > > >> >> > > If this has to be changed, let's wait for AI
> 3.0
> > > > > >> >>> > > >> >> > >
> > > > > >> >>> > > >> >> > > Regards,
> > > > > >> >>> > > >> >> > > --
> > > > > >> >>> > > >> >> > > Ilya Kasnacheev
> > > > > >> >>> > > >> >> > >
> > > > > >> >>> > > >> >> > >
> > > > > >> >>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <
> > > > > >> >>> > > >> [hidden email]>:
> > > > > >> >>> > > >> >> > >
> > > > > >> >>> > > >> >> > > > Hello Emmanouil,
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > > As discussed earlier, I discussed with
> Apache
> > > > Bahir
> > > > > >> >>> community
> > > > > >> >>> > > and
> > > > > >> >>> > > >> >> they
> > > > > >> >>> > > >> >> > > are
> > > > > >> >>> > > >> >> > > > interested to have Apache Ignite extensions
> as
> > > > part
> > > > > of
> > > > > >> >>> Apache
> > > > > >> >>> > > >> Bahir
> > > > > >> >>> > > >> >> > > > project.
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > > Can you please share Pub/Sub streamer
> > > > implementation
> > > > > >> >>> details
> > > > > >> >>> > > with
> > > > > >> >>> > > >> >> > Apache
> > > > > >> >>> > > >> >> > > > Bahir community and request for feedback.
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > > https://bahir.apache.org/
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > > I have also requested for contributor access
> > in
> > > > Jira
> > > > > >> for
> > > > > >> >>> > Apache
> > > > > >> >>> > > >> >> Bahir
> > > > > >> >>> > > >> >> > > > project so that I can create issues and
> assign
> > > to
> > > > > >> myself.
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > > I can help with code reviews as well.
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > > Here is the email thread for reference
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > >
> > > > https://www.mail-archive.com/dev@.../msg02703.html
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > > Regards,
> > > > > >> >>> > > >> >> > > > Saikat
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat
> Maitra
> > <
> > > > > >> >>> > > >> >> [hidden email]
> > > > > >> >>> > > >> >> > >
> > > > > >> >>> > > >> >> > > > wrote:
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > > > Hello,
> > > > > >> >>> > > >> >> > > > >
> > > > > >> >>> > > >> >> > > > > As discussed I have created following
> > > discussion
> > > > > >> >>> threads on
> > > > > >> >>> > > our
> > > > > >> >>> > > >> >> > > migration
> > > > > >> >>> > > >> >> > > > > proposals for Apache Ignite extensions:
> > > > > >> >>> > > >> >> > > > >
> > > > > >> >>> > > >> >> > > > >
> > > > > >> >>> > > >> >> > > > >
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > >
> > > > > >> >>> > > >> >> >
> > > > > >> >>> > > >> >>
> > > > > >> >>> > > >>
> > > > > >> >>> > >
> > > > > >> >>> >
> > > > > >> >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> > > > > >> >>> > > >> >> > > > >
> > > > > >> >>> > > >> >> > > > >
> > > > > >> >>> > > >>
> > > > > >> https://www.mail-archive.com/dev@.../msg02703.html
> > > > > >> >>> > > >> >> > > > >
> > > > > >> >>> > > >> >> > > > > I will share update as I hear more
> > > information.
> > > > > >> >>> > > >> >> > > > >
> > > > > >> >>> > > >> >> > > > > Regards,
> > > > > >> >>> > > >> >> > > > > Saikat
> > > > > >> >>> > > >> >> > > > >
> > > > > >> >>> > > >> >> > > > >
> > > > > >> >>> > > >> >> > > > >
> > > > > >> >>> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat
> > Maitra
> > > <
> > > > > >> >>> > > >> >> > [hidden email]
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > > > wrote:
> > > > > >> >>> > > >> >> > > > >
> > > > > >> >>> > > >> >> > > > >> Hello Denis,
> > > > > >> >>> > > >> >> > > > >>
> > > > > >> >>> > > >> >> > > > >> Sure, sounds good. I will create a
> separate
> > > > > >> discussion
> > > > > >> >>> > > thread
> > > > > >> >>> > > >> to
> > > > > >> >>> > > >> >> get
> > > > > >> >>> > > >> >> > > > >> community feedback on whether we
> > > > > >> >>> > > >> >> > > > >> should join the Bahir or kick-off "Ignite
> > > > > >> Extensions"
> > > > > >> >>> > > project.
> > > > > >> >>> > > >> >> > > > >>
> > > > > >> >>> > > >> >> > > > >> Regards,
> > > > > >> >>> > > >> >> > > > >> Saikat
> > > > > >> >>> > > >> >> > > > >>
> > > > > >> >>> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis
> > Magda <
> > > > > >> >>> > > >> [hidden email]>
> > > > > >> >>> > > >> >> > > wrote:
> > > > > >> >>> > > >> >> > > > >>
> > > > > >> >>> > > >> >> > > > >>> Folks,
> > > > > >> >>> > > >> >> > > > >>>
> > > > > >> >>> > > >> >> > > > >>> The concept of Apache Bahir is precisely
> > > what
> > > > we
> > > > > >> >>> need!
> > > > > >> >>> > > >> Saikat,
> > > > > >> >>> > > >> >> > thanks
> > > > > >> >>> > > >> >> > > > for
> > > > > >> >>> > > >> >> > > > >>> doing the research. Why I believe the *
> > > idea*
> > > > > >> fits us
> > > > > >> >>> > well:
> > > > > >> >>> > > >> >> > > > >>>
> > > > > >> >>> > > >> >> > > > >>>    - All integrations can be stored in
> > > > separate
> > > > > >> >>> Github
> > > > > >> >>> > > >> >> repositories
> > > > > >> >>> > > >> >> > > and
> > > > > >> >>> > > >> >> > > > >>>    have their dev lifecycles. We've not
> > > > obliged
> > > > > to
> > > > > >> >>> couple
> > > > > >> >>> > > all
> > > > > >> >>> > > >> >> the
> > > > > >> >>> > > >> >> > > > >>> integrations
> > > > > >> >>> > > >> >> > > > >>>    in a single repo. For instance, Spark
> > can
> > > > be
> > > > > >> >>> located
> > > > > >> >>> > in
> > > > > >> >>> > > a
> > > > > >> >>> > > >> >> > > dedicated
> > > > > >> >>> > > >> >> > > > >>> repo
> > > > > >> >>> > > >> >> > > > >>>    while streaming integrations might be
> > in
> > > a
> > > > > >> single
> > > > > >> >>> one.
> > > > > >> >>> > > >> It's
> > > > > >> >>> > > >> >> up
> > > > > >> >>> > > >> >> > to
> > > > > >> >>> > > >> >> > > > us.
> > > > > >> >>> > > >> >> > > > >>>    - All the repositories and
> integrations
> > > > > belong
> > > > > >> to
> > > > > >> >>> ASF.
> > > > > >> >>> > > >> We're
> > > > > >> >>> > > >> >> not
> > > > > >> >>> > > >> >> > > > >>> dumping
> > > > > >> >>> > > >> >> > > > >>>    them on Github but rather keep
> > supporting
> > > > and
> > > > > >> >>> > developing
> > > > > >> >>> > > >> in
> > > > > >> >>> > > >> >> > > > >>> accordance with
> > > > > >> >>> > > >> >> > > > >>>    ASF vision and practices.
> > > > > >> >>> > > >> >> > > > >>>    - There is a way to reward
> contributors
> > > via
> > > > > >> >>> > > committership
> > > > > >> >>> > > >> and
> > > > > >> >>> > > >> >> > PMC
> > > > > >> >>> > > >> >> > > > >>>    membership. You won't get this if a
> > > project
> > > > > is
> > > > > >> >>> just
> > > > > >> >>> > one
> > > > > >> >>> > > of
> > > > > >> >>> > > >> >> the
> > > > > >> >>> > > >> >> > > > >>> millions of
> > > > > >> >>> > > >> >> > > > >>>    Github projects.
> > > > > >> >>> > > >> >> > > > >>>
> > > > > >> >>> > > >> >> > > > >>> Saikat, as Ignite PMC member, could you
> > > please
> > > > > >> >>> kick-off a
> > > > > >> >>> > > >> >> separate
> > > > > >> >>> > > >> >> > > > >>> dev-list
> > > > > >> >>> > > >> >> > > > >>> discussion to involve more community
> > members
> > > > and
> > > > > >> >>> > referring
> > > > > >> >>> > > to
> > > > > >> >>> > > >> >> this
> > > > > >> >>> > > >> >> > > > >>> thread?
> > > > > >> >>> > > >> >> > > > >>> I think the primary question the
> community
> > > > needs
> > > > > >> to
> > > > > >> >>> > answer
> > > > > >> >>> > > >> >> whether
> > > > > >> >>> > > >> >> > we
> > > > > >> >>> > > >> >> > > > >>> should join the Bahir or kick-off
> "Ignite
> > > > > >> Extensions"
> > > > > >> >>> > > >> project?
> > > > > >> >>> > > >> >> > > > >>>
> > > > > >> >>> > > >> >> > > > >>> -
> > > > > >> >>> > > >> >> > > > >>> Denis
> > > > > >> >>> > > >> >> > > > >>>
> > > > > >> >>> > > >> >> > > > >>>
> > > > > >> >>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey
> > > > Zinoviev
> > > > > <
> > > > > >> >>> > > >> >> > > > [hidden email]>
> > > > > >> >>> > > >> >> > > > >>> wrote:
> > > > > >> >>> > > >> >> > > > >>>
> > > > > >> >>> > > >> >> > > > >>> > Maybe we could move all our Streaming
> > > > > >> Integrations
> > > > > >> >>> > there,
> > > > > >> >>> > > >> but
> > > > > >> >>> > > >> >> > what
> > > > > >> >>> > > >> >> > > is
> > > > > >> >>> > > >> >> > > > >>> about
> > > > > >> >>> > > >> >> > > > >>> > maintaining and committer permissions
> to
> > > the
> > > > > new
> > > > > >> >>> > > >> repositories?
> > > > > >> >>> > > >> >> > > > >>> > I see the list of the committers and
> PMC
> > > > > members
> > > > > >> >>> there
> > > > > >> >>> > > >> >> > > > >>> >
> > > https://bahir.apache.org/community-members/
> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> >>> > > >> >> > > > >>> > Could somebody from Ignite community
> be
> > > > added
> > > > > to
> > > > > >> >>> this
> > > > > >> >>> > > list
> > > > > >> >>> > > >> as
> > > > > >> >>> > > >> >> a
> > > > > >> >>> > > >> >> > > > >>> > PMC/committer to maintain Ignite
> > > > integrations?
> > > > > >> >>> > > >> >> > > > >>> > If yes, I'd happy to join this project
> > and
> > > > > >> >>> collaborate
> > > > > >> >>> > > with
> > > > > >> >>> > > >> >> these
> > > > > >> >>> > > >> >> > > > guys
> > > > > >> >>> > > >> >> > > > >>> > together
> > > > > >> >>> > > >> >> > > > >>> > If no, and we should start from the
> zero
> > > > level
> > > > > >> with
> > > > > >> >>> > > >> external
> > > > > >> >>> > > >> >> PRs
> > > > > >> >>> > > >> >> > -
> > > > > >> >>> > > >> >> > > > >>> hmmm,
> > > > > >> >>> > > >> >> > > > >>> > it's better to have our own external
> > > > > repository
> > > > > >> >>> with
> > > > > >> >>> > > >> >> ApacheBahirr
> > > > > >> >>> > > >> >> > > > >>> ideology.
> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> >>> > > >> >> > > > >>> > Also, I agree, that all connectors
> could
> > > be
> > > > > >> moved
> > > > > >> >>> there
> > > > > >> >>> > > (in
> > > > > >> >>> > > >> >> > > > >>> ApacheBahirr
> > > > > >> >>> > > >> >> > > > >>> > like repository), but not all modules
> > from
> > > > the
> > > > > >> page
> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> >>> > > >> >> > > > >>>
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > >
> > > > > >> >>> > > >> >> >
> > > > > >> >>> > > >> >>
> > > > > >> >>> > > >>
> > > > > >> >>> > >
> > > > > >> >>> >
> > > > > >> >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > > >> >>> > > >> >> > > > >>> > because
> > > > > >> >>> > > >> >> > > > >>> > they use different parts of core
> modules
> > > and
> > > > > >> could
> > > > > >> >>> be
> > > > > >> >>> > > >> >> developed
> > > > > >> >>> > > >> >> > > with
> > > > > >> >>> > > >> >> > > > >>> > different velocity.
> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> >>> > > >> >> > > > >>> > In reality, before creation of new
> > > > > repositories
> > > > > >> we
> > > > > >> >>> need
> > > > > >> >>> > > >> wide
> > > > > >> >>> > > >> >> > > > discussion
> > > > > >> >>> > > >> >> > > > >>> > based on the document mentioned above.
> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> >>> > > >> >> > > > >>> > P.S. Of course, we could experiment in
> > the
> > > > > next
> > > > > >> >>> release
> > > > > >> >>> > > 2.8
> > > > > >> >>> > > >> >> with
> > > > > >> >>> > > >> >> > > one
> > > > > >> >>> > > >> >> > > > >>> or two
> > > > > >> >>> > > >> >> > > > >>> > integrations like twitter or ZeroMQ
> > > > > >> >>> > > >> >> > > > >>> > Also, I'd like Denis Magda idea for
> the
> > > > > separate
> > > > > >> >>> > > >> repositories
> > > > > >> >>> > > >> >> in
> > > > > >> >>> > > >> >> > > > Apache
> > > > > >> >>> > > >> >> > > > >>> > Github to maintain them by anybody.
> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> >>> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat
> > > Maitra <
> > > > > >> >>> > > >> >> > > [hidden email]
> > > > > >> >>> > > >> >> > > > >:
> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> >>> > > >> >> > > > >>> > > Hi Denis, Emmanouil
> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> >>> > > >> >> > > > >>> > > Thank you for your email. I think
> > > > creating a
> > > > > >> >>> separate
> > > > > >> >>> > > >> >> > integration
> > > > > >> >>> > > >> >> > > > >>> project
> > > > > >> >>> > > >> >> > > > >>> > > in github and also proposing it as
> an
> > > > apache
> > > > > >> >>> > incubator
> > > > > >> >>> > > >> >> project
> > > > > >> >>> > > >> >> > > will
> > > > > >> >>> > > >> >> > > > >>> be a
> > > > > >> >>> > > >> >> > > > >>> > > good move. The other separate
> > > integration
> > > > > >> modules
> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> >>> > > >> >> > > > >>>
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > >
> > > > > >> >>> > > >> >> >
> > > > > >> >>> > > >> >>
> > > > > >> >>> > > >>
> > > > > >> >>> > >
> > > > > >> >>> >
> > > > > >> >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > > >> >>> > > >> >> > > > >>> > > can
> > > > > >> >>> > > >> >> > > > >>> > > be moved to this new apache
> incubator
> > > > > project.
> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> >>> > > >> >> > > > >>> > > There are similar integration
> > available
> > > > for
> > > > > >> >>> Flink and
> > > > > >> >>> > > >> Spark
> > > > > >> >>> > > >> >> in
> > > > > >> >>> > > >> >> > > > Apache
> > > > > >> >>> > > >> >> > > > >>> > Bahir
> > > > > >> >>> > > >> >> > > > >>> > > https://bahir.apache.org/
> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> >>> > > >> >> > > > >>> > > Do you think we should reach out to
> > > Apache
> > > > > >> Bahir
> > > > > >> >>> > > >> community
> > > > > >> >>> > > >> >> > with a
> > > > > >> >>> > > >> >> > > > >>> > proposal
> > > > > >> >>> > > >> >> > > > >>> > > or we should plan to create a
> separate
> > > > > Apache
> > > > > >> >>> > Incubator
> > > > > >> >>> > > >> >> > project?
> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> >>> > > >> >> > > > >>> > > Regards,
> > > > > >> >>> > > >> >> > > > >>> > > Saikat
> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> >>> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM
> > > Emmanouil
> > > > > >> >>> > Gkatziouras <
> > > > > >> >>> > > >> >> > > > >>> > > [hidden email]>
> > > > > >> >>> > > >> >> > > > >>> > > wrote:
> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> >>> > > >> >> > > > >>> > > > Hi all!
> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> >>> > > >> >> > > > >>> > > > Based on the discussions most
> > probably
> > > > > this
> > > > > >> is
> > > > > >> >>> > going
> > > > > >> >>> > > to
> > > > > >> >>> > > >> >> be on
> > > > > >> >>> > > >> >> > > > >>> another
> > > > > >> >>> > > >> >> > > > >>> > > > GitHub repo. Therefore I will
> > prepare
> > > a
> > > > > >> >>> standalone
> > > > > >> >>> > > >> project
> > > > > >> >>> > > >> >> > with
> > > > > >> >>> > > >> >> > > > the
> > > > > >> >>> > > >> >> > > > >>> > > Pub/Sub
> > > > > >> >>> > > >> >> > > > >>> > > > module and once the repository is
> > > > created
> > > > > a
> > > > > >> >>> pull
> > > > > >> >>> > > >> request
> > > > > >> >>> > > >> >> > shall
> > > > > >> >>> > > >> >> > > be
> > > > > >> >>> > > >> >> > > > >>> > issued.
> > > > > >> >>> > > >> >> > > > >>> > > > If there is anything else I can do
> > in
> > > > > order
> > > > > >> to
> > > > > >> >>> > > >> facilitate
> > > > > >> >>> > > >> >> > this
> > > > > >> >>> > > >> >> > > > >>> please
> > > > > >> >>> > > >> >> > > > >>> > let
> > > > > >> >>> > > >> >> > > > >>> > > > me know.
> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> >>> > > >> >> > > > >>> > > > If a ticket is created for this
> > > issue, I
> > > > > >> shall
> > > > > >> >>> be
> > > > > >> >>> > > able
> > > > > >> >>> > > >> to
> > > > > >> >>> > > >> >> > > create
> > > > > >> >>> > > >> >> > > > a
> > > > > >> >>> > > >> >> > > > >>> pull
> > > > > >> >>> > > >> >> > > > >>> > > > request and thus a build will take
> > > place
> > > > > on
> > > > > >> >>> > ignite's
> > > > > >> >>> > > >> CI.
> > > > > >> >>> > > >> >> > > > >>> > > > If everyone is aligned with
> adding a
> > > > > Pub/Sub
> > > > > >> >>> > > >> >> implementation
> > > > > >> >>> > > >> >> > > may I
> > > > > >> >>> > > >> >> > > > >>> > create
> > > > > >> >>> > > >> >> > > > >>> > > a
> > > > > >> >>> > > >> >> > > > >>> > > > ticket on JIRA?
> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> >>> > > >> >> > > > >>> > > > My next question has to do on how
> > > Ignite
> > > > > >> >>> configures
> > > > > >> >>> > > the
> > > > > >> >>> > > >> >> build
> > > > > >> >>> > > >> >> > > > jobs
> > > > > >> >>> > > >> >> > > > >>> in
> > > > > >> >>> > > >> >> > > > >>> > > Team
> > > > > >> >>> > > >> >> > > > >>> > > > City. The unit tests for my
> > > > implementation
> > > > > >> >>> need a
> > > > > >> >>> > > local
> > > > > >> >>> > > >> >> > Pub/Sub
> > > > > >> >>> > > >> >> > > > >>> server
> > > > > >> >>> > > >> >> > > > >>> > up
> > > > > >> >>> > > >> >> > > > >>> > > > and running (they provide one for
> > > > testing
> > > > > >> >>> > purposes).
> > > > > >> >>> > > >> Will
> > > > > >> >>> > > >> >> > this
> > > > > >> >>> > > >> >> > > > take
> > > > > >> >>> > > >> >> > > > >>> > > effect
> > > > > >> >>> > > >> >> > > > >>> > > > in a form of a build script inside
> > my
> > > > > >> >>> > implementation
> > > > > >> >>> > > or
> > > > > >> >>> > > >> >> is it
> > > > > >> >>> > > >> >> > > > >>> something
> > > > > >> >>> > > >> >> > > > >>> > > > that should be configured on Team
> > > City?
> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> >>> > > >> >> > > > >>> > > > Kind regards,
> > > > > >> >>> > > >> >> > > > >>> > > > Emmanouil
> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> >>> > > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
> > > > > >> >>> > > >> >> > > > >>> > > > https://egkatzioura.com/ |
> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > >> >>> > > >> >> > > > >>> > > > https://github.com/gkatzioura
> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> >>> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27,
> Denis
> > > > Magda
> > > > > <
> > > > > >> >>> > > >> >> [hidden email]
> > > > > >> >>> > > >> >> > >
> > > > > >> >>> > > >> >> > > > >>> wrote:
> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > After contemplating on this, let
> > me
> > > > > >> propose
> > > > > >> >>> > another
> > > > > >> >>> > > >> way
> > > > > >> >>> > > >> >> > with
> > > > > >> >>> > > >> >> > > > this
> > > > > >> >>> > > >> >> > > > >>> > > > > particular integration that is
> > > aligned
> > > > > >> with
> > > > > >> >>> our
> > > > > >> >>> > > >> >> > > modularization
> > > > > >> >>> > > >> >> > > > >>> > > endeavor.
> > > > > >> >>> > > >> >> > > > >>> > > > > The modularization [1] defines
> the
> > > > > Ignite
> > > > > >> >>> core,
> > > > > >> >>> > > >> Ignite
> > > > > >> >>> > > >> >> > > modules,
> > > > > >> >>> > > >> >> > > > >>> and
> > > > > >> >>> > > >> >> > > > >>> > > > > independent integrations. The
> > first
> > > > two
> > > > > >> are
> > > > > >> >>> to be
> > > > > >> >>> > > >> >> located
> > > > > >> >>> > > >> >> > in
> > > > > >> >>> > > >> >> > > > >>> Ignite
> > > > > >> >>> > > >> >> > > > >>> > > > > repositories and to be
> > > tested/updated
> > > > by
> > > > > >> the
> > > > > >> >>> > > >> community
> > > > > >> >>> > > >> >> for
> > > > > >> >>> > > >> >> > > > every
> > > > > >> >>> > > >> >> > > > >>> > > release.
> > > > > >> >>> > > >> >> > > > >>> > > > > The modules are a subset of
> > existing
> > > > > >> >>> integrations
> > > > > >> >>> > > >> that
> > > > > >> >>> > > >> >> are
> > > > > >> >>> > > >> >> > > > >>> adopted
> > > > > >> >>> > > >> >> > > > >>> > > widely
> > > > > >> >>> > > >> >> > > > >>> > > > > and require community attention.
> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > As for the independent
> > integrations
> > > > [1],
> > > > > >> >>> those
> > > > > >> >>> > are
> > > > > >> >>> > > >> >> supposed
> > > > > >> >>> > > >> >> > > to
> > > > > >> >>> > > >> >> > > > be
> > > > > >> >>> > > >> >> > > > >>> > > located
> > > > > >> >>> > > >> >> > > > >>> > > > > in a separate GitHub repo. I
> would
> > > > > advise
> > > > > >> us
> > > > > >> >>> to
> > > > > >> >>> > > start
> > > > > >> >>> > > >> >> > > > >>> implementing
> > > > > >> >>> > > >> >> > > > >>> > this
> > > > > >> >>> > > >> >> > > > >>> > > > > concept with the Pub/Sub
> > > integration.
> > > > > Even
> > > > > >> >>> though
> > > > > >> >>> > > it
> > > > > >> >>> > > >> >> will
> > > > > >> >>> > > >> >> > be
> > > > > >> >>> > > >> >> > > > >>> located
> > > > > >> >>> > > >> >> > > > >>> > > in a
> > > > > >> >>> > > >> >> > > > >>> > > > > separate Github repo, it will be
> > > > listed
> > > > > >> among
> > > > > >> >>> > > >> officially
> > > > > >> >>> > > >> >> > > > >>> available
> > > > > >> >>> > > >> >> > > > >>> > > > > integrations on the Ignite
> website
> > > > > >> featuring
> > > > > >> >>> > > >> Emmanouil
> > > > > >> >>> > > >> >> as a
> > > > > >> >>> > > >> >> > > > >>> > > contributor.
> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > What do you think?
> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > [1]
> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> >>> > > >> >> > > > >>>
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > >
> > > > > >> >>> > > >> >> >
> > > > > >> >>> > > >> >>
> > > > > >> >>> > > >>
> > > > > >> >>> > >
> > > > > >> >>> >
> > > > > >> >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > -
> > > > > >> >>> > > >> >> > > > >>> > > > > Denis
> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM
> > > Saikat
> > > > > >> >>> Maitra <
> > > > > >> >>> > > >> >> > > > >>> > [hidden email]
> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > wrote:
> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > > Hi Emmanouil,
> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > > The changes looks good to me.
> I
> > > > wanted
> > > > > >> to
> > > > > >> >>> check
> > > > > >> >>> > > if
> > > > > >> >>> > > >> you
> > > > > >> >>> > > >> >> > are
> > > > > >> >>> > > >> >> > > > also
> > > > > >> >>> > > >> >> > > > >>> > able
> > > > > >> >>> > > >> >> > > > >>> > > to
> > > > > >> >>> > > >> >> > > > >>> > > > > > validate that the once data is
> > > added
> > > > > to
> > > > > >> >>> Ignite
> > > > > >> >>> > > >> cluster
> > > > > >> >>> > > >> >> > you
> > > > > >> >>> > > >> >> > > > are
> > > > > >> >>> > > >> >> > > > >>> also
> > > > > >> >>> > > >> >> > > > >>> > > > able
> > > > > >> >>> > > >> >> > > > >>> > > > > to
> > > > > >> >>> > > >> >> > > > >>> > > > > > access it using another Ignite
> > > > client
> > > > > or
> > > > > >> >>> using
> > > > > >> >>> > > rest
> > > > > >> >>> > > >> >> > > > endpoints.
> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > > We use Teamcity for CI
> process,
> > > the
> > > > > >> >>> details are
> > > > > >> >>> > > as
> > > > > >> >>> > > >> >> > follows
> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > > Trigger validation of those
> test
> > > > > suites
> > > > > >> >>> that
> > > > > >> >>> > have
> > > > > >> >>> > > >> been
> > > > > >> >>> > > >> >> > > > >>> affected by
> > > > > >> >>> > > >> >> > > > >>> > > your
> > > > > >> >>> > > >> >> > > > >>> > > > > > changes on TeamCity <
> > > > > >> >>> > > http://ci.ignite.apache.org/>:
> > > > > >> >>> > > >> <
> > > > > >> >>> > > >> >> > > > >>> > > > > http://204.14.53.153/
> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Locate a test suite you
> > have
> > > to
> > > > > >> check,
> > > > > >> >>> > press
> > > > > >> >>> > > >> >> button
> > > > > >> >>> > > >> >> > > > named
> > > > > >> >>> > > >> >> > > > >>> > "..."
> > > > > >> >>> > > >> >> > > > >>> > > > that
> > > > > >> >>> > > >> >> > > > >>> > > > > >    is located on the left of
> > "Run"
> > > > > >> button.
> > > > > >> >>> "Run
> > > > > >> >>> > > >> custom
> > > > > >> >>> > > >> >> > > build"
> > > > > >> >>> > > >> >> > > > >>> > dialog
> > > > > >> >>> > > >> >> > > > >>> > > > will
> > > > > >> >>> > > >> >> > > > >>> > > > > >    appear;
> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Go to "Changes" tab and
> > > choose
> > > > > >> >>> > > >> >> > > > >>> > "pull/<pull-request-number>/head"
> > > > > >> >>> > > >> >> > > > >>> > > > in
> > > > > >> >>> > > >> >> > > > >>> > > > > >    "Build branch" dropdown
> list;
> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Press "Run build" button
> > and
> > > > > >> monitor
> > > > > >> >>> tests
> > > > > >> >>> > > >> >> results.
> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > > @Roman [hidden email] -
> can
> > > you
> > > > > also
> > > > > >> >>> please
> > > > > >> >>> > > >> take a
> > > > > >> >>> > > >> >> > > look?
> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > > Regards,
> > > > > >> >>> > > >> >> > > > >>> > > > > > Saikat
> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20
> PM
> > > > > >> Emmanouil
> > > > > >> >>> > > >> Gkatziouras
> > > > > >> >>> > > >> >> <
> > > > > >> >>> > > >> >> > > > >>> > > > > > [hidden email]>
> > > > > >> >>> > > >> >> > > > >>> > > > > > wrote:
> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > > > I have my implementation on
> an
> > > > > Ignite
> > > > > >> >>> fork on
> > > > > >> >>> > > my
> > > > > >> >>> > > >> >> > personal
> > > > > >> >>> > > >> >> > > > >>> GitHub
> > > > > >> >>> > > >> >> > > > >>> > > > > account
> > > > > >> >>> > > >> >> > > > >>> > > > > > > [1]
> > > > > >> >>> > > >> >> > > > >>> > > > > > > It is based on the Kafka
> > > > > >> implementation
> > > > > >> >>> which
> > > > > >> >>> > > is
> > > > > >> >>> > > >> of
> > > > > >> >>> > > >> >> a
> > > > > >> >>> > > >> >> > > > polling
> > > > > >> >>> > > >> >> > > > >>> > > nature.
> > > > > >> >>> > > >> >> > > > >>> > > > > > > GCP does not provide a
> service
> > > > like
> > > > > >> >>> Kafka.
> > > > > >> >>> > > >> Instead
> > > > > >> >>> > > >> >> it
> > > > > >> >>> > > >> >> > > > >>> provides us
> > > > > >> >>> > > >> >> > > > >>> > > > with
> > > > > >> >>> > > >> >> > > > >>> > > > > > > Pub/Sub which is just like
> > > Kafka.
> > > > > >> >>> > > >> >> > > > >>> > > > > > > Therefore using the Kafka
> > > streamer
> > > > > as
> > > > > >> a
> > > > > >> >>> > > guideline
> > > > > >> >>> > > >> >> made
> > > > > >> >>> > > >> >> > > more
> > > > > >> >>> > > >> >> > > > >>> sense
> > > > > >> >>> > > >> >> > > > >>> > > to
> > > > > >> >>> > > >> >> > > > >>> > > > > me.
> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > > > Apart from the
> implementation
> > I
> > > > have
> > > > > >> also
> > > > > >> >>> > > added a
> > > > > >> >>> > > >> >> unit
> > > > > >> >>> > > >> >> > > > test.
> > > > > >> >>> > > >> >> > > > >>> > > > > > > It definitely needs some
> > > polishing
> > > > > >> and a
> > > > > >> >>> > double
> > > > > >> >>> > > >> >> check
> > > > > >> >>> > > >> >> > on
> > > > > >> >>> > > >> >> > > my
> > > > > >> >>> > > >> >> > > > >>> side
> > > > > >> >>> > > >> >> > > > >>> > to
> > > > > >> >>> > > >> >> > > > >>> > > > > make
> > > > > >> >>> > > >> >> > > > >>> > > > > > > the review smooth.
> > > > > >> >>> > > >> >> > > > >>> > > > > > > For the tests a local
> Pub/Sub
> > > [2]
> > > > > >> server
> > > >

[Saikat Maitra]

Hello,

I have made changes in Ignite Extensions build configurations.

It is now running the tests as expected.

https://ci.ignite.apache.org/viewLog.html?buildId=4804480&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead

Regards,
Saikat

On Thu, Nov 28, 2019 at 1:29 PM Saikat Maitra <[hidden email]>
wrote:

> Hi,
>
> We do run build with -DskipTests and then configure selective test suites
> to run.
>
> I will check why these test suites are not getting executed.
>
> Regards
> Saikat
>
> On Wed, 27 Nov 2019 at 3:05 PM, Emmanouil Gkatziouras <
> [hidden email]> wrote:
>
>> Hi all,
>>
>> I did add a test suite and run the build with the corresponding arguments,
>> however the tests did not run [1].
>> I checked the Flink's build logs and the same message is displayed there
>> too `No tests to run` [2].
>>
>> [1]
>>
>> https://ci.ignite.apache.org/viewLog.html?buildId=4798203&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=248
>> [2]
>>
>> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4789199&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=1256
>>
>> Kind regards
>> *Emmanouil Gkatziouras*
>> https://egkatzioura.com/ |
>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> https://github.com/gkatzioura
>>
>>
>> On Wed, 27 Nov 2019 at 18:58, Saikat Maitra <[hidden email]>
>> wrote:
>>
>> > Hi,
>> >
>> > We will need to add test suites for tests to be executed in build. We
>> pass
>> > them as TestSuites param. I had added Flink sink and source testsuites.
>> >
>> > Can you please review and confirm.
>> >
>> > Regards
>> > Saikat
>> >
>> > On Wed, 27 Nov 2019 at 6:00 AM, Emmanouil Gkatziouras <
>> > [hidden email]>
>> > wrote:
>> >
>> > > Hi all!
>> > >
>> > > I was successful building and running my pull requests. It seems that
>> the
>> > > tests do not run both for Flink and Pub/Sub [1]
>> > > If there is something I cannot do to make them discoverable please
>> let me
>> > > know.
>> > >
>> > > [1]
>> > >
>> > >
>> >
>> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4796722&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=283
>> > >
>> > > *Emmanouil Gkatziouras*
>> > > https://egkatzioura.com/ |
>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > https://github.com/gkatzioura
>> > >
>> > >
>> > > On Wed, 27 Nov 2019 at 04:57, Saikat Maitra <[hidden email]>
>> > > wrote:
>> > >
>> > > > Hi Emmanouil,
>> > > >
>> > > > I have added you as contributor in Ignite Extensions project. Can
>> you
>> > > > please check and confirm if you are able to see the project and
>> execute
>> > > > build on your pull request.
>> > > >
>> > > > Regards,
>> > > > Saikat
>> > > >
>> > > > On Tue, Nov 26, 2019 at 12:49 PM Emmanouil Gkatziouras <
>> > > > [hidden email]>
>> > > > wrote:
>> > > >
>> > > > > Hi all!
>> > > > >
>> > > > > If someone could give me read access to the Job of Ignite
>> Extensions
>> > on
>> > > > > Team City will greatly help me (username gkatzioura).
>> > > > > I suppose the builds get triggered automatically on a pull
>> request.
>> > > > >
>> > > > > Kind regards,
>> > > > > Emmanouil
>> > > > >
>> > > > > *Emmanouil Gkatziouras*
>> > > > > https://egkatzioura.com/ |
>> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > > https://github.com/gkatzioura
>> > > > >
>> > > > >
>> > > > > On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras <
>> > > > [hidden email]>
>> > > > > wrote:
>> > > > >
>> > > > > > Hi Saikat!
>> > > > > >
>> > > > > > I just rebased with the flink branch. Unfortunately I do not
>> have
>> > > read
>> > > > > > access to the team city link you provided and thus evaluate the
>> > tests
>> > > > for
>> > > > > > my pull request.
>> > > > > > I guess it has to do with ignite extensions being new.
>> > > > > > My username is gkatzioura and email the one I am using on this
>> > email
>> > > > > >
>> > > > > > Thank you for your time.
>> > > > > >
>> > > > > >
>> > > > > > *Emmanouil Gkatziouras*
>> > > > > > https://egkatzioura.com/ |
>> > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > > > https://github.com/gkatzioura
>> > > > > >
>> > > > > >
>> > > > > > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <
>> > [hidden email]
>> > > >
>> > > > > > wrote:
>> > > > > >
>> > > > > >> Hi Emmanouil,
>> > > > > >>
>> > > > > >> The latest build on teamcity has passed on Flink pull request.
>> > > > > >>
>> > > > > >>
>> > > > > >>
>> > > > >
>> > > >
>> > >
>> >
>> https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
>> > > > > >>
>> > > > > >> you should be able to take the changes I made in my PR and run
>> > build
>> > > > on
>> > > > > >> your pull request.
>> > > > > >>
>> > > > > >> Please let me know if you have any questions.
>> > > > > >>
>> > > > > >> Regards,
>> > > > > >> Saikat
>> > > > > >>
>> > > > > >> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <
>> > > > [hidden email]
>> > > > > >
>> > > > > >> wrote:
>> > > > > >>
>> > > > > >> > Hi Emmanouil,
>> > > > > >> >
>> > > > > >> > I have fixed the Flink module testsuites and have setup new
>> > > project
>> > > > in
>> > > > > >> > teamcity.
>> > > > > >> >
>> > > > > >> >
>> > > > > >> >
>> > > > > >>
>> > > > >
>> > > >
>> > >
>> >
>> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
>> > > > > >> >
>> > > > > >> > I am looking into setting up the build.
>> > > > > >> >
>> > > > > >> > Regards,
>> > > > > >> > Saikat
>> > > > > >> >
>> > > > > >> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <
>> > > > > [hidden email]>
>> > > > > >> > wrote:
>> > > > > >> >
>> > > > > >> >> Hi Emmanouil,
>> > > > > >> >>
>> > > > > >> >> Thank you for your email. Yes, the plan is once the Flink
>> PR is
>> > > > > merged
>> > > > > >> it
>> > > > > >> >> will provide the Licence, parent POMs etc and you can rebase
>> > from
>> > > > > >> master
>> > > > > >> >> branch and apply the changes on top of it.
>> > > > > >> >>
>> > > > > >> >> I am getting some issues with test failures in local with
>> > > > > >> >> GridTestProperties as the test.properties is not present in
>> > this
>> > > > new
>> > > > > >> >> project but should be available from dependencies.
>> > > > > >> >>
>> > > > > >> >> Once I address this issue, I will go ahead and merge the
>> > changes
>> > > > and
>> > > > > >> then
>> > > > > >> >> we can take it from there.
>> > > > > >> >>
>> > > > > >> >> Regards,
>> > > > > >> >> Saikat
>> > > > > >> >>
>> > > > > >> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
>> > > > > >> >> [hidden email]> wrote:
>> > > > > >> >>
>> > > > > >> >>> Hi all,
>> > > > > >> >>>
>> > > > > >> >>> I made my first pull request [1]. Since this project is
>> brand
>> > > new
>> > > > > (no
>> > > > > >> >>> parent poms, licensing), I reckoned it was best to use
>> > Saikat's
>> > > > > >> branch on
>> > > > > >> >>> Flink.
>> > > > > >> >>> I suppose the Flink branch will be merged first. If not
>> please
>> > > > give
>> > > > > me
>> > > > > >> >>> guidelines on how I should proceed next.
>> > > > > >> >>>
>> > > > > >> >>> Kind regards
>> > > > > >> >>> Emmanouil
>> > > > > >> >>>
>> > > > > >> >>> [1] https://github.com/apache/ignite-extensions/pull/2
>> > > > > >> >>>
>> > > > > >> >>> *Emmanouil Gkatziouras*
>> > > > > >> >>> https://egkatzioura.com/ |
>> > > > > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > > >> >>> https://github.com/gkatzioura
>> > > > > >> >>>
>> > > > > >> >>>
>> > > > > >> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <
>> [hidden email]>
>> > > > > wrote:
>> > > > > >> >>>
>> > > > > >> >>> > Awesome, ping us whenever you're ready!
>> > > > > >> >>> >
>> > > > > >> >>> > -
>> > > > > >> >>> > Denis
>> > > > > >> >>> >
>> > > > > >> >>> >
>> > > > > >> >>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
>> > > > > >> >>> > [hidden email]>
>> > > > > >> >>> > wrote:
>> > > > > >> >>> >
>> > > > > >> >>> > > Hi all!
>> > > > > >> >>> > >
>> > > > > >> >>> > > I am sorry for being late on that. I was trying to
>> > refactor
>> > > > the
>> > > > > >> test
>> > > > > >> >>> in
>> > > > > >> >>> > > order not to be in need of any external tools or
>> spinning
>> > > up a
>> > > > > >> >>> server.
>> > > > > >> >>> > > I did forked the new repo and indeed my changes there,
>> so
>> > a
>> > > > pull
>> > > > > >> >>> request
>> > > > > >> >>> > is
>> > > > > >> >>> > > a matter of time!
>> > > > > >> >>> > >
>> > > > > >> >>> > > Kind regards
>> > > > > >> >>> > >
>> > > > > >> >>> > > *Emmanouil Gkatziouras*
>> > > > > >> >>> > > https://egkatzioura.com/ |
>> > > > > >> >>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > > >> >>> > > https://github.com/gkatzioura
>> > > > > >> >>> > >
>> > > > > >> >>> > >
>> > > > > >> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <
>> > > [hidden email]>
>> > > > > >> wrote:
>> > > > > >> >>> > >
>> > > > > >> >>> > > > Hi Emmanouil,
>> > > > > >> >>> > > >
>> > > > > >> >>> > > > Do you have any questions or need any support from
>> the
>> > > > > >> community?
>> > > > > >> >>> > > >
>> > > > > >> >>> > > > -
>> > > > > >> >>> > > > Denis
>> > > > > >> >>> > > >
>> > > > > >> >>> > > >
>> > > > > >> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
>> > > > > >> >>> [hidden email]
>> > > > > >> >>> > >
>> > > > > >> >>> > > > wrote:
>> > > > > >> >>> > > >
>> > > > > >> >>> > > >> Hi Emmanouil,
>> > > > > >> >>> > > >>
>> > > > > >> >>> > > >> Can you please take a looks at dev utils, if this is
>> > > > > something
>> > > > > >> >>> you are
>> > > > > >> >>> > > >> looking for
>> > > > > >> >>> > > >>
>> > > > > https://github.com/apache/ignite/tree/master/modules/dev-utils
>> > > > > >> ?
>> > > > > >> >>> > > >>
>> > > > > >> >>> > > >> IMO, if you can release Pub/Sub server in maven and
>> > then
>> > > > use
>> > > > > >> it as
>> > > > > >> >>> > > >> dependency, that would be great.
>> > > > > >> >>> > > >>
>> > > > > >> >>> > > >> Regards,
>> > > > > >> >>> > > >> Saikat
>> > > > > >> >>> > > >>
>> > > > > >> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
>> > > > > >> >>> > [hidden email]>
>> > > > > >> >>> > > >> wrote:
>> > > > > >> >>> > > >>
>> > > > > >> >>> > > >> > Hi Emmanouil,
>> > > > > >> >>> > > >> >
>> > > > > >> >>> > > >> > The master branch has been initialized. I have
>> > raised a
>> > > > PR
>> > > > > to
>> > > > > >> >>> > migrate
>> > > > > >> >>> > > >> > Flink module to ignite-extensions.
>> > > > > >> >>> > > >> >
>> > > > > >> >>> > > >> > PR
>> > https://github.com/apache/ignite-extensions/pull/1
>> > > > > >> >>> > > >> > Jira :
>> > > > https://issues.apache.org/jira/browse/IGNITE-12356
>> > > > > >> >>> > > >> >
>> > > > > >> >>> > > >> > Regards,
>> > > > > >> >>> > > >> > Saikat
>> > > > > >> >>> > > >> >
>> > > > > >> >>> > > >> >
>> > > > > >> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil
>> > Gkatziouras <
>> > > > > >> >>> > > >> > [hidden email]> wrote:
>> > > > > >> >>> > > >> >
>> > > > > >> >>> > > >> >> Hi all!
>> > > > > >> >>> > > >> >>
>> > > > > >> >>> > > >> >> This is an update. The integration is tested with
>> > > mocks
>> > > > > >> based
>> > > > > >> >>> on
>> > > > > >> >>> > the
>> > > > > >> >>> > > >> >> original Ignite repo [1].
>> > > > > >> >>> > > >> >> I am currently waiting for the first commits on
>> the
>> > > new
>> > > > > >> >>> integration
>> > > > > >> >>> > > >> >> project
>> > > > > >> >>> > > >> >> to take place in order to create my pull request
>> > > > > accordingly
>> > > > > >> >>> to the
>> > > > > >> >>> > > >> >> initial
>> > > > > >> >>> > > >> >> commits.
>> > > > > >> >>> > > >> >> In order to simulate the Pub/Sub server in a
>> test I
>> > > > used a
>> > > > > >> test
>> > > > > >> >>> > > server
>> > > > > >> >>> > > >> >> implementation [2]. Unfortunately they don't have
>> > this
>> > > > > util
>> > > > > >> >>> > uploaded
>> > > > > >> >>> > > >> in a
>> > > > > >> >>> > > >> >> maven repo, thus it needs to be hosted on a maven
>> > repo
>> > > > > >> (Apache
>> > > > > >> >>> > > License
>> > > > > >> >>> > > >> >> 2.0). Is there a repo for utilities that Ignite
>> uses
>> > > for
>> > > > > the
>> > > > > >> >>> > > >> test/build?
>> > > > > >> >>> > > >> >> I have also created a ticket [3]
>> > > > > >> >>> > > >> >>
>> > > > > >> >>> > > >> >> Kind regards
>> > > > > >> >>> > > >> >>
>> > > > > >> >>> > > >> >> [1]
>> > > > > >> >>> >
>> > > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
>> > > > > >> >>> > > >> >> [2]
>> > > > > >> >>>
>> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
>> > > > > >> >>> > > >> >> [3]
>> > > https://issues.apache.org/jira/browse/IGNITE-12262
>> > > > > >> >>> > > >> >>
>> > > > > >> >>> > > >> >> *Emmanouil Gkatziouras*
>> > > > > >> >>> > > >> >> https://egkatzioura.com/ |
>> > > > > >> >>> > > >> >>
>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > > >> >>> > > >> >> https://github.com/gkatzioura
>> > > > > >> >>> > > >> >>
>> > > > > >> >>> > > >> >>
>> > > > > >> >>> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
>> > > > > >> >>> > [hidden email]
>> > > > > >> >>> > > >
>> > > > > >> >>> > > >> >> wrote:
>> > > > > >> >>> > > >> >>
>> > > > > >> >>> > > >> >> > Hello Ilya,
>> > > > > >> >>> > > >> >> >
>> > > > > >> >>> > > >> >> > Thank you for your email. Yes, before we remove
>> > any
>> > > > > >> >>> integration
>> > > > > >> >>> > we
>> > > > > >> >>> > > >> will
>> > > > > >> >>> > > >> >> > move them to another repository and release.
>> > > > > >> >>> > > >> >> >
>> > > > > >> >>> > > >> >> > We are thinking we will start with Pub/Sub
>> > Streamer
>> > > > > >> >>> > implementation
>> > > > > >> >>> > > >> and
>> > > > > >> >>> > > >> >> > continue to move other integration as
>> mentioned in
>> > > the
>> > > > > >> >>> following
>> > > > > >> >>> > > doc.
>> > > > > >> >>> > > >> >> >
>> > > > > >> >>> > > >> >> >
>> > > > > >> >>> > > >> >> >
>> > > > > >> >>> > > >> >>
>> > > > > >> >>> > > >>
>> > > > > >> >>> > >
>> > > > > >> >>> >
>> > > > > >> >>>
>> > > > > >>
>> > > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> > > > > >> >>> > > >> >> >
>> > > > > >> >>> > > >> >> > Regards,
>> > > > > >> >>> > > >> >> > Saikat
>> > > > > >> >>> > > >> >> >
>> > > > > >> >>> > > >> >> >
>> > > > > >> >>> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya
>> Kasnacheev <
>> > > > > >> >>> > > >> >> > [hidden email]>
>> > > > > >> >>> > > >> >> > wrote:
>> > > > > >> >>> > > >> >> >
>> > > > > >> >>> > > >> >> > > Hello!
>> > > > > >> >>> > > >> >> > >
>> > > > > >> >>> > > >> >> > > My take on this: we want to offload some of
>> > > > > >> integrations,
>> > > > > >> >>> but
>> > > > > >> >>> > > >> please
>> > > > > >> >>> > > >> >> > commit
>> > > > > >> >>> > > >> >> > > and release them to another project first,
>> and
>> > > only
>> > > > > >> then we
>> > > > > >> >>> > will
>> > > > > >> >>> > > >> >> decide
>> > > > > >> >>> > > >> >> > to
>> > > > > >> >>> > > >> >> > > remove them from our repository.
>> > > > > >> >>> > > >> >> > >
>> > > > > >> >>> > > >> >> > > Please also make sure they are available with
>> > same
>> > > > > >> artifact
>> > > > > >> >>> > name
>> > > > > >> >>> > > >> and
>> > > > > >> >>> > > >> >> API.
>> > > > > >> >>> > > >> >> > > If this has to be changed, let's wait for AI
>> 3.0
>> > > > > >> >>> > > >> >> > >
>> > > > > >> >>> > > >> >> > > Regards,
>> > > > > >> >>> > > >> >> > > --
>> > > > > >> >>> > > >> >> > > Ilya Kasnacheev
>> > > > > >> >>> > > >> >> > >
>> > > > > >> >>> > > >> >> > >
>> > > > > >> >>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <
>> > > > > >> >>> > > >> [hidden email]>:
>> > > > > >> >>> > > >> >> > >
>> > > > > >> >>> > > >> >> > > > Hello Emmanouil,
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > > As discussed earlier, I discussed with
>> Apache
>> > > > Bahir
>> > > > > >> >>> community
>> > > > > >> >>> > > and
>> > > > > >> >>> > > >> >> they
>> > > > > >> >>> > > >> >> > > are
>> > > > > >> >>> > > >> >> > > > interested to have Apache Ignite
>> extensions as
>> > > > part
>> > > > > of
>> > > > > >> >>> Apache
>> > > > > >> >>> > > >> Bahir
>> > > > > >> >>> > > >> >> > > > project.
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > > Can you please share Pub/Sub streamer
>> > > > implementation
>> > > > > >> >>> details
>> > > > > >> >>> > > with
>> > > > > >> >>> > > >> >> > Apache
>> > > > > >> >>> > > >> >> > > > Bahir community and request for feedback.
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > > https://bahir.apache.org/
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > > I have also requested for contributor
>> access
>> > in
>> > > > Jira
>> > > > > >> for
>> > > > > >> >>> > Apache
>> > > > > >> >>> > > >> >> Bahir
>> > > > > >> >>> > > >> >> > > > project so that I can create issues and
>> assign
>> > > to
>> > > > > >> myself.
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > > I can help with code reviews as well.
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > > Here is the email thread for reference
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > >
>> > > > https://www.mail-archive.com/dev@.../msg02703.html
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > > Regards,
>> > > > > >> >>> > > >> >> > > > Saikat
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat
>> Maitra
>> > <
>> > > > > >> >>> > > >> >> [hidden email]
>> > > > > >> >>> > > >> >> > >
>> > > > > >> >>> > > >> >> > > > wrote:
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > > > Hello,
>> > > > > >> >>> > > >> >> > > > >
>> > > > > >> >>> > > >> >> > > > > As discussed I have created following
>> > > discussion
>> > > > > >> >>> threads on
>> > > > > >> >>> > > our
>> > > > > >> >>> > > >> >> > > migration
>> > > > > >> >>> > > >> >> > > > > proposals for Apache Ignite extensions:
>> > > > > >> >>> > > >> >> > > > >
>> > > > > >> >>> > > >> >> > > > >
>> > > > > >> >>> > > >> >> > > > >
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > >
>> > > > > >> >>> > > >> >> >
>> > > > > >> >>> > > >> >>
>> > > > > >> >>> > > >>
>> > > > > >> >>> > >
>> > > > > >> >>> >
>> > > > > >> >>>
>> > > > > >>
>> > > > >
>> > > >
>> > >
>> >
>> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
>> > > > > >> >>> > > >> >> > > > >
>> > > > > >> >>> > > >> >> > > > >
>> > > > > >> >>> > > >>
>> > > > > >>
>> https://www.mail-archive.com/dev@.../msg02703.html
>> > > > > >> >>> > > >> >> > > > >
>> > > > > >> >>> > > >> >> > > > > I will share update as I hear more
>> > > information.
>> > > > > >> >>> > > >> >> > > > >
>> > > > > >> >>> > > >> >> > > > > Regards,
>> > > > > >> >>> > > >> >> > > > > Saikat
>> > > > > >> >>> > > >> >> > > > >
>> > > > > >> >>> > > >> >> > > > >
>> > > > > >> >>> > > >> >> > > > >
>> > > > > >> >>> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat
>> > Maitra
>> > > <
>> > > > > >> >>> > > >> >> > [hidden email]
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > > > wrote:
>> > > > > >> >>> > > >> >> > > > >
>> > > > > >> >>> > > >> >> > > > >> Hello Denis,
>> > > > > >> >>> > > >> >> > > > >>
>> > > > > >> >>> > > >> >> > > > >> Sure, sounds good. I will create a
>> separate
>> > > > > >> discussion
>> > > > > >> >>> > > thread
>> > > > > >> >>> > > >> to
>> > > > > >> >>> > > >> >> get
>> > > > > >> >>> > > >> >> > > > >> community feedback on whether we
>> > > > > >> >>> > > >> >> > > > >> should join the Bahir or kick-off
>> "Ignite
>> > > > > >> Extensions"
>> > > > > >> >>> > > project.
>> > > > > >> >>> > > >> >> > > > >>
>> > > > > >> >>> > > >> >> > > > >> Regards,
>> > > > > >> >>> > > >> >> > > > >> Saikat
>> > > > > >> >>> > > >> >> > > > >>
>> > > > > >> >>> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis
>> > Magda <
>> > > > > >> >>> > > >> [hidden email]>
>> > > > > >> >>> > > >> >> > > wrote:
>> > > > > >> >>> > > >> >> > > > >>
>> > > > > >> >>> > > >> >> > > > >>> Folks,
>> > > > > >> >>> > > >> >> > > > >>>
>> > > > > >> >>> > > >> >> > > > >>> The concept of Apache Bahir is
>> precisely
>> > > what
>> > > > we
>> > > > > >> >>> need!
>> > > > > >> >>> > > >> Saikat,
>> > > > > >> >>> > > >> >> > thanks
>> > > > > >> >>> > > >> >> > > > for
>> > > > > >> >>> > > >> >> > > > >>> doing the research. Why I believe the *
>> > > idea*
>> > > > > >> fits us
>> > > > > >> >>> > well:
>> > > > > >> >>> > > >> >> > > > >>>
>> > > > > >> >>> > > >> >> > > > >>>    - All integrations can be stored in
>> > > > separate
>> > > > > >> >>> Github
>> > > > > >> >>> > > >> >> repositories
>> > > > > >> >>> > > >> >> > > and
>> > > > > >> >>> > > >> >> > > > >>>    have their dev lifecycles. We've not
>> > > > obliged
>> > > > > to
>> > > > > >> >>> couple
>> > > > > >> >>> > > all
>> > > > > >> >>> > > >> >> the
>> > > > > >> >>> > > >> >> > > > >>> integrations
>> > > > > >> >>> > > >> >> > > > >>>    in a single repo. For instance,
>> Spark
>> > can
>> > > > be
>> > > > > >> >>> located
>> > > > > >> >>> > in
>> > > > > >> >>> > > a
>> > > > > >> >>> > > >> >> > > dedicated
>> > > > > >> >>> > > >> >> > > > >>> repo
>> > > > > >> >>> > > >> >> > > > >>>    while streaming integrations might
>> be
>> > in
>> > > a
>> > > > > >> single
>> > > > > >> >>> one.
>> > > > > >> >>> > > >> It's
>> > > > > >> >>> > > >> >> up
>> > > > > >> >>> > > >> >> > to
>> > > > > >> >>> > > >> >> > > > us.
>> > > > > >> >>> > > >> >> > > > >>>    - All the repositories and
>> integrations
>> > > > > belong
>> > > > > >> to
>> > > > > >> >>> ASF.
>> > > > > >> >>> > > >> We're
>> > > > > >> >>> > > >> >> not
>> > > > > >> >>> > > >> >> > > > >>> dumping
>> > > > > >> >>> > > >> >> > > > >>>    them on Github but rather keep
>> > supporting
>> > > > and
>> > > > > >> >>> > developing
>> > > > > >> >>> > > >> in
>> > > > > >> >>> > > >> >> > > > >>> accordance with
>> > > > > >> >>> > > >> >> > > > >>>    ASF vision and practices.
>> > > > > >> >>> > > >> >> > > > >>>    - There is a way to reward
>> contributors
>> > > via
>> > > > > >> >>> > > committership
>> > > > > >> >>> > > >> and
>> > > > > >> >>> > > >> >> > PMC
>> > > > > >> >>> > > >> >> > > > >>>    membership. You won't get this if a
>> > > project
>> > > > > is
>> > > > > >> >>> just
>> > > > > >> >>> > one
>> > > > > >> >>> > > of
>> > > > > >> >>> > > >> >> the
>> > > > > >> >>> > > >> >> > > > >>> millions of
>> > > > > >> >>> > > >> >> > > > >>>    Github projects.
>> > > > > >> >>> > > >> >> > > > >>>
>> > > > > >> >>> > > >> >> > > > >>> Saikat, as Ignite PMC member, could you
>> > > please
>> > > > > >> >>> kick-off a
>> > > > > >> >>> > > >> >> separate
>> > > > > >> >>> > > >> >> > > > >>> dev-list
>> > > > > >> >>> > > >> >> > > > >>> discussion to involve more community
>> > members
>> > > > and
>> > > > > >> >>> > referring
>> > > > > >> >>> > > to
>> > > > > >> >>> > > >> >> this
>> > > > > >> >>> > > >> >> > > > >>> thread?
>> > > > > >> >>> > > >> >> > > > >>> I think the primary question the
>> community
>> > > > needs
>> > > > > >> to
>> > > > > >> >>> > answer
>> > > > > >> >>> > > >> >> whether
>> > > > > >> >>> > > >> >> > we
>> > > > > >> >>> > > >> >> > > > >>> should join the Bahir or kick-off
>> "Ignite
>> > > > > >> Extensions"
>> > > > > >> >>> > > >> project?
>> > > > > >> >>> > > >> >> > > > >>>
>> > > > > >> >>> > > >> >> > > > >>> -
>> > > > > >> >>> > > >> >> > > > >>> Denis
>> > > > > >> >>> > > >> >> > > > >>>
>> > > > > >> >>> > > >> >> > > > >>>
>> > > > > >> >>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey
>> > > > Zinoviev
>> > > > > <
>> > > > > >> >>> > > >> >> > > > [hidden email]>
>> > > > > >> >>> > > >> >> > > > >>> wrote:
>> > > > > >> >>> > > >> >> > > > >>>
>> > > > > >> >>> > > >> >> > > > >>> > Maybe we could move all our Streaming
>> > > > > >> Integrations
>> > > > > >> >>> > there,
>> > > > > >> >>> > > >> but
>> > > > > >> >>> > > >> >> > what
>> > > > > >> >>> > > >> >> > > is
>> > > > > >> >>> > > >> >> > > > >>> about
>> > > > > >> >>> > > >> >> > > > >>> > maintaining and committer
>> permissions to
>> > > the
>> > > > > new
>> > > > > >> >>> > > >> repositories?
>> > > > > >> >>> > > >> >> > > > >>> > I see the list of the committers and
>> PMC
>> > > > > members
>> > > > > >> >>> there
>> > > > > >> >>> > > >> >> > > > >>> >
>> > > https://bahir.apache.org/community-members/
>> > > > > >> >>> > > >> >> > > > >>> >
>> > > > > >> >>> > > >> >> > > > >>> > Could somebody from Ignite community
>> be
>> > > > added
>> > > > > to
>> > > > > >> >>> this
>> > > > > >> >>> > > list
>> > > > > >> >>> > > >> as
>> > > > > >> >>> > > >> >> a
>> > > > > >> >>> > > >> >> > > > >>> > PMC/committer to maintain Ignite
>> > > > integrations?
>> > > > > >> >>> > > >> >> > > > >>> > If yes, I'd happy to join this
>> project
>> > and
>> > > > > >> >>> collaborate
>> > > > > >> >>> > > with
>> > > > > >> >>> > > >> >> these
>> > > > > >> >>> > > >> >> > > > guys
>> > > > > >> >>> > > >> >> > > > >>> > together
>> > > > > >> >>> > > >> >> > > > >>> > If no, and we should start from the
>> zero
>> > > > level
>> > > > > >> with
>> > > > > >> >>> > > >> external
>> > > > > >> >>> > > >> >> PRs
>> > > > > >> >>> > > >> >> > -
>> > > > > >> >>> > > >> >> > > > >>> hmmm,
>> > > > > >> >>> > > >> >> > > > >>> > it's better to have our own external
>> > > > > repository
>> > > > > >> >>> with
>> > > > > >> >>> > > >> >> ApacheBahirr
>> > > > > >> >>> > > >> >> > > > >>> ideology.
>> > > > > >> >>> > > >> >> > > > >>> >
>> > > > > >> >>> > > >> >> > > > >>> > Also, I agree, that all connectors
>> could
>> > > be
>> > > > > >> moved
>> > > > > >> >>> there
>> > > > > >> >>> > > (in
>> > > > > >> >>> > > >> >> > > > >>> ApacheBahirr
>> > > > > >> >>> > > >> >> > > > >>> > like repository), but not all modules
>> > from
>> > > > the
>> > > > > >> page
>> > > > > >> >>> > > >> >> > > > >>> >
>> > > > > >> >>> > > >> >> > > > >>> >
>> > > > > >> >>> > > >> >> > > > >>>
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > >
>> > > > > >> >>> > > >> >> >
>> > > > > >> >>> > > >> >>
>> > > > > >> >>> > > >>
>> > > > > >> >>> > >
>> > > > > >> >>> >
>> > > > > >> >>>
>> > > > > >>
>> > > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> > > > > >> >>> > > >> >> > > > >>> > because
>> > > > > >> >>> > > >> >> > > > >>> > they use different parts of core
>> modules
>> > > and
>> > > > > >> could
>> > > > > >> >>> be
>> > > > > >> >>> > > >> >> developed
>> > > > > >> >>> > > >> >> > > with
>> > > > > >> >>> > > >> >> > > > >>> > different velocity.
>> > > > > >> >>> > > >> >> > > > >>> >
>> > > > > >> >>> > > >> >> > > > >>> > In reality, before creation of new
>> > > > > repositories
>> > > > > >> we
>> > > > > >> >>> need
>> > > > > >> >>> > > >> wide
>> > > > > >> >>> > > >> >> > > > discussion
>> > > > > >> >>> > > >> >> > > > >>> > based on the document mentioned
>> above.
>> > > > > >> >>> > > >> >> > > > >>> >
>> > > > > >> >>> > > >> >> > > > >>> > P.S. Of course, we could experiment
>> in
>> > the
>> > > > > next
>> > > > > >> >>> release
>> > > > > >> >>> > > 2.8
>> > > > > >> >>> > > >> >> with
>> > > > > >> >>> > > >> >> > > one
>> > > > > >> >>> > > >> >> > > > >>> or two
>> > > > > >> >>> > > >> >> > > > >>> > integrations like twitter or ZeroMQ
>> > > > > >> >>> > > >> >> > > > >>> > Also, I'd like Denis Magda idea for
>> the
>> > > > > separate
>> > > > > >> >>> > > >> repositories
>> > > > > >> >>> > > >> >> in
>> > > > > >> >>> > > >> >> > > > Apache
>> > > > > >> >>> > > >> >> > > > >>> > Github to maintain them by anybody.
>> > > > > >> >>> > > >> >> > > > >>> >
>> > > > > >> >>> > > >> >> > > > >>> >
>> > > > > >> >>> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat
>> > > Maitra <
>> > > > > >> >>> > > >> >> > > [hidden email]
>> > > > > >> >>> > > >> >> > > > >:
>> > > > > >> >>> > > >> >> > > > >>> >
>> > > > > >> >>> > > >> >> > > > >>> > > Hi Denis, Emmanouil
>> > > > > >> >>> > > >> >> > > > >>> > >
>> > > > > >> >>> > > >> >> > > > >>> > > Thank you for your email. I think
>> > > > creating a
>> > > > > >> >>> separate
>> > > > > >> >>> > > >> >> > integration
>> > > > > >> >>> > > >> >> > > > >>> project
>> > > > > >> >>> > > >> >> > > > >>> > > in github and also proposing it as
>> an
>> > > > apache
>> > > > > >> >>> > incubator
>> > > > > >> >>> > > >> >> project
>> > > > > >> >>> > > >> >> > > will
>> > > > > >> >>> > > >> >> > > > >>> be a
>> > > > > >> >>> > > >> >> > > > >>> > > good move. The other separate
>> > > integration
>> > > > > >> modules
>> > > > > >> >>> > > >> >> > > > >>> > >
>> > > > > >> >>> > > >> >> > > > >>> > >
>> > > > > >> >>> > > >> >> > > > >>> >
>> > > > > >> >>> > > >> >> > > > >>>
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > >
>> > > > > >> >>> > > >> >> >
>> > > > > >> >>> > > >> >>
>> > > > > >> >>> > > >>
>> > > > > >> >>> > >
>> > > > > >> >>> >
>> > > > > >> >>>
>> > > > > >>
>> > > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> > > > > >> >>> > > >> >> > > > >>> > > can
>> > > > > >> >>> > > >> >> > > > >>> > > be moved to this new apache
>> incubator
>> > > > > project.
>> > > > > >> >>> > > >> >> > > > >>> > >
>> > > > > >> >>> > > >> >> > > > >>> > > There are similar integration
>> > available
>> > > > for
>> > > > > >> >>> Flink and
>> > > > > >> >>> > > >> Spark
>> > > > > >> >>> > > >> >> in
>> > > > > >> >>> > > >> >> > > > Apache
>> > > > > >> >>> > > >> >> > > > >>> > Bahir
>> > > > > >> >>> > > >> >> > > > >>> > > https://bahir.apache.org/
>> > > > > >> >>> > > >> >> > > > >>> > >
>> > > > > >> >>> > > >> >> > > > >>> > > Do you think we should reach out to
>> > > Apache
>> > > > > >> Bahir
>> > > > > >> >>> > > >> community
>> > > > > >> >>> > > >> >> > with a
>> > > > > >> >>> > > >> >> > > > >>> > proposal
>> > > > > >> >>> > > >> >> > > > >>> > > or we should plan to create a
>> separate
>> > > > > Apache
>> > > > > >> >>> > Incubator
>> > > > > >> >>> > > >> >> > project?
>> > > > > >> >>> > > >> >> > > > >>> > >
>> > > > > >> >>> > > >> >> > > > >>> > > Regards,
>> > > > > >> >>> > > >> >> > > > >>> > > Saikat
>> > > > > >> >>> > > >> >> > > > >>> > >
>> > > > > >> >>> > > >> >> > > > >>> > >
>> > > > > >> >>> > > >> >> > > > >>> > >
>> > > > > >> >>> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM
>> > > Emmanouil
>> > > > > >> >>> > Gkatziouras <
>> > > > > >> >>> > > >> >> > > > >>> > > [hidden email]>
>> > > > > >> >>> > > >> >> > > > >>> > > wrote:
>> > > > > >> >>> > > >> >> > > > >>> > >
>> > > > > >> >>> > > >> >> > > > >>> > > > Hi all!
>> > > > > >> >>> > > >> >> > > > >>> > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > Based on the discussions most
>> > probably
>> > > > > this
>> > > > > >> is
>> > > > > >> >>> > going
>> > > > > >> >>> > > to
>> > > > > >> >>> > > >> >> be on
>> > > > > >> >>> > > >> >> > > > >>> another
>> > > > > >> >>> > > >> >> > > > >>> > > > GitHub repo. Therefore I will
>> > prepare
>> > > a
>> > > > > >> >>> standalone
>> > > > > >> >>> > > >> project
>> > > > > >> >>> > > >> >> > with
>> > > > > >> >>> > > >> >> > > > the
>> > > > > >> >>> > > >> >> > > > >>> > > Pub/Sub
>> > > > > >> >>> > > >> >> > > > >>> > > > module and once the repository is
>> > > > created
>> > > > > a
>> > > > > >> >>> pull
>> > > > > >> >>> > > >> request
>> > > > > >> >>> > > >> >> > shall
>> > > > > >> >>> > > >> >> > > be
>> > > > > >> >>> > > >> >> > > > >>> > issued.
>> > > > > >> >>> > > >> >> > > > >>> > > > If there is anything else I can
>> do
>> > in
>> > > > > order
>> > > > > >> to
>> > > > > >> >>> > > >> facilitate
>> > > > > >> >>> > > >> >> > this
>> > > > > >> >>> > > >> >> > > > >>> please
>> > > > > >> >>> > > >> >> > > > >>> > let
>> > > > > >> >>> > > >> >> > > > >>> > > > me know.
>> > > > > >> >>> > > >> >> > > > >>> > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > If a ticket is created for this
>> > > issue, I
>> > > > > >> shall
>> > > > > >> >>> be
>> > > > > >> >>> > > able
>> > > > > >> >>> > > >> to
>> > > > > >> >>> > > >> >> > > create
>> > > > > >> >>> > > >> >> > > > a
>> > > > > >> >>> > > >> >> > > > >>> pull
>> > > > > >> >>> > > >> >> > > > >>> > > > request and thus a build will
>> take
>> > > place
>> > > > > on
>> > > > > >> >>> > ignite's
>> > > > > >> >>> > > >> CI.
>> > > > > >> >>> > > >> >> > > > >>> > > > If everyone is aligned with
>> adding a
>> > > > > Pub/Sub
>> > > > > >> >>> > > >> >> implementation
>> > > > > >> >>> > > >> >> > > may I
>> > > > > >> >>> > > >> >> > > > >>> > create
>> > > > > >> >>> > > >> >> > > > >>> > > a
>> > > > > >> >>> > > >> >> > > > >>> > > > ticket on JIRA?
>> > > > > >> >>> > > >> >> > > > >>> > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > My next question has to do on how
>> > > Ignite
>> > > > > >> >>> configures
>> > > > > >> >>> > > the
>> > > > > >> >>> > > >> >> build
>> > > > > >> >>> > > >> >> > > > jobs
>> > > > > >> >>> > > >> >> > > > >>> in
>> > > > > >> >>> > > >> >> > > > >>> > > Team
>> > > > > >> >>> > > >> >> > > > >>> > > > City. The unit tests for my
>> > > > implementation
>> > > > > >> >>> need a
>> > > > > >> >>> > > local
>> > > > > >> >>> > > >> >> > Pub/Sub
>> > > > > >> >>> > > >> >> > > > >>> server
>> > > > > >> >>> > > >> >> > > > >>> > up
>> > > > > >> >>> > > >> >> > > > >>> > > > and running (they provide one for
>> > > > testing
>> > > > > >> >>> > purposes).
>> > > > > >> >>> > > >> Will
>> > > > > >> >>> > > >> >> > this
>> > > > > >> >>> > > >> >> > > > take
>> > > > > >> >>> > > >> >> > > > >>> > > effect
>> > > > > >> >>> > > >> >> > > > >>> > > > in a form of a build script
>> inside
>> > my
>> > > > > >> >>> > implementation
>> > > > > >> >>> > > or
>> > > > > >> >>> > > >> >> is it
>> > > > > >> >>> > > >> >> > > > >>> something
>> > > > > >> >>> > > >> >> > > > >>> > > > that should be configured on Team
>> > > City?
>> > > > > >> >>> > > >> >> > > > >>> > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > Kind regards,
>> > > > > >> >>> > > >> >> > > > >>> > > > Emmanouil
>> > > > > >> >>> > > >> >> > > > >>> > > >
>> > > > > >> >>> > > >> >> > > > >>> > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
>> > > > > >> >>> > > >> >> > > > >>> > > > https://egkatzioura.com/ |
>> > > > > >> >>> > > >> >> > > > >>> > > >
>> > > > > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > > >> >>> > > >> >> > > > >>> > > > https://github.com/gkatzioura
>> > > > > >> >>> > > >> >> > > > >>> > > >
>> > > > > >> >>> > > >> >> > > > >>> > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27,
>> Denis
>> > > > Magda
>> > > > > <
>> > > > > >> >>> > > >> >> [hidden email]
>> > > > > >> >>> > > >> >> > >
>> > > > > >> >>> > > >> >> > > > >>> wrote:
>> > > > > >> >>> > > >> >> > > > >>> > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
>> > > > > >> >>> > > >> >> > > > >>> > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > After contemplating on this,
>> let
>> > me
>> > > > > >> propose
>> > > > > >> >>> > another
>> > > > > >> >>> > > >> way
>> > > > > >> >>> > > >> >> > with
>> > > > > >> >>> > > >> >> > > > this
>> > > > > >> >>> > > >> >> > > > >>> > > > > particular integration that is
>> > > aligned
>> > > > > >> with
>> > > > > >> >>> our
>> > > > > >> >>> > > >> >> > > modularization
>> > > > > >> >>> > > >> >> > > > >>> > > endeavor.
>> > > > > >> >>> > > >> >> > > > >>> > > > > The modularization [1] defines
>> the
>> > > > > Ignite
>> > > > > >> >>> core,
>> > > > > >> >>> > > >> Ignite
>> > > > > >> >>> > > >> >> > > modules,
>> > > > > >> >>> > > >> >> > > > >>> and
>> > > > > >> >>> > > >> >> > > > >>> > > > > independent integrations. The
>> > first
>> > > > two
>> > > > > >> are
>> > > > > >> >>> to be
>> > > > > >> >>> > > >> >> located
>> > > > > >> >>> > > >> >> > in
>> > > > > >> >>> > > >> >> > > > >>> Ignite
>> > > > > >> >>> > > >> >> > > > >>> > > > > repositories and to be
>> > > tested/updated
>> > > > by
>> > > > > >> the
>> > > > > >> >>> > > >> community
>> > > > > >> >>> > > >> >> for
>> > > > > >> >>> > > >> >> > > > every
>> > > > > >> >>> > > >> >> > > > >>> > > release.
>> > > > > >> >>> > > >> >> > > > >>> > > > > The modules are a subset of
>> > existing
>> > > > > >> >>> integrations
>> > > > > >> >>> > > >> that
>> > > > > >> >>> > > >> >> are
>> > > > > >> >>> > > >> >> > > > >>> adopted
>> > > > > >> >>> > > >> >> > > > >>> > > widely
>> > > > > >> >>> > > >> >> > > > >>> > > > > and require community
>> attention.
>> > > > > >> >>> > > >> >> > > > >>> > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > As for the independent
>> > integrations
>> > > > [1],
>> > > > > >> >>> those
>> > > > > >> >>> > are
>> > > > > >> >>> > > >> >> supposed
>> > > > > >> >>> > > >> >> > > to
>> > > > > >> >>> > > >> >> > > > be
>> > > > > >> >>> > > >> >> > > > >>> > > located
>> > > > > >> >>> > > >> >> > > > >>> > > > > in a separate GitHub repo. I
>> would
>> > > > > advise
>> > > > > >> us
>> > > > > >> >>> to
>> > > > > >> >>> > > start
>> > > > > >> >>> > > >> >> > > > >>> implementing
>> > > > > >> >>> > > >> >> > > > >>> > this
>> > > > > >> >>> > > >> >> > > > >>> > > > > concept with the Pub/Sub
>> > > integration.
>> > > > > Even
>> > > > > >> >>> though
>> > > > > >> >>> > > it
>> > > > > >> >>> > > >> >> will
>> > > > > >> >>> > > >> >> > be
>> > > > > >> >>> > > >> >> > > > >>> located
>> > > > > >> >>> > > >> >> > > > >>> > > in a
>> > > > > >> >>> > > >> >> > > > >>> > > > > separate Github repo, it will
>> be
>> > > > listed
>> > > > > >> among
>> > > > > >> >>> > > >> officially
>> > > > > >> >>> > > >> >> > > > >>> available
>> > > > > >> >>> > > >> >> > > > >>> > > > > integrations on the Ignite
>> website
>> > > > > >> featuring
>> > > > > >> >>> > > >> Emmanouil
>> > > > > >> >>> > > >> >> as a
>> > > > > >> >>> > > >> >> > > > >>> > > contributor.
>> > > > > >> >>> > > >> >> > > > >>> > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > What do you think?
>> > > > > >> >>> > > >> >> > > > >>> > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > [1]
>> > > > > >> >>> > > >> >> > > > >>> > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > >
>> > > > > >> >>> > > >> >> > > > >>> > >
>> > > > > >> >>> > > >> >> > > > >>> >
>> > > > > >> >>> > > >> >> > > > >>>
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > >
>> > > > > >> >>> > > >> >> >
>> > > > > >> >>> > > >> >>
>> > > > > >> >>> > > >>
>> > > > > >> >>> > >
>> > > > > >> >>> >
>> > > > > >> >>>
>> > > > > >>
>> > > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> > > > > >> >>> > > >> >> > > > >>> > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > -
>> > > > > >> >>> > > >> >> > > > >>> > > > > Denis
>> > > > > >> >>> > > >> >> > > > >>> > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM
>> > > Saikat
>> > > > > >> >>> Maitra <
>> > > > > >> >>> > > >> >> > > > >>> > [hidden email]
>> > > > > >> >>> > > >> >> > > > >>> > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > wrote:
>> > > > > >> >>> > > >> >> > > > >>> > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > > Hi Emmanouil,
>> > > > > >> >>> > > >> >> > > > >>> > > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > > The changes looks good to
>> me. I
>> > > > wanted
>> > > > > >> to
>> > > > > >> >>> check
>> > > > > >> >>> > > if
>> > > > > >> >>> > > >> you
>> > > > > >> >>> > > >> >> > are
>> > > > > >> >>> > > >> >> > > > also
>> > > > > >> >>> > > >> >> > > > >>> > able
>> > > > > >> >>> > > >> >> > > > >>> > > to
>> > > > > >> >>> > > >> >> > > > >>> > > > > > validate that the once data
>> is
>> > > added
>> > > > > to
>> > > > > >> >>> Ignite
>> > > > > >> >>> > > >> cluster
>> > > > > >> >>> > > >> >> > you
>> > > > > >> >>> > > >> >> > > > are
>> > > > > >> >>> > > >> >> > > > >>> also
>> > > > > >> >>> > > >> >> > > > >>> > > > able
>> > > > > >> >>> > > >> >> > > > >>> > > > > to
>> > > > > >> >>> > > >> >> > > > >>> > > > > > access it using another
>> Ignite
>> > > > client
>> > > > > or
>> > > > > >> >>> using
>> > > > > >> >>> > > rest
>> > > > > >> >>> > > >> >> > > > endpoints.
>> > > > > >> >>> > > >> >> > > > >>> > > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > > We use Teamcity for CI
>> process,
>> > > the
>> > > > > >> >>> details are
>> > > > > >> >>> > > as
>> > > > > >> >>> > > >> >> > follows
>> > > > > >> >>> > > >> >> > > > >>> > > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > > Trigger validation of those
>> test
>> > > > > suites
>> > > > > >> >>> that
>> > > > > >> >>> > have
>> > > > > >> >>> > > >> been
>> > > > > >> >>> > > >> >> > > > >>> affected by
>> > > > > >> >>> > > >> >> > > > >>> > > your
>> > > > > >> >>> > > >> >> > > > >>> > > > > > changes on TeamCity <
>> > > > > >> >>> > > http://ci.ignite.apache.org/>:
>> > > > > >> >>> > > >> <
>> > > > > >> >>> > > >> >> > > > >>> > > > > http://204.14.53.153/
>> > > > > >> >>> > > >> >> > > > >>> > > > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Locate a test suite you
>> > have
>> > > to
>> > > > > >> check,
>> > > > > >> >>> > press
>> > > > > >> >>> > > >> >> button
>> > > > > >> >>> > > >> >> > > > named
>> > > > > >> >>> > > >> >> > > > >>> > "..."
>> > > > > >> >>> > > >> >> > > > >>> > > > that
>> > > > > >> >>> > > >> >> > > > >>> > > > > >    is located on the left of
>> > "Run"
>> > > > > >> button.
>> > > > > >> >>> "Run
>> > > > > >> >>> > > >> custom
>> > > > > >> >>> > > >> >> > > build"
>> > > > > >> >>> > > >> >> > > > >>> > dialog
>> > > > > >> >>> > > >> >> > > > >>> > > > will
>> > > > > >> >>> > > >> >> > > > >>> > > > > >    appear;
>> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Go to "Changes" tab and
>> > > choose
>> > > > > >> >>> > > >> >> > > > >>> > "pull/<pull-request-number>/head"
>> > > > > >> >>> > > >> >> > > > >>> > > > in
>> > > > > >> >>> > > >> >> > > > >>> > > > > >    "Build branch" dropdown
>> list;
>> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Press "Run build" button
>> > and
>> > > > > >> monitor
>> > > > > >> >>> tests
>> > > > > >> >>> > > >> >> results.
>> > > > > >> >>> > > >> >> > > > >>> > > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > > @Roman [hidden email] -
>> can
>> > > you
>> > > > > also
>> > > > > >> >>> please
>> > > > > >> >>> > > >> take a
>> > > > > >> >>> > > >> >> > > look?
>> > > > > >> >>> > > >> >> > > > >>> > > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > > Regards,
>> > > > > >> >>> > > >> >> > > > >>> > > > > > Saikat
>> > > > > >> >>> > > >> >> > > > >>> > > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20
>> PM
>> > > > > >> Emmanouil
>> > > > > >> >>> > > >> Gkatziouras
>> > > > > >> >>> > > >> >> <
>> > > > > >> >>> > > >> >> > > > >>> > > > > > [hidden email]>
>> > > > > >> >>> > > >> >> > > > >>> > > > > > wrote:
>> > > > > >> >>> > > >> >> > > > >>> > > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
>> > > > > >> >>> > > >> >> > > > >>> > > > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > > > I have my implementation
>> on an
>> > > > > Ignite
>> > > > > >> >>> fork on
>> > > > > >> >>> > > my
>> > > > > >> >>> > > >> >> > personal
>> > > > > >> >>> > > >> >> > > > >>> GitHub
>> > > > > >> >>> > > >> >> > > > >>> > > > > account
>> > > > > >> >>> > > >> >> > > > >>> > > > > > > [1]
>> > > > > >> >>> > > >> >> > > > >>> > > > > > > It is based on the Kafka
>> > > > > >> implementation
>> > > > > >> >>> which
>> > > > > >> >>> > > is
>> > > > > >> >>> > > >> of
>> > > > > >> >>> > > >> >> a
>> > > > > >> >>> > > >> >> > > > polling
>> > > > > >> >>> > > >> >> > > > >>> > > nature.
>> > > > > >> >>> > > >> >> > > > >>> > > > > > > GCP does not provide a
>> service
>> > > > like
>> > > > > >> >>> Kafka.
>> > > > > >> >>> > > >> Instead
>> > > > > >> >>> > > >> >> it
>> > > > > >> >>> > > >> >> > > > >>> provides us
>> > > > > >> >>> > > >> >> > > > >>> > > > with
>> > > > > >> >>> > > >> >> > > > >>> > > > > > > Pub/Sub which is just like
>> > > Kafka.
>> > > > > >> >>> > > >> >> > > > >>> > > > > > > Therefore using the Kafka
>> > > streamer
>> > > > > as
>> > > > > >> a
>> > > > > >> >>> > > guideline
>> > > > > >> >>> > > >> >> made
>> > > > > >> >>> > > >> >> > > more
>> > > > > >> >>> > > >> >> > > > >>> sense
>> > > > > >> >>> > > >> >> > > > >>> > > to
>> > > > > >> >>> > > >> >> > > > >>> > > > > me.
>> > > > > >> >>> > > >> >> > > > >>> > > > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > > > Apart from the
>> implementation
>> > I
>> > > > have
>> > > > > >> also
>> > > > > >> >>> > > added a
>> > > > > >> >>> > > >> >> unit
>> > > > > >> >>> > > >> >> > > > test.
>> > > > > >> >>> > > >> >> > > > >>> > > > > > > It definitely needs some
>> > > polishing
>> > > > > >> and a
>> > > > > >> >>> > double
>> > > > > >> >>> > > >> >> check
>> > > > > >> >>> > > >> >> > on
>> > > > > >> >>> > > >> >> > > my
>> > > > > >> >>> > > >> >> > > > >>> side
>> > > > > >> >>> > > >> >> > > > >>> > to
>> > > > > >> >>> > > >> >> > > > >>> > > > > make
>> > > > > >> >>> > > >> >> > > > >>> > > > > > > the review smooth.
>> > > > > >> >>> > > >> >> > > > >>> > > > > > > For the tests a local
>> Pub/Sub
>> > > [2]
>> > > > > >> server
>> > > >
>
>

[Emmanouil Gkatziouras]

Hi Saikat!

From the logs[1] it seems that TC cannot find the project. However I did
include the project on the parent pom (modules sections) [2]
I tried to reproduce locally unfortunately I am not aware of the full
context on TC. It seems as if a different parent pom is being picked up.

Kind regards

[1]
https://ci.ignite.apache.org/viewLog.html?buildId=4804893&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=1704
[2]
https://github.com/gkatzioura/ignite-extensions/blob/d6a8beee7feff28f59d9a12be692016305564d25/pom.xml#L46

*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Sat, 30 Nov 2019 at 22:20, Saikat Maitra <[hidden email]> wrote:

> Hello,
>
> I have made changes in Ignite Extensions build configurations.
>
> It is now running the tests as expected.
>
>
> https://ci.ignite.apache.org/viewLog.html?buildId=4804480&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
>
> Regards,
> Saikat
>
> On Thu, Nov 28, 2019 at 1:29 PM Saikat Maitra <[hidden email]>
> wrote:
>
> > Hi,
> >
> > We do run build with -DskipTests and then configure selective test suites
> > to run.
> >
> > I will check why these test suites are not getting executed.
> >
> > Regards
> > Saikat
> >
> > On Wed, 27 Nov 2019 at 3:05 PM, Emmanouil Gkatziouras <
> > [hidden email]> wrote:
> >
> >> Hi all,
> >>
> >> I did add a test suite and run the build with the corresponding
> arguments,
> >> however the tests did not run [1].
> >> I checked the Flink's build logs and the same message is displayed there
> >> too `No tests to run` [2].
> >>
> >> [1]
> >>
> >>
> https://ci.ignite.apache.org/viewLog.html?buildId=4798203&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=248
> >> [2]
> >>
> >>
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4789199&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=1256
> >>
> >> Kind regards
> >> *Emmanouil Gkatziouras*
> >> https://egkatzioura.com/ |
> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> https://github.com/gkatzioura
> >>
> >>
> >> On Wed, 27 Nov 2019 at 18:58, Saikat Maitra <[hidden email]>
> >> wrote:
> >>
> >> > Hi,
> >> >
> >> > We will need to add test suites for tests to be executed in build. We
> >> pass
> >> > them as TestSuites param. I had added Flink sink and source
> testsuites.
> >> >
> >> > Can you please review and confirm.
> >> >
> >> > Regards
> >> > Saikat
> >> >
> >> > On Wed, 27 Nov 2019 at 6:00 AM, Emmanouil Gkatziouras <
> >> > [hidden email]>
> >> > wrote:
> >> >
> >> > > Hi all!
> >> > >
> >> > > I was successful building and running my pull requests. It seems
> that
> >> the
> >> > > tests do not run both for Flink and Pub/Sub [1]
> >> > > If there is something I cannot do to make them discoverable please
> >> let me
> >> > > know.
> >> > >
> >> > > [1]
> >> > >
> >> > >
> >> >
> >>
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4796722&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=283
> >> > >
> >> > > *Emmanouil Gkatziouras*
> >> > > https://egkatzioura.com/ |
> >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > https://github.com/gkatzioura
> >> > >
> >> > >
> >> > > On Wed, 27 Nov 2019 at 04:57, Saikat Maitra <
> [hidden email]>
> >> > > wrote:
> >> > >
> >> > > > Hi Emmanouil,
> >> > > >
> >> > > > I have added you as contributor in Ignite Extensions project. Can
> >> you
> >> > > > please check and confirm if you are able to see the project and
> >> execute
> >> > > > build on your pull request.
> >> > > >
> >> > > > Regards,
> >> > > > Saikat
> >> > > >
> >> > > > On Tue, Nov 26, 2019 at 12:49 PM Emmanouil Gkatziouras <
> >> > > > [hidden email]>
> >> > > > wrote:
> >> > > >
> >> > > > > Hi all!
> >> > > > >
> >> > > > > If someone could give me read access to the Job of Ignite
> >> Extensions
> >> > on
> >> > > > > Team City will greatly help me (username gkatzioura).
> >> > > > > I suppose the builds get triggered automatically on a pull
> >> request.
> >> > > > >
> >> > > > > Kind regards,
> >> > > > > Emmanouil
> >> > > > >
> >> > > > > *Emmanouil Gkatziouras*
> >> > > > > https://egkatzioura.com/ |
> >> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > > > https://github.com/gkatzioura
> >> > > > >
> >> > > > >
> >> > > > > On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras <
> >> > > > [hidden email]>
> >> > > > > wrote:
> >> > > > >
> >> > > > > > Hi Saikat!
> >> > > > > >
> >> > > > > > I just rebased with the flink branch. Unfortunately I do not
> >> have
> >> > > read
> >> > > > > > access to the team city link you provided and thus evaluate
> the
> >> > tests
> >> > > > for
> >> > > > > > my pull request.
> >> > > > > > I guess it has to do with ignite extensions being new.
> >> > > > > > My username is gkatzioura and email the one I am using on this
> >> > email
> >> > > > > >
> >> > > > > > Thank you for your time.
> >> > > > > >
> >> > > > > >
> >> > > > > > *Emmanouil Gkatziouras*
> >> > > > > > https://egkatzioura.com/ |
> >> > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > > > > https://github.com/gkatzioura
> >> > > > > >
> >> > > > > >
> >> > > > > > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <
> >> > [hidden email]
> >> > > >
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > >> Hi Emmanouil,
> >> > > > > >>
> >> > > > > >> The latest build on teamcity has passed on Flink pull
> request.
> >> > > > > >>
> >> > > > > >>
> >> > > > > >>
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> >> > > > > >>
> >> > > > > >> you should be able to take the changes I made in my PR and
> run
> >> > build
> >> > > > on
> >> > > > > >> your pull request.
> >> > > > > >>
> >> > > > > >> Please let me know if you have any questions.
> >> > > > > >>
> >> > > > > >> Regards,
> >> > > > > >> Saikat
> >> > > > > >>
> >> > > > > >> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <
> >> > > > [hidden email]
> >> > > > > >
> >> > > > > >> wrote:
> >> > > > > >>
> >> > > > > >> > Hi Emmanouil,
> >> > > > > >> >
> >> > > > > >> > I have fixed the Flink module testsuites and have setup new
> >> > > project
> >> > > > in
> >> > > > > >> > teamcity.
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >>
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
> >> > > > > >> >
> >> > > > > >> > I am looking into setting up the build.
> >> > > > > >> >
> >> > > > > >> > Regards,
> >> > > > > >> > Saikat
> >> > > > > >> >
> >> > > > > >> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <
> >> > > > > [hidden email]>
> >> > > > > >> > wrote:
> >> > > > > >> >
> >> > > > > >> >> Hi Emmanouil,
> >> > > > > >> >>
> >> > > > > >> >> Thank you for your email. Yes, the plan is once the Flink
> >> PR is
> >> > > > > merged
> >> > > > > >> it
> >> > > > > >> >> will provide the Licence, parent POMs etc and you can
> rebase
> >> > from
> >> > > > > >> master
> >> > > > > >> >> branch and apply the changes on top of it.
> >> > > > > >> >>
> >> > > > > >> >> I am getting some issues with test failures in local with
> >> > > > > >> >> GridTestProperties as the test.properties is not present
> in
> >> > this
> >> > > > new
> >> > > > > >> >> project but should be available from dependencies.
> >> > > > > >> >>
> >> > > > > >> >> Once I address this issue, I will go ahead and merge the
> >> > changes
> >> > > > and
> >> > > > > >> then
> >> > > > > >> >> we can take it from there.
> >> > > > > >> >>
> >> > > > > >> >> Regards,
> >> > > > > >> >> Saikat
> >> > > > > >> >>
> >> > > > > >> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
> >> > > > > >> >> [hidden email]> wrote:
> >> > > > > >> >>
> >> > > > > >> >>> Hi all,
> >> > > > > >> >>>
> >> > > > > >> >>> I made my first pull request [1]. Since this project is
> >> brand
> >> > > new
> >> > > > > (no
> >> > > > > >> >>> parent poms, licensing), I reckoned it was best to use
> >> > Saikat's
> >> > > > > >> branch on
> >> > > > > >> >>> Flink.
> >> > > > > >> >>> I suppose the Flink branch will be merged first. If not
> >> please
> >> > > > give
> >> > > > > me
> >> > > > > >> >>> guidelines on how I should proceed next.
> >> > > > > >> >>>
> >> > > > > >> >>> Kind regards
> >> > > > > >> >>> Emmanouil
> >> > > > > >> >>>
> >> > > > > >> >>> [1] https://github.com/apache/ignite-extensions/pull/2
> >> > > > > >> >>>
> >> > > > > >> >>> *Emmanouil Gkatziouras*
> >> > > > > >> >>> https://egkatzioura.com/ |
> >> > > > > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > > > >> >>> https://github.com/gkatzioura
> >> > > > > >> >>>
> >> > > > > >> >>>
> >> > > > > >> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <
> >> [hidden email]>
> >> > > > > wrote:
> >> > > > > >> >>>
> >> > > > > >> >>> > Awesome, ping us whenever you're ready!
> >> > > > > >> >>> >
> >> > > > > >> >>> > -
> >> > > > > >> >>> > Denis
> >> > > > > >> >>> >
> >> > > > > >> >>> >
> >> > > > > >> >>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras
> <
> >> > > > > >> >>> > [hidden email]>
> >> > > > > >> >>> > wrote:
> >> > > > > >> >>> >
> >> > > > > >> >>> > > Hi all!
> >> > > > > >> >>> > >
> >> > > > > >> >>> > > I am sorry for being late on that. I was trying to
> >> > refactor
> >> > > > the
> >> > > > > >> test
> >> > > > > >> >>> in
> >> > > > > >> >>> > > order not to be in need of any external tools or
> >> spinning
> >> > > up a
> >> > > > > >> >>> server.
> >> > > > > >> >>> > > I did forked the new repo and indeed my changes
> there,
> >> so
> >> > a
> >> > > > pull
> >> > > > > >> >>> request
> >> > > > > >> >>> > is
> >> > > > > >> >>> > > a matter of time!
> >> > > > > >> >>> > >
> >> > > > > >> >>> > > Kind regards
> >> > > > > >> >>> > >
> >> > > > > >> >>> > > *Emmanouil Gkatziouras*
> >> > > > > >> >>> > > https://egkatzioura.com/ |
> >> > > > > >> >>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > > > >> >>> > > https://github.com/gkatzioura
> >> > > > > >> >>> > >
> >> > > > > >> >>> > >
> >> > > > > >> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <
> >> > > [hidden email]>
> >> > > > > >> wrote:
> >> > > > > >> >>> > >
> >> > > > > >> >>> > > > Hi Emmanouil,
> >> > > > > >> >>> > > >
> >> > > > > >> >>> > > > Do you have any questions or need any support from
> >> the
> >> > > > > >> community?
> >> > > > > >> >>> > > >
> >> > > > > >> >>> > > > -
> >> > > > > >> >>> > > > Denis
> >> > > > > >> >>> > > >
> >> > > > > >> >>> > > >
> >> > > > > >> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
> >> > > > > >> >>> [hidden email]
> >> > > > > >> >>> > >
> >> > > > > >> >>> > > > wrote:
> >> > > > > >> >>> > > >
> >> > > > > >> >>> > > >> Hi Emmanouil,
> >> > > > > >> >>> > > >>
> >> > > > > >> >>> > > >> Can you please take a looks at dev utils, if this
> is
> >> > > > > something
> >> > > > > >> >>> you are
> >> > > > > >> >>> > > >> looking for
> >> > > > > >> >>> > > >>
> >> > > > > https://github.com/apache/ignite/tree/master/modules/dev-utils
> >> > > > > >> ?
> >> > > > > >> >>> > > >>
> >> > > > > >> >>> > > >> IMO, if you can release Pub/Sub server in maven
> and
> >> > then
> >> > > > use
> >> > > > > >> it as
> >> > > > > >> >>> > > >> dependency, that would be great.
> >> > > > > >> >>> > > >>
> >> > > > > >> >>> > > >> Regards,
> >> > > > > >> >>> > > >> Saikat
> >> > > > > >> >>> > > >>
> >> > > > > >> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
> >> > > > > >> >>> > [hidden email]>
> >> > > > > >> >>> > > >> wrote:
> >> > > > > >> >>> > > >>
> >> > > > > >> >>> > > >> > Hi Emmanouil,
> >> > > > > >> >>> > > >> >
> >> > > > > >> >>> > > >> > The master branch has been initialized. I have
> >> > raised a
> >> > > > PR
> >> > > > > to
> >> > > > > >> >>> > migrate
> >> > > > > >> >>> > > >> > Flink module to ignite-extensions.
> >> > > > > >> >>> > > >> >
> >> > > > > >> >>> > > >> > PR
> >> > https://github.com/apache/ignite-extensions/pull/1
> >> > > > > >> >>> > > >> > Jira :
> >> > > > https://issues.apache.org/jira/browse/IGNITE-12356
> >> > > > > >> >>> > > >> >
> >> > > > > >> >>> > > >> > Regards,
> >> > > > > >> >>> > > >> > Saikat
> >> > > > > >> >>> > > >> >
> >> > > > > >> >>> > > >> >
> >> > > > > >> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil
> >> > Gkatziouras <
> >> > > > > >> >>> > > >> > [hidden email]> wrote:
> >> > > > > >> >>> > > >> >
> >> > > > > >> >>> > > >> >> Hi all!
> >> > > > > >> >>> > > >> >>
> >> > > > > >> >>> > > >> >> This is an update. The integration is tested
> with
> >> > > mocks
> >> > > > > >> based
> >> > > > > >> >>> on
> >> > > > > >> >>> > the
> >> > > > > >> >>> > > >> >> original Ignite repo [1].
> >> > > > > >> >>> > > >> >> I am currently waiting for the first commits on
> >> the
> >> > > new
> >> > > > > >> >>> integration
> >> > > > > >> >>> > > >> >> project
> >> > > > > >> >>> > > >> >> to take place in order to create my pull
> request
> >> > > > > accordingly
> >> > > > > >> >>> to the
> >> > > > > >> >>> > > >> >> initial
> >> > > > > >> >>> > > >> >> commits.
> >> > > > > >> >>> > > >> >> In order to simulate the Pub/Sub server in a
> >> test I
> >> > > > used a
> >> > > > > >> test
> >> > > > > >> >>> > > server
> >> > > > > >> >>> > > >> >> implementation [2]. Unfortunately they don't
> have
> >> > this
> >> > > > > util
> >> > > > > >> >>> > uploaded
> >> > > > > >> >>> > > >> in a
> >> > > > > >> >>> > > >> >> maven repo, thus it needs to be hosted on a
> maven
> >> > repo
> >> > > > > >> (Apache
> >> > > > > >> >>> > > License
> >> > > > > >> >>> > > >> >> 2.0). Is there a repo for utilities that Ignite
> >> uses
> >> > > for
> >> > > > > the
> >> > > > > >> >>> > > >> test/build?
> >> > > > > >> >>> > > >> >> I have also created a ticket [3]
> >> > > > > >> >>> > > >> >>
> >> > > > > >> >>> > > >> >> Kind regards
> >> > > > > >> >>> > > >> >>
> >> > > > > >> >>> > > >> >> [1]
> >> > > > > >> >>> >
> >> > > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> >> > > > > >> >>> > > >> >> [2]
> >> > > > > >> >>>
> >> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> >> > > > > >> >>> > > >> >> [3]
> >> > > https://issues.apache.org/jira/browse/IGNITE-12262
> >> > > > > >> >>> > > >> >>
> >> > > > > >> >>> > > >> >> *Emmanouil Gkatziouras*
> >> > > > > >> >>> > > >> >> https://egkatzioura.com/ |
> >> > > > > >> >>> > > >> >>
> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > > > >> >>> > > >> >> https://github.com/gkatzioura
> >> > > > > >> >>> > > >> >>
> >> > > > > >> >>> > > >> >>
> >> > > > > >> >>> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
> >> > > > > >> >>> > [hidden email]
> >> > > > > >> >>> > > >
> >> > > > > >> >>> > > >> >> wrote:
> >> > > > > >> >>> > > >> >>
> >> > > > > >> >>> > > >> >> > Hello Ilya,
> >> > > > > >> >>> > > >> >> >
> >> > > > > >> >>> > > >> >> > Thank you for your email. Yes, before we
> remove
> >> > any
> >> > > > > >> >>> integration
> >> > > > > >> >>> > we
> >> > > > > >> >>> > > >> will
> >> > > > > >> >>> > > >> >> > move them to another repository and release.
> >> > > > > >> >>> > > >> >> >
> >> > > > > >> >>> > > >> >> > We are thinking we will start with Pub/Sub
> >> > Streamer
> >> > > > > >> >>> > implementation
> >> > > > > >> >>> > > >> and
> >> > > > > >> >>> > > >> >> > continue to move other integration as
> >> mentioned in
> >> > > the
> >> > > > > >> >>> following
> >> > > > > >> >>> > > doc.
> >> > > > > >> >>> > > >> >> >
> >> > > > > >> >>> > > >> >> >
> >> > > > > >> >>> > > >> >> >
> >> > > > > >> >>> > > >> >>
> >> > > > > >> >>> > > >>
> >> > > > > >> >>> > >
> >> > > > > >> >>> >
> >> > > > > >> >>>
> >> > > > > >>
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> > > > > >> >>> > > >> >> >
> >> > > > > >> >>> > > >> >> > Regards,
> >> > > > > >> >>> > > >> >> > Saikat
> >> > > > > >> >>> > > >> >> >
> >> > > > > >> >>> > > >> >> >
> >> > > > > >> >>> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya
> >> Kasnacheev <
> >> > > > > >> >>> > > >> >> > [hidden email]>
> >> > > > > >> >>> > > >> >> > wrote:
> >> > > > > >> >>> > > >> >> >
> >> > > > > >> >>> > > >> >> > > Hello!
> >> > > > > >> >>> > > >> >> > >
> >> > > > > >> >>> > > >> >> > > My take on this: we want to offload some of
> >> > > > > >> integrations,
> >> > > > > >> >>> but
> >> > > > > >> >>> > > >> please
> >> > > > > >> >>> > > >> >> > commit
> >> > > > > >> >>> > > >> >> > > and release them to another project first,
> >> and
> >> > > only
> >> > > > > >> then we
> >> > > > > >> >>> > will
> >> > > > > >> >>> > > >> >> decide
> >> > > > > >> >>> > > >> >> > to
> >> > > > > >> >>> > > >> >> > > remove them from our repository.
> >> > > > > >> >>> > > >> >> > >
> >> > > > > >> >>> > > >> >> > > Please also make sure they are available
> with
> >> > same
> >> > > > > >> artifact
> >> > > > > >> >>> > name
> >> > > > > >> >>> > > >> and
> >> > > > > >> >>> > > >> >> API.
> >> > > > > >> >>> > > >> >> > > If this has to be changed, let's wait for
> AI
> >> 3.0
> >> > > > > >> >>> > > >> >> > >
> >> > > > > >> >>> > > >> >> > > Regards,
> >> > > > > >> >>> > > >> >> > > --
> >> > > > > >> >>> > > >> >> > > Ilya Kasnacheev
> >> > > > > >> >>> > > >> >> > >
> >> > > > > >> >>> > > >> >> > >
> >> > > > > >> >>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra
> <
> >> > > > > >> >>> > > >> [hidden email]>:
> >> > > > > >> >>> > > >> >> > >
> >> > > > > >> >>> > > >> >> > > > Hello Emmanouil,
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > > As discussed earlier, I discussed with
> >> Apache
> >> > > > Bahir
> >> > > > > >> >>> community
> >> > > > > >> >>> > > and
> >> > > > > >> >>> > > >> >> they
> >> > > > > >> >>> > > >> >> > > are
> >> > > > > >> >>> > > >> >> > > > interested to have Apache Ignite
> >> extensions as
> >> > > > part
> >> > > > > of
> >> > > > > >> >>> Apache
> >> > > > > >> >>> > > >> Bahir
> >> > > > > >> >>> > > >> >> > > > project.
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > > Can you please share Pub/Sub streamer
> >> > > > implementation
> >> > > > > >> >>> details
> >> > > > > >> >>> > > with
> >> > > > > >> >>> > > >> >> > Apache
> >> > > > > >> >>> > > >> >> > > > Bahir community and request for feedback.
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > > https://bahir.apache.org/
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > > I have also requested for contributor
> >> access
> >> > in
> >> > > > Jira
> >> > > > > >> for
> >> > > > > >> >>> > Apache
> >> > > > > >> >>> > > >> >> Bahir
> >> > > > > >> >>> > > >> >> > > > project so that I can create issues and
> >> assign
> >> > > to
> >> > > > > >> myself.
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > > I can help with code reviews as well.
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > > Here is the email thread for reference
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > >
> >> > > > https://www.mail-archive.com/dev@.../msg02703.html
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > > Regards,
> >> > > > > >> >>> > > >> >> > > > Saikat
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat
> >> Maitra
> >> > <
> >> > > > > >> >>> > > >> >> [hidden email]
> >> > > > > >> >>> > > >> >> > >
> >> > > > > >> >>> > > >> >> > > > wrote:
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > > > Hello,
> >> > > > > >> >>> > > >> >> > > > >
> >> > > > > >> >>> > > >> >> > > > > As discussed I have created following
> >> > > discussion
> >> > > > > >> >>> threads on
> >> > > > > >> >>> > > our
> >> > > > > >> >>> > > >> >> > > migration
> >> > > > > >> >>> > > >> >> > > > > proposals for Apache Ignite extensions:
> >> > > > > >> >>> > > >> >> > > > >
> >> > > > > >> >>> > > >> >> > > > >
> >> > > > > >> >>> > > >> >> > > > >
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > >
> >> > > > > >> >>> > > >> >> >
> >> > > > > >> >>> > > >> >>
> >> > > > > >> >>> > > >>
> >> > > > > >> >>> > >
> >> > > > > >> >>> >
> >> > > > > >> >>>
> >> > > > > >>
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> >> > > > > >> >>> > > >> >> > > > >
> >> > > > > >> >>> > > >> >> > > > >
> >> > > > > >> >>> > > >>
> >> > > > > >>
> >> https://www.mail-archive.com/dev@.../msg02703.html
> >> > > > > >> >>> > > >> >> > > > >
> >> > > > > >> >>> > > >> >> > > > > I will share update as I hear more
> >> > > information.
> >> > > > > >> >>> > > >> >> > > > >
> >> > > > > >> >>> > > >> >> > > > > Regards,
> >> > > > > >> >>> > > >> >> > > > > Saikat
> >> > > > > >> >>> > > >> >> > > > >
> >> > > > > >> >>> > > >> >> > > > >
> >> > > > > >> >>> > > >> >> > > > >
> >> > > > > >> >>> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat
> >> > Maitra
> >> > > <
> >> > > > > >> >>> > > >> >> > [hidden email]
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > > > wrote:
> >> > > > > >> >>> > > >> >> > > > >
> >> > > > > >> >>> > > >> >> > > > >> Hello Denis,
> >> > > > > >> >>> > > >> >> > > > >>
> >> > > > > >> >>> > > >> >> > > > >> Sure, sounds good. I will create a
> >> separate
> >> > > > > >> discussion
> >> > > > > >> >>> > > thread
> >> > > > > >> >>> > > >> to
> >> > > > > >> >>> > > >> >> get
> >> > > > > >> >>> > > >> >> > > > >> community feedback on whether we
> >> > > > > >> >>> > > >> >> > > > >> should join the Bahir or kick-off
> >> "Ignite
> >> > > > > >> Extensions"
> >> > > > > >> >>> > > project.
> >> > > > > >> >>> > > >> >> > > > >>
> >> > > > > >> >>> > > >> >> > > > >> Regards,
> >> > > > > >> >>> > > >> >> > > > >> Saikat
> >> > > > > >> >>> > > >> >> > > > >>
> >> > > > > >> >>> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis
> >> > Magda <
> >> > > > > >> >>> > > >> [hidden email]>
> >> > > > > >> >>> > > >> >> > > wrote:
> >> > > > > >> >>> > > >> >> > > > >>
> >> > > > > >> >>> > > >> >> > > > >>> Folks,
> >> > > > > >> >>> > > >> >> > > > >>>
> >> > > > > >> >>> > > >> >> > > > >>> The concept of Apache Bahir is
> >> precisely
> >> > > what
> >> > > > we
> >> > > > > >> >>> need!
> >> > > > > >> >>> > > >> Saikat,
> >> > > > > >> >>> > > >> >> > thanks
> >> > > > > >> >>> > > >> >> > > > for
> >> > > > > >> >>> > > >> >> > > > >>> doing the research. Why I believe
> the *
> >> > > idea*
> >> > > > > >> fits us
> >> > > > > >> >>> > well:
> >> > > > > >> >>> > > >> >> > > > >>>
> >> > > > > >> >>> > > >> >> > > > >>>    - All integrations can be stored
> in
> >> > > > separate
> >> > > > > >> >>> Github
> >> > > > > >> >>> > > >> >> repositories
> >> > > > > >> >>> > > >> >> > > and
> >> > > > > >> >>> > > >> >> > > > >>>    have their dev lifecycles. We've
> not
> >> > > > obliged
> >> > > > > to
> >> > > > > >> >>> couple
> >> > > > > >> >>> > > all
> >> > > > > >> >>> > > >> >> the
> >> > > > > >> >>> > > >> >> > > > >>> integrations
> >> > > > > >> >>> > > >> >> > > > >>>    in a single repo. For instance,
> >> Spark
> >> > can
> >> > > > be
> >> > > > > >> >>> located
> >> > > > > >> >>> > in
> >> > > > > >> >>> > > a
> >> > > > > >> >>> > > >> >> > > dedicated
> >> > > > > >> >>> > > >> >> > > > >>> repo
> >> > > > > >> >>> > > >> >> > > > >>>    while streaming integrations might
> >> be
> >> > in
> >> > > a
> >> > > > > >> single
> >> > > > > >> >>> one.
> >> > > > > >> >>> > > >> It's
> >> > > > > >> >>> > > >> >> up
> >> > > > > >> >>> > > >> >> > to
> >> > > > > >> >>> > > >> >> > > > us.
> >> > > > > >> >>> > > >> >> > > > >>>    - All the repositories and
> >> integrations
> >> > > > > belong
> >> > > > > >> to
> >> > > > > >> >>> ASF.
> >> > > > > >> >>> > > >> We're
> >> > > > > >> >>> > > >> >> not
> >> > > > > >> >>> > > >> >> > > > >>> dumping
> >> > > > > >> >>> > > >> >> > > > >>>    them on Github but rather keep
> >> > supporting
> >> > > > and
> >> > > > > >> >>> > developing
> >> > > > > >> >>> > > >> in
> >> > > > > >> >>> > > >> >> > > > >>> accordance with
> >> > > > > >> >>> > > >> >> > > > >>>    ASF vision and practices.
> >> > > > > >> >>> > > >> >> > > > >>>    - There is a way to reward
> >> contributors
> >> > > via
> >> > > > > >> >>> > > committership
> >> > > > > >> >>> > > >> and
> >> > > > > >> >>> > > >> >> > PMC
> >> > > > > >> >>> > > >> >> > > > >>>    membership. You won't get this if
> a
> >> > > project
> >> > > > > is
> >> > > > > >> >>> just
> >> > > > > >> >>> > one
> >> > > > > >> >>> > > of
> >> > > > > >> >>> > > >> >> the
> >> > > > > >> >>> > > >> >> > > > >>> millions of
> >> > > > > >> >>> > > >> >> > > > >>>    Github projects.
> >> > > > > >> >>> > > >> >> > > > >>>
> >> > > > > >> >>> > > >> >> > > > >>> Saikat, as Ignite PMC member, could
> you
> >> > > please
> >> > > > > >> >>> kick-off a
> >> > > > > >> >>> > > >> >> separate
> >> > > > > >> >>> > > >> >> > > > >>> dev-list
> >> > > > > >> >>> > > >> >> > > > >>> discussion to involve more community
> >> > members
> >> > > > and
> >> > > > > >> >>> > referring
> >> > > > > >> >>> > > to
> >> > > > > >> >>> > > >> >> this
> >> > > > > >> >>> > > >> >> > > > >>> thread?
> >> > > > > >> >>> > > >> >> > > > >>> I think the primary question the
> >> community
> >> > > > needs
> >> > > > > >> to
> >> > > > > >> >>> > answer
> >> > > > > >> >>> > > >> >> whether
> >> > > > > >> >>> > > >> >> > we
> >> > > > > >> >>> > > >> >> > > > >>> should join the Bahir or kick-off
> >> "Ignite
> >> > > > > >> Extensions"
> >> > > > > >> >>> > > >> project?
> >> > > > > >> >>> > > >> >> > > > >>>
> >> > > > > >> >>> > > >> >> > > > >>> -
> >> > > > > >> >>> > > >> >> > > > >>> Denis
> >> > > > > >> >>> > > >> >> > > > >>>
> >> > > > > >> >>> > > >> >> > > > >>>
> >> > > > > >> >>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM
> Alexey
> >> > > > Zinoviev
> >> > > > > <
> >> > > > > >> >>> > > >> >> > > > [hidden email]>
> >> > > > > >> >>> > > >> >> > > > >>> wrote:
> >> > > > > >> >>> > > >> >> > > > >>>
> >> > > > > >> >>> > > >> >> > > > >>> > Maybe we could move all our
> Streaming
> >> > > > > >> Integrations
> >> > > > > >> >>> > there,
> >> > > > > >> >>> > > >> but
> >> > > > > >> >>> > > >> >> > what
> >> > > > > >> >>> > > >> >> > > is
> >> > > > > >> >>> > > >> >> > > > >>> about
> >> > > > > >> >>> > > >> >> > > > >>> > maintaining and committer
> >> permissions to
> >> > > the
> >> > > > > new
> >> > > > > >> >>> > > >> repositories?
> >> > > > > >> >>> > > >> >> > > > >>> > I see the list of the committers
> and
> >> PMC
> >> > > > > members
> >> > > > > >> >>> there
> >> > > > > >> >>> > > >> >> > > > >>> >
> >> > > https://bahir.apache.org/community-members/
> >> > > > > >> >>> > > >> >> > > > >>> >
> >> > > > > >> >>> > > >> >> > > > >>> > Could somebody from Ignite
> community
> >> be
> >> > > > added
> >> > > > > to
> >> > > > > >> >>> this
> >> > > > > >> >>> > > list
> >> > > > > >> >>> > > >> as
> >> > > > > >> >>> > > >> >> a
> >> > > > > >> >>> > > >> >> > > > >>> > PMC/committer to maintain Ignite
> >> > > > integrations?
> >> > > > > >> >>> > > >> >> > > > >>> > If yes, I'd happy to join this
> >> project
> >> > and
> >> > > > > >> >>> collaborate
> >> > > > > >> >>> > > with
> >> > > > > >> >>> > > >> >> these
> >> > > > > >> >>> > > >> >> > > > guys
> >> > > > > >> >>> > > >> >> > > > >>> > together
> >> > > > > >> >>> > > >> >> > > > >>> > If no, and we should start from the
> >> zero
> >> > > > level
> >> > > > > >> with
> >> > > > > >> >>> > > >> external
> >> > > > > >> >>> > > >> >> PRs
> >> > > > > >> >>> > > >> >> > -
> >> > > > > >> >>> > > >> >> > > > >>> hmmm,
> >> > > > > >> >>> > > >> >> > > > >>> > it's better to have our own
> external
> >> > > > > repository
> >> > > > > >> >>> with
> >> > > > > >> >>> > > >> >> ApacheBahirr
> >> > > > > >> >>> > > >> >> > > > >>> ideology.
> >> > > > > >> >>> > > >> >> > > > >>> >
> >> > > > > >> >>> > > >> >> > > > >>> > Also, I agree, that all connectors
> >> could
> >> > > be
> >> > > > > >> moved
> >> > > > > >> >>> there
> >> > > > > >> >>> > > (in
> >> > > > > >> >>> > > >> >> > > > >>> ApacheBahirr
> >> > > > > >> >>> > > >> >> > > > >>> > like repository), but not all
> modules
> >> > from
> >> > > > the
> >> > > > > >> page
> >> > > > > >> >>> > > >> >> > > > >>> >
> >> > > > > >> >>> > > >> >> > > > >>> >
> >> > > > > >> >>> > > >> >> > > > >>>
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > >
> >> > > > > >> >>> > > >> >> >
> >> > > > > >> >>> > > >> >>
> >> > > > > >> >>> > > >>
> >> > > > > >> >>> > >
> >> > > > > >> >>> >
> >> > > > > >> >>>
> >> > > > > >>
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> > > > > >> >>> > > >> >> > > > >>> > because
> >> > > > > >> >>> > > >> >> > > > >>> > they use different parts of core
> >> modules
> >> > > and
> >> > > > > >> could
> >> > > > > >> >>> be
> >> > > > > >> >>> > > >> >> developed
> >> > > > > >> >>> > > >> >> > > with
> >> > > > > >> >>> > > >> >> > > > >>> > different velocity.
> >> > > > > >> >>> > > >> >> > > > >>> >
> >> > > > > >> >>> > > >> >> > > > >>> > In reality, before creation of new
> >> > > > > repositories
> >> > > > > >> we
> >> > > > > >> >>> need
> >> > > > > >> >>> > > >> wide
> >> > > > > >> >>> > > >> >> > > > discussion
> >> > > > > >> >>> > > >> >> > > > >>> > based on the document mentioned
> >> above.
> >> > > > > >> >>> > > >> >> > > > >>> >
> >> > > > > >> >>> > > >> >> > > > >>> > P.S. Of course, we could experiment
> >> in
> >> > the
> >> > > > > next
> >> > > > > >> >>> release
> >> > > > > >> >>> > > 2.8
> >> > > > > >> >>> > > >> >> with
> >> > > > > >> >>> > > >> >> > > one
> >> > > > > >> >>> > > >> >> > > > >>> or two
> >> > > > > >> >>> > > >> >> > > > >>> > integrations like twitter or ZeroMQ
> >> > > > > >> >>> > > >> >> > > > >>> > Also, I'd like Denis Magda idea for
> >> the
> >> > > > > separate
> >> > > > > >> >>> > > >> repositories
> >> > > > > >> >>> > > >> >> in
> >> > > > > >> >>> > > >> >> > > > Apache
> >> > > > > >> >>> > > >> >> > > > >>> > Github to maintain them by anybody.
> >> > > > > >> >>> > > >> >> > > > >>> >
> >> > > > > >> >>> > > >> >> > > > >>> >
> >> > > > > >> >>> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat
> >> > > Maitra <
> >> > > > > >> >>> > > >> >> > > [hidden email]
> >> > > > > >> >>> > > >> >> > > > >:
> >> > > > > >> >>> > > >> >> > > > >>> >
> >> > > > > >> >>> > > >> >> > > > >>> > > Hi Denis, Emmanouil
> >> > > > > >> >>> > > >> >> > > > >>> > >
> >> > > > > >> >>> > > >> >> > > > >>> > > Thank you for your email. I think
> >> > > > creating a
> >> > > > > >> >>> separate
> >> > > > > >> >>> > > >> >> > integration
> >> > > > > >> >>> > > >> >> > > > >>> project
> >> > > > > >> >>> > > >> >> > > > >>> > > in github and also proposing it
> as
> >> an
> >> > > > apache
> >> > > > > >> >>> > incubator
> >> > > > > >> >>> > > >> >> project
> >> > > > > >> >>> > > >> >> > > will
> >> > > > > >> >>> > > >> >> > > > >>> be a
> >> > > > > >> >>> > > >> >> > > > >>> > > good move. The other separate
> >> > > integration
> >> > > > > >> modules
> >> > > > > >> >>> > > >> >> > > > >>> > >
> >> > > > > >> >>> > > >> >> > > > >>> > >
> >> > > > > >> >>> > > >> >> > > > >>> >
> >> > > > > >> >>> > > >> >> > > > >>>
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > >
> >> > > > > >> >>> > > >> >> >
> >> > > > > >> >>> > > >> >>
> >> > > > > >> >>> > > >>
> >> > > > > >> >>> > >
> >> > > > > >> >>> >
> >> > > > > >> >>>
> >> > > > > >>
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> > > > > >> >>> > > >> >> > > > >>> > > can
> >> > > > > >> >>> > > >> >> > > > >>> > > be moved to this new apache
> >> incubator
> >> > > > > project.
> >> > > > > >> >>> > > >> >> > > > >>> > >
> >> > > > > >> >>> > > >> >> > > > >>> > > There are similar integration
> >> > available
> >> > > > for
> >> > > > > >> >>> Flink and
> >> > > > > >> >>> > > >> Spark
> >> > > > > >> >>> > > >> >> in
> >> > > > > >> >>> > > >> >> > > > Apache
> >> > > > > >> >>> > > >> >> > > > >>> > Bahir
> >> > > > > >> >>> > > >> >> > > > >>> > > https://bahir.apache.org/
> >> > > > > >> >>> > > >> >> > > > >>> > >
> >> > > > > >> >>> > > >> >> > > > >>> > > Do you think we should reach out
> to
> >> > > Apache
> >> > > > > >> Bahir
> >> > > > > >> >>> > > >> community
> >> > > > > >> >>> > > >> >> > with a
> >> > > > > >> >>> > > >> >> > > > >>> > proposal
> >> > > > > >> >>> > > >> >> > > > >>> > > or we should plan to create a
> >> separate
> >> > > > > Apache
> >> > > > > >> >>> > Incubator
> >> > > > > >> >>> > > >> >> > project?
> >> > > > > >> >>> > > >> >> > > > >>> > >
> >> > > > > >> >>> > > >> >> > > > >>> > > Regards,
> >> > > > > >> >>> > > >> >> > > > >>> > > Saikat
> >> > > > > >> >>> > > >> >> > > > >>> > >
> >> > > > > >> >>> > > >> >> > > > >>> > >
> >> > > > > >> >>> > > >> >> > > > >>> > >
> >> > > > > >> >>> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM
> >> > > Emmanouil
> >> > > > > >> >>> > Gkatziouras <
> >> > > > > >> >>> > > >> >> > > > >>> > > [hidden email]>
> >> > > > > >> >>> > > >> >> > > > >>> > > wrote:
> >> > > > > >> >>> > > >> >> > > > >>> > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > Hi all!
> >> > > > > >> >>> > > >> >> > > > >>> > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > Based on the discussions most
> >> > probably
> >> > > > > this
> >> > > > > >> is
> >> > > > > >> >>> > going
> >> > > > > >> >>> > > to
> >> > > > > >> >>> > > >> >> be on
> >> > > > > >> >>> > > >> >> > > > >>> another
> >> > > > > >> >>> > > >> >> > > > >>> > > > GitHub repo. Therefore I will
> >> > prepare
> >> > > a
> >> > > > > >> >>> standalone
> >> > > > > >> >>> > > >> project
> >> > > > > >> >>> > > >> >> > with
> >> > > > > >> >>> > > >> >> > > > the
> >> > > > > >> >>> > > >> >> > > > >>> > > Pub/Sub
> >> > > > > >> >>> > > >> >> > > > >>> > > > module and once the repository
> is
> >> > > > created
> >> > > > > a
> >> > > > > >> >>> pull
> >> > > > > >> >>> > > >> request
> >> > > > > >> >>> > > >> >> > shall
> >> > > > > >> >>> > > >> >> > > be
> >> > > > > >> >>> > > >> >> > > > >>> > issued.
> >> > > > > >> >>> > > >> >> > > > >>> > > > If there is anything else I can
> >> do
> >> > in
> >> > > > > order
> >> > > > > >> to
> >> > > > > >> >>> > > >> facilitate
> >> > > > > >> >>> > > >> >> > this
> >> > > > > >> >>> > > >> >> > > > >>> please
> >> > > > > >> >>> > > >> >> > > > >>> > let
> >> > > > > >> >>> > > >> >> > > > >>> > > > me know.
> >> > > > > >> >>> > > >> >> > > > >>> > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > If a ticket is created for this
> >> > > issue, I
> >> > > > > >> shall
> >> > > > > >> >>> be
> >> > > > > >> >>> > > able
> >> > > > > >> >>> > > >> to
> >> > > > > >> >>> > > >> >> > > create
> >> > > > > >> >>> > > >> >> > > > a
> >> > > > > >> >>> > > >> >> > > > >>> pull
> >> > > > > >> >>> > > >> >> > > > >>> > > > request and thus a build will
> >> take
> >> > > place
> >> > > > > on
> >> > > > > >> >>> > ignite's
> >> > > > > >> >>> > > >> CI.
> >> > > > > >> >>> > > >> >> > > > >>> > > > If everyone is aligned with
> >> adding a
> >> > > > > Pub/Sub
> >> > > > > >> >>> > > >> >> implementation
> >> > > > > >> >>> > > >> >> > > may I
> >> > > > > >> >>> > > >> >> > > > >>> > create
> >> > > > > >> >>> > > >> >> > > > >>> > > a
> >> > > > > >> >>> > > >> >> > > > >>> > > > ticket on JIRA?
> >> > > > > >> >>> > > >> >> > > > >>> > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > My next question has to do on
> how
> >> > > Ignite
> >> > > > > >> >>> configures
> >> > > > > >> >>> > > the
> >> > > > > >> >>> > > >> >> build
> >> > > > > >> >>> > > >> >> > > > jobs
> >> > > > > >> >>> > > >> >> > > > >>> in
> >> > > > > >> >>> > > >> >> > > > >>> > > Team
> >> > > > > >> >>> > > >> >> > > > >>> > > > City. The unit tests for my
> >> > > > implementation
> >> > > > > >> >>> need a
> >> > > > > >> >>> > > local
> >> > > > > >> >>> > > >> >> > Pub/Sub
> >> > > > > >> >>> > > >> >> > > > >>> server
> >> > > > > >> >>> > > >> >> > > > >>> > up
> >> > > > > >> >>> > > >> >> > > > >>> > > > and running (they provide one
> for
> >> > > > testing
> >> > > > > >> >>> > purposes).
> >> > > > > >> >>> > > >> Will
> >> > > > > >> >>> > > >> >> > this
> >> > > > > >> >>> > > >> >> > > > take
> >> > > > > >> >>> > > >> >> > > > >>> > > effect
> >> > > > > >> >>> > > >> >> > > > >>> > > > in a form of a build script
> >> inside
> >> > my
> >> > > > > >> >>> > implementation
> >> > > > > >> >>> > > or
> >> > > > > >> >>> > > >> >> is it
> >> > > > > >> >>> > > >> >> > > > >>> something
> >> > > > > >> >>> > > >> >> > > > >>> > > > that should be configured on
> Team
> >> > > City?
> >> > > > > >> >>> > > >> >> > > > >>> > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > Kind regards,
> >> > > > > >> >>> > > >> >> > > > >>> > > > Emmanouil
> >> > > > > >> >>> > > >> >> > > > >>> > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
> >> > > > > >> >>> > > >> >> > > > >>> > > > https://egkatzioura.com/ |
> >> > > > > >> >>> > > >> >> > > > >>> > > >
> >> > > > > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > > > >> >>> > > >> >> > > > >>> > > > https://github.com/gkatzioura
> >> > > > > >> >>> > > >> >> > > > >>> > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27,
> >> Denis
> >> > > > Magda
> >> > > > > <
> >> > > > > >> >>> > > >> >> [hidden email]
> >> > > > > >> >>> > > >> >> > >
> >> > > > > >> >>> > > >> >> > > > >>> wrote:
> >> > > > > >> >>> > > >> >> > > > >>> > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
> >> > > > > >> >>> > > >> >> > > > >>> > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > After contemplating on this,
> >> let
> >> > me
> >> > > > > >> propose
> >> > > > > >> >>> > another
> >> > > > > >> >>> > > >> way
> >> > > > > >> >>> > > >> >> > with
> >> > > > > >> >>> > > >> >> > > > this
> >> > > > > >> >>> > > >> >> > > > >>> > > > > particular integration that
> is
> >> > > aligned
> >> > > > > >> with
> >> > > > > >> >>> our
> >> > > > > >> >>> > > >> >> > > modularization
> >> > > > > >> >>> > > >> >> > > > >>> > > endeavor.
> >> > > > > >> >>> > > >> >> > > > >>> > > > > The modularization [1]
> defines
> >> the
> >> > > > > Ignite
> >> > > > > >> >>> core,
> >> > > > > >> >>> > > >> Ignite
> >> > > > > >> >>> > > >> >> > > modules,
> >> > > > > >> >>> > > >> >> > > > >>> and
> >> > > > > >> >>> > > >> >> > > > >>> > > > > independent integrations. The
> >> > first
> >> > > > two
> >> > > > > >> are
> >> > > > > >> >>> to be
> >> > > > > >> >>> > > >> >> located
> >> > > > > >> >>> > > >> >> > in
> >> > > > > >> >>> > > >> >> > > > >>> Ignite
> >> > > > > >> >>> > > >> >> > > > >>> > > > > repositories and to be
> >> > > tested/updated
> >> > > > by
> >> > > > > >> the
> >> > > > > >> >>> > > >> community
> >> > > > > >> >>> > > >> >> for
> >> > > > > >> >>> > > >> >> > > > every
> >> > > > > >> >>> > > >> >> > > > >>> > > release.
> >> > > > > >> >>> > > >> >> > > > >>> > > > > The modules are a subset of
> >> > existing
> >> > > > > >> >>> integrations
> >> > > > > >> >>> > > >> that
> >> > > > > >> >>> > > >> >> are
> >> > > > > >> >>> > > >> >> > > > >>> adopted
> >> > > > > >> >>> > > >> >> > > > >>> > > widely
> >> > > > > >> >>> > > >> >> > > > >>> > > > > and require community
> >> attention.
> >> > > > > >> >>> > > >> >> > > > >>> > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > As for the independent
> >> > integrations
> >> > > > [1],
> >> > > > > >> >>> those
> >> > > > > >> >>> > are
> >> > > > > >> >>> > > >> >> supposed
> >> > > > > >> >>> > > >> >> > > to
> >> > > > > >> >>> > > >> >> > > > be
> >> > > > > >> >>> > > >> >> > > > >>> > > located
> >> > > > > >> >>> > > >> >> > > > >>> > > > > in a separate GitHub repo. I
> >> would
> >> > > > > advise
> >> > > > > >> us
> >> > > > > >> >>> to
> >> > > > > >> >>> > > start
> >> > > > > >> >>> > > >> >> > > > >>> implementing
> >> > > > > >> >>> > > >> >> > > > >>> > this
> >> > > > > >> >>> > > >> >> > > > >>> > > > > concept with the Pub/Sub
> >> > > integration.
> >> > > > > Even
> >> > > > > >> >>> though
> >> > > > > >> >>> > > it
> >> > > > > >> >>> > > >> >> will
> >> > > > > >> >>> > > >> >> > be
> >> > > > > >> >>> > > >> >> > > > >>> located
> >> > > > > >> >>> > > >> >> > > > >>> > > in a
> >> > > > > >> >>> > > >> >> > > > >>> > > > > separate Github repo, it will
> >> be
> >> > > > listed
> >> > > > > >> among
> >> > > > > >> >>> > > >> officially
> >> > > > > >> >>> > > >> >> > > > >>> available
> >> > > > > >> >>> > > >> >> > > > >>> > > > > integrations on the Ignite
> >> website
> >> > > > > >> featuring
> >> > > > > >> >>> > > >> Emmanouil
> >> > > > > >> >>> > > >> >> as a
> >> > > > > >> >>> > > >> >> > > > >>> > > contributor.
> >> > > > > >> >>> > > >> >> > > > >>> > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > What do you think?
> >> > > > > >> >>> > > >> >> > > > >>> > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > [1]
> >> > > > > >> >>> > > >> >> > > > >>> > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > >
> >> > > > > >> >>> > > >> >> > > > >>> > >
> >> > > > > >> >>> > > >> >> > > > >>> >
> >> > > > > >> >>> > > >> >> > > > >>>
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > >
> >> > > > > >> >>> > > >> >> >
> >> > > > > >> >>> > > >> >>
> >> > > > > >> >>> > > >>
> >> > > > > >> >>> > >
> >> > > > > >> >>> >
> >> > > > > >> >>>
> >> > > > > >>
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> > > > > >> >>> > > >> >> > > > >>> > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > -
> >> > > > > >> >>> > > >> >> > > > >>> > > > > Denis
> >> > > > > >> >>> > > >> >> > > > >>> > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47
> PM
> >> > > Saikat
> >> > > > > >> >>> Maitra <
> >> > > > > >> >>> > > >> >> > > > >>> > [hidden email]
> >> > > > > >> >>> > > >> >> > > > >>> > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > wrote:
> >> > > > > >> >>> > > >> >> > > > >>> > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > Hi Emmanouil,
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > The changes looks good to
> >> me. I
> >> > > > wanted
> >> > > > > >> to
> >> > > > > >> >>> check
> >> > > > > >> >>> > > if
> >> > > > > >> >>> > > >> you
> >> > > > > >> >>> > > >> >> > are
> >> > > > > >> >>> > > >> >> > > > also
> >> > > > > >> >>> > > >> >> > > > >>> > able
> >> > > > > >> >>> > > >> >> > > > >>> > > to
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > validate that the once data
> >> is
> >> > > added
> >> > > > > to
> >> > > > > >> >>> Ignite
> >> > > > > >> >>> > > >> cluster
> >> > > > > >> >>> > > >> >> > you
> >> > > > > >> >>> > > >> >> > > > are
> >> > > > > >> >>> > > >> >> > > > >>> also
> >> > > > > >> >>> > > >> >> > > > >>> > > > able
> >> > > > > >> >>> > > >> >> > > > >>> > > > > to
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > access it using another
> >> Ignite
> >> > > > client
> >> > > > > or
> >> > > > > >> >>> using
> >> > > > > >> >>> > > rest
> >> > > > > >> >>> > > >> >> > > > endpoints.
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > We use Teamcity for CI
> >> process,
> >> > > the
> >> > > > > >> >>> details are
> >> > > > > >> >>> > > as
> >> > > > > >> >>> > > >> >> > follows
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > Trigger validation of those
> >> test
> >> > > > > suites
> >> > > > > >> >>> that
> >> > > > > >> >>> > have
> >> > > > > >> >>> > > >> been
> >> > > > > >> >>> > > >> >> > > > >>> affected by
> >> > > > > >> >>> > > >> >> > > > >>> > > your
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > changes on TeamCity <
> >> > > > > >> >>> > > http://ci.ignite.apache.org/>:
> >> > > > > >> >>> > > >> <
> >> > > > > >> >>> > > >> >> > > > >>> > > > > http://204.14.53.153/
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Locate a test suite
> you
> >> > have
> >> > > to
> >> > > > > >> check,
> >> > > > > >> >>> > press
> >> > > > > >> >>> > > >> >> button
> >> > > > > >> >>> > > >> >> > > > named
> >> > > > > >> >>> > > >> >> > > > >>> > "..."
> >> > > > > >> >>> > > >> >> > > > >>> > > > that
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >    is located on the left
> of
> >> > "Run"
> >> > > > > >> button.
> >> > > > > >> >>> "Run
> >> > > > > >> >>> > > >> custom
> >> > > > > >> >>> > > >> >> > > build"
> >> > > > > >> >>> > > >> >> > > > >>> > dialog
> >> > > > > >> >>> > > >> >> > > > >>> > > > will
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >    appear;
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Go to "Changes" tab
> and
> >> > > choose
> >> > > > > >> >>> > > >> >> > > > >>> > "pull/<pull-request-number>/head"
> >> > > > > >> >>> > > >> >> > > > >>> > > > in
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >    "Build branch" dropdown
> >> list;
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Press "Run build"
> button
> >> > and
> >> > > > > >> monitor
> >> > > > > >> >>> tests
> >> > > > > >> >>> > > >> >> results.
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > @Roman [hidden email] -
> >> can
> >> > > you
> >> > > > > also
> >> > > > > >> >>> please
> >> > > > > >> >>> > > >> take a
> >> > > > > >> >>> > > >> >> > > look?
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > Regards,
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > Saikat
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at
> 6:20
> >> PM
> >> > > > > >> Emmanouil
> >> > > > > >> >>> > > >> Gkatziouras
> >> > > > > >> >>> > > >> >> <
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > [hidden email]>
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > wrote:
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > > I have my implementation
> >> on an
> >> > > > > Ignite
> >> > > > > >> >>> fork on
> >> > > > > >> >>> > > my
> >> > > > > >> >>> > > >> >> > personal
> >> > > > > >> >>> > > >> >> > > > >>> GitHub
> >> > > > > >> >>> > > >> >> > > > >>> > > > > account
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > > [1]
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > > It is based on the Kafka
> >> > > > > >> implementation
> >> > > > > >> >>> which
> >> > > > > >> >>> > > is
> >> > > > > >> >>> > > >> of
> >> > > > > >> >>> > > >> >> a
> >> > > > > >> >>> > > >> >> > > > polling
> >> > > > > >> >>> > > >> >> > > > >>> > > nature.
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > > GCP does not provide a
> >> service
> >> > > > like
> >> > > > > >> >>> Kafka.
> >> > > > > >> >>> > > >> Instead
> >> > > > > >> >>> > > >> >> it
> >> > > > > >> >>> > > >> >> > > > >>> provides us
> >> > > > > >> >>> > > >> >> > > > >>> > > > with
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Pub/Sub which is just
> like
> >> > > Kafka.
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Therefore using the Kafka
> >> > > streamer
> >> > > > > as
> >> > > > > >> a
> >> > > > > >> >>> > > guideline
> >> > > > > >> >>> > > >> >> made
> >> > > > > >> >>> > > >> >> > > more
> >> > > > > >> >>> > > >> >> > > > >>> sense
> >> > > > > >> >>> > > >> >> > > > >>> > > to
> >> > > > > >> >>> > > >> >> > > > >>> > > > > me.
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Apart from the
> >> implementation
> >> > I
> >> > > > have
> >> > > > > >> also
> >> > > > > >> >>> > > added a
> >> > > > > >> >>> > > >> >> unit
> >> > > > > >> >>> > > >> >> > > > test.
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > > It definitely needs some
> >> > > polishing
> >> > > > > >> and a
> >> > > > > >> >>> > double
> >> > > > > >> >>> > > >> >> check
> >> > > > > >> >>> > > >> >> > on
> >> > > > > >> >>> > > >> >> > > my
> >> > > > > >> >>> > > >> >> > > > >>> side
> >> > > > > >> >>> > > >> >> > > > >>> > to
> >> > > > > >> >>> > > >> >> > > > >>> > > > > make
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > > the review smooth.
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > > For the tests a local
> >> Pub/Sub
> >> > > [2]
> >> > > > > >> server
> >> > > >
> >
> >
>

[Saikat Maitra]

Hi Emmanouil,

I looked into the build logs and I observed that since I added artifacts
dependencies to "Build apache Ignite"[1] it was able to pull the required
dependencies and was able to run the existing tests. It is however not
identifying new modules like I changed from flink to flink-ext or pub-sub.

[1]
https://ci.ignite.apache.org/admin/editDependencies.html?id=buildType:IgniteExtensions_Build

I will look into the issue and debug further.

Regards,
Saikat


On Sat, Nov 30, 2019 at 5:35 PM Emmanouil Gkatziouras <[hidden email]>
wrote:

> Hi Saikat!
>
> From the logs[1] it seems that TC cannot find the project. However I did
> include the project on the parent pom (modules sections) [2]
> I tried to reproduce locally unfortunately I am not aware of the full
> context on TC. It seems as if a different parent pom is being picked up.
>
> Kind regards
>
> [1]
>
> https://ci.ignite.apache.org/viewLog.html?buildId=4804893&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=1704
> [2]
>
> https://github.com/gkatzioura/ignite-extensions/blob/d6a8beee7feff28f59d9a12be692016305564d25/pom.xml#L46
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Sat, 30 Nov 2019 at 22:20, Saikat Maitra <[hidden email]>
> wrote:
>
> > Hello,
> >
> > I have made changes in Ignite Extensions build configurations.
> >
> > It is now running the tests as expected.
> >
> >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4804480&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> >
> > Regards,
> > Saikat
> >
> > On Thu, Nov 28, 2019 at 1:29 PM Saikat Maitra <[hidden email]>
> > wrote:
> >
> > > Hi,
> > >
> > > We do run build with -DskipTests and then configure selective test
> suites
> > > to run.
> > >
> > > I will check why these test suites are not getting executed.
> > >
> > > Regards
> > > Saikat
> > >
> > > On Wed, 27 Nov 2019 at 3:05 PM, Emmanouil Gkatziouras <
> > > [hidden email]> wrote:
> > >
> > >> Hi all,
> > >>
> > >> I did add a test suite and run the build with the corresponding
> > arguments,
> > >> however the tests did not run [1].
> > >> I checked the Flink's build logs and the same message is displayed
> there
> > >> too `No tests to run` [2].
> > >>
> > >> [1]
> > >>
> > >>
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4798203&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=248
> > >> [2]
> > >>
> > >>
> >
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4789199&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=1256
> > >>
> > >> Kind regards
> > >> *Emmanouil Gkatziouras*
> > >> https://egkatzioura.com/ |
> > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> https://github.com/gkatzioura
> > >>
> > >>
> > >> On Wed, 27 Nov 2019 at 18:58, Saikat Maitra <[hidden email]>
> > >> wrote:
> > >>
> > >> > Hi,
> > >> >
> > >> > We will need to add test suites for tests to be executed in build.
> We
> > >> pass
> > >> > them as TestSuites param. I had added Flink sink and source
> > testsuites.
> > >> >
> > >> > Can you please review and confirm.
> > >> >
> > >> > Regards
> > >> > Saikat
> > >> >
> > >> > On Wed, 27 Nov 2019 at 6:00 AM, Emmanouil Gkatziouras <
> > >> > [hidden email]>
> > >> > wrote:
> > >> >
> > >> > > Hi all!
> > >> > >
> > >> > > I was successful building and running my pull requests. It seems
> > that
> > >> the
> > >> > > tests do not run both for Flink and Pub/Sub [1]
> > >> > > If there is something I cannot do to make them discoverable please
> > >> let me
> > >> > > know.
> > >> > >
> > >> > > [1]
> > >> > >
> > >> > >
> > >> >
> > >>
> >
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4796722&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=283
> > >> > >
> > >> > > *Emmanouil Gkatziouras*
> > >> > > https://egkatzioura.com/ |
> > >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> > > https://github.com/gkatzioura
> > >> > >
> > >> > >
> > >> > > On Wed, 27 Nov 2019 at 04:57, Saikat Maitra <
> > [hidden email]>
> > >> > > wrote:
> > >> > >
> > >> > > > Hi Emmanouil,
> > >> > > >
> > >> > > > I have added you as contributor in Ignite Extensions project.
> Can
> > >> you
> > >> > > > please check and confirm if you are able to see the project and
> > >> execute
> > >> > > > build on your pull request.
> > >> > > >
> > >> > > > Regards,
> > >> > > > Saikat
> > >> > > >
> > >> > > > On Tue, Nov 26, 2019 at 12:49 PM Emmanouil Gkatziouras <
> > >> > > > [hidden email]>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > Hi all!
> > >> > > > >
> > >> > > > > If someone could give me read access to the Job of Ignite
> > >> Extensions
> > >> > on
> > >> > > > > Team City will greatly help me (username gkatzioura).
> > >> > > > > I suppose the builds get triggered automatically on a pull
> > >> request.
> > >> > > > >
> > >> > > > > Kind regards,
> > >> > > > > Emmanouil
> > >> > > > >
> > >> > > > > *Emmanouil Gkatziouras*
> > >> > > > > https://egkatzioura.com/ |
> > >> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> > > > > https://github.com/gkatzioura
> > >> > > > >
> > >> > > > >
> > >> > > > > On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras <
> > >> > > > [hidden email]>
> > >> > > > > wrote:
> > >> > > > >
> > >> > > > > > Hi Saikat!
> > >> > > > > >
> > >> > > > > > I just rebased with the flink branch. Unfortunately I do not
> > >> have
> > >> > > read
> > >> > > > > > access to the team city link you provided and thus evaluate
> > the
> > >> > tests
> > >> > > > for
> > >> > > > > > my pull request.
> > >> > > > > > I guess it has to do with ignite extensions being new.
> > >> > > > > > My username is gkatzioura and email the one I am using on
> this
> > >> > email
> > >> > > > > >
> > >> > > > > > Thank you for your time.
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > *Emmanouil Gkatziouras*
> > >> > > > > > https://egkatzioura.com/ |
> > >> > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> > > > > > https://github.com/gkatzioura
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <
> > >> > [hidden email]
> > >> > > >
> > >> > > > > > wrote:
> > >> > > > > >
> > >> > > > > >> Hi Emmanouil,
> > >> > > > > >>
> > >> > > > > >> The latest build on teamcity has passed on Flink pull
> > request.
> > >> > > > > >>
> > >> > > > > >>
> > >> > > > > >>
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> > >> > > > > >>
> > >> > > > > >> you should be able to take the changes I made in my PR and
> > run
> > >> > build
> > >> > > > on
> > >> > > > > >> your pull request.
> > >> > > > > >>
> > >> > > > > >> Please let me know if you have any questions.
> > >> > > > > >>
> > >> > > > > >> Regards,
> > >> > > > > >> Saikat
> > >> > > > > >>
> > >> > > > > >> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <
> > >> > > > [hidden email]
> > >> > > > > >
> > >> > > > > >> wrote:
> > >> > > > > >>
> > >> > > > > >> > Hi Emmanouil,
> > >> > > > > >> >
> > >> > > > > >> > I have fixed the Flink module testsuites and have setup
> new
> > >> > > project
> > >> > > > in
> > >> > > > > >> > teamcity.
> > >> > > > > >> >
> > >> > > > > >> >
> > >> > > > > >> >
> > >> > > > > >>
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
> > >> > > > > >> >
> > >> > > > > >> > I am looking into setting up the build.
> > >> > > > > >> >
> > >> > > > > >> > Regards,
> > >> > > > > >> > Saikat
> > >> > > > > >> >
> > >> > > > > >> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <
> > >> > > > > [hidden email]>
> > >> > > > > >> > wrote:
> > >> > > > > >> >
> > >> > > > > >> >> Hi Emmanouil,
> > >> > > > > >> >>
> > >> > > > > >> >> Thank you for your email. Yes, the plan is once the
> Flink
> > >> PR is
> > >> > > > > merged
> > >> > > > > >> it
> > >> > > > > >> >> will provide the Licence, parent POMs etc and you can
> > rebase
> > >> > from
> > >> > > > > >> master
> > >> > > > > >> >> branch and apply the changes on top of it.
> > >> > > > > >> >>
> > >> > > > > >> >> I am getting some issues with test failures in local
> with
> > >> > > > > >> >> GridTestProperties as the test.properties is not present
> > in
> > >> > this
> > >> > > > new
> > >> > > > > >> >> project but should be available from dependencies.
> > >> > > > > >> >>
> > >> > > > > >> >> Once I address this issue, I will go ahead and merge the
> > >> > changes
> > >> > > > and
> > >> > > > > >> then
> > >> > > > > >> >> we can take it from there.
> > >> > > > > >> >>
> > >> > > > > >> >> Regards,
> > >> > > > > >> >> Saikat
> > >> > > > > >> >>
> > >> > > > > >> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
> > >> > > > > >> >> [hidden email]> wrote:
> > >> > > > > >> >>
> > >> > > > > >> >>> Hi all,
> > >> > > > > >> >>>
> > >> > > > > >> >>> I made my first pull request [1]. Since this project is
> > >> brand
> > >> > > new
> > >> > > > > (no
> > >> > > > > >> >>> parent poms, licensing), I reckoned it was best to use
> > >> > Saikat's
> > >> > > > > >> branch on
> > >> > > > > >> >>> Flink.
> > >> > > > > >> >>> I suppose the Flink branch will be merged first. If not
> > >> please
> > >> > > > give
> > >> > > > > me
> > >> > > > > >> >>> guidelines on how I should proceed next.
> > >> > > > > >> >>>
> > >> > > > > >> >>> Kind regards
> > >> > > > > >> >>> Emmanouil
> > >> > > > > >> >>>
> > >> > > > > >> >>> [1] https://github.com/apache/ignite-extensions/pull/2
> > >> > > > > >> >>>
> > >> > > > > >> >>> *Emmanouil Gkatziouras*
> > >> > > > > >> >>> https://egkatzioura.com/ |
> > >> > > > > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> > > > > >> >>> https://github.com/gkatzioura
> > >> > > > > >> >>>
> > >> > > > > >> >>>
> > >> > > > > >> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <
> > >> [hidden email]>
> > >> > > > > wrote:
> > >> > > > > >> >>>
> > >> > > > > >> >>> > Awesome, ping us whenever you're ready!
> > >> > > > > >> >>> >
> > >> > > > > >> >>> > -
> > >> > > > > >> >>> > Denis
> > >> > > > > >> >>> >
> > >> > > > > >> >>> >
> > >> > > > > >> >>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil
> Gkatziouras
> > <
> > >> > > > > >> >>> > [hidden email]>
> > >> > > > > >> >>> > wrote:
> > >> > > > > >> >>> >
> > >> > > > > >> >>> > > Hi all!
> > >> > > > > >> >>> > >
> > >> > > > > >> >>> > > I am sorry for being late on that. I was trying to
> > >> > refactor
> > >> > > > the
> > >> > > > > >> test
> > >> > > > > >> >>> in
> > >> > > > > >> >>> > > order not to be in need of any external tools or
> > >> spinning
> > >> > > up a
> > >> > > > > >> >>> server.
> > >> > > > > >> >>> > > I did forked the new repo and indeed my changes
> > there,
> > >> so
> > >> > a
> > >> > > > pull
> > >> > > > > >> >>> request
> > >> > > > > >> >>> > is
> > >> > > > > >> >>> > > a matter of time!
> > >> > > > > >> >>> > >
> > >> > > > > >> >>> > > Kind regards
> > >> > > > > >> >>> > >
> > >> > > > > >> >>> > > *Emmanouil Gkatziouras*
> > >> > > > > >> >>> > > https://egkatzioura.com/ |
> > >> > > > > >> >>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> > > > > >> >>> > > https://github.com/gkatzioura
> > >> > > > > >> >>> > >
> > >> > > > > >> >>> > >
> > >> > > > > >> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <
> > >> > > [hidden email]>
> > >> > > > > >> wrote:
> > >> > > > > >> >>> > >
> > >> > > > > >> >>> > > > Hi Emmanouil,
> > >> > > > > >> >>> > > >
> > >> > > > > >> >>> > > > Do you have any questions or need any support
> from
> > >> the
> > >> > > > > >> community?
> > >> > > > > >> >>> > > >
> > >> > > > > >> >>> > > > -
> > >> > > > > >> >>> > > > Denis
> > >> > > > > >> >>> > > >
> > >> > > > > >> >>> > > >
> > >> > > > > >> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
> > >> > > > > >> >>> [hidden email]
> > >> > > > > >> >>> > >
> > >> > > > > >> >>> > > > wrote:
> > >> > > > > >> >>> > > >
> > >> > > > > >> >>> > > >> Hi Emmanouil,
> > >> > > > > >> >>> > > >>
> > >> > > > > >> >>> > > >> Can you please take a looks at dev utils, if
> this
> > is
> > >> > > > > something
> > >> > > > > >> >>> you are
> > >> > > > > >> >>> > > >> looking for
> > >> > > > > >> >>> > > >>
> > >> > > > >
> https://github.com/apache/ignite/tree/master/modules/dev-utils
> > >> > > > > >> ?
> > >> > > > > >> >>> > > >>
> > >> > > > > >> >>> > > >> IMO, if you can release Pub/Sub server in maven
> > and
> > >> > then
> > >> > > > use
> > >> > > > > >> it as
> > >> > > > > >> >>> > > >> dependency, that would be great.
> > >> > > > > >> >>> > > >>
> > >> > > > > >> >>> > > >> Regards,
> > >> > > > > >> >>> > > >> Saikat
> > >> > > > > >> >>> > > >>
> > >> > > > > >> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
> > >> > > > > >> >>> > [hidden email]>
> > >> > > > > >> >>> > > >> wrote:
> > >> > > > > >> >>> > > >>
> > >> > > > > >> >>> > > >> > Hi Emmanouil,
> > >> > > > > >> >>> > > >> >
> > >> > > > > >> >>> > > >> > The master branch has been initialized. I have
> > >> > raised a
> > >> > > > PR
> > >> > > > > to
> > >> > > > > >> >>> > migrate
> > >> > > > > >> >>> > > >> > Flink module to ignite-extensions.
> > >> > > > > >> >>> > > >> >
> > >> > > > > >> >>> > > >> > PR
> > >> > https://github.com/apache/ignite-extensions/pull/1
> > >> > > > > >> >>> > > >> > Jira :
> > >> > > > https://issues.apache.org/jira/browse/IGNITE-12356
> > >> > > > > >> >>> > > >> >
> > >> > > > > >> >>> > > >> > Regards,
> > >> > > > > >> >>> > > >> > Saikat
> > >> > > > > >> >>> > > >> >
> > >> > > > > >> >>> > > >> >
> > >> > > > > >> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil
> > >> > Gkatziouras <
> > >> > > > > >> >>> > > >> > [hidden email]> wrote:
> > >> > > > > >> >>> > > >> >
> > >> > > > > >> >>> > > >> >> Hi all!
> > >> > > > > >> >>> > > >> >>
> > >> > > > > >> >>> > > >> >> This is an update. The integration is tested
> > with
> > >> > > mocks
> > >> > > > > >> based
> > >> > > > > >> >>> on
> > >> > > > > >> >>> > the
> > >> > > > > >> >>> > > >> >> original Ignite repo [1].
> > >> > > > > >> >>> > > >> >> I am currently waiting for the first commits
> on
> > >> the
> > >> > > new
> > >> > > > > >> >>> integration
> > >> > > > > >> >>> > > >> >> project
> > >> > > > > >> >>> > > >> >> to take place in order to create my pull
> > request
> > >> > > > > accordingly
> > >> > > > > >> >>> to the
> > >> > > > > >> >>> > > >> >> initial
> > >> > > > > >> >>> > > >> >> commits.
> > >> > > > > >> >>> > > >> >> In order to simulate the Pub/Sub server in a
> > >> test I
> > >> > > > used a
> > >> > > > > >> test
> > >> > > > > >> >>> > > server
> > >> > > > > >> >>> > > >> >> implementation [2]. Unfortunately they don't
> > have
> > >> > this
> > >> > > > > util
> > >> > > > > >> >>> > uploaded
> > >> > > > > >> >>> > > >> in a
> > >> > > > > >> >>> > > >> >> maven repo, thus it needs to be hosted on a
> > maven
> > >> > repo
> > >> > > > > >> (Apache
> > >> > > > > >> >>> > > License
> > >> > > > > >> >>> > > >> >> 2.0). Is there a repo for utilities that
> Ignite
> > >> uses
> > >> > > for
> > >> > > > > the
> > >> > > > > >> >>> > > >> test/build?
> > >> > > > > >> >>> > > >> >> I have also created a ticket [3]
> > >> > > > > >> >>> > > >> >>
> > >> > > > > >> >>> > > >> >> Kind regards
> > >> > > > > >> >>> > > >> >>
> > >> > > > > >> >>> > > >> >> [1]
> > >> > > > > >> >>> >
> > >> > > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > >> > > > > >> >>> > > >> >> [2]
> > >> > > > > >> >>>
> > >> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> > >> > > > > >> >>> > > >> >> [3]
> > >> > > https://issues.apache.org/jira/browse/IGNITE-12262
> > >> > > > > >> >>> > > >> >>
> > >> > > > > >> >>> > > >> >> *Emmanouil Gkatziouras*
> > >> > > > > >> >>> > > >> >> https://egkatzioura.com/ |
> > >> > > > > >> >>> > > >> >>
> > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> > > > > >> >>> > > >> >> https://github.com/gkatzioura
> > >> > > > > >> >>> > > >> >>
> > >> > > > > >> >>> > > >> >>
> > >> > > > > >> >>> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
> > >> > > > > >> >>> > [hidden email]
> > >> > > > > >> >>> > > >
> > >> > > > > >> >>> > > >> >> wrote:
> > >> > > > > >> >>> > > >> >>
> > >> > > > > >> >>> > > >> >> > Hello Ilya,
> > >> > > > > >> >>> > > >> >> >
> > >> > > > > >> >>> > > >> >> > Thank you for your email. Yes, before we
> > remove
> > >> > any
> > >> > > > > >> >>> integration
> > >> > > > > >> >>> > we
> > >> > > > > >> >>> > > >> will
> > >> > > > > >> >>> > > >> >> > move them to another repository and
> release.
> > >> > > > > >> >>> > > >> >> >
> > >> > > > > >> >>> > > >> >> > We are thinking we will start with Pub/Sub
> > >> > Streamer
> > >> > > > > >> >>> > implementation
> > >> > > > > >> >>> > > >> and
> > >> > > > > >> >>> > > >> >> > continue to move other integration as
> > >> mentioned in
> > >> > > the
> > >> > > > > >> >>> following
> > >> > > > > >> >>> > > doc.
> > >> > > > > >> >>> > > >> >> >
> > >> > > > > >> >>> > > >> >> >
> > >> > > > > >> >>> > > >> >> >
> > >> > > > > >> >>> > > >> >>
> > >> > > > > >> >>> > > >>
> > >> > > > > >> >>> > >
> > >> > > > > >> >>> >
> > >> > > > > >> >>>
> > >> > > > > >>
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > >> > > > > >> >>> > > >> >> >
> > >> > > > > >> >>> > > >> >> > Regards,
> > >> > > > > >> >>> > > >> >> > Saikat
> > >> > > > > >> >>> > > >> >> >
> > >> > > > > >> >>> > > >> >> >
> > >> > > > > >> >>> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya
> > >> Kasnacheev <
> > >> > > > > >> >>> > > >> >> > [hidden email]>
> > >> > > > > >> >>> > > >> >> > wrote:
> > >> > > > > >> >>> > > >> >> >
> > >> > > > > >> >>> > > >> >> > > Hello!
> > >> > > > > >> >>> > > >> >> > >
> > >> > > > > >> >>> > > >> >> > > My take on this: we want to offload some
> of
> > >> > > > > >> integrations,
> > >> > > > > >> >>> but
> > >> > > > > >> >>> > > >> please
> > >> > > > > >> >>> > > >> >> > commit
> > >> > > > > >> >>> > > >> >> > > and release them to another project
> first,
> > >> and
> > >> > > only
> > >> > > > > >> then we
> > >> > > > > >> >>> > will
> > >> > > > > >> >>> > > >> >> decide
> > >> > > > > >> >>> > > >> >> > to
> > >> > > > > >> >>> > > >> >> > > remove them from our repository.
> > >> > > > > >> >>> > > >> >> > >
> > >> > > > > >> >>> > > >> >> > > Please also make sure they are available
> > with
> > >> > same
> > >> > > > > >> artifact
> > >> > > > > >> >>> > name
> > >> > > > > >> >>> > > >> and
> > >> > > > > >> >>> > > >> >> API.
> > >> > > > > >> >>> > > >> >> > > If this has to be changed, let's wait for
> > AI
> > >> 3.0
> > >> > > > > >> >>> > > >> >> > >
> > >> > > > > >> >>> > > >> >> > > Regards,
> > >> > > > > >> >>> > > >> >> > > --
> > >> > > > > >> >>> > > >> >> > > Ilya Kasnacheev
> > >> > > > > >> >>> > > >> >> > >
> > >> > > > > >> >>> > > >> >> > >
> > >> > > > > >> >>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat
> Maitra
> > <
> > >> > > > > >> >>> > > >> [hidden email]>:
> > >> > > > > >> >>> > > >> >> > >
> > >> > > > > >> >>> > > >> >> > > > Hello Emmanouil,
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > > As discussed earlier, I discussed with
> > >> Apache
> > >> > > > Bahir
> > >> > > > > >> >>> community
> > >> > > > > >> >>> > > and
> > >> > > > > >> >>> > > >> >> they
> > >> > > > > >> >>> > > >> >> > > are
> > >> > > > > >> >>> > > >> >> > > > interested to have Apache Ignite
> > >> extensions as
> > >> > > > part
> > >> > > > > of
> > >> > > > > >> >>> Apache
> > >> > > > > >> >>> > > >> Bahir
> > >> > > > > >> >>> > > >> >> > > > project.
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > > Can you please share Pub/Sub streamer
> > >> > > > implementation
> > >> > > > > >> >>> details
> > >> > > > > >> >>> > > with
> > >> > > > > >> >>> > > >> >> > Apache
> > >> > > > > >> >>> > > >> >> > > > Bahir community and request for
> feedback.
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > > https://bahir.apache.org/
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > > I have also requested for contributor
> > >> access
> > >> > in
> > >> > > > Jira
> > >> > > > > >> for
> > >> > > > > >> >>> > Apache
> > >> > > > > >> >>> > > >> >> Bahir
> > >> > > > > >> >>> > > >> >> > > > project so that I can create issues and
> > >> assign
> > >> > > to
> > >> > > > > >> myself.
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > > I can help with code reviews as well.
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > > Here is the email thread for reference
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > >
> > >> > > > https://www.mail-archive.com/dev@.../msg02703.html
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > > Regards,
> > >> > > > > >> >>> > > >> >> > > > Saikat
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat
> > >> Maitra
> > >> > <
> > >> > > > > >> >>> > > >> >> [hidden email]
> > >> > > > > >> >>> > > >> >> > >
> > >> > > > > >> >>> > > >> >> > > > wrote:
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > > > Hello,
> > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > >> >>> > > >> >> > > > > As discussed I have created following
> > >> > > discussion
> > >> > > > > >> >>> threads on
> > >> > > > > >> >>> > > our
> > >> > > > > >> >>> > > >> >> > > migration
> > >> > > > > >> >>> > > >> >> > > > > proposals for Apache Ignite
> extensions:
> > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > >
> > >> > > > > >> >>> > > >> >> >
> > >> > > > > >> >>> > > >> >>
> > >> > > > > >> >>> > > >>
> > >> > > > > >> >>> > >
> > >> > > > > >> >>> >
> > >> > > > > >> >>>
> > >> > > > > >>
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > >> >>> > > >>
> > >> > > > > >>
> > >> https://www.mail-archive.com/dev@.../msg02703.html
> > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > >> >>> > > >> >> > > > > I will share update as I hear more
> > >> > > information.
> > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > >> >>> > > >> >> > > > > Regards,
> > >> > > > > >> >>> > > >> >> > > > > Saikat
> > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > >> >>> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM
> Saikat
> > >> > Maitra
> > >> > > <
> > >> > > > > >> >>> > > >> >> > [hidden email]
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > > > wrote:
> > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > >> >>> > > >> >> > > > >> Hello Denis,
> > >> > > > > >> >>> > > >> >> > > > >>
> > >> > > > > >> >>> > > >> >> > > > >> Sure, sounds good. I will create a
> > >> separate
> > >> > > > > >> discussion
> > >> > > > > >> >>> > > thread
> > >> > > > > >> >>> > > >> to
> > >> > > > > >> >>> > > >> >> get
> > >> > > > > >> >>> > > >> >> > > > >> community feedback on whether we
> > >> > > > > >> >>> > > >> >> > > > >> should join the Bahir or kick-off
> > >> "Ignite
> > >> > > > > >> Extensions"
> > >> > > > > >> >>> > > project.
> > >> > > > > >> >>> > > >> >> > > > >>
> > >> > > > > >> >>> > > >> >> > > > >> Regards,
> > >> > > > > >> >>> > > >> >> > > > >> Saikat
> > >> > > > > >> >>> > > >> >> > > > >>
> > >> > > > > >> >>> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM
> Denis
> > >> > Magda <
> > >> > > > > >> >>> > > >> [hidden email]>
> > >> > > > > >> >>> > > >> >> > > wrote:
> > >> > > > > >> >>> > > >> >> > > > >>
> > >> > > > > >> >>> > > >> >> > > > >>> Folks,
> > >> > > > > >> >>> > > >> >> > > > >>>
> > >> > > > > >> >>> > > >> >> > > > >>> The concept of Apache Bahir is
> > >> precisely
> > >> > > what
> > >> > > > we
> > >> > > > > >> >>> need!
> > >> > > > > >> >>> > > >> Saikat,
> > >> > > > > >> >>> > > >> >> > thanks
> > >> > > > > >> >>> > > >> >> > > > for
> > >> > > > > >> >>> > > >> >> > > > >>> doing the research. Why I believe
> > the *
> > >> > > idea*
> > >> > > > > >> fits us
> > >> > > > > >> >>> > well:
> > >> > > > > >> >>> > > >> >> > > > >>>
> > >> > > > > >> >>> > > >> >> > > > >>>    - All integrations can be stored
> > in
> > >> > > > separate
> > >> > > > > >> >>> Github
> > >> > > > > >> >>> > > >> >> repositories
> > >> > > > > >> >>> > > >> >> > > and
> > >> > > > > >> >>> > > >> >> > > > >>>    have their dev lifecycles. We've
> > not
> > >> > > > obliged
> > >> > > > > to
> > >> > > > > >> >>> couple
> > >> > > > > >> >>> > > all
> > >> > > > > >> >>> > > >> >> the
> > >> > > > > >> >>> > > >> >> > > > >>> integrations
> > >> > > > > >> >>> > > >> >> > > > >>>    in a single repo. For instance,
> > >> Spark
> > >> > can
> > >> > > > be
> > >> > > > > >> >>> located
> > >> > > > > >> >>> > in
> > >> > > > > >> >>> > > a
> > >> > > > > >> >>> > > >> >> > > dedicated
> > >> > > > > >> >>> > > >> >> > > > >>> repo
> > >> > > > > >> >>> > > >> >> > > > >>>    while streaming integrations
> might
> > >> be
> > >> > in
> > >> > > a
> > >> > > > > >> single
> > >> > > > > >> >>> one.
> > >> > > > > >> >>> > > >> It's
> > >> > > > > >> >>> > > >> >> up
> > >> > > > > >> >>> > > >> >> > to
> > >> > > > > >> >>> > > >> >> > > > us.
> > >> > > > > >> >>> > > >> >> > > > >>>    - All the repositories and
> > >> integrations
> > >> > > > > belong
> > >> > > > > >> to
> > >> > > > > >> >>> ASF.
> > >> > > > > >> >>> > > >> We're
> > >> > > > > >> >>> > > >> >> not
> > >> > > > > >> >>> > > >> >> > > > >>> dumping
> > >> > > > > >> >>> > > >> >> > > > >>>    them on Github but rather keep
> > >> > supporting
> > >> > > > and
> > >> > > > > >> >>> > developing
> > >> > > > > >> >>> > > >> in
> > >> > > > > >> >>> > > >> >> > > > >>> accordance with
> > >> > > > > >> >>> > > >> >> > > > >>>    ASF vision and practices.
> > >> > > > > >> >>> > > >> >> > > > >>>    - There is a way to reward
> > >> contributors
> > >> > > via
> > >> > > > > >> >>> > > committership
> > >> > > > > >> >>> > > >> and
> > >> > > > > >> >>> > > >> >> > PMC
> > >> > > > > >> >>> > > >> >> > > > >>>    membership. You won't get this
> if
> > a
> > >> > > project
> > >> > > > > is
> > >> > > > > >> >>> just
> > >> > > > > >> >>> > one
> > >> > > > > >> >>> > > of
> > >> > > > > >> >>> > > >> >> the
> > >> > > > > >> >>> > > >> >> > > > >>> millions of
> > >> > > > > >> >>> > > >> >> > > > >>>    Github projects.
> > >> > > > > >> >>> > > >> >> > > > >>>
> > >> > > > > >> >>> > > >> >> > > > >>> Saikat, as Ignite PMC member, could
> > you
> > >> > > please
> > >> > > > > >> >>> kick-off a
> > >> > > > > >> >>> > > >> >> separate
> > >> > > > > >> >>> > > >> >> > > > >>> dev-list
> > >> > > > > >> >>> > > >> >> > > > >>> discussion to involve more
> community
> > >> > members
> > >> > > > and
> > >> > > > > >> >>> > referring
> > >> > > > > >> >>> > > to
> > >> > > > > >> >>> > > >> >> this
> > >> > > > > >> >>> > > >> >> > > > >>> thread?
> > >> > > > > >> >>> > > >> >> > > > >>> I think the primary question the
> > >> community
> > >> > > > needs
> > >> > > > > >> to
> > >> > > > > >> >>> > answer
> > >> > > > > >> >>> > > >> >> whether
> > >> > > > > >> >>> > > >> >> > we
> > >> > > > > >> >>> > > >> >> > > > >>> should join the Bahir or kick-off
> > >> "Ignite
> > >> > > > > >> Extensions"
> > >> > > > > >> >>> > > >> project?
> > >> > > > > >> >>> > > >> >> > > > >>>
> > >> > > > > >> >>> > > >> >> > > > >>> -
> > >> > > > > >> >>> > > >> >> > > > >>> Denis
> > >> > > > > >> >>> > > >> >> > > > >>>
> > >> > > > > >> >>> > > >> >> > > > >>>
> > >> > > > > >> >>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM
> > Alexey
> > >> > > > Zinoviev
> > >> > > > > <
> > >> > > > > >> >>> > > >> >> > > > [hidden email]>
> > >> > > > > >> >>> > > >> >> > > > >>> wrote:
> > >> > > > > >> >>> > > >> >> > > > >>>
> > >> > > > > >> >>> > > >> >> > > > >>> > Maybe we could move all our
> > Streaming
> > >> > > > > >> Integrations
> > >> > > > > >> >>> > there,
> > >> > > > > >> >>> > > >> but
> > >> > > > > >> >>> > > >> >> > what
> > >> > > > > >> >>> > > >> >> > > is
> > >> > > > > >> >>> > > >> >> > > > >>> about
> > >> > > > > >> >>> > > >> >> > > > >>> > maintaining and committer
> > >> permissions to
> > >> > > the
> > >> > > > > new
> > >> > > > > >> >>> > > >> repositories?
> > >> > > > > >> >>> > > >> >> > > > >>> > I see the list of the committers
> > and
> > >> PMC
> > >> > > > > members
> > >> > > > > >> >>> there
> > >> > > > > >> >>> > > >> >> > > > >>> >
> > >> > > https://bahir.apache.org/community-members/
> > >> > > > > >> >>> > > >> >> > > > >>> >
> > >> > > > > >> >>> > > >> >> > > > >>> > Could somebody from Ignite
> > community
> > >> be
> > >> > > > added
> > >> > > > > to
> > >> > > > > >> >>> this
> > >> > > > > >> >>> > > list
> > >> > > > > >> >>> > > >> as
> > >> > > > > >> >>> > > >> >> a
> > >> > > > > >> >>> > > >> >> > > > >>> > PMC/committer to maintain Ignite
> > >> > > > integrations?
> > >> > > > > >> >>> > > >> >> > > > >>> > If yes, I'd happy to join this
> > >> project
> > >> > and
> > >> > > > > >> >>> collaborate
> > >> > > > > >> >>> > > with
> > >> > > > > >> >>> > > >> >> these
> > >> > > > > >> >>> > > >> >> > > > guys
> > >> > > > > >> >>> > > >> >> > > > >>> > together
> > >> > > > > >> >>> > > >> >> > > > >>> > If no, and we should start from
> the
> > >> zero
> > >> > > > level
> > >> > > > > >> with
> > >> > > > > >> >>> > > >> external
> > >> > > > > >> >>> > > >> >> PRs
> > >> > > > > >> >>> > > >> >> > -
> > >> > > > > >> >>> > > >> >> > > > >>> hmmm,
> > >> > > > > >> >>> > > >> >> > > > >>> > it's better to have our own
> > external
> > >> > > > > repository
> > >> > > > > >> >>> with
> > >> > > > > >> >>> > > >> >> ApacheBahirr
> > >> > > > > >> >>> > > >> >> > > > >>> ideology.
> > >> > > > > >> >>> > > >> >> > > > >>> >
> > >> > > > > >> >>> > > >> >> > > > >>> > Also, I agree, that all
> connectors
> > >> could
> > >> > > be
> > >> > > > > >> moved
> > >> > > > > >> >>> there
> > >> > > > > >> >>> > > (in
> > >> > > > > >> >>> > > >> >> > > > >>> ApacheBahirr
> > >> > > > > >> >>> > > >> >> > > > >>> > like repository), but not all
> > modules
> > >> > from
> > >> > > > the
> > >> > > > > >> page
> > >> > > > > >> >>> > > >> >> > > > >>> >
> > >> > > > > >> >>> > > >> >> > > > >>> >
> > >> > > > > >> >>> > > >> >> > > > >>>
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > >
> > >> > > > > >> >>> > > >> >> >
> > >> > > > > >> >>> > > >> >>
> > >> > > > > >> >>> > > >>
> > >> > > > > >> >>> > >
> > >> > > > > >> >>> >
> > >> > > > > >> >>>
> > >> > > > > >>
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > >> > > > > >> >>> > > >> >> > > > >>> > because
> > >> > > > > >> >>> > > >> >> > > > >>> > they use different parts of core
> > >> modules
> > >> > > and
> > >> > > > > >> could
> > >> > > > > >> >>> be
> > >> > > > > >> >>> > > >> >> developed
> > >> > > > > >> >>> > > >> >> > > with
> > >> > > > > >> >>> > > >> >> > > > >>> > different velocity.
> > >> > > > > >> >>> > > >> >> > > > >>> >
> > >> > > > > >> >>> > > >> >> > > > >>> > In reality, before creation of
> new
> > >> > > > > repositories
> > >> > > > > >> we
> > >> > > > > >> >>> need
> > >> > > > > >> >>> > > >> wide
> > >> > > > > >> >>> > > >> >> > > > discussion
> > >> > > > > >> >>> > > >> >> > > > >>> > based on the document mentioned
> > >> above.
> > >> > > > > >> >>> > > >> >> > > > >>> >
> > >> > > > > >> >>> > > >> >> > > > >>> > P.S. Of course, we could
> experiment
> > >> in
> > >> > the
> > >> > > > > next
> > >> > > > > >> >>> release
> > >> > > > > >> >>> > > 2.8
> > >> > > > > >> >>> > > >> >> with
> > >> > > > > >> >>> > > >> >> > > one
> > >> > > > > >> >>> > > >> >> > > > >>> or two
> > >> > > > > >> >>> > > >> >> > > > >>> > integrations like twitter or
> ZeroMQ
> > >> > > > > >> >>> > > >> >> > > > >>> > Also, I'd like Denis Magda idea
> for
> > >> the
> > >> > > > > separate
> > >> > > > > >> >>> > > >> repositories
> > >> > > > > >> >>> > > >> >> in
> > >> > > > > >> >>> > > >> >> > > > Apache
> > >> > > > > >> >>> > > >> >> > > > >>> > Github to maintain them by
> anybody.
> > >> > > > > >> >>> > > >> >> > > > >>> >
> > >> > > > > >> >>> > > >> >> > > > >>> >
> > >> > > > > >> >>> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02,
> Saikat
> > >> > > Maitra <
> > >> > > > > >> >>> > > >> >> > > [hidden email]
> > >> > > > > >> >>> > > >> >> > > > >:
> > >> > > > > >> >>> > > >> >> > > > >>> >
> > >> > > > > >> >>> > > >> >> > > > >>> > > Hi Denis, Emmanouil
> > >> > > > > >> >>> > > >> >> > > > >>> > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > Thank you for your email. I
> think
> > >> > > > creating a
> > >> > > > > >> >>> separate
> > >> > > > > >> >>> > > >> >> > integration
> > >> > > > > >> >>> > > >> >> > > > >>> project
> > >> > > > > >> >>> > > >> >> > > > >>> > > in github and also proposing it
> > as
> > >> an
> > >> > > > apache
> > >> > > > > >> >>> > incubator
> > >> > > > > >> >>> > > >> >> project
> > >> > > > > >> >>> > > >> >> > > will
> > >> > > > > >> >>> > > >> >> > > > >>> be a
> > >> > > > > >> >>> > > >> >> > > > >>> > > good move. The other separate
> > >> > > integration
> > >> > > > > >> modules
> > >> > > > > >> >>> > > >> >> > > > >>> > >
> > >> > > > > >> >>> > > >> >> > > > >>> > >
> > >> > > > > >> >>> > > >> >> > > > >>> >
> > >> > > > > >> >>> > > >> >> > > > >>>
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > >
> > >> > > > > >> >>> > > >> >> >
> > >> > > > > >> >>> > > >> >>
> > >> > > > > >> >>> > > >>
> > >> > > > > >> >>> > >
> > >> > > > > >> >>> >
> > >> > > > > >> >>>
> > >> > > > > >>
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > >> > > > > >> >>> > > >> >> > > > >>> > > can
> > >> > > > > >> >>> > > >> >> > > > >>> > > be moved to this new apache
> > >> incubator
> > >> > > > > project.
> > >> > > > > >> >>> > > >> >> > > > >>> > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > There are similar integration
> > >> > available
> > >> > > > for
> > >> > > > > >> >>> Flink and
> > >> > > > > >> >>> > > >> Spark
> > >> > > > > >> >>> > > >> >> in
> > >> > > > > >> >>> > > >> >> > > > Apache
> > >> > > > > >> >>> > > >> >> > > > >>> > Bahir
> > >> > > > > >> >>> > > >> >> > > > >>> > > https://bahir.apache.org/
> > >> > > > > >> >>> > > >> >> > > > >>> > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > Do you think we should reach
> out
> > to
> > >> > > Apache
> > >> > > > > >> Bahir
> > >> > > > > >> >>> > > >> community
> > >> > > > > >> >>> > > >> >> > with a
> > >> > > > > >> >>> > > >> >> > > > >>> > proposal
> > >> > > > > >> >>> > > >> >> > > > >>> > > or we should plan to create a
> > >> separate
> > >> > > > > Apache
> > >> > > > > >> >>> > Incubator
> > >> > > > > >> >>> > > >> >> > project?
> > >> > > > > >> >>> > > >> >> > > > >>> > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > Regards,
> > >> > > > > >> >>> > > >> >> > > > >>> > > Saikat
> > >> > > > > >> >>> > > >> >> > > > >>> > >
> > >> > > > > >> >>> > > >> >> > > > >>> > >
> > >> > > > > >> >>> > > >> >> > > > >>> > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM
> > >> > > Emmanouil
> > >> > > > > >> >>> > Gkatziouras <
> > >> > > > > >> >>> > > >> >> > > > >>> > > [hidden email]>
> > >> > > > > >> >>> > > >> >> > > > >>> > > wrote:
> > >> > > > > >> >>> > > >> >> > > > >>> > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > Hi all!
> > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > Based on the discussions most
> > >> > probably
> > >> > > > > this
> > >> > > > > >> is
> > >> > > > > >> >>> > going
> > >> > > > > >> >>> > > to
> > >> > > > > >> >>> > > >> >> be on
> > >> > > > > >> >>> > > >> >> > > > >>> another
> > >> > > > > >> >>> > > >> >> > > > >>> > > > GitHub repo. Therefore I will
> > >> > prepare
> > >> > > a
> > >> > > > > >> >>> standalone
> > >> > > > > >> >>> > > >> project
> > >> > > > > >> >>> > > >> >> > with
> > >> > > > > >> >>> > > >> >> > > > the
> > >> > > > > >> >>> > > >> >> > > > >>> > > Pub/Sub
> > >> > > > > >> >>> > > >> >> > > > >>> > > > module and once the
> repository
> > is
> > >> > > > created
> > >> > > > > a
> > >> > > > > >> >>> pull
> > >> > > > > >> >>> > > >> request
> > >> > > > > >> >>> > > >> >> > shall
> > >> > > > > >> >>> > > >> >> > > be
> > >> > > > > >> >>> > > >> >> > > > >>> > issued.
> > >> > > > > >> >>> > > >> >> > > > >>> > > > If there is anything else I
> can
> > >> do
> > >> > in
> > >> > > > > order
> > >> > > > > >> to
> > >> > > > > >> >>> > > >> facilitate
> > >> > > > > >> >>> > > >> >> > this
> > >> > > > > >> >>> > > >> >> > > > >>> please
> > >> > > > > >> >>> > > >> >> > > > >>> > let
> > >> > > > > >> >>> > > >> >> > > > >>> > > > me know.
> > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > If a ticket is created for
> this
> > >> > > issue, I
> > >> > > > > >> shall
> > >> > > > > >> >>> be
> > >> > > > > >> >>> > > able
> > >> > > > > >> >>> > > >> to
> > >> > > > > >> >>> > > >> >> > > create
> > >> > > > > >> >>> > > >> >> > > > a
> > >> > > > > >> >>> > > >> >> > > > >>> pull
> > >> > > > > >> >>> > > >> >> > > > >>> > > > request and thus a build will
> > >> take
> > >> > > place
> > >> > > > > on
> > >> > > > > >> >>> > ignite's
> > >> > > > > >> >>> > > >> CI.
> > >> > > > > >> >>> > > >> >> > > > >>> > > > If everyone is aligned with
> > >> adding a
> > >> > > > > Pub/Sub
> > >> > > > > >> >>> > > >> >> implementation
> > >> > > > > >> >>> > > >> >> > > may I
> > >> > > > > >> >>> > > >> >> > > > >>> > create
> > >> > > > > >> >>> > > >> >> > > > >>> > > a
> > >> > > > > >> >>> > > >> >> > > > >>> > > > ticket on JIRA?
> > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > My next question has to do on
> > how
> > >> > > Ignite
> > >> > > > > >> >>> configures
> > >> > > > > >> >>> > > the
> > >> > > > > >> >>> > > >> >> build
> > >> > > > > >> >>> > > >> >> > > > jobs
> > >> > > > > >> >>> > > >> >> > > > >>> in
> > >> > > > > >> >>> > > >> >> > > > >>> > > Team
> > >> > > > > >> >>> > > >> >> > > > >>> > > > City. The unit tests for my
> > >> > > > implementation
> > >> > > > > >> >>> need a
> > >> > > > > >> >>> > > local
> > >> > > > > >> >>> > > >> >> > Pub/Sub
> > >> > > > > >> >>> > > >> >> > > > >>> server
> > >> > > > > >> >>> > > >> >> > > > >>> > up
> > >> > > > > >> >>> > > >> >> > > > >>> > > > and running (they provide one
> > for
> > >> > > > testing
> > >> > > > > >> >>> > purposes).
> > >> > > > > >> >>> > > >> Will
> > >> > > > > >> >>> > > >> >> > this
> > >> > > > > >> >>> > > >> >> > > > take
> > >> > > > > >> >>> > > >> >> > > > >>> > > effect
> > >> > > > > >> >>> > > >> >> > > > >>> > > > in a form of a build script
> > >> inside
> > >> > my
> > >> > > > > >> >>> > implementation
> > >> > > > > >> >>> > > or
> > >> > > > > >> >>> > > >> >> is it
> > >> > > > > >> >>> > > >> >> > > > >>> something
> > >> > > > > >> >>> > > >> >> > > > >>> > > > that should be configured on
> > Team
> > >> > > City?
> > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > Kind regards,
> > >> > > > > >> >>> > > >> >> > > > >>> > > > Emmanouil
> > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
> > >> > > > > >> >>> > > >> >> > > > >>> > > > https://egkatzioura.com/ |
> > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > >> > > > > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> > > > > >> >>> > > >> >> > > > >>> > > >
> https://github.com/gkatzioura
> > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27,
> > >> Denis
> > >> > > > Magda
> > >> > > > > <
> > >> > > > > >> >>> > > >> >> [hidden email]
> > >> > > > > >> >>> > > >> >> > >
> > >> > > > > >> >>> > > >> >> > > > >>> wrote:
> > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
> > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > After contemplating on
> this,
> > >> let
> > >> > me
> > >> > > > > >> propose
> > >> > > > > >> >>> > another
> > >> > > > > >> >>> > > >> way
> > >> > > > > >> >>> > > >> >> > with
> > >> > > > > >> >>> > > >> >> > > > this
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > particular integration that
> > is
> > >> > > aligned
> > >> > > > > >> with
> > >> > > > > >> >>> our
> > >> > > > > >> >>> > > >> >> > > modularization
> > >> > > > > >> >>> > > >> >> > > > >>> > > endeavor.
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > The modularization [1]
> > defines
> > >> the
> > >> > > > > Ignite
> > >> > > > > >> >>> core,
> > >> > > > > >> >>> > > >> Ignite
> > >> > > > > >> >>> > > >> >> > > modules,
> > >> > > > > >> >>> > > >> >> > > > >>> and
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > independent integrations.
> The
> > >> > first
> > >> > > > two
> > >> > > > > >> are
> > >> > > > > >> >>> to be
> > >> > > > > >> >>> > > >> >> located
> > >> > > > > >> >>> > > >> >> > in
> > >> > > > > >> >>> > > >> >> > > > >>> Ignite
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > repositories and to be
> > >> > > tested/updated
> > >> > > > by
> > >> > > > > >> the
> > >> > > > > >> >>> > > >> community
> > >> > > > > >> >>> > > >> >> for
> > >> > > > > >> >>> > > >> >> > > > every
> > >> > > > > >> >>> > > >> >> > > > >>> > > release.
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > The modules are a subset of
> > >> > existing
> > >> > > > > >> >>> integrations
> > >> > > > > >> >>> > > >> that
> > >> > > > > >> >>> > > >> >> are
> > >> > > > > >> >>> > > >> >> > > > >>> adopted
> > >> > > > > >> >>> > > >> >> > > > >>> > > widely
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > and require community
> > >> attention.
> > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > As for the independent
> > >> > integrations
> > >> > > > [1],
> > >> > > > > >> >>> those
> > >> > > > > >> >>> > are
> > >> > > > > >> >>> > > >> >> supposed
> > >> > > > > >> >>> > > >> >> > > to
> > >> > > > > >> >>> > > >> >> > > > be
> > >> > > > > >> >>> > > >> >> > > > >>> > > located
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > in a separate GitHub repo.
> I
> > >> would
> > >> > > > > advise
> > >> > > > > >> us
> > >> > > > > >> >>> to
> > >> > > > > >> >>> > > start
> > >> > > > > >> >>> > > >> >> > > > >>> implementing
> > >> > > > > >> >>> > > >> >> > > > >>> > this
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > concept with the Pub/Sub
> > >> > > integration.
> > >> > > > > Even
> > >> > > > > >> >>> though
> > >> > > > > >> >>> > > it
> > >> > > > > >> >>> > > >> >> will
> > >> > > > > >> >>> > > >> >> > be
> > >> > > > > >> >>> > > >> >> > > > >>> located
> > >> > > > > >> >>> > > >> >> > > > >>> > > in a
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > separate Github repo, it
> will
> > >> be
> > >> > > > listed
> > >> > > > > >> among
> > >> > > > > >> >>> > > >> officially
> > >> > > > > >> >>> > > >> >> > > > >>> available
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > integrations on the Ignite
> > >> website
> > >> > > > > >> featuring
> > >> > > > > >> >>> > > >> Emmanouil
> > >> > > > > >> >>> > > >> >> as a
> > >> > > > > >> >>> > > >> >> > > > >>> > > contributor.
> > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > What do you think?
> > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > [1]
> > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > >
> > >> > > > > >> >>> > > >> >> > > > >>> >
> > >> > > > > >> >>> > > >> >> > > > >>>
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > >
> > >> > > > > >> >>> > > >> >> >
> > >> > > > > >> >>> > > >> >>
> > >> > > > > >> >>> > > >>
> > >> > > > > >> >>> > >
> > >> > > > > >> >>> >
> > >> > > > > >> >>>
> > >> > > > > >>
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > -
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > Denis
> > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at
> 9:47
> > PM
> > >> > > Saikat
> > >> > > > > >> >>> Maitra <
> > >> > > > > >> >>> > > >> >> > > > >>> > [hidden email]
> > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > wrote:
> > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > Hi Emmanouil,
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > The changes looks good to
> > >> me. I
> > >> > > > wanted
> > >> > > > > >> to
> > >> > > > > >> >>> check
> > >> > > > > >> >>> > > if
> > >> > > > > >> >>> > > >> you
> > >> > > > > >> >>> > > >> >> > are
> > >> > > > > >> >>> > > >> >> > > > also
> > >> > > > > >> >>> > > >> >> > > > >>> > able
> > >> > > > > >> >>> > > >> >> > > > >>> > > to
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > validate that the once
> data
> > >> is
> > >> > > added
> > >> > > > > to
> > >> > > > > >> >>> Ignite
> > >> > > > > >> >>> > > >> cluster
> > >> > > > > >> >>> > > >> >> > you
> > >> > > > > >> >>> > > >> >> > > > are
> > >> > > > > >> >>> > > >> >> > > > >>> also
> > >> > > > > >> >>> > > >> >> > > > >>> > > > able
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > to
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > access it using another
> > >> Ignite
> > >> > > > client
> > >> > > > > or
> > >> > > > > >> >>> using
> > >> > > > > >> >>> > > rest
> > >> > > > > >> >>> > > >> >> > > > endpoints.
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > We use Teamcity for CI
> > >> process,
> > >> > > the
> > >> > > > > >> >>> details are
> > >> > > > > >> >>> > > as
> > >> > > > > >> >>> > > >> >> > follows
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > Trigger validation of
> those
> > >> test
> > >> > > > > suites
> > >> > > > > >> >>> that
> > >> > > > > >> >>> > have
> > >> > > > > >> >>> > > >> been
> > >> > > > > >> >>> > > >> >> > > > >>> affected by
> > >> > > > > >> >>> > > >> >> > > > >>> > > your
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > changes on TeamCity <
> > >> > > > > >> >>> > > http://ci.ignite.apache.org/>:
> > >> > > > > >> >>> > > >> <
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > http://204.14.53.153/
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Locate a test suite
> > you
> > >> > have
> > >> > > to
> > >> > > > > >> check,
> > >> > > > > >> >>> > press
> > >> > > > > >> >>> > > >> >> button
> > >> > > > > >> >>> > > >> >> > > > named
> > >> > > > > >> >>> > > >> >> > > > >>> > "..."
> > >> > > > > >> >>> > > >> >> > > > >>> > > > that
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    is located on the left
> > of
> > >> > "Run"
> > >> > > > > >> button.
> > >> > > > > >> >>> "Run
> > >> > > > > >> >>> > > >> custom
> > >> > > > > >> >>> > > >> >> > > build"
> > >> > > > > >> >>> > > >> >> > > > >>> > dialog
> > >> > > > > >> >>> > > >> >> > > > >>> > > > will
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    appear;
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Go to "Changes" tab
> > and
> > >> > > choose
> > >> > > > > >> >>> > > >> >> > > > >>> > "pull/<pull-request-number>/head"
> > >> > > > > >> >>> > > >> >> > > > >>> > > > in
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    "Build branch"
> dropdown
> > >> list;
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Press "Run build"
> > button
> > >> > and
> > >> > > > > >> monitor
> > >> > > > > >> >>> tests
> > >> > > > > >> >>> > > >> >> results.
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > @Roman [hidden email]
> -
> > >> can
> > >> > > you
> > >> > > > > also
> > >> > > > > >> >>> please
> > >> > > > > >> >>> > > >> take a
> > >> > > > > >> >>> > > >> >> > > look?
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > Regards,
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > Saikat
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at
> > 6:20
> > >> PM
> > >> > > > > >> Emmanouil
> > >> > > > > >> >>> > > >> Gkatziouras
> > >> > > > > >> >>> > > >> >> <
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > [hidden email]>
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > wrote:
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > I have my
> implementation
> > >> on an
> > >> > > > > Ignite
> > >> > > > > >> >>> fork on
> > >> > > > > >> >>> > > my
> > >> > > > > >> >>> > > >> >> > personal
> > >> > > > > >> >>> > > >> >> > > > >>> GitHub
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > account
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > [1]
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > It is based on the
> Kafka
> > >> > > > > >> implementation
> > >> > > > > >> >>> which
> > >> > > > > >> >>> > > is
> > >> > > > > >> >>> > > >> of
> > >> > > > > >> >>> > > >> >> a
> > >> > > > > >> >>> > > >> >> > > > polling
> > >> > > > > >> >>> > > >> >> > > > >>> > > nature.
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > GCP does not provide a
> > >> service
> > >> > > > like
> > >> > > > > >> >>> Kafka.
> > >> > > > > >> >>> > > >> Instead
> > >> > > > > >> >>> > > >> >> it
> > >> > > > > >> >>> > > >> >> > > > >>> provides us
> > >> > > > > >> >>> > > >> >> > > > >>> > > > with
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Pub/Sub which is just
> > like
> > >> > > Kafka.
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Therefore using the
> Kafka
> > >> > > streamer
> > >> > > > > as
> > >> > > > > >> a
> > >> > > > > >> >>> > > guideline
> > >> > > > > >> >>> > > >> >> made
> > >> > > > > >> >>> > > >> >> > > more
> > >> > > > > >> >>> > > >> >> > > > >>> sense
> > >> > > > > >> >>> > > >> >> > > > >>> > > to
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > me.
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Apart from the
> > >> implementation
> > >> > I
> > >> > > > have
> > >> > > > > >> also
> > >> > > > > >> >>> > > added a
> > >> > > > > >> >>> > > >> >> unit
> > >> > > > > >> >>> > > >> >> > > > test.
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > It definitely needs
> some
> > >> > > polishing
> > >> > > > > >> and a
> > >> > > > > >> >>> > double
> > >> > > > > >> >>> > > >> >> check
> > >> > > > > >> >>> > > >> >> > on
> > >> > > > > >> >>> > > >> >> > > my
> > >> > > > > >> >>> > > >> >> > > > >>> side
> > >> > > > > >> >>> > > >> >> > > > >>> > to
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > make
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > the review smooth.
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > For the tests a local
> > >> Pub/Sub
> > >> > > [2]
> > >> > > > > >> server
> > >> > > >
> > >
> > >
> >
>

[Emmanouil Gkatziouras]

Hi Saikat!

Thank you for your assistance on that!

Kind regards

*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Sat, 30 Nov 2019 at 23:51, Saikat Maitra <[hidden email]> wrote:

> Hi Emmanouil,
>
> I looked into the build logs and I observed that since I added artifacts
> dependencies to "Build apache Ignite"[1] it was able to pull the required
> dependencies and was able to run the existing tests. It is however not
> identifying new modules like I changed from flink to flink-ext or pub-sub.
>
> [1]
>
> https://ci.ignite.apache.org/admin/editDependencies.html?id=buildType:IgniteExtensions_Build
>
> I will look into the issue and debug further.
>
> Regards,
> Saikat
>
>
> On Sat, Nov 30, 2019 at 5:35 PM Emmanouil Gkatziouras <
> [hidden email]>
> wrote:
>
> > Hi Saikat!
> >
> > From the logs[1] it seems that TC cannot find the project. However I did
> > include the project on the parent pom (modules sections) [2]
> > I tried to reproduce locally unfortunately I am not aware of the full
> > context on TC. It seems as if a different parent pom is being picked up.
> >
> > Kind regards
> >
> > [1]
> >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4804893&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=1704
> > [2]
> >
> >
> https://github.com/gkatzioura/ignite-extensions/blob/d6a8beee7feff28f59d9a12be692016305564d25/pom.xml#L46
> >
> > *Emmanouil Gkatziouras*
> > https://egkatzioura.com/ |
> > https://www.linkedin.com/in/gkatziourasemmanouil/
> > https://github.com/gkatzioura
> >
> >
> > On Sat, 30 Nov 2019 at 22:20, Saikat Maitra <[hidden email]>
> > wrote:
> >
> > > Hello,
> > >
> > > I have made changes in Ignite Extensions build configurations.
> > >
> > > It is now running the tests as expected.
> > >
> > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4804480&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> > >
> > > Regards,
> > > Saikat
> > >
> > > On Thu, Nov 28, 2019 at 1:29 PM Saikat Maitra <[hidden email]
> >
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > We do run build with -DskipTests and then configure selective test
> > suites
> > > > to run.
> > > >
> > > > I will check why these test suites are not getting executed.
> > > >
> > > > Regards
> > > > Saikat
> > > >
> > > > On Wed, 27 Nov 2019 at 3:05 PM, Emmanouil Gkatziouras <
> > > > [hidden email]> wrote:
> > > >
> > > >> Hi all,
> > > >>
> > > >> I did add a test suite and run the build with the corresponding
> > > arguments,
> > > >> however the tests did not run [1].
> > > >> I checked the Flink's build logs and the same message is displayed
> > there
> > > >> too `No tests to run` [2].
> > > >>
> > > >> [1]
> > > >>
> > > >>
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4798203&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=248
> > > >> [2]
> > > >>
> > > >>
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4789199&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=1256
> > > >>
> > > >> Kind regards
> > > >> *Emmanouil Gkatziouras*
> > > >> https://egkatzioura.com/ |
> > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> https://github.com/gkatzioura
> > > >>
> > > >>
> > > >> On Wed, 27 Nov 2019 at 18:58, Saikat Maitra <
> [hidden email]>
> > > >> wrote:
> > > >>
> > > >> > Hi,
> > > >> >
> > > >> > We will need to add test suites for tests to be executed in build.
> > We
> > > >> pass
> > > >> > them as TestSuites param. I had added Flink sink and source
> > > testsuites.
> > > >> >
> > > >> > Can you please review and confirm.
> > > >> >
> > > >> > Regards
> > > >> > Saikat
> > > >> >
> > > >> > On Wed, 27 Nov 2019 at 6:00 AM, Emmanouil Gkatziouras <
> > > >> > [hidden email]>
> > > >> > wrote:
> > > >> >
> > > >> > > Hi all!
> > > >> > >
> > > >> > > I was successful building and running my pull requests. It seems
> > > that
> > > >> the
> > > >> > > tests do not run both for Flink and Pub/Sub [1]
> > > >> > > If there is something I cannot do to make them discoverable
> please
> > > >> let me
> > > >> > > know.
> > > >> > >
> > > >> > > [1]
> > > >> > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4796722&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=283
> > > >> > >
> > > >> > > *Emmanouil Gkatziouras*
> > > >> > > https://egkatzioura.com/ |
> > > >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> > > https://github.com/gkatzioura
> > > >> > >
> > > >> > >
> > > >> > > On Wed, 27 Nov 2019 at 04:57, Saikat Maitra <
> > > [hidden email]>
> > > >> > > wrote:
> > > >> > >
> > > >> > > > Hi Emmanouil,
> > > >> > > >
> > > >> > > > I have added you as contributor in Ignite Extensions project.
> > Can
> > > >> you
> > > >> > > > please check and confirm if you are able to see the project
> and
> > > >> execute
> > > >> > > > build on your pull request.
> > > >> > > >
> > > >> > > > Regards,
> > > >> > > > Saikat
> > > >> > > >
> > > >> > > > On Tue, Nov 26, 2019 at 12:49 PM Emmanouil Gkatziouras <
> > > >> > > > [hidden email]>
> > > >> > > > wrote:
> > > >> > > >
> > > >> > > > > Hi all!
> > > >> > > > >
> > > >> > > > > If someone could give me read access to the Job of Ignite
> > > >> Extensions
> > > >> > on
> > > >> > > > > Team City will greatly help me (username gkatzioura).
> > > >> > > > > I suppose the builds get triggered automatically on a pull
> > > >> request.
> > > >> > > > >
> > > >> > > > > Kind regards,
> > > >> > > > > Emmanouil
> > > >> > > > >
> > > >> > > > > *Emmanouil Gkatziouras*
> > > >> > > > > https://egkatzioura.com/ |
> > > >> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> > > > > https://github.com/gkatzioura
> > > >> > > > >
> > > >> > > > >
> > > >> > > > > On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras <
> > > >> > > > [hidden email]>
> > > >> > > > > wrote:
> > > >> > > > >
> > > >> > > > > > Hi Saikat!
> > > >> > > > > >
> > > >> > > > > > I just rebased with the flink branch. Unfortunately I do
> not
> > > >> have
> > > >> > > read
> > > >> > > > > > access to the team city link you provided and thus
> evaluate
> > > the
> > > >> > tests
> > > >> > > > for
> > > >> > > > > > my pull request.
> > > >> > > > > > I guess it has to do with ignite extensions being new.
> > > >> > > > > > My username is gkatzioura and email the one I am using on
> > this
> > > >> > email
> > > >> > > > > >
> > > >> > > > > > Thank you for your time.
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > > *Emmanouil Gkatziouras*
> > > >> > > > > > https://egkatzioura.com/ |
> > > >> > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> > > > > > https://github.com/gkatzioura
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <
> > > >> > [hidden email]
> > > >> > > >
> > > >> > > > > > wrote:
> > > >> > > > > >
> > > >> > > > > >> Hi Emmanouil,
> > > >> > > > > >>
> > > >> > > > > >> The latest build on teamcity has passed on Flink pull
> > > request.
> > > >> > > > > >>
> > > >> > > > > >>
> > > >> > > > > >>
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> > > >> > > > > >>
> > > >> > > > > >> you should be able to take the changes I made in my PR
> and
> > > run
> > > >> > build
> > > >> > > > on
> > > >> > > > > >> your pull request.
> > > >> > > > > >>
> > > >> > > > > >> Please let me know if you have any questions.
> > > >> > > > > >>
> > > >> > > > > >> Regards,
> > > >> > > > > >> Saikat
> > > >> > > > > >>
> > > >> > > > > >> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <
> > > >> > > > [hidden email]
> > > >> > > > > >
> > > >> > > > > >> wrote:
> > > >> > > > > >>
> > > >> > > > > >> > Hi Emmanouil,
> > > >> > > > > >> >
> > > >> > > > > >> > I have fixed the Flink module testsuites and have setup
> > new
> > > >> > > project
> > > >> > > > in
> > > >> > > > > >> > teamcity.
> > > >> > > > > >> >
> > > >> > > > > >> >
> > > >> > > > > >> >
> > > >> > > > > >>
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
> > > >> > > > > >> >
> > > >> > > > > >> > I am looking into setting up the build.
> > > >> > > > > >> >
> > > >> > > > > >> > Regards,
> > > >> > > > > >> > Saikat
> > > >> > > > > >> >
> > > >> > > > > >> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <
> > > >> > > > > [hidden email]>
> > > >> > > > > >> > wrote:
> > > >> > > > > >> >
> > > >> > > > > >> >> Hi Emmanouil,
> > > >> > > > > >> >>
> > > >> > > > > >> >> Thank you for your email. Yes, the plan is once the
> > Flink
> > > >> PR is
> > > >> > > > > merged
> > > >> > > > > >> it
> > > >> > > > > >> >> will provide the Licence, parent POMs etc and you can
> > > rebase
> > > >> > from
> > > >> > > > > >> master
> > > >> > > > > >> >> branch and apply the changes on top of it.
> > > >> > > > > >> >>
> > > >> > > > > >> >> I am getting some issues with test failures in local
> > with
> > > >> > > > > >> >> GridTestProperties as the test.properties is not
> present
> > > in
> > > >> > this
> > > >> > > > new
> > > >> > > > > >> >> project but should be available from dependencies.
> > > >> > > > > >> >>
> > > >> > > > > >> >> Once I address this issue, I will go ahead and merge
> the
> > > >> > changes
> > > >> > > > and
> > > >> > > > > >> then
> > > >> > > > > >> >> we can take it from there.
> > > >> > > > > >> >>
> > > >> > > > > >> >> Regards,
> > > >> > > > > >> >> Saikat
> > > >> > > > > >> >>
> > > >> > > > > >> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras
> <
> > > >> > > > > >> >> [hidden email]> wrote:
> > > >> > > > > >> >>
> > > >> > > > > >> >>> Hi all,
> > > >> > > > > >> >>>
> > > >> > > > > >> >>> I made my first pull request [1]. Since this project
> is
> > > >> brand
> > > >> > > new
> > > >> > > > > (no
> > > >> > > > > >> >>> parent poms, licensing), I reckoned it was best to
> use
> > > >> > Saikat's
> > > >> > > > > >> branch on
> > > >> > > > > >> >>> Flink.
> > > >> > > > > >> >>> I suppose the Flink branch will be merged first. If
> not
> > > >> please
> > > >> > > > give
> > > >> > > > > me
> > > >> > > > > >> >>> guidelines on how I should proceed next.
> > > >> > > > > >> >>>
> > > >> > > > > >> >>> Kind regards
> > > >> > > > > >> >>> Emmanouil
> > > >> > > > > >> >>>
> > > >> > > > > >> >>> [1]
> https://github.com/apache/ignite-extensions/pull/2
> > > >> > > > > >> >>>
> > > >> > > > > >> >>> *Emmanouil Gkatziouras*
> > > >> > > > > >> >>> https://egkatzioura.com/ |
> > > >> > > > > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> > > > > >> >>> https://github.com/gkatzioura
> > > >> > > > > >> >>>
> > > >> > > > > >> >>>
> > > >> > > > > >> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <
> > > >> [hidden email]>
> > > >> > > > > wrote:
> > > >> > > > > >> >>>
> > > >> > > > > >> >>> > Awesome, ping us whenever you're ready!
> > > >> > > > > >> >>> >
> > > >> > > > > >> >>> > -
> > > >> > > > > >> >>> > Denis
> > > >> > > > > >> >>> >
> > > >> > > > > >> >>> >
> > > >> > > > > >> >>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil
> > Gkatziouras
> > > <
> > > >> > > > > >> >>> > [hidden email]>
> > > >> > > > > >> >>> > wrote:
> > > >> > > > > >> >>> >
> > > >> > > > > >> >>> > > Hi all!
> > > >> > > > > >> >>> > >
> > > >> > > > > >> >>> > > I am sorry for being late on that. I was trying
> to
> > > >> > refactor
> > > >> > > > the
> > > >> > > > > >> test
> > > >> > > > > >> >>> in
> > > >> > > > > >> >>> > > order not to be in need of any external tools or
> > > >> spinning
> > > >> > > up a
> > > >> > > > > >> >>> server.
> > > >> > > > > >> >>> > > I did forked the new repo and indeed my changes
> > > there,
> > > >> so
> > > >> > a
> > > >> > > > pull
> > > >> > > > > >> >>> request
> > > >> > > > > >> >>> > is
> > > >> > > > > >> >>> > > a matter of time!
> > > >> > > > > >> >>> > >
> > > >> > > > > >> >>> > > Kind regards
> > > >> > > > > >> >>> > >
> > > >> > > > > >> >>> > > *Emmanouil Gkatziouras*
> > > >> > > > > >> >>> > > https://egkatzioura.com/ |
> > > >> > > > > >> >>> > >
> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> > > > > >> >>> > > https://github.com/gkatzioura
> > > >> > > > > >> >>> > >
> > > >> > > > > >> >>> > >
> > > >> > > > > >> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <
> > > >> > > [hidden email]>
> > > >> > > > > >> wrote:
> > > >> > > > > >> >>> > >
> > > >> > > > > >> >>> > > > Hi Emmanouil,
> > > >> > > > > >> >>> > > >
> > > >> > > > > >> >>> > > > Do you have any questions or need any support
> > from
> > > >> the
> > > >> > > > > >> community?
> > > >> > > > > >> >>> > > >
> > > >> > > > > >> >>> > > > -
> > > >> > > > > >> >>> > > > Denis
> > > >> > > > > >> >>> > > >
> > > >> > > > > >> >>> > > >
> > > >> > > > > >> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
> > > >> > > > > >> >>> [hidden email]
> > > >> > > > > >> >>> > >
> > > >> > > > > >> >>> > > > wrote:
> > > >> > > > > >> >>> > > >
> > > >> > > > > >> >>> > > >> Hi Emmanouil,
> > > >> > > > > >> >>> > > >>
> > > >> > > > > >> >>> > > >> Can you please take a looks at dev utils, if
> > this
> > > is
> > > >> > > > > something
> > > >> > > > > >> >>> you are
> > > >> > > > > >> >>> > > >> looking for
> > > >> > > > > >> >>> > > >>
> > > >> > > > >
> > https://github.com/apache/ignite/tree/master/modules/dev-utils
> > > >> > > > > >> ?
> > > >> > > > > >> >>> > > >>
> > > >> > > > > >> >>> > > >> IMO, if you can release Pub/Sub server in
> maven
> > > and
> > > >> > then
> > > >> > > > use
> > > >> > > > > >> it as
> > > >> > > > > >> >>> > > >> dependency, that would be great.
> > > >> > > > > >> >>> > > >>
> > > >> > > > > >> >>> > > >> Regards,
> > > >> > > > > >> >>> > > >> Saikat
> > > >> > > > > >> >>> > > >>
> > > >> > > > > >> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra
> <
> > > >> > > > > >> >>> > [hidden email]>
> > > >> > > > > >> >>> > > >> wrote:
> > > >> > > > > >> >>> > > >>
> > > >> > > > > >> >>> > > >> > Hi Emmanouil,
> > > >> > > > > >> >>> > > >> >
> > > >> > > > > >> >>> > > >> > The master branch has been initialized. I
> have
> > > >> > raised a
> > > >> > > > PR
> > > >> > > > > to
> > > >> > > > > >> >>> > migrate
> > > >> > > > > >> >>> > > >> > Flink module to ignite-extensions.
> > > >> > > > > >> >>> > > >> >
> > > >> > > > > >> >>> > > >> > PR
> > > >> > https://github.com/apache/ignite-extensions/pull/1
> > > >> > > > > >> >>> > > >> > Jira :
> > > >> > > > https://issues.apache.org/jira/browse/IGNITE-12356
> > > >> > > > > >> >>> > > >> >
> > > >> > > > > >> >>> > > >> > Regards,
> > > >> > > > > >> >>> > > >> > Saikat
> > > >> > > > > >> >>> > > >> >
> > > >> > > > > >> >>> > > >> >
> > > >> > > > > >> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil
> > > >> > Gkatziouras <
> > > >> > > > > >> >>> > > >> > [hidden email]> wrote:
> > > >> > > > > >> >>> > > >> >
> > > >> > > > > >> >>> > > >> >> Hi all!
> > > >> > > > > >> >>> > > >> >>
> > > >> > > > > >> >>> > > >> >> This is an update. The integration is
> tested
> > > with
> > > >> > > mocks
> > > >> > > > > >> based
> > > >> > > > > >> >>> on
> > > >> > > > > >> >>> > the
> > > >> > > > > >> >>> > > >> >> original Ignite repo [1].
> > > >> > > > > >> >>> > > >> >> I am currently waiting for the first
> commits
> > on
> > > >> the
> > > >> > > new
> > > >> > > > > >> >>> integration
> > > >> > > > > >> >>> > > >> >> project
> > > >> > > > > >> >>> > > >> >> to take place in order to create my pull
> > > request
> > > >> > > > > accordingly
> > > >> > > > > >> >>> to the
> > > >> > > > > >> >>> > > >> >> initial
> > > >> > > > > >> >>> > > >> >> commits.
> > > >> > > > > >> >>> > > >> >> In order to simulate the Pub/Sub server in
> a
> > > >> test I
> > > >> > > > used a
> > > >> > > > > >> test
> > > >> > > > > >> >>> > > server
> > > >> > > > > >> >>> > > >> >> implementation [2]. Unfortunately they
> don't
> > > have
> > > >> > this
> > > >> > > > > util
> > > >> > > > > >> >>> > uploaded
> > > >> > > > > >> >>> > > >> in a
> > > >> > > > > >> >>> > > >> >> maven repo, thus it needs to be hosted on a
> > > maven
> > > >> > repo
> > > >> > > > > >> (Apache
> > > >> > > > > >> >>> > > License
> > > >> > > > > >> >>> > > >> >> 2.0). Is there a repo for utilities that
> > Ignite
> > > >> uses
> > > >> > > for
> > > >> > > > > the
> > > >> > > > > >> >>> > > >> test/build?
> > > >> > > > > >> >>> > > >> >> I have also created a ticket [3]
> > > >> > > > > >> >>> > > >> >>
> > > >> > > > > >> >>> > > >> >> Kind regards
> > > >> > > > > >> >>> > > >> >>
> > > >> > > > > >> >>> > > >> >> [1]
> > > >> > > > > >> >>> >
> > > >> > > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > > >> > > > > >> >>> > > >> >> [2]
> > > >> > > > > >> >>>
> > > >> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> > > >> > > > > >> >>> > > >> >> [3]
> > > >> > > https://issues.apache.org/jira/browse/IGNITE-12262
> > > >> > > > > >> >>> > > >> >>
> > > >> > > > > >> >>> > > >> >> *Emmanouil Gkatziouras*
> > > >> > > > > >> >>> > > >> >> https://egkatzioura.com/ |
> > > >> > > > > >> >>> > > >> >>
> > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> > > > > >> >>> > > >> >> https://github.com/gkatzioura
> > > >> > > > > >> >>> > > >> >>
> > > >> > > > > >> >>> > > >> >>
> > > >> > > > > >> >>> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat
> Maitra <
> > > >> > > > > >> >>> > [hidden email]
> > > >> > > > > >> >>> > > >
> > > >> > > > > >> >>> > > >> >> wrote:
> > > >> > > > > >> >>> > > >> >>
> > > >> > > > > >> >>> > > >> >> > Hello Ilya,
> > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > >> >>> > > >> >> > Thank you for your email. Yes, before we
> > > remove
> > > >> > any
> > > >> > > > > >> >>> integration
> > > >> > > > > >> >>> > we
> > > >> > > > > >> >>> > > >> will
> > > >> > > > > >> >>> > > >> >> > move them to another repository and
> > release.
> > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > >> >>> > > >> >> > We are thinking we will start with
> Pub/Sub
> > > >> > Streamer
> > > >> > > > > >> >>> > implementation
> > > >> > > > > >> >>> > > >> and
> > > >> > > > > >> >>> > > >> >> > continue to move other integration as
> > > >> mentioned in
> > > >> > > the
> > > >> > > > > >> >>> following
> > > >> > > > > >> >>> > > doc.
> > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > >> >>> > > >> >>
> > > >> > > > > >> >>> > > >>
> > > >> > > > > >> >>> > >
> > > >> > > > > >> >>> >
> > > >> > > > > >> >>>
> > > >> > > > > >>
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > >> >>> > > >> >> > Regards,
> > > >> > > > > >> >>> > > >> >> > Saikat
> > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > >> >>> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya
> > > >> Kasnacheev <
> > > >> > > > > >> >>> > > >> >> > [hidden email]>
> > > >> > > > > >> >>> > > >> >> > wrote:
> > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > >> >>> > > >> >> > > Hello!
> > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > >> >>> > > >> >> > > My take on this: we want to offload
> some
> > of
> > > >> > > > > >> integrations,
> > > >> > > > > >> >>> but
> > > >> > > > > >> >>> > > >> please
> > > >> > > > > >> >>> > > >> >> > commit
> > > >> > > > > >> >>> > > >> >> > > and release them to another project
> > first,
> > > >> and
> > > >> > > only
> > > >> > > > > >> then we
> > > >> > > > > >> >>> > will
> > > >> > > > > >> >>> > > >> >> decide
> > > >> > > > > >> >>> > > >> >> > to
> > > >> > > > > >> >>> > > >> >> > > remove them from our repository.
> > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > >> >>> > > >> >> > > Please also make sure they are
> available
> > > with
> > > >> > same
> > > >> > > > > >> artifact
> > > >> > > > > >> >>> > name
> > > >> > > > > >> >>> > > >> and
> > > >> > > > > >> >>> > > >> >> API.
> > > >> > > > > >> >>> > > >> >> > > If this has to be changed, let's wait
> for
> > > AI
> > > >> 3.0
> > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > >> >>> > > >> >> > > Regards,
> > > >> > > > > >> >>> > > >> >> > > --
> > > >> > > > > >> >>> > > >> >> > > Ilya Kasnacheev
> > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > >> >>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat
> > Maitra
> > > <
> > > >> > > > > >> >>> > > >> [hidden email]>:
> > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > >> >>> > > >> >> > > > Hello Emmanouil,
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > > As discussed earlier, I discussed
> with
> > > >> Apache
> > > >> > > > Bahir
> > > >> > > > > >> >>> community
> > > >> > > > > >> >>> > > and
> > > >> > > > > >> >>> > > >> >> they
> > > >> > > > > >> >>> > > >> >> > > are
> > > >> > > > > >> >>> > > >> >> > > > interested to have Apache Ignite
> > > >> extensions as
> > > >> > > > part
> > > >> > > > > of
> > > >> > > > > >> >>> Apache
> > > >> > > > > >> >>> > > >> Bahir
> > > >> > > > > >> >>> > > >> >> > > > project.
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > > Can you please share Pub/Sub streamer
> > > >> > > > implementation
> > > >> > > > > >> >>> details
> > > >> > > > > >> >>> > > with
> > > >> > > > > >> >>> > > >> >> > Apache
> > > >> > > > > >> >>> > > >> >> > > > Bahir community and request for
> > feedback.
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > > https://bahir.apache.org/
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > > I have also requested for contributor
> > > >> access
> > > >> > in
> > > >> > > > Jira
> > > >> > > > > >> for
> > > >> > > > > >> >>> > Apache
> > > >> > > > > >> >>> > > >> >> Bahir
> > > >> > > > > >> >>> > > >> >> > > > project so that I can create issues
> and
> > > >> assign
> > > >> > > to
> > > >> > > > > >> myself.
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > > I can help with code reviews as well.
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > > Here is the email thread for
> reference
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > >
> > > >> > > >
> https://www.mail-archive.com/dev@.../msg02703.html
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > > Regards,
> > > >> > > > > >> >>> > > >> >> > > > Saikat
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM
> Saikat
> > > >> Maitra
> > > >> > <
> > > >> > > > > >> >>> > > >> >> [hidden email]
> > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > >> >>> > > >> >> > > > wrote:
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > > > Hello,
> > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > >> >>> > > >> >> > > > > As discussed I have created
> following
> > > >> > > discussion
> > > >> > > > > >> >>> threads on
> > > >> > > > > >> >>> > > our
> > > >> > > > > >> >>> > > >> >> > > migration
> > > >> > > > > >> >>> > > >> >> > > > > proposals for Apache Ignite
> > extensions:
> > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > >> >>> > > >> >>
> > > >> > > > > >> >>> > > >>
> > > >> > > > > >> >>> > >
> > > >> > > > > >> >>> >
> > > >> > > > > >> >>>
> > > >> > > > > >>
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > >> >>> > > >>
> > > >> > > > > >>
> > > >> https://www.mail-archive.com/dev@.../msg02703.html
> > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > >> >>> > > >> >> > > > > I will share update as I hear more
> > > >> > > information.
> > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > >> >>> > > >> >> > > > > Regards,
> > > >> > > > > >> >>> > > >> >> > > > > Saikat
> > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > >> >>> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM
> > Saikat
> > > >> > Maitra
> > > >> > > <
> > > >> > > > > >> >>> > > >> >> > [hidden email]
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > > > wrote:
> > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >> Hello Denis,
> > > >> > > > > >> >>> > > >> >> > > > >>
> > > >> > > > > >> >>> > > >> >> > > > >> Sure, sounds good. I will create a
> > > >> separate
> > > >> > > > > >> discussion
> > > >> > > > > >> >>> > > thread
> > > >> > > > > >> >>> > > >> to
> > > >> > > > > >> >>> > > >> >> get
> > > >> > > > > >> >>> > > >> >> > > > >> community feedback on whether we
> > > >> > > > > >> >>> > > >> >> > > > >> should join the Bahir or kick-off
> > > >> "Ignite
> > > >> > > > > >> Extensions"
> > > >> > > > > >> >>> > > project.
> > > >> > > > > >> >>> > > >> >> > > > >>
> > > >> > > > > >> >>> > > >> >> > > > >> Regards,
> > > >> > > > > >> >>> > > >> >> > > > >> Saikat
> > > >> > > > > >> >>> > > >> >> > > > >>
> > > >> > > > > >> >>> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM
> > Denis
> > > >> > Magda <
> > > >> > > > > >> >>> > > >> [hidden email]>
> > > >> > > > > >> >>> > > >> >> > > wrote:
> > > >> > > > > >> >>> > > >> >> > > > >>
> > > >> > > > > >> >>> > > >> >> > > > >>> Folks,
> > > >> > > > > >> >>> > > >> >> > > > >>>
> > > >> > > > > >> >>> > > >> >> > > > >>> The concept of Apache Bahir is
> > > >> precisely
> > > >> > > what
> > > >> > > > we
> > > >> > > > > >> >>> need!
> > > >> > > > > >> >>> > > >> Saikat,
> > > >> > > > > >> >>> > > >> >> > thanks
> > > >> > > > > >> >>> > > >> >> > > > for
> > > >> > > > > >> >>> > > >> >> > > > >>> doing the research. Why I believe
> > > the *
> > > >> > > idea*
> > > >> > > > > >> fits us
> > > >> > > > > >> >>> > well:
> > > >> > > > > >> >>> > > >> >> > > > >>>
> > > >> > > > > >> >>> > > >> >> > > > >>>    - All integrations can be
> stored
> > > in
> > > >> > > > separate
> > > >> > > > > >> >>> Github
> > > >> > > > > >> >>> > > >> >> repositories
> > > >> > > > > >> >>> > > >> >> > > and
> > > >> > > > > >> >>> > > >> >> > > > >>>    have their dev lifecycles.
> We've
> > > not
> > > >> > > > obliged
> > > >> > > > > to
> > > >> > > > > >> >>> couple
> > > >> > > > > >> >>> > > all
> > > >> > > > > >> >>> > > >> >> the
> > > >> > > > > >> >>> > > >> >> > > > >>> integrations
> > > >> > > > > >> >>> > > >> >> > > > >>>    in a single repo. For
> instance,
> > > >> Spark
> > > >> > can
> > > >> > > > be
> > > >> > > > > >> >>> located
> > > >> > > > > >> >>> > in
> > > >> > > > > >> >>> > > a
> > > >> > > > > >> >>> > > >> >> > > dedicated
> > > >> > > > > >> >>> > > >> >> > > > >>> repo
> > > >> > > > > >> >>> > > >> >> > > > >>>    while streaming integrations
> > might
> > > >> be
> > > >> > in
> > > >> > > a
> > > >> > > > > >> single
> > > >> > > > > >> >>> one.
> > > >> > > > > >> >>> > > >> It's
> > > >> > > > > >> >>> > > >> >> up
> > > >> > > > > >> >>> > > >> >> > to
> > > >> > > > > >> >>> > > >> >> > > > us.
> > > >> > > > > >> >>> > > >> >> > > > >>>    - All the repositories and
> > > >> integrations
> > > >> > > > > belong
> > > >> > > > > >> to
> > > >> > > > > >> >>> ASF.
> > > >> > > > > >> >>> > > >> We're
> > > >> > > > > >> >>> > > >> >> not
> > > >> > > > > >> >>> > > >> >> > > > >>> dumping
> > > >> > > > > >> >>> > > >> >> > > > >>>    them on Github but rather keep
> > > >> > supporting
> > > >> > > > and
> > > >> > > > > >> >>> > developing
> > > >> > > > > >> >>> > > >> in
> > > >> > > > > >> >>> > > >> >> > > > >>> accordance with
> > > >> > > > > >> >>> > > >> >> > > > >>>    ASF vision and practices.
> > > >> > > > > >> >>> > > >> >> > > > >>>    - There is a way to reward
> > > >> contributors
> > > >> > > via
> > > >> > > > > >> >>> > > committership
> > > >> > > > > >> >>> > > >> and
> > > >> > > > > >> >>> > > >> >> > PMC
> > > >> > > > > >> >>> > > >> >> > > > >>>    membership. You won't get this
> > if
> > > a
> > > >> > > project
> > > >> > > > > is
> > > >> > > > > >> >>> just
> > > >> > > > > >> >>> > one
> > > >> > > > > >> >>> > > of
> > > >> > > > > >> >>> > > >> >> the
> > > >> > > > > >> >>> > > >> >> > > > >>> millions of
> > > >> > > > > >> >>> > > >> >> > > > >>>    Github projects.
> > > >> > > > > >> >>> > > >> >> > > > >>>
> > > >> > > > > >> >>> > > >> >> > > > >>> Saikat, as Ignite PMC member,
> could
> > > you
> > > >> > > please
> > > >> > > > > >> >>> kick-off a
> > > >> > > > > >> >>> > > >> >> separate
> > > >> > > > > >> >>> > > >> >> > > > >>> dev-list
> > > >> > > > > >> >>> > > >> >> > > > >>> discussion to involve more
> > community
> > > >> > members
> > > >> > > > and
> > > >> > > > > >> >>> > referring
> > > >> > > > > >> >>> > > to
> > > >> > > > > >> >>> > > >> >> this
> > > >> > > > > >> >>> > > >> >> > > > >>> thread?
> > > >> > > > > >> >>> > > >> >> > > > >>> I think the primary question the
> > > >> community
> > > >> > > > needs
> > > >> > > > > >> to
> > > >> > > > > >> >>> > answer
> > > >> > > > > >> >>> > > >> >> whether
> > > >> > > > > >> >>> > > >> >> > we
> > > >> > > > > >> >>> > > >> >> > > > >>> should join the Bahir or kick-off
> > > >> "Ignite
> > > >> > > > > >> Extensions"
> > > >> > > > > >> >>> > > >> project?
> > > >> > > > > >> >>> > > >> >> > > > >>>
> > > >> > > > > >> >>> > > >> >> > > > >>> -
> > > >> > > > > >> >>> > > >> >> > > > >>> Denis
> > > >> > > > > >> >>> > > >> >> > > > >>>
> > > >> > > > > >> >>> > > >> >> > > > >>>
> > > >> > > > > >> >>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM
> > > Alexey
> > > >> > > > Zinoviev
> > > >> > > > > <
> > > >> > > > > >> >>> > > >> >> > > > [hidden email]>
> > > >> > > > > >> >>> > > >> >> > > > >>> wrote:
> > > >> > > > > >> >>> > > >> >> > > > >>>
> > > >> > > > > >> >>> > > >> >> > > > >>> > Maybe we could move all our
> > > Streaming
> > > >> > > > > >> Integrations
> > > >> > > > > >> >>> > there,
> > > >> > > > > >> >>> > > >> but
> > > >> > > > > >> >>> > > >> >> > what
> > > >> > > > > >> >>> > > >> >> > > is
> > > >> > > > > >> >>> > > >> >> > > > >>> about
> > > >> > > > > >> >>> > > >> >> > > > >>> > maintaining and committer
> > > >> permissions to
> > > >> > > the
> > > >> > > > > new
> > > >> > > > > >> >>> > > >> repositories?
> > > >> > > > > >> >>> > > >> >> > > > >>> > I see the list of the
> committers
> > > and
> > > >> PMC
> > > >> > > > > members
> > > >> > > > > >> >>> there
> > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > >> > > https://bahir.apache.org/community-members/
> > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > >> > > > > >> >>> > > >> >> > > > >>> > Could somebody from Ignite
> > > community
> > > >> be
> > > >> > > > added
> > > >> > > > > to
> > > >> > > > > >> >>> this
> > > >> > > > > >> >>> > > list
> > > >> > > > > >> >>> > > >> as
> > > >> > > > > >> >>> > > >> >> a
> > > >> > > > > >> >>> > > >> >> > > > >>> > PMC/committer to maintain
> Ignite
> > > >> > > > integrations?
> > > >> > > > > >> >>> > > >> >> > > > >>> > If yes, I'd happy to join this
> > > >> project
> > > >> > and
> > > >> > > > > >> >>> collaborate
> > > >> > > > > >> >>> > > with
> > > >> > > > > >> >>> > > >> >> these
> > > >> > > > > >> >>> > > >> >> > > > guys
> > > >> > > > > >> >>> > > >> >> > > > >>> > together
> > > >> > > > > >> >>> > > >> >> > > > >>> > If no, and we should start from
> > the
> > > >> zero
> > > >> > > > level
> > > >> > > > > >> with
> > > >> > > > > >> >>> > > >> external
> > > >> > > > > >> >>> > > >> >> PRs
> > > >> > > > > >> >>> > > >> >> > -
> > > >> > > > > >> >>> > > >> >> > > > >>> hmmm,
> > > >> > > > > >> >>> > > >> >> > > > >>> > it's better to have our own
> > > external
> > > >> > > > > repository
> > > >> > > > > >> >>> with
> > > >> > > > > >> >>> > > >> >> ApacheBahirr
> > > >> > > > > >> >>> > > >> >> > > > >>> ideology.
> > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > >> > > > > >> >>> > > >> >> > > > >>> > Also, I agree, that all
> > connectors
> > > >> could
> > > >> > > be
> > > >> > > > > >> moved
> > > >> > > > > >> >>> there
> > > >> > > > > >> >>> > > (in
> > > >> > > > > >> >>> > > >> >> > > > >>> ApacheBahirr
> > > >> > > > > >> >>> > > >> >> > > > >>> > like repository), but not all
> > > modules
> > > >> > from
> > > >> > > > the
> > > >> > > > > >> page
> > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > >> > > > > >> >>> > > >> >> > > > >>>
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > >> >>> > > >> >>
> > > >> > > > > >> >>> > > >>
> > > >> > > > > >> >>> > >
> > > >> > > > > >> >>> >
> > > >> > > > > >> >>>
> > > >> > > > > >>
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > >> > > > > >> >>> > > >> >> > > > >>> > because
> > > >> > > > > >> >>> > > >> >> > > > >>> > they use different parts of
> core
> > > >> modules
> > > >> > > and
> > > >> > > > > >> could
> > > >> > > > > >> >>> be
> > > >> > > > > >> >>> > > >> >> developed
> > > >> > > > > >> >>> > > >> >> > > with
> > > >> > > > > >> >>> > > >> >> > > > >>> > different velocity.
> > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > >> > > > > >> >>> > > >> >> > > > >>> > In reality, before creation of
> > new
> > > >> > > > > repositories
> > > >> > > > > >> we
> > > >> > > > > >> >>> need
> > > >> > > > > >> >>> > > >> wide
> > > >> > > > > >> >>> > > >> >> > > > discussion
> > > >> > > > > >> >>> > > >> >> > > > >>> > based on the document mentioned
> > > >> above.
> > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > >> > > > > >> >>> > > >> >> > > > >>> > P.S. Of course, we could
> > experiment
> > > >> in
> > > >> > the
> > > >> > > > > next
> > > >> > > > > >> >>> release
> > > >> > > > > >> >>> > > 2.8
> > > >> > > > > >> >>> > > >> >> with
> > > >> > > > > >> >>> > > >> >> > > one
> > > >> > > > > >> >>> > > >> >> > > > >>> or two
> > > >> > > > > >> >>> > > >> >> > > > >>> > integrations like twitter or
> > ZeroMQ
> > > >> > > > > >> >>> > > >> >> > > > >>> > Also, I'd like Denis Magda idea
> > for
> > > >> the
> > > >> > > > > separate
> > > >> > > > > >> >>> > > >> repositories
> > > >> > > > > >> >>> > > >> >> in
> > > >> > > > > >> >>> > > >> >> > > > Apache
> > > >> > > > > >> >>> > > >> >> > > > >>> > Github to maintain them by
> > anybody.
> > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > >> > > > > >> >>> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02,
> > Saikat
> > > >> > > Maitra <
> > > >> > > > > >> >>> > > >> >> > > [hidden email]
> > > >> > > > > >> >>> > > >> >> > > > >:
> > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > Hi Denis, Emmanouil
> > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > Thank you for your email. I
> > think
> > > >> > > > creating a
> > > >> > > > > >> >>> separate
> > > >> > > > > >> >>> > > >> >> > integration
> > > >> > > > > >> >>> > > >> >> > > > >>> project
> > > >> > > > > >> >>> > > >> >> > > > >>> > > in github and also proposing
> it
> > > as
> > > >> an
> > > >> > > > apache
> > > >> > > > > >> >>> > incubator
> > > >> > > > > >> >>> > > >> >> project
> > > >> > > > > >> >>> > > >> >> > > will
> > > >> > > > > >> >>> > > >> >> > > > >>> be a
> > > >> > > > > >> >>> > > >> >> > > > >>> > > good move. The other separate
> > > >> > > integration
> > > >> > > > > >> modules
> > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > >> > > > > >> >>> > > >> >> > > > >>>
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > >> >>> > > >> >>
> > > >> > > > > >> >>> > > >>
> > > >> > > > > >> >>> > >
> > > >> > > > > >> >>> >
> > > >> > > > > >> >>>
> > > >> > > > > >>
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > >> > > > > >> >>> > > >> >> > > > >>> > > can
> > > >> > > > > >> >>> > > >> >> > > > >>> > > be moved to this new apache
> > > >> incubator
> > > >> > > > > project.
> > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > There are similar integration
> > > >> > available
> > > >> > > > for
> > > >> > > > > >> >>> Flink and
> > > >> > > > > >> >>> > > >> Spark
> > > >> > > > > >> >>> > > >> >> in
> > > >> > > > > >> >>> > > >> >> > > > Apache
> > > >> > > > > >> >>> > > >> >> > > > >>> > Bahir
> > > >> > > > > >> >>> > > >> >> > > > >>> > > https://bahir.apache.org/
> > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > Do you think we should reach
> > out
> > > to
> > > >> > > Apache
> > > >> > > > > >> Bahir
> > > >> > > > > >> >>> > > >> community
> > > >> > > > > >> >>> > > >> >> > with a
> > > >> > > > > >> >>> > > >> >> > > > >>> > proposal
> > > >> > > > > >> >>> > > >> >> > > > >>> > > or we should plan to create a
> > > >> separate
> > > >> > > > > Apache
> > > >> > > > > >> >>> > Incubator
> > > >> > > > > >> >>> > > >> >> > project?
> > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > Regards,
> > > >> > > > > >> >>> > > >> >> > > > >>> > > Saikat
> > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21
> AM
> > > >> > > Emmanouil
> > > >> > > > > >> >>> > Gkatziouras <
> > > >> > > > > >> >>> > > >> >> > > > >>> > > [hidden email]>
> > > >> > > > > >> >>> > > >> >> > > > >>> > > wrote:
> > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > Hi all!
> > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > Based on the discussions
> most
> > > >> > probably
> > > >> > > > > this
> > > >> > > > > >> is
> > > >> > > > > >> >>> > going
> > > >> > > > > >> >>> > > to
> > > >> > > > > >> >>> > > >> >> be on
> > > >> > > > > >> >>> > > >> >> > > > >>> another
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > GitHub repo. Therefore I
> will
> > > >> > prepare
> > > >> > > a
> > > >> > > > > >> >>> standalone
> > > >> > > > > >> >>> > > >> project
> > > >> > > > > >> >>> > > >> >> > with
> > > >> > > > > >> >>> > > >> >> > > > the
> > > >> > > > > >> >>> > > >> >> > > > >>> > > Pub/Sub
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > module and once the
> > repository
> > > is
> > > >> > > > created
> > > >> > > > > a
> > > >> > > > > >> >>> pull
> > > >> > > > > >> >>> > > >> request
> > > >> > > > > >> >>> > > >> >> > shall
> > > >> > > > > >> >>> > > >> >> > > be
> > > >> > > > > >> >>> > > >> >> > > > >>> > issued.
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > If there is anything else I
> > can
> > > >> do
> > > >> > in
> > > >> > > > > order
> > > >> > > > > >> to
> > > >> > > > > >> >>> > > >> facilitate
> > > >> > > > > >> >>> > > >> >> > this
> > > >> > > > > >> >>> > > >> >> > > > >>> please
> > > >> > > > > >> >>> > > >> >> > > > >>> > let
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > me know.
> > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > If a ticket is created for
> > this
> > > >> > > issue, I
> > > >> > > > > >> shall
> > > >> > > > > >> >>> be
> > > >> > > > > >> >>> > > able
> > > >> > > > > >> >>> > > >> to
> > > >> > > > > >> >>> > > >> >> > > create
> > > >> > > > > >> >>> > > >> >> > > > a
> > > >> > > > > >> >>> > > >> >> > > > >>> pull
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > request and thus a build
> will
> > > >> take
> > > >> > > place
> > > >> > > > > on
> > > >> > > > > >> >>> > ignite's
> > > >> > > > > >> >>> > > >> CI.
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > If everyone is aligned with
> > > >> adding a
> > > >> > > > > Pub/Sub
> > > >> > > > > >> >>> > > >> >> implementation
> > > >> > > > > >> >>> > > >> >> > > may I
> > > >> > > > > >> >>> > > >> >> > > > >>> > create
> > > >> > > > > >> >>> > > >> >> > > > >>> > > a
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > ticket on JIRA?
> > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > My next question has to do
> on
> > > how
> > > >> > > Ignite
> > > >> > > > > >> >>> configures
> > > >> > > > > >> >>> > > the
> > > >> > > > > >> >>> > > >> >> build
> > > >> > > > > >> >>> > > >> >> > > > jobs
> > > >> > > > > >> >>> > > >> >> > > > >>> in
> > > >> > > > > >> >>> > > >> >> > > > >>> > > Team
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > City. The unit tests for my
> > > >> > > > implementation
> > > >> > > > > >> >>> need a
> > > >> > > > > >> >>> > > local
> > > >> > > > > >> >>> > > >> >> > Pub/Sub
> > > >> > > > > >> >>> > > >> >> > > > >>> server
> > > >> > > > > >> >>> > > >> >> > > > >>> > up
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > and running (they provide
> one
> > > for
> > > >> > > > testing
> > > >> > > > > >> >>> > purposes).
> > > >> > > > > >> >>> > > >> Will
> > > >> > > > > >> >>> > > >> >> > this
> > > >> > > > > >> >>> > > >> >> > > > take
> > > >> > > > > >> >>> > > >> >> > > > >>> > > effect
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > in a form of a build script
> > > >> inside
> > > >> > my
> > > >> > > > > >> >>> > implementation
> > > >> > > > > >> >>> > > or
> > > >> > > > > >> >>> > > >> >> is it
> > > >> > > > > >> >>> > > >> >> > > > >>> something
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > that should be configured
> on
> > > Team
> > > >> > > City?
> > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > Kind regards,
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > Emmanouil
> > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > https://egkatzioura.com/ |
> > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > >> > > > > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > https://github.com/gkatzioura
> > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at
> 23:27,
> > > >> Denis
> > > >> > > > Magda
> > > >> > > > > <
> > > >> > > > > >> >>> > > >> >> [hidden email]
> > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > >> >>> > > >> >> > > > >>> wrote:
> > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > After contemplating on
> > this,
> > > >> let
> > > >> > me
> > > >> > > > > >> propose
> > > >> > > > > >> >>> > another
> > > >> > > > > >> >>> > > >> way
> > > >> > > > > >> >>> > > >> >> > with
> > > >> > > > > >> >>> > > >> >> > > > this
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > particular integration
> that
> > > is
> > > >> > > aligned
> > > >> > > > > >> with
> > > >> > > > > >> >>> our
> > > >> > > > > >> >>> > > >> >> > > modularization
> > > >> > > > > >> >>> > > >> >> > > > >>> > > endeavor.
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > The modularization [1]
> > > defines
> > > >> the
> > > >> > > > > Ignite
> > > >> > > > > >> >>> core,
> > > >> > > > > >> >>> > > >> Ignite
> > > >> > > > > >> >>> > > >> >> > > modules,
> > > >> > > > > >> >>> > > >> >> > > > >>> and
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > independent integrations.
> > The
> > > >> > first
> > > >> > > > two
> > > >> > > > > >> are
> > > >> > > > > >> >>> to be
> > > >> > > > > >> >>> > > >> >> located
> > > >> > > > > >> >>> > > >> >> > in
> > > >> > > > > >> >>> > > >> >> > > > >>> Ignite
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > repositories and to be
> > > >> > > tested/updated
> > > >> > > > by
> > > >> > > > > >> the
> > > >> > > > > >> >>> > > >> community
> > > >> > > > > >> >>> > > >> >> for
> > > >> > > > > >> >>> > > >> >> > > > every
> > > >> > > > > >> >>> > > >> >> > > > >>> > > release.
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > The modules are a subset
> of
> > > >> > existing
> > > >> > > > > >> >>> integrations
> > > >> > > > > >> >>> > > >> that
> > > >> > > > > >> >>> > > >> >> are
> > > >> > > > > >> >>> > > >> >> > > > >>> adopted
> > > >> > > > > >> >>> > > >> >> > > > >>> > > widely
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > and require community
> > > >> attention.
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > As for the independent
> > > >> > integrations
> > > >> > > > [1],
> > > >> > > > > >> >>> those
> > > >> > > > > >> >>> > are
> > > >> > > > > >> >>> > > >> >> supposed
> > > >> > > > > >> >>> > > >> >> > > to
> > > >> > > > > >> >>> > > >> >> > > > be
> > > >> > > > > >> >>> > > >> >> > > > >>> > > located
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > in a separate GitHub
> repo.
> > I
> > > >> would
> > > >> > > > > advise
> > > >> > > > > >> us
> > > >> > > > > >> >>> to
> > > >> > > > > >> >>> > > start
> > > >> > > > > >> >>> > > >> >> > > > >>> implementing
> > > >> > > > > >> >>> > > >> >> > > > >>> > this
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > concept with the Pub/Sub
> > > >> > > integration.
> > > >> > > > > Even
> > > >> > > > > >> >>> though
> > > >> > > > > >> >>> > > it
> > > >> > > > > >> >>> > > >> >> will
> > > >> > > > > >> >>> > > >> >> > be
> > > >> > > > > >> >>> > > >> >> > > > >>> located
> > > >> > > > > >> >>> > > >> >> > > > >>> > > in a
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > separate Github repo, it
> > will
> > > >> be
> > > >> > > > listed
> > > >> > > > > >> among
> > > >> > > > > >> >>> > > >> officially
> > > >> > > > > >> >>> > > >> >> > > > >>> available
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > integrations on the
> Ignite
> > > >> website
> > > >> > > > > >> featuring
> > > >> > > > > >> >>> > > >> Emmanouil
> > > >> > > > > >> >>> > > >> >> as a
> > > >> > > > > >> >>> > > >> >> > > > >>> > > contributor.
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > What do you think?
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > [1]
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > >> > > > > >> >>> > > >> >> > > > >>>
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > >> >>> > > >> >>
> > > >> > > > > >> >>> > > >>
> > > >> > > > > >> >>> > >
> > > >> > > > > >> >>> >
> > > >> > > > > >> >>>
> > > >> > > > > >>
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > -
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > Denis
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at
> > 9:47
> > > PM
> > > >> > > Saikat
> > > >> > > > > >> >>> Maitra <
> > > >> > > > > >> >>> > > >> >> > > > >>> > [hidden email]
> > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > wrote:
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > Hi Emmanouil,
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > The changes looks good
> to
> > > >> me. I
> > > >> > > > wanted
> > > >> > > > > >> to
> > > >> > > > > >> >>> check
> > > >> > > > > >> >>> > > if
> > > >> > > > > >> >>> > > >> you
> > > >> > > > > >> >>> > > >> >> > are
> > > >> > > > > >> >>> > > >> >> > > > also
> > > >> > > > > >> >>> > > >> >> > > > >>> > able
> > > >> > > > > >> >>> > > >> >> > > > >>> > > to
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > validate that the once
> > data
> > > >> is
> > > >> > > added
> > > >> > > > > to
> > > >> > > > > >> >>> Ignite
> > > >> > > > > >> >>> > > >> cluster
> > > >> > > > > >> >>> > > >> >> > you
> > > >> > > > > >> >>> > > >> >> > > > are
> > > >> > > > > >> >>> > > >> >> > > > >>> also
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > able
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > to
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > access it using another
> > > >> Ignite
> > > >> > > > client
> > > >> > > > > or
> > > >> > > > > >> >>> using
> > > >> > > > > >> >>> > > rest
> > > >> > > > > >> >>> > > >> >> > > > endpoints.
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > We use Teamcity for CI
> > > >> process,
> > > >> > > the
> > > >> > > > > >> >>> details are
> > > >> > > > > >> >>> > > as
> > > >> > > > > >> >>> > > >> >> > follows
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > Trigger validation of
> > those
> > > >> test
> > > >> > > > > suites
> > > >> > > > > >> >>> that
> > > >> > > > > >> >>> > have
> > > >> > > > > >> >>> > > >> been
> > > >> > > > > >> >>> > > >> >> > > > >>> affected by
> > > >> > > > > >> >>> > > >> >> > > > >>> > > your
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > changes on TeamCity <
> > > >> > > > > >> >>> > > http://ci.ignite.apache.org/>:
> > > >> > > > > >> >>> > > >> <
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > http://204.14.53.153/
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Locate a test
> suite
> > > you
> > > >> > have
> > > >> > > to
> > > >> > > > > >> check,
> > > >> > > > > >> >>> > press
> > > >> > > > > >> >>> > > >> >> button
> > > >> > > > > >> >>> > > >> >> > > > named
> > > >> > > > > >> >>> > > >> >> > > > >>> > "..."
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > that
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    is located on the
> left
> > > of
> > > >> > "Run"
> > > >> > > > > >> button.
> > > >> > > > > >> >>> "Run
> > > >> > > > > >> >>> > > >> custom
> > > >> > > > > >> >>> > > >> >> > > build"
> > > >> > > > > >> >>> > > >> >> > > > >>> > dialog
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > will
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    appear;
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Go to "Changes"
> tab
> > > and
> > > >> > > choose
> > > >> > > > > >> >>> > > >> >> > > > >>> >
> "pull/<pull-request-number>/head"
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > in
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    "Build branch"
> > dropdown
> > > >> list;
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Press "Run build"
> > > button
> > > >> > and
> > > >> > > > > >> monitor
> > > >> > > > > >> >>> tests
> > > >> > > > > >> >>> > > >> >> results.
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > @Roman
> [hidden email]
> > -
> > > >> can
> > > >> > > you
> > > >> > > > > also
> > > >> > > > > >> >>> please
> > > >> > > > > >> >>> > > >> take a
> > > >> > > > > >> >>> > > >> >> > > look?
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > Regards,
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > Saikat
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at
> > > 6:20
> > > >> PM
> > > >> > > > > >> Emmanouil
> > > >> > > > > >> >>> > > >> Gkatziouras
> > > >> > > > > >> >>> > > >> >> <
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > [hidden email]>
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > wrote:
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > I have my
> > implementation
> > > >> on an
> > > >> > > > > Ignite
> > > >> > > > > >> >>> fork on
> > > >> > > > > >> >>> > > my
> > > >> > > > > >> >>> > > >> >> > personal
> > > >> > > > > >> >>> > > >> >> > > > >>> GitHub
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > account
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > [1]
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > It is based on the
> > Kafka
> > > >> > > > > >> implementation
> > > >> > > > > >> >>> which
> > > >> > > > > >> >>> > > is
> > > >> > > > > >> >>> > > >> of
> > > >> > > > > >> >>> > > >> >> a
> > > >> > > > > >> >>> > > >> >> > > > polling
> > > >> > > > > >> >>> > > >> >> > > > >>> > > nature.
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > GCP does not provide
> a
> > > >> service
> > > >> > > > like
> > > >> > > > > >> >>> Kafka.
> > > >> > > > > >> >>> > > >> Instead
> > > >> > > > > >> >>> > > >> >> it
> > > >> > > > > >> >>> > > >> >> > > > >>> provides us
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > with
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Pub/Sub which is just
> > > like
> > > >> > > Kafka.
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Therefore using the
> > Kafka
> > > >> > > streamer
> > > >> > > > > as
> > > >> > > > > >> a
> > > >> > > > > >> >>> > > guideline
> > > >> > > > > >> >>> > > >> >> made
> > > >> > > > > >> >>> > > >> >> > > more
> > > >> > > > > >> >>> > > >> >> > > > >>> sense
> > > >> > > > > >> >>> > > >> >> > > > >>> > > to
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > me.
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Apart from the
> > > >> implementation
> > > >> > I
> > > >> > > > have
> > > >> > > > > >> also
> > > >> > > > > >> >>> > > added a
> > > >> > > > > >> >>> > > >> >> unit
> > > >> > > > > >> >>> > > >> >> > > > test.
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > It definitely needs
> > some
> > > >> > > polishing
> > > >> > > > > >> and a
> > > >> > > > > >> >>> > double
> > > >> > > > > >> >>> > > >> >> check
> > > >> > > > > >> >>> > > >> >> > on
> > > >> > > > > >> >>> > > >> >> > > my
> > > >> > > > > >> >>> > > >> >> > > > >>> side
> > > >> > > > > >> >>> > > >> >> > > > >>> > to
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > make
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > the review smooth.
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > For the tests a local
> > > >> Pub/Sub
> > > >> > > [2]
> > > >> > > > > >> server
> > > >> > > >
> > > >
> > > >
> > >
> >
>