[DISCUSS] Ignite Update Checker

[dsetrakyan]

Igniters,

There has been lots of talk of proposals about various usage metrics for
Ignite and nothing came of it. I would like to resurrect the topic and
propose something very simple and non-intrusive.

1. Update Checker
The main purpose of the update checker is not to collect metrics, but to
notify users about a new version of Ignite by accessing maven.org and
getting the version out of the metadata file:
http://repo2.maven.org/maven2/org/apache/ignite/ignite-core/maven-metadata.xml

This way we do not send any information anywhere and, at the same time,
urge our users to download and start using the latest version of Ignite.

2. Startup Counter
This piece is optional, but we can also get an insight in how many times a
certain Ignite release gets started. This is just a cool metric for the
community to gauge the project popularity. You can think of it as of a page
visit counter shown on many websites. We can even decide to display this
counter on the Ignite website as well.

To do this, we can simply add a JAR in maven for every release, e.g.
ignite-start-counter.jar, which will contain only 1 byte. Every time an
Ignite node starts, it will download this JAR in the background. Then we
will be able to view the number of the total downloads for this JAR in
Maven Central, which is essentially the number of starts of Ignite nodes.

*Note that neither of the above suggestions require Ignite to send or track
any user information whatsoever.*

Please reply suggesting weather you are OK with this approach.

D.

[dsetrakyan]

Is everyone OK with this approach? Should I file a ticket on it?

On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <[hidden email]>
wrote:

> Igniters,
>
> There has been lots of talk of proposals about various usage metrics for
> Ignite and nothing came of it. I would like to resurrect the topic and
> propose something very simple and non-intrusive.
>
> 1. Update Checker
> The main purpose of the update checker is not to collect metrics, but to
> notify users about a new version of Ignite by accessing maven.org and
> getting the version out of the metadata file:
> http://repo2.maven.org/maven2/org/apache/ignite/ignite-core/
> maven-metadata.xml
>
> This way we do not send any information anywhere and, at the same time,
> urge our users to download and start using the latest version of Ignite.
>
> 2. Startup Counter
> This piece is optional, but we can also get an insight in how many times a
> certain Ignite release gets started. This is just a cool metric for the
> community to gauge the project popularity. You can think of it as of a page
> visit counter shown on many websites. We can even decide to display this
> counter on the Ignite website as well.
>
> To do this, we can simply add a JAR in maven for every release, e.g.
> ignite-start-counter.jar, which will contain only 1 byte. Every time an
> Ignite node starts, it will download this JAR in the background. Then we
> will be able to view the number of the total downloads for this JAR in
> Maven Central, which is essentially the number of starts of Ignite nodes.
>
> *Note that neither of the above suggestions require Ignite to send or
> track any user information whatsoever.*
>
> Please reply suggesting weather you are OK with this approach.
>
> D.
>

[dmagda]

I see nothing wrong with this approach.

Cos, Roman, Raul, as Apache veterans, what do you think? Is it good to go?

—
Denis

> On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan <[hidden email]> wrote:
>
> Is everyone OK with this approach? Should I file a ticket on it?
>
> On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <[hidden email]>
> wrote:
>
>> Igniters,
>>
>> There has been lots of talk of proposals about various usage metrics for
>> Ignite and nothing came of it. I would like to resurrect the topic and
>> propose something very simple and non-intrusive.
>>
>> 1. Update Checker
>> The main purpose of the update checker is not to collect metrics, but to
>> notify users about a new version of Ignite by accessing maven.org and
>> getting the version out of the metadata file:
>> http://repo2.maven.org/maven2/org/apache/ignite/ignite-core/
>> maven-metadata.xml
>>
>> This way we do not send any information anywhere and, at the same time,
>> urge our users to download and start using the latest version of Ignite.
>>
>> 2. Startup Counter
>> This piece is optional, but we can also get an insight in how many times a
>> certain Ignite release gets started. This is just a cool metric for the
>> community to gauge the project popularity. You can think of it as of a page
>> visit counter shown on many websites. We can even decide to display this
>> counter on the Ignite website as well.
>>
>> To do this, we can simply add a JAR in maven for every release, e.g.
>> ignite-start-counter.jar, which will contain only 1 byte. Every time an
>> Ignite node starts, it will download this JAR in the background. Then we
>> will be able to view the number of the total downloads for this JAR in
>> Maven Central, which is essentially the number of starts of Ignite nodes.
>>
>> *Note that neither of the above suggestions require Ignite to send or
>> track any user information whatsoever.*
>>
>> Please reply suggesting weather you are OK with this approach.
>>
>> D.
>>

[Raul Kripalani]

Hey guys

In my opinion, maven.org is still owned by a third party (Sonatype). We
don't know what kind of data analysis or intelligence extraction they run.

If Ignite servers all over the world were hitting maven.org periodically
asking for an Ignite Maven artifact, it gives Sonatype a clear indication
of who is running an Ignite server.

They could reverse lookup the IP address and find out what corporation it
is.

How about having Ignite check the ASF Git directly?

We could use the Git ssh API, but that would require a new dependency,
which I advise against.

Alternatively, we could have it scrape this HTML for new Git tags:
https://git-wip-us.apache.org/repos/asf?p=ignite.git

Another option is to place a txt file in the root of the master branch (e.g
LATEST), containing a list of the latest GA versions for each major version
line (1.x, 2.x).

I would advocate this last option, but it requires somebody remembering to
update the file with every release, unless we automate it with a Maven
plugin.

Hope that helps!


On 24 Aug 2017 19:37, "Denis Magda" <[hidden email]> wrote:

I see nothing wrong with this approach.

Cos, Roman, Raul, as Apache veterans, what do you think? Is it good to go?

—
Denis

> On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan <[hidden email]>
wrote:

>
> Is everyone OK with this approach? Should I file a ticket on it?
>
> On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <[hidden email]>
> wrote:
>
>> Igniters,
>>
>> There has been lots of talk of proposals about various usage metrics for
>> Ignite and nothing came of it. I would like to resurrect the topic and
>> propose something very simple and non-intrusive.
>>
>> 1. Update Checker
>> The main purpose of the update checker is not to collect metrics, but to
>> notify users about a new version of Ignite by accessing maven.org and
>> getting the version out of the metadata file:
>> http://repo2.maven.org/maven2/org/apache/ignite/ignite-core/
>> maven-metadata.xml
>>
>> This way we do not send any information anywhere and, at the same time,
>> urge our users to download and start using the latest version of Ignite.
>>
>> 2. Startup Counter
>> This piece is optional, but we can also get an insight in how many times

a
>> certain Ignite release gets started. This is just a cool metric for the
>> community to gauge the project popularity. You can think of it as of a
page

>> visit counter shown on many websites. We can even decide to display this
>> counter on the Ignite website as well.
>>
>> To do this, we can simply add a JAR in maven for every release, e.g.
>> ignite-start-counter.jar, which will contain only 1 byte. Every time an
>> Ignite node starts, it will download this JAR in the background. Then we
>> will be able to view the number of the total downloads for this JAR in
>> Maven Central, which is essentially the number of starts of Ignite nodes.
>>
>> *Note that neither of the above suggestions require Ignite to send or
>> track any user information whatsoever.*
>>
>> Please reply suggesting weather you are OK with this approach.
>>
>> D.
>>

[yzhdanov]

Raul, any ideas on startup counter?

BTW, we have google analytics applied to Ignite site. Can it be used to
count node starts?

--Yakov

[Konstantin Boudnik-2]

I agree with Raul.
- providing a ping-back address to a 3rd party might be frown upon by some.
  And might have a consequences like stats collection about users'
  infrastructure.
- checking an ASF git-repo is easy and won't download any binary data:
  everything is clear text and could be easily monitored by any of the network
  diagnostic tools, shall it be required. But it involves a bit of the release
  discipline.
- the binary data download in the runtime is my main concern. This is the
  vector of MMA. What if someone gains the control over the repository and
  replaces the file with some malicious content.

As for the particular mechanism: IIRC Ignite used to make a call to an
external script to check upon the atest software version available for
download. In the past, the endpoint was running on a 3rd party server, I
believe the best way would be to put this script on ASF infra and have the
"update checker" running in a completely controlled environment. Actually, I
recall we had this very discussion during the Incubation; I can probably dig
out the corresponding thread.

Thoughts?
  Cok

On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote:

> Hey guys
>
> In my opinion, maven.org is still owned by a third party (Sonatype). We
> don't know what kind of data analysis or intelligence extraction they run.
>
> If Ignite servers all over the world were hitting maven.org periodically
> asking for an Ignite Maven artifact, it gives Sonatype a clear indication
> of who is running an Ignite server.
>
> They could reverse lookup the IP address and find out what corporation it
> is.
>
> How about having Ignite check the ASF Git directly?
>
> We could use the Git ssh API, but that would require a new dependency,
> which I advise against.
>
> Alternatively, we could have it scrape this HTML for new Git tags:
> https://git-wip-us.apache.org/repos/asf?p=ignite.git
>
> Another option is to place a txt file in the root of the master branch (e.g
> LATEST), containing a list of the latest GA versions for each major version
> line (1.x, 2.x).
>
> I would advocate this last option, but it requires somebody remembering to
> update the file with every release, unless we automate it with a Maven
> plugin.
>
> Hope that helps!
>
>
> On 24 Aug 2017 19:37, "Denis Magda" <[hidden email]> wrote:
>
> I see nothing wrong with this approach.
>
> Cos, Roman, Raul, as Apache veterans, what do you think? Is it good to go?
>
> —
> Denis
>
> > On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan <[hidden email]>
> wrote:
> >
> > Is everyone OK with this approach? Should I file a ticket on it?
> >
> > On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <[hidden email]>
> > wrote:
> >
> >> Igniters,
> >>
> >> There has been lots of talk of proposals about various usage metrics for
> >> Ignite and nothing came of it. I would like to resurrect the topic and
> >> propose something very simple and non-intrusive.
> >>
> >> 1. Update Checker
> >> The main purpose of the update checker is not to collect metrics, but to
> >> notify users about a new version of Ignite by accessing maven.org and
> >> getting the version out of the metadata file:
> >> http://repo2.maven.org/maven2/org/apache/ignite/ignite-core/
> >> maven-metadata.xml
> >>
> >> This way we do not send any information anywhere and, at the same time,
> >> urge our users to download and start using the latest version of Ignite.
> >>
> >> 2. Startup Counter
> >> This piece is optional, but we can also get an insight in how many times
> a
> >> certain Ignite release gets started. This is just a cool metric for the
> >> community to gauge the project popularity. You can think of it as of a
> page
> >> visit counter shown on many websites. We can even decide to display this
> >> counter on the Ignite website as well.
> >>
> >> To do this, we can simply add a JAR in maven for every release, e.g.
> >> ignite-start-counter.jar, which will contain only 1 byte. Every time an
> >> Ignite node starts, it will download this JAR in the background. Then we
> >> will be able to view the number of the total downloads for this JAR in
> >> Maven Central, which is essentially the number of starts of Ignite nodes.
> >>
> >> *Note that neither of the above suggestions require Ignite to send or
> >> track any user information whatsoever.*
> >>
> >> Please reply suggesting weather you are OK with this approach.
> >>
> >> D.
> >>

[dsetrakyan]

Cos, Raul,

Thanks for the feedback. I completely agree about Maven Central being a 3rd
party repo (did not think about that initially). All your suggestions make
sense, but I would like to keep it as simple as possible, and so far
everything suggested required GIT dependencies and extra work.

How about Yakov's suggestion. We simply add a page to the Ignite website
which will have only the latest version. Every time a node starts, it
receives the latest version from the page and suggests that users upgrade
if needed.

Also, since we have GA enabled for the website, we can track how many times
this page was accessed, which will be equal to the number of starts. This
way, the counter information is tracked and monitored by the Ignite PMC.

This approach looks pretty innocent to me and everything is kept and
managed within Apache.

Thoughts?

D.


On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik <[hidden email]> wrote:

> I agree with Raul.
> - providing a ping-back address to a 3rd party might be frown upon by some.
>   And might have a consequences like stats collection about users'
>   infrastructure.
> - checking an ASF git-repo is easy and won't download any binary data:
>   everything is clear text and could be easily monitored by any of the
> network
>   diagnostic tools, shall it be required. But it involves a bit of the
> release
>   discipline.
> - the binary data download in the runtime is my main concern. This is the
>   vector of MMA. What if someone gains the control over the repository and
>   replaces the file with some malicious content.
>
> As for the particular mechanism: IIRC Ignite used to make a call to an
> external script to check upon the atest software version available for
> download. In the past, the endpoint was running on a 3rd party server, I
> believe the best way would be to put this script on ASF infra and have the
> "update checker" running in a completely controlled environment. Actually,
> I
> recall we had this very discussion during the Incubation; I can probably
> dig
> out the corresponding thread.
>
> Thoughts?
>   Cok
>
> On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote:
> > Hey guys
> >
> > In my opinion, maven.org is still owned by a third party (Sonatype). We
> > don't know what kind of data analysis or intelligence extraction they
> run.
> >
> > If Ignite servers all over the world were hitting maven.org periodically
> > asking for an Ignite Maven artifact, it gives Sonatype a clear indication
> > of who is running an Ignite server.
> >
> > They could reverse lookup the IP address and find out what corporation it
> > is.
> >
> > How about having Ignite check the ASF Git directly?
> >
> > We could use the Git ssh API, but that would require a new dependency,
> > which I advise against.
> >
> > Alternatively, we could have it scrape this HTML for new Git tags:
> > https://git-wip-us.apache.org/repos/asf?p=ignite.git
> >
> > Another option is to place a txt file in the root of the master branch
> (e.g
> > LATEST), containing a list of the latest GA versions for each major
> version
> > line (1.x, 2.x).
> >
> > I would advocate this last option, but it requires somebody remembering
> to
> > update the file with every release, unless we automate it with a Maven
> > plugin.
> >
> > Hope that helps!
> >
> >
> > On 24 Aug 2017 19:37, "Denis Magda" <[hidden email]> wrote:
> >
> > I see nothing wrong with this approach.
> >
> > Cos, Roman, Raul, as Apache veterans, what do you think? Is it good to
> go?
> >
> > —
> > Denis
> >
> > > On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan <[hidden email]
> >
> > wrote:
> > >
> > > Is everyone OK with this approach? Should I file a ticket on it?
> > >
> > > On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <
> [hidden email]>
> > > wrote:
> > >
> > >> Igniters,
> > >>
> > >> There has been lots of talk of proposals about various usage metrics
> for
> > >> Ignite and nothing came of it. I would like to resurrect the topic and
> > >> propose something very simple and non-intrusive.
> > >>
> > >> 1. Update Checker
> > >> The main purpose of the update checker is not to collect metrics, but
> to
> > >> notify users about a new version of Ignite by accessing maven.org and
> > >> getting the version out of the metadata file:
> > >> http://repo2.maven.org/maven2/org/apache/ignite/ignite-core/
> > >> maven-metadata.xml
> > >>
> > >> This way we do not send any information anywhere and, at the same
> time,
> > >> urge our users to download and start using the latest version of
> Ignite.
> > >>
> > >> 2. Startup Counter
> > >> This piece is optional, but we can also get an insight in how many
> times
> > a
> > >> certain Ignite release gets started. This is just a cool metric for
> the
> > >> community to gauge the project popularity. You can think of it as of a
> > page
> > >> visit counter shown on many websites. We can even decide to display
> this
> > >> counter on the Ignite website as well.
> > >>
> > >> To do this, we can simply add a JAR in maven for every release, e.g.
> > >> ignite-start-counter.jar, which will contain only 1 byte. Every time
> an
> > >> Ignite node starts, it will download this JAR in the background. Then
> we
> > >> will be able to view the number of the total downloads for this JAR in
> > >> Maven Central, which is essentially the number of starts of Ignite
> nodes.
> > >>
> > >> *Note that neither of the above suggestions require Ignite to send or
> > >> track any user information whatsoever.*
> > >>
> > >> Please reply suggesting weather you are OK with this approach.
> > >>
> > >> D.
> > >>
>
>

[Raúl Kripalani]

Hey Dmitriy and all

Also, since we have GA enabled for the website, we can track how many times
> this page was accessed, which will be equal to the number of starts. This
> way, the counter information is tracked and monitored by the Ignite PMC.


Unfortunately this won't work because GA is loaded via Javascript on the
browser, so Google will never receive the page hit.

Given the constraints, the most viable solution is an HTTPS endpoint
running on ASF infra that Ignite invokes via a GET or POST request. The
simplest thing is to write each request in a log file, along with the
timestamp, the version reported by the client, maybe the IP (not sure about
the ASF rules about this concerning privacy, I guess it's OK if you make it
an opt-in) and a unique node identifier, i.e. a UUID the node creates on
first startup or something.

This endpoint would need some basic DDoS protection and blacklisting to
prevent data spoofing.

If we'll be implementing this endpoint anyway, then there's no point
placing another file on Git or elsewhere for reporting the latest versions:
the endpoint itself can return them.

WDYT?

Cheers.

On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan <[hidden email]>
wrote:

> Cos, Raul,
>
> Thanks for the feedback. I completely agree about Maven Central being a 3rd
> party repo (did not think about that initially). All your suggestions make
> sense, but I would like to keep it as simple as possible, and so far
> everything suggested required GIT dependencies and extra work.
>
> How about Yakov's suggestion. We simply add a page to the Ignite website
> which will have only the latest version. Every time a node starts, it
> receives the latest version from the page and suggests that users upgrade
> if needed.
>
> Also, since we have GA enabled for the website, we can track how many times
> this page was accessed, which will be equal to the number of starts. This
> way, the counter information is tracked and monitored by the Ignite PMC.
>
> This approach looks pretty innocent to me and everything is kept and
> managed within Apache.
>
> Thoughts?
>
> D.
>
>
> On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik <[hidden email]>
> wrote:
>
> > I agree with Raul.
> > - providing a ping-back address to a 3rd party might be frown upon by
> some.
> >   And might have a consequences like stats collection about users'
> >   infrastructure.
> > - checking an ASF git-repo is easy and won't download any binary data:
> >   everything is clear text and could be easily monitored by any of the
> > network
> >   diagnostic tools, shall it be required. But it involves a bit of the
> > release
> >   discipline.
> > - the binary data download in the runtime is my main concern. This is the
> >   vector of MMA. What if someone gains the control over the repository
> and
> >   replaces the file with some malicious content.
> >
> > As for the particular mechanism: IIRC Ignite used to make a call to an
> > external script to check upon the atest software version available for
> > download. In the past, the endpoint was running on a 3rd party server, I
> > believe the best way would be to put this script on ASF infra and have
> the
> > "update checker" running in a completely controlled environment.
> Actually,
> > I
> > recall we had this very discussion during the Incubation; I can probably
> > dig
> > out the corresponding thread.
> >
> > Thoughts?
> >   Cok
> >
> > On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote:
> > > Hey guys
> > >
> > > In my opinion, maven.org is still owned by a third party (Sonatype).
> We
> > > don't know what kind of data analysis or intelligence extraction they
> > run.
> > >
> > > If Ignite servers all over the world were hitting maven.org
> periodically
> > > asking for an Ignite Maven artifact, it gives Sonatype a clear
> indication
> > > of who is running an Ignite server.
> > >
> > > They could reverse lookup the IP address and find out what corporation
> it
> > > is.
> > >
> > > How about having Ignite check the ASF Git directly?
> > >
> > > We could use the Git ssh API, but that would require a new dependency,
> > > which I advise against.
> > >
> > > Alternatively, we could have it scrape this HTML for new Git tags:
> > > https://git-wip-us.apache.org/repos/asf?p=ignite.git
> > >
> > > Another option is to place a txt file in the root of the master branch
> > (e.g
> > > LATEST), containing a list of the latest GA versions for each major
> > version
> > > line (1.x, 2.x).
> > >
> > > I would advocate this last option, but it requires somebody remembering
> > to
> > > update the file with every release, unless we automate it with a Maven
> > > plugin.
> > >
> > > Hope that helps!
> > >
> > >
> > > On 24 Aug 2017 19:37, "Denis Magda" <[hidden email]> wrote:
> > >
> > > I see nothing wrong with this approach.
> > >
> > > Cos, Roman, Raul, as Apache veterans, what do you think? Is it good to
> > go?
> > >
> > > —
> > > Denis
> > >
> > > > On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan <
> [hidden email]
> > >
> > > wrote:
> > > >
> > > > Is everyone OK with this approach? Should I file a ticket on it?
> > > >
> > > > On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <
> > [hidden email]>
> > > > wrote:
> > > >
> > > >> Igniters,
> > > >>
> > > >> There has been lots of talk of proposals about various usage metrics
> > for
> > > >> Ignite and nothing came of it. I would like to resurrect the topic
> and
> > > >> propose something very simple and non-intrusive.
> > > >>
> > > >> 1. Update Checker
> > > >> The main purpose of the update checker is not to collect metrics,
> but
> > to
> > > >> notify users about a new version of Ignite by accessing maven.org
> and
> > > >> getting the version out of the metadata file:
> > > >> http://repo2.maven.org/maven2/org/apache/ignite/ignite-core/
> > > >> maven-metadata.xml
> > > >>
> > > >> This way we do not send any information anywhere and, at the same
> > time,
> > > >> urge our users to download and start using the latest version of
> > Ignite.
> > > >>
> > > >> 2. Startup Counter
> > > >> This piece is optional, but we can also get an insight in how many
> > times
> > > a
> > > >> certain Ignite release gets started. This is just a cool metric for
> > the
> > > >> community to gauge the project popularity. You can think of it as
> of a
> > > page
> > > >> visit counter shown on many websites. We can even decide to display
> > this
> > > >> counter on the Ignite website as well.
> > > >>
> > > >> To do this, we can simply add a JAR in maven for every release, e.g.
> > > >> ignite-start-counter.jar, which will contain only 1 byte. Every time
> > an
> > > >> Ignite node starts, it will download this JAR in the background.
> Then
> > we
> > > >> will be able to view the number of the total downloads for this JAR
> in
> > > >> Maven Central, which is essentially the number of starts of Ignite
> > nodes.
> > > >>
> > > >> *Note that neither of the above suggestions require Ignite to send
> or
> > > >> track any user information whatsoever.*
> > > >>
> > > >> Please reply suggesting weather you are OK with this approach.
> > > >>
> > > >> D.
> > > >>
> >
> >
>

[Konstantin Boudnik-2]

This will surely works to get the update info to the nodes. And it
sounds like a legit approach.
I cannot judge on the feasibility of the GA though - don't know squat
about it ;)

Thanks!
--
  With regards,
Konstantin (Cos) Boudnik
2CAC 8312 4870 D885 8616  6115 220F 6980 1F27 E622

Disclaimer: Opinions expressed in this email are those of the author,
and do not necessarily represent the views of any company the author
might be affiliated with at the moment of writing.


On Fri, Aug 25, 2017 at 1:56 PM, Dmitriy Setrakyan
<[hidden email]> wrote:

> Cos, Raul,
>
> Thanks for the feedback. I completely agree about Maven Central being a 3rd
> party repo (did not think about that initially). All your suggestions make
> sense, but I would like to keep it as simple as possible, and so far
> everything suggested required GIT dependencies and extra work.
>
> How about Yakov's suggestion. We simply add a page to the Ignite website
> which will have only the latest version. Every time a node starts, it
> receives the latest version from the page and suggests that users upgrade
> if needed.
>
> Also, since we have GA enabled for the website, we can track how many times
> this page was accessed, which will be equal to the number of starts. This
> way, the counter information is tracked and monitored by the Ignite PMC.
>
> This approach looks pretty innocent to me and everything is kept and
> managed within Apache.
>
> Thoughts?
>
> D.
>
>
> On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik <[hidden email]> wrote:
>
>> I agree with Raul.
>> - providing a ping-back address to a 3rd party might be frown upon by some.
>>   And might have a consequences like stats collection about users'
>>   infrastructure.
>> - checking an ASF git-repo is easy and won't download any binary data:
>>   everything is clear text and could be easily monitored by any of the
>> network
>>   diagnostic tools, shall it be required. But it involves a bit of the
>> release
>>   discipline.
>> - the binary data download in the runtime is my main concern. This is the
>>   vector of MMA. What if someone gains the control over the repository and
>>   replaces the file with some malicious content.
>>
>> As for the particular mechanism: IIRC Ignite used to make a call to an
>> external script to check upon the atest software version available for
>> download. In the past, the endpoint was running on a 3rd party server, I
>> believe the best way would be to put this script on ASF infra and have the
>> "update checker" running in a completely controlled environment. Actually,
>> I
>> recall we had this very discussion during the Incubation; I can probably
>> dig
>> out the corresponding thread.
>>
>> Thoughts?
>>   Cok
>>
>> On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote:
>> > Hey guys
>> >
>> > In my opinion, maven.org is still owned by a third party (Sonatype). We
>> > don't know what kind of data analysis or intelligence extraction they
>> run.
>> >
>> > If Ignite servers all over the world were hitting maven.org periodically
>> > asking for an Ignite Maven artifact, it gives Sonatype a clear indication
>> > of who is running an Ignite server.
>> >
>> > They could reverse lookup the IP address and find out what corporation it
>> > is.
>> >
>> > How about having Ignite check the ASF Git directly?
>> >
>> > We could use the Git ssh API, but that would require a new dependency,
>> > which I advise against.
>> >
>> > Alternatively, we could have it scrape this HTML for new Git tags:
>> > https://git-wip-us.apache.org/repos/asf?p=ignite.git
>> >
>> > Another option is to place a txt file in the root of the master branch
>> (e.g
>> > LATEST), containing a list of the latest GA versions for each major
>> version
>> > line (1.x, 2.x).
>> >
>> > I would advocate this last option, but it requires somebody remembering
>> to
>> > update the file with every release, unless we automate it with a Maven
>> > plugin.
>> >
>> > Hope that helps!
>> >
>> >
>> > On 24 Aug 2017 19:37, "Denis Magda" <[hidden email]> wrote:
>> >
>> > I see nothing wrong with this approach.
>> >
>> > Cos, Roman, Raul, as Apache veterans, what do you think? Is it good to
>> go?
>> >
>> > —
>> > Denis
>> >
>> > > On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan <[hidden email]
>> >
>> > wrote:
>> > >
>> > > Is everyone OK with this approach? Should I file a ticket on it?
>> > >
>> > > On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <
>> [hidden email]>
>> > > wrote:
>> > >
>> > >> Igniters,
>> > >>
>> > >> There has been lots of talk of proposals about various usage metrics
>> for
>> > >> Ignite and nothing came of it. I would like to resurrect the topic and
>> > >> propose something very simple and non-intrusive.
>> > >>
>> > >> 1. Update Checker
>> > >> The main purpose of the update checker is not to collect metrics, but
>> to
>> > >> notify users about a new version of Ignite by accessing maven.org and
>> > >> getting the version out of the metadata file:
>> > >> http://repo2.maven.org/maven2/org/apache/ignite/ignite-core/
>> > >> maven-metadata.xml
>> > >>
>> > >> This way we do not send any information anywhere and, at the same
>> time,
>> > >> urge our users to download and start using the latest version of
>> Ignite.
>> > >>
>> > >> 2. Startup Counter
>> > >> This piece is optional, but we can also get an insight in how many
>> times
>> > a
>> > >> certain Ignite release gets started. This is just a cool metric for
>> the
>> > >> community to gauge the project popularity. You can think of it as of a
>> > page
>> > >> visit counter shown on many websites. We can even decide to display
>> this
>> > >> counter on the Ignite website as well.
>> > >>
>> > >> To do this, we can simply add a JAR in maven for every release, e.g.
>> > >> ignite-start-counter.jar, which will contain only 1 byte. Every time
>> an
>> > >> Ignite node starts, it will download this JAR in the background. Then
>> we
>> > >> will be able to view the number of the total downloads for this JAR in
>> > >> Maven Central, which is essentially the number of starts of Ignite
>> nodes.
>> > >>
>> > >> *Note that neither of the above suggestions require Ignite to send or
>> > >> track any user information whatsoever.*
>> > >>
>> > >> Please reply suggesting weather you are OK with this approach.
>> > >>
>> > >> D.
>> > >>
>>
>>

[dsetrakyan]

Raul,

Could point about Javascript, it will not work because it executes in the
browser. This means we need a server-side script, like CGI we are using on
our download page.

How about this approach. We create something like ignite-version.cgi script
which will invoke a call to GA and then return the latest version.

This should work, right?

D.

On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani <[hidden email]>
wrote:

> Hey Dmitriy and all
>
> Also, since we have GA enabled for the website, we can track how many times
> > this page was accessed, which will be equal to the number of starts. This
> > way, the counter information is tracked and monitored by the Ignite PMC.
>
>
> Unfortunately this won't work because GA is loaded via Javascript on the
> browser, so Google will never receive the page hit.
>
> Given the constraints, the most viable solution is an HTTPS endpoint
> running on ASF infra that Ignite invokes via a GET or POST request. The
> simplest thing is to write each request in a log file, along with the
> timestamp, the version reported by the client, maybe the IP (not sure about
> the ASF rules about this concerning privacy, I guess it's OK if you make it
> an opt-in) and a unique node identifier, i.e. a UUID the node creates on
> first startup or something.
>
> This endpoint would need some basic DDoS protection and blacklisting to
> prevent data spoofing.
>
> If we'll be implementing this endpoint anyway, then there's no point
> placing another file on Git or elsewhere for reporting the latest versions:
> the endpoint itself can return them.
>
> WDYT?
>
> Cheers.
>
> On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan <[hidden email]>
> wrote:
>
> > Cos, Raul,
> >
> > Thanks for the feedback. I completely agree about Maven Central being a
> 3rd
> > party repo (did not think about that initially). All your suggestions
> make
> > sense, but I would like to keep it as simple as possible, and so far
> > everything suggested required GIT dependencies and extra work.
> >
> > How about Yakov's suggestion. We simply add a page to the Ignite website
> > which will have only the latest version. Every time a node starts, it
> > receives the latest version from the page and suggests that users upgrade
> > if needed.
> >
> > Also, since we have GA enabled for the website, we can track how many
> times
> > this page was accessed, which will be equal to the number of starts. This
> > way, the counter information is tracked and monitored by the Ignite PMC.
> >
> > This approach looks pretty innocent to me and everything is kept and
> > managed within Apache.
> >
> > Thoughts?
> >
> > D.
> >
> >
> > On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik <[hidden email]>
> > wrote:
> >
> > > I agree with Raul.
> > > - providing a ping-back address to a 3rd party might be frown upon by
> > some.
> > >   And might have a consequences like stats collection about users'
> > >   infrastructure.
> > > - checking an ASF git-repo is easy and won't download any binary data:
> > >   everything is clear text and could be easily monitored by any of the
> > > network
> > >   diagnostic tools, shall it be required. But it involves a bit of the
> > > release
> > >   discipline.
> > > - the binary data download in the runtime is my main concern. This is
> the
> > >   vector of MMA. What if someone gains the control over the repository
> > and
> > >   replaces the file with some malicious content.
> > >
> > > As for the particular mechanism: IIRC Ignite used to make a call to an
> > > external script to check upon the atest software version available for
> > > download. In the past, the endpoint was running on a 3rd party server,
> I
> > > believe the best way would be to put this script on ASF infra and have
> > the
> > > "update checker" running in a completely controlled environment.
> > Actually,
> > > I
> > > recall we had this very discussion during the Incubation; I can
> probably
> > > dig
> > > out the corresponding thread.
> > >
> > > Thoughts?
> > >   Cok
> > >
> > > On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote:
> > > > Hey guys
> > > >
> > > > In my opinion, maven.org is still owned by a third party (Sonatype).
> > We
> > > > don't know what kind of data analysis or intelligence extraction they
> > > run.
> > > >
> > > > If Ignite servers all over the world were hitting maven.org
> > periodically
> > > > asking for an Ignite Maven artifact, it gives Sonatype a clear
> > indication
> > > > of who is running an Ignite server.
> > > >
> > > > They could reverse lookup the IP address and find out what
> corporation
> > it
> > > > is.
> > > >
> > > > How about having Ignite check the ASF Git directly?
> > > >
> > > > We could use the Git ssh API, but that would require a new
> dependency,
> > > > which I advise against.
> > > >
> > > > Alternatively, we could have it scrape this HTML for new Git tags:
> > > > https://git-wip-us.apache.org/repos/asf?p=ignite.git
> > > >
> > > > Another option is to place a txt file in the root of the master
> branch
> > > (e.g
> > > > LATEST), containing a list of the latest GA versions for each major
> > > version
> > > > line (1.x, 2.x).
> > > >
> > > > I would advocate this last option, but it requires somebody
> remembering
> > > to
> > > > update the file with every release, unless we automate it with a
> Maven
> > > > plugin.
> > > >
> > > > Hope that helps!
> > > >
> > > >
> > > > On 24 Aug 2017 19:37, "Denis Magda" <[hidden email]> wrote:
> > > >
> > > > I see nothing wrong with this approach.
> > > >
> > > > Cos, Roman, Raul, as Apache veterans, what do you think? Is it good
> to
> > > go?
> > > >
> > > > —
> > > > Denis
> > > >
> > > > > On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan <
> > [hidden email]
> > > >
> > > > wrote:
> > > > >
> > > > > Is everyone OK with this approach? Should I file a ticket on it?
> > > > >
> > > > > On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <
> > > [hidden email]>
> > > > > wrote:
> > > > >
> > > > >> Igniters,
> > > > >>
> > > > >> There has been lots of talk of proposals about various usage
> metrics
> > > for
> > > > >> Ignite and nothing came of it. I would like to resurrect the topic
> > and
> > > > >> propose something very simple and non-intrusive.
> > > > >>
> > > > >> 1. Update Checker
> > > > >> The main purpose of the update checker is not to collect metrics,
> > but
> > > to
> > > > >> notify users about a new version of Ignite by accessing maven.org
> > and
> > > > >> getting the version out of the metadata file:
> > > > >> http://repo2.maven.org/maven2/org/apache/ignite/ignite-core/
> > > > >> maven-metadata.xml
> > > > >>
> > > > >> This way we do not send any information anywhere and, at the same
> > > time,
> > > > >> urge our users to download and start using the latest version of
> > > Ignite.
> > > > >>
> > > > >> 2. Startup Counter
> > > > >> This piece is optional, but we can also get an insight in how many
> > > times
> > > > a
> > > > >> certain Ignite release gets started. This is just a cool metric
> for
> > > the
> > > > >> community to gauge the project popularity. You can think of it as
> > of a
> > > > page
> > > > >> visit counter shown on many websites. We can even decide to
> display
> > > this
> > > > >> counter on the Ignite website as well.
> > > > >>
> > > > >> To do this, we can simply add a JAR in maven for every release,
> e.g.
> > > > >> ignite-start-counter.jar, which will contain only 1 byte. Every
> time
> > > an
> > > > >> Ignite node starts, it will download this JAR in the background.
> > Then
> > > we
> > > > >> will be able to view the number of the total downloads for this
> JAR
> > in
> > > > >> Maven Central, which is essentially the number of starts of Ignite
> > > nodes.
> > > > >>
> > > > >> *Note that neither of the above suggestions require Ignite to send
> > or
> > > > >> track any user information whatsoever.*
> > > > >>
> > > > >> Please reply suggesting weather you are OK with this approach.
> > > > >>
> > > > >> D.
> > > > >>
> > >
> > >
> >
>

[Raúl Kripalani]

Yeah, I guess that's doable as well and requires less management effort
than my suggestion. We could use events [1] to store payload data (e.g. IP,
version, etc.) What the download page CGI developed in? PHP?

However, I'm not sure whether storing this data in a 3rd party (Google) is
compliant with the ASF policy. I guess it's no biggie, but if there's doubt
in the PMC, it's better to ask legal@. I think Cos said it's OK; maybe
Roman can pitch in.

Cheers.

[1]
https://developers.google.com/analytics/devguides/collection/analyticsjs/events

On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan <[hidden email]>
wrote:

> Raul,
>
> Could point about Javascript, it will not work because it executes in the
> browser. This means we need a server-side script, like CGI we are using on
> our download page.
>
> How about this approach. We create something like ignite-version.cgi script
> which will invoke a call to GA and then return the latest version.
>
> This should work, right?
>
> D.
>
> On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani <[hidden email]>
> wrote:
>
> > Hey Dmitriy and all
> >
> > Also, since we have GA enabled for the website, we can track how many
> times
> > > this page was accessed, which will be equal to the number of starts.
> This
> > > way, the counter information is tracked and monitored by the Ignite
> PMC.
> >
> >
> > Unfortunately this won't work because GA is loaded via Javascript on the
> > browser, so Google will never receive the page hit.
> >
> > Given the constraints, the most viable solution is an HTTPS endpoint
> > running on ASF infra that Ignite invokes via a GET or POST request. The
> > simplest thing is to write each request in a log file, along with the
> > timestamp, the version reported by the client, maybe the IP (not sure
> about
> > the ASF rules about this concerning privacy, I guess it's OK if you make
> it
> > an opt-in) and a unique node identifier, i.e. a UUID the node creates on
> > first startup or something.
> >
> > This endpoint would need some basic DDoS protection and blacklisting to
> > prevent data spoofing.
> >
> > If we'll be implementing this endpoint anyway, then there's no point
> > placing another file on Git or elsewhere for reporting the latest
> versions:
> > the endpoint itself can return them.
> >
> > WDYT?
> >
> > Cheers.
> >
> > On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan <
> [hidden email]>
> > wrote:
> >
> > > Cos, Raul,
> > >
> > > Thanks for the feedback. I completely agree about Maven Central being a
> > 3rd
> > > party repo (did not think about that initially). All your suggestions
> > make
> > > sense, but I would like to keep it as simple as possible, and so far
> > > everything suggested required GIT dependencies and extra work.
> > >
> > > How about Yakov's suggestion. We simply add a page to the Ignite
> website
> > > which will have only the latest version. Every time a node starts, it
> > > receives the latest version from the page and suggests that users
> upgrade
> > > if needed.
> > >
> > > Also, since we have GA enabled for the website, we can track how many
> > times
> > > this page was accessed, which will be equal to the number of starts.
> This
> > > way, the counter information is tracked and monitored by the Ignite
> PMC.
> > >
> > > This approach looks pretty innocent to me and everything is kept and
> > > managed within Apache.
> > >
> > > Thoughts?
> > >
> > > D.
> > >
> > >
> > > On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik <[hidden email]>
> > > wrote:
> > >
> > > > I agree with Raul.
> > > > - providing a ping-back address to a 3rd party might be frown upon by
> > > some.
> > > >   And might have a consequences like stats collection about users'
> > > >   infrastructure.
> > > > - checking an ASF git-repo is easy and won't download any binary
> data:
> > > >   everything is clear text and could be easily monitored by any of
> the
> > > > network
> > > >   diagnostic tools, shall it be required. But it involves a bit of
> the
> > > > release
> > > >   discipline.
> > > > - the binary data download in the runtime is my main concern. This is
> > the
> > > >   vector of MMA. What if someone gains the control over the
> repository
> > > and
> > > >   replaces the file with some malicious content.
> > > >
> > > > As for the particular mechanism: IIRC Ignite used to make a call to
> an
> > > > external script to check upon the atest software version available
> for
> > > > download. In the past, the endpoint was running on a 3rd party
> server,
> > I
> > > > believe the best way would be to put this script on ASF infra and
> have
> > > the
> > > > "update checker" running in a completely controlled environment.
> > > Actually,
> > > > I
> > > > recall we had this very discussion during the Incubation; I can
> > probably
> > > > dig
> > > > out the corresponding thread.
> > > >
> > > > Thoughts?
> > > >   Cok
> > > >
> > > > On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote:
> > > > > Hey guys
> > > > >
> > > > > In my opinion, maven.org is still owned by a third party
> (Sonatype).
> > > We
> > > > > don't know what kind of data analysis or intelligence extraction
> they
> > > > run.
> > > > >
> > > > > If Ignite servers all over the world were hitting maven.org
> > > periodically
> > > > > asking for an Ignite Maven artifact, it gives Sonatype a clear
> > > indication
> > > > > of who is running an Ignite server.
> > > > >
> > > > > They could reverse lookup the IP address and find out what
> > corporation
> > > it
> > > > > is.
> > > > >
> > > > > How about having Ignite check the ASF Git directly?
> > > > >
> > > > > We could use the Git ssh API, but that would require a new
> > dependency,
> > > > > which I advise against.
> > > > >
> > > > > Alternatively, we could have it scrape this HTML for new Git tags:
> > > > > https://git-wip-us.apache.org/repos/asf?p=ignite.git
> > > > >
> > > > > Another option is to place a txt file in the root of the master
> > branch
> > > > (e.g
> > > > > LATEST), containing a list of the latest GA versions for each major
> > > > version
> > > > > line (1.x, 2.x).
> > > > >
> > > > > I would advocate this last option, but it requires somebody
> > remembering
> > > > to
> > > > > update the file with every release, unless we automate it with a
> > Maven
> > > > > plugin.
> > > > >
> > > > > Hope that helps!
> > > > >
> > > > >
> > > > > On 24 Aug 2017 19:37, "Denis Magda" <[hidden email]> wrote:
> > > > >
> > > > > I see nothing wrong with this approach.
> > > > >
> > > > > Cos, Roman, Raul, as Apache veterans, what do you think? Is it good
> > to
> > > > go?
> > > > >
> > > > > —
> > > > > Denis
> > > > >
> > > > > > On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan <
> > > [hidden email]
> > > > >
> > > > > wrote:
> > > > > >
> > > > > > Is everyone OK with this approach? Should I file a ticket on it?
> > > > > >
> > > > > > On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <
> > > > [hidden email]>
> > > > > > wrote:
> > > > > >
> > > > > >> Igniters,
> > > > > >>
> > > > > >> There has been lots of talk of proposals about various usage
> > metrics
> > > > for
> > > > > >> Ignite and nothing came of it. I would like to resurrect the
> topic
> > > and
> > > > > >> propose something very simple and non-intrusive.
> > > > > >>
> > > > > >> 1. Update Checker
> > > > > >> The main purpose of the update checker is not to collect
> metrics,
> > > but
> > > > to
> > > > > >> notify users about a new version of Ignite by accessing
> maven.org
> > > and
> > > > > >> getting the version out of the metadata file:
> > > > > >> http://repo2.maven.org/maven2/org/apache/ignite/ignite-core/
> > > > > >> maven-metadata.xml
> > > > > >>
> > > > > >> This way we do not send any information anywhere and, at the
> same
> > > > time,
> > > > > >> urge our users to download and start using the latest version of
> > > > Ignite.
> > > > > >>
> > > > > >> 2. Startup Counter
> > > > > >> This piece is optional, but we can also get an insight in how
> many
> > > > times
> > > > > a
> > > > > >> certain Ignite release gets started. This is just a cool metric
> > for
> > > > the
> > > > > >> community to gauge the project popularity. You can think of it
> as
> > > of a
> > > > > page
> > > > > >> visit counter shown on many websites. We can even decide to
> > display
> > > > this
> > > > > >> counter on the Ignite website as well.
> > > > > >>
> > > > > >> To do this, we can simply add a JAR in maven for every release,
> > e.g.
> > > > > >> ignite-start-counter.jar, which will contain only 1 byte. Every
> > time
> > > > an
> > > > > >> Ignite node starts, it will download this JAR in the background.
> > > Then
> > > > we
> > > > > >> will be able to view the number of the total downloads for this
> > JAR
> > > in
> > > > > >> Maven Central, which is essentially the number of starts of
> Ignite
> > > > nodes.
> > > > > >>
> > > > > >> *Note that neither of the above suggestions require Ignite to
> send
> > > or
> > > > > >> track any user information whatsoever.*
> > > > > >>
> > > > > >> Please reply suggesting weather you are OK with this approach.
> > > > > >>
> > > > > >> D.
> > > > > >>
> > > >
> > > >
> > >
> >
>

[dsetrakyan]

On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani <[hidden email]>
wrote:

> Yeah, I guess that's doable as well and requires less management effort
> than my suggestion. We could use events [1] to store payload data (e.g. IP,
> version, etc.)


Yes, we could use events or some other similar API provided by GA.


> What the download page CGI developed in? PHP?
>

To be honest, no clue. I guess someone in the community can figure it out:
https://svn.apache.org/repos/asf/ignite/site/trunk/download.html


> However, I'm not sure whether storing this data in a 3rd party (Google) is
> compliant with the ASF policy. I guess it's no biggie, but if there's doubt
> in the PMC, it's better to ask legal@.


I am not sure there is anything to ask about. The whole Ignite website is
GA enabled, and all we are doing is accessing a standard web page from the
Ignite web site. The information gathered from GA is available only to the
Ignite PMC. Frankly, I think legal@ will be very confused by this question.

Even ASF website itself uses GA:
https://www.apache.org/foundation/policies/privacy.html


> I think Cos said it's OK; maybe Roman can pitch in.
>

 Sure, would be nice to hear from Roman as well.

> Cheers.
>
> [1]
> https://developers.google.com/analytics/devguides/
> collection/analyticsjs/events
>
> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan <[hidden email]
> >
> wrote:
>
> > Raul,
> >
> > Could point about Javascript, it will not work because it executes in the
> > browser. This means we need a server-side script, like CGI we are using
> on
> > our download page.
> >
> > How about this approach. We create something like ignite-version.cgi
> script
> > which will invoke a call to GA and then return the latest version.
> >
> > This should work, right?
> >
> > D.
> >
> > On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani <[hidden email]>
> > wrote:
> >
> > > Hey Dmitriy and all
> > >
> > > Also, since we have GA enabled for the website, we can track how many
> > times
> > > > this page was accessed, which will be equal to the number of starts.
> > This
> > > > way, the counter information is tracked and monitored by the Ignite
> > PMC.
> > >
> > >
> > > Unfortunately this won't work because GA is loaded via Javascript on
> the
> > > browser, so Google will never receive the page hit.
> > >
> > > Given the constraints, the most viable solution is an HTTPS endpoint
> > > running on ASF infra that Ignite invokes via a GET or POST request. The
> > > simplest thing is to write each request in a log file, along with the
> > > timestamp, the version reported by the client, maybe the IP (not sure
> > about
> > > the ASF rules about this concerning privacy, I guess it's OK if you
> make
> > it
> > > an opt-in) and a unique node identifier, i.e. a UUID the node creates
> on
> > > first startup or something.
> > >
> > > This endpoint would need some basic DDoS protection and blacklisting to
> > > prevent data spoofing.
> > >
> > > If we'll be implementing this endpoint anyway, then there's no point
> > > placing another file on Git or elsewhere for reporting the latest
> > versions:
> > > the endpoint itself can return them.
> > >
> > > WDYT?
> > >
> > > Cheers.
> > >
> > > On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan <
> > [hidden email]>
> > > wrote:
> > >
> > > > Cos, Raul,
> > > >
> > > > Thanks for the feedback. I completely agree about Maven Central
> being a
> > > 3rd
> > > > party repo (did not think about that initially). All your suggestions
> > > make
> > > > sense, but I would like to keep it as simple as possible, and so far
> > > > everything suggested required GIT dependencies and extra work.
> > > >
> > > > How about Yakov's suggestion. We simply add a page to the Ignite
> > website
> > > > which will have only the latest version. Every time a node starts, it
> > > > receives the latest version from the page and suggests that users
> > upgrade
> > > > if needed.
> > > >
> > > > Also, since we have GA enabled for the website, we can track how many
> > > times
> > > > this page was accessed, which will be equal to the number of starts.
> > This
> > > > way, the counter information is tracked and monitored by the Ignite
> > PMC.
> > > >
> > > > This approach looks pretty innocent to me and everything is kept and
> > > > managed within Apache.
> > > >
> > > > Thoughts?
> > > >
> > > > D.
> > > >
> > > >
> > > > On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik <[hidden email]
> >
> > > > wrote:
> > > >
> > > > > I agree with Raul.
> > > > > - providing a ping-back address to a 3rd party might be frown upon
> by
> > > > some.
> > > > >   And might have a consequences like stats collection about users'
> > > > >   infrastructure.
> > > > > - checking an ASF git-repo is easy and won't download any binary
> > data:
> > > > >   everything is clear text and could be easily monitored by any of
> > the
> > > > > network
> > > > >   diagnostic tools, shall it be required. But it involves a bit of
> > the
> > > > > release
> > > > >   discipline.
> > > > > - the binary data download in the runtime is my main concern. This
> is
> > > the
> > > > >   vector of MMA. What if someone gains the control over the
> > repository
> > > > and
> > > > >   replaces the file with some malicious content.
> > > > >
> > > > > As for the particular mechanism: IIRC Ignite used to make a call to
> > an
> > > > > external script to check upon the atest software version available
> > for
> > > > > download. In the past, the endpoint was running on a 3rd party
> > server,
> > > I
> > > > > believe the best way would be to put this script on ASF infra and
> > have
> > > > the
> > > > > "update checker" running in a completely controlled environment.
> > > > Actually,
> > > > > I
> > > > > recall we had this very discussion during the Incubation; I can
> > > probably
> > > > > dig
> > > > > out the corresponding thread.
> > > > >
> > > > > Thoughts?
> > > > >   Cok
> > > > >
> > > > > On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote:
> > > > > > Hey guys
> > > > > >
> > > > > > In my opinion, maven.org is still owned by a third party
> > (Sonatype).
> > > > We
> > > > > > don't know what kind of data analysis or intelligence extraction
> > they
> > > > > run.
> > > > > >
> > > > > > If Ignite servers all over the world were hitting maven.org
> > > > periodically
> > > > > > asking for an Ignite Maven artifact, it gives Sonatype a clear
> > > > indication
> > > > > > of who is running an Ignite server.
> > > > > >
> > > > > > They could reverse lookup the IP address and find out what
> > > corporation
> > > > it
> > > > > > is.
> > > > > >
> > > > > > How about having Ignite check the ASF Git directly?
> > > > > >
> > > > > > We could use the Git ssh API, but that would require a new
> > > dependency,
> > > > > > which I advise against.
> > > > > >
> > > > > > Alternatively, we could have it scrape this HTML for new Git
> tags:
> > > > > > https://git-wip-us.apache.org/repos/asf?p=ignite.git
> > > > > >
> > > > > > Another option is to place a txt file in the root of the master
> > > branch
> > > > > (e.g
> > > > > > LATEST), containing a list of the latest GA versions for each
> major
> > > > > version
> > > > > > line (1.x, 2.x).
> > > > > >
> > > > > > I would advocate this last option, but it requires somebody
> > > remembering
> > > > > to
> > > > > > update the file with every release, unless we automate it with a
> > > Maven
> > > > > > plugin.
> > > > > >
> > > > > > Hope that helps!
> > > > > >
> > > > > >
> > > > > > On 24 Aug 2017 19:37, "Denis Magda" <[hidden email]> wrote:
> > > > > >
> > > > > > I see nothing wrong with this approach.
> > > > > >
> > > > > > Cos, Roman, Raul, as Apache veterans, what do you think? Is it
> good
> > > to
> > > > > go?
> > > > > >
> > > > > > —
> > > > > > Denis
> > > > > >
> > > > > > > On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan <
> > > > [hidden email]
> > > > > >
> > > > > > wrote:
> > > > > > >
> > > > > > > Is everyone OK with this approach? Should I file a ticket on
> it?
> > > > > > >
> > > > > > > On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <
> > > > > [hidden email]>
> > > > > > > wrote:
> > > > > > >
> > > > > > >> Igniters,
> > > > > > >>
> > > > > > >> There has been lots of talk of proposals about various usage
> > > metrics
> > > > > for
> > > > > > >> Ignite and nothing came of it. I would like to resurrect the
> > topic
> > > > and
> > > > > > >> propose something very simple and non-intrusive.
> > > > > > >>
> > > > > > >> 1. Update Checker
> > > > > > >> The main purpose of the update checker is not to collect
> > metrics,
> > > > but
> > > > > to
> > > > > > >> notify users about a new version of Ignite by accessing
> > maven.org
> > > > and
> > > > > > >> getting the version out of the metadata file:
> > > > > > >> http://repo2.maven.org/maven2/org/apache/ignite/ignite-core/
> > > > > > >> maven-metadata.xml
> > > > > > >>
> > > > > > >> This way we do not send any information anywhere and, at the
> > same
> > > > > time,
> > > > > > >> urge our users to download and start using the latest version
> of
> > > > > Ignite.
> > > > > > >>
> > > > > > >> 2. Startup Counter
> > > > > > >> This piece is optional, but we can also get an insight in how
> > many
> > > > > times
> > > > > > a
> > > > > > >> certain Ignite release gets started. This is just a cool
> metric
> > > for
> > > > > the
> > > > > > >> community to gauge the project popularity. You can think of it
> > as
> > > > of a
> > > > > > page
> > > > > > >> visit counter shown on many websites. We can even decide to
> > > display
> > > > > this
> > > > > > >> counter on the Ignite website as well.
> > > > > > >>
> > > > > > >> To do this, we can simply add a JAR in maven for every
> release,
> > > e.g.
> > > > > > >> ignite-start-counter.jar, which will contain only 1 byte.
> Every
> > > time
> > > > > an
> > > > > > >> Ignite node starts, it will download this JAR in the
> background.
> > > > Then
> > > > > we
> > > > > > >> will be able to view the number of the total downloads for
> this
> > > JAR
> > > > in
> > > > > > >> Maven Central, which is essentially the number of starts of
> > Ignite
> > > > > nodes.
> > > > > > >>
> > > > > > >> *Note that neither of the above suggestions require Ignite to
> > send
> > > > or
> > > > > > >> track any user information whatsoever.*
> > > > > > >>
> > > > > > >> Please reply suggesting weather you are OK with this approach.
> > > > > > >>
> > > > > > >> D.
> > > > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>

[dsetrakyan]

I think it is safe to assume at this point that everyone is in general
agreement, since there are no active objections.

I have filed a ticket for the 2.3 release. Let's try to make it happen:
https://issues.apache.org/jira/browse/IGNITE-6305

D.

On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan <[hidden email]>
wrote:

>
>
> On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani <[hidden email]>
> wrote:
>
>> Yeah, I guess that's doable as well and requires less management effort
>> than my suggestion. We could use events [1] to store payload data (e.g.
>> IP,
>> version, etc.)
>
>
> Yes, we could use events or some other similar API provided by GA.
>
>
>> What the download page CGI developed in? PHP?
>>
>
> To be honest, no clue. I guess someone in the community can figure it out:
> https://svn.apache.org/repos/asf/ignite/site/trunk/download.html
>
>
>> However, I'm not sure whether storing this data in a 3rd party (Google) is
>> compliant with the ASF policy. I guess it's no biggie, but if there's
>> doubt
>> in the PMC, it's better to ask legal@.
>
>
> I am not sure there is anything to ask about. The whole Ignite website is
> GA enabled, and all we are doing is accessing a standard web page from the
> Ignite web site. The information gathered from GA is available only to the
> Ignite PMC. Frankly, I think legal@ will be very confused by this
> question.
>
> Even ASF website itself uses GA: https://www.apache.org/
> foundation/policies/privacy.html
>
>
>> I think Cos said it's OK; maybe Roman can pitch in.
>>
>
>  Sure, would be nice to hear from Roman as well.
>
>
>> Cheers.
>>
>> [1]
>> https://developers.google.com/analytics/devguides/collection
>> /analyticsjs/events
>>
>> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan <
>> [hidden email]>
>> wrote:
>>
>> > Raul,
>> >
>> > Could point about Javascript, it will not work because it executes in
>> the
>> > browser. This means we need a server-side script, like CGI we are using
>> on
>> > our download page.
>> >
>> > How about this approach. We create something like ignite-version.cgi
>> script
>> > which will invoke a call to GA and then return the latest version.
>> >
>> > This should work, right?
>> >
>> > D.
>> >
>> > On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani <[hidden email]>
>> > wrote:
>> >
>> > > Hey Dmitriy and all
>> > >
>> > > Also, since we have GA enabled for the website, we can track how many
>> > times
>> > > > this page was accessed, which will be equal to the number of starts.
>> > This
>> > > > way, the counter information is tracked and monitored by the Ignite
>> > PMC.
>> > >
>> > >
>> > > Unfortunately this won't work because GA is loaded via Javascript on
>> the
>> > > browser, so Google will never receive the page hit.
>> > >
>> > > Given the constraints, the most viable solution is an HTTPS endpoint
>> > > running on ASF infra that Ignite invokes via a GET or POST request.
>> The
>> > > simplest thing is to write each request in a log file, along with the
>> > > timestamp, the version reported by the client, maybe the IP (not sure
>> > about
>> > > the ASF rules about this concerning privacy, I guess it's OK if you
>> make
>> > it
>> > > an opt-in) and a unique node identifier, i.e. a UUID the node creates
>> on
>> > > first startup or something.
>> > >
>> > > This endpoint would need some basic DDoS protection and blacklisting
>> to
>> > > prevent data spoofing.
>> > >
>> > > If we'll be implementing this endpoint anyway, then there's no point
>> > > placing another file on Git or elsewhere for reporting the latest
>> > versions:
>> > > the endpoint itself can return them.
>> > >
>> > > WDYT?
>> > >
>> > > Cheers.
>> > >
>> > > On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan <
>> > [hidden email]>
>> > > wrote:
>> > >
>> > > > Cos, Raul,
>> > > >
>> > > > Thanks for the feedback. I completely agree about Maven Central
>> being a
>> > > 3rd
>> > > > party repo (did not think about that initially). All your
>> suggestions
>> > > make
>> > > > sense, but I would like to keep it as simple as possible, and so far
>> > > > everything suggested required GIT dependencies and extra work.
>> > > >
>> > > > How about Yakov's suggestion. We simply add a page to the Ignite
>> > website
>> > > > which will have only the latest version. Every time a node starts,
>> it
>> > > > receives the latest version from the page and suggests that users
>> > upgrade
>> > > > if needed.
>> > > >
>> > > > Also, since we have GA enabled for the website, we can track how
>> many
>> > > times
>> > > > this page was accessed, which will be equal to the number of starts.
>> > This
>> > > > way, the counter information is tracked and monitored by the Ignite
>> > PMC.
>> > > >
>> > > > This approach looks pretty innocent to me and everything is kept and
>> > > > managed within Apache.
>> > > >
>> > > > Thoughts?
>> > > >
>> > > > D.
>> > > >
>> > > >
>> > > > On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik <
>> [hidden email]>
>> > > > wrote:
>> > > >
>> > > > > I agree with Raul.
>> > > > > - providing a ping-back address to a 3rd party might be frown
>> upon by
>> > > > some.
>> > > > >   And might have a consequences like stats collection about users'
>> > > > >   infrastructure.
>> > > > > - checking an ASF git-repo is easy and won't download any binary
>> > data:
>> > > > >   everything is clear text and could be easily monitored by any of
>> > the
>> > > > > network
>> > > > >   diagnostic tools, shall it be required. But it involves a bit of
>> > the
>> > > > > release
>> > > > >   discipline.
>> > > > > - the binary data download in the runtime is my main concern.
>> This is
>> > > the
>> > > > >   vector of MMA. What if someone gains the control over the
>> > repository
>> > > > and
>> > > > >   replaces the file with some malicious content.
>> > > > >
>> > > > > As for the particular mechanism: IIRC Ignite used to make a call
>> to
>> > an
>> > > > > external script to check upon the atest software version available
>> > for
>> > > > > download. In the past, the endpoint was running on a 3rd party
>> > server,
>> > > I
>> > > > > believe the best way would be to put this script on ASF infra and
>> > have
>> > > > the
>> > > > > "update checker" running in a completely controlled environment.
>> > > > Actually,
>> > > > > I
>> > > > > recall we had this very discussion during the Incubation; I can
>> > > probably
>> > > > > dig
>> > > > > out the corresponding thread.
>> > > > >
>> > > > > Thoughts?
>> > > > >   Cok
>> > > > >
>> > > > > On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote:
>> > > > > > Hey guys
>> > > > > >
>> > > > > > In my opinion, maven.org is still owned by a third party
>> > (Sonatype).
>> > > > We
>> > > > > > don't know what kind of data analysis or intelligence extraction
>> > they
>> > > > > run.
>> > > > > >
>> > > > > > If Ignite servers all over the world were hitting maven.org
>> > > > periodically
>> > > > > > asking for an Ignite Maven artifact, it gives Sonatype a clear
>> > > > indication
>> > > > > > of who is running an Ignite server.
>> > > > > >
>> > > > > > They could reverse lookup the IP address and find out what
>> > > corporation
>> > > > it
>> > > > > > is.
>> > > > > >
>> > > > > > How about having Ignite check the ASF Git directly?
>> > > > > >
>> > > > > > We could use the Git ssh API, but that would require a new
>> > > dependency,
>> > > > > > which I advise against.
>> > > > > >
>> > > > > > Alternatively, we could have it scrape this HTML for new Git
>> tags:
>> > > > > > https://git-wip-us.apache.org/repos/asf?p=ignite.git
>> > > > > >
>> > > > > > Another option is to place a txt file in the root of the master
>> > > branch
>> > > > > (e.g
>> > > > > > LATEST), containing a list of the latest GA versions for each
>> major
>> > > > > version
>> > > > > > line (1.x, 2.x).
>> > > > > >
>> > > > > > I would advocate this last option, but it requires somebody
>> > > remembering
>> > > > > to
>> > > > > > update the file with every release, unless we automate it with a
>> > > Maven
>> > > > > > plugin.
>> > > > > >
>> > > > > > Hope that helps!
>> > > > > >
>> > > > > >
>> > > > > > On 24 Aug 2017 19:37, "Denis Magda" <[hidden email]> wrote:
>> > > > > >
>> > > > > > I see nothing wrong with this approach.
>> > > > > >
>> > > > > > Cos, Roman, Raul, as Apache veterans, what do you think? Is it
>> good
>> > > to
>> > > > > go?
>> > > > > >
>> > > > > > —
>> > > > > > Denis
>> > > > > >
>> > > > > > > On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan <
>> > > > [hidden email]
>> > > > > >
>> > > > > > wrote:
>> > > > > > >
>> > > > > > > Is everyone OK with this approach? Should I file a ticket on
>> it?
>> > > > > > >
>> > > > > > > On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <
>> > > > > [hidden email]>
>> > > > > > > wrote:
>> > > > > > >
>> > > > > > >> Igniters,
>> > > > > > >>
>> > > > > > >> There has been lots of talk of proposals about various usage
>> > > metrics
>> > > > > for
>> > > > > > >> Ignite and nothing came of it. I would like to resurrect the
>> > topic
>> > > > and
>> > > > > > >> propose something very simple and non-intrusive.
>> > > > > > >>
>> > > > > > >> 1. Update Checker
>> > > > > > >> The main purpose of the update checker is not to collect
>> > metrics,
>> > > > but
>> > > > > to
>> > > > > > >> notify users about a new version of Ignite by accessing
>> > maven.org
>> > > > and
>> > > > > > >> getting the version out of the metadata file:
>> > > > > > >> http://repo2.maven.org/maven2/org/apache/ignite/ignite-core/
>> > > > > > >> maven-metadata.xml
>> > > > > > >>
>> > > > > > >> This way we do not send any information anywhere and, at the
>> > same
>> > > > > time,
>> > > > > > >> urge our users to download and start using the latest
>> version of
>> > > > > Ignite.
>> > > > > > >>
>> > > > > > >> 2. Startup Counter
>> > > > > > >> This piece is optional, but we can also get an insight in how
>> > many
>> > > > > times
>> > > > > > a
>> > > > > > >> certain Ignite release gets started. This is just a cool
>> metric
>> > > for
>> > > > > the
>> > > > > > >> community to gauge the project popularity. You can think of
>> it
>> > as
>> > > > of a
>> > > > > > page
>> > > > > > >> visit counter shown on many websites. We can even decide to
>> > > display
>> > > > > this
>> > > > > > >> counter on the Ignite website as well.
>> > > > > > >>
>> > > > > > >> To do this, we can simply add a JAR in maven for every
>> release,
>> > > e.g.
>> > > > > > >> ignite-start-counter.jar, which will contain only 1 byte.
>> Every
>> > > time
>> > > > > an
>> > > > > > >> Ignite node starts, it will download this JAR in the
>> background.
>> > > > Then
>> > > > > we
>> > > > > > >> will be able to view the number of the total downloads for
>> this
>> > > JAR
>> > > > in
>> > > > > > >> Maven Central, which is essentially the number of starts of
>> > Ignite
>> > > > > nodes.
>> > > > > > >>
>> > > > > > >> *Note that neither of the above suggestions require Ignite to
>> > send
>> > > > or
>> > > > > > >> track any user information whatsoever.*
>> > > > > > >>
>> > > > > > >> Please reply suggesting weather you are OK with this
>> approach.
>> > > > > > >>
>> > > > > > >> D.
>> > > > > > >>
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>

[Vladimir Ozerov]

Folks,

Can we add version of current node to web request? This way we will better
understand version distribution, what might help us with certain API
update/deprecate decisions
E.g. http://ignite.apache.org/latest.cgi&ver=2.2.0

Vladimir.

On Fri, Sep 8, 2017 at 7:06 AM, Dmitriy Setrakyan <[hidden email]>
wrote:

> I think it is safe to assume at this point that everyone is in general
> agreement, since there are no active objections.
>
> I have filed a ticket for the 2.3 release. Let's try to make it happen:
> https://issues.apache.org/jira/browse/IGNITE-6305
>
> D.
>
> On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan <[hidden email]>
> wrote:
>
> >
> >
> > On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani <[hidden email]>
> > wrote:
> >
> >> Yeah, I guess that's doable as well and requires less management effort
> >> than my suggestion. We could use events [1] to store payload data (e.g.
> >> IP,
> >> version, etc.)
> >
> >
> > Yes, we could use events or some other similar API provided by GA.
> >
> >
> >> What the download page CGI developed in? PHP?
> >>
> >
> > To be honest, no clue. I guess someone in the community can figure it
> out:
> > https://svn.apache.org/repos/asf/ignite/site/trunk/download.html
> >
> >
> >> However, I'm not sure whether storing this data in a 3rd party (Google)
> is
> >> compliant with the ASF policy. I guess it's no biggie, but if there's
> >> doubt
> >> in the PMC, it's better to ask legal@.
> >
> >
> > I am not sure there is anything to ask about. The whole Ignite website is
> > GA enabled, and all we are doing is accessing a standard web page from
> the
> > Ignite web site. The information gathered from GA is available only to
> the
> > Ignite PMC. Frankly, I think legal@ will be very confused by this
> > question.
> >
> > Even ASF website itself uses GA: https://www.apache.org/
> > foundation/policies/privacy.html
> >
> >
> >> I think Cos said it's OK; maybe Roman can pitch in.
> >>
> >
> >  Sure, would be nice to hear from Roman as well.
> >
> >
> >> Cheers.
> >>
> >> [1]
> >> https://developers.google.com/analytics/devguides/collection
> >> /analyticsjs/events
> >>
> >> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan <
> >> [hidden email]>
> >> wrote:
> >>
> >> > Raul,
> >> >
> >> > Could point about Javascript, it will not work because it executes in
> >> the
> >> > browser. This means we need a server-side script, like CGI we are
> using
> >> on
> >> > our download page.
> >> >
> >> > How about this approach. We create something like ignite-version.cgi
> >> script
> >> > which will invoke a call to GA and then return the latest version.
> >> >
> >> > This should work, right?
> >> >
> >> > D.
> >> >
> >> > On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani <[hidden email]
> >
> >> > wrote:
> >> >
> >> > > Hey Dmitriy and all
> >> > >
> >> > > Also, since we have GA enabled for the website, we can track how
> many
> >> > times
> >> > > > this page was accessed, which will be equal to the number of
> starts.
> >> > This
> >> > > > way, the counter information is tracked and monitored by the
> Ignite
> >> > PMC.
> >> > >
> >> > >
> >> > > Unfortunately this won't work because GA is loaded via Javascript on
> >> the
> >> > > browser, so Google will never receive the page hit.
> >> > >
> >> > > Given the constraints, the most viable solution is an HTTPS endpoint
> >> > > running on ASF infra that Ignite invokes via a GET or POST request.
> >> The
> >> > > simplest thing is to write each request in a log file, along with
> the
> >> > > timestamp, the version reported by the client, maybe the IP (not
> sure
> >> > about
> >> > > the ASF rules about this concerning privacy, I guess it's OK if you
> >> make
> >> > it
> >> > > an opt-in) and a unique node identifier, i.e. a UUID the node
> creates
> >> on
> >> > > first startup or something.
> >> > >
> >> > > This endpoint would need some basic DDoS protection and blacklisting
> >> to
> >> > > prevent data spoofing.
> >> > >
> >> > > If we'll be implementing this endpoint anyway, then there's no point
> >> > > placing another file on Git or elsewhere for reporting the latest
> >> > versions:
> >> > > the endpoint itself can return them.
> >> > >
> >> > > WDYT?
> >> > >
> >> > > Cheers.
> >> > >
> >> > > On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan <
> >> > [hidden email]>
> >> > > wrote:
> >> > >
> >> > > > Cos, Raul,
> >> > > >
> >> > > > Thanks for the feedback. I completely agree about Maven Central
> >> being a
> >> > > 3rd
> >> > > > party repo (did not think about that initially). All your
> >> suggestions
> >> > > make
> >> > > > sense, but I would like to keep it as simple as possible, and so
> far
> >> > > > everything suggested required GIT dependencies and extra work.
> >> > > >
> >> > > > How about Yakov's suggestion. We simply add a page to the Ignite
> >> > website
> >> > > > which will have only the latest version. Every time a node starts,
> >> it
> >> > > > receives the latest version from the page and suggests that users
> >> > upgrade
> >> > > > if needed.
> >> > > >
> >> > > > Also, since we have GA enabled for the website, we can track how
> >> many
> >> > > times
> >> > > > this page was accessed, which will be equal to the number of
> starts.
> >> > This
> >> > > > way, the counter information is tracked and monitored by the
> Ignite
> >> > PMC.
> >> > > >
> >> > > > This approach looks pretty innocent to me and everything is kept
> and
> >> > > > managed within Apache.
> >> > > >
> >> > > > Thoughts?
> >> > > >
> >> > > > D.
> >> > > >
> >> > > >
> >> > > > On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik <
> >> [hidden email]>
> >> > > > wrote:
> >> > > >
> >> > > > > I agree with Raul.
> >> > > > > - providing a ping-back address to a 3rd party might be frown
> >> upon by
> >> > > > some.
> >> > > > >   And might have a consequences like stats collection about
> users'
> >> > > > >   infrastructure.
> >> > > > > - checking an ASF git-repo is easy and won't download any binary
> >> > data:
> >> > > > >   everything is clear text and could be easily monitored by any
> of
> >> > the
> >> > > > > network
> >> > > > >   diagnostic tools, shall it be required. But it involves a bit
> of
> >> > the
> >> > > > > release
> >> > > > >   discipline.
> >> > > > > - the binary data download in the runtime is my main concern.
> >> This is
> >> > > the
> >> > > > >   vector of MMA. What if someone gains the control over the
> >> > repository
> >> > > > and
> >> > > > >   replaces the file with some malicious content.
> >> > > > >
> >> > > > > As for the particular mechanism: IIRC Ignite used to make a call
> >> to
> >> > an
> >> > > > > external script to check upon the atest software version
> available
> >> > for
> >> > > > > download. In the past, the endpoint was running on a 3rd party
> >> > server,
> >> > > I
> >> > > > > believe the best way would be to put this script on ASF infra
> and
> >> > have
> >> > > > the
> >> > > > > "update checker" running in a completely controlled environment.
> >> > > > Actually,
> >> > > > > I
> >> > > > > recall we had this very discussion during the Incubation; I can
> >> > > probably
> >> > > > > dig
> >> > > > > out the corresponding thread.
> >> > > > >
> >> > > > > Thoughts?
> >> > > > >   Cok
> >> > > > >
> >> > > > > On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote:
> >> > > > > > Hey guys
> >> > > > > >
> >> > > > > > In my opinion, maven.org is still owned by a third party
> >> > (Sonatype).
> >> > > > We
> >> > > > > > don't know what kind of data analysis or intelligence
> extraction
> >> > they
> >> > > > > run.
> >> > > > > >
> >> > > > > > If Ignite servers all over the world were hitting maven.org
> >> > > > periodically
> >> > > > > > asking for an Ignite Maven artifact, it gives Sonatype a clear
> >> > > > indication
> >> > > > > > of who is running an Ignite server.
> >> > > > > >
> >> > > > > > They could reverse lookup the IP address and find out what
> >> > > corporation
> >> > > > it
> >> > > > > > is.
> >> > > > > >
> >> > > > > > How about having Ignite check the ASF Git directly?
> >> > > > > >
> >> > > > > > We could use the Git ssh API, but that would require a new
> >> > > dependency,
> >> > > > > > which I advise against.
> >> > > > > >
> >> > > > > > Alternatively, we could have it scrape this HTML for new Git
> >> tags:
> >> > > > > > https://git-wip-us.apache.org/repos/asf?p=ignite.git
> >> > > > > >
> >> > > > > > Another option is to place a txt file in the root of the
> master
> >> > > branch
> >> > > > > (e.g
> >> > > > > > LATEST), containing a list of the latest GA versions for each
> >> major
> >> > > > > version
> >> > > > > > line (1.x, 2.x).
> >> > > > > >
> >> > > > > > I would advocate this last option, but it requires somebody
> >> > > remembering
> >> > > > > to
> >> > > > > > update the file with every release, unless we automate it
> with a
> >> > > Maven
> >> > > > > > plugin.
> >> > > > > >
> >> > > > > > Hope that helps!
> >> > > > > >
> >> > > > > >
> >> > > > > > On 24 Aug 2017 19:37, "Denis Magda" <[hidden email]>
> wrote:
> >> > > > > >
> >> > > > > > I see nothing wrong with this approach.
> >> > > > > >
> >> > > > > > Cos, Roman, Raul, as Apache veterans, what do you think? Is it
> >> good
> >> > > to
> >> > > > > go?
> >> > > > > >
> >> > > > > > —
> >> > > > > > Denis
> >> > > > > >
> >> > > > > > > On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan <
> >> > > > [hidden email]
> >> > > > > >
> >> > > > > > wrote:
> >> > > > > > >
> >> > > > > > > Is everyone OK with this approach? Should I file a ticket on
> >> it?
> >> > > > > > >
> >> > > > > > > On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <
> >> > > > > [hidden email]>
> >> > > > > > > wrote:
> >> > > > > > >
> >> > > > > > >> Igniters,
> >> > > > > > >>
> >> > > > > > >> There has been lots of talk of proposals about various
> usage
> >> > > metrics
> >> > > > > for
> >> > > > > > >> Ignite and nothing came of it. I would like to resurrect
> the
> >> > topic
> >> > > > and
> >> > > > > > >> propose something very simple and non-intrusive.
> >> > > > > > >>
> >> > > > > > >> 1. Update Checker
> >> > > > > > >> The main purpose of the update checker is not to collect
> >> > metrics,
> >> > > > but
> >> > > > > to
> >> > > > > > >> notify users about a new version of Ignite by accessing
> >> > maven.org
> >> > > > and
> >> > > > > > >> getting the version out of the metadata file:
> >> > > > > > >> http://repo2.maven.org/maven2/
> org/apache/ignite/ignite-core/
> >> > > > > > >> maven-metadata.xml
> >> > > > > > >>
> >> > > > > > >> This way we do not send any information anywhere and, at
> the
> >> > same
> >> > > > > time,
> >> > > > > > >> urge our users to download and start using the latest
> >> version of
> >> > > > > Ignite.
> >> > > > > > >>
> >> > > > > > >> 2. Startup Counter
> >> > > > > > >> This piece is optional, but we can also get an insight in
> how
> >> > many
> >> > > > > times
> >> > > > > > a
> >> > > > > > >> certain Ignite release gets started. This is just a cool
> >> metric
> >> > > for
> >> > > > > the
> >> > > > > > >> community to gauge the project popularity. You can think of
> >> it
> >> > as
> >> > > > of a
> >> > > > > > page
> >> > > > > > >> visit counter shown on many websites. We can even decide to
> >> > > display
> >> > > > > this
> >> > > > > > >> counter on the Ignite website as well.
> >> > > > > > >>
> >> > > > > > >> To do this, we can simply add a JAR in maven for every
> >> release,
> >> > > e.g.
> >> > > > > > >> ignite-start-counter.jar, which will contain only 1 byte.
> >> Every
> >> > > time
> >> > > > > an
> >> > > > > > >> Ignite node starts, it will download this JAR in the
> >> background.
> >> > > > Then
> >> > > > > we
> >> > > > > > >> will be able to view the number of the total downloads for
> >> this
> >> > > JAR
> >> > > > in
> >> > > > > > >> Maven Central, which is essentially the number of starts of
> >> > Ignite
> >> > > > > nodes.
> >> > > > > > >>
> >> > > > > > >> *Note that neither of the above suggestions require Ignite
> to
> >> > send
> >> > > > or
> >> > > > > > >> track any user information whatsoever.*
> >> > > > > > >>
> >> > > > > > >> Please reply suggesting weather you are OK with this
> >> approach.
> >> > > > > > >>
> >> > > > > > >> D.
> >> > > > > > >>
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>

[dsetrakyan]

On Tue, Sep 26, 2017 at 6:20 AM, Vladimir Ozerov <[hidden email]>
wrote:

> Folks,
>
> Can we add version of current node to web request? This way we will better
> understand version distribution, what might help us with certain API
> update/deprecate decisions
> E.g. http://ignite.apache.org/latest.cgi&ver=2.2.0


Vladimir, I personally do not see a problem with that, as sending the
current version to the update checker seems very innocent to me. At the
same time, it will allow us to examine the usage of each release and make
decisions about dropping backward compatibility or spotting bugs for a
certain release.

Cos, Raul, any thoughts?

>
>
> Vladimir.
>
> On Fri, Sep 8, 2017 at 7:06 AM, Dmitriy Setrakyan <[hidden email]>
> wrote:
>
> > I think it is safe to assume at this point that everyone is in general
> > agreement, since there are no active objections.
> >
> > I have filed a ticket for the 2.3 release. Let's try to make it happen:
> > https://issues.apache.org/jira/browse/IGNITE-6305
> >
> > D.
> >
> > On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan <
> [hidden email]>
> > wrote:
> >
> > >
> > >
> > > On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani <[hidden email]>
> > > wrote:
> > >
> > >> Yeah, I guess that's doable as well and requires less management
> effort
> > >> than my suggestion. We could use events [1] to store payload data
> (e.g.
> > >> IP,
> > >> version, etc.)
> > >
> > >
> > > Yes, we could use events or some other similar API provided by GA.
> > >
> > >
> > >> What the download page CGI developed in? PHP?
> > >>
> > >
> > > To be honest, no clue. I guess someone in the community can figure it
> > out:
> > > https://svn.apache.org/repos/asf/ignite/site/trunk/download.html
> > >
> > >
> > >> However, I'm not sure whether storing this data in a 3rd party
> (Google)
> > is
> > >> compliant with the ASF policy. I guess it's no biggie, but if there's
> > >> doubt
> > >> in the PMC, it's better to ask legal@.
> > >
> > >
> > > I am not sure there is anything to ask about. The whole Ignite website
> is
> > > GA enabled, and all we are doing is accessing a standard web page from
> > the
> > > Ignite web site. The information gathered from GA is available only to
> > the
> > > Ignite PMC. Frankly, I think legal@ will be very confused by this
> > > question.
> > >
> > > Even ASF website itself uses GA: https://www.apache.org/
> > > foundation/policies/privacy.html
> > >
> > >
> > >> I think Cos said it's OK; maybe Roman can pitch in.
> > >>
> > >
> > >  Sure, would be nice to hear from Roman as well.
> > >
> > >
> > >> Cheers.
> > >>
> > >> [1]
> > >> https://developers.google.com/analytics/devguides/collection
> > >> /analyticsjs/events
> > >>
> > >> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan <
> > >> [hidden email]>
> > >> wrote:
> > >>
> > >> > Raul,
> > >> >
> > >> > Could point about Javascript, it will not work because it executes
> in
> > >> the
> > >> > browser. This means we need a server-side script, like CGI we are
> > using
> > >> on
> > >> > our download page.
> > >> >
> > >> > How about this approach. We create something like ignite-version.cgi
> > >> script
> > >> > which will invoke a call to GA and then return the latest version.
> > >> >
> > >> > This should work, right?
> > >> >
> > >> > D.
> > >> >
> > >> > On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani <
> [hidden email]
> > >
> > >> > wrote:
> > >> >
> > >> > > Hey Dmitriy and all
> > >> > >
> > >> > > Also, since we have GA enabled for the website, we can track how
> > many
> > >> > times
> > >> > > > this page was accessed, which will be equal to the number of
> > starts.
> > >> > This
> > >> > > > way, the counter information is tracked and monitored by the
> > Ignite
> > >> > PMC.
> > >> > >
> > >> > >
> > >> > > Unfortunately this won't work because GA is loaded via Javascript
> on
> > >> the
> > >> > > browser, so Google will never receive the page hit.
> > >> > >
> > >> > > Given the constraints, the most viable solution is an HTTPS
> endpoint
> > >> > > running on ASF infra that Ignite invokes via a GET or POST
> request.
> > >> The
> > >> > > simplest thing is to write each request in a log file, along with
> > the
> > >> > > timestamp, the version reported by the client, maybe the IP (not
> > sure
> > >> > about
> > >> > > the ASF rules about this concerning privacy, I guess it's OK if
> you
> > >> make
> > >> > it
> > >> > > an opt-in) and a unique node identifier, i.e. a UUID the node
> > creates
> > >> on
> > >> > > first startup or something.
> > >> > >
> > >> > > This endpoint would need some basic DDoS protection and
> blacklisting
> > >> to
> > >> > > prevent data spoofing.
> > >> > >
> > >> > > If we'll be implementing this endpoint anyway, then there's no
> point
> > >> > > placing another file on Git or elsewhere for reporting the latest
> > >> > versions:
> > >> > > the endpoint itself can return them.
> > >> > >
> > >> > > WDYT?
> > >> > >
> > >> > > Cheers.
> > >> > >
> > >> > > On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan <
> > >> > [hidden email]>
> > >> > > wrote:
> > >> > >
> > >> > > > Cos, Raul,
> > >> > > >
> > >> > > > Thanks for the feedback. I completely agree about Maven Central
> > >> being a
> > >> > > 3rd
> > >> > > > party repo (did not think about that initially). All your
> > >> suggestions
> > >> > > make
> > >> > > > sense, but I would like to keep it as simple as possible, and so
> > far
> > >> > > > everything suggested required GIT dependencies and extra work.
> > >> > > >
> > >> > > > How about Yakov's suggestion. We simply add a page to the Ignite
> > >> > website
> > >> > > > which will have only the latest version. Every time a node
> starts,
> > >> it
> > >> > > > receives the latest version from the page and suggests that
> users
> > >> > upgrade
> > >> > > > if needed.
> > >> > > >
> > >> > > > Also, since we have GA enabled for the website, we can track how
> > >> many
> > >> > > times
> > >> > > > this page was accessed, which will be equal to the number of
> > starts.
> > >> > This
> > >> > > > way, the counter information is tracked and monitored by the
> > Ignite
> > >> > PMC.
> > >> > > >
> > >> > > > This approach looks pretty innocent to me and everything is kept
> > and
> > >> > > > managed within Apache.
> > >> > > >
> > >> > > > Thoughts?
> > >> > > >
> > >> > > > D.
> > >> > > >
> > >> > > >
> > >> > > > On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik <
> > >> [hidden email]>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > I agree with Raul.
> > >> > > > > - providing a ping-back address to a 3rd party might be frown
> > >> upon by
> > >> > > > some.
> > >> > > > >   And might have a consequences like stats collection about
> > users'
> > >> > > > >   infrastructure.
> > >> > > > > - checking an ASF git-repo is easy and won't download any
> binary
> > >> > data:
> > >> > > > >   everything is clear text and could be easily monitored by
> any
> > of
> > >> > the
> > >> > > > > network
> > >> > > > >   diagnostic tools, shall it be required. But it involves a
> bit
> > of
> > >> > the
> > >> > > > > release
> > >> > > > >   discipline.
> > >> > > > > - the binary data download in the runtime is my main concern.
> > >> This is
> > >> > > the
> > >> > > > >   vector of MMA. What if someone gains the control over the
> > >> > repository
> > >> > > > and
> > >> > > > >   replaces the file with some malicious content.
> > >> > > > >
> > >> > > > > As for the particular mechanism: IIRC Ignite used to make a
> call
> > >> to
> > >> > an
> > >> > > > > external script to check upon the atest software version
> > available
> > >> > for
> > >> > > > > download. In the past, the endpoint was running on a 3rd party
> > >> > server,
> > >> > > I
> > >> > > > > believe the best way would be to put this script on ASF infra
> > and
> > >> > have
> > >> > > > the
> > >> > > > > "update checker" running in a completely controlled
> environment.
> > >> > > > Actually,
> > >> > > > > I
> > >> > > > > recall we had this very discussion during the Incubation; I
> can
> > >> > > probably
> > >> > > > > dig
> > >> > > > > out the corresponding thread.
> > >> > > > >
> > >> > > > > Thoughts?
> > >> > > > >   Cok
> > >> > > > >
> > >> > > > > On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote:
> > >> > > > > > Hey guys
> > >> > > > > >
> > >> > > > > > In my opinion, maven.org is still owned by a third party
> > >> > (Sonatype).
> > >> > > > We
> > >> > > > > > don't know what kind of data analysis or intelligence
> > extraction
> > >> > they
> > >> > > > > run.
> > >> > > > > >
> > >> > > > > > If Ignite servers all over the world were hitting maven.org
> > >> > > > periodically
> > >> > > > > > asking for an Ignite Maven artifact, it gives Sonatype a
> clear
> > >> > > > indication
> > >> > > > > > of who is running an Ignite server.
> > >> > > > > >
> > >> > > > > > They could reverse lookup the IP address and find out what
> > >> > > corporation
> > >> > > > it
> > >> > > > > > is.
> > >> > > > > >
> > >> > > > > > How about having Ignite check the ASF Git directly?
> > >> > > > > >
> > >> > > > > > We could use the Git ssh API, but that would require a new
> > >> > > dependency,
> > >> > > > > > which I advise against.
> > >> > > > > >
> > >> > > > > > Alternatively, we could have it scrape this HTML for new Git
> > >> tags:
> > >> > > > > > https://git-wip-us.apache.org/repos/asf?p=ignite.git
> > >> > > > > >
> > >> > > > > > Another option is to place a txt file in the root of the
> > master
> > >> > > branch
> > >> > > > > (e.g
> > >> > > > > > LATEST), containing a list of the latest GA versions for
> each
> > >> major
> > >> > > > > version
> > >> > > > > > line (1.x, 2.x).
> > >> > > > > >
> > >> > > > > > I would advocate this last option, but it requires somebody
> > >> > > remembering
> > >> > > > > to
> > >> > > > > > update the file with every release, unless we automate it
> > with a
> > >> > > Maven
> > >> > > > > > plugin.
> > >> > > > > >
> > >> > > > > > Hope that helps!
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > On 24 Aug 2017 19:37, "Denis Magda" <[hidden email]>
> > wrote:
> > >> > > > > >
> > >> > > > > > I see nothing wrong with this approach.
> > >> > > > > >
> > >> > > > > > Cos, Roman, Raul, as Apache veterans, what do you think? Is
> it
> > >> good
> > >> > > to
> > >> > > > > go?
> > >> > > > > >
> > >> > > > > > —
> > >> > > > > > Denis
> > >> > > > > >
> > >> > > > > > > On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan <
> > >> > > > [hidden email]
> > >> > > > > >
> > >> > > > > > wrote:
> > >> > > > > > >
> > >> > > > > > > Is everyone OK with this approach? Should I file a ticket
> on
> > >> it?
> > >> > > > > > >
> > >> > > > > > > On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <
> > >> > > > > [hidden email]>
> > >> > > > > > > wrote:
> > >> > > > > > >
> > >> > > > > > >> Igniters,
> > >> > > > > > >>
> > >> > > > > > >> There has been lots of talk of proposals about various
> > usage
> > >> > > metrics
> > >> > > > > for
> > >> > > > > > >> Ignite and nothing came of it. I would like to resurrect
> > the
> > >> > topic
> > >> > > > and
> > >> > > > > > >> propose something very simple and non-intrusive.
> > >> > > > > > >>
> > >> > > > > > >> 1. Update Checker
> > >> > > > > > >> The main purpose of the update checker is not to collect
> > >> > metrics,
> > >> > > > but
> > >> > > > > to
> > >> > > > > > >> notify users about a new version of Ignite by accessing
> > >> > maven.org
> > >> > > > and
> > >> > > > > > >> getting the version out of the metadata file:
> > >> > > > > > >> http://repo2.maven.org/maven2/
> > org/apache/ignite/ignite-core/
> > >> > > > > > >> maven-metadata.xml
> > >> > > > > > >>
> > >> > > > > > >> This way we do not send any information anywhere and, at
> > the
> > >> > same
> > >> > > > > time,
> > >> > > > > > >> urge our users to download and start using the latest
> > >> version of
> > >> > > > > Ignite.
> > >> > > > > > >>
> > >> > > > > > >> 2. Startup Counter
> > >> > > > > > >> This piece is optional, but we can also get an insight in
> > how
> > >> > many
> > >> > > > > times
> > >> > > > > > a
> > >> > > > > > >> certain Ignite release gets started. This is just a cool
> > >> metric
> > >> > > for
> > >> > > > > the
> > >> > > > > > >> community to gauge the project popularity. You can think
> of
> > >> it
> > >> > as
> > >> > > > of a
> > >> > > > > > page
> > >> > > > > > >> visit counter shown on many websites. We can even decide
> to
> > >> > > display
> > >> > > > > this
> > >> > > > > > >> counter on the Ignite website as well.
> > >> > > > > > >>
> > >> > > > > > >> To do this, we can simply add a JAR in maven for every
> > >> release,
> > >> > > e.g.
> > >> > > > > > >> ignite-start-counter.jar, which will contain only 1 byte.
> > >> Every
> > >> > > time
> > >> > > > > an
> > >> > > > > > >> Ignite node starts, it will download this JAR in the
> > >> background.
> > >> > > > Then
> > >> > > > > we
> > >> > > > > > >> will be able to view the number of the total downloads
> for
> > >> this
> > >> > > JAR
> > >> > > > in
> > >> > > > > > >> Maven Central, which is essentially the number of starts
> of
> > >> > Ignite
> > >> > > > > nodes.
> > >> > > > > > >>
> > >> > > > > > >> *Note that neither of the above suggestions require
> Ignite
> > to
> > >> > send
> > >> > > > or
> > >> > > > > > >> track any user information whatsoever.*
> > >> > > > > > >>
> > >> > > > > > >> Please reply suggesting weather you are OK with this
> > >> approach.
> > >> > > > > > >>
> > >> > > > > > >> D.
> > >> > > > > > >>
> > >> > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>

[Konstantin Boudnik-2]

I don't see an issue with node version either. 

Just one more, and it might be slightly irrelevant, issue though... I looked at the Ignite's site and found the following ads and trackers (that are indeed getting disabled by my browser).

Why are googleads or doubleclick are permitted?

​

Thanks,

  Cos
​

On Tue, Sep 26, 2017 at 3:21 PM, Dmitriy Setrakyan <[hidden email]> wrote:

On Tue, Sep 26, 2017 at 6:20 AM, Vladimir Ozerov <[hidden email]>
wrote:

> Folks,
>
> Can we add version of current node to web request? This way we will better
> understand version distribution, what might help us with certain API
> update/deprecate decisions
> E.g. http://ignite.apache.org/latest.cgi&ver=2.2.0


Vladimir, I personally do not see a problem with that, as sending the
current version to the update checker seems very innocent to me. At the
same time, it will allow us to examine the usage of each release and make
decisions about dropping backward compatibility or spotting bugs for a
certain release.

Cos, Raul, any thoughts?

>
>
> Vladimir.
>
> On Fri, Sep 8, 2017 at 7:06 AM, Dmitriy Setrakyan <[hidden email]>
> wrote:
>
> > I think it is safe to assume at this point that everyone is in general
> > agreement, since there are no active objections.
> >
> > I have filed a ticket for the 2.3 release. Let's try to make it happen:
> > https://issues.apache.org/jira/browse/IGNITE-6305
> >
> > D.
> >
> > On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan <
> [hidden email]>
> > wrote:
> >
> > >
> > >
> > > On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani <[hidden email]>
> > > wrote:
> > >
> > >> Yeah, I guess that's doable as well and requires less management
> effort
> > >> than my suggestion. We could use events [1] to store payload data
> (e.g.
> > >> IP,
> > >> version, etc.)
> > >
> > >
> > > Yes, we could use events or some other similar API provided by GA.
> > >
> > >
> > >> What the download page CGI developed in? PHP?
> > >>
> > >
> > > To be honest, no clue. I guess someone in the community can figure it
> > out:
> > > https://svn.apache.org/repos/asf/ignite/site/trunk/download.html
> > >
> > >
> > >> However, I'm not sure whether storing this data in a 3rd party
> (Google)
> > is
> > >> compliant with the ASF policy. I guess it's no biggie, but if there's
> > >> doubt
> > >> in the PMC, it's better to ask legal@.
> > >
> > >
> > > I am not sure there is anything to ask about. The whole Ignite website
> is
> > > GA enabled, and all we are doing is accessing a standard web page from
> > the
> > > Ignite web site. The information gathered from GA is available only to
> > the
> > > Ignite PMC. Frankly, I think legal@ will be very confused by this
> > > question.
> > >
> > > Even ASF website itself uses GA: https://www.apache.org/
> > > foundation/policies/privacy.html
> > >
> > >
> > >> I think Cos said it's OK; maybe Roman can pitch in.
> > >>
> > >
> > >  Sure, would be nice to hear from Roman as well.
> > >
> > >
> > >> Cheers.
> > >>
> > >> [1]
> > >> https://developers.google.com/analytics/devguides/collection
> > >> /analyticsjs/events
> > >>
> > >> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan <
> > >> [hidden email]>
> > >> wrote:
> > >>
> > >> > Raul,
> > >> >
> > >> > Could point about Javascript, it will not work because it executes
> in
> > >> the
> > >> > browser. This means we need a server-side script, like CGI we are
> > using
> > >> on
> > >> > our download page.
> > >> >
> > >> > How about this approach. We create something like ignite-version.cgi
> > >> script
> > >> > which will invoke a call to GA and then return the latest version.
> > >> >
> > >> > This should work, right?
> > >> >
> > >> > D.
> > >> >
> > >> > On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani <
> [hidden email]
> > >
> > >> > wrote:
> > >> >
> > >> > > Hey Dmitriy and all
> > >> > >
> > >> > > Also, since we have GA enabled for the website, we can track how
> > many
> > >> > times
> > >> > > > this page was accessed, which will be equal to the number of
> > starts.
> > >> > This
> > >> > > > way, the counter information is tracked and monitored by the
> > Ignite
> > >> > PMC.
> > >> > >
> > >> > >
> > >> > > Unfortunately this won't work because GA is loaded via Javascript
> on
> > >> the
> > >> > > browser, so Google will never receive the page hit.
> > >> > >
> > >> > > Given the constraints, the most viable solution is an HTTPS
> endpoint
> > >> > > running on ASF infra that Ignite invokes via a GET or POST
> request.
> > >> The
> > >> > > simplest thing is to write each request in a log file, along with
> > the
> > >> > > timestamp, the version reported by the client, maybe the IP (not
> > sure
> > >> > about
> > >> > > the ASF rules about this concerning privacy, I guess it's OK if
> you
> > >> make
> > >> > it
> > >> > > an opt-in) and a unique node identifier, i.e. a UUID the node
> > creates
> > >> on
> > >> > > first startup or something.
> > >> > >
> > >> > > This endpoint would need some basic DDoS protection and
> blacklisting
> > >> to
> > >> > > prevent data spoofing.
> > >> > >
> > >> > > If we'll be implementing this endpoint anyway, then there's no
> point
> > >> > > placing another file on Git or elsewhere for reporting the latest
> > >> > versions:
> > >> > > the endpoint itself can return them.
> > >> > >
> > >> > > WDYT?
> > >> > >
> > >> > > Cheers.
> > >> > >
> > >> > > On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan <
> > >> > [hidden email]>
> > >> > > wrote:
> > >> > >
> > >> > > > Cos, Raul,
> > >> > > >
> > >> > > > Thanks for the feedback. I completely agree about Maven Central
> > >> being a
> > >> > > 3rd
> > >> > > > party repo (did not think about that initially). All your
> > >> suggestions
> > >> > > make
> > >> > > > sense, but I would like to keep it as simple as possible, and so
> > far
> > >> > > > everything suggested required GIT dependencies and extra work.
> > >> > > >
> > >> > > > How about Yakov's suggestion. We simply add a page to the Ignite
> > >> > website
> > >> > > > which will have only the latest version. Every time a node
> starts,
> > >> it
> > >> > > > receives the latest version from the page and suggests that
> users
> > >> > upgrade
> > >> > > > if needed.
> > >> > > >
> > >> > > > Also, since we have GA enabled for the website, we can track how
> > >> many
> > >> > > times
> > >> > > > this page was accessed, which will be equal to the number of
> > starts.
> > >> > This
> > >> > > > way, the counter information is tracked and monitored by the
> > Ignite
> > >> > PMC.
> > >> > > >
> > >> > > > This approach looks pretty innocent to me and everything is kept
> > and
> > >> > > > managed within Apache.
> > >> > > >
> > >> > > > Thoughts?
> > >> > > >
> > >> > > > D.
> > >> > > >
> > >> > > >
> > >> > > > On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik <
> > >> [hidden email]>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > I agree with Raul.
> > >> > > > > - providing a ping-back address to a 3rd party might be frown
> > >> upon by
> > >> > > > some.
> > >> > > > >   And might have a consequences like stats collection about
> > users'
> > >> > > > >   infrastructure.
> > >> > > > > - checking an ASF git-repo is easy and won't download any
> binary
> > >> > data:
> > >> > > > >   everything is clear text and could be easily monitored by
> any
> > of
> > >> > the
> > >> > > > > network
> > >> > > > >   diagnostic tools, shall it be required. But it involves a
> bit
> > of
> > >> > the
> > >> > > > > release
> > >> > > > >   discipline.
> > >> > > > > - the binary data download in the runtime is my main concern.
> > >> This is
> > >> > > the
> > >> > > > >   vector of MMA. What if someone gains the control over the
> > >> > repository
> > >> > > > and
> > >> > > > >   replaces the file with some malicious content.
> > >> > > > >
> > >> > > > > As for the particular mechanism: IIRC Ignite used to make a
> call
> > >> to
> > >> > an
> > >> > > > > external script to check upon the atest software version
> > available
> > >> > for
> > >> > > > > download. In the past, the endpoint was running on a 3rd party
> > >> > server,
> > >> > > I
> > >> > > > > believe the best way would be to put this script on ASF infra
> > and
> > >> > have
> > >> > > > the
> > >> > > > > "update checker" running in a completely controlled
> environment.
> > >> > > > Actually,
> > >> > > > > I
> > >> > > > > recall we had this very discussion during the Incubation; I
> can
> > >> > > probably
> > >> > > > > dig
> > >> > > > > out the corresponding thread.
> > >> > > > >
> > >> > > > > Thoughts?
> > >> > > > >   Cok
> > >> > > > >
> > >> > > > > On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote:
> > >> > > > > > Hey guys
> > >> > > > > >
> > >> > > > > > In my opinion, maven.org is still owned by a third party
> > >> > (Sonatype).
> > >> > > > We
> > >> > > > > > don't know what kind of data analysis or intelligence
> > extraction
> > >> > they
> > >> > > > > run.
> > >> > > > > >
> > >> > > > > > If Ignite servers all over the world were hitting maven.org
> > >> > > > periodically
> > >> > > > > > asking for an Ignite Maven artifact, it gives Sonatype a
> clear
> > >> > > > indication
> > >> > > > > > of who is running an Ignite server.
> > >> > > > > >
> > >> > > > > > They could reverse lookup the IP address and find out what
> > >> > > corporation
> > >> > > > it
> > >> > > > > > is.
> > >> > > > > >
> > >> > > > > > How about having Ignite check the ASF Git directly?
> > >> > > > > >
> > >> > > > > > We could use the Git ssh API, but that would require a new
> > >> > > dependency,
> > >> > > > > > which I advise against.
> > >> > > > > >
> > >> > > > > > Alternatively, we could have it scrape this HTML for new Git
> > >> tags:
> > >> > > > > > https://git-wip-us.apache.org/repos/asf?p=ignite.git
> > >> > > > > >
> > >> > > > > > Another option is to place a txt file in the root of the
> > master
> > >> > > branch
> > >> > > > > (e.g
> > >> > > > > > LATEST), containing a list of the latest GA versions for
> each
> > >> major
> > >> > > > > version
> > >> > > > > > line (1.x, 2.x).
> > >> > > > > >
> > >> > > > > > I would advocate this last option, but it requires somebody
> > >> > > remembering
> > >> > > > > to
> > >> > > > > > update the file with every release, unless we automate it
> > with a
> > >> > > Maven
> > >> > > > > > plugin.
> > >> > > > > >
> > >> > > > > > Hope that helps!
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > On 24 Aug 2017 19:37, "Denis Magda" <[hidden email]>
> > wrote:
> > >> > > > > >
> > >> > > > > > I see nothing wrong with this approach.
> > >> > > > > >
> > >> > > > > > Cos, Roman, Raul, as Apache veterans, what do you think? Is
> it
> > >> good
> > >> > > to
> > >> > > > > go?
> > >> > > > > >
> > >> > > > > > —
> > >> > > > > > Denis
> > >> > > > > >
> > >> > > > > > > On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan <
> > >> > > > [hidden email]
> > >> > > > > >
> > >> > > > > > wrote:
> > >> > > > > > >
> > >> > > > > > > Is everyone OK with this approach? Should I file a ticket
> on
> > >> it?
> > >> > > > > > >
> > >> > > > > > > On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <
> > >> > > > > [hidden email]>
> > >> > > > > > > wrote:
> > >> > > > > > >
> > >> > > > > > >> Igniters,
> > >> > > > > > >>
> > >> > > > > > >> There has been lots of talk of proposals about various
> > usage
> > >> > > metrics
> > >> > > > > for
> > >> > > > > > >> Ignite and nothing came of it. I would like to resurrect
> > the
> > >> > topic
> > >> > > > and
> > >> > > > > > >> propose something very simple and non-intrusive.
> > >> > > > > > >>
> > >> > > > > > >> 1. Update Checker
> > >> > > > > > >> The main purpose of the update checker is not to collect
> > >> > metrics,
> > >> > > > but
> > >> > > > > to
> > >> > > > > > >> notify users about a new version of Ignite by accessing
> > >> > maven.org
> > >> > > > and
> > >> > > > > > >> getting the version out of the metadata file:
> > >> > > > > > >> http://repo2.maven.org/maven2/
> > org/apache/ignite/ignite-core/
> > >> > > > > > >> maven-metadata.xml
> > >> > > > > > >>
> > >> > > > > > >> This way we do not send any information anywhere and, at
> > the
> > >> > same
> > >> > > > > time,
> > >> > > > > > >> urge our users to download and start using the latest
> > >> version of
> > >> > > > > Ignite.
> > >> > > > > > >>
> > >> > > > > > >> 2. Startup Counter
> > >> > > > > > >> This piece is optional, but we can also get an insight in
> > how
> > >> > many
> > >> > > > > times
> > >> > > > > > a
> > >> > > > > > >> certain Ignite release gets started. This is just a cool
> > >> metric
> > >> > > for
> > >> > > > > the
> > >> > > > > > >> community to gauge the project popularity. You can think
> of
> > >> it
> > >> > as
> > >> > > > of a
> > >> > > > > > page
> > >> > > > > > >> visit counter shown on many websites. We can even decide
> to
> > >> > > display
> > >> > > > > this
> > >> > > > > > >> counter on the Ignite website as well.
> > >> > > > > > >>
> > >> > > > > > >> To do this, we can simply add a JAR in maven for every
> > >> release,
> > >> > > e.g.
> > >> > > > > > >> ignite-start-counter.jar, which will contain only 1 byte.
> > >> Every
> > >> > > time
> > >> > > > > an
> > >> > > > > > >> Ignite node starts, it will download this JAR in the
> > >> background.
> > >> > > > Then
> > >> > > > > we
> > >> > > > > > >> will be able to view the number of the total downloads
> for
> > >> this
> > >> > > JAR
> > >> > > > in
> > >> > > > > > >> Maven Central, which is essentially the number of starts
> of
> > >> > Ignite
> > >> > > > > nodes.
> > >> > > > > > >>
> > >> > > > > > >> *Note that neither of the above suggestions require
> Ignite
> > to
> > >> > send
> > >> > > > or
> > >> > > > > > >> track any user information whatsoever.*
> > >> > > > > > >>
> > >> > > > > > >> Please reply suggesting weather you are OK with this
> > >> approach.
> > >> > > > > > >>
> > >> > > > > > >> D.
> > >> > > > > > >>
> > >> > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>

[dmagda]

Cos,

The screenshot was not attached. Could you share it some other way (google drive, etc.)? I’ve never seen any commercial on the site.

—
Denis

> On Sep 28, 2017, at 7:23 AM, Konstantin Boudnik <[hidden email]> wrote:
>
> I don't see an issue with node version either.
>
> Just one more, and it might be slightly irrelevant, issue though... I looked at the Ignite's site and found the following ads and trackers (that are indeed getting disabled by my browser).
> Why are googleads or doubleclick are permitted?
>
>
> ​
> Thanks,
>   Cos
> ​
>
> --
>   With regards,
> Konstantin (Cos) Boudnik
> 2CAC 8312 4870 D885 8616  6115 220F 6980 1F27 E622
>
> Disclaimer: Opinions expressed in this email are those of the author, and do not necessarily represent the views of any company the author might be affiliated with at the moment of writing.
>
> On Tue, Sep 26, 2017 at 3:21 PM, Dmitriy Setrakyan <[hidden email] <mailto:[hidden email]>> wrote:
> On Tue, Sep 26, 2017 at 6:20 AM, Vladimir Ozerov <[hidden email] <mailto:[hidden email]>>
> wrote:
>
> > Folks,
> >
> > Can we add version of current node to web request? This way we will better
> > understand version distribution, what might help us with certain API
> > update/deprecate decisions
> > E.g. http://ignite.apache.org/latest.cgi&ver=2.2.0 <http://ignite.apache.org/latest.cgi&ver=2.2.0>
>
>
> Vladimir, I personally do not see a problem with that, as sending the
> current version to the update checker seems very innocent to me. At the
> same time, it will allow us to examine the usage of each release and make
> decisions about dropping backward compatibility or spotting bugs for a
> certain release.
>
> Cos, Raul, any thoughts?
>
>
> >
> >
> > Vladimir.
> >
> > On Fri, Sep 8, 2017 at 7:06 AM, Dmitriy Setrakyan <[hidden email] <mailto:[hidden email]>>
> > wrote:
> >
> > > I think it is safe to assume at this point that everyone is in general
> > > agreement, since there are no active objections.
> > >
> > > I have filed a ticket for the 2.3 release. Let's try to make it happen:
> > > https://issues.apache.org/jira/browse/IGNITE-6305 <https://issues.apache.org/jira/browse/IGNITE-6305>
> > >
> > > D.
> > >
> > > On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan <
> > [hidden email] <mailto:[hidden email]>>
> > > wrote:
> > >
> > > >
> > > >
> > > > On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani <[hidden email] <mailto:[hidden email]>>
> > > > wrote:
> > > >
> > > >> Yeah, I guess that's doable as well and requires less management
> > effort
> > > >> than my suggestion. We could use events [1] to store payload data
> > (e.g.
> > > >> IP,
> > > >> version, etc.)
> > > >
> > > >
> > > > Yes, we could use events or some other similar API provided by GA.
> > > >
> > > >
> > > >> What the download page CGI developed in? PHP?
> > > >>
> > > >
> > > > To be honest, no clue. I guess someone in the community can figure it
> > > out:
> > > > https://svn.apache.org/repos/asf/ignite/site/trunk/download.html <https://svn.apache.org/repos/asf/ignite/site/trunk/download.html>
> > > >
> > > >
> > > >> However, I'm not sure whether storing this data in a 3rd party
> > (Google)
> > > is
> > > >> compliant with the ASF policy. I guess it's no biggie, but if there's
> > > >> doubt
> > > >> in the PMC, it's better to ask legal@.
> > > >
> > > >
> > > > I am not sure there is anything to ask about. The whole Ignite website
> > is
> > > > GA enabled, and all we are doing is accessing a standard web page from
> > > the
> > > > Ignite web site. The information gathered from GA is available only to
> > > the
> > > > Ignite PMC. Frankly, I think legal@ will be very confused by this
> > > > question.
> > > >
> > > > Even ASF website itself uses GA: https://www.apache.org/ <https://www.apache.org/>
> > > > foundation/policies/privacy.html
> > > >
> > > >
> > > >> I think Cos said it's OK; maybe Roman can pitch in.
> > > >>
> > > >
> > > >  Sure, would be nice to hear from Roman as well.
> > > >
> > > >
> > > >> Cheers.
> > > >>
> > > >> [1]
> > > >> https://developers.google.com/analytics/devguides/collection <https://developers.google.com/analytics/devguides/collection>
> > > >> /analyticsjs/events
> > > >>
> > > >> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan <
> > > >> [hidden email] <mailto:[hidden email]>>
> > > >> wrote:
> > > >>
> > > >> > Raul,
> > > >> >
> > > >> > Could point about Javascript, it will not work because it executes
> > in
> > > >> the
> > > >> > browser. This means we need a server-side script, like CGI we are
> > > using
> > > >> on
> > > >> > our download page.
> > > >> >
> > > >> > How about this approach. We create something like ignite-version.cgi
> > > >> script
> > > >> > which will invoke a call to GA and then return the latest version.
> > > >> >
> > > >> > This should work, right?
> > > >> >
> > > >> > D.
> > > >> >
> > > >> > On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani <
> > [hidden email] <mailto:[hidden email]>
> > > >
> > > >> > wrote:
> > > >> >
> > > >> > > Hey Dmitriy and all
> > > >> > >
> > > >> > > Also, since we have GA enabled for the website, we can track how
> > > many
> > > >> > times
> > > >> > > > this page was accessed, which will be equal to the number of
> > > starts.
> > > >> > This
> > > >> > > > way, the counter information is tracked and monitored by the
> > > Ignite
> > > >> > PMC.
> > > >> > >
> > > >> > >
> > > >> > > Unfortunately this won't work because GA is loaded via Javascript
> > on
> > > >> the
> > > >> > > browser, so Google will never receive the page hit.
> > > >> > >
> > > >> > > Given the constraints, the most viable solution is an HTTPS
> > endpoint
> > > >> > > running on ASF infra that Ignite invokes via a GET or POST
> > request.
> > > >> The
> > > >> > > simplest thing is to write each request in a log file, along with
> > > the
> > > >> > > timestamp, the version reported by the client, maybe the IP (not
> > > sure
> > > >> > about
> > > >> > > the ASF rules about this concerning privacy, I guess it's OK if
> > you
> > > >> make
> > > >> > it
> > > >> > > an opt-in) and a unique node identifier, i.e. a UUID the node
> > > creates
> > > >> on
> > > >> > > first startup or something.
> > > >> > >
> > > >> > > This endpoint would need some basic DDoS protection and
> > blacklisting
> > > >> to
> > > >> > > prevent data spoofing.
> > > >> > >
> > > >> > > If we'll be implementing this endpoint anyway, then there's no
> > point
> > > >> > > placing another file on Git or elsewhere for reporting the latest
> > > >> > versions:
> > > >> > > the endpoint itself can return them.
> > > >> > >
> > > >> > > WDYT?
> > > >> > >
> > > >> > > Cheers.
> > > >> > >
> > > >> > > On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan <
> > > >> > [hidden email] <mailto:[hidden email]>>
> > > >> > > wrote:
> > > >> > >
> > > >> > > > Cos, Raul,
> > > >> > > >
> > > >> > > > Thanks for the feedback. I completely agree about Maven Central
> > > >> being a
> > > >> > > 3rd
> > > >> > > > party repo (did not think about that initially). All your
> > > >> suggestions
> > > >> > > make
> > > >> > > > sense, but I would like to keep it as simple as possible, and so
> > > far
> > > >> > > > everything suggested required GIT dependencies and extra work.
> > > >> > > >
> > > >> > > > How about Yakov's suggestion. We simply add a page to the Ignite
> > > >> > website
> > > >> > > > which will have only the latest version. Every time a node
> > starts,
> > > >> it
> > > >> > > > receives the latest version from the page and suggests that
> > users
> > > >> > upgrade
> > > >> > > > if needed.
> > > >> > > >
> > > >> > > > Also, since we have GA enabled for the website, we can track how
> > > >> many
> > > >> > > times
> > > >> > > > this page was accessed, which will be equal to the number of
> > > starts.
> > > >> > This
> > > >> > > > way, the counter information is tracked and monitored by the
> > > Ignite
> > > >> > PMC.
> > > >> > > >
> > > >> > > > This approach looks pretty innocent to me and everything is kept
> > > and
> > > >> > > > managed within Apache.
> > > >> > > >
> > > >> > > > Thoughts?
> > > >> > > >
> > > >> > > > D.
> > > >> > > >
> > > >> > > >
> > > >> > > > On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik <
> > > >> [hidden email] <mailto:[hidden email]>>
> > > >> > > > wrote:
> > > >> > > >
> > > >> > > > > I agree with Raul.
> > > >> > > > > - providing a ping-back address to a 3rd party might be frown
> > > >> upon by
> > > >> > > > some.
> > > >> > > > >   And might have a consequences like stats collection about
> > > users'
> > > >> > > > >   infrastructure.
> > > >> > > > > - checking an ASF git-repo is easy and won't download any
> > binary
> > > >> > data:
> > > >> > > > >   everything is clear text and could be easily monitored by
> > any
> > > of
> > > >> > the
> > > >> > > > > network
> > > >> > > > >   diagnostic tools, shall it be required. But it involves a
> > bit
> > > of
> > > >> > the
> > > >> > > > > release
> > > >> > > > >   discipline.
> > > >> > > > > - the binary data download in the runtime is my main concern.
> > > >> This is
> > > >> > > the
> > > >> > > > >   vector of MMA. What if someone gains the control over the
> > > >> > repository
> > > >> > > > and
> > > >> > > > >   replaces the file with some malicious content.
> > > >> > > > >
> > > >> > > > > As for the particular mechanism: IIRC Ignite used to make a
> > call
> > > >> to
> > > >> > an
> > > >> > > > > external script to check upon the atest software version
> > > available
> > > >> > for
> > > >> > > > > download. In the past, the endpoint was running on a 3rd party
> > > >> > server,
> > > >> > > I
> > > >> > > > > believe the best way would be to put this script on ASF infra
> > > and
> > > >> > have
> > > >> > > > the
> > > >> > > > > "update checker" running in a completely controlled
> > environment.
> > > >> > > > Actually,
> > > >> > > > > I
> > > >> > > > > recall we had this very discussion during the Incubation; I
> > can
> > > >> > > probably
> > > >> > > > > dig
> > > >> > > > > out the corresponding thread.
> > > >> > > > >
> > > >> > > > > Thoughts?
> > > >> > > > >   Cok
> > > >> > > > >
> > > >> > > > > On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote:
> > > >> > > > > > Hey guys
> > > >> > > > > >
> > > >> > > > > > In my opinion, maven.org <http://maven.org/> is still owned by a third party
> > > >> > (Sonatype).
> > > >> > > > We
> > > >> > > > > > don't know what kind of data analysis or intelligence
> > > extraction
> > > >> > they
> > > >> > > > > run.
> > > >> > > > > >
> > > >> > > > > > If Ignite servers all over the world were hitting maven.org <http://maven.org/>
> > > >> > > > periodically
> > > >> > > > > > asking for an Ignite Maven artifact, it gives Sonatype a
> > clear
> > > >> > > > indication
> > > >> > > > > > of who is running an Ignite server.
> > > >> > > > > >
> > > >> > > > > > They could reverse lookup the IP address and find out what
> > > >> > > corporation
> > > >> > > > it
> > > >> > > > > > is.
> > > >> > > > > >
> > > >> > > > > > How about having Ignite check the ASF Git directly?
> > > >> > > > > >
> > > >> > > > > > We could use the Git ssh API, but that would require a new
> > > >> > > dependency,
> > > >> > > > > > which I advise against.
> > > >> > > > > >
> > > >> > > > > > Alternatively, we could have it scrape this HTML for new Git
> > > >> tags:
> > > >> > > > > > https://git-wip-us.apache.org/repos/asf?p=ignite.git <https://git-wip-us.apache.org/repos/asf?p=ignite.git>
> > > >> > > > > >
> > > >> > > > > > Another option is to place a txt file in the root of the
> > > master
> > > >> > > branch
> > > >> > > > > (e.g
> > > >> > > > > > LATEST), containing a list of the latest GA versions for
> > each
> > > >> major
> > > >> > > > > version
> > > >> > > > > > line (1.x, 2.x).
> > > >> > > > > >
> > > >> > > > > > I would advocate this last option, but it requires somebody
> > > >> > > remembering
> > > >> > > > > to
> > > >> > > > > > update the file with every release, unless we automate it
> > > with a
> > > >> > > Maven
> > > >> > > > > > plugin.
> > > >> > > > > >
> > > >> > > > > > Hope that helps!
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > > On 24 Aug 2017 19:37, "Denis Magda" <[hidden email] <mailto:[hidden email]>>
> > > wrote:
> > > >> > > > > >
> > > >> > > > > > I see nothing wrong with this approach.
> > > >> > > > > >
> > > >> > > > > > Cos, Roman, Raul, as Apache veterans, what do you think? Is
> > it
> > > >> good
> > > >> > > to
> > > >> > > > > go?
> > > >> > > > > >
> > > >> > > > > > —
> > > >> > > > > > Denis
> > > >> > > > > >
> > > >> > > > > > > On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan <
> > > >> > > > [hidden email] <mailto:[hidden email]>
> > > >> > > > > >
> > > >> > > > > > wrote:
> > > >> > > > > > >
> > > >> > > > > > > Is everyone OK with this approach? Should I file a ticket
> > on
> > > >> it?
> > > >> > > > > > >
> > > >> > > > > > > On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <
> > > >> > > > > [hidden email] <mailto:[hidden email]>>
> > > >> > > > > > > wrote:
> > > >> > > > > > >
> > > >> > > > > > >> Igniters,
> > > >> > > > > > >>
> > > >> > > > > > >> There has been lots of talk of proposals about various
> > > usage
> > > >> > > metrics
> > > >> > > > > for
> > > >> > > > > > >> Ignite and nothing came of it. I would like to resurrect
> > > the
> > > >> > topic
> > > >> > > > and
> > > >> > > > > > >> propose something very simple and non-intrusive.
> > > >> > > > > > >>
> > > >> > > > > > >> 1. Update Checker
> > > >> > > > > > >> The main purpose of the update checker is not to collect
> > > >> > metrics,
> > > >> > > > but
> > > >> > > > > to
> > > >> > > > > > >> notify users about a new version of Ignite by accessing
> > > >> > maven.org <http://maven.org/>
> > > >> > > > and
> > > >> > > > > > >> getting the version out of the metadata file:
> > > >> > > > > > >> http://repo2.maven.org/maven2/ <http://repo2.maven.org/maven2/>
> > > org/apache/ignite/ignite-core/
> > > >> > > > > > >> maven-metadata.xml
> > > >> > > > > > >>
> > > >> > > > > > >> This way we do not send any information anywhere and, at
> > > the
> > > >> > same
> > > >> > > > > time,
> > > >> > > > > > >> urge our users to download and start using the latest
> > > >> version of
> > > >> > > > > Ignite.
> > > >> > > > > > >>
> > > >> > > > > > >> 2. Startup Counter
> > > >> > > > > > >> This piece is optional, but we can also get an insight in
> > > how
> > > >> > many
> > > >> > > > > times
> > > >> > > > > > a
> > > >> > > > > > >> certain Ignite release gets started. This is just a cool
> > > >> metric
> > > >> > > for
> > > >> > > > > the
> > > >> > > > > > >> community to gauge the project popularity. You can think
> > of
> > > >> it
> > > >> > as
> > > >> > > > of a
> > > >> > > > > > page
> > > >> > > > > > >> visit counter shown on many websites. We can even decide
> > to
> > > >> > > display
> > > >> > > > > this
> > > >> > > > > > >> counter on the Ignite website as well.
> > > >> > > > > > >>
> > > >> > > > > > >> To do this, we can simply add a JAR in maven for every
> > > >> release,
> > > >> > > e.g.
> > > >> > > > > > >> ignite-start-counter.jar, which will contain only 1 byte.
> > > >> Every
> > > >> > > time
> > > >> > > > > an
> > > >> > > > > > >> Ignite node starts, it will download this JAR in the
> > > >> background.
> > > >> > > > Then
> > > >> > > > > we
> > > >> > > > > > >> will be able to view the number of the total downloads
> > for
> > > >> this
> > > >> > > JAR
> > > >> > > > in
> > > >> > > > > > >> Maven Central, which is essentially the number of starts
> > of
> > > >> > Ignite
> > > >> > > > > nodes.
> > > >> > > > > > >>
> > > >> > > > > > >> *Note that neither of the above suggestions require
> > Ignite
> > > to
> > > >> > send
> > > >> > > > or
> > > >> > > > > > >> track any user information whatsoever.*
> > > >> > > > > > >>
> > > >> > > > > > >> Please reply suggesting weather you are OK with this
> > > >> approach.
> > > >> > > > > > >>
> > > >> > > > > > >> D.
> > > >> > > > > > >>
> > > >> > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>

[Konstantin Boudnik-2]

Sorry guys - I neglected the fact that our lists don't permit
attachments. I have put the screenshot to an external server [1]

[1] https://imgur.com/a/p9FJ9

Thank you!
--
  With regards,
Konstantin (Cos) Boudnik
2CAC 8312 4870 D885 8616  6115 220F 6980 1F27 E622

Disclaimer: Opinions expressed in this email are those of the author,
and do not necessarily represent the views of any company the author
might be affiliated with at the moment of writing.


On Thu, Sep 28, 2017 at 1:37 PM, Denis Magda <[hidden email]> wrote:

> Cos,
>
> The screenshot was not attached. Could you share it some other way (google drive, etc.)? I’ve never seen any commercial on the site.
>
> —
> Denis
>
>> On Sep 28, 2017, at 7:23 AM, Konstantin Boudnik <[hidden email]> wrote:
>>
>> I don't see an issue with node version either.
>>
>> Just one more, and it might be slightly irrelevant, issue though... I looked at the Ignite's site and found the following ads and trackers (that are indeed getting disabled by my browser).
>> Why are googleads or doubleclick are permitted?
>>
>>
>>
>> Thanks,
>>   Cos
>>
>>
>> --
>>   With regards,
>> Konstantin (Cos) Boudnik
>> 2CAC 8312 4870 D885 8616  6115 220F 6980 1F27 E622
>>
>> Disclaimer: Opinions expressed in this email are those of the author, and do not necessarily represent the views of any company the author might be affiliated with at the moment of writing.
>>
>> On Tue, Sep 26, 2017 at 3:21 PM, Dmitriy Setrakyan <[hidden email] <mailto:[hidden email]>> wrote:
>> On Tue, Sep 26, 2017 at 6:20 AM, Vladimir Ozerov <[hidden email] <mailto:[hidden email]>>
>> wrote:
>>
>> > Folks,
>> >
>> > Can we add version of current node to web request? This way we will better
>> > understand version distribution, what might help us with certain API
>> > update/deprecate decisions
>> > E.g. http://ignite.apache.org/latest.cgi&ver=2.2.0 <http://ignite.apache.org/latest.cgi&ver=2.2.0>
>>
>>
>> Vladimir, I personally do not see a problem with that, as sending the
>> current version to the update checker seems very innocent to me. At the
>> same time, it will allow us to examine the usage of each release and make
>> decisions about dropping backward compatibility or spotting bugs for a
>> certain release.
>>
>> Cos, Raul, any thoughts?
>>
>>
>> >
>> >
>> > Vladimir.
>> >
>> > On Fri, Sep 8, 2017 at 7:06 AM, Dmitriy Setrakyan <[hidden email] <mailto:[hidden email]>>
>> > wrote:
>> >
>> > > I think it is safe to assume at this point that everyone is in general
>> > > agreement, since there are no active objections.
>> > >
>> > > I have filed a ticket for the 2.3 release. Let's try to make it happen:
>> > > https://issues.apache.org/jira/browse/IGNITE-6305 <https://issues.apache.org/jira/browse/IGNITE-6305>
>> > >
>> > > D.
>> > >
>> > > On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan <
>> > [hidden email] <mailto:[hidden email]>>
>> > > wrote:
>> > >
>> > > >
>> > > >
>> > > > On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani <[hidden email] <mailto:[hidden email]>>
>> > > > wrote:
>> > > >
>> > > >> Yeah, I guess that's doable as well and requires less management
>> > effort
>> > > >> than my suggestion. We could use events [1] to store payload data
>> > (e.g.
>> > > >> IP,
>> > > >> version, etc.)
>> > > >
>> > > >
>> > > > Yes, we could use events or some other similar API provided by GA.
>> > > >
>> > > >
>> > > >> What the download page CGI developed in? PHP?
>> > > >>
>> > > >
>> > > > To be honest, no clue. I guess someone in the community can figure it
>> > > out:
>> > > > https://svn.apache.org/repos/asf/ignite/site/trunk/download.html <https://svn.apache.org/repos/asf/ignite/site/trunk/download.html>
>> > > >
>> > > >
>> > > >> However, I'm not sure whether storing this data in a 3rd party
>> > (Google)
>> > > is
>> > > >> compliant with the ASF policy. I guess it's no biggie, but if there's
>> > > >> doubt
>> > > >> in the PMC, it's better to ask legal@.
>> > > >
>> > > >
>> > > > I am not sure there is anything to ask about. The whole Ignite website
>> > is
>> > > > GA enabled, and all we are doing is accessing a standard web page from
>> > > the
>> > > > Ignite web site. The information gathered from GA is available only to
>> > > the
>> > > > Ignite PMC. Frankly, I think legal@ will be very confused by this
>> > > > question.
>> > > >
>> > > > Even ASF website itself uses GA: https://www.apache.org/ <https://www.apache.org/>
>> > > > foundation/policies/privacy.html
>> > > >
>> > > >
>> > > >> I think Cos said it's OK; maybe Roman can pitch in.
>> > > >>
>> > > >
>> > > >  Sure, would be nice to hear from Roman as well.
>> > > >
>> > > >
>> > > >> Cheers.
>> > > >>
>> > > >> [1]
>> > > >> https://developers.google.com/analytics/devguides/collection <https://developers.google.com/analytics/devguides/collection>
>> > > >> /analyticsjs/events
>> > > >>
>> > > >> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan <
>> > > >> [hidden email] <mailto:[hidden email]>>
>> > > >> wrote:
>> > > >>
>> > > >> > Raul,
>> > > >> >
>> > > >> > Could point about Javascript, it will not work because it executes
>> > in
>> > > >> the
>> > > >> > browser. This means we need a server-side script, like CGI we are
>> > > using
>> > > >> on
>> > > >> > our download page.
>> > > >> >
>> > > >> > How about this approach. We create something like ignite-version.cgi
>> > > >> script
>> > > >> > which will invoke a call to GA and then return the latest version.
>> > > >> >
>> > > >> > This should work, right?
>> > > >> >
>> > > >> > D.
>> > > >> >
>> > > >> > On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani <
>> > [hidden email] <mailto:[hidden email]>
>> > > >
>> > > >> > wrote:
>> > > >> >
>> > > >> > > Hey Dmitriy and all
>> > > >> > >
>> > > >> > > Also, since we have GA enabled for the website, we can track how
>> > > many
>> > > >> > times
>> > > >> > > > this page was accessed, which will be equal to the number of
>> > > starts.
>> > > >> > This
>> > > >> > > > way, the counter information is tracked and monitored by the
>> > > Ignite
>> > > >> > PMC.
>> > > >> > >
>> > > >> > >
>> > > >> > > Unfortunately this won't work because GA is loaded via Javascript
>> > on
>> > > >> the
>> > > >> > > browser, so Google will never receive the page hit.
>> > > >> > >
>> > > >> > > Given the constraints, the most viable solution is an HTTPS
>> > endpoint
>> > > >> > > running on ASF infra that Ignite invokes via a GET or POST
>> > request.
>> > > >> The
>> > > >> > > simplest thing is to write each request in a log file, along with
>> > > the
>> > > >> > > timestamp, the version reported by the client, maybe the IP (not
>> > > sure
>> > > >> > about
>> > > >> > > the ASF rules about this concerning privacy, I guess it's OK if
>> > you
>> > > >> make
>> > > >> > it
>> > > >> > > an opt-in) and a unique node identifier, i.e. a UUID the node
>> > > creates
>> > > >> on
>> > > >> > > first startup or something.
>> > > >> > >
>> > > >> > > This endpoint would need some basic DDoS protection and
>> > blacklisting
>> > > >> to
>> > > >> > > prevent data spoofing.
>> > > >> > >
>> > > >> > > If we'll be implementing this endpoint anyway, then there's no
>> > point
>> > > >> > > placing another file on Git or elsewhere for reporting the latest
>> > > >> > versions:
>> > > >> > > the endpoint itself can return them.
>> > > >> > >
>> > > >> > > WDYT?
>> > > >> > >
>> > > >> > > Cheers.
>> > > >> > >
>> > > >> > > On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan <
>> > > >> > [hidden email] <mailto:[hidden email]>>
>> > > >> > > wrote:
>> > > >> > >
>> > > >> > > > Cos, Raul,
>> > > >> > > >
>> > > >> > > > Thanks for the feedback. I completely agree about Maven Central
>> > > >> being a
>> > > >> > > 3rd
>> > > >> > > > party repo (did not think about that initially). All your
>> > > >> suggestions
>> > > >> > > make
>> > > >> > > > sense, but I would like to keep it as simple as possible, and so
>> > > far
>> > > >> > > > everything suggested required GIT dependencies and extra work.
>> > > >> > > >
>> > > >> > > > How about Yakov's suggestion. We simply add a page to the Ignite
>> > > >> > website
>> > > >> > > > which will have only the latest version. Every time a node
>> > starts,
>> > > >> it
>> > > >> > > > receives the latest version from the page and suggests that
>> > users
>> > > >> > upgrade
>> > > >> > > > if needed.
>> > > >> > > >
>> > > >> > > > Also, since we have GA enabled for the website, we can track how
>> > > >> many
>> > > >> > > times
>> > > >> > > > this page was accessed, which will be equal to the number of
>> > > starts.
>> > > >> > This
>> > > >> > > > way, the counter information is tracked and monitored by the
>> > > Ignite
>> > > >> > PMC.
>> > > >> > > >
>> > > >> > > > This approach looks pretty innocent to me and everything is kept
>> > > and
>> > > >> > > > managed within Apache.
>> > > >> > > >
>> > > >> > > > Thoughts?
>> > > >> > > >
>> > > >> > > > D.
>> > > >> > > >
>> > > >> > > >
>> > > >> > > > On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik <
>> > > >> [hidden email] <mailto:[hidden email]>>
>> > > >> > > > wrote:
>> > > >> > > >
>> > > >> > > > > I agree with Raul.
>> > > >> > > > > - providing a ping-back address to a 3rd party might be frown
>> > > >> upon by
>> > > >> > > > some.
>> > > >> > > > >   And might have a consequences like stats collection about
>> > > users'
>> > > >> > > > >   infrastructure.
>> > > >> > > > > - checking an ASF git-repo is easy and won't download any
>> > binary
>> > > >> > data:
>> > > >> > > > >   everything is clear text and could be easily monitored by
>> > any
>> > > of
>> > > >> > the
>> > > >> > > > > network
>> > > >> > > > >   diagnostic tools, shall it be required. But it involves a
>> > bit
>> > > of
>> > > >> > the
>> > > >> > > > > release
>> > > >> > > > >   discipline.
>> > > >> > > > > - the binary data download in the runtime is my main concern.
>> > > >> This is
>> > > >> > > the
>> > > >> > > > >   vector of MMA. What if someone gains the control over the
>> > > >> > repository
>> > > >> > > > and
>> > > >> > > > >   replaces the file with some malicious content.
>> > > >> > > > >
>> > > >> > > > > As for the particular mechanism: IIRC Ignite used to make a
>> > call
>> > > >> to
>> > > >> > an
>> > > >> > > > > external script to check upon the atest software version
>> > > available
>> > > >> > for
>> > > >> > > > > download. In the past, the endpoint was running on a 3rd party
>> > > >> > server,
>> > > >> > > I
>> > > >> > > > > believe the best way would be to put this script on ASF infra
>> > > and
>> > > >> > have
>> > > >> > > > the
>> > > >> > > > > "update checker" running in a completely controlled
>> > environment.
>> > > >> > > > Actually,
>> > > >> > > > > I
>> > > >> > > > > recall we had this very discussion during the Incubation; I
>> > can
>> > > >> > > probably
>> > > >> > > > > dig
>> > > >> > > > > out the corresponding thread.
>> > > >> > > > >
>> > > >> > > > > Thoughts?
>> > > >> > > > >   Cok
>> > > >> > > > >
>> > > >> > > > > On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote:
>> > > >> > > > > > Hey guys
>> > > >> > > > > >
>> > > >> > > > > > In my opinion, maven.org <http://maven.org/> is still owned by a third party
>> > > >> > (Sonatype).
>> > > >> > > > We
>> > > >> > > > > > don't know what kind of data analysis or intelligence
>> > > extraction
>> > > >> > they
>> > > >> > > > > run.
>> > > >> > > > > >
>> > > >> > > > > > If Ignite servers all over the world were hitting maven.org <http://maven.org/>
>> > > >> > > > periodically
>> > > >> > > > > > asking for an Ignite Maven artifact, it gives Sonatype a
>> > clear
>> > > >> > > > indication
>> > > >> > > > > > of who is running an Ignite server.
>> > > >> > > > > >
>> > > >> > > > > > They could reverse lookup the IP address and find out what
>> > > >> > > corporation
>> > > >> > > > it
>> > > >> > > > > > is.
>> > > >> > > > > >
>> > > >> > > > > > How about having Ignite check the ASF Git directly?
>> > > >> > > > > >
>> > > >> > > > > > We could use the Git ssh API, but that would require a new
>> > > >> > > dependency,
>> > > >> > > > > > which I advise against.
>> > > >> > > > > >
>> > > >> > > > > > Alternatively, we could have it scrape this HTML for new Git
>> > > >> tags:
>> > > >> > > > > > https://git-wip-us.apache.org/repos/asf?p=ignite.git <https://git-wip-us.apache.org/repos/asf?p=ignite.git>
>> > > >> > > > > >
>> > > >> > > > > > Another option is to place a txt file in the root of the
>> > > master
>> > > >> > > branch
>> > > >> > > > > (e.g
>> > > >> > > > > > LATEST), containing a list of the latest GA versions for
>> > each
>> > > >> major
>> > > >> > > > > version
>> > > >> > > > > > line (1.x, 2.x).
>> > > >> > > > > >
>> > > >> > > > > > I would advocate this last option, but it requires somebody
>> > > >> > > remembering
>> > > >> > > > > to
>> > > >> > > > > > update the file with every release, unless we automate it
>> > > with a
>> > > >> > > Maven
>> > > >> > > > > > plugin.
>> > > >> > > > > >
>> > > >> > > > > > Hope that helps!
>> > > >> > > > > >
>> > > >> > > > > >
>> > > >> > > > > > On 24 Aug 2017 19:37, "Denis Magda" <[hidden email] <mailto:[hidden email]>>
>> > > wrote:
>> > > >> > > > > >
>> > > >> > > > > > I see nothing wrong with this approach.
>> > > >> > > > > >
>> > > >> > > > > > Cos, Roman, Raul, as Apache veterans, what do you think? Is
>> > it
>> > > >> good
>> > > >> > > to
>> > > >> > > > > go?
>> > > >> > > > > >
>> > > >> > > > > > —
>> > > >> > > > > > Denis
>> > > >> > > > > >
>> > > >> > > > > > > On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan <
>> > > >> > > > [hidden email] <mailto:[hidden email]>
>> > > >> > > > > >
>> > > >> > > > > > wrote:
>> > > >> > > > > > >
>> > > >> > > > > > > Is everyone OK with this approach? Should I file a ticket
>> > on
>> > > >> it?
>> > > >> > > > > > >
>> > > >> > > > > > > On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <
>> > > >> > > > > [hidden email] <mailto:[hidden email]>>
>> > > >> > > > > > > wrote:
>> > > >> > > > > > >
>> > > >> > > > > > >> Igniters,
>> > > >> > > > > > >>
>> > > >> > > > > > >> There has been lots of talk of proposals about various
>> > > usage
>> > > >> > > metrics
>> > > >> > > > > for
>> > > >> > > > > > >> Ignite and nothing came of it. I would like to resurrect
>> > > the
>> > > >> > topic
>> > > >> > > > and
>> > > >> > > > > > >> propose something very simple and non-intrusive.
>> > > >> > > > > > >>
>> > > >> > > > > > >> 1. Update Checker
>> > > >> > > > > > >> The main purpose of the update checker is not to collect
>> > > >> > metrics,
>> > > >> > > > but
>> > > >> > > > > to
>> > > >> > > > > > >> notify users about a new version of Ignite by accessing
>> > > >> > maven.org <http://maven.org/>
>> > > >> > > > and
>> > > >> > > > > > >> getting the version out of the metadata file:
>> > > >> > > > > > >> http://repo2.maven.org/maven2/ <http://repo2.maven.org/maven2/>
>> > > org/apache/ignite/ignite-core/
>> > > >> > > > > > >> maven-metadata.xml
>> > > >> > > > > > >>
>> > > >> > > > > > >> This way we do not send any information anywhere and, at
>> > > the
>> > > >> > same
>> > > >> > > > > time,
>> > > >> > > > > > >> urge our users to download and start using the latest
>> > > >> version of
>> > > >> > > > > Ignite.
>> > > >> > > > > > >>
>> > > >> > > > > > >> 2. Startup Counter
>> > > >> > > > > > >> This piece is optional, but we can also get an insight in
>> > > how
>> > > >> > many
>> > > >> > > > > times
>> > > >> > > > > > a
>> > > >> > > > > > >> certain Ignite release gets started. This is just a cool
>> > > >> metric
>> > > >> > > for
>> > > >> > > > > the
>> > > >> > > > > > >> community to gauge the project popularity. You can think
>> > of
>> > > >> it
>> > > >> > as
>> > > >> > > > of a
>> > > >> > > > > > page
>> > > >> > > > > > >> visit counter shown on many websites. We can even decide
>> > to
>> > > >> > > display
>> > > >> > > > > this
>> > > >> > > > > > >> counter on the Ignite website as well.
>> > > >> > > > > > >>
>> > > >> > > > > > >> To do this, we can simply add a JAR in maven for every
>> > > >> release,
>> > > >> > > e.g.
>> > > >> > > > > > >> ignite-start-counter.jar, which will contain only 1 byte.
>> > > >> Every
>> > > >> > > time
>> > > >> > > > > an
>> > > >> > > > > > >> Ignite node starts, it will download this JAR in the
>> > > >> background.
>> > > >> > > > Then
>> > > >> > > > > we
>> > > >> > > > > > >> will be able to view the number of the total downloads
>> > for
>> > > >> this
>> > > >> > > JAR
>> > > >> > > > in
>> > > >> > > > > > >> Maven Central, which is essentially the number of starts
>> > of
>> > > >> > Ignite
>> > > >> > > > > nodes.
>> > > >> > > > > > >>
>> > > >> > > > > > >> *Note that neither of the above suggestions require
>> > Ignite
>> > > to
>> > > >> > send
>> > > >> > > > or
>> > > >> > > > > > >> track any user information whatsoever.*
>> > > >> > > > > > >>
>> > > >> > > > > > >> Please reply suggesting weather you are OK with this
>> > > >> approach.
>> > > >> > > > > > >>
>> > > >> > > > > > >> D.
>> > > >> > > > > > >>
>> > > >> > > > >
>> > > >> > > > >
>> > > >> > > >
>> > > >> > >
>> > > >> >
>> > > >>
>> > > >
>> > > >
>> > >
>> >
>>
>

[Alexey Kuznetsov]

Cos, Denis.

I think it is because we have embedded videos (on YouTube).
Mauricio or Denis, please check my idea.

On Fri, Sep 29, 2017 at 8:02 AM, Konstantin Boudnik <[hidden email]> wrote:

> Sorry guys - I neglected the fact that our lists don't permit
> attachments. I have put the screenshot to an external server [1]
>
> [1] https://imgur.com/a/p9FJ9
>
> Thank you!
> --
>   With regards,
> Konstantin (Cos) Boudnik
> 2CAC 8312 4870 D885 8616  6115 220F 6980 1F27 E622
>
> Disclaimer: Opinions expressed in this email are those of the author,
> and do not necessarily represent the views of any company the author
> might be affiliated with at the moment of writing.
>
>
> On Thu, Sep 28, 2017 at 1:37 PM, Denis Magda <[hidden email]> wrote:
> > Cos,
> >
> > The screenshot was not attached. Could you share it some other way
> (google drive, etc.)? I’ve never seen any commercial on the site.
> >
> > —
> > Denis
> >
> >> On Sep 28, 2017, at 7:23 AM, Konstantin Boudnik <[hidden email]> wrote:
> >>
> >> I don't see an issue with node version either.
> >>
> >> Just one more, and it might be slightly irrelevant, issue though... I
> looked at the Ignite's site and found the following ads and trackers (that
> are indeed getting disabled by my browser).
> >> Why are googleads or doubleclick are permitted?
> >>
> >>
> >>
> >> Thanks,
> >>   Cos
> >>
> >>
> >> --
> >>   With regards,
> >> Konstantin (Cos) Boudnik
> >> 2CAC 8312 4870 D885 8616  6115 220F 6980 1F27 E622
> >>
> >> Disclaimer: Opinions expressed in this email are those of the author,
> and do not necessarily represent the views of any company the author might
> be affiliated with at the moment of writing.
> >>
> >> On Tue, Sep 26, 2017 at 3:21 PM, Dmitriy Setrakyan <
> [hidden email] <mailto:[hidden email]>> wrote:
> >> On Tue, Sep 26, 2017 at 6:20 AM, Vladimir Ozerov <[hidden email]
> <mailto:[hidden email]>>
> >> wrote:
> >>
> >> > Folks,
> >> >
> >> > Can we add version of current node to web request? This way we will
> better
> >> > understand version distribution, what might help us with certain API
> >> > update/deprecate decisions
> >> > E.g. http://ignite.apache.org/latest.cgi&ver=2.2.0 <
> http://ignite.apache.org/latest.cgi&ver=2.2.0>
> >>
> >>
> >> Vladimir, I personally do not see a problem with that, as sending the
> >> current version to the update checker seems very innocent to me. At the
> >> same time, it will allow us to examine the usage of each release and
> make
> >> decisions about dropping backward compatibility or spotting bugs for a
> >> certain release.
> >>
> >> Cos, Raul, any thoughts?
> >>
> >>
> >> >
> >> >
> >> > Vladimir.
> >> >
> >> > On Fri, Sep 8, 2017 at 7:06 AM, Dmitriy Setrakyan <
> [hidden email] <mailto:[hidden email]>>
> >> > wrote:
> >> >
> >> > > I think it is safe to assume at this point that everyone is in
> general
> >> > > agreement, since there are no active objections.
> >> > >
> >> > > I have filed a ticket for the 2.3 release. Let's try to make it
> happen:
> >> > > https://issues.apache.org/jira/browse/IGNITE-6305 <
> https://issues.apache.org/jira/browse/IGNITE-6305>
> >> > >
> >> > > D.
> >> > >
> >> > > On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan <
> >> > [hidden email] <mailto:[hidden email]>>
> >> > > wrote:
> >> > >
> >> > > >
> >> > > >
> >> > > > On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani <
> [hidden email] <mailto:[hidden email]>>
> >> > > > wrote:
> >> > > >
> >> > > >> Yeah, I guess that's doable as well and requires less management
> >> > effort
> >> > > >> than my suggestion. We could use events [1] to store payload data
> >> > (e.g.
> >> > > >> IP,
> >> > > >> version, etc.)
> >> > > >
> >> > > >
> >> > > > Yes, we could use events or some other similar API provided by GA.
> >> > > >
> >> > > >
> >> > > >> What the download page CGI developed in? PHP?
> >> > > >>
> >> > > >
> >> > > > To be honest, no clue. I guess someone in the community can
> figure it
> >> > > out:
> >> > > > https://svn.apache.org/repos/asf/ignite/site/trunk/download.html
> <https://svn.apache.org/repos/asf/ignite/site/trunk/download.html>
> >> > > >
> >> > > >
> >> > > >> However, I'm not sure whether storing this data in a 3rd party
> >> > (Google)
> >> > > is
> >> > > >> compliant with the ASF policy. I guess it's no biggie, but if
> there's
> >> > > >> doubt
> >> > > >> in the PMC, it's better to ask legal@.
> >> > > >
> >> > > >
> >> > > > I am not sure there is anything to ask about. The whole Ignite
> website
> >> > is
> >> > > > GA enabled, and all we are doing is accessing a standard web page
> from
> >> > > the
> >> > > > Ignite web site. The information gathered from GA is available
> only to
> >> > > the
> >> > > > Ignite PMC. Frankly, I think legal@ will be very confused by this
> >> > > > question.
> >> > > >
> >> > > > Even ASF website itself uses GA: https://www.apache.org/ <
> https://www.apache.org/>
> >> > > > foundation/policies/privacy.html
> >> > > >
> >> > > >
> >> > > >> I think Cos said it's OK; maybe Roman can pitch in.
> >> > > >>
> >> > > >
> >> > > >  Sure, would be nice to hear from Roman as well.
> >> > > >
> >> > > >
> >> > > >> Cheers.
> >> > > >>
> >> > > >> [1]
> >> > > >> https://developers.google.com/analytics/devguides/collection <
> https://developers.google.com/analytics/devguides/collection>
> >> > > >> /analyticsjs/events
> >> > > >>
> >> > > >> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan <
> >> > > >> [hidden email] <mailto:[hidden email]>>
> >> > > >> wrote:
> >> > > >>
> >> > > >> > Raul,
> >> > > >> >
> >> > > >> > Could point about Javascript, it will not work because it
> executes
> >> > in
> >> > > >> the
> >> > > >> > browser. This means we need a server-side script, like CGI we
> are
> >> > > using
> >> > > >> on
> >> > > >> > our download page.
> >> > > >> >
> >> > > >> > How about this approach. We create something like
> ignite-version.cgi
> >> > > >> script
> >> > > >> > which will invoke a call to GA and then return the latest
> version.
> >> > > >> >
> >> > > >> > This should work, right?
> >> > > >> >
> >> > > >> > D.
> >> > > >> >
> >> > > >> > On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani <
> >> > [hidden email] <mailto:[hidden email]>
> >> > > >
> >> > > >> > wrote:
> >> > > >> >
> >> > > >> > > Hey Dmitriy and all
> >> > > >> > >
> >> > > >> > > Also, since we have GA enabled for the website, we can track
> how
> >> > > many
> >> > > >> > times
> >> > > >> > > > this page was accessed, which will be equal to the number
> of
> >> > > starts.
> >> > > >> > This
> >> > > >> > > > way, the counter information is tracked and monitored by
> the
> >> > > Ignite
> >> > > >> > PMC.
> >> > > >> > >
> >> > > >> > >
> >> > > >> > > Unfortunately this won't work because GA is loaded via
> Javascript
> >> > on
> >> > > >> the
> >> > > >> > > browser, so Google will never receive the page hit.
> >> > > >> > >
> >> > > >> > > Given the constraints, the most viable solution is an HTTPS
> >> > endpoint
> >> > > >> > > running on ASF infra that Ignite invokes via a GET or POST
> >> > request.
> >> > > >> The
> >> > > >> > > simplest thing is to write each request in a log file, along
> with
> >> > > the
> >> > > >> > > timestamp, the version reported by the client, maybe the IP
> (not
> >> > > sure
> >> > > >> > about
> >> > > >> > > the ASF rules about this concerning privacy, I guess it's OK
> if
> >> > you
> >> > > >> make
> >> > > >> > it
> >> > > >> > > an opt-in) and a unique node identifier, i.e. a UUID the node
> >> > > creates
> >> > > >> on
> >> > > >> > > first startup or something.
> >> > > >> > >
> >> > > >> > > This endpoint would need some basic DDoS protection and
> >> > blacklisting
> >> > > >> to
> >> > > >> > > prevent data spoofing.
> >> > > >> > >
> >> > > >> > > If we'll be implementing this endpoint anyway, then there's
> no
> >> > point
> >> > > >> > > placing another file on Git or elsewhere for reporting the
> latest
> >> > > >> > versions:
> >> > > >> > > the endpoint itself can return them.
> >> > > >> > >
> >> > > >> > > WDYT?
> >> > > >> > >
> >> > > >> > > Cheers.
> >> > > >> > >
> >> > > >> > > On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan <
> >> > > >> > [hidden email] <mailto:[hidden email]>>
> >> > > >> > > wrote:
> >> > > >> > >
> >> > > >> > > > Cos, Raul,
> >> > > >> > > >
> >> > > >> > > > Thanks for the feedback. I completely agree about Maven
> Central
> >> > > >> being a
> >> > > >> > > 3rd
> >> > > >> > > > party repo (did not think about that initially). All your
> >> > > >> suggestions
> >> > > >> > > make
> >> > > >> > > > sense, but I would like to keep it as simple as possible,
> and so
> >> > > far
> >> > > >> > > > everything suggested required GIT dependencies and extra
> work.
> >> > > >> > > >
> >> > > >> > > > How about Yakov's suggestion. We simply add a page to the
> Ignite
> >> > > >> > website
> >> > > >> > > > which will have only the latest version. Every time a node
> >> > starts,
> >> > > >> it
> >> > > >> > > > receives the latest version from the page and suggests that
> >> > users
> >> > > >> > upgrade
> >> > > >> > > > if needed.
> >> > > >> > > >
> >> > > >> > > > Also, since we have GA enabled for the website, we can
> track how
> >> > > >> many
> >> > > >> > > times
> >> > > >> > > > this page was accessed, which will be equal to the number
> of
> >> > > starts.
> >> > > >> > This
> >> > > >> > > > way, the counter information is tracked and monitored by
> the
> >> > > Ignite
> >> > > >> > PMC.
> >> > > >> > > >
> >> > > >> > > > This approach looks pretty innocent to me and everything
> is kept
> >> > > and
> >> > > >> > > > managed within Apache.
> >> > > >> > > >
> >> > > >> > > > Thoughts?
> >> > > >> > > >
> >> > > >> > > > D.
> >> > > >> > > >
> >> > > >> > > >
> >> > > >> > > > On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik <
> >> > > >> [hidden email] <mailto:[hidden email]>>
> >> > > >> > > > wrote:
> >> > > >> > > >
> >> > > >> > > > > I agree with Raul.
> >> > > >> > > > > - providing a ping-back address to a 3rd party might be
> frown
> >> > > >> upon by
> >> > > >> > > > some.
> >> > > >> > > > >   And might have a consequences like stats collection
> about
> >> > > users'
> >> > > >> > > > >   infrastructure.
> >> > > >> > > > > - checking an ASF git-repo is easy and won't download any
> >> > binary
> >> > > >> > data:
> >> > > >> > > > >   everything is clear text and could be easily monitored
> by
> >> > any
> >> > > of
> >> > > >> > the
> >> > > >> > > > > network
> >> > > >> > > > >   diagnostic tools, shall it be required. But it
> involves a
> >> > bit
> >> > > of
> >> > > >> > the
> >> > > >> > > > > release
> >> > > >> > > > >   discipline.
> >> > > >> > > > > - the binary data download in the runtime is my main
> concern.
> >> > > >> This is
> >> > > >> > > the
> >> > > >> > > > >   vector of MMA. What if someone gains the control over
> the
> >> > > >> > repository
> >> > > >> > > > and
> >> > > >> > > > >   replaces the file with some malicious content.
> >> > > >> > > > >
> >> > > >> > > > > As for the particular mechanism: IIRC Ignite used to
> make a
> >> > call
> >> > > >> to
> >> > > >> > an
> >> > > >> > > > > external script to check upon the atest software version
> >> > > available
> >> > > >> > for
> >> > > >> > > > > download. In the past, the endpoint was running on a 3rd
> party
> >> > > >> > server,
> >> > > >> > > I
> >> > > >> > > > > believe the best way would be to put this script on ASF
> infra
> >> > > and
> >> > > >> > have
> >> > > >> > > > the
> >> > > >> > > > > "update checker" running in a completely controlled
> >> > environment.
> >> > > >> > > > Actually,
> >> > > >> > > > > I
> >> > > >> > > > > recall we had this very discussion during the
> Incubation; I
> >> > can
> >> > > >> > > probably
> >> > > >> > > > > dig
> >> > > >> > > > > out the corresponding thread.
> >> > > >> > > > >
> >> > > >> > > > > Thoughts?
> >> > > >> > > > >   Cok
> >> > > >> > > > >
> >> > > >> > > > > On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote:
> >> > > >> > > > > > Hey guys
> >> > > >> > > > > >
> >> > > >> > > > > > In my opinion, maven.org <http://maven.org/> is still
> owned by a third party
> >> > > >> > (Sonatype).
> >> > > >> > > > We
> >> > > >> > > > > > don't know what kind of data analysis or intelligence
> >> > > extraction
> >> > > >> > they
> >> > > >> > > > > run.
> >> > > >> > > > > >
> >> > > >> > > > > > If Ignite servers all over the world were hitting
> maven.org <http://maven.org/>
> >> > > >> > > > periodically
> >> > > >> > > > > > asking for an Ignite Maven artifact, it gives Sonatype
> a
> >> > clear
> >> > > >> > > > indication
> >> > > >> > > > > > of who is running an Ignite server.
> >> > > >> > > > > >
> >> > > >> > > > > > They could reverse lookup the IP address and find out
> what
> >> > > >> > > corporation
> >> > > >> > > > it
> >> > > >> > > > > > is.
> >> > > >> > > > > >
> >> > > >> > > > > > How about having Ignite check the ASF Git directly?
> >> > > >> > > > > >
> >> > > >> > > > > > We could use the Git ssh API, but that would require a
> new
> >> > > >> > > dependency,
> >> > > >> > > > > > which I advise against.
> >> > > >> > > > > >
> >> > > >> > > > > > Alternatively, we could have it scrape this HTML for
> new Git
> >> > > >> tags:
> >> > > >> > > > > > https://git-wip-us.apache.org/repos/asf?p=ignite.git <
> https://git-wip-us.apache.org/repos/asf?p=ignite.git>
> >> > > >> > > > > >
> >> > > >> > > > > > Another option is to place a txt file in the root of
> the
> >> > > master
> >> > > >> > > branch
> >> > > >> > > > > (e.g
> >> > > >> > > > > > LATEST), containing a list of the latest GA versions
> for
> >> > each
> >> > > >> major
> >> > > >> > > > > version
> >> > > >> > > > > > line (1.x, 2.x).
> >> > > >> > > > > >
> >> > > >> > > > > > I would advocate this last option, but it requires
> somebody
> >> > > >> > > remembering
> >> > > >> > > > > to
> >> > > >> > > > > > update the file with every release, unless we automate
> it
> >> > > with a
> >> > > >> > > Maven
> >> > > >> > > > > > plugin.
> >> > > >> > > > > >
> >> > > >> > > > > > Hope that helps!
> >> > > >> > > > > >
> >> > > >> > > > > >
> >> > > >> > > > > > On 24 Aug 2017 19:37, "Denis Magda" <[hidden email]
> <mailto:[hidden email]>>
> >> > > wrote:
> >> > > >> > > > > >
> >> > > >> > > > > > I see nothing wrong with this approach.
> >> > > >> > > > > >
> >> > > >> > > > > > Cos, Roman, Raul, as Apache veterans, what do you
> think? Is
> >> > it
> >> > > >> good
> >> > > >> > > to
> >> > > >> > > > > go?
> >> > > >> > > > > >
> >> > > >> > > > > > —
> >> > > >> > > > > > Denis
> >> > > >> > > > > >
> >> > > >> > > > > > > On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan <
> >> > > >> > > > [hidden email] <mailto:[hidden email]>
> >> > > >> > > > > >
> >> > > >> > > > > > wrote:
> >> > > >> > > > > > >
> >> > > >> > > > > > > Is everyone OK with this approach? Should I file a
> ticket
> >> > on
> >> > > >> it?
> >> > > >> > > > > > >
> >> > > >> > > > > > > On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <
> >> > > >> > > > > [hidden email] <mailto:[hidden email]>>
> >> > > >> > > > > > > wrote:
> >> > > >> > > > > > >
> >> > > >> > > > > > >> Igniters,
> >> > > >> > > > > > >>
> >> > > >> > > > > > >> There has been lots of talk of proposals about
> various
> >> > > usage
> >> > > >> > > metrics
> >> > > >> > > > > for
> >> > > >> > > > > > >> Ignite and nothing came of it. I would like to
> resurrect
> >> > > the
> >> > > >> > topic
> >> > > >> > > > and
> >> > > >> > > > > > >> propose something very simple and non-intrusive.
> >> > > >> > > > > > >>
> >> > > >> > > > > > >> 1. Update Checker
> >> > > >> > > > > > >> The main purpose of the update checker is not to
> collect
> >> > > >> > metrics,
> >> > > >> > > > but
> >> > > >> > > > > to
> >> > > >> > > > > > >> notify users about a new version of Ignite by
> accessing
> >> > > >> > maven.org <http://maven.org/>
> >> > > >> > > > and
> >> > > >> > > > > > >> getting the version out of the metadata file:
> >> > > >> > > > > > >> http://repo2.maven.org/maven2/ <
> http://repo2.maven.org/maven2/>
> >> > > org/apache/ignite/ignite-core/
> >> > > >> > > > > > >> maven-metadata.xml
> >> > > >> > > > > > >>
> >> > > >> > > > > > >> This way we do not send any information anywhere
> and, at
> >> > > the
> >> > > >> > same
> >> > > >> > > > > time,
> >> > > >> > > > > > >> urge our users to download and start using the
> latest
> >> > > >> version of
> >> > > >> > > > > Ignite.
> >> > > >> > > > > > >>
> >> > > >> > > > > > >> 2. Startup Counter
> >> > > >> > > > > > >> This piece is optional, but we can also get an
> insight in
> >> > > how
> >> > > >> > many
> >> > > >> > > > > times
> >> > > >> > > > > > a
> >> > > >> > > > > > >> certain Ignite release gets started. This is just a
> cool
> >> > > >> metric
> >> > > >> > > for
> >> > > >> > > > > the
> >> > > >> > > > > > >> community to gauge the project popularity. You can
> think
> >> > of
> >> > > >> it
> >> > > >> > as
> >> > > >> > > > of a
> >> > > >> > > > > > page
> >> > > >> > > > > > >> visit counter shown on many websites. We can even
> decide
> >> > to
> >> > > >> > > display
> >> > > >> > > > > this
> >> > > >> > > > > > >> counter on the Ignite website as well.
> >> > > >> > > > > > >>
> >> > > >> > > > > > >> To do this, we can simply add a JAR in maven for
> every
> >> > > >> release,
> >> > > >> > > e.g.
> >> > > >> > > > > > >> ignite-start-counter.jar, which will contain only 1
> byte.
> >> > > >> Every
> >> > > >> > > time
> >> > > >> > > > > an
> >> > > >> > > > > > >> Ignite node starts, it will download this JAR in the
> >> > > >> background.
> >> > > >> > > > Then
> >> > > >> > > > > we
> >> > > >> > > > > > >> will be able to view the number of the total
> downloads
> >> > for
> >> > > >> this
> >> > > >> > > JAR
> >> > > >> > > > in
> >> > > >> > > > > > >> Maven Central, which is essentially the number of
> starts
> >> > of
> >> > > >> > Ignite
> >> > > >> > > > > nodes.
> >> > > >> > > > > > >>
> >> > > >> > > > > > >> *Note that neither of the above suggestions require
> >> > Ignite
> >> > > to
> >> > > >> > send
> >> > > >> > > > or
> >> > > >> > > > > > >> track any user information whatsoever.*
> >> > > >> > > > > > >>
> >> > > >> > > > > > >> Please reply suggesting weather you are OK with this
> >> > > >> approach.
> >> > > >> > > > > > >>
> >> > > >> > > > > > >> D.
> >> > > >> > > > > > >>
> >> > > >> > > > >
> >> > > >> > > > >
> >> > > >> > > >
> >> > > >> > >
> >> > > >> >
> >> > > >>
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> >
>

--
Alexey Kuznetsov

[Denis Magda]

That’s definitely worthwhile checking. Prachi, as the one who embedded the screencast, would you check the theory?

—
Denis

> On Sep 28, 2017, at 11:50 PM, Alexey Kuznetsov <[hidden email]> wrote:
>
> Cos, Denis.
>
> I think it is because we have embedded videos (on YouTube).
> Mauricio or Denis, please check my idea.
>
> On Fri, Sep 29, 2017 at 8:02 AM, Konstantin Boudnik <[hidden email] <mailto:[hidden email]>> wrote:
> Sorry guys - I neglected the fact that our lists don't permit
> attachments. I have put the screenshot to an external server [1]
>
> [1] https://imgur.com/a/p9FJ9 <https://imgur.com/a/p9FJ9>
>
> Thank you!
> --
>   With regards,
> Konstantin (Cos) Boudnik
> 2CAC 8312 4870 D885 8616  6115 220F 6980 1F27 E622
>
> Disclaimer: Opinions expressed in this email are those of the author,
> and do not necessarily represent the views of any company the author
> might be affiliated with at the moment of writing.
>
>
> On Thu, Sep 28, 2017 at 1:37 PM, Denis Magda <[hidden email] <mailto:[hidden email]>> wrote:
> > Cos,
> >
> > The screenshot was not attached. Could you share it some other way (google drive, etc.)? I’ve never seen any commercial on the site.
> >
> > —
> > Denis
> >
> >> On Sep 28, 2017, at 7:23 AM, Konstantin Boudnik <[hidden email] <mailto:[hidden email]>> wrote:
> >>
> >> I don't see an issue with node version either.
> >>
> >> Just one more, and it might be slightly irrelevant, issue though... I looked at the Ignite's site and found the following ads and trackers (that are indeed getting disabled by my browser).
> >> Why are googleads or doubleclick are permitted?
> >>
> >>
> >>
> >> Thanks,
> >>   Cos
> >>
> >>
> >> --
> >>   With regards,
> >> Konstantin (Cos) Boudnik
> >> 2CAC 8312 4870 D885 8616  6115 220F 6980 1F27 E622
> >>
> >> Disclaimer: Opinions expressed in this email are those of the author, and do not necessarily represent the views of any company the author might be affiliated with at the moment of writing.
> >>
> >> On Tue, Sep 26, 2017 at 3:21 PM, Dmitriy Setrakyan <[hidden email] <mailto:[hidden email]> <mailto:[hidden email] <mailto:[hidden email]>>> wrote:
> >> On Tue, Sep 26, 2017 at 6:20 AM, Vladimir Ozerov <[hidden email] <mailto:[hidden email]> <mailto:[hidden email] <mailto:[hidden email]>>>
> >> wrote:
> >>
> >> > Folks,
> >> >
> >> > Can we add version of current node to web request? This way we will better
> >> > understand version distribution, what might help us with certain API
> >> > update/deprecate decisions
> >> > E.g. http://ignite.apache.org/latest.cgi&ver=2.2.0 <http://ignite.apache.org/latest.cgi&ver=2.2.0> <http://ignite.apache.org/latest.cgi&ver=2.2.0 <http://ignite.apache.org/latest.cgi&ver=2.2.0>>
> >>
> >>
> >> Vladimir, I personally do not see a problem with that, as sending the
> >> current version to the update checker seems very innocent to me. At the
> >> same time, it will allow us to examine the usage of each release and make
> >> decisions about dropping backward compatibility or spotting bugs for a
> >> certain release.
> >>
> >> Cos, Raul, any thoughts?
> >>
> >>
> >> >
> >> >
> >> > Vladimir.
> >> >
> >> > On Fri, Sep 8, 2017 at 7:06 AM, Dmitriy Setrakyan <[hidden email] <mailto:[hidden email]> <mailto:[hidden email] <mailto:[hidden email]>>>
> >> > wrote:
> >> >
> >> > > I think it is safe to assume at this point that everyone is in general
> >> > > agreement, since there are no active objections.
> >> > >
> >> > > I have filed a ticket for the 2.3 release. Let's try to make it happen:
> >> > > https://issues.apache.org/jira/browse/IGNITE-6305 <https://issues.apache.org/jira/browse/IGNITE-6305> <https://issues.apache.org/jira/browse/IGNITE-6305 <https://issues.apache.org/jira/browse/IGNITE-6305>>
> >> > >
> >> > > D.
> >> > >
> >> > > On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan <
> >> > [hidden email] <mailto:[hidden email]> <mailto:[hidden email] <mailto:[hidden email]>>>
> >> > > wrote:
> >> > >
> >> > > >
> >> > > >
> >> > > > On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani <[hidden email] <mailto:[hidden email]> <mailto:[hidden email] <mailto:[hidden email]>>>
> >> > > > wrote:
> >> > > >
> >> > > >> Yeah, I guess that's doable as well and requires less management
> >> > effort
> >> > > >> than my suggestion. We could use events [1] to store payload data
> >> > (e.g.
> >> > > >> IP,
> >> > > >> version, etc.)
> >> > > >
> >> > > >
> >> > > > Yes, we could use events or some other similar API provided by GA.
> >> > > >
> >> > > >
> >> > > >> What the download page CGI developed in? PHP?
> >> > > >>
> >> > > >
> >> > > > To be honest, no clue. I guess someone in the community can figure it
> >> > > out:
> >> > > > https://svn.apache.org/repos/asf/ignite/site/trunk/download.html <https://svn.apache.org/repos/asf/ignite/site/trunk/download.html> <https://svn.apache.org/repos/asf/ignite/site/trunk/download.html <https://svn.apache.org/repos/asf/ignite/site/trunk/download.html>>
> >> > > >
> >> > > >
> >> > > >> However, I'm not sure whether storing this data in a 3rd party
> >> > (Google)
> >> > > is
> >> > > >> compliant with the ASF policy. I guess it's no biggie, but if there's
> >> > > >> doubt
> >> > > >> in the PMC, it's better to ask legal@.
> >> > > >
> >> > > >
> >> > > > I am not sure there is anything to ask about. The whole Ignite website
> >> > is
> >> > > > GA enabled, and all we are doing is accessing a standard web page from
> >> > > the
> >> > > > Ignite web site. The information gathered from GA is available only to
> >> > > the
> >> > > > Ignite PMC. Frankly, I think legal@ will be very confused by this
> >> > > > question.
> >> > > >
> >> > > > Even ASF website itself uses GA: https://www.apache.org/ <https://www.apache.org/> <https://www.apache.org/ <https://www.apache.org/>>
> >> > > > foundation/policies/privacy.html
> >> > > >
> >> > > >
> >> > > >> I think Cos said it's OK; maybe Roman can pitch in.
> >> > > >>
> >> > > >
> >> > > >  Sure, would be nice to hear from Roman as well.
> >> > > >
> >> > > >
> >> > > >> Cheers.
> >> > > >>
> >> > > >> [1]
> >> > > >> https://developers.google.com/analytics/devguides/collection <https://developers.google.com/analytics/devguides/collection> <https://developers.google.com/analytics/devguides/collection <https://developers.google.com/analytics/devguides/collection>>
> >> > > >> /analyticsjs/events
> >> > > >>
> >> > > >> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan <
> >> > > >> [hidden email] <mailto:[hidden email]> <mailto:[hidden email] <mailto:[hidden email]>>>
> >> > > >> wrote:
> >> > > >>
> >> > > >> > Raul,
> >> > > >> >
> >> > > >> > Could point about Javascript, it will not work because it executes
> >> > in
> >> > > >> the
> >> > > >> > browser. This means we need a server-side script, like CGI we are
> >> > > using
> >> > > >> on
> >> > > >> > our download page.
> >> > > >> >
> >> > > >> > How about this approach. We create something like ignite-version.cgi
> >> > > >> script
> >> > > >> > which will invoke a call to GA and then return the latest version.
> >> > > >> >
> >> > > >> > This should work, right?
> >> > > >> >
> >> > > >> > D.
> >> > > >> >
> >> > > >> > On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani <
> >> > [hidden email] <mailto:[hidden email]> <mailto:[hidden email] <mailto:[hidden email]>>
> >> > > >
> >> > > >> > wrote:
> >> > > >> >
> >> > > >> > > Hey Dmitriy and all
> >> > > >> > >
> >> > > >> > > Also, since we have GA enabled for the website, we can track how
> >> > > many
> >> > > >> > times
> >> > > >> > > > this page was accessed, which will be equal to the number of
> >> > > starts.
> >> > > >> > This
> >> > > >> > > > way, the counter information is tracked and monitored by the
> >> > > Ignite
> >> > > >> > PMC.
> >> > > >> > >
> >> > > >> > >
> >> > > >> > > Unfortunately this won't work because GA is loaded via Javascript
> >> > on
> >> > > >> the
> >> > > >> > > browser, so Google will never receive the page hit.
> >> > > >> > >
> >> > > >> > > Given the constraints, the most viable solution is an HTTPS
> >> > endpoint
> >> > > >> > > running on ASF infra that Ignite invokes via a GET or POST
> >> > request.
> >> > > >> The
> >> > > >> > > simplest thing is to write each request in a log file, along with
> >> > > the
> >> > > >> > > timestamp, the version reported by the client, maybe the IP (not
> >> > > sure
> >> > > >> > about
> >> > > >> > > the ASF rules about this concerning privacy, I guess it's OK if
> >> > you
> >> > > >> make
> >> > > >> > it
> >> > > >> > > an opt-in) and a unique node identifier, i.e. a UUID the node
> >> > > creates
> >> > > >> on
> >> > > >> > > first startup or something.
> >> > > >> > >
> >> > > >> > > This endpoint would need some basic DDoS protection and
> >> > blacklisting
> >> > > >> to
> >> > > >> > > prevent data spoofing.
> >> > > >> > >
> >> > > >> > > If we'll be implementing this endpoint anyway, then there's no
> >> > point
> >> > > >> > > placing another file on Git or elsewhere for reporting the latest
> >> > > >> > versions:
> >> > > >> > > the endpoint itself can return them.
> >> > > >> > >
> >> > > >> > > WDYT?
> >> > > >> > >
> >> > > >> > > Cheers.
> >> > > >> > >
> >> > > >> > > On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan <
> >> > > >> > [hidden email] <mailto:[hidden email]> <mailto:[hidden email] <mailto:[hidden email]>>>
> >> > > >> > > wrote:
> >> > > >> > >
> >> > > >> > > > Cos, Raul,
> >> > > >> > > >
> >> > > >> > > > Thanks for the feedback. I completely agree about Maven Central
> >> > > >> being a
> >> > > >> > > 3rd
> >> > > >> > > > party repo (did not think about that initially). All your
> >> > > >> suggestions
> >> > > >> > > make
> >> > > >> > > > sense, but I would like to keep it as simple as possible, and so
> >> > > far
> >> > > >> > > > everything suggested required GIT dependencies and extra work.
> >> > > >> > > >
> >> > > >> > > > How about Yakov's suggestion. We simply add a page to the Ignite
> >> > > >> > website
> >> > > >> > > > which will have only the latest version. Every time a node
> >> > starts,
> >> > > >> it
> >> > > >> > > > receives the latest version from the page and suggests that
> >> > users
> >> > > >> > upgrade
> >> > > >> > > > if needed.
> >> > > >> > > >
> >> > > >> > > > Also, since we have GA enabled for the website, we can track how
> >> > > >> many
> >> > > >> > > times
> >> > > >> > > > this page was accessed, which will be equal to the number of
> >> > > starts.
> >> > > >> > This
> >> > > >> > > > way, the counter information is tracked and monitored by the
> >> > > Ignite
> >> > > >> > PMC.
> >> > > >> > > >
> >> > > >> > > > This approach looks pretty innocent to me and everything is kept
> >> > > and
> >> > > >> > > > managed within Apache.
> >> > > >> > > >
> >> > > >> > > > Thoughts?
> >> > > >> > > >
> >> > > >> > > > D.
> >> > > >> > > >
> >> > > >> > > >
> >> > > >> > > > On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik <
> >> > > >> [hidden email] <mailto:[hidden email]> <mailto:[hidden email] <mailto:[hidden email]>>>
> >> > > >> > > > wrote:
> >> > > >> > > >
> >> > > >> > > > > I agree with Raul.
> >> > > >> > > > > - providing a ping-back address to a 3rd party might be frown
> >> > > >> upon by
> >> > > >> > > > some.
> >> > > >> > > > >   And might have a consequences like stats collection about
> >> > > users'
> >> > > >> > > > >   infrastructure.
> >> > > >> > > > > - checking an ASF git-repo is easy and won't download any
> >> > binary
> >> > > >> > data:
> >> > > >> > > > >   everything is clear text and could be easily monitored by
> >> > any
> >> > > of
> >> > > >> > the
> >> > > >> > > > > network
> >> > > >> > > > >   diagnostic tools, shall it be required. But it involves a
> >> > bit
> >> > > of
> >> > > >> > the
> >> > > >> > > > > release
> >> > > >> > > > >   discipline.
> >> > > >> > > > > - the binary data download in the runtime is my main concern.
> >> > > >> This is
> >> > > >> > > the
> >> > > >> > > > >   vector of MMA. What if someone gains the control over the
> >> > > >> > repository
> >> > > >> > > > and
> >> > > >> > > > >   replaces the file with some malicious content.
> >> > > >> > > > >
> >> > > >> > > > > As for the particular mechanism: IIRC Ignite used to make a
> >> > call
> >> > > >> to
> >> > > >> > an
> >> > > >> > > > > external script to check upon the atest software version
> >> > > available
> >> > > >> > for
> >> > > >> > > > > download. In the past, the endpoint was running on a 3rd party
> >> > > >> > server,
> >> > > >> > > I
> >> > > >> > > > > believe the best way would be to put this script on ASF infra
> >> > > and
> >> > > >> > have
> >> > > >> > > > the
> >> > > >> > > > > "update checker" running in a completely controlled
> >> > environment.
> >> > > >> > > > Actually,
> >> > > >> > > > > I
> >> > > >> > > > > recall we had this very discussion during the Incubation; I
> >> > can
> >> > > >> > > probably
> >> > > >> > > > > dig
> >> > > >> > > > > out the corresponding thread.
> >> > > >> > > > >
> >> > > >> > > > > Thoughts?
> >> > > >> > > > >   Cok
> >> > > >> > > > >
> >> > > >> > > > > On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote:
> >> > > >> > > > > > Hey guys
> >> > > >> > > > > >
> >> > > >> > > > > > In my opinion, maven.org <http://maven.org/> <http://maven.org/ <http://maven.org/>> is still owned by a third party
> >> > > >> > (Sonatype).
> >> > > >> > > > We
> >> > > >> > > > > > don't know what kind of data analysis or intelligence
> >> > > extraction
> >> > > >> > they
> >> > > >> > > > > run.
> >> > > >> > > > > >
> >> > > >> > > > > > If Ignite servers all over the world were hitting maven.org <http://maven.org/> <http://maven.org/ <http://maven.org/>>
> >> > > >> > > > periodically
> >> > > >> > > > > > asking for an Ignite Maven artifact, it gives Sonatype a
> >> > clear
> >> > > >> > > > indication
> >> > > >> > > > > > of who is running an Ignite server.
> >> > > >> > > > > >
> >> > > >> > > > > > They could reverse lookup the IP address and find out what
> >> > > >> > > corporation
> >> > > >> > > > it
> >> > > >> > > > > > is.
> >> > > >> > > > > >
> >> > > >> > > > > > How about having Ignite check the ASF Git directly?
> >> > > >> > > > > >
> >> > > >> > > > > > We could use the Git ssh API, but that would require a new
> >> > > >> > > dependency,
> >> > > >> > > > > > which I advise against.
> >> > > >> > > > > >
> >> > > >> > > > > > Alternatively, we could have it scrape this HTML for new Git
> >> > > >> tags:
> >> > > >> > > > > > https://git-wip-us.apache.org/repos/asf?p=ignite.git <https://git-wip-us.apache.org/repos/asf?p=ignite.git> <https://git-wip-us.apache.org/repos/asf?p=ignite.git <https://git-wip-us.apache.org/repos/asf?p=ignite.git>>
> >> > > >> > > > > >
> >> > > >> > > > > > Another option is to place a txt file in the root of the
> >> > > master
> >> > > >> > > branch
> >> > > >> > > > > (e.g
> >> > > >> > > > > > LATEST), containing a list of the latest GA versions for
> >> > each
> >> > > >> major
> >> > > >> > > > > version
> >> > > >> > > > > > line (1.x, 2.x).
> >> > > >> > > > > >
> >> > > >> > > > > > I would advocate this last option, but it requires somebody
> >> > > >> > > remembering
> >> > > >> > > > > to
> >> > > >> > > > > > update the file with every release, unless we automate it
> >> > > with a
> >> > > >> > > Maven
> >> > > >> > > > > > plugin.
> >> > > >> > > > > >
> >> > > >> > > > > > Hope that helps!
> >> > > >> > > > > >
> >> > > >> > > > > >
> >> > > >> > > > > > On 24 Aug 2017 19:37, "Denis Magda" <[hidden email] <mailto:[hidden email]> <mailto:[hidden email] <mailto:[hidden email]>>>
> >> > > wrote:
> >> > > >> > > > > >
> >> > > >> > > > > > I see nothing wrong with this approach.
> >> > > >> > > > > >
> >> > > >> > > > > > Cos, Roman, Raul, as Apache veterans, what do you think? Is
> >> > it
> >> > > >> good
> >> > > >> > > to
> >> > > >> > > > > go?
> >> > > >> > > > > >
> >> > > >> > > > > > —
> >> > > >> > > > > > Denis
> >> > > >> > > > > >
> >> > > >> > > > > > > On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan <
> >> > > >> > > > [hidden email] <mailto:[hidden email]> <mailto:[hidden email] <mailto:[hidden email]>>
> >> > > >> > > > > >
> >> > > >> > > > > > wrote:
> >> > > >> > > > > > >
> >> > > >> > > > > > > Is everyone OK with this approach? Should I file a ticket
> >> > on
> >> > > >> it?
> >> > > >> > > > > > >
> >> > > >> > > > > > > On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <
> >> > > >> > > > > [hidden email] <mailto:[hidden email]> <mailto:[hidden email] <mailto:[hidden email]>>>
> >> > > >> > > > > > > wrote:
> >> > > >> > > > > > >
> >> > > >> > > > > > >> Igniters,
> >> > > >> > > > > > >>
> >> > > >> > > > > > >> There has been lots of talk of proposals about various
> >> > > usage
> >> > > >> > > metrics
> >> > > >> > > > > for
> >> > > >> > > > > > >> Ignite and nothing came of it. I would like to resurrect
> >> > > the
> >> > > >> > topic
> >> > > >> > > > and
> >> > > >> > > > > > >> propose something very simple and non-intrusive.
> >> > > >> > > > > > >>
> >> > > >> > > > > > >> 1. Update Checker
> >> > > >> > > > > > >> The main purpose of the update checker is not to collect
> >> > > >> > metrics,
> >> > > >> > > > but
> >> > > >> > > > > to
> >> > > >> > > > > > >> notify users about a new version of Ignite by accessing
> >> > > >> > maven.org <http://maven.org/> <http://maven.org/ <http://maven.org/>>
> >> > > >> > > > and
> >> > > >> > > > > > >> getting the version out of the metadata file:
> >> > > >> > > > > > >> http://repo2.maven.org/maven2/ <http://repo2.maven.org/maven2/> <http://repo2.maven.org/maven2/ <http://repo2.maven.org/maven2/>>
> >> > > org/apache/ignite/ignite-core/
> >> > > >> > > > > > >> maven-metadata.xml
> >> > > >> > > > > > >>
> >> > > >> > > > > > >> This way we do not send any information anywhere and, at
> >> > > the
> >> > > >> > same
> >> > > >> > > > > time,
> >> > > >> > > > > > >> urge our users to download and start using the latest
> >> > > >> version of
> >> > > >> > > > > Ignite.
> >> > > >> > > > > > >>
> >> > > >> > > > > > >> 2. Startup Counter
> >> > > >> > > > > > >> This piece is optional, but we can also get an insight in
> >> > > how
> >> > > >> > many
> >> > > >> > > > > times
> >> > > >> > > > > > a
> >> > > >> > > > > > >> certain Ignite release gets started. This is just a cool
> >> > > >> metric
> >> > > >> > > for
> >> > > >> > > > > the
> >> > > >> > > > > > >> community to gauge the project popularity. You can think
> >> > of
> >> > > >> it
> >> > > >> > as
> >> > > >> > > > of a
> >> > > >> > > > > > page
> >> > > >> > > > > > >> visit counter shown on many websites. We can even decide
> >> > to
> >> > > >> > > display
> >> > > >> > > > > this
> >> > > >> > > > > > >> counter on the Ignite website as well.
> >> > > >> > > > > > >>
> >> > > >> > > > > > >> To do this, we can simply add a JAR in maven for every
> >> > > >> release,
> >> > > >> > > e.g.
> >> > > >> > > > > > >> ignite-start-counter.jar, which will contain only 1 byte.
> >> > > >> Every
> >> > > >> > > time
> >> > > >> > > > > an
> >> > > >> > > > > > >> Ignite node starts, it will download this JAR in the
> >> > > >> background.
> >> > > >> > > > Then
> >> > > >> > > > > we
> >> > > >> > > > > > >> will be able to view the number of the total downloads
> >> > for
> >> > > >> this
> >> > > >> > > JAR
> >> > > >> > > > in
> >> > > >> > > > > > >> Maven Central, which is essentially the number of starts
> >> > of
> >> > > >> > Ignite
> >> > > >> > > > > nodes.
> >> > > >> > > > > > >>
> >> > > >> > > > > > >> *Note that neither of the above suggestions require
> >> > Ignite
> >> > > to
> >> > > >> > send
> >> > > >> > > > or
> >> > > >> > > > > > >> track any user information whatsoever.*
> >> > > >> > > > > > >>
> >> > > >> > > > > > >> Please reply suggesting weather you are OK with this
> >> > > >> approach.
> >> > > >> > > > > > >>
> >> > > >> > > > > > >> D.
> >> > > >> > > > > > >>
> >> > > >> > > > >
> >> > > >> > > > >
> >> > > >> > > >
> >> > > >> > >
> >> > > >> >
> >> > > >>
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> >
>
>
>
> --
> Alexey Kuznetsov