Apache Ignite Developers - Legacy Mail Archive

Development process

Classic

List

Threaded

16 messages Options

yzhdanov

Development process

Igniters,

We have got pretty cool changes in current ignite-sprint-4 branch (already
merged or are ready to be merged):
* Added Google Compute Engine TCP discovery IP finder.
* Added generic cloud TCP discovery IP finder (based on jcoulds).
* Added SortedEvictionPolicy.
* Added chaining for IgniteConfiguration and CacheConfiguration setters.
* Improved expiry policy handling.
* Fixed job continuations.
* Fixed compilation and runtime with OpenJDK 7 & 8
* Many stability and fault-tolerance fixes.

Let's try releasing it as apache-ignite-1.1!

Please continue new functionality development in ignite-sprint-5 (whoever
has anything to commit to will create this branch).

Make sure that no new functionality ignite-sprint-4.

Release instructions can be found in DEVNOTES.txt - Ignite Release
Instructions section. in git -
https://git-wip-us.apache.org/repos/asf?p=incubator-ignite.git;a=blob_plain;f=DEVNOTES.txt;hb=refs/heads/ignite-sprint-4

--Yakov

Yakov Zhdanov-2

Fwd: Development process

dsetrakyan

Re: Development process

In reply to this post by yzhdanov

I think one of the biggest improvements in this release is fully automated
build process. As Yakov suggested, the instructions to run the build are in
the DEVNOTES.txt in sprint-4 branch:

http://s.apache.org/caf

Brane, Cos, please take a look and let us know if you have any comments.

D.

On Wed, Apr 29, 2015 at 4:03 AM, Yakov Zhdanov <[hidden email]> wrote:

> Igniters,
>
> We have got pretty cool changes in current ignite-sprint-4 branch (already
> merged or are ready to be merged):
> * Added Google Compute Engine TCP discovery IP finder.
> * Added generic cloud TCP discovery IP finder (based on jcoulds).
> * Added SortedEvictionPolicy.
> * Added chaining for IgniteConfiguration and CacheConfiguration setters.
> * Improved expiry policy handling.
> * Fixed job continuations.
> * Fixed compilation and runtime with OpenJDK 7 & 8
> * Many stability and fault-tolerance fixes.
>
> Let's try releasing it as apache-ignite-1.1!
>
> Please continue new functionality development in ignite-sprint-5 (whoever
> has anything to commit to will create this branch).
>
> Make sure that no new functionality ignite-sprint-4.
>
> Release instructions can be found in DEVNOTES.txt - Ignite Release
> Instructions section. in git -
>
> https://git-wip-us.apache.org/repos/asf?p=incubator-ignite.git;a=blob_plain;f=DEVNOTES.txt;hb=refs/heads/ignite-sprint-4
>
> --Yakov
>

Konstantin Boudnik-2

Re: Development process

In reply to this post by yzhdanov

That's great! I am personally looking for this fix IGNITE-791 as Bigtop 1.0
is blocked by it ;( Thanks!

Cos

On Wed, Apr 29, 2015 at 12:03PM, Yakov Zhdanov wrote:

> Igniters,
>
> We have got pretty cool changes in current ignite-sprint-4 branch (already
> merged or are ready to be merged):
> * Added Google Compute Engine TCP discovery IP finder.
> * Added generic cloud TCP discovery IP finder (based on jcoulds).
> * Added SortedEvictionPolicy.
> * Added chaining for IgniteConfiguration and CacheConfiguration setters.
> * Improved expiry policy handling.
> * Fixed job continuations.
> * Fixed compilation and runtime with OpenJDK 7 & 8
> * Many stability and fault-tolerance fixes.
>
> Let's try releasing it as apache-ignite-1.1!
>
> Please continue new functionality development in ignite-sprint-5 (whoever
> has anything to commit to will create this branch).
>
> Make sure that no new functionality ignite-sprint-4.
>
> Release instructions can be found in DEVNOTES.txt - Ignite Release
> Instructions section. in git -
> https://git-wip-us.apache.org/repos/asf?p=incubator-ignite.git;a=blob_plain;f=DEVNOTES.txt;hb=refs/heads/ignite-sprint-4
>
> --Yakov

Branko Čibej

Re: Development process

In reply to this post by dsetrakyan

On 29.04.2015 18:10, Dmitriy Setrakyan wrote:
> I think one of the biggest improvements in this release is fully automated
> build process. As Yakov suggested, the instructions to run the build are in
> the DEVNOTES.txt in sprint-4 branch:
>
> http://s.apache.org/caf
>
> Brane, Cos, please take a look and let us know if you have any comments.

Ah, thanks for reminding me. I do have a few comments.

Fist of all, the release instructions don't say anything about release
artefact naming. I got the impression that the package names were more
or less an accident; certainly they've been less than consistent in the
last couple release attempts! It's also not clear from the instructions
how to correctly create the source package.

It's important that the source package creation and naming is consistent
and not an accident of how somebody happens to have their work area set up.

> Deploy Ignite release to maven repository and site:
> ...
> Start vote based on people.apache.org/~<username>/ignite-version.

That's the wrong way around. Vote first, then release. If there's a
/staging/ area in the maven repo, then that's different (and equivalent
to the staging area on dist.apache.org), so please make that clear in
the release docs. It's not clear to me, although it's sort of implied by:

> Release nexus staging when version accepted.

but ... how do you do that? Document it; it's not obvious that everyone
should be completely familiar with Maven repository management.

> Configure access to people.apache.org/~<username>. It necessary to upload sources to site.

I think I mentioned this before but I'd /really/ recommend that you use
the dist repository for this, not some random home directory, even if it
is on people.apache.org.

Currently, you put final releases in

https://dist.apache.org/repos/dist/release/incubator/ignite

which is per spec. I strongly recommend that you put source packages
that are ready for release voting into

https://dist.apache.org/repos/dist/dev/incubator/ignite

because then, once a release has been approved, a single remote "svn
copy" operation will move the package to the dist/release area, leaving
no room for error in this step. It also makes the release process
independent of any single person's login on people.apache.org.

Oh, by the way, the release instructions say nothing about putting the
source release on dist/release ...

> You might need to allow connection to people.apache.org. Just do following at console:
>
> ssh people.apache.org
>
> and type "yes" + enter.

Do not ever again recommend bad security practices in your docs! You
must *not* "type yes + enter", you must double-check the server key
fingerprint before trusting the server.

-- Brane

Konstantin Boudnik-2

Git practices [Was: Development process]

In reply to this post by yzhdanov

Guys,

I don't want to hijack the original thread, so I am forking it ;)

I've been watching the git tree for the sprint branches and I have one general
comment, which I always pushing for when it gets to correct use of git. Here
it is:
- during the development on a branch one has to re-sync with the main
integration branch (ie ignite-sprint-X)
- git has two options of doing this: merge and rebase
- merge preserves the history of the, well, branch merges - potentially
producing a complex intervened picture of branches like

│ │ ● │ minor
│ │ ●━┑ Merge remote-tracking branch 'origin/ignite-sprint-4' into ignite-sprint-4
│ │ │ ●━┑ merge from ignite-790 to ignite-sprint-4
│ │ │ │ ● ignite-790: improvements after the review
│ │ │ │ ● review
│ │ │ │ ●━┑ Merge branches 'ignite-790' and 'ignite-sprint-4' of https://git-wip-us.apache.org/repo~
│ │ │ │ ● │ ignite-790: several times performance improvements
│ │ │ │ ● │ ignite-790: added example configs for clouds
│ │ │ │ ● │ ignite-790: default configs
│ │ │ │ ● │ ignite-790: fixes, improvements, tests
│ │ │ │ ● │ ignite-709: read private key for GCE from the file
│ │ │ │ ● │ temporal commit, forgot to switch to the branch
│ │ │ ●━┑ │ Merge branch 'ignite-734' into ignite-sprint-4
│ │ │ │ ● │ review
│ │ │ │ ●━┪ Merge branches 'ignite-734' and 'ignite-sprint-4' of https://git-wip-us.apache.org/repo~
│ │ │ │ ● │ ignite-734: finished implementation, provided tests
│ │ │ │ ● │ ignite-734: google cloud storage ipfinder is fully implemented
│ │ │ │ ● │ ignite-734: implemented ip finder
│ │ │ │ ● │ ignite-734: start implementing google ip finder
│ │ ● │ │ │ fixed NPE
│ ● │ │ │ │ javadoc fix
●━┑ │ │ │ │ Merge branches 'ignite-839' and 'ignite-sprint-4' of https://git-wip-us.apache.org/repo~

- rebase rewrites the new commits on top of a specified HEAD, hence producing
a flat line of development
- merge makes sense for feature branches integration, automatic branch
management by CI, etc.
- rebase makes sense when you're working on your feature for some time and
don't want to show the whole world how many times you were synching with
the integration branch. Keep in mind, that rebase rewrites the commits
hence changing their sha1 hashes. Be considerate about this and never
rebase something that has been already pushed to the remote repo
- In the example above perhaps no one would either care for the fact that
there was 5 consequent commits for ignite-790. This particular situation
could be fixes with 'squash' (a form of interactive rebase), but I will
leave this exercise for the readers.

These simple rules allow to produce a much cleaner git history, which is
easier to work with, bisect if needed, and in general do any sorts of tracing.

Please let me know if you have any questions.
Cos

On Wed, Apr 29, 2015 at 12:03PM, Yakov Zhdanov wrote:

> Igniters,
>
> We have got pretty cool changes in current ignite-sprint-4 branch (already
> merged or are ready to be merged):
> * Added Google Compute Engine TCP discovery IP finder.
> * Added generic cloud TCP discovery IP finder (based on jcoulds).
> * Added SortedEvictionPolicy.
> * Added chaining for IgniteConfiguration and CacheConfiguration setters.
> * Improved expiry policy handling.
> * Fixed job continuations.
> * Fixed compilation and runtime with OpenJDK 7 & 8
> * Many stability and fault-tolerance fixes.
>
> Let's try releasing it as apache-ignite-1.1!
>
> Please continue new functionality development in ignite-sprint-5 (whoever
> has anything to commit to will create this branch).
>
> Make sure that no new functionality ignite-sprint-4.
>
> Release instructions can be found in DEVNOTES.txt - Ignite Release
> Instructions section. in git -
> https://git-wip-us.apache.org/repos/asf?p=incubator-ignite.git;a=blob_plain;f=DEVNOTES.txt;hb=refs/heads/ignite-sprint-4
>
> --Yakov

Branko Čibej

Re: Git practices [Was: Development process]

On 30.04.2015 00:48, Konstantin Boudnik wrote:
> These simple rules allow to produce a much cleaner git history, which is
> easier to work with, bisect if needed, and in general do any sorts of tracing.

I have to point out that the goal of version control is not to have a
"clean git history" but to have an audit trail of what went into the
code, when and by whom. Rebase destroys that information. While I can
marginally understand people wanting to look nice in public and rebasing
stuff in their local repository clones, rebasing anything that's already
been pushed upstream is a recipe for disaster (and repository corruption).

We quite recently had a case where we could not trace the history of
jdk8 backports and therefore could not decide whether they were public
domain or GPL. Really, that's a horrible way to do version control.

Maybe instead of trying to fudge the history to make it "cleaner", you
could consider a different workflow that doesn't involve creating a
branch fore every line of code you write. That's a bit like sayin, Oh, I
have a hammer, let's go find as many nails as I can, instead of just
hammering in those nails you actually need. And use a wrench for the
nuts, please, a hammer just ruins them.

-- Brane

P.S.: Before someone accuses me of an anti-git bias, let me just mention
that I've been studying version control workflows for 20 years and
writing a version control system for 15 ... this isn't about git, it's
about rational development process.

Konstantin Boudnik-2

Re: Git practices [Was: Development process]

On Thu, Apr 30, 2015 at 08:05AM, Branko Čibej wrote:

Never at any point I have advocated that. More over, I have explicitly warned
not to do it. The only case for rebases when you
- don't need to expose that you did 26 merges with the master branch while
working on your fix - this info is irrelevant to anyone
- need to squash a bunch of small commits into a set of larger logically
separated commits (e.g. per JIRA tickets or something)

Any tool can be abused. I am only suggesting that polluting the history with
useless merges doesn't help anyone with anything.

Cos

> We quite recently had a case where we could not trace the history of
> jdk8 backports and therefore could not decide whether they were public
> domain or GPL. Really, that's a horrible way to do version control.
>
> Maybe instead of trying to fudge the history to make it "cleaner", you
> could consider a different workflow that doesn't involve creating a
> branch fore every line of code you write. That's a bit like sayin, Oh, I
> have a hammer, let's go find as many nails as I can, instead of just
> hammering in those nails you actually need. And use a wrench for the
> nuts, please, a hammer just ruins them.
>
> -- Brane
>
> P.S.: Before someone accuses me of an anti-git bias, let me just mention
> that I've been studying version control workflows for 20 years and
> writing a version control system for 15 ... this isn't about git, it's
> about rational development process.

Anton Vinogradov

Re: Development process

In reply to this post by Branko Čibej

Brane, thanks for review

Release artifact name template is specified at main pom.xml:

<ignite.zip.pattern>ignite-${ignite.edition}-${project.version}-incubating</ignite.zip.pattern>

This property value can be changed, for example by command line parameter.
I've added this to instruction.

Source package can be created by executing the following:

mvn package -P apache-release

Source package name is ignite-${project.version}-source-release.zip. This
template specified at parent apache:16 pom.xml.

Release procedure will be changed at sprint-5 according to your comments.

Please check changes at sprint-5 DEVNOTES.txt.
https://git-wip-us.apache.org/repos/asf?p=incubator-ignite.git;a=blob_plain;f=DEVNOTES.txt;hb=refs/heads/ignite-sprint-5

On Thu, Apr 30, 2015 at 1:03 AM, Branko Čibej <[hidden email]> wrote:

> On 29.04.2015 18:10, Dmitriy Setrakyan wrote:
> > I think one of the biggest improvements in this release is fully
> automated
> > build process. As Yakov suggested, the instructions to run the build are
> in
> > the DEVNOTES.txt in sprint-4 branch:
> >
> > http://s.apache.org/caf
> >
> > Brane, Cos, please take a look and let us know if you have any comments.
>
>
> Ah, thanks for reminding me. I do have a few comments.
>
>
> Fist of all, the release instructions don't say anything about release
> artefact naming. I got the impression that the package names were more
> or less an accident; certainly they've been less than consistent in the
> last couple release attempts! It's also not clear from the instructions
> how to correctly create the source package.
>
> It's important that the source package creation and naming is consistent
> and not an accident of how somebody happens to have their work area set up.
>
>
> > Deploy Ignite release to maven repository and site:
> > ...
> > Start vote based on people.apache.org/~<username>/ignite-version.
>
> That's the wrong way around. Vote first, then release. If there's a
> /staging/ area in the maven repo, then that's different (and equivalent
> to the staging area on dist.apache.org), so please make that clear in
> the release docs. It's not clear to me, although it's sort of implied by:
>
> > Release nexus staging when version accepted.
>
> but ... how do you do that? Document it; it's not obvious that everyone
> should be completely familiar with Maven repository management.
>
> > Configure access to people.apache.org/~<username>. It necessary to
> upload sources to site.
>
> I think I mentioned this before but I'd /really/ recommend that you use
> the dist repository for this, not some random home directory, even if it
> is on people.apache.org.
>
> Currently, you put final releases in
>
> https://dist.apache.org/repos/dist/release/incubator/ignite
>
> which is per spec. I strongly recommend that you put source packages
> that are ready for release voting into
>
> https://dist.apache.org/repos/dist/dev/incubator/ignite
>
> because then, once a release has been approved, a single remote "svn
> copy" operation will move the package to the dist/release area, leaving
> no room for error in this step. It also makes the release process
> independent of any single person's login on people.apache.org.
>
>
> Oh, by the way, the release instructions say nothing about putting the
> source release on dist/release ...
>
>
> > You might need to allow connection to people.apache.org. Just do
> following at console:
> >
> > ssh people.apache.org
> >
> > and type "yes" + enter.
>
> Do not ever again recommend bad security practices in your docs! You
> must *not* "type yes + enter", you must double-check the server key
> fingerprint before trusting the server.
>
> -- Brane
>
>

dsetrakyan

Re: Development process

Anton,

Please try to put your answers inline going forward... It is hard to gage
which Brane's comments are addressed in sprint-4 and which in sprint-5.

Thanks for looking into this.

D.

On Thu, Apr 30, 2015 at 11:04 AM, Anton Vinogradov <[hidden email]
> wrote:

> Brane, thanks for review
>
> Release artifact name template is specified at main pom.xml:
>
>
> <ignite.zip.pattern>ignite-${ignite.edition}-${project.version}-incubating</ignite.zip.pattern>
>
> This property value can be changed, for example by command line parameter.
> I've added this to instruction.
>
> Source package can be created by executing the following:
>
> mvn package -P apache-release
>
> Source package name is ignite-${project.version}-source-release.zip. This
> template specified at parent apache:16 pom.xml.
>
> Release procedure will be changed at sprint-5 according to your comments.
>
> Please check changes at sprint-5 DEVNOTES.txt.
>
> https://git-wip-us.apache.org/repos/asf?p=incubator-ignite.git;a=blob_plain;f=DEVNOTES.txt;hb=refs/heads/ignite-sprint-5
>
> On Thu, Apr 30, 2015 at 1:03 AM, Branko Čibej <[hidden email]> wrote:
>
> > On 29.04.2015 18:10, Dmitriy Setrakyan wrote:
> > > I think one of the biggest improvements in this release is fully
> > automated
> > > build process. As Yakov suggested, the instructions to run the build
> are
> > in
> > > the DEVNOTES.txt in sprint-4 branch:
> > >
> > > http://s.apache.org/caf
> > >
> > > Brane, Cos, please take a look and let us know if you have any
> comments.
> >
> >
> > Ah, thanks for reminding me. I do have a few comments.
> >
> >
> > Fist of all, the release instructions don't say anything about release
> > artefact naming. I got the impression that the package names were more
> > or less an accident; certainly they've been less than consistent in the
> > last couple release attempts! It's also not clear from the instructions
> > how to correctly create the source package.
> >
> > It's important that the source package creation and naming is consistent
> > and not an accident of how somebody happens to have their work area set
> up.
> >
> >
> > > Deploy Ignite release to maven repository and site:
> > > ...
> > > Start vote based on people.apache.org/~<username>/ignite-version.
> >
> > That's the wrong way around. Vote first, then release. If there's a
> > /staging/ area in the maven repo, then that's different (and equivalent
> > to the staging area on dist.apache.org), so please make that clear in
> > the release docs. It's not clear to me, although it's sort of implied by:
> >
> > > Release nexus staging when version accepted.
> >
> > but ... how do you do that? Document it; it's not obvious that everyone
> > should be completely familiar with Maven repository management.
> >
> > > Configure access to people.apache.org/~<username>. It necessary to
> > upload sources to site.
> >
> > I think I mentioned this before but I'd /really/ recommend that you use
> > the dist repository for this, not some random home directory, even if it
> > is on people.apache.org.
> >
> > Currently, you put final releases in
> >
> > https://dist.apache.org/repos/dist/release/incubator/ignite
> >
> > which is per spec. I strongly recommend that you put source packages
> > that are ready for release voting into
> >
> > https://dist.apache.org/repos/dist/dev/incubator/ignite
> >
> > because then, once a release has been approved, a single remote "svn
> > copy" operation will move the package to the dist/release area, leaving
> > no room for error in this step. It also makes the release process
> > independent of any single person's login on people.apache.org.
> >
> >
> > Oh, by the way, the release instructions say nothing about putting the
> > source release on dist/release ...
> >
> >
> > > You might need to allow connection to people.apache.org. Just do
> > following at console:
> > >
> > > ssh people.apache.org
> > >
> > > and type "yes" + enter.
> >
> > Do not ever again recommend bad security practices in your docs! You
> > must *not* "type yes + enter", you must double-check the server key
> > fingerprint before trusting the server.
> >
> > -- Brane
> >
> >
>

dsetrakyan

Re: Git practices [Was: Development process]

In reply to this post by Branko Čibej

On Thu, Apr 30, 2015 at 1:05 AM, Branko Čibej <[hidden email]> wrote:

> On 30.04.2015 00:48, Konstantin Boudnik wrote:
> > These simple rules allow to produce a much cleaner git history, which is
> > easier to work with, bisect if needed, and in general do any sorts of
> tracing.
>
> I have to point out that the goal of version control is not to have a
> "clean git history" but to have an audit trail of what went into the
> code, when and by whom. Rebase destroys that information. While I can
> marginally understand people wanting to look nice in public and rebasing
> stuff in their local repository clones, rebasing anything that's already
> been pushed upstream is a recipe for disaster (and repository corruption).
>
> We quite recently had a case where we could not trace the history of
> jdk8 backports and therefore could not decide whether they were public
> domain or GPL. Really, that's a horrible way to do version control.
>
> Maybe instead of trying to fudge the history to make it "cleaner", you
> could consider a different workflow that doesn't involve creating a
> branch fore every line of code you write. That's a bit like sayin, Oh, I
> have a hammer, let's go find as many nails as I can, instead of just
> hammering in those nails you actually need. And use a wrench for the
> nuts, please, a hammer just ruins them.
>

Brane, the only reason a branch is created for every ticket is CI. We need
to make sure that all tests pass before in developer's branch on CI before
the final merge. In current scheme, CI build runs every time there is a
push to any branch. Such scheme allows us to make sure that sprint branches
are always green and all the tests pass.

Do you have another scheme in mind? Happy to discuss.

>
> -- Brane
>
> P.S.: Before someone accuses me of an anti-git bias, let me just mention
> that I've been studying version control workflows for 20 years and
> writing a version control system for 15 ... this isn't about git, it's
> about rational development process.
>

Konstantin Boudnik-2

Re: Development process

In reply to this post by yzhdanov

As a part of next release (1.2?), may I offer an idea to drop md5 sums? It's
pretty weak stuff and we've been using it mainly because a number of ASF
projects keep doing this. But it just occurred to me that md5 doesn't serve
any purpose really. Even sha1 is questionable. Perhaps just sha512 and gpg
signature?

Thoughts?
Cos

On Wed, Apr 29, 2015 at 12:03PM, Yakov Zhdanov wrote:

> Igniters,
>
> We have got pretty cool changes in current ignite-sprint-4 branch (already
> merged or are ready to be merged):
> * Added Google Compute Engine TCP discovery IP finder.
> * Added generic cloud TCP discovery IP finder (based on jcoulds).
> * Added SortedEvictionPolicy.
> * Added chaining for IgniteConfiguration and CacheConfiguration setters.
> * Improved expiry policy handling.
> * Fixed job continuations.
> * Fixed compilation and runtime with OpenJDK 7 & 8
> * Many stability and fault-tolerance fixes.
>
> Let's try releasing it as apache-ignite-1.1!
>
> Please continue new functionality development in ignite-sprint-5 (whoever
> has anything to commit to will create this branch).
>
> Make sure that no new functionality ignite-sprint-4.
>
> Release instructions can be found in DEVNOTES.txt - Ignite Release
> Instructions section. in git -
> https://git-wip-us.apache.org/repos/asf?p=incubator-ignite.git;a=blob_plain;f=DEVNOTES.txt;hb=refs/heads/ignite-sprint-4
>
> --Yakov

Branko Čibej

Re: Development process

In reply to this post by Anton Vinogradov

On 30.04.2015 18:04, Anton Vinogradov wrote:

> Brane, thanks for review
>
> Release artifact name template is specified at main pom.xml:
>
> <ignite.zip.pattern>ignite-${ignite.edition}-${project.version}-incubating</ignite.zip.pattern>
>
> This property value can be changed, for example by command line parameter.
> I've added this to instruction.
>
> Source package can be created by executing the following:
>
> mvn package -P apache-release
>
> Source package name is ignite-${project.version}-source-release.zip. This
> template specified at parent apache:16 pom.xml.
>
> Release procedure will be changed at sprint-5 according to your comments.
>
> Please check changes at sprint-5 DEVNOTES.txt.
> https://git-wip-us.apache.org/repos/asf?p=incubator-ignite.git;a=blob_plain;f=DEVNOTES.txt;hb=refs/heads/ignite-sprint-5

Looks good.

I'd add a note about the created tag name and mention that the vote mail
should include the tag name, so that people can easily compare the
sources with the repository.

-- Brane

> On Thu, Apr 30, 2015 at 1:03 AM, Branko Čibej <[hidden email]> wrote:
>
>> On 29.04.2015 18:10, Dmitriy Setrakyan wrote:
>>> I think one of the biggest improvements in this release is fully
>> automated
>>> build process. As Yakov suggested, the instructions to run the build are
>> in
>>> the DEVNOTES.txt in sprint-4 branch:
>>>
>>> http://s.apache.org/caf
>>>
>>> Brane, Cos, please take a look and let us know if you have any comments.
>>
>> Ah, thanks for reminding me. I do have a few comments.
>>
>>
>> Fist of all, the release instructions don't say anything about release
>> artefact naming. I got the impression that the package names were more
>> or less an accident; certainly they've been less than consistent in the
>> last couple release attempts! It's also not clear from the instructions
>> how to correctly create the source package.
>>
>> It's important that the source package creation and naming is consistent
>> and not an accident of how somebody happens to have their work area set up.
>>
>>
>>> Deploy Ignite release to maven repository and site:
>>> ...
>>> Start vote based on people.apache.org/~<username>/ignite-version.
>> That's the wrong way around. Vote first, then release. If there's a
>> /staging/ area in the maven repo, then that's different (and equivalent
>> to the staging area on dist.apache.org), so please make that clear in
>> the release docs. It's not clear to me, although it's sort of implied by:
>>
>>> Release nexus staging when version accepted.
>> but ... how do you do that? Document it; it's not obvious that everyone
>> should be completely familiar with Maven repository management.
>>
>>> Configure access to people.apache.org/~<username>. It necessary to
>> upload sources to site.
>>
>> I think I mentioned this before but I'd /really/ recommend that you use
>> the dist repository for this, not some random home directory, even if it
>> is on people.apache.org.
>>
>> Currently, you put final releases in
>>
>> https://dist.apache.org/repos/dist/release/incubator/ignite
>>
>> which is per spec. I strongly recommend that you put source packages
>> that are ready for release voting into
>>
>> https://dist.apache.org/repos/dist/dev/incubator/ignite
>>
>> because then, once a release has been approved, a single remote "svn
>> copy" operation will move the package to the dist/release area, leaving
>> no room for error in this step. It also makes the release process
>> independent of any single person's login on people.apache.org.
>>
>>
>> Oh, by the way, the release instructions say nothing about putting the
>> source release on dist/release ...
>>
>>
>>> You might need to allow connection to people.apache.org. Just do
>> following at console:
>>> ssh people.apache.org
>>>
>>> and type "yes" + enter.
>> Do not ever again recommend bad security practices in your docs! You
>> must *not* "type yes + enter", you must double-check the server key
>> fingerprint before trusting the server.
>>
>> -- Brane
>>
>>

Konstantin Boudnik-2

Re: Git practices [Was: Development process]

In reply to this post by Konstantin Boudnik-2

Another bit that confuses me a lot is the fact that there's no "master" branch
in this project. It is very hard for a layman to find where to start and what
to use. Anyone with even a rudimentary git experience is normally to start
looking at the master branch: try to build it, run tests on it, etc.

I got a private chat from a new user, who's looking to start contributing to
the project but couldn't even get past running tests on master. And indeed,
the master is two month behind.

Another problem this model possesses is that there's no common point to
branch-off release branches when the time comes. Hence, making maintenance
releases would be quite a bit of hassle. It might be less problematic now as
we go in the straight line, but later it could be a necessity to supplant
fixes for older release.

Once again, I'd like to propose to consider this git branching model
http://nvie.com/posts/a-successful-git-branching-model/
which proved to be very clean and development friendly in multiple projects
I've used it at. I am happy to have a video call with anyone who wants to
discuss it p2p any further.

Thoughts?
Cos

On Wed, Apr 29, 2015 at 03:48PM, Konstantin Boudnik wrote:

> Guys,
>
> I don't want to hijack the original thread, so I am forking it ;)
>
> I've been watching the git tree for the sprint branches and I have one general
> comment, which I always pushing for when it gets to correct use of git. Here
> it is:
> - during the development on a branch one has to re-sync with the main
> integration branch (ie ignite-sprint-X)
> - git has two options of doing this: merge and rebase
> - merge preserves the history of the, well, branch merges - potentially
> producing a complex intervened picture of branches like
>
> │ │ ● │ minor
> │ │ ●━┑ Merge remote-tracking branch 'origin/ignite-sprint-4' into ignite-sprint-4
> │ │ │ ●━┑ merge from ignite-790 to ignite-sprint-4
> │ │ │ │ ● ignite-790: improvements after the review
> │ │ │ │ ● review
> │ │ │ │ ●━┑ Merge branches 'ignite-790' and 'ignite-sprint-4' of https://git-wip-us.apache.org/repo~
> │ │ │ │ ● │ ignite-790: several times performance improvements
> │ │ │ │ ● │ ignite-790: added example configs for clouds
> │ │ │ │ ● │ ignite-790: default configs
> │ │ │ │ ● │ ignite-790: fixes, improvements, tests
> │ │ │ │ ● │ ignite-709: read private key for GCE from the file
> │ │ │ │ ● │ temporal commit, forgot to switch to the branch
> │ │ │ ●━┑ │ Merge branch 'ignite-734' into ignite-sprint-4
> │ │ │ │ ● │ review
> │ │ │ │ ●━┪ Merge branches 'ignite-734' and 'ignite-sprint-4' of https://git-wip-us.apache.org/repo~
> │ │ │ │ ● │ ignite-734: finished implementation, provided tests
> │ │ │ │ ● │ ignite-734: google cloud storage ipfinder is fully implemented
> │ │ │ │ ● │ ignite-734: implemented ip finder
> │ │ │ │ ● │ ignite-734: start implementing google ip finder
> │ │ ● │ │ │ fixed NPE
> │ ● │ │ │ │ javadoc fix
> ●━┑ │ │ │ │ Merge branches 'ignite-839' and 'ignite-sprint-4' of https://git-wip-us.apache.org/repo~
>
> - rebase rewrites the new commits on top of a specified HEAD, hence producing
> a flat line of development
> - merge makes sense for feature branches integration, automatic branch
> management by CI, etc.
> - rebase makes sense when you're working on your feature for some time and
> don't want to show the whole world how many times you were synching with
> the integration branch. Keep in mind, that rebase rewrites the commits
> hence changing their sha1 hashes. Be considerate about this and never
> rebase something that has been already pushed to the remote repo
> - In the example above perhaps no one would either care for the fact that
> there was 5 consequent commits for ignite-790. This particular situation
> could be fixes with 'squash' (a form of interactive rebase), but I will
> leave this exercise for the readers.
>
> These simple rules allow to produce a much cleaner git history, which is
> easier to work with, bisect if needed, and in general do any sorts of tracing.
>
> Please let me know if you have any questions.
> Cos
>
> On Wed, Apr 29, 2015 at 12:03PM, Yakov Zhdanov wrote:
> > Igniters,
> >
> > We have got pretty cool changes in current ignite-sprint-4 branch (already
> > merged or are ready to be merged):
> > * Added Google Compute Engine TCP discovery IP finder.
> > * Added generic cloud TCP discovery IP finder (based on jcoulds).
> > * Added SortedEvictionPolicy.
> > * Added chaining for IgniteConfiguration and CacheConfiguration setters.
> > * Improved expiry policy handling.
> > * Fixed job continuations.
> > * Fixed compilation and runtime with OpenJDK 7 & 8
> > * Many stability and fault-tolerance fixes.
> >
> > Let's try releasing it as apache-ignite-1.1!
> >
> > Please continue new functionality development in ignite-sprint-5 (whoever
> > has anything to commit to will create this branch).
> >
> > Make sure that no new functionality ignite-sprint-4.
> >
> > Release instructions can be found in DEVNOTES.txt - Ignite Release
> > Instructions section. in git -
> > https://git-wip-us.apache.org/repos/asf?p=incubator-ignite.git;a=blob_plain;f=DEVNOTES.txt;hb=refs/heads/ignite-sprint-4
> >
> > --Yakov

dsetrakyan

Re: Git practices [Was: Development process]

On Thu, May 7, 2015 at 2:10 PM, Konstantin Boudnik <[hidden email]> wrote:

> Another bit that confuses me a lot is the fact that there's no "master"
> branch
> in this project. It is very hard for a layman to find where to start and
> what
> to use. Anyone with even a rudimentary git experience is normally to start
> looking at the master branch: try to build it, run tests on it, etc.
>
> I got a private chat from a new user, who's looking to start contributing
> to
> the project but couldn't even get past running tests on master. And indeed,
> the master is two month behind.
>
> Another problem this model possesses is that there's no common point to
> branch-off release branches when the time comes. Hence, making maintenance
> releases would be quite a bit of hassle. It might be less problematic now
> as
> we go in the straight line, but later it could be a necessity to supplant
> fixes for older release.
>
> Once again, I'd like to propose to consider this git branching model
> http://nvie.com/posts/a-successful-git-branching-model/
> which proved to be very clean and development friendly in multiple projects
> I've used it at. I am happy to have a video call with anyone who wants to
> discuss it p2p any further.
>

I am looking at the model you are suggesting and it is pretty close to what
we do right now. The master should be as old as 1.0, which is normal since
that was our last release. When we release 1.1.0, we will merge it back
again. I think this is consistent with the diagram you are suggesting, no?

We could have another policy, and merge back to master with every sprint. I
think it sounds like you would prefer that better, right?

>
> Thoughts?
> Cos
>
> On Wed, Apr 29, 2015 at 03:48PM, Konstantin Boudnik wrote:
> > Guys,
> >
> > I don't want to hijack the original thread, so I am forking it ;)
> >
> > I've been watching the git tree for the sprint branches and I have one
> general
> > comment, which I always pushing for when it gets to correct use of git.
> Here
> > it is:
> > - during the development on a branch one has to re-sync with the main
> > integration branch (ie ignite-sprint-X)
> > - git has two options of doing this: merge and rebase
> > - merge preserves the history of the, well, branch merges - potentially
> > producing a complex intervened picture of branches like
> >
> > │ │ ● │ minor
> > │ │ ●━┑ Merge remote-tracking branch 'origin/ignite-sprint-4' into
> ignite-sprint-4
> > │ │ │ ●━┑ merge from ignite-790 to ignite-sprint-4
> > │ │ │ │ ● ignite-790: improvements after the review
> > │ │ │ │ ● review
> > │ │ │ │ ●━┑ Merge branches 'ignite-790' and 'ignite-sprint-4' of
> https://git-wip-us.apache.org/repo~
> > │ │ │ │ ● │ ignite-790: several times performance improvements
> > │ │ │ │ ● │ ignite-790: added example configs for clouds
> > │ │ │ │ ● │ ignite-790: default configs
> > │ │ │ │ ● │ ignite-790: fixes, improvements, tests
> > │ │ │ │ ● │ ignite-709: read private key for GCE from the file
> > │ │ │ │ ● │ temporal commit, forgot to switch to the branch
> > │ │ │ ●━┑ │ Merge branch 'ignite-734' into ignite-sprint-4
> > │ │ │ │ ● │ review
> > │ │ │ │ ●━┪ Merge branches 'ignite-734' and 'ignite-sprint-4' of
> https://git-wip-us.apache.org/repo~
> > │ │ │ │ ● │ ignite-734: finished implementation, provided tests
> > │ │ │ │ ● │ ignite-734: google cloud storage ipfinder is fully
> implemented
> > │ │ │ │ ● │ ignite-734: implemented ip finder
> > │ │ │ │ ● │ ignite-734: start implementing google ip finder
> > │ │ ● │ │ │ fixed NPE
> > │ ● │ │ │ │ javadoc fix
> > ●━┑ │ │ │ │ Merge branches 'ignite-839' and 'ignite-sprint-4' of
> https://git-wip-us.apache.org/repo~
> >
> > - rebase rewrites the new commits on top of a specified HEAD, hence
> producing
> > a flat line of development
> > - merge makes sense for feature branches integration, automatic branch
> > management by CI, etc.
> > - rebase makes sense when you're working on your feature for some time
> and
> > don't want to show the whole world how many times you were synching
> with
> > the integration branch. Keep in mind, that rebase rewrites the commits
> > hence changing their sha1 hashes. Be considerate about this and never
> > rebase something that has been already pushed to the remote repo
> > - In the example above perhaps no one would either care for the fact
> that
> > there was 5 consequent commits for ignite-790. This particular
> situation
> > could be fixes with 'squash' (a form of interactive rebase), but I
> will
> > leave this exercise for the readers.
> >
> > These simple rules allow to produce a much cleaner git history, which is
> > easier to work with, bisect if needed, and in general do any sorts of
> tracing.
> >
> > Please let me know if you have any questions.
> > Cos
> >
> > On Wed, Apr 29, 2015 at 12:03PM, Yakov Zhdanov wrote:
> > > Igniters,
> > >
> > > We have got pretty cool changes in current ignite-sprint-4 branch
> (already
> > > merged or are ready to be merged):
> > > * Added Google Compute Engine TCP discovery IP finder.
> > > * Added generic cloud TCP discovery IP finder (based on jcoulds).
> > > * Added SortedEvictionPolicy.
> > > * Added chaining for IgniteConfiguration and CacheConfiguration
> setters.
> > > * Improved expiry policy handling.
> > > * Fixed job continuations.
> > > * Fixed compilation and runtime with OpenJDK 7 & 8
> > > * Many stability and fault-tolerance fixes.
> > >
> > > Let's try releasing it as apache-ignite-1.1!
> > >
> > > Please continue new functionality development in ignite-sprint-5
> (whoever
> > > has anything to commit to will create this branch).
> > >
> > > Make sure that no new functionality ignite-sprint-4.
> > >
> > > Release instructions can be found in DEVNOTES.txt - Ignite Release
> > > Instructions section. in git -
> > >
> https://git-wip-us.apache.org/repos/asf?p=incubator-ignite.git;a=blob_plain;f=DEVNOTES.txt;hb=refs/heads/ignite-sprint-4
> > >
> > > --Yakov
>

Konstantin Boudnik-2

Re: Git practices [Was: Development process]

On Thu, May 07, 2015 at 02:53PM, Dmitriy Setrakyan wrote:

> On Thu, May 7, 2015 at 2:10 PM, Konstantin Boudnik <[hidden email]> wrote:
>
> > Another bit that confuses me a lot is the fact that there's no "master"
> > branch
> > in this project. It is very hard for a layman to find where to start and
> > what
> > to use. Anyone with even a rudimentary git experience is normally to start
> > looking at the master branch: try to build it, run tests on it, etc.
> >
> > I got a private chat from a new user, who's looking to start contributing
> > to
> > the project but couldn't even get past running tests on master. And indeed,
> > the master is two month behind.
> >
> > Another problem this model possesses is that there's no common point to
> > branch-off release branches when the time comes. Hence, making maintenance
> > releases would be quite a bit of hassle. It might be less problematic now
> > as
> > we go in the straight line, but later it could be a necessity to supplant
> > fixes for older release.
> >
> > Once again, I'd like to propose to consider this git branching model
> > http://nvie.com/posts/a-successful-git-branching-model/
> > which proved to be very clean and development friendly in multiple projects
> > I've used it at. I am happy to have a video call with anyone who wants to
> > discuss it p2p any further.
> >
>
> I am looking at the model you are suggesting and it is pretty close to what
> we do right now. The master should be as old as 1.0, which is normal since
> that was our last release. When we release 1.1.0, we will merge it back
> again. I think this is consistent with the diagram you are suggesting, no?

Yup, it is. I just was curious about the state of the master as the tests were
hanging on it. I will check back with the fella to make sure it wasn't a
particular computer environment issue.

> We could have another policy, and merge back to master with every sprint. I
> think it sounds like you would prefer that better, right?

Doesn't really matter what I prefer as far as it is consistent and documented.
Either way has its own merits: the dev@ community needs to be comfortable with
it and new-comers have to have a way to learn it easily.

Thanks for the clarification!
Cos

> > Thoughts?
> > Cos
> >
> > On Wed, Apr 29, 2015 at 03:48PM, Konstantin Boudnik wrote:
> > > Guys,
> > >
> > > I don't want to hijack the original thread, so I am forking it ;)
> > >
> > > I've been watching the git tree for the sprint branches and I have one
> > general
> > > comment, which I always pushing for when it gets to correct use of git.
> > Here
> > > it is:
> > > - during the development on a branch one has to re-sync with the main
> > > integration branch (ie ignite-sprint-X)
> > > - git has two options of doing this: merge and rebase
> > > - merge preserves the history of the, well, branch merges - potentially
> > > producing a complex intervened picture of branches like
> > >
> > > │ │ ● │ minor
> > > │ │ ●━┑ Merge remote-tracking branch 'origin/ignite-sprint-4' into
> > ignite-sprint-4
> > > │ │ │ ●━┑ merge from ignite-790 to ignite-sprint-4
> > > │ │ │ │ ● ignite-790: improvements after the review
> > > │ │ │ │ ● review
> > > │ │ │ │ ●━┑ Merge branches 'ignite-790' and 'ignite-sprint-4' of
> > https://git-wip-us.apache.org/repo~
> > > │ │ │ │ ● │ ignite-790: several times performance improvements
> > > │ │ │ │ ● │ ignite-790: added example configs for clouds
> > > │ │ │ │ ● │ ignite-790: default configs
> > > │ │ │ │ ● │ ignite-790: fixes, improvements, tests
> > > │ │ │ │ ● │ ignite-709: read private key for GCE from the file
> > > │ │ │ │ ● │ temporal commit, forgot to switch to the branch
> > > │ │ │ ●━┑ │ Merge branch 'ignite-734' into ignite-sprint-4
> > > │ │ │ │ ● │ review
> > > │ │ │ │ ●━┪ Merge branches 'ignite-734' and 'ignite-sprint-4' of
> > https://git-wip-us.apache.org/repo~
> > > │ │ │ │ ● │ ignite-734: finished implementation, provided tests
> > > │ │ │ │ ● │ ignite-734: google cloud storage ipfinder is fully
> > implemented
> > > │ │ │ │ ● │ ignite-734: implemented ip finder
> > > │ │ │ │ ● │ ignite-734: start implementing google ip finder
> > > │ │ ● │ │ │ fixed NPE
> > > │ ● │ │ │ │ javadoc fix
> > > ●━┑ │ │ │ │ Merge branches 'ignite-839' and 'ignite-sprint-4' of
> > https://git-wip-us.apache.org/repo~
> > >
> > > - rebase rewrites the new commits on top of a specified HEAD, hence
> > producing
> > > a flat line of development
> > > - merge makes sense for feature branches integration, automatic branch
> > > management by CI, etc.
> > > - rebase makes sense when you're working on your feature for some time
> > and
> > > don't want to show the whole world how many times you were synching
> > with
> > > the integration branch. Keep in mind, that rebase rewrites the commits
> > > hence changing their sha1 hashes. Be considerate about this and never
> > > rebase something that has been already pushed to the remote repo
> > > - In the example above perhaps no one would either care for the fact
> > that
> > > there was 5 consequent commits for ignite-790. This particular
> > situation
> > > could be fixes with 'squash' (a form of interactive rebase), but I
> > will
> > > leave this exercise for the readers.
> > >
> > > These simple rules allow to produce a much cleaner git history, which is
> > > easier to work with, bisect if needed, and in general do any sorts of
> > tracing.
> > >
> > > Please let me know if you have any questions.
> > > Cos
> > >
> > > On Wed, Apr 29, 2015 at 12:03PM, Yakov Zhdanov wrote:
> > > > Igniters,
> > > >
> > > > We have got pretty cool changes in current ignite-sprint-4 branch
> > (already
> > > > merged or are ready to be merged):
> > > > * Added Google Compute Engine TCP discovery IP finder.
> > > > * Added generic cloud TCP discovery IP finder (based on jcoulds).
> > > > * Added SortedEvictionPolicy.
> > > > * Added chaining for IgniteConfiguration and CacheConfiguration
> > setters.
> > > > * Improved expiry policy handling.
> > > > * Fixed job continuations.
> > > > * Fixed compilation and runtime with OpenJDK 7 & 8
> > > > * Many stability and fault-tolerance fixes.
> > > >
> > > > Let's try releasing it as apache-ignite-1.1!
> > > >
> > > > Please continue new functionality development in ignite-sprint-5
> > (whoever
> > > > has anything to commit to will create this branch).
> > > >
> > > > Make sure that no new functionality ignite-sprint-4.
> > > >
> > > > Release instructions can be found in DEVNOTES.txt - Ignite Release
> > > > Instructions section. in git -
> > > >
> > https://git-wip-us.apache.org/repos/asf?p=incubator-ignite.git;a=blob_plain;f=DEVNOTES.txt;hb=refs/heads/ignite-sprint-4
> > > >
> > > > --Yakov
> >