Change TC trigger from Run All to per-project basis

[Pavel Tupitsyn]

Igniters,

Currently we have a TeamCity trigger only on Run All aggregating project:
https://ci.ignite.apache.org/admin/editTriggers.html?id=buildType:IgniteTests24Java8_RunAll

Each change triggers it's own full run. If new changes are added while some
builds are queued,
these changes are not picked up, and a separate run is scheduled.

At this very moment I see over 600 queued builds, all on master branch from
some trigger.
Previously we had a trigger per-project, which seems to be more efficient.

Thoughts?

Pavel

[Dmitriy Pavlov]

Hi Pavel,

About queue: all these builds were manually triggered by me. It is like
experiment under MakeTeamCityGreenAgain. I am trying to accumulate
statistics about failing tests in master. Feel free to reoder your build to
top.

Statistic is necessary to find out how much tests are actually failing. For
example in run 1: failed test A in run 2 test A and D, in run 3 B, C
As result we actually have A, B, C, D tests failing, but one chain results
shows us it is only 1 or 2. I'll share results later.

Per project triggers were actually located in template. It was hard to
manage.
Now Run Basic test chain has VCS trigger (basic&licenses&compute), other
builds inlcuded into Run All. Run All does not have VCS triggers and runs
every nigth. No one checks results more often then daily.

If somethig you may suggest to be included into Run Basic sub-set please
share. It should be fast & stable suite without flaky tests because
RunBasic now has email notifications.

Sincerely,
Dmitriy Pavlov



пн, 19 февр. 2018 г. в 0:39, Pavel Tupitsyn <[hidden email]>:

> Igniters,
>
> Currently we have a TeamCity trigger only on Run All aggregating project:
>
> https://ci.ignite.apache.org/admin/editTriggers.html?id=buildType:IgniteTests24Java8_RunAll
>
> Each change triggers it's own full run. If new changes are added while some
> builds are queued,
> these changes are not picked up, and a separate run is scheduled.
>
> At this very moment I see over 600 queued builds, all on master branch from
> some trigger.
> Previously we had a trigger per-project, which seems to be more efficient.
>
> Thoughts?
>
> Pavel
>

[Dmitriy Pavlov]

Hi Pavel,

WDYT,
1) is it needed to prepare wiki article describing tests and chains we
have, e.g. describe
https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_RunBasicTests
and
its difference with RunAll; which chain to use in each case?

2) Which .Net test suite may be applicable to be included into 'run basic
tests' (requires <30 min to run, no failed test, no flaky test with fail
rate >1%) ?

Sincerely,
Dmitriy Pavlov

пн, 19 февр. 2018 г. в 0:39, Pavel Tupitsyn <[hidden email]>:

> Igniters,
>
> Currently we have a TeamCity trigger only on Run All aggregating project:
>
> https://ci.ignite.apache.org/admin/editTriggers.html?id=buildType:IgniteTests24Java8_RunAll
>
> Each change triggers it's own full run. If new changes are added while some
> builds are queued,
> these changes are not picked up, and a separate run is scheduled.
>
> At this very moment I see over 600 queued builds, all on master branch from
> some trigger.
> Previously we had a trigger per-project, which seems to be more efficient.
>
> Thoughts?
>
> Pavel
>

[Pavel Tupitsyn]

Dmitry, thanks for the explanation, this makes sense.

1) Certainly can be helpful.
2) I would include "Platform .NET Core Linux": does not require win agent,
runs 15 min, covers most functionality.

Pavel

On Wed, Feb 21, 2018 at 12:12 AM, Dmitry Pavlov <[hidden email]>
wrote:

> Hi Pavel,
>
> WDYT,
> 1) is it needed to prepare wiki article describing tests and chains we
> have, e.g. describe
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_
> RunBasicTests
> and
> its difference with RunAll; which chain to use in each case?
>
> 2) Which .Net test suite may be applicable to be included into 'run basic
> tests' (requires <30 min to run, no failed test, no flaky test with fail
> rate >1%) ?
>
> Sincerely,
> Dmitriy Pavlov
>
> пн, 19 февр. 2018 г. в 0:39, Pavel Tupitsyn <[hidden email]>:
>
> > Igniters,
> >
> > Currently we have a TeamCity trigger only on Run All aggregating project:
> >
> > https://ci.ignite.apache.org/admin/editTriggers.html?id=
> buildType:IgniteTests24Java8_RunAll
> >
> > Each change triggers it's own full run. If new changes are added while
> some
> > builds are queued,
> > these changes are not picked up, and a separate run is scheduled.
> >
> > At this very moment I see over 600 queued builds, all on master branch
> from
> > some trigger.
> > Previously we had a trigger per-project, which seems to be more
> efficient.
> >
> > Thoughts?
> >
> > Pavel
> >
>

[Dmitriy Pavlov]

Hi Pavel,

Tests are passing now.

I've inluded
https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_IgnitePlatformNetCoreLinux&branch_IgniteTests24Java8=%3Cdefault%3E&tab=buildTypeStatusDiv
to
Build Basic chain to run suite on per commit basis.

Let's see if it tests would be stable

Sincerely,
Dmitriy Pavlov

ср, 21 февр. 2018 г. в 10:26, Pavel Tupitsyn <[hidden email]>:

> Dmitry, thanks for the explanation, this makes sense.
>
> 1) Certainly can be helpful.
> 2) I would include "Platform .NET Core Linux": does not require win agent,
> runs 15 min, covers most functionality.
>
> Pavel
>
> On Wed, Feb 21, 2018 at 12:12 AM, Dmitry Pavlov <[hidden email]>
> wrote:
>
> > Hi Pavel,
> >
> > WDYT,
> > 1) is it needed to prepare wiki article describing tests and chains we
> > have, e.g. describe
> >
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_
> > RunBasicTests
> > and
> > its difference with RunAll; which chain to use in each case?
> >
> > 2) Which .Net test suite may be applicable to be included into 'run basic
> > tests' (requires <30 min to run, no failed test, no flaky test with fail
> > rate >1%) ?
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > пн, 19 февр. 2018 г. в 0:39, Pavel Tupitsyn <[hidden email]>:
> >
> > > Igniters,
> > >
> > > Currently we have a TeamCity trigger only on Run All aggregating
> project:
> > >
> > > https://ci.ignite.apache.org/admin/editTriggers.html?id=
> > buildType:IgniteTests24Java8_RunAll
> > >
> > > Each change triggers it's own full run. If new changes are added while
> > some
> > > builds are queued,
> > > these changes are not picked up, and a separate run is scheduled.
> > >
> > > At this very moment I see over 600 queued builds, all on master branch
> > from
> > > some trigger.
> > > Previously we had a trigger per-project, which seems to be more
> > efficient.
> > >
> > > Thoughts?
> > >
> > > Pavel
> > >
> >
>

[Dmitriy Pavlov]

Hi Pavel,

regarding point 1, I've created draft here
https://cwiki.apache.org/confluence/display/IGNITE/Continuous+Integration

I'll create new topic once it will be ready.

SIncerely,
Dmitriy Pavlov

ср, 21 февр. 2018 г. в 10:26, Pavel Tupitsyn <[hidden email]>:

> Dmitry, thanks for the explanation, this makes sense.
>
> 1) Certainly can be helpful.
> 2) I would include "Platform .NET Core Linux": does not require win agent,
> runs 15 min, covers most functionality.
>
> Pavel
>
> On Wed, Feb 21, 2018 at 12:12 AM, Dmitry Pavlov <[hidden email]>
> wrote:
>
> > Hi Pavel,
> >
> > WDYT,
> > 1) is it needed to prepare wiki article describing tests and chains we
> > have, e.g. describe
> >
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_
> > RunBasicTests
> > and
> > its difference with RunAll; which chain to use in each case?
> >
> > 2) Which .Net test suite may be applicable to be included into 'run basic
> > tests' (requires <30 min to run, no failed test, no flaky test with fail
> > rate >1%) ?
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > пн, 19 февр. 2018 г. в 0:39, Pavel Tupitsyn <[hidden email]>:
> >
> > > Igniters,
> > >
> > > Currently we have a TeamCity trigger only on Run All aggregating
> project:
> > >
> > > https://ci.ignite.apache.org/admin/editTriggers.html?id=
> > buildType:IgniteTests24Java8_RunAll
> > >
> > > Each change triggers it's own full run. If new changes are added while
> > some
> > > builds are queued,
> > > these changes are not picked up, and a separate run is scheduled.
> > >
> > > At this very moment I see over 600 queued builds, all on master branch
> > from
> > > some trigger.
> > > Previously we had a trigger per-project, which seems to be more
> > efficient.
> > >
> > > Thoughts?
> > >
> > > Pavel
> > >
> >
>

[Pavel Tupitsyn]

Dmitry, awesome start!

Just a note, not every TC project is about JUnit ;)

Thanks,
Pavel

On Fri, Feb 23, 2018 at 12:49 AM, Dmitry Pavlov <[hidden email]>
wrote:

> Hi Pavel,
>
> regarding point 1, I've created draft here
> https://cwiki.apache.org/confluence/display/IGNITE/Continuous+Integration
>
> I'll create new topic once it will be ready.
>
> SIncerely,
> Dmitriy Pavlov
>
> ср, 21 февр. 2018 г. в 10:26, Pavel Tupitsyn <[hidden email]>:
>
> > Dmitry, thanks for the explanation, this makes sense.
> >
> > 1) Certainly can be helpful.
> > 2) I would include "Platform .NET Core Linux": does not require win
> agent,
> > runs 15 min, covers most functionality.
> >
> > Pavel
> >
> > On Wed, Feb 21, 2018 at 12:12 AM, Dmitry Pavlov <[hidden email]>
> > wrote:
> >
> > > Hi Pavel,
> > >
> > > WDYT,
> > > 1) is it needed to prepare wiki article describing tests and chains we
> > > have, e.g. describe
> > >
> > https://ci.ignite.apache.org/viewType.html?buildTypeId=
> IgniteTests24Java8_
> > > RunBasicTests
> > > and
> > > its difference with RunAll; which chain to use in each case?
> > >
> > > 2) Which .Net test suite may be applicable to be included into 'run
> basic
> > > tests' (requires <30 min to run, no failed test, no flaky test with
> fail
> > > rate >1%) ?
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > пн, 19 февр. 2018 г. в 0:39, Pavel Tupitsyn <[hidden email]>:
> > >
> > > > Igniters,
> > > >
> > > > Currently we have a TeamCity trigger only on Run All aggregating
> > project:
> > > >
> > > > https://ci.ignite.apache.org/admin/editTriggers.html?id=
> > > buildType:IgniteTests24Java8_RunAll
> > > >
> > > > Each change triggers it's own full run. If new changes are added
> while
> > > some
> > > > builds are queued,
> > > > these changes are not picked up, and a separate run is scheduled.
> > > >
> > > > At this very moment I see over 600 queued builds, all on master
> branch
> > > from
> > > > some trigger.
> > > > Previously we had a trigger per-project, which seems to be more
> > > efficient.
> > > >
> > > > Thoughts?
> > > >
> > > > Pavel
> > > >
> > >
> >
>