New TC builds adding process

[Ivan Pavlukhin]

Hi Igniters,

On Friday I felt a real inconvenience (pain in ass) with running builds for
a ticked I worked on [1]. There were two the most frustrating moments:
1. Estimated build completion time was 11 hours after the start.
2. There are newly added suites which are failing constantly in recent runs.

I compared builds count in RunAll chain for mentioned ticked. On Oct 6
there were 105 builds, and on Oct 19 where were 113. Then I compared
differences and found suspicious items.

First of all there were 2 Windows suites PlatformNetCoverage [2] and
PlatformCWindowsX86 [3]. Former seems to be something added recently and
latter as I know was decided to be excluded from RunAll some time ago but
strangely reappeared. I suspect that these 2 suites could increase build
run time because they are quite lengthy and require Windows slaves which
are in limited amount.

Also there were 2 rather problematic builds InspectionsAop [4] and
InspectionsCore [5] which seems to fail constantly. Presence of such builds
brings a noise into analyzing build results. TC Bot treats them as possible
blockers.

So, here are my suggestions:
1. A contributor should add new build to RunAll after estimating build
running time impact carefully. If impact is noticeable such builds should
be announced and discussed on dev list. It sounds good idea to test drive
new heavy build running it on a scheduled basis first before adding to
RunAll.
2. We should not have constantly failing builds like mentioned Inspections.

What are your thoughts?

[1] https://issues.apache.org/jira/browse/IGNITE-5935
[2]
https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_PlatformNetCoverage
[3]
https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_PlatformCWindowsX86
[4]
https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_InspectionsAop
[5]
https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_InspectionsCore

--
Best regards,
Ivan Pavlukhin

[Dmitriy Pavlov]

Hi Ivan,

I believe every time any suite is added or significantly reconfigured it
should be mentioned in the dev list.

So we don't need to approve any change by discussion, but keeping community
member posted seems to be really helpful here.

Currently, I don't know why and which suites were added.

Sincerely,
Dmitriy Pavlov

сб, 20 окт. 2018 г. в 9:25, Павлухин Иван <[hidden email]>:

> Hi Igniters,
>
> On Friday I felt a real inconvenience (pain in ass) with running builds for
> a ticked I worked on [1]. There were two the most frustrating moments:
> 1. Estimated build completion time was 11 hours after the start.
> 2. There are newly added suites which are failing constantly in recent
> runs.
>
> I compared builds count in RunAll chain for mentioned ticked. On Oct 6
> there were 105 builds, and on Oct 19 where were 113. Then I compared
> differences and found suspicious items.
>
> First of all there were 2 Windows suites PlatformNetCoverage [2] and
> PlatformCWindowsX86 [3]. Former seems to be something added recently and
> latter as I know was decided to be excluded from RunAll some time ago but
> strangely reappeared. I suspect that these 2 suites could increase build
> run time because they are quite lengthy and require Windows slaves which
> are in limited amount.
>
> Also there were 2 rather problematic builds InspectionsAop [4] and
> InspectionsCore [5] which seems to fail constantly. Presence of such builds
> brings a noise into analyzing build results. TC Bot treats them as possible
> blockers.
>
> So, here are my suggestions:
> 1. A contributor should add new build to RunAll after estimating build
> running time impact carefully. If impact is noticeable such builds should
> be announced and discussed on dev list. It sounds good idea to test drive
> new heavy build running it on a scheduled basis first before adding to
> RunAll.
> 2. We should not have constantly failing builds like mentioned Inspections.
>
> What are your thoughts?
>
> [1] https://issues.apache.org/jira/browse/IGNITE-5935
> [2]
>
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_PlatformNetCoverage
> [3]
>
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_PlatformCWindowsX86
> [4]
>
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_InspectionsAop
> [5]
>
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_InspectionsCore
>
> --
> Best regards,
> Ivan Pavlukhin
>

[vveider]

Ivan,


Please check dependencies list now.

> On 21 Oct 2018, at 11:39, Dmitriy Pavlov <[hidden email]> wrote:
>
> Hi Ivan,
>
> I believe every time any suite is added or significantly reconfigured it
> should be mentioned in the dev list.
>
> So we don't need to approve any change by discussion, but keeping community
> member posted seems to be really helpful here.
>
> Currently, I don't know why and which suites were added.
>
> Sincerely,
> Dmitriy Pavlov
>
> сб, 20 окт. 2018 г. в 9:25, Павлухин Иван <[hidden email]>:
>
>> Hi Igniters,
>>
>> On Friday I felt a real inconvenience (pain in ass) with running builds for
>> a ticked I worked on [1]. There were two the most frustrating moments:
>> 1. Estimated build completion time was 11 hours after the start.
>> 2. There are newly added suites which are failing constantly in recent
>> runs.
>>
>> I compared builds count in RunAll chain for mentioned ticked. On Oct 6
>> there were 105 builds, and on Oct 19 where were 113. Then I compared
>> differences and found suspicious items.
>>
>> First of all there were 2 Windows suites PlatformNetCoverage [2] and
>> PlatformCWindowsX86 [3]. Former seems to be something added recently and
>> latter as I know was decided to be excluded from RunAll some time ago but
>> strangely reappeared. I suspect that these 2 suites could increase build
>> run time because they are quite lengthy and require Windows slaves which
>> are in limited amount.
>>
>> Also there were 2 rather problematic builds InspectionsAop [4] and
>> InspectionsCore [5] which seems to fail constantly. Presence of such builds
>> brings a noise into analyzing build results. TC Bot treats them as possible
>> blockers.
>>
>> So, here are my suggestions:
>> 1. A contributor should add new build to RunAll after estimating build
>> running time impact carefully. If impact is noticeable such builds should
>> be announced and discussed on dev list. It sounds good idea to test drive
>> new heavy build running it on a scheduled basis first before adding to
>> RunAll.
>> 2. We should not have constantly failing builds like mentioned Inspections.
>>
>> What are your thoughts?
>>
>> [1] https://issues.apache.org/jira/browse/IGNITE-5935
>> [2]
>>
>> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_PlatformNetCoverage
>> [3]
>>
>> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_PlatformCWindowsX86
>> [4]
>>
>> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_InspectionsAop
>> [5]
>>
>> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_InspectionsCore
>>
>> --
>> Best regards,
>> Ivan Pavlukhin
>>

[Ivan Pavlukhin]

Peter, thanks! Looks good.

Dmitriy, I agree with your proposal. Making community aware sounds the most
important thing, especially for "heavy" builds.
And we can omit any approval to avoid unnecessary slowdown of the
development process.

пн, 22 окт. 2018 г. в 10:41, Petr Ivanov <[hidden email]>:

> Ivan,
>
>
> Please check dependencies list now.
>
> > On 21 Oct 2018, at 11:39, Dmitriy Pavlov <[hidden email]> wrote:
> >
> > Hi Ivan,
> >
> > I believe every time any suite is added or significantly reconfigured it
> > should be mentioned in the dev list.
> >
> > So we don't need to approve any change by discussion, but keeping
> community
> > member posted seems to be really helpful here.
> >
> > Currently, I don't know why and which suites were added.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > сб, 20 окт. 2018 г. в 9:25, Павлухин Иван <[hidden email]>:
> >
> >> Hi Igniters,
> >>
> >> On Friday I felt a real inconvenience (pain in ass) with running builds
> for
> >> a ticked I worked on [1]. There were two the most frustrating moments:
> >> 1. Estimated build completion time was 11 hours after the start.
> >> 2. There are newly added suites which are failing constantly in recent
> >> runs.
> >>
> >> I compared builds count in RunAll chain for mentioned ticked. On Oct 6
> >> there were 105 builds, and on Oct 19 where were 113. Then I compared
> >> differences and found suspicious items.
> >>
> >> First of all there were 2 Windows suites PlatformNetCoverage [2] and
> >> PlatformCWindowsX86 [3]. Former seems to be something added recently and
> >> latter as I know was decided to be excluded from RunAll some time ago
> but
> >> strangely reappeared. I suspect that these 2 suites could increase build
> >> run time because they are quite lengthy and require Windows slaves which
> >> are in limited amount.
> >>
> >> Also there were 2 rather problematic builds InspectionsAop [4] and
> >> InspectionsCore [5] which seems to fail constantly. Presence of such
> builds
> >> brings a noise into analyzing build results. TC Bot treats them as
> possible
> >> blockers.
> >>
> >> So, here are my suggestions:
> >> 1. A contributor should add new build to RunAll after estimating build
> >> running time impact carefully. If impact is noticeable such builds
> should
> >> be announced and discussed on dev list. It sounds good idea to test
> drive
> >> new heavy build running it on a scheduled basis first before adding to
> >> RunAll.
> >> 2. We should not have constantly failing builds like mentioned
> Inspections.
> >>
> >> What are your thoughts?
> >>
> >> [1] https://issues.apache.org/jira/browse/IGNITE-5935
> >> [2]
> >>
> >>
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_PlatformNetCoverage
> >> [3]
> >>
> >>
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_PlatformCWindowsX86
> >> [4]
> >>
> >>
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_InspectionsAop
> >> [5]
> >>
> >>
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_InspectionsCore
> >>
> >> --
> >> Best regards,
> >> Ivan Pavlukhin
> >>
>
>

--
Best regards,
Ivan Pavlukhin