[DISCUSSION] Mirroring Teamcity settings in a GitHub repository

[Dmitry Pavlov]

Hi Igniters,

During preparing of the release I've faced with lack of information related
to required steps. This was one reason of too long release preparation.
This process was automated by several Ignite contributors (thank you, BTW).
Now, these valued automation results are more or less stored using TeamCity
settings at ci.ignite.apache.org.

First of all, changes done in these steps before were not always
consistent. The TC's built-in audit wasn't useful, because there were tons
of changes. So it is not clear who and why changed something.

Secondly, Ignite release is performed outside of ASF infrastructure. It is
not a problem itself (and we're grateful to GridGain to sponsoring
infrastructure). But I believe knowledge about release should be backed up
somehow inside of ASF infra.

The last issue related not to release, but to our test suites. It is now
about 90 suites, which may have different settings. There is no easy and
clear way to grep, check consistency, check possible options we've used.

So let's consider 2 possible options to solve these issues.

Option A: store settings in VCS
https://www.jetbrains.com/help/teamcity/2019.1/storing-project-settings-in-version-control.html


Option B: use DSL
https://www.jetbrains.com/help/teamcity/2019.1/storing-project-settings-in-version-control.html


Please share your vision.

Sincerely,
Dmitriy Pavlov

PS. to see how DSL can look like you can use
https://ci.ignite.apache.org/admin/editBuild.html?id=buildType:Releases_ApacheIgniteMain_ReleaseBuild

and click view DSL.

[Nikolay Izhikov-2]

Hello, Dmitriy.

Thanks, for starting this discussion.
I think almost all of community members don't know the difference between two options.

I vote for the simplest solution with the human readable format.


В Пн, 17/06/2019 в 21:07 +0300, Dmitriy Pavlov пишет:

> Hi Igniters,
>
> During preparing of the release I've faced with lack of information related
> to required steps. This was one reason of too long release preparation.
> This process was automated by several Ignite contributors (thank you, BTW).
> Now, these valued automation results are more or less stored using TeamCity
> settings at ci.ignite.apache.org.
>
> First of all, changes done in these steps before were not always
> consistent. The TC's built-in audit wasn't useful, because there were tons
> of changes. So it is not clear who and why changed something.
>
> Secondly, Ignite release is performed outside of ASF infrastructure. It is
> not a problem itself (and we're grateful to GridGain to sponsoring
> infrastructure). But I believe knowledge about release should be backed up
> somehow inside of ASF infra.
>
> The last issue related not to release, but to our test suites. It is now
> about 90 suites, which may have different settings. There is no easy and
> clear way to grep, check consistency, check possible options we've used.
>
> So let's consider 2 possible options to solve these issues.
>
> Option A: store settings in VCS
> https://www.jetbrains.com/help/teamcity/2019.1/storing-project-settings-in-version-control.html
>
>
> Option B: use DSL
> https://www.jetbrains.com/help/teamcity/2019.1/storing-project-settings-in-version-control.html
>
>
> Please share your vision.
>
> Sincerely,
> Dmitriy Pavlov
>
> PS. to see how DSL can look like you can use
> https://ci.ignite.apache.org/admin/editBuild.html?id=buildType:Releases_ApacheIgniteMain_ReleaseBuild
>
> and click view DSL.

[Ivan Pavlukhin]

Dmitriy,

It is a very important topic for Ignite CI. I think it would be great
if all significant changes in a build configuration goes with a
meaningful description (a commit message). As for me, DSL sounds
promising.

BTW, a special rights are needed to view
https://ci.ignite.apache.org/admin/editBuild.html?id=buildType:Releases_ApacheIgniteMain_ReleaseBuild
(I do not have such).

вт, 18 июн. 2019 г. в 14:30, Nikolay Izhikov <[hidden email]>:

>
> Hello, Dmitriy.
>
> Thanks, for starting this discussion.
> I think almost all of community members don't know the difference between two options.
>
> I vote for the simplest solution with the human readable format.
>
>
> В Пн, 17/06/2019 в 21:07 +0300, Dmitriy Pavlov пишет:
> > Hi Igniters,
> >
> > During preparing of the release I've faced with lack of information related
> > to required steps. This was one reason of too long release preparation.
> > This process was automated by several Ignite contributors (thank you, BTW).
> > Now, these valued automation results are more or less stored using TeamCity
> > settings at ci.ignite.apache.org.
> >
> > First of all, changes done in these steps before were not always
> > consistent. The TC's built-in audit wasn't useful, because there were tons
> > of changes. So it is not clear who and why changed something.
> >
> > Secondly, Ignite release is performed outside of ASF infrastructure. It is
> > not a problem itself (and we're grateful to GridGain to sponsoring
> > infrastructure). But I believe knowledge about release should be backed up
> > somehow inside of ASF infra.
> >
> > The last issue related not to release, but to our test suites. It is now
> > about 90 suites, which may have different settings. There is no easy and
> > clear way to grep, check consistency, check possible options we've used.
> >
> > So let's consider 2 possible options to solve these issues.
> >
> > Option A: store settings in VCS
> > https://www.jetbrains.com/help/teamcity/2019.1/storing-project-settings-in-version-control.html
> >
> >
> > Option B: use DSL
> > https://www.jetbrains.com/help/teamcity/2019.1/storing-project-settings-in-version-control.html
> >
> >
> > Please share your vision.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > PS. to see how DSL can look like you can use
> > https://ci.ignite.apache.org/admin/editBuild.html?id=buildType:Releases_ApacheIgniteMain_ReleaseBuild
> >
> > and click view DSL.

--
Best regards,
Ivan Pavlukhin

[Ivan Pavlukhin]

For curious ones I copied a RunAll DSL configuration to gist [1].

[1] https://gist.github.com/pavlukhin/e788bbebafe69ce9d9a3c9e9871ab8c4

чт, 20 июн. 2019 г. в 08:42, Павлухин Иван <[hidden email]>:

>
> Dmitriy,
>
> It is a very important topic for Ignite CI. I think it would be great
> if all significant changes in a build configuration goes with a
> meaningful description (a commit message). As for me, DSL sounds
> promising.
>
> BTW, a special rights are needed to view
> https://ci.ignite.apache.org/admin/editBuild.html?id=buildType:Releases_ApacheIgniteMain_ReleaseBuild
> (I do not have such).
>
> вт, 18 июн. 2019 г. в 14:30, Nikolay Izhikov <[hidden email]>:
> >
> > Hello, Dmitriy.
> >
> > Thanks, for starting this discussion.
> > I think almost all of community members don't know the difference between two options.
> >
> > I vote for the simplest solution with the human readable format.
> >
> >
> > В Пн, 17/06/2019 в 21:07 +0300, Dmitriy Pavlov пишет:
> > > Hi Igniters,
> > >
> > > During preparing of the release I've faced with lack of information related
> > > to required steps. This was one reason of too long release preparation.
> > > This process was automated by several Ignite contributors (thank you, BTW).
> > > Now, these valued automation results are more or less stored using TeamCity
> > > settings at ci.ignite.apache.org.
> > >
> > > First of all, changes done in these steps before were not always
> > > consistent. The TC's built-in audit wasn't useful, because there were tons
> > > of changes. So it is not clear who and why changed something.
> > >
> > > Secondly, Ignite release is performed outside of ASF infrastructure. It is
> > > not a problem itself (and we're grateful to GridGain to sponsoring
> > > infrastructure). But I believe knowledge about release should be backed up
> > > somehow inside of ASF infra.
> > >
> > > The last issue related not to release, but to our test suites. It is now
> > > about 90 suites, which may have different settings. There is no easy and
> > > clear way to grep, check consistency, check possible options we've used.
> > >
> > > So let's consider 2 possible options to solve these issues.
> > >
> > > Option A: store settings in VCS
> > > https://www.jetbrains.com/help/teamcity/2019.1/storing-project-settings-in-version-control.html
> > >
> > >
> > > Option B: use DSL
> > > https://www.jetbrains.com/help/teamcity/2019.1/storing-project-settings-in-version-control.html
> > >
> > >
> > > Please share your vision.
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > PS. to see how DSL can look like you can use
> > > https://ci.ignite.apache.org/admin/editBuild.html?id=buildType:Releases_ApacheIgniteMain_ReleaseBuild
> > >
> > > and click view DSL.
>
>
>
> --
> Best regards,
> Ivan Pavlukhin

--
Best regards,
Ivan Pavlukhin

[Dmitry Pavlov]

Ivan, sorry, it seems PMCs forgot to add your account to Committer group.
I've changed rights settings.

чт, 20 июн. 2019 г. в 08:42, Павлухин Иван <[hidden email]>:

> Dmitriy,
>
> It is a very important topic for Ignite CI. I think it would be great
> if all significant changes in a build configuration goes with a
> meaningful description (a commit message). As for me, DSL sounds
> promising.
>
> BTW, a special rights are needed to view
>
> https://ci.ignite.apache.org/admin/editBuild.html?id=buildType:Releases_ApacheIgniteMain_ReleaseBuild
> (I do not have such).
>
> вт, 18 июн. 2019 г. в 14:30, Nikolay Izhikov <[hidden email]>:
> >
> > Hello, Dmitriy.
> >
> > Thanks, for starting this discussion.
> > I think almost all of community members don't know the difference
> between two options.
> >
> > I vote for the simplest solution with the human readable format.
> >
> >
> > В Пн, 17/06/2019 в 21:07 +0300, Dmitriy Pavlov пишет:
> > > Hi Igniters,
> > >
> > > During preparing of the release I've faced with lack of information
> related
> > > to required steps. This was one reason of too long release preparation.
> > > This process was automated by several Ignite contributors (thank you,
> BTW).
> > > Now, these valued automation results are more or less stored using
> TeamCity
> > > settings at ci.ignite.apache.org.
> > >
> > > First of all, changes done in these steps before were not always
> > > consistent. The TC's built-in audit wasn't useful, because there were
> tons
> > > of changes. So it is not clear who and why changed something.
> > >
> > > Secondly, Ignite release is performed outside of ASF infrastructure.
> It is
> > > not a problem itself (and we're grateful to GridGain to sponsoring
> > > infrastructure). But I believe knowledge about release should be
> backed up
> > > somehow inside of ASF infra.
> > >
> > > The last issue related not to release, but to our test suites. It is
> now
> > > about 90 suites, which may have different settings. There is no easy
> and
> > > clear way to grep, check consistency, check possible options we've
> used.
> > >
> > > So let's consider 2 possible options to solve these issues.
> > >
> > > Option A: store settings in VCS
> > >
> https://www.jetbrains.com/help/teamcity/2019.1/storing-project-settings-in-version-control.html
> > >
> > >
> > > Option B: use DSL
> > >
> https://www.jetbrains.com/help/teamcity/2019.1/storing-project-settings-in-version-control.html
> > >
> > >
> > > Please share your vision.
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > PS. to see how DSL can look like you can use
> > >
> https://ci.ignite.apache.org/admin/editBuild.html?id=buildType:Releases_ApacheIgniteMain_ReleaseBuild
> > >
> > > and click view DSL.
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>

[Dmitry Pavlov]

One more blog I found today related to this topic

https://blog.jetbrains.com/teamcity/2019/03/configuration-as-code-part-1-getting-started-with-kotlin-dsl/


If DSL may bring conditional steps, it may be used for various
optimizations in suites. Moreover, it is a git-based change management
opportunity.

чт, 20 июн. 2019 г. в 08:46, Павлухин Иван <[hidden email]>:

> For curious ones I copied a RunAll DSL configuration to gist [1].
>
> [1] https://gist.github.com/pavlukhin/e788bbebafe69ce9d9a3c9e9871ab8c4
>
> чт, 20 июн. 2019 г. в 08:42, Павлухин Иван <[hidden email]>:
> >
> > Dmitriy,
> >
> > It is a very important topic for Ignite CI. I think it would be great
> > if all significant changes in a build configuration goes with a
> > meaningful description (a commit message). As for me, DSL sounds
> > promising.
> >
> > BTW, a special rights are needed to view
> >
> https://ci.ignite.apache.org/admin/editBuild.html?id=buildType:Releases_ApacheIgniteMain_ReleaseBuild
> > (I do not have such).
> >
> > вт, 18 июн. 2019 г. в 14:30, Nikolay Izhikov <[hidden email]>:
> > >
> > > Hello, Dmitriy.
> > >
> > > Thanks, for starting this discussion.
> > > I think almost all of community members don't know the difference
> between two options.
> > >
> > > I vote for the simplest solution with the human readable format.
> > >
> > >
> > > В Пн, 17/06/2019 в 21:07 +0300, Dmitriy Pavlov пишет:
> > > > Hi Igniters,
> > > >
> > > > During preparing of the release I've faced with lack of information
> related
> > > > to required steps. This was one reason of too long release
> preparation.
> > > > This process was automated by several Ignite contributors (thank
> you, BTW).
> > > > Now, these valued automation results are more or less stored using
> TeamCity
> > > > settings at ci.ignite.apache.org.
> > > >
> > > > First of all, changes done in these steps before were not always
> > > > consistent. The TC's built-in audit wasn't useful, because there
> were tons
> > > > of changes. So it is not clear who and why changed something.
> > > >
> > > > Secondly, Ignite release is performed outside of ASF infrastructure.
> It is
> > > > not a problem itself (and we're grateful to GridGain to sponsoring
> > > > infrastructure). But I believe knowledge about release should be
> backed up
> > > > somehow inside of ASF infra.
> > > >
> > > > The last issue related not to release, but to our test suites. It is
> now
> > > > about 90 suites, which may have different settings. There is no easy
> and
> > > > clear way to grep, check consistency, check possible options we've
> used.
> > > >
> > > > So let's consider 2 possible options to solve these issues.
> > > >
> > > > Option A: store settings in VCS
> > > >
> https://www.jetbrains.com/help/teamcity/2019.1/storing-project-settings-in-version-control.html
> > > >
> > > >
> > > > Option B: use DSL
> > > >
> https://www.jetbrains.com/help/teamcity/2019.1/storing-project-settings-in-version-control.html
> > > >
> > > >
> > > > Please share your vision.
> > > >
> > > > Sincerely,
> > > > Dmitriy Pavlov
> > > >
> > > > PS. to see how DSL can look like you can use
> > > >
> https://ci.ignite.apache.org/admin/editBuild.html?id=buildType:Releases_ApacheIgniteMain_ReleaseBuild
> > > >
> > > > and click view DSL.
> >
> >
> >
> > --
> > Best regards,
> > Ivan Pavlukhin
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>