Make Teamcity Green Again
Locked
 
|
Hi Igniters,
We all know that there are a lot of open tasks in Ignite project. To be able to handle all these tasks, we need to grow our community. And to achieve that, we should pay special attention to new community members. As recent newcomer I’ve faced the following problem: There is no way to find out by yourself whether your pull request changes are correct or not. Contributor can not distinguish pull request test failures and master test failures. Experienced Ignite contributor can estimate the damage, but how it can be done by new community member? Once failing test is introduced to master it is multiplexed to all later PRs. As a result, we have tons of failures and suites timeouts, which waste Teamcity agents time. I suggest to start new activity - MakeTeamcityGreenAgain. I’m sure that there are a lot of experienced developers in the community who understand issues with tests very quickly. Please contact me if you are ready to help the community to fix current test issues. I would appreciate any ideas and sharing your vision how we can achieve green teamcity. If you are a newcomer, please share your feeling: are you confused by test failures on Teamcity? Sincerely, Dmitriy Pavlov |
|
I think the right question to ask is why do we have RED tests to begin
with. In my view, TC should be green. If there are tests that do not pass, we should disable them and file blocker tickets for the next releases. D. On Thu, Jul 20, 2017 at 1:20 PM, Dmitry Pavlov <[hidden email]> wrote: > Hi Igniters, > > We all know that there are a lot of open tasks in Ignite project. To be > able to handle all these tasks, we need to grow our community. And to > achieve that, we should pay special attention to new community members. > > As recent newcomer I’ve faced the following problem: There is no way to > find out by yourself whether your pull request changes are correct or not. > Contributor can not distinguish pull request test failures and master test > failures. Experienced Ignite contributor can estimate the damage, but how > it can be done by new community member? > > Once failing test is introduced to master it is multiplexed to all later > PRs. As a result, we have tons of failures and suites timeouts, which waste > Teamcity agents time. > > I suggest to start new activity - MakeTeamcityGreenAgain. I’m sure that > there are a lot of experienced developers in the community who understand > issues with tests very quickly. Please contact me if you are ready to help > the community to fix current test issues. > > I would appreciate any ideas and sharing your vision how we can achieve > green teamcity. > > If you are a newcomer, please share your feeling: are you confused by test > failures on Teamcity? > > Sincerely, > > Dmitriy Pavlov > |
Re: Make Teamcity Green Again
|
+1 to Dmitry Pavlov from me.
Green team city would be very helpful. > If you are a newcomer, please share your feeling: are you confused by test > failures on Teamcity? I'm working on my very first ignite issue and it's not easy to understand TC build results. 21.07.2017 00:30, Dmitriy Setrakyan пишет: > I think the right question to ask is why do we have RED tests to begin > with. In my view, TC should be green. If there are tests that do not pass, > we should disable them and file blocker tickets for the next releases. > > D. > > On Thu, Jul 20, 2017 at 1:20 PM, Dmitry Pavlov <[hidden email]> > wrote: > >> Hi Igniters, >> >> We all know that there are a lot of open tasks in Ignite project. To be >> able to handle all these tasks, we need to grow our community. And to >> achieve that, we should pay special attention to new community members. >> >> As recent newcomer I’ve faced the following problem: There is no way to >> find out by yourself whether your pull request changes are correct or not. >> Contributor can not distinguish pull request test failures and master test >> failures. Experienced Ignite contributor can estimate the damage, but how >> it can be done by new community member? >> >> Once failing test is introduced to master it is multiplexed to all later >> PRs. As a result, we have tons of failures and suites timeouts, which waste >> Teamcity agents time. >> >> I suggest to start new activity - MakeTeamcityGreenAgain. I’m sure that >> there are a lot of experienced developers in the community who understand >> issues with tests very quickly. Please contact me if you are ready to help >> the community to fix current test issues. >> >> I would appreciate any ideas and sharing your vision how we can achieve >> green teamcity. >> >> If you are a newcomer, please share your feeling: are you confused by test >> failures on Teamcity? >> >> Sincerely, >> >> Dmitriy Pavlov >> > |
Re: Make Teamcity Green Again
|
|
Nikolay,
That's also a big problem for me, as reviewer, to accept changes and merge them to master. Dmitriy, Currently we have some contributions *from community* blocked by red Teamcity. On Fri, Jul 21, 2017 at 12:47 AM, Nikolay Izhikov <[hidden email]> wrote: > +1 to Dmitry Pavlov from me. > > Green team city would be very helpful. > > > > If you are a newcomer, please share your feeling: are you confused by > test > > failures on Teamcity? > > I'm working on my very first ignite issue > and it's not easy to understand TC build results. > > > 21.07.2017 00:30, Dmitriy Setrakyan пишет: > > I think the right question to ask is why do we have RED tests to begin >> with. In my view, TC should be green. If there are tests that do not pass, >> we should disable them and file blocker tickets for the next releases. >> >> D. >> >> On Thu, Jul 20, 2017 at 1:20 PM, Dmitry Pavlov <[hidden email]> >> wrote: >> >> Hi Igniters, >>> >>> We all know that there are a lot of open tasks in Ignite project. To be >>> able to handle all these tasks, we need to grow our community. And to >>> achieve that, we should pay special attention to new community members. >>> >>> As recent newcomer I’ve faced the following problem: There is no way to >>> find out by yourself whether your pull request changes are correct or >>> not. >>> Contributor can not distinguish pull request test failures and master >>> test >>> failures. Experienced Ignite contributor can estimate the damage, but how >>> it can be done by new community member? >>> >>> Once failing test is introduced to master it is multiplexed to all later >>> PRs. As a result, we have tons of failures and suites timeouts, which >>> waste >>> Teamcity agents time. >>> >>> I suggest to start new activity - MakeTeamcityGreenAgain. I’m sure that >>> there are a lot of experienced developers in the community who understand >>> issues with tests very quickly. Please contact me if you are ready to >>> help >>> the community to fix current test issues. >>> >>> I would appreciate any ideas and sharing your vision how we can achieve >>> green teamcity. >>> >>> If you are a newcomer, please share your feeling: are you confused by >>> test >>> failures on Teamcity? >>> >>> Sincerely, >>> >>> Dmitriy Pavlov >>> >>> >> |
|
|
Hi guys.
I vote for #MakeTeamcityGreenAgain. :-) FYI: it had been described and supported previously[1] After the completion of my current task I will try to help with this activity. [1] http://apache-ignite-developers.2346864.n4.nabble.com/Test-failures-td14353.html 2017-07-21 12:39 GMT+03:00 Anton Vinogradov <[hidden email]>: > Nikolay, > > That's also a big problem for me, as reviewer, to accept changes and merge > them to master. > > Dmitriy, > > Currently we have some contributions *from community* blocked by red > Teamcity. > > On Fri, Jul 21, 2017 at 12:47 AM, Nikolay Izhikov <[hidden email]> > wrote: > > > +1 to Dmitry Pavlov from me. > > > > Green team city would be very helpful. > > > > > > > If you are a newcomer, please share your feeling: are you confused by > > test > > > failures on Teamcity? > > > > I'm working on my very first ignite issue > > and it's not easy to understand TC build results. > > > > > > 21.07.2017 00:30, Dmitriy Setrakyan пишет: > > > > I think the right question to ask is why do we have RED tests to begin > >> with. In my view, TC should be green. If there are tests that do not > pass, > >> we should disable them and file blocker tickets for the next releases. > >> > >> D. > >> > >> On Thu, Jul 20, 2017 at 1:20 PM, Dmitry Pavlov <[hidden email]> > >> wrote: > >> > >> Hi Igniters, > >>> > >>> We all know that there are a lot of open tasks in Ignite project. To be > >>> able to handle all these tasks, we need to grow our community. And to > >>> achieve that, we should pay special attention to new community members. > >>> > >>> As recent newcomer I’ve faced the following problem: There is no way to > >>> find out by yourself whether your pull request changes are correct or > >>> not. > >>> Contributor can not distinguish pull request test failures and master > >>> test > >>> failures. Experienced Ignite contributor can estimate the damage, but > how > >>> it can be done by new community member? > >>> > >>> Once failing test is introduced to master it is multiplexed to all > later > >>> PRs. As a result, we have tons of failures and suites timeouts, which > >>> waste > >>> Teamcity agents time. > >>> > >>> I suggest to start new activity - MakeTeamcityGreenAgain. I’m sure that > >>> there are a lot of experienced developers in the community who > understand > >>> issues with tests very quickly. Please contact me if you are ready to > >>> help > >>> the community to fix current test issues. > >>> > >>> I would appreciate any ideas and sharing your vision how we can achieve > >>> green teamcity. > >>> > >>> If you are a newcomer, please share your feeling: are you confused by > >>> test > >>> failures on Teamcity? > >>> > >>> Sincerely, > >>> > >>> Dmitriy Pavlov > >>> > >>> > >> > -- Best Regards, Vyacheslav D. |
Re: Make Teamcity Green Again
|
|
Hello, Igniters.
Also ready to help to #MakeTeamcityGreenAgain ! 21 июля 2017 г. 12:56 PM пользователь "Vyacheslav Daradur" < [hidden email]> написал: > Hi guys. > > I vote for #MakeTeamcityGreenAgain. :-) > > FYI: it had been described and supported previously[1] > > After the completion of my current task I will try to help with this > activity. > > [1] > http://apache-ignite-developers.2346864.n4.nabble. > com/Test-failures-td14353.html > > 2017-07-21 12:39 GMT+03:00 Anton Vinogradov <[hidden email]>: > > > Nikolay, > > > > That's also a big problem for me, as reviewer, to accept changes and > merge > > them to master. > > > > Dmitriy, > > > > Currently we have some contributions *from community* blocked by red > > Teamcity. > > > > On Fri, Jul 21, 2017 at 12:47 AM, Nikolay Izhikov < > [hidden email]> > > wrote: > > > > > +1 to Dmitry Pavlov from me. > > > > > > Green team city would be very helpful. > > > > > > > > > > If you are a newcomer, please share your feeling: are you confused by > > > test > > > > failures on Teamcity? > > > > > > I'm working on my very first ignite issue > > > and it's not easy to understand TC build results. > > > > > > > > > 21.07.2017 00:30, Dmitriy Setrakyan пишет: > > > > > > I think the right question to ask is why do we have RED tests to begin > > >> with. In my view, TC should be green. If there are tests that do not > > pass, > > >> we should disable them and file blocker tickets for the next releases. > > >> > > >> D. > > >> > > >> On Thu, Jul 20, 2017 at 1:20 PM, Dmitry Pavlov <[hidden email] > > > > >> wrote: > > >> > > >> Hi Igniters, > > >>> > > >>> We all know that there are a lot of open tasks in Ignite project. To > be > > >>> able to handle all these tasks, we need to grow our community. And to > > >>> achieve that, we should pay special attention to new community > members. > > >>> > > >>> As recent newcomer I’ve faced the following problem: There is no way > to > > >>> find out by yourself whether your pull request changes are correct or > > >>> not. > > >>> Contributor can not distinguish pull request test failures and master > > >>> test > > >>> failures. Experienced Ignite contributor can estimate the damage, but > > how > > >>> it can be done by new community member? > > >>> > > >>> Once failing test is introduced to master it is multiplexed to all > > later > > >>> PRs. As a result, we have tons of failures and suites timeouts, which > > >>> waste > > >>> Teamcity agents time. > > >>> > > >>> I suggest to start new activity - MakeTeamcityGreenAgain. I’m sure > that > > >>> there are a lot of experienced developers in the community who > > understand > > >>> issues with tests very quickly. Please contact me if you are ready to > > >>> help > > >>> the community to fix current test issues. > > >>> > > >>> I would appreciate any ideas and sharing your vision how we can > achieve > > >>> green teamcity. > > >>> > > >>> If you are a newcomer, please share your feeling: are you confused by > > >>> test > > >>> failures on Teamcity? > > >>> > > >>> Sincerely, > > >>> > > >>> Dmitriy Pavlov > > >>> > > >>> > > >> > > > > > > -- > Best Regards, Vyacheslav D. > |
|
> Green Again
> Again As if it ever was green :) Of course +1 on this, let me know if you see any .NET-related failures. On Fri, Jul 21, 2017 at 1:06 PM, Николай Ижиков <[hidden email]> wrote: > Hello, Igniters. > > Also ready to help to #MakeTeamcityGreenAgain ! > > 21 июля 2017 г. 12:56 PM пользователь "Vyacheslav Daradur" < > [hidden email]> написал: > > > Hi guys. > > > > I vote for #MakeTeamcityGreenAgain. :-) > > > > FYI: it had been described and supported previously[1] > > > > After the completion of my current task I will try to help with this > > activity. > > > > [1] > > http://apache-ignite-developers.2346864.n4.nabble. > > com/Test-failures-td14353.html > > > > 2017-07-21 12:39 GMT+03:00 Anton Vinogradov <[hidden email]>: > > > > > Nikolay, > > > > > > That's also a big problem for me, as reviewer, to accept changes and > > merge > > > them to master. > > > > > > Dmitriy, > > > > > > Currently we have some contributions *from community* blocked by red > > > Teamcity. > > > > > > On Fri, Jul 21, 2017 at 12:47 AM, Nikolay Izhikov < > > [hidden email]> > > > wrote: > > > > > > > +1 to Dmitry Pavlov from me. > > > > > > > > Green team city would be very helpful. > > > > > > > > > > > > > If you are a newcomer, please share your feeling: are you confused > by > > > > test > > > > > failures on Teamcity? > > > > > > > > I'm working on my very first ignite issue > > > > and it's not easy to understand TC build results. > > > > > > > > > > > > 21.07.2017 00:30, Dmitriy Setrakyan пишет: > > > > > > > > I think the right question to ask is why do we have RED tests to > begin > > > >> with. In my view, TC should be green. If there are tests that do not > > > pass, > > > >> we should disable them and file blocker tickets for the next > releases. > > > >> > > > >> D. > > > >> > > > >> On Thu, Jul 20, 2017 at 1:20 PM, Dmitry Pavlov < > [hidden email] > > > > > > >> wrote: > > > >> > > > >> Hi Igniters, > > > >>> > > > >>> We all know that there are a lot of open tasks in Ignite project. > To > > be > > > >>> able to handle all these tasks, we need to grow our community. And > to > > > >>> achieve that, we should pay special attention to new community > > members. > > > >>> > > > >>> As recent newcomer I’ve faced the following problem: There is no > way > > to > > > >>> find out by yourself whether your pull request changes are correct > or > > > >>> not. > > > >>> Contributor can not distinguish pull request test failures and > master > > > >>> test > > > >>> failures. Experienced Ignite contributor can estimate the damage, > but > > > how > > > >>> it can be done by new community member? > > > >>> > > > >>> Once failing test is introduced to master it is multiplexed to all > > > later > > > >>> PRs. As a result, we have tons of failures and suites timeouts, > which > > > >>> waste > > > >>> Teamcity agents time. > > > >>> > > > >>> I suggest to start new activity - MakeTeamcityGreenAgain. I’m sure > > that > > > >>> there are a lot of experienced developers in the community who > > > understand > > > >>> issues with tests very quickly. Please contact me if you are ready > to > > > >>> help > > > >>> the community to fix current test issues. > > > >>> > > > >>> I would appreciate any ideas and sharing your vision how we can > > achieve > > > >>> green teamcity. > > > >>> > > > >>> If you are a newcomer, please share your feeling: are you confused > by > > > >>> test > > > >>> failures on Teamcity? > > > >>> > > > >>> Sincerely, > > > >>> > > > >>> Dmitriy Pavlov > > > >>> > > > >>> > > > >> > > > > > > > > > > > -- > > Best Regards, Vyacheslav D. > > > |
|
|
Hi Igniters,
I am happy to report that the activity of monitoring and fixing the tests brought the first significant victory! Today tests launch running all Ignite tests (Ignite 2.0 Tests->RunAll) is completely green: https://ci.ignite.apache.org/viewLog.html?buildId=930953 This is an reason for being pride, but not a reason to stop this activity. Unfortunately, there are still a number of tests with unpredictable failures (flaky, 234 tests), and a number of tests have been muted. Full project scope is available in the article https://cwiki.apache.org/confluence/display/IGNITE/Make+Teamcity+Green+Again In any case, thank to all of you who helped and continues to help the project to correct the tests and bugs. Sincerely, Pavlov Dmitry пт, 21 июл. 2017 г. в 14:29, Pavel Tupitsyn <[hidden email]>: > > Green Again > > Again > As if it ever was green :) > > Of course +1 on this, let me know if you see any .NET-related failures. > > On Fri, Jul 21, 2017 at 1:06 PM, Николай Ижиков <[hidden email]> > wrote: > > > Hello, Igniters. > > > > Also ready to help to #MakeTeamcityGreenAgain ! > > > > 21 июля 2017 г. 12:56 PM пользователь "Vyacheslav Daradur" < > > [hidden email]> написал: > > > > > Hi guys. > > > > > > I vote for #MakeTeamcityGreenAgain. :-) > > > > > > FYI: it had been described and supported previously[1] > > > > > > After the completion of my current task I will try to help with this > > > activity. > > > > > > [1] > > > http://apache-ignite-developers.2346864.n4.nabble. > > > com/Test-failures-td14353.html > > > > > > 2017-07-21 12:39 GMT+03:00 Anton Vinogradov <[hidden email]>: > > > > > > > Nikolay, > > > > > > > > That's also a big problem for me, as reviewer, to accept changes and > > > merge > > > > them to master. > > > > > > > > Dmitriy, > > > > > > > > Currently we have some contributions *from community* blocked by red > > > > Teamcity. > > > > > > > > On Fri, Jul 21, 2017 at 12:47 AM, Nikolay Izhikov < > > > [hidden email]> > > > > wrote: > > > > > > > > > +1 to Dmitry Pavlov from me. > > > > > > > > > > Green team city would be very helpful. > > > > > > > > > > > > > > > > If you are a newcomer, please share your feeling: are you > confused > > by > > > > > test > > > > > > failures on Teamcity? > > > > > > > > > > I'm working on my very first ignite issue > > > > > and it's not easy to understand TC build results. > > > > > > > > > > > > > > > 21.07.2017 00:30, Dmitriy Setrakyan пишет: > > > > > > > > > > I think the right question to ask is why do we have RED tests to > > begin > > > > >> with. In my view, TC should be green. If there are tests that do > not > > > > pass, > > > > >> we should disable them and file blocker tickets for the next > > releases. > > > > >> > > > > >> D. > > > > >> > > > > >> On Thu, Jul 20, 2017 at 1:20 PM, Dmitry Pavlov < > > [hidden email] > > > > > > > > >> wrote: > > > > >> > > > > >> Hi Igniters, > > > > >>> > > > > >>> We all know that there are a lot of open tasks in Ignite project. > > To > > > be > > > > >>> able to handle all these tasks, we need to grow our community. > And > > to > > > > >>> achieve that, we should pay special attention to new community > > > members. > > > > >>> > > > > >>> As recent newcomer I’ve faced the following problem: There is no > > way > > > to > > > > >>> find out by yourself whether your pull request changes are > correct > > or > > > > >>> not. > > > > >>> Contributor can not distinguish pull request test failures and > > master > > > > >>> test > > > > >>> failures. Experienced Ignite contributor can estimate the damage, > > but > > > > how > > > > >>> it can be done by new community member? > > > > >>> > > > > >>> Once failing test is introduced to master it is multiplexed to > all > > > > later > > > > >>> PRs. As a result, we have tons of failures and suites timeouts, > > which > > > > >>> waste > > > > >>> Teamcity agents time. > > > > >>> > > > > >>> I suggest to start new activity - MakeTeamcityGreenAgain. I’m > sure > > > that > > > > >>> there are a lot of experienced developers in the community who > > > > understand > > > > >>> issues with tests very quickly. Please contact me if you are > ready > > to > > > > >>> help > > > > >>> the community to fix current test issues. > > > > >>> > > > > >>> I would appreciate any ideas and sharing your vision how we can > > > achieve > > > > >>> green teamcity. > > > > >>> > > > > >>> If you are a newcomer, please share your feeling: are you > confused > > by > > > > >>> test > > > > >>> failures on Teamcity? > > > > >>> > > > > >>> Sincerely, > > > > >>> > > > > >>> Dmitriy Pavlov > > > > >>> > > > > >>> > > > > >> > > > > > > > > > > > > > > > > -- > > > Best Regards, Vyacheslav D. > > > > > > |
|
|
Dmitriy,
Now it’s time to fix a pretty decent number of muted and ignored tests. BTW, what’s the difference between “ignored” and “muted”? I’ve seen the former before. — Denis > On Nov 5, 2017, at 9:58 AM, Dmitry Pavlov <[hidden email]> wrote: > > Hi Igniters, > > I am happy to report that the activity of monitoring and fixing the tests > brought the first significant victory! > > Today tests launch running all Ignite tests (Ignite 2.0 Tests->RunAll) is > completely green: https://ci.ignite.apache.org/viewLog.html?buildId=930953 > > This is an reason for being pride, but not a reason to stop this activity. > Unfortunately, there are still a number of tests with unpredictable > failures (flaky, 234 tests), and a number of tests have been muted. Full > project scope is available in the article > https://cwiki.apache.org/confluence/display/IGNITE/Make+Teamcity+Green+Again > > In any case, thank to all of you who helped and continues to help the > project to correct the tests and bugs. > > Sincerely, > Pavlov Dmitry > > > пт, 21 июл. 2017 г. в 14:29, Pavel Tupitsyn <[hidden email]>: > >>> Green Again >>> Again >> As if it ever was green :) >> >> Of course +1 on this, let me know if you see any .NET-related failures. >> >> On Fri, Jul 21, 2017 at 1:06 PM, Николай Ижиков <[hidden email]> >> wrote: >> >>> Hello, Igniters. >>> >>> Also ready to help to #MakeTeamcityGreenAgain ! >>> >>> 21 июля 2017 г. 12:56 PM пользователь "Vyacheslav Daradur" < >>> [hidden email]> написал: >>> >>>> Hi guys. >>>> >>>> I vote for #MakeTeamcityGreenAgain. :-) >>>> >>>> FYI: it had been described and supported previously[1] >>>> >>>> After the completion of my current task I will try to help with this >>>> activity. >>>> >>>> [1] >>>> http://apache-ignite-developers.2346864.n4.nabble. >>>> com/Test-failures-td14353.html >>>> >>>> 2017-07-21 12:39 GMT+03:00 Anton Vinogradov <[hidden email]>: >>>> >>>>> Nikolay, >>>>> >>>>> That's also a big problem for me, as reviewer, to accept changes and >>>> merge >>>>> them to master. >>>>> >>>>> Dmitriy, >>>>> >>>>> Currently we have some contributions *from community* blocked by red >>>>> Teamcity. >>>>> >>>>> On Fri, Jul 21, 2017 at 12:47 AM, Nikolay Izhikov < >>>> [hidden email]> >>>>> wrote: >>>>> >>>>>> +1 to Dmitry Pavlov from me. >>>>>> >>>>>> Green team city would be very helpful. >>>>>> >>>>>> >>>>>>> If you are a newcomer, please share your feeling: are you >> confused >>> by >>>>>> test >>>>>>> failures on Teamcity? >>>>>> >>>>>> I'm working on my very first ignite issue >>>>>> and it's not easy to understand TC build results. >>>>>> >>>>>> >>>>>> 21.07.2017 00:30, Dmitriy Setrakyan пишет: >>>>>> >>>>>> I think the right question to ask is why do we have RED tests to >>> begin >>>>>>> with. In my view, TC should be green. If there are tests that do >> not >>>>> pass, >>>>>>> we should disable them and file blocker tickets for the next >>> releases. >>>>>>> >>>>>>> D. >>>>>>> >>>>>>> On Thu, Jul 20, 2017 at 1:20 PM, Dmitry Pavlov < >>> [hidden email] >>>>> >>>>>>> wrote: >>>>>>> >>>>>>> Hi Igniters, >>>>>>>> >>>>>>>> We all know that there are a lot of open tasks in Ignite project. >>> To >>>> be >>>>>>>> able to handle all these tasks, we need to grow our community. >> And >>> to >>>>>>>> achieve that, we should pay special attention to new community >>>> members. >>>>>>>> >>>>>>>> As recent newcomer I’ve faced the following problem: There is no >>> way >>>> to >>>>>>>> find out by yourself whether your pull request changes are >> correct >>> or >>>>>>>> not. >>>>>>>> Contributor can not distinguish pull request test failures and >>> master >>>>>>>> test >>>>>>>> failures. Experienced Ignite contributor can estimate the damage, >>> but >>>>> how >>>>>>>> it can be done by new community member? >>>>>>>> >>>>>>>> Once failing test is introduced to master it is multiplexed to >> all >>>>> later >>>>>>>> PRs. As a result, we have tons of failures and suites timeouts, >>> which >>>>>>>> waste >>>>>>>> Teamcity agents time. >>>>>>>> >>>>>>>> I suggest to start new activity - MakeTeamcityGreenAgain. I’m >> sure >>>> that >>>>>>>> there are a lot of experienced developers in the community who >>>>> understand >>>>>>>> issues with tests very quickly. Please contact me if you are >> ready >>> to >>>>>>>> help >>>>>>>> the community to fix current test issues. >>>>>>>> >>>>>>>> I would appreciate any ideas and sharing your vision how we can >>>> achieve >>>>>>>> green teamcity. >>>>>>>> >>>>>>>> If you are a newcomer, please share your feeling: are you >> confused >>> by >>>>>>>> test >>>>>>>> failures on Teamcity? >>>>>>>> >>>>>>>> Sincerely, >>>>>>>> >>>>>>>> Dmitriy Pavlov >>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> Best Regards, Vyacheslav D. >>>> >>> >> |
|
|
"Ignored" means that the test is disabled from the code, using annotation
@Ignore. Not sure it works for our JUnit3 style Java tests. It is used for .NET. "Muted" test is disabled in the interface TeamCity, but test code is actually executed (which is why hanging tests needs to be failed in addition to mute). The fact that the run all now became green means all the tests that fall at this point are flaky. If it is about 200 non-muted flaky tests having the probability of passing 98%, then the whole chain will pass only with the probability 0.98 ^ 200 = 1.7% I suggest continue to identify, fix, or mute such flaky tests to get stable green RunAll for correct code. пн, 6 нояб. 2017 г. в 22:57, Denis Magda <[hidden email]>: > Dmitriy, > > Now it’s time to fix a pretty decent number of muted and ignored tests. > BTW, what’s the difference between “ignored” and “muted”? I’ve seen the > former before. > > — > Denis > > > On Nov 5, 2017, at 9:58 AM, Dmitry Pavlov <[hidden email]> wrote: > > > > Hi Igniters, > > > > I am happy to report that the activity of monitoring and fixing the tests > > brought the first significant victory! > > > > Today tests launch running all Ignite tests (Ignite 2.0 Tests->RunAll) is > > completely green: > https://ci.ignite.apache.org/viewLog.html?buildId=930953 > > > > This is an reason for being pride, but not a reason to stop this > activity. > > Unfortunately, there are still a number of tests with unpredictable > > failures (flaky, 234 tests), and a number of tests have been muted. Full > > project scope is available in the article > > > https://cwiki.apache.org/confluence/display/IGNITE/Make+Teamcity+Green+Again > > > > In any case, thank to all of you who helped and continues to help the > > project to correct the tests and bugs. > > > > Sincerely, > > Pavlov Dmitry > > > > > > пт, 21 июл. 2017 г. в 14:29, Pavel Tupitsyn <[hidden email]>: > > > >>> Green Again > >>> Again > >> As if it ever was green :) > >> > >> Of course +1 on this, let me know if you see any .NET-related failures. > >> > >> On Fri, Jul 21, 2017 at 1:06 PM, Николай Ижиков <[hidden email] > > > >> wrote: > >> > >>> Hello, Igniters. > >>> > >>> Also ready to help to #MakeTeamcityGreenAgain ! > >>> > >>> 21 июля 2017 г. 12:56 PM пользователь "Vyacheslav Daradur" < > >>> [hidden email]> написал: > >>> > >>>> Hi guys. > >>>> > >>>> I vote for #MakeTeamcityGreenAgain. :-) > >>>> > >>>> FYI: it had been described and supported previously[1] > >>>> > >>>> After the completion of my current task I will try to help with this > >>>> activity. > >>>> > >>>> [1] > >>>> http://apache-ignite-developers.2346864.n4.nabble. > >>>> com/Test-failures-td14353.html > >>>> > >>>> 2017-07-21 12:39 GMT+03:00 Anton Vinogradov <[hidden email]>: > >>>> > >>>>> Nikolay, > >>>>> > >>>>> That's also a big problem for me, as reviewer, to accept changes and > >>>> merge > >>>>> them to master. > >>>>> > >>>>> Dmitriy, > >>>>> > >>>>> Currently we have some contributions *from community* blocked by red > >>>>> Teamcity. > >>>>> > >>>>> On Fri, Jul 21, 2017 at 12:47 AM, Nikolay Izhikov < > >>>> [hidden email]> > >>>>> wrote: > >>>>> > >>>>>> +1 to Dmitry Pavlov from me. > >>>>>> > >>>>>> Green team city would be very helpful. > >>>>>> > >>>>>> > >>>>>>> If you are a newcomer, please share your feeling: are you > >> confused > >>> by > >>>>>> test > >>>>>>> failures on Teamcity? > >>>>>> > >>>>>> I'm working on my very first ignite issue > >>>>>> and it's not easy to understand TC build results. > >>>>>> > >>>>>> > >>>>>> 21.07.2017 00:30, Dmitriy Setrakyan пишет: > >>>>>> > >>>>>> I think the right question to ask is why do we have RED tests to > >>> begin > >>>>>>> with. In my view, TC should be green. If there are tests that do > >> not > >>>>> pass, > >>>>>>> we should disable them and file blocker tickets for the next > >>> releases. > >>>>>>> > >>>>>>> D. > >>>>>>> > >>>>>>> On Thu, Jul 20, 2017 at 1:20 PM, Dmitry Pavlov < > >>> [hidden email] > >>>>> > >>>>>>> wrote: > >>>>>>> > >>>>>>> Hi Igniters, > >>>>>>>> > >>>>>>>> We all know that there are a lot of open tasks in Ignite project. > >>> To > >>>> be > >>>>>>>> able to handle all these tasks, we need to grow our community. > >> And > >>> to > >>>>>>>> achieve that, we should pay special attention to new community > >>>> members. > >>>>>>>> > >>>>>>>> As recent newcomer I’ve faced the following problem: There is no > >>> way > >>>> to > >>>>>>>> find out by yourself whether your pull request changes are > >> correct > >>> or > >>>>>>>> not. > >>>>>>>> Contributor can not distinguish pull request test failures and > >>> master > >>>>>>>> test > >>>>>>>> failures. Experienced Ignite contributor can estimate the damage, > >>> but > >>>>> how > >>>>>>>> it can be done by new community member? > >>>>>>>> > >>>>>>>> Once failing test is introduced to master it is multiplexed to > >> all > >>>>> later > >>>>>>>> PRs. As a result, we have tons of failures and suites timeouts, > >>> which > >>>>>>>> waste > >>>>>>>> Teamcity agents time. > >>>>>>>> > >>>>>>>> I suggest to start new activity - MakeTeamcityGreenAgain. I’m > >> sure > >>>> that > >>>>>>>> there are a lot of experienced developers in the community who > >>>>> understand > >>>>>>>> issues with tests very quickly. Please contact me if you are > >> ready > >>> to > >>>>>>>> help > >>>>>>>> the community to fix current test issues. > >>>>>>>> > >>>>>>>> I would appreciate any ideas and sharing your vision how we can > >>>> achieve > >>>>>>>> green teamcity. > >>>>>>>> > >>>>>>>> If you are a newcomer, please share your feeling: are you > >> confused > >>> by > >>>>>>>> test > >>>>>>>> failures on Teamcity? > >>>>>>>> > >>>>>>>> Sincerely, > >>>>>>>> > >>>>>>>> Dmitriy Pavlov > >>>>>>>> > >>>>>>>> > >>>>>>> > >>>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> Best Regards, Vyacheslav D. > >>>> > >>> > >> > > |
|
|
Hi Folks,
I want to resurrect this process and start to fix tests. Currently approx. 60-80 tests are failed each run on master. As one from the first point I suggest to use TC investigations for brief research of test failures. If you can help with research please create and assign investigation on TC (on test fail select dropdown menu, then 'Investigate / Mute' then 'Investigated by: me' & 'Resolve: auto/manually'; Runs https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_RunAll&branch_IgniteTests24Java8=%3Cdefault%3E ). Also help with existing fixes review/merge would be very appreciated. Issue creation is still desired, but investigation will help us to identify tests failures which are under research and failures not researched for now. Sincerely, Dmitriy Pavlov вт, 7 нояб. 2017 г. в 0:06, Dmitry Pavlov <[hidden email]>: > "Ignored" means that the test is disabled from the code, using annotation > @Ignore. Not sure it works for our JUnit3 style Java tests. It is used for > .NET. > "Muted" test is disabled in the interface TeamCity, but test code is > actually executed (which is why hanging tests needs to be failed in > addition to mute). > > The fact that the run all now became green means all the tests that fall > at this point are flaky. If it is about 200 non-muted flaky tests having > the probability of passing 98%, then the whole chain will pass only with > the probability 0.98 ^ 200 = 1.7% > > I suggest continue to identify, fix, or mute such flaky tests to get > stable green RunAll for correct code. > > пн, 6 нояб. 2017 г. в 22:57, Denis Magda <[hidden email]>: > >> Dmitriy, >> >> Now it’s time to fix a pretty decent number of muted and ignored tests. >> BTW, what’s the difference between “ignored” and “muted”? I’ve seen the >> former before. >> >> — >> Denis >> >> > On Nov 5, 2017, at 9:58 AM, Dmitry Pavlov <[hidden email]> >> wrote: >> > >> > Hi Igniters, >> > >> > I am happy to report that the activity of monitoring and fixing the >> tests >> > brought the first significant victory! >> > >> > Today tests launch running all Ignite tests (Ignite 2.0 Tests->RunAll) >> is >> > completely green: >> https://ci.ignite.apache.org/viewLog.html?buildId=930953 >> > >> > This is an reason for being pride, but not a reason to stop this >> activity. >> > Unfortunately, there are still a number of tests with unpredictable >> > failures (flaky, 234 tests), and a number of tests have been muted. Full >> > project scope is available in the article >> > >> https://cwiki.apache.org/confluence/display/IGNITE/Make+Teamcity+Green+Again >> > >> > In any case, thank to all of you who helped and continues to help the >> > project to correct the tests and bugs. >> > >> > Sincerely, >> > Pavlov Dmitry >> > >> > >> > пт, 21 июл. 2017 г. в 14:29, Pavel Tupitsyn <[hidden email]>: >> > >> >>> Green Again >> >>> Again >> >> As if it ever was green :) >> >> >> >> Of course +1 on this, let me know if you see any .NET-related failures. >> >> >> >> On Fri, Jul 21, 2017 at 1:06 PM, Николай Ижиков < >> [hidden email]> >> >> wrote: >> >> >> >>> Hello, Igniters. >> >>> >> >>> Also ready to help to #MakeTeamcityGreenAgain ! >> >>> >> >>> 21 июля 2017 г. 12:56 PM пользователь "Vyacheslav Daradur" < >> >>> [hidden email]> написал: >> >>> >> >>>> Hi guys. >> >>>> >> >>>> I vote for #MakeTeamcityGreenAgain. :-) >> >>>> >> >>>> FYI: it had been described and supported previously[1] >> >>>> >> >>>> After the completion of my current task I will try to help with this >> >>>> activity. >> >>>> >> >>>> [1] >> >>>> http://apache-ignite-developers.2346864.n4.nabble. >> >>>> com/Test-failures-td14353.html >> >>>> >> >>>> 2017-07-21 12:39 GMT+03:00 Anton Vinogradov <[hidden email]>: >> >>>> >> >>>>> Nikolay, >> >>>>> >> >>>>> That's also a big problem for me, as reviewer, to accept changes and >> >>>> merge >> >>>>> them to master. >> >>>>> >> >>>>> Dmitriy, >> >>>>> >> >>>>> Currently we have some contributions *from community* blocked by red >> >>>>> Teamcity. >> >>>>> >> >>>>> On Fri, Jul 21, 2017 at 12:47 AM, Nikolay Izhikov < >> >>>> [hidden email]> >> >>>>> wrote: >> >>>>> >> >>>>>> +1 to Dmitry Pavlov from me. >> >>>>>> >> >>>>>> Green team city would be very helpful. >> >>>>>> >> >>>>>> >> >>>>>>> If you are a newcomer, please share your feeling: are you >> >> confused >> >>> by >> >>>>>> test >> >>>>>>> failures on Teamcity? >> >>>>>> >> >>>>>> I'm working on my very first ignite issue >> >>>>>> and it's not easy to understand TC build results. >> >>>>>> >> >>>>>> >> >>>>>> 21.07.2017 00:30, Dmitriy Setrakyan пишет: >> >>>>>> >> >>>>>> I think the right question to ask is why do we have RED tests to >> >>> begin >> >>>>>>> with. In my view, TC should be green. If there are tests that do >> >> not >> >>>>> pass, >> >>>>>>> we should disable them and file blocker tickets for the next >> >>> releases. >> >>>>>>> >> >>>>>>> D. >> >>>>>>> >> >>>>>>> On Thu, Jul 20, 2017 at 1:20 PM, Dmitry Pavlov < >> >>> [hidden email] >> >>>>> >> >>>>>>> wrote: >> >>>>>>> >> >>>>>>> Hi Igniters, >> >>>>>>>> >> >>>>>>>> We all know that there are a lot of open tasks in Ignite project. >> >>> To >> >>>> be >> >>>>>>>> able to handle all these tasks, we need to grow our community. >> >> And >> >>> to >> >>>>>>>> achieve that, we should pay special attention to new community >> >>>> members. >> >>>>>>>> >> >>>>>>>> As recent newcomer I’ve faced the following problem: There is no >> >>> way >> >>>> to >> >>>>>>>> find out by yourself whether your pull request changes are >> >> correct >> >>> or >> >>>>>>>> not. >> >>>>>>>> Contributor can not distinguish pull request test failures and >> >>> master >> >>>>>>>> test >> >>>>>>>> failures. Experienced Ignite contributor can estimate the damage, >> >>> but >> >>>>> how >> >>>>>>>> it can be done by new community member? >> >>>>>>>> >> >>>>>>>> Once failing test is introduced to master it is multiplexed to >> >> all >> >>>>> later >> >>>>>>>> PRs. As a result, we have tons of failures and suites timeouts, >> >>> which >> >>>>>>>> waste >> >>>>>>>> Teamcity agents time. >> >>>>>>>> >> >>>>>>>> I suggest to start new activity - MakeTeamcityGreenAgain. I’m >> >> sure >> >>>> that >> >>>>>>>> there are a lot of experienced developers in the community who >> >>>>> understand >> >>>>>>>> issues with tests very quickly. Please contact me if you are >> >> ready >> >>> to >> >>>>>>>> help >> >>>>>>>> the community to fix current test issues. >> >>>>>>>> >> >>>>>>>> I would appreciate any ideas and sharing your vision how we can >> >>>> achieve >> >>>>>>>> green teamcity. >> >>>>>>>> >> >>>>>>>> If you are a newcomer, please share your feeling: are you >> >> confused >> >>> by >> >>>>>>>> test >> >>>>>>>> failures on Teamcity? >> >>>>>>>> >> >>>>>>>> Sincerely, >> >>>>>>>> >> >>>>>>>> Dmitriy Pavlov >> >>>>>>>> >> >>>>>>>> >> >>>>>>> >> >>>>> >> >>>> >> >>>> >> >>>> >> >>>> -- >> >>>> Best Regards, Vyacheslav D. >> >>>> >> >>> >> >> >> >> |
|
|
Dmitriy, thanks for pushing this! We have to bring the number of the
failing tests to zero. I hope that the community gets behind this effort. D. On Mon, Feb 12, 2018 at 8:26 AM, Dmitry Pavlov <[hidden email]> wrote: > Hi Folks, > > I want to resurrect this process and start to fix tests. Currently approx. > 60-80 tests are failed each run on master. > > As one from the first point I suggest to use TC investigations for brief > research of test failures. > > If you can help with research please create and assign investigation on TC > (on test fail select dropdown menu, then 'Investigate / Mute' > then 'Investigated by: me' & 'Resolve: auto/manually'; Runs > https://ci.ignite.apache.org/viewType.html?buildTypeId= > IgniteTests24Java8_RunAll&branch_IgniteTests24Java8=%3Cdefault%3E > ). Also help with existing fixes review/merge would be very appreciated. > > Issue creation is still desired, but investigation will help us to identify > tests failures which are under research and failures not researched for > now. > > Sincerely, > Dmitriy Pavlov > > вт, 7 нояб. 2017 г. в 0:06, Dmitry Pavlov <[hidden email]>: > > > "Ignored" means that the test is disabled from the code, using annotation > > @Ignore. Not sure it works for our JUnit3 style Java tests. It is used > for > > .NET. > > "Muted" test is disabled in the interface TeamCity, but test code is > > actually executed (which is why hanging tests needs to be failed in > > addition to mute). > > > > The fact that the run all now became green means all the tests that fall > > at this point are flaky. If it is about 200 non-muted flaky tests having > > the probability of passing 98%, then the whole chain will pass only with > > the probability 0.98 ^ 200 = 1.7% > > > > I suggest continue to identify, fix, or mute such flaky tests to get > > stable green RunAll for correct code. > > > > пн, 6 нояб. 2017 г. в 22:57, Denis Magda <[hidden email]>: > > > >> Dmitriy, > >> > >> Now it’s time to fix a pretty decent number of muted and ignored tests. > >> BTW, what’s the difference between “ignored” and “muted”? I’ve seen the > >> former before. > >> > >> — > >> Denis > >> > >> > On Nov 5, 2017, at 9:58 AM, Dmitry Pavlov <[hidden email]> > >> wrote: > >> > > >> > Hi Igniters, > >> > > >> > I am happy to report that the activity of monitoring and fixing the > >> tests > >> > brought the first significant victory! > >> > > >> > Today tests launch running all Ignite tests (Ignite 2.0 Tests->RunAll) > >> is > >> > completely green: > >> https://ci.ignite.apache.org/viewLog.html?buildId=930953 > >> > > >> > This is an reason for being pride, but not a reason to stop this > >> activity. > >> > Unfortunately, there are still a number of tests with unpredictable > >> > failures (flaky, 234 tests), and a number of tests have been muted. > Full > >> > project scope is available in the article > >> > > >> https://cwiki.apache.org/confluence/display/IGNITE/ > Make+Teamcity+Green+Again > >> > > >> > In any case, thank to all of you who helped and continues to help the > >> > project to correct the tests and bugs. > >> > > >> > Sincerely, > >> > Pavlov Dmitry > >> > > >> > > >> > пт, 21 июл. 2017 г. в 14:29, Pavel Tupitsyn <[hidden email]>: > >> > > >> >>> Green Again > >> >>> Again > >> >> As if it ever was green :) > >> >> > >> >> Of course +1 on this, let me know if you see any .NET-related > failures. > >> >> > >> >> On Fri, Jul 21, 2017 at 1:06 PM, Николай Ижиков < > >> [hidden email]> > >> >> wrote: > >> >> > >> >>> Hello, Igniters. > >> >>> > >> >>> Also ready to help to #MakeTeamcityGreenAgain ! > >> >>> > >> >>> 21 июля 2017 г. 12:56 PM пользователь "Vyacheslav Daradur" < > >> >>> [hidden email]> написал: > >> >>> > >> >>>> Hi guys. > >> >>>> > >> >>>> I vote for #MakeTeamcityGreenAgain. :-) > >> >>>> > >> >>>> FYI: it had been described and supported previously[1] > >> >>>> > >> >>>> After the completion of my current task I will try to help with > this > >> >>>> activity. > >> >>>> > >> >>>> [1] > >> >>>> http://apache-ignite-developers.2346864.n4.nabble. > >> >>>> com/Test-failures-td14353.html > >> >>>> > >> >>>> 2017-07-21 12:39 GMT+03:00 Anton Vinogradov <[hidden email]>: > >> >>>> > >> >>>>> Nikolay, > >> >>>>> > >> >>>>> That's also a big problem for me, as reviewer, to accept changes > and > >> >>>> merge > >> >>>>> them to master. > >> >>>>> > >> >>>>> Dmitriy, > >> >>>>> > >> >>>>> Currently we have some contributions *from community* blocked by > red > >> >>>>> Teamcity. > >> >>>>> > >> >>>>> On Fri, Jul 21, 2017 at 12:47 AM, Nikolay Izhikov < > >> >>>> [hidden email]> > >> >>>>> wrote: > >> >>>>> > >> >>>>>> +1 to Dmitry Pavlov from me. > >> >>>>>> > >> >>>>>> Green team city would be very helpful. > >> >>>>>> > >> >>>>>> > >> >>>>>>> If you are a newcomer, please share your feeling: are you > >> >> confused > >> >>> by > >> >>>>>> test > >> >>>>>>> failures on Teamcity? > >> >>>>>> > >> >>>>>> I'm working on my very first ignite issue > >> >>>>>> and it's not easy to understand TC build results. > >> >>>>>> > >> >>>>>> > >> >>>>>> 21.07.2017 00:30, Dmitriy Setrakyan пишет: > >> >>>>>> > >> >>>>>> I think the right question to ask is why do we have RED tests to > >> >>> begin > >> >>>>>>> with. In my view, TC should be green. If there are tests that do > >> >> not > >> >>>>> pass, > >> >>>>>>> we should disable them and file blocker tickets for the next > >> >>> releases. > >> >>>>>>> > >> >>>>>>> D. > >> >>>>>>> > >> >>>>>>> On Thu, Jul 20, 2017 at 1:20 PM, Dmitry Pavlov < > >> >>> [hidden email] > >> >>>>> > >> >>>>>>> wrote: > >> >>>>>>> > >> >>>>>>> Hi Igniters, > >> >>>>>>>> > >> >>>>>>>> We all know that there are a lot of open tasks in Ignite > project. > >> >>> To > >> >>>> be > >> >>>>>>>> able to handle all these tasks, we need to grow our community. > >> >> And > >> >>> to > >> >>>>>>>> achieve that, we should pay special attention to new community > >> >>>> members. > >> >>>>>>>> > >> >>>>>>>> As recent newcomer I’ve faced the following problem: There is > no > >> >>> way > >> >>>> to > >> >>>>>>>> find out by yourself whether your pull request changes are > >> >> correct > >> >>> or > >> >>>>>>>> not. > >> >>>>>>>> Contributor can not distinguish pull request test failures and > >> >>> master > >> >>>>>>>> test > >> >>>>>>>> failures. Experienced Ignite contributor can estimate the > damage, > >> >>> but > >> >>>>> how > >> >>>>>>>> it can be done by new community member? > >> >>>>>>>> > >> >>>>>>>> Once failing test is introduced to master it is multiplexed to > >> >> all > >> >>>>> later > >> >>>>>>>> PRs. As a result, we have tons of failures and suites timeouts, > >> >>> which > >> >>>>>>>> waste > >> >>>>>>>> Teamcity agents time. > >> >>>>>>>> > >> >>>>>>>> I suggest to start new activity - MakeTeamcityGreenAgain. I’m > >> >> sure > >> >>>> that > >> >>>>>>>> there are a lot of experienced developers in the community who > >> >>>>> understand > >> >>>>>>>> issues with tests very quickly. Please contact me if you are > >> >> ready > >> >>> to > >> >>>>>>>> help > >> >>>>>>>> the community to fix current test issues. > >> >>>>>>>> > >> >>>>>>>> I would appreciate any ideas and sharing your vision how we can > >> >>>> achieve > >> >>>>>>>> green teamcity. > >> >>>>>>>> > >> >>>>>>>> If you are a newcomer, please share your feeling: are you > >> >> confused > >> >>> by > >> >>>>>>>> test > >> >>>>>>>> failures on Teamcity? > >> >>>>>>>> > >> >>>>>>>> Sincerely, > >> >>>>>>>> > >> >>>>>>>> Dmitriy Pavlov > >> >>>>>>>> > >> >>>>>>>> > >> >>>>>>> > >> >>>>> > >> >>>> > >> >>>> > >> >>>> > >> >>>> -- > >> >>>> Best Regards, Vyacheslav D. > >> >>>> > >> >>> > >> >> > >> > >> > |
|
|
Forgot to share link to actual investigations list
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&tab=investigations Please feel free to create and assign investigations if you have assumption about causes of failure. вт, 13 февр. 2018 г. в 1:34, Dmitriy Setrakyan <[hidden email]>: > Dmitriy, thanks for pushing this! We have to bring the number of the > failing tests to zero. I hope that the community gets behind this effort. > > D. > > On Mon, Feb 12, 2018 at 8:26 AM, Dmitry Pavlov <[hidden email]> > wrote: > > > Hi Folks, > > > > I want to resurrect this process and start to fix tests. Currently > approx. > > 60-80 tests are failed each run on master. > > > > As one from the first point I suggest to use TC investigations for brief > > research of test failures. > > > > If you can help with research please create and assign investigation on > TC > > (on test fail select dropdown menu, then 'Investigate / Mute' > > then 'Investigated by: me' & 'Resolve: auto/manually'; Runs > > https://ci.ignite.apache.org/viewType.html?buildTypeId= > > IgniteTests24Java8_RunAll&branch_IgniteTests24Java8=%3Cdefault%3E > > ). Also help with existing fixes review/merge would be very appreciated. > > > > Issue creation is still desired, but investigation will help us to > identify > > tests failures which are under research and failures not researched for > > now. > > > > Sincerely, > > Dmitriy Pavlov > > > > вт, 7 нояб. 2017 г. в 0:06, Dmitry Pavlov <[hidden email]>: > > > > > "Ignored" means that the test is disabled from the code, using > annotation > > > @Ignore. Not sure it works for our JUnit3 style Java tests. It is used > > for > > > .NET. > > > "Muted" test is disabled in the interface TeamCity, but test code is > > > actually executed (which is why hanging tests needs to be failed in > > > addition to mute). > > > > > > The fact that the run all now became green means all the tests that > fall > > > at this point are flaky. If it is about 200 non-muted flaky tests > having > > > the probability of passing 98%, then the whole chain will pass only > with > > > the probability 0.98 ^ 200 = 1.7% > > > > > > I suggest continue to identify, fix, or mute such flaky tests to get > > > stable green RunAll for correct code. > > > > > > пн, 6 нояб. 2017 г. в 22:57, Denis Magda <[hidden email]>: > > > > > >> Dmitriy, > > >> > > >> Now it’s time to fix a pretty decent number of muted and ignored > tests. > > >> BTW, what’s the difference between “ignored” and “muted”? I’ve seen > the > > >> former before. > > >> > > >> — > > >> Denis > > >> > > >> > On Nov 5, 2017, at 9:58 AM, Dmitry Pavlov <[hidden email]> > > >> wrote: > > >> > > > >> > Hi Igniters, > > >> > > > >> > I am happy to report that the activity of monitoring and fixing the > > >> tests > > >> > brought the first significant victory! > > >> > > > >> > Today tests launch running all Ignite tests (Ignite 2.0 > Tests->RunAll) > > >> is > > >> > completely green: > > >> https://ci.ignite.apache.org/viewLog.html?buildId=930953 > > >> > > > >> > This is an reason for being pride, but not a reason to stop this > > >> activity. > > >> > Unfortunately, there are still a number of tests with unpredictable > > >> > failures (flaky, 234 tests), and a number of tests have been muted. > > Full > > >> > project scope is available in the article > > >> > > > >> https://cwiki.apache.org/confluence/display/IGNITE/ > > Make+Teamcity+Green+Again > > >> > > > >> > In any case, thank to all of you who helped and continues to help > the > > >> > project to correct the tests and bugs. > > >> > > > >> > Sincerely, > > >> > Pavlov Dmitry > > >> > > > >> > > > >> > пт, 21 июл. 2017 г. в 14:29, Pavel Tupitsyn <[hidden email]>: > > >> > > > >> >>> Green Again > > >> >>> Again > > >> >> As if it ever was green :) > > >> >> > > >> >> Of course +1 on this, let me know if you see any .NET-related > > failures. > > >> >> > > >> >> On Fri, Jul 21, 2017 at 1:06 PM, Николай Ижиков < > > >> [hidden email]> > > >> >> wrote: > > >> >> > > >> >>> Hello, Igniters. > > >> >>> > > >> >>> Also ready to help to #MakeTeamcityGreenAgain ! > > >> >>> > > >> >>> 21 июля 2017 г. 12:56 PM пользователь "Vyacheslav Daradur" < > > >> >>> [hidden email]> написал: > > >> >>> > > >> >>>> Hi guys. > > >> >>>> > > >> >>>> I vote for #MakeTeamcityGreenAgain. :-) > > >> >>>> > > >> >>>> FYI: it had been described and supported previously[1] > > >> >>>> > > >> >>>> After the completion of my current task I will try to help with > > this > > >> >>>> activity. > > >> >>>> > > >> >>>> [1] > > >> >>>> http://apache-ignite-developers.2346864.n4.nabble. > > >> >>>> com/Test-failures-td14353.html > > >> >>>> > > >> >>>> 2017-07-21 12:39 GMT+03:00 Anton Vinogradov <[hidden email]>: > > >> >>>> > > >> >>>>> Nikolay, > > >> >>>>> > > >> >>>>> That's also a big problem for me, as reviewer, to accept changes > > and > > >> >>>> merge > > >> >>>>> them to master. > > >> >>>>> > > >> >>>>> Dmitriy, > > >> >>>>> > > >> >>>>> Currently we have some contributions *from community* blocked by > > red > > >> >>>>> Teamcity. > > >> >>>>> > > >> >>>>> On Fri, Jul 21, 2017 at 12:47 AM, Nikolay Izhikov < > > >> >>>> [hidden email]> > > >> >>>>> wrote: > > >> >>>>> > > >> >>>>>> +1 to Dmitry Pavlov from me. > > >> >>>>>> > > >> >>>>>> Green team city would be very helpful. > > >> >>>>>> > > >> >>>>>> > > >> >>>>>>> If you are a newcomer, please share your feeling: are you > > >> >> confused > > >> >>> by > > >> >>>>>> test > > >> >>>>>>> failures on Teamcity? > > >> >>>>>> > > >> >>>>>> I'm working on my very first ignite issue > > >> >>>>>> and it's not easy to understand TC build results. > > >> >>>>>> > > >> >>>>>> > > >> >>>>>> 21.07.2017 00:30, Dmitriy Setrakyan пишет: > > >> >>>>>> > > >> >>>>>> I think the right question to ask is why do we have RED tests > to > > >> >>> begin > > >> >>>>>>> with. In my view, TC should be green. If there are tests that > do > > >> >> not > > >> >>>>> pass, > > >> >>>>>>> we should disable them and file blocker tickets for the next > > >> >>> releases. > > >> >>>>>>> > > >> >>>>>>> D. > > >> >>>>>>> > > >> >>>>>>> On Thu, Jul 20, 2017 at 1:20 PM, Dmitry Pavlov < > > >> >>> [hidden email] > > >> >>>>> > > >> >>>>>>> wrote: > > >> >>>>>>> > > >> >>>>>>> Hi Igniters, > > >> >>>>>>>> > > >> >>>>>>>> We all know that there are a lot of open tasks in Ignite > > project. > > >> >>> To > > >> >>>> be > > >> >>>>>>>> able to handle all these tasks, we need to grow our > community. > > >> >> And > > >> >>> to > > >> >>>>>>>> achieve that, we should pay special attention to new > community > > >> >>>> members. > > >> >>>>>>>> > > >> >>>>>>>> As recent newcomer I’ve faced the following problem: There is > > no > > >> >>> way > > >> >>>> to > > >> >>>>>>>> find out by yourself whether your pull request changes are > > >> >> correct > > >> >>> or > > >> >>>>>>>> not. > > >> >>>>>>>> Contributor can not distinguish pull request test failures > and > > >> >>> master > > >> >>>>>>>> test > > >> >>>>>>>> failures. Experienced Ignite contributor can estimate the > > damage, > > >> >>> but > > >> >>>>> how > > >> >>>>>>>> it can be done by new community member? > > >> >>>>>>>> > > >> >>>>>>>> Once failing test is introduced to master it is multiplexed > to > > >> >> all > > >> >>>>> later > > >> >>>>>>>> PRs. As a result, we have tons of failures and suites > timeouts, > > >> >>> which > > >> >>>>>>>> waste > > >> >>>>>>>> Teamcity agents time. > > >> >>>>>>>> > > >> >>>>>>>> I suggest to start new activity - MakeTeamcityGreenAgain. I’m > > >> >> sure > > >> >>>> that > > >> >>>>>>>> there are a lot of experienced developers in the community > who > > >> >>>>> understand > > >> >>>>>>>> issues with tests very quickly. Please contact me if you are > > >> >> ready > > >> >>> to > > >> >>>>>>>> help > > >> >>>>>>>> the community to fix current test issues. > > >> >>>>>>>> > > >> >>>>>>>> I would appreciate any ideas and sharing your vision how we > can > > >> >>>> achieve > > >> >>>>>>>> green teamcity. > > >> >>>>>>>> > > >> >>>>>>>> If you are a newcomer, please share your feeling: are you > > >> >> confused > > >> >>> by > > >> >>>>>>>> test > > >> >>>>>>>> failures on Teamcity? > > >> >>>>>>>> > > >> >>>>>>>> Sincerely, > > >> >>>>>>>> > > >> >>>>>>>> Dmitriy Pavlov > > >> >>>>>>>> > > >> >>>>>>>> > > >> >>>>>>> > > >> >>>>> > > >> >>>> > > >> >>>> > > >> >>>> > > >> >>>> -- > > >> >>>> Best Regards, Vyacheslav D. > > >> >>>> > > >> >>> > > >> >> > > >> > > >> > > > |
«
Return to Apache Ignite Developers - Legacy Mail Archive
|
1 view|%1 views
| Free forum by Nabble | Edit this page |