Test naming on TC JUnit 5

[Ivan Fedotov]

Hi, igniters!

In the context of IEP-30 and IGNITE-10958 in particular [1], the JUnit5
tests are going to appear in master.

During migration Ignite on JUnit5 I faced that the test naming will be
changed.
The names are different, in the version before path consists from "path to
suite: path to test" [2] and in the version after just "path to test" [3].
I want to inform you that it will lead to obsolete current history for some
tests: see tests with label "History for base branch is absent" [4]. It
means that TC bot shows those tests as blockers because of a different
name, but indeed tests were red on JUnit 4 version also. After some time in
the new version of JUnit we will get a history for mentioned tests again.

Do have some of us have questions/comments according to migration?

[1] https://issues.apache.org/jira/browse/IGNITE-10973
[2]
https://ci.ignite.apache.org/project.html?tab=testDetails&branch_IgniteTests24Java8=%3Cdefault%3E&projectId=IgniteTests24Java8&testNameId=680055112352967935&page=1
[3]
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=8375541702052870683&branch=%3Cdefault%3E&tab=testDetails&branch_IgniteTests24Java8=__all_branches__
[4]
https://mtcga.gridgain.com/pr.html?serverId=apache&suiteId=IgniteTests24Java8_RunAll&branchForTc=pull/6606/head&action=Latest
--
Ivan Fedotov.

[hidden email]

[Dmitry Pavlov]

Hi Ivan.

Yes, test name changes (reported by TC), would be a short term issue, but
TC bot will accumulate statistics quite fast.

We can consider applying changes before a weekend, so the maximum number of
re-runs occur; so the maximum statistic is the outcome.

Moreover, the test name change may have a positive effect in long-term,
because of a re-run of a particular suite on TC has exactly the same impact
on test name. It will be a path to test case wihtout suite.
So for now, Re-run possible blockers activate the whole suite, but when all
tests change their names, TC bot could activate only some test case(s) with
a test(s) failure to rerun. It could also bring faster detection of master
failures.

Sincerely,
Dmitriy Pavlov

пн, 22 июл. 2019 г. в 20:10, Ivan Fedotov <[hidden email]>:

> Hi, igniters!
>
> In the context of IEP-30 and IGNITE-10958 in particular [1], the JUnit5
> tests are going to appear in master.
>
> During migration Ignite on JUnit5 I faced that the test naming will be
> changed.
> The names are different, in the version before path consists from "path to
> suite: path to test" [2] and in the version after just "path to test" [3].
> I want to inform you that it will lead to obsolete current history for some
> tests: see tests with label "History for base branch is absent" [4]. It
> means that TC bot shows those tests as blockers because of a different
> name, but indeed tests were red on JUnit 4 version also. After some time in
> the new version of JUnit we will get a history for mentioned tests again.
>
> Do have some of us have questions/comments according to migration?
>
> [1] https://issues.apache.org/jira/browse/IGNITE-10973
> [2]
>
> https://ci.ignite.apache.org/project.html?tab=testDetails&branch_IgniteTests24Java8=%3Cdefault%3E&projectId=IgniteTests24Java8&testNameId=680055112352967935&page=1
> [3]
>
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=8375541702052870683&branch=%3Cdefault%3E&tab=testDetails&branch_IgniteTests24Java8=__all_branches__
> [4]
>
> https://mtcga.gridgain.com/pr.html?serverId=apache&suiteId=IgniteTests24Java8_RunAll&branchForTc=pull/6606/head&action=Latest
> --
> Ivan Fedotov.
>
> [hidden email]
>

[Ivan Fedotov]

Hi Dmitriy.

Thank you for your reply, re-running concrete test cases, not a suite, is
another one utility of JUnit 5.

I want to pay your (and other Igniters) attention on .net tests. I faced
with failure "The filename or extension is too long" in ExecutableTest [1].
The problem is in jvmClasspath because other args are hardcoded. After
comparison log files from my branch [2] and master [3] I found that
jvmClasspath variable contains all maven dependencies. I added more
dependencies to parent/pom file and mentioned string becomes too long for
system .net function.

I think that the problem is in Classpath.cs#CreateClasspath method: if we
want to add more dependencies to Ignite project (not necessary JUnit, any
dependency) it will lead to the same problem. Furthermore, jvmClasspath
variable contains an enumeration of all jar files.

Is it correct behavior of CreateClasspath method or we need to change its
implementation?

[1]
https://github.com/apache/ignite/blob/master/modules/platforms/dotnet/Apache.Ignite.Core.Tests/ExecutableTest.cs#L148
[2]
https://ci.ignite.apache.org/viewLog.html?buildId=4318302&tab=buildResultsDiv&buildTypeId=IgniteTests24Java8_PlatformNetLongRunning
[3]
https://ci.ignite.apache.org/viewLog.html?buildId=4394312&buildTypeId=IgniteTests24Java8_PlatformNetLongRunning&tab=testsInfo


чт, 25 июл. 2019 г. в 10:11, Dmitriy Pavlov <[hidden email]>:

> Hi Ivan.
>
> Yes, test name changes (reported by TC), would be a short term issue, but
> TC bot will accumulate statistics quite fast.
>
> We can consider applying changes before a weekend, so the maximum number of
> re-runs occur; so the maximum statistic is the outcome.
>
> Moreover, the test name change may have a positive effect in long-term,
> because of a re-run of a particular suite on TC has exactly the same impact
> on test name. It will be a path to test case wihtout suite.
> So for now, Re-run possible blockers activate the whole suite, but when all
> tests change their names, TC bot could activate only some test case(s) with
> a test(s) failure to rerun. It could also bring faster detection of master
> failures.
>
> Sincerely,
> Dmitriy Pavlov
>
> пн, 22 июл. 2019 г. в 20:10, Ivan Fedotov <[hidden email]>:
>
> > Hi, igniters!
> >
> > In the context of IEP-30 and IGNITE-10958 in particular [1], the JUnit5
> > tests are going to appear in master.
> >
> > During migration Ignite on JUnit5 I faced that the test naming will be
> > changed.
> > The names are different, in the version before path consists from "path
> to
> > suite: path to test" [2] and in the version after just "path to test"
> [3].
> > I want to inform you that it will lead to obsolete current history for
> some
> > tests: see tests with label "History for base branch is absent" [4]. It
> > means that TC bot shows those tests as blockers because of a different
> > name, but indeed tests were red on JUnit 4 version also. After some time
> in
> > the new version of JUnit we will get a history for mentioned tests again.
> >
> > Do have some of us have questions/comments according to migration?
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-10973
> > [2]
> >
> >
> https://ci.ignite.apache.org/project.html?tab=testDetails&branch_IgniteTests24Java8=%3Cdefault%3E&projectId=IgniteTests24Java8&testNameId=680055112352967935&page=1
> > [3]
> >
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=8375541702052870683&branch=%3Cdefault%3E&tab=testDetails&branch_IgniteTests24Java8=__all_branches__
> > [4]
> >
> >
> https://mtcga.gridgain.com/pr.html?serverId=apache&suiteId=IgniteTests24Java8_RunAll&branchForTc=pull/6606/head&action=Latest
> > --
> > Ivan Fedotov.
> >
> > [hidden email]
> >
>

--
Ivan Fedotov.

[hidden email]

[Ivan Fedotov]

Also, according to IGNITE-10973 [1] (in nutshell: maven dependencies +
migration Examples module), we are ready for JUnit 5.

What is a strategy for massive migration?

[1] https://issues.apache.org/jira/browse/IGNITE-10973

чт, 25 июл. 2019 г. в 14:33, Ivan Fedotov <[hidden email]>:

> Hi Dmitriy.
>
> Thank you for your reply, re-running concrete test cases, not a suite, is
> another one utility of JUnit 5.
>
> I want to pay your (and other Igniters) attention on .net tests. I faced
> with failure "The filename or extension is too long" in ExecutableTest [1].
> The problem is in jvmClasspath because other args are hardcoded. After
> comparison log files from my branch [2] and master [3] I found that
> jvmClasspath variable contains all maven dependencies. I added more
> dependencies to parent/pom file and mentioned string becomes too long for
> system .net function.
>
> I think that the problem is in Classpath.cs#CreateClasspath method: if we
> want to add more dependencies to Ignite project (not necessary JUnit, any
> dependency) it will lead to the same problem. Furthermore, jvmClasspath
> variable contains an enumeration of all jar files.
>
> Is it correct behavior of CreateClasspath method or we need to change its
> implementation?
>
> [1]
> https://github.com/apache/ignite/blob/master/modules/platforms/dotnet/Apache.Ignite.Core.Tests/ExecutableTest.cs#L148
> [2]
> https://ci.ignite.apache.org/viewLog.html?buildId=4318302&tab=buildResultsDiv&buildTypeId=IgniteTests24Java8_PlatformNetLongRunning
> [3]
> https://ci.ignite.apache.org/viewLog.html?buildId=4394312&buildTypeId=IgniteTests24Java8_PlatformNetLongRunning&tab=testsInfo
>
>
> чт, 25 июл. 2019 г. в 10:11, Dmitriy Pavlov <[hidden email]>:
>
>> Hi Ivan.
>>
>> Yes, test name changes (reported by TC), would be a short term issue, but
>> TC bot will accumulate statistics quite fast.
>>
>> We can consider applying changes before a weekend, so the maximum number
>> of
>> re-runs occur; so the maximum statistic is the outcome.
>>
>> Moreover, the test name change may have a positive effect in long-term,
>> because of a re-run of a particular suite on TC has exactly the same
>> impact
>> on test name. It will be a path to test case wihtout suite.
>> So for now, Re-run possible blockers activate the whole suite, but when
>> all
>> tests change their names, TC bot could activate only some test case(s)
>> with
>> a test(s) failure to rerun. It could also bring faster detection of master
>> failures.
>>
>> Sincerely,
>> Dmitriy Pavlov
>>
>> пн, 22 июл. 2019 г. в 20:10, Ivan Fedotov <[hidden email]>:
>>
>> > Hi, igniters!
>> >
>> > In the context of IEP-30 and IGNITE-10958 in particular [1], the JUnit5
>> > tests are going to appear in master.
>> >
>> > During migration Ignite on JUnit5 I faced that the test naming will be
>> > changed.
>> > The names are different, in the version before path consists from "path
>> to
>> > suite: path to test" [2] and in the version after just "path to test"
>> [3].
>> > I want to inform you that it will lead to obsolete current history for
>> some
>> > tests: see tests with label "History for base branch is absent" [4]. It
>> > means that TC bot shows those tests as blockers because of a different
>> > name, but indeed tests were red on JUnit 4 version also. After some
>> time in
>> > the new version of JUnit we will get a history for mentioned tests
>> again.
>> >
>> > Do have some of us have questions/comments according to migration?
>> >
>> > [1] https://issues.apache.org/jira/browse/IGNITE-10973
>> > [2]
>> >
>> >
>> https://ci.ignite.apache.org/project.html?tab=testDetails&branch_IgniteTests24Java8=%3Cdefault%3E&projectId=IgniteTests24Java8&testNameId=680055112352967935&page=1
>> > [3]
>> >
>> >
>> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=8375541702052870683&branch=%3Cdefault%3E&tab=testDetails&branch_IgniteTests24Java8=__all_branches__
>> > [4]
>> >
>> >
>> https://mtcga.gridgain.com/pr.html?serverId=apache&suiteId=IgniteTests24Java8_RunAll&branchForTc=pull/6606/head&action=Latest
>> > --
>> > Ivan Fedotov.
>> >
>> > [hidden email]
>> >
>>
>
>
> --
> Ivan Fedotov.
>
> [hidden email]
>

--
Ivan Fedotov.

[hidden email]