Tests coverage report of Ignite 2.5 core module

[daradurvs]

Hi, Igniters!

I’ve made tests coverage report of core module using Jacoco 0.8.1

I’d like to share results with the community. Here is a link:
https://www.dropbox.com/s/rdgs1svvojm757x/ignite-2.5-core-module-tests-coverage-report-rev-d83f1ec.zip?dl=0

For viewing the report unpack the archive and open: coverage-report/index.html

Also, there is ‘tests-classes.txt’ which includes names of tests
classes which have been used.

The report was generated from branch: ‘ignite-2-5’, commit:
https://github.com/apache/ignite/tree/d83f1ec0a4d010e57fecc12ffd3d1f8346ded61c


--
Best Regards, Vyacheslav D.

[Dmitriy Pavlov]

Hi Vyacheslav,

https://www.dropbox.com/s/rdgs1svvojm757x/ignite-2.5-core-module-tests-coverage-report-rev-d83f1ec.zip?dl=0&file_subpath=%2Fcoverage-report%2Findex.html

says error 400 for me.

Sincerely,
Dmitriy Pavlov

пн, 14 мая 2018 г. в 17:06, Vyacheslav Daradur <[hidden email]>:

> Hi, Igniters!
>
> I’ve made tests coverage report of core module using Jacoco 0.8.1
>
> I’d like to share results with the community. Here is a link:
>
> https://www.dropbox.com/s/rdgs1svvojm757x/ignite-2.5-core-module-tests-coverage-report-rev-d83f1ec.zip?dl=0
>
> For viewing the report unpack the archive and open:
> coverage-report/index.html
>
> Also, there is ‘tests-classes.txt’ which includes names of tests
> classes which have been used.
>
> The report was generated from branch: ‘ignite-2-5’, commit:
>
> https://github.com/apache/ignite/tree/d83f1ec0a4d010e57fecc12ffd3d1f8346ded61c
>
>
> --
> Best Regards, Vyacheslav D.
>

[daradurvs]

Hi, Dmitry,

Download and unpack the archive first:
https://www.dropbox.com/s/rdgs1svvojm757x/ignite-2.5-core-module-tests-coverage-report-rev-d83f1ec.zip


On Mon, May 14, 2018 at 5:12 PM, Dmitry Pavlov <[hidden email]> wrote:

> Hi Vyacheslav,
>
> https://www.dropbox.com/s/rdgs1svvojm757x/ignite-2.5-core-module-tests-coverage-report-rev-d83f1ec.zip?dl=0&file_subpath=%2Fcoverage-report%2Findex.html
>
> says error 400 for me.
>
> Sincerely,
> Dmitriy Pavlov
>
> пн, 14 мая 2018 г. в 17:06, Vyacheslav Daradur <[hidden email]>:
>
>> Hi, Igniters!
>>
>> I’ve made tests coverage report of core module using Jacoco 0.8.1
>>
>> I’d like to share results with the community. Here is a link:
>>
>> https://www.dropbox.com/s/rdgs1svvojm757x/ignite-2.5-core-module-tests-coverage-report-rev-d83f1ec.zip?dl=0
>>
>> For viewing the report unpack the archive and open:
>> coverage-report/index.html
>>
>> Also, there is ‘tests-classes.txt’ which includes names of tests
>> classes which have been used.
>>
>> The report was generated from branch: ‘ignite-2-5’, commit:
>>
>> https://github.com/apache/ignite/tree/d83f1ec0a4d010e57fecc12ffd3d1f8346ded61c
>>
>>
>> --
>> Best Regards, Vyacheslav D.
>>

--
Best Regards, Vyacheslav D.

[Dmitriy Pavlov]

Ok, thank you. I missed that I should download the archive first.

I've found for example that method truncate in FSyncWalManager is not
covered.

Which test suites were executed?

пн, 14 мая 2018 г. в 17:16, Vyacheslav Daradur <[hidden email]>:

> Hi, Dmitry,
>
> Download and unpack the archive first:
>
> https://www.dropbox.com/s/rdgs1svvojm757x/ignite-2.5-core-module-tests-coverage-report-rev-d83f1ec.zip
>
>
> On Mon, May 14, 2018 at 5:12 PM, Dmitry Pavlov <[hidden email]>
> wrote:
> > Hi Vyacheslav,
> >
> >
> https://www.dropbox.com/s/rdgs1svvojm757x/ignite-2.5-core-module-tests-coverage-report-rev-d83f1ec.zip?dl=0&file_subpath=%2Fcoverage-report%2Findex.html
> >
> > says error 400 for me.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > пн, 14 мая 2018 г. в 17:06, Vyacheslav Daradur <[hidden email]>:
> >
> >> Hi, Igniters!
> >>
> >> I’ve made tests coverage report of core module using Jacoco 0.8.1
> >>
> >> I’d like to share results with the community. Here is a link:
> >>
> >>
> https://www.dropbox.com/s/rdgs1svvojm757x/ignite-2.5-core-module-tests-coverage-report-rev-d83f1ec.zip?dl=0
> >>
> >> For viewing the report unpack the archive and open:
> >> coverage-report/index.html
> >>
> >> Also, there is ‘tests-classes.txt’ which includes names of tests
> >> classes which have been used.
> >>
> >> The report was generated from branch: ‘ignite-2-5’, commit:
> >>
> >>
> https://github.com/apache/ignite/tree/d83f1ec0a4d010e57fecc12ffd3d1f8346ded61c
> >>
> >>
> >> --
> >> Best Regards, Vyacheslav D.
> >>
>
>
>
> --
> Best Regards, Vyacheslav D.
>

[daradurvs]

TestSuites weren't used.

Set of tests classes was created as result of finding not abstract
classes which names contain '*Test*' except '*TestCase*' in
'/src/test/java/org/apache/ignite' as Maven does [1].

Each tests class was executed and covered separately.

You can see used tests classes in the file 'tests-classes.txt' which
was included in the archive.


[1] http://maven.apache.org/surefire/maven-surefire-plugin/examples/inclusion-exclusion.html



On Mon, May 14, 2018 at 5:30 PM, Dmitry Pavlov <[hidden email]> wrote:

> Ok, thank you. I missed that I should download the archive first.
>
> I've found for example that method truncate in FSyncWalManager is not
> covered.
>
> Which test suites were executed?
>
> пн, 14 мая 2018 г. в 17:16, Vyacheslav Daradur <[hidden email]>:
>
>> Hi, Dmitry,
>>
>> Download and unpack the archive first:
>>
>> https://www.dropbox.com/s/rdgs1svvojm757x/ignite-2.5-core-module-tests-coverage-report-rev-d83f1ec.zip
>>
>>
>> On Mon, May 14, 2018 at 5:12 PM, Dmitry Pavlov <[hidden email]>
>> wrote:
>> > Hi Vyacheslav,
>> >
>> >
>> https://www.dropbox.com/s/rdgs1svvojm757x/ignite-2.5-core-module-tests-coverage-report-rev-d83f1ec.zip?dl=0&file_subpath=%2Fcoverage-report%2Findex.html
>> >
>> > says error 400 for me.
>> >
>> > Sincerely,
>> > Dmitriy Pavlov
>> >
>> > пн, 14 мая 2018 г. в 17:06, Vyacheslav Daradur <[hidden email]>:
>> >
>> >> Hi, Igniters!
>> >>
>> >> I’ve made tests coverage report of core module using Jacoco 0.8.1
>> >>
>> >> I’d like to share results with the community. Here is a link:
>> >>
>> >>
>> https://www.dropbox.com/s/rdgs1svvojm757x/ignite-2.5-core-module-tests-coverage-report-rev-d83f1ec.zip?dl=0
>> >>
>> >> For viewing the report unpack the archive and open:
>> >> coverage-report/index.html
>> >>
>> >> Also, there is ‘tests-classes.txt’ which includes names of tests
>> >> classes which have been used.
>> >>
>> >> The report was generated from branch: ‘ignite-2-5’, commit:
>> >>
>> >>
>> https://github.com/apache/ignite/tree/d83f1ec0a4d010e57fecc12ffd3d1f8346ded61c
>> >>
>> >>
>> >> --
>> >> Best Regards, Vyacheslav D.
>> >>
>>
>>
>>
>> --
>> Best Regards, Vyacheslav D.
>>

--
Best Regards, Vyacheslav D.

[Александр Меньшиков]

Hi Vyacheslav,
Great job!

I specifically happy to see jacoco.csv file into the archive.
I think I can range classes by some complex criterion, like
coverage/complexity.
That classes could be the first in the queue for testing.

2018-05-14 17:49 GMT+03:00 Vyacheslav Daradur <[hidden email]>:

> TestSuites weren't used.
>
> Set of tests classes was created as result of finding not abstract
> classes which names contain '*Test*' except '*TestCase*' in
> '/src/test/java/org/apache/ignite' as Maven does [1].
>
> Each tests class was executed and covered separately.
>
> You can see used tests classes in the file 'tests-classes.txt' which
> was included in the archive.
>
>
> [1] http://maven.apache.org/surefire/maven-surefire-
> plugin/examples/inclusion-exclusion.html
>
>
>
> On Mon, May 14, 2018 at 5:30 PM, Dmitry Pavlov <[hidden email]>
> wrote:
> > Ok, thank you. I missed that I should download the archive first.
> >
> > I've found for example that method truncate in FSyncWalManager is not
> > covered.
> >
> > Which test suites were executed?
> >
> > пн, 14 мая 2018 г. в 17:16, Vyacheslav Daradur <[hidden email]>:
> >
> >> Hi, Dmitry,
> >>
> >> Download and unpack the archive first:
> >>
> >> https://www.dropbox.com/s/rdgs1svvojm757x/ignite-2.5-
> core-module-tests-coverage-report-rev-d83f1ec.zip
> >>
> >>
> >> On Mon, May 14, 2018 at 5:12 PM, Dmitry Pavlov <[hidden email]>
> >> wrote:
> >> > Hi Vyacheslav,
> >> >
> >> >
> >> https://www.dropbox.com/s/rdgs1svvojm757x/ignite-2.5-
> core-module-tests-coverage-report-rev-d83f1ec.zip?dl=0&
> file_subpath=%2Fcoverage-report%2Findex.html
> >> >
> >> > says error 400 for me.
> >> >
> >> > Sincerely,
> >> > Dmitriy Pavlov
> >> >
> >> > пн, 14 мая 2018 г. в 17:06, Vyacheslav Daradur <[hidden email]>:
> >> >
> >> >> Hi, Igniters!
> >> >>
> >> >> I’ve made tests coverage report of core module using Jacoco 0.8.1
> >> >>
> >> >> I’d like to share results with the community. Here is a link:
> >> >>
> >> >>
> >> https://www.dropbox.com/s/rdgs1svvojm757x/ignite-2.5-
> core-module-tests-coverage-report-rev-d83f1ec.zip?dl=0
> >> >>
> >> >> For viewing the report unpack the archive and open:
> >> >> coverage-report/index.html
> >> >>
> >> >> Also, there is ‘tests-classes.txt’ which includes names of tests
> >> >> classes which have been used.
> >> >>
> >> >> The report was generated from branch: ‘ignite-2-5’, commit:
> >> >>
> >> >>
> >> https://github.com/apache/ignite/tree/d83f1ec0a4d010e57fecc12ffd3d1f
> 8346ded61c
> >> >>
> >> >>
> >> >> --
> >> >> Best Regards, Vyacheslav D.
> >> >>
> >>
> >>
> >>
> >> --
> >> Best Regards, Vyacheslav D.
> >>
>
>
>
> --
> Best Regards, Vyacheslav D.
>

[Александр Меньшиков]

Vyacheslav,
I have made a simple ranking for your data for highlighting classes which
more than other needs testing.

Please take a look at the list which I have done (36 classes):
https://drive.google.com/file/d/1RJrtnN77MeEWLTShFzqaLInCdOSb9WdC/view?usp=sharing


Below you can read how did it in details:

First I have removed all small classes (less than 200 lines), classes with
more than 80% coverage, and with 0% (too strange result).
After that, I have extracted subset which dominates other classes by two
parameters: coverage and complexity.
It means all other classes have not less coverage and not greater
complexity, and also better at least at one of these parameters.

Finally, I have sorted this list by value "coverage/complexity".

[Dmitriy Pavlov]

Hi Alexander,

I did not quite understand how this list of top tests was obtained. It does
not look like sorted.

Can you clarify?

Sincerely,
Dmitriy Pavlov

пн, 14 мая 2018 г. в 20:48, Александр Меньшиков <[hidden email]>:

> Vyacheslav,
> I have made a simple ranking for your data for highlighting classes which
> more than other needs testing.
>
> Please take a look at the list which I have done (36 classes):
>
> https://drive.google.com/file/d/1RJrtnN77MeEWLTShFzqaLInCdOSb9WdC/view?usp=sharing
>
>
> Below you can read how did it in details:
>
> First I have removed all small classes (less than 200 lines), classes with
> more than 80% coverage, and with 0% (too strange result).
> After that, I have extracted subset which dominates other classes by two
> parameters: coverage and complexity.
> It means all other classes have not less coverage and not greater
> complexity, and also better at least at one of these parameters.
>
> Finally, I have sorted this list by value "coverage/complexity".
>

[Александр Меньшиков]

Hi Dmitry,
You are right this is not a sorting, this is a ranking by two parameters
where both of them equal in priority.
For such task in math, we have an idea of "dominating subset".
It simply too understand by example:

      P1, P2
E1  2, 2
E2  1, 2
E3  2, 1
E4  1, 1

Here element E1 dominate all other elements. E2 and E3 dominate E4, and E4
is a loser.
So we can do a ranking like that: E1 has rank 1, E2 and E3 have rank 2, and
E4 has rank 3.

I have done the same with JoCoCo results.
Where less coverage is better, and greate complexity is better (because we
are looking for classes with the problems).
In my list, there are all classes with rank 1 (except well covered, small,
and with a strange result of 0% coverage).

Finally, I have sorted this list by one value "coverage/complexity" -- just
for better viewing.

I have written R script for this, and if you think about some other
parameters or filtering -- let me know.


2018-05-14 20:54 GMT+03:00 Dmitry Pavlov <[hidden email]>:

> Hi Alexander,
>
> I did not quite understand how this list of top tests was obtained. It does
> not look like sorted.
>
> Can you clarify?
>
> Sincerely,
> Dmitriy Pavlov
>
> пн, 14 мая 2018 г. в 20:48, Александр Меньшиков <[hidden email]>:
>
> > Vyacheslav,
> > I have made a simple ranking for your data for highlighting classes which
> > more than other needs testing.
> >
> > Please take a look at the list which I have done (36 classes):
> >
> > https://drive.google.com/file/d/1RJrtnN77MeEWLTShFzqaLInCdOSb9
> WdC/view?usp=sharing
> >
> >
> > Below you can read how did it in details:
> >
> > First I have removed all small classes (less than 200 lines), classes
> with
> > more than 80% coverage, and with 0% (too strange result).
> > After that, I have extracted subset which dominates other classes by two
> > parameters: coverage and complexity.
> > It means all other classes have not less coverage and not greater
> > complexity, and also better at least at one of these parameters.
> >
> > Finally, I have sorted this list by value "coverage/complexity".
> >
>