Empty test results for Security Test Suite ran against master branch on Team City

[Shishkov Ilya-2]

Hello, Igniters!

As I see, there are no test results for Security Test Suite run against
'master' branch [1], for 'ignite-2.10' Security tests results are present
as expected [2].
Is this situation correct?

1.
https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Security/5977426?buildTab=log&focusLine=1878&linesState=1579
2.
https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Security/5976111?buildTab=log&focusLine=67783&linesState=1599.68439.68448

[Dmitry Pavlov]

Hi,

I guess it 's not correct since master tests count is 0. No reason to run a suite without tests.

Folks, why it could be the case? I've checked SUITE_NAME and suite in the code and it seems to be mached.

Sincerely,
Dmitriy Pavlov

On 2021/04/22 14:29:37, Shishkov Ilya <[hidden email]> wrote:

> Hello, Igniters!
>
> As I see, there are no test results for Security Test Suite run against
> 'master' branch [1], for 'ignite-2.10' Security tests results are present
> as expected [2].
> Is this situation correct?
>
> 1.
> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Security/5977426?buildTab=log&focusLine=1878&linesState=1579
> 2.
> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Security/5976111?buildTab=log&focusLine=67783&linesState=1599.68439.68448
>

[Dmitry Pavlov]

Up!

Folks, who is owner/expert for this suite?

We have a number of tests not running.

чт, 22 апр. 2021 г., 18:01 Dmitry Pavlov <[hidden email]>:

> Hi,
>
> I guess it 's not correct since master tests count is 0. No reason to run
> a suite without tests.
>
> Folks, why it could be the case? I've checked SUITE_NAME and suite in the
> code and it seems to be mached.
>
> Sincerely,
> Dmitriy Pavlov
>
> On 2021/04/22 14:29:37, Shishkov Ilya <[hidden email]> wrote:
> > Hello, Igniters!
> >
> > As I see, there are no test results for Security Test Suite run against
> > 'master' branch [1], for 'ignite-2.10' Security tests results are present
> > as expected [2].
> > Is this situation correct?
> >
> > 1.
> >
> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Security/5977426?buildTab=log&focusLine=1878&linesState=1579
> > 2.
> >
> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Security/5976111?buildTab=log&focusLine=67783&linesState=1599.68439.68448
> >
>

[Ilya Kasnacheev]

Hello!

Indeed this looks like a regression.

Can you please file a ticket about it against IGNITE jira?

Regards,
--
Ilya Kasnacheev


чт, 22 апр. 2021 г. в 17:36, Shishkov Ilya <[hidden email]>:

> Hello, Igniters!
>
> As I see, there are no test results for Security Test Suite run against
> 'master' branch [1], for 'ignite-2.10' Security tests results are present
> as expected [2].
> Is this situation correct?
>
> 1.
>
> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Security/5977426?buildTab=log&focusLine=1878&linesState=1579
> 2.
>
> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Security/5976111?buildTab=log&focusLine=67783&linesState=1599.68439.68448
>

[Ilya Kasnacheev]

Hello!

It seems that we have broke this suite in our "tests not in suites" commit:

commit 62cc45861c0273286e00eb6fbc510c0bb5589e52
Author: Maksim Timonin <[hidden email]>
Date:   Fri Feb 5 19:10:18 2021 +0300

   IGNITE-13512 Add travis check that tests are not in suites - Fixes
#8367.

   Signed-off-by: Ilya Kasnacheev <[hidden email]>

Regards,
--
Ilya Kasnacheev


ср, 28 апр. 2021 г. в 11:01, Ilya Kasnacheev <[hidden email]>:

> Hello!
>
> Indeed this looks like a regression.
>
> Can you please file a ticket about it against IGNITE jira?
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> чт, 22 апр. 2021 г. в 17:36, Shishkov Ilya <[hidden email]>:
>
>> Hello, Igniters!
>>
>> As I see, there are no test results for Security Test Suite run against
>> 'master' branch [1], for 'ignite-2.10' Security tests results are present
>> as expected [2].
>> Is this situation correct?
>>
>> 1.
>>
>> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Security/5977426?buildTab=log&focusLine=1878&linesState=1579
>> 2.
>>
>> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Security/5976111?buildTab=log&focusLine=67783&linesState=1599.68439.68448
>>
>

[Ilya Kasnacheev]

Hello!

I have filed a ticket https://issues.apache.org/jira/browse/IGNITE-14665

Regards,
--
Ilya Kasnacheev


ср, 28 апр. 2021 г. в 14:10, Ilya Kasnacheev <[hidden email]>:

> Hello!
>
> It seems that we have broke this suite in our "tests not in suites" commit:
>
> commit 62cc45861c0273286e00eb6fbc510c0bb5589e52
> Author: Maksim Timonin <[hidden email]>
> Date:   Fri Feb 5 19:10:18 2021 +0300
>
>    IGNITE-13512 Add travis check that tests are not in suites - Fixes
> #8367.
>
>    Signed-off-by: Ilya Kasnacheev <[hidden email]>
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 28 апр. 2021 г. в 11:01, Ilya Kasnacheev <[hidden email]>:
>
>> Hello!
>>
>> Indeed this looks like a regression.
>>
>> Can you please file a ticket about it against IGNITE jira?
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> чт, 22 апр. 2021 г. в 17:36, Shishkov Ilya <[hidden email]>:
>>
>>> Hello, Igniters!
>>>
>>> As I see, there are no test results for Security Test Suite run against
>>> 'master' branch [1], for 'ignite-2.10' Security tests results are present
>>> as expected [2].
>>> Is this situation correct?
>>>
>>> 1.
>>>
>>> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Security/5977426?buildTab=log&focusLine=1878&linesState=1579
>>> 2.
>>>
>>> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Security/5976111?buildTab=log&focusLine=67783&linesState=1599.68439.68448
>>>
>>

[Maksim Timonin]

Hi! I've started to investigate the issue. Try to back soon.

On Wed, Apr 28, 2021 at 4:41 PM Ilya Kasnacheev <[hidden email]>
wrote:

> Hello!
>
> I have filed a ticket https://issues.apache.org/jira/browse/IGNITE-14665
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 28 апр. 2021 г. в 14:10, Ilya Kasnacheev <[hidden email]>:
>
>> Hello!
>>
>> It seems that we have broke this suite in our "tests not in suites"
>> commit:
>>
>> commit 62cc45861c0273286e00eb6fbc510c0bb5589e52
>> Author: Maksim Timonin <[hidden email]>
>> Date:   Fri Feb 5 19:10:18 2021 +0300
>>
>>    IGNITE-13512 Add travis check that tests are not in suites - Fixes
>> #8367.
>>
>>    Signed-off-by: Ilya Kasnacheev <[hidden email]>
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> ср, 28 апр. 2021 г. в 11:01, Ilya Kasnacheev <[hidden email]>:
>>
>>> Hello!
>>>
>>> Indeed this looks like a regression.
>>>
>>> Can you please file a ticket about it against IGNITE jira?
>>>
>>> Regards,
>>> --
>>> Ilya Kasnacheev
>>>
>>>
>>> чт, 22 апр. 2021 г. в 17:36, Shishkov Ilya <[hidden email]>:
>>>
>>>> Hello, Igniters!
>>>>
>>>> As I see, there are no test results for Security Test Suite run against
>>>> 'master' branch [1], for 'ignite-2.10' Security tests results are
>>>> present
>>>> as expected [2].
>>>> Is this situation correct?
>>>>
>>>> 1.
>>>>
>>>> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Security/5977426?buildTab=log&focusLine=1878&linesState=1579
>>>> 2.
>>>>
>>>> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Security/5976111?buildTab=log&focusLine=67783&linesState=1599.68439.68448
>>>>
>>>

[Maksim Timonin]

Hi, I've found a bug, the security suite runs with my fix, check TC [1].

The reason is the wrong maven dependency version for the profile
"surefire-fork-count-1". I fixed this and triggered MTCGA RunAll. In case
any other TC suite uses this profile. So after it is finished I'll ask for
review.

Actually, I see that TC runs with the setting failIfNoTests=false. WDYT if
we change it to true? It looks weird if a TC test suite doesn't run any
test and then says that everything is OK.


[1]
https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Security/5985778

On Wed, Apr 28, 2021 at 5:21 PM Maksim Timonin <[hidden email]>
wrote:

> Hi! I've started to investigate the issue. Try to back soon.
>
> On Wed, Apr 28, 2021 at 4:41 PM Ilya Kasnacheev <[hidden email]>
> wrote:
>
>> Hello!
>>
>> I have filed a ticket https://issues.apache.org/jira/browse/IGNITE-14665
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> ср, 28 апр. 2021 г. в 14:10, Ilya Kasnacheev <[hidden email]>:
>>
>>> Hello!
>>>
>>> It seems that we have broke this suite in our "tests not in suites"
>>> commit:
>>>
>>> commit 62cc45861c0273286e00eb6fbc510c0bb5589e52
>>> Author: Maksim Timonin <[hidden email]>
>>> Date:   Fri Feb 5 19:10:18 2021 +0300
>>>
>>>    IGNITE-13512 Add travis check that tests are not in suites - Fixes
>>> #8367.
>>>
>>>    Signed-off-by: Ilya Kasnacheev <[hidden email]>
>>>
>>> Regards,
>>> --
>>> Ilya Kasnacheev
>>>
>>>
>>> ср, 28 апр. 2021 г. в 11:01, Ilya Kasnacheev <[hidden email]
>>> >:
>>>
>>>> Hello!
>>>>
>>>> Indeed this looks like a regression.
>>>>
>>>> Can you please file a ticket about it against IGNITE jira?
>>>>
>>>> Regards,
>>>> --
>>>> Ilya Kasnacheev
>>>>
>>>>
>>>> чт, 22 апр. 2021 г. в 17:36, Shishkov Ilya <[hidden email]>:
>>>>
>>>>> Hello, Igniters!
>>>>>
>>>>> As I see, there are no test results for Security Test Suite run against
>>>>> 'master' branch [1], for 'ignite-2.10' Security tests results are
>>>>> present
>>>>> as expected [2].
>>>>> Is this situation correct?
>>>>>
>>>>> 1.
>>>>>
>>>>> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Security/5977426?buildTab=log&focusLine=1878&linesState=1579
>>>>> 2.
>>>>>
>>>>> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Security/5976111?buildTab=log&focusLine=67783&linesState=1599.68439.68448
>>>>>
>>>>

[Ilya Kasnacheev]

Hello!

This is usually done since there is more than one module in the project and
all of them may fail due to failIfNoTests=true.

But you may re-check what happens if you run the same without
this parameter.
Please mention me on the ticket for quick review.

Regards,
--
Ilya Kasnacheev


ср, 28 апр. 2021 г. в 18:36, Maksim Timonin <[hidden email]>:

> Hi, I've found a bug, the security suite runs with my fix, check TC [1].
>
> The reason is the wrong maven dependency version for the profile
> "surefire-fork-count-1". I fixed this and triggered MTCGA RunAll. In case
> any other TC suite uses this profile. So after it is finished I'll ask for
> review.
>
> Actually, I see that TC runs with the setting failIfNoTests=false. WDYT if
> we change it to true? It looks weird if a TC test suite doesn't run any
> test and then says that everything is OK.
>
>
> [1]
> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Security/5985778
>
> On Wed, Apr 28, 2021 at 5:21 PM Maksim Timonin <[hidden email]>
> wrote:
>
>> Hi! I've started to investigate the issue. Try to back soon.
>>
>> On Wed, Apr 28, 2021 at 4:41 PM Ilya Kasnacheev <
>> [hidden email]> wrote:
>>
>>> Hello!
>>>
>>> I have filed a ticket https://issues.apache.org/jira/browse/IGNITE-14665
>>>
>>> Regards,
>>> --
>>> Ilya Kasnacheev
>>>
>>>
>>> ср, 28 апр. 2021 г. в 14:10, Ilya Kasnacheev <[hidden email]
>>> >:
>>>
>>>> Hello!
>>>>
>>>> It seems that we have broke this suite in our "tests not in suites"
>>>> commit:
>>>>
>>>> commit 62cc45861c0273286e00eb6fbc510c0bb5589e52
>>>> Author: Maksim Timonin <[hidden email]>
>>>> Date:   Fri Feb 5 19:10:18 2021 +0300
>>>>
>>>>    IGNITE-13512 Add travis check that tests are not in suites - Fixes
>>>> #8367.
>>>>
>>>>    Signed-off-by: Ilya Kasnacheev <[hidden email]>
>>>>
>>>> Regards,
>>>> --
>>>> Ilya Kasnacheev
>>>>
>>>>
>>>> ср, 28 апр. 2021 г. в 11:01, Ilya Kasnacheev <[hidden email]
>>>> >:
>>>>
>>>>> Hello!
>>>>>
>>>>> Indeed this looks like a regression.
>>>>>
>>>>> Can you please file a ticket about it against IGNITE jira?
>>>>>
>>>>> Regards,
>>>>> --
>>>>> Ilya Kasnacheev
>>>>>
>>>>>
>>>>> чт, 22 апр. 2021 г. в 17:36, Shishkov Ilya <[hidden email]>:
>>>>>
>>>>>> Hello, Igniters!
>>>>>>
>>>>>> As I see, there are no test results for Security Test Suite run
>>>>>> against
>>>>>> 'master' branch [1], for 'ignite-2.10' Security tests results are
>>>>>> present
>>>>>> as expected [2].
>>>>>> Is this situation correct?
>>>>>>
>>>>>> 1.
>>>>>>
>>>>>> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Security/5977426?buildTab=log&focusLine=1878&linesState=1579
>>>>>> 2.
>>>>>>
>>>>>> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Security/5976111?buildTab=log&focusLine=67783&linesState=1599.68439.68448
>>>>>>
>>>>>