Ignite master: OOMEs and test failure rate raised

[Dmitriy Pavlov]

Hi Igniters,

I've noticed in latest run we have ~ 985 test failures and 3 suites with
OOME in master.

Who knows if it is temporary or infra problem or we have some bug in code?
Could you please share ticket numbers?

https://ci.ignite.apache.org/viewLog.html?buildId=1429088
https://ci.ignite.apache.org/viewLog.html?buildId=1429108
https://ci.ignite.apache.org/viewLog.html?buildId=1428437

Sincerely,
Dmitriy Pavlov

[Andrew Mashenkov]

Hi,
I think we have to check if all PDS tests has limited region sizes and
checkpoint buffer sizes.

Also, seems, there is a leakage when using scan query. I'll file a ticket
with details tomorrow.

ср, 27 июн. 2018 г., 18:55 Dmitry Pavlov <[hidden email]>:

> Hi Igniters,
>
> I've noticed in latest run we have ~ 985 test failures and 3 suites with
> OOME in master.
>
> Who knows if it is temporary or infra problem or we have some bug in code?
> Could you please share ticket numbers?
>
> https://ci.ignite.apache.org/viewLog.html?buildId=1429088
> https://ci.ignite.apache.org/viewLog.html?buildId=1429108
> https://ci.ignite.apache.org/viewLog.html?buildId=1428437
>
> Sincerely,
> Dmitriy Pavlov
>

[vveider]

Some OOMEs seems like infrastructure problem, one of them has failed due to 4Gb limit to artifact size.
I think we should reconsider log and work artifacts collection after build and try to shrink working logs even more.

Dmitriy, can you aggregate such problems at least for one week and create ticket for investigation?

> On 27 Jun 2018, at 18:55, Dmitry Pavlov <[hidden email]> wrote:
>
> Hi Igniters,
>
> I've noticed in latest run we have ~ 985 test failures and 3 suites with
> OOME in master.
>
> Who knows if it is temporary or infra problem or we have some bug in code?
> Could you please share ticket numbers?
>
> https://ci.ignite.apache.org/viewLog.html?buildId=1429088
> https://ci.ignite.apache.org/viewLog.html?buildId=1429108
> https://ci.ignite.apache.org/viewLog.html?buildId=1428437
>
> Sincerely,
> Dmitriy Pavlov