Re: Wrong backup size with persistence + data expriation

[Ilya Kasnacheev]

Hello!

Indeed I can observe this problem:

Baseline: [Server1, Server2, Server3]
10000 records inserted
2019-02-26T16:50:00.603 :: [0] all=20000, primary=10000, backup=10000,
query=10000
2019-02-26T16:50:10.692 :: [1] all=20000, primary=10000, backup=10000,
query=10000
2019-02-26T16:50:20.750 :: [2] all=20000, primary=10000, backup=10000,
query=10000
2019-02-26T16:50:30.775 :: [3] all=20000, primary=10000, backup=10000,
query=10000
2019-02-26T16:50:40.851 :: [4] all=18564, primary=9285, backup=9262,
query=9122
2019-02-26T16:50:50.897 :: [5] all=10143, primary=5073, backup=5061,
query=4752
2019-02-26T16:51:00.925 :: [6] all=173, primary=86, backup=72, query=0
2019-02-26T16:51:10.934 :: [7] all=9, primary=0, backup=9, query=0
2019-02-26T16:51:20.952 :: [8] all=9, primary=0, backup=9, query=0
2019-02-26T16:51:30.964 :: [9] all=9, primary=0, backup=9, query=0
2019-02-26T16:51:40.976 :: [10] all=9, primary=0, backup=9, query=0
2019-02-26T16:51:50.988 :: [11] all=9, primary=0, backup=9, query=0
2019-02-26T16:52:01.015 :: [12] all=9, primary=0, backup=9, query=0
2019-02-26T16:52:11.040 :: [13] all=9, primary=0, backup=9, query=0
2019-02-26T16:52:21.053 :: [14] all=9, primary=0, backup=9, query=0
2019-02-26T16:52:31.072 :: [15] all=9, primary=0, backup=9, query=0
[16:52:35] Ignite node stopped OK [uptime=00:03:02.734]

As far as my understanding goes, all entries should expire and both primary
and backup be reduced to zero, but it does not happen.

Can anyone comment this? Should a ticket be filed?

Regards,
--
Ilya Kasnacheev


пт, 8 февр. 2019 г. в 22:53, Ruslan Kamashev <[hidden email]>:

> Hi,
>
> I try to use data expiration with persistence, but I got wrong backup size
> after cache records are expired.
> Also sometimes I got result when expired records didn't deleted but I
> can't reproduce this case again.
>
> Could you share more details about using data expiration with persistence?
>
> Source: https://github.com/rynffoll/ignite-tests
> Logs:
> https://github.com/rynffoll/ignite-tests/tree/master/logs/persistance_with_data_expiration
>
>

[dmagda]

Evgeniy, Igniters,

Is this somehow related to the TTL issue we observed with other use cases
recently?

Ruslan, please share your configuration.

-
Denis


On Tue, Feb 26, 2019 at 5:56 AM Ilya Kasnacheev <[hidden email]>
wrote:

> Hello!
>
> Indeed I can observe this problem:
>
> Baseline: [Server1, Server2, Server3]
> 10000 records inserted
> 2019-02-26T16:50:00.603 :: [0] all=20000, primary=10000, backup=10000,
> query=10000
> 2019-02-26T16:50:10.692 :: [1] all=20000, primary=10000, backup=10000,
> query=10000
> 2019-02-26T16:50:20.750 :: [2] all=20000, primary=10000, backup=10000,
> query=10000
> 2019-02-26T16:50:30.775 :: [3] all=20000, primary=10000, backup=10000,
> query=10000
> 2019-02-26T16:50:40.851 :: [4] all=18564, primary=9285, backup=9262,
> query=9122
> 2019-02-26T16:50:50.897 :: [5] all=10143, primary=5073, backup=5061,
> query=4752
> 2019-02-26T16:51:00.925 :: [6] all=173, primary=86, backup=72, query=0
> 2019-02-26T16:51:10.934 :: [7] all=9, primary=0, backup=9, query=0
> 2019-02-26T16:51:20.952 :: [8] all=9, primary=0, backup=9, query=0
> 2019-02-26T16:51:30.964 :: [9] all=9, primary=0, backup=9, query=0
> 2019-02-26T16:51:40.976 :: [10] all=9, primary=0, backup=9, query=0
> 2019-02-26T16:51:50.988 :: [11] all=9, primary=0, backup=9, query=0
> 2019-02-26T16:52:01.015 :: [12] all=9, primary=0, backup=9, query=0
> 2019-02-26T16:52:11.040 :: [13] all=9, primary=0, backup=9, query=0
> 2019-02-26T16:52:21.053 :: [14] all=9, primary=0, backup=9, query=0
> 2019-02-26T16:52:31.072 :: [15] all=9, primary=0, backup=9, query=0
> [16:52:35] Ignite node stopped OK [uptime=00:03:02.734]
>
> As far as my understanding goes, all entries should expire and both primary
> and backup be reduced to zero, but it does not happen.
>
> Can anyone comment this? Should a ticket be filed?
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пт, 8 февр. 2019 г. в 22:53, Ruslan Kamashev <[hidden email]>:
>
> > Hi,
> >
> > I try to use data expiration with persistence, but I got wrong backup
> size
> > after cache records are expired.
> > Also sometimes I got result when expired records didn't deleted but I
> > can't reproduce this case again.
> >
> > Could you share more details about using data expiration with
> persistence?
> >
> > Source: https://github.com/rynffoll/ignite-tests
> > Logs:
> >
> https://github.com/rynffoll/ignite-tests/tree/master/logs/persistance_with_data_expiration
> >
> >
>

[slava.koptilin]

Hi Denis, Ruslan,

It looks like we found out the cause of that issue
https://issues.apache.org/jira/browse/IGNITE-11438.
Implementation details can be found here:
https://github.com/apache/ignite/pull/6197
I hope that the fix will be merged into master soon.

Thanks,
S.

ср, 27 февр. 2019 г. в 01:48, Denis Magda <[hidden email]>:

> Evgeniy, Igniters,
>
> Is this somehow related to the TTL issue we observed with other use cases
> recently?
>
> Ruslan, please share your configuration.
>
> -
> Denis
>
>
> On Tue, Feb 26, 2019 at 5:56 AM Ilya Kasnacheev <[hidden email]
> >
> wrote:
>
> > Hello!
> >
> > Indeed I can observe this problem:
> >
> > Baseline: [Server1, Server2, Server3]
> > 10000 records inserted
> > 2019-02-26T16:50:00.603 :: [0] all=20000, primary=10000, backup=10000,
> > query=10000
> > 2019-02-26T16:50:10.692 :: [1] all=20000, primary=10000, backup=10000,
> > query=10000
> > 2019-02-26T16:50:20.750 :: [2] all=20000, primary=10000, backup=10000,
> > query=10000
> > 2019-02-26T16:50:30.775 :: [3] all=20000, primary=10000, backup=10000,
> > query=10000
> > 2019-02-26T16:50:40.851 :: [4] all=18564, primary=9285, backup=9262,
> > query=9122
> > 2019-02-26T16:50:50.897 :: [5] all=10143, primary=5073, backup=5061,
> > query=4752
> > 2019-02-26T16:51:00.925 :: [6] all=173, primary=86, backup=72, query=0
> > 2019-02-26T16:51:10.934 :: [7] all=9, primary=0, backup=9, query=0
> > 2019-02-26T16:51:20.952 :: [8] all=9, primary=0, backup=9, query=0
> > 2019-02-26T16:51:30.964 :: [9] all=9, primary=0, backup=9, query=0
> > 2019-02-26T16:51:40.976 :: [10] all=9, primary=0, backup=9, query=0
> > 2019-02-26T16:51:50.988 :: [11] all=9, primary=0, backup=9, query=0
> > 2019-02-26T16:52:01.015 :: [12] all=9, primary=0, backup=9, query=0
> > 2019-02-26T16:52:11.040 :: [13] all=9, primary=0, backup=9, query=0
> > 2019-02-26T16:52:21.053 :: [14] all=9, primary=0, backup=9, query=0
> > 2019-02-26T16:52:31.072 :: [15] all=9, primary=0, backup=9, query=0
> > [16:52:35] Ignite node stopped OK [uptime=00:03:02.734]
> >
> > As far as my understanding goes, all entries should expire and both
> primary
> > and backup be reduced to zero, but it does not happen.
> >
> > Can anyone comment this? Should a ticket be filed?
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > пт, 8 февр. 2019 г. в 22:53, Ruslan Kamashev <[hidden email]>:
> >
> > > Hi,
> > >
> > > I try to use data expiration with persistence, but I got wrong backup
> > size
> > > after cache records are expired.
> > > Also sometimes I got result when expired records didn't deleted but I
> > > can't reproduce this case again.
> > >
> > > Could you share more details about using data expiration with
> > persistence?
> > >
> > > Source: https://github.com/rynffoll/ignite-tests
> > > Logs:
> > >
> >
> https://github.com/rynffoll/ignite-tests/tree/master/logs/persistance_with_data_expiration
> > >
> > >
> >
>