Re: Cache expiry policy not deleting records from disk(native persistence)
Locked
 
|
Shiva,
Hopefully, someone from the dev community will pick this ticket up soon and solve the task. In the meantime, Artem, would you mind documenting this limitation referring to ticket 10862? - Denis On Tue, Sep 10, 2019 at 12:50 AM Shiva Kumar <[hidden email]> wrote: > I have filed a bug https://issues.apache.org/jira/browse/IGNITE-12152 but > this is same as https://issues.apache.org/jira/browse/IGNITE-10862 > Any idea on the timeline of these tickets? > In the documentation > https://apacheignite.readme.io/v2.7/docs/expiry-policies > it says when native persistence is enabled "*expired entries are removed > from both memory and disk tiers*" but in the disk it just mark the pages > as unwanted pages and same disk space used by these unwanted pages will be > used to store new pages but it will not remove unwanted pages from disk and > so it will not release disk space used by these unwanted pages. > > here is the developer's discussion link > > http://apache-ignite-developers.2346864.n4.nabble.com/How-to-free-up-space-on-disc-after-removing-entries-from-IgniteCache-with-enabled-PDS-td39839.html > > > On Mon, Sep 9, 2019 at 11:53 PM Shiva Kumar <[hidden email]> > wrote: > >> Hi >> I have deployed ignite on kubernetes and configured two seperate >> persistent volume for WAL and persistence. >> The issue Iam facing is same as >> https://issues.apache.org/jira/browse/IGNITE-10862 >> >> Thanks >> Shiva >> >> On Mon, 9 Sep, 2019, 10:47 PM Andrei Aleksandrov, < >> [hidden email]> wrote: >> >>> Hello, >>> >>> I guess that generated WAL will take this disk space. Please read about >>> WAL here: >>> >>> https://apacheignite.readme.io/docs/write-ahead-log >>> >>> Please provide the size of every folder under /opt/ignite/persistence. >>> >>> BR, >>> Andrei >>> 9/6/2019 9:45 PM, Shiva Kumar пишет: >>> >>> Hi all, >>> I have set cache expiry policy like this >>> >>> >>> </property> >>> <property name="cacheConfiguration"> >>> <list> >>> <bean id="cache-template-bean" abstract="true" >>> class="org.apache.ignite.configuration.CacheConfiguration"> >>> <property name="name" value="templateEternal*"/> >>> <property name="cacheMode" value="PARTITIONED"/> >>> <property name="backups" value="1"/> >>> <property name="groupName" value="groupEternal"/> >>> <property name="expiryPolicyFactory"> >>> <bean class="javax.cache.expiry.CreatedExpiryPolicy" >>> factory-method="factoryOf"> >>> <constructor-arg> >>> <bean class="javax.cache.expiry.Duration"> >>> <constructor-arg value="MINUTES"/> >>> <constructor-arg value="10"/> >>> </bean> >>> </constructor-arg> >>> </bean> >>> </property> >>> >>> </bean> >>> </list> >>> </property> >>> >>> >>> And batch inserting records to one of the table which is created with >>> above cache template. >>> Around 10 minutes, I ingested ~1.5GB of data and after 10 minutes >>> records started reducing(expiring) when I monitored from sqlline. >>> >>> 0: jdbc:ignite:thin://192.168.*.*:10800> select count(ID) from >>> DIMENSIONS; >>> -------------------------------- >>> >>> COUNT(ID) >>> -------------------------------- >>> >>> 248896 >>> -------------------------------- >>> 1 row selected (0.86 seconds) >>> 0: jdbc:ignite:thin://192.168.*.*:10800> select count(ID) from >>> DIMENSIONS; >>> -------------------------------- >>> >>> COUNT(ID) >>> -------------------------------- >>> >>> 222174 >>> -------------------------------- >>> 1 row selected (0.313 seconds) >>> 0: jdbc:ignite:thin://192.168.*.*:10800> select count(ID) from >>> DIMENSIONS; >>> -------------------------------- >>> >>> COUNT(ID) >>> -------------------------------- >>> >>> 118154 >>> -------------------------------- >>> 1 row selected (0.15 seconds) >>> 0: jdbc:ignite:thin://192.168.*.*:10800> >>> 0: jdbc:ignite:thin://192.168.*.*:10800> select count(ID) from >>> DIMENSIONS; >>> -------------------------------- >>> >>> COUNT(ID) >>> -------------------------------- >>> >>> 76061 >>> -------------------------------- >>> 1 row selected (0.106 seconds) >>> 0: jdbc:ignite:thin://192.168.*.*:10800> >>> 0: jdbc:ignite:thin://192.168.*.*:10800> select count(ID) from >>> DIMENSIONS; >>> -------------------------------- >>> >>> COUNT(ID) >>> -------------------------------- >>> >>> 41671 >>> -------------------------------- >>> 1 row selected (0.063 seconds) >>> 0: jdbc:ignite:thin://192.168.*.*:10800> select count(ID) from >>> DIMENSIONS; >>> -------------------------------- >>> >>> COUNT(ID) >>> -------------------------------- >>> >>> 18455 >>> -------------------------------- >>> 1 row selected (0.037 seconds) >>> 0: jdbc:ignite:thin://192.168.*.*:10800> select count(ID) from >>> DIMENSIONS; >>> -------------------------------- >>> >>> COUNT(ID) >>> -------------------------------- >>> >>> 0 >>> -------------------------------- >>> 1 row selected (0.014 seconds) >>> >>> >>> But in the meantime, the disk space used by the persistence store was in >>> the same usage level instead of decreasing. >>> >>> >>> [ignite@ignite-cluster-ign-shiv-0 ignite]$ while true ; do df -h >>> /opt/ignite/persistence/; sleep 1s; done >>> Filesystem Size Used Avail Use% Mounted on >>> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >>> Filesystem Size Used Avail Use% Mounted on >>> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >>> Filesystem Size Used Avail Use% Mounted on >>> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >>> Filesystem Size Used Avail Use% Mounted on >>> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >>> Filesystem Size Used Avail Use% Mounted on >>> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >>> Filesystem Size Used Avail Use% Mounted on >>> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >>> Filesystem Size Used Avail Use% Mounted on >>> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >>> >>> >>> >>> This means that expiry policy not deleting records from the disk, but >>> ignite document says when expiry policy is set and native persistence is >>> enabled then it deletes records from disk as well. >>> Am I missing some configuration? >>> Any help is appreciated. >>> >>> Shiva >>> >>> |
Re: Cache expiry policy not deleting records from disk(native persistence)
|
|
Hi Denis,
I updated the page about eviction policies. Not freeing up space on disk was not implemented for reasons explained in the dev-list thread. I'll update the page once more if/when a solution is implemented. Artem On 13.09.2019 00:34, Denis Magda wrote: > Shiva, > > Hopefully, someone from the dev community will pick this ticket up > soon and solve the task. In the meantime, Artem, would you mind > documenting this limitation referring to ticket 10862? > > - > Denis > > > On Tue, Sep 10, 2019 at 12:50 AM Shiva Kumar <[hidden email] > <mailto:[hidden email]>> wrote: > > I have filed a bug > https://issues.apache.org/jira/browse/IGNITE-12152 but this is > same as https://issues.apache.org/jira/browse/IGNITE-10862 > Any idea on the timeline of these tickets? > In the documentation > https://apacheignite.readme.io/v2.7/docs/expiry-policies > it says when native persistence is enabled "*expired entries are > removed from both memory and disk tiers*" but in the disk it just > mark the pages as unwanted pages and same disk space used by these > unwanted pages will be used to store new pages but it will not > remove unwanted pages from disk and so it will not release disk > space used by these unwanted pages. > > here is the developer's discussion link > http://apache-ignite-developers.2346864.n4.nabble.com/How-to-free-up-space-on-disc-after-removing-entries-from-IgniteCache-with-enabled-PDS-td39839.html > > > On Mon, Sep 9, 2019 at 11:53 PM Shiva Kumar > <[hidden email] <mailto:[hidden email]>> wrote: > > Hi > I have deployed ignite on kubernetes and configured two > seperate persistent volume for WAL and persistence. > The issue Iam facing is same as > https://issues.apache.org/jira/browse/IGNITE-10862 > > Thanks > Shiva > > On Mon, 9 Sep, 2019, 10:47 PM Andrei Aleksandrov, > <[hidden email] <mailto:[hidden email]>> wrote: > > Hello, > > I guess that generated WAL will take this disk space. > Please read about WAL here: > > https://apacheignite.readme.io/docs/write-ahead-log > > Please provide the size of every folder under > /opt/ignite/persistence. > > BR, > Andrei > > 9/6/2019 9:45 PM, Shiva Kumar пишет: >> Hi all, >> I have set cache expiry policy like this >> >> >> </property> >> <property name="cacheConfiguration"> >> <list> >> <bean id="cache-template-bean" >> abstract="true" >> class="org.apache.ignite.configuration.CacheConfiguration"> >> <property name="name" >> value="templateEternal*"/> >> <property name="cacheMode" >> value="PARTITIONED"/> >> <property name="backups" value="1"/> >> <property name="groupName" >> value="groupEternal"/> >> <property name="expiryPolicyFactory"> >> <bean >> class="javax.cache.expiry.CreatedExpiryPolicy" >> factory-method="factoryOf"> >> <constructor-arg> >> <bean >> class="javax.cache.expiry.Duration"> >> <constructor-arg value="MINUTES"/> >> <constructor-arg value="10"/> >> </bean> >> </constructor-arg> >> </bean> >> </property> >> >> </bean> >> </list> >> </property> >> >> >> And batch inserting records to one of the table which is >> created with above cache template. >> Around 10 minutes, I ingested ~1.5GB of data and after 10 >> minutes records started reducing(expiring) when I >> monitored from sqlline. >> >> 0: jdbc:ignite:thin://192.168.*.*:10800> select count(ID) >> from DIMENSIONS; >> -------------------------------- >> >> COUNT(ID) >> -------------------------------- >> >> 248896 >> -------------------------------- >> 1 row selected (0.86 seconds) >> 0: jdbc:ignite:thin://192.168.*.*:10800> select count(ID) >> from DIMENSIONS; >> -------------------------------- >> >> COUNT(ID) >> -------------------------------- >> >> 222174 >> -------------------------------- >> 1 row selected (0.313 seconds) >> 0: jdbc:ignite:thin://192.168.*.*:10800> select count(ID) >> from DIMENSIONS; >> -------------------------------- >> >> COUNT(ID) >> -------------------------------- >> >> 118154 >> -------------------------------- >> 1 row selected (0.15 seconds) >> 0: jdbc:ignite:thin://192.168.*.*:10800> >> 0: jdbc:ignite:thin://192.168.*.*:10800> select count(ID) >> from DIMENSIONS; >> -------------------------------- >> >> COUNT(ID) >> -------------------------------- >> >> 76061 >> -------------------------------- >> 1 row selected (0.106 seconds) >> 0: jdbc:ignite:thin://192.168.*.*:10800> >> 0: jdbc:ignite:thin://192.168.*.*:10800> select count(ID) >> from DIMENSIONS; >> -------------------------------- >> >> COUNT(ID) >> -------------------------------- >> >> 41671 >> -------------------------------- >> 1 row selected (0.063 seconds) >> 0: jdbc:ignite:thin://192.168.*.*:10800> select count(ID) >> from DIMENSIONS; >> -------------------------------- >> >> COUNT(ID) >> -------------------------------- >> >> 18455 >> -------------------------------- >> 1 row selected (0.037 seconds) >> 0: jdbc:ignite:thin://192.168.*.*:10800> select count(ID) >> from DIMENSIONS; >> -------------------------------- >> >> COUNT(ID) >> -------------------------------- >> >> 0 >> -------------------------------- >> 1 row selected (0.014 seconds) >> >> >> But in the meantime, the disk space used by the >> persistence store was in the same usage level instead of >> decreasing. >> >> >> [ignite@ignite-cluster-ign-shiv-0 ignite]$ while true ; >> do df -h /opt/ignite/persistence/; sleep 1s; done >> Filesystem Size Used Avail Use% Mounted on >> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >> Filesystem Size Used Avail Use% Mounted on >> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >> Filesystem Size Used Avail Use% Mounted on >> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >> Filesystem Size Used Avail Use% Mounted on >> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >> Filesystem Size Used Avail Use% Mounted on >> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >> Filesystem Size Used Avail Use% Mounted on >> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >> Filesystem Size Used Avail Use% Mounted on >> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >> >> >> >> This means that expiry policy not deleting records from >> the disk, but ignite document says when expiry policy is >> set and native persistence is enabled then it deletes >> records from disk as well. >> Am I missing some configuration? >> Any help is appreciated. >> >> Shiva > |
|
|
Artem, thanks, could you please share a reference to the updated page?
Can't find anything here: https://apacheignite.readme.io/docs/evictions Shiva, I've restarted the discussion on the dev list, to get to the bottom of this gap and how it can be addressed: http://apache-ignite-developers.2346864.n4.nabble.com/How-to-free-up-space-on-disc-after-removing-entries-from-IgniteCache-with-enabled-PDS-td39839.html - Denis On Fri, Sep 13, 2019 at 7:05 AM Artem Budnikov <[hidden email]> wrote: > Hi Denis, > > I updated the page about eviction policies. Not freeing up space on disk > was not implemented for reasons explained in the dev-list thread. I'll > update the page once more if/when a solution is implemented. > > Artem > On 13.09.2019 00:34, Denis Magda wrote: > > Shiva, > > Hopefully, someone from the dev community will pick this ticket up soon > and solve the task. In the meantime, Artem, would you mind documenting this > limitation referring to ticket 10862? > > - > Denis > > > On Tue, Sep 10, 2019 at 12:50 AM Shiva Kumar <[hidden email]> > wrote: > >> I have filed a bug https://issues.apache.org/jira/browse/IGNITE-12152 but >> this is same as https://issues.apache.org/jira/browse/IGNITE-10862 >> Any idea on the timeline of these tickets? >> In the documentation >> https://apacheignite.readme.io/v2.7/docs/expiry-policies >> it says when native persistence is enabled "*expired entries are removed >> from both memory and disk tiers*" but in the disk it just mark the pages >> as unwanted pages and same disk space used by these unwanted pages will be >> used to store new pages but it will not remove unwanted pages from disk and >> so it will not release disk space used by these unwanted pages. >> >> here is the developer's discussion link >> >> http://apache-ignite-developers.2346864.n4.nabble.com/How-to-free-up-space-on-disc-after-removing-entries-from-IgniteCache-with-enabled-PDS-td39839.html >> >> >> On Mon, Sep 9, 2019 at 11:53 PM Shiva Kumar <[hidden email]> >> wrote: >> >>> Hi >>> I have deployed ignite on kubernetes and configured two seperate >>> persistent volume for WAL and persistence. >>> The issue Iam facing is same as >>> https://issues.apache.org/jira/browse/IGNITE-10862 >>> >>> Thanks >>> Shiva >>> >>> On Mon, 9 Sep, 2019, 10:47 PM Andrei Aleksandrov, < >>> [hidden email]> wrote: >>> >>>> Hello, >>>> >>>> I guess that generated WAL will take this disk space. Please read about >>>> WAL here: >>>> >>>> https://apacheignite.readme.io/docs/write-ahead-log >>>> >>>> Please provide the size of every folder under /opt/ignite/persistence. >>>> >>>> BR, >>>> Andrei >>>> 9/6/2019 9:45 PM, Shiva Kumar пишет: >>>> >>>> Hi all, >>>> I have set cache expiry policy like this >>>> >>>> >>>> </property> >>>> <property name="cacheConfiguration"> >>>> <list> >>>> <bean id="cache-template-bean" abstract="true" >>>> class="org.apache.ignite.configuration.CacheConfiguration"> >>>> <property name="name" value="templateEternal*"/> >>>> <property name="cacheMode" value="PARTITIONED"/> >>>> <property name="backups" value="1"/> >>>> <property name="groupName" value="groupEternal"/> >>>> <property name="expiryPolicyFactory"> >>>> <bean >>>> class="javax.cache.expiry.CreatedExpiryPolicy" factory-method="factoryOf"> >>>> <constructor-arg> >>>> <bean class="javax.cache.expiry.Duration"> >>>> <constructor-arg value="MINUTES"/> >>>> <constructor-arg value="10"/> >>>> </bean> >>>> </constructor-arg> >>>> </bean> >>>> </property> >>>> >>>> </bean> >>>> </list> >>>> </property> >>>> >>>> >>>> And batch inserting records to one of the table which is created with >>>> above cache template. >>>> Around 10 minutes, I ingested ~1.5GB of data and after 10 minutes >>>> records started reducing(expiring) when I monitored from sqlline. >>>> >>>> 0: jdbc:ignite:thin://192.168.*.*:10800> select count(ID) from >>>> DIMENSIONS; >>>> -------------------------------- >>>> >>>> COUNT(ID) >>>> -------------------------------- >>>> >>>> 248896 >>>> -------------------------------- >>>> 1 row selected (0.86 seconds) >>>> 0: jdbc:ignite:thin://192.168.*.*:10800> select count(ID) from >>>> DIMENSIONS; >>>> -------------------------------- >>>> >>>> COUNT(ID) >>>> -------------------------------- >>>> >>>> 222174 >>>> -------------------------------- >>>> 1 row selected (0.313 seconds) >>>> 0: jdbc:ignite:thin://192.168.*.*:10800> select count(ID) from >>>> DIMENSIONS; >>>> -------------------------------- >>>> >>>> COUNT(ID) >>>> -------------------------------- >>>> >>>> 118154 >>>> -------------------------------- >>>> 1 row selected (0.15 seconds) >>>> 0: jdbc:ignite:thin://192.168.*.*:10800> >>>> 0: jdbc:ignite:thin://192.168.*.*:10800> select count(ID) from >>>> DIMENSIONS; >>>> -------------------------------- >>>> >>>> COUNT(ID) >>>> -------------------------------- >>>> >>>> 76061 >>>> -------------------------------- >>>> 1 row selected (0.106 seconds) >>>> 0: jdbc:ignite:thin://192.168.*.*:10800> >>>> 0: jdbc:ignite:thin://192.168.*.*:10800> select count(ID) from >>>> DIMENSIONS; >>>> -------------------------------- >>>> >>>> COUNT(ID) >>>> -------------------------------- >>>> >>>> 41671 >>>> -------------------------------- >>>> 1 row selected (0.063 seconds) >>>> 0: jdbc:ignite:thin://192.168.*.*:10800> select count(ID) from >>>> DIMENSIONS; >>>> -------------------------------- >>>> >>>> COUNT(ID) >>>> -------------------------------- >>>> >>>> 18455 >>>> -------------------------------- >>>> 1 row selected (0.037 seconds) >>>> 0: jdbc:ignite:thin://192.168.*.*:10800> select count(ID) from >>>> DIMENSIONS; >>>> -------------------------------- >>>> >>>> COUNT(ID) >>>> -------------------------------- >>>> >>>> 0 >>>> -------------------------------- >>>> 1 row selected (0.014 seconds) >>>> >>>> >>>> But in the meantime, the disk space used by the persistence store was >>>> in the same usage level instead of decreasing. >>>> >>>> >>>> [ignite@ignite-cluster-ign-shiv-0 ignite]$ while true ; do df -h >>>> /opt/ignite/persistence/; sleep 1s; done >>>> Filesystem Size Used Avail Use% Mounted on >>>> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >>>> Filesystem Size Used Avail Use% Mounted on >>>> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >>>> Filesystem Size Used Avail Use% Mounted on >>>> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >>>> Filesystem Size Used Avail Use% Mounted on >>>> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >>>> Filesystem Size Used Avail Use% Mounted on >>>> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >>>> Filesystem Size Used Avail Use% Mounted on >>>> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >>>> Filesystem Size Used Avail Use% Mounted on >>>> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >>>> >>>> >>>> >>>> This means that expiry policy not deleting records from the disk, but >>>> ignite document says when expiry policy is set and native persistence is >>>> enabled then it deletes records from disk as well. >>>> Am I missing some configuration? >>>> Any help is appreciated. >>>> >>>> Shiva >>>> >>>> |
Re: Cache expiry policy not deleting records from disk(native persistence)
|
|
Hi Denis,
That's on the expiry policies page: https://apacheignite.readme.io/docs/expiry-policies On 13.09.2019 19:46, Denis Magda wrote: > Artem, thanks, could you please share a reference to the updated page? > Can't find anything here: > https://apacheignite.readme.io/docs/evictions > > Shiva, I've restarted the discussion on the dev list, to get to the > bottom of this gap and how it can be addressed: > http://apache-ignite-developers.2346864.n4.nabble.com/How-to-free-up-space-on-disc-after-removing-entries-from-IgniteCache-with-enabled-PDS-td39839.html > > - > Denis > > > On Fri, Sep 13, 2019 at 7:05 AM Artem Budnikov > <[hidden email] <mailto:[hidden email]>> wrote: > > Hi Denis, > > I updated the page about eviction policies. Not freeing up space > on disk was not implemented for reasons explained in the dev-list > thread. I'll update the page once more if/when a solution is > implemented. > > Artem > > On 13.09.2019 00:34, Denis Magda wrote: >> Shiva, >> >> Hopefully, someone from the dev community will pick this ticket >> up soon and solve the task. In the meantime, Artem, would you >> mind documenting this limitation referring to ticket 10862? >> >> - >> Denis >> >> >> On Tue, Sep 10, 2019 at 12:50 AM Shiva Kumar >> <[hidden email] <mailto:[hidden email]>> wrote: >> >> I have filed a bug >> https://issues.apache.org/jira/browse/IGNITE-12152 but this >> is same as https://issues.apache.org/jira/browse/IGNITE-10862 >> Any idea on the timeline of these tickets? >> In the documentation >> https://apacheignite.readme.io/v2.7/docs/expiry-policies >> it says when native persistence is enabled "*expired entries >> are removed from both memory and disk tiers*" but in the disk >> it just mark the pages as unwanted pages and same disk space >> used by these unwanted pages will be used to store new pages >> but it will not remove unwanted pages from disk and so it >> will not release disk space used by these unwanted pages. >> >> here is the developer's discussion link >> http://apache-ignite-developers.2346864.n4.nabble.com/How-to-free-up-space-on-disc-after-removing-entries-from-IgniteCache-with-enabled-PDS-td39839.html >> >> >> On Mon, Sep 9, 2019 at 11:53 PM Shiva Kumar >> <[hidden email] <mailto:[hidden email]>> >> wrote: >> >> Hi >> I have deployed ignite on kubernetes and configured two >> seperate persistent volume for WAL and persistence. >> The issue Iam facing is same as >> https://issues.apache.org/jira/browse/IGNITE-10862 >> >> Thanks >> Shiva >> >> On Mon, 9 Sep, 2019, 10:47 PM Andrei Aleksandrov, >> <[hidden email] >> <mailto:[hidden email]>> wrote: >> >> Hello, >> >> I guess that generated WAL will take this disk space. >> Please read about WAL here: >> >> https://apacheignite.readme.io/docs/write-ahead-log >> >> Please provide the size of every folder under >> /opt/ignite/persistence. >> >> BR, >> Andrei >> >> 9/6/2019 9:45 PM, Shiva Kumar пишет: >>> Hi all, >>> I have set cache expiry policy like this >>> >>> >>> </property> >>> <property name="cacheConfiguration"> >>> <list> >>> <bean id="cache-template-bean" >>> abstract="true" >>> class="org.apache.ignite.configuration.CacheConfiguration"> >>> <property name="name" >>> value="templateEternal*"/> >>> <property name="cacheMode" >>> value="PARTITIONED"/> >>> <property name="backups" value="1"/> >>> <property name="groupName" >>> value="groupEternal"/> >>> <property name="expiryPolicyFactory"> >>> <bean >>> class="javax.cache.expiry.CreatedExpiryPolicy" >>> factory-method="factoryOf"> >>> <constructor-arg> >>> <bean >>> class="javax.cache.expiry.Duration"> >>> <constructor-arg value="MINUTES"/> >>> <constructor-arg value="10"/> >>> </bean> >>> </constructor-arg> >>> </bean> >>> </property> >>> >>> </bean> >>> </list> >>> </property> >>> >>> >>> And batch inserting records to one of the table >>> which is created with above cache template. >>> Around 10 minutes, I ingested ~1.5GB of data and >>> after 10 minutes records started reducing(expiring) >>> when I monitored from sqlline. >>> >>> 0: jdbc:ignite:thin://192.168.*.*:10800> select >>> count(ID) from DIMENSIONS; >>> -------------------------------- >>> >>> COUNT(ID) >>> -------------------------------- >>> >>> 248896 >>> -------------------------------- >>> 1 row selected (0.86 seconds) >>> 0: jdbc:ignite:thin://192.168.*.*:10800> select >>> count(ID) from DIMENSIONS; >>> -------------------------------- >>> >>> COUNT(ID) >>> -------------------------------- >>> >>> 222174 >>> -------------------------------- >>> 1 row selected (0.313 seconds) >>> 0: jdbc:ignite:thin://192.168.*.*:10800> select >>> count(ID) from DIMENSIONS; >>> -------------------------------- >>> >>> COUNT(ID) >>> -------------------------------- >>> >>> 118154 >>> -------------------------------- >>> 1 row selected (0.15 seconds) >>> 0: jdbc:ignite:thin://192.168.*.*:10800> >>> 0: jdbc:ignite:thin://192.168.*.*:10800> select >>> count(ID) from DIMENSIONS; >>> -------------------------------- >>> >>> COUNT(ID) >>> -------------------------------- >>> >>> 76061 >>> -------------------------------- >>> 1 row selected (0.106 seconds) >>> 0: jdbc:ignite:thin://192.168.*.*:10800> >>> 0: jdbc:ignite:thin://192.168.*.*:10800> select >>> count(ID) from DIMENSIONS; >>> -------------------------------- >>> >>> COUNT(ID) >>> -------------------------------- >>> >>> 41671 >>> -------------------------------- >>> 1 row selected (0.063 seconds) >>> 0: jdbc:ignite:thin://192.168.*.*:10800> select >>> count(ID) from DIMENSIONS; >>> -------------------------------- >>> >>> COUNT(ID) >>> -------------------------------- >>> >>> 18455 >>> -------------------------------- >>> 1 row selected (0.037 seconds) >>> 0: jdbc:ignite:thin://192.168.*.*:10800> select >>> count(ID) from DIMENSIONS; >>> -------------------------------- >>> >>> COUNT(ID) >>> -------------------------------- >>> >>> 0 >>> -------------------------------- >>> 1 row selected (0.014 seconds) >>> >>> >>> But in the meantime, the disk space used by the >>> persistence store was in the same usage level >>> instead of decreasing. >>> >>> >>> [ignite@ignite-cluster-ign-shiv-0 ignite]$ while >>> true ; do df -h /opt/ignite/persistence/; sleep 1s; done >>> Filesystem Size Used Avail Use% Mounted on >>> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >>> Filesystem Size Used Avail Use% Mounted on >>> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >>> Filesystem Size Used Avail Use% Mounted on >>> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >>> Filesystem Size Used Avail Use% Mounted on >>> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >>> Filesystem Size Used Avail Use% Mounted on >>> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >>> Filesystem Size Used Avail Use% Mounted on >>> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >>> Filesystem Size Used Avail Use% Mounted on >>> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >>> >>> >>> >>> This means that expiry policy not deleting records >>> from the disk, but ignite document says when expiry >>> policy is set and native persistence is enabled then >>> it deletes records from disk as well. >>> Am I missing some configuration? >>> Any help is appreciated. >>> >>> Shiva >> |
«
Return to Apache Ignite Developers - Legacy Mail Archive
|
1 view|%1 views
| Free forum by Nabble | Edit this page |