Sergey Chugunov created IGNITE-13143:
----------------------------------------
Summary: Persistent store defragmentation
Key: IGNITE-13143
URL:
https://issues.apache.org/jira/browse/IGNITE-13143 Project: Ignite
Issue Type: New Feature
Reporter: Sergey Chugunov
Persistent store enables users to store data of their caches in a durable fashion on disk still benefiting from in-memory nature of Apache Ignite. Data of caches is stored in files created for every primary or backup partition assigned to that node and in an additional file for all user indexes.
Files in filesystem are allocated lazily (only if some data is actually stored to particular partition) and grow automatically when more data is added to the cache. But the problem is that files cannot shrink even if all data is removed.
This umbrella ticket covers all other tasks needed to implement simple yet effective approach to defragmentation. Detailed discussion could be found in [IEP-47|
https://cwiki.apache.org/confluence/display/IGNITE/IEP-47%3A+Native+persistence+defragmentation] and in corresponding [dev-list discussion|
http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-IEP-47-Native-persistence-defragmentation-td47717.html] but core ideas are as follows:
# Defragmentation is performed in a special _maintenance_ mode when node starts, provides access to some APIs like metrics or JMX management but doesn't join the cluster.
# It is performed by copying all data from all partitions on node to new files with automatic compaction. After successful copy old partition files are deleted.
# Metrics on progress of the operation are provided to the user.
# Operation is fault-tolerant and in case of node failure proceeds after node restart.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Locked
