[jira] [Created] (IGNITE-11641) Server node copies a lot of WAL files in WAL archive after restart

[Anton Vinogradov (Jira)]

Dmitriy Govorukhin created IGNITE-11641:
-------------------------------------------

             Summary: Server node copies a lot of WAL files in WAL archive after restart
                 Key: IGNITE-11641
                 URL: https://issues.apache.org/jira/browse/IGNITE-11641
             Project: Ignite
          Issue Type: Bug
            Reporter: Dmitriy Govorukhin


Pre-condition: PDS is enabled, wal_path and wal_archive_path are set in config file.

1. Cluster is up and running. Some data uploaded into caches.
2. Start load to generate a lot of files in wal archive (more than files in wal directory).
3. Stop some node and delete all files from wal archive.
4. Start node.

In this case node copies WAL files from WAL dir into wal archive dir again and again until the amount of files will be the same it was in wal archive before stop.

Here is information from server node log

{code}
10:10:17,054][INFO][main][GridCacheDatabaseSharedManager] Restoring partition state for local groups.
[10:10:18,522][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000000.wal, dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000000.wal]
[10:10:18,523][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Starting to copy WAL segment [absIdx=1, segIdx=1, origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000001.wal, dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000001.wal]
[10:10:20,631][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000001.wal, dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000001.wal]
[10:10:20,632][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Starting to copy WAL segment [absIdx=2, segIdx=2, origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000002.wal, dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000002.wal]
[10:10:23,276][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000002.wal, dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000002.wal]
[10:10:23,276][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Starting to copy WAL segment [absIdx=3, segIdx=3, origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000003.wal, dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000003.wal]
[10:10:23,995][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000003.wal, dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000003.wal]
[10:10:23,996][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Starting to copy WAL segment [absIdx=4, segIdx=4, origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000004.wal, dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000004.wal]
[10:10:24,644][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000004.wal, dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000004.wal]
[10:10:24,645][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Starting to copy WAL segment [absIdx=5, segIdx=5, origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000005.wal, dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000005.wal]
[10:10:25,301][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000005.wal, dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000005.wal]
[10:10:25,301][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Starting to copy WAL segment [absIdx=6, segIdx=6, origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000006.wal, dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000006.wal]
[10:10:26,043][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000006.wal, dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000006.wal]
[10:10:26,044][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Starting to copy WAL segment [absIdx=7, segIdx=7, origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000007.wal, dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000007.wal]
[10:10:26,768][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000007.wal, dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000007.wal]
[10:10:26,769][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Starting to copy WAL segment [absIdx=8, segIdx=8, origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000008.wal, dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000008.wal]
[10:10:27,600][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000008.wal, dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000008.wal]
[10:10:27,601][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Starting to copy WAL segment [absIdx=9, segIdx=9, origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000009.wal, dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000009.wal]
[10:10:28,315][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000009.wal, dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000009.wal]
[10:10:28,316][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Starting to copy WAL segment [absIdx=10, segIdx=0, origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000000.wal, dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000010.wal]
[10:10:29,092][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000000.wal, dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000010.wal]
[10:10:29,092][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Starting to copy WAL segment [absIdx=11, segIdx=1, origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000001.wal, dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000011.wal]
[10:10:29,908][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000001.wal, dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000011.wal]
[10:10:29,908][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Starting to copy WAL segment [absIdx=12, segIdx=2, origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000002.wal, dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000012.wal]
[10:10:30,716][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000002.wal, dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000012.wal]
[10:10:30,716][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Starting to copy WAL segment [absIdx=13, segIdx=3, origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000003.wal, dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000013.wal]
[10:10:31,402][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000003.wal, dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000013.wal]
[10:10:31,403][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Starting to copy WAL segment [absIdx=14, segIdx=4, origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000004.wal, dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000014.wal]
[10:10:32,052][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000004.wal, dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000014.wal]
[10:10:32,053][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Starting to copy WAL segment [absIdx=15, segIdx=5, origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000005.wal, dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000015.wal]
[10:10:32,679][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000005.wal, dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000015.wal]
[10:10:33,986][INFO][main][GridCacheDatabaseSharedManager] Finished restoring partition state for local groups [groupsProcessed39partitionsProcessed=15056, time=16928ms]
[10:10:34,269][INFO][main][TcpDiscoverySpi] Connection check threshold is calculated: 2000
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)