Re: Speed up the removal of WAL files

On Sat, Nov 18, 2017 at 2:57 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
> On Fri, Nov 17, 2017 at 5:20 PM, Tsunakawa, Takayuki
> <tsunakawa.takay@jp.fujitsu.com> wrote:
>> From: Kyotaro HORIGUCHI [mailto:horiguchi.kyotaro@lab.ntt.co.jp]
>>> The orinal code recycles some of the to-be-removed files, but the patch
>>> removes all the victims.  This may impact on performance.
>>
>> Yes, I noticed it after submitting the patch and was wondering what to do.  Thinking simply, I think it would be
justenough to replace durable_unlink/durable_rename in RemoveXLogFile() with unlink/rename, and sync the pg_wal
directoryonce in RemoveNonParentXlogFiles() and RemoveOldXlogFiles().  This will benefit the failover time when fast
promotionis not performed.  What do you think? 

Note that durable_rename() also flushes the origin file to make sure
that it does not show up again after a crash.

> It seems not good idea to just replace durable_rename() with rename()
> in RemoveOldXlogFiles()->RemoveXlogFiles()->InstallXLogFileSegment().
> Because that change seems to be able to cause the following problem.

If archiving is enabled, RemoveOldXlogFiles() would create a .ready
flag for all segments that have reappeared. Oops, it archived again.
--
Michael