Re: archive status ".ready" files may be created too early
От | Bossart, Nathan |
---|---|
Тема | Re: archive status ".ready" files may be created too early |
Дата | |
Msg-id | 45E14987-0E25-4E5D-BDA5-B94EDA28A778@amazon.com обсуждение исходный текст |
Ответ на | Re: archive status ".ready" files may be created too early ("Bossart, Nathan" <bossartn@amazon.com>) |
Ответы |
RE: archive status ".ready" files may be created too early
|
Список | pgsql-hackers |
Sorry for the long delay. I've finally gotten to a new approach that I think is promising. My previous attempts to fix this within XLogWrite() or within the associated code paths all seemed to miss corner cases or to add far too much complexity. The new proof-of-concept patch that I have attached is much different. Instead of trying to adjust the ready- for-archive logic in the XLogWrite() code paths, I propose relocating the ready-for-archive logic to a separate process. The v3 patch is a proof-of-concept patch that moves the ready-for- archive logic to the WAL writer process. We mark files as ready-for- archive when the WAL flush pointer has advanced beyond a known WAL record boundary. In this patch, I am using the WAL insert location as the known WAL record boundary. The main idea is that it should be safe to archive a segment once we know the last WAL record for the WAL segment, which may overflow into the following segment, has been completely written to disk. There are many things missing from this proof-of-concept patch that will need to be handled if this approach seems reasonable. For example, I haven't looked into any adjustments needed for the archive_timeout parameter, I haven't added a way to persist the "latest segment marked ready-for-archive" through crashes, I haven't tried reducing the frequency of retrieving the WAL locations, and I'm not sure the WAL writer process is even the right location for this logic. However, these remaining problems all seem tractable to me. I would appreciate your feedback on whether you believe this approach is worth pursuing. Nathan
Вложения
В списке pgsql-hackers по дате отправления: