WAL format changes
От | Heikki Linnakangas |
---|---|
Тема | WAL format changes |
Дата | |
Msg-id | 4FDA5136.6080206@enterprisedb.com обсуждение исходный текст |
Ответы |
Re: WAL format changes
Re: WAL format changes Re: WAL format changes Re: WAL format changes |
Список | pgsql-hackers |
As I threatened earlier (http://archives.postgresql.org/message-id/4FD0B1AB.3090405@enterprisedb.com), here are three patches that change the WAL format. The goal is to change the format so that when you're inserting a WAL record of a given size, you know exactly how much space it requires in the WAL. 1. Use a 64-bit segment number, instead of the log/seg combination. And don't waste the last segment on each logical 4 GB log file. The concept of a "logical log file" is now completely gone. XLogRecPtr is unchanged, but it should now be understood as a plain 64-bit value, just split into two 32-bit integers for historical reasons. On disk, this means that there will be log files ending in FF, those were skipped before. 2. Always include the xl_rem_len field, used for continuation records, in the xlog page header. A continuation log record only contained that one field, it's now included straight in the page header, so the concept of a continuation record doesn't exist anymore. Because of alignment, this wastes 4 bytes on every page that contains continued data from a previous record, and 8 bytes on pages that don't. That's not very much, and the next step will buy that back: 3. Allow WAL record header to be split across pages. Per Tom's suggestion, move xl_tot_len to be the first field in XLogRecord, so that even if the header is split, xl_tot_len is always on the first page. xl_crc is moved to be the last field, and xl_prev is the second to last. This has the advantage that you can calculate the CRC for all the other fields before acquiring WALInsertLock. For xl_prev, you need to know where exactly the record is inserted, so it's handy that it's the last field before CRC. This patch doesn't try to take advantage of that, however, and I'm not sure if that makes any difference once I finish the patch to make XLogInsert scale better, which is the ultimate goal of all this. Those are the three patches I'd like to get committed in this commitfest. To see where all this is leading to, I've included a rough WIP version of the XLogInsert scaling patch. This version is quite different from the one I posted in spring, it takes advantage of the WAL format changes, and I'm also experimenting with a different method of tracking how far each WAL insertion has progressed. But more on that later. (Note to self: remember to bump XLOG_PAGE_MAGIC) -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
Вложения
В списке pgsql-hackers по дате отправления: