Re: Read-ahead and parallelism in redo recovery

Поиск
Список
Период
Сортировка
От Aidan Van Dyk
Тема Re: Read-ahead and parallelism in redo recovery
Дата
Msg-id 20080229205940.GN17067@yugib.highrise.ca
обсуждение исходный текст
Ответ на Re: Read-ahead and parallelism in redo recovery  (Tom Lane <tgl@sss.pgh.pa.us>)
Ответы Re: Read-ahead and parallelism in redo recovery  ("Heikki Linnakangas" <heikki@enterprisedb.com>)
Список pgsql-hackers
* Tom Lane <tgl@sss.pgh.pa.us> [080229 15:49]:
> 
> If that isn't entirely useless, you need a better kernel.  The system
> should *certainly* be bright enough to do read-ahead for our reads of
> the source xlog file.  The fetches that are likely to be problematic are
> the ones for pages in the data area, which will be a lot less regular
> for typical workloads.

How difficult is it to parse the WAL logs with enough knowledge to know
what heap page (file/offset) a wal record contains (I haven't looked
into any wal code)?

There are "compression/decompression" archive_command/restore_command
programs with rudimentary knowledge of the WAL record formats.  Would a
"restore_command" be able to parse the wal records as it copies them
over noting which file pages need to be read, and the just before it
exits, fork() and read each page in order.

This child doesn't need to do anything with the blocks it reads - it
just needs to read them to "pre-warm" the kernel buffer cache...  If the
restoration is doing any writing, this dumb reader would hopefully be
able to keep a block ahead...  And since it's separated enough from the
backend, any experiments in async_io/fadvise could easily be done.

-- 
Aidan Van Dyk                                             Create like a god,
aidan@highrise.ca                                       command like a king,
http://www.highrise.ca/                                   work like a slave.

В списке pgsql-hackers по дате отправления:

Предыдущее
От: Tom Lane
Дата:
Сообщение: Re: Read-ahead and parallelism in redo recovery
Следующее
От: "Heikki Linnakangas"
Дата:
Сообщение: Re: "could not open relation 1663/16384/16584: No such file or directory" in a specific combination of transactions with temp tables