Logical archiving
От | Andrey Borodin |
---|---|
Тема | Logical archiving |
Дата | |
Msg-id | 20D9328B-A189-43D1-80E2-EB25B9284AD6@yandex-team.ru обсуждение исходный текст |
Ответы |
Re: Logical archiving
Re: Logical archiving |
Список | pgsql-hackers |
Hi all I was discussing problems of CDC with scientific community and they asked this simple question: "So you have efficient WALarchive on a very cheap storage, why don't you have a logical archive too?" This seems like a wild idea. But really, we have a super expensive NVMe drives for OLTP workload. And use this devices tostore buffer for data to be dumped into MapReduce\YT analytical system. If OLAP cannot consume data fast enough - we are out of space due to repl slot. If we have a WAL HA switchover - OLAP has a hole in the stream and have to resync data from the scratch. If we could just run archive command ```archive-tool wal-push 0000000900000F2C000000E1.logical``` with contents of logicalreplication - this would be super cool for OLAP. I'd prefer even avoid writing 0000000900000F2C000000E1.logical todisk, i.e. push data on stdio or something like that. What do you think? Best regards, Andrey Borodin.
В списке pgsql-hackers по дате отправления: