Re: [PoC PATCH] Parallel dump to /dev/null

Hi,

On Thu, Feb 01, 2018 at 02:24:46PM +0100, Michael Banck wrote:
> dumping a database to /dev/null via pg_dump is (AFAIK) one recommended
> way to check for corruption.  However, dumping to /dev/null is currently
> not supported in directory mode which makes it not possible to dump to
> /dev/null in parallel.
> 
> I had a look at this, and it appears it would suffice to just override
> the few spots in pg_backup_directory.c which assemble filenames in the
> target directory to revert to '/dev/null' (or 'NUL' on Windows).
> 
> The attached proof-of-concept patch does that, and it seems to work; I'm
> getting some nice speedups on a 170 GB test database:
> 
> $ time pg_dump -Fc -Z0 -f /dev/null TESTDB
> real    32m45.120s
> [...]
> $ time pg_dump -Fd -j8 -Z0 -f /dev/null TESTDB
> real    9m28.357s

I have added this patch to the next commitfest:

https://commitfest.postgresql.org/17/1576/

I also tried to add a TAP test, but as this patch produces no dump
output, I had to exclude it from all restores tests and then got an
off-by-one between the number of tests that were expected vs. those that
were run. I've attached it if somebody wants to take a look, but I hope
with Stephen's refactoring of the pg_dump TAP tests, this might be
easier.


Michael