Re: Verify Option with pg_dump

On Wed, Nov 30, 2016 at 12:00:07PM +0000, Howard News wrote:

> recently I had problems with a corrupt pg_dump file. The problem with the
> file was due to a faulty disk. The trouble with this is that I was unaware
> of the disk problem and the pg_dump file corruption so I did not have a full
> valid backup. In order to reduce the chances of this I was hoping that there
> could be a verify option as in SQL Server for the backups. This could be as
> simple as checking the CRC/MD5 as the stream is created. So pg_dump |
> crc_save
>
> The idea being that the pg_dump is crc'd before it is streamed to disk, and
> then the file re-read from disk to check the CRC.
>
> Is there a linux utility to do this or would it be simple to modify pg_dump
> to do this?

You can try to suitably combine "pg_dump --format=plain" with
"tee" and "md5sum" such that the output stream is diverted to
both a file and a pipe-into-CRC-algorithm and eventually
compare the pipe's sum with the sum generated from the file.

But the better solution might be to stream to a filesystem
that verifies disk writes immediately. Or to a suitable RAID
array.

Regards,
Karsten
--
GPG key ID E4071346 @ eu.pool.sks-keyservers.net
E167 67FD A291 2BEA 73BD  4537 78B9 A9F9 E407 1346