Re: Improving compressibility of WAL files

>>> Greg Smith <gsmith@gregsmith.com> wrote: 
> The main use case I'm concerned about losing support for is:
> 
> 1) Two systems connected by a WAN with significant transmit latency
> 2) The secondary system runs a warm standby aimed at disaster
recovery
> 3) Business requirements want the standby to never be more than (say)
5 
> minutes behind the primary, presuming the WAN is up
> 4) WAN traffic is "expensive" (money==bandwidth, one of the two is
scarce)
> 
> This seems a pretty common scenario in my experience.  Right now,
this 
> case is served quite well like this:
> 
> -archive_timeout='5 minutes'
> -[pglesslog|pg_clearxlogtail] | gzip | rsync
You've come pretty close to describing our environment, other than
having 72 primaries each using rsync to push the WAL files to another
server at the same site while a server at the central site uses rsync
to pull them back.  We don't run warm standby on the backup server at
the site of origin, and don't want to have to do so.
It is critically important that the flow of xlog data never hold up
the primary databases, and that failure to copy xlog to either of the
targets not interfere with copying to the other.  (We have WAN
failures surprising often, sometimes for days at a time, and the
backup server on-site is in the same rack of the same cabinet as the
database server.)
Compression of xlog data is important not only for WAN transmission,
but for storage space.  We keep two weeks of WAL files to allow
recovery from either of the last two weekly backups, and we archive
the first weekly backup of each month, with the WAL files needed for
recovery, for one year.
So it appears we care about somewhat similar issues.
-Kevin