On 03-03-2016 14:44, Magnus Hagander wrote: > On Thu, Mar 3, 2016 at 6:34 PM, Andres Freund <andres@anarazel.de > <mailto:andres@anarazel.de>> wrote: > > On 2016-03-03 18:31:03 +0100, Magnus Hagander wrote: > > I think we want it at protocol level rather than pg_basebackup level. > > I think we may want both eventually, but I do agree that protocol level > has a lot higher "priority" than that. Something like protocol level > compression has a bit of different tradeofs than compressing base > backups, and it's nice not to compress, uncompress, compress again. > > > > Yeah, good point, we definitely want both. Based on the field experience > I've had (which might differ from others), having it protocol level > would help more people tough, so should be higher prio. > Some time ago, I started a thread [1] to implement compression at protocol level. The use cases are data load over slow links and reduce bandwidth consumption during replication.
At that time, there wasn't a consensus about which compression algorithm to choose. After the WAL compression feature, I think we can do some POC with LZ compression (that is already available in common).
I'll try to update the code and do some benchmarks.
+1 to protocol level compression. In our case the primary reasons why we use thirdparty magic networking appliances as a middle man between our offices is to compress postgres network traffic (which is very compress-able that is > 95% reduction is normal). And the presence of those devices introduces all kinds of weird additional error cases and administrative overhead (+ of course cost). So I would personally consider protocol level compression to be bigger killer feature than any other feature that has made itself into postgres since the 9.2 release. But of course YMMV ;-)