Re: Are there plans to add data compression feature to postgresql?

Sam Mason wrote:
> On Mon, Nov 03, 2008 at 08:18:54AM +0900, Craig Ringer wrote:
>> Joris Dobbelsteen wrote:
>>> Also I still have to see an compression algorithm that can sustain over
>>> (or even anything close to, for that matter) 100MB/s on todays COTS
>>> hardware. As TOAST provides compression, maybe that data can be
>>> transmitted in compressed manner  (without recompression).
>
>> I get 19 Mbit/s from gzip (deflate) on my 2.4GHz Core 2 Duo laptop. With
>> lzop (LZO) the machine achieves 45 Mbit/s. In both cases only a single
>> core is used. With 7zip (LZMA) it only manages 3.1 Mb/s using BOTH cores
>> together.
>
> Your lzop numbers look *very* low; the paper suggests
> compression going up to ~0.3GB/s on a 2GHz Opteron.

Er ... ENOCOFFEE? . s/Mb(it)?/MB/g . And I'm normally *so* careful about
Mb/MB etc; this was just a complete thinko at some level. My apologies,
and thanks for catching that stupid error.

The paragraph should've read:

I get 19 MB/s (152 Mb/s) from gzip (deflate) on my 2.4GHz Core 2 Duo
laptop. With lzop (LZO) the machine achieves 45 MB/s (360 Mb/s). In both
cases only a single core is used. With 7zip (LZMA) it only manages 3.1
MB/s (24.8 Mb/s) using BOTH cores together.

So - it's potentially even worth compressing the wire protocol for use
on a 100 megabit LAN if a lightweight scheme like LZO can be used.

--
Craig Ringer