Re: BUG #14384: pg_dump uses excessive amounts of memory for LOBs

Tom Lane wrote:
>
> TL;DR: blobs are not exactly lightweight objects.  If you want something
> with less overhead, maybe you should just store the data in a plain
> bytea column.
>

Yes, I tested this case with bytea/TOAST in place of LOB column and
pg_dump used just a few MB .

Normally (when dumping a database with hundreds of tables) it uses
insignificant amount of memory so I was surprised when it allocated
about 26GB while processing a 100GB database with 30*10^6 LOBs (caused
swapping and almost triggered OOM killer).

The 'obvious' (for database this size) and easy solution was to use
pg_basebackup - and it sped up the process almost 40 times .

I found a similar case here
(https://www.postgresql.org/message-id/524C3163.1050502%40iqbuzz.ru) so
I think it would be good to warn users about this behavior in pg_dump
documentation (to save a few hours of the third person who triggers this
issue in ... October 2019).

Thank you,
Bolek Ziobrowski