Re: pg_stat_*_columns?

On Sat, Jun 20, 2015 at 6:15 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Robert Haas <robertmhaas@gmail.com> writes:
>> One idea would be to advertise a DSM ID in the main shared memory
>> segment, and have the individual backends read that value and attach
>> to it.  When new stats are generated, the stats collector creates a
>> new DSM (which might be bigger or smaller than the old one), writes
>> the new stats in there, and then advertises the new DSM ID in the main
>> shared memory segment.  Backends that still have the old segment
>> attached can still use it, and it will go away automatically once they
>> all drop off.
>
> Hmmm.  This sounds attractive, but what happens if we fail to create
> a new DSM when needed?

Presumably you keep using the old one and retry later.  I mean, out of
memory is out of memory; you can't move a variable-size data structure
into shared memory without the possibility of running into OOM
failures at some point.

>> But I'm not sure how this would work with the new per-database split
>> of the stats file.  I don't think it'll work to have one DSM per
>> database; we don't support enough DSMs for that.
>
> AFAIR, that per-database split exists only to try to reduce the amount of
> traffic written to disk.  We could lose it cheerfully if the communication
> all happens in shared memory.

If we arranged things so that the processes could use the data in the
DSM directly rather than having to copy it out, we'd presumably save
quite a bit of memory, since the whole structure would be shared
rather than each backend having its own copy.  But if the structure
got too big to map (on a 32-bit system), then you'd be sort of hosed,
because there's no way to attach just part of it.  That might not be
worth worrying about, but it depends on how big it's likely to get - a
32-bit system is very likely to choke on a 1GB mapping, and maybe even
on a much smaller one.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company