Обсуждение: excessive WAL activity

Поиск
Список
Период
Сортировка

excessive WAL activity

От
Sean Dillon
Дата:
Just turned on WAL archiving to an S3 bucket for a small database - total size of perhaps 2-4G.  After turning on
achiving,we're seeing WAL logs written to S3 at the rate of about 1G every 3 minutes.  That seems completely
unreasonablegiven usage of the db.  I can even see that nearly nothing is happening with this: 

select datname, usename, procpid, client_addr, waiting, query_start, current_query from pg_stat_activity;

Nearly every time I run that, all 20 connections have current_query = '<IDLE>'.  Does current_query include inserts,
updates,and deletes or just select statements? 

Any ideas what to look for or how to solve this?



Re: excessive WAL activity

От
bricklen
Дата:
On Wed, Jun 19, 2013 at 4:22 PM, Sean Dillon <sean@dillonsoftware.com> wrote:
Just turned on WAL archiving to an S3 bucket for a small database - total size of perhaps 2-4G.  After turning on achiving, we're seeing WAL logs written to S3 at the rate of about 1G every 3 minutes.  That seems completely unreasonable given usage of the db.  I can even see that nearly nothing is happening with this:

select datname, usename, procpid, client_addr, waiting, query_start, current_query from pg_stat_activity;

Nearly every time I run that, all 20 connections have current_query = '<IDLE>'.  Does current_query include inserts, updates, and deletes or just select statements?

Any ideas what to look for or how to solve this?

Can you show the results from:

SELECT name, current_setting(name), source
FROM pg_settings
WHERE source NOT IN ('default', 'override')
UNION ALL
SELECT 'version' as name, version(), null;

Perhaps there is a setting there (like archive_timeout) which could lead to an answer. In your $PGDATA/pg_xlog directory, look at the timestamps of the WAL segments, are there lots of files generated per minute?
 

Re: excessive WAL activity

От
Albe Laurenz
Дата:
bricklen wrote:
> On Wed, Jun 19, 2013 at 4:22 PM, Sean Dillon <sean@dillonsoftware.com> wrote:
>> Just turned on WAL archiving to an S3 bucket for a small database - total size of perhaps 2-4G.
>> After turning on achiving, we're seeing WAL logs written to S3 at the rate of about 1G every 3
>> minutes.  That seems completely unreasonable given usage of the db.  I can even see that nearly
>> nothing is happening with this:
>> 
>> select datname, usename, procpid, client_addr, waiting, query_start, current_query from
>> pg_stat_activity;
>> 
>> Nearly every time I run that, all 20 connections have current_query = '<IDLE>'.  Does
>> current_query include inserts, updates, and deletes or just select statements?
>> 
>> Any ideas what to look for or how to solve this?
> 
> Can you show the results from:
> 
> SELECT name, current_setting(name), source
> FROM pg_settings
> WHERE source NOT IN ('default', 'override')
> UNION ALL
> SELECT 'version' as name, version(), null;
> 
> 
> Perhaps there is a setting there (like archive_timeout) which could lead to an answer. In your
> $PGDATA/pg_xlog directory, look at the timestamps of the WAL segments, are there lots of files
> generated per minute?

Also, it might be good to check how much is really going on.
Try log_statement=all and see how much gets logged.

Yours,
Laurenz Albe