Обсуждение: Urgent! Log full with inexistent PID processes

Поиск
Список
Период
Сортировка

Urgent! Log full with inexistent PID processes

От
Iñigo Martinez Lasala
Дата:
Hi everybody.

We have a postgres 8.4.5-2 database under Debian Lenny 64bit.
This database has been working without any problem, but this afternoon server has got out of space due to a log dump.

Log file gets full with this entry until it reaches several gigabytes:

2011-06-27 13:25:07 CEST LOG:  PID 21822 in cancel request did not match any process
2011-06-27 13:25:07 CEST LOG:  PID 21821 in cancel request did not match any process
2011-06-27 13:25:07 CEST LOG:  PID 21820 in cancel request did not match any process
2011-06-27 13:25:07 CEST LOG:  PID 21816 in cancel request did not match any process
2011-06-27 13:25:07 CEST LOG:  PID 21802 in cancel request did not match any process


Of course, we have restarted postmaster and rebooted machine but problem remains.

Any clue about this?

pg_hba only allows connection from localhost and, of course, there are no processes with these PIDs in the server...

Thanks in advance. 

Re: Urgent! Log full with inexistent PID processes

От
Adarsh Sharma
Дата:


Iñigo Martinez Lasala wrote:
Hi everybody.

We have a postgres 8.4.5-2 database under Debian Lenny 64bit.
This database has been working without any problem, but this afternoon server has got out of space due to a log dump.

Log file gets full with this entry until it reaches several gigabytes:

2011-06-27 13:25:07 CEST LOG:  PID 21822 in cancel request did not match any process
2011-06-27 13:25:07 CEST LOG:  PID 21821 in cancel request did not match any process
2011-06-27 13:25:07 CEST LOG:  PID 21820 in cancel request did not match any process
2011-06-27 13:25:07 CEST LOG:  PID 21816 in cancel request did not match any process
2011-06-27 13:25:07 CEST LOG:  PID 21802 in cancel request did not match any process


First check the space occupied by pg_log directory.
Just delete the previous logs that are 1 month older.
rm -rf pg_log******


Of course, we have restarted postmaster and rebooted machine but problem remains.

Any clue about this?

pg_hba only allows connection from localhost and, of course, there are no processes with these PIDs in the server...

Thanks in advance. 

Re: Urgent! Log full with inexistent PID processes

От
Florian Weimer
Дата:
* Iñigo Martinez Lasala:

> Any clue about this?

There is a misbehaving client which tries to issue broken cancel
requests.  You need to find it.

> pg_hba only allows connection from localhost and, of course, there are
> no processes with these PIDs in the server...

Do you use TCP or UNIX domain sockets for the database connections?

--
Florian Weimer                <fweimer@bfk.de>
BFK edv-consulting GmbH       http://www.bfk.de/
Kriegsstraße 100              tel: +49-721-96201-1
D-76133 Karlsruhe             fax: +49-721-96201-99

Re: Urgent! Log full with inexistent PID processes

От
Iñigo Martinez Lasala
Дата:
Hi Florian.

There is are no connections to DB. Only app using database is a web app that has been stopped also.

-----Original Message-----
From: Florian Weimer <fweimer@bfk.de>
To: Iñigo Martinez Lasala <imartinez@vectorsf.com>
Cc: pgsql-admin@postgresql.org
Subject: Re: [ADMIN] Urgent! Log full with inexistent PID processes
Date: Mon, 27 Jun 2011 13:40:49 +0000

* Iñigo Martinez Lasala:

> Any clue about this?

There is a misbehaving client which tries to issue broken cancel
requests.  You need to find it.

> pg_hba only allows connection from localhost and, of course, there are
> no processes with these PIDs in the server... 

Do you use TCP or UNIX domain sockets for the database connections?

Re: Urgent! Log full with inexistent PID processes

От
Florian Weimer
Дата:
* Iñigo Martinez Lasala:

> There is are no connections to DB. Only app using database is a web app
> that has been stopped also.

Uh-oh.  Can you "strace -f" the postmaster?  Surely something is
connecting to it.

--
Florian Weimer                <fweimer@bfk.de>
BFK edv-consulting GmbH       http://www.bfk.de/
Kriegsstraße 100              tel: +49-721-96201-1
D-76133 Karlsruhe             fax: +49-721-96201-99

Re: Urgent! Log full with inexistent PID processes

От
Iñigo Martinez Lasala
Дата:
You are my man, bud. :)

A remote server was launching a process against this database. We had limited access via pg_hba.conf (or at least we thought access was limited). Since a database admin was modifiying config in paralell with me, we have a mess with config and so access restriction was not applying.

Thank you very mucho.

-----Original Message-----
From: Florian Weimer <fweimer@bfk.de>
To: Iñigo Martinez Lasala <imartinez@vectorsf.com>
Cc: pgsql-admin@postgresql.org
Subject: Re: [ADMIN] Urgent! Log full with inexistent PID processes
Date: Tue, 28 Jun 2011 09:25:26 +0000

* Iñigo Martinez Lasala:

> There is are no connections to DB. Only app using database is a web app
> that has been stopped also.

Uh-oh.  Can you "strace -f" the postmaster?  Surely something is
connecting to it.