Re: URGENT: Database keeps crashing - suspect damaged RAM

"Markus Wollny" <Markus.Wollny@computec.de> writes:
> So: Is it bad RAM? How can I make sure? What else could it be?

Have you tried running memtest86?  I've never used that myself but
some folks on the list say it works well.

> Here's a small excerpt from the logfile:

> 2002-08-06 17:36:23 [17296]  DEBUG:  _mdfd_blind_getseg: couldn't open
> /var/lib/pgsql/data/base/base/16596/16671: Cannot allocate memory

Is it possible that you are running with inadequate swap space, a small
data segment limit (ulimit -d), or something else that would make the
kernel refuse to give memory to a backend process?

> 2002-08-06 17:40:53 [16530]  DEBUG:  connection startup failed (fork
> failure): Cannot allocate memory
> 2002-08-06 17:52:50 [16530]  DEBUG:  connection startup failed (fork
> failure): Cannot allocate memory

Still looks like inadequate memory --- but now I'm thinking that it's a
system-wide condition, ie, you just plain haven't got enough RAM for the
number of processes you're trying to start.

> 2002-08-06 17:52:54 [16530]  DEBUG:  server process (pid 18237) was
> terminated by signal 9

Postgres never issues any kill -9 on itself, but I've heard that the
Linux kernel may start killing processes when it's desperately low on
memory.

Other than the signal 9, everything I see in this trace is either a
cannot-allocate-memory failure or followup effects from one.  How many
backends are you trying to start up, anyway?  Might you have a runaway
client that keeps opening new backend connections?

            regards, tom lane