Michael Adler <adler@pobox.com> writes:
> I'm investigating a problem that happened last night and I would
> appreciate any recommendations. The logs indicate that the disks were
> full, but I truly doubt that since we only use about 14GB out of the
> available 65GB.
> I found entries like this in the logs:
> ERROR: could not write block 2354 of temporary file: No space left on device
> HINT: Perhaps out of disk space?
> ....
> ERROR: could not extend relation "parent_table": No space left on device
> HINT: Check free disk space.
> ....
> LOG: could not close temporary statistics file "/var/lib/postgres/data/global/pgstat.tmp.1464": No space left on
device
> According to the logs, the problem went away after a reboot. I wonder
> if the kernel or the RAID device got confused and postgres was simply
> echoing what it was told. We run a couple hundred postgres servers and
> we have not seen this before (except when the disks truly were full).
I'm inclined to think that a query created a 50GB temporary file ...
the postmaster cleans out temp files when restarted, so that would
have destroyed the evidence.
regards, tom lane