Re: Too many open files (was Re: spinlock problems reported earlier)

Tom Lane wrote:
> 
> Department of Things that Fell Through the Cracks:
> 
> Back in August we had concluded that it is a bad idea to trust
> "sysconf(_SC_OPEN_MAX)" as an indicator of how many files each backend
> can safely open.  FreeBSD was reported to return 4136, and I have
> since noticed that LinuxPPC returns 1024.  Both of those are
> unreasonably large fractions of the actual kernel file table size.
> A few dozen backends opening hundreds of files apiece will fill the
> kernel file table on most Unix platforms.
> 
> I'm not sure why this didn't get dealt with, but I think it's a "must
> fix" kind of problem for 7.1.  The dbadmin has *got* to be able to
> limit Postgres' appetite for open file descriptors.
> 
> I propose we add a new configuration parameter, MAX_FILES_PER_PROCESS,
> with a default value of about 100.  A new backend would set its
> max-files setting to the smaller of this parameter or
> sysconf(_SC_OPEN_MAX).
> 
> An alternative approach would be to make the parameter be total open files
> across the whole installation, and divide it by MaxBackends to arrive at
> the per-backend limit.  However, it'd be much harder to pick a reasonable
> default value if we did it that way.
> 
> Comments?

On Linux, at least, the 1024 file limit is a per process limit, the
system wide limit defaults to 4096 and can be easily changed by 

echo 16384 > /proc/sys/fs/file-max

(16384 is arbitrary and can be much larger)

I am all for having the ability to tune behavior over the system
reported values, but I think it should be an option which defaults to
the previous behavior.

-- 
http://www.mohawksoft.com