On Sat, Jul 25, 2015 at 8:04 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Craig James <cjames@emolecules.com> writes: > ... This would result in a thousand > or so Postgres connections on a machine with 32 CPUs.
> So the question is: do idle connections impact performance?
Yes. Those connections have to be examined when gathering snapshot information, since you don't know that they're idle until you look. So the cost of taking a snapshot is proportional to the total number of connections, even when most are idle. This sort of situation is known to aggravate contention for the ProcArrayLock, which is a performance bottleneck if you've got lots of CPUs.
You'd be a lot better off with a pooler.
OK, thanks for the info, that answers the question.
Another choice we have, since all schemas are in the same database, is to use a single "super user" connection that has access to every schema. Each fast-CGI would then only need a single connection. That's a lot more work, as it requires altering our security, altering all of the SQL statements, etc. It moves the security model from the database to the application.
A pooler isn't an idea solution here, because there is still overhead associated with each connection. Persistent connections are blazingly fast (we already use them in a more limited fast-CGI application).
Craig
(There has been, and continues to be, interest in getting rid of this bottleneck ... but it's a problem in all existing Postgres versions.)