Re: BUG #6629: Creating a gist index fails with "too many LWLocks taken"

On 11.05.2012 18:18, Simon Riggs wrote:
> On 11 May 2012 15:14, Heikki Linnakangas
> <heikki.linnakangas@enterprisedb.com>  wrote:
>> On 11.05.2012 16:56, Simon Riggs wrote:
>>>
>>> On 11 May 2012 11:07, Heikki Linnakangas
>>> <heikki.linnakangas@enterprisedb.com>    wrote:
>>>
>>>> I wonder if we should reserve a few of the lwlock "slots" for critical
>>>> sections, to make this less likely to happen. Not only in this case, but
>>>> in
>>>> general. We haven't seen this problem often, but it would be quite
>>>> trivial
>>>> to reserve a few slots.
>>>
>>>
>>> Why reserve them solely for critical sections?
>>
>> Because if you run out of lwlocks in a critical section, you get a PANIC.
>
> Yes, but why reserve them solely for critical sections? If you have an
> escape hatch you use it, always

Well, no, because a PANIC is much worse than an ERROR. Think of this as
a spare oxygen tank while diving, rather than an escape hatch. A spare
tank can save your life if you run out of oxygen while ascending, but if
you run out of oxygen on the way down, you don't continue going down
with just the spare tank.

Imagine that you have a process that does something like this:

for (i=0; i < 99; i++)
   LWLockAcquire(foolock[i])

START_CRIT_SECTION();

XLogInsert(...)

END_CRIT_SECTION();

What happens at the moment is that XLogInsert hits the limit when it
tries to acquire WALInsertLock, so you get a PANIC. If we reserved, say,
5 locks for critical sections, so that you could hold 95 locks while
outside a critical section, and 5 more within on, you would get an error
earlier, outside the critical section, while acquiring the "foolocks".
Or if the number of foolocks acquired was less than 95, you would not
get error at all. That avoids the PANIC.

You can argue for just raising the limit, but that just moves the
problem around. It's still possible to hit the limit within a critical
section, and PANIC. Likewise, if we lower the limit, that helps us find
the problems earlier in the development cycle, but doesn't change the
fact that if you run too close to the edge, you run out of locks within
a critical section and PANIC.

Of course, nothing stops you from writing (bad) code that acquires a lot
of locks within a critical section, in which case you're screwed anyway.

--
   Heikki Linnakangas
   EnterpriseDB   http://www.enterprisedb.com