Re: How to make ResourceOwnerForgetBuffer() O(1), instead of O(N^2) scale

On 10/03/2014 07:08 AM, Kouhei Kaigai wrote:
> Hello,
> 
> I recently got a trouble on development of my extension that utilizes
> the shared buffer when it released each buffer page.
> 
> This extension transfers contents of the shared buffers to GPU device
> using DMA feature, then kicks a device kernel code.

Wow, that sounds crazy.

> Once backend/extension calls ReadBuffer(), resowner.c tracks which
> buffer was referenced by the current resource owner, to ensure these
> buffers being released at end of the transaction.
> However, it seems to me implementation of resowner.c didn't assume
> many buffers are referenced by a particular resource owner simultaneously.
> It manages the buffer index using an expandable array, then looks up
> the target buffer by sequential walk but from the tail because recently
> pinned buffer tends to be released first.
> It made a trouble in my case. My extension pinned multiple thousands
> buffers, so owner->buffers[] were enlarged and takes expensive cost
> to walk on.
> In my measurement, ResourceOwnerForgetBuffer() takes 36 seconds in
> total during hash-joining 2M rows; even though hash-joining itself
> takes less than 16 seconds.
> 
> What is the best way to solve the problem?

How about creating a separate ResourceOwner for these buffer pins, and
doing a wholesale ResourceOwnerRelease() on it when you're done?

- Heikki