Re: postgres 8.4, COPY, and high concurrency

On Wed, Nov 14, 2012 at 1:01 PM, Jeff Janes <jeff.janes@gmail.com> wrote:
> On Wed, Nov 14, 2012 at 6:41 AM, Jon Nelson <jnelson+pgsql@jamponi.net> wrote:
>>
>> UPDATE: I have been able to replicate the issue. The parent table (the
>> one referenced in the LIKE portion of the CREATE TABLE statement) had
>> three indices.
>>
>> Now that I've been able to replicate the issue, are there tests that I
>> can perform that would be useful to people?
>> I will also try to build a stand-alone test.
>
> While the WAL is suppressed for the table inserts, it is not
> suppressed for the index inserts, and the index WAL traffic is enough
> to lead to contention.

Aha!

> I don't know why that is the case, it seems like the same method that
> allows us to bypass WAL for the table would work for the indices as
> well.  Maybe it is just that no one bothered to implement it.  After
> all, building the index after the copy will be even more efficient
> than building it before but by-passing WAL.

> But it does seem like the docs could at least be clarified here.

In general, then, would it be safe to say that concurrent (parallel)
index creation may be a source of significant WAL contention? I was
planning on taking advantage of this due to modern, beefy boxes with
10's of CPUs all just sitting there bored.


--
Jon