Re: performance for high-volume log insertion

On Wed, Apr 22, 2009 at 4:37 PM, Stephen Frost <sfrost@snowman.net> wrote:
> Thanks for doing the work.  I had been intending to but hadn't gotten to
> it yet.

I'd done similar tests recently, for some batch import code, so it was
just a matter of recreating it.

>> separate inserts, no transaction: 21.21s
>> separate inserts, same transaction: 1.89s
>> 40 inserts, 100 rows/insert: 0.18s
>> one 40000-value insert: 0.16s
>> 40 prepared inserts, 100 rows/insert: 0.15s
>> COPY (text): 0.10s
>> COPY (binary): 0.10s
>
> What about 40000 individual prepared inserts?  Just curious about it.

40000 inserts, one prepared statement each (constructing the prepared
statement only once), in a single transaction: 1.68s

I'm surprised that there's any win here at all.

> Also, kind of suprised about COPY text vs. binary.  What was the data
> type in the table..?  If text, that makes sense, if it was an integer or
> something else, I'm kind of suprised.

Each row had one integer column.  I expect strings to be harder to
parse, since it's allocating buffers and parsing escapes, which is
usually more expensive than parsing an integer out of a string.  I'd
expect the difference to be negligible either way, though, and I'd be
interested in hearing about use cases where binary copying is enough
of a win to be worth the development cost of maintaining the feature.

On Wed, Apr 22, 2009 at 4:49 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Also, just to be clear: were the "40 insert" cases 40 separate
> transactions or one transaction?  I think the latter was meant but
> it's not 100% clear.

One transaction--the only case where I ran more than one transaction
was the first.

--
Glenn Maynard