Re: basic pgbench runs with various performance-related patches

On Mon, Jan 23, 2012 at 1:53 PM, Robert Haas <robertmhaas@gmail.com> wrote:

> Results are the median of three five-minute test runs

> checkpoint_timeout = 15min

Test duration is important for tests that don't relate to pure
contention reduction, which is every patch apart from XLogInsert.
We've discussed that before, so not sure what value you assign to
these results. Very little, is my view, so I'm a little disappointed
to see this post and the associated comments.

I'm very happy to see that your personal work has resulted in gains
and these results are valid tests of that work, IMHO. If you only
measure throughput you're only measuring half of what users care
about. We've not yet seen any tests that confirm that other important
issues have not been made worse.

Before commenting on individual patches its clear that the tests
you've run aren't even designed to highlight the BufFreelistLock
contention that is present in different configs, so that alone is
sufficient to throw most of this away.

On particular patches....

* background-clean-slru-v2 related very directly to reducing the
response time spikes you showed us in your last set of results. Why
not repeat those same tests??

* removebufmgrfreelist-v1 related to the impact of dropping
tables/index/databases, so given the variability of the results, that
at least shows it has no effect in the general case.

> And here are the results.  For everything against master, I've also
> included the percentage speedup or slowdown vs. the same test run
> against master.  Many of these numbers are likely not statistically
> significant, though some clearly are.

> with one exception: buffreelistlock-reduction-v1 crapped
> out during one of the test runs with the following errors

That patch comes with the proviso, stated in comments:
"We didn't get the lock, but read the value anyway on the assumption
that reading this value is atomic."
So we seem to have proved that reading it without the lock isn't safe.

The remaining patch you tested was withdrawn and not submitted to the CF.

Sigh.

--
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services