Re: Perf regression in 2.6.32 (Ubuntu 10.04 LTS)

On 28/09/10 04:28, Robert Haas wrote: <blockquote
cite="mid:AANLkTimPCHCb56Tdid_h9v6jZtCw5dn-1Sjz_r_=5+C2@mail.gmail.com"type="cite"><pre wrap="">On Mon, Sep 13, 2010 at
12:05PM, Greg Smith <a class="moz-txt-link-rfc2396E"
href="mailto:greg@2ndquadrant.com"><greg@2ndquadrant.com></a>wrote: </pre><blockquote type="cite"><pre
wrap="">DomasMituzas wrote:   </pre><blockquote type="cite"><pre wrap="">
 
I've been playing around today a lot with sysbench, and observed that
2.6.32 kernel supplied by Ubuntu is having perf regression with PG (which
does not affect MySQL), compared to 2.6.28 builds I have.
What I observed can be seen in a paste at
<a class="moz-txt-link-freetext"
href="http://p.defau.lt/?8_GQV82Pz3_SDZbNOdP93Q">http://p.defau.lt/?8_GQV82Pz3_SDZbNOdP93Q</a>(db12 is 2.6.28, db20 is
2.6.32-
 
2.6.32-24-server).
Machines are two socket quad-opterons 2356s.
oprofile output can be seen at <a class="moz-txt-link-freetext"
href="http://p.defau.lt/?OIR1vDFK4cze_fmBTQbV9w">http://p.defau.lt/?OIR1vDFK4cze_fmBTQbV9w</a>-
 
system has >20% of idle cpu, which is somewhere in the top symbol :)
     </pre></blockquote><pre wrap="">
Are you using the same filesystem setup on both setups?  And regardless,
what is that filesystem?  We know that between 2.6.28 and 2.6.32 the kernel
improved how it handles fsync requests in a good way from a reliability
perspective (to fix bugs that could cause data loss before), particularly on
ext4, so it's possible the regression you're seeing is just the expense of
handling things properly.

If you already have sysbench on there, I'd suggest comparing the two systems
by seeing how fast each can execute fsync requests:

sysbench --test=fileio --file-fsync-freq=1 --file-num=1
--file-total-size=16384 --file-test-mode=rndwr run | grep "Requests/sec"

To help distinguish whether this regression might be coming from the already
known changes in that area, or if it's instead from something that's
impacting CPU efficiency.

Also, it's easy to see a performance change of this size just from the
database files being on a different part of the disk if you didn't control
for that.  Disks are almost twice as fast at their beginning than their end
nowadays.   </pre></blockquote><pre wrap="">
Greg, have you run into any other evidence suggesting a problem with 2.6.32?
 </pre></blockquote><font size="-1"><font face="Helvetica"><br /> Not Greg (sorry), but this might be worth a look:<br
/><br/><a class="moz-txt-link-freetext"
href="http://www.spinics.net/lists/linux-ext4/msg20299.html">http://www.spinics.net/lists/linux-ext4/msg20299.html</a><br
/><br/> regards<br /><br /> Mark<br /></font></font>