Re: Further pg_upgrade analysis for many tables

On Mon, Nov 12, 2012 at 03:59:27PM -0500, Bruce Momjian wrote:
> > The second approach would be to simply try to copy the fsm, vm, and
> > extent files, and ignore any ENOEXIST errors.  This allows code
> > simplification.  The downside is that it doesn't pull all files with
> > matching prefixes --- it requires pg_upgrade to _know_ what suffixes
> > might exist in that directory.  Second, it assumes there can be no
> > number gaps in the file extent numbering (is that safe?).
> > 
> > I need recommendations on which direction to persue;  this would only be
> > for 9.3.
> 
> I went with the second idea, patch attached.  Here are the times:
> 
>              ----------  9.2 ----------  ------------ 9.3 --------
>              -- normal -- -- bin-up --   -- normal -- -- bin-up --  pg_upgrade
>              dump   rest   dump   rest   dump   rest   dump   rest   git   patch
>       1      0.12   0.06   0.12   0.06   0.11   0.07   0.11   0.07  11.11  11.02
>    1000      7.22   2.40   4.74   2.78   2.20   2.43   4.04   2.86  19.60  19.25
>    2000      5.67   5.10   8.82   5.57   4.50   4.97   8.07   5.69  30.55  26.67
>    4000     13.34  11.13  25.16  12.52   8.95  11.24  16.75  12.16  60.70  52.31
>    8000     29.12  25.98  59.60  28.08  16.68  24.02  30.63  27.08 123.05 102.78
>   16000     87.36  53.16 189.38  62.72  31.38  55.37  61.55  62.66 365.71 286.00
> 
> You can see a significant speedup with those loops removed.  The 16k
> case is improved, but still not linear.  The 16k dump/restore scale
> looks fine, so it must be something in pg_upgrade, or in the kernel.

It is possible that the poor 16k pg_upgrade value is caused by the poor
9.2 binary-upgrade number (189.38).  Perhaps I need to hack up
pg_upgrade to allow a 9.3 to 9.3 upgrade to test this.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + It's impossible for everything to be true. +