Обсуждение: pg_upgrade support is broken?
Hi
While reading the setup script, I noticed that to initdb the new cluster
for pg_upgrade runs the following:
# Perform initdb on the new server
$PGENGINE/postgresql92-setup initdb
Also, PREVPGENGINE isn't referenced; instead the pg_upgrade code uses
PGPREVENGINE .
This suggests that pg_upgrade support via the initscript has been broken
throughout 9.3 and is still broken in 9.4, even for direct upgrades from
the immediately previous major.
It also uses 'su' instead of 'runuser', so I think it'd have issues on
SELinux.
It doesn't look like upgrades from other than the immediately previous
major could ever have worked:
- Support for specifying a non-default version to upgrade seems broken.
Lots of the code continues to use $PREVMAJORVERSION despite it not being
set if the user passes a $OLD_SERVICE_NAME argument.
- it assumes that the previous binaries are in
/usr/pgsql-$PREVMAJORVERSION/bin but they aren't if the user passed a
$OLD_SERVICE_NAME other than that of $PREVMAJORVERSION
- The issue with PREVPGENGINE vs PGPREVENGINE above
Also, $OLD_SERVICE_NAME must be the basename of the initscript/service
control file, e.g. postgresql-9.3 , not just "9.3". This isn't super
obvious from the help.
Support for upgrading from the immediately prev version looks to be
fixable by replacing:
# Perform initdb on the new server
$PGENGINE/postgresql92-setup initdb
with
# Perform initdb on the new server
$PGENGINE/postgresql${PREVMAJORVERSION/.}-setup initdb
+ using "$SU" instead of "su"
Fixing upgrading from earlier versions also needs PREVMAJORVERSION to be
overridden if the user specifies a control file explicitly and a bunch
of other fixes.
I attach a majorly reworked script for your review and comment. I
haven't done much testing yet.
It requires the rpm builds script to sub in some new variables:
PREVMAJORVERSION=%{prevmajorversion}
PGUNITNAME=%{unitname}
PGPACKAGENAME=%{name}
so a wrapper like:
substitute_vars() {
# Insert RPM variables into shell scripts
sed -e 's|^PGVERSION=.*$|PGVERSION=%{version}|' \
-e 's|^PGENGINE=.*$|PGENGINE=%{pgbaseinstdir}/bin|' \
-e 's|^PGUNITNAME=.*$|PGUNITNAME=%{unitname}|' \
-e 's|^PREVMAJORVERSION=.*$|PREVMAJORVERSION=%{prevmajorversion}|' \
-e 's|^PGPACKAGENAME=.*$|PGPACKAGENAME=%{name}|' \
-e 's|^PGDOCDIR=.*$|PGDOCDIR=%{_pkgdocdir}|' \
< "$1" > "$2"
}
# prep the setup script, including insertion of some values it needs
substitute_vars "%{SOURCE17}" "%{name}-setup"
might be useful.
BTW, the README.rpm-dist looks like it's a victim of some copy-and-paste
too; see:
```
In 9.1+, the RPMs also support in-place upgrade from th immediately
previous major release. Currently, you can upgrade in-place
from 9.2.x to 9.4.x. Just run:
$ /usr/pgsql-9.4/bin/postgresql94-setup upgrade
Please note that 9.2 and 9.4 contrib RPMs needs to be installed for this
feature to work.
```
That should be "9.3" not "9.2" and there's a typo in "th[e]".
Anyway. Thoughts on the updated scripts? I've combined the sysv and
systemd scripts to reduce duplication.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Вложения
Hi Craig, On Fri, 2014-08-01 at 22:15 +0800, Craig Ringer wrote: > Anyway. Thoughts on the updated scripts? I've combined the sysv and > systemd scripts to reduce duplication. What is "__RPM_PGENGINE_ " and other similar stuff in both of these patches? I am ok to apply these, once I have an understading how they are expanded into real paths or so. Cheers, -- Devrim GÜNDÜZ Principal Systems Engineer @ EnterpriseDB: http://www.enterprisedb.com PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer Twitter: @DevrimGunduz , @DevrimGunduzTR
Вложения
On 09/11/2014 05:09 AM, Devrim Gündüz wrote: > > Hi Craig, > > On Fri, 2014-08-01 at 22:15 +0800, Craig Ringer wrote: > >> Anyway. Thoughts on the updated scripts? I've combined the sysv and >> systemd scripts to reduce duplication. > > What is "__RPM_PGENGINE_ " and other similar stuff in both of these > patches? I am ok to apply these, once I have an understading how they > are expanded into real paths or so. Best hold off a little, I have a considerably cleaned up and updated version nearly ready, based off the BDR release packages with the BDR stuff stripped out. I'll move all the substitutions into var declarations at the top of each script for clarity. Should've done it earlier. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
Hi, On Thu, 2014-09-11 at 12:11 +0800, Craig Ringer wrote: > Best hold off a little, I have a considerably cleaned up and updated > version nearly ready, based off the BDR release packages with the BDR > stuff stripped out. > > I'll move all the substitutions into var declarations at the top of > each script for clarity. Should've done it earlier. Thanks. Do you think you can complete this in next few weeks? I don't think 9.4.0 is too far away. Regards, -- Devrim GÜNDÜZ Principal Systems Engineer @ EnterpriseDB: http://www.enterprisedb.com PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer Twitter: @DevrimGunduz , @DevrimGunduzTR
Вложения
On 09/23/2014 06:42 PM, Devrim Gündüz wrote: > > Hi, > > On Thu, 2014-09-11 at 12:11 +0800, Craig Ringer wrote: >> Best hold off a little, I have a considerably cleaned up and >> updated version nearly ready, based off the BDR release packages >> with the BDR stuff stripped out. >> >> I'll move all the substitutions into var declarations at the top >> of each script for clarity. Should've done it earlier. > > Thanks. Do you think you can complete this in next few weeks? I > don't think 9.4.0 is too far away. Yes. It's been on my "tomorrow" list for days now, but urgent things keep popping up. I'll make an hour or two for it tomorrow one way or the other. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On Fri, 2014-08-01 at 22:15 +0800, Craig Ringer wrote: Hi, you may be interested about Pavel's recent work on postgresql-setup [1] that we are using in Fedora and planing to use in RHEL. There is a lot of work to support migration scenarios from different versions. [1] https://praiskup.fedorapeople.org/proj/postgresql-setup/ > Hi > > While reading the setup script, I noticed that to initdb the new > cluster > for pg_upgrade runs the following: > > # Perform initdb on the new server > $PGENGINE/postgresql92-setup initdb > > > Also, PREVPGENGINE isn't referenced; instead the pg_upgrade code uses > PGPREVENGINE . > > This suggests that pg_upgrade support via the initscript has been > broken > throughout 9.3 and is still broken in 9.4, even for direct upgrades > from > the immediately previous major. > > It also uses 'su' instead of 'runuser', so I think it'd have issues > on > SELinux. > > > > It doesn't look like upgrades from other than the immediately > previous > major could ever have worked: > > - Support for specifying a non-default version to upgrade seems > broken. > Lots of the code continues to use $PREVMAJORVERSION despite it not > being > set if the user passes a $OLD_SERVICE_NAME argument. > > - it assumes that the previous binaries are in > /usr/pgsql-$PREVMAJORVERSION/bin but they aren't if the user passed a > $OLD_SERVICE_NAME other than that of $PREVMAJORVERSION > > - The issue with PREVPGENGINE vs PGPREVENGINE above > > > > > Also, $OLD_SERVICE_NAME must be the basename of the > initscript/service > control file, e.g. postgresql-9.3 , not just "9.3". This isn't super > obvious from the help. > > > Support for upgrading from the immediately prev version looks to be > fixable by replacing: > > > # Perform initdb on the new server > $PGENGINE/postgresql92-setup initdb > > with > > # Perform initdb on the new server > $PGENGINE/postgresql${PREVMAJORVERSION/.}-setup initdb > > + using "$SU" instead of "su" > > > Fixing upgrading from earlier versions also needs PREVMAJORVERSION to > be > overridden if the user specifies a control file explicitly and a > bunch > of other fixes. > > I attach a majorly reworked script for your review and comment. I > haven't done much testing yet. > > It requires the rpm builds script to sub in some new variables: > > PREVMAJORVERSION=%{prevmajorversion} > PGUNITNAME=%{unitname} > PGPACKAGENAME=%{name} > > > so a wrapper like: > > substitute_vars() { > # Insert RPM variables into shell scripts > sed -e 's|^PGVERSION=.*$|PGVERSION=%{version}|' \ > -e 's|^PGENGINE=.*$|PGENGINE=%{pgbaseinstdir}/bin|' \ > -e 's|^PGUNITNAME=.*$|PGUNITNAME=%{unitname}|' \ > -e > 's|^PREVMAJORVERSION=.*$|PREVMAJORVERSION=%{prevmajorversion}|' \ > -e 's|^PGPACKAGENAME=.*$|PGPACKAGENAME=%{name}|' \ > -e 's|^PGDOCDIR=.*$|PGDOCDIR=%{_pkgdocdir}|' \ > < "$1" > "$2" > } > > # prep the setup script, including insertion of some values it needs > substitute_vars "%{SOURCE17}" "%{name}-setup" > > > > might be useful. > > > > > > > BTW, the README.rpm-dist looks like it's a victim of some copy-and > -paste > too; see: > > > ``` > In 9.1+, the RPMs also support in-place upgrade from th immediately > previous major release. Currently, you can upgrade in-place > from 9.2.x to 9.4.x. Just run: > $ /usr/pgsql-9.4/bin/postgresql94-setup upgrade > > Please note that 9.2 and 9.4 contrib RPMs needs to be installed for > this > feature to work. > ``` > > That should be "9.3" not "9.2" and there's a typo in "th[e]". > > > > Anyway. Thoughts on the updated scripts? I've combined the sysv and > systemd scripts to reduce duplication. >
On Wed, 2015-05-13 at 08:52 +0200, Jozef Mlich wrote: Sorry, I didn't noticed, that I am replying on old message .. > On Fri, 2014-08-01 at 22:15 +0800, Craig Ringer wrote: > > > Hi, > > you may be interested about Pavel's recent work on postgresql-setup > [1] > that we are using in Fedora and planing to use in RHEL. > > There is a lot of work to support migration scenarios from different > versions. > > [1] https://praiskup.fedorapeople.org/proj/postgresql-setup/ > > > > Hi > > > > While reading the setup script, I noticed that to initdb the new > > cluster > > for pg_upgrade runs the following: > > > > # Perform initdb on the new server > > $PGENGINE/postgresql92-setup initdb > > > > > > Also, PREVPGENGINE isn't referenced; instead the pg_upgrade code > > uses > > PGPREVENGINE . > > > > This suggests that pg_upgrade support via the initscript has been > > broken > > throughout 9.3 and is still broken in 9.4, even for direct upgrades > > > > from > > the immediately previous major. > > > > It also uses 'su' instead of 'runuser', so I think it'd have issues > > > > on > > SELinux. > > > > > > > > It doesn't look like upgrades from other than the immediately > > previous > > major could ever have worked: > > > > - Support for specifying a non-default version to upgrade seems > > broken. > > Lots of the code continues to use $PREVMAJORVERSION despite it not > > being > > set if the user passes a $OLD_SERVICE_NAME argument. > > > > - it assumes that the previous binaries are in > > /usr/pgsql-$PREVMAJORVERSION/bin but they aren't if the user passed > > a > > $OLD_SERVICE_NAME other than that of $PREVMAJORVERSION > > > > - The issue with PREVPGENGINE vs PGPREVENGINE above > > > > > > > > > > Also, $OLD_SERVICE_NAME must be the basename of the > > initscript/service > > control file, e.g. postgresql-9.3 , not just "9.3". This isn't > > super > > obvious from the help. > > > > > > Support for upgrading from the immediately prev version looks to be > > fixable by replacing: > > > > > > # Perform initdb on the new server > > $PGENGINE/postgresql92-setup initdb > > > > with > > > > # Perform initdb on the new server > > $PGENGINE/postgresql${PREVMAJORVERSION/.}-setup initdb > > > > + using "$SU" instead of "su" > > > > > > Fixing upgrading from earlier versions also needs PREVMAJORVERSION > > to > > be > > overridden if the user specifies a control file explicitly and a > > bunch > > of other fixes. > > > > I attach a majorly reworked script for your review and comment. I > > haven't done much testing yet. > > > > It requires the rpm builds script to sub in some new variables: > > > > PREVMAJORVERSION=%{prevmajorversion} > > PGUNITNAME=%{unitname} > > PGPACKAGENAME=%{name} > > > > > > so a wrapper like: > > > > substitute_vars() { > > # Insert RPM variables into shell scripts > > sed -e 's|^PGVERSION=.*$|PGVERSION=%{version}|' \ > > -e 's|^PGENGINE=.*$|PGENGINE=%{pgbaseinstdir}/bin|' \ > > -e 's|^PGUNITNAME=.*$|PGUNITNAME=%{unitname}|' \ > > -e > > 's|^PREVMAJORVERSION=.*$|PREVMAJORVERSION=%{prevmajorversion}|' \ > > -e 's|^PGPACKAGENAME=.*$|PGPACKAGENAME=%{name}|' \ > > -e 's|^PGDOCDIR=.*$|PGDOCDIR=%{_pkgdocdir}|' \ > > < "$1" > "$2" > > } > > > > # prep the setup script, including insertion of some values it > > needs > > substitute_vars "%{SOURCE17}" "%{name}-setup" > > > > > > > > might be useful. > > > > > > > > > > > > > > BTW, the README.rpm-dist looks like it's a victim of some copy-and > > -paste > > too; see: > > > > > > ``` > > In 9.1+, the RPMs also support in-place upgrade from th immediately > > previous major release. Currently, you can upgrade in-place > > from 9.2.x to 9.4.x. Just run: > > $ /usr/pgsql-9.4/bin/postgresql94-setup upgrade > > > > Please note that 9.2 and 9.4 contrib RPMs needs to be installed for > > > > this > > feature to work. > > ``` > > > > That should be "9.3" not "9.2" and there's a typo in "th[e]". > > > > > > > > Anyway. Thoughts on the updated scripts? I've combined the sysv and > > systemd scripts to reduce duplication. > > > > >
Hi, On Wed, 2015-05-13 at 08:53 +0200, Jozef Mlich wrote: > Sorry, I didn't noticed, that I am replying on old message .. It is ok. Our upgrade script is very broken already. A new approach would be good. I'll check. Regards, -- Devrim GÜNDÜZ Principal Systems Engineer @ EnterpriseDB: http://www.enterprisedb.com PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer Twitter: @DevrimGunduz , @DevrimGunduzTR