Обсуждение: Unifying the spec files?

Hi

While looking over the spec files for the releases I noticed that the
_vast_ majority of content is the same for each spec.

Where differences exist it isn't always clear why. For example, EL-7
seems to generate an init script differently to EL-6, and the EL-7
packages have postgres_fdw and sepgsql but EL-6 doesn't.

I'm a little curious about why the spec files seem to be copied and
edited, rather than having a single spec for the package across all
supported versions and using conditionals. I'm sure you've thought of
that, so there'll be good reasons - did you try this and run into issues?

As I'm facing the need to do multi-distro patch releases (hotfixes,
basically) where I'll need to rapidly produce new RPMs for each
supported distro, these differences are not fun to deal with, so if
there's a way to use a single spec file and just use conditionals I'd be
pretty happy.

e.g. SuSE defines a %{suse_version} macro, RHEL defines %{rhel} for the
RHEL release, %{fedora} for the fedora release.


rpmbuild --showrc will report defined macros.

If used for builds, mock will define appropriate macros either in the
chroot or by calling rpmbuild.

If I prep a unified spec for postgresql would you be willing to review
and give it a go?



Useful info:


http://rpm5.org/docs/rpm-guide.html#id3039571
http://en.opensuse.org/openSUSE:RPM_conditional_builds

http://fedoraproject.org/wiki/Packaging:DistTag

https://fedoraproject.org/wiki/Packaging:RPMMacros?rd=Packaging/RPMMacros

http://backreference.org/2011/09/17/some-tips-on-rpm-conditional-macros/

--
 Craig Ringer                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

Hi Craig,

On Tue, 2014-07-29 at 09:46 +0800, Craig Ringer wrote:

> While looking over the spec files for the releases I noticed that the
> _vast_ majority of content is the same for each spec.
>
> Where differences exist it isn't always clear why. For example, EL-7
> seems to generate an init script differently to EL-6,

EL-7 does not have init script ;) It uses systemd'd unit file.
Basically, right now EL-5 and EL-6 is one group, and EL-7,F-19 and F-20
are another group when it comes to features/init scripts, etc.

> and the EL-7 packages have postgres_fdw and sepgsql but EL-6 doesn't.

AFAICS EL-6 has postgres_fdw, too.

The selinux-devel version is EL-6 does not satisfy the needs for sepgsql
module. That is the reason why we don't build it there. There are slight
other differences IIRC.

> I'm a little curious about why the spec files seem to be copied and
> edited, rather than having a single spec for the package across all
> supported versions and using conditionals. I'm sure you've thought of
> that, so there'll be good reasons - did you try this and run into
> issues?

I have tried this *years* ago, but it did not work, since some
conditionals did not work across distros.

I just looked at a few spec files in Fedora/EPEL, and it seems that
issue has been fixed now.

> If I prep a unified spec for postgresql would you be willing to review
> and give it a go?

Of course, appreciated. Please let me know if you need help.

Thanks!

Regards,
--
Devrim GÜNDÜZ
Principal Systems Engineer @ EnterpriseDB: http://www.enterprisedb.com
PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer
Twitter: @DevrimGunduz , @DevrimGunduzTR

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 07/29/2014 01:06 PM, Devrim Gündüz wrote:
>
> Hi Craig,
>
> On Tue, 2014-07-29 at 09:46 +0800, Craig Ringer wrote:
>
>> While looking over the spec files for the releases I noticed
>> that the _vast_ majority of content is the same for each spec.
>>
>> Where differences exist it isn't always clear why. For example,
>> EL-7 seems to generate an init script differently to EL-6,
>
> EL-7 does not have init script ;) It uses systemd'd unit file.
> Basically, right now EL-5 and EL-6 is one group, and EL-7,F-19 and
> F-20 are another group when it comes to features/init scripts,
> etc.

Yeah, I worked that one out after posting.

> The selinux-devel version is EL-6 does not satisfy the needs for
> sepgsql module. That is the reason why we don't build it there.
> There are slight other differences IIRC.

OK, makes sense.

>> I'm a little curious about why the spec files seem to be copied
>> and edited, rather than having a single spec for the package
>> across all supported versions and using conditionals. I'm sure
>> you've thought of that, so there'll be good reasons - did you
>> try this and run into issues?
>
> I have tried this *years* ago, but it did not work, since some
> conditionals did not work across distros.

Yes, it seems lots of things really work best with rpm5.

Take a look at the attached, which is a first cut unification attempt.
I'm going to do mock builds and test installs for each arch now, but
figured I'd send it for comment as well.

I've versioned the Provides: and made some other rpmlint fixes while I
was at it.

I'd also like to parameterise the package basename ("postgresql") to
allow variants that still have "Provides: postgresql" and "Provides:
postgresql-%{majorversion}" but live in a separate datadir. The
immediate reason is to permit me to package BDR, which is 9.4 but has
a few catalog changes that mean it can't run from a straight 9.4
datadir, without stomping on PGDG packages and without the need to
rebuild all the dependencies. Thoughts?

- --
 Craig Ringer                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJT2ISYAAoJELBXNkqjr+S2bBgIAJoXCUnQNAGDR5OGYj/SbiQq
VJu6bGWB8BRX8gmDqxd22qGlV7e8DC423jOKwXClcMoNor+whKFS0nRwB3BW43m9
14IqyXECpCSnXFHB/MRTkG/Tww+TqJDQ7e0ywX6XCwVsHIRqUMm2Bycy43NEDPoA
cpLFd7kF3Np+UuT5MxnjxyilJ3ga2OiE5+Rt5gNBNObEmPj4OTAhV0lNi3n2frum
62qu9uNREAWBjxTjH0ahKVdQVMYZPji4G29mT4UZI5gjqhM+emwKniEZqesC6y4Q
V/VuEJWW8kRPOX+9yoQ2/Ymra82Z0oSqOza/g5HZLX9pGMOsVMfm13GXVDV0kjU=
=1DOT
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

(Whoops, attached the full tar file instead of just the spec).

On 07/29/2014 01:06 PM, Devrim wrote:
>
> Hi Craig,
>
> On Tue, 2014-07-29 at 09:46 +0800, Craig Ringer wrote:
>
>> While looking over the spec files for the releases I noticed that
>> the _vast_ majority of content is the same for each spec.
>>
>> Where differences exist it isn't always clear why. For example,
>> EL-7 seems to generate an init script differently to EL-6,
>
> EL-7 does not have init script ;) It uses systemd'd unit file.
> Basically, right now EL-5 and EL-6 is one group, and EL-7,F-19 and
>  F-20 are another group when it comes to features/init scripts,
> etc.

Yeah, I worked that one out after posting.

> The selinux-devel version is EL-6 does not satisfy the needs for
> sepgsql module. That is the reason why we don't build it there.
> There are slight other differences IIRC.

OK, makes sense.

>> I'm a little curious about why the spec files seem to be copied
>> and edited, rather than having a single spec for the package
>> across all supported versions and using conditionals. I'm sure
>> you've thought of that, so there'll be good reasons - did you try
>> this and run into issues?
>
> I have tried this *years* ago, but it did not work, since some
> conditionals did not work across distros.

Yes, it seems lots of things really work best with rpm5.

Take a look at the attached, which is a first cut unification attempt.
I'm going to do mock builds and test installs for each arch now, but
figured I'd send it for comment as well.

I've versioned the Provides: and made some other rpmlint fixes while I
was at it.

I'd also like to parameterise the package basename ("postgresql") to
allow variants that still have "Provides: postgresql" and "Provides:
postgresql-%{majorversion}" but live in a separate datadir. The
immediate reason is to permit me to package BDR, which is 9.4 but has
a few catalog changes that mean it can't run from a straight 9.4
datadir, without stomping on PGDG packages and without the need to
rebuild all the dependencies. Thoughts?

- --
 Craig Ringer                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJT2IkzAAoJELBXNkqjr+S2iqoH/jIlEpvK+hNqQqB7wttJU1Uo
Qm0ier4Y5aJwU7e8OOBPASfodZSxtpwSzn+ttnSfY7x+6/WnIgn5mkCA4KmeVAQO
u8hv//0apNF5Op8w5PKjKw6ds7MDQmfDfVcpf+w9767zqTCk6m9xSoIDX05h0w1S
l0ctD0A7QKb2+D98RyYtFhWF5cBC7Mo08wiIA2X/Htu9sZf7t6SlxpHJksCkHC/D
0uchYXaHdx5KYbzdOQfti2zOAjDw56oqbBehVfVZxjlIBM6usutxW54oUN5t4A/o
7H+ZYSejf0Ik1FBy2hgdqtjzeVaIHCYhIynlPwIR7ZJsMN6vIpjmUEq0QoLE9O4=
=ylcF
-----END PGP SIGNATURE-----

On Wed, 2014-07-30 at 13:37 +0800, Craig Ringer wrote:
>
> Take a look at the attached, which is a first cut unification attempt.
> I'm going to do mock builds and test installs for each arch now, but
> figured I'd send it for comment as well.


Thanks ;) This is *really* great work!

I built the packages on Fedora 20. Here are some initial comments:

============
# It should be supplied externally from mock/koju, rpmbuild macros, etc.
s/koju/koji
============

============
$ rpm -qp --scripts postgresql94-server-9.4beta2-3PGDG.f20.x86_64.rpm |grep postgresql94
    /bin/systemctl --no-reload disable postgresql94-9.4.service >/dev/null 2>&1 || :
    /bin/systemctl stop postgresql94-9.4.service >/dev/null 2>&1 || :
    /bin/systemctl try-restart postgresql94-9.4.service >/dev/null 2>&1 || :

* Unit file name is postgresql-9.4.service.
* I think using /usr/bin/systemctl would be much better.
============

============
While working on it, can you please get rid of %kerbdir variable?
Currently:
%{!?kerbdir:%define kerbdir "/usr"}
    --with-includes=%{kerbdir}/include \
    --with-libraries=%{kerbdir}/%{_lib} \

If we remove them, it will be
    --with-includes=%{_includedir} \
    --with-libraries=%{_libdir} \
============

============
In line 80 it says:
%if  0%{?rhel} >= 7 || 0%{?fedora}

Let's change it to

%if  0%{?rhel} >= 7 || 0%{?fedora} => 19

for consistency.
=============

=============
While working on it can you please change

BuildRequires:  perl(ExtUtils::MakeMaker)
to
BuildRequires:  perl-ExtUtils-MakeMaker

for cosmetic reasons :-)
=============

> I've versioned the Provides:

I am not whether this is right or not. OS-related packages may be
broken, since we supply versioned Provides... Did you try installing
libreoffice, for example?

> and made some other rpmlint fixes while I was at it.

Thanks!

> I'd also like to parameterise the package basename ("postgresql") to
> allow variants that still have "Provides: postgresql" and "Provides:
> postgresql-%{majorversion}" but live in a separate datadir. The
> immediate reason is to permit me to package BDR, which is 9.4 but has
> a few catalog changes that mean it can't run from a straight 9.4
> datadir, without stomping on PGDG packages and without the need to
> rebuild all the dependencies. Thoughts?

Can you please explain this more briefly?

Regards,

--
Devrim GÜNDÜZ
Principal Systems Engineer @ EnterpriseDB: http://www.enterprisedb.com
PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer
Twitter: @DevrimGunduz , @DevrimGunduzTR

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 07/30/2014 04:35 PM, Devrim Gündüz wrote:
> On Wed, 2014-07-30 at 13:37 +0800, Craig Ringer wrote:
>>
>> Take a look at the attached, which is a first cut unification
>> attempt. I'm going to do mock builds and test installs for each
>> arch now, but figured I'd send it for comment as well.
>
>
> Thanks ;) This is *really* great work!

Glad to hear it.

> # It should be supplied externally from mock/koju, rpmbuild
> macros, etc. s/koju/koji

Er. Rather.

> ============ $ rpm -qp --scripts
> postgresql94-server-9.4beta2-3PGDG.f20.x86_64.rpm |grep
> postgresql94 /bin/systemctl --no-reload disable
> postgresql94-9.4.service >/dev/null 2>&1 || : /bin/systemctl stop
> postgresql94-9.4.service >/dev/null 2>&1 || : /bin/systemctl
> try-restart postgresql94-9.4.service >/dev/null 2>&1 || :
>
> * Unit file name is postgresql-9.4.service. * I think using
> /usr/bin/systemctl would be much better. ============

I think both those were original, unless I mangled it when merging.
Will fix, anyway.

> ... series of other edits ...

No problem.

>> I've versioned the Provides:
>
> I am not whether this is right or not. OS-related packages may be
> broken, since we supply versioned Provides... Did you try
> installing libreoffice, for example?

I'll need to do more testing on this. It shouldn't be an issue;
unversioned Requires: will work as-is, at least, and versioned ones
that require >= should be fine.

I guess if someone Requires: postgresql = 9.3 it might be an issue.
I'll see if I can figure out how to query RPM's dependency graph so I
can see what people are declaring dependencies on.

>> I'd also like to parameterise the package basename
>> ("postgresql") to allow variants that still have "Provides:
>> postgresql" and "Provides: postgresql-%{majorversion}" but live
>> in a separate datadir. The immediate reason is to permit me to
>> package BDR, which is 9.4 but has a few catalog changes that mean
>> it can't run from a straight 9.4 datadir, without stomping on
>> PGDG packages and without the need to rebuild all the
>> dependencies. Thoughts?
>
> Can you please explain this more briefly?

Sure.

Sometimes I need to push a package containing a hotfix to someone
before it hits mainline and gets officially packaged and released in
the next point release, or need to supply a backported fix for a
version that's no longer under official support. That's not really any
fuss, as it's only minor changes and these sorts of things don't
affect the datadir format etc. So for things like this I just set the
Release: tag to reflect that it's a 2ndQuadrant package when I rebuild.

I'm in the process of prepping packages for BDR, and it's more
complicated there. BDR changes the catalog format to add support for
the sequence access method infrastructure that didn't make 9.4, so
while it's PostgreSQL 9.4 as far as the client libraries, protocol,
etc go the datadir format is _slightly_ different.

Because of that I'm not comfortable doing what I'd usually do and
rolling a "postgresql94-9.4beta2-1bdr" like I'd usually prepare a
"postgresql94-9.4beta2-1_2ndquadrant_lwlocks" or whatever. It's not
just a release of essentially the same code; it has a new extension
and enough core changes that it's not 100% on disk compatible.

So I'm trying to figure out how to do a release that doesn't create
problems and still inter-operates with all the existing packages in
yum.postgresql.org so it isn't necessary to rebuild and maintain
*everything*, especially as BDR is destined to merge back into core
over 9.5/9.6.

The libpq etc in BDR are totally compatible, it's only the
server/datadir that isn't. Some arguments are added to pg_dump and
pg_resetxlog, but nothing was removed/broken and no existing output
was changed. So, except for packaged backend extensions, it should be
fine to just install existing packages.

OTOH, it's also clearly wrong to permit an install of BDR to overwrite
a regular 9.4 install and attempt to use the same datadir.

So what I'm trying to do is find a way to produce packages that:

- - Meet Requires: for anyone who needs PostgreSQL clients, libs, etc

- - Use a different data directory

- - Conflict with the official upstream postgresql-server, or co-exist
  with it using different service/initscript names, etc.

and do so with minimal mess and confusion.


I'm thinking of giving it a different package name like
postgresql-bdr94 and adding lines like:

  Provides: postgresql94

for everything except postgresql-server, which won't provide
postgresql94-server and will conflict with it.

There are lots of assumptions burned into the package about the naming
of datadir, paths, etc, though, so I'm not sure how practical a new
top package name will be.

If I can make it work, I'd like to parameterise the top package name
in the specfile, so it can be easily overridden by releasers.



- --
 Craig Ringer                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJT2LYCAAoJELBXNkqjr+S2sucIALVHT49XQjuz35EbbBDTqrEL
ux68PDjN9Hn8YltedA5z/pLABO4oh+JwjNkjOGVEV3y1mgXI2u4Rar0upp1rfsfI
QmbKPU/VXZT/IFcZbFtr7LorfDnaHS1Bsp9t1tEQSa7GcNsO3DqkqZTccUvlHxG1
0C3zYilgbYMOTCMxU+1Yvdodijh3NSJ2DEuFbaMz+/gOmJmWHTBLzXMCLn8sYZvW
9Twklog4EmGgP0/pIg9i3GoIJfPrsOiU7fN9rNxLfyzqDmPFV39Gznhd+71MNxJW
SoscQu/NtSl/82m3KP4/vcQpbmkrVI9MzJyCZ8s7HhOfq/brPvJXu1Cl3SvPBnk=
=/Kla
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 07/30/2014 05:08 PM, Craig Ringer wrote:
> On 07/30/2014 04:35 PM, Devrim Gündüz wrote:
>> On Wed, 2014-07-30 at 13:37 +0800, Craig Ringer wrote:
>>>
>>> Take a look at the attached, which is a first cut unification
>>> attempt. I'm going to do mock builds and test installs for each
>>>  arch now, but figured I'd send it for comment as well.
>
>
>> Thanks ;) This is *really* great work!
>
> Glad to hear it.

Just a quick update on this.

So far it's looking really promising.

Because of
https://bugzilla.redhat.com/show_bug.cgi?id=490613
it's necessary to still do separate .src.rpm builds for older
RHEL/CentOS and for Fedora. mock can take care of that though, just

   mock -r epel-5-i386 \
        --buildsrpm \
        --spec rpm/postgresql-x.y.spec \
        --sources rpm/

or whatever.

Still some bugs to fix in the unified package and more testing to do
but I'm happy with how things are looking. I'll probably need to make
new mock targets too, of course, but that looks trivial.

- --
 Craig Ringer                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJT2RGsAAoJELBXNkqjr+S2tAkIAIY182WgXo2wsIjtxc4ABTCZ
ehHyhs07u9RupQkqbU1xIKzKEUomTgI4d22FNpGCfkAldXt7WTewr2rAD7LitMi7
pgerZZPXPmtlbyy+E/7fMOCg3vxUmROD0FR6g+MK4O7/q03Rs8gxAjivXsJWkG9G
GfWYmuxVu0km3M62rpoK4B/6BVIe59J4Blr1F2EYmSi4nW4SJUGuylpNlXxLAW4n
ZZ2F1VDoWL1WdL9niQIC8paRH/ZRplaC4oRlLwPmFQsAPAD/bmuG0AhQLCfBMrJr
zc4GvdnZGgJb0TFZCE9rDkmm8/7y08uYgRczrKbRvm8i8d8jSZeyvwkzx9H1rnY=
=BULG
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 07/30/2014 04:35 PM, Devrim Gündüz wrote:
> * Unit file name is postgresql-9.4.service.

It's probably a bit late, but should it be?

If the package is postgresql94, such that %{name} is postgresql94,
shouldn't the unit file be postgresql94.service?

I'll change in the spec anyway, just interested in comments on that.

- --
 Craig Ringer                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJT2hLGAAoJELBXNkqjr+S2CJ4H+wcUysQEnMshqmk0Ape6zmqG
8jLQG7tlIjw5B1x8XTSnLdvhL8fnXL1pFL+0phFvruxldXmGb2Ma/QIjUGQExQFS
mVXGcSpJSPu1ypSkBqS+pvEWBg4cgsvQFIpLVbxscQYFq0ZkrqFsWh0b7i80KIR4
nYsdDCdX935+BT62vSxipynZ3XIm2y48YbsH13ctKHTHpERGsD3g+7gXq4Yv4XIM
M3kUZVkRZ4MYG7cW+Nsq4oGfVMb5m3wUKXHvYtY2jSZFTp7jyRI95HiRxK/BJNtF
hLnX3KU5FxwS69MQ+uFtgnYoXzPV7i7YxzMHpWpeVDoVijMU8iSRU18qXTYwDHU=
=EQBf
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 07/30/2014 04:35 PM, Devrim Gündüz wrote:

> While working on it can you please change
>
> BuildRequires:  perl(ExtUtils::MakeMaker) to BuildRequires:
> perl-ExtUtils-MakeMaker
>
> for cosmetic reasons :-)

I think it's meant to be the other way around actually.

The first says "I want a package named this".

The second says "I want this Perl module".

so I'd rather

BuildRequires:  perl(ExtUtils::Embed)
BuildRequires:  perl(ExtUtils::MakeMaker)

>> I've versioned the Provides:
>
> I am not whether this is right or not. OS-related packages may be
> broken, since we supply versioned Provides... Did you try
> installing libreoffice, for example?

Haven't tested this yet, still dealing with some other work items.

- --
 Craig Ringer                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJT2iXlAAoJELBXNkqjr+S2AGYH/1BVX+My0uaEZnNdi4FgHpZn
2W5gjlOTGCfNXEOZkLLBNLKZt2+LnFJKEM2SPSIEiIPcEdvkDtUUozW7PM0nYIAO
fiRUBY8/V4MU1crvt10NJVuNUeOkf9rrzgrSUIaMSdozuwdcWY2MrKT9edtnfZ5V
EjvVnvRpPSegLhcDJfud5tY2cdnkT0gX0E/5INX0RGMCI09RRwQYXwPmgHsDXZbb
FrCMDtMeP6zws6BzaaXMYoCFetMz09cpcZiDCTfn1PNxTATM8S2ncJKNHY2lf6Id
mlwsOt+jX3Sb7wj8ziM1P5dC3cHsx5xuCu3XVXf7CjvwUjdWyGX4mAUzYS/x2ik=
=8piV
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 07/30/2014 11:39 PM, Craig Ringer wrote:
> Still some bugs to fix in the unified package and more testing to
> do but I'm happy with how things are looking. I'll probably need to
> make new mock targets too, of course, but that looks trivial.

I'm down to testing now. Looking good so far.

I removed the addition of -Wl,--as-needed  as configure.in knows to
test for it and add it if it's safe. Forcing it broke RHEL5 builds
because of

    https://sourceware.org/ml/binutils/2005-05/msg00616.html and
    https://bugzilla.redhat.com/show_bug.cgi?id=499837

and was unnecessary anyway.

Updated spec attached with the changes we discussed.

I renamed files source  containing the Pg version to omit it too, so
it's easier to repackage for different versions. So the sources are:

ecpg_config.h
filter-requires-perl-Pg.sh
Makefile.regress
pg_config.h
postgresql-9.4.spec
postgresql-check-db-dir
postgresql.init
postgresql-logging.patch
postgresql.pam
postgresql-perl-rpath.patch
postgresql-prefer-ncurses.patch
postgresql.service
postgresql-setup
README.rpm-dist
rpm-pgsql.patch

with the file contents unchanged, just renamed.

Still got some work to do on making the base name, datadir, etc
configurable, but that's for once the basics are working.

To build, I use mock to produce the srpm, then build the rpms for it.
e.g.:

mock -r fedora-20-x86_64 --buildsrpm \
- --spec rpm/postgresql-x.y.spec --sources rpm/ \
- --resultdir mock-"%(dist)s"-srpm/

for arch in x86_64 i386
do
mock -r fedora-20-$arch \
- --rebuild mock-fedora-20-srpm/postgresql94-9.4beta2-3PGDG.fc20.src.rpm \
- --resultdir mock-"%(dist)s"-"%(target_arch)s"-rpm
done


for the targets in /etc/mock/ . I've used the EPEL targets for RHEL;
strictly I should be targeting stock RHEL + the PGDG RPMs with a new
target file; will try that soon, but it doesn't matter for Pg its self
as it doesn't depend on anything outside RHEL its self.

Now I need to figure out if versioned provides are safe, and look into
making it possible to build variant packages.

Hope you like what you see so far.

- --
 Craig Ringer                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJT2n5UAAoJELBXNkqjr+S2bIYH/j7+7lIRqqRuvg+JCEAu2pYB
VwMPp/b9KdpPBvh1gfedBYTacY5/Tdo+2omdC01QSRbW9Au/YVp4MgN4Kf3DA/dt
NgE/EZPIldfDnhfeYTdN1wO4aBvT2Njn+gBii7Vs462RYCXiVBCLeUl9exmaXlEy
L/QwaloGaBFvheAaSFrwVHc/V76+StG8pYpy2Z/q5OjAL043rzSRxJm9X0QFTJnm
8GeZ+KVDKWGjAjpmu7z1YWU9wSiz6Oatnip1bNnMWwlBhQ256MO3/FhedrrKUGQV
CVVvqmdOQmyu9CeV2tluX6MBETdtrqK0CVRUGc/Gj13W/u3Wd258Oiu+DeILW0M=
=gR/D
-----END PGP SIGNATURE-----

On 08/01/2014 01:35 AM, Craig Ringer wrote:
> On 07/30/2014 11:39 PM, Craig Ringer wrote:
>> Still some bugs to fix in the unified package and more testing to
>> do but I'm happy with how things are looking. I'll probably need to
>> make new mock targets too, of course, but that looks trivial.
>
> I'm down to testing now. Looking good so far.

OK, I think I'm largely done with this. I've done the most recent round
of testing on BDR RPMs based on this work, but the stock Pg RPMs are
only trivially different so it should be fine.

I've made a lot of changes since the last update:

- Fixed a bunch of package bugs, including that Perl dependency bug

- Used a single sed statement to insert placeholders in all the
  scripts, instead of individual statements with different placeholders
  for each script

- in the %check section, verify no placeholders are left over after
  package build

- I've gone through and parameterized all references to the package
  name, version, etc, and tested that these can be changed by modifying
  defines in the package. These changes are substituted into any
  scripts built for the package, and I'm using these changes in BDR
  package testing.

- Produced a build automation script. It's not intended for long term
  use, but it's good for testing. Koji should be used for in the medium-
  long term. It also does some post-build sanity checks and I'm setting
  up some scripts with vagrant to do proper ones.

  The script also creates a yum repo with yum metadata and repoview
  data.

- New mock targets that include PGDG repositories (and, for RHEL/CentOS,
  EPEL repositories). They're disabled by default but can be enabled
  on demand to install specific packages during mock setup, though this
  needs a patched mock from http://github.com/ringerc/mock . Upstream
  is merging the patches.

- Inline documentation in the spec file

- Several script fixes

Outstanding issues:

- Need to change the RHEL5 package to build against libuuid

- Should expose a new Provides: that can be depended on by extensions
  to allow finer control of what extensions will install on what
  servers.

- More testing, specially on CentOS/RHEL and on sysvinit systems in
  general.

- pg_upgrade support is disabled. It's been broken in PGDG since 9.2
  anyway, and I suspect it should simply be removed from
  postgresql94-setup in favour of documenting how to do it properly
  by hand. The code's there, but hasn't been tested.

  If this is to be re-enabled we need test automation to make sure
  it actually works for a variety of scenarios.


Quickstart to try it out, on a freshly installed clean Fedora 20 box
(use EC2, vagrant, real hw, whatever), assuming that the scripts are in
a directory named 'packaging':

    # Install tools
    yum groupinstall "Development Tools"
    yum install yum-utils rpmdevtools repoview createrepo
    yum-builddep mock
    yum install git svn
    # You only need this to make the postgresql dist tarball
    # because configure has to run:
    yum-builddep postgresql
    # and for the docs in the dist tarball:
    yum install openjade jadetex docbook-dtds docbook5-schemas
docbook-style-dsssl docbook-style-xsl

    # Install patched mock
    git clone https://github.com/ringerc/mock.git
    pushd mock
    git checkout target-paths
    ./autogen.sh
    ./configure
    make rpm
    yum install noarch/*
    popd

    # Grab sources and create source tarball
    git clone git://git.postgresql.org/git/postgresql.git
    pushd postgresql
    # whatever bdr you want to build; remember to change the
    # SRPM to match
    git checkout REL9_4_STABLE
    # No need for any options, it's only to create the makefiles
    # for "make dist".
    ./configure
    make dist
    popd

    pushd packaging
    pushd unified-rpm
    mv ../../postgresql/*.bz2 .
    spectool -gf postgresql94.spec
    # If needed, edit unified-rpm/postgresql94.spec, particularly
    # the version defines and the Source0 line for the Pg sources;
    # then:
    popd
    ./mock-build-all.sh

The first run will take longer because it has to set up and cache the
mock chroots.

You can limit the build to only some targets, see the targets variable
in mock-build-all.sh .

If you want to test the packages you built locally, without pushing to a
remote repo, you need only define a repo in yum with the file:// URL to
your locally built repo.



I'll put the WIP unified packages in a public git repo soon; right now
they're in a private tree associated with BDR work. For now, the current
versions are attached.

--
 Craig Ringer                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services