Обсуждение: snapshot generation broken
Hi all! It seems that the git move has broken the generation of the automatic snapshot tarballs - has anybody yet looked into what it would take to move those to fetching from git? Stefan
On 22/09/10 11:33, Stefan Kaltenbrunner wrote: > Hi all! > > It seems that the git move has broken the generation of the automatic > snapshot tarballs - has anybody yet looked into what it would take to > move those to fetching from git? BTW, there is a nice command called in git to create tarballs: git archive master | gzip > postgresql.tar.gz I doubt we can use it because we add some generated files to our tarballs that are not in the repository, but I thought I'd mention it. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
On Wed, Sep 22, 2010 at 10:47, Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> wrote: > On 22/09/10 11:33, Stefan Kaltenbrunner wrote: >> >> Hi all! >> >> It seems that the git move has broken the generation of the automatic >> snapshot tarballs - has anybody yet looked into what it would take to >> move those to fetching from git? > > BTW, there is a nice command called in git to create tarballs: > > git archive master | gzip > postgresql.tar.gz > > I doubt we can use it because we add some generated files to our tarballs > that are not in the repository, but I thought I'd mention it. It is useful, yes, but not for this. I'll take a peek at the tarball generation today, unless beaten to it (meaning if someone wants to star tlooking into it, send me an email or ping me on irc to make sure we don't duplicate the work) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
On ons, 2010-09-22 at 10:33 +0200, Stefan Kaltenbrunner wrote: > It seems that the git move has broken the generation of the automatic > snapshot tarballs - has anybody yet looked into what it would take to > move those to fetching from git? Depends on what's "broken" about it, but I notice that the developer docs and the NLS builds are also not updating. Perhaps something wrong with the anoncvs service.
On Wed, Sep 22, 2010 at 11:29, Peter Eisentraut <peter_e@gmx.net> wrote: > On ons, 2010-09-22 at 10:33 +0200, Stefan Kaltenbrunner wrote: >> It seems that the git move has broken the generation of the automatic >> snapshot tarballs - has anybody yet looked into what it would take to >> move those to fetching from git? > > Depends on what's "broken" about it, but I notice that the developer > docs and the NLS builds are also not updating. Perhaps something wrong > with the anoncvs service. anoncvs still serves the old cvs repository. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
On Wed, Sep 22, 2010 at 11:11 AM, Magnus Hagander <magnus@hagander.net> wrote: > On Wed, Sep 22, 2010 at 11:29, Peter Eisentraut <peter_e@gmx.net> wrote: >> On ons, 2010-09-22 at 10:33 +0200, Stefan Kaltenbrunner wrote: >>> It seems that the git move has broken the generation of the automatic >>> snapshot tarballs - has anybody yet looked into what it would take to >>> move those to fetching from git? >> >> Depends on what's "broken" about it, but I notice that the developer >> docs and the NLS builds are also not updating. Perhaps something wrong >> with the anoncvs service. > > anoncvs still serves the old cvs repository. Whens that due to be fixed? I imagine most of the buildfarm is testing that still... -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise Postgres Company
Peter Eisentraut wrote: > On ons, 2010-09-22 at 10:33 +0200, Stefan Kaltenbrunner wrote: >> It seems that the git move has broken the generation of the automatic >> snapshot tarballs - has anybody yet looked into what it would take to >> move those to fetching from git? > > Depends on what's "broken" about it, but I notice that the developer > docs and the NLS builds are also not updating. Perhaps something wrong > with the anoncvs service. heh - well nagios complained that the current tarballs are outdated, I have not actually investigated further :) But iirc snapshot generation is running against the main cvs repo and not anoncvs. Stefan
On Wed, Sep 22, 2010 at 12:19, Dave Page <dpage@pgadmin.org> wrote: > On Wed, Sep 22, 2010 at 11:11 AM, Magnus Hagander <magnus@hagander.net> wrote: >> On Wed, Sep 22, 2010 at 11:29, Peter Eisentraut <peter_e@gmx.net> wrote: >>> On ons, 2010-09-22 at 10:33 +0200, Stefan Kaltenbrunner wrote: >>>> It seems that the git move has broken the generation of the automatic >>>> snapshot tarballs - has anybody yet looked into what it would take to >>>> move those to fetching from git? >>> >>> Depends on what's "broken" about it, but I notice that the developer >>> docs and the NLS builds are also not updating. Perhaps something wrong >>> with the anoncvs service. >> >> anoncvs still serves the old cvs repository. > > Whens that due to be fixed? I imagine most of the buildfarm is testing > that still... Don't have an actual time for that. Figured we'd leave the old one running until things have settled down. Mind you, I think bf members will need tweaking anyway, since they will assume the old style cvs layout. AFAICT, git-cvsserver will export each branch as a module, rather than one module as pgsql. We can obviously get that working for the HEAD branch (by just mapping it to a branch called pgsql), but I don't know how doable that is for backbranches. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
On Wed, Sep 22, 2010 at 10:33, Stefan Kaltenbrunner <stefan@kaltenbrunner.cc> wrote: > Hi all! > > It seems that the git move has broken the generation of the automatic > snapshot tarballs - has anybody yet looked into what it would take to move > those to fetching from git? I think this should be working now. I made this change to mk-snapshot, and a similar one to mk-stable_snapshot. I have run the snapshot generation for master, and the one for 8.4 should finish in a minute or so. I have disabled the cronjobs until someone else has verified the tarballs. When I do, I'll also enable generation of snapshots of REL9_0_STABLE. We currently do back to 8.2 - should we keep that, or drop 8.2? //Magnus *** old/mk-snapshot Wed Sep 22 06:02:02 2010 --- mk-snapshot Wed Sep 22 07:09:13 2010 *************** *** 8,14 **** export MAKE=gmake export MD5SUM=md5 ! export CVSROOT=:ext:scrappy@cvs.postgresql.org:/cvsroot export MAKEFLAGS=VERSION=snapshot export TMPDIR=/usr/local/pgsql/snapshot --- 8,14 ---- export MAKE=gmake export MD5SUM=md5 ! export GIT_DIR=/usr/local/pgsql/git/postgresql/.git export MAKEFLAGS=VERSION=snapshot export TMPDIR=/usr/local/pgsql/snapshot *************** *** 19,25 **** then rm -fr pgsql fi ! /usr/bin/cvs -q export -rHEAD pgsql cd pgsql ./configure export OUTPUTFILE=$($MAKE -s $MAKEFLAGS distdir-location) --- 19,28 ---- then rm -fr pgsql fi ! mkdir pgsql ! git fetch ! git archive origin/master | tar xf - -C pgsql ! cd pgsql ./configure export OUTPUTFILE=$($MAKE -s $MAKEFLAGS distdir-location) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
Magnus Hagander wrote: > On Wed, Sep 22, 2010 at 10:33, Stefan Kaltenbrunner > <stefan@kaltenbrunner.cc> wrote: >> Hi all! >> >> It seems that the git move has broken the generation of the automatic >> snapshot tarballs - has anybody yet looked into what it would take to move >> those to fetching from git? > > I think this should be working now. I made this change to mk-snapshot, > and a similar one to mk-stable_snapshot. I have run the snapshot > generation for master, and the one for 8.4 should finish in a > minute or so. I have disabled the cronjobs until someone else has > verified the tarballs. > > When I do, I'll also enable generation of snapshots of REL9_0_STABLE. > We currently do back to 8.2 - should we keep that, or drop 8.2? hmm I would say we should do all supported branches because those are the ones that people might be interested in when looking for a fix on a branch... Stefan
On ons, 2010-09-22 at 12:22 +0200, Magnus Hagander wrote: > Mind you, I think bf members will need tweaking anyway, since they > will assume the old style cvs layout. AFAICT, git-cvsserver will > export each branch as a module, rather than one module as pgsql. We > can obviously get that working for the HEAD branch (by just mapping it > to a branch called pgsql), but I don't know how doable that is for > backbranches. We need to have a fix for this before we can do any more backbranch releases.
On Wed, Sep 22, 2010 at 13:20, Peter Eisentraut <peter_e@gmx.net> wrote: > On ons, 2010-09-22 at 12:22 +0200, Magnus Hagander wrote: >> Mind you, I think bf members will need tweaking anyway, since they >> will assume the old style cvs layout. AFAICT, git-cvsserver will >> export each branch as a module, rather than one module as pgsql. We >> can obviously get that working for the HEAD branch (by just mapping it >> to a branch called pgsql), but I don't know how doable that is for >> backbranches. > > We need to have a fix for this before we can do any more backbranch > releases. Fix for what, exactly? (assuming that people update their bf animals of course) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
Peter Eisentraut wrote: > On ons, 2010-09-22 at 12:22 +0200, Magnus Hagander wrote: >> Mind you, I think bf members will need tweaking anyway, since they >> will assume the old style cvs layout. AFAICT, git-cvsserver will >> export each branch as a module, rather than one module as pgsql. We >> can obviously get that working for the HEAD branch (by just mapping it >> to a branch called pgsql), but I don't know how doable that is for >> backbranches. > > We need to have a fix for this before we can do any more backbranch > releases. yeah having to touch ever BF member despite the fact that we have a git->cvs gateway seems heavily annoying and not really expected. However if this must be done oen should send a proper headsupand howto to the buildfarm member list (people need to add the 9.0 branch as well so we better tell them this now instead of having to do work twice). Stefan
Magnus Hagander wrote: > On Wed, Sep 22, 2010 at 13:20, Peter Eisentraut <peter_e@gmx.net> wrote: >> On ons, 2010-09-22 at 12:22 +0200, Magnus Hagander wrote: >>> Mind you, I think bf members will need tweaking anyway, since they >>> will assume the old style cvs layout. AFAICT, git-cvsserver will >>> export each branch as a module, rather than one module as pgsql. We >>> can obviously get that working for the HEAD branch (by just mapping it >>> to a branch called pgsql), but I don't know how doable that is for >>> backbranches. >> We need to have a fix for this before we can do any more backbranch >> releases. > > Fix for what, exactly? (assuming that people update their bf animals of course) how exactly should people fix their BF members - is there any howto or such on how to deal with the new git2cvs gateway? Switching to git entirely is rather hard for some of my members for example - at least not without investing hours of work... Stefan
On ons, 2010-09-22 at 13:25 +0200, Magnus Hagander wrote: > On Wed, Sep 22, 2010 at 13:20, Peter Eisentraut <peter_e@gmx.net> wrote: > > We need to have a fix for this before we can do any more backbranch > > releases. > > Fix for what, exactly? (assuming that people update their bf animals of course) For the NLS workflow, we need to have some kind of CVS server that serves up all supported branches. Changing the URL or the module name or whatever is fine.
On Wed, Sep 22, 2010 at 13:28, Stefan Kaltenbrunner <stefan@kaltenbrunner.cc> wrote: > Magnus Hagander wrote: >> >> On Wed, Sep 22, 2010 at 13:20, Peter Eisentraut <peter_e@gmx.net> wrote: >>> >>> On ons, 2010-09-22 at 12:22 +0200, Magnus Hagander wrote: >>>> >>>> Mind you, I think bf members will need tweaking anyway, since they >>>> will assume the old style cvs layout. AFAICT, git-cvsserver will >>>> export each branch as a module, rather than one module as pgsql. We >>>> can obviously get that working for the HEAD branch (by just mapping it >>>> to a branch called pgsql), but I don't know how doable that is for >>>> backbranches. >>> >>> We need to have a fix for this before we can do any more backbranch >>> releases. >> >> Fix for what, exactly? (assuming that people update their bf animals of >> course) > > how exactly should people fix their BF members - is there any howto or such > on how to deal with the new git2cvs gateway? Andrew has posted info on how to convert them to git, I believe. I don't think there are any for the git2cvs gateway, since it's not up yet. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
On Wed, Sep 22, 2010 at 13:38, Peter Eisentraut <peter_e@gmx.net> wrote: > On ons, 2010-09-22 at 13:25 +0200, Magnus Hagander wrote: >> On Wed, Sep 22, 2010 at 13:20, Peter Eisentraut <peter_e@gmx.net> wrote: >> > We need to have a fix for this before we can do any more backbranch >> > releases. >> >> Fix for what, exactly? (assuming that people update their bf animals of course) > > For the NLS workflow, we need to have some kind of CVS server that > serves up all supported branches. Changing the URL or the module name > or whatever is fine. I thought you had/were going to convert those to work natively with git? I guess I misunderstood you. Anyway, it's on my list to get done pretty quickly. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
On ons, 2010-09-22 at 13:45 +0200, Magnus Hagander wrote: > On Wed, Sep 22, 2010 at 13:38, Peter Eisentraut <peter_e@gmx.net> wrote: > > For the NLS workflow, we need to have some kind of CVS server that > > serves up all supported branches. Changing the URL or the module name > > or whatever is fine. > > I thought you had/were going to convert those to work natively with > git? I guess I misunderstood you. I think CVS should be eliminated from that process during the 9.1 release cycle, but we are looking for backbranch releases within the next few weeks, and there is no time to get that done by then.
On 09/22/2010 06:22 AM, Magnus Hagander wrote: > Mind you, I think bf members will need tweaking anyway, since they > will assume the old style cvs layout. AFAICT, git-cvsserver will > export each branch as a module, rather than one module as pgsql. We > can obviously get that working for the HEAD branch (by just mapping it > to a branch called pgsql), but I don't know how doable that is for > backbranches. > Buildfarm owners should be moving aggressively towards using git. I'm not aware of any platform it can't be used on. I was planning on doing an audit around the end of the week and following up with people who haven't migrated. cheers andrew
On Wed, Sep 22, 2010 at 13:45, Magnus Hagander <magnus@hagander.net> wrote: > On Wed, Sep 22, 2010 at 13:38, Peter Eisentraut <peter_e@gmx.net> wrote: >> On ons, 2010-09-22 at 13:25 +0200, Magnus Hagander wrote: >>> On Wed, Sep 22, 2010 at 13:20, Peter Eisentraut <peter_e@gmx.net> wrote: >>> > We need to have a fix for this before we can do any more backbranch >>> > releases. >>> >>> Fix for what, exactly? (assuming that people update their bf animals of course) >> >> For the NLS workflow, we need to have some kind of CVS server that >> serves up all supported branches. Changing the URL or the module name >> or whatever is fine. > > I thought you had/were going to convert those to work natively with > git? I guess I misunderstood you. > > Anyway, it's on my list to get done pretty quickly. Started again on this. Whenever I try to do something useful, it crashes with sqlite errors that appear to be FreeBSD specific. I'm going to try to upgrade all the ports on the box and give it another try. Unfortunately, that tends to take a couple of hours - but it'll get done eventually :-) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
Stefan Kaltenbrunner <stefan@kaltenbrunner.cc> writes:
> Magnus Hagander wrote:
>> I think this should be working now. I made this change to mk-snapshot,
>> and a similar one to mk-stable_snapshot. I have run the snapshot
>> generation for master, and the one for 8.4 should finish in a
>> minute or so. I have disabled the cronjobs until someone else has
>> verified the tarballs.
Do you still need someone to do that, and what do you want done exactly?
>> When I do, I'll also enable generation of snapshots of REL9_0_STABLE.
>> We currently do back to 8.2 - should we keep that, or drop 8.2?
> hmm I would say we should do all supported branches because those are
> the ones that people might be interested in when looking for a fix on a
> branch...
FWIW, I think back to 8.2 is sufficient. The branches earlier than that
are going to be unsupported in a matter of weeks anyway. And if no one
has complained about the lack of nightly tarballs for them to date,
it's unlikely that they'll start complaining now.
regards, tom lane
On Wed, Sep 22, 2010 at 16:50, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Stefan Kaltenbrunner <stefan@kaltenbrunner.cc> writes: >> Magnus Hagander wrote: >>> I think this should be working now. I made this change to mk-snapshot, >>> and a similar one to mk-stable_snapshot. I have run the snapshot >>> generation for master, and the one for 8.4 should finish in a >>> minute or so. I have disabled the cronjobs until someone else has >>> verified the tarballs. > > Do you still need someone to do that, and what do you want done exactly? Just a second set of eyes that the output looks reasonable for being a snapshot generated now. >>> When I do, I'll also enable generation of snapshots of REL9_0_STABLE. >>> We currently do back to 8.2 - should we keep that, or drop 8.2? > >> hmm I would say we should do all supported branches because those are >> the ones that people might be interested in when looking for a fix on a >> branch... > > FWIW, I think back to 8.2 is sufficient. The branches earlier than that > are going to be unsupported in a matter of weeks anyway. And if no one > has complained about the lack of nightly tarballs for them to date, > it's unlikely that they'll start complaining now. Ok. 8.2 it is then - and I'll add 9.0 -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
Magnus Hagander <magnus@hagander.net> writes:
> On Wed, Sep 22, 2010 at 16:50, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Do you still need someone to do that, and what do you want done exactly?
> Just a second set of eyes that the output looks reasonable for being a
> snapshot generated now.
Sure, I can look. Where are these tarballs anyway?
regards, tom lane
On Wed, Sep 22, 2010 at 17:08, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Magnus Hagander <magnus@hagander.net> writes: >> On Wed, Sep 22, 2010 at 16:50, Tom Lane <tgl@sss.pgh.pa.us> wrote: >>> Do you still need someone to do that, and what do you want done exactly? > >> Just a second set of eyes that the output looks reasonable for being a >> snapshot generated now. > > Sure, I can look. Where are these tarballs anyway? They're up on the ftp site - ftp://ftp.postgresql.org/pub/snapshot/. dev and stable/8.4 are updated. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
Magnus Hagander <magnus@hagander.net> writes:
> On Wed, Sep 22, 2010 at 17:08, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Sure, I can look. �Where are these tarballs anyway?
> They're up on the ftp site - ftp://ftp.postgresql.org/pub/snapshot/.
> dev and stable/8.4 are updated.
Both of those look pretty sane from here.
regards, tom lane
On Wed, Sep 22, 2010 at 19:23, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Magnus Hagander <magnus@hagander.net> writes: >> On Wed, Sep 22, 2010 at 17:08, Tom Lane <tgl@sss.pgh.pa.us> wrote: >>> Sure, I can look. Where are these tarballs anyway? > >> They're up on the ftp site - ftp://ftp.postgresql.org/pub/snapshot/. >> dev and stable/8.4 are updated. > > Both of those look pretty sane from here. Ok, I've re-enabled the cron job. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
On ons, 2010-09-22 at 12:29 +0300, Peter Eisentraut wrote: > On ons, 2010-09-22 at 10:33 +0200, Stefan Kaltenbrunner wrote: > > It seems that the git move has broken the generation of the automatic > > snapshot tarballs - has anybody yet looked into what it would take to > > move those to fetching from git? > > Depends on what's "broken" about it, but I notice that the developer > docs and the NLS builds are also not updating. Perhaps something wrong > with the anoncvs service. Developer docs are now building again.
On tor, 2010-09-23 at 11:07 +0300, Peter Eisentraut wrote: > On ons, 2010-09-22 at 12:29 +0300, Peter Eisentraut wrote: > > On ons, 2010-09-22 at 10:33 +0200, Stefan Kaltenbrunner wrote: > > > It seems that the git move has broken the generation of the automatic > > > snapshot tarballs - has anybody yet looked into what it would take to > > > move those to fetching from git? > > > > Depends on what's "broken" about it, but I notice that the developer > > docs and the NLS builds are also not updating. Perhaps something wrong > > with the anoncvs service. > > Developer docs are now building again. And NLS is also fixed.
On Fri, Sep 24, 2010 at 07:15, Peter Eisentraut <peter_e@gmx.net> wrote: > On tor, 2010-09-23 at 11:07 +0300, Peter Eisentraut wrote: >> On ons, 2010-09-22 at 12:29 +0300, Peter Eisentraut wrote: >> > On ons, 2010-09-22 at 10:33 +0200, Stefan Kaltenbrunner wrote: >> > > It seems that the git move has broken the generation of the automatic >> > > snapshot tarballs - has anybody yet looked into what it would take to >> > > move those to fetching from git? >> > >> > Depends on what's "broken" about it, but I notice that the developer >> > docs and the NLS builds are also not updating. Perhaps something wrong >> > with the anoncvs service. >> >> Developer docs are now building again. > > And NLS is also fixed. Great. Thanks - that takes one more thing off the cvs requirement ;) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
Magnus Hagander <magnus@hagander.net> writes:
> On Fri, Sep 24, 2010 at 07:15, Peter Eisentraut <peter_e@gmx.net> wrote:
>> And NLS is also fixed.
> Great. Thanks - that takes one more thing off the cvs requirement ;)
Yeah. Maybe we don't need a cvsserver after all. Would it make more
sense to help Stefan get git running on his BSD boxes? If Bruce and
I could get it to work on our pet dinosaurs, I think it likely can
be gotten to work on spoonbill too.
regards, tom lane
On 09/24/2010 03:47 PM, Tom Lane wrote: > Yeah. Maybe we don't need a cvsserver after all. Would it make more > sense to help Stefan get git running on his BSD boxes? If Bruce and > I could get it to work on our pet dinosaurs, I think it likely can > be gotten to work on spoonbill too. Yeah, I find it hard to believe that a machine can build postgres, with openssl, but can't build git. The problem is that Stefan doesn't have time to work on it, and I gather he's unable to give anyone else access. cheers andrew
On 09/24/2010 09:54 PM, Andrew Dunstan wrote: > > > On 09/24/2010 03:47 PM, Tom Lane wrote: > >> Yeah. Maybe we don't need a cvsserver after all. Would it make more >> sense to help Stefan get git running on his BSD boxes? If Bruce and >> I could get it to work on our pet dinosaurs, I think it likely can >> be gotten to work on spoonbill too. > > Yeah, I find it hard to believe that a machine can build postgres, with > openssl, but can't build git. The problem is that Stefan doesn't have > time to work on it, and I gather he's unable to give anyone else access. yeah don't worry too much, I will find a way to fix it will just take a few more days until I get a bit of spare time... Stefan