Обсуждение: archive_command vs. cp -i
Hi, while setting up a new linux postgresql server, I checked my archive_command. I noticed that the usual "cp -i f1 f2 < /dev/null" did NOT as expected: It did not overwrite the file (PASS), but it returned zero (FAIL, should return error). This could be reproduced on CentOS 5.6, RHEL4, and Ubuntu 10.04. This is my test: echo huhu > file1 echo hello > file2 /bin/cp -i file2 file1 < /dev/null echo $? # should return != 0 diff -s file1 file2 Am I missing something here? Should the BACKUP-ARCHIVING-WAL doc be changed? Regards, Martin -- NEU: FreePhone - kostenlos mobil telefonieren! Jetzt informieren: http://www.gmx.net/de/go/freephone
"Martin Münstermann"<mmuenst@gmx.de> wrote: > while setting up a new linux postgresql server, I checked my > archive_command. I noticed that the usual "cp -i f1 f2 < > /dev/null" did NOT as expected: It did not overwrite the file > (PASS), but it returned zero (FAIL, should return error). > > This could be reproduced on CentOS 5.6, RHEL4, and Ubuntu 10.04. I've confirmed on SLES 10 and Ubuntu 9, too. > Should the BACKUP-ARCHIVING-WAL doc be changed? I think so. Given the wide variety of platforms on which the example could silently cause data loss, I *really* don't think we want that in our docs. Someone could blithely copy it into production without testing and not know they had it wrong until a backup failed to restore. -Kevin
Technically the documentation covers its self by saying:
"This is an example, not a recommendation, and might not work on all platforms."
What would be nice is if the documentation could be made more useful by providing some more complex example scripts.
What do our readers use for WAL archiving?
On Fri, Jun 17, 2011 at 12:32 PM, Kevin Grittner <Kevin.Grittner@wicourts.gov> wrote:
"Martin Münstermann"<mmuenst@gmx.de> wrote:I've confirmed on SLES 10 and Ubuntu 9, too.
> while setting up a new linux postgresql server, I checked my
> archive_command. I noticed that the usual "cp -i f1 f2 <
> /dev/null" did NOT as expected: It did not overwrite the file
> (PASS), but it returned zero (FAIL, should return error).
>
> This could be reproduced on CentOS 5.6, RHEL4, and Ubuntu 10.04.I think so. Given the wide variety of platforms on which the
> Should the BACKUP-ARCHIVING-WAL doc be changed?
example could silently cause data loss, I *really* don't think we
want that in our docs. Someone could blithely copy it into
production without testing and not know they had it wrong until a
backup failed to restore.
-Kevin
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
Hi all: Corrected the top posting. On Fri, Jun 17, 2011 at 12:55:07PM -0400, Tim wrote: > On Fri, Jun 17, 2011 at 12:32 PM, Kevin Grittner > Kevin.Grittner@wicourts.gov> wrote: > > "Martin Münstermann"<mmuenst@gmx.de> wrote: > > > while setting up a new linux postgresql server, I checked my > > > archive_command. I noticed that the usual "cp -i f1 f2 < > > > /dev/null" did NOT as expected: It did not overwrite the file > > > (PASS), but it returned zero (FAIL, should return error). > > > > > > This could be reproduced on CentOS 5.6, RHEL4, and Ubuntu 10.04. > > I've confirmed on SLES 10 and Ubuntu 9, too. Not really surprising since choosing to not copy a file isn't really an indication of an error when copying a file so... I get the same result (0 exit status) on solaris 8 and solaris 10 with /bin/cp. I wonder what platform the example worked on. > > > Should the BACKUP-ARCHIVING-WAL doc be changed? > > > > I think so. Given the wide variety of platforms on which the > > example could silently cause data loss, I *really* don't think we > > want that in our docs. Someone could blithely copy it into > > production without testing and not know they had it wrong until a > > backup failed to restore. For what it's worth I strongly agree. > Technically the documentation covers its self by saying: > "This is an example, not a recommendation, and might not work on all > platforms." > What would be nice is if the documentation could be made more useful by > providing some more complex example scripts. > What do our readers use for WAL archiving? 'test ! -f /var/bak/pgsql/%f && cp %p /var/bak/pgsql/%f' Which does have a slight race condition as the output file could be created between the test finishing and the cp starting. But the only way that is likely to happen is if postgresql tried to run two identical archive commands in parallel, and I think I would have other issues if that was happening. The following may be safer if you are using a shell with noclobber, or it may not be. Depends on the race conditions in how the shell implements the noclobber code. 'sh -c "set -o noclobber; cat %p > /var/bak/pgsql/%f"' -- -- rouilj John Rouillard System Administrator Renesys Corporation
John Rouillard <rouilj@renesys.com> writes: >> On Fri, Jun 17, 2011 at 12:32 PM, Kevin Grittner >> Kevin.Grittner@wicourts.gov> wrote: >>>> This could be reproduced on CentOS 5.6, RHEL4, and Ubuntu 10.04. >>> I've confirmed on SLES 10 and Ubuntu 9, too. > Not really surprising since choosing to not copy a file isn't really > an indication of an error when copying a file so... > I get the same result (0 exit status) on solaris 8 and solaris 10 with > /bin/cp. I wonder what platform the example worked on. cp does behave as the example suggests for me, on OS X and HPUX. I suspect Bruce tested it on some BSD variant before putting it in the docs. That would suggest that it probably works that way on most (all?) BSDen. Still, if the GNU version doesn't act that way, we have a problem. The test-and-cp approach seems the most likely to be portable. regards, tom lane
I wrote: > John Rouillard <rouilj@renesys.com> writes: >> I get the same result (0 exit status) on solaris 8 and solaris 10 with >> /bin/cp. I wonder what platform the example worked on. > cp does behave as the example suggests for me, on OS X and HPUX. I > suspect Bruce tested it on some BSD variant before putting it in the > docs. That would suggest that it probably works that way on most (all?) > BSDen. Still, if the GNU version doesn't act that way, we have a > problem. > The test-and-cp approach seems the most likely to be portable. I've applied patches for this. It turns out that before 9.0, we did have a warning that cp -i wasn't very trustworthy: It is advisable to test your proposed archive command to ensure that it indeed does not overwrite an existing file, <emphasis>and that it returns nonzero status in this case</>. We have found that <literal>cp -i</> does this correctly on some platforms but not others. If the chosen command but somebody "improved" that text in a way that made it sound like you could expect the command to work most places. On the whole it seems like a better idea to provide a sample command that is safe most everywhere, and then mention cp -i as a nonportable simplification. regards, tom lane