Обсуждение: Rename backup_label to recovery_control

Hackers,

I was recently discussing the complexities of dealing with pg_control 
and backup_label with some hackers at PGConf NYC, when David Christensen 
commented that backup_label was not a very good name since it gives the 
impression of being informational and therefore something the user can 
delete. In fact, we see this happen quite a lot, and there have been 
some other discussions about it recently, see [1] and [2]. I bounced the 
idea of a rename off various hackers at the conference and in general 
people seemed to think it was a good idea.

Attached is a patch to rename backup_label to recovery_control. The 
purpose is to make it more obvious that the file should not be deleted. 
I'm open to other names, e.g. recovery.control. That makes the naming 
distinct from tablespace_map, which is perhaps a good thing, but is also 
more likely to be confused with recovery.signal.

I did a pretty straight-forward search and replace on comments and 
documentation with only light editing. If this seems like a good idea 
and we choose a final name, I'll do a more thorough pass through the 
comments and docs to try and make the usage more consistent.

Note that there is one usage of backup label that remains, i.e. the text 
that the user can set to describe the backup.

Regards,
-David

[1] 
https://www.postgresql.org/message-id/1330cb48-4e47-03ca-f2fb-b144b49514d8%40pgmasters.net
[2] 

https://www.postgresql.org/message-id/flat/CAM_vCudkSjr7NsNKSdjwtfAm9dbzepY6beZ5DP177POKy8%3D2aw%40mail.gmail.com#746e492bfcd2667635634f1477a61288

At Sat, 14 Oct 2023 14:19:42 -0400, David Steele <david@pgmasters.net> wrote in 
> I was recently discussing the complexities of dealing with pg_control
> and backup_label with some hackers at PGConf NYC, when David
> Christensen commented that backup_label was not a very good name since
> it gives the impression of being informational and therefore something
> the user can delete. In fact, we see this happen quite a lot, and
> there have been some other discussions about it recently, see [1] and
> [2]. I bounced the idea of a rename off various hackers at the
> conference and in general people seemed to think it was a good idea.
> 
> Attached is a patch to rename backup_label to recovery_control. The

Just an idea in a slightly different direction, but I'm wondering if
we can simply merge the content of backup_label into control file.
The file is 8192 bytes long, yet only 256 bytes are used. As a result,
we anticipate no overhead.  Sucha configuration would forcibly prevent
uses from from removing the backup information.

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center

At Mon, 16 Oct 2023 13:16:42 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in 
> Just an idea in a slightly different direction, but I'm wondering if
> we can simply merge the content of backup_label into control file.
> The file is 8192 bytes long, yet only 256 bytes are used. As a result,
> we anticipate no overhead.  Sucha configuration would forcibly prevent
> uses from from removing the backup information.

In second thought, that would break the case of file-system level
backups, which require backup information separately from control
data.

Sorry for the noise.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center

On Mon, Oct 16, 2023 at 01:16:42PM +0900, Kyotaro Horiguchi wrote:
> Just an idea in a slightly different direction, but I'm wondering if
> we can simply merge the content of backup_label into control file.
> The file is 8192 bytes long, yet only 256 bytes are used. As a result,
> we anticipate no overhead.  Sucha configuration would forcibly prevent
> uses from from removing the backup information.

With the critical assumptions behind PG_CONTROL_MAX_SAFE_SIZE, that
does not sound like a good idea to me.  And that's without the fact
that base backup labels could make the control file theoretically even
larger than PG_CONTROL_FILE_SIZE, even if that's unlikely going to
happen in practice.
--
Michael

On 10/16/23 00:26, Kyotaro Horiguchi wrote:
> At Mon, 16 Oct 2023 13:16:42 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in
>> Just an idea in a slightly different direction, but I'm wondering if
>> we can simply merge the content of backup_label into control file.
>> The file is 8192 bytes long, yet only 256 bytes are used. As a result,
>> we anticipate no overhead.  Sucha configuration would forcibly prevent
>> uses from from removing the backup information.
> 
> In second thought, that would break the case of file-system level
> backups, which require backup information separately from control
> data.

Exactly -- but we do have a proposal to do the opposite and embed 
pg_control into backup_label [1] (or hopefully recovery_control).

Regards,
-David

[1] 
https://www.postgresql.org/message-id/1330cb48-4e47-03ca-f2fb-b144b49514d8%40pgmasters.net

On Sat, Oct 14, 2023 at 2:22 PM David Steele <david@pgmasters.net> wrote:
> I was recently discussing the complexities of dealing with pg_control
> and backup_label with some hackers at PGConf NYC, when David Christensen
> commented that backup_label was not a very good name since it gives the
> impression of being informational and therefore something the user can
> delete. In fact, we see this happen quite a lot, and there have been
> some other discussions about it recently, see [1] and [2]. I bounced the
> idea of a rename off various hackers at the conference and in general
> people seemed to think it was a good idea.

Personally, I feel like this is an area where we keep moving the parts
around but I'm not sure we're really getting to anything better. We
got rid of recovery.conf. We got rid of exclusive backup mode. We
replaced pg_start_backup with pg_backup_start. It feels like every
other release or so we whack something around here, but I'm not
convinced that any of it is really making much of an impact. If
there's been any decrease in people screwing up their backups, I
haven't noticed it.

To be fair, I will grant that renaming pg_clog to pg_xact_status and
pg_xlog to pg_wal does seem to have reduced the incidence of people
nuking those directories, at least IME. So maybe this change would
help too, for similar reasons. But I'm still concerned that we're
doing too much superficial tinkering in this area. Breaking
compatibility is not without cost.

I also do wonder with recovery_control is really a better name. Maybe
I just have backup_label too firmly stuck in my head, but is what that
file does really best described as recovery control? I'm not so sure
about that.

--
Robert Haas
EDB: http://www.enterprisedb.com

On 10/16/23 10:19, Robert Haas wrote:
> On Sat, Oct 14, 2023 at 2:22 PM David Steele <david@pgmasters.net> wrote:
>> I was recently discussing the complexities of dealing with pg_control
>> and backup_label with some hackers at PGConf NYC, when David Christensen
>> commented that backup_label was not a very good name since it gives the
>> impression of being informational and therefore something the user can
>> delete. In fact, we see this happen quite a lot, and there have been
>> some other discussions about it recently, see [1] and [2]. I bounced the
>> idea of a rename off various hackers at the conference and in general
>> people seemed to think it was a good idea.
> 
> Personally, I feel like this is an area where we keep moving the parts
> around but I'm not sure we're really getting to anything better. We
> got rid of recovery.conf. 

I agree that this was not an improvement. I was fine with bringing the 
recovery options into the GUC fold but never really liked forcing them 
into postgresql.auto.conf. But I lost that argument.

> We got rid of exclusive backup mode. We
> replaced pg_start_backup with pg_backup_start. 

I do think this was an improvement. For example it allows us to do [1], 
which I believe is a better overall solution to the problem of torn 
reads of pg_control. With exclusive backup we would not have this option.

> It feels like every
> other release or so we whack something around here, but I'm not
> convinced that any of it is really making much of an impact. If
> there's been any decrease in people screwing up their backups, I
> haven't noticed it.

It's pretty subjective, but I feel much the same way. However, I think 
the *areas* that people are messing up are changing as we remove 
obstacles and I feel like we should address them. backup_label has 
always been a bit of a problem -- basically deciding should it be deleted?

With the removal of exclusive backup we removed the only valid use case 
(I think) for removing backup_label manually. Now, it should probably 
never be removed manually, so we need to make adjustments to make that 
clearer to the user, also see [1].

Better messaging may also help, and I am also thinking about that.

> To be fair, I will grant that renaming pg_clog to pg_xact_status and
> pg_xlog to pg_wal does seem to have reduced the incidence of people
> nuking those directories, at least IME. So maybe this change would
> help too, for similar reasons. But I'm still concerned that we're
> doing too much superficial tinkering in this area. Breaking
> compatibility is not without cost.

True enough, but ISTM that we have gotten few (or any) actual complaints 
outside of hackers speculating that there will be complaints. For the 
various maintainers of backup software this is just business as usual. 
The changes to pg_basebackup are also pretty trivial.

> I also do wonder with recovery_control is really a better name. Maybe
> I just have backup_label too firmly stuck in my head, but is what that
> file does really best described as recovery control? I'm not so sure
> about that.

The thing it does that describes it as "recovery control" in my view is 
that it contains the LSN where Postgres must start recovery (plus TLI, 
backup method, etc.). There is some other informational stuff in there, 
but the important fields are all about ensuring consistent recovery.

At the end of the day the entire point of backup *is* recovery and users 
will interact with this file primarily in recovery scenarios.

Regards,
-David

---

[1] 
https://www.postgresql.org/message-id/1330cb48-4e47-03ca-f2fb-b144b49514d8%40pgmasters.net

On Mon, Oct 16, 2023 at 11:15:53AM -0400, David Steele wrote:
> On 10/16/23 10:19, Robert Haas wrote:
> > We got rid of exclusive backup mode. We replaced pg_start_backup
> > with pg_backup_start.
> 
> I do think this was an improvement. For example it allows us to do
> [1], which I believe is a better overall solution to the problem of
> torn reads of pg_control. With exclusive backup we would not have this
> option.

Well maybe, but it also seems to mean that any other 3rd party (i.e. not
Postgres-specific) backup tool seems to only support Postgres up till
version 14, as they cannot deal with non-exclusive mode - they are used
to a simple pre/post-script approach.

Not sure what to do about this, but as people/companies start moving to
15, I am afraid we will get people complaining about this. I think
having exclusive mode still be the default for pg_start_backup() (albeit
deprecated) in one release and then dropping it in the next was too
fast.

Or is somebody helping those "enterprise" backup solutions along in
implementing non-exclusive Postgres backups?


Michael

On Mon, Oct 16, 2023 at 12:06 PM Michael Banck <mbanck@gmx.net> wrote:
> Not sure what to do about this, but as people/companies start moving to
> 15, I am afraid we will get people complaining about this. I think
> having exclusive mode still be the default for pg_start_backup() (albeit
> deprecated) in one release and then dropping it in the next was too
> fast.

I completely agree, and I said so at the time, but got shouted down. I
think the argument that exclusive backups were breaking anything at
all was very weak. Nobody was being forced to use them, and they broke
nothing for people who didn't.

--
Robert Haas
EDB: http://www.enterprisedb.com

On 10/16/23 12:06, Michael Banck wrote:
> On Mon, Oct 16, 2023 at 11:15:53AM -0400, David Steele wrote:
>> On 10/16/23 10:19, Robert Haas wrote:
>>> We got rid of exclusive backup mode. We replaced pg_start_backup
>>> with pg_backup_start.
>>
>> I do think this was an improvement. For example it allows us to do
>> [1], which I believe is a better overall solution to the problem of
>> torn reads of pg_control. With exclusive backup we would not have this
>> option.
> 
> Well maybe, but it also seems to mean that any other 3rd party (i.e. not
> Postgres-specific) backup tool seems to only support Postgres up till
> version 14, as they cannot deal with non-exclusive mode - they are used
> to a simple pre/post-script approach.

I'd be curious to know what enterprise solutions currently depend on 
this method. At the very least they'd need to manage a WAL archive since 
copying pg_wal is not a safe thing to do (without a snapshot), so it's 
not just a matter of using start/stop scripts. And you'd probably want 
PITR, etc.

> Not sure what to do about this, but as people/companies start moving to
> 15, I am afraid we will get people complaining about this. I think
> having exclusive mode still be the default for pg_start_backup() (albeit
> deprecated) in one release and then dropping it in the next was too
> fast.

But lots of companies are on PG15 and lots of hosting providers support 
it, apparently with no issues. Perhaps the companies you are referring 
to are lagging in adoption (a pretty common scenario) but I still see no 
evidence that there is a big problem looming.

Exclusive backup was deprecated for six releases, which should have been 
ample time to switch over. All the backup solutions I am familiar with 
have supported non-exclusive backup for years.

> Or is somebody helping those "enterprise" backup solutions along in
> implementing non-exclusive Postgres backups?

I couldn't say, but there are many examples in open source projects of 
how to do this. Somebody (Laurenz, I believe) also wrote a shell script 
to simulate exclusive backup behavior for those that want to continue 
using it. Not what I would recommend, but he showed that it was possible.

Regards,
-David

On 10/16/23 12:12, Robert Haas wrote:
> On Mon, Oct 16, 2023 at 12:06 PM Michael Banck <mbanck@gmx.net> wrote:
>> Not sure what to do about this, but as people/companies start moving to
>> 15, I am afraid we will get people complaining about this. I think
>> having exclusive mode still be the default for pg_start_backup() (albeit
>> deprecated) in one release and then dropping it in the next was too
>> fast.
> 
> I completely agree, and I said so at the time, but got shouted down. I
> think the argument that exclusive backups were breaking anything at
> all was very weak. Nobody was being forced to use them, and they broke
> nothing for people who didn't.

My argument then (and now) is that exclusive backup prevented us from 
making material improvements in backup and recovery. It was complicated, 
duplicative (in code and docs), and entirely untested.

So you are correct that it was only dangerous to the people who were 
using it (even if they did not know they were), but it was also a 
barrier to progress.

Regards,
-David

On Mon, 2023-10-16 at 12:12 -0400, Robert Haas wrote:
> On Mon, Oct 16, 2023 at 12:06 PM Michael Banck <mbanck@gmx.net> wrote:
> > Not sure what to do about this, but as people/companies start moving to
> > 15, I am afraid we will get people complaining about this. I think
> > having exclusive mode still be the default for pg_start_backup() (albeit
> > deprecated) in one release and then dropping it in the next was too
> > fast.
>
> I completely agree, and I said so at the time, but got shouted down. I
> think the argument that exclusive backups were breaking anything at
> all was very weak. Nobody was being forced to use them, and they broke
> nothing for people who didn't.

+1

Yours,
Laurenz Albe

On 16.10.23 17:15, David Steele wrote:
>> I also do wonder with recovery_control is really a better name. Maybe
>> I just have backup_label too firmly stuck in my head, but is what that
>> file does really best described as recovery control? I'm not so sure
>> about that.
> 
> The thing it does that describes it as "recovery control" in my view is 
> that it contains the LSN where Postgres must start recovery (plus TLI, 
> backup method, etc.). There is some other informational stuff in there, 
> but the important fields are all about ensuring consistent recovery.
> 
> At the end of the day the entire point of backup *is* recovery and users 
> will interact with this file primarily in recovery scenarios.

Maybe "restore" is better than "recovery", since recovery also happens 
separate from backups, but restoring is something you do with a backup 
(and there is also restore_command etc.).

On 10/18/23 03:07, Peter Eisentraut wrote:
> On 16.10.23 17:15, David Steele wrote:
>>> I also do wonder with recovery_control is really a better name. Maybe
>>> I just have backup_label too firmly stuck in my head, but is what that
>>> file does really best described as recovery control? I'm not so sure
>>> about that.
>>
>> The thing it does that describes it as "recovery control" in my view 
>> is that it contains the LSN where Postgres must start recovery (plus 
>> TLI, backup method, etc.). There is some other informational stuff in 
>> there, but the important fields are all about ensuring consistent 
>> recovery.
>>
>> At the end of the day the entire point of backup *is* recovery and 
>> users will interact with this file primarily in recovery scenarios.
> 
> Maybe "restore" is better than "recovery", since recovery also happens 
> separate from backups, but restoring is something you do with a backup 
> (and there is also restore_command etc.).

I would not object to restore (there is restore_command) but I do think 
of what PostgreSQL does as "recovery" as opposed to "restore", which 
comes before the recovery. Recovery is used a lot in the docs and there 
is also recovery.signal.

But based on the discussion in [1] I think we might be able to do away 
with backup_label entirely, which would make this change moot.

Regards,
-David

[1] 
https://www.postgresql.org/message-id/0f948866-7caf-0759-d53c-93c3e266ec3f%40pgmasters.net