Re: Online base backup from the hot-standby

Thanks for the review!

On Tue, Oct 25, 2011 at 12:24 AM, Heikki Linnakangas
<heikki.linnakangas@enterprisedb.com> wrote:
> On 24.10.2011 15:29, Fujii Masao wrote:
>>
>> +    <listitem>
>> +     <para>
>> +      Copy the pg_control file from the cluster directory to the global
>> +      sub-directory of the backup. For example:
>> + <programlisting>
>> + cp $PGDATA/global/pg_control /mnt/server/backupdir/global
>> + </programlisting>
>> +     </para>
>> +    </listitem>
>
> Why is this step required? The control file is overwritten by information
> from the backup_label anyway, no?

Yes, when recovery starts, the control file is overwritten. But before that,
we retrieve the minimum recovery point from the control file. Then it's used
as the backup end location.

During recovery, pg_stop_backup() cannot write an end-of-backup record.
So, in standby-only backup, other way to retrieve the backup end location
(instead of an end-of-backup record) is required. Ishiduka-san used the
control file as that, according to your suggestion ;)
http://archives.postgresql.org/pgsql-hackers/2011-05/msg01405.php

>> +    <listitem>
>> +     <para>
>> +      Again connect to the database as a superuser, and execute
>> +      <function>pg_stop_backup</>. This terminates the backup mode, but
>> does not
>> +      perform a switch to the next WAL segment, create a backup history
>> file and
>> +      wait for all required WAL segments to be archived,
>> +      unlike that during normal processing.
>> +     </para>
>> +    </listitem>
>
> How do you ensure that all the required WAL segments have been archived,
> then?

The patch doesn't provide any capability to ensure that, IOW assumes that's
a user responsibility. If a user wants to ensure that, he/she needs to calculate
the backup start and end WAL files from the result of pg_start_backup()
and pg_stop_backup() respectively, and needs to wait until those files have
appeared in the archive. Also if the required WAL file has not been archived
yet, a user might need to execute pg_switch_xlog() in the master.

If we change pg_stop_backup() so that, even during recovery, it waits until
all required WAL files have been archived, we would need to WAL-log
the completion of WAL archiving in the master. This enables the standby to
check whether specified WAL files have been archived. We should change
the patch in this way? But even if we change, you still might need to execute
pg_switch_xlog() in the master additionally, and pg_stop_backup() might keep
waiting infinitely if the master is not in progress.

>> +   </orderedlist>
>> +    </para>
>> +
>> +    <para>
>> +     You cannot use the <application>pg_basebackup</> tool to take the
>> backup
>> +     from the standby.
>> +    </para>
>
> Why not? We have cascading replication now.

Because no one has implemented that feature.

Yeah, we have cascading replication, but without adopting the standby-only
backup patch, pg_basebackup cannot execute do_pg_start_backup() and
do_pg_stop_backup() during recovery. So we can think that the patch that
Ishiduka-san proposed is the first step to extend pg_basebackup so that it
can take backup from the standby.

Regards,

--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center