Re: WIP/PoC for parallel backup

On Tue, Apr 7, 2020 at 10:14 PM Asif Rehman <asifr.rehman@gmail.com> wrote:

Hi,

Thanks, Kashif and Rajkumar. I have fixed the reported issues.

I have added the shared state as previously described. The new grammar changes

are as follows:

START_BACKUP [LABEL '<label>'] [FAST] [MAX_RATE %d]

    - This will generate a unique backupid using pg_strong_random(16) and hex-encoded

      it. which is then returned as the result set.

    - It will also create a shared state and add it to the hashtable. The hash table size is set

      to BACKUP_HASH_SIZE=10, but since hashtable can expand dynamically, I think it's

      sufficient initial size. max_wal_senders is not used, because it can be set to quite a 

      large values.

JOIN_BACKUP 'backup_id'

    - finds 'backup_id' in hashtable and attaches it to server process.

SEND_FILE '(' 'FILE' ')' [NOVERIFY_CHECKSUMS]

    - renamed SEND_FILES to SEND_FILE

    - removed START_WAL_LOCATION from this because 'startptr' is now accessible through

      shared state.

There is no change in other commands:

STOP_BACKUP [NOWAIT]
LIST_TABLESPACES [PROGRESS]
LIST_FILES [TABLESPACE]
LIST_WAL_FILES [START_WAL_LOCATION 'X/X'] [END_WAL_LOCATION 'X/X']

The current patches (v11) have been rebased to the latest master. The backup manifest is enabled

by default, so I have disabled it for parallel backup mode and have generated a warning so that

user is aware of it and not expect it in the backup.

So, are you working on to make it work? I don't think a parallel backup feature should be creating a backup with no manifest.