BUG #16032: pg_basebackup when running on Windows doesn't clean up on failure correctly

The following bug has been logged on the website:

Bug reference:      16032
Logged by:          Rob Emery
Email address:      postgresql@mintsoft.net
PostgreSQL version: 11.5
Operating system:   Windows
Description:        

Hello,

When running a pg_basebackup in tar mode on windows against a PG 9.5 box we
are finding that the backup doesn't get deleted successfully when it
fails.
Based on
https://github.com/postgres/postgres/commit/9083353b15c3cf8e7bbac104a81ad42281178cdf#diff-473d306e41ee616e92fb58ac128a9dcc
we would expect it to?

To reproduce start running pg_basebackup like so:

PS C:\Users\developer> & "$PgBinPath\pg_basebackup.exe" --host="PGSERVER"
--pgdata="D:\Backups\rofl" --format=tar --wal-method=fetch
--username="backup_user"

from a psql on the box run: 
`SELECT pg_terminate_backend(pid) FROM pg_stat_replication WHERE state =
'backup';`
this kills the backup.

If you run pg_basebackup on a linux box it nicely deletes the base.tar and
the directory, whereas on windows we get the following error:
```
pg_basebackup: could not read COPY data: SSL SYSCALL error: EOF detected
pg_basebackup: removing data directory "D:\Backups\rofl"
could not remove file or directory "D:\Backups\rofl/base.tar": Permission
denied
could not remove file or directory "D:\Backups\rofl": Directory not empty
pg_basebackup: failed to remove data directory
```
Once pg_basebackup has exited, the file lock seems to be gone and we can
delete it ourselves, 
it looks like pg_basebackup is contending with itself but only on Windows.

We've reproduced this on 2 different Windows machines so we don't think it's
something like AntiVirus getting in the way or similiar.

Thanks,
Rob