Re: Backup solution over unreliable network

On 30/11/18 8:22 μ.μ., Evan Bauer wrote:
> Achilleas,
>
> I may be over-simplifying your situation, but have you considered breaking the problem into two pieces?  First
backingup to a local drive and then using rsync to move those files over the unreliable network to the remote site.
 
>
> Like other who have responded, I can heartily recommend pgbackrest.  But if the network stinks, then I’d break the
problemin two and leave PostgreSQL out of the network equation.
 


Pretty good idea, but :

1) those rsync transfers have to be somehow db aware, otherwise lots of 
things might break, checksums, order of WALs, etc. There would be the 
need to write a whole solution and end up ... getting one of the 
established solutions

2) the rsync part would go basically unattended, meaning no smart 
software would be taking care of it, monitoring it, sending alerts, etc. 
Also we had our issues with rsync in the past with unreliable networks 
like getting error messages for which google returns one or no results 
(no pgsql stuff, just system scripts) . No wonder more and more PgSQL 
backup solutions move away from rsync.

Am I missing something or exaggerating?


>
> Cheers,
>
> Evan
>
> Sent from my iPhone
>
>> On Nov 30, 2018, at 05:17, Achilleas Mantzios <achill@matrix.gatewaynet.com> wrote:
>>
>> Hello,
>> we've been running our backup solution for the last 5 months to a second site which has an unreliable network
connection.We had problems with barman, since it doesn't support backup resume, also no option to disable the
replicationslot, in the sense, that it is better to sacrifice the backup rather than fill up the primary with WALs and
bringthe primary down. Another issue is now supporting entirely backing up from the secondary. With barman this is not
possible,streaming (or archiving) must originate from the primary.So I want to ask two things here :
 
>> - Backing up to a remote site over an unreliable channel is a limited use case by itself, it is useful for local
PITRrestores on specific tables/data, or in case the whole primary suffers a disaster. Is there any other benefit that
wouldjustify building a solution for it?
 
>> - I have only read the best reviews about PgBackRest, can PgBackRest address those issues?
>>
>> Thank you!
>>
>> -- 
>> Achilleas Mantzios
>> IT DEV Lead
>> IT DEPT
>> Dynacom Tankers Mgmt
>>
>>
>