Re: parallel pg_restore - WIP patch

Andrew Dunstan wrote:
> 
> 
> Andrew Dunstan wrote:
>>
>>
>>>
>>> this works better but there is something fishy still - using the same 
>>> dump file I get a proper restore using pg_restore normally. If I 
>>> however use -m for a parallel one I only get parts (in this case only 
>>> 243 of the 709 tables) of the database restored ...
>>>
>>>
>>>
>>
>> Yes, there are several funny things going on, including some stuff 
>> with dependencies. I'll have a new patch tomorrow with luck. Thanks 
>> for testing.
>>
>>
> 
> OK, in this version a whole heap of bugs are fixed, mainly those to do 
> with dependencies and saved state. I get identical row counts in the 
> source and destination now, quite reliably.

this looks much better (for a restore that usually takes 180min I can 
get down to 72min using -m 4) - however especially with higher 
concurrency I'm sometimes running into restore failures due to deadlocks 
happening during constraint restoration (slightly redacted):

pg_restore: [archiver (db)] Error from TOC entry 7765; 2606 1460743180 
FK CONSTRAINT fk_av_relations_av db_owner
pg_restore: [archiver (db)] could not execute query: ERROR:  deadlock 
detected
DETAIL:  Process 18100 waits for AccessExclusiveLock on relation 
1460818342 of database 1460815284; blocked by process 18103.
Process 18103 waits for AccessExclusiveLock on relation 1460818336 of 
database 1460815284; blocked by process 18100.
HINT:  See server log for query details.

ALTER TABLE ONLY foo    ADD CONSTRAINT fk_av_relations_av FOREIGN KEY (vs_id) REFERENCES 
bar ...