Re: [PATCH] Fix Proposal - Deadlock Issue in Single User Mode When IOFailure Occurs

On Sun, Jan 20, 2019 at 4:45 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> On Sat, Dec 1, 2018 at 2:30 AM Chengchao Yu <chengyu@microsoft.com> wrote:
> > Recently, we hit a few occurrences of deadlock when IO failure (including disk full, random remote disk IO
failures)happens in single user mode. We found the issue exists on both Linux and Windows in multiple postgres
versions.
> >
> > 3.       Because the unable to write relation data scenario is difficult to hit naturally even reserved space is
turnedoff, I have prepared a small patch (see attachment “emulate-error.patch”) to force an error when PG tries to
writedata to relation files. We can just apply the patch and there is no need to put efforts flooding data to disk any
more.
>
> I have one question related to the way you have tried to emulate the error.
>
> @@ -840,6 +840,10 @@ mdwrite(SMgrRelation reln, ForkNumber forknum,
> BlockNumber blocknum,
> nbytes,
> BLCKSZ);
> + ereport(ERROR,
> + (errcode(ERRCODE_INTERNAL_ERROR),
> + errmsg("Emulate exception in mdwrite() when writing to disk")));
> +
>
> We generally reserve the space in a relation before attempting to
> write, so not sure how you are able to hit the disk full situation via
> mdwrite.  If you see the description of the function, that also
> indicates same.

Presumably ZFS or BTRFS or something more exotic could still get
ENOSPC here, and of course any filesystem could give us EIO here
(because the disk is on fire or the remote NFS server is rebooting due
to an automatic Windows update).

--
Thomas Munro
http://www.enterprisedb.com