Re: pg_read_file() with virtual files returns empty string

On 7/1/20 6:22 PM, Tom Lane wrote:
> Joe Conway <mail@joeconway.com> writes:
>> The only downside is that the max filesize is reduced to (MaxAllocSize -
>> MIN_READ_SIZE - 1) compared to MaxAllocSize with the old method.
>
> Hm, I was expecting that the last successful iteration of
> enlargeStringInfo would increase the buffer size to MaxAllocSize,
> so that we'd really only be losing one byte (which we can't avoid
> if we use stringinfo).  But you're right that it's most likely moot
> since later manipulations of such a result would risk hitting overflows.
>
> I marked the CF entry as RFC.

Sorry to open this can of worms again, but I couldn't get my head past the fact
that reading the entire file would have a different size limit than reading the
exact number of bytes in the file.

So, inspired by what you did (and StringInfo itself) I came up with the
attached. This version performs equivalently to your patch (and HEAD), and
allows files up to and including (MaxAllocSize - VARHDRSZ) -- i.e. exactly the
same as the specified-length case and legacy behavior for the full file read.

But if you object I will just go with your version barring any other opinions.

Joe

--
Crunchy Data - http://crunchydata.com
PostgreSQL Support for Secure Enterprises
Consulting, Training, & Open Source Development