On 6/28/20 6:00 PM, Tom Lane wrote:
> Joe Conway <mail@joeconway.com> writes:
>> All good stuff -- I believe the attached checks all the boxes.
>
> Looks okay to me, except I think you want
>
> ! if (bytes_to_read > 0)
>
> to be
>
> ! if (bytes_to_read >= 0)
Yep -- thanks.
I did some performance testing of the worst case/largest possible file and found
that skipping the stat and bulk read does cause a significant regression.
Current HEAD takes about 400ms on my desktop, and with that version of the patch
more like 1100ms.
In the attached patch I was able to get most of the performance degradation back
-- ~600ms. Hopefully you don't think what I did was "too cute by half" :-). Do
you think this is good enough or should we go back to using the stat file size
when it is > 0?
As noted in the comment, the downside of that method is that the largest
supported file size is 1 byte smaller when "reading the entire file" versus
"reading a specified size" due to StringInfo reserving the last byte for a
trailing null.
Joe
--
Crunchy Data - http://crunchydata.com
PostgreSQL Support for Secure Enterprises
Consulting, Training, & Open Source Development