Re: Fdw batch insert error out when set batch_size > 65535

On 6/13/21 5:25 PM, Bharath Rupireddy wrote:
> On Sun, Jun 13, 2021 at 6:10 AM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>>
>> On 2021-Jun-12, Tomas Vondra wrote:
>>
>>> There's one caveat, though - for regular builds the slowdown is pretty
>>> much eliminated. But with valgrind it's still considerably slower. For
>>> postgres_fdw the "make check" used to take ~5 minutes for me, now it
>>> takes >1h. And yes, this is entirely due to the new test case which is
>>> generating / inserting 70k rows. So maybe the test case is not worth it
>>> after all, and we should get rid of it.
>>
>> Hmm, what if the table is made 1600 columns wide -- would inserting 41
>> rows be sufficient to trigger the problem case?  If it does, maybe it
>> would reduce the runtime for valgrind/cache-clobber animals enough that
>> it's no longer a concern.
> 
> Yeah, that's a good idea. PSA patch that creates the table of 1600
> columns and inserts 41 rows into the foreign table. If the batch_size
> adjustment fix isn't there, we will hit the error. On my dev system,
> postgres_fdw contrib regression tests execution time: with and without
> the attached patch 4.5 sec and 5.7 sec respectively.
> 

But we're discussing cases with valgrind and/or CLOBBER_CACHE_ALWAYS.


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company