Re: BUG #17005: Enhancement request: Improve walsender throughput by aggregating multiple messages in one send

Andres Freund писал 2021-05-17 01:44:
> Hi,
> 
> On 2021-05-13 00:31:53 +0000, PG Bug reporting form wrote:
>> I measured the throughput of reading the logical replication slot and 
>> found
>> that in smaller row size (512 bytes) the throughput is 50% lower 
>> compared to
>> 1024 bytes.
> 
> Huh, that is interesting.
> 
> 
>> tcpdump shows that ethernet packets sent by the replication server 
>> contain
>> only one message per packet (see tcpdump output below).
>> May be this is the intended design to achieve low latency but this is 
>> not
>> favorable in application that requires high throughput.
> 
> What kind of network is this? I would have expected that if the network
> can't keep up the small sends would end up getting aggregated into
> larger packets anyway? Are you hitting a PPS limit due to the small
> packages, but not yet the throughput limit?

I believe the reason is more in sys-call and kernel cpu time overhead 
than in
network throughput. Especially in this "after meltdown+spectre" time.

> 
>> Is it possible for PostgreSQL to enable Nagle's algorithm on the 
>> streaming
>> socket for replication?
>> Or aggegate the messages manually before sending them in one send()?
> 
> I think we can probably do better in cases a transaction is more than a
> single change - but I don't think either enabling nagle's or 
> aggregation
> are really an option in case of single row transactions. The latency
> impact for some scenarios seems too high.

We have commit_siblings and commit_delay options. Probably similar could
be introduced for replication slot?

regards
Yura