Re: Experimental patch for inter-page delay in VACUUM

Ang Chin Han wrote:

> Christopher Browne wrote:
>
>> Centuries ago, Nostradamus foresaw when "Stephen" 
>> <jleelim@xxxxxxx.com> would write:
>>
>>> As it turns out. With vacuum_page_delay = 0, VACUUM took 1m20s (80s)
>>> to complete, with vacuum_page_delay = 1 and vacuum_page_delay = 10,
>>> both VACUUMs completed in 18m3s (1080 sec). A factor of 13 times! 
>>> This is for a single 350 MB table.
>>
>> While it is unfortunate that the minimum quanta seems to commonly be
>> 10ms, it doesn't strike me as an enormous difficulty from a practical
>> perspective.
>
> If we can't lower the minimum quanta, we could always vacuum 2 pages 
> before sleeping 10ms, effectively sleeping 5ms.

Right, I think this is what Jan has done already.

> What would be interesting would be pg_autovacuum changing these values 
> per table, depending on current I/O load.
>
> Hmmm. Looks like there's a lot of interesting things pg_autovacuum can 
> do:
> 1. When on low I/O load, running multiple vacuums on different, 
> smaller tables on full speed, careful to note that these vacuums will 
> increase the I/O load as well.
> 2. When on high I/O load, vacuum big, busy tables slowly.

I'm not sure how practacle any of this is.  How will pg_autovacuum 
surmise the current I/O load of the system?  Keeping in mind that 
postgres is not the only cause of I/O.  Also, the optimum delay for a 
long running vacuum might change dramatically while it's running.  If 
there is a way to judge the current I/O load, it might be better for 
vacuum to auto-tune itself while it's running, perhaps based on some 
hints given to it by pg_autovacuum or manually by a user.   For example, 
a delay hint of 0 should always be zero no matter what.  A delay hint of 
1 will scale up slower than a delay hint of 2 which would scale up 
slower than 5 etc.... 

Of course this is all wild conjecture if there isn't an easy way to 
surmise the system I/O load.  Thoughts?