Re: Vacuum rate limit in KBps

On Jan 19, 2012, at 4:23 PM, Greg Smith wrote:
> On 1/18/12 4:18 PM, Jim Nasby wrote:
>> What about doing away with all the arbitrary numbers completely, and just state data rate limits for hit/miss/dirty?
>
> Since many workloads will have a mix of all three, it still seems like there's some need for weighing these
individually,even if they each got their own rates.  If someone says read=8MB/s and write=4MB/s (the current effective
defaults),I doubt they would be happy with seeing 12MB/s happen. 
>
>> BTW, this is a case where it would be damn handy to know if the miss was really a miss or not... in the case where
we'realready rate limiting vacuum, could we afford the cost of get_time_of_day() to see if a miss actually did have to
comefrom disk? 
>
> We certainly might if it's a system where timing information is reasonably cheap, and measuring that exact area will
beeasy if the timing test contrib module submitted into this CF gets committed.  I could see using that to re-classify
somemisses as hits if the read returns fast enough. 
>
> There's not an obvious way to draw that line though.  The "fast=hit" vs. "slow=miss" transition happens at very
differentplace on SSD vs. regular disks, as the simplest example.  I don't see any way to wander down this path that
doesn'tend up introducing multiple new GUCs, which is the opposite of what I'd hoped to do--which was at worst to keep
thesame number, but reduce how many were likely to be touched. 

Your two comments together made me realize something... at the end of the day people don't care about MB/s. They care
aboutimpact to other read and write activity in the database. 

What would be interesting is if we could monitor how long all *foreground* IO requests took. If they start exceeding
somenumber, that means the system is at or near full capacity, and we'd like background stuff to slow down. 

Dealing with SSDs vs real media would be a bit challenging... though, I think it would only be an issue if the two were
randomlymixed together. Kept separately I would expect them to have distinct behavior patterns that could be measured
andidentified. 
--
Jim C. Nasby, Database Architect                   jim@nasby.net
512.569.9461 (cell)                         http://jim.nasby.net