Re: WAL Rate Limiting

On 1/16/14, 9:19 PM, Craig Ringer wrote:
> On 01/16/2014 11:52 PM, Tom Lane wrote:
>> Heikki Linnakangas <hlinnakangas@vmware.com> writes:
>>> On 01/16/2014 05:39 PM, Andres Freund wrote:
>>>> Do you see a reasonable way to implement this generically for all
>>>> commands? I don't.
>>
>>> In suitable safe places, check if you've written too much WAL, and sleep
>>> if so. Call it CHECK_FOR_WAL_BUDGET(), along the lines of
>>> CHECK_FOR_INTERRUPTS(), but called less frequently. Or maybe
>>> vacuum_delay_point() is a better analogy. Hopefully you don't need to
>>> sprinkle them in too many places to be useful.
>>
>> We probably don't really need to implement it for "all" commands; I think
>> everyone can agree that, say, ALTER TABLE RENAME COLUMN isn't going to
>> emit enough WAL to really matter.  My point was that we should try to hit
>> everything that potentially *could* generate lots of WAL, rather than
>> arbitrarily deciding that some are utility commands and some are not.
>
> Then you land up needing a configuration mechanism to control *which*
> commands get affected, too, to handle the original use case of "I want
> to rate limit vaccuum and index rebuilds, while everything else runs
> full tilt".
>
> Or do you propose to just do this per-session? If so, what about autovacuum?

Tom was proposing per-session.

For autovac, don't we already have some method of changing the GUCs it uses? If not, we should...
-- 
Jim C. Nasby, Data Architect                       jim@nasby.net
512.569.9461 (cell)                         http://jim.nasby.net