Re: Load Distributed Checkpoints, take 3

Heikki Linnakangas  writes:
> Tom Lane wrote:
>> I still think you've not demonstrated a need to expose this parameter.

> Greg Smith wanted to explicitly control the I/O rate, and let the 
> checkpoint duration vary. I personally think that fixing the checkpoint 
> duration is better because it's easier to tune.

> But if we only do that, you might end up with ridiculously long 
> checkpoints when there's not many dirty pages. If we want to avoid that, 
> we need some way of telling what's a safe minimum rate to write at, 
> because that can vary greatly depending on your hardware.

As long as the minimum rate is at least 1/bgdelay, I don't think this is
a big problem.

> But maybe we don't care about prolonging checkpoints, and don't really 
> need any GUCs at all. We could then just hardcode writes_per_nap to some 
> low value, and target duration close to 1.0. You would have a checkpoint 
> running practically all the time, and you would use 
> checkpoint_timeout/checkpoint_segments to control how long it takes. I'm 
> a bit worried about jumping to such a completely different regime, 
> though. For example, pg_start_backup needs to create a new checkpoint, 
> so it would need to wait on average 1.5 * checkpoint_timeout/segments, 

Maybe I misread the patch, but I thought that if someone requested an
immediate checkpoint, the checkpoint-in-progress would effectively flip
to immediate mode.  So that could be handled by offering an immediate vs
extended checkpoint option in pg_start_backup.  I'm not sure it's a
problem though, since as previously noted you probably want
pg_start_backup to be noninvasive.  Also, one could do a manual
CHECKPOINT command then immediately pg_start_backup if one wanted
as-fast-as-possible (CHECKPOINT requests immediate checkpoint, right?)

> and recovery would need to process on average 1.5 as much WAL as before. 
> Though with LDC, you should get away with shorter checkpoint intervals 
> than before, because the checkpoints aren't as invasive.

No, you still want a pretty long checkpoint interval, because of the
increase in WAL traffic due to more page images being dumped when the
interval is short.

> If we do that, we should remove bgwriter_all_* settings. They wouldn't 
> do much because we would have checkpoint running all the time, writing 
> out dirty pages.

Yeah, I'm not sure that we've thought through the interactions with the
existing bgwriter behavior.

			regards, tom lane