Re: Set hint bits upon eviction from BufMgr

On Fri, Mar 25, 2011 at 10:34 AM, Jim Nasby <jim@nasby.net> wrote:
> On Mar 25, 2011, at 9:52 AM, Merlin Moncure wrote:
>> Without this bit, the only way to set hint bits going during bufmgr
>> eviction is to do a visibility check on every tuple, which would
>> probably be prohibitively expensive.  Since OLTP environments would
>> rarely see this bit, they would not have to pay for the check.
>
> IIRC one of the biggest costs is accessing the CLOG, but what if the bufmgr.c/bgwriter didn't use the same CLOG
lookupmechanism as backends did? Unlike when a backend is inspecting visibility, it's not necessary for something like
bgwriterto know exact visibility as long as it doesn't mark something as visible when it shouldn't. If it uses a
differentCLOG caching/accessing method that lags behind the real CLOG then the worst-case scenario is that there's a
delayon setting hint bits. But getting grwiter to dothis would likely still be a huge win over forcing backends to
worryabout it. It's also possible that the visibility check itself could be simplified. 
>
> BTW, I don't think you want to play these games when a backend is evicting a page because you'll be slowing a real
backenddown. 

Well, I'm not so sure -- as noted above, you only pay for the check
above when all the records in a page are new, and only once per page,
not once per tuple.  Basically, only when you are bulk jamming records
through the buffers.  The amoritized cost of the clog lookup is going
to be near zero (maybe you could put a fuse in that would get tripped
if there weren't enough tuples in the page to justify the check).

If you are bulk loading more data than you have shared buffers, then
you get zero benefit.  However, you might having the makings of a
strategy of dealing with hint bit i/o in user land.  (by breaking up
transactions, tweaking shared buffers, etc).

merlin