Re: ice-broker scan thread

Qingqing Zhou wrote:

>I am considering add an "ice-broker scan thread" to accelerate PostgreSQL
>sequential scan IO speed. The basic idea of this thread is just like the
>"read-ahead" method, but the difference is this one does not read the data
>into shared buffer pool directly, instead, it reads the data into file
>system cache, which makes the integration easy and this is unique to
>PostgreSQL.
>  
>
Interesting, and I wondered about this too. But for my taste the 
demonstrated benefit really
isn't large enough to make it worthwhile.
BTW, I heard a long time ago that NTFS has quite fancy read-ahead, where 
it attempts
to detect the application's access pattern including if it is reading 
sequentially and even
if there is a 'stride' to the accesses when they're not contiguous. I 
would imagine that
other filesystems attempt similar tricks. So one might expect a simple 
linear prefectch
to not help much in the presence of such a filesystem.

Were you worried about the icebreaker thread getting too far ahead of 
the scan ?
If it did it might page out the data you're about to read, I think. Of 
course this could
be fixed by having the read ahead thread perodically check the current 
location being
read by the query thread and pausing if it's got too far ahead.

Anyway, the recent performance thread has been intersting to me because 
in all my career
I've never seen a database that scanned scads of data from disk to 
process a query.
Typically the problems I work on arrange to read the entire database 
into memory.
I think I need to get out more... ;)