Re: ice-broker scan thread

"David Boreham" <david_list@boreham.org> wrote
>>
> I don't think your NT overlapped I/O code is quite right. At least
> I think it will issue reads at a high rate without waiting for any of them
> to complete. Beyond some point that has to give the kernel gut-rot.
>

[also with reply to Gavin] look up dictionary for "gut-rot", got it ... Uh, 
this behavior is intended - I try to push enough requests shortly to kernel 
so that it understands that I am doing sequential scan, so it would pull the 
data from disk to file system cache more efficiently. Some file systems may 
have "free-behind" mechanism, but our main thread (who really process the 
query) should be fast enough before the data vanished.

>
> You could re-write your program to have a single thread but use aio.
> In that case it should show the same read ahead benefit that you see
> with the thread.
>

I guess this is also Gavin's point - I understand that will be two different 
methodologies to handle "read-ahead". If no other thread/process involved, 
then the main thread will be responsible to grab a free buffer page from 
bufferpool and ask the kernel to put the data there by sync IO (current 
PostgreSQL does) or async IOs. And that's what I want to avoid. I'd like to 
use a dedicated thread/process to "break the ice" only, i.e., pull data from 
disk to file system cache, so that the main thread will only issue *logical* 
read.

Regards,
Qingqing