Re: [HACKERS] Re: [QUESTIONS] Does Storage Manager support >2GB tables?

Bruce Momjian wrote:

> Yes, the threading topic has come up before, and I have never considered
> it a big win.  We want to remove the exec() from the startup, so we just
> do a fork.  Will save 0.001 seconds of startup.

As I was always biased to threading I would note that in many cases it
is a big win. First of all, today it's the IO which is usually the
slowest part of the database. Most of good IO designs have different
optimisations like tag queuing etc. But, to make use of all this stuff
you have to generate your IO requests as soon as possible. If you
process your query in one thread, most of the time your processor waits
for the IO, but sometimes your IO waits for the processor. If you
populate your IO system with a bunch of IO requests paralelly,
controllers may try to requeue them to optimise heads movements etc.
However, if you do some IO, then some processing, then some IO.... you
loose the capability of optimising your requests. Unless your system is
heavily loaded. In this case, it actually doesn't matter how you
schedule IO requests. But if your system is not loaded too heavily, it's
good to parallelize IO tasks. And the easiest way to accomplish it is to
use threads for parallel execution of tasks.

But I notice that many people still think of threads as a replacement
for fork. Of course, in such case it's pretty useless since fork is fast
enough. But the key to the success is to parallelize single queries not
only to leverage processor usage, but also to push IO to it's maximum.

> That is a very easy win for us.  I hadn't considered the synchonization
> problems with palloc/pfree, and that could be a real problem.

Few months ago I was thinking about it. Actually I don't see much
problems with things like palloc/pfree. I don't see any problems with
any heap data that is used locally. But it is a big problem when you
take a look at global variables and global data that is accessed and
modified in many places. This is potential source of troubles.

Mike

--
WWW: http://www.lodz.pdi.net/~mimo  tel: Int. Acc. Code + 48 42 148340
add: Michal Mosiewicz  *  Bugaj 66 m.54 *  95-200 Pabianice  *  POLAND