Re: Threaded Sorting

On Fri, 4 Oct 2002, Bruce Momjian wrote:

> Hans-Jürgen Schönig wrote:
> > Did anybody think about threaded sorting so far?
> > Assume an SMP machine. In the case of building an index or in the case 
> > of sorting a lot of data there is just one backend working. Therefore 
> > just one CPU is used.
> > What about starting a thread for every temporary file being created? 
> > This way CREATE INDEX could use many CPUs.
> > Maybe this is worth thinking about because it will speed up huge 
> > databases and enterprise level computing.
> 
> We haven't thought about it yet because there are too many buggy thread
> implementations.  We are probably just now getting to a point where we
> can consider it.  However, lots of databases have moved to threads for
> all sorts of things and ended up with a royal mess of code.  Threads
> can only improve things in a few areas of the backend so it would be
> nice if we could limit the exposure to threads to those areas;  sorting
> could certainly be one of them, but frankly, I think disk I/O is our
> limiting factore there.  I would be interested to see some tests that
> showed otherwise.

Wouldn't the type of disk subsystem really make a big difference here?

With a couple of U160 cards and a dozen 15krpm hard drives, I would 
imagine I/O would no longer be as much of an issue as a single drive 
system would be.

It seems like sometimes we consider these issues more from the one or two 
SCSI drives perspective insted of the big box o drives perspective.