Re: Parallel postgresql

Bruce Momjian wrote:
> Martin Rusoff wrote:
> 
>>I was just contemplating how to make postgres parallel (for DSS 
>>applications)... Has anyone done work on this? It looks to me like there 
>>are a couple of obvious places to add parallel operation:
>>
>>Stage 1) I/O , perhaps through MPIO - would improve tablescanning and 
>>load/unload operations. One (or more) Postgresql servers would use 
>>MPIO/ROMIO to access a parallel file system like PVFS or GPFS(IBM).
>>
>>Stage 2) Parallel Postgres Servers, with the postmaster spawning off the 
>>server on a different node (possibly borrowing some code from GNU queue) 
>>and doing any buffer twiddling with RPC for that connection, The client 
>>connection would still be through the proxy on the postmaster node? (kind 
>>of like MOSIX)
> 
> 
> One idea would be to throw parts of the executor (like a table sort) to
> different machines or to different processors on the same machine,
> perhaps via dblink.  You could use threads to send several requests and
> wait for their results.
> 
> Threading the entire backend would be hard, but we could thread some
> parts of it by having slave backends doing some of the work in parallel.



This would be nice - especially for huge queries needed in warehouses.
Maybe it could even make sense to do things in par. if there is just one 
machine (e.g. computing a function while a sort process is waiting for 
I/O or so).

Which operations can run in par.? What do you think?
I guess implementing something like that means 20 years more work on the 
planner ...

By the way: NCR has a quite nice solution for problems like that. 
Teradata has been designed to run everything on multiple nodes (they 
call it AMPs).
Teradata has been designed for A LOT OF data and reporting purposes.
There are just three problems:- not Open Source- ~$70k / node- runs on Windows and NCR's UNIX implementation.

Is anybody familiar with Teradata?
Hans







-- 
Cybertec Geschwinde u Schoenig
Ludo-Hartmannplatz 1/14, A-1160 Vienna, Austria
Tel: +43/2952/30706 or +43/660/816 40 77
www.cybertec.at, www.postgresql.at, kernel.cybertec.at