Re: Parallel query execution

On Wed, Jan 16, 2013 at 09:11:20AM +0900, Michael Paquier wrote:
> 
> 
> On Wed, Jan 16, 2013 at 7:14 AM, Bruce Momjian <bruce@momjian.us> wrote:
> 
>     I mentioned last year that I wanted to start working on parallelism:
> 
>             https://wiki.postgresql.org/wiki/Parallel_Query_Execution
> 
>     Years ago I added thread-safety to libpq.  Recently I added two parallel
>     execution paths to pg_upgrade.  The first parallel path allows execution
>     of external binaries pg_dump and psql (to restore).  The second parallel
>     path does copy/link by calling fork/thread-safe C functions.  I was able
>     to do each in 2-3 days.
> 
>     I believe it is time to start adding parallel execution to the backend.
>     We already have some parallelism in the backend:
>     effective_io_concurrency and helper processes.  I think it is time we
>     start to consider additional options.
> 
>     Parallelism isn't going to help all queries, in fact it might be just a
>     small subset, but it will be the larger queries.  The pg_upgrade
>     parallelism only helps clusters with multiple databases or tablespaces,
>     but the improvements are significant.
> 
>     I have summarized my ideas by updating our Parallel Query Execution wiki
>     page:
> 
>             https://wiki.postgresql.org/wiki/Parallel_Query_Execution
> 
>     Please consider updating the page yourself or posting your ideas to this
>     thread.  Thanks.
> 
> Honestly that would be a great feature, and I would be happy helping working on
> it.
> Taking advantage of parallelism in a server with multiple core, especially for
> things like large sorting operations would be great.
> Just thinking loudly, but wouldn't it be the role of the planner to determine
> if such or such query is worth using parallelism? The executor would then be in
> charge of actually firing the tasks in parallel that planner has determined
> necessary to do.

Yes, it would probably be driven off of the optimizer statistics.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + It's impossible for everything to be true. +