Re: Slow concurrent processing

On 03/11/2013 08:55 PM, Misa Simic wrote:
> Hi all,
>
> We have one table with list of "records for processing"...
>
> We loop trough that table and call one long runing function:
>
>  do_the_math_for_record(record_id)...<snip>...
>
> but - if replace do_the_math_and_save_results with pg_sleep(1); To
> simulate long running function so processing each row - last 1 sec...
>
> Sequential processing last as expected 300 seconds!
>
> Concurrent processing last faster with higher number of
> concurrent connections - about 30 seconds with 30 connections! (much
> faster - and expected...)
>
> however, if we return our: do_the_math_and_save_results - we can't get
> better results in concurrent processing...

Sleep will not have any significant impact on CPU, memory or disk use
and thus is not a simulation of actual processing.

All you have really shown us so far is a black box. Please provide an
overview of your schemas and the type of processing you are attempting
on them.

Cheers,
Steve