Обсуждение: coypu: "FATAL: sorry, too many clients already"

Поиск
Список
Период
Сортировка

coypu: "FATAL: sorry, too many clients already"

От
Thomas Munro
Дата:
Hello,

Several times on master[1] beginning with an initial occurrence 36
days ago, and every time on REL_12_STABLE[2], but not on older
branches, build farm animal coypu has failed in the regression tests
with the error given in the subject.  How can there be too many if
there are only 20 in a parallel group?

[1] https://buildfarm.postgresql.org/cgi-bin/show_history.pl?nm=coypu&br=HEAD
[2] https://buildfarm.postgresql.org/cgi-bin/show_history.pl?nm=coypu&br=REL_12_STABLE

-- 
Thomas Munro
https://enterprisedb.com



Re: coypu: "FATAL: sorry, too many clients already"

От
Tom Lane
Дата:
Thomas Munro <thomas.munro@gmail.com> writes:
> Several times on master[1] beginning with an initial occurrence 36
> days ago, and every time on REL_12_STABLE[2], but not on older
> branches, build farm animal coypu has failed in the regression tests
> with the error given in the subject.  How can there be too many if
> there are only 20 in a parallel group?

Well, (a) according to the initdb steps in successful runs, coypu
is running with very resource-starved settings, because initdb picks

selecting default max_connections ... 20
selecting default shared_buffers ... 128MB

meaning there is exactly no headroom above the 20 parallel tests
that we sometimes try to launch.  (Digging in the buildfarm logs
shows that it used to pick 30, so something's changed there.)

(b) coypu is running with force_parallel_mode = regress, which
comes close to doubling its requirement for backend processes.
(It didn't use to do that, either.)

(c) session disconnect is asynchronous, so previous test backends might
still be cleaning up when new ones try to launch.

The wonder is not that coypu sometimes fails but that it ever succeeds.

I don't see a really good reason to be using force_parallel_mode on
such a low-end box, and would recommend taking that out in any case.
If the box's SysV IPC limits can't be raised, it might be a good idea
to restrict the maximum test parallelism.  For instance, I run
prairiedog with

    'build_env' => {
                                    'MAX_CONNECTIONS' => '3',
                   },

because I determined a long time ago that it got through the parallel
tests the fastest that way.  (Perhaps this setting is no longer optimal,
but the exact value is not very relevant here.)

            regards, tom lane



Re: coypu: "FATAL: sorry, too many clients already"

От
Rémi Zara
Дата:

> Le 10 juil. 2019 à 04:09, Tom Lane <tgl@sss.pgh.pa.us> a écrit :
>
> I don't see a really good reason to be using force_parallel_mode on
> such a low-end box, and would recommend taking that out in any case.
> If the box's SysV IPC limits can't be raised, it might be a good idea
> to restrict the maximum test parallelism.  For instance, I run
> prairiedog with
>
>    'build_env' => {
>                                    'MAX_CONNECTIONS' => '3',
>                   },
>

Hi,

The difference of behavior might be explained by an upgraded OS and upgraded buildfarm script.
I’ll implement the suggested setting as soon as I can powercycle the machine which I fail to contact right now.

Regards,

Rémi Zara