Re: Performance bottleneck. High active sessions but postmaster kernel threads are in a sleep state, low CPU utilization

Hi,

You showed a lot of detail from the operating system perspective, but nothing from pg_stat_activity or pg_locks during those contention times where the cpu is low but lot of concurrent activity in the database.

I would guess that there is a lot of lightweight and/or spinlocks going on. This activity usually occurs when there appears to be no heavy cpu activity, no cpu io wait activity, but the db is being hit big time with a bunch of concurrent active connections contending over the same resources as Laurenz mentioned.

You can identify lightweight locks/spinlocks with the general wait type: wait_event_type = LWLock. The wait_event column will drill down even further.  You can also help yourself by installing and using pg_wait_sampling extension.  Because locks come and go so fast in pg_stat_activity, this historical perspective from the sampling extension may provide a better picture.
https://github.com/postgrespro/pg_wait_sampling

Regards,
Michael Vitale

Laurenz Albe wrote on 3/31/2021 5:47 AM:

On Tue, 2021-03-30 at 12:22 -0600, Hotmail wrote:

We are trying to gain some insight into a performance bottleneck that we are hitting while load testing Postgres on 11.11.
[hundreds of active sessions, but CPU is not maxed out]

With that many active sessions you are probably hitting some contention inside
the database.  Look at "wait_event" and "wait_event_type" in pg_stat_activity.

Yours,
Laurenz Albe