Lots of memory allocated when reassigning Large Objects
| От | Guillaume Lelarge |
|---|---|
| Тема | Lots of memory allocated when reassigning Large Objects |
| Дата | |
| Msg-id | CAECtzeW2DAoioEGBRjR=CzHP6TdL=yosGku8qZxfX9hhtrBB0Q@mail.gmail.com обсуждение исходный текст |
| Ответы |
Re: Lots of memory allocated when reassigning Large Objects
|
| Список | pgsql-hackers |
Hi,
One of our customers had a very bad issue while trying to reassign objects from user A to user B. He had a lot of them, and the backend got very hungry for memory. It finally all went down when the linux kernel decided to kill the backend (-9 of course).
I attach three shell scripts showing the issue. create.sh creates a database and two users. Then it imports a million Large Objects in this new database. There's no drop.sh as it is a simple "dropdb foodb".
session1_monitor.sh will start logging memory usage in the server log file every second. So it needs v14, but our customer is in v11. While this script runs, you can start session2_reindex.sh. This script will only run a reassign from one user to another.
Here is what I get in the server log file:
$ grep "Grand total" 14.log
LOG: Grand total: 15560832 bytes...
LOG: Grand total: 68710528 bytes...
LOG: Grand total: 119976064 bytes..
LOG: Grand total: 171626624 bytes...
LOG: Grand total: 224211072 bytes...
LOG: Grand total: 276615296 bytes...
LOG: Grand total: 325611648 bytes...
LOG: Grand total: 378196096 bytes...
LOG: Grand total: 429838464 bytes...
LOG: Grand total: 481104000 bytes...
LOG: Grand total: 68710528 bytes...
LOG: Grand total: 119976064 bytes..
LOG: Grand total: 171626624 bytes...
LOG: Grand total: 224211072 bytes...
LOG: Grand total: 276615296 bytes...
LOG: Grand total: 325611648 bytes...
LOG: Grand total: 378196096 bytes...
LOG: Grand total: 429838464 bytes...
LOG: Grand total: 481104000 bytes...
IOW, it's asking for at least 481MB to reassign 1 million empty LO. It strikes me as odd.
FWIW, the biggest memory context is this one:
LOG: level: 2; PortalContext: 479963904 total in 58590 blocks; 2662328 free (32567 chunks); 477301576 used: <unnamed>
Thanks.
Regards.
PS : we've found a workaround to make it work for our customer (executing all the required ALTER LARGE OBJECT ... OWNER TO ...), but I'm still amazed by this weird behaviour.
--
Guillaume.
Вложения
В списке pgsql-hackers по дате отправления: