Performance problem in aset.c
От | Tom Lane |
---|---|
Тема | Performance problem in aset.c |
Дата | |
Msg-id | 18059.963373725@sss.pgh.pa.us обсуждение исходный текст |
Ответы |
Re: Performance problem in aset.c
Re: Performance problem in aset.c |
Список | pgsql-hackers |
While testing the just-committed executor memory leak fixes, I observe that there are still slow leaks in some operations such as sorts on large text fields. With a little digging, it seems that the leak must be blamed on the behavior of aset.c for large chunks. Chunks between 1K and 64K (ALLOC_SMALLCHUNK_LIMIT and ALLOC_BIGCHUNK_LIMIT) are all kept in the same freelist, and when a request is made for an amount of memory in that range, aset.c gives back the first chunk that's big enough from that freelist. For example, let's say you are allocating and freeing roughly equal numbers of 2K and 10K blocks. Over time, about half of the 2K requests will be answered by returning a 10K block --- which will prevent the next 10K request from being filled by recycling that 10K block, causing a new 10K block to be allocated. If aset.c had given back a 2K block whenever possible, the net memory usage would be static, but since it doesn't, the usage gradually creeps up as more and more chunks are used inefficiently. Our actual maximum memory usage might be m * 2K plus n * 10K but the allocated space will creep towards (m + n) * 10K where *all* the active blocks are the larger size. A straightforward solution would be to scan the entire freelist and give back the smallest block that's big enough for the request. That could be pretty slow (and induce lots of swap thrashing) so I don't like it much. Another idea is to split the returned chunk and put the wasted part back as a smaller free chunk, but I don't think this solves the problem; it just means that the wasted space ends up on a small-chunk freelist, not that you can actually do anything with it. But maybe someone can figure out a variant that works better. It might be that this behavior won't be seen much in practice and we shouldn't slow down aset.c at all to try to deal with it. But I think it's worth looking for solutions that won't slow typical cases much. regards, tom lane
В списке pgsql-hackers по дате отправления: