Re: Improving the memory allocator

On Tuesday, April 26, 2011 01:24:20 AM Robert Haas wrote:
> On Mon, Apr 25, 2011 at 5:45 PM, Andres Freund <andres@anarazel.de> wrote:
> > [ lots of awesome test results ]
> 
> Very interesting work.  I think it's interesting that there is so much
> allocation happening inside MessageContext; I barely knew that
> existed, let alone that it was a hotspot for memory allocation.
Yes, I was surprised as well. I haven't checked yet what the callsites for 
that are.
As I just wrote to TOm the hotspots are wildly different if you use slightly 
more complex statements. Which doesn't mean its not worthy of improvement ;)

> I
> think it would be interesting to break out the calls to new_list() by
> caller.  It's been vaguely bothering me for a long time that we store
> so many things in using List structures.  That is not a particularly
> efficient representation, because a List of N nodes requires 2N+1
> memory allocations - N nodes, N ListCell objects, and the List object
> itself.  It might be worth experimenting with some kind of array/list
> hybridy thing, like this:
> ...
While I aggree that that might be beneficial I think in this case nearly all 
of the lists are of 1 element length because its callers seem to look mostly 
like:
{{{
List *
lappend(List *list, void *datum)
{Assert(IsPointerList(list));
if (list == NIL)    list = new_list(T_List);else    new_tail_cell(list);
lfirst(list->tail) = datum;check_list_invariants(list);return list;
}
}}}

Hm. Seems worthy of some further investigation...

Thanks,

Andres