mdnblocks is an amazing time sink in huge relations
От | Tom Lane |
---|---|
Тема | mdnblocks is an amazing time sink in huge relations |
Дата | |
Msg-id | 15265.939701548@sss.pgh.pa.us обсуждение исходный текст |
Ответы |
RE: [HACKERS] mdnblocks is an amazing time sink in huge relations
Re: [HACKERS] mdnblocks is an amazing time sink in huge relations |
Список | pgsql-hackers |
I have been doing some testing with multi-segment relations. (Not having sixty gigabytes laying around, like some folk I've been talking to recently, I rebuilt with a very small RELSEG_SIZE for testing...) I have discovered that performance goes to zero as the number of segments gets large --- in particular, if the number of segments exceeds the number of kernel file descriptors that fd.c thinks it can use, you can take a coffee break while inserting a few tuples :-(. The main problem is mdnblocks() in md.c, which is called for each tuple inserted; it insists on touching every segment of the relation to verify its length via lseek(). That's an unreasonable number of kernel calls in any case, and if you add a file open and close to each touch because of file-descriptor thrashing, it's coffee break time. It seems to me that mdnblocks should assume that all segments previously verified to be of length RELSEG_SIZE are still of that length, and only do an _mdnblocks() call on the last element of the segment chain. That way the number of kernel interactions needed is a constant independent of the number of segments in the table. (This doesn't make truncation of the relation by a different backend any more or less dangerous; we're still hosed if we don't get a notification before we try to do something with the relation. I think that is fixed, and if it's not this doesn't make it worse.) Any objections? regards, tom lane
В списке pgsql-hackers по дате отправления: