Speeding up GIST index creation for tsvectors
От | Amit Khandekar |
---|---|
Тема | Speeding up GIST index creation for tsvectors |
Дата | |
Msg-id | CAJ3gD9eGvKkoZ5+3mkM9jmw1kUhKwQpMEgE-fo8uTSqHMnsmqg@mail.gmail.com обсуждение исходный текст |
Ответы |
Re: Speeding up GIST index creation for tsvectors
|
Список | pgsql-hackers |
Hi, In hemdistsign() of tsgistidx.c, if we process the bits in 64-bit chunks rather than byte-by-byte, we get an overall speed up in Gist index creation for tsvector types. With default siglen (124), the speed up is 12-20%. With siglen=700, it is 30-50%. So with longer signature lengths, we get higher percentage speed-up. The attached patch 0001 has the required changes. In the patch 0001, rather than using xor operator on char values, xor is operated on 64-bit chunks. And since the chunks are 64-bit, popcount64() is used on each of the chunks. I have checked that the two bitvector pointer arguments of hemdistsign() are not always 64-bit aligned. So process the leading mis-aligned bits and the trailing remainder bits char-by-char, leaving the middle 64-bit chunks for popcount64() usage. We might extend this to the hemdistsign() definitions at other places in the code. But for now, we can start with gist. I haven't tried other places. ------------- While working on this, I observed that on platforms other than x86_64, we still declare pg_popcount64() as a function pointer, even though we don't use the runtime selection of right function using__get_cpuid() as is done on x86. The other patch i.e. 0002 is a general optimization that avoids this function pointer for pg_popcount32/64() call. The patch arranges for direct function call so as to get rid of function pointer dereferencing each time pg_popcount32/64 is called. To do this, define pg_popcount64 to another function name (pg_popcount64_nonasm) rather than a function pointer, whenever USE_POPCNT_ASM is not defined. And let pg_popcount64_nonasm() be a static inline function so that whenever pg_popcount64() is called, directly the __builtin_popcount() gets called. For platforms not supporting __builtin_popcount(), continue using the slow version as is the current behaviour. Tested this 0002 patch on ARM64, with patch 0001 already applied, and the gist index creation for tsvectors *further* speeds up by 6% for default siglen (=124), and by 12% with siglen=700. ------------- Schema : CREATE TABLE test_tsvector(t text, a tsvector); -- Attached tsearch.data (a bigger version of -- src/test/regress/data/tsearch.data) \COPY test_tsvector FROM 'tsearch.data'; Test case that shows improvement : CREATE INDEX wowidx6 ON test_tsvector USING gist (a); Time taken by the above create-index command, in seconds, along with % speed-up w.r.t. HEAD : A) siglen=124 (Default) head 0001.patch 0001+0002.patch x86 .827 .737 (12%) ..... arm 1.098 .912 (20%) .861 (28%) B) siglen=700 (... USING gist (a tsvector_ops(siglen=700)) head 0001.patch 0001+0002.patch x86 1.121 .847 (32%) ..... arm 1.751 1.191 (47%) 1.062 (65%) -- Thanks, -Amit Khandekar Huawei Technologies
Вложения
В списке pgsql-hackers по дате отправления: