GROUP BY / ORDER BY string is very slow
От | Oleg Broytmann |
---|---|
Тема | GROUP BY / ORDER BY string is very slow |
Дата | |
Msg-id | Pine.SOL2.3.96.SK.990121140402.12992A-100000@sun.med.ru обсуждение исходный текст |
Список | pgsql-hackers |
Hello! I use a modified version of contrib/apache_logginig. The table I am using: CREATE TABLE combinedlog ( host text, accdate abstime, request text, authuser text, cookie text, referer text, useragenttext, stime int2, status int2, bytes int4 ); Once a month I run a very simple script to put WWW logs into the table. The very loading is not fast (I am running postmaster with -F, I use BEGIN/END and I drop indicies before loading. I remember when I started using BEGIN/END loading speed up a bit, but not significantly), but is not my biggest concern. What is worse is spped of my queries. After inserting into the table, I run this shell script: sel_f() { field=$1 psql -d ce_wwwlog -c "SELECT COUNT($field), $field FROM raw_combinedlog GROUP BY 2 ORDER BY 1 DESC;" } for i in host request referer useragent; do sel_f $i > by-$i done This works very, very slow. I tried to use indicies: CREATE INDEX host ON combinedlog (host); CREATE INDEX request ON combinedlog (request); CREATE INDEX referer ON combinedlog (referer); CREATE INDEX useragent ON combinedlog (useragent); but indicies do not help much, and indexing time is so big, that sum of CREATE INDEX + SELECT is even bigger :( Why is it so slow? How can I speed it up? I am running postgres compiled with --enable-locale. that is, for every string comparision there are 2 (two) malloc calls and one strcoll. Can I increase speed turning strcoll off? If so, postgres need a SET command to turn localization temporary off. I can hack in, as I already rewrote localization stuff a year ago. The only thing I want to hear from postgres community (hackers, actually :) is how it should be named: SET STRCOLL=off or such? I remember when I submitted my locale patch there was a discussion on how to do it the Right Way, but I didn't remember the conlusion. What finally we decided? I want to add command (if I should to) that is compliant with other stuff here. Oleg. ---- Oleg Broytmann http://members.tripod.com/~phd2/ phd2@earthling.net Programmers don't die, they justGOSUB without RETURN.
В списке pgsql-hackers по дате отправления: