DBMirror.pl performance change
От | Achilleus Mantzios |
---|---|
Тема | DBMirror.pl performance change |
Дата | |
Msg-id | Pine.LNX.4.44.0601231327390.18872-200000@matrix.gatewaynet.com обсуждение исходный текст |
Ответы |
Re: DBMirror.pl performance change
|
Список | pgsql-general |
I discovered a problem in DBMirror.pl, performance wise. pending.c stores data in a way very similar to the PgSQL input "\" escaped format. When the field is of type bytea, and the source of data is binary, then this produces 2 additional backslashes for every unprintable char. The performance in function extractData in DBMirror.pl, really suffers from this condition, since it breaks data in chunks of "\" delimited strings. Informally speaking, performance tends to be O(n) where n is the size of the data. This can be remedied if we break data in chunks of "'" rather than "\". "'" happens much more infrequently in common binary files (bz2, tiff, jpg, pdf etc..), and if we notice that odd number of contained "\", signals an intermidiate "'", whereas even number of "\" signals the final "'", then we can make this routine run much faster. I attach the new extractData function. Now replicating a 400 k tiff takes 3 seconds instead of 12 minutes it used to do. I am wondering about the state of http://www.whitebeam.org/library/guide/TechNotes/replicate.rhtm Please feel free for any comments. Pete could you test this new DBMirror.pl, to see how it behaves in comparison with your C++ solution? -- -Achilleus
Вложения
В списке pgsql-general по дате отправления: