Обсуждение: Base64 decode/encode performance
Hi, I have been experimenting with some base64 encoding/decoding implementation. I find out that the one at http://code.google.com/p/stringencoders is the best obtaining a 1.3 speedup vs the postgres one. Do you think is worth to submit a patch that replaces the postgres base64 implementation with this one? Regards Gaetano Mendola
On 9/10/08, Gaetano Mendola <mendola@gmail.com> wrote: > I have been experimenting with some base64 encoding/decoding implementation. > > I find out that the one at http://code.google.com/p/stringencoders is the best > obtaining a 1.3 speedup vs the postgres one. > > Do you think is worth to submit a patch that replaces the postgres base64 implementation > with this one? (Note: the b64encode there reads 3 chars at a time, b64decode int32 at a time.) There are 2 killer problems: - decode does not seem to handle architectures that segfault on unaligned int32 accesses. - decode does not allow whitespace in input string. If those are fixed it's question of if the 1.3x speed if worth more complex code with big lookup tables. If you want to optimize, it seems more worthwhile to add additional loop to current code that reads 3 or 4 chars at a time, before the current single-char loop. The decode loop may thus even optimize to int32 fetching on x86/64 with reasonable compiler. Handling whitespace with such code is doable, but will the code be clear enough? -- marko
Marko Kreen wrote: > (Note: the b64encode there reads 3 chars at a time, b64decode int32 > at a time.) > > There are 2 killer problems: > > - decode does not seem to handle architectures that segfault > on unaligned int32 accesses. Out of curiosity - does this problem exist on any platform for which PostgreSQL is currently ported and supported? Cheers, mark -- Mark Mielke <mark@mielke.cc>
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, Sep 10, 2008 at 10:44:00AM -0400, Mark Mielke wrote: > Marko Kreen wrote: [...] >> - decode does not seem to handle architectures that segfault >> on unaligned int32 accesses. > > Out of curiosity - does this problem exist on any platform for which > PostgreSQL is currently ported and supported? HP PA is one of them. Besides, some others hide that behind a painful (more than a factor of 1.3 -- we are talking software emulation here) performance hit. More modeern architectures (Alpha, Itanium, IA-64) take this route. Regards - -- tomás -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFIx+ggBcgs9XrR2kYRAha9AJ0Xy6Zg/m76H2H4Uzta3pSXJh/D2gCfS8PF vpDQMU8gg2BahURgSI97GSk= =KnIm -----END PGP SIGNATURE-----
On Wed, Sep 10, 2008 at 10:44:00AM -0400, Mark Mielke wrote: > >There are 2 killer problems: > > > >- decode does not seem to handle architectures that segfault > > on unaligned int32 accesses. > > Out of curiosity - does this problem exist on any platform for which > PostgreSQL is currently ported and supported? It exists on most CPUs actually like Alpha/Sparc/MIPS, just not on Intel chips, which is why you don't see them very often. Unaligned accesses do take twice as long to execute though, even on Intel chips. On some OSes the unaligned access is trapped and emulated by the OS, which doesn't do much for performance. Have a nice day, -- Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/ > Please line up in a tree and maintain the heap invariant while > boarding. Thank you for flying nlogn airlines.