Re: libpq and Binary Data Formats

Wilhansen Li wrote:
> Basically, better support for binary formats which includes, but not 
> limited
> to:
> 1) functions for converting to and from various datatypes
> 2) reducing the need to convert to and from network byte order
> 3) better documentation
> 
> My suggestion on using ASN.1 was merely a naive suggestion on how in can be
> implemented properly without breaking (future) compatibility because that
> seems to be the main problem which prevents the use of binary formats.

Well, it sounds to me like this is two separate items: (1+2), (3).

For (3) there is the pgsql-docs mailing list. If you have 
additions/changes, that's the place you want. Submissions in text are 
fine, you don't need to worry about SGML formatting, but do discuss them 
first. The documentation relies on people saying "I don't think this bit 
is clear", so help is always welcome.

For (1+2) it sounds like what you actually want is a "native binary for 
my application" protocol rather than "internal binary format" which is 
sort of what's available now. Clearly "application binary" is an 
addition rather than a replacement (unless everyone using binary 
transfers thinks it's so much better they're happy to switch immediately).

A few obvious questions leap out at me:
1. What languages are you seeking to target: just "C"?
2. What platforms are you seeking to target: intel 32 bit? 64 bit? 
powerpc? arm?
3. How much do I gain (and lose) over text transfer, and under what 
circumstances?
4. What will happen with custom/user-defined types? Will they need their 
own "adaptor" written to support this?

Crucially, I think you want to demonstrate #3 - that there's a clear 
gain for all the work that's involved in defining a separate transfer 
encoding. If you can demonstrate the gains are felt by all the 
Perl/PHP/Java applications too that'd obviously help.

Bear in mind I'm just another user of PostgreSQL, not a developer, so 
you could do everything I've said and still not interest core in making 
changes. However, I've seen a lot of changes come and go and I think 
you'll need to make progress on those 4 points to get anywhere.

--   Richard Huxton  Archonet Ltd