I have finished applying Mike Ansley's changes for long queries, along
with a bunch of my own. The current status is:
* You can send a query string of indefinite length to the backend. (This is poorly tested for MULTIBYTE, though; would
someonewho uses MULTIBYTE more than I do try it out?)
* You can get back an EXPLAIN or error message string of indefinite length.
* Single lexical tokens within a query are currently limited to 64k because of the lexer's use of YY_REJECT. I have
notcommitted any of Leon's proposed lexer changes, since that issue still seems controversial. I would like to see us
agreeon a solution. (ecpg's lexer has the same problem, of course.)
Although I think the backend is in fairly good shape, there are still
a few minor trouble spots. (The rule deparser will blow up at 8K for
example --- I have some work to do in there and will fix it when
I get a chance.)
In the frontend libraries and clients, both libpq and psql are length-
limit-free. I have not looked much at any of the other frontend
interface libraries. I suspect that at least odbc and the python
interface need work, because quick glimpse searches show suspicious-
looking
constants:MAX_QUERY_SIZEERROR_MSG_LENGTHSQL_PACKET_SIZEMAX_MESSAGE_LENTEXT_FIELD_SIZEMAX_VARCHAR_SIZEDRV_VARCHAR_SIZEDRV_LONGVARCHAR_SIZEMAX_BUFFER_SIZEMAX_FIELDS
The real problem in the clients is that pg_dump blithely assumes it
will never need to deal with strings over MAX_QUERY_SIZE. This is
a bad idea --- it ought to be rewritten to use the expansible-string-
buffer facility that now exists in libpq. There may be restrictions
in the other programs in bin/ as well, though glimpse didn't turn up
any red flags.
I would like to encourage the odbc and python folks to get rid of the
length limitations in their modules; I don't use either and have no
intention of touching either. I'd like to find a volunteer other than
myself to fix pg_dump, too.
Now, all we need is someone to implement multiple-disk-block tuples ;-)
regards, tom lane