Re: Doing psql's lexing with flex

Tom Lane wrote:
> I got interested enough in the psql-with-flex problem to go off and
> solve it.  Attached is a working patch, which I'm now debating whether
> to apply.  Comments solicited...
>
> The patch removes about 200 lines of very spaghetti-ish code in
> mainloop.c.  However, it adds an 875-line flex source file, which
> might be thought a bad tradeoff :-(.  One bright spot is that about
> half of that total is a direct copy of the main backend lexer, so
> it's not really as much new, separately maintainable code as all that.
> Also, Andrew Dunstan's patch for supporting dollar-quoting would add
> about 100 lines to mainloop.c, versus only a dozen or so lines in the
> flex implementation.  Once that's taken into account I don't think there
> is a lot of difference in effective SLOC to maintain.  I'm also of the
> opinion that the new C code in psqlscan.l is much more straightforward
> than the code removed from mainloop.c, though having just written it,
> I'm no doubt pretty biased.
>
> Bruce was asking about speed.  On normal-size queries I cannot measure
> any difference at all.  For testing purposes I made up a file containing
> a single 750K query (just a "SELECT big-honking-string-constant", with
> the string literal broken into lines of 75 bytes).  The client-side
> (psql) CPU time to run this file looks about like this on my machine:
>
>               PGCLIENTENCODING
>             UNICODE        SJIS
>
> CVS tip            1.57        1.82
>
> flex implementation    0.93        2.33

Looks good.  I withdraw my performance concerns.  Thanks for testing.

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073