Re: Parser Cruft in gram.y

Dimitri Fontaine <dimitri@2ndQuadrant.fr> writes:
> Now, what about splitting those grammars in order to freely add any new
> production rules with or without new keywords for administrative
> commands, without a blink of though about the main parser grammar.

Let me explain to you why there will never be a situation where we can
consider new keywords to be zero-cost.

$ size src/backend/parser/gram.o    text    data     bss     dec     hex filename952864     104       0  952968   e8a88
src/backend/parser/gram.o
$ size src/backend/postgres       text    data     bss     dec     hex filename
6815102  123416  239356 7177874  6d8692 src/backend/postgres

That is, the grammar tables already amount to 14% of the total bulk of
the server executable.  (The above numbers exclude debug symbols BTW.)
That bloat is not free; for one thing, it's competing for L1 cache with
all the actual code in the backend.  And the main cause of it is that
we have lots-and-lots of keywords, because the parser tables are
basically number-of-tokens wide by number-of-states high.  (In HEAD
there are 433 tokens known to the grammar, all but 30 of which are
keywords, and 4367 states.)

Splitting the grammar into multiple grammars is unlikely to do much to
improve this --- in fact, it could easily make matters worse due to
duplication.  Rather, we just have to be careful about adding new
keywords.  In this connection, I quite like the fact that recent syntax
extensions such as EXPLAIN (...options...) have managed to avoid making
the option names into grammar keywords at all.
        regards, tom lane