Re: scanner/parser minimization

On 13.03.2013 10:50, Simon Riggs wrote:
> On 2 March 2013 18:47, Heikki Linnakangas<hlinnakangas@vmware.com>  wrote:
>> Interestingly, the yy_transition array generated by flex used to be much
>> smaller:
>>
>> 8.3: 22072 elements
>> 8.4: 62623 elements
>> master: 64535 elements
>>
>> The big jump between 8.3 and 8.4 was caused by introduction of the unicode
>> escapes: U&'foo' [UESCAPE 'x'] . And in particular, the "error rule" for the
>> UESCAPE, which we use to avoid backtracking.
>>
>> I experimented with a patch that uses two extra flex states to shorten the
>> error rules, see attached. The idea is that after lexing a unicode literal
>> like "U&'foo'", you enter a new state, in which you check whether an
>> "UESCAPE 'x'" follows. This slashes the size of the array to 36581 elements.
>
> +1 to do this sooner rather than later

I hear no objection, so committed. (after fixing some small bugs in the 
patch, and adding some comments)

- Heikki