Re: pg should ignore u+200b zero width space

Heikki Linnakangas <hlinnaka@iki.fi> writes:
> On 03/11/2020 15:41, James Cloos wrote:
>> pg should treat a no break space after whitespace as just more
>> whitespace.

> Hmm. I'm not sure if change the behavior is a good idea, but a hint in 
> the error message would be nice. Something like:

The difficulty with doing anything in this space --- whether it be
ignoring, throwing an error, or whatever --- is that it makes the
lexer's behavior encoding-sensitive and potentially locale-sensitive.
That's problematic for all sorts of reasons.  One of the worst is
that frontend programs such as psql and ecpg also have SQL lexers,
and there'd be no way to keep their behavior in precise sync with
the backend's.  (They might not even be running in the same encoding,
never mind locale.)  It might even be possible to build security
holes around that; recall the fun we've had with trying to lock
down quoting rules in encodings where backslash can be part of a
multibyte character :-(.

Perhaps it'd be all right to confine the change in behavior to
just modifying the error text in cases where we were going to
throw an error anyway.  But I think this is much harder than
it sounds to do in a valid, safe way.

            regards, tom lane