Re: Mac OS: invalid byte sequence for encoding "UTF8"

On 2016-02-10 17:00, Tom Lane wrote:
> Larry Rosenman <ler@lerctr.org> writes:
>> On 2016-02-10 16:19, Tom Lane wrote:
>>> I looked into the OS X sources, and found that indeed you are right:
>>> *scanf processes the input a byte at a time, and applies isspace() to
>>> each byte separately, even when the locale is such that that's a
>>> clearly insane thing to do.  Since this code was derived from 
>>> FreeBSD,
>>> FreeBSD has or once had the same issue.  (A look at the freebsd 
>>> project
>>> on github says it still does, assuming that's the authoritative 
>>> repo.)
>>> Not sure about other BSDen.
> 
>> Definitive FreeBSD Sources:
>> https://svnweb.freebsd.org/base/
> 
> Ah, thanks for the link.  I'm not totally sure which branch is most
> current, but at least on this one, it's still clearly wrong:
> https://svnweb.freebsd.org/base/stable/10/lib/libc/stdio/vfscanf.c?revision=291336&view=markup
> convert_string(), which handles %s, applies isspace() to individual 
> bytes
> regardless of locale.  convert_wstring(), which handles %ls, does it 
> more
> intelligently ... but as I said upthread, relying on %ls would just 
> give
> us a different set of portability problems.
> 
> It looks like Artur's patch is indeed what we need to do, along with
> looking around for other *scanf() uses that are vulnerable.
> 
>             regards, tom lane

that would be the current 10.x tree, production, and getting ready for 
10.3 which is in code slush.

If you want, file a bug at https://bugs.freebsd.org/bugzilla


-- 
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 214-642-9640                 E-Mail: ler@lerctr.org
US Mail: 7011 W Parmer Ln, Apt 1115, Austin, TX 78729-6961