Re: [GENERAL] Postgres, apps, special characters and UTF-8 encoding

On 03/07/2017 03:20 PM, Ken Tanzer wrote:
> Hi.  I've got a recurring problem with character encoding for a
> Postgres-based web PHP app, and am hoping someone can clue me in or at
> least point me in the right direction.  I'll confess upfront my
> understanding of encoding issues is extremely limited.  Here goes.
>
> The app uses a Postgres database, UTF-8 encoded.  Through their
> browsers, users can add and edit records often including text.  Most of
> the time this works fine.  Though sometimes this will fail with Postgres
> complaining, for example, "Could query with ... , The error text was:
> ERROR: invalid byte sequence for encoding "UTF8": 0xe9 0x20 0x67"
>
> So this generally happens when people copy and paste things out of their
> word documents and such.
>
> As I understand it, those are likely encoded in something non-UTF-8,
> like WIN-1251 or something.  And that one way or another, the encoding
> needs to be translated before it can be placed into the database.  I'm
> not clear how this is supposed to happen though.  Automatically by the
> browser?  Done in the app?  Some other way?  And if in the app, how is
> one supposed to know what the incoming encoding is?

I don't use PHP, but found this:

http://www.php.net/manual/en/function.mb-detect-encoding.php

and this:

http://php.net/manual/en/function.mb-convert-encoding.php


>
> Thanks in advance for any help or pointers.
>
> Ken
>
>
> --
> AGENCY Software
> A Free Software data system
> By and for non-profits
> /http://agency-software.org//
> /https://agency-software.org/demo/client/
> ken.tanzer@agency-software.org <mailto:ken.tanzer@agency-software.org>
> (253) 245-3801
>
> Subscribe to the mailing list
> <mailto:agency-general-request@lists.sourceforge.net?body=subscribe> to
> learn more about AGENCY or
> follow the discussion.


--
Adrian Klaver
adrian.klaver@aklaver.com