Re: [PATCH] Generalized JSON output functions

On 07/13/2015 05:41 AM, Shulgin, Oleksandr wrote:
> On Mon, Jul 13, 2015 at 9:44 AM, Pavel Stehule 
> <pavel.stehule@gmail.com <mailto:pavel.stehule@gmail.com>> wrote:
>
>
>         To reiterate: for my problem, that is escaping numerics that
>         can potentially overflow[1] under ECMAScript standard, I want
>         to be able to override the code that outputs the numeric
>         converted to string.  There is no way in current
>         implementation to do that *at all*, short of copying all the
>         code involved in producing JSON output and changing it at
>         certain points.  One could try re-parsing JSON instead, but
>         that doesn't actually solve the issue, because type
>         information is lost forever at that point.
>
>         The whitespace unification was a mere side-effect of the
>         original effort on this patch.
>
>
>     The dynamic type change is some what I would not to do in
>     database, really :)
>
>     If you afraid about overflow, then convert numeric to string
>     immediately - in this case, the client have to support both
>     variant - so immediate cast should not be a problem.
>
>
> Yeah, but how would you do that in context of a logical replication 
> decoding plugin?  I've tried a number of tricks for that, including, 
> but not limited to registering phony types to wrap numeric type and 
> replacing the OID of numeric with this custom type OID in TupleDesc, 
> but then again one has to register that as known record type, etc.
>
>     Anyway this check on max number should be implemented in our
>     JSON(b) out functions (as warning?).
>
>
> Not really, since this is a problem of ECMAScript standard, not JSON 
> spec.  For example, Python module for handling JSON doesn't suffer 
> from this overflow problem,
>
> The thing is, we cannot know which clients are going to consume the 
> stream of decoded events, and if it's some implementation of 
> JavaScript, it can suffer silent data corruption if we don't guard 
> against that in the logical decoding plugin.
>
> Hope that makes it clear. :-)
>
>

Yes, but I think the plugin is the right place to do it. What is more, 
this won't actually prevent you completely from producing non-ECMAScript 
compliant JSON, since json or jsonb values containing offending numerics 
won't be caught, AIUI. But a fairly simple to write function that 
reparsed and fixed the JSON inside the decoder would work.

cheers

andrew