Re: Options OUTPUT_PLUGIN_* controlling format are confusing (Was: Misleading error message in logical decoding)

On 2014-09-23 12:40:25 +0900, Michael Paquier wrote:
> On Tue, Sep 23, 2014 at 4:59 AM, Robert Haas <robertmhaas@gmail.com> wrote:
> > On Thu, Sep 18, 2014 at 11:21 AM, Michael Paquier
> > <michael.paquier@gmail.com> wrote:
> >> On Thu, Sep 18, 2014 at 9:56 AM, Andres Freund <andres@2ndquadrant.com> wrote:
> >>> On 2014-09-18 09:50:38 -0500, Michael Paquier wrote:
> >>>> > Do you see the difference between what your doc patch states and the
> >>>> > explanation I've given nearby in this thread?
> >>>> Perhaps that's the lack of documentation...
> >>>
> >>> Man. I've explained it to you about three times. The previous attempts
> >>> at doing so didn't seem to help. If my explanations don't explain it so
> >>> you can understand it adding them to the docs won't change a thing.
> >>> That's why I ask whether you see the difference?
> >> Urg sorry for the misunderstanding. The patch stated that this
> >> parameter only influences the output of the SQL functions while it
> >> defines if "the output plugin requires the output method to support
> >> binary data"?
> >
> > I'm not sure exactly what that sentence means.
> >
> > But here's the point: every series of bytes is a valid bytea, except
> > maybe if it's over 1GB and runs afould of MaxAllocSize.  But a series
> > of bytes is only a valid text datum if it's a valid sequence of
> > characters according to the database encoding.  We like to think of
> > text as being an arbitrary series of bytes, but it isn't.  It can't
> > contain any \0 bytes, and it can't contain anything that's invalid in
> > the database encoding.  bytea isn't subject to either of those
> > restrictions.
> >
> > So if we were going to have one universal output format for output
> > plugins, it would have to be bytea because that, really, can be
> > anything.  We could make users convert from that to text or whatever
> > they like.  But that's unappealing for several reasons: bytea output
> > is printed as unreadable hexademical garbage, and encoding conversions
> > are expensive.  So what we do instead is provide a text output method
> > and a binary output method.  That way, plugins that want to return
> > binary data are free to do so, and output methods that are happy to
> > return text can *declare* that what they return is legal text - and
> > then we just assume that to be true, and need not do an encoding
> > conversion.
> Aha, thanks. That's all clear then!

What about the attached patch then?

Greetings,

Andres Freund

--
 Andres Freund                       http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services