Re: [HACKERS] Built-in plugin for logical decoding output

On 23/09/17 00:28, Gregory Brail wrote:
> We have been working on a project that makes extensive use of logical 
> replication for use inside Apigee (which is a very small part of Google):
>
> https://github.com/apigee-labs/transicator
>
> In order to do this, we had to write our own logical replication 
> plugin because the supplied "test_decoding" plugin from the "contrib" 
> directory was hard for us to work with. Primarily:
>
> 1) It doesn't include all the fields that we need for Transicator 
> (most importantly we need the LSN and the 64-bit transaction ID).
> 2) It outputs a text format that is hard to parse.
>
> I imagine that other users of logical decoding are facing similar 
> problems.
>
> Would the community support the development of another plugin that is 
> distributed as part of "contrib" that addresses these issues? I'd be 
> happy to submit a patch, or GitHub repo, or whatever works best as an 
> example. (Also, although Transicator uses protobuf, I'm happy to have 
> it output a simple binary format as well.)
>
> As a side note, doing this would also help making logical decoding a 
> useful feature for customers of Amazon and Google's built-in Postgres 
> hosting options. In those environments, there's no way to add a custom 
> plugin to Postgres, so anything not built in the product tends to be 
> harder for someone to consume.
>
> If anyone is interested in looking more:
>
> The plugin code is here:
> https://github.com/apigee-labs/transicator/tree/master/pgoutput
>
> and produces output defined by the "ChangePb" structure defined here:
> https://github.com/apigee-labs/transicator/blob/master/common/transicator.proto
    How about using pgoutput, which is included by default in 
PostgreSQL 10, as the basis for logical replication?

    Cheers,
    Álvaro


-- 

Alvaro Hernandez


-----------
OnGres



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers