Обсуждение: Make pgoutput documentation easier to find
Hi, The current documentation for pgoutput is buried in the logical streaming replication protocol section (in protocol.sgml), and there's no index entry for it. This makes it hard to discover and access, for example, when trying to look up the options it supports. I've often struggled to locate this information myself, so I'd like to propose moving the pgoutput documentation to the logical decoding section and adding an index entry. The attached patch does that. I think this change will make it much easier for users to find the relevant details. Thoughts? Regards, -- Fujii Masao
Вложения
On 03.08.25 03:32, Fujii Masao wrote: > The current documentation for pgoutput is buried in the logical streaming > replication protocol section (in protocol.sgml), and there's no index entry > for it. This makes it hard to discover and access, for example, when trying > to look up the options it supports. > > I've often struggled to locate this information myself, so I'd like to > propose moving the pgoutput documentation to the logical decoding section > and adding an index entry. The attached patch does that. I think this change > will make it much easier for users to find the relevant details. This would move the documentation of pgoutput from "Internals" to "Server Programming". So it's a question of whether this is something we want to advertise that people can use directly. In the past, pgoutput was an implementation detail of logical replication. But I gather people are using it for other things now? Still not clear what kind of guarantees we want to give about its interfaces, for example.
On Wed, Aug 6, 2025 at 8:36 PM Peter Eisentraut <peter@eisentraut.org> wrote: > > On 03.08.25 03:32, Fujii Masao wrote: > > The current documentation for pgoutput is buried in the logical streaming > > replication protocol section (in protocol.sgml), and there's no index entry > > for it. This makes it hard to discover and access, for example, when trying > > to look up the options it supports. > > > > I've often struggled to locate this information myself, so I'd like to > > propose moving the pgoutput documentation to the logical decoding section > > and adding an index entry. The attached patch does that. I think this change > > will make it much easier for users to find the relevant details. > > This would move the documentation of pgoutput from "Internals" to > "Server Programming". So it's a question of whether this is something > we want to advertise that people can use directly. In the past, > pgoutput was an implementation detail of logical replication. But I > gather people are using it for other things now? I've heard that Debezium users, a tool for change data capture, can use pgoutput as the logical decoding plugin. I also know users, including some of my colleagues, who use pgoutput with pg_recvlogical to capture messages inserted via pg_logical_emit_message(). So I think making the pgoutput documentation easier to find would be helpful for those users. Regards, -- Fujii Masao
On Wed, Aug 6, 2025, at 10:48 AM, Fujii Masao wrote: > On Wed, Aug 6, 2025 at 8:36 PM Peter Eisentraut <peter@eisentraut.org> wrote: >> >> On 03.08.25 03:32, Fujii Masao wrote: >> > The current documentation for pgoutput is buried in the logical streaming >> > replication protocol section (in protocol.sgml), and there's no index entry >> > for it. This makes it hard to discover and access, for example, when trying >> > to look up the options it supports. >> > >> > I've often struggled to locate this information myself, so I'd like to >> > propose moving the pgoutput documentation to the logical decoding section >> > and adding an index entry. The attached patch does that. I think this change >> > will make it much easier for users to find the relevant details. >> >> This would move the documentation of pgoutput from "Internals" to >> "Server Programming". So it's a question of whether this is something >> we want to advertise that people can use directly. In the past, >> pgoutput was an implementation detail of logical replication. But I >> gather people are using it for other things now? > > I've heard that Debezium users, a tool for change data capture, can use > pgoutput as the logical decoding plugin. I also know users, including > some of my colleagues, who use pgoutput with pg_recvlogical to capture > messages inserted via pg_logical_emit_message(). > pgoutput is our defacto output plugin and I think we consider it a mature piece of code. If you are worried about compatibility, version 18 added the pg_get_loaded_modules() function to find the loadable module version -- see commit 55527368bd07. There might be cases where you need to debug logical replication (conflicts?) and you need to pass the exact output plugin options via SQL (pg_logical_slot_*_changes) to obtain the changes. If we keep it in the "Internals" chapter, it seems it is something to PostgreSQL developers (as the chapter description says); it is not. Advanced users can make good use of this. The new links will help users to find the moved information if you are searching it into the "Internals" section. Since you are proposing this change, I'm wondering if we shouldn't move "Writing Logical Decoding Output Plugins" to "Internals" chapter. It contains information to PostgreSQL developers. It is a bit off-topic for this thread but we can also move part of the "Background Worker Processes", part of the "Archive Modules" and part of the "OAuth Validator Modules" section (2 subsections that describe the API) into "Internals" chapter. -- Euler Taveira EDB https://www.enterprisedb.com/
On Thu, Aug 7, 2025 at 10:35 PM Euler Taveira <euler@eulerto.com> wrote: > Since you are proposing this change, I'm wondering if we shouldn't move > "Writing Logical Decoding Output Plugins" to "Internals" chapter. It contains > information to PostgreSQL developers. Maybe, but I don’t have a strong opinion. If we decide to do it, it would be better to make the change in a separate patch. Regards, -- Fujii Masao
Dear Fujii-san, > The current documentation for pgoutput is buried in the logical streaming > replication protocol section (in protocol.sgml), and there's no index entry > for it. This makes it hard to discover and access, for example, when trying > to look up the options it supports. > > I've often struggled to locate this information myself, so I'd like to > propose moving the pgoutput documentation to the logical decoding section > and adding an index entry. The attached patch does that. I think this change > will make it much easier for users to find the relevant details. Personally considered pgoutput is not the user-facing and extension developers can dig by themselves, but this is not a huge complaint. DBA may check their settings via pg_recvlogical, which is a use-case. I did agree to add new chapter for plugin developers, but it could be done at different thread. Since pgoutput can only generate the binary output, it pg_logical_slot_get_changes() cannot be used with this plugin. I felt this should be also documented. Another point: test_decoding does not have descriptions for their options neither. Should we also add as well? Best regards, Hayato Kuroda FUJITSU LIMITED
On Thu, Aug 14, 2025 at 9:12 PM Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com> wrote: > Since pgoutput can only generate the binary output, it pg_logical_slot_get_changes() > cannot be used with this plugin. I felt this should be also documented. OK, I added this note in the 0002 patch attached earlier. > Another point: test_decoding does not have descriptions for their options neither. > Should we also add as well? Yes, I think it's worth doing this (but it's better to do as a separate patch). TBH this was already on my TODO list. Regards, -- Fujii Masao