Make pgoutput documentation easier to find
Hi,
The current documentation for pgoutput is buried in the logical streaming
replication protocol section (in protocol.sgml), and there's no index entry
for it. This makes it hard to discover and access, for example, when trying
to look up the options it supports.
I've often struggled to locate this information myself, so I'd like to
propose moving the pgoutput documentation to the logical decoding section
and adding an index entry. The attached patch does that. I think this change
will make it much easier for users to find the relevant details.
Thoughts?
Regards,
--
Fujii Masao
Attachments:
v1-0001-doc-Improve-documentation-discoverability-for-pgo.patchapplication/octet-stream; name=v1-0001-doc-Improve-documentation-discoverability-for-pgo.patchDownload
From 6a0b1c15260a1089d250a4b4d1c148832b5d3c35 Mon Sep 17 00:00:00 2001
From: Fujii Masao <fujii@postgresql.org>
Date: Sun, 3 Aug 2025 10:14:59 +0900
Subject: [PATCH v1] doc: Improve documentation discoverability for pgoutput.
Previously, the documentation for pgoutput was located in the section on
the logical streaming replication protocol, and there was no index entry
for it. As a result, users had difficulty finding information about pgoutput.
This commit moves the pgoutput documentation under the logical decoding
section and adds an index entry, making it easier for users to locate and
access this information.
---
doc/src/sgml/logical-replication.sgml | 8 +-
doc/src/sgml/logicaldecoding.sgml | 146 ++++++++++++++++++++++++++
doc/src/sgml/protocol.sgml | 131 ++---------------------
3 files changed, 156 insertions(+), 129 deletions(-)
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index fcac55aefe6..7de4a59747d 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -542,8 +542,8 @@
it manually before the subscription can be activated. The steps to create
the slot and activate the subscription are shown in the following examples.
These examples specify the standard logical decoding output plugin
- (<literal>pgoutput</literal>), which is what the built-in logical
- replication uses.
+ (<xref linkend="logicaldecoding-pgoutput"/>),
+ which is what the built-in logical replication uses.
</para>
<para>
First, create a publication for the examples to use.
@@ -2157,8 +2157,8 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
implemented by <literal>walsender</literal> and <literal>apply</literal>
processes. The walsender process starts logical decoding (described
in <xref linkend="logicaldecoding"/>) of the WAL and loads the standard
- logical decoding output plugin (<literal>pgoutput</literal>). The plugin
- transforms the changes read
+ logical decoding output plugin (<xref linkend="logicaldecoding-pgoutput"/>).
+ The plugin transforms the changes read
from WAL to the logical replication protocol
(see <xref linkend="protocol-logical-replication"/>) and filters the data
according to the publication specification. The data is then continuously
diff --git a/doc/src/sgml/logicaldecoding.sgml b/doc/src/sgml/logicaldecoding.sgml
index 593f784b69d..4516650983d 100644
--- a/doc/src/sgml/logicaldecoding.sgml
+++ b/doc/src/sgml/logicaldecoding.sgml
@@ -574,6 +574,152 @@ postgres=# select * from pg_logical_slot_get_changes('regression_slot', NULL, NU
<sect1 id="logicaldecoding-output-plugin">
<title>Logical Decoding Output Plugins</title>
+
+ <para>
+ <productname>PostgreSQL</productname> provides two logical decoding
+ output plugins, <xref linkend="logicaldecoding-pgoutput"/> and
+ <xref linkend="test-decoding"/>. You can also write custom output plugins
+ (see <xref linkend="logicaldecoding-output-plugin-writing"/> for details).
+ </para>
+
+ <sect2 id="logicaldecoding-pgoutput" xreflabel="pgoutput">
+ <title>pgoutput — Standard Logical Decoding Output Plugin</title>
+
+ <indexterm zone="logicaldecoding-pgoutput">
+ <primary>pgoutput</primary>
+ </indexterm>
+
+ <para>
+ <filename>pgoutput</filename> is the standard logical decoding output
+ plugin provided by <productname>PostgreSQL</productname>.
+ It's used for the built-in <link linkend="logical-replication">
+ logical replication</link>.
+ </para>
+
+ <sect3 id="logicaldecoding-pgoutput-options">
+ <title>Options</title>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ proto_version
+ </term>
+ <listitem>
+ <para>
+ Protocol version. Currently versions <literal>1</literal>, <literal>2</literal>,
+ <literal>3</literal>, and <literal>4</literal> are supported. A valid
+ version is required.
+ </para>
+ <para>
+ Version <literal>2</literal> is supported only for server version 14
+ and above, and it allows streaming of large in-progress transactions.
+ </para>
+ <para>
+ Version <literal>3</literal> is supported only for server version 15
+ and above, and it allows streaming of two-phase commits.
+ </para>
+ <para>
+ Version <literal>4</literal> is supported only for server version 16
+ and above, and it allows streams of large in-progress transactions to
+ be applied in parallel.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ publication_names
+ </term>
+ <listitem>
+ <para>
+ Comma-separated list of publication names for which to subscribe
+ (receive changes). The individual publication names are treated
+ as standard objects names and can be quoted the same as needed.
+ At least one publication name is required.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ binary
+ </term>
+ <listitem>
+ <para>
+ Boolean option to use binary transfer mode. Binary mode is faster
+ than the text mode but slightly less robust.
+ The default is <literal>off</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ messages
+ </term>
+ <listitem>
+ <para>
+ Boolean option to enable sending the messages that are written
+ by <function>pg_logical_emit_message</function>.
+ The default is <literal>off</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ streaming
+ </term>
+ <listitem>
+ <para>
+ Option to enable streaming of in-progress transactions. Valid values are
+ <literal>off</literal> (the default), <literal>on</literal> and
+ <literal>parallel</literal>. The setting <literal>parallel</literal>
+ enables sending extra information with some messages to be used for
+ parallelization. Minimum protocol version 2 is required to turn it
+ <literal>on</literal>. Minimum protocol version 4 is required for the
+ <literal>parallel</literal> value.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ two_phase
+ </term>
+ <listitem>
+ <para>
+ Boolean option to enable two-phase transactions. Minimum protocol
+ version 3 is required to turn it on.
+ The default is <literal>off</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ origin
+ </term>
+ <listitem>
+ <para>
+ Option to send changes by their origin. Possible values are
+ <literal>none</literal> to only send the changes that have no origin
+ associated, or <literal>any</literal>
+ to send the changes regardless of their origin. This can be used
+ to avoid loops (infinite replication of the same data) among
+ replication nodes.
+ The default is <literal>any</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+ </sect3>
+ </sect2>
+ </sect1>
+
+ <sect1 id="logicaldecoding-output-plugin-writing">
+ <title>Writing Logical Decoding Output Plugins</title>
<para>
An example output plugin can be found in the
<link linkend="test-decoding">
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index e56eac8fd0f..9fe48c5acad 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -2955,7 +2955,7 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;"
<listitem>
<para>
The name of an option passed to the slot's logical decoding output
- plugin. See <xref linkend="protocol-logical-replication"/> for
+ plugin. See <xref linkend="logicaldecoding-pgoutput-options"/> for
options that are accepted by the standard (<literal>pgoutput</literal>)
plugin.
</para>
@@ -3523,134 +3523,15 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;"
the physical streaming replication protocol.
</para>
- <para>
- <productname>PostgreSQL</productname> logical decoding supports output
- plugins. <literal>pgoutput</literal> is the standard one used for
- the built-in logical replication.
- </para>
-
<sect2 id="protocol-logical-replication-params">
<title>Logical Streaming Replication Parameters</title>
<para>
- Using the <literal>START_REPLICATION</literal> command,
- <literal>pgoutput</literal> accepts the following options:
-
- <variablelist>
- <varlistentry>
- <term>
- proto_version
- </term>
- <listitem>
- <para>
- Protocol version. Currently versions <literal>1</literal>, <literal>2</literal>,
- <literal>3</literal>, and <literal>4</literal> are supported. A valid
- version is required.
- </para>
- <para>
- Version <literal>2</literal> is supported only for server version 14
- and above, and it allows streaming of large in-progress transactions.
- </para>
- <para>
- Version <literal>3</literal> is supported only for server version 15
- and above, and it allows streaming of two-phase commits.
- </para>
- <para>
- Version <literal>4</literal> is supported only for server version 16
- and above, and it allows streams of large in-progress transactions to
- be applied in parallel.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term>
- publication_names
- </term>
- <listitem>
- <para>
- Comma-separated list of publication names for which to subscribe
- (receive changes). The individual publication names are treated
- as standard objects names and can be quoted the same as needed.
- At least one publication name is required.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term>
- binary
- </term>
- <listitem>
- <para>
- Boolean option to use binary transfer mode. Binary mode is faster
- than the text mode but slightly less robust.
- The default is <literal>off</literal>.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term>
- messages
- </term>
- <listitem>
- <para>
- Boolean option to enable sending the messages that are written
- by <function>pg_logical_emit_message</function>.
- The default is <literal>off</literal>.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term>
- streaming
- </term>
- <listitem>
- <para>
- Option to enable streaming of in-progress transactions. Valid values are
- <literal>off</literal> (the default), <literal>on</literal> and
- <literal>parallel</literal>. The setting <literal>parallel</literal>
- enables sending extra information with some messages to be used for
- parallelization. Minimum protocol version 2 is required to turn it
- <literal>on</literal>. Minimum protocol version 4 is required for the
- <literal>parallel</literal> value.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term>
- two_phase
- </term>
- <listitem>
- <para>
- Boolean option to enable two-phase transactions. Minimum protocol
- version 3 is required to turn it on.
- The default is <literal>off</literal>.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term>
- origin
- </term>
- <listitem>
- <para>
- Option to send changes by their origin. Possible values are
- <literal>none</literal> to only send the changes that have no origin
- associated, or <literal>any</literal>
- to send the changes regardless of their origin. This can be used
- to avoid loops (infinite replication of the same data) among
- replication nodes.
- The default is <literal>any</literal>.
- </para>
- </listitem>
- </varlistentry>
- </variablelist>
-
+ The <literal>START_REPLICATION</literal> command allows us to
+ pass options to the logical decoding output plugin associated with
+ the replication slot specified in the command.
+ See <xref linkend="logicaldecoding-pgoutput-options"/> for options
+ that are accepted by the standard (<literal>pgoutput</literal>) plugin.
</para>
</sect2>
--
2.50.1
On 03.08.25 03:32, Fujii Masao wrote:
The current documentation for pgoutput is buried in the logical streaming
replication protocol section (in protocol.sgml), and there's no index entry
for it. This makes it hard to discover and access, for example, when trying
to look up the options it supports.I've often struggled to locate this information myself, so I'd like to
propose moving the pgoutput documentation to the logical decoding section
and adding an index entry. The attached patch does that. I think this change
will make it much easier for users to find the relevant details.
This would move the documentation of pgoutput from "Internals" to
"Server Programming". So it's a question of whether this is something
we want to advertise that people can use directly. In the past,
pgoutput was an implementation detail of logical replication. But I
gather people are using it for other things now? Still not clear what
kind of guarantees we want to give about its interfaces, for example.
On Wed, Aug 6, 2025 at 8:36 PM Peter Eisentraut <peter@eisentraut.org> wrote:
On 03.08.25 03:32, Fujii Masao wrote:
The current documentation for pgoutput is buried in the logical streaming
replication protocol section (in protocol.sgml), and there's no index entry
for it. This makes it hard to discover and access, for example, when trying
to look up the options it supports.I've often struggled to locate this information myself, so I'd like to
propose moving the pgoutput documentation to the logical decoding section
and adding an index entry. The attached patch does that. I think this change
will make it much easier for users to find the relevant details.This would move the documentation of pgoutput from "Internals" to
"Server Programming". So it's a question of whether this is something
we want to advertise that people can use directly. In the past,
pgoutput was an implementation detail of logical replication. But I
gather people are using it for other things now?
I've heard that Debezium users, a tool for change data capture, can use
pgoutput as the logical decoding plugin. I also know users, including
some of my colleagues, who use pgoutput with pg_recvlogical to capture
messages inserted via pg_logical_emit_message().
So I think making the pgoutput documentation easier to find would be
helpful for those users.
Regards,
--
Fujii Masao
On Wed, Aug 6, 2025, at 10:48 AM, Fujii Masao wrote:
On Wed, Aug 6, 2025 at 8:36 PM Peter Eisentraut <peter@eisentraut.org> wrote:
On 03.08.25 03:32, Fujii Masao wrote:
The current documentation for pgoutput is buried in the logical streaming
replication protocol section (in protocol.sgml), and there's no index entry
for it. This makes it hard to discover and access, for example, when trying
to look up the options it supports.I've often struggled to locate this information myself, so I'd like to
propose moving the pgoutput documentation to the logical decoding section
and adding an index entry. The attached patch does that. I think this change
will make it much easier for users to find the relevant details.This would move the documentation of pgoutput from "Internals" to
"Server Programming". So it's a question of whether this is something
we want to advertise that people can use directly. In the past,
pgoutput was an implementation detail of logical replication. But I
gather people are using it for other things now?I've heard that Debezium users, a tool for change data capture, can use
pgoutput as the logical decoding plugin. I also know users, including
some of my colleagues, who use pgoutput with pg_recvlogical to capture
messages inserted via pg_logical_emit_message().
pgoutput is our defacto output plugin and I think we consider it a mature piece
of code. If you are worried about compatibility, version 18 added the
pg_get_loaded_modules() function to find the loadable module version -- see
commit 55527368bd07.
There might be cases where you need to debug logical replication (conflicts?)
and you need to pass the exact output plugin options via SQL
(pg_logical_slot_*_changes) to obtain the changes.
If we keep it in the "Internals" chapter, it seems it is something to
PostgreSQL developers (as the chapter description says); it is not. Advanced
users can make good use of this. The new links will help users to find the
moved information if you are searching it into the "Internals" section.
Since you are proposing this change, I'm wondering if we shouldn't move
"Writing Logical Decoding Output Plugins" to "Internals" chapter. It contains
information to PostgreSQL developers. It is a bit off-topic for this thread but
we can also move part of the "Background Worker Processes", part of the
"Archive Modules" and part of the "OAuth Validator Modules" section (2
subsections that describe the API) into "Internals" chapter.
--
Euler Taveira
EDB https://www.enterprisedb.com/
On Thu, Aug 7, 2025 at 10:35 PM Euler Taveira <euler@eulerto.com> wrote:
Since you are proposing this change, I'm wondering if we shouldn't move
"Writing Logical Decoding Output Plugins" to "Internals" chapter. It contains
information to PostgreSQL developers.
Maybe, but I don’t have a strong opinion. If we decide to do it, it would
be better to make the change in a separate patch.
Regards,
--
Fujii Masao
I applied the batch and built the html docs. I tested that all hyperlinks worked for me.
There are a few nit comments about wording:
diff --git a/doc/src/sgml/logicaldecoding.sgml b/doc/src/sgml/logicaldecoding.sgml
index 593f784b69d..4516650983d 100644
--- a/doc/src/sgml/logicaldecoding.sgml
+++ b/doc/src/sgml/logicaldecoding.sgml+ <xref linkend="test-decoding"/>. You can also write custom output plugins
nit: "write" -> "develop"
+ Protocol version. Currently versions <literal>1</literal>, <literal>2</literal>,
"Protocol version." -> "Specifies the protocol version."
+ Comma-separated list of publication names for which to subscribe
+ (receive changes). The individual publication names are treatedA comma-separated list of publication names to subscribe to. The individual publication names are treated
+ Boolean option to use binary transfer mode. Binary mode is faster
nit: "use" -> "enable", as other places all use "enable"
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index e56eac8fd0f..9fe48c5acad 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml+ The <literal>START_REPLICATION</literal> command allows us to
+ pass options to the logical decoding output plugin associated with
+ the replication slot specified in the command.
+ See <xref linkend="logicaldecoding-pgoutput-options"/> for options
+ that are accepted by the standard (<literal>pgoutput</literal>) plugin.nit: The START_REPLICATION command can pass options to the logical decoding output plugin associated with the specified replication slot. See Section 47.6.1.1 for the options supported by the standard (pgoutput) plugin.
The new status of this patch is: Waiting on Author
Dear Fujii-san,
The current documentation for pgoutput is buried in the logical streaming
replication protocol section (in protocol.sgml), and there's no index entry
for it. This makes it hard to discover and access, for example, when trying
to look up the options it supports.I've often struggled to locate this information myself, so I'd like to
propose moving the pgoutput documentation to the logical decoding section
and adding an index entry. The attached patch does that. I think this change
will make it much easier for users to find the relevant details.
Personally considered pgoutput is not the user-facing and extension developers
can dig by themselves, but this is not a huge complaint.
DBA may check their settings via pg_recvlogical, which is a use-case.
I did agree to add new chapter for plugin developers, but it could be done at
different thread.
Since pgoutput can only generate the binary output, it pg_logical_slot_get_changes()
cannot be used with this plugin. I felt this should be also documented.
Another point: test_decoding does not have descriptions for their options neither.
Should we also add as well?
Best regards,
Hayato Kuroda
FUJITSU LIMITED
On Thu, Aug 14, 2025 at 1:05 PM Chao Li <li.evan.chao@gmail.com> wrote:
I applied the batch and built the html docs. I tested that all hyperlinks worked for me.
Thanks for the review and testing!
+ <xref linkend="test-decoding"/>. You can also write custom output plugins
nit: "write" -> "develop"
OK, updated. I've attached the updated version of the patch (0001).
+ Protocol version. Currently versions <literal>1</literal>, <literal>2</literal>,
"Protocol version." -> "Specifies the protocol version."
+ Comma-separated list of publication names for which to subscribe + (receive changes). The individual publication names are treatedA comma-separated list of publication names to subscribe to. The individual publication names are treated
+ Boolean option to use binary transfer mode. Binary mode is faster
nit: "use" -> "enable", as other places all use "enable"
The 0001 patch mainly moves the pgoutput docs to a more visible location.
On the other hand, it looks like you're also suggesting further improvements
to the docs. I agree this is worth doing, but I think it should be a separate
patch. So I've created 0002, which updates the docs based on your comments
and also improves several other descriptions.
+ The <literal>START_REPLICATION</literal> command allows us to + pass options to the logical decoding output plugin associated with + the replication slot specified in the command. + See <xref linkend="logicaldecoding-pgoutput-options"/> for options + that are accepted by the standard (<literal>pgoutput</literal>) plugin.nit: The START_REPLICATION command can pass options to the logical decoding output plugin associated with the specified replication slot. See Section 47.6.1.1 for the options supported by the standard (pgoutput) plugin.
OK, updated (0001 patch).
Regards,
--
Fujii Masao
Attachments:
v2-0002-doc-Improve-pgoutput-docs.patchapplication/octet-stream; name=v2-0002-doc-Improve-pgoutput-docs.patchDownload
From 74af8c56578cb737d6afa1ceaa70e2bd4a48d58e Mon Sep 17 00:00:00 2001
From: Fujii Masao <fujii@postgresql.org>
Date: Thu, 14 Aug 2025 23:33:52 +0900
Subject: [PATCH v2 2/2] doc: Improve pgoutput docs.
---
doc/src/sgml/func/func-admin.sgml | 2 +-
doc/src/sgml/logicaldecoding.sgml | 118 +++++++++++++++++-------------
2 files changed, 69 insertions(+), 51 deletions(-)
diff --git a/doc/src/sgml/func/func-admin.sgml b/doc/src/sgml/func/func-admin.sgml
index 446fdfe56f4..6347fe60b0c 100644
--- a/doc/src/sgml/func/func-admin.sgml
+++ b/doc/src/sgml/func/func-admin.sgml
@@ -1224,7 +1224,7 @@ postgres=# SELECT '0/0'::pg_lsn + pd.segment_number * ps.setting::int + :offset
</row>
<row>
- <entry role="func_table_entry"><para role="func_signature">
+ <entry id="pg-logical-slot-peek-binary-changes" role="func_table_entry"><para role="func_signature">
<indexterm>
<primary>pg_logical_slot_peek_binary_changes</primary>
</indexterm>
diff --git a/doc/src/sgml/logicaldecoding.sgml b/doc/src/sgml/logicaldecoding.sgml
index 1e3ccc84e65..a1f2efb2420 100644
--- a/doc/src/sgml/logicaldecoding.sgml
+++ b/doc/src/sgml/logicaldecoding.sgml
@@ -600,109 +600,112 @@ DETAIL: Synchronization could lead to data loss, because the remote slot needs
<title>Options</title>
<variablelist>
- <varlistentry>
- <term>
- proto_version
- </term>
+ <varlistentry id="pgoutput-options-proto-version">
+ <term><varname>proto_version</varname> (<type>integer</type>)</term>
<listitem>
<para>
- Protocol version. Currently versions <literal>1</literal>, <literal>2</literal>,
+ Specifies the protocol version.
+ Currently versions <literal>1</literal>, <literal>2</literal>,
<literal>3</literal>, and <literal>4</literal> are supported. A valid
version is required.
</para>
<para>
- Version <literal>2</literal> is supported only for server version 14
- and above, and it allows streaming of large in-progress transactions.
+ Version <literal>2</literal> is supported on server version 14
+ and above, and is required when <varname>streaming</varname>
+ is set to <literal>on</literal> to stream large in-progress
+ transactions.
</para>
<para>
- Version <literal>3</literal> is supported only for server version 15
- and above, and it allows streaming of two-phase commits.
+ Version <literal>3</literal> is supported on server version 15
+ and above, and is required when <varname>two_phase</varname>
+ is enabled to stream two-phase commits.
</para>
<para>
- Version <literal>4</literal> is supported only for server version 16
- and above, and it allows streams of large in-progress transactions to
- be applied in parallel.
+ Version <literal>4</literal> is supported on server version 16
+ and above, and is required when <varname>streaming</varname>
+ is set to <literal>parallel</literal> to stream large in-progress
+ transactions to be applied in parallel.
</para>
</listitem>
</varlistentry>
- <varlistentry>
- <term>
- publication_names
- </term>
+ <varlistentry id="pgoutput-options-publication-names">
+ <term><varname>publication_names</varname> (<type>string</type>)</term>
<listitem>
<para>
- Comma-separated list of publication names for which to subscribe
- (receive changes). The individual publication names are treated
+ A comma-separated list of publication names to subscribe to.
+ The individual publication names are treated
as standard objects names and can be quoted the same as needed.
At least one publication name is required.
</para>
</listitem>
</varlistentry>
- <varlistentry>
- <term>
- binary
- </term>
+ <varlistentry id="pgoutput-options-binary">
+ <term><varname>binary</varname> (<type>boolean</type>)</term>
<listitem>
<para>
- Boolean option to use binary transfer mode. Binary mode is faster
+ Enables binary transfer mode. Binary mode is faster
than the text mode but slightly less robust.
The default is <literal>off</literal>.
</para>
</listitem>
</varlistentry>
- <varlistentry>
- <term>
- messages
- </term>
+ <varlistentry id="pgoutput-options-messages">
+ <term><varname>messages</varname> (<type>boolean</type>)</term>
<listitem>
<para>
- Boolean option to enable sending the messages that are written
- by <function>pg_logical_emit_message</function>.
+ Enables sending the messages that are written by
+ <function><link linkend="pg-logical-emit-message">pg_logical_emit_message</link></function>.
The default is <literal>off</literal>.
</para>
</listitem>
</varlistentry>
- <varlistentry>
- <term>
- streaming
- </term>
+ <varlistentry id="pgoutput-options-streaming">
+ <term><varname>streaming</varname> (<type>enum</type>)</term>
<listitem>
<para>
- Option to enable streaming of in-progress transactions. Valid values are
+ Enables streaming of in-progress transactions. Valid values are
<literal>off</literal> (the default), <literal>on</literal> and
- <literal>parallel</literal>. The setting <literal>parallel</literal>
- enables sending extra information with some messages to be used for
- parallelization. Minimum protocol version 2 is required to turn it
- <literal>on</literal>. Minimum protocol version 4 is required for the
- <literal>parallel</literal> value.
+ <literal>parallel</literal>.
+ </para>
+ <para>
+ When set to <literal>off</literal>, <filename>pgoutput</filename>
+ fully decodes a transaction before sending it as a whole.
+ This mode works with any protocol version.
+ </para>
+ <para>
+ When set to <literal>on</literal>, <filename>pgoutput</filename>
+ streams large in-progress transactions.
+ This requires protocol version 2 or higher.
+ </para>
+ <para>
+ When set to <literal>parallel</literal>, <filename>pgoutput</filename>
+ streams large in-progress transactions and also sends
+ extra information in some messages to support parallel processing.
+ This requires protocol version 4 or higher.
</para>
</listitem>
</varlistentry>
- <varlistentry>
- <term>
- two_phase
- </term>
+ <varlistentry id="pgoutput-options-two-phase">
+ <term><varname>two_phase</varname> (<type>boolean</type>)</term>
<listitem>
<para>
- Boolean option to enable two-phase transactions. Minimum protocol
- version 3 is required to turn it on.
+ Enables sending two-phase transactions.
+ Minimum protocol version 3 is required to turn it on.
The default is <literal>off</literal>.
</para>
</listitem>
</varlistentry>
- <varlistentry>
- <term>
- origin
- </term>
+ <varlistentry id="pgoutput-options-origin">
+ <term><varname>origin</varname> (<type>enum</type>)</term>
<listitem>
<para>
- Option to send changes by their origin. Possible values are
+ Specifies whether to send changes by their origin. Possible values are
<literal>none</literal> to only send the changes that have no origin
associated, or <literal>any</literal>
to send the changes regardless of their origin. This can be used
@@ -715,6 +718,21 @@ DETAIL: Synchronization could lead to data loss, because the remote slot needs
</variablelist>
</sect3>
+
+ <sect3 id="logicaldecoding-pgoutput-notes">
+ <title>Notes</title>
+
+ <para>
+ <filename>pgoutput</filename> produces binary output,
+ so functions expecting textual data (
+ <function><link linkend="pg-logical-slot-peek-changes">pg_logical_slot_peek_changes</link></function> and
+ <function><link linkend="pg-logical-slot-get-changes">pg_logical_slot_get_changes</link></function>)
+ cannot be used with it. Use
+ <function><link linkend="pg-logical-slot-peek-binary-changes">pg_logical_slot_peek_binary_changes</link></function> or
+ <function><link linkend="pg-logical-slot-get-binary-changes">pg_logical_slot_get_binary_changes</link></function>
+ instead.
+ </para>
+ </sect3>
</sect2>
</sect1>
--
2.50.1
v2-0001-doc-Improve-documentation-discoverability-for-pgo.patchapplication/octet-stream; name=v2-0001-doc-Improve-documentation-discoverability-for-pgo.patchDownload
From d79c67530d9df817dc29f3c6d91b15b111e680a2 Mon Sep 17 00:00:00 2001
From: Fujii Masao <fujii@postgresql.org>
Date: Thu, 14 Aug 2025 13:20:05 +0900
Subject: [PATCH v2 1/2] doc: Improve documentation discoverability for
pgoutput.
Previously, the documentation for pgoutput was located in the section on
the logical streaming replication protocol, and there was no index entry
for it. As a result, users had difficulty finding information about pgoutput.
This commit moves the pgoutput documentation under the logical decoding
section and adds an index entry, making it easier for users to locate and
access this information.
Discussion: https://postgr.es/m/CAHGQGwFJTbygdhhjR_iP4Oem=Lo1xsptWWOq825uoW+hG_Lfnw@mail.gmail.com
---
doc/src/sgml/logical-replication.sgml | 8 +-
doc/src/sgml/logicaldecoding.sgml | 146 ++++++++++++++++++++++++++
doc/src/sgml/protocol.sgml | 131 ++---------------------
3 files changed, 156 insertions(+), 129 deletions(-)
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index a0761cfee3f..0ac29928f17 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -542,8 +542,8 @@
it manually before the subscription can be activated. The steps to create
the slot and activate the subscription are shown in the following examples.
These examples specify the standard logical decoding output plugin
- (<literal>pgoutput</literal>), which is what the built-in logical
- replication uses.
+ (<xref linkend="logicaldecoding-pgoutput"/>),
+ which is what the built-in logical replication uses.
</para>
<para>
First, create a publication for the examples to use.
@@ -2173,8 +2173,8 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
implemented by <literal>walsender</literal> and <literal>apply</literal>
processes. The walsender process starts logical decoding (described
in <xref linkend="logicaldecoding"/>) of the WAL and loads the standard
- logical decoding output plugin (<literal>pgoutput</literal>). The plugin
- transforms the changes read
+ logical decoding output plugin (<xref linkend="logicaldecoding-pgoutput"/>).
+ The plugin transforms the changes read
from WAL to the logical replication protocol
(see <xref linkend="protocol-logical-replication"/>) and filters the data
according to the publication specification. The data is then continuously
diff --git a/doc/src/sgml/logicaldecoding.sgml b/doc/src/sgml/logicaldecoding.sgml
index 77c720c422c..1e3ccc84e65 100644
--- a/doc/src/sgml/logicaldecoding.sgml
+++ b/doc/src/sgml/logicaldecoding.sgml
@@ -574,6 +574,152 @@ DETAIL: Synchronization could lead to data loss, because the remote slot needs
<sect1 id="logicaldecoding-output-plugin">
<title>Logical Decoding Output Plugins</title>
+
+ <para>
+ <productname>PostgreSQL</productname> provides two logical decoding
+ output plugins, <xref linkend="logicaldecoding-pgoutput"/> and
+ <xref linkend="test-decoding"/>. You can also develop custom output plugins
+ (see <xref linkend="logicaldecoding-output-plugin-writing"/> for details).
+ </para>
+
+ <sect2 id="logicaldecoding-pgoutput" xreflabel="pgoutput">
+ <title>pgoutput — Standard Logical Decoding Output Plugin</title>
+
+ <indexterm zone="logicaldecoding-pgoutput">
+ <primary>pgoutput</primary>
+ </indexterm>
+
+ <para>
+ <filename>pgoutput</filename> is the standard logical decoding output
+ plugin provided by <productname>PostgreSQL</productname>.
+ It's used for the built-in <link linkend="logical-replication">
+ logical replication</link>.
+ </para>
+
+ <sect3 id="logicaldecoding-pgoutput-options">
+ <title>Options</title>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ proto_version
+ </term>
+ <listitem>
+ <para>
+ Protocol version. Currently versions <literal>1</literal>, <literal>2</literal>,
+ <literal>3</literal>, and <literal>4</literal> are supported. A valid
+ version is required.
+ </para>
+ <para>
+ Version <literal>2</literal> is supported only for server version 14
+ and above, and it allows streaming of large in-progress transactions.
+ </para>
+ <para>
+ Version <literal>3</literal> is supported only for server version 15
+ and above, and it allows streaming of two-phase commits.
+ </para>
+ <para>
+ Version <literal>4</literal> is supported only for server version 16
+ and above, and it allows streams of large in-progress transactions to
+ be applied in parallel.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ publication_names
+ </term>
+ <listitem>
+ <para>
+ Comma-separated list of publication names for which to subscribe
+ (receive changes). The individual publication names are treated
+ as standard objects names and can be quoted the same as needed.
+ At least one publication name is required.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ binary
+ </term>
+ <listitem>
+ <para>
+ Boolean option to use binary transfer mode. Binary mode is faster
+ than the text mode but slightly less robust.
+ The default is <literal>off</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ messages
+ </term>
+ <listitem>
+ <para>
+ Boolean option to enable sending the messages that are written
+ by <function>pg_logical_emit_message</function>.
+ The default is <literal>off</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ streaming
+ </term>
+ <listitem>
+ <para>
+ Option to enable streaming of in-progress transactions. Valid values are
+ <literal>off</literal> (the default), <literal>on</literal> and
+ <literal>parallel</literal>. The setting <literal>parallel</literal>
+ enables sending extra information with some messages to be used for
+ parallelization. Minimum protocol version 2 is required to turn it
+ <literal>on</literal>. Minimum protocol version 4 is required for the
+ <literal>parallel</literal> value.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ two_phase
+ </term>
+ <listitem>
+ <para>
+ Boolean option to enable two-phase transactions. Minimum protocol
+ version 3 is required to turn it on.
+ The default is <literal>off</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ origin
+ </term>
+ <listitem>
+ <para>
+ Option to send changes by their origin. Possible values are
+ <literal>none</literal> to only send the changes that have no origin
+ associated, or <literal>any</literal>
+ to send the changes regardless of their origin. This can be used
+ to avoid loops (infinite replication of the same data) among
+ replication nodes.
+ The default is <literal>any</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+ </sect3>
+ </sect2>
+ </sect1>
+
+ <sect1 id="logicaldecoding-output-plugin-writing">
+ <title>Writing Logical Decoding Output Plugins</title>
<para>
An example output plugin can be found in the
<link linkend="test-decoding">
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index cc5c8dc574c..e63647c093b 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -2955,7 +2955,7 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;"
<listitem>
<para>
The name of an option passed to the slot's logical decoding output
- plugin. See <xref linkend="protocol-logical-replication"/> for
+ plugin. See <xref linkend="logicaldecoding-pgoutput-options"/> for
options that are accepted by the standard (<literal>pgoutput</literal>)
plugin.
</para>
@@ -3523,134 +3523,15 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;"
the physical streaming replication protocol.
</para>
- <para>
- <productname>PostgreSQL</productname> logical decoding supports output
- plugins. <literal>pgoutput</literal> is the standard one used for
- the built-in logical replication.
- </para>
-
<sect2 id="protocol-logical-replication-params">
<title>Logical Streaming Replication Parameters</title>
<para>
- Using the <literal>START_REPLICATION</literal> command,
- <literal>pgoutput</literal> accepts the following options:
-
- <variablelist>
- <varlistentry>
- <term>
- proto_version
- </term>
- <listitem>
- <para>
- Protocol version. Currently versions <literal>1</literal>, <literal>2</literal>,
- <literal>3</literal>, and <literal>4</literal> are supported. A valid
- version is required.
- </para>
- <para>
- Version <literal>2</literal> is supported only for server version 14
- and above, and it allows streaming of large in-progress transactions.
- </para>
- <para>
- Version <literal>3</literal> is supported only for server version 15
- and above, and it allows streaming of two-phase commits.
- </para>
- <para>
- Version <literal>4</literal> is supported only for server version 16
- and above, and it allows streams of large in-progress transactions to
- be applied in parallel.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term>
- publication_names
- </term>
- <listitem>
- <para>
- Comma-separated list of publication names for which to subscribe
- (receive changes). The individual publication names are treated
- as standard objects names and can be quoted the same as needed.
- At least one publication name is required.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term>
- binary
- </term>
- <listitem>
- <para>
- Boolean option to use binary transfer mode. Binary mode is faster
- than the text mode but slightly less robust.
- The default is <literal>off</literal>.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term>
- messages
- </term>
- <listitem>
- <para>
- Boolean option to enable sending the messages that are written
- by <function>pg_logical_emit_message</function>.
- The default is <literal>off</literal>.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term>
- streaming
- </term>
- <listitem>
- <para>
- Option to enable streaming of in-progress transactions. Valid values are
- <literal>off</literal> (the default), <literal>on</literal> and
- <literal>parallel</literal>. The setting <literal>parallel</literal>
- enables sending extra information with some messages to be used for
- parallelization. Minimum protocol version 2 is required to turn it
- <literal>on</literal>. Minimum protocol version 4 is required for the
- <literal>parallel</literal> value.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term>
- two_phase
- </term>
- <listitem>
- <para>
- Boolean option to enable two-phase transactions. Minimum protocol
- version 3 is required to turn it on.
- The default is <literal>off</literal>.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term>
- origin
- </term>
- <listitem>
- <para>
- Option to send changes by their origin. Possible values are
- <literal>none</literal> to only send the changes that have no origin
- associated, or <literal>any</literal>
- to send the changes regardless of their origin. This can be used
- to avoid loops (infinite replication of the same data) among
- replication nodes.
- The default is <literal>any</literal>.
- </para>
- </listitem>
- </varlistentry>
- </variablelist>
-
+ The <literal>START_REPLICATION</literal> command can pass
+ options to the logical decoding output plugin associated
+ with the specified replication slot.
+ See <xref linkend="logicaldecoding-pgoutput-options"/> for options
+ that are accepted by the standard (<literal>pgoutput</literal>) plugin.
</para>
</sect2>
--
2.50.1
On Thu, Aug 14, 2025 at 9:12 PM Hayato Kuroda (Fujitsu)
<kuroda.hayato@fujitsu.com> wrote:
Since pgoutput can only generate the binary output, it pg_logical_slot_get_changes()
cannot be used with this plugin. I felt this should be also documented.
OK, I added this note in the 0002 patch attached earlier.
Another point: test_decoding does not have descriptions for their options neither.
Should we also add as well?
Yes, I think it's worth doing this (but it's better to do as a separate patch).
TBH this was already on my TODO list.
Regards,
--
Fujii Masao
The following review has been posted through the commitfest application:
make installcheck-world: not tested
Implements feature: not tested
Spec compliant: not tested
Documentation: tested, passed
Thanks for addressing my comments. I downloaded the new patch files and tested again. Now this patch looks good to me.
BTW, in the pushed commit, when you add me as a reviewer, would you please use my company email address: "Chao Li <lic@highgo.com>". Thanks a lot.
The new status of this patch is: Ready for Committer
Dear Fujii-san,
Thanks for updating the patch. Your patches look good to me.
Best regards,
Hayato Kuroda
FUJITSU LIMITED
On Fri, Aug 15, 2025 at 1:46 PM Hayato Kuroda (Fujitsu)
<kuroda.hayato@fujitsu.com> wrote:
Dear Fujii-san,
Thanks for updating the patch. Your patches look good to me.
Thanks for the review! Unless there are objections, I'll commit these
two patches.
Regards,
--
Fujii Masao
On Mon, Aug 18, 2025 at 11:55 AM Fujii Masao <masao.fujii@gmail.com> wrote:
On Fri, Aug 15, 2025 at 1:46 PM Hayato Kuroda (Fujitsu)
<kuroda.hayato@fujitsu.com> wrote:Dear Fujii-san,
Thanks for updating the patch. Your patches look good to me.
Thanks for the review! Unless there are objections, I'll commit these
two patches.
I've pushed the patches. Thanks!
Regards,
--
Fujii Masao