Doc patch for Logical Replication Message Formats (PG14)
Hello Hackers,
while amending Npgsql to account for the Logical Streaming Replication
Protocol changes in PostgreSQL 14 I stumbled upon two documentation
inaccuracies in the Logical Replication Message Formats documentation
(https://www.postgresql.org/docs/devel/protocol-logicalrep-message-formats.html)
that have been introduced (or rather omitted) with the recent changes to
allow pgoutput to send logical decoding messages
(https://github.com/postgres/postgres/commit/ac4645c0157fc5fcef0af8ff571512aa284a2cec)
and to allow logical replication to transfer data in binary format
(https://github.com/postgres/postgres/commit/9de77b5453130242654ff0b30a551c9c862ed661).
1. The content of the logical decoding message in the 'Message' message
is prefixed with a length field (Int32) which isn't documented yet.
See
https://github.com/postgres/postgres/blob/69a58bfe4ab05567a8fab8bdce7f3095ed06b99c/src/backend/replication/logical/proto.c#L388
2. The TupleData may now contain the byte 'b' as indicator for binary
data which isn't documented yet. See
https://github.com/postgres/postgres/blob/69a58bfe4ab05567a8fab8bdce7f3095ed06b99c/src/include/replication/logicalproto.h#L83
and
https://github.com/postgres/postgres/blob/69a58bfe4ab05567a8fab8bdce7f3095ed06b99c/src/backend/replication/logical/proto.c#L558.
The attached documentation patch fixes both.
Best regards,
Brar
Attachments:
protocol-logicalrep-message-formats-pg14-fix.patchtext/plain; charset=UTF-8; name=protocol-logicalrep-message-formats-pg14-fix.patchDownload
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index bc2a2feb0b..53b3f1f651 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -6498,6 +6498,18 @@ Message
</para>
</listitem>
</varlistentry>
+
+<varlistentry>
+<term>
+ Int32
+</term>
+<listitem>
+<para>
+ Length of the content.
+</para>
+</listitem>
+</varlistentry>
+
<varlistentry>
<term>
Byte<replaceable>n</replaceable>
@@ -7430,6 +7442,19 @@ TupleData
</para>
</listitem>
</varlistentry>
+</variablelist>
+ Or
+<variablelist>
+<varlistentry>
+<term>
+ Byte1('b')
+</term>
+<listitem>
+<para>
+ Identifies the data as binary value.
+</para>
+</listitem>
+</varlistentry>
<varlistentry>
<term>
Int32
On Mon, Jun 21, 2021 at 12:26 PM Brar Piening <brar@gmx.de> wrote:
Hello Hackers,
while amending Npgsql to account for the Logical Streaming Replication
Protocol changes in PostgreSQL 14 I stumbled upon two documentation
inaccuracies in the Logical Replication Message Formats documentation
(https://www.postgresql.org/docs/devel/protocol-logicalrep-message-formats.html)
that have been introduced (or rather omitted) with the recent changes to
allow pgoutput to send logical decoding messages
(https://github.com/postgres/postgres/commit/ac4645c0157fc5fcef0af8ff571512aa284a2cec)
and to allow logical replication to transfer data in binary format
(https://github.com/postgres/postgres/commit/9de77b5453130242654ff0b30a551c9c862ed661).1. The content of the logical decoding message in the 'Message' message
is prefixed with a length field (Int32) which isn't documented yet.
See
https://github.com/postgres/postgres/blob/69a58bfe4ab05567a8fab8bdce7f3095ed06b99c/src/backend/replication/logical/proto.c#L388
2. The TupleData may now contain the byte 'b' as indicator for binary
data which isn't documented yet. See
https://github.com/postgres/postgres/blob/69a58bfe4ab05567a8fab8bdce7f3095ed06b99c/src/include/replication/logicalproto.h#L83
and
https://github.com/postgres/postgres/blob/69a58bfe4ab05567a8fab8bdce7f3095ed06b99c/src/backend/replication/logical/proto.c#L558.The attached documentation patch fixes both.
Yeah, I think these should be fixed and your patch looks good to me in
that regard.
--
With Regards,
Amit Kapila.
Amit Kapila wrote:
On Mon, Jun 21, 2021 at 12:26 PM Brar Piening <brar@gmx.de> wrote:
Hello Hackers,
while amending Npgsql to account for the Logical Streaming Replication
Protocol changes in PostgreSQL 14 I stumbled upon two documentation
inaccuracies in the Logical Replication Message Formats documentation
(https://www.postgresql.org/docs/devel/protocol-logicalrep-message-formats.html)
that have been introduced (or rather omitted) with the recent changes to
allow pgoutput to send logical decoding messages
(https://github.com/postgres/postgres/commit/ac4645c0157fc5fcef0af8ff571512aa284a2cec)
and to allow logical replication to transfer data in binary format
(https://github.com/postgres/postgres/commit/9de77b5453130242654ff0b30a551c9c862ed661).1. The content of the logical decoding message in the 'Message' message
is prefixed with a length field (Int32) which isn't documented yet.
See
https://github.com/postgres/postgres/blob/69a58bfe4ab05567a8fab8bdce7f3095ed06b99c/src/backend/replication/logical/proto.c#L388
2. The TupleData may now contain the byte 'b' as indicator for binary
data which isn't documented yet. See
https://github.com/postgres/postgres/blob/69a58bfe4ab05567a8fab8bdce7f3095ed06b99c/src/include/replication/logicalproto.h#L83
and
https://github.com/postgres/postgres/blob/69a58bfe4ab05567a8fab8bdce7f3095ed06b99c/src/backend/replication/logical/proto.c#L558.The attached documentation patch fixes both.
Yeah, I think these should be fixed and your patch looks good to me in
that regard.
After looking at the docs once again I have another minor amendment (new
patch attached).
Attachments:
protocol-logicalrep-message-formats-pg14-fix_v2.patchtext/plain; charset=UTF-8; name=protocol-logicalrep-message-formats-pg14-fix_v2.patchDownload
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index bc2a2feb0b..7d29308abc 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -6498,6 +6498,18 @@ Message
</para>
</listitem>
</varlistentry>
+
+<varlistentry>
+<term>
+ Int32
+</term>
+<listitem>
+<para>
+ Length of the content.
+</para>
+</listitem>
+</varlistentry>
+
<varlistentry>
<term>
Byte<replaceable>n</replaceable>
@@ -7430,6 +7442,19 @@ TupleData
</para>
</listitem>
</varlistentry>
+</variablelist>
+ Or
+<variablelist>
+<varlistentry>
+<term>
+ Byte1('b')
+</term>
+<listitem>
+<para>
+ Identifies the data as binary value.
+</para>
+</listitem>
+</varlistentry>
<varlistentry>
<term>
Int32
@@ -7446,8 +7471,8 @@ TupleData
</term>
<listitem>
<para>
- The value of the column, in text format. (A future release
- might support additional formats.)
+ The value of the column, eiter in binary or in text format.
+ (As specified in the preceding format byte).
<replaceable>n</replaceable> is the above length.
</para>
On Mon, Jun 21, 2021 at 4:13 PM Brar Piening <brar@gmx.de> wrote:
Amit Kapila wrote:
After looking at the docs once again I have another minor amendment (new
patch attached).
+ The value of the column, eiter in binary or in text format.
Typo. /eiter/either
--
With Regards,
Amit Kapila.
Amit Kapila schrieb:
On Mon, Jun 21, 2021 at 4:13 PM Brar Piening <brar@gmx.de> wrote:
Amit Kapila wrote:
After looking at the docs once again I have another minor amendment (new
patch attached).+ The value of the column, eiter in binary or in text format.
Typo. /eiter/either
Fixed - thanks!
Attachments:
protocol-logicalrep-message-formats-pg14-fix_v3.patchtext/plain; charset=UTF-8; name=protocol-logicalrep-message-formats-pg14-fix_v3.patchDownload
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index bc2a2feb0b..7d29308abc 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -6498,6 +6498,18 @@ Message
</para>
</listitem>
</varlistentry>
+
+<varlistentry>
+<term>
+ Int32
+</term>
+<listitem>
+<para>
+ Length of the content.
+</para>
+</listitem>
+</varlistentry>
+
<varlistentry>
<term>
Byte<replaceable>n</replaceable>
@@ -7430,6 +7442,19 @@ TupleData
</para>
</listitem>
</varlistentry>
+</variablelist>
+ Or
+<variablelist>
+<varlistentry>
+<term>
+ Byte1('b')
+</term>
+<listitem>
+<para>
+ Identifies the data as binary value.
+</para>
+</listitem>
+</varlistentry>
<varlistentry>
<term>
Int32
@@ -7446,8 +7471,8 @@ TupleData
</term>
<listitem>
<para>
- The value of the column, in text format. (A future release
- might support additional formats.)
+ The value of the column, either in binary or in text format.
+ (As specified in the preceding format byte).
<replaceable>n</replaceable> is the above length.
</para>
On Mon, Jun 21, 2021 at 4:41 PM Brar Piening <brar@gmx.de> wrote:
Amit Kapila schrieb:
On Mon, Jun 21, 2021 at 4:13 PM Brar Piening <brar@gmx.de> wrote:
Amit Kapila wrote:
After looking at the docs once again I have another minor amendment (new
patch attached).+ The value of the column, eiter in binary or in text format.
Typo. /eiter/either
Fixed - thanks!
Thanks for the report and patch. Pushed.
--
With Regards,
Amit Kapila.