missing documentation for streaming in-progress transactions

Started by Ajin Cherianalmost 5 years ago12 messages

itsajin@gmail.com

almost 5 years ago

1 attachment(s)

Hi,

Found that some documentation hasn't been updated for the changes made as
part of
streaming large in-progress transactions. Attached a patch that adds the
missing changes. Let me know if anything more needs to be added or any
comments on this change.

regards,
Ajin Cherian
Fujitsu Australia

Attachments:

v1-0001-Add-missing-documentation-of-streaming-in-progres.patchapplication/octet-stream; name=v1-0001-Add-missing-documentation-of-streaming-in-progres.patchDownload

From d4c6205a3a6749173c608c2393e5cc60aef561a2 Mon Sep 17 00:00:00 2001
From: Ajin Cherian <ajinc@fast.au.fujitsu.com>
Date: Wed, 7 Apr 2021 03:19:19 -0400
Subject: [PATCH v1] Add missing documentation of streaming in-progress
 transactions.

Updated documentation for new messages added for streaming in-progress
transactions, as well as changes made to the existing messages. Also updated
the protocl versions supported for logical replication.
---
 doc/src/sgml/protocol.sgml | 250 ++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 249 insertions(+), 1 deletion(-)

diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index 380be5f..2a93a6b 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -2797,7 +2797,7 @@ The commands accepted in replication mode are:
      </term>
      <listitem>
       <para>
-       Protocol version. Currently only version <literal>1</literal> is
+       Protocol version. Currently versions <literal>1</literal> and <literal>2</literal> are
        supported.
       </para>
      </listitem>
@@ -6647,6 +6647,16 @@ Relation
 </term>
 <listitem>
 <para>
+               Xid of the transaction (only present for streamed transactions).
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
                 ID of the relation.
 </para>
 </listitem>
@@ -6768,6 +6778,16 @@ Type
 </term>
 <listitem>
 <para>
+               Xid of the transaction (only present for streamed transactions).
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
                 ID of the data type.
 </para>
 </listitem>
@@ -6822,6 +6842,16 @@ Insert
 </term>
 <listitem>
 <para>
+                Xid of the transaction (only present for streamed transactions).
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
                 ID of the relation corresponding to the ID in the relation
                 message.
 </para>
@@ -6878,6 +6908,16 @@ Update
 </term>
 <listitem>
 <para>
+               Xid of the transaction (only present for streamed transactions).
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
                 ID of the relation corresponding to the ID in the relation
                 message.
 </para>
@@ -6981,6 +7021,16 @@ Delete
 </term>
 <listitem>
 <para>
+               Xid of the transaction (only present for streamed transactions).
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
                 ID of the relation corresponding to the ID in the relation
                 message.
 </para>
@@ -7059,6 +7109,16 @@ Truncate
 </term>
 <listitem>
 <para>
+               Xid of the transaction (only present for streamed transactions).
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
                 Number of relations
 </para>
 </listitem>
@@ -7091,6 +7151,194 @@ Truncate
 </listitem>
 </varlistentry>
 
+<varlistentry>
+<term>
+Stream Start
+</term>
+<listitem>
+<para>
+
+<variablelist>
+<varlistentry>
+<term>
+        Byte1('S')
+</term>
+<listitem>
+<para>
+                Identifies the message as a stream start message.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
+                Transaction id.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int8
+</term>
+<listitem>
+<para>
+                Flag to indicate if this is the first stream segment for this XID.
+                1 if first segment, 0 if not.
+</para>
+</listitem>
+</varlistentry>
+
+</variablelist>
+</para>
+</listitem>
+</varlistentry>
+
+<varlistentry>
+<term>
+Stream Stop
+</term>
+<listitem>
+<para>
+
+<variablelist>
+<varlistentry>
+<term>
+        Byte1('E')
+</term>
+<listitem>
+<para>
+                Identifies the message as a stream stop message.
+</para>
+</listitem>
+</varlistentry>
+
+</variablelist>
+</para>
+</listitem>
+</varlistentry>
+
+<varlistentry>
+<term>
+Stream Commit
+</term>
+<listitem>
+<para>
+
+<variablelist>
+<varlistentry>
+<term>
+        Byte1('c')
+</term>
+<listitem>
+<para>
+                Identifies the message as a stream commit message.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
+                Transaction id.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int8
+</term>
+<listitem>
+<para>
+                Flags; currently unused (must be 0).
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int64
+</term>
+<listitem>
+<para>
+                The LSN of the commit.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int64
+</term>
+<listitem>
+<para>
+                The end LSN of the transaction.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int64
+</term>
+<listitem>
+<para>
+                Commit timestamp of the transaction. The value is in number
+                of microseconds since PostgreSQL epoch (2000-01-01).
+</para>
+</listitem>
+</varlistentry>
+
+</variablelist>
+</para>
+</listitem>
+</varlistentry>
+
+<varlistentry>
+<term>
+Stream Abort
+</term>
+<listitem>
+<para>
+
+<variablelist>
+<varlistentry>
+<term>
+        Byte1('A')
+</term>
+<listitem>
+<para>
+                Identifies the message as a stream abort message.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
+                Transaction id.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
+                Sub-transaction id (will be same as transaction id for top-level transactions).
+</para>
+</listitem>
+</varlistentry>
+
+</variablelist>
+</para>
+</listitem>
+</varlistentry>
+
 </variablelist>
 
 <para>
-- 
1.8.3.1

Amit Kapila

amit.kapila16@gmail.com

almost 5 years ago

In reply to: Ajin Cherian (#1)

1 attachment(s)

Re: missing documentation for streaming in-progress transactions

On Wed, Apr 7, 2021 at 1:11 PM Ajin Cherian <itsajin@gmail.com> wrote:

Hi,

Found that some documentation hasn't been updated for the changes made as part of
streaming large in-progress transactions. Attached a patch that adds the missing changes. Let me know if anything more needs to be added or any comments on this change.

Thanks, this mostly looks good to me. I have slightly modified it.
See, what do you think of the attached?

--
With Regards,
Amit Kapila.

Attachments:

v2-0001-doc-Update-information-of-new-messages-for-logica.patchapplication/octet-stream; name=v2-0001-doc-Update-information-of-new-messages-for-logica.patchDownload

From ddc58cd3bca01722ea0b493812af4a0ea94119f7 Mon Sep 17 00:00:00 2001
From: Ajin Cherian <ajinc@fast.au.fujitsu.com>
Date: Wed, 7 Apr 2021 03:19:19 -0400
Subject: [PATCH v2] doc: Update information of new messages for logical
 replication.

Updated documentation for new messages added for streaming in-progress
transactions, as well as changes made to the existing messages. It also
updates the information of protocol versions supported for logical
replication.

Author: Ajin Cherian
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/CAFPTHDYHN9m=MZZct-B=BYg_TETvv+kXvL9RD2DpaBS5pGxGYg@mail.gmail.com
---
 doc/src/sgml/protocol.sgml | 265 ++++++++++++++++++++++++++++++++++++-
 1 file changed, 259 insertions(+), 6 deletions(-)

diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index 380be5fb17c..51a6eae0c8a 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -2797,9 +2797,11 @@ The commands accepted in replication mode are:
      </term>
      <listitem>
       <para>
-       Protocol version. Currently only version <literal>1</literal> is
-       supported.
-      </para>
+       Protocol version. Currently versions <literal>1</literal> and
+       <literal>2</literal> are supported. The version <literal>2</literal>
+       is supported only for server versions 14 and above, and is used to allow
+       streaming of large in-progress transactions.
+     </para>
      </listitem>
     </varlistentry>
 
@@ -2855,7 +2857,10 @@ The commands accepted in replication mode are:
   <para>
    The logical replication protocol sends individual transactions one by one.
    This means that all messages between a pair of Begin and Commit messages
-   belong to the same transaction.
+   belong to the same transaction. It also sends changes of large in-progress
+   transactions between a pair of Stream Start and Stream Stop messages. The
+   last stream of such a transaction contains Stream Commit or Stream Abort
+   message.
   </para>
 
   <para>
@@ -6457,8 +6462,7 @@ Message
 </term>
 <listitem>
 <para>
-                Xid of the transaction. The XID is sent only when user has
-                requested streaming of in-progress transactions.
+                Xid of the transaction (only present for streamed transactions).
 </para>
 </listitem>
 </varlistentry>
@@ -6646,6 +6650,16 @@ Relation
         Int32
 </term>
 <listitem>
+<para>
+               Xid of the transaction (only present for streamed transactions).
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
 <para>
                 ID of the relation.
 </para>
@@ -6767,6 +6781,16 @@ Type
         Int32
 </term>
 <listitem>
+<para>
+               Xid of the transaction (only present for streamed transactions).
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
 <para>
                 ID of the data type.
 </para>
@@ -6821,6 +6845,16 @@ Insert
         Int32
 </term>
 <listitem>
+<para>
+                Xid of the transaction (only present for streamed transactions).
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
 <para>
                 ID of the relation corresponding to the ID in the relation
                 message.
@@ -6877,6 +6911,16 @@ Update
         Int32
 </term>
 <listitem>
+<para>
+               Xid of the transaction (only present for streamed transactions).
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
 <para>
                 ID of the relation corresponding to the ID in the relation
                 message.
@@ -6980,6 +7024,16 @@ Delete
         Int32
 </term>
 <listitem>
+<para>
+               Xid of the transaction (only present for streamed transactions).
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
 <para>
                 ID of the relation corresponding to the ID in the relation
                 message.
@@ -7058,6 +7112,16 @@ Truncate
         Int32
 </term>
 <listitem>
+<para>
+               Xid of the transaction (only present for streamed transactions).
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
 <para>
                 Number of relations
 </para>
@@ -7091,6 +7155,195 @@ Truncate
 </listitem>
 </varlistentry>
 
+<varlistentry>
+<term>
+Stream Start
+</term>
+<listitem>
+<para>
+
+<variablelist>
+<varlistentry>
+<term>
+        Byte1('S')
+</term>
+<listitem>
+<para>
+                Identifies the message as a stream start message.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
+                Transaction id.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int8
+</term>
+<listitem>
+<para>
+                A value of 1 indicates this is the first stream segment for
+                this XID, 0 for any other stream segment.
+</para>
+</listitem>
+</varlistentry>
+
+</variablelist>
+</para>
+</listitem>
+</varlistentry>
+
+<varlistentry>
+<term>
+Stream Stop
+</term>
+<listitem>
+<para>
+
+<variablelist>
+<varlistentry>
+<term>
+        Byte1('E')
+</term>
+<listitem>
+<para>
+                Identifies the message as a stream stop message.
+</para>
+</listitem>
+</varlistentry>
+
+</variablelist>
+</para>
+</listitem>
+</varlistentry>
+
+<varlistentry>
+<term>
+Stream Commit
+</term>
+<listitem>
+<para>
+
+<variablelist>
+<varlistentry>
+<term>
+        Byte1('c')
+</term>
+<listitem>
+<para>
+                Identifies the message as a stream commit message.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
+                Transaction id.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int8
+</term>
+<listitem>
+<para>
+                Flags; currently unused (must be 0).
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int64
+</term>
+<listitem>
+<para>
+                The LSN of the commit.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int64
+</term>
+<listitem>
+<para>
+                The end LSN of the transaction.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int64
+</term>
+<listitem>
+<para>
+                Commit timestamp of the transaction. The value is in number
+                of microseconds since PostgreSQL epoch (2000-01-01).
+</para>
+</listitem>
+</varlistentry>
+
+</variablelist>
+</para>
+</listitem>
+</varlistentry>
+
+<varlistentry>
+<term>
+Stream Abort
+</term>
+<listitem>
+<para>
+
+<variablelist>
+<varlistentry>
+<term>
+        Byte1('A')
+</term>
+<listitem>
+<para>
+                Identifies the message as a stream abort message.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
+                Transaction id.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
+                Sub-transaction id (will be same as transaction id for top-level
+                transactions).
+</para>
+</listitem>
+</varlistentry>
+
+</variablelist>
+</para>
+</listitem>
+</varlistentry>
+
 </variablelist>
 
 <para>
-- 
2.28.0.windows.1

Peter Smith

smithpb2250@gmail.com

almost 5 years ago

In reply to: Amit Kapila (#2)

Re: missing documentation for streaming in-progress transactions

On Wed, Apr 7, 2021 at 10:15 PM Amit Kapila <amit.kapila16@gmail.com> wrote:

On Wed, Apr 7, 2021 at 1:11 PM Ajin Cherian <itsajin@gmail.com> wrote:

Hi,

Found that some documentation hasn't been updated for the changes made as part of
streaming large in-progress transactions. Attached a patch that adds the missing changes. Let me know if anything more needs to be added or any comments on this change.

Thanks, this mostly looks good to me. I have slightly modified it.
See, what do you think of the attached?

1.
I felt that this protocol documentation needs to include something
like a "Since: 2" notation (e.g. see how the javadoc API [1]https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/String.html does it)
otherwise with more message types and more protocol versions it is
quickly going to become too complicated to know what message or
message attribute belongs with what protocol.

2.
There are inconsistent terms used for a transaction id.
e.g.1 Sometimes called " Transaction id."
e.g.2 Sometimes called "Xid of the transaction"
I think there should be consistent terminology used on this page

3.
There is inconsistent wording for what seems to be the same condition.
e.g.1 The existing documentation [2]https://www.postgresql.org/docs/devel/protocol-logicalrep-message-formats.html says "Xid of the transaction. The
XID is sent only when user has requested streaming of in-progress
transactions."
e.g.2 For a similar case the patch says "Xid of the transaction (only
present for streamed transactions)."
I think there should be consistent wording used on this page where possible.

------
[1]: https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/String.html
[2]: https://www.postgresql.org/docs/devel/protocol-logicalrep-message-formats.html

Kind Regards,
Peter Smith.
Fujitsu Australia

Amit Kapila

amit.kapila16@gmail.com

almost 5 years ago

In reply to: Peter Smith (#3)

Re: missing documentation for streaming in-progress transactions

On Thu, Apr 8, 2021 at 3:49 AM Peter Smith <smithpb2250@gmail.com> wrote:

On Wed, Apr 7, 2021 at 10:15 PM Amit Kapila <amit.kapila16@gmail.com> wrote:

On Wed, Apr 7, 2021 at 1:11 PM Ajin Cherian <itsajin@gmail.com> wrote:

3.
There is inconsistent wording for what seems to be the same condition.
e.g.1 The existing documentation [2] says "Xid of the transaction. The
XID is sent only when user has requested streaming of in-progress
transactions."
e.g.2 For a similar case the patch says "Xid of the transaction (only
present for streamed transactions)."
I think there should be consistent wording used on this page where possible.

I think this is already modified as below in the patch. Is there any
other place you are referring to?

@@ -6457,8 +6462,7 @@ Message
 </term>
 <listitem>
 <para>
-                Xid of the transaction. The XID is sent only when user has
-                requested streaming of in-progress transactions.
+                Xid of the transaction (only present for streamed
transactions).

--
With Regards,
Amit Kapila.

Peter Smith

smithpb2250@gmail.com

almost 5 years ago

In reply to: Amit Kapila (#4)

Re: missing documentation for streaming in-progress transactions

On Thu, Apr 8, 2021 at 12:56 PM Amit Kapila <amit.kapila16@gmail.com> wrote:

On Thu, Apr 8, 2021 at 3:49 AM Peter Smith <smithpb2250@gmail.com> wrote:

On Wed, Apr 7, 2021 at 10:15 PM Amit Kapila <amit.kapila16@gmail.com> wrote:

On Wed, Apr 7, 2021 at 1:11 PM Ajin Cherian <itsajin@gmail.com> wrote:

3.
There is inconsistent wording for what seems to be the same condition.
e.g.1 The existing documentation [2] says "Xid of the transaction. The
XID is sent only when user has requested streaming of in-progress
transactions."
e.g.2 For a similar case the patch says "Xid of the transaction (only
present for streamed transactions)."
I think there should be consistent wording used on this page where possible.

I think this is already modified as below in the patch. Is there any
other place you are referring to?

No. My mistake. Sorry for the false alarm.

------
KInd Regards,
Peter Smith.
Fujitsu Australia

Ajin Cherian

itsajin@gmail.com

almost 5 years ago

In reply to: Peter Smith (#3)

1 attachment(s)

Re: missing documentation for streaming in-progress transactions

On Thu, Apr 8, 2021 at 8:19 AM Peter Smith <smithpb2250@gmail.com> wrote:

1.
I felt that this protocol documentation needs to include something
like a "Since: 2" notation (e.g. see how the javadoc API [1] does it)
otherwise with more message types and more protocol versions it is
quickly going to become too complicated to know what message or
message attribute belongs with what protocol.

Updated.

2.
There are inconsistent terms used for a transaction id.
e.g.1 Sometimes called " Transaction id."
e.g.2 Sometimes called "Xid of the transaction"
I think there should be consistent terminology used on this page

Updated.

regards,
Ajin Cherian
Fujitsu Australia

Attachments:

v3-0001-doc-Update-information-of-new-messages-for-logica.patchapplication/octet-stream; name=v3-0001-doc-Update-information-of-new-messages-for-logica.patchDownload

From 4d6a1f234c57f5be94da7b28435615b5700eb517 Mon Sep 17 00:00:00 2001
From: Ajin Cherian <ajinc@fast.au.fujitsu.com>
Date: Thu, 8 Apr 2021 03:13:41 -0400
Subject: [PATCH v3] doc: Update information of new messages for logical
 replication.

Updated documentation for new messages added for streaming in-progress
transactions, as well as changes made to the existing messages. It also
updates the information of protocol versions supported for logical
replication.

Author: Ajin Cherian
Reviewed-by: Amit Kapila, Peter Smith
Discussion: https://postgr.es/m/CAFPTHDYHN9m=MZZct-B=BYg_TETvv+kXvL9RD2DpaBS5pGxGYg@mail.gmail.com
---
 doc/src/sgml/protocol.sgml | 298 ++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 292 insertions(+), 6 deletions(-)

diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index 380be5f..6f4931e 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -2797,9 +2797,11 @@ The commands accepted in replication mode are:
      </term>
      <listitem>
       <para>
-       Protocol version. Currently only version <literal>1</literal> is
-       supported.
-      </para>
+       Protocol version. Currently versions <literal>1</literal> and
+       <literal>2</literal> are supported. The version <literal>2</literal>
+       is supported only for server versions 14 and above, and is used to allow
+       streaming of large in-progress transactions.
+     </para>
      </listitem>
     </varlistentry>
 
@@ -2855,7 +2857,10 @@ The commands accepted in replication mode are:
   <para>
    The logical replication protocol sends individual transactions one by one.
    This means that all messages between a pair of Begin and Commit messages
-   belong to the same transaction.
+   belong to the same transaction. It also sends changes of large in-progress
+   transactions between a pair of Stream Start and Stream Stop messages. The
+   last stream of such a transaction contains Stream Commit or Stream Abort
+   message.
   </para>
 
   <para>
@@ -6457,8 +6462,10 @@ Message
 </term>
 <listitem>
 <para>
-                Xid of the transaction. The XID is sent only when user has
-                requested streaming of in-progress transactions.
+               Since: 2
+</para>
+<para>
+                Xid of the transaction (only present for streamed transactions).
 </para>
 </listitem>
 </varlistentry>
@@ -6647,6 +6654,19 @@ Relation
 </term>
 <listitem>
 <para>
+              Since: 2
+</para>
+<para>
+               Xid of the transaction (only present for streamed transactions).
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
                 ID of the relation.
 </para>
 </listitem>
@@ -6768,6 +6788,19 @@ Type
 </term>
 <listitem>
 <para>
+              Since: 2
+</para>
+<para>
+               Xid of the transaction (only present for streamed transactions).
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
                 ID of the data type.
 </para>
 </listitem>
@@ -6822,6 +6855,19 @@ Insert
 </term>
 <listitem>
 <para>
+               Since: 2
+</para>
+<para>
+                Xid of the transaction (only present for streamed transactions).
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
                 ID of the relation corresponding to the ID in the relation
                 message.
 </para>
@@ -6878,6 +6924,19 @@ Update
 </term>
 <listitem>
 <para>
+              Since: 2
+</para>
+<para>
+               Xid of the transaction (only present for streamed transactions).
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
                 ID of the relation corresponding to the ID in the relation
                 message.
 </para>
@@ -6981,6 +7040,19 @@ Delete
 </term>
 <listitem>
 <para>
+              Since: 2
+</para>
+<para>
+               Xid of the transaction (only present for streamed transactions).
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
                 ID of the relation corresponding to the ID in the relation
                 message.
 </para>
@@ -7059,6 +7131,19 @@ Truncate
 </term>
 <listitem>
 <para>
+              Since: 2
+</para>
+<para>
+               Xid of the transaction (only present for streamed transactions).
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
                 Number of relations
 </para>
 </listitem>
@@ -7091,6 +7176,207 @@ Truncate
 </listitem>
 </varlistentry>
 
+<varlistentry>
+<term>
+Stream Start
+</term>
+<listitem>
+<para>
+
+<variablelist>
+<varlistentry>
+<term>
+        Byte1('S')
+</term>
+<listitem>
+<para>
+               Since: 2
+</para>
+<para>
+                Identifies the message as a stream start message.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
+                Xid of the transaction.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int8
+</term>
+<listitem>
+<para>
+                A value of 1 indicates this is the first stream segment for
+                this XID, 0 for any other stream segment.
+</para>
+</listitem>
+</varlistentry>
+
+</variablelist>
+</para>
+</listitem>
+</varlistentry>
+
+<varlistentry>
+<term>
+Stream Stop
+</term>
+<listitem>
+<para>
+
+<variablelist>
+<varlistentry>
+<term>
+        Byte1('E')
+</term>
+<listitem>
+<para>
+               Since: 2
+</para>
+<para>
+                Identifies the message as a stream stop message.
+</para>
+</listitem>
+</varlistentry>
+
+</variablelist>
+</para>
+</listitem>
+</varlistentry>
+
+<varlistentry>
+<term>
+Stream Commit
+</term>
+<listitem>
+<para>
+
+<variablelist>
+<varlistentry>
+<term>
+        Byte1('c')
+</term>
+<listitem>
+<para>
+               Since: 2
+</para>
+<para>
+                Identifies the message as a stream commit message.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
+                Xid of the transaction.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int8
+</term>
+<listitem>
+<para>
+                Flags; currently unused (must be 0).
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int64
+</term>
+<listitem>
+<para>
+                The LSN of the commit.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int64
+</term>
+<listitem>
+<para>
+                The end LSN of the transaction.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int64
+</term>
+<listitem>
+<para>
+                Commit timestamp of the transaction. The value is in number
+                of microseconds since PostgreSQL epoch (2000-01-01).
+</para>
+</listitem>
+</varlistentry>
+
+</variablelist>
+</para>
+</listitem>
+</varlistentry>
+
+<varlistentry>
+<term>
+Stream Abort
+</term>
+<listitem>
+<para>
+
+<variablelist>
+<varlistentry>
+<term>
+        Byte1('A')
+</term>
+<listitem>
+<para>
+               Since: 2
+</para>
+<para>
+                Identifies the message as a stream abort message.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
+                Xid of the transaction.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
+                Xid of the sub-transaction (will be same as xid of the transaction for top-level
+                transactions).
+</para>
+</listitem>
+</varlistentry>
+
+</variablelist>
+</para>
+</listitem>
+</varlistentry>
+
 </variablelist>
 
 <para>
-- 
1.8.3.1

Euler Taveira

euler@eulerto.com

almost 5 years ago

In reply to: Ajin Cherian (#6)

Re: missing documentation for streaming in-progress transactions

On Thu, Apr 8, 2021, at 4:25 AM, Ajin Cherian wrote:

Updated.

-       Protocol version. Currently only version <literal>1</literal> is
-       supported.
-      </para>
+       Protocol version. Currently versions <literal>1</literal> and
+       <literal>2</literal> are supported. The version <literal>2</literal>
+       is supported only for server versions 14 and above, and is used to allow
+       streaming of large in-progress transactions.
+     </para>

s/server versions/server version/

I suggest that the last part of the sentence might be "and it allows streaming
of large in-progress transactions"

+              Since: 2
+</para>
+<para>

I didn't like this style because it is not descriptive enough. It is also not a
style adopted by Postgres. I suggest to add something like "This field was
introduced in version 2" or "This field is available since version 2" after the
field description.

+                Xid of the sub-transaction (will be same as xid of the transaction for top-level
+                transactions).
+</para>

Although, sub-transaction is also used in the documentation, I suggest to use
subtransaction. Maybe change the other sub-transaction occurrences too.

--
Euler Taveira
EDB https://www.enterprisedb.com/

Ajin Cherian

itsajin@gmail.com

almost 5 years ago

In reply to: Euler Taveira (#7)

1 attachment(s)

Re: missing documentation for streaming in-progress transactions

On Fri, Apr 9, 2021 at 10:23 AM Euler Taveira <euler@eulerto.com> wrote:

I didn't like this style because it is not descriptive enough. It is also
not a
style adopted by Postgres. I suggest to add something like "This field was
introduced in version 2" or "This field is available since version 2"
after the
field description.

I have updated this to "Since protocol version 2"

+                Xid of the sub-transaction (will be same as xid of the
transaction for top-level
+                transactions).
+</para>
Although, sub-transaction is also used in the documentation, I suggest to
use
subtransaction. Maybe change the other sub-transaction occurrences too.

Updated.

regards,
Ajin Cherian
Fujitsu Australia

Attachments:

v4-0001-doc-Update-information-of-new-messages-for-logica.patchapplication/octet-stream; name=v4-0001-doc-Update-information-of-new-messages-for-logica.patchDownload

From 502c87886770205cc39d72d1e0ed9eb220e5ff21 Mon Sep 17 00:00:00 2001
From: Ajin Cherian <ajinc@fast.au.fujitsu.com>
Date: Thu, 8 Apr 2021 22:44:50 -0400
Subject: [PATCH v4] doc: Update information of new messages for logical
 replication.

Updated documentation for new messages added for streaming in-progress
transactions, as well as changes made to the existing messages. It also
updates the information of protocol versions supported for logical
replication.

Author: Ajin Cherian
Reviewed-by: Amit Kapila, Peter Smith
Discussion: https://postgr.es/m/CAFPTHDYHN9m=MZZct-B=BYg_TETvv+kXvL9RD2DpaBS5pGxGYg@mail.gmail.com
---
 doc/src/sgml/protocol.sgml | 298 ++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 292 insertions(+), 6 deletions(-)

diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index 380be5f..bf7d737 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -2797,9 +2797,11 @@ The commands accepted in replication mode are:
      </term>
      <listitem>
       <para>
-       Protocol version. Currently only version <literal>1</literal> is
-       supported.
-      </para>
+       Protocol version. Currently versions <literal>1</literal> and
+       <literal>2</literal> are supported. The version <literal>2</literal>
+       is supported only for server versions 14 and above, and is used to allow
+       streaming of large in-progress transactions.
+     </para>
      </listitem>
     </varlistentry>
 
@@ -2855,7 +2857,10 @@ The commands accepted in replication mode are:
   <para>
    The logical replication protocol sends individual transactions one by one.
    This means that all messages between a pair of Begin and Commit messages
-   belong to the same transaction.
+   belong to the same transaction. It also sends changes of large in-progress
+   transactions between a pair of Stream Start and Stream Stop messages. The
+   last stream of such a transaction contains Stream Commit or Stream Abort
+   message.
   </para>
 
   <para>
@@ -6457,8 +6462,10 @@ Message
 </term>
 <listitem>
 <para>
-                Xid of the transaction. The XID is sent only when user has
-                requested streaming of in-progress transactions.
+               Since protocol version 2
+</para>
+<para>
+                Xid of the transaction (only present for streamed transactions).
 </para>
 </listitem>
 </varlistentry>
@@ -6647,6 +6654,19 @@ Relation
 </term>
 <listitem>
 <para>
+              Since protocol version 2
+</para>
+<para>
+               Xid of the transaction (only present for streamed transactions).
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
                 ID of the relation.
 </para>
 </listitem>
@@ -6768,6 +6788,19 @@ Type
 </term>
 <listitem>
 <para>
+              Since protocol version 2
+</para>
+<para>
+               Xid of the transaction (only present for streamed transactions).
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
                 ID of the data type.
 </para>
 </listitem>
@@ -6822,6 +6855,19 @@ Insert
 </term>
 <listitem>
 <para>
+               Since protocol version 2
+</para>
+<para>
+                Xid of the transaction (only present for streamed transactions).
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
                 ID of the relation corresponding to the ID in the relation
                 message.
 </para>
@@ -6878,6 +6924,19 @@ Update
 </term>
 <listitem>
 <para>
+              Since protocol version 2
+</para>
+<para>
+               Xid of the transaction (only present for streamed transactions).
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
                 ID of the relation corresponding to the ID in the relation
                 message.
 </para>
@@ -6981,6 +7040,19 @@ Delete
 </term>
 <listitem>
 <para>
+              Since protocol version 2
+</para>
+<para>
+               Xid of the transaction (only present for streamed transactions).
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
                 ID of the relation corresponding to the ID in the relation
                 message.
 </para>
@@ -7059,6 +7131,19 @@ Truncate
 </term>
 <listitem>
 <para>
+              Since protocol version 2
+</para>
+<para>
+               Xid of the transaction (only present for streamed transactions).
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
                 Number of relations
 </para>
 </listitem>
@@ -7091,6 +7176,207 @@ Truncate
 </listitem>
 </varlistentry>
 
+<varlistentry>
+<term>
+Stream Start
+</term>
+<listitem>
+<para>
+
+<variablelist>
+<varlistentry>
+<term>
+        Byte1('S')
+</term>
+<listitem>
+<para>
+               Since protocol version 2
+</para>
+<para>
+                Identifies the message as a stream start message.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
+                Xid of the transaction.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int8
+</term>
+<listitem>
+<para>
+                A value of 1 indicates this is the first stream segment for
+                this XID, 0 for any other stream segment.
+</para>
+</listitem>
+</varlistentry>
+
+</variablelist>
+</para>
+</listitem>
+</varlistentry>
+
+<varlistentry>
+<term>
+Stream Stop
+</term>
+<listitem>
+<para>
+
+<variablelist>
+<varlistentry>
+<term>
+        Byte1('E')
+</term>
+<listitem>
+<para>
+               Since protocol version 2
+</para>
+<para>
+                Identifies the message as a stream stop message.
+</para>
+</listitem>
+</varlistentry>
+
+</variablelist>
+</para>
+</listitem>
+</varlistentry>
+
+<varlistentry>
+<term>
+Stream Commit
+</term>
+<listitem>
+<para>
+
+<variablelist>
+<varlistentry>
+<term>
+        Byte1('c')
+</term>
+<listitem>
+<para>
+               Since protocol version 2
+</para>
+<para>
+                Identifies the message as a stream commit message.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
+                Xid of the transaction.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int8
+</term>
+<listitem>
+<para>
+                Flags; currently unused (must be 0).
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int64
+</term>
+<listitem>
+<para>
+                The LSN of the commit.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int64
+</term>
+<listitem>
+<para>
+                The end LSN of the transaction.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int64
+</term>
+<listitem>
+<para>
+                Commit timestamp of the transaction. The value is in number
+                of microseconds since PostgreSQL epoch (2000-01-01).
+</para>
+</listitem>
+</varlistentry>
+
+</variablelist>
+</para>
+</listitem>
+</varlistentry>
+
+<varlistentry>
+<term>
+Stream Abort
+</term>
+<listitem>
+<para>
+
+<variablelist>
+<varlistentry>
+<term>
+        Byte1('A')
+</term>
+<listitem>
+<para>
+               Since protocol version 2
+</para>
+<para>
+                Identifies the message as a stream abort message.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
+                Xid of the transaction.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
+                Xid of the subtransaction (will be same as xid of the transaction for top-level
+                transactions).
+</para>
+</listitem>
+</varlistentry>
+
+</variablelist>
+</para>
+</listitem>
+</varlistentry>
+
 </variablelist>
 
 <para>
-- 
1.8.3.1

Amit Kapila

amit.kapila16@gmail.com

almost 5 years ago

In reply to: Ajin Cherian (#8)

1 attachment(s)

Re: missing documentation for streaming in-progress transactions

On Fri, Apr 9, 2021 at 8:29 AM Ajin Cherian <itsajin@gmail.com> wrote:

On Fri, Apr 9, 2021 at 10:23 AM Euler Taveira <euler@eulerto.com> wrote:

I didn't like this style because it is not descriptive enough. It is also not a
style adopted by Postgres. I suggest to add something like "This field was
introduced in version 2" or "This field is available since version 2" after the
field description.

I have updated this to "Since protocol version 2"
+                Xid of the sub-transaction (will be same as xid of the transaction for top-level
+                transactions).
+</para>
Although, sub-transaction is also used in the documentation, I suggest to use
subtransaction. Maybe change the other sub-transaction occurrences too.
Updated.

I don't like repeating the same thing for all new messages. So added
separate para for the same and few other changes. See what do you
think of the attached?

--
With Regards,
Amit Kapila.

Attachments:

v5-0001-doc-Update-information-of-new-messages-for-logica.patchapplication/octet-stream; name=v5-0001-doc-Update-information-of-new-messages-for-logica.patchDownload

From 4052343d34d1f066ff1b9252288d49926e44a1fe Mon Sep 17 00:00:00 2001
From: Amit Kapila <akapila@postgresql.org>
Date: Fri, 9 Apr 2021 09:17:42 +0530
Subject: [PATCH v5] doc: Update information of new messages for logical
 replication.

Updated documentation for new messages added for streaming in-progress
transactions, as well as changes made to the existing messages. It also
updates the information of protocol versions supported for logical
replication.

Author: Ajin Cherian
Reviewed-by: Amit Kapila, Peter Smith, Euler Taveira
Discussion: https://postgr.es/m/CAFPTHDYHN9m=MZZct-B=BYg_TETvv+kXvL9RD2DpaBS5pGxGYg@mail.gmail.com
---
 doc/src/sgml/protocol.sgml | 283 ++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 277 insertions(+), 6 deletions(-)

diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index 380be5f..2f4dde3 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -2797,9 +2797,11 @@ The commands accepted in replication mode are:
      </term>
      <listitem>
       <para>
-       Protocol version. Currently only version <literal>1</literal> is
-       supported.
-      </para>
+       Protocol version. Currently versions <literal>1</literal> and
+       <literal>2</literal> are supported. The version <literal>2</literal>
+       is supported only for server version 14 and above, and it allows
+       streaming of large in-progress transactions.
+     </para>
      </listitem>
     </varlistentry>
 
@@ -2855,7 +2857,10 @@ The commands accepted in replication mode are:
   <para>
    The logical replication protocol sends individual transactions one by one.
    This means that all messages between a pair of Begin and Commit messages
-   belong to the same transaction.
+   belong to the same transaction. It also sends changes of large in-progress
+   transactions between a pair of Stream Start and Stream Stop messages. The
+   last stream of such a transaction contains Stream Commit or Stream Abort
+   message.
   </para>
 
   <para>
@@ -6457,8 +6462,8 @@ Message
 </term>
 <listitem>
 <para>
-                Xid of the transaction. The XID is sent only when user has
-                requested streaming of in-progress transactions.
+                Xid of the transaction (only present for streamed transactions).
+                This field is available since protocol version 2.
 </para>
 </listitem>
 </varlistentry>
@@ -6647,6 +6652,17 @@ Relation
 </term>
 <listitem>
 <para>
+               Xid of the transaction (only present for streamed transactions).
+               This field is available since protocol version 2.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
                 ID of the relation.
 </para>
 </listitem>
@@ -6768,6 +6784,17 @@ Type
 </term>
 <listitem>
 <para>
+               Xid of the transaction (only present for streamed transactions).
+               This field is available since protocol version 2.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
                 ID of the data type.
 </para>
 </listitem>
@@ -6822,6 +6849,17 @@ Insert
 </term>
 <listitem>
 <para>
+                Xid of the transaction (only present for streamed transactions).
+                This field is available since protocol version 2.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
                 ID of the relation corresponding to the ID in the relation
                 message.
 </para>
@@ -6878,6 +6916,17 @@ Update
 </term>
 <listitem>
 <para>
+               Xid of the transaction (only present for streamed transactions).
+               This field is available since protocol version 2.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
                 ID of the relation corresponding to the ID in the relation
                 message.
 </para>
@@ -6981,6 +7030,17 @@ Delete
 </term>
 <listitem>
 <para>
+               Xid of the transaction (only present for streamed transactions).
+               This field is available since protocol version 2.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
                 ID of the relation corresponding to the ID in the relation
                 message.
 </para>
@@ -7059,6 +7119,17 @@ Truncate
 </term>
 <listitem>
 <para>
+               Xid of the transaction (only present for streamed transactions).
+               This field is available since protocol version 2.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
                 Number of relations
 </para>
 </listitem>
@@ -7095,6 +7166,206 @@ Truncate
 
 <para>
 
+The following messages (Stream Start, Stream End, Stream Commit, and
+Stream Abort) are available since protocol version 2.
+
+</para>
+
+<variablelist>
+
+<varlistentry>
+<term>
+Stream Start
+</term>
+<listitem>
+<para>
+
+<variablelist>
+<varlistentry>
+<term>
+        Byte1('S')
+</term>
+<listitem>
+<para>
+                Identifies the message as a stream start message.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
+                Xid of the transaction.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int8
+</term>
+<listitem>
+<para>
+                A value of 1 indicates this is the first stream segment for
+                this XID, 0 for any other stream segment.
+</para>
+</listitem>
+</varlistentry>
+
+</variablelist>
+</para>
+</listitem>
+</varlistentry>
+
+<varlistentry>
+<term>
+Stream Stop
+</term>
+<listitem>
+<para>
+
+<variablelist>
+<varlistentry>
+<term>
+        Byte1('E')
+</term>
+<listitem>
+<para>
+                Identifies the message as a stream stop message.
+</para>
+</listitem>
+</varlistentry>
+
+</variablelist>
+</para>
+</listitem>
+</varlistentry>
+
+<varlistentry>
+<term>
+Stream Commit
+</term>
+<listitem>
+<para>
+
+<variablelist>
+<varlistentry>
+<term>
+        Byte1('c')
+</term>
+<listitem>
+<para>
+                Identifies the message as a stream commit message.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
+                Xid of the transaction.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int8
+</term>
+<listitem>
+<para>
+                Flags; currently unused (must be 0).
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int64
+</term>
+<listitem>
+<para>
+                The LSN of the commit.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int64
+</term>
+<listitem>
+<para>
+                The end LSN of the transaction.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int64
+</term>
+<listitem>
+<para>
+                Commit timestamp of the transaction. The value is in number
+                of microseconds since PostgreSQL epoch (2000-01-01).
+</para>
+</listitem>
+</varlistentry>
+
+</variablelist>
+</para>
+</listitem>
+</varlistentry>
+
+<varlistentry>
+<term>
+Stream Abort
+</term>
+<listitem>
+<para>
+
+<variablelist>
+<varlistentry>
+<term>
+        Byte1('A')
+</term>
+<listitem>
+<para>
+                Identifies the message as a stream abort message.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
+                Xid of the transaction.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+        Int32
+</term>
+<listitem>
+<para>
+                Xid of the subtransaction (will be same as xid of the transaction for top-level
+                transactions).
+</para>
+</listitem>
+</varlistentry>
+
+</variablelist>
+</para>
+</listitem>
+</varlistentry>
+
+</variablelist>
+
+<para>
+
 The following message parts are shared by the above messages.
 
 </para>
-- 
1.8.3.1

#10

Justin Pryzby

pryzby@telsasoft.com

almost 5 years ago

In reply to: Amit Kapila (#2)

Re: missing documentation for streaming in-progress transactions

On Wed, Apr 07, 2021 at 05:45:16PM +0530, Amit Kapila wrote:

On Wed, Apr 7, 2021 at 1:11 PM Ajin Cherian <itsajin@gmail.com> wrote:

Found that some documentation hasn't been updated for the changes made as part of
streaming large in-progress transactions. Attached a patch that adds the missing changes. Let me know if anything more needs to be added or any comments on this change.

Thanks, this mostly looks good to me. I have slightly modified it.
See, what do you think of the attached?

+       Protocol version. Currently versions <literal>1</literal> and
+       <literal>2</literal> are supported. The version <literal>2</literal>
+       is supported only for server versions 14 and above, and is used to allow
+       streaming of large in-progress transactions.

The diff briefly confused me, since this is in protocol.sgml, and since the
libpq protocol version is 1/2/3, with 2 being removed in v14 (3174d69fb).
I suggest to say "replication protocol version 2".

I realize that the headings make this more clear when reading the .html, so
maybe there's no issue.

--
Justin

#11

Amit Kapila

amit.kapila16@gmail.com

almost 5 years ago

In reply to: Justin Pryzby (#10)

Re: missing documentation for streaming in-progress transactions

On Fri, Apr 9, 2021 at 9:58 AM Justin Pryzby <pryzby@telsasoft.com> wrote:

On Wed, Apr 07, 2021 at 05:45:16PM +0530, Amit Kapila wrote:

On Wed, Apr 7, 2021 at 1:11 PM Ajin Cherian <itsajin@gmail.com> wrote:

Found that some documentation hasn't been updated for the changes made as part of
streaming large in-progress transactions. Attached a patch that adds the missing changes. Let me know if anything more needs to be added or any comments on this change.

Thanks, this mostly looks good to me. I have slightly modified it.
See, what do you think of the attached?
+       Protocol version. Currently versions <literal>1</literal> and
+       <literal>2</literal> are supported. The version <literal>2</literal>
+       is supported only for server versions 14 and above, and is used to allow
+       streaming of large in-progress transactions.
The diff briefly confused me, since this is in protocol.sgml, and since the
libpq protocol version is 1/2/3, with 2 being removed in v14 (3174d69fb).
I suggest to say "replication protocol version 2".

I realize that the headings make this more clear when reading the .html, so
maybe there's no issue.

Yeah, this was the reason to not include replication. If one is
reading the document or even *.sgml, there shouldn't be any confusion
but if you or others feel so, we can use 'replication' as well.

--
With Regards,
Amit Kapila.

#12

Amit Kapila

amit.kapila16@gmail.com

almost 5 years ago

In reply to: Amit Kapila (#9)

Re: missing documentation for streaming in-progress transactions

On Fri, Apr 9, 2021 at 9:39 AM Amit Kapila <amit.kapila16@gmail.com> wrote:

I don't like repeating the same thing for all new messages. So added
separate para for the same and few other changes. See what do you
think of the attached?

Pushed.

--
With Regards,
Amit Kapila.