Missing empty transaction optimization in pgoutput plugin

On Mon, Dec 8, 2025 at 12:19 PM Ashutosh Bapat
<ashutosh.bapat.oss@gmail.com> wrote:

test_decoding output plugin skips sending stream_start and stream_end
if no change from a segment of a streamed transaction is sent
downstream. If no change has been sent downstream across all the
streamed segments it doesn't send commit or abort for that streamed
transaction. But I don't see similar changes in stream related
callbacks in pgoutput output plugin.

While the changes discussed in thread [1] were committed, those didn't
include this optimization for test_decoding plugin as well. The
optimization was discussed in [2] separately as a bug fix. Maybe
because it wasn't considered as an optimization that time, we may not
have considered it for pgoutput plugin which doesn't have explicit
skip-empty-xacts option. I think the optimization would be useful in
setups where multiple publications and long transactions are common.
Any reason not to have it.

I don't remember any reason why we can't have the same optimization
for streamed transactions. IIRC, we implemented this optimization
because we got reports for extra messages for short empty-transactions
but if you see any practical cases where it could be useful for
streamed transactions as well then feel free to propose a patch.

--
With Regards,
Amit Kapila.

On Monday, December 8, 2025 2:49 PM Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> wrote:

Hi All,
test_decoding output plugin skips sending stream_start and stream_end if no
change from a segment of a streamed transaction is sent downstream. If no
change has been sent downstream across all the streamed segments it
doesn't send commit or abort for that streamed transaction. But I don't see
similar changes in stream related callbacks in pgoutput output plugin.

While the changes discussed in thread [1] were committed, those didn't
include this optimization for test_decoding plugin as well. The optimization
was discussed in [2] separately as a bug fix. Maybe because it wasn't
considered as an optimization that time, we may not have considered it for
pgoutput plugin which doesn't have explicit skip-empty-xacts option. I think
the optimization would be useful in setups where multiple publications and
long transactions are common.
Any reason not to have it.

One possible reason we did not implement the optimization to avoid sending empty stream
blocks along with commit d5a9d86 is the handling of prepared transactions, for
which STREAM PREPARE is sent. The logical replication worker currently relies on
receiving at least one STREAM START/STREAM STOP prior to this; otherwise, it
might encounter errors when applying STREAM PREPARE. Although the optimization
is feasible, it requires some modifications to ensure the replication worker works
correctly, such as sending an empty stream block before issuing STREAM PREPARE.

One might think that we could avoid sending STREAM PREPARE for empty txn as
well, but that could be tricky, see the comment atop of (typedef struct
PGOutputTxnData) for details.

[1] /messages/by-id/688b0b7f-2f6c-d827-c27b-
216a8e3ea700%402ndquadrant.com
[2] https://www.postgresql.org/message-
id/flat/CAA4eK1%2BOqgFNZkf7%3DETe_y5ntjgDk3T0wcdkd4Sot_u1hySGfw%
40mail.gmail.com

Best Regards,
Hou zj

On Wed, Dec 10, 2025 at 9:17 AM Zhijie Hou (Fujitsu)
<houzj.fnst@fujitsu.com> wrote:

On Monday, December 8, 2025 2:49 PM Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> wrote:

Hi All,
test_decoding output plugin skips sending stream_start and stream_end if no
change from a segment of a streamed transaction is sent downstream. If no
change has been sent downstream across all the streamed segments it
doesn't send commit or abort for that streamed transaction. But I don't see
similar changes in stream related callbacks in pgoutput output plugin.

While the changes discussed in thread [1] were committed, those didn't
include this optimization for test_decoding plugin as well. The optimization
was discussed in [2] separately as a bug fix. Maybe because it wasn't
considered as an optimization that time, we may not have considered it for
pgoutput plugin which doesn't have explicit skip-empty-xacts option. I think
the optimization would be useful in setups where multiple publications and
long transactions are common.
Any reason not to have it.

One possible reason we did not implement the optimization to avoid sending empty stream
blocks along with commit d5a9d86 is the handling of prepared transactions, for
which STREAM PREPARE is sent. The logical replication worker currently relies on
receiving at least one STREAM START/STREAM STOP prior to this; otherwise, it
might encounter errors when applying STREAM PREPARE. Although the optimization
is feasible, it requires some modifications to ensure the replication worker works
correctly, such as sending an empty stream block before issuing STREAM PREPARE.

One might think that we could avoid sending STREAM PREPARE for empty txn as
well, but that could be tricky, see the comment atop of (typedef struct
PGOutputTxnData) for details.

[1] /messages/by-id/688b0b7f-2f6c-d827-c27b-
216a8e3ea700%402ndquadrant.com
[2] https://www.postgresql.org/message-
id/flat/CAA4eK1%2BOqgFNZkf7%3DETe_y5ntjgDk3T0wcdkd4Sot_u1hySGfw%
40mail.gmail.com

Thanks a lot for the details. I see it's much larger change than just
a few lines like test_decoding.c.

--
Best Wishes,
Ashutosh Bapat