pgsql: Add decoding of sequences to test_decoding

Hi,

On 2022-02-11 23:52:20 +0000, Tomas Vondra wrote:

Add decoding of sequences to test_decoding

Commit 0da92dc530 improved the logical decoding infrastructure to handle
sequences, and did various changes to related parts (WAL logging etc.).
But it did not include any implementation of the new callbacks added to
OutputPluginCallbacks.

This extends test_decoding with two callbacks to decode sequences. The
decoding of sequences may be disabled using 'include-sequences', a new
option of the output plugin.

Author: Tomas Vondra, Cary Huang
Reviewed-by: Peter Eisentraut, Hannu Krosing, Andres Freund
Discussion: /messages/by-id/d045f3c2-6cfb-06d3-5540-e63c320df8bc@enterprisedb.com
Discussion: /messages/by-id/1710ed7e13b.cd7177461430746.3372264562543607781@highgo.ca

The tests fail on slow machines (valgrind in this case), with some spurious
output differences:

https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink&dt=2022-02-12%2010%3A24%3A22

diff -U3 /mnt/resource/bf/build/skink-master/HEAD/pgsql/contrib/test_decoding/expected/sequence.out /mnt/resource/bf/build/skink-master/HEAD/pgsql.build/contrib/test_decoding/results/sequence.out
--- /mnt/resource/bf/build/skink-master/HEAD/pgsql/contrib/test_decoding/expected/sequence.out	2022-02-12 00:40:25.431913374 +0000
+++ /mnt/resource/bf/build/skink-master/HEAD/pgsql.build/contrib/test_decoding/results/sequence.out	2022-02-12 11:07:19.177487256 +0000
@@ -263,12 +263,14 @@
                                           data
 -----------------------------------------------------------------------------------------
  BEGIN
+ COMMIT
+ BEGIN
  sequence public.test_table_a_seq: transactional:1 last_value: 1 log_cnt: 0 is_called:0
  sequence public.test_table_a_seq: transactional:1 last_value: 33 log_cnt: 0 is_called:1
  table public.test_table: INSERT: a[integer]:1 b[integer]:100
  table public.test_table: INSERT: a[integer]:2 b[integer]:200
  COMMIT
-(6 rows)
+(8 rows)

I don't think it's a good idea to use skip-empty-xacts = 0 in tests, there
always can be an autovacuum or such.

Greetings,

Andres Freund

On 2/12/22 23:04, Andres Freund wrote:

Hi,

On 2022-02-11 23:52:20 +0000, Tomas Vondra wrote:

Add decoding of sequences to test_decoding

Commit 0da92dc530 improved the logical decoding infrastructure to handle
sequences, and did various changes to related parts (WAL logging etc.).
But it did not include any implementation of the new callbacks added to
OutputPluginCallbacks.

This extends test_decoding with two callbacks to decode sequences. The
decoding of sequences may be disabled using 'include-sequences', a new
option of the output plugin.

Author: Tomas Vondra, Cary Huang
Reviewed-by: Peter Eisentraut, Hannu Krosing, Andres Freund
Discussion: /messages/by-id/d045f3c2-6cfb-06d3-5540-e63c320df8bc@enterprisedb.com
Discussion: /messages/by-id/1710ed7e13b.cd7177461430746.3372264562543607781@highgo.ca

The tests fail on slow machines (valgrind in this case), with some spurious
output differences:

https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink&dt=2022-02-12%2010%3A24%3A22

diff -U3 /mnt/resource/bf/build/skink-master/HEAD/pgsql/contrib/test_decoding/expected/sequence.out /mnt/resource/bf/build/skink-master/HEAD/pgsql.build/contrib/test_decoding/results/sequence.out
--- /mnt/resource/bf/build/skink-master/HEAD/pgsql/contrib/test_decoding/expected/sequence.out	2022-02-12 00:40:25.431913374 +0000
+++ /mnt/resource/bf/build/skink-master/HEAD/pgsql.build/contrib/test_decoding/results/sequence.out	2022-02-12 11:07:19.177487256 +0000
@@ -263,12 +263,14 @@
data
-----------------------------------------------------------------------------------------
BEGIN
+ COMMIT
+ BEGIN
sequence public.test_table_a_seq: transactional:1 last_value: 1 log_cnt: 0 is_called:0
sequence public.test_table_a_seq: transactional:1 last_value: 33 log_cnt: 0 is_called:1
table public.test_table: INSERT: a[integer]:1 b[integer]:100
table public.test_table: INSERT: a[integer]:2 b[integer]:200
COMMIT
-(6 rows)
+(8 rows)

I don't think it's a good idea to use skip-empty-xacts = 0 in tests, there
always can be an autovacuum or such.

Thanks, I pushed a fix. Turns out the bug was a bit worse, because the
sequence callbacks did not actually deal with empty xacts correctly, so
I fixed that too.

regards

--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company