Add contrib/pg_logicalsnapinspect

Started by Bertrand Drouvotover 1 year ago81 messages
#1Bertrand Drouvot
bertranddrouvot.pg@gmail.com
1 attachment(s)

Hi hackers,

Please find attached a patch to $SUBJECT.

This module provides SQL functions to inspect the contents of serialized logical
snapshots of a running database cluster, which I think could be useful for
debugging or educational purposes.

It's currently made of 2 functions, one to return the metadata:

postgres=# SELECT * FROM pg_get_logical_snapshot_meta('0/40796E18');
-[ RECORD 1 ]--------
magic | 1369563137
checksum | 1028045905
version | 6

and one to return more information:

postgres=# SELECT * FROM pg_get_logical_snapshot_info('0/40796E18');
-[ RECORD 1 ]------------+-----------
state | 2
xmin | 751
xmax | 751
start_decoding_at | 0/40796AF8
two_phase_at | 0/40796AF8
initial_xmin_horizon | 0
building_full_snapshot | f
in_slot_creation | f
last_serialized_snapshot | 0/0
next_phase_at | 0
committed_count | 0
committed_xip |
catchange_count | 2
catchange_xip | {751,752}

The LSN used as argument is extracted from the snapshot file name:

postgres=# select * from pg_ls_logicalsnapdir();
name | size | modification
-----------------+------+------------------------
0-40796E18.snap | 152 | 2024-08-14 16:36:32+00
(1 row)

A few remarks:

1. The "state" field is linked to the SnapBuildState enum (snapbuild.h). I've the
feeling that that's fine to display it as int but could write an helper function
to display strings instead ('SNAPBUILD_BUILDING_SNAPSHOT',...).

2. The SnapBuildOnDisk and SnapBuild structs are now exposed to public. Means
we should now pay much more attention when changing their contents but I think
it's worth it.

3. The pg_get_logical_snapshot_info() function mainly displays the SnapBuild
content extracted from the logical snapshot file.

4. I think that providing SQL functions is enough and that it's not needed to
also create a related binary tool.

5. A few PGDLLIMPORT have been added (Windows CI was failing).

6. Related documentation has been added.

7. A test has been added.

8. I don't like the module name that much but it follows the same as for
pg_walinspect.

Looking forward to your feedback,

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachments:

v1-0001-Add-contrib-pg_logicalsnapinspect.patchtext/x-diff; charset=us-asciiDownload
From cc57b8cda2ec7d7bcd5122f20f2b2c9950840998 Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Wed, 14 Aug 2024 08:46:05 +0000
Subject: [PATCH v1] Add contrib/pg_logicalsnapinspect

Provides SQL functions that allow to inspect the contents of serialized logical
snapshots of a running database cluster, which is useful for debugging or
educational purposes.
---
 contrib/Makefile                              |   1 +
 contrib/meson.build                           |   1 +
 contrib/pg_logicalsnapinspect/.gitignore      |   4 +
 contrib/pg_logicalsnapinspect/Makefile        |  31 +++
 .../expected/logical_snapshot_inspect.out     |  52 ++++
 .../logicalsnapinspect.conf                   |   1 +
 contrib/pg_logicalsnapinspect/meson.build     |  39 +++
 .../pg_logicalsnapinspect--1.0.sql            |  43 +++
 .../pg_logicalsnapinspect.c                   | 249 ++++++++++++++++++
 .../pg_logicalsnapinspect.control             |   5 +
 .../specs/logical_snapshot_inspect.spec       |  34 +++
 doc/src/sgml/contrib.sgml                     |   1 +
 doc/src/sgml/filelist.sgml                    |   1 +
 doc/src/sgml/pglogicalsnapinspect.sgml        | 144 ++++++++++
 src/backend/replication/logical/snapbuild.c   | 189 +------------
 src/include/port/pg_crc32c.h                  |  16 +-
 src/include/replication/snapbuild.h           | 186 ++++++++++++-
 17 files changed, 800 insertions(+), 197 deletions(-)
   7.7% contrib/pg_logicalsnapinspect/expected/
   5.8% contrib/pg_logicalsnapinspect/specs/
  33.2% contrib/pg_logicalsnapinspect/
  13.4% doc/src/sgml/
  17.5% src/backend/replication/logical/
   4.2% src/include/port/
  17.7% src/include/replication/

diff --git a/contrib/Makefile b/contrib/Makefile
index abd780f277..a379ce30c8 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -32,6 +32,7 @@ SUBDIRS = \
 		passwordcheck	\
 		pg_buffercache	\
 		pg_freespacemap \
+		pg_logicalsnapinspect \
 		pg_prewarm	\
 		pg_stat_statements \
 		pg_surgery	\
diff --git a/contrib/meson.build b/contrib/meson.build
index 14a8906865..d54009bfe5 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -46,6 +46,7 @@ subdir('passwordcheck')
 subdir('pg_buffercache')
 subdir('pgcrypto')
 subdir('pg_freespacemap')
+subdir('pg_logicalsnapinspect')
 subdir('pg_prewarm')
 subdir('pgrowlocks')
 subdir('pg_stat_statements')
diff --git a/contrib/pg_logicalsnapinspect/.gitignore b/contrib/pg_logicalsnapinspect/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/contrib/pg_logicalsnapinspect/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/contrib/pg_logicalsnapinspect/Makefile b/contrib/pg_logicalsnapinspect/Makefile
new file mode 100644
index 0000000000..aef1d9aa87
--- /dev/null
+++ b/contrib/pg_logicalsnapinspect/Makefile
@@ -0,0 +1,31 @@
+# contrib/pg_logicalsnapinspect/Makefile
+
+MODULE_big = pg_logicalsnapinspect
+OBJS = \
+	$(WIN32RES) \
+	pg_logicalsnapinspect.o
+PGFILEDESC = "pg_logicalsnapinspect - functions to inspect logical snapshots"
+
+EXTENSION = pg_logicalsnapinspect
+DATA = pg_logicalsnapinspect--1.0.sql
+
+EXTRA_INSTALL = contrib/test_decoding
+
+ISOLATION = logical_snapshot_inspect
+
+ISOLATION_OPTS = --temp-config $(top_srcdir)/contrib/pg_logicalsnapinspect/logicalsnapinspect.conf
+
+# Disabled because these tests require "wal_level=logical", which
+# some installcheck users do not have (e.g. buildfarm clients).
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/pg_logicalsnapinspect
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/pg_logicalsnapinspect/expected/logical_snapshot_inspect.out b/contrib/pg_logicalsnapinspect/expected/logical_snapshot_inspect.out
new file mode 100644
index 0000000000..749cd4642d
--- /dev/null
+++ b/contrib/pg_logicalsnapinspect/expected/logical_snapshot_inspect.out
@@ -0,0 +1,52 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s0_init s0_begin s0_savepoint s0_truncate s1_checkpoint s1_get_changes s0_commit s0_begin s0_insert s1_checkpoint s1_get_changes s0_commit s1_get_changes s1_get_logical_snapshot_info s1_get_logical_snapshot_meta
+step s0_init: SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding');
+?column?
+--------
+init    
+(1 row)
+
+step s0_begin: BEGIN;
+step s0_savepoint: SAVEPOINT sp1;
+step s0_truncate: TRUNCATE tbl1;
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data
+----
+(0 rows)
+
+step s0_commit: COMMIT;
+step s0_begin: BEGIN;
+step s0_insert: INSERT INTO tbl1 VALUES (1);
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                   
+---------------------------------------
+BEGIN                                  
+table public.tbl1: TRUNCATE: (no-flags)
+COMMIT                                 
+(3 rows)
+
+step s0_commit: COMMIT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                                         
+-------------------------------------------------------------
+BEGIN                                                        
+table public.tbl1: INSERT: val1[integer]:1 val2[integer]:null
+COMMIT                                                       
+(3 rows)
+
+step s1_get_logical_snapshot_info: SELECT (pg_get_logical_snapshot_info(f.name::pg_lsn)).state,(pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_xip,1),(pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_xip,1) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f ORDER BY 2;
+state|catchange_count|array_length|committed_count|array_length
+-----+---------------+------------+---------------+------------
+    2|              0|            |              2|           2
+    2|              2|           2|              0|            
+(2 rows)
+
+step s1_get_logical_snapshot_meta: SELECT COUNT((pg_get_logical_snapshot_meta(f.name::pg_lsn))) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f;
+count
+-----
+    2
+(1 row)
+
diff --git a/contrib/pg_logicalsnapinspect/logicalsnapinspect.conf b/contrib/pg_logicalsnapinspect/logicalsnapinspect.conf
new file mode 100644
index 0000000000..e3d257315f
--- /dev/null
+++ b/contrib/pg_logicalsnapinspect/logicalsnapinspect.conf
@@ -0,0 +1 @@
+wal_level = logical
diff --git a/contrib/pg_logicalsnapinspect/meson.build b/contrib/pg_logicalsnapinspect/meson.build
new file mode 100644
index 0000000000..9f2c2bb45b
--- /dev/null
+++ b/contrib/pg_logicalsnapinspect/meson.build
@@ -0,0 +1,39 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+pg_logicalsnapinspect_sources = files('pg_logicalsnapinspect.c')
+
+if host_system == 'windows'
+  pg_logicalsnapinspect_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+    '--NAME', 'pg_logicalsnapinspect',
+    '--FILEDESC', 'pg_logicalsnapinspect - functions to inspect contents of logical snapshots',])
+endif
+
+pg_logicalsnapinspect = shared_module('pg_logicalsnapinspect',
+  pg_logicalsnapinspect_sources,
+  kwargs: contrib_mod_args + {
+      'dependencies': contrib_mod_args['dependencies'],
+  },
+)
+contrib_targets += pg_logicalsnapinspect
+
+install_data(
+  'pg_logicalsnapinspect.control',
+  'pg_logicalsnapinspect--1.0.sql',
+  kwargs: contrib_data_args,
+)
+
+tests += {
+  'name': 'pg_logicalsnapinspect',
+  'sd': meson.current_source_dir(),
+  'bd': meson.current_build_dir(),
+  'isolation': {
+    'specs': [
+      'logical_snapshot_inspect',
+    ],
+    'regress_args': [
+      '--temp-config', files('logicalsnapinspect.conf'),
+    ],
+    # see above
+    'runningcheck': false,
+  },
+}
diff --git a/contrib/pg_logicalsnapinspect/pg_logicalsnapinspect--1.0.sql b/contrib/pg_logicalsnapinspect/pg_logicalsnapinspect--1.0.sql
new file mode 100644
index 0000000000..0fcc8aa816
--- /dev/null
+++ b/contrib/pg_logicalsnapinspect/pg_logicalsnapinspect--1.0.sql
@@ -0,0 +1,43 @@
+/* contrib/pg_logicalsnapinspect/pg_logicalsnapinspect--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_logicalsnapinspect" to load this file. \quit
+
+--
+-- pg_get_logical_snapshot_meta()
+--
+CREATE FUNCTION pg_get_logical_snapshot_meta(IN in_lsn pg_lsn,
+    OUT magic int4,
+    OUT checksum int4,
+    OUT version int4
+)
+AS 'MODULE_PATHNAME', 'pg_get_logical_snapshot_meta'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_logical_snapshot_meta(pg_lsn) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_logical_snapshot_meta(pg_lsn) TO pg_read_server_files;
+
+--
+-- pg_get_logical_snapshot_info()
+--
+CREATE FUNCTION pg_get_logical_snapshot_info(IN in_lsn pg_lsn,
+    OUT state int2,
+    OUT xmin xid,
+    OUT xmax xid,
+    OUT start_decoding_at pg_lsn,
+    OUT two_phase_at pg_lsn,
+    OUT initial_xmin_horizon xid,
+    OUT building_full_snapshot boolean,
+    OUT in_slot_creation boolean,
+    OUT last_serialized_snapshot pg_lsn,
+    OUT next_phase_at xid,
+    OUT committed_count int8,
+    OUT committed_xip xid[],
+    OUT catchange_count int8,
+    OUT catchange_xip xid[]
+)
+AS 'MODULE_PATHNAME', 'pg_get_logical_snapshot_info'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_logical_snapshot_info(pg_lsn) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_logical_snapshot_info(pg_lsn) TO pg_read_server_files;
diff --git a/contrib/pg_logicalsnapinspect/pg_logicalsnapinspect.c b/contrib/pg_logicalsnapinspect/pg_logicalsnapinspect.c
new file mode 100644
index 0000000000..874129d01f
--- /dev/null
+++ b/contrib/pg_logicalsnapinspect/pg_logicalsnapinspect.c
@@ -0,0 +1,249 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_logicalsnapinspect.c
+ *		  Functions to inspect contents of PostgreSQL logical snapshots
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ *		  contrib/pg_logicalsnapinspect/pg_logicalsnapinspect.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "funcapi.h"
+#include "port/pg_crc32c.h"
+#include "replication/snapbuild.h"
+#include "utils/array.h"
+#include "utils/pg_lsn.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_meta);
+PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_info);
+
+static void ValidateSnapshotFile(XLogRecPtr lsn, SnapBuildOnDisk *ondisk,
+								 const char *path);
+
+/*
+ * NOTE: For any code change or issue fix here, it is highly recommended to
+ * give a thought about doing the same in SnapBuildRestore() as well.
+ */
+
+/*
+ * Validate the logical snapshot file.
+ */
+static void
+ValidateSnapshotFile(XLogRecPtr lsn, SnapBuildOnDisk *ondisk, const char *path)
+{
+	int			fd;
+	Size		sz;
+	pg_crc32c	checksum;
+	MemoryContext context;
+
+	context = AllocSetContextCreate(CurrentMemoryContext,
+									"logicalsnapshot inspect context",
+									ALLOCSET_DEFAULT_SIZES);
+
+	fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
+
+	if (fd < 0 && errno == ENOENT)
+		ereport(ERROR,
+				errmsg("file \"%s\" does not exist", path));
+	else if (fd < 0)
+		ereport(ERROR,
+				(errcode_for_file_access(),
+				 errmsg("could not open file \"%s\": %m", path)));
+
+	/* ----
+	 * Make sure the snapshot had been stored safely to disk, that's normally
+	 * cheap.
+	 * Note that we do not need PANIC here, nobody will be able to use the
+	 * slot without fsyncing, and saving it won't succeed without an fsync()
+	 * either...
+	 * ----
+	 */
+	fsync_fname(path, false);
+	fsync_fname("pg_logical/snapshots", true);
+
+
+	/* read statically sized portion of snapshot */
+	SnapBuildRestoreContents(fd, (char *) ondisk, SnapBuildOnDiskConstantSize, path);
+
+	if (ondisk->magic != SNAPBUILD_MAGIC)
+		ereport(ERROR,
+				(errcode(ERRCODE_DATA_CORRUPTED),
+				 errmsg("snapbuild state file \"%s\" has wrong magic number: %u instead of %u",
+						path, ondisk->magic, SNAPBUILD_MAGIC)));
+
+	if (ondisk->version != SNAPBUILD_VERSION)
+		ereport(ERROR,
+				(errcode(ERRCODE_DATA_CORRUPTED),
+				 errmsg("snapbuild state file \"%s\" has unsupported version: %u instead of %u",
+						path, ondisk->version, SNAPBUILD_VERSION)));
+
+	INIT_CRC32C(checksum);
+	COMP_CRC32C(checksum,
+				((char *) ondisk) + SnapBuildOnDiskNotChecksummedSize,
+				SnapBuildOnDiskConstantSize - SnapBuildOnDiskNotChecksummedSize);
+
+	/* read SnapBuild */
+	SnapBuildRestoreContents(fd, (char *) &ondisk->builder, sizeof(SnapBuild), path);
+	COMP_CRC32C(checksum, &ondisk->builder, sizeof(SnapBuild));
+
+	ondisk->builder.context = context;
+
+	/* restore committed xacts information */
+	if (ondisk->builder.committed.xcnt > 0)
+	{
+		sz = sizeof(TransactionId) * ondisk->builder.committed.xcnt;
+		ondisk->builder.committed.xip = MemoryContextAllocZero(ondisk->builder.context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.committed.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.committed.xip, sz);
+	}
+
+	/* restore catalog modifying xacts information */
+	if (ondisk->builder.catchange.xcnt > 0)
+	{
+		sz = sizeof(TransactionId) * ondisk->builder.catchange.xcnt;
+		ondisk->builder.catchange.xip = MemoryContextAllocZero(ondisk->builder.context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.catchange.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.catchange.xip, sz);
+	}
+
+	if (CloseTransientFile(fd) != 0)
+		ereport(ERROR,
+				(errcode_for_file_access(),
+				 errmsg("could not close file \"%s\": %m", path)));
+
+	FIN_CRC32C(checksum);
+
+	/* verify checksum of what we've read */
+	if (!EQ_CRC32C(checksum, ondisk->checksum))
+		ereport(ERROR,
+				(errcode(ERRCODE_DATA_CORRUPTED),
+				 errmsg("checksum mismatch for snapbuild state file \"%s\": is %u, should be %u",
+						path, checksum, ondisk->checksum)));
+}
+
+/*
+ * Retrieve the logical snapshot file metadata.
+ */
+Datum
+pg_get_logical_snapshot_meta(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_META_COLS 3
+	SnapBuildOnDisk ondisk;
+	XLogRecPtr	lsn;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_META_COLS];
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_META_COLS];
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+
+	lsn = PG_GETARG_LSN(0);
+
+	sprintf(path, "pg_logical/snapshots/%X-%X.snap",
+			LSN_FORMAT_ARGS(lsn));
+
+	ValidateSnapshotFile(lsn, &ondisk, path);
+
+	/* Build a tuple descriptor for our result type. */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	memset(nulls, 0, sizeof(nulls));
+
+	values[0] = Int32GetDatum(ondisk.magic);
+	values[1] = Int32GetDatum(ondisk.checksum);
+	values[2] = Int32GetDatum(ondisk.version);
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	MemoryContextReset(ondisk.builder.context);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_META_COLS
+}
+
+Datum
+pg_get_logical_snapshot_info(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_INFO_COLS 14
+	SnapBuildOnDisk ondisk;
+	XLogRecPtr	lsn;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS];
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS];
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+
+	lsn = PG_GETARG_LSN(0);
+
+	sprintf(path, "pg_logical/snapshots/%X-%X.snap",
+			LSN_FORMAT_ARGS(lsn));
+
+	ValidateSnapshotFile(lsn, &ondisk, path);
+
+	/* Build a tuple descriptor for our result type. */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	memset(nulls, 0, sizeof(nulls));
+
+	values[0] = Int16GetDatum(ondisk.builder.state);
+	values[1] = TransactionIdGetDatum(ondisk.builder.xmin);
+	values[2] = TransactionIdGetDatum(ondisk.builder.xmax);
+	values[3] = LSNGetDatum(ondisk.builder.start_decoding_at);
+	values[4] = LSNGetDatum(ondisk.builder.two_phase_at);
+	values[5] = TransactionIdGetDatum(ondisk.builder.initial_xmin_horizon);
+	values[6] = BoolGetDatum(ondisk.builder.building_full_snapshot);
+	values[7] = BoolGetDatum(ondisk.builder.in_slot_creation);
+	values[8] = LSNGetDatum(ondisk.builder.last_serialized_snapshot);
+	values[9] = TransactionIdGetDatum(ondisk.builder.next_phase_at);
+	values[10] = Int64GetDatum(ondisk.builder.committed.xcnt);
+
+	if (ondisk.builder.committed.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+		int			narrayelems;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.committed.xcnt * sizeof(Datum));
+		narrayelems = 0;
+
+		for (narrayelems = 0; narrayelems < ondisk.builder.committed.xcnt; narrayelems++)
+			arrayelems[narrayelems] = Int64GetDatum((int64) ondisk.builder.committed.xip[narrayelems]);
+
+		values[11] = PointerGetDatum(construct_array_builtin(arrayelems, narrayelems, INT8OID));
+	}
+	else
+		nulls[11] = true;
+
+	values[12] = Int64GetDatum(ondisk.builder.catchange.xcnt);
+
+	if (ondisk.builder.catchange.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+		int			narrayelems;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.catchange.xcnt * sizeof(Datum));
+		narrayelems = 0;
+
+		for (narrayelems = 0; narrayelems < ondisk.builder.catchange.xcnt; narrayelems++)
+			arrayelems[narrayelems] = Int64GetDatum((int64) ondisk.builder.catchange.xip[narrayelems]);
+
+		values[13] = PointerGetDatum(construct_array_builtin(arrayelems, narrayelems, INT8OID));
+	}
+	else
+		nulls[13] = true;
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	MemoryContextReset(ondisk.builder.context);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_INFO_COLS
+}
diff --git a/contrib/pg_logicalsnapinspect/pg_logicalsnapinspect.control b/contrib/pg_logicalsnapinspect/pg_logicalsnapinspect.control
new file mode 100644
index 0000000000..b366ccb10c
--- /dev/null
+++ b/contrib/pg_logicalsnapinspect/pg_logicalsnapinspect.control
@@ -0,0 +1,5 @@
+# pg_logicalsnapinspect extension
+comment = 'functions to inspect contents of logical snapshot'
+default_version = '1.0'
+module_pathname = '$libdir/pg_logicalsnapinspect'
+relocatable = true
diff --git a/contrib/pg_logicalsnapinspect/specs/logical_snapshot_inspect.spec b/contrib/pg_logicalsnapinspect/specs/logical_snapshot_inspect.spec
new file mode 100644
index 0000000000..6fd2c338ca
--- /dev/null
+++ b/contrib/pg_logicalsnapinspect/specs/logical_snapshot_inspect.spec
@@ -0,0 +1,34 @@
+# Test the pg_logicalsnapinspect functions: that needs some permutation to
+# ensure that we are creating multiple logical snapshots and that one of them
+# contains ongoing catalogs changes.
+setup
+{
+    DROP TABLE IF EXISTS tbl1;
+    CREATE TABLE tbl1 (val1 integer, val2 integer);
+	CREATE EXTENSION pg_logicalsnapinspect;
+}
+
+teardown
+{
+    DROP TABLE tbl1;
+    SELECT 'stop' FROM pg_drop_replication_slot('isolation_slot');
+	DROP EXTENSION pg_logicalsnapinspect;
+}
+
+session "s0"
+setup { SET synchronous_commit=on; }
+step "s0_init" { SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding'); }
+step "s0_begin" { BEGIN; }
+step "s0_savepoint" { SAVEPOINT sp1; }
+step "s0_truncate" { TRUNCATE tbl1; }
+step "s0_insert" { INSERT INTO tbl1 VALUES (1); }
+step "s0_commit" { COMMIT; }
+
+session "s1"
+setup { SET synchronous_commit=on; }
+step "s1_checkpoint" { CHECKPOINT; }
+step "s1_get_changes" { SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0'); }
+step "s1_get_logical_snapshot_meta" { SELECT COUNT((pg_get_logical_snapshot_meta(f.name::pg_lsn))) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f; }
+step "s1_get_logical_snapshot_info" { SELECT (pg_get_logical_snapshot_info(f.name::pg_lsn)).state,(pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_xip,1),(pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_xip,1) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f ORDER BY 2; }
+
+permutation "s0_init" "s0_begin" "s0_savepoint" "s0_truncate" "s1_checkpoint" "s1_get_changes" "s0_commit" "s0_begin" "s0_insert" "s1_checkpoint" "s1_get_changes" "s0_commit" "s1_get_changes" "s1_get_logical_snapshot_info" "s1_get_logical_snapshot_meta"
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index 44639a8dca..f7b1cd85ee 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -154,6 +154,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
  &pgbuffercache;
  &pgcrypto;
  &pgfreespacemap;
+ &pglogicalsnapinspect;
  &pgprewarm;
  &pgrowlocks;
  &pgstatstatements;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index a7ff5f8264..94b650915d 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -143,6 +143,7 @@
 <!ENTITY pgbuffercache   SYSTEM "pgbuffercache.sgml">
 <!ENTITY pgcrypto        SYSTEM "pgcrypto.sgml">
 <!ENTITY pgfreespacemap  SYSTEM "pgfreespacemap.sgml">
+<!ENTITY pglogicalsnapinspect  SYSTEM "pglogicalsnapinspect.sgml">
 <!ENTITY pgprewarm       SYSTEM "pgprewarm.sgml">
 <!ENTITY pgrowlocks      SYSTEM "pgrowlocks.sgml">
 <!ENTITY pgstatstatements SYSTEM "pgstatstatements.sgml">
diff --git a/doc/src/sgml/pglogicalsnapinspect.sgml b/doc/src/sgml/pglogicalsnapinspect.sgml
new file mode 100644
index 0000000000..5e005ab124
--- /dev/null
+++ b/doc/src/sgml/pglogicalsnapinspect.sgml
@@ -0,0 +1,144 @@
+<!-- doc/src/sgml/pglogicalsnapinspect.sgml -->
+
+<sect1 id="pglogicalsnapinspect" xreflabel="pg_logicalsnapinspect">
+ <title>pg_logicalsnapinspect &mdash; logical snapshot inspection</title>
+
+ <indexterm zone="pglogicalsnapinspect">
+  <primary>pg_logicalsnapinspect</primary>
+ </indexterm>
+
+ <para>
+  The <filename>pg_logicalsnapinspect</filename> module provides SQL functions
+  that allow you to inspect the contents of serialized logical snapshots of a
+  running <productname>PostgreSQL</productname> database cluster, which is useful
+  for debugging or educational purposes.
+ </para>
+
+ <note>
+  <para>
+   The <filename>pg_logicalsnapinspect</filename> functions are called
+   using an LSN argument that can be extracted from the output name of the
+   <function>pg_ls_logicalsnapdir</function>() function.
+  </para>
+ </note>
+
+ <sect2 id="pglogicalsnapinspect-funcs">
+  <title>General Functions</title>
+
+  <variablelist>
+   <varlistentry id="pglogicalsnapinspect-funcs-pg-get-logical-snapshot-meta">
+    <term>
+     <function>pg_get_logical_snapshot_meta(in_lsn pg_lsn) returns record</function>
+    </term>
+
+    <listitem>
+     <para>
+      Gets logical snapshot metadata about a snapshot file that is located in
+      the <filename>pg_logical/snapshots</filename> directory.
+      The <replaceable>in_lsn</replaceable> argument can be extracted from the
+      snapshot file name.
+      example:
+<screen>
+postgres=# SELECT * FROM pg_ls_logicalsnapdir();
+-[ RECORD 1 ]+-----------------------
+name         | 0-40796E18.snap
+size         | 152
+modification | 2024-08-14 16:36:32+00
+
+postgres=# SELECT * FROM pg_get_logical_snapshot_meta('0/40796E18');
+-[ RECORD 1 ]--------
+magic    | 1369563137
+checksum | 1028045905
+version  | 6
+
+postgres=# SELECT (pg_get_logical_snapshot_meta(f.name::pg_lsn)).*
+           FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name
+                 FROM pg_ls_logicalsnapdir()) AS f;
+-[ RECORD 1 ]--------
+magic    | 1369563137
+checksum | 1028045905
+version  | 6
+</screen>
+     </para>
+     <para>
+      If <replaceable>in_lsn</replaceable> does not match a snapshot file, the
+      function raises an error.
+     </para>
+    </listitem>
+   </varlistentry>
+
+   <varlistentry id="pglogicalsnapinspect-funcs-pg-get-logical-snapshot-info">
+    <term>
+     <function>pg_get_logical_snapshot_info(in_lsn pg_lsn) returns record</function>
+    </term>
+
+    <listitem>
+     <para>
+      Gets logical snapshot information about a snapshot file that is located in
+      the <filename>pg_logical/snapshots</filename> directory.
+      The <replaceable>in_lsn</replaceable> argument can be extracted from the
+      snapshot file name.
+      example:
+<screen>
+postgres=# SELECT * FROM pg_ls_logicalsnapdir();
+-[ RECORD 1 ]+-----------------------
+name         | 0-40796E18.snap
+size         | 152
+modification | 2024-08-14 16:36:32+00
+
+postgres=# SELECT * FROM pg_get_logical_snapshot_info('0/40796E18');
+-[ RECORD 1 ]------------+-----------
+state                    | 2
+xmin                     | 751
+xmax                     | 751
+start_decoding_at        | 0/40796AF8
+two_phase_at             | 0/40796AF8
+initial_xmin_horizon     | 0
+building_full_snapshot   | f
+in_slot_creation         | f
+last_serialized_snapshot | 0/0
+next_phase_at            | 0
+committed_count          | 0
+committed_xip            |
+catchange_count          | 2
+catchange_xip            | {751,752}
+
+postgres=# SELECT (pg_get_logical_snapshot_info(f.name::pg_lsn)).*
+           FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name
+                 FROM pg_ls_logicalsnapdir()) AS f;
+-[ RECORD 1 ]------------+-----------
+state                    | 2
+xmin                     | 751
+xmax                     | 751
+start_decoding_at        | 0/40796AF8
+two_phase_at             | 0/40796AF8
+initial_xmin_horizon     | 0
+building_full_snapshot   | f
+in_slot_creation         | f
+last_serialized_snapshot | 0/0
+next_phase_at            | 0
+committed_count          | 0
+committed_xip            |
+catchange_count          | 2
+catchange_xip            | {751,752}
+</screen>
+     </para>
+     <para>
+      If <replaceable>in_lsn</replaceable> does not match a snapshot file, the
+      function raises an error.
+     </para>
+    </listitem>
+   </varlistentry>
+
+  </variablelist>
+ </sect2>
+
+ <sect2 id="pglogicalsnapinspect-author">
+  <title>Author</title>
+
+  <para>
+   Bertrand Drouvot <email>bertranddrouvot.pg@gmail.com</email>
+  </para>
+ </sect2>
+
+</sect1>
diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c
index ae676145e6..b9b8e894b6 100644
--- a/src/backend/replication/logical/snapbuild.c
+++ b/src/backend/replication/logical/snapbuild.c
@@ -143,146 +143,6 @@
 #include "utils/memutils.h"
 #include "utils/snapmgr.h"
 #include "utils/snapshot.h"
-
-/*
- * This struct contains the current state of the snapshot building
- * machinery. Besides a forward declaration in the header, it is not exposed
- * to the public, so we can easily change its contents.
- */
-struct SnapBuild
-{
-	/* how far are we along building our first full snapshot */
-	SnapBuildState state;
-
-	/* private memory context used to allocate memory for this module. */
-	MemoryContext context;
-
-	/* all transactions < than this have committed/aborted */
-	TransactionId xmin;
-
-	/* all transactions >= than this are uncommitted */
-	TransactionId xmax;
-
-	/*
-	 * Don't replay commits from an LSN < this LSN. This can be set externally
-	 * but it will also be advanced (never retreat) from within snapbuild.c.
-	 */
-	XLogRecPtr	start_decoding_at;
-
-	/*
-	 * LSN at which two-phase decoding was enabled or LSN at which we found a
-	 * consistent point at the time of slot creation.
-	 *
-	 * The prepared transactions, that were skipped because previously
-	 * two-phase was not enabled or are not covered by initial snapshot, need
-	 * to be sent later along with commit prepared and they must be before
-	 * this point.
-	 */
-	XLogRecPtr	two_phase_at;
-
-	/*
-	 * Don't start decoding WAL until the "xl_running_xacts" information
-	 * indicates there are no running xids with an xid smaller than this.
-	 */
-	TransactionId initial_xmin_horizon;
-
-	/* Indicates if we are building full snapshot or just catalog one. */
-	bool		building_full_snapshot;
-
-	/*
-	 * Indicates if we are using the snapshot builder for the creation of a
-	 * logical replication slot. If it's true, the start point for decoding
-	 * changes is not determined yet. So we skip snapshot restores to properly
-	 * find the start point. See SnapBuildFindSnapshot() for details.
-	 */
-	bool		in_slot_creation;
-
-	/*
-	 * Snapshot that's valid to see the catalog state seen at this moment.
-	 */
-	Snapshot	snapshot;
-
-	/*
-	 * LSN of the last location we are sure a snapshot has been serialized to.
-	 */
-	XLogRecPtr	last_serialized_snapshot;
-
-	/*
-	 * The reorderbuffer we need to update with usable snapshots et al.
-	 */
-	ReorderBuffer *reorder;
-
-	/*
-	 * TransactionId at which the next phase of initial snapshot building will
-	 * happen. InvalidTransactionId if not known (i.e. SNAPBUILD_START), or
-	 * when no next phase necessary (SNAPBUILD_CONSISTENT).
-	 */
-	TransactionId next_phase_at;
-
-	/*
-	 * Array of transactions which could have catalog changes that committed
-	 * between xmin and xmax.
-	 */
-	struct
-	{
-		/* number of committed transactions */
-		size_t		xcnt;
-
-		/* available space for committed transactions */
-		size_t		xcnt_space;
-
-		/*
-		 * Until we reach a CONSISTENT state, we record commits of all
-		 * transactions, not just the catalog changing ones. Record when that
-		 * changes so we know we cannot export a snapshot safely anymore.
-		 */
-		bool		includes_all_transactions;
-
-		/*
-		 * Array of committed transactions that have modified the catalog.
-		 *
-		 * As this array is frequently modified we do *not* keep it in
-		 * xidComparator order. Instead we sort the array when building &
-		 * distributing a snapshot.
-		 *
-		 * TODO: It's unclear whether that reasoning has much merit. Every
-		 * time we add something here after becoming consistent will also
-		 * require distributing a snapshot. Storing them sorted would
-		 * potentially also make it easier to purge (but more complicated wrt
-		 * wraparound?). Should be improved if sorting while building the
-		 * snapshot shows up in profiles.
-		 */
-		TransactionId *xip;
-	}			committed;
-
-	/*
-	 * Array of transactions and subtransactions that had modified catalogs
-	 * and were running when the snapshot was serialized.
-	 *
-	 * We normally rely on some WAL record types such as HEAP2_NEW_CID to know
-	 * if the transaction has changed the catalog. But it could happen that
-	 * the logical decoding decodes only the commit record of the transaction
-	 * after restoring the previously serialized snapshot in which case we
-	 * will miss adding the xid to the snapshot and end up looking at the
-	 * catalogs with the wrong snapshot.
-	 *
-	 * Now to avoid the above problem, we serialize the transactions that had
-	 * modified the catalogs and are still running at the time of snapshot
-	 * serialization. We fill this array while restoring the snapshot and then
-	 * refer it while decoding commit to ensure if the xact has modified the
-	 * catalog. We discard this array when all the xids in the list become old
-	 * enough to matter. See SnapBuildPurgeOlderTxn for details.
-	 */
-	struct
-	{
-		/* number of transactions */
-		size_t		xcnt;
-
-		/* This array must be sorted in xidComparator order */
-		TransactionId *xip;
-	}			catchange;
-};
-
 /*
  * Starting a transaction -- which we need to do while exporting a snapshot --
  * removes knowledge about the previously used resowner, so we save it here.
@@ -312,7 +172,6 @@ static void SnapBuildWaitSnapshot(xl_running_xacts *running, TransactionId cutof
 /* serialization functions */
 static void SnapBuildSerialize(SnapBuild *builder, XLogRecPtr lsn);
 static bool SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn);
-static void SnapBuildRestoreContents(int fd, char *dest, Size size, const char *path);
 
 /*
  * Allocate a new snapshot builder.
@@ -1557,48 +1416,6 @@ SnapBuildWaitSnapshot(xl_running_xacts *running, TransactionId cutoff)
 	}
 }
 
-/* -----------------------------------
- * Snapshot serialization support
- * -----------------------------------
- */
-
-/*
- * We store current state of struct SnapBuild on disk in the following manner:
- *
- * struct SnapBuildOnDisk;
- * TransactionId * committed.xcnt; (*not xcnt_space*)
- * TransactionId * catchange.xcnt;
- *
- */
-typedef struct SnapBuildOnDisk
-{
-	/* first part of this struct needs to be version independent */
-
-	/* data not covered by checksum */
-	uint32		magic;
-	pg_crc32c	checksum;
-
-	/* data covered by checksum */
-
-	/* version, in case we want to support pg_upgrade */
-	uint32		version;
-	/* how large is the on disk data, excluding the constant sized part */
-	uint32		length;
-
-	/* version dependent part */
-	SnapBuild	builder;
-
-	/* variable amount of TransactionIds follows */
-} SnapBuildOnDisk;
-
-#define SnapBuildOnDiskConstantSize \
-	offsetof(SnapBuildOnDisk, builder)
-#define SnapBuildOnDiskNotChecksummedSize \
-	offsetof(SnapBuildOnDisk, version)
-
-#define SNAPBUILD_MAGIC 0x51A1E001
-#define SNAPBUILD_VERSION 6
-
 /*
  * Store/Load a snapshot from disk, depending on the snapshot builder's state.
  *
@@ -1857,6 +1674,10 @@ out:
 /*
  * Restore a snapshot into 'builder' if previously one has been stored at the
  * location indicated by 'lsn'. Returns true if successful, false otherwise.
+ *
+ * NOTE: For any code change or issue fix here, it is highly recommended to
+ * give a thought about doing the same in pg_logicalsnapinspect contrib module
+ * as well.
  */
 static bool
 SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
@@ -2030,7 +1851,7 @@ snapshot_not_interesting:
 /*
  * Read the contents of the serialized snapshot to 'dest'.
  */
-static void
+void
 SnapBuildRestoreContents(int fd, char *dest, Size size, const char *path)
 {
 	int			readBytes;
diff --git a/src/include/port/pg_crc32c.h b/src/include/port/pg_crc32c.h
index 63c8e3a00b..cfc8c07944 100644
--- a/src/include/port/pg_crc32c.h
+++ b/src/include/port/pg_crc32c.h
@@ -47,7 +47,7 @@ typedef uint32 pg_crc32c;
 	((crc) = pg_comp_crc32c_sse42((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
 
 #elif defined(USE_ARMV8_CRC32C)
 /* Use ARMv8 CRC Extension instructions. */
@@ -56,7 +56,7 @@ extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t le
 	((crc) = pg_comp_crc32c_armv8((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
 
 #elif defined(USE_LOONGARCH_CRC32C)
 /* Use LoongArch CRCC instructions. */
@@ -65,7 +65,7 @@ extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t le
 	((crc) = pg_comp_crc32c_loongarch((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len);
 
 #elif defined(USE_SSE42_CRC32C_WITH_RUNTIME_CHECK) || defined(USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK)
 
@@ -77,14 +77,14 @@ extern pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_
 	((crc) = pg_comp_crc32c((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
-extern pg_crc32c (*pg_comp_crc32c) (pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c (*pg_comp_crc32c) (pg_crc32c crc, const void *data, size_t len);
 
 #ifdef USE_SSE42_CRC32C_WITH_RUNTIME_CHECK
-extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
 #endif
 #ifdef USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK
-extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
 #endif
 
 #else
@@ -103,7 +103,7 @@ extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t le
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 #endif
 
-extern pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
 
 #endif
 
diff --git a/src/include/replication/snapbuild.h b/src/include/replication/snapbuild.h
index caa5113ff8..a4617c5197 100644
--- a/src/include/replication/snapbuild.h
+++ b/src/include/replication/snapbuild.h
@@ -13,8 +13,22 @@
 #define SNAPBUILD_H
 
 #include "access/xlogdefs.h"
+#include "replication/reorderbuffer.h"
 #include "utils/snapmgr.h"
 
+/* -----------------------------------
+ * Snapshot serialization support
+ * -----------------------------------
+ */
+
+#define SnapBuildOnDiskConstantSize \
+	offsetof(SnapBuildOnDisk, builder)
+#define SnapBuildOnDiskNotChecksummedSize \
+	offsetof(SnapBuildOnDisk, version)
+
+#define SNAPBUILD_MAGIC 0x51A1E001
+#define SNAPBUILD_VERSION 6
+
 typedef enum
 {
 	/*
@@ -46,12 +60,173 @@ typedef enum
 	SNAPBUILD_CONSISTENT = 2,
 } SnapBuildState;
 
-/* forward declare so we don't have to expose the struct to the public */
-struct SnapBuild;
-typedef struct SnapBuild SnapBuild;
+/*
+ * This struct contains the current state of the snapshot building
+ * machinery. It is exposed to the public, so pay attention when changing its
+ * contents.
+ */
+typedef struct SnapBuild
+{
+	/* how far are we along building our first full snapshot */
+	SnapBuildState state;
+
+	/* private memory context used to allocate memory for this module. */
+	MemoryContext context;
+
+	/* all transactions < than this have committed/aborted */
+	TransactionId xmin;
+
+	/* all transactions >= than this are uncommitted */
+	TransactionId xmax;
+
+	/*
+	 * Don't replay commits from an LSN < this LSN. This can be set externally
+	 * but it will also be advanced (never retreat) from within snapbuild.c.
+	 */
+	XLogRecPtr	start_decoding_at;
+
+	/*
+	 * LSN at which two-phase decoding was enabled or LSN at which we found a
+	 * consistent point at the time of slot creation.
+	 *
+	 * The prepared transactions, that were skipped because previously
+	 * two-phase was not enabled or are not covered by initial snapshot, need
+	 * to be sent later along with commit prepared and they must be before
+	 * this point.
+	 */
+	XLogRecPtr	two_phase_at;
+
+	/*
+	 * Don't start decoding WAL until the "xl_running_xacts" information
+	 * indicates there are no running xids with an xid smaller than this.
+	 */
+	TransactionId initial_xmin_horizon;
+
+	/* Indicates if we are building full snapshot or just catalog one. */
+	bool		building_full_snapshot;
+
+	/*
+	 * Indicates if we are using the snapshot builder for the creation of a
+	 * logical replication slot. If it's true, the start point for decoding
+	 * changes is not determined yet. So we skip snapshot restores to properly
+	 * find the start point. See SnapBuildFindSnapshot() for details.
+	 */
+	bool		in_slot_creation;
+
+	/*
+	 * Snapshot that's valid to see the catalog state seen at this moment.
+	 */
+	Snapshot	snapshot;
+
+	/*
+	 * LSN of the last location we are sure a snapshot has been serialized to.
+	 */
+	XLogRecPtr	last_serialized_snapshot;
+
+	/*
+	 * The reorderbuffer we need to update with usable snapshots et al.
+	 */
+	ReorderBuffer *reorder;
+
+	/*
+	 * TransactionId at which the next phase of initial snapshot building will
+	 * happen. InvalidTransactionId if not known (i.e. SNAPBUILD_START), or
+	 * when no next phase necessary (SNAPBUILD_CONSISTENT).
+	 */
+	TransactionId next_phase_at;
+
+	/*
+	 * Array of transactions which could have catalog changes that committed
+	 * between xmin and xmax.
+	 */
+	struct
+	{
+		/* number of committed transactions */
+		size_t		xcnt;
+
+		/* available space for committed transactions */
+		size_t		xcnt_space;
+
+		/*
+		 * Until we reach a CONSISTENT state, we record commits of all
+		 * transactions, not just the catalog changing ones. Record when that
+		 * changes so we know we cannot export a snapshot safely anymore.
+		 */
+		bool		includes_all_transactions;
+
+		/*
+		 * Array of committed transactions that have modified the catalog.
+		 *
+		 * As this array is frequently modified we do *not* keep it in
+		 * xidComparator order. Instead we sort the array when building &
+		 * distributing a snapshot.
+		 *
+		 * TODO: It's unclear whether that reasoning has much merit. Every
+		 * time we add something here after becoming consistent will also
+		 * require distributing a snapshot. Storing them sorted would
+		 * potentially also make it easier to purge (but more complicated wrt
+		 * wraparound?). Should be improved if sorting while building the
+		 * snapshot shows up in profiles.
+		 */
+		TransactionId *xip;
+	}			committed;
+
+	/*
+	 * Array of transactions and subtransactions that had modified catalogs
+	 * and were running when the snapshot was serialized.
+	 *
+	 * We normally rely on some WAL record types such as HEAP2_NEW_CID to know
+	 * if the transaction has changed the catalog. But it could happen that
+	 * the logical decoding decodes only the commit record of the transaction
+	 * after restoring the previously serialized snapshot in which case we
+	 * will miss adding the xid to the snapshot and end up looking at the
+	 * catalogs with the wrong snapshot.
+	 *
+	 * Now to avoid the above problem, we serialize the transactions that had
+	 * modified the catalogs and are still running at the time of snapshot
+	 * serialization. We fill this array while restoring the snapshot and then
+	 * refer it while decoding commit to ensure if the xact has modified the
+	 * catalog. We discard this array when all the xids in the list become old
+	 * enough to matter. See SnapBuildPurgeOlderTxn for details.
+	 */
+	struct
+	{
+		/* number of transactions */
+		size_t		xcnt;
+
+		/* This array must be sorted in xidComparator order */
+		TransactionId *xip;
+	}			catchange;
+} SnapBuild;
+
+/*
+ * We store current state of struct SnapBuild on disk in the following manner:
+ *
+ * struct SnapBuildOnDisk;
+ * TransactionId * committed.xcnt; (*not xcnt_space*)
+ * TransactionId * catchange.xcnt;
+ *
+ */
+typedef struct SnapBuildOnDisk
+{
+	/* first part of this struct needs to be version independent */
+
+	/* data not covered by checksum */
+	uint32		magic;
+	pg_crc32c	checksum;
+
+	/* data covered by checksum */
+
+	/* version, in case we want to support pg_upgrade */
+	uint32		version;
+	/* how large is the on disk data, excluding the constant sized part */
+	uint32		length;
+
+	/* version dependent part */
+	SnapBuild	builder;
 
-/* forward declare so we don't have to include reorderbuffer.h */
-struct ReorderBuffer;
+	/* variable amount of TransactionIds follows */
+} SnapBuildOnDisk;
 
 /* forward declare so we don't have to include heapam_xlog.h */
 struct xl_heap_new_cid;
@@ -94,4 +269,5 @@ extern void SnapBuildSerializationPoint(SnapBuild *builder, XLogRecPtr lsn);
 
 extern bool SnapBuildSnapshotExists(XLogRecPtr lsn);
 
+extern void SnapBuildRestoreContents(int fd, char *dest, Size size, const char *path);
 #endif							/* SNAPBUILD_H */
-- 
2.34.1

#2Amit Kapila
amit.kapila16@gmail.com
In reply to: Bertrand Drouvot (#1)
Re: Add contrib/pg_logicalsnapinspect

On Thu, Aug 22, 2024 at 5:56 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Please find attached a patch to $SUBJECT.

This module provides SQL functions to inspect the contents of serialized logical
snapshots of a running database cluster, which I think could be useful for
debugging or educational purposes.

+1. I see it could be a good debugging aid.

2. The SnapBuildOnDisk and SnapBuild structs are now exposed to public. Means
we should now pay much more attention when changing their contents but I think
it's worth it.

Is it possible to avoid exposing these structures? Can we expose some
function from snapbuild.c that provides the required information?

--
With Regards,
Amit Kapila.

#3Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: Amit Kapila (#2)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Mon, Aug 26, 2024 at 07:05:27PM +0530, Amit Kapila wrote:

On Thu, Aug 22, 2024 at 5:56 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Please find attached a patch to $SUBJECT.

This module provides SQL functions to inspect the contents of serialized logical
snapshots of a running database cluster, which I think could be useful for
debugging or educational purposes.

+1. I see it could be a good debugging aid.

Thanks for the feedback!

2. The SnapBuildOnDisk and SnapBuild structs are now exposed to public. Means
we should now pay much more attention when changing their contents but I think
it's worth it.

Is it possible to avoid exposing these structures? Can we expose some
function from snapbuild.c that provides the required information?

Yeah, that's an option if we don't want to expose those structs to public.

I think we could 1/ create a function that would return a formed HeapTuple, or
2/ we could create multiple functions (about 15) that would return the values
we are interested in.

I think 2/ is fine as it would give more flexiblity (no need to retrieve a whole
tuple if one is interested to only one value).

What do you think? Did you have something else in mind?

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

#4Amit Kapila
amit.kapila16@gmail.com
In reply to: Bertrand Drouvot (#3)
Re: Add contrib/pg_logicalsnapinspect

On Wed, Aug 28, 2024 at 1:25 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

On Mon, Aug 26, 2024 at 07:05:27PM +0530, Amit Kapila wrote:

On Thu, Aug 22, 2024 at 5:56 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

2. The SnapBuildOnDisk and SnapBuild structs are now exposed to public. Means
we should now pay much more attention when changing their contents but I think
it's worth it.

Is it possible to avoid exposing these structures? Can we expose some
function from snapbuild.c that provides the required information?

Yeah, that's an option if we don't want to expose those structs to public.

I think we could 1/ create a function that would return a formed HeapTuple, or
2/ we could create multiple functions (about 15) that would return the values
we are interested in.

I think 2/ is fine as it would give more flexiblity (no need to retrieve a whole
tuple if one is interested to only one value).

True, but OTOH, each time we add a new field to these structures, a
new function has to be exposed. I don't have a strong opinion on this
but seeing current use cases, it seems okay to expose a single
function.

What do you think? Did you have something else in mind?

On similar lines, we can also provide a function to get the slot's
on-disk data. IIRC, Bharath had previously proposed a tool to achieve
the same. It is fine if we don't want to add that as part of this
patch but I mentioned it because by having that we can have a set of
functions to view logical decoding data.

--
With Regards,
Amit Kapila.

#5Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: Amit Kapila (#4)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Thu, Aug 29, 2024 at 02:51:36PM +0530, Amit Kapila wrote:

On Wed, Aug 28, 2024 at 1:25 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

On Mon, Aug 26, 2024 at 07:05:27PM +0530, Amit Kapila wrote:

On Thu, Aug 22, 2024 at 5:56 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

2. The SnapBuildOnDisk and SnapBuild structs are now exposed to public. Means
we should now pay much more attention when changing their contents but I think
it's worth it.

Is it possible to avoid exposing these structures? Can we expose some
function from snapbuild.c that provides the required information?

Yeah, that's an option if we don't want to expose those structs to public.

I think we could 1/ create a function that would return a formed HeapTuple, or
2/ we could create multiple functions (about 15) that would return the values
we are interested in.

I think 2/ is fine as it would give more flexiblity (no need to retrieve a whole
tuple if one is interested to only one value).

True, but OTOH, each time we add a new field to these structures, a
new function has to be exposed. I don't have a strong opinion on this
but seeing current use cases, it seems okay to expose a single
function.

Yeah that's fair. And now I'm wondering if we need an extra module. I think
we could "simply" expose 2 new functions in core, thoughts?

What do you think? Did you have something else in mind?

On similar lines, we can also provide a function to get the slot's
on-disk data.

Yeah, having a way to expose the data from the disk makes fully sense to me.

IIRC, Bharath had previously proposed a tool to achieve
the same. It is fine if we don't want to add that as part of this
patch but I mentioned it because by having that we can have a set of
functions to view logical decoding data.

That's right. I think this one would be simply enough to expose one or two
functions in core too (and probably would not need an extra module).

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

#6Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: Bertrand Drouvot (#5)
Re: Add contrib/pg_logicalsnapinspect

On Thu, Aug 29, 2024 at 3:44 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Yeah that's fair. And now I'm wondering if we need an extra module. I think
we could "simply" expose 2 new functions in core, thoughts?

What do you think? Did you have something else in mind?

On similar lines, we can also provide a function to get the slot's
on-disk data.

Yeah, having a way to expose the data from the disk makes fully sense to me.

IIRC, Bharath had previously proposed a tool to achieve
the same. It is fine if we don't want to add that as part of this
patch but I mentioned it because by having that we can have a set of
functions to view logical decoding data.

That's right. I think this one would be simply enough to expose one or two
functions in core too (and probably would not need an extra module).

+1 for functions in core unless this extra module
pg_logicalsnapinspect works as a tool to be helpful even when the
server is down.

FWIW, I wrote pg_replslotdata as a tool, not as an extension for
reading on-disk replication slot data to help when the server is down
- /messages/by-id/CALj2ACW0rV5gWK8A3m6_X62qH+Vfaq5hznC=i0R5Wojt5+yhyw@mail.gmail.com.
When the server is running, pg_get_replication_slots() pretty much
gives the on-disk contents.

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

#7Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: Bharath Rupireddy (#6)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Thu, Aug 29, 2024 at 06:33:19PM +0530, Bharath Rupireddy wrote:

On Thu, Aug 29, 2024 at 3:44 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

That's right. I think this one would be simply enough to expose one or two
functions in core too (and probably would not need an extra module).

+1 for functions in core unless this extra module
pg_logicalsnapinspect works as a tool to be helpful even when the
server is down.

Thanks for the feedback!

I don't see any use case where it could be useful when the server is down. So,
I think I'll move forward with in core functions (unless someone has a different
opinion).

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

#8Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: Bertrand Drouvot (#7)
1 attachment(s)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Thu, Aug 29, 2024 at 02:15:47PM +0000, Bertrand Drouvot wrote:

I don't see any use case where it could be useful when the server is down. So,
I think I'll move forward with in core functions (unless someone has a different
opinion).

Please find v2 attached that creates the 2 new in core functions.

Note that once those new functions are in (or maybe sooner), I'll submit an
additional patch to get rid of the code duplication between the new
ValidateSnapshotFile() and SnapBuildRestore().

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachments:

v2-0001-Functions-to-get-ondisk-logical-snapshots-details.patchtext/x-diff; charset=us-asciiDownload
From 6c5a1ad66b203036739aae955932b8e3813c71e3 Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Thu, 29 Aug 2024 20:51:31 +0000
Subject: [PATCH v2] Functions to get ondisk logical snapshots details

Provides SQL functions that allow to inspect the contents of serialized logical
snapshots of a running database cluster, which is useful for debugging or
educational purposes.
---
 contrib/test_decoding/Makefile                |   2 +-
 .../expected/get_ondisk_snapshot_info.out     |  57 +++++
 contrib/test_decoding/meson.build             |   1 +
 .../specs/get_ondisk_snapshot_info.spec       |  32 +++
 doc/src/sgml/func.sgml                        |  53 +++++
 src/backend/replication/logical/snapbuild.c   | 225 ++++++++++++++++++
 src/include/catalog/pg_proc.dat               |  16 ++
 7 files changed, 385 insertions(+), 1 deletion(-)
  17.3% contrib/test_decoding/expected/
  12.6% contrib/test_decoding/specs/
  17.3% doc/src/sgml/
  45.3% src/backend/replication/logical/
   6.6% src/include/catalog/

diff --git a/contrib/test_decoding/Makefile b/contrib/test_decoding/Makefile
index a4ba1a509a..b1b8ffa9e8 100644
--- a/contrib/test_decoding/Makefile
+++ b/contrib/test_decoding/Makefile
@@ -9,7 +9,7 @@ REGRESS = ddl xact rewrite toast permissions decoding_in_xact \
 ISOLATION = mxact delayed_startup ondisk_startup concurrent_ddl_dml \
 	oldest_xmin snapshot_transfer subxact_without_top concurrent_stream \
 	twophase_snapshot slot_creation_error catalog_change_snapshot \
-	skip_snapshot_restore
+	skip_snapshot_restore get_ondisk_snapshot_info
 
 REGRESS_OPTS = --temp-config $(top_srcdir)/contrib/test_decoding/logical.conf
 ISOLATION_OPTS = --temp-config $(top_srcdir)/contrib/test_decoding/logical.conf
diff --git a/contrib/test_decoding/expected/get_ondisk_snapshot_info.out b/contrib/test_decoding/expected/get_ondisk_snapshot_info.out
new file mode 100644
index 0000000000..b676ccd528
--- /dev/null
+++ b/contrib/test_decoding/expected/get_ondisk_snapshot_info.out
@@ -0,0 +1,57 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s0_init s0_begin s0_savepoint s0_truncate s1_checkpoint s1_get_changes s0_commit s0_begin s0_insert s1_checkpoint s1_get_changes s0_commit s1_get_changes s1_get_logical_snapshot_info s1_get_logical_snapshot_meta
+step s0_init: SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding');
+?column?
+--------
+init    
+(1 row)
+
+step s0_begin: BEGIN;
+step s0_savepoint: SAVEPOINT sp1;
+step s0_truncate: TRUNCATE tbl1;
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data
+----
+(0 rows)
+
+step s0_commit: COMMIT;
+step s0_begin: BEGIN;
+step s0_insert: INSERT INTO tbl1 VALUES (1);
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                   
+---------------------------------------
+BEGIN                                  
+table public.tbl1: TRUNCATE: (no-flags)
+COMMIT                                 
+(3 rows)
+
+step s0_commit: COMMIT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                                         
+-------------------------------------------------------------
+BEGIN                                                        
+table public.tbl1: INSERT: val1[integer]:1 val2[integer]:null
+COMMIT                                                       
+(3 rows)
+
+step s1_get_logical_snapshot_info: SELECT (pg_get_logical_snapshot_info(f.name::pg_lsn)).state,(pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_xip,1),(pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_xip,1) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f ORDER BY 2;
+state|catchange_count|array_length|committed_count|array_length
+-----+---------------+------------+---------------+------------
+    2|              0|            |              2|           2
+    2|              2|           2|              0|            
+(2 rows)
+
+step s1_get_logical_snapshot_meta: SELECT COUNT((pg_get_logical_snapshot_meta(f.name::pg_lsn))) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f;
+count
+-----
+    2
+(1 row)
+
+?column?
+--------
+stop    
+(1 row)
+
diff --git a/contrib/test_decoding/meson.build b/contrib/test_decoding/meson.build
index f643dc81a2..d6e784cbdd 100644
--- a/contrib/test_decoding/meson.build
+++ b/contrib/test_decoding/meson.build
@@ -63,6 +63,7 @@ tests += {
       'twophase_snapshot',
       'slot_creation_error',
       'skip_snapshot_restore',
+      'get_ondisk_snapshot_info',
     ],
     'regress_args': [
       '--temp-config', files('logical.conf'),
diff --git a/contrib/test_decoding/specs/get_ondisk_snapshot_info.spec b/contrib/test_decoding/specs/get_ondisk_snapshot_info.spec
new file mode 100644
index 0000000000..39c2ee1430
--- /dev/null
+++ b/contrib/test_decoding/specs/get_ondisk_snapshot_info.spec
@@ -0,0 +1,32 @@
+# Test the functions that retrieve ondisk logical snapshots informations.
+# That needs some permutation to ensure that we are creating multiple logical
+# snapshots and that one of them contains ongoing catalogs changes.
+setup
+{
+    DROP TABLE IF EXISTS tbl1;
+    CREATE TABLE tbl1 (val1 integer, val2 integer);
+}
+
+teardown
+{
+    DROP TABLE tbl1;
+    SELECT 'stop' FROM pg_drop_replication_slot('isolation_slot');
+}
+
+session "s0"
+setup { SET synchronous_commit=on; }
+step "s0_init" { SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding'); }
+step "s0_begin" { BEGIN; }
+step "s0_savepoint" { SAVEPOINT sp1; }
+step "s0_truncate" { TRUNCATE tbl1; }
+step "s0_insert" { INSERT INTO tbl1 VALUES (1); }
+step "s0_commit" { COMMIT; }
+
+session "s1"
+setup { SET synchronous_commit=on; }
+step "s1_checkpoint" { CHECKPOINT; }
+step "s1_get_changes" { SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0'); }
+step "s1_get_logical_snapshot_meta" { SELECT COUNT((pg_get_logical_snapshot_meta(f.name::pg_lsn))) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f; }
+step "s1_get_logical_snapshot_info" { SELECT (pg_get_logical_snapshot_info(f.name::pg_lsn)).state,(pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_xip,1),(pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_xip,1) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f ORDER BY 2; }
+
+permutation "s0_init" "s0_begin" "s0_savepoint" "s0_truncate" "s1_checkpoint" "s1_get_changes" "s0_commit" "s0_begin" "s0_insert" "s1_checkpoint" "s1_get_changes" "s0_commit" "s1_get_changes" "s1_get_logical_snapshot_info" "s1_get_logical_snapshot_meta"
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 461fc3f437..8d233f092d 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -29682,6 +29682,59 @@ DETAIL:  Make sure pg_wal_replay_wait() isn't called within a transaction with a
       </entry>
       </row>
 
+      <row>
+       <entry id="pg-get-logical-snapshot-meta" role="func_table_entry"><para role="func_signature">
+        <indexterm>
+         <primary>pg_get_logical_snapshot_meta</primary>
+        </indexterm>
+        <function>pg_get_logical_snapshot_meta</function> ( <parameter>in_lsn</parameter> <type>pg_lsn</type> )
+        <returnvalue>record</returnvalue>
+        ( <parameter>magic</parameter> <type>int</type>,
+        <parameter>checksum</parameter> <type>int</type>,
+        <parameter>version</parameter> <type>int</type> )
+       </para>
+       <para>
+        Gets logical snapshot metadata about a snapshot file that is located in
+        the <filename>pg_logical/snapshots</filename> directory.
+        The <replaceable>in_lsn</replaceable> argument can be extracted from the
+        snapshot file name. The aim of this function is mainly for debugging or
+        educational purposes.
+       </para>
+      </entry>
+      </row>
+
+      <row>
+       <entry id="pg-get-logical-snapshot-info" role="func_table_entry"><para role="func_signature">
+        <indexterm>
+         <primary>pg_get_logical_snapshot_info</primary>
+        </indexterm>
+        <function>pg_get_logical_snapshot_info</function> ( <parameter>in_lsn</parameter> <type>pg_lsn</type> )
+        <returnvalue>record</returnvalue>
+        ( <parameter>state</parameter> <type>smallint</type>,
+        <parameter>xmin</parameter> <type>xid</type>,
+        <parameter>xmax</parameter> <type>xid</type>,
+        <parameter>start_decoding_at</parameter> <type>pg_lsn</type>,
+        <parameter>two_phase_at</parameter> <type>pg_lsn</type>,
+        <parameter>initial_xmin_horizon</parameter> <type>xid</type>,
+        <parameter>building_full_snapshot</parameter> <type>boolean</type>,
+        <parameter>in_slot_creation</parameter> <type>boolean</type>,
+        <parameter>last_serialized_snapshot</parameter> <type>pg_lsn</type>,
+        <parameter>next_phase_at</parameter> <type>xid</type>,
+        <parameter>committed_count</parameter> <type>bigint</type>,
+        <parameter>committed_xip</parameter> <type>xid[]</type>,
+        <parameter>catchange_count</parameter> <type>bigint</type>,
+        <parameter>catchange_xip</parameter> <type>xid[]</type> )
+       </para>
+       <para>
+        Gets logical snapshot information about a snapshot file that is located
+        in the <filename>pg_logical/snapshots</filename> directory.
+        The <replaceable>in_lsn</replaceable> argument can be extracted from the
+        snapshot file name. The aim of this function is mainly for debugging or
+        educational purposes.
+       </para>
+      </entry>
+      </row>
+
      </tbody>
     </tgroup>
    </table>
diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c
index 0450f94ba8..b4704dd5b4 100644
--- a/src/backend/replication/logical/snapbuild.c
+++ b/src/backend/replication/logical/snapbuild.c
@@ -129,6 +129,7 @@
 #include "access/transam.h"
 #include "access/xact.h"
 #include "common/file_utils.h"
+#include "funcapi.h"
 #include "miscadmin.h"
 #include "pgstat.h"
 #include "replication/logical.h"
@@ -139,8 +140,10 @@
 #include "storage/proc.h"
 #include "storage/procarray.h"
 #include "storage/standby.h"
+#include "utils/array.h"
 #include "utils/builtins.h"
 #include "utils/memutils.h"
+#include "utils/pg_lsn.h"
 #include "utils/snapmgr.h"
 #include "utils/snapshot.h"
 
@@ -1599,6 +1602,9 @@ typedef struct SnapBuildOnDisk
 #define SNAPBUILD_MAGIC 0x51A1E001
 #define SNAPBUILD_VERSION 6
 
+static void ValidateSnapshotFile(XLogRecPtr lsn, SnapBuildOnDisk *ondisk,
+								 const char *path);
+
 /*
  * Store/Load a snapshot from disk, depending on the snapshot builder's state.
  *
@@ -2178,3 +2184,222 @@ SnapBuildSnapshotExists(XLogRecPtr lsn)
 
 	return ret == 0;
 }
+
+static void
+ValidateSnapshotFile(XLogRecPtr lsn, SnapBuildOnDisk *ondisk, const char *path)
+{
+	int			fd;
+	Size		sz;
+	pg_crc32c	checksum;
+	MemoryContext context;
+
+	context = AllocSetContextCreate(CurrentMemoryContext,
+									"ondisk logical snapshot inspect context",
+									ALLOCSET_DEFAULT_SIZES);
+
+	fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
+
+	if (fd < 0 && errno == ENOENT)
+		ereport(ERROR,
+				errmsg("file \"%s\" does not exist", path));
+	else if (fd < 0)
+		ereport(ERROR,
+				(errcode_for_file_access(),
+				 errmsg("could not open file \"%s\": %m", path)));
+
+	/* ----
+	 * Make sure the snapshot had been stored safely to disk, that's normally
+	 * cheap.
+	 * Note that we do not need PANIC here, nobody will be able to use the
+	 * slot without fsyncing, and saving it won't succeed without an fsync()
+	 * either...
+	 * ----
+	 */
+	fsync_fname(path, false);
+	fsync_fname(PG_LOGICAL_SNAPSHOTS_DIR, true);
+
+
+	/* read statically sized portion of snapshot */
+	SnapBuildRestoreContents(fd, (char *) ondisk, SnapBuildOnDiskConstantSize, path);
+
+	if (ondisk->magic != SNAPBUILD_MAGIC)
+		ereport(ERROR,
+				(errcode(ERRCODE_DATA_CORRUPTED),
+				 errmsg("snapbuild state file \"%s\" has wrong magic number: %u instead of %u",
+						path, ondisk->magic, SNAPBUILD_MAGIC)));
+
+	if (ondisk->version != SNAPBUILD_VERSION)
+		ereport(ERROR,
+				(errcode(ERRCODE_DATA_CORRUPTED),
+				 errmsg("snapbuild state file \"%s\" has unsupported version: %u instead of %u",
+						path, ondisk->version, SNAPBUILD_VERSION)));
+
+	INIT_CRC32C(checksum);
+	COMP_CRC32C(checksum,
+				((char *) ondisk) + SnapBuildOnDiskNotChecksummedSize,
+				SnapBuildOnDiskConstantSize - SnapBuildOnDiskNotChecksummedSize);
+
+	/* read SnapBuild */
+	SnapBuildRestoreContents(fd, (char *) &ondisk->builder, sizeof(SnapBuild), path);
+	COMP_CRC32C(checksum, &ondisk->builder, sizeof(SnapBuild));
+
+	ondisk->builder.context = context;
+
+	/* restore committed xacts information */
+	if (ondisk->builder.committed.xcnt > 0)
+	{
+		sz = sizeof(TransactionId) * ondisk->builder.committed.xcnt;
+		ondisk->builder.committed.xip = MemoryContextAllocZero(ondisk->builder.context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.committed.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.committed.xip, sz);
+	}
+
+	/* restore catalog modifying xacts information */
+	if (ondisk->builder.catchange.xcnt > 0)
+	{
+		sz = sizeof(TransactionId) * ondisk->builder.catchange.xcnt;
+		ondisk->builder.catchange.xip = MemoryContextAllocZero(ondisk->builder.context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.catchange.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.catchange.xip, sz);
+	}
+
+	if (CloseTransientFile(fd) != 0)
+		ereport(ERROR,
+				(errcode_for_file_access(),
+				 errmsg("could not close file \"%s\": %m", path)));
+
+	FIN_CRC32C(checksum);
+
+	/* verify checksum of what we've read */
+	if (!EQ_CRC32C(checksum, ondisk->checksum))
+		ereport(ERROR,
+				(errcode(ERRCODE_DATA_CORRUPTED),
+				 errmsg("checksum mismatch for snapbuild state file \"%s\": is %u, should be %u",
+						path, checksum, ondisk->checksum)));
+}
+
+/*
+ * Retrieve the logical snapshot file metadata.
+ */
+Datum
+pg_get_logical_snapshot_meta(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_META_COLS 3
+	SnapBuildOnDisk ondisk;
+	XLogRecPtr	lsn;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_META_COLS];
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_META_COLS];
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+
+	lsn = PG_GETARG_LSN(0);
+
+	sprintf(path, "%s/%X-%X.snap",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			LSN_FORMAT_ARGS(lsn));
+
+	ValidateSnapshotFile(lsn, &ondisk, path);
+
+	/* Build a tuple descriptor for our result type. */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	memset(nulls, 0, sizeof(nulls));
+
+	values[0] = Int32GetDatum(ondisk.magic);
+	values[1] = Int32GetDatum(ondisk.checksum);
+	values[2] = Int32GetDatum(ondisk.version);
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	MemoryContextReset(ondisk.builder.context);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_META_COLS
+}
+
+/*
+ * Retrieve the logical snapshot file data.
+ */
+Datum
+pg_get_logical_snapshot_info(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_INFO_COLS 14
+	SnapBuildOnDisk ondisk;
+	XLogRecPtr	lsn;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS];
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS];
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+
+	lsn = PG_GETARG_LSN(0);
+
+	sprintf(path, "%s/%X-%X.snap",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			LSN_FORMAT_ARGS(lsn));
+
+	ValidateSnapshotFile(lsn, &ondisk, path);
+
+	/* Build a tuple descriptor for our result type. */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	memset(nulls, 0, sizeof(nulls));
+
+	values[0] = Int16GetDatum(ondisk.builder.state);
+	values[1] = TransactionIdGetDatum(ondisk.builder.xmin);
+	values[2] = TransactionIdGetDatum(ondisk.builder.xmax);
+	values[3] = LSNGetDatum(ondisk.builder.start_decoding_at);
+	values[4] = LSNGetDatum(ondisk.builder.two_phase_at);
+	values[5] = TransactionIdGetDatum(ondisk.builder.initial_xmin_horizon);
+	values[6] = BoolGetDatum(ondisk.builder.building_full_snapshot);
+	values[7] = BoolGetDatum(ondisk.builder.in_slot_creation);
+	values[8] = LSNGetDatum(ondisk.builder.last_serialized_snapshot);
+	values[9] = TransactionIdGetDatum(ondisk.builder.next_phase_at);
+	values[10] = Int64GetDatum(ondisk.builder.committed.xcnt);
+
+	if (ondisk.builder.committed.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+		int			narrayelems;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.committed.xcnt * sizeof(Datum));
+		narrayelems = 0;
+
+		for (narrayelems = 0; narrayelems < ondisk.builder.committed.xcnt; narrayelems++)
+			arrayelems[narrayelems] = Int64GetDatum((int64) ondisk.builder.committed.xip[narrayelems]);
+
+		values[11] = PointerGetDatum(construct_array_builtin(arrayelems, narrayelems, INT8OID));
+	}
+	else
+		nulls[11] = true;
+
+	values[12] = Int64GetDatum(ondisk.builder.catchange.xcnt);
+
+	if (ondisk.builder.catchange.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+		int			narrayelems;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.catchange.xcnt * sizeof(Datum));
+		narrayelems = 0;
+
+		for (narrayelems = 0; narrayelems < ondisk.builder.catchange.xcnt; narrayelems++)
+			arrayelems[narrayelems] = Int64GetDatum((int64) ondisk.builder.catchange.xip[narrayelems]);
+
+		values[13] = PointerGetDatum(construct_array_builtin(arrayelems, narrayelems, INT8OID));
+	}
+	else
+		nulls[13] = true;
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	MemoryContextReset(ondisk.builder.context);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_INFO_COLS
+}
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 85f42be1b3..76a8c00ba0 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11264,6 +11264,22 @@
   proargmodes => '{i,i,i,v,o,o,o}',
   proargnames => '{slot_name,upto_lsn,upto_nchanges,options,lsn,xid,data}',
   prosrc => 'pg_logical_slot_get_changes' },
+{ oid => '9080', descr => 'get logical snapshot file data',
+  proname => 'pg_get_logical_snapshot_info',
+  provolatile => 'i', proparallel => 's',
+  prorettype => 'record', proargtypes => 'pg_lsn',
+  proallargtypes => '{pg_lsn,int2,xid,xid,pg_lsn,pg_lsn,xid,bool,bool,pg_lsn,xid,int8,_int8,int8,_int8}',
+  proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+  proargnames => '{in_lsn,state,xmin,xmax,start_decoding_at,two_phase_at,initial_xmin_horizon,building_full_snapshot,in_slot_creation,last_serialized_snapshot,next_phase_at,committed_count,committed_xip,catchange_count,catchange_xip}',
+  prosrc => 'pg_get_logical_snapshot_info' },
+{ oid => '9095', descr => 'get logical snapshot file metadata',
+  proname => 'pg_get_logical_snapshot_meta',
+  provolatile => 'i', proparallel => 's',
+  prorettype => 'record', proargtypes => 'pg_lsn',
+  proallargtypes => '{pg_lsn,int4,int4,int4}',
+  proargmodes => '{i,o,o,o}',
+  proargnames => '{in_lsn,magic,checksum,version}',
+  prosrc => 'pg_get_logical_snapshot_meta' },
 { oid => '3783', descr => 'get binary changes from replication slot',
   proname => 'pg_logical_slot_get_binary_changes', procost => '1000',
   prorows => '1000', provariadic => 'text', proisstrict => 'f',
-- 
2.34.1

#9Amit Kapila
amit.kapila16@gmail.com
In reply to: Bharath Rupireddy (#6)
Re: Add contrib/pg_logicalsnapinspect

On Thu, Aug 29, 2024 at 6:33 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:

On Thu, Aug 29, 2024 at 3:44 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Yeah that's fair. And now I'm wondering if we need an extra module. I think
we could "simply" expose 2 new functions in core, thoughts?

What do you think? Did you have something else in mind?

On similar lines, we can also provide a function to get the slot's
on-disk data.

Yeah, having a way to expose the data from the disk makes fully sense to me.

IIRC, Bharath had previously proposed a tool to achieve
the same. It is fine if we don't want to add that as part of this
patch but I mentioned it because by having that we can have a set of
functions to view logical decoding data.

That's right. I think this one would be simply enough to expose one or two
functions in core too (and probably would not need an extra module).

+1 for functions in core unless this extra module
pg_logicalsnapinspect works as a tool to be helpful even when the
server is down.

We have an example of pageinspect which provides low-level functions
to aid debugging. The proposal for these APIs also seems to fall in
the same category, so why go for the core for these functions?

--
With Regards,
Amit Kapila.

#10Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: Amit Kapila (#9)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Fri, Aug 30, 2024 at 03:43:12PM +0530, Amit Kapila wrote:

On Thu, Aug 29, 2024 at 6:33 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:

On Thu, Aug 29, 2024 at 3:44 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Yeah that's fair. And now I'm wondering if we need an extra module. I think
we could "simply" expose 2 new functions in core, thoughts?

What do you think? Did you have something else in mind?

On similar lines, we can also provide a function to get the slot's
on-disk data.

Yeah, having a way to expose the data from the disk makes fully sense to me.

IIRC, Bharath had previously proposed a tool to achieve
the same. It is fine if we don't want to add that as part of this
patch but I mentioned it because by having that we can have a set of
functions to view logical decoding data.

That's right. I think this one would be simply enough to expose one or two
functions in core too (and probably would not need an extra module).

+1 for functions in core unless this extra module
pg_logicalsnapinspect works as a tool to be helpful even when the
server is down.

We have an example of pageinspect which provides low-level functions
to aid debugging. The proposal for these APIs also seems to fall in
the same category,

That's right, but...

so why go for the core for these functions?

as we decided not to expose the SnapBuildOnDisk and SnapBuild structs to public
and to create/expose 2 new functions in snapbuild.c then the functions in the
module would do nothing but expose the data coming from the snapbuild.c's
functions (get the tuple and send it to the client). That sounds weird to me to
create a module that would "only" do so, that's why I thought that in core
functions taking care of everything make more sense.

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

#11Amit Kapila
amit.kapila16@gmail.com
In reply to: Bertrand Drouvot (#10)
Re: Add contrib/pg_logicalsnapinspect

On Fri, Aug 30, 2024 at 5:18 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

On Fri, Aug 30, 2024 at 03:43:12PM +0530, Amit Kapila wrote:

On Thu, Aug 29, 2024 at 6:33 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:

On Thu, Aug 29, 2024 at 3:44 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Yeah that's fair. And now I'm wondering if we need an extra module. I think
we could "simply" expose 2 new functions in core, thoughts?

What do you think? Did you have something else in mind?

On similar lines, we can also provide a function to get the slot's
on-disk data.

Yeah, having a way to expose the data from the disk makes fully sense to me.

IIRC, Bharath had previously proposed a tool to achieve
the same. It is fine if we don't want to add that as part of this
patch but I mentioned it because by having that we can have a set of
functions to view logical decoding data.

That's right. I think this one would be simply enough to expose one or two
functions in core too (and probably would not need an extra module).

+1 for functions in core unless this extra module
pg_logicalsnapinspect works as a tool to be helpful even when the
server is down.

We have an example of pageinspect which provides low-level functions
to aid debugging. The proposal for these APIs also seems to fall in
the same category,

That's right, but...

so why go for the core for these functions?

as we decided not to expose the SnapBuildOnDisk and SnapBuild structs to public
and to create/expose 2 new functions in snapbuild.c then the functions in the
module would do nothing but expose the data coming from the snapbuild.c's
functions (get the tuple and send it to the client). That sounds weird to me to
create a module that would "only" do so, that's why I thought that in core
functions taking care of everything make more sense.

I see your point. Does anyone else have an opinion on the need for
these functions and whether to expose them from a contrib module or
have them as core functions?

--
With Regards,
Amit Kapila.

#12Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: Amit Kapila (#11)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Mon, Sep 09, 2024 at 04:24:09PM +0530, Amit Kapila wrote:

On Fri, Aug 30, 2024 at 5:18 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

as we decided not to expose the SnapBuildOnDisk and SnapBuild structs to public
and to create/expose 2 new functions in snapbuild.c then the functions in the
module would do nothing but expose the data coming from the snapbuild.c's
functions (get the tuple and send it to the client). That sounds weird to me to
create a module that would "only" do so, that's why I thought that in core
functions taking care of everything make more sense.

I see your point. Does anyone else have an opinion on the need for
these functions and whether to expose them from a contrib module or
have them as core functions?

I looked at when the SNAPBUILD_VERSION has been changed:

ec5896aed3 (2014)
a975ff4980 (2021)
8bdb1332eb (2021)
7f13ac8123 (2022)
bb19b70081 (2024)

So it's not like we are changing the SnapBuildOnDisk or SnapBuild structs that
frequently. Furthermore, those structs are serialized and so we have to preserve
their on-disk compatibility (means we can change them only in a major release
if we need to).

So, I think it would not be that much of an issue to expose those structs and
create a new contrib module (as v1 did propose) instead of in core new functions.

If we want to insist that external modules "should" not rely on those structs then
we could put them into a new internal_snapbuild.h file (instead of snapbuild.h
as proposed in v1).

At the end, I think that creating a contrib module and exposing those structs in
internal_snapbuild.h make more sense (as compared to in core functions).

Thoughts?

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

#13Amit Kapila
amit.kapila16@gmail.com
In reply to: Bertrand Drouvot (#12)
Re: Add contrib/pg_logicalsnapinspect

On Tue, Sep 10, 2024 at 8:56 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

On Mon, Sep 09, 2024 at 04:24:09PM +0530, Amit Kapila wrote:

On Fri, Aug 30, 2024 at 5:18 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

as we decided not to expose the SnapBuildOnDisk and SnapBuild structs to public
and to create/expose 2 new functions in snapbuild.c then the functions in the
module would do nothing but expose the data coming from the snapbuild.c's
functions (get the tuple and send it to the client). That sounds weird to me to
create a module that would "only" do so, that's why I thought that in core
functions taking care of everything make more sense.

I see your point. Does anyone else have an opinion on the need for
these functions and whether to expose them from a contrib module or
have them as core functions?

I looked at when the SNAPBUILD_VERSION has been changed:

ec5896aed3 (2014)
a975ff4980 (2021)
8bdb1332eb (2021)
7f13ac8123 (2022)
bb19b70081 (2024)

So it's not like we are changing the SnapBuildOnDisk or SnapBuild structs that
frequently. Furthermore, those structs are serialized and so we have to preserve
their on-disk compatibility (means we can change them only in a major release
if we need to).

So, I think it would not be that much of an issue to expose those structs and
create a new contrib module (as v1 did propose) instead of in core new functions.

If we want to insist that external modules "should" not rely on those structs then
we could put them into a new internal_snapbuild.h file (instead of snapbuild.h
as proposed in v1).

Adding snapbuild_internal.h sounds like a good idea.

At the end, I think that creating a contrib module and exposing those structs in
internal_snapbuild.h make more sense (as compared to in core functions).

Fail enough. We can keep the module name as logicalinspect so that we
can extend it for other logical decoding/replication-related files in
the future.

--
With Regards,
Amit Kapila.

#14Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: Amit Kapila (#13)
1 attachment(s)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Wed, Sep 11, 2024 at 10:30:37AM +0530, Amit Kapila wrote:

On Tue, Sep 10, 2024 at 8:56 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

On Mon, Sep 09, 2024 at 04:24:09PM +0530, Amit Kapila wrote:

On Fri, Aug 30, 2024 at 5:18 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

as we decided not to expose the SnapBuildOnDisk and SnapBuild structs to public
and to create/expose 2 new functions in snapbuild.c then the functions in the
module would do nothing but expose the data coming from the snapbuild.c's
functions (get the tuple and send it to the client). That sounds weird to me to
create a module that would "only" do so, that's why I thought that in core
functions taking care of everything make more sense.

I see your point. Does anyone else have an opinion on the need for
these functions and whether to expose them from a contrib module or
have them as core functions?

I looked at when the SNAPBUILD_VERSION has been changed:

ec5896aed3 (2014)
a975ff4980 (2021)
8bdb1332eb (2021)
7f13ac8123 (2022)
bb19b70081 (2024)

So it's not like we are changing the SnapBuildOnDisk or SnapBuild structs that
frequently. Furthermore, those structs are serialized and so we have to preserve
their on-disk compatibility (means we can change them only in a major release
if we need to).

So, I think it would not be that much of an issue to expose those structs and
create a new contrib module (as v1 did propose) instead of in core new functions.

If we want to insist that external modules "should" not rely on those structs then
we could put them into a new internal_snapbuild.h file (instead of snapbuild.h
as proposed in v1).

Adding snapbuild_internal.h sounds like a good idea.

Thanks for the feedback!

At the end, I think that creating a contrib module and exposing those structs in
internal_snapbuild.h make more sense (as compared to in core functions).

Fail enough. We can keep the module name as logicalinspect so that we
can extend it for other logical decoding/replication-related files in
the future.

Yeah, good idea. Done that way in v3 attached.

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachments:

v3-0001-Add-contrib-pg_logicalinspect.patchtext/x-diff; charset=us-asciiDownload
From b51d403fa62af10f090c1e2339627e3ee29af524 Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Wed, 14 Aug 2024 08:46:05 +0000
Subject: [PATCH v3] Add contrib/pg_logicalinspect

Provides SQL functions that allow to inspect logical decoding components.

It currently allows to inspect the contents of serialized logical snapshots of
a running database cluster, which is useful for debugging or educational
purposes.
---
 contrib/Makefile                              |   1 +
 contrib/meson.build                           |   1 +
 contrib/pg_logicalinspect/.gitignore          |   4 +
 contrib/pg_logicalinspect/Makefile            |  31 +++
 .../expected/logical_inspect.out              |  52 ++++
 contrib/pg_logicalinspect/logicalinspect.conf |   1 +
 contrib/pg_logicalinspect/meson.build         |  39 +++
 .../pg_logicalinspect--1.0.sql                |  43 +++
 contrib/pg_logicalinspect/pg_logicalinspect.c | 249 ++++++++++++++++++
 .../pg_logicalinspect.control                 |   5 +
 .../specs/logical_inspect.spec                |  34 +++
 doc/src/sgml/contrib.sgml                     |   1 +
 doc/src/sgml/filelist.sgml                    |   1 +
 doc/src/sgml/pglogicalinspect.sgml            | 145 ++++++++++
 src/backend/replication/logical/snapbuild.c   | 190 +------------
 src/include/port/pg_crc32c.h                  |  16 +-
 src/include/replication/internal_snapbuild.h  | 204 ++++++++++++++
 src/include/replication/snapbuild.h           |   2 +-
 18 files changed, 826 insertions(+), 193 deletions(-)
   7.6% contrib/pg_logicalinspect/expected/
   5.7% contrib/pg_logicalinspect/specs/
  32.4% contrib/pg_logicalinspect/
  13.3% doc/src/sgml/
  17.4% src/backend/replication/logical/
   4.1% src/include/port/
  18.9% src/include/replication/

diff --git a/contrib/Makefile b/contrib/Makefile
index abd780f277..952855d9b6 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -32,6 +32,7 @@ SUBDIRS = \
 		passwordcheck	\
 		pg_buffercache	\
 		pg_freespacemap \
+		pg_logicalinspect \
 		pg_prewarm	\
 		pg_stat_statements \
 		pg_surgery	\
diff --git a/contrib/meson.build b/contrib/meson.build
index 14a8906865..159ff41555 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -46,6 +46,7 @@ subdir('passwordcheck')
 subdir('pg_buffercache')
 subdir('pgcrypto')
 subdir('pg_freespacemap')
+subdir('pg_logicalinspect')
 subdir('pg_prewarm')
 subdir('pgrowlocks')
 subdir('pg_stat_statements')
diff --git a/contrib/pg_logicalinspect/.gitignore b/contrib/pg_logicalinspect/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/contrib/pg_logicalinspect/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/contrib/pg_logicalinspect/Makefile b/contrib/pg_logicalinspect/Makefile
new file mode 100644
index 0000000000..55124514d4
--- /dev/null
+++ b/contrib/pg_logicalinspect/Makefile
@@ -0,0 +1,31 @@
+# contrib/pg_logicalinspect/Makefile
+
+MODULE_big = pg_logicalinspect
+OBJS = \
+	$(WIN32RES) \
+	pg_logicalinspect.o
+PGFILEDESC = "pg_logicalinspect - functions to inspect logical decoding components"
+
+EXTENSION = pg_logicalinspect
+DATA = pg_logicalinspect--1.0.sql
+
+EXTRA_INSTALL = contrib/test_decoding
+
+ISOLATION = logical_inspect
+
+ISOLATION_OPTS = --temp-config $(top_srcdir)/contrib/pg_logicalinspect/logicalinspect.conf
+
+# Disabled because these tests require "wal_level=logical", which
+# some installcheck users do not have (e.g. buildfarm clients).
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/pg_logicalinspect
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/pg_logicalinspect/expected/logical_inspect.out b/contrib/pg_logicalinspect/expected/logical_inspect.out
new file mode 100644
index 0000000000..749cd4642d
--- /dev/null
+++ b/contrib/pg_logicalinspect/expected/logical_inspect.out
@@ -0,0 +1,52 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s0_init s0_begin s0_savepoint s0_truncate s1_checkpoint s1_get_changes s0_commit s0_begin s0_insert s1_checkpoint s1_get_changes s0_commit s1_get_changes s1_get_logical_snapshot_info s1_get_logical_snapshot_meta
+step s0_init: SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding');
+?column?
+--------
+init    
+(1 row)
+
+step s0_begin: BEGIN;
+step s0_savepoint: SAVEPOINT sp1;
+step s0_truncate: TRUNCATE tbl1;
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data
+----
+(0 rows)
+
+step s0_commit: COMMIT;
+step s0_begin: BEGIN;
+step s0_insert: INSERT INTO tbl1 VALUES (1);
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                   
+---------------------------------------
+BEGIN                                  
+table public.tbl1: TRUNCATE: (no-flags)
+COMMIT                                 
+(3 rows)
+
+step s0_commit: COMMIT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                                         
+-------------------------------------------------------------
+BEGIN                                                        
+table public.tbl1: INSERT: val1[integer]:1 val2[integer]:null
+COMMIT                                                       
+(3 rows)
+
+step s1_get_logical_snapshot_info: SELECT (pg_get_logical_snapshot_info(f.name::pg_lsn)).state,(pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_xip,1),(pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_xip,1) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f ORDER BY 2;
+state|catchange_count|array_length|committed_count|array_length
+-----+---------------+------------+---------------+------------
+    2|              0|            |              2|           2
+    2|              2|           2|              0|            
+(2 rows)
+
+step s1_get_logical_snapshot_meta: SELECT COUNT((pg_get_logical_snapshot_meta(f.name::pg_lsn))) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f;
+count
+-----
+    2
+(1 row)
+
diff --git a/contrib/pg_logicalinspect/logicalinspect.conf b/contrib/pg_logicalinspect/logicalinspect.conf
new file mode 100644
index 0000000000..e3d257315f
--- /dev/null
+++ b/contrib/pg_logicalinspect/logicalinspect.conf
@@ -0,0 +1 @@
+wal_level = logical
diff --git a/contrib/pg_logicalinspect/meson.build b/contrib/pg_logicalinspect/meson.build
new file mode 100644
index 0000000000..b787dafc9b
--- /dev/null
+++ b/contrib/pg_logicalinspect/meson.build
@@ -0,0 +1,39 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+pg_logicalinspect_sources = files('pg_logicalinspect.c')
+
+if host_system == 'windows'
+  pg_logicalinspect_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+    '--NAME', 'pg_logicalinspect',
+    '--FILEDESC', 'pg_logicalinspect - functions to inspect contents of logical snapshots',])
+endif
+
+pg_logicalinspect = shared_module('pg_logicalinspect',
+  pg_logicalinspect_sources,
+  kwargs: contrib_mod_args + {
+      'dependencies': contrib_mod_args['dependencies'],
+  },
+)
+contrib_targets += pg_logicalinspect
+
+install_data(
+  'pg_logicalinspect.control',
+  'pg_logicalinspect--1.0.sql',
+  kwargs: contrib_data_args,
+)
+
+tests += {
+  'name': 'pg_logicalinspect',
+  'sd': meson.current_source_dir(),
+  'bd': meson.current_build_dir(),
+  'isolation': {
+    'specs': [
+      'logical_inspect',
+    ],
+    'regress_args': [
+      '--temp-config', files('logicalinspect.conf'),
+    ],
+    # see above
+    'runningcheck': false,
+  },
+}
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql b/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
new file mode 100644
index 0000000000..51713ed53e
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
@@ -0,0 +1,43 @@
+/* contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_logicalinspect" to load this file. \quit
+
+--
+-- pg_get_logical_snapshot_meta()
+--
+CREATE FUNCTION pg_get_logical_snapshot_meta(IN in_lsn pg_lsn,
+    OUT magic int4,
+    OUT checksum int4,
+    OUT version int4
+)
+AS 'MODULE_PATHNAME', 'pg_get_logical_snapshot_meta'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_logical_snapshot_meta(pg_lsn) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_logical_snapshot_meta(pg_lsn) TO pg_read_server_files;
+
+--
+-- pg_get_logical_snapshot_info()
+--
+CREATE FUNCTION pg_get_logical_snapshot_info(IN in_lsn pg_lsn,
+    OUT state int2,
+    OUT xmin xid,
+    OUT xmax xid,
+    OUT start_decoding_at pg_lsn,
+    OUT two_phase_at pg_lsn,
+    OUT initial_xmin_horizon xid,
+    OUT building_full_snapshot boolean,
+    OUT in_slot_creation boolean,
+    OUT last_serialized_snapshot pg_lsn,
+    OUT next_phase_at xid,
+    OUT committed_count int8,
+    OUT committed_xip xid[],
+    OUT catchange_count int8,
+    OUT catchange_xip xid[]
+)
+AS 'MODULE_PATHNAME', 'pg_get_logical_snapshot_info'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_logical_snapshot_info(pg_lsn) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_logical_snapshot_info(pg_lsn) TO pg_read_server_files;
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.c b/contrib/pg_logicalinspect/pg_logicalinspect.c
new file mode 100644
index 0000000000..f3c1a0178e
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.c
@@ -0,0 +1,249 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_logicalinspect.c
+ *		  Functions to inspect contents of PostgreSQL logical snapshots
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ *		  contrib/pg_logicalinspect/pg_logicalinspect.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "funcapi.h"
+#include "port/pg_crc32c.h"
+#include "replication/internal_snapbuild.h"
+#include "utils/array.h"
+#include "utils/pg_lsn.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_meta);
+PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_info);
+
+static void ValidateSnapshotFile(XLogRecPtr lsn, SnapBuildOnDisk *ondisk,
+								 const char *path);
+
+/*
+ * NOTE: For any code change or issue fix here, it is highly recommended to
+ * give a thought about doing the same in SnapBuildRestore() as well.
+ */
+
+/*
+ * Validate the logical snapshot file.
+ */
+static void
+ValidateSnapshotFile(XLogRecPtr lsn, SnapBuildOnDisk *ondisk, const char *path)
+{
+	int			fd;
+	Size		sz;
+	pg_crc32c	checksum;
+	MemoryContext context;
+
+	context = AllocSetContextCreate(CurrentMemoryContext,
+									"logicalsnapshot inspect context",
+									ALLOCSET_DEFAULT_SIZES);
+
+	fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
+
+	if (fd < 0 && errno == ENOENT)
+		ereport(ERROR,
+				errmsg("file \"%s\" does not exist", path));
+	else if (fd < 0)
+		ereport(ERROR,
+				(errcode_for_file_access(),
+				 errmsg("could not open file \"%s\": %m", path)));
+
+	/* ----
+	 * Make sure the snapshot had been stored safely to disk, that's normally
+	 * cheap.
+	 * Note that we do not need PANIC here, nobody will be able to use the
+	 * slot without fsyncing, and saving it won't succeed without an fsync()
+	 * either...
+	 * ----
+	 */
+	fsync_fname(path, false);
+	fsync_fname("pg_logical/snapshots", true);
+
+
+	/* read statically sized portion of snapshot */
+	SnapBuildRestoreContents(fd, (char *) ondisk, SnapBuildOnDiskConstantSize, path);
+
+	if (ondisk->magic != SNAPBUILD_MAGIC)
+		ereport(ERROR,
+				(errcode(ERRCODE_DATA_CORRUPTED),
+				 errmsg("snapbuild state file \"%s\" has wrong magic number: %u instead of %u",
+						path, ondisk->magic, SNAPBUILD_MAGIC)));
+
+	if (ondisk->version != SNAPBUILD_VERSION)
+		ereport(ERROR,
+				(errcode(ERRCODE_DATA_CORRUPTED),
+				 errmsg("snapbuild state file \"%s\" has unsupported version: %u instead of %u",
+						path, ondisk->version, SNAPBUILD_VERSION)));
+
+	INIT_CRC32C(checksum);
+	COMP_CRC32C(checksum,
+				((char *) ondisk) + SnapBuildOnDiskNotChecksummedSize,
+				SnapBuildOnDiskConstantSize - SnapBuildOnDiskNotChecksummedSize);
+
+	/* read SnapBuild */
+	SnapBuildRestoreContents(fd, (char *) &ondisk->builder, sizeof(SnapBuild), path);
+	COMP_CRC32C(checksum, &ondisk->builder, sizeof(SnapBuild));
+
+	ondisk->builder.context = context;
+
+	/* restore committed xacts information */
+	if (ondisk->builder.committed.xcnt > 0)
+	{
+		sz = sizeof(TransactionId) * ondisk->builder.committed.xcnt;
+		ondisk->builder.committed.xip = MemoryContextAllocZero(ondisk->builder.context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.committed.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.committed.xip, sz);
+	}
+
+	/* restore catalog modifying xacts information */
+	if (ondisk->builder.catchange.xcnt > 0)
+	{
+		sz = sizeof(TransactionId) * ondisk->builder.catchange.xcnt;
+		ondisk->builder.catchange.xip = MemoryContextAllocZero(ondisk->builder.context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.catchange.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.catchange.xip, sz);
+	}
+
+	if (CloseTransientFile(fd) != 0)
+		ereport(ERROR,
+				(errcode_for_file_access(),
+				 errmsg("could not close file \"%s\": %m", path)));
+
+	FIN_CRC32C(checksum);
+
+	/* verify checksum of what we've read */
+	if (!EQ_CRC32C(checksum, ondisk->checksum))
+		ereport(ERROR,
+				(errcode(ERRCODE_DATA_CORRUPTED),
+				 errmsg("checksum mismatch for snapbuild state file \"%s\": is %u, should be %u",
+						path, checksum, ondisk->checksum)));
+}
+
+/*
+ * Retrieve the logical snapshot file metadata.
+ */
+Datum
+pg_get_logical_snapshot_meta(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_META_COLS 3
+	SnapBuildOnDisk ondisk;
+	XLogRecPtr	lsn;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_META_COLS];
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_META_COLS];
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+
+	lsn = PG_GETARG_LSN(0);
+
+	sprintf(path, "pg_logical/snapshots/%X-%X.snap",
+			LSN_FORMAT_ARGS(lsn));
+
+	ValidateSnapshotFile(lsn, &ondisk, path);
+
+	/* Build a tuple descriptor for our result type. */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	memset(nulls, 0, sizeof(nulls));
+
+	values[0] = Int32GetDatum(ondisk.magic);
+	values[1] = Int32GetDatum(ondisk.checksum);
+	values[2] = Int32GetDatum(ondisk.version);
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	MemoryContextReset(ondisk.builder.context);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_META_COLS
+}
+
+Datum
+pg_get_logical_snapshot_info(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_INFO_COLS 14
+	SnapBuildOnDisk ondisk;
+	XLogRecPtr	lsn;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS];
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS];
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+
+	lsn = PG_GETARG_LSN(0);
+
+	sprintf(path, "pg_logical/snapshots/%X-%X.snap",
+			LSN_FORMAT_ARGS(lsn));
+
+	ValidateSnapshotFile(lsn, &ondisk, path);
+
+	/* Build a tuple descriptor for our result type. */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	memset(nulls, 0, sizeof(nulls));
+
+	values[0] = Int16GetDatum(ondisk.builder.state);
+	values[1] = TransactionIdGetDatum(ondisk.builder.xmin);
+	values[2] = TransactionIdGetDatum(ondisk.builder.xmax);
+	values[3] = LSNGetDatum(ondisk.builder.start_decoding_at);
+	values[4] = LSNGetDatum(ondisk.builder.two_phase_at);
+	values[5] = TransactionIdGetDatum(ondisk.builder.initial_xmin_horizon);
+	values[6] = BoolGetDatum(ondisk.builder.building_full_snapshot);
+	values[7] = BoolGetDatum(ondisk.builder.in_slot_creation);
+	values[8] = LSNGetDatum(ondisk.builder.last_serialized_snapshot);
+	values[9] = TransactionIdGetDatum(ondisk.builder.next_phase_at);
+	values[10] = Int64GetDatum(ondisk.builder.committed.xcnt);
+
+	if (ondisk.builder.committed.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+		int			narrayelems;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.committed.xcnt * sizeof(Datum));
+		narrayelems = 0;
+
+		for (narrayelems = 0; narrayelems < ondisk.builder.committed.xcnt; narrayelems++)
+			arrayelems[narrayelems] = Int64GetDatum((int64) ondisk.builder.committed.xip[narrayelems]);
+
+		values[11] = PointerGetDatum(construct_array_builtin(arrayelems, narrayelems, INT8OID));
+	}
+	else
+		nulls[11] = true;
+
+	values[12] = Int64GetDatum(ondisk.builder.catchange.xcnt);
+
+	if (ondisk.builder.catchange.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+		int			narrayelems;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.catchange.xcnt * sizeof(Datum));
+		narrayelems = 0;
+
+		for (narrayelems = 0; narrayelems < ondisk.builder.catchange.xcnt; narrayelems++)
+			arrayelems[narrayelems] = Int64GetDatum((int64) ondisk.builder.catchange.xip[narrayelems]);
+
+		values[13] = PointerGetDatum(construct_array_builtin(arrayelems, narrayelems, INT8OID));
+	}
+	else
+		nulls[13] = true;
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	MemoryContextReset(ondisk.builder.context);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_INFO_COLS
+}
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.control b/contrib/pg_logicalinspect/pg_logicalinspect.control
new file mode 100644
index 0000000000..b4a70e57ba
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.control
@@ -0,0 +1,5 @@
+# pg_logicalinspect extension
+comment = 'functions to inspect logical decoding components'
+default_version = '1.0'
+module_pathname = '$libdir/pg_logicalinspect'
+relocatable = true
diff --git a/contrib/pg_logicalinspect/specs/logical_inspect.spec b/contrib/pg_logicalinspect/specs/logical_inspect.spec
new file mode 100644
index 0000000000..e11eb63615
--- /dev/null
+++ b/contrib/pg_logicalinspect/specs/logical_inspect.spec
@@ -0,0 +1,34 @@
+# Test the pg_logicalinspect functions: that needs some permutation to
+# ensure that we are creating multiple logical snapshots and that one of them
+# contains ongoing catalogs changes.
+setup
+{
+    DROP TABLE IF EXISTS tbl1;
+    CREATE TABLE tbl1 (val1 integer, val2 integer);
+	CREATE EXTENSION pg_logicalinspect;
+}
+
+teardown
+{
+    DROP TABLE tbl1;
+    SELECT 'stop' FROM pg_drop_replication_slot('isolation_slot');
+	DROP EXTENSION pg_logicalinspect;
+}
+
+session "s0"
+setup { SET synchronous_commit=on; }
+step "s0_init" { SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding'); }
+step "s0_begin" { BEGIN; }
+step "s0_savepoint" { SAVEPOINT sp1; }
+step "s0_truncate" { TRUNCATE tbl1; }
+step "s0_insert" { INSERT INTO tbl1 VALUES (1); }
+step "s0_commit" { COMMIT; }
+
+session "s1"
+setup { SET synchronous_commit=on; }
+step "s1_checkpoint" { CHECKPOINT; }
+step "s1_get_changes" { SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0'); }
+step "s1_get_logical_snapshot_meta" { SELECT COUNT((pg_get_logical_snapshot_meta(f.name::pg_lsn))) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f; }
+step "s1_get_logical_snapshot_info" { SELECT (pg_get_logical_snapshot_info(f.name::pg_lsn)).state,(pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_xip,1),(pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_xip,1) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f ORDER BY 2; }
+
+permutation "s0_init" "s0_begin" "s0_savepoint" "s0_truncate" "s1_checkpoint" "s1_get_changes" "s0_commit" "s0_begin" "s0_insert" "s1_checkpoint" "s1_get_changes" "s0_commit" "s1_get_changes" "s1_get_logical_snapshot_info" "s1_get_logical_snapshot_meta"
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index 44639a8dca..7c381949a5 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -154,6 +154,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
  &pgbuffercache;
  &pgcrypto;
  &pgfreespacemap;
+ &pglogicalinspect;
  &pgprewarm;
  &pgrowlocks;
  &pgstatstatements;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index a7ff5f8264..66e6dccd4c 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -143,6 +143,7 @@
 <!ENTITY pgbuffercache   SYSTEM "pgbuffercache.sgml">
 <!ENTITY pgcrypto        SYSTEM "pgcrypto.sgml">
 <!ENTITY pgfreespacemap  SYSTEM "pgfreespacemap.sgml">
+<!ENTITY pglogicalinspect  SYSTEM "pglogicalinspect.sgml">
 <!ENTITY pgprewarm       SYSTEM "pgprewarm.sgml">
 <!ENTITY pgrowlocks      SYSTEM "pgrowlocks.sgml">
 <!ENTITY pgstatstatements SYSTEM "pgstatstatements.sgml">
diff --git a/doc/src/sgml/pglogicalinspect.sgml b/doc/src/sgml/pglogicalinspect.sgml
new file mode 100644
index 0000000000..7767de263f
--- /dev/null
+++ b/doc/src/sgml/pglogicalinspect.sgml
@@ -0,0 +1,145 @@
+<!-- doc/src/sgml/pglogicalinspect.sgml -->
+
+<sect1 id="pglogicalinspect" xreflabel="pg_logicalinspect">
+ <title>pg_logicalinspect &mdash; logical decoding components inspection</title>
+
+ <indexterm zone="pglogicalinspect">
+  <primary>pg_logicalinspect</primary>
+ </indexterm>
+
+ <para>
+  The <filename>pg_logicalinspect</filename> module provides SQL functions
+  that allow you to inspect the contents of logical decoding components. It
+  allows to inspect serialized logical snapshots of a running
+  <productname>PostgreSQL</productname> database cluster, which is useful
+  for debugging or educational purposes.
+ </para>
+
+ <note>
+  <para>
+   The <filename>pg_logicalinspect</filename> functions are called
+   using an LSN argument that can be extracted from the output name of the
+   <function>pg_ls_logicalsnapdir</function>() function.
+  </para>
+ </note>
+
+ <sect2 id="pglogicalinspect-funcs">
+  <title>General Functions</title>
+
+  <variablelist>
+   <varlistentry id="pglogicalinspect-funcs-pg-get-logical-snapshot-meta">
+    <term>
+     <function>pg_get_logical_snapshot_meta(in_lsn pg_lsn) returns record</function>
+    </term>
+
+    <listitem>
+     <para>
+      Gets logical snapshot metadata about a snapshot file that is located in
+      the <filename>pg_logical/snapshots</filename> directory.
+      The <replaceable>in_lsn</replaceable> argument can be extracted from the
+      snapshot file name.
+      example:
+<screen>
+postgres=# SELECT * FROM pg_ls_logicalsnapdir();
+-[ RECORD 1 ]+-----------------------
+name         | 0-40796E18.snap
+size         | 152
+modification | 2024-08-14 16:36:32+00
+
+postgres=# SELECT * FROM pg_get_logical_snapshot_meta('0/40796E18');
+-[ RECORD 1 ]--------
+magic    | 1369563137
+checksum | 1028045905
+version  | 6
+
+postgres=# SELECT (pg_get_logical_snapshot_meta(f.name::pg_lsn)).*
+           FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name
+                 FROM pg_ls_logicalsnapdir()) AS f;
+-[ RECORD 1 ]--------
+magic    | 1369563137
+checksum | 1028045905
+version  | 6
+</screen>
+     </para>
+     <para>
+      If <replaceable>in_lsn</replaceable> does not match a snapshot file, the
+      function raises an error.
+     </para>
+    </listitem>
+   </varlistentry>
+
+   <varlistentry id="pglogicalinspect-funcs-pg-get-logical-snapshot-info">
+    <term>
+     <function>pg_get_logical_snapshot_info(in_lsn pg_lsn) returns record</function>
+    </term>
+
+    <listitem>
+     <para>
+      Gets logical snapshot information about a snapshot file that is located in
+      the <filename>pg_logical/snapshots</filename> directory.
+      The <replaceable>in_lsn</replaceable> argument can be extracted from the
+      snapshot file name.
+      example:
+<screen>
+postgres=# SELECT * FROM pg_ls_logicalsnapdir();
+-[ RECORD 1 ]+-----------------------
+name         | 0-40796E18.snap
+size         | 152
+modification | 2024-08-14 16:36:32+00
+
+postgres=# SELECT * FROM pg_get_logical_snapshot_info('0/40796E18');
+-[ RECORD 1 ]------------+-----------
+state                    | 2
+xmin                     | 751
+xmax                     | 751
+start_decoding_at        | 0/40796AF8
+two_phase_at             | 0/40796AF8
+initial_xmin_horizon     | 0
+building_full_snapshot   | f
+in_slot_creation         | f
+last_serialized_snapshot | 0/0
+next_phase_at            | 0
+committed_count          | 0
+committed_xip            |
+catchange_count          | 2
+catchange_xip            | {751,752}
+
+postgres=# SELECT (pg_get_logical_snapshot_info(f.name::pg_lsn)).*
+           FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name
+                 FROM pg_ls_logicalsnapdir()) AS f;
+-[ RECORD 1 ]------------+-----------
+state                    | 2
+xmin                     | 751
+xmax                     | 751
+start_decoding_at        | 0/40796AF8
+two_phase_at             | 0/40796AF8
+initial_xmin_horizon     | 0
+building_full_snapshot   | f
+in_slot_creation         | f
+last_serialized_snapshot | 0/0
+next_phase_at            | 0
+committed_count          | 0
+committed_xip            |
+catchange_count          | 2
+catchange_xip            | {751,752}
+</screen>
+     </para>
+     <para>
+      If <replaceable>in_lsn</replaceable> does not match a snapshot file, the
+      function raises an error.
+     </para>
+    </listitem>
+   </varlistentry>
+
+  </variablelist>
+ </sect2>
+
+ <sect2 id="pglogicalinspect-author">
+  <title>Author</title>
+
+  <para>
+   Bertrand Drouvot <email>bertranddrouvot.pg@gmail.com</email>
+  </para>
+ </sect2>
+
+</sect1>
diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c
index 0450f94ba8..a48c09c184 100644
--- a/src/backend/replication/logical/snapbuild.c
+++ b/src/backend/replication/logical/snapbuild.c
@@ -131,6 +131,7 @@
 #include "common/file_utils.h"
 #include "miscadmin.h"
 #include "pgstat.h"
+#include "replication/internal_snapbuild.h"
 #include "replication/logical.h"
 #include "replication/reorderbuffer.h"
 #include "replication/snapbuild.h"
@@ -143,146 +144,6 @@
 #include "utils/memutils.h"
 #include "utils/snapmgr.h"
 #include "utils/snapshot.h"
-
-/*
- * This struct contains the current state of the snapshot building
- * machinery. Besides a forward declaration in the header, it is not exposed
- * to the public, so we can easily change its contents.
- */
-struct SnapBuild
-{
-	/* how far are we along building our first full snapshot */
-	SnapBuildState state;
-
-	/* private memory context used to allocate memory for this module. */
-	MemoryContext context;
-
-	/* all transactions < than this have committed/aborted */
-	TransactionId xmin;
-
-	/* all transactions >= than this are uncommitted */
-	TransactionId xmax;
-
-	/*
-	 * Don't replay commits from an LSN < this LSN. This can be set externally
-	 * but it will also be advanced (never retreat) from within snapbuild.c.
-	 */
-	XLogRecPtr	start_decoding_at;
-
-	/*
-	 * LSN at which two-phase decoding was enabled or LSN at which we found a
-	 * consistent point at the time of slot creation.
-	 *
-	 * The prepared transactions, that were skipped because previously
-	 * two-phase was not enabled or are not covered by initial snapshot, need
-	 * to be sent later along with commit prepared and they must be before
-	 * this point.
-	 */
-	XLogRecPtr	two_phase_at;
-
-	/*
-	 * Don't start decoding WAL until the "xl_running_xacts" information
-	 * indicates there are no running xids with an xid smaller than this.
-	 */
-	TransactionId initial_xmin_horizon;
-
-	/* Indicates if we are building full snapshot or just catalog one. */
-	bool		building_full_snapshot;
-
-	/*
-	 * Indicates if we are using the snapshot builder for the creation of a
-	 * logical replication slot. If it's true, the start point for decoding
-	 * changes is not determined yet. So we skip snapshot restores to properly
-	 * find the start point. See SnapBuildFindSnapshot() for details.
-	 */
-	bool		in_slot_creation;
-
-	/*
-	 * Snapshot that's valid to see the catalog state seen at this moment.
-	 */
-	Snapshot	snapshot;
-
-	/*
-	 * LSN of the last location we are sure a snapshot has been serialized to.
-	 */
-	XLogRecPtr	last_serialized_snapshot;
-
-	/*
-	 * The reorderbuffer we need to update with usable snapshots et al.
-	 */
-	ReorderBuffer *reorder;
-
-	/*
-	 * TransactionId at which the next phase of initial snapshot building will
-	 * happen. InvalidTransactionId if not known (i.e. SNAPBUILD_START), or
-	 * when no next phase necessary (SNAPBUILD_CONSISTENT).
-	 */
-	TransactionId next_phase_at;
-
-	/*
-	 * Array of transactions which could have catalog changes that committed
-	 * between xmin and xmax.
-	 */
-	struct
-	{
-		/* number of committed transactions */
-		size_t		xcnt;
-
-		/* available space for committed transactions */
-		size_t		xcnt_space;
-
-		/*
-		 * Until we reach a CONSISTENT state, we record commits of all
-		 * transactions, not just the catalog changing ones. Record when that
-		 * changes so we know we cannot export a snapshot safely anymore.
-		 */
-		bool		includes_all_transactions;
-
-		/*
-		 * Array of committed transactions that have modified the catalog.
-		 *
-		 * As this array is frequently modified we do *not* keep it in
-		 * xidComparator order. Instead we sort the array when building &
-		 * distributing a snapshot.
-		 *
-		 * TODO: It's unclear whether that reasoning has much merit. Every
-		 * time we add something here after becoming consistent will also
-		 * require distributing a snapshot. Storing them sorted would
-		 * potentially also make it easier to purge (but more complicated wrt
-		 * wraparound?). Should be improved if sorting while building the
-		 * snapshot shows up in profiles.
-		 */
-		TransactionId *xip;
-	}			committed;
-
-	/*
-	 * Array of transactions and subtransactions that had modified catalogs
-	 * and were running when the snapshot was serialized.
-	 *
-	 * We normally rely on some WAL record types such as HEAP2_NEW_CID to know
-	 * if the transaction has changed the catalog. But it could happen that
-	 * the logical decoding decodes only the commit record of the transaction
-	 * after restoring the previously serialized snapshot in which case we
-	 * will miss adding the xid to the snapshot and end up looking at the
-	 * catalogs with the wrong snapshot.
-	 *
-	 * Now to avoid the above problem, we serialize the transactions that had
-	 * modified the catalogs and are still running at the time of snapshot
-	 * serialization. We fill this array while restoring the snapshot and then
-	 * refer it while decoding commit to ensure if the xact has modified the
-	 * catalog. We discard this array when all the xids in the list become old
-	 * enough to matter. See SnapBuildPurgeOlderTxn for details.
-	 */
-	struct
-	{
-		/* number of transactions */
-		size_t		xcnt;
-
-		/* This array must be sorted in xidComparator order */
-		TransactionId *xip;
-	}			catchange;
-};
-
 /*
  * Starting a transaction -- which we need to do while exporting a snapshot --
  * removes knowledge about the previously used resowner, so we save it here.
@@ -312,7 +173,6 @@ static void SnapBuildWaitSnapshot(xl_running_xacts *running, TransactionId cutof
 /* serialization functions */
 static void SnapBuildSerialize(SnapBuild *builder, XLogRecPtr lsn);
 static bool SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn);
-static void SnapBuildRestoreContents(int fd, char *dest, Size size, const char *path);
 
 /*
  * Allocate a new snapshot builder.
@@ -1557,48 +1417,6 @@ SnapBuildWaitSnapshot(xl_running_xacts *running, TransactionId cutoff)
 	}
 }
 
-/* -----------------------------------
- * Snapshot serialization support
- * -----------------------------------
- */
-
-/*
- * We store current state of struct SnapBuild on disk in the following manner:
- *
- * struct SnapBuildOnDisk;
- * TransactionId * committed.xcnt; (*not xcnt_space*)
- * TransactionId * catchange.xcnt;
- *
- */
-typedef struct SnapBuildOnDisk
-{
-	/* first part of this struct needs to be version independent */
-
-	/* data not covered by checksum */
-	uint32		magic;
-	pg_crc32c	checksum;
-
-	/* data covered by checksum */
-
-	/* version, in case we want to support pg_upgrade */
-	uint32		version;
-	/* how large is the on disk data, excluding the constant sized part */
-	uint32		length;
-
-	/* version dependent part */
-	SnapBuild	builder;
-
-	/* variable amount of TransactionIds follows */
-} SnapBuildOnDisk;
-
-#define SnapBuildOnDiskConstantSize \
-	offsetof(SnapBuildOnDisk, builder)
-#define SnapBuildOnDiskNotChecksummedSize \
-	offsetof(SnapBuildOnDisk, version)
-
-#define SNAPBUILD_MAGIC 0x51A1E001
-#define SNAPBUILD_VERSION 6
-
 /*
  * Store/Load a snapshot from disk, depending on the snapshot builder's state.
  *
@@ -1859,6 +1677,10 @@ out:
 /*
  * Restore a snapshot into 'builder' if previously one has been stored at the
  * location indicated by 'lsn'. Returns true if successful, false otherwise.
+ *
+ * NOTE: For any code change or issue fix here, it is highly recommended to
+ * give a thought about doing the same in pg_logicalinspect contrib module
+ * as well.
  */
 static bool
 SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
@@ -2033,7 +1855,7 @@ snapshot_not_interesting:
 /*
  * Read the contents of the serialized snapshot to 'dest'.
  */
-static void
+void
 SnapBuildRestoreContents(int fd, char *dest, Size size, const char *path)
 {
 	int			readBytes;
diff --git a/src/include/port/pg_crc32c.h b/src/include/port/pg_crc32c.h
index 63c8e3a00b..cfc8c07944 100644
--- a/src/include/port/pg_crc32c.h
+++ b/src/include/port/pg_crc32c.h
@@ -47,7 +47,7 @@ typedef uint32 pg_crc32c;
 	((crc) = pg_comp_crc32c_sse42((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
 
 #elif defined(USE_ARMV8_CRC32C)
 /* Use ARMv8 CRC Extension instructions. */
@@ -56,7 +56,7 @@ extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t le
 	((crc) = pg_comp_crc32c_armv8((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
 
 #elif defined(USE_LOONGARCH_CRC32C)
 /* Use LoongArch CRCC instructions. */
@@ -65,7 +65,7 @@ extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t le
 	((crc) = pg_comp_crc32c_loongarch((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len);
 
 #elif defined(USE_SSE42_CRC32C_WITH_RUNTIME_CHECK) || defined(USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK)
 
@@ -77,14 +77,14 @@ extern pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_
 	((crc) = pg_comp_crc32c((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
-extern pg_crc32c (*pg_comp_crc32c) (pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c (*pg_comp_crc32c) (pg_crc32c crc, const void *data, size_t len);
 
 #ifdef USE_SSE42_CRC32C_WITH_RUNTIME_CHECK
-extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
 #endif
 #ifdef USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK
-extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
 #endif
 
 #else
@@ -103,7 +103,7 @@ extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t le
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 #endif
 
-extern pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
 
 #endif
 
diff --git a/src/include/replication/internal_snapbuild.h b/src/include/replication/internal_snapbuild.h
new file mode 100644
index 0000000000..19a501c30f
--- /dev/null
+++ b/src/include/replication/internal_snapbuild.h
@@ -0,0 +1,204 @@
+/*-------------------------------------------------------------------------
+ *
+ * internal_snapbuild.h
+ *    This file contains declarations for logical decoding utility
+ *    functions for internal use.
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * src/include/replication/internal_snapbuild.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef INTERNAL_SNAPBUILD_H
+#define INTERNAL_SNAPBUILD_H
+
+#include "port/pg_crc32c.h"
+#include "replication/reorderbuffer.h"
+#include "replication/snapbuild.h"
+
+/* -----------------------------------
+ * Snapshot serialization support
+ * -----------------------------------
+ */
+
+#define SnapBuildOnDiskConstantSize \
+	offsetof(SnapBuildOnDisk, builder)
+#define SnapBuildOnDiskNotChecksummedSize \
+	offsetof(SnapBuildOnDisk, version)
+
+#define SNAPBUILD_MAGIC 0x51A1E001
+#define SNAPBUILD_VERSION 6
+
+/*
+ * This struct contains the current state of the snapshot building
+ * machinery. It is exposed to the public, so pay attention when changing its
+ * contents.
+ */
+typedef struct SnapBuild
+{
+	/* how far are we along building our first full snapshot */
+	SnapBuildState state;
+
+	/* private memory context used to allocate memory for this module. */
+	MemoryContext context;
+
+	/* all transactions < than this have committed/aborted */
+	TransactionId xmin;
+
+	/* all transactions >= than this are uncommitted */
+	TransactionId xmax;
+
+	/*
+	 * Don't replay commits from an LSN < this LSN. This can be set externally
+	 * but it will also be advanced (never retreat) from within snapbuild.c.
+	 */
+	XLogRecPtr	start_decoding_at;
+
+	/*
+	 * LSN at which two-phase decoding was enabled or LSN at which we found a
+	 * consistent point at the time of slot creation.
+	 *
+	 * The prepared transactions, that were skipped because previously
+	 * two-phase was not enabled or are not covered by initial snapshot, need
+	 * to be sent later along with commit prepared and they must be before
+	 * this point.
+	 */
+	XLogRecPtr	two_phase_at;
+
+	/*
+	 * Don't start decoding WAL until the "xl_running_xacts" information
+	 * indicates there are no running xids with an xid smaller than this.
+	 */
+	TransactionId initial_xmin_horizon;
+
+	/* Indicates if we are building full snapshot or just catalog one. */
+	bool		building_full_snapshot;
+
+	/*
+	 * Indicates if we are using the snapshot builder for the creation of a
+	 * logical replication slot. If it's true, the start point for decoding
+	 * changes is not determined yet. So we skip snapshot restores to properly
+	 * find the start point. See SnapBuildFindSnapshot() for details.
+	 */
+	bool		in_slot_creation;
+
+	/*
+	 * Snapshot that's valid to see the catalog state seen at this moment.
+	 */
+	Snapshot	snapshot;
+
+	/*
+	 * LSN of the last location we are sure a snapshot has been serialized to.
+	 */
+	XLogRecPtr	last_serialized_snapshot;
+
+	/*
+	 * The reorderbuffer we need to update with usable snapshots et al.
+	 */
+	ReorderBuffer *reorder;
+
+	/*
+	 * TransactionId at which the next phase of initial snapshot building will
+	 * happen. InvalidTransactionId if not known (i.e. SNAPBUILD_START), or
+	 * when no next phase necessary (SNAPBUILD_CONSISTENT).
+	 */
+	TransactionId next_phase_at;
+
+	/*
+	 * Array of transactions which could have catalog changes that committed
+	 * between xmin and xmax.
+	 */
+	struct
+	{
+		/* number of committed transactions */
+		size_t		xcnt;
+
+		/* available space for committed transactions */
+		size_t		xcnt_space;
+
+		/*
+		 * Until we reach a CONSISTENT state, we record commits of all
+		 * transactions, not just the catalog changing ones. Record when that
+		 * changes so we know we cannot export a snapshot safely anymore.
+		 */
+		bool		includes_all_transactions;
+
+		/*
+		 * Array of committed transactions that have modified the catalog.
+		 *
+		 * As this array is frequently modified we do *not* keep it in
+		 * xidComparator order. Instead we sort the array when building &
+		 * distributing a snapshot.
+		 *
+		 * TODO: It's unclear whether that reasoning has much merit. Every
+		 * time we add something here after becoming consistent will also
+		 * require distributing a snapshot. Storing them sorted would
+		 * potentially also make it easier to purge (but more complicated wrt
+		 * wraparound?). Should be improved if sorting while building the
+		 * snapshot shows up in profiles.
+		 */
+		TransactionId *xip;
+	}			committed;
+
+	/*
+	 * Array of transactions and subtransactions that had modified catalogs
+	 * and were running when the snapshot was serialized.
+	 *
+	 * We normally rely on some WAL record types such as HEAP2_NEW_CID to know
+	 * if the transaction has changed the catalog. But it could happen that
+	 * the logical decoding decodes only the commit record of the transaction
+	 * after restoring the previously serialized snapshot in which case we
+	 * will miss adding the xid to the snapshot and end up looking at the
+	 * catalogs with the wrong snapshot.
+	 *
+	 * Now to avoid the above problem, we serialize the transactions that had
+	 * modified the catalogs and are still running at the time of snapshot
+	 * serialization. We fill this array while restoring the snapshot and then
+	 * refer it while decoding commit to ensure if the xact has modified the
+	 * catalog. We discard this array when all the xids in the list become old
+	 * enough to matter. See SnapBuildPurgeOlderTxn for details.
+	 */
+	struct
+	{
+		/* number of transactions */
+		size_t		xcnt;
+
+		/* This array must be sorted in xidComparator order */
+		TransactionId *xip;
+	}			catchange;
+} SnapBuild;
+
+/*
+ * We store current state of struct SnapBuild on disk in the following manner:
+ *
+ * struct SnapBuildOnDisk;
+ * TransactionId * committed.xcnt; (*not xcnt_space*)
+ * TransactionId * catchange.xcnt;
+ *
+ */
+typedef struct SnapBuildOnDisk
+{
+	/* first part of this struct needs to be version independent */
+
+	/* data not covered by checksum */
+	uint32		magic;
+	pg_crc32c	checksum;
+
+	/* data covered by checksum */
+
+	/* version, in case we want to support pg_upgrade */
+	uint32		version;
+	/* how large is the on disk data, excluding the constant sized part */
+	uint32		length;
+
+	/* version dependent part */
+	SnapBuild	builder;
+
+	/* variable amount of TransactionIds follows */
+} SnapBuildOnDisk;
+
+extern void SnapBuildRestoreContents(int fd, char *dest, Size size, const char *path);
+
+#endif							/* INTERNAL_SNAPBUILD_H */
diff --git a/src/include/replication/snapbuild.h b/src/include/replication/snapbuild.h
index caa5113ff8..fbe0798a41 100644
--- a/src/include/replication/snapbuild.h
+++ b/src/include/replication/snapbuild.h
@@ -46,7 +46,7 @@ typedef enum
 	SNAPBUILD_CONSISTENT = 2,
 } SnapBuildState;
 
-/* forward declare so we don't have to expose the struct to the public */
+/* forward declare so we don't have to include internal_snapbuild.h */
 struct SnapBuild;
 typedef struct SnapBuild SnapBuild;
 
-- 
2.34.1

#15shveta malik
shveta.malik@gmail.com
In reply to: Bertrand Drouvot (#14)
Re: Add contrib/pg_logicalsnapinspect

On Wed, Sep 11, 2024 at 4:21 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Yeah, good idea. Done that way in v3 attached.

Thanks for the patch. +1 on the patch's idea. I have started
reviewing/testing it. It is WIP but please find few initial comments:

src/backend/replication/logical/snapbuild.c:

1)
+ fsync_fname("pg_logical/snapshots", true);

Should we use macros PG_LOGICAL_DIR and PG_LOGICAL_SNAPSHOTS_DIR in
ValidateSnapshotFile(), instead of hard coding the path

2)
Same as above in pg_get_logical_snapshot_meta() and
pg_get_logical_snapshot_info()

+ sprintf(path, "pg_logical/snapshots/%X-%X.snap",
+ LSN_FORMAT_ARGS(lsn));                        LSN_FORMAT_ARGS(lsn));

3)
+#include "replication/internal_snapbuild.h"

Shall we name new file as 'snapbuild_internal.h' instead of
'internal_snapbuild.h'. Please see other files' name under
'./src/include/replication':
worker_internal.h
walsender_private.h

4)
+static void ValidateSnapshotFile(XLogRecPtr lsn, SnapBuildOnDisk *ondisk,
+ const char *path);

Is it required? We generally don't add declaration unless required by
compiler. Since definition is prior to usage, it is not needed?

thanks
Shveta

#16Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: shveta malik (#15)
1 attachment(s)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Mon, Sep 16, 2024 at 04:02:51PM +0530, shveta malik wrote:

On Wed, Sep 11, 2024 at 4:21 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Yeah, good idea. Done that way in v3 attached.

Thanks for the patch. +1 on the patch's idea. I have started
reviewing/testing it. It is WIP but please find few initial comments:

Thanks for sharing your thoughts and for the review!

src/backend/replication/logical/snapbuild.c:

1)
+ fsync_fname("pg_logical/snapshots", true);

Should we use macros PG_LOGICAL_DIR and PG_LOGICAL_SNAPSHOTS_DIR in
ValidateSnapshotFile(), instead of hard coding the path

2)
Same as above in pg_get_logical_snapshot_meta() and
pg_get_logical_snapshot_info()

+ sprintf(path, "pg_logical/snapshots/%X-%X.snap",
+ LSN_FORMAT_ARGS(lsn));                        LSN_FORMAT_ARGS(lsn));

Doh! Yeah, agree that we should use those macros. They are coming from c39afc38cf
which has been introduced after v1 of this patch. I thought I took care of it once
c39afc38cf went in, but it looks like I missed it somehow. Done in v4 attached,
Thanks!

3)
+#include "replication/internal_snapbuild.h"

Shall we name new file as 'snapbuild_internal.h' instead of
'internal_snapbuild.h'. Please see other files' name under
'./src/include/replication':
worker_internal.h
walsender_private.h

Agree, that should be snapbuild_internal.h, done in v4.

4)
+static void ValidateSnapshotFile(XLogRecPtr lsn, SnapBuildOnDisk *ondisk,
+ const char *path);

Is it required? We generally don't add declaration unless required by
compiler. Since definition is prior to usage, it is not needed?

I personally prefer to add them even if not required by the compiler. I did not
pay attention that "We generally don't add declaration unless required by compiler"
and (after a quick check) I did not find any reference in the coding style
documentation [1]https://www.postgresql.org/docs/current/source.html. That said, I don't have a strong opinion about that and so
removed in v4. Worth to add a mention in the coding convention doc?

[1]: https://www.postgresql.org/docs/current/source.html

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachments:

v4-0001-Add-contrib-pg_logicalinspect.patchtext/x-diff; charset=us-asciiDownload
From 7f824aae484f7275bb4119e1d5b060bce74c3058 Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Wed, 14 Aug 2024 08:46:05 +0000
Subject: [PATCH v4] Add contrib/pg_logicalinspect

Provides SQL functions that allow to inspect logical decoding components.

It currently allows to inspect the contents of serialized logical snapshots of
a running database cluster, which is useful for debugging or educational
purposes.
---
 contrib/Makefile                              |   1 +
 contrib/meson.build                           |   1 +
 contrib/pg_logicalinspect/.gitignore          |   4 +
 contrib/pg_logicalinspect/Makefile            |  31 +++
 .../expected/logical_inspect.out              |  52 ++++
 contrib/pg_logicalinspect/logicalinspect.conf |   1 +
 contrib/pg_logicalinspect/meson.build         |  39 +++
 .../pg_logicalinspect--1.0.sql                |  43 +++
 contrib/pg_logicalinspect/pg_logicalinspect.c | 248 ++++++++++++++++++
 .../pg_logicalinspect.control                 |   5 +
 .../specs/logical_inspect.spec                |  34 +++
 doc/src/sgml/contrib.sgml                     |   1 +
 doc/src/sgml/filelist.sgml                    |   1 +
 doc/src/sgml/pglogicalinspect.sgml            | 145 ++++++++++
 src/backend/replication/logical/snapbuild.c   | 190 +-------------
 src/include/port/pg_crc32c.h                  |  16 +-
 src/include/replication/snapbuild.h           |   2 +-
 src/include/replication/snapbuild_internal.h  | 204 ++++++++++++++
 18 files changed, 825 insertions(+), 193 deletions(-)
   7.6% contrib/pg_logicalinspect/expected/
   5.7% contrib/pg_logicalinspect/specs/
  32.3% contrib/pg_logicalinspect/
  13.3% doc/src/sgml/
  17.5% src/backend/replication/logical/
   4.1% src/include/port/
  19.0% src/include/replication/

diff --git a/contrib/Makefile b/contrib/Makefile
index abd780f277..952855d9b6 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -32,6 +32,7 @@ SUBDIRS = \
 		passwordcheck	\
 		pg_buffercache	\
 		pg_freespacemap \
+		pg_logicalinspect \
 		pg_prewarm	\
 		pg_stat_statements \
 		pg_surgery	\
diff --git a/contrib/meson.build b/contrib/meson.build
index 14a8906865..159ff41555 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -46,6 +46,7 @@ subdir('passwordcheck')
 subdir('pg_buffercache')
 subdir('pgcrypto')
 subdir('pg_freespacemap')
+subdir('pg_logicalinspect')
 subdir('pg_prewarm')
 subdir('pgrowlocks')
 subdir('pg_stat_statements')
diff --git a/contrib/pg_logicalinspect/.gitignore b/contrib/pg_logicalinspect/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/contrib/pg_logicalinspect/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/contrib/pg_logicalinspect/Makefile b/contrib/pg_logicalinspect/Makefile
new file mode 100644
index 0000000000..55124514d4
--- /dev/null
+++ b/contrib/pg_logicalinspect/Makefile
@@ -0,0 +1,31 @@
+# contrib/pg_logicalinspect/Makefile
+
+MODULE_big = pg_logicalinspect
+OBJS = \
+	$(WIN32RES) \
+	pg_logicalinspect.o
+PGFILEDESC = "pg_logicalinspect - functions to inspect logical decoding components"
+
+EXTENSION = pg_logicalinspect
+DATA = pg_logicalinspect--1.0.sql
+
+EXTRA_INSTALL = contrib/test_decoding
+
+ISOLATION = logical_inspect
+
+ISOLATION_OPTS = --temp-config $(top_srcdir)/contrib/pg_logicalinspect/logicalinspect.conf
+
+# Disabled because these tests require "wal_level=logical", which
+# some installcheck users do not have (e.g. buildfarm clients).
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/pg_logicalinspect
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/pg_logicalinspect/expected/logical_inspect.out b/contrib/pg_logicalinspect/expected/logical_inspect.out
new file mode 100644
index 0000000000..749cd4642d
--- /dev/null
+++ b/contrib/pg_logicalinspect/expected/logical_inspect.out
@@ -0,0 +1,52 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s0_init s0_begin s0_savepoint s0_truncate s1_checkpoint s1_get_changes s0_commit s0_begin s0_insert s1_checkpoint s1_get_changes s0_commit s1_get_changes s1_get_logical_snapshot_info s1_get_logical_snapshot_meta
+step s0_init: SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding');
+?column?
+--------
+init    
+(1 row)
+
+step s0_begin: BEGIN;
+step s0_savepoint: SAVEPOINT sp1;
+step s0_truncate: TRUNCATE tbl1;
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data
+----
+(0 rows)
+
+step s0_commit: COMMIT;
+step s0_begin: BEGIN;
+step s0_insert: INSERT INTO tbl1 VALUES (1);
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                   
+---------------------------------------
+BEGIN                                  
+table public.tbl1: TRUNCATE: (no-flags)
+COMMIT                                 
+(3 rows)
+
+step s0_commit: COMMIT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                                         
+-------------------------------------------------------------
+BEGIN                                                        
+table public.tbl1: INSERT: val1[integer]:1 val2[integer]:null
+COMMIT                                                       
+(3 rows)
+
+step s1_get_logical_snapshot_info: SELECT (pg_get_logical_snapshot_info(f.name::pg_lsn)).state,(pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_xip,1),(pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_xip,1) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f ORDER BY 2;
+state|catchange_count|array_length|committed_count|array_length
+-----+---------------+------------+---------------+------------
+    2|              0|            |              2|           2
+    2|              2|           2|              0|            
+(2 rows)
+
+step s1_get_logical_snapshot_meta: SELECT COUNT((pg_get_logical_snapshot_meta(f.name::pg_lsn))) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f;
+count
+-----
+    2
+(1 row)
+
diff --git a/contrib/pg_logicalinspect/logicalinspect.conf b/contrib/pg_logicalinspect/logicalinspect.conf
new file mode 100644
index 0000000000..e3d257315f
--- /dev/null
+++ b/contrib/pg_logicalinspect/logicalinspect.conf
@@ -0,0 +1 @@
+wal_level = logical
diff --git a/contrib/pg_logicalinspect/meson.build b/contrib/pg_logicalinspect/meson.build
new file mode 100644
index 0000000000..b787dafc9b
--- /dev/null
+++ b/contrib/pg_logicalinspect/meson.build
@@ -0,0 +1,39 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+pg_logicalinspect_sources = files('pg_logicalinspect.c')
+
+if host_system == 'windows'
+  pg_logicalinspect_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+    '--NAME', 'pg_logicalinspect',
+    '--FILEDESC', 'pg_logicalinspect - functions to inspect contents of logical snapshots',])
+endif
+
+pg_logicalinspect = shared_module('pg_logicalinspect',
+  pg_logicalinspect_sources,
+  kwargs: contrib_mod_args + {
+      'dependencies': contrib_mod_args['dependencies'],
+  },
+)
+contrib_targets += pg_logicalinspect
+
+install_data(
+  'pg_logicalinspect.control',
+  'pg_logicalinspect--1.0.sql',
+  kwargs: contrib_data_args,
+)
+
+tests += {
+  'name': 'pg_logicalinspect',
+  'sd': meson.current_source_dir(),
+  'bd': meson.current_build_dir(),
+  'isolation': {
+    'specs': [
+      'logical_inspect',
+    ],
+    'regress_args': [
+      '--temp-config', files('logicalinspect.conf'),
+    ],
+    # see above
+    'runningcheck': false,
+  },
+}
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql b/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
new file mode 100644
index 0000000000..51713ed53e
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
@@ -0,0 +1,43 @@
+/* contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_logicalinspect" to load this file. \quit
+
+--
+-- pg_get_logical_snapshot_meta()
+--
+CREATE FUNCTION pg_get_logical_snapshot_meta(IN in_lsn pg_lsn,
+    OUT magic int4,
+    OUT checksum int4,
+    OUT version int4
+)
+AS 'MODULE_PATHNAME', 'pg_get_logical_snapshot_meta'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_logical_snapshot_meta(pg_lsn) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_logical_snapshot_meta(pg_lsn) TO pg_read_server_files;
+
+--
+-- pg_get_logical_snapshot_info()
+--
+CREATE FUNCTION pg_get_logical_snapshot_info(IN in_lsn pg_lsn,
+    OUT state int2,
+    OUT xmin xid,
+    OUT xmax xid,
+    OUT start_decoding_at pg_lsn,
+    OUT two_phase_at pg_lsn,
+    OUT initial_xmin_horizon xid,
+    OUT building_full_snapshot boolean,
+    OUT in_slot_creation boolean,
+    OUT last_serialized_snapshot pg_lsn,
+    OUT next_phase_at xid,
+    OUT committed_count int8,
+    OUT committed_xip xid[],
+    OUT catchange_count int8,
+    OUT catchange_xip xid[]
+)
+AS 'MODULE_PATHNAME', 'pg_get_logical_snapshot_info'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_logical_snapshot_info(pg_lsn) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_logical_snapshot_info(pg_lsn) TO pg_read_server_files;
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.c b/contrib/pg_logicalinspect/pg_logicalinspect.c
new file mode 100644
index 0000000000..74311b9d3b
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.c
@@ -0,0 +1,248 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_logicalinspect.c
+ *		  Functions to inspect contents of PostgreSQL logical snapshots
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ *		  contrib/pg_logicalinspect/pg_logicalinspect.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "funcapi.h"
+#include "port/pg_crc32c.h"
+#include "replication/snapbuild_internal.h"
+#include "utils/array.h"
+#include "utils/pg_lsn.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_meta);
+PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_info);
+
+/*
+ * NOTE: For any code change or issue fix here, it is highly recommended to
+ * give a thought about doing the same in SnapBuildRestore() as well.
+ */
+
+/*
+ * Validate the logical snapshot file.
+ */
+static void
+ValidateSnapshotFile(XLogRecPtr lsn, SnapBuildOnDisk *ondisk, const char *path)
+{
+	int			fd;
+	Size		sz;
+	pg_crc32c	checksum;
+	MemoryContext context;
+
+	context = AllocSetContextCreate(CurrentMemoryContext,
+									"logicalsnapshot inspect context",
+									ALLOCSET_DEFAULT_SIZES);
+
+	fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
+
+	if (fd < 0 && errno == ENOENT)
+		ereport(ERROR,
+				errmsg("file \"%s\" does not exist", path));
+	else if (fd < 0)
+		ereport(ERROR,
+				(errcode_for_file_access(),
+				 errmsg("could not open file \"%s\": %m", path)));
+
+	/* ----
+	 * Make sure the snapshot had been stored safely to disk, that's normally
+	 * cheap.
+	 * Note that we do not need PANIC here, nobody will be able to use the
+	 * slot without fsyncing, and saving it won't succeed without an fsync()
+	 * either...
+	 * ----
+	 */
+	fsync_fname(path, false);
+	fsync_fname(PG_LOGICAL_SNAPSHOTS_DIR, true);
+
+
+	/* read statically sized portion of snapshot */
+	SnapBuildRestoreContents(fd, (char *) ondisk, SnapBuildOnDiskConstantSize, path);
+
+	if (ondisk->magic != SNAPBUILD_MAGIC)
+		ereport(ERROR,
+				(errcode(ERRCODE_DATA_CORRUPTED),
+				 errmsg("snapbuild state file \"%s\" has wrong magic number: %u instead of %u",
+						path, ondisk->magic, SNAPBUILD_MAGIC)));
+
+	if (ondisk->version != SNAPBUILD_VERSION)
+		ereport(ERROR,
+				(errcode(ERRCODE_DATA_CORRUPTED),
+				 errmsg("snapbuild state file \"%s\" has unsupported version: %u instead of %u",
+						path, ondisk->version, SNAPBUILD_VERSION)));
+
+	INIT_CRC32C(checksum);
+	COMP_CRC32C(checksum,
+				((char *) ondisk) + SnapBuildOnDiskNotChecksummedSize,
+				SnapBuildOnDiskConstantSize - SnapBuildOnDiskNotChecksummedSize);
+
+	/* read SnapBuild */
+	SnapBuildRestoreContents(fd, (char *) &ondisk->builder, sizeof(SnapBuild), path);
+	COMP_CRC32C(checksum, &ondisk->builder, sizeof(SnapBuild));
+
+	ondisk->builder.context = context;
+
+	/* restore committed xacts information */
+	if (ondisk->builder.committed.xcnt > 0)
+	{
+		sz = sizeof(TransactionId) * ondisk->builder.committed.xcnt;
+		ondisk->builder.committed.xip = MemoryContextAllocZero(ondisk->builder.context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.committed.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.committed.xip, sz);
+	}
+
+	/* restore catalog modifying xacts information */
+	if (ondisk->builder.catchange.xcnt > 0)
+	{
+		sz = sizeof(TransactionId) * ondisk->builder.catchange.xcnt;
+		ondisk->builder.catchange.xip = MemoryContextAllocZero(ondisk->builder.context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.catchange.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.catchange.xip, sz);
+	}
+
+	if (CloseTransientFile(fd) != 0)
+		ereport(ERROR,
+				(errcode_for_file_access(),
+				 errmsg("could not close file \"%s\": %m", path)));
+
+	FIN_CRC32C(checksum);
+
+	/* verify checksum of what we've read */
+	if (!EQ_CRC32C(checksum, ondisk->checksum))
+		ereport(ERROR,
+				(errcode(ERRCODE_DATA_CORRUPTED),
+				 errmsg("checksum mismatch for snapbuild state file \"%s\": is %u, should be %u",
+						path, checksum, ondisk->checksum)));
+}
+
+/*
+ * Retrieve the logical snapshot file metadata.
+ */
+Datum
+pg_get_logical_snapshot_meta(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_META_COLS 3
+	SnapBuildOnDisk ondisk;
+	XLogRecPtr	lsn;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_META_COLS];
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_META_COLS];
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+
+	lsn = PG_GETARG_LSN(0);
+
+	sprintf(path, "%s/%X-%X.snap",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			LSN_FORMAT_ARGS(lsn));
+
+	ValidateSnapshotFile(lsn, &ondisk, path);
+
+	/* Build a tuple descriptor for our result type. */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	memset(nulls, 0, sizeof(nulls));
+
+	values[0] = Int32GetDatum(ondisk.magic);
+	values[1] = Int32GetDatum(ondisk.checksum);
+	values[2] = Int32GetDatum(ondisk.version);
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	MemoryContextReset(ondisk.builder.context);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_META_COLS
+}
+
+Datum
+pg_get_logical_snapshot_info(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_INFO_COLS 14
+	SnapBuildOnDisk ondisk;
+	XLogRecPtr	lsn;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS];
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS];
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+
+	lsn = PG_GETARG_LSN(0);
+
+	sprintf(path, "%s/%X-%X.snap",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			LSN_FORMAT_ARGS(lsn));
+
+	ValidateSnapshotFile(lsn, &ondisk, path);
+
+	/* Build a tuple descriptor for our result type. */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	memset(nulls, 0, sizeof(nulls));
+
+	values[0] = Int16GetDatum(ondisk.builder.state);
+	values[1] = TransactionIdGetDatum(ondisk.builder.xmin);
+	values[2] = TransactionIdGetDatum(ondisk.builder.xmax);
+	values[3] = LSNGetDatum(ondisk.builder.start_decoding_at);
+	values[4] = LSNGetDatum(ondisk.builder.two_phase_at);
+	values[5] = TransactionIdGetDatum(ondisk.builder.initial_xmin_horizon);
+	values[6] = BoolGetDatum(ondisk.builder.building_full_snapshot);
+	values[7] = BoolGetDatum(ondisk.builder.in_slot_creation);
+	values[8] = LSNGetDatum(ondisk.builder.last_serialized_snapshot);
+	values[9] = TransactionIdGetDatum(ondisk.builder.next_phase_at);
+	values[10] = Int64GetDatum(ondisk.builder.committed.xcnt);
+
+	if (ondisk.builder.committed.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+		int			narrayelems;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.committed.xcnt * sizeof(Datum));
+		narrayelems = 0;
+
+		for (narrayelems = 0; narrayelems < ondisk.builder.committed.xcnt; narrayelems++)
+			arrayelems[narrayelems] = Int64GetDatum((int64) ondisk.builder.committed.xip[narrayelems]);
+
+		values[11] = PointerGetDatum(construct_array_builtin(arrayelems, narrayelems, INT8OID));
+	}
+	else
+		nulls[11] = true;
+
+	values[12] = Int64GetDatum(ondisk.builder.catchange.xcnt);
+
+	if (ondisk.builder.catchange.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+		int			narrayelems;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.catchange.xcnt * sizeof(Datum));
+		narrayelems = 0;
+
+		for (narrayelems = 0; narrayelems < ondisk.builder.catchange.xcnt; narrayelems++)
+			arrayelems[narrayelems] = Int64GetDatum((int64) ondisk.builder.catchange.xip[narrayelems]);
+
+		values[13] = PointerGetDatum(construct_array_builtin(arrayelems, narrayelems, INT8OID));
+	}
+	else
+		nulls[13] = true;
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	MemoryContextReset(ondisk.builder.context);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_INFO_COLS
+}
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.control b/contrib/pg_logicalinspect/pg_logicalinspect.control
new file mode 100644
index 0000000000..b4a70e57ba
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.control
@@ -0,0 +1,5 @@
+# pg_logicalinspect extension
+comment = 'functions to inspect logical decoding components'
+default_version = '1.0'
+module_pathname = '$libdir/pg_logicalinspect'
+relocatable = true
diff --git a/contrib/pg_logicalinspect/specs/logical_inspect.spec b/contrib/pg_logicalinspect/specs/logical_inspect.spec
new file mode 100644
index 0000000000..e11eb63615
--- /dev/null
+++ b/contrib/pg_logicalinspect/specs/logical_inspect.spec
@@ -0,0 +1,34 @@
+# Test the pg_logicalinspect functions: that needs some permutation to
+# ensure that we are creating multiple logical snapshots and that one of them
+# contains ongoing catalogs changes.
+setup
+{
+    DROP TABLE IF EXISTS tbl1;
+    CREATE TABLE tbl1 (val1 integer, val2 integer);
+	CREATE EXTENSION pg_logicalinspect;
+}
+
+teardown
+{
+    DROP TABLE tbl1;
+    SELECT 'stop' FROM pg_drop_replication_slot('isolation_slot');
+	DROP EXTENSION pg_logicalinspect;
+}
+
+session "s0"
+setup { SET synchronous_commit=on; }
+step "s0_init" { SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding'); }
+step "s0_begin" { BEGIN; }
+step "s0_savepoint" { SAVEPOINT sp1; }
+step "s0_truncate" { TRUNCATE tbl1; }
+step "s0_insert" { INSERT INTO tbl1 VALUES (1); }
+step "s0_commit" { COMMIT; }
+
+session "s1"
+setup { SET synchronous_commit=on; }
+step "s1_checkpoint" { CHECKPOINT; }
+step "s1_get_changes" { SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0'); }
+step "s1_get_logical_snapshot_meta" { SELECT COUNT((pg_get_logical_snapshot_meta(f.name::pg_lsn))) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f; }
+step "s1_get_logical_snapshot_info" { SELECT (pg_get_logical_snapshot_info(f.name::pg_lsn)).state,(pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_xip,1),(pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_xip,1) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f ORDER BY 2; }
+
+permutation "s0_init" "s0_begin" "s0_savepoint" "s0_truncate" "s1_checkpoint" "s1_get_changes" "s0_commit" "s0_begin" "s0_insert" "s1_checkpoint" "s1_get_changes" "s0_commit" "s1_get_changes" "s1_get_logical_snapshot_info" "s1_get_logical_snapshot_meta"
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index 44639a8dca..7c381949a5 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -154,6 +154,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
  &pgbuffercache;
  &pgcrypto;
  &pgfreespacemap;
+ &pglogicalinspect;
  &pgprewarm;
  &pgrowlocks;
  &pgstatstatements;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index a7ff5f8264..66e6dccd4c 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -143,6 +143,7 @@
 <!ENTITY pgbuffercache   SYSTEM "pgbuffercache.sgml">
 <!ENTITY pgcrypto        SYSTEM "pgcrypto.sgml">
 <!ENTITY pgfreespacemap  SYSTEM "pgfreespacemap.sgml">
+<!ENTITY pglogicalinspect  SYSTEM "pglogicalinspect.sgml">
 <!ENTITY pgprewarm       SYSTEM "pgprewarm.sgml">
 <!ENTITY pgrowlocks      SYSTEM "pgrowlocks.sgml">
 <!ENTITY pgstatstatements SYSTEM "pgstatstatements.sgml">
diff --git a/doc/src/sgml/pglogicalinspect.sgml b/doc/src/sgml/pglogicalinspect.sgml
new file mode 100644
index 0000000000..7767de263f
--- /dev/null
+++ b/doc/src/sgml/pglogicalinspect.sgml
@@ -0,0 +1,145 @@
+<!-- doc/src/sgml/pglogicalinspect.sgml -->
+
+<sect1 id="pglogicalinspect" xreflabel="pg_logicalinspect">
+ <title>pg_logicalinspect &mdash; logical decoding components inspection</title>
+
+ <indexterm zone="pglogicalinspect">
+  <primary>pg_logicalinspect</primary>
+ </indexterm>
+
+ <para>
+  The <filename>pg_logicalinspect</filename> module provides SQL functions
+  that allow you to inspect the contents of logical decoding components. It
+  allows to inspect serialized logical snapshots of a running
+  <productname>PostgreSQL</productname> database cluster, which is useful
+  for debugging or educational purposes.
+ </para>
+
+ <note>
+  <para>
+   The <filename>pg_logicalinspect</filename> functions are called
+   using an LSN argument that can be extracted from the output name of the
+   <function>pg_ls_logicalsnapdir</function>() function.
+  </para>
+ </note>
+
+ <sect2 id="pglogicalinspect-funcs">
+  <title>General Functions</title>
+
+  <variablelist>
+   <varlistentry id="pglogicalinspect-funcs-pg-get-logical-snapshot-meta">
+    <term>
+     <function>pg_get_logical_snapshot_meta(in_lsn pg_lsn) returns record</function>
+    </term>
+
+    <listitem>
+     <para>
+      Gets logical snapshot metadata about a snapshot file that is located in
+      the <filename>pg_logical/snapshots</filename> directory.
+      The <replaceable>in_lsn</replaceable> argument can be extracted from the
+      snapshot file name.
+      example:
+<screen>
+postgres=# SELECT * FROM pg_ls_logicalsnapdir();
+-[ RECORD 1 ]+-----------------------
+name         | 0-40796E18.snap
+size         | 152
+modification | 2024-08-14 16:36:32+00
+
+postgres=# SELECT * FROM pg_get_logical_snapshot_meta('0/40796E18');
+-[ RECORD 1 ]--------
+magic    | 1369563137
+checksum | 1028045905
+version  | 6
+
+postgres=# SELECT (pg_get_logical_snapshot_meta(f.name::pg_lsn)).*
+           FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name
+                 FROM pg_ls_logicalsnapdir()) AS f;
+-[ RECORD 1 ]--------
+magic    | 1369563137
+checksum | 1028045905
+version  | 6
+</screen>
+     </para>
+     <para>
+      If <replaceable>in_lsn</replaceable> does not match a snapshot file, the
+      function raises an error.
+     </para>
+    </listitem>
+   </varlistentry>
+
+   <varlistentry id="pglogicalinspect-funcs-pg-get-logical-snapshot-info">
+    <term>
+     <function>pg_get_logical_snapshot_info(in_lsn pg_lsn) returns record</function>
+    </term>
+
+    <listitem>
+     <para>
+      Gets logical snapshot information about a snapshot file that is located in
+      the <filename>pg_logical/snapshots</filename> directory.
+      The <replaceable>in_lsn</replaceable> argument can be extracted from the
+      snapshot file name.
+      example:
+<screen>
+postgres=# SELECT * FROM pg_ls_logicalsnapdir();
+-[ RECORD 1 ]+-----------------------
+name         | 0-40796E18.snap
+size         | 152
+modification | 2024-08-14 16:36:32+00
+
+postgres=# SELECT * FROM pg_get_logical_snapshot_info('0/40796E18');
+-[ RECORD 1 ]------------+-----------
+state                    | 2
+xmin                     | 751
+xmax                     | 751
+start_decoding_at        | 0/40796AF8
+two_phase_at             | 0/40796AF8
+initial_xmin_horizon     | 0
+building_full_snapshot   | f
+in_slot_creation         | f
+last_serialized_snapshot | 0/0
+next_phase_at            | 0
+committed_count          | 0
+committed_xip            |
+catchange_count          | 2
+catchange_xip            | {751,752}
+
+postgres=# SELECT (pg_get_logical_snapshot_info(f.name::pg_lsn)).*
+           FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name
+                 FROM pg_ls_logicalsnapdir()) AS f;
+-[ RECORD 1 ]------------+-----------
+state                    | 2
+xmin                     | 751
+xmax                     | 751
+start_decoding_at        | 0/40796AF8
+two_phase_at             | 0/40796AF8
+initial_xmin_horizon     | 0
+building_full_snapshot   | f
+in_slot_creation         | f
+last_serialized_snapshot | 0/0
+next_phase_at            | 0
+committed_count          | 0
+committed_xip            |
+catchange_count          | 2
+catchange_xip            | {751,752}
+</screen>
+     </para>
+     <para>
+      If <replaceable>in_lsn</replaceable> does not match a snapshot file, the
+      function raises an error.
+     </para>
+    </listitem>
+   </varlistentry>
+
+  </variablelist>
+ </sect2>
+
+ <sect2 id="pglogicalinspect-author">
+  <title>Author</title>
+
+  <para>
+   Bertrand Drouvot <email>bertranddrouvot.pg@gmail.com</email>
+  </para>
+ </sect2>
+
+</sect1>
diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c
index 0450f94ba8..bb7def9440 100644
--- a/src/backend/replication/logical/snapbuild.c
+++ b/src/backend/replication/logical/snapbuild.c
@@ -134,6 +134,7 @@
 #include "replication/logical.h"
 #include "replication/reorderbuffer.h"
 #include "replication/snapbuild.h"
+#include "replication/snapbuild_internal.h"
 #include "storage/fd.h"
 #include "storage/lmgr.h"
 #include "storage/proc.h"
@@ -143,146 +144,6 @@
 #include "utils/memutils.h"
 #include "utils/snapmgr.h"
 #include "utils/snapshot.h"
-
-/*
- * This struct contains the current state of the snapshot building
- * machinery. Besides a forward declaration in the header, it is not exposed
- * to the public, so we can easily change its contents.
- */
-struct SnapBuild
-{
-	/* how far are we along building our first full snapshot */
-	SnapBuildState state;
-
-	/* private memory context used to allocate memory for this module. */
-	MemoryContext context;
-
-	/* all transactions < than this have committed/aborted */
-	TransactionId xmin;
-
-	/* all transactions >= than this are uncommitted */
-	TransactionId xmax;
-
-	/*
-	 * Don't replay commits from an LSN < this LSN. This can be set externally
-	 * but it will also be advanced (never retreat) from within snapbuild.c.
-	 */
-	XLogRecPtr	start_decoding_at;
-
-	/*
-	 * LSN at which two-phase decoding was enabled or LSN at which we found a
-	 * consistent point at the time of slot creation.
-	 *
-	 * The prepared transactions, that were skipped because previously
-	 * two-phase was not enabled or are not covered by initial snapshot, need
-	 * to be sent later along with commit prepared and they must be before
-	 * this point.
-	 */
-	XLogRecPtr	two_phase_at;
-
-	/*
-	 * Don't start decoding WAL until the "xl_running_xacts" information
-	 * indicates there are no running xids with an xid smaller than this.
-	 */
-	TransactionId initial_xmin_horizon;
-
-	/* Indicates if we are building full snapshot or just catalog one. */
-	bool		building_full_snapshot;
-
-	/*
-	 * Indicates if we are using the snapshot builder for the creation of a
-	 * logical replication slot. If it's true, the start point for decoding
-	 * changes is not determined yet. So we skip snapshot restores to properly
-	 * find the start point. See SnapBuildFindSnapshot() for details.
-	 */
-	bool		in_slot_creation;
-
-	/*
-	 * Snapshot that's valid to see the catalog state seen at this moment.
-	 */
-	Snapshot	snapshot;
-
-	/*
-	 * LSN of the last location we are sure a snapshot has been serialized to.
-	 */
-	XLogRecPtr	last_serialized_snapshot;
-
-	/*
-	 * The reorderbuffer we need to update with usable snapshots et al.
-	 */
-	ReorderBuffer *reorder;
-
-	/*
-	 * TransactionId at which the next phase of initial snapshot building will
-	 * happen. InvalidTransactionId if not known (i.e. SNAPBUILD_START), or
-	 * when no next phase necessary (SNAPBUILD_CONSISTENT).
-	 */
-	TransactionId next_phase_at;
-
-	/*
-	 * Array of transactions which could have catalog changes that committed
-	 * between xmin and xmax.
-	 */
-	struct
-	{
-		/* number of committed transactions */
-		size_t		xcnt;
-
-		/* available space for committed transactions */
-		size_t		xcnt_space;
-
-		/*
-		 * Until we reach a CONSISTENT state, we record commits of all
-		 * transactions, not just the catalog changing ones. Record when that
-		 * changes so we know we cannot export a snapshot safely anymore.
-		 */
-		bool		includes_all_transactions;
-
-		/*
-		 * Array of committed transactions that have modified the catalog.
-		 *
-		 * As this array is frequently modified we do *not* keep it in
-		 * xidComparator order. Instead we sort the array when building &
-		 * distributing a snapshot.
-		 *
-		 * TODO: It's unclear whether that reasoning has much merit. Every
-		 * time we add something here after becoming consistent will also
-		 * require distributing a snapshot. Storing them sorted would
-		 * potentially also make it easier to purge (but more complicated wrt
-		 * wraparound?). Should be improved if sorting while building the
-		 * snapshot shows up in profiles.
-		 */
-		TransactionId *xip;
-	}			committed;
-
-	/*
-	 * Array of transactions and subtransactions that had modified catalogs
-	 * and were running when the snapshot was serialized.
-	 *
-	 * We normally rely on some WAL record types such as HEAP2_NEW_CID to know
-	 * if the transaction has changed the catalog. But it could happen that
-	 * the logical decoding decodes only the commit record of the transaction
-	 * after restoring the previously serialized snapshot in which case we
-	 * will miss adding the xid to the snapshot and end up looking at the
-	 * catalogs with the wrong snapshot.
-	 *
-	 * Now to avoid the above problem, we serialize the transactions that had
-	 * modified the catalogs and are still running at the time of snapshot
-	 * serialization. We fill this array while restoring the snapshot and then
-	 * refer it while decoding commit to ensure if the xact has modified the
-	 * catalog. We discard this array when all the xids in the list become old
-	 * enough to matter. See SnapBuildPurgeOlderTxn for details.
-	 */
-	struct
-	{
-		/* number of transactions */
-		size_t		xcnt;
-
-		/* This array must be sorted in xidComparator order */
-		TransactionId *xip;
-	}			catchange;
-};
-
 /*
  * Starting a transaction -- which we need to do while exporting a snapshot --
  * removes knowledge about the previously used resowner, so we save it here.
@@ -312,7 +173,6 @@ static void SnapBuildWaitSnapshot(xl_running_xacts *running, TransactionId cutof
 /* serialization functions */
 static void SnapBuildSerialize(SnapBuild *builder, XLogRecPtr lsn);
 static bool SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn);
-static void SnapBuildRestoreContents(int fd, char *dest, Size size, const char *path);
 
 /*
  * Allocate a new snapshot builder.
@@ -1557,48 +1417,6 @@ SnapBuildWaitSnapshot(xl_running_xacts *running, TransactionId cutoff)
 	}
 }
 
-/* -----------------------------------
- * Snapshot serialization support
- * -----------------------------------
- */
-
-/*
- * We store current state of struct SnapBuild on disk in the following manner:
- *
- * struct SnapBuildOnDisk;
- * TransactionId * committed.xcnt; (*not xcnt_space*)
- * TransactionId * catchange.xcnt;
- *
- */
-typedef struct SnapBuildOnDisk
-{
-	/* first part of this struct needs to be version independent */
-
-	/* data not covered by checksum */
-	uint32		magic;
-	pg_crc32c	checksum;
-
-	/* data covered by checksum */
-
-	/* version, in case we want to support pg_upgrade */
-	uint32		version;
-	/* how large is the on disk data, excluding the constant sized part */
-	uint32		length;
-
-	/* version dependent part */
-	SnapBuild	builder;
-
-	/* variable amount of TransactionIds follows */
-} SnapBuildOnDisk;
-
-#define SnapBuildOnDiskConstantSize \
-	offsetof(SnapBuildOnDisk, builder)
-#define SnapBuildOnDiskNotChecksummedSize \
-	offsetof(SnapBuildOnDisk, version)
-
-#define SNAPBUILD_MAGIC 0x51A1E001
-#define SNAPBUILD_VERSION 6
-
 /*
  * Store/Load a snapshot from disk, depending on the snapshot builder's state.
  *
@@ -1859,6 +1677,10 @@ out:
 /*
  * Restore a snapshot into 'builder' if previously one has been stored at the
  * location indicated by 'lsn'. Returns true if successful, false otherwise.
+ *
+ * NOTE: For any code change or issue fix here, it is highly recommended to
+ * give a thought about doing the same in pg_logicalinspect contrib module
+ * as well.
  */
 static bool
 SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
@@ -2033,7 +1855,7 @@ snapshot_not_interesting:
 /*
  * Read the contents of the serialized snapshot to 'dest'.
  */
-static void
+void
 SnapBuildRestoreContents(int fd, char *dest, Size size, const char *path)
 {
 	int			readBytes;
diff --git a/src/include/port/pg_crc32c.h b/src/include/port/pg_crc32c.h
index 63c8e3a00b..cfc8c07944 100644
--- a/src/include/port/pg_crc32c.h
+++ b/src/include/port/pg_crc32c.h
@@ -47,7 +47,7 @@ typedef uint32 pg_crc32c;
 	((crc) = pg_comp_crc32c_sse42((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
 
 #elif defined(USE_ARMV8_CRC32C)
 /* Use ARMv8 CRC Extension instructions. */
@@ -56,7 +56,7 @@ extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t le
 	((crc) = pg_comp_crc32c_armv8((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
 
 #elif defined(USE_LOONGARCH_CRC32C)
 /* Use LoongArch CRCC instructions. */
@@ -65,7 +65,7 @@ extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t le
 	((crc) = pg_comp_crc32c_loongarch((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len);
 
 #elif defined(USE_SSE42_CRC32C_WITH_RUNTIME_CHECK) || defined(USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK)
 
@@ -77,14 +77,14 @@ extern pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_
 	((crc) = pg_comp_crc32c((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
-extern pg_crc32c (*pg_comp_crc32c) (pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c (*pg_comp_crc32c) (pg_crc32c crc, const void *data, size_t len);
 
 #ifdef USE_SSE42_CRC32C_WITH_RUNTIME_CHECK
-extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
 #endif
 #ifdef USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK
-extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
 #endif
 
 #else
@@ -103,7 +103,7 @@ extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t le
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 #endif
 
-extern pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
 
 #endif
 
diff --git a/src/include/replication/snapbuild.h b/src/include/replication/snapbuild.h
index caa5113ff8..dbb4bc2f4b 100644
--- a/src/include/replication/snapbuild.h
+++ b/src/include/replication/snapbuild.h
@@ -46,7 +46,7 @@ typedef enum
 	SNAPBUILD_CONSISTENT = 2,
 } SnapBuildState;
 
-/* forward declare so we don't have to expose the struct to the public */
+/* forward declare so we don't have to include snapbuild_internal.h */
 struct SnapBuild;
 typedef struct SnapBuild SnapBuild;
 
diff --git a/src/include/replication/snapbuild_internal.h b/src/include/replication/snapbuild_internal.h
new file mode 100644
index 0000000000..4d4d688470
--- /dev/null
+++ b/src/include/replication/snapbuild_internal.h
@@ -0,0 +1,204 @@
+/*-------------------------------------------------------------------------
+ *
+ * snapbuild_internal.h
+ *    This file contains declarations for logical decoding utility
+ *    functions for internal use.
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * src/include/replication/snapbuild_internal.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef INTERNAL_SNAPBUILD_H
+#define INTERNAL_SNAPBUILD_H
+
+#include "port/pg_crc32c.h"
+#include "replication/reorderbuffer.h"
+#include "replication/snapbuild.h"
+
+/* -----------------------------------
+ * Snapshot serialization support
+ * -----------------------------------
+ */
+
+#define SnapBuildOnDiskConstantSize \
+	offsetof(SnapBuildOnDisk, builder)
+#define SnapBuildOnDiskNotChecksummedSize \
+	offsetof(SnapBuildOnDisk, version)
+
+#define SNAPBUILD_MAGIC 0x51A1E001
+#define SNAPBUILD_VERSION 6
+
+/*
+ * This struct contains the current state of the snapshot building
+ * machinery. It is exposed to the public, so pay attention when changing its
+ * contents.
+ */
+typedef struct SnapBuild
+{
+	/* how far are we along building our first full snapshot */
+	SnapBuildState state;
+
+	/* private memory context used to allocate memory for this module. */
+	MemoryContext context;
+
+	/* all transactions < than this have committed/aborted */
+	TransactionId xmin;
+
+	/* all transactions >= than this are uncommitted */
+	TransactionId xmax;
+
+	/*
+	 * Don't replay commits from an LSN < this LSN. This can be set externally
+	 * but it will also be advanced (never retreat) from within snapbuild.c.
+	 */
+	XLogRecPtr	start_decoding_at;
+
+	/*
+	 * LSN at which two-phase decoding was enabled or LSN at which we found a
+	 * consistent point at the time of slot creation.
+	 *
+	 * The prepared transactions, that were skipped because previously
+	 * two-phase was not enabled or are not covered by initial snapshot, need
+	 * to be sent later along with commit prepared and they must be before
+	 * this point.
+	 */
+	XLogRecPtr	two_phase_at;
+
+	/*
+	 * Don't start decoding WAL until the "xl_running_xacts" information
+	 * indicates there are no running xids with an xid smaller than this.
+	 */
+	TransactionId initial_xmin_horizon;
+
+	/* Indicates if we are building full snapshot or just catalog one. */
+	bool		building_full_snapshot;
+
+	/*
+	 * Indicates if we are using the snapshot builder for the creation of a
+	 * logical replication slot. If it's true, the start point for decoding
+	 * changes is not determined yet. So we skip snapshot restores to properly
+	 * find the start point. See SnapBuildFindSnapshot() for details.
+	 */
+	bool		in_slot_creation;
+
+	/*
+	 * Snapshot that's valid to see the catalog state seen at this moment.
+	 */
+	Snapshot	snapshot;
+
+	/*
+	 * LSN of the last location we are sure a snapshot has been serialized to.
+	 */
+	XLogRecPtr	last_serialized_snapshot;
+
+	/*
+	 * The reorderbuffer we need to update with usable snapshots et al.
+	 */
+	ReorderBuffer *reorder;
+
+	/*
+	 * TransactionId at which the next phase of initial snapshot building will
+	 * happen. InvalidTransactionId if not known (i.e. SNAPBUILD_START), or
+	 * when no next phase necessary (SNAPBUILD_CONSISTENT).
+	 */
+	TransactionId next_phase_at;
+
+	/*
+	 * Array of transactions which could have catalog changes that committed
+	 * between xmin and xmax.
+	 */
+	struct
+	{
+		/* number of committed transactions */
+		size_t		xcnt;
+
+		/* available space for committed transactions */
+		size_t		xcnt_space;
+
+		/*
+		 * Until we reach a CONSISTENT state, we record commits of all
+		 * transactions, not just the catalog changing ones. Record when that
+		 * changes so we know we cannot export a snapshot safely anymore.
+		 */
+		bool		includes_all_transactions;
+
+		/*
+		 * Array of committed transactions that have modified the catalog.
+		 *
+		 * As this array is frequently modified we do *not* keep it in
+		 * xidComparator order. Instead we sort the array when building &
+		 * distributing a snapshot.
+		 *
+		 * TODO: It's unclear whether that reasoning has much merit. Every
+		 * time we add something here after becoming consistent will also
+		 * require distributing a snapshot. Storing them sorted would
+		 * potentially also make it easier to purge (but more complicated wrt
+		 * wraparound?). Should be improved if sorting while building the
+		 * snapshot shows up in profiles.
+		 */
+		TransactionId *xip;
+	}			committed;
+
+	/*
+	 * Array of transactions and subtransactions that had modified catalogs
+	 * and were running when the snapshot was serialized.
+	 *
+	 * We normally rely on some WAL record types such as HEAP2_NEW_CID to know
+	 * if the transaction has changed the catalog. But it could happen that
+	 * the logical decoding decodes only the commit record of the transaction
+	 * after restoring the previously serialized snapshot in which case we
+	 * will miss adding the xid to the snapshot and end up looking at the
+	 * catalogs with the wrong snapshot.
+	 *
+	 * Now to avoid the above problem, we serialize the transactions that had
+	 * modified the catalogs and are still running at the time of snapshot
+	 * serialization. We fill this array while restoring the snapshot and then
+	 * refer it while decoding commit to ensure if the xact has modified the
+	 * catalog. We discard this array when all the xids in the list become old
+	 * enough to matter. See SnapBuildPurgeOlderTxn for details.
+	 */
+	struct
+	{
+		/* number of transactions */
+		size_t		xcnt;
+
+		/* This array must be sorted in xidComparator order */
+		TransactionId *xip;
+	}			catchange;
+} SnapBuild;
+
+/*
+ * We store current state of struct SnapBuild on disk in the following manner:
+ *
+ * struct SnapBuildOnDisk;
+ * TransactionId * committed.xcnt; (*not xcnt_space*)
+ * TransactionId * catchange.xcnt;
+ *
+ */
+typedef struct SnapBuildOnDisk
+{
+	/* first part of this struct needs to be version independent */
+
+	/* data not covered by checksum */
+	uint32		magic;
+	pg_crc32c	checksum;
+
+	/* data covered by checksum */
+
+	/* version, in case we want to support pg_upgrade */
+	uint32		version;
+	/* how large is the on disk data, excluding the constant sized part */
+	uint32		length;
+
+	/* version dependent part */
+	SnapBuild	builder;
+
+	/* variable amount of TransactionIds follows */
+} SnapBuildOnDisk;
+
+extern void SnapBuildRestoreContents(int fd, char *dest, Size size, const char *path);
+
+#endif							/* INTERNAL_SNAPBUILD_H */
-- 
2.34.1

#17shveta malik
shveta.malik@gmail.com
In reply to: Bertrand Drouvot (#16)
Re: Add contrib/pg_logicalsnapinspect

On Mon, Sep 16, 2024 at 8:03 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Hi,

On Mon, Sep 16, 2024 at 04:02:51PM +0530, shveta malik wrote:

On Wed, Sep 11, 2024 at 4:21 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Yeah, good idea. Done that way in v3 attached.

Thanks for the patch. +1 on the patch's idea. I have started
reviewing/testing it. It is WIP but please find few initial comments:

Thanks for sharing your thoughts and for the review!

src/backend/replication/logical/snapbuild.c:

1)
+ fsync_fname("pg_logical/snapshots", true);

Should we use macros PG_LOGICAL_DIR and PG_LOGICAL_SNAPSHOTS_DIR in
ValidateSnapshotFile(), instead of hard coding the path

2)
Same as above in pg_get_logical_snapshot_meta() and
pg_get_logical_snapshot_info()

+ sprintf(path, "pg_logical/snapshots/%X-%X.snap",
+ LSN_FORMAT_ARGS(lsn));                        LSN_FORMAT_ARGS(lsn));

Doh! Yeah, agree that we should use those macros. They are coming from c39afc38cf
which has been introduced after v1 of this patch. I thought I took care of it once
c39afc38cf went in, but it looks like I missed it somehow. Done in v4 attached,
Thanks!

3)
+#include "replication/internal_snapbuild.h"

Shall we name new file as 'snapbuild_internal.h' instead of
'internal_snapbuild.h'. Please see other files' name under
'./src/include/replication':
worker_internal.h
walsender_private.h

Agree, that should be snapbuild_internal.h, done in v4.

4)
+static void ValidateSnapshotFile(XLogRecPtr lsn, SnapBuildOnDisk *ondisk,
+ const char *path);

Is it required? We generally don't add declaration unless required by
compiler. Since definition is prior to usage, it is not needed?

I personally prefer to add them even if not required by the compiler. I did not
pay attention that "We generally don't add declaration unless required by compiler"
and (after a quick check) I did not find any reference in the coding style
documentation [1]. That said, I don't have a strong opinion about that and so
removed in v4. Worth to add a mention in the coding convention doc?

Okay. I was somehow under the impression that this is the way in the
postgres i.e. not add redundant declarations. Will be good to know
what others think on this.

Thanks for addressing the comments. I have not started reviewing v4
yet, but here are few more comments on v3:

1)
+#include "port/pg_crc32c.h"

It is not needed in pg_logicalinspect.c as it is already included in
internal_snapbuild.h

2)
+ values[0] = Int16GetDatum(ondisk.builder.state);
........
+ values[8] = LSNGetDatum(ondisk.builder.last_serialized_snapshot);
+ values[9] = TransactionIdGetDatum(ondisk.builder.next_phase_at);
+ values[10] = Int64GetDatum(ondisk.builder.committed.xcnt);

We can have values[i++] in all the places and later we can check :
Assert(i == PG_GET_LOGICAL_SNAPSHOT_INFO_COLS);
Then we need not to keep track of number even in later part of code,
as it goes till 14.

3)
Similar change can be done here:

+ values[0] = Int32GetDatum(ondisk.magic);
+ values[1] = Int32GetDatum(ondisk.checksum);
+ values[2] = Int32GetDatum(ondisk.version);

check at the end will be: Assert(i == PG_GET_LOGICAL_SNAPSHOT_META_COLS);

4)
Most of the output columns in pg_get_logical_snapshot_info() look
self-explanatory except 'state'. Should we have meaningful 'text' here
corresponding to SnapBuildState? Similar to what we do for
'invalidation_reason' in pg_replication_slots. (SlotInvalidationCauses
for ReplicationSlotInvalidationCause)

thanks
Shveta

#18shveta malik
shveta.malik@gmail.com
In reply to: shveta malik (#17)
Re: Add contrib/pg_logicalsnapinspect

On Tue, Sep 17, 2024 at 10:18 AM shveta malik <shveta.malik@gmail.com> wrote:

Thanks for addressing the comments. I have not started reviewing v4
yet, but here are few more comments on v3:

I just noticed that when we pass NULL input, both the new functions
give 1 row as output, all cols as NULL:

newdb1=# SELECT * FROM pg_get_logical_snapshot_meta(NULL);
magic | checksum | version
-------+----------+---------
| |

(1 row)

Similar behavior with pg_get_logical_snapshot_info(). While the
existing 'pg_ls_logicalsnapdir' function gives this error, which looks
more meaningful:

newdb1=# select * from pg_ls_logicalsnapdir(NULL);
ERROR: function pg_ls_logicalsnapdir(unknown) does not exist
LINE 1: select * from pg_ls_logicalsnapdir(NULL);
HINT: No function matches the given name and argument types. You
might need to add explicit type casts.

Shouldn't the new functions have same behavior?

thanks
Shveta

#19David G. Johnston
david.g.johnston@gmail.com
In reply to: shveta malik (#18)
Re: Add contrib/pg_logicalsnapinspect

On Monday, September 16, 2024, shveta malik <shveta.malik@gmail.com> wrote:

On Tue, Sep 17, 2024 at 10:18 AM shveta malik <shveta.malik@gmail.com>
wrote:

Thanks for addressing the comments. I have not started reviewing v4
yet, but here are few more comments on v3:

I just noticed that when we pass NULL input, both the new functions
give 1 row as output, all cols as NULL:

newdb1=# SELECT * FROM pg_get_logical_snapshot_meta(NULL);
magic | checksum | version
-------+----------+---------
| |

(1 row)

Similar behavior with pg_get_logical_snapshot_info(). While the
existing 'pg_ls_logicalsnapdir' function gives this error, which looks
more meaningful:

newdb1=# select * from pg_ls_logicalsnapdir(NULL);
ERROR: function pg_ls_logicalsnapdir(unknown) does not exist
LINE 1: select * from pg_ls_logicalsnapdir(NULL);
HINT: No function matches the given name and argument types. You
might need to add explicit type casts.

Shouldn't the new functions have same behavior?

No. Since the name pg_ls_logicalsnapdir has zero single-argument
implementations passing a null value as an argument is indeed attempt to
invoke a function signature that doesn’t exist.

If there is exactly one single input argument function of the given name
the parser is going to cast the null literal to the data type of the single
argument and invoke the function. It will not and cannot be convinced to
fail to find a matching function.

I can see an argument that they should produce an empty set instead of a
single all-null row, but the idea that they wouldn’t even be found is
contrary to a core design of the system.

David J.

#20Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: David G. Johnston (#19)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Mon, Sep 16, 2024 at 10:16:16PM -0700, David G. Johnston wrote:

On Monday, September 16, 2024, shveta malik <shveta.malik@gmail.com> wrote:

On Tue, Sep 17, 2024 at 10:18 AM shveta malik <shveta.malik@gmail.com>
wrote:

Thanks for addressing the comments. I have not started reviewing v4
yet, but here are few more comments on v3:

I just noticed that when we pass NULL input, both the new functions
give 1 row as output, all cols as NULL:

newdb1=# SELECT * FROM pg_get_logical_snapshot_meta(NULL);
magic | checksum | version
-------+----------+---------
| |

(1 row)

Similar behavior with pg_get_logical_snapshot_info(). While the
existing 'pg_ls_logicalsnapdir' function gives this error, which looks
more meaningful:

newdb1=# select * from pg_ls_logicalsnapdir(NULL);
ERROR: function pg_ls_logicalsnapdir(unknown) does not exist
LINE 1: select * from pg_ls_logicalsnapdir(NULL);
HINT: No function matches the given name and argument types. You
might need to add explicit type casts.

Shouldn't the new functions have same behavior?

No. Since the name pg_ls_logicalsnapdir has zero single-argument
implementations passing a null value as an argument is indeed attempt to
invoke a function signature that doesn’t exist.

Agree.

I can see an argument that they should produce an empty set instead of a
single all-null row,

Yeah, it's outside the scope of this patch but I've seen different behavior
in this area.

For example:

postgres=# select * from pg_ls_replslotdir(NULL);
name | size | modification
------+------+--------------
(0 rows)

as compared to:

postgres=# select * from pg_walfile_name_offset(NULL);
file_name | file_offset
-----------+-------------
|
(1 row)

I thought that it might be linked to the volatility but it is not:

postgres=# select * from pg_stat_get_xact_blocks_fetched(NULL);
pg_stat_get_xact_blocks_fetched
---------------------------------

(1 row)

postgres=# select * from pg_get_multixact_members(NULL);
xid | mode
-----+------
(0 rows)

while both are volatile.

I think both make sense: It's "empty" or we "don't know the values of the fields".
I don't know if there is any reason about this "inconsistency".

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

#21Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: shveta malik (#17)
1 attachment(s)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Tue, Sep 17, 2024 at 10:18:35AM +0530, shveta malik wrote:

Thanks for addressing the comments. I have not started reviewing v4
yet, but here are few more comments on v3:

1)
+#include "port/pg_crc32c.h"

It is not needed in pg_logicalinspect.c as it is already included in
internal_snapbuild.h

Yeap, forgot to remove that one when creating the new "internal".h file, done
in v5 attached, thanks!

2)
+ values[0] = Int16GetDatum(ondisk.builder.state);
........
+ values[8] = LSNGetDatum(ondisk.builder.last_serialized_snapshot);
+ values[9] = TransactionIdGetDatum(ondisk.builder.next_phase_at);
+ values[10] = Int64GetDatum(ondisk.builder.committed.xcnt);

We can have values[i++] in all the places and later we can check :
Assert(i == PG_GET_LOGICAL_SNAPSHOT_INFO_COLS);
Then we need not to keep track of number even in later part of code,
as it goes till 14.

Right, let's do it that way (as it is done in pg_walinspect for example).

4)
Most of the output columns in pg_get_logical_snapshot_info() look
self-explanatory except 'state'. Should we have meaningful 'text' here
corresponding to SnapBuildState? Similar to what we do for
'invalidation_reason' in pg_replication_slots. (SlotInvalidationCauses
for ReplicationSlotInvalidationCause)

Yeah we could. I was not sure about that (and that was my first remark in [1]/messages/by-id/ZscuZ92uGh3wm4tW@ip-10-97-1-34.eu-west-3.compute.internal)
, as the module is mainly for debugging purpose, I was thinking that the one
using it could refer to "snapbuild.h". Let's see what others think.

[1]: /messages/by-id/ZscuZ92uGh3wm4tW@ip-10-97-1-34.eu-west-3.compute.internal

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachments:

v5-0001-Add-contrib-pg_logicalinspect.patchtext/x-diff; charset=us-asciiDownload
From ef6ce23b7145d3a1639dc776c56f0b94ae33bff0 Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Wed, 14 Aug 2024 08:46:05 +0000
Subject: [PATCH v5] Add contrib/pg_logicalinspect

Provides SQL functions that allow to inspect logical decoding components.

It currently allows to inspect the contents of serialized logical snapshots of
a running database cluster, which is useful for debugging or educational
purposes.
---
 contrib/Makefile                              |   1 +
 contrib/meson.build                           |   1 +
 contrib/pg_logicalinspect/.gitignore          |   4 +
 contrib/pg_logicalinspect/Makefile            |  31 +++
 .../expected/logical_inspect.out              |  52 ++++
 contrib/pg_logicalinspect/logicalinspect.conf |   1 +
 contrib/pg_logicalinspect/meson.build         |  39 +++
 .../pg_logicalinspect--1.0.sql                |  43 +++
 contrib/pg_logicalinspect/pg_logicalinspect.c | 253 ++++++++++++++++++
 .../pg_logicalinspect.control                 |   5 +
 .../specs/logical_inspect.spec                |  34 +++
 doc/src/sgml/contrib.sgml                     |   1 +
 doc/src/sgml/filelist.sgml                    |   1 +
 doc/src/sgml/pglogicalinspect.sgml            | 145 ++++++++++
 src/backend/replication/logical/snapbuild.c   | 190 +------------
 src/include/port/pg_crc32c.h                  |  16 +-
 src/include/replication/snapbuild.h           |   2 +-
 src/include/replication/snapbuild_internal.h  | 204 ++++++++++++++
 18 files changed, 830 insertions(+), 193 deletions(-)
   7.6% contrib/pg_logicalinspect/expected/
   5.7% contrib/pg_logicalinspect/specs/
  32.6% contrib/pg_logicalinspect/
  13.2% doc/src/sgml/
  17.4% src/backend/replication/logical/
   4.1% src/include/port/
  18.9% src/include/replication/

diff --git a/contrib/Makefile b/contrib/Makefile
index abd780f277..952855d9b6 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -32,6 +32,7 @@ SUBDIRS = \
 		passwordcheck	\
 		pg_buffercache	\
 		pg_freespacemap \
+		pg_logicalinspect \
 		pg_prewarm	\
 		pg_stat_statements \
 		pg_surgery	\
diff --git a/contrib/meson.build b/contrib/meson.build
index 14a8906865..159ff41555 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -46,6 +46,7 @@ subdir('passwordcheck')
 subdir('pg_buffercache')
 subdir('pgcrypto')
 subdir('pg_freespacemap')
+subdir('pg_logicalinspect')
 subdir('pg_prewarm')
 subdir('pgrowlocks')
 subdir('pg_stat_statements')
diff --git a/contrib/pg_logicalinspect/.gitignore b/contrib/pg_logicalinspect/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/contrib/pg_logicalinspect/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/contrib/pg_logicalinspect/Makefile b/contrib/pg_logicalinspect/Makefile
new file mode 100644
index 0000000000..55124514d4
--- /dev/null
+++ b/contrib/pg_logicalinspect/Makefile
@@ -0,0 +1,31 @@
+# contrib/pg_logicalinspect/Makefile
+
+MODULE_big = pg_logicalinspect
+OBJS = \
+	$(WIN32RES) \
+	pg_logicalinspect.o
+PGFILEDESC = "pg_logicalinspect - functions to inspect logical decoding components"
+
+EXTENSION = pg_logicalinspect
+DATA = pg_logicalinspect--1.0.sql
+
+EXTRA_INSTALL = contrib/test_decoding
+
+ISOLATION = logical_inspect
+
+ISOLATION_OPTS = --temp-config $(top_srcdir)/contrib/pg_logicalinspect/logicalinspect.conf
+
+# Disabled because these tests require "wal_level=logical", which
+# some installcheck users do not have (e.g. buildfarm clients).
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/pg_logicalinspect
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/pg_logicalinspect/expected/logical_inspect.out b/contrib/pg_logicalinspect/expected/logical_inspect.out
new file mode 100644
index 0000000000..749cd4642d
--- /dev/null
+++ b/contrib/pg_logicalinspect/expected/logical_inspect.out
@@ -0,0 +1,52 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s0_init s0_begin s0_savepoint s0_truncate s1_checkpoint s1_get_changes s0_commit s0_begin s0_insert s1_checkpoint s1_get_changes s0_commit s1_get_changes s1_get_logical_snapshot_info s1_get_logical_snapshot_meta
+step s0_init: SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding');
+?column?
+--------
+init    
+(1 row)
+
+step s0_begin: BEGIN;
+step s0_savepoint: SAVEPOINT sp1;
+step s0_truncate: TRUNCATE tbl1;
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data
+----
+(0 rows)
+
+step s0_commit: COMMIT;
+step s0_begin: BEGIN;
+step s0_insert: INSERT INTO tbl1 VALUES (1);
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                   
+---------------------------------------
+BEGIN                                  
+table public.tbl1: TRUNCATE: (no-flags)
+COMMIT                                 
+(3 rows)
+
+step s0_commit: COMMIT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                                         
+-------------------------------------------------------------
+BEGIN                                                        
+table public.tbl1: INSERT: val1[integer]:1 val2[integer]:null
+COMMIT                                                       
+(3 rows)
+
+step s1_get_logical_snapshot_info: SELECT (pg_get_logical_snapshot_info(f.name::pg_lsn)).state,(pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_xip,1),(pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_xip,1) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f ORDER BY 2;
+state|catchange_count|array_length|committed_count|array_length
+-----+---------------+------------+---------------+------------
+    2|              0|            |              2|           2
+    2|              2|           2|              0|            
+(2 rows)
+
+step s1_get_logical_snapshot_meta: SELECT COUNT((pg_get_logical_snapshot_meta(f.name::pg_lsn))) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f;
+count
+-----
+    2
+(1 row)
+
diff --git a/contrib/pg_logicalinspect/logicalinspect.conf b/contrib/pg_logicalinspect/logicalinspect.conf
new file mode 100644
index 0000000000..e3d257315f
--- /dev/null
+++ b/contrib/pg_logicalinspect/logicalinspect.conf
@@ -0,0 +1 @@
+wal_level = logical
diff --git a/contrib/pg_logicalinspect/meson.build b/contrib/pg_logicalinspect/meson.build
new file mode 100644
index 0000000000..b787dafc9b
--- /dev/null
+++ b/contrib/pg_logicalinspect/meson.build
@@ -0,0 +1,39 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+pg_logicalinspect_sources = files('pg_logicalinspect.c')
+
+if host_system == 'windows'
+  pg_logicalinspect_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+    '--NAME', 'pg_logicalinspect',
+    '--FILEDESC', 'pg_logicalinspect - functions to inspect contents of logical snapshots',])
+endif
+
+pg_logicalinspect = shared_module('pg_logicalinspect',
+  pg_logicalinspect_sources,
+  kwargs: contrib_mod_args + {
+      'dependencies': contrib_mod_args['dependencies'],
+  },
+)
+contrib_targets += pg_logicalinspect
+
+install_data(
+  'pg_logicalinspect.control',
+  'pg_logicalinspect--1.0.sql',
+  kwargs: contrib_data_args,
+)
+
+tests += {
+  'name': 'pg_logicalinspect',
+  'sd': meson.current_source_dir(),
+  'bd': meson.current_build_dir(),
+  'isolation': {
+    'specs': [
+      'logical_inspect',
+    ],
+    'regress_args': [
+      '--temp-config', files('logicalinspect.conf'),
+    ],
+    # see above
+    'runningcheck': false,
+  },
+}
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql b/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
new file mode 100644
index 0000000000..51713ed53e
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
@@ -0,0 +1,43 @@
+/* contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_logicalinspect" to load this file. \quit
+
+--
+-- pg_get_logical_snapshot_meta()
+--
+CREATE FUNCTION pg_get_logical_snapshot_meta(IN in_lsn pg_lsn,
+    OUT magic int4,
+    OUT checksum int4,
+    OUT version int4
+)
+AS 'MODULE_PATHNAME', 'pg_get_logical_snapshot_meta'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_logical_snapshot_meta(pg_lsn) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_logical_snapshot_meta(pg_lsn) TO pg_read_server_files;
+
+--
+-- pg_get_logical_snapshot_info()
+--
+CREATE FUNCTION pg_get_logical_snapshot_info(IN in_lsn pg_lsn,
+    OUT state int2,
+    OUT xmin xid,
+    OUT xmax xid,
+    OUT start_decoding_at pg_lsn,
+    OUT two_phase_at pg_lsn,
+    OUT initial_xmin_horizon xid,
+    OUT building_full_snapshot boolean,
+    OUT in_slot_creation boolean,
+    OUT last_serialized_snapshot pg_lsn,
+    OUT next_phase_at xid,
+    OUT committed_count int8,
+    OUT committed_xip xid[],
+    OUT catchange_count int8,
+    OUT catchange_xip xid[]
+)
+AS 'MODULE_PATHNAME', 'pg_get_logical_snapshot_info'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_logical_snapshot_info(pg_lsn) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_logical_snapshot_info(pg_lsn) TO pg_read_server_files;
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.c b/contrib/pg_logicalinspect/pg_logicalinspect.c
new file mode 100644
index 0000000000..dc9041a619
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.c
@@ -0,0 +1,253 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_logicalinspect.c
+ *		  Functions to inspect contents of PostgreSQL logical snapshots
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ *		  contrib/pg_logicalinspect/pg_logicalinspect.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "funcapi.h"
+#include "replication/snapbuild_internal.h"
+#include "utils/array.h"
+#include "utils/pg_lsn.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_meta);
+PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_info);
+
+/*
+ * NOTE: For any code change or issue fix here, it is highly recommended to
+ * give a thought about doing the same in SnapBuildRestore() as well.
+ */
+
+/*
+ * Validate the logical snapshot file.
+ */
+static void
+ValidateSnapshotFile(XLogRecPtr lsn, SnapBuildOnDisk *ondisk, const char *path)
+{
+	int			fd;
+	Size		sz;
+	pg_crc32c	checksum;
+	MemoryContext context;
+
+	context = AllocSetContextCreate(CurrentMemoryContext,
+									"logicalsnapshot inspect context",
+									ALLOCSET_DEFAULT_SIZES);
+
+	fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
+
+	if (fd < 0 && errno == ENOENT)
+		ereport(ERROR,
+				errmsg("file \"%s\" does not exist", path));
+	else if (fd < 0)
+		ereport(ERROR,
+				(errcode_for_file_access(),
+				 errmsg("could not open file \"%s\": %m", path)));
+
+	/* ----
+	 * Make sure the snapshot had been stored safely to disk, that's normally
+	 * cheap.
+	 * Note that we do not need PANIC here, nobody will be able to use the
+	 * slot without fsyncing, and saving it won't succeed without an fsync()
+	 * either...
+	 * ----
+	 */
+	fsync_fname(path, false);
+	fsync_fname(PG_LOGICAL_SNAPSHOTS_DIR, true);
+
+
+	/* read statically sized portion of snapshot */
+	SnapBuildRestoreContents(fd, (char *) ondisk, SnapBuildOnDiskConstantSize, path);
+
+	if (ondisk->magic != SNAPBUILD_MAGIC)
+		ereport(ERROR,
+				(errcode(ERRCODE_DATA_CORRUPTED),
+				 errmsg("snapbuild state file \"%s\" has wrong magic number: %u instead of %u",
+						path, ondisk->magic, SNAPBUILD_MAGIC)));
+
+	if (ondisk->version != SNAPBUILD_VERSION)
+		ereport(ERROR,
+				(errcode(ERRCODE_DATA_CORRUPTED),
+				 errmsg("snapbuild state file \"%s\" has unsupported version: %u instead of %u",
+						path, ondisk->version, SNAPBUILD_VERSION)));
+
+	INIT_CRC32C(checksum);
+	COMP_CRC32C(checksum,
+				((char *) ondisk) + SnapBuildOnDiskNotChecksummedSize,
+				SnapBuildOnDiskConstantSize - SnapBuildOnDiskNotChecksummedSize);
+
+	/* read SnapBuild */
+	SnapBuildRestoreContents(fd, (char *) &ondisk->builder, sizeof(SnapBuild), path);
+	COMP_CRC32C(checksum, &ondisk->builder, sizeof(SnapBuild));
+
+	ondisk->builder.context = context;
+
+	/* restore committed xacts information */
+	if (ondisk->builder.committed.xcnt > 0)
+	{
+		sz = sizeof(TransactionId) * ondisk->builder.committed.xcnt;
+		ondisk->builder.committed.xip = MemoryContextAllocZero(ondisk->builder.context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.committed.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.committed.xip, sz);
+	}
+
+	/* restore catalog modifying xacts information */
+	if (ondisk->builder.catchange.xcnt > 0)
+	{
+		sz = sizeof(TransactionId) * ondisk->builder.catchange.xcnt;
+		ondisk->builder.catchange.xip = MemoryContextAllocZero(ondisk->builder.context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.catchange.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.catchange.xip, sz);
+	}
+
+	if (CloseTransientFile(fd) != 0)
+		ereport(ERROR,
+				(errcode_for_file_access(),
+				 errmsg("could not close file \"%s\": %m", path)));
+
+	FIN_CRC32C(checksum);
+
+	/* verify checksum of what we've read */
+	if (!EQ_CRC32C(checksum, ondisk->checksum))
+		ereport(ERROR,
+				(errcode(ERRCODE_DATA_CORRUPTED),
+				 errmsg("checksum mismatch for snapbuild state file \"%s\": is %u, should be %u",
+						path, checksum, ondisk->checksum)));
+}
+
+/*
+ * Retrieve the logical snapshot file metadata.
+ */
+Datum
+pg_get_logical_snapshot_meta(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_META_COLS 3
+	SnapBuildOnDisk ondisk;
+	XLogRecPtr	lsn;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_META_COLS];
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_META_COLS];
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+	int			i = 0;
+
+	lsn = PG_GETARG_LSN(0);
+
+	sprintf(path, "%s/%X-%X.snap",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			LSN_FORMAT_ARGS(lsn));
+
+	ValidateSnapshotFile(lsn, &ondisk, path);
+
+	/* Build a tuple descriptor for our result type. */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	memset(nulls, 0, sizeof(nulls));
+
+	values[i++] = Int32GetDatum(ondisk.magic);
+	values[i++] = Int32GetDatum(ondisk.checksum);
+	values[i++] = Int32GetDatum(ondisk.version);
+
+	Assert(i == PG_GET_LOGICAL_SNAPSHOT_META_COLS);
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	MemoryContextReset(ondisk.builder.context);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_META_COLS
+}
+
+Datum
+pg_get_logical_snapshot_info(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_INFO_COLS 14
+	SnapBuildOnDisk ondisk;
+	XLogRecPtr	lsn;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS];
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS];
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+	int			i = 0;
+
+	lsn = PG_GETARG_LSN(0);
+
+	sprintf(path, "%s/%X-%X.snap",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			LSN_FORMAT_ARGS(lsn));
+
+	ValidateSnapshotFile(lsn, &ondisk, path);
+
+	/* Build a tuple descriptor for our result type. */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	memset(nulls, 0, sizeof(nulls));
+
+	values[i++] = Int16GetDatum(ondisk.builder.state);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.xmin);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.xmax);
+	values[i++] = LSNGetDatum(ondisk.builder.start_decoding_at);
+	values[i++] = LSNGetDatum(ondisk.builder.two_phase_at);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.initial_xmin_horizon);
+	values[i++] = BoolGetDatum(ondisk.builder.building_full_snapshot);
+	values[i++] = BoolGetDatum(ondisk.builder.in_slot_creation);
+	values[i++] = LSNGetDatum(ondisk.builder.last_serialized_snapshot);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.next_phase_at);
+	values[i++] = Int64GetDatum(ondisk.builder.committed.xcnt);
+
+	if (ondisk.builder.committed.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+		int			narrayelems;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.committed.xcnt * sizeof(Datum));
+		narrayelems = 0;
+
+		for (narrayelems = 0; narrayelems < ondisk.builder.committed.xcnt; narrayelems++)
+			arrayelems[narrayelems] = Int64GetDatum((int64) ondisk.builder.committed.xip[narrayelems]);
+
+		values[i++] = PointerGetDatum(construct_array_builtin(arrayelems, narrayelems, INT8OID));
+	}
+	else
+		nulls[i++] = true;
+
+	values[i++] = Int64GetDatum(ondisk.builder.catchange.xcnt);
+
+	if (ondisk.builder.catchange.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+		int			narrayelems;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.catchange.xcnt * sizeof(Datum));
+		narrayelems = 0;
+
+		for (narrayelems = 0; narrayelems < ondisk.builder.catchange.xcnt; narrayelems++)
+			arrayelems[narrayelems] = Int64GetDatum((int64) ondisk.builder.catchange.xip[narrayelems]);
+
+		values[i++] = PointerGetDatum(construct_array_builtin(arrayelems, narrayelems, INT8OID));
+	}
+	else
+		nulls[i++] = true;
+
+	Assert(i == PG_GET_LOGICAL_SNAPSHOT_INFO_COLS);
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	MemoryContextReset(ondisk.builder.context);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_INFO_COLS
+}
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.control b/contrib/pg_logicalinspect/pg_logicalinspect.control
new file mode 100644
index 0000000000..b4a70e57ba
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.control
@@ -0,0 +1,5 @@
+# pg_logicalinspect extension
+comment = 'functions to inspect logical decoding components'
+default_version = '1.0'
+module_pathname = '$libdir/pg_logicalinspect'
+relocatable = true
diff --git a/contrib/pg_logicalinspect/specs/logical_inspect.spec b/contrib/pg_logicalinspect/specs/logical_inspect.spec
new file mode 100644
index 0000000000..e11eb63615
--- /dev/null
+++ b/contrib/pg_logicalinspect/specs/logical_inspect.spec
@@ -0,0 +1,34 @@
+# Test the pg_logicalinspect functions: that needs some permutation to
+# ensure that we are creating multiple logical snapshots and that one of them
+# contains ongoing catalogs changes.
+setup
+{
+    DROP TABLE IF EXISTS tbl1;
+    CREATE TABLE tbl1 (val1 integer, val2 integer);
+	CREATE EXTENSION pg_logicalinspect;
+}
+
+teardown
+{
+    DROP TABLE tbl1;
+    SELECT 'stop' FROM pg_drop_replication_slot('isolation_slot');
+	DROP EXTENSION pg_logicalinspect;
+}
+
+session "s0"
+setup { SET synchronous_commit=on; }
+step "s0_init" { SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding'); }
+step "s0_begin" { BEGIN; }
+step "s0_savepoint" { SAVEPOINT sp1; }
+step "s0_truncate" { TRUNCATE tbl1; }
+step "s0_insert" { INSERT INTO tbl1 VALUES (1); }
+step "s0_commit" { COMMIT; }
+
+session "s1"
+setup { SET synchronous_commit=on; }
+step "s1_checkpoint" { CHECKPOINT; }
+step "s1_get_changes" { SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0'); }
+step "s1_get_logical_snapshot_meta" { SELECT COUNT((pg_get_logical_snapshot_meta(f.name::pg_lsn))) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f; }
+step "s1_get_logical_snapshot_info" { SELECT (pg_get_logical_snapshot_info(f.name::pg_lsn)).state,(pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_xip,1),(pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_xip,1) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f ORDER BY 2; }
+
+permutation "s0_init" "s0_begin" "s0_savepoint" "s0_truncate" "s1_checkpoint" "s1_get_changes" "s0_commit" "s0_begin" "s0_insert" "s1_checkpoint" "s1_get_changes" "s0_commit" "s1_get_changes" "s1_get_logical_snapshot_info" "s1_get_logical_snapshot_meta"
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index 44639a8dca..7c381949a5 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -154,6 +154,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
  &pgbuffercache;
  &pgcrypto;
  &pgfreespacemap;
+ &pglogicalinspect;
  &pgprewarm;
  &pgrowlocks;
  &pgstatstatements;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index a7ff5f8264..66e6dccd4c 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -143,6 +143,7 @@
 <!ENTITY pgbuffercache   SYSTEM "pgbuffercache.sgml">
 <!ENTITY pgcrypto        SYSTEM "pgcrypto.sgml">
 <!ENTITY pgfreespacemap  SYSTEM "pgfreespacemap.sgml">
+<!ENTITY pglogicalinspect  SYSTEM "pglogicalinspect.sgml">
 <!ENTITY pgprewarm       SYSTEM "pgprewarm.sgml">
 <!ENTITY pgrowlocks      SYSTEM "pgrowlocks.sgml">
 <!ENTITY pgstatstatements SYSTEM "pgstatstatements.sgml">
diff --git a/doc/src/sgml/pglogicalinspect.sgml b/doc/src/sgml/pglogicalinspect.sgml
new file mode 100644
index 0000000000..7767de263f
--- /dev/null
+++ b/doc/src/sgml/pglogicalinspect.sgml
@@ -0,0 +1,145 @@
+<!-- doc/src/sgml/pglogicalinspect.sgml -->
+
+<sect1 id="pglogicalinspect" xreflabel="pg_logicalinspect">
+ <title>pg_logicalinspect &mdash; logical decoding components inspection</title>
+
+ <indexterm zone="pglogicalinspect">
+  <primary>pg_logicalinspect</primary>
+ </indexterm>
+
+ <para>
+  The <filename>pg_logicalinspect</filename> module provides SQL functions
+  that allow you to inspect the contents of logical decoding components. It
+  allows to inspect serialized logical snapshots of a running
+  <productname>PostgreSQL</productname> database cluster, which is useful
+  for debugging or educational purposes.
+ </para>
+
+ <note>
+  <para>
+   The <filename>pg_logicalinspect</filename> functions are called
+   using an LSN argument that can be extracted from the output name of the
+   <function>pg_ls_logicalsnapdir</function>() function.
+  </para>
+ </note>
+
+ <sect2 id="pglogicalinspect-funcs">
+  <title>General Functions</title>
+
+  <variablelist>
+   <varlistentry id="pglogicalinspect-funcs-pg-get-logical-snapshot-meta">
+    <term>
+     <function>pg_get_logical_snapshot_meta(in_lsn pg_lsn) returns record</function>
+    </term>
+
+    <listitem>
+     <para>
+      Gets logical snapshot metadata about a snapshot file that is located in
+      the <filename>pg_logical/snapshots</filename> directory.
+      The <replaceable>in_lsn</replaceable> argument can be extracted from the
+      snapshot file name.
+      example:
+<screen>
+postgres=# SELECT * FROM pg_ls_logicalsnapdir();
+-[ RECORD 1 ]+-----------------------
+name         | 0-40796E18.snap
+size         | 152
+modification | 2024-08-14 16:36:32+00
+
+postgres=# SELECT * FROM pg_get_logical_snapshot_meta('0/40796E18');
+-[ RECORD 1 ]--------
+magic    | 1369563137
+checksum | 1028045905
+version  | 6
+
+postgres=# SELECT (pg_get_logical_snapshot_meta(f.name::pg_lsn)).*
+           FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name
+                 FROM pg_ls_logicalsnapdir()) AS f;
+-[ RECORD 1 ]--------
+magic    | 1369563137
+checksum | 1028045905
+version  | 6
+</screen>
+     </para>
+     <para>
+      If <replaceable>in_lsn</replaceable> does not match a snapshot file, the
+      function raises an error.
+     </para>
+    </listitem>
+   </varlistentry>
+
+   <varlistentry id="pglogicalinspect-funcs-pg-get-logical-snapshot-info">
+    <term>
+     <function>pg_get_logical_snapshot_info(in_lsn pg_lsn) returns record</function>
+    </term>
+
+    <listitem>
+     <para>
+      Gets logical snapshot information about a snapshot file that is located in
+      the <filename>pg_logical/snapshots</filename> directory.
+      The <replaceable>in_lsn</replaceable> argument can be extracted from the
+      snapshot file name.
+      example:
+<screen>
+postgres=# SELECT * FROM pg_ls_logicalsnapdir();
+-[ RECORD 1 ]+-----------------------
+name         | 0-40796E18.snap
+size         | 152
+modification | 2024-08-14 16:36:32+00
+
+postgres=# SELECT * FROM pg_get_logical_snapshot_info('0/40796E18');
+-[ RECORD 1 ]------------+-----------
+state                    | 2
+xmin                     | 751
+xmax                     | 751
+start_decoding_at        | 0/40796AF8
+two_phase_at             | 0/40796AF8
+initial_xmin_horizon     | 0
+building_full_snapshot   | f
+in_slot_creation         | f
+last_serialized_snapshot | 0/0
+next_phase_at            | 0
+committed_count          | 0
+committed_xip            |
+catchange_count          | 2
+catchange_xip            | {751,752}
+
+postgres=# SELECT (pg_get_logical_snapshot_info(f.name::pg_lsn)).*
+           FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name
+                 FROM pg_ls_logicalsnapdir()) AS f;
+-[ RECORD 1 ]------------+-----------
+state                    | 2
+xmin                     | 751
+xmax                     | 751
+start_decoding_at        | 0/40796AF8
+two_phase_at             | 0/40796AF8
+initial_xmin_horizon     | 0
+building_full_snapshot   | f
+in_slot_creation         | f
+last_serialized_snapshot | 0/0
+next_phase_at            | 0
+committed_count          | 0
+committed_xip            |
+catchange_count          | 2
+catchange_xip            | {751,752}
+</screen>
+     </para>
+     <para>
+      If <replaceable>in_lsn</replaceable> does not match a snapshot file, the
+      function raises an error.
+     </para>
+    </listitem>
+   </varlistentry>
+
+  </variablelist>
+ </sect2>
+
+ <sect2 id="pglogicalinspect-author">
+  <title>Author</title>
+
+  <para>
+   Bertrand Drouvot <email>bertranddrouvot.pg@gmail.com</email>
+  </para>
+ </sect2>
+
+</sect1>
diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c
index 0450f94ba8..bb7def9440 100644
--- a/src/backend/replication/logical/snapbuild.c
+++ b/src/backend/replication/logical/snapbuild.c
@@ -134,6 +134,7 @@
 #include "replication/logical.h"
 #include "replication/reorderbuffer.h"
 #include "replication/snapbuild.h"
+#include "replication/snapbuild_internal.h"
 #include "storage/fd.h"
 #include "storage/lmgr.h"
 #include "storage/proc.h"
@@ -143,146 +144,6 @@
 #include "utils/memutils.h"
 #include "utils/snapmgr.h"
 #include "utils/snapshot.h"
-
-/*
- * This struct contains the current state of the snapshot building
- * machinery. Besides a forward declaration in the header, it is not exposed
- * to the public, so we can easily change its contents.
- */
-struct SnapBuild
-{
-	/* how far are we along building our first full snapshot */
-	SnapBuildState state;
-
-	/* private memory context used to allocate memory for this module. */
-	MemoryContext context;
-
-	/* all transactions < than this have committed/aborted */
-	TransactionId xmin;
-
-	/* all transactions >= than this are uncommitted */
-	TransactionId xmax;
-
-	/*
-	 * Don't replay commits from an LSN < this LSN. This can be set externally
-	 * but it will also be advanced (never retreat) from within snapbuild.c.
-	 */
-	XLogRecPtr	start_decoding_at;
-
-	/*
-	 * LSN at which two-phase decoding was enabled or LSN at which we found a
-	 * consistent point at the time of slot creation.
-	 *
-	 * The prepared transactions, that were skipped because previously
-	 * two-phase was not enabled or are not covered by initial snapshot, need
-	 * to be sent later along with commit prepared and they must be before
-	 * this point.
-	 */
-	XLogRecPtr	two_phase_at;
-
-	/*
-	 * Don't start decoding WAL until the "xl_running_xacts" information
-	 * indicates there are no running xids with an xid smaller than this.
-	 */
-	TransactionId initial_xmin_horizon;
-
-	/* Indicates if we are building full snapshot or just catalog one. */
-	bool		building_full_snapshot;
-
-	/*
-	 * Indicates if we are using the snapshot builder for the creation of a
-	 * logical replication slot. If it's true, the start point for decoding
-	 * changes is not determined yet. So we skip snapshot restores to properly
-	 * find the start point. See SnapBuildFindSnapshot() for details.
-	 */
-	bool		in_slot_creation;
-
-	/*
-	 * Snapshot that's valid to see the catalog state seen at this moment.
-	 */
-	Snapshot	snapshot;
-
-	/*
-	 * LSN of the last location we are sure a snapshot has been serialized to.
-	 */
-	XLogRecPtr	last_serialized_snapshot;
-
-	/*
-	 * The reorderbuffer we need to update with usable snapshots et al.
-	 */
-	ReorderBuffer *reorder;
-
-	/*
-	 * TransactionId at which the next phase of initial snapshot building will
-	 * happen. InvalidTransactionId if not known (i.e. SNAPBUILD_START), or
-	 * when no next phase necessary (SNAPBUILD_CONSISTENT).
-	 */
-	TransactionId next_phase_at;
-
-	/*
-	 * Array of transactions which could have catalog changes that committed
-	 * between xmin and xmax.
-	 */
-	struct
-	{
-		/* number of committed transactions */
-		size_t		xcnt;
-
-		/* available space for committed transactions */
-		size_t		xcnt_space;
-
-		/*
-		 * Until we reach a CONSISTENT state, we record commits of all
-		 * transactions, not just the catalog changing ones. Record when that
-		 * changes so we know we cannot export a snapshot safely anymore.
-		 */
-		bool		includes_all_transactions;
-
-		/*
-		 * Array of committed transactions that have modified the catalog.
-		 *
-		 * As this array is frequently modified we do *not* keep it in
-		 * xidComparator order. Instead we sort the array when building &
-		 * distributing a snapshot.
-		 *
-		 * TODO: It's unclear whether that reasoning has much merit. Every
-		 * time we add something here after becoming consistent will also
-		 * require distributing a snapshot. Storing them sorted would
-		 * potentially also make it easier to purge (but more complicated wrt
-		 * wraparound?). Should be improved if sorting while building the
-		 * snapshot shows up in profiles.
-		 */
-		TransactionId *xip;
-	}			committed;
-
-	/*
-	 * Array of transactions and subtransactions that had modified catalogs
-	 * and were running when the snapshot was serialized.
-	 *
-	 * We normally rely on some WAL record types such as HEAP2_NEW_CID to know
-	 * if the transaction has changed the catalog. But it could happen that
-	 * the logical decoding decodes only the commit record of the transaction
-	 * after restoring the previously serialized snapshot in which case we
-	 * will miss adding the xid to the snapshot and end up looking at the
-	 * catalogs with the wrong snapshot.
-	 *
-	 * Now to avoid the above problem, we serialize the transactions that had
-	 * modified the catalogs and are still running at the time of snapshot
-	 * serialization. We fill this array while restoring the snapshot and then
-	 * refer it while decoding commit to ensure if the xact has modified the
-	 * catalog. We discard this array when all the xids in the list become old
-	 * enough to matter. See SnapBuildPurgeOlderTxn for details.
-	 */
-	struct
-	{
-		/* number of transactions */
-		size_t		xcnt;
-
-		/* This array must be sorted in xidComparator order */
-		TransactionId *xip;
-	}			catchange;
-};
-
 /*
  * Starting a transaction -- which we need to do while exporting a snapshot --
  * removes knowledge about the previously used resowner, so we save it here.
@@ -312,7 +173,6 @@ static void SnapBuildWaitSnapshot(xl_running_xacts *running, TransactionId cutof
 /* serialization functions */
 static void SnapBuildSerialize(SnapBuild *builder, XLogRecPtr lsn);
 static bool SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn);
-static void SnapBuildRestoreContents(int fd, char *dest, Size size, const char *path);
 
 /*
  * Allocate a new snapshot builder.
@@ -1557,48 +1417,6 @@ SnapBuildWaitSnapshot(xl_running_xacts *running, TransactionId cutoff)
 	}
 }
 
-/* -----------------------------------
- * Snapshot serialization support
- * -----------------------------------
- */
-
-/*
- * We store current state of struct SnapBuild on disk in the following manner:
- *
- * struct SnapBuildOnDisk;
- * TransactionId * committed.xcnt; (*not xcnt_space*)
- * TransactionId * catchange.xcnt;
- *
- */
-typedef struct SnapBuildOnDisk
-{
-	/* first part of this struct needs to be version independent */
-
-	/* data not covered by checksum */
-	uint32		magic;
-	pg_crc32c	checksum;
-
-	/* data covered by checksum */
-
-	/* version, in case we want to support pg_upgrade */
-	uint32		version;
-	/* how large is the on disk data, excluding the constant sized part */
-	uint32		length;
-
-	/* version dependent part */
-	SnapBuild	builder;
-
-	/* variable amount of TransactionIds follows */
-} SnapBuildOnDisk;
-
-#define SnapBuildOnDiskConstantSize \
-	offsetof(SnapBuildOnDisk, builder)
-#define SnapBuildOnDiskNotChecksummedSize \
-	offsetof(SnapBuildOnDisk, version)
-
-#define SNAPBUILD_MAGIC 0x51A1E001
-#define SNAPBUILD_VERSION 6
-
 /*
  * Store/Load a snapshot from disk, depending on the snapshot builder's state.
  *
@@ -1859,6 +1677,10 @@ out:
 /*
  * Restore a snapshot into 'builder' if previously one has been stored at the
  * location indicated by 'lsn'. Returns true if successful, false otherwise.
+ *
+ * NOTE: For any code change or issue fix here, it is highly recommended to
+ * give a thought about doing the same in pg_logicalinspect contrib module
+ * as well.
  */
 static bool
 SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
@@ -2033,7 +1855,7 @@ snapshot_not_interesting:
 /*
  * Read the contents of the serialized snapshot to 'dest'.
  */
-static void
+void
 SnapBuildRestoreContents(int fd, char *dest, Size size, const char *path)
 {
 	int			readBytes;
diff --git a/src/include/port/pg_crc32c.h b/src/include/port/pg_crc32c.h
index 63c8e3a00b..cfc8c07944 100644
--- a/src/include/port/pg_crc32c.h
+++ b/src/include/port/pg_crc32c.h
@@ -47,7 +47,7 @@ typedef uint32 pg_crc32c;
 	((crc) = pg_comp_crc32c_sse42((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
 
 #elif defined(USE_ARMV8_CRC32C)
 /* Use ARMv8 CRC Extension instructions. */
@@ -56,7 +56,7 @@ extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t le
 	((crc) = pg_comp_crc32c_armv8((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
 
 #elif defined(USE_LOONGARCH_CRC32C)
 /* Use LoongArch CRCC instructions. */
@@ -65,7 +65,7 @@ extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t le
 	((crc) = pg_comp_crc32c_loongarch((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len);
 
 #elif defined(USE_SSE42_CRC32C_WITH_RUNTIME_CHECK) || defined(USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK)
 
@@ -77,14 +77,14 @@ extern pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_
 	((crc) = pg_comp_crc32c((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
-extern pg_crc32c (*pg_comp_crc32c) (pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c (*pg_comp_crc32c) (pg_crc32c crc, const void *data, size_t len);
 
 #ifdef USE_SSE42_CRC32C_WITH_RUNTIME_CHECK
-extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
 #endif
 #ifdef USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK
-extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
 #endif
 
 #else
@@ -103,7 +103,7 @@ extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t le
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 #endif
 
-extern pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
 
 #endif
 
diff --git a/src/include/replication/snapbuild.h b/src/include/replication/snapbuild.h
index caa5113ff8..dbb4bc2f4b 100644
--- a/src/include/replication/snapbuild.h
+++ b/src/include/replication/snapbuild.h
@@ -46,7 +46,7 @@ typedef enum
 	SNAPBUILD_CONSISTENT = 2,
 } SnapBuildState;
 
-/* forward declare so we don't have to expose the struct to the public */
+/* forward declare so we don't have to include snapbuild_internal.h */
 struct SnapBuild;
 typedef struct SnapBuild SnapBuild;
 
diff --git a/src/include/replication/snapbuild_internal.h b/src/include/replication/snapbuild_internal.h
new file mode 100644
index 0000000000..4d4d688470
--- /dev/null
+++ b/src/include/replication/snapbuild_internal.h
@@ -0,0 +1,204 @@
+/*-------------------------------------------------------------------------
+ *
+ * snapbuild_internal.h
+ *    This file contains declarations for logical decoding utility
+ *    functions for internal use.
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * src/include/replication/snapbuild_internal.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef INTERNAL_SNAPBUILD_H
+#define INTERNAL_SNAPBUILD_H
+
+#include "port/pg_crc32c.h"
+#include "replication/reorderbuffer.h"
+#include "replication/snapbuild.h"
+
+/* -----------------------------------
+ * Snapshot serialization support
+ * -----------------------------------
+ */
+
+#define SnapBuildOnDiskConstantSize \
+	offsetof(SnapBuildOnDisk, builder)
+#define SnapBuildOnDiskNotChecksummedSize \
+	offsetof(SnapBuildOnDisk, version)
+
+#define SNAPBUILD_MAGIC 0x51A1E001
+#define SNAPBUILD_VERSION 6
+
+/*
+ * This struct contains the current state of the snapshot building
+ * machinery. It is exposed to the public, so pay attention when changing its
+ * contents.
+ */
+typedef struct SnapBuild
+{
+	/* how far are we along building our first full snapshot */
+	SnapBuildState state;
+
+	/* private memory context used to allocate memory for this module. */
+	MemoryContext context;
+
+	/* all transactions < than this have committed/aborted */
+	TransactionId xmin;
+
+	/* all transactions >= than this are uncommitted */
+	TransactionId xmax;
+
+	/*
+	 * Don't replay commits from an LSN < this LSN. This can be set externally
+	 * but it will also be advanced (never retreat) from within snapbuild.c.
+	 */
+	XLogRecPtr	start_decoding_at;
+
+	/*
+	 * LSN at which two-phase decoding was enabled or LSN at which we found a
+	 * consistent point at the time of slot creation.
+	 *
+	 * The prepared transactions, that were skipped because previously
+	 * two-phase was not enabled or are not covered by initial snapshot, need
+	 * to be sent later along with commit prepared and they must be before
+	 * this point.
+	 */
+	XLogRecPtr	two_phase_at;
+
+	/*
+	 * Don't start decoding WAL until the "xl_running_xacts" information
+	 * indicates there are no running xids with an xid smaller than this.
+	 */
+	TransactionId initial_xmin_horizon;
+
+	/* Indicates if we are building full snapshot or just catalog one. */
+	bool		building_full_snapshot;
+
+	/*
+	 * Indicates if we are using the snapshot builder for the creation of a
+	 * logical replication slot. If it's true, the start point for decoding
+	 * changes is not determined yet. So we skip snapshot restores to properly
+	 * find the start point. See SnapBuildFindSnapshot() for details.
+	 */
+	bool		in_slot_creation;
+
+	/*
+	 * Snapshot that's valid to see the catalog state seen at this moment.
+	 */
+	Snapshot	snapshot;
+
+	/*
+	 * LSN of the last location we are sure a snapshot has been serialized to.
+	 */
+	XLogRecPtr	last_serialized_snapshot;
+
+	/*
+	 * The reorderbuffer we need to update with usable snapshots et al.
+	 */
+	ReorderBuffer *reorder;
+
+	/*
+	 * TransactionId at which the next phase of initial snapshot building will
+	 * happen. InvalidTransactionId if not known (i.e. SNAPBUILD_START), or
+	 * when no next phase necessary (SNAPBUILD_CONSISTENT).
+	 */
+	TransactionId next_phase_at;
+
+	/*
+	 * Array of transactions which could have catalog changes that committed
+	 * between xmin and xmax.
+	 */
+	struct
+	{
+		/* number of committed transactions */
+		size_t		xcnt;
+
+		/* available space for committed transactions */
+		size_t		xcnt_space;
+
+		/*
+		 * Until we reach a CONSISTENT state, we record commits of all
+		 * transactions, not just the catalog changing ones. Record when that
+		 * changes so we know we cannot export a snapshot safely anymore.
+		 */
+		bool		includes_all_transactions;
+
+		/*
+		 * Array of committed transactions that have modified the catalog.
+		 *
+		 * As this array is frequently modified we do *not* keep it in
+		 * xidComparator order. Instead we sort the array when building &
+		 * distributing a snapshot.
+		 *
+		 * TODO: It's unclear whether that reasoning has much merit. Every
+		 * time we add something here after becoming consistent will also
+		 * require distributing a snapshot. Storing them sorted would
+		 * potentially also make it easier to purge (but more complicated wrt
+		 * wraparound?). Should be improved if sorting while building the
+		 * snapshot shows up in profiles.
+		 */
+		TransactionId *xip;
+	}			committed;
+
+	/*
+	 * Array of transactions and subtransactions that had modified catalogs
+	 * and were running when the snapshot was serialized.
+	 *
+	 * We normally rely on some WAL record types such as HEAP2_NEW_CID to know
+	 * if the transaction has changed the catalog. But it could happen that
+	 * the logical decoding decodes only the commit record of the transaction
+	 * after restoring the previously serialized snapshot in which case we
+	 * will miss adding the xid to the snapshot and end up looking at the
+	 * catalogs with the wrong snapshot.
+	 *
+	 * Now to avoid the above problem, we serialize the transactions that had
+	 * modified the catalogs and are still running at the time of snapshot
+	 * serialization. We fill this array while restoring the snapshot and then
+	 * refer it while decoding commit to ensure if the xact has modified the
+	 * catalog. We discard this array when all the xids in the list become old
+	 * enough to matter. See SnapBuildPurgeOlderTxn for details.
+	 */
+	struct
+	{
+		/* number of transactions */
+		size_t		xcnt;
+
+		/* This array must be sorted in xidComparator order */
+		TransactionId *xip;
+	}			catchange;
+} SnapBuild;
+
+/*
+ * We store current state of struct SnapBuild on disk in the following manner:
+ *
+ * struct SnapBuildOnDisk;
+ * TransactionId * committed.xcnt; (*not xcnt_space*)
+ * TransactionId * catchange.xcnt;
+ *
+ */
+typedef struct SnapBuildOnDisk
+{
+	/* first part of this struct needs to be version independent */
+
+	/* data not covered by checksum */
+	uint32		magic;
+	pg_crc32c	checksum;
+
+	/* data covered by checksum */
+
+	/* version, in case we want to support pg_upgrade */
+	uint32		version;
+	/* how large is the on disk data, excluding the constant sized part */
+	uint32		length;
+
+	/* version dependent part */
+	SnapBuild	builder;
+
+	/* variable amount of TransactionIds follows */
+} SnapBuildOnDisk;
+
+extern void SnapBuildRestoreContents(int fd, char *dest, Size size, const char *path);
+
+#endif							/* INTERNAL_SNAPBUILD_H */
-- 
2.34.1

#22shveta malik
shveta.malik@gmail.com
In reply to: David G. Johnston (#19)
Re: Add contrib/pg_logicalsnapinspect

On Tue, Sep 17, 2024 at 10:46 AM David G. Johnston
<david.g.johnston@gmail.com> wrote:

On Monday, September 16, 2024, shveta malik <shveta.malik@gmail.com> wrote:

On Tue, Sep 17, 2024 at 10:18 AM shveta malik <shveta.malik@gmail.com> wrote:

Thanks for addressing the comments. I have not started reviewing v4
yet, but here are few more comments on v3:

I just noticed that when we pass NULL input, both the new functions
give 1 row as output, all cols as NULL:

newdb1=# SELECT * FROM pg_get_logical_snapshot_meta(NULL);
magic | checksum | version
-------+----------+---------
| |

(1 row)

Similar behavior with pg_get_logical_snapshot_info(). While the
existing 'pg_ls_logicalsnapdir' function gives this error, which looks
more meaningful:

newdb1=# select * from pg_ls_logicalsnapdir(NULL);
ERROR: function pg_ls_logicalsnapdir(unknown) does not exist
LINE 1: select * from pg_ls_logicalsnapdir(NULL);
HINT: No function matches the given name and argument types. You
might need to add explicit type casts.

Shouldn't the new functions have same behavior?

No. Since the name pg_ls_logicalsnapdir has zero single-argument implementations passing a null value as an argument is indeed attempt to invoke a function signature that doesn’t exist.

If there is exactly one single input argument function of the given name the parser is going to cast the null literal to the data type of the single argument and invoke the function. It will not and cannot be convinced to fail to find a matching function.

Okay, understood. Thanks for explaining.

I can see an argument that they should produce an empty set instead of a single all-null row, but the idea that they wouldn’t even be found is contrary to a core design of the system.

Okay, a single row can be investigated if it comes under this scope.
But I see why 'ERROR' is not a possibility here.

thanks
Shveta

#23shveta malik
shveta.malik@gmail.com
In reply to: Bertrand Drouvot (#21)
Re: Add contrib/pg_logicalsnapinspect

On Tue, Sep 17, 2024 at 12:44 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Hi,

On Tue, Sep 17, 2024 at 10:18:35AM +0530, shveta malik wrote:

Thanks for addressing the comments. I have not started reviewing v4
yet, but here are few more comments on v3:

1)
+#include "port/pg_crc32c.h"

It is not needed in pg_logicalinspect.c as it is already included in
internal_snapbuild.h

Yeap, forgot to remove that one when creating the new "internal".h file, done
in v5 attached, thanks!

2)
+ values[0] = Int16GetDatum(ondisk.builder.state);
........
+ values[8] = LSNGetDatum(ondisk.builder.last_serialized_snapshot);
+ values[9] = TransactionIdGetDatum(ondisk.builder.next_phase_at);
+ values[10] = Int64GetDatum(ondisk.builder.committed.xcnt);

We can have values[i++] in all the places and later we can check :
Assert(i == PG_GET_LOGICAL_SNAPSHOT_INFO_COLS);
Then we need not to keep track of number even in later part of code,
as it goes till 14.

Right, let's do it that way (as it is done in pg_walinspect for example).

4)
Most of the output columns in pg_get_logical_snapshot_info() look
self-explanatory except 'state'. Should we have meaningful 'text' here
corresponding to SnapBuildState? Similar to what we do for
'invalidation_reason' in pg_replication_slots. (SlotInvalidationCauses
for ReplicationSlotInvalidationCause)

Yeah we could. I was not sure about that (and that was my first remark in [1])
, as the module is mainly for debugging purpose, I was thinking that the one
using it could refer to "snapbuild.h". Let's see what others think.

okay, makes sense. lets wait what others have to say.

Thanks for the patch. Few trivial things:

1)
May be we shall change 'INTERNAL_SNAPBUILD_H' in snapbuild_internal.h
to 'SNAPBUILD_INTERNAL_H'?

2)
ValidateSnapshotFile()

It is not only validating, but loading the content as well. So may be
we can rename to ValidateAndRestoreSnapshotFile?

3) sgml:
a)
+ The pg_logicalinspect functions are called using an LSN argument
that can be extracted from the output name of the
pg_ls_logicalsnapdir() function.

Is it possible to give link to pg_ls_logicalsnapdir function here?

b)
+ Gets logical snapshot metadata about a snapshot file that is located
in the pg_logical/snapshots directory.

located in server's pg_logical/snapshots directory
(i.e. use server keyword, similar to how pg_ls_logicalsnapdir ,
pg_ls_logicalmapdir explains it)

thanks
Shveta

#24Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: shveta malik (#23)
1 attachment(s)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Wed, Sep 18, 2024 at 11:33:08AM +0530, shveta malik wrote:

Thanks for the patch. Few trivial things:

1)
May be we shall change 'INTERNAL_SNAPBUILD_H' in snapbuild_internal.h
to 'SNAPBUILD_INTERNAL_H'?

Indeed, done in v6 attached, thanks!

2)
ValidateSnapshotFile()

It is not only validating, but loading the content as well. So may be
we can rename to ValidateAndRestoreSnapshotFile?

I see what you mean, we're also populating the SnapBuildOnDisk. I think your
proposal makes sense, done that way in v6.

3) sgml:
a)
+ The pg_logicalinspect functions are called using an LSN argument
that can be extracted from the output name of the
pg_ls_logicalsnapdir() function.

Is it possible to give link to pg_ls_logicalsnapdir function here?

Yes but I'm not sure that's needed. A quick "git grep "<function>" "*.sgml""
seems to show that providing a link is not that common.

b)
+ Gets logical snapshot metadata about a snapshot file that is located
in the pg_logical/snapshots directory.

located in server's pg_logical/snapshots directory
(i.e. use server keyword, similar to how pg_ls_logicalsnapdir ,
pg_ls_logicalmapdir explains it)

Agree, done that way in v6.

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachments:

v6-0001-Add-contrib-pg_logicalinspect.patchtext/x-diff; charset=us-asciiDownload
From a4e1432f9ccd02b999fb62619dcde3b0d9ad164f Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Wed, 14 Aug 2024 08:46:05 +0000
Subject: [PATCH v6] Add contrib/pg_logicalinspect

Provides SQL functions that allow to inspect logical decoding components.

It currently allows to inspect the contents of serialized logical snapshots of
a running database cluster, which is useful for debugging or educational
purposes.
---
 contrib/Makefile                              |   1 +
 contrib/meson.build                           |   1 +
 contrib/pg_logicalinspect/.gitignore          |   4 +
 contrib/pg_logicalinspect/Makefile            |  31 +++
 .../expected/logical_inspect.out              |  52 ++++
 contrib/pg_logicalinspect/logicalinspect.conf |   1 +
 contrib/pg_logicalinspect/meson.build         |  39 +++
 .../pg_logicalinspect--1.0.sql                |  43 +++
 contrib/pg_logicalinspect/pg_logicalinspect.c | 254 ++++++++++++++++++
 .../pg_logicalinspect.control                 |   5 +
 .../specs/logical_inspect.spec                |  34 +++
 doc/src/sgml/contrib.sgml                     |   1 +
 doc/src/sgml/filelist.sgml                    |   1 +
 doc/src/sgml/pglogicalinspect.sgml            | 145 ++++++++++
 src/backend/replication/logical/snapbuild.c   | 190 +------------
 src/include/port/pg_crc32c.h                  |  16 +-
 src/include/replication/snapbuild.h           |   2 +-
 src/include/replication/snapbuild_internal.h  | 204 ++++++++++++++
 18 files changed, 831 insertions(+), 193 deletions(-)
   7.6% contrib/pg_logicalinspect/expected/
   5.7% contrib/pg_logicalinspect/specs/
  32.6% contrib/pg_logicalinspect/
  13.2% doc/src/sgml/
  17.4% src/backend/replication/logical/
   4.1% src/include/port/
  18.9% src/include/replication/

diff --git a/contrib/Makefile b/contrib/Makefile
index abd780f277..952855d9b6 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -32,6 +32,7 @@ SUBDIRS = \
 		passwordcheck	\
 		pg_buffercache	\
 		pg_freespacemap \
+		pg_logicalinspect \
 		pg_prewarm	\
 		pg_stat_statements \
 		pg_surgery	\
diff --git a/contrib/meson.build b/contrib/meson.build
index 14a8906865..159ff41555 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -46,6 +46,7 @@ subdir('passwordcheck')
 subdir('pg_buffercache')
 subdir('pgcrypto')
 subdir('pg_freespacemap')
+subdir('pg_logicalinspect')
 subdir('pg_prewarm')
 subdir('pgrowlocks')
 subdir('pg_stat_statements')
diff --git a/contrib/pg_logicalinspect/.gitignore b/contrib/pg_logicalinspect/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/contrib/pg_logicalinspect/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/contrib/pg_logicalinspect/Makefile b/contrib/pg_logicalinspect/Makefile
new file mode 100644
index 0000000000..55124514d4
--- /dev/null
+++ b/contrib/pg_logicalinspect/Makefile
@@ -0,0 +1,31 @@
+# contrib/pg_logicalinspect/Makefile
+
+MODULE_big = pg_logicalinspect
+OBJS = \
+	$(WIN32RES) \
+	pg_logicalinspect.o
+PGFILEDESC = "pg_logicalinspect - functions to inspect logical decoding components"
+
+EXTENSION = pg_logicalinspect
+DATA = pg_logicalinspect--1.0.sql
+
+EXTRA_INSTALL = contrib/test_decoding
+
+ISOLATION = logical_inspect
+
+ISOLATION_OPTS = --temp-config $(top_srcdir)/contrib/pg_logicalinspect/logicalinspect.conf
+
+# Disabled because these tests require "wal_level=logical", which
+# some installcheck users do not have (e.g. buildfarm clients).
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/pg_logicalinspect
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/pg_logicalinspect/expected/logical_inspect.out b/contrib/pg_logicalinspect/expected/logical_inspect.out
new file mode 100644
index 0000000000..749cd4642d
--- /dev/null
+++ b/contrib/pg_logicalinspect/expected/logical_inspect.out
@@ -0,0 +1,52 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s0_init s0_begin s0_savepoint s0_truncate s1_checkpoint s1_get_changes s0_commit s0_begin s0_insert s1_checkpoint s1_get_changes s0_commit s1_get_changes s1_get_logical_snapshot_info s1_get_logical_snapshot_meta
+step s0_init: SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding');
+?column?
+--------
+init    
+(1 row)
+
+step s0_begin: BEGIN;
+step s0_savepoint: SAVEPOINT sp1;
+step s0_truncate: TRUNCATE tbl1;
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data
+----
+(0 rows)
+
+step s0_commit: COMMIT;
+step s0_begin: BEGIN;
+step s0_insert: INSERT INTO tbl1 VALUES (1);
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                   
+---------------------------------------
+BEGIN                                  
+table public.tbl1: TRUNCATE: (no-flags)
+COMMIT                                 
+(3 rows)
+
+step s0_commit: COMMIT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                                         
+-------------------------------------------------------------
+BEGIN                                                        
+table public.tbl1: INSERT: val1[integer]:1 val2[integer]:null
+COMMIT                                                       
+(3 rows)
+
+step s1_get_logical_snapshot_info: SELECT (pg_get_logical_snapshot_info(f.name::pg_lsn)).state,(pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_xip,1),(pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_xip,1) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f ORDER BY 2;
+state|catchange_count|array_length|committed_count|array_length
+-----+---------------+------------+---------------+------------
+    2|              0|            |              2|           2
+    2|              2|           2|              0|            
+(2 rows)
+
+step s1_get_logical_snapshot_meta: SELECT COUNT((pg_get_logical_snapshot_meta(f.name::pg_lsn))) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f;
+count
+-----
+    2
+(1 row)
+
diff --git a/contrib/pg_logicalinspect/logicalinspect.conf b/contrib/pg_logicalinspect/logicalinspect.conf
new file mode 100644
index 0000000000..e3d257315f
--- /dev/null
+++ b/contrib/pg_logicalinspect/logicalinspect.conf
@@ -0,0 +1 @@
+wal_level = logical
diff --git a/contrib/pg_logicalinspect/meson.build b/contrib/pg_logicalinspect/meson.build
new file mode 100644
index 0000000000..b787dafc9b
--- /dev/null
+++ b/contrib/pg_logicalinspect/meson.build
@@ -0,0 +1,39 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+pg_logicalinspect_sources = files('pg_logicalinspect.c')
+
+if host_system == 'windows'
+  pg_logicalinspect_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+    '--NAME', 'pg_logicalinspect',
+    '--FILEDESC', 'pg_logicalinspect - functions to inspect contents of logical snapshots',])
+endif
+
+pg_logicalinspect = shared_module('pg_logicalinspect',
+  pg_logicalinspect_sources,
+  kwargs: contrib_mod_args + {
+      'dependencies': contrib_mod_args['dependencies'],
+  },
+)
+contrib_targets += pg_logicalinspect
+
+install_data(
+  'pg_logicalinspect.control',
+  'pg_logicalinspect--1.0.sql',
+  kwargs: contrib_data_args,
+)
+
+tests += {
+  'name': 'pg_logicalinspect',
+  'sd': meson.current_source_dir(),
+  'bd': meson.current_build_dir(),
+  'isolation': {
+    'specs': [
+      'logical_inspect',
+    ],
+    'regress_args': [
+      '--temp-config', files('logicalinspect.conf'),
+    ],
+    # see above
+    'runningcheck': false,
+  },
+}
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql b/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
new file mode 100644
index 0000000000..51713ed53e
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
@@ -0,0 +1,43 @@
+/* contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_logicalinspect" to load this file. \quit
+
+--
+-- pg_get_logical_snapshot_meta()
+--
+CREATE FUNCTION pg_get_logical_snapshot_meta(IN in_lsn pg_lsn,
+    OUT magic int4,
+    OUT checksum int4,
+    OUT version int4
+)
+AS 'MODULE_PATHNAME', 'pg_get_logical_snapshot_meta'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_logical_snapshot_meta(pg_lsn) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_logical_snapshot_meta(pg_lsn) TO pg_read_server_files;
+
+--
+-- pg_get_logical_snapshot_info()
+--
+CREATE FUNCTION pg_get_logical_snapshot_info(IN in_lsn pg_lsn,
+    OUT state int2,
+    OUT xmin xid,
+    OUT xmax xid,
+    OUT start_decoding_at pg_lsn,
+    OUT two_phase_at pg_lsn,
+    OUT initial_xmin_horizon xid,
+    OUT building_full_snapshot boolean,
+    OUT in_slot_creation boolean,
+    OUT last_serialized_snapshot pg_lsn,
+    OUT next_phase_at xid,
+    OUT committed_count int8,
+    OUT committed_xip xid[],
+    OUT catchange_count int8,
+    OUT catchange_xip xid[]
+)
+AS 'MODULE_PATHNAME', 'pg_get_logical_snapshot_info'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_logical_snapshot_info(pg_lsn) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_logical_snapshot_info(pg_lsn) TO pg_read_server_files;
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.c b/contrib/pg_logicalinspect/pg_logicalinspect.c
new file mode 100644
index 0000000000..8fba5cbda6
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.c
@@ -0,0 +1,254 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_logicalinspect.c
+ *		  Functions to inspect contents of PostgreSQL logical snapshots
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ *		  contrib/pg_logicalinspect/pg_logicalinspect.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "funcapi.h"
+#include "replication/snapbuild_internal.h"
+#include "utils/array.h"
+#include "utils/pg_lsn.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_meta);
+PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_info);
+
+/*
+ * NOTE: For any code change or issue fix here, it is highly recommended to
+ * give a thought about doing the same in SnapBuildRestore() as well.
+ */
+
+/*
+ * Validate the logical snapshot file.
+ */
+static void
+ValidateAndRestoreSnapshotFile(XLogRecPtr lsn, SnapBuildOnDisk *ondisk,
+							   const char *path)
+{
+	int			fd;
+	Size		sz;
+	pg_crc32c	checksum;
+	MemoryContext context;
+
+	context = AllocSetContextCreate(CurrentMemoryContext,
+									"logicalsnapshot inspect context",
+									ALLOCSET_DEFAULT_SIZES);
+
+	fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
+
+	if (fd < 0 && errno == ENOENT)
+		ereport(ERROR,
+				errmsg("file \"%s\" does not exist", path));
+	else if (fd < 0)
+		ereport(ERROR,
+				(errcode_for_file_access(),
+				 errmsg("could not open file \"%s\": %m", path)));
+
+	/* ----
+	 * Make sure the snapshot had been stored safely to disk, that's normally
+	 * cheap.
+	 * Note that we do not need PANIC here, nobody will be able to use the
+	 * slot without fsyncing, and saving it won't succeed without an fsync()
+	 * either...
+	 * ----
+	 */
+	fsync_fname(path, false);
+	fsync_fname(PG_LOGICAL_SNAPSHOTS_DIR, true);
+
+
+	/* read statically sized portion of snapshot */
+	SnapBuildRestoreContents(fd, (char *) ondisk, SnapBuildOnDiskConstantSize, path);
+
+	if (ondisk->magic != SNAPBUILD_MAGIC)
+		ereport(ERROR,
+				(errcode(ERRCODE_DATA_CORRUPTED),
+				 errmsg("snapbuild state file \"%s\" has wrong magic number: %u instead of %u",
+						path, ondisk->magic, SNAPBUILD_MAGIC)));
+
+	if (ondisk->version != SNAPBUILD_VERSION)
+		ereport(ERROR,
+				(errcode(ERRCODE_DATA_CORRUPTED),
+				 errmsg("snapbuild state file \"%s\" has unsupported version: %u instead of %u",
+						path, ondisk->version, SNAPBUILD_VERSION)));
+
+	INIT_CRC32C(checksum);
+	COMP_CRC32C(checksum,
+				((char *) ondisk) + SnapBuildOnDiskNotChecksummedSize,
+				SnapBuildOnDiskConstantSize - SnapBuildOnDiskNotChecksummedSize);
+
+	/* read SnapBuild */
+	SnapBuildRestoreContents(fd, (char *) &ondisk->builder, sizeof(SnapBuild), path);
+	COMP_CRC32C(checksum, &ondisk->builder, sizeof(SnapBuild));
+
+	ondisk->builder.context = context;
+
+	/* restore committed xacts information */
+	if (ondisk->builder.committed.xcnt > 0)
+	{
+		sz = sizeof(TransactionId) * ondisk->builder.committed.xcnt;
+		ondisk->builder.committed.xip = MemoryContextAllocZero(ondisk->builder.context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.committed.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.committed.xip, sz);
+	}
+
+	/* restore catalog modifying xacts information */
+	if (ondisk->builder.catchange.xcnt > 0)
+	{
+		sz = sizeof(TransactionId) * ondisk->builder.catchange.xcnt;
+		ondisk->builder.catchange.xip = MemoryContextAllocZero(ondisk->builder.context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.catchange.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.catchange.xip, sz);
+	}
+
+	if (CloseTransientFile(fd) != 0)
+		ereport(ERROR,
+				(errcode_for_file_access(),
+				 errmsg("could not close file \"%s\": %m", path)));
+
+	FIN_CRC32C(checksum);
+
+	/* verify checksum of what we've read */
+	if (!EQ_CRC32C(checksum, ondisk->checksum))
+		ereport(ERROR,
+				(errcode(ERRCODE_DATA_CORRUPTED),
+				 errmsg("checksum mismatch for snapbuild state file \"%s\": is %u, should be %u",
+						path, checksum, ondisk->checksum)));
+}
+
+/*
+ * Retrieve the logical snapshot file metadata.
+ */
+Datum
+pg_get_logical_snapshot_meta(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_META_COLS 3
+	SnapBuildOnDisk ondisk;
+	XLogRecPtr	lsn;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_META_COLS];
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_META_COLS];
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+	int			i = 0;
+
+	lsn = PG_GETARG_LSN(0);
+
+	sprintf(path, "%s/%X-%X.snap",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			LSN_FORMAT_ARGS(lsn));
+
+	ValidateAndRestoreSnapshotFile(lsn, &ondisk, path);
+
+	/* Build a tuple descriptor for our result type. */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	memset(nulls, 0, sizeof(nulls));
+
+	values[i++] = Int32GetDatum(ondisk.magic);
+	values[i++] = Int32GetDatum(ondisk.checksum);
+	values[i++] = Int32GetDatum(ondisk.version);
+
+	Assert(i == PG_GET_LOGICAL_SNAPSHOT_META_COLS);
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	MemoryContextReset(ondisk.builder.context);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_META_COLS
+}
+
+Datum
+pg_get_logical_snapshot_info(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_INFO_COLS 14
+	SnapBuildOnDisk ondisk;
+	XLogRecPtr	lsn;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS];
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS];
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+	int			i = 0;
+
+	lsn = PG_GETARG_LSN(0);
+
+	sprintf(path, "%s/%X-%X.snap",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			LSN_FORMAT_ARGS(lsn));
+
+	ValidateAndRestoreSnapshotFile(lsn, &ondisk, path);
+
+	/* Build a tuple descriptor for our result type. */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	memset(nulls, 0, sizeof(nulls));
+
+	values[i++] = Int16GetDatum(ondisk.builder.state);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.xmin);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.xmax);
+	values[i++] = LSNGetDatum(ondisk.builder.start_decoding_at);
+	values[i++] = LSNGetDatum(ondisk.builder.two_phase_at);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.initial_xmin_horizon);
+	values[i++] = BoolGetDatum(ondisk.builder.building_full_snapshot);
+	values[i++] = BoolGetDatum(ondisk.builder.in_slot_creation);
+	values[i++] = LSNGetDatum(ondisk.builder.last_serialized_snapshot);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.next_phase_at);
+	values[i++] = Int64GetDatum(ondisk.builder.committed.xcnt);
+
+	if (ondisk.builder.committed.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+		int			narrayelems;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.committed.xcnt * sizeof(Datum));
+		narrayelems = 0;
+
+		for (narrayelems = 0; narrayelems < ondisk.builder.committed.xcnt; narrayelems++)
+			arrayelems[narrayelems] = Int64GetDatum((int64) ondisk.builder.committed.xip[narrayelems]);
+
+		values[i++] = PointerGetDatum(construct_array_builtin(arrayelems, narrayelems, INT8OID));
+	}
+	else
+		nulls[i++] = true;
+
+	values[i++] = Int64GetDatum(ondisk.builder.catchange.xcnt);
+
+	if (ondisk.builder.catchange.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+		int			narrayelems;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.catchange.xcnt * sizeof(Datum));
+		narrayelems = 0;
+
+		for (narrayelems = 0; narrayelems < ondisk.builder.catchange.xcnt; narrayelems++)
+			arrayelems[narrayelems] = Int64GetDatum((int64) ondisk.builder.catchange.xip[narrayelems]);
+
+		values[i++] = PointerGetDatum(construct_array_builtin(arrayelems, narrayelems, INT8OID));
+	}
+	else
+		nulls[i++] = true;
+
+	Assert(i == PG_GET_LOGICAL_SNAPSHOT_INFO_COLS);
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	MemoryContextReset(ondisk.builder.context);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_INFO_COLS
+}
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.control b/contrib/pg_logicalinspect/pg_logicalinspect.control
new file mode 100644
index 0000000000..b4a70e57ba
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.control
@@ -0,0 +1,5 @@
+# pg_logicalinspect extension
+comment = 'functions to inspect logical decoding components'
+default_version = '1.0'
+module_pathname = '$libdir/pg_logicalinspect'
+relocatable = true
diff --git a/contrib/pg_logicalinspect/specs/logical_inspect.spec b/contrib/pg_logicalinspect/specs/logical_inspect.spec
new file mode 100644
index 0000000000..e11eb63615
--- /dev/null
+++ b/contrib/pg_logicalinspect/specs/logical_inspect.spec
@@ -0,0 +1,34 @@
+# Test the pg_logicalinspect functions: that needs some permutation to
+# ensure that we are creating multiple logical snapshots and that one of them
+# contains ongoing catalogs changes.
+setup
+{
+    DROP TABLE IF EXISTS tbl1;
+    CREATE TABLE tbl1 (val1 integer, val2 integer);
+	CREATE EXTENSION pg_logicalinspect;
+}
+
+teardown
+{
+    DROP TABLE tbl1;
+    SELECT 'stop' FROM pg_drop_replication_slot('isolation_slot');
+	DROP EXTENSION pg_logicalinspect;
+}
+
+session "s0"
+setup { SET synchronous_commit=on; }
+step "s0_init" { SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding'); }
+step "s0_begin" { BEGIN; }
+step "s0_savepoint" { SAVEPOINT sp1; }
+step "s0_truncate" { TRUNCATE tbl1; }
+step "s0_insert" { INSERT INTO tbl1 VALUES (1); }
+step "s0_commit" { COMMIT; }
+
+session "s1"
+setup { SET synchronous_commit=on; }
+step "s1_checkpoint" { CHECKPOINT; }
+step "s1_get_changes" { SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0'); }
+step "s1_get_logical_snapshot_meta" { SELECT COUNT((pg_get_logical_snapshot_meta(f.name::pg_lsn))) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f; }
+step "s1_get_logical_snapshot_info" { SELECT (pg_get_logical_snapshot_info(f.name::pg_lsn)).state,(pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_xip,1),(pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_xip,1) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f ORDER BY 2; }
+
+permutation "s0_init" "s0_begin" "s0_savepoint" "s0_truncate" "s1_checkpoint" "s1_get_changes" "s0_commit" "s0_begin" "s0_insert" "s1_checkpoint" "s1_get_changes" "s0_commit" "s1_get_changes" "s1_get_logical_snapshot_info" "s1_get_logical_snapshot_meta"
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index 44639a8dca..7c381949a5 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -154,6 +154,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
  &pgbuffercache;
  &pgcrypto;
  &pgfreespacemap;
+ &pglogicalinspect;
  &pgprewarm;
  &pgrowlocks;
  &pgstatstatements;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index a7ff5f8264..66e6dccd4c 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -143,6 +143,7 @@
 <!ENTITY pgbuffercache   SYSTEM "pgbuffercache.sgml">
 <!ENTITY pgcrypto        SYSTEM "pgcrypto.sgml">
 <!ENTITY pgfreespacemap  SYSTEM "pgfreespacemap.sgml">
+<!ENTITY pglogicalinspect  SYSTEM "pglogicalinspect.sgml">
 <!ENTITY pgprewarm       SYSTEM "pgprewarm.sgml">
 <!ENTITY pgrowlocks      SYSTEM "pgrowlocks.sgml">
 <!ENTITY pgstatstatements SYSTEM "pgstatstatements.sgml">
diff --git a/doc/src/sgml/pglogicalinspect.sgml b/doc/src/sgml/pglogicalinspect.sgml
new file mode 100644
index 0000000000..ce3c688b4f
--- /dev/null
+++ b/doc/src/sgml/pglogicalinspect.sgml
@@ -0,0 +1,145 @@
+<!-- doc/src/sgml/pglogicalinspect.sgml -->
+
+<sect1 id="pglogicalinspect" xreflabel="pg_logicalinspect">
+ <title>pg_logicalinspect &mdash; logical decoding components inspection</title>
+
+ <indexterm zone="pglogicalinspect">
+  <primary>pg_logicalinspect</primary>
+ </indexterm>
+
+ <para>
+  The <filename>pg_logicalinspect</filename> module provides SQL functions
+  that allow you to inspect the contents of logical decoding components. It
+  allows to inspect serialized logical snapshots of a running
+  <productname>PostgreSQL</productname> database cluster, which is useful
+  for debugging or educational purposes.
+ </para>
+
+ <note>
+  <para>
+   The <filename>pg_logicalinspect</filename> functions are called
+   using an LSN argument that can be extracted from the output name of the
+   <function>pg_ls_logicalsnapdir</function>() function.
+  </para>
+ </note>
+
+ <sect2 id="pglogicalinspect-funcs">
+  <title>General Functions</title>
+
+  <variablelist>
+   <varlistentry id="pglogicalinspect-funcs-pg-get-logical-snapshot-meta">
+    <term>
+     <function>pg_get_logical_snapshot_meta(in_lsn pg_lsn) returns record</function>
+    </term>
+
+    <listitem>
+     <para>
+      Gets logical snapshot metadata about a snapshot file that is located in
+      the server's <filename>pg_logical/snapshots</filename> directory.
+      The <replaceable>in_lsn</replaceable> argument can be extracted from the
+      snapshot file name.
+      example:
+<screen>
+postgres=# SELECT * FROM pg_ls_logicalsnapdir();
+-[ RECORD 1 ]+-----------------------
+name         | 0-40796E18.snap
+size         | 152
+modification | 2024-08-14 16:36:32+00
+
+postgres=# SELECT * FROM pg_get_logical_snapshot_meta('0/40796E18');
+-[ RECORD 1 ]--------
+magic    | 1369563137
+checksum | 1028045905
+version  | 6
+
+postgres=# SELECT (pg_get_logical_snapshot_meta(f.name::pg_lsn)).*
+           FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name
+                 FROM pg_ls_logicalsnapdir()) AS f;
+-[ RECORD 1 ]--------
+magic    | 1369563137
+checksum | 1028045905
+version  | 6
+</screen>
+     </para>
+     <para>
+      If <replaceable>in_lsn</replaceable> does not match a snapshot file, the
+      function raises an error.
+     </para>
+    </listitem>
+   </varlistentry>
+
+   <varlistentry id="pglogicalinspect-funcs-pg-get-logical-snapshot-info">
+    <term>
+     <function>pg_get_logical_snapshot_info(in_lsn pg_lsn) returns record</function>
+    </term>
+
+    <listitem>
+     <para>
+      Gets logical snapshot information about a snapshot file that is located in
+      the <filename>pg_logical/snapshots</filename> directory.
+      The <replaceable>in_lsn</replaceable> argument can be extracted from the
+      snapshot file name.
+      example:
+<screen>
+postgres=# SELECT * FROM pg_ls_logicalsnapdir();
+-[ RECORD 1 ]+-----------------------
+name         | 0-40796E18.snap
+size         | 152
+modification | 2024-08-14 16:36:32+00
+
+postgres=# SELECT * FROM pg_get_logical_snapshot_info('0/40796E18');
+-[ RECORD 1 ]------------+-----------
+state                    | 2
+xmin                     | 751
+xmax                     | 751
+start_decoding_at        | 0/40796AF8
+two_phase_at             | 0/40796AF8
+initial_xmin_horizon     | 0
+building_full_snapshot   | f
+in_slot_creation         | f
+last_serialized_snapshot | 0/0
+next_phase_at            | 0
+committed_count          | 0
+committed_xip            |
+catchange_count          | 2
+catchange_xip            | {751,752}
+
+postgres=# SELECT (pg_get_logical_snapshot_info(f.name::pg_lsn)).*
+           FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name
+                 FROM pg_ls_logicalsnapdir()) AS f;
+-[ RECORD 1 ]------------+-----------
+state                    | 2
+xmin                     | 751
+xmax                     | 751
+start_decoding_at        | 0/40796AF8
+two_phase_at             | 0/40796AF8
+initial_xmin_horizon     | 0
+building_full_snapshot   | f
+in_slot_creation         | f
+last_serialized_snapshot | 0/0
+next_phase_at            | 0
+committed_count          | 0
+committed_xip            |
+catchange_count          | 2
+catchange_xip            | {751,752}
+</screen>
+     </para>
+     <para>
+      If <replaceable>in_lsn</replaceable> does not match a snapshot file, the
+      function raises an error.
+     </para>
+    </listitem>
+   </varlistentry>
+
+  </variablelist>
+ </sect2>
+
+ <sect2 id="pglogicalinspect-author">
+  <title>Author</title>
+
+  <para>
+   Bertrand Drouvot <email>bertranddrouvot.pg@gmail.com</email>
+  </para>
+ </sect2>
+
+</sect1>
diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c
index 0450f94ba8..bb7def9440 100644
--- a/src/backend/replication/logical/snapbuild.c
+++ b/src/backend/replication/logical/snapbuild.c
@@ -134,6 +134,7 @@
 #include "replication/logical.h"
 #include "replication/reorderbuffer.h"
 #include "replication/snapbuild.h"
+#include "replication/snapbuild_internal.h"
 #include "storage/fd.h"
 #include "storage/lmgr.h"
 #include "storage/proc.h"
@@ -143,146 +144,6 @@
 #include "utils/memutils.h"
 #include "utils/snapmgr.h"
 #include "utils/snapshot.h"
-
-/*
- * This struct contains the current state of the snapshot building
- * machinery. Besides a forward declaration in the header, it is not exposed
- * to the public, so we can easily change its contents.
- */
-struct SnapBuild
-{
-	/* how far are we along building our first full snapshot */
-	SnapBuildState state;
-
-	/* private memory context used to allocate memory for this module. */
-	MemoryContext context;
-
-	/* all transactions < than this have committed/aborted */
-	TransactionId xmin;
-
-	/* all transactions >= than this are uncommitted */
-	TransactionId xmax;
-
-	/*
-	 * Don't replay commits from an LSN < this LSN. This can be set externally
-	 * but it will also be advanced (never retreat) from within snapbuild.c.
-	 */
-	XLogRecPtr	start_decoding_at;
-
-	/*
-	 * LSN at which two-phase decoding was enabled or LSN at which we found a
-	 * consistent point at the time of slot creation.
-	 *
-	 * The prepared transactions, that were skipped because previously
-	 * two-phase was not enabled or are not covered by initial snapshot, need
-	 * to be sent later along with commit prepared and they must be before
-	 * this point.
-	 */
-	XLogRecPtr	two_phase_at;
-
-	/*
-	 * Don't start decoding WAL until the "xl_running_xacts" information
-	 * indicates there are no running xids with an xid smaller than this.
-	 */
-	TransactionId initial_xmin_horizon;
-
-	/* Indicates if we are building full snapshot or just catalog one. */
-	bool		building_full_snapshot;
-
-	/*
-	 * Indicates if we are using the snapshot builder for the creation of a
-	 * logical replication slot. If it's true, the start point for decoding
-	 * changes is not determined yet. So we skip snapshot restores to properly
-	 * find the start point. See SnapBuildFindSnapshot() for details.
-	 */
-	bool		in_slot_creation;
-
-	/*
-	 * Snapshot that's valid to see the catalog state seen at this moment.
-	 */
-	Snapshot	snapshot;
-
-	/*
-	 * LSN of the last location we are sure a snapshot has been serialized to.
-	 */
-	XLogRecPtr	last_serialized_snapshot;
-
-	/*
-	 * The reorderbuffer we need to update with usable snapshots et al.
-	 */
-	ReorderBuffer *reorder;
-
-	/*
-	 * TransactionId at which the next phase of initial snapshot building will
-	 * happen. InvalidTransactionId if not known (i.e. SNAPBUILD_START), or
-	 * when no next phase necessary (SNAPBUILD_CONSISTENT).
-	 */
-	TransactionId next_phase_at;
-
-	/*
-	 * Array of transactions which could have catalog changes that committed
-	 * between xmin and xmax.
-	 */
-	struct
-	{
-		/* number of committed transactions */
-		size_t		xcnt;
-
-		/* available space for committed transactions */
-		size_t		xcnt_space;
-
-		/*
-		 * Until we reach a CONSISTENT state, we record commits of all
-		 * transactions, not just the catalog changing ones. Record when that
-		 * changes so we know we cannot export a snapshot safely anymore.
-		 */
-		bool		includes_all_transactions;
-
-		/*
-		 * Array of committed transactions that have modified the catalog.
-		 *
-		 * As this array is frequently modified we do *not* keep it in
-		 * xidComparator order. Instead we sort the array when building &
-		 * distributing a snapshot.
-		 *
-		 * TODO: It's unclear whether that reasoning has much merit. Every
-		 * time we add something here after becoming consistent will also
-		 * require distributing a snapshot. Storing them sorted would
-		 * potentially also make it easier to purge (but more complicated wrt
-		 * wraparound?). Should be improved if sorting while building the
-		 * snapshot shows up in profiles.
-		 */
-		TransactionId *xip;
-	}			committed;
-
-	/*
-	 * Array of transactions and subtransactions that had modified catalogs
-	 * and were running when the snapshot was serialized.
-	 *
-	 * We normally rely on some WAL record types such as HEAP2_NEW_CID to know
-	 * if the transaction has changed the catalog. But it could happen that
-	 * the logical decoding decodes only the commit record of the transaction
-	 * after restoring the previously serialized snapshot in which case we
-	 * will miss adding the xid to the snapshot and end up looking at the
-	 * catalogs with the wrong snapshot.
-	 *
-	 * Now to avoid the above problem, we serialize the transactions that had
-	 * modified the catalogs and are still running at the time of snapshot
-	 * serialization. We fill this array while restoring the snapshot and then
-	 * refer it while decoding commit to ensure if the xact has modified the
-	 * catalog. We discard this array when all the xids in the list become old
-	 * enough to matter. See SnapBuildPurgeOlderTxn for details.
-	 */
-	struct
-	{
-		/* number of transactions */
-		size_t		xcnt;
-
-		/* This array must be sorted in xidComparator order */
-		TransactionId *xip;
-	}			catchange;
-};
-
 /*
  * Starting a transaction -- which we need to do while exporting a snapshot --
  * removes knowledge about the previously used resowner, so we save it here.
@@ -312,7 +173,6 @@ static void SnapBuildWaitSnapshot(xl_running_xacts *running, TransactionId cutof
 /* serialization functions */
 static void SnapBuildSerialize(SnapBuild *builder, XLogRecPtr lsn);
 static bool SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn);
-static void SnapBuildRestoreContents(int fd, char *dest, Size size, const char *path);
 
 /*
  * Allocate a new snapshot builder.
@@ -1557,48 +1417,6 @@ SnapBuildWaitSnapshot(xl_running_xacts *running, TransactionId cutoff)
 	}
 }
 
-/* -----------------------------------
- * Snapshot serialization support
- * -----------------------------------
- */
-
-/*
- * We store current state of struct SnapBuild on disk in the following manner:
- *
- * struct SnapBuildOnDisk;
- * TransactionId * committed.xcnt; (*not xcnt_space*)
- * TransactionId * catchange.xcnt;
- *
- */
-typedef struct SnapBuildOnDisk
-{
-	/* first part of this struct needs to be version independent */
-
-	/* data not covered by checksum */
-	uint32		magic;
-	pg_crc32c	checksum;
-
-	/* data covered by checksum */
-
-	/* version, in case we want to support pg_upgrade */
-	uint32		version;
-	/* how large is the on disk data, excluding the constant sized part */
-	uint32		length;
-
-	/* version dependent part */
-	SnapBuild	builder;
-
-	/* variable amount of TransactionIds follows */
-} SnapBuildOnDisk;
-
-#define SnapBuildOnDiskConstantSize \
-	offsetof(SnapBuildOnDisk, builder)
-#define SnapBuildOnDiskNotChecksummedSize \
-	offsetof(SnapBuildOnDisk, version)
-
-#define SNAPBUILD_MAGIC 0x51A1E001
-#define SNAPBUILD_VERSION 6
-
 /*
  * Store/Load a snapshot from disk, depending on the snapshot builder's state.
  *
@@ -1859,6 +1677,10 @@ out:
 /*
  * Restore a snapshot into 'builder' if previously one has been stored at the
  * location indicated by 'lsn'. Returns true if successful, false otherwise.
+ *
+ * NOTE: For any code change or issue fix here, it is highly recommended to
+ * give a thought about doing the same in pg_logicalinspect contrib module
+ * as well.
  */
 static bool
 SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
@@ -2033,7 +1855,7 @@ snapshot_not_interesting:
 /*
  * Read the contents of the serialized snapshot to 'dest'.
  */
-static void
+void
 SnapBuildRestoreContents(int fd, char *dest, Size size, const char *path)
 {
 	int			readBytes;
diff --git a/src/include/port/pg_crc32c.h b/src/include/port/pg_crc32c.h
index 63c8e3a00b..cfc8c07944 100644
--- a/src/include/port/pg_crc32c.h
+++ b/src/include/port/pg_crc32c.h
@@ -47,7 +47,7 @@ typedef uint32 pg_crc32c;
 	((crc) = pg_comp_crc32c_sse42((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
 
 #elif defined(USE_ARMV8_CRC32C)
 /* Use ARMv8 CRC Extension instructions. */
@@ -56,7 +56,7 @@ extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t le
 	((crc) = pg_comp_crc32c_armv8((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
 
 #elif defined(USE_LOONGARCH_CRC32C)
 /* Use LoongArch CRCC instructions. */
@@ -65,7 +65,7 @@ extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t le
 	((crc) = pg_comp_crc32c_loongarch((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len);
 
 #elif defined(USE_SSE42_CRC32C_WITH_RUNTIME_CHECK) || defined(USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK)
 
@@ -77,14 +77,14 @@ extern pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_
 	((crc) = pg_comp_crc32c((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
-extern pg_crc32c (*pg_comp_crc32c) (pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c (*pg_comp_crc32c) (pg_crc32c crc, const void *data, size_t len);
 
 #ifdef USE_SSE42_CRC32C_WITH_RUNTIME_CHECK
-extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
 #endif
 #ifdef USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK
-extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
 #endif
 
 #else
@@ -103,7 +103,7 @@ extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t le
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 #endif
 
-extern pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
 
 #endif
 
diff --git a/src/include/replication/snapbuild.h b/src/include/replication/snapbuild.h
index caa5113ff8..dbb4bc2f4b 100644
--- a/src/include/replication/snapbuild.h
+++ b/src/include/replication/snapbuild.h
@@ -46,7 +46,7 @@ typedef enum
 	SNAPBUILD_CONSISTENT = 2,
 } SnapBuildState;
 
-/* forward declare so we don't have to expose the struct to the public */
+/* forward declare so we don't have to include snapbuild_internal.h */
 struct SnapBuild;
 typedef struct SnapBuild SnapBuild;
 
diff --git a/src/include/replication/snapbuild_internal.h b/src/include/replication/snapbuild_internal.h
new file mode 100644
index 0000000000..2b332303c2
--- /dev/null
+++ b/src/include/replication/snapbuild_internal.h
@@ -0,0 +1,204 @@
+/*-------------------------------------------------------------------------
+ *
+ * snapbuild_internal.h
+ *    This file contains declarations for logical decoding utility
+ *    functions for internal use.
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * src/include/replication/snapbuild_internal.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef SNAPBUILD_INTERNAL_H
+#define SNAPBUILD_INTERNAL_H
+
+#include "port/pg_crc32c.h"
+#include "replication/reorderbuffer.h"
+#include "replication/snapbuild.h"
+
+/* -----------------------------------
+ * Snapshot serialization support
+ * -----------------------------------
+ */
+
+#define SnapBuildOnDiskConstantSize \
+	offsetof(SnapBuildOnDisk, builder)
+#define SnapBuildOnDiskNotChecksummedSize \
+	offsetof(SnapBuildOnDisk, version)
+
+#define SNAPBUILD_MAGIC 0x51A1E001
+#define SNAPBUILD_VERSION 6
+
+/*
+ * This struct contains the current state of the snapshot building
+ * machinery. It is exposed to the public, so pay attention when changing its
+ * contents.
+ */
+typedef struct SnapBuild
+{
+	/* how far are we along building our first full snapshot */
+	SnapBuildState state;
+
+	/* private memory context used to allocate memory for this module. */
+	MemoryContext context;
+
+	/* all transactions < than this have committed/aborted */
+	TransactionId xmin;
+
+	/* all transactions >= than this are uncommitted */
+	TransactionId xmax;
+
+	/*
+	 * Don't replay commits from an LSN < this LSN. This can be set externally
+	 * but it will also be advanced (never retreat) from within snapbuild.c.
+	 */
+	XLogRecPtr	start_decoding_at;
+
+	/*
+	 * LSN at which two-phase decoding was enabled or LSN at which we found a
+	 * consistent point at the time of slot creation.
+	 *
+	 * The prepared transactions, that were skipped because previously
+	 * two-phase was not enabled or are not covered by initial snapshot, need
+	 * to be sent later along with commit prepared and they must be before
+	 * this point.
+	 */
+	XLogRecPtr	two_phase_at;
+
+	/*
+	 * Don't start decoding WAL until the "xl_running_xacts" information
+	 * indicates there are no running xids with an xid smaller than this.
+	 */
+	TransactionId initial_xmin_horizon;
+
+	/* Indicates if we are building full snapshot or just catalog one. */
+	bool		building_full_snapshot;
+
+	/*
+	 * Indicates if we are using the snapshot builder for the creation of a
+	 * logical replication slot. If it's true, the start point for decoding
+	 * changes is not determined yet. So we skip snapshot restores to properly
+	 * find the start point. See SnapBuildFindSnapshot() for details.
+	 */
+	bool		in_slot_creation;
+
+	/*
+	 * Snapshot that's valid to see the catalog state seen at this moment.
+	 */
+	Snapshot	snapshot;
+
+	/*
+	 * LSN of the last location we are sure a snapshot has been serialized to.
+	 */
+	XLogRecPtr	last_serialized_snapshot;
+
+	/*
+	 * The reorderbuffer we need to update with usable snapshots et al.
+	 */
+	ReorderBuffer *reorder;
+
+	/*
+	 * TransactionId at which the next phase of initial snapshot building will
+	 * happen. InvalidTransactionId if not known (i.e. SNAPBUILD_START), or
+	 * when no next phase necessary (SNAPBUILD_CONSISTENT).
+	 */
+	TransactionId next_phase_at;
+
+	/*
+	 * Array of transactions which could have catalog changes that committed
+	 * between xmin and xmax.
+	 */
+	struct
+	{
+		/* number of committed transactions */
+		size_t		xcnt;
+
+		/* available space for committed transactions */
+		size_t		xcnt_space;
+
+		/*
+		 * Until we reach a CONSISTENT state, we record commits of all
+		 * transactions, not just the catalog changing ones. Record when that
+		 * changes so we know we cannot export a snapshot safely anymore.
+		 */
+		bool		includes_all_transactions;
+
+		/*
+		 * Array of committed transactions that have modified the catalog.
+		 *
+		 * As this array is frequently modified we do *not* keep it in
+		 * xidComparator order. Instead we sort the array when building &
+		 * distributing a snapshot.
+		 *
+		 * TODO: It's unclear whether that reasoning has much merit. Every
+		 * time we add something here after becoming consistent will also
+		 * require distributing a snapshot. Storing them sorted would
+		 * potentially also make it easier to purge (but more complicated wrt
+		 * wraparound?). Should be improved if sorting while building the
+		 * snapshot shows up in profiles.
+		 */
+		TransactionId *xip;
+	}			committed;
+
+	/*
+	 * Array of transactions and subtransactions that had modified catalogs
+	 * and were running when the snapshot was serialized.
+	 *
+	 * We normally rely on some WAL record types such as HEAP2_NEW_CID to know
+	 * if the transaction has changed the catalog. But it could happen that
+	 * the logical decoding decodes only the commit record of the transaction
+	 * after restoring the previously serialized snapshot in which case we
+	 * will miss adding the xid to the snapshot and end up looking at the
+	 * catalogs with the wrong snapshot.
+	 *
+	 * Now to avoid the above problem, we serialize the transactions that had
+	 * modified the catalogs and are still running at the time of snapshot
+	 * serialization. We fill this array while restoring the snapshot and then
+	 * refer it while decoding commit to ensure if the xact has modified the
+	 * catalog. We discard this array when all the xids in the list become old
+	 * enough to matter. See SnapBuildPurgeOlderTxn for details.
+	 */
+	struct
+	{
+		/* number of transactions */
+		size_t		xcnt;
+
+		/* This array must be sorted in xidComparator order */
+		TransactionId *xip;
+	}			catchange;
+} SnapBuild;
+
+/*
+ * We store current state of struct SnapBuild on disk in the following manner:
+ *
+ * struct SnapBuildOnDisk;
+ * TransactionId * committed.xcnt; (*not xcnt_space*)
+ * TransactionId * catchange.xcnt;
+ *
+ */
+typedef struct SnapBuildOnDisk
+{
+	/* first part of this struct needs to be version independent */
+
+	/* data not covered by checksum */
+	uint32		magic;
+	pg_crc32c	checksum;
+
+	/* data covered by checksum */
+
+	/* version, in case we want to support pg_upgrade */
+	uint32		version;
+	/* how large is the on disk data, excluding the constant sized part */
+	uint32		length;
+
+	/* version dependent part */
+	SnapBuild	builder;
+
+	/* variable amount of TransactionIds follows */
+} SnapBuildOnDisk;
+
+extern void SnapBuildRestoreContents(int fd, char *dest, Size size, const char *path);
+
+#endif							/* SNAPBUILD_INTERNAL_H */
-- 
2.34.1

#25Peter Smith
smithpb2250@gmail.com
In reply to: Bertrand Drouvot (#21)
1 attachment(s)
Re: Add contrib/pg_logicalsnapinspect

HI, here are some mostly minor review comments for the patch v5-0001.

======
Commit message

1.
Do you think you should also name the new functions here?

======
contrib/pg_logicalinspect/pg_logicalinspect.c

2.
Regarding the question about static function declarations:

Shveta wrote: I was somehow under the impression that this is the way
in the postgres i.e. not add redundant declarations. Will be good to
know what others think on this.

FWIW, my understanding is the convention is just to be consistent with
whatever the module currently does. If it declares static functions,
then declare them all (redundant or not). If it doesn't declare static
functions, then don't add one. But, in the current case, since this is
a new module, I guess it is entirely up to you whatever you want to
do.

~~~

3.
+/*
+ * NOTE: For any code change or issue fix here, it is highly recommended to
+ * give a thought about doing the same in SnapBuildRestore() as well.
+ */
+

nit - I think this NOTE should be part of this module's header
comment. (e.g. like the tablesync.c NOTES)

~~~

ValidateSnapshotFile:

4.
+ValidateSnapshotFile(XLogRecPtr lsn, SnapBuildOnDisk *ondisk, const char *path)
+{
+ int fd;
+ Size sz;

nit - The 'sz' is overwritten a few times. I thnk declaring it at each
scope where used would be tidier.

~~~

5.
+ fsync_fname(path, false);
+ fsync_fname(PG_LOGICAL_SNAPSHOTS_DIR, true);
+
+

nit - remove some excessive blank lines

~~~

6.
+ /* read statically sized portion of snapshot */
+ SnapBuildRestoreContents(fd, (char *) ondisk,
SnapBuildOnDiskConstantSize, path);

Should that say "fixed size portion"?

~~~

pg_get_logical_snapshot_info:

7.
+ if (ondisk.builder.committed.xcnt > 0)
+ {
+ Datum    *arrayelems;
+ int narrayelems;
+
+ arrayelems = (Datum *) palloc(ondisk.builder.committed.xcnt * sizeof(Datum));
+ narrayelems = 0;
+
+ for (narrayelems = 0; narrayelems < ondisk.builder.committed.xcnt;
narrayelems++)
+ arrayelems[narrayelems] = Int64GetDatum((int64)
ondisk.builder.committed.xip[narrayelems]);

nit - Why the double assignment of narrayelems = 0? It is simpler to
assign at the declaration and then remove both others.

~~~

8.
+ if (ondisk.builder.catchange.xcnt > 0)
+ {
+ Datum    *arrayelems;
+ int narrayelems;
+
+ arrayelems = (Datum *) palloc(ondisk.builder.catchange.xcnt * sizeof(Datum));
+ narrayelems = 0;
+
+ for (narrayelems = 0; narrayelems < ondisk.builder.catchange.xcnt;
narrayelems++)
+ arrayelems[narrayelems] = Int64GetDatum((int64)
ondisk.builder.catchange.xip[narrayelems]);

nit - ditto previous comment

======
doc/src/sgml/pglogicalinspect.sgml

9.
+ <para>
+  The <filename>pg_logicalinspect</filename> module provides SQL functions
+  that allow you to inspect the contents of logical decoding components. It
+  allows to inspect serialized logical snapshots of a running
+  <productname>PostgreSQL</productname> database cluster, which is useful
+  for debugging or educational purposes.
+ </para>

nit - /It allows to inspect/It allows the inspection of/

~~~

10.
+ example:

nit - /example:/For example:/ (this is in a couple of places)

======
src/include/replication/snapbuild_internal.h

11.
+#ifndef INTERNAL_SNAPBUILD_H
+#define INTERNAL_SNAPBUILD_H

Shouldn't these be SNAPBUILD_INTERNAL_H to match the filename?

~~~

12.
The contents of the snapbuild.c that got moved into
snapbuild_internal.h also got shuffled around a bit.

e.g. originally the typedef struct SnapBuildOnDisk:

+/*
+ * We store current state of struct SnapBuild on disk in the following manner:
+ *
+ * struct SnapBuildOnDisk;
+ * TransactionId * committed.xcnt; (*not xcnt_space*)
+ * TransactionId * catchange.xcnt;
+ *
+ */
+typedef struct SnapBuildOnDisk

was directly beneath the comment:
-/* -----------------------------------
- * Snapshot serialization support
- * -----------------------------------
- */
-

The macros were also defined immediately after the SnapBuildOnDisk
fields they referred to.

Wasn't that original ordering better than how it is now ordered in
snapshot_internal.h?

======

Please also see the attachment, which implements some of those nits
mentioned above.

======
Kind Regards,
Peter Smith.
Fujitsu Australia

Attachments:

PS_NITPICKS_V5.txttext/plain; charset=US-ASCII; name=PS_NITPICKS_V5.txtDownload
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.c b/contrib/pg_logicalinspect/pg_logicalinspect.c
index dc9041a..2111202 100644
--- a/contrib/pg_logicalinspect/pg_logicalinspect.c
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.c
@@ -1,13 +1,17 @@
 /*-------------------------------------------------------------------------
  *
  * pg_logicalinspect.c
- *		  Functions to inspect contents of PostgreSQL logical snapshots
+ *		Functions to inspect contents of PostgreSQL logical snapshots
  *
  * Copyright (c) 2024, PostgreSQL Global Development Group
  *
  * IDENTIFICATION
- *		  contrib/pg_logicalinspect/pg_logicalinspect.c
+ *		contrib/pg_logicalinspect/pg_logicalinspect.c
  *
+ *
+ * NOTES
+ * 		For any code change or issue fix here, it is highly recommended to
+ * 		give a thought about doing the same in SnapBuildRestore() as well.
  *-------------------------------------------------------------------------
  */
 #include "postgres.h"
@@ -23,18 +27,12 @@ PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_meta);
 PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_info);
 
 /*
- * NOTE: For any code change or issue fix here, it is highly recommended to
- * give a thought about doing the same in SnapBuildRestore() as well.
- */
-
-/*
  * Validate the logical snapshot file.
  */
 static void
 ValidateSnapshotFile(XLogRecPtr lsn, SnapBuildOnDisk *ondisk, const char *path)
 {
 	int			fd;
-	Size		sz;
 	pg_crc32c	checksum;
 	MemoryContext context;
 
@@ -63,7 +61,6 @@ ValidateSnapshotFile(XLogRecPtr lsn, SnapBuildOnDisk *ondisk, const char *path)
 	fsync_fname(path, false);
 	fsync_fname(PG_LOGICAL_SNAPSHOTS_DIR, true);
 
-
 	/* read statically sized portion of snapshot */
 	SnapBuildRestoreContents(fd, (char *) ondisk, SnapBuildOnDiskConstantSize, path);
 
@@ -93,7 +90,7 @@ ValidateSnapshotFile(XLogRecPtr lsn, SnapBuildOnDisk *ondisk, const char *path)
 	/* restore committed xacts information */
 	if (ondisk->builder.committed.xcnt > 0)
 	{
-		sz = sizeof(TransactionId) * ondisk->builder.committed.xcnt;
+		Size sz = sizeof(TransactionId) * ondisk->builder.committed.xcnt;
 		ondisk->builder.committed.xip = MemoryContextAllocZero(ondisk->builder.context, sz);
 		SnapBuildRestoreContents(fd, (char *) ondisk->builder.committed.xip, sz, path);
 		COMP_CRC32C(checksum, ondisk->builder.committed.xip, sz);
@@ -102,7 +99,7 @@ ValidateSnapshotFile(XLogRecPtr lsn, SnapBuildOnDisk *ondisk, const char *path)
 	/* restore catalog modifying xacts information */
 	if (ondisk->builder.catchange.xcnt > 0)
 	{
-		sz = sizeof(TransactionId) * ondisk->builder.catchange.xcnt;
+		Size sz = sizeof(TransactionId) * ondisk->builder.catchange.xcnt;
 		ondisk->builder.catchange.xip = MemoryContextAllocZero(ondisk->builder.context, sz);
 		SnapBuildRestoreContents(fd, (char *) ondisk->builder.catchange.xip, sz, path);
 		COMP_CRC32C(checksum, ondisk->builder.catchange.xip, sz);
@@ -210,12 +207,11 @@ pg_get_logical_snapshot_info(PG_FUNCTION_ARGS)
 	if (ondisk.builder.committed.xcnt > 0)
 	{
 		Datum	   *arrayelems;
-		int			narrayelems;
+		int			narrayelems = 0;
 
 		arrayelems = (Datum *) palloc(ondisk.builder.committed.xcnt * sizeof(Datum));
-		narrayelems = 0;
 
-		for (narrayelems = 0; narrayelems < ondisk.builder.committed.xcnt; narrayelems++)
+		for (; narrayelems < ondisk.builder.committed.xcnt; narrayelems++)
 			arrayelems[narrayelems] = Int64GetDatum((int64) ondisk.builder.committed.xip[narrayelems]);
 
 		values[i++] = PointerGetDatum(construct_array_builtin(arrayelems, narrayelems, INT8OID));
@@ -228,12 +224,11 @@ pg_get_logical_snapshot_info(PG_FUNCTION_ARGS)
 	if (ondisk.builder.catchange.xcnt > 0)
 	{
 		Datum	   *arrayelems;
-		int			narrayelems;
+		int			narrayelems = 0;
 
 		arrayelems = (Datum *) palloc(ondisk.builder.catchange.xcnt * sizeof(Datum));
-		narrayelems = 0;
 
-		for (narrayelems = 0; narrayelems < ondisk.builder.catchange.xcnt; narrayelems++)
+		for (; narrayelems < ondisk.builder.catchange.xcnt; narrayelems++)
 			arrayelems[narrayelems] = Int64GetDatum((int64) ondisk.builder.catchange.xip[narrayelems]);
 
 		values[i++] = PointerGetDatum(construct_array_builtin(arrayelems, narrayelems, INT8OID));
diff --git a/doc/src/sgml/pglogicalinspect.sgml b/doc/src/sgml/pglogicalinspect.sgml
index 7767de2..3cc7742 100644
--- a/doc/src/sgml/pglogicalinspect.sgml
+++ b/doc/src/sgml/pglogicalinspect.sgml
@@ -10,7 +10,7 @@
  <para>
   The <filename>pg_logicalinspect</filename> module provides SQL functions
   that allow you to inspect the contents of logical decoding components. It
-  allows to inspect serialized logical snapshots of a running
+  allows the inspection of serialized logical snapshots of a running
   <productname>PostgreSQL</productname> database cluster, which is useful
   for debugging or educational purposes.
  </para>
@@ -38,7 +38,7 @@
       the <filename>pg_logical/snapshots</filename> directory.
       The <replaceable>in_lsn</replaceable> argument can be extracted from the
       snapshot file name.
-      example:
+      For example:
 <screen>
 postgres=# SELECT * FROM pg_ls_logicalsnapdir();
 -[ RECORD 1 ]+-----------------------
@@ -79,7 +79,7 @@ version  | 6
       the <filename>pg_logical/snapshots</filename> directory.
       The <replaceable>in_lsn</replaceable> argument can be extracted from the
       snapshot file name.
-      example:
+      For example:
 <screen>
 postgres=# SELECT * FROM pg_ls_logicalsnapdir();
 -[ RECORD 1 ]+-----------------------
#26Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: Peter Smith (#25)
1 attachment(s)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Wed, Sep 18, 2024 at 07:52:51PM +1000, Peter Smith wrote:

HI, here are some mostly minor review comments for the patch v5-0001.

Thanks for the review!

======
Commit message

1.
Do you think you should also name the new functions here?

Not sure about this one. It has not been done in 2258e76f90 for example.

======
contrib/pg_logicalinspect/pg_logicalinspect.c

2.
Regarding the question about static function declarations:

Shveta wrote: I was somehow under the impression that this is the way
in the postgres i.e. not add redundant declarations. Will be good to
know what others think on this.

FWIW, my understanding is the convention is just to be consistent with
whatever the module currently does. If it declares static functions,
then declare them all (redundant or not). If it doesn't declare static
functions, then don't add one. But, in the current case, since this is
a new module, I guess it is entirely up to you whatever you want to
do.

Thanks for the feedback and sharing your thoughts. I don't have a strong opinion
on this (though I tend to write the declaration(s)). I see it as a Nit, so let
just keep it as done in v6.

~~~

3.
+/*
+ * NOTE: For any code change or issue fix here, it is highly recommended to
+ * give a thought about doing the same in SnapBuildRestore() as well.
+ */
+

nit - I think this NOTE should be part of this module's header
comment. (e.g. like the tablesync.c NOTES)

Not sure about this one. I took pg_walinspect.c as an example. And we may want
to add more functionalities in the future that could have nothing to do with
SnapBuildRestore(). I think that I prefer where it is located currently (near the
code that "looks like" SnapBuildRestore()).

~~~

ValidateSnapshotFile:

4.
+ValidateSnapshotFile(XLogRecPtr lsn, SnapBuildOnDisk *ondisk, const char *path)
+{
+ int fd;
+ Size sz;

nit - The 'sz' is overwritten a few times. I thnk declaring it at each
scope where used would be tidier.

I see what you mean. I think it's a matter of taste and I generally also prefer
to do it the way you propose. That said, it's mainly inspired from SnapBuildRestore(),
so I think it's better to try to differ from it the less that we can.

~~~

5.
+ fsync_fname(path, false);
+ fsync_fname(PG_LOGICAL_SNAPSHOTS_DIR, true);
+
+

nit - remove some excessive blank lines

Agree. While it's the same as in SnapBuildRestore(), I'm ok to do this particular
change, done in v7 attached.

~~~

6.
+ /* read statically sized portion of snapshot */
+ SnapBuildRestoreContents(fd, (char *) ondisk,
SnapBuildOnDiskConstantSize, path);

Should that say "fixed size portion"?

Maybe, but same remark as for 4. (though that's only a comment).

~~~

pg_get_logical_snapshot_info:

7.
+ if (ondisk.builder.committed.xcnt > 0)
+ {
+ Datum    *arrayelems;
+ int narrayelems;
+
+ arrayelems = (Datum *) palloc(ondisk.builder.committed.xcnt * sizeof(Datum));
+ narrayelems = 0;
+
+ for (narrayelems = 0; narrayelems < ondisk.builder.committed.xcnt;
narrayelems++)
+ arrayelems[narrayelems] = Int64GetDatum((int64)
ondisk.builder.committed.xip[narrayelems]);

nit - Why the double assignment of narrayelems = 0?

Probably fat fingers when writting this part.

assign at the declaration and then remove both others.

yeah, done.

======
doc/src/sgml/pglogicalinspect.sgml

9.
+ <para>
+  The <filename>pg_logicalinspect</filename> module provides SQL functions
+  that allow you to inspect the contents of logical decoding components. It
+  allows to inspect serialized logical snapshots of a running
+  <productname>PostgreSQL</productname> database cluster, which is useful
+  for debugging or educational purposes.
+ </para>

nit - /It allows to inspect/It allows the inspection of/

Done.

~~~

10.
+ example:

nit - /example:/For example:/ (this is in a couple of places)

Done.

======
src/include/replication/snapbuild_internal.h

11.
+#ifndef INTERNAL_SNAPBUILD_H
+#define INTERNAL_SNAPBUILD_H

Shouldn't these be SNAPBUILD_INTERNAL_H to match the filename?

Yeah, was already mentioned by Shveta up-thread and fixed in v6.

~~~

12.
The contents of the snapbuild.c that got moved into
snapbuild_internal.h also got shuffled around a bit.

e.g. originally the typedef struct SnapBuildOnDisk:

+/*
+ * We store current state of struct SnapBuild on disk in the following manner:
+ *
+ * struct SnapBuildOnDisk;
+ * TransactionId * committed.xcnt; (*not xcnt_space*)
+ * TransactionId * catchange.xcnt;
+ *
+ */
+typedef struct SnapBuildOnDisk

was directly beneath the comment:
-/* -----------------------------------
- * Snapshot serialization support
- * -----------------------------------
- */
-

Moving it to the same place in v7.

The macros were also defined immediately after the SnapBuildOnDisk
fields they referred to.

Moving them to the same place in v7.

Please also see the attachment, which implements some of those nits
mentioned above.

While I did not implement all the nits you mentioned, I really appreciate that
you added this attachment, thanks! That's a nice idea and I will try to do the
same for my reviews..

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachments:

v7-0001-Add-contrib-pg_logicalinspect.patchtext/x-diff; charset=us-asciiDownload
From dc62ef7632874a60ff388e68d5c0cb67675c87a5 Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Wed, 14 Aug 2024 08:46:05 +0000
Subject: [PATCH v7] Add contrib/pg_logicalinspect

Provides SQL functions that allow to inspect logical decoding components.

It currently allows to inspect the contents of serialized logical snapshots of
a running database cluster, which is useful for debugging or educational
purposes.
---
 contrib/Makefile                              |   1 +
 contrib/meson.build                           |   1 +
 contrib/pg_logicalinspect/.gitignore          |   4 +
 contrib/pg_logicalinspect/Makefile            |  31 +++
 .../expected/logical_inspect.out              |  52 ++++
 contrib/pg_logicalinspect/logicalinspect.conf |   1 +
 contrib/pg_logicalinspect/meson.build         |  39 +++
 .../pg_logicalinspect--1.0.sql                |  43 +++
 contrib/pg_logicalinspect/pg_logicalinspect.c | 251 ++++++++++++++++++
 .../pg_logicalinspect.control                 |   5 +
 .../specs/logical_inspect.spec                |  34 +++
 doc/src/sgml/contrib.sgml                     |   1 +
 doc/src/sgml/filelist.sgml                    |   1 +
 doc/src/sgml/pglogicalinspect.sgml            | 145 ++++++++++
 src/backend/replication/logical/snapbuild.c   | 190 +------------
 src/include/port/pg_crc32c.h                  |  16 +-
 src/include/replication/snapbuild.h           |   2 +-
 src/include/replication/snapbuild_internal.h  | 204 ++++++++++++++
 18 files changed, 828 insertions(+), 193 deletions(-)
   7.6% contrib/pg_logicalinspect/expected/
   5.7% contrib/pg_logicalinspect/specs/
  32.5% contrib/pg_logicalinspect/
  13.3% doc/src/sgml/
  17.4% src/backend/replication/logical/
   4.1% src/include/port/
  18.9% src/include/replication/

diff --git a/contrib/Makefile b/contrib/Makefile
index abd780f277..952855d9b6 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -32,6 +32,7 @@ SUBDIRS = \
 		passwordcheck	\
 		pg_buffercache	\
 		pg_freespacemap \
+		pg_logicalinspect \
 		pg_prewarm	\
 		pg_stat_statements \
 		pg_surgery	\
diff --git a/contrib/meson.build b/contrib/meson.build
index 14a8906865..159ff41555 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -46,6 +46,7 @@ subdir('passwordcheck')
 subdir('pg_buffercache')
 subdir('pgcrypto')
 subdir('pg_freespacemap')
+subdir('pg_logicalinspect')
 subdir('pg_prewarm')
 subdir('pgrowlocks')
 subdir('pg_stat_statements')
diff --git a/contrib/pg_logicalinspect/.gitignore b/contrib/pg_logicalinspect/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/contrib/pg_logicalinspect/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/contrib/pg_logicalinspect/Makefile b/contrib/pg_logicalinspect/Makefile
new file mode 100644
index 0000000000..55124514d4
--- /dev/null
+++ b/contrib/pg_logicalinspect/Makefile
@@ -0,0 +1,31 @@
+# contrib/pg_logicalinspect/Makefile
+
+MODULE_big = pg_logicalinspect
+OBJS = \
+	$(WIN32RES) \
+	pg_logicalinspect.o
+PGFILEDESC = "pg_logicalinspect - functions to inspect logical decoding components"
+
+EXTENSION = pg_logicalinspect
+DATA = pg_logicalinspect--1.0.sql
+
+EXTRA_INSTALL = contrib/test_decoding
+
+ISOLATION = logical_inspect
+
+ISOLATION_OPTS = --temp-config $(top_srcdir)/contrib/pg_logicalinspect/logicalinspect.conf
+
+# Disabled because these tests require "wal_level=logical", which
+# some installcheck users do not have (e.g. buildfarm clients).
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/pg_logicalinspect
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/pg_logicalinspect/expected/logical_inspect.out b/contrib/pg_logicalinspect/expected/logical_inspect.out
new file mode 100644
index 0000000000..749cd4642d
--- /dev/null
+++ b/contrib/pg_logicalinspect/expected/logical_inspect.out
@@ -0,0 +1,52 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s0_init s0_begin s0_savepoint s0_truncate s1_checkpoint s1_get_changes s0_commit s0_begin s0_insert s1_checkpoint s1_get_changes s0_commit s1_get_changes s1_get_logical_snapshot_info s1_get_logical_snapshot_meta
+step s0_init: SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding');
+?column?
+--------
+init    
+(1 row)
+
+step s0_begin: BEGIN;
+step s0_savepoint: SAVEPOINT sp1;
+step s0_truncate: TRUNCATE tbl1;
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data
+----
+(0 rows)
+
+step s0_commit: COMMIT;
+step s0_begin: BEGIN;
+step s0_insert: INSERT INTO tbl1 VALUES (1);
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                   
+---------------------------------------
+BEGIN                                  
+table public.tbl1: TRUNCATE: (no-flags)
+COMMIT                                 
+(3 rows)
+
+step s0_commit: COMMIT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                                         
+-------------------------------------------------------------
+BEGIN                                                        
+table public.tbl1: INSERT: val1[integer]:1 val2[integer]:null
+COMMIT                                                       
+(3 rows)
+
+step s1_get_logical_snapshot_info: SELECT (pg_get_logical_snapshot_info(f.name::pg_lsn)).state,(pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_xip,1),(pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_xip,1) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f ORDER BY 2;
+state|catchange_count|array_length|committed_count|array_length
+-----+---------------+------------+---------------+------------
+    2|              0|            |              2|           2
+    2|              2|           2|              0|            
+(2 rows)
+
+step s1_get_logical_snapshot_meta: SELECT COUNT((pg_get_logical_snapshot_meta(f.name::pg_lsn))) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f;
+count
+-----
+    2
+(1 row)
+
diff --git a/contrib/pg_logicalinspect/logicalinspect.conf b/contrib/pg_logicalinspect/logicalinspect.conf
new file mode 100644
index 0000000000..e3d257315f
--- /dev/null
+++ b/contrib/pg_logicalinspect/logicalinspect.conf
@@ -0,0 +1 @@
+wal_level = logical
diff --git a/contrib/pg_logicalinspect/meson.build b/contrib/pg_logicalinspect/meson.build
new file mode 100644
index 0000000000..b787dafc9b
--- /dev/null
+++ b/contrib/pg_logicalinspect/meson.build
@@ -0,0 +1,39 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+pg_logicalinspect_sources = files('pg_logicalinspect.c')
+
+if host_system == 'windows'
+  pg_logicalinspect_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+    '--NAME', 'pg_logicalinspect',
+    '--FILEDESC', 'pg_logicalinspect - functions to inspect contents of logical snapshots',])
+endif
+
+pg_logicalinspect = shared_module('pg_logicalinspect',
+  pg_logicalinspect_sources,
+  kwargs: contrib_mod_args + {
+      'dependencies': contrib_mod_args['dependencies'],
+  },
+)
+contrib_targets += pg_logicalinspect
+
+install_data(
+  'pg_logicalinspect.control',
+  'pg_logicalinspect--1.0.sql',
+  kwargs: contrib_data_args,
+)
+
+tests += {
+  'name': 'pg_logicalinspect',
+  'sd': meson.current_source_dir(),
+  'bd': meson.current_build_dir(),
+  'isolation': {
+    'specs': [
+      'logical_inspect',
+    ],
+    'regress_args': [
+      '--temp-config', files('logicalinspect.conf'),
+    ],
+    # see above
+    'runningcheck': false,
+  },
+}
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql b/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
new file mode 100644
index 0000000000..51713ed53e
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
@@ -0,0 +1,43 @@
+/* contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_logicalinspect" to load this file. \quit
+
+--
+-- pg_get_logical_snapshot_meta()
+--
+CREATE FUNCTION pg_get_logical_snapshot_meta(IN in_lsn pg_lsn,
+    OUT magic int4,
+    OUT checksum int4,
+    OUT version int4
+)
+AS 'MODULE_PATHNAME', 'pg_get_logical_snapshot_meta'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_logical_snapshot_meta(pg_lsn) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_logical_snapshot_meta(pg_lsn) TO pg_read_server_files;
+
+--
+-- pg_get_logical_snapshot_info()
+--
+CREATE FUNCTION pg_get_logical_snapshot_info(IN in_lsn pg_lsn,
+    OUT state int2,
+    OUT xmin xid,
+    OUT xmax xid,
+    OUT start_decoding_at pg_lsn,
+    OUT two_phase_at pg_lsn,
+    OUT initial_xmin_horizon xid,
+    OUT building_full_snapshot boolean,
+    OUT in_slot_creation boolean,
+    OUT last_serialized_snapshot pg_lsn,
+    OUT next_phase_at xid,
+    OUT committed_count int8,
+    OUT committed_xip xid[],
+    OUT catchange_count int8,
+    OUT catchange_xip xid[]
+)
+AS 'MODULE_PATHNAME', 'pg_get_logical_snapshot_info'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_logical_snapshot_info(pg_lsn) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_logical_snapshot_info(pg_lsn) TO pg_read_server_files;
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.c b/contrib/pg_logicalinspect/pg_logicalinspect.c
new file mode 100644
index 0000000000..185f36a72c
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.c
@@ -0,0 +1,251 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_logicalinspect.c
+ *		  Functions to inspect contents of PostgreSQL logical snapshots
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ *		  contrib/pg_logicalinspect/pg_logicalinspect.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "funcapi.h"
+#include "replication/snapbuild_internal.h"
+#include "utils/array.h"
+#include "utils/pg_lsn.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_meta);
+PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_info);
+
+/*
+ * NOTE: For any code change or issue fix here, it is highly recommended to
+ * give a thought about doing the same in SnapBuildRestore() as well.
+ */
+
+/*
+ * Validate the logical snapshot file.
+ */
+static void
+ValidateAndRestoreSnapshotFile(XLogRecPtr lsn, SnapBuildOnDisk *ondisk,
+							   const char *path)
+{
+	int			fd;
+	Size		sz;
+	pg_crc32c	checksum;
+	MemoryContext context;
+
+	context = AllocSetContextCreate(CurrentMemoryContext,
+									"logicalsnapshot inspect context",
+									ALLOCSET_DEFAULT_SIZES);
+
+	fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
+
+	if (fd < 0 && errno == ENOENT)
+		ereport(ERROR,
+				errmsg("file \"%s\" does not exist", path));
+	else if (fd < 0)
+		ereport(ERROR,
+				(errcode_for_file_access(),
+				 errmsg("could not open file \"%s\": %m", path)));
+
+	/* ----
+	 * Make sure the snapshot had been stored safely to disk, that's normally
+	 * cheap.
+	 * Note that we do not need PANIC here, nobody will be able to use the
+	 * slot without fsyncing, and saving it won't succeed without an fsync()
+	 * either...
+	 * ----
+	 */
+	fsync_fname(path, false);
+	fsync_fname(PG_LOGICAL_SNAPSHOTS_DIR, true);
+
+	/* read statically sized portion of snapshot */
+	SnapBuildRestoreContents(fd, (char *) ondisk, SnapBuildOnDiskConstantSize, path);
+
+	if (ondisk->magic != SNAPBUILD_MAGIC)
+		ereport(ERROR,
+				(errcode(ERRCODE_DATA_CORRUPTED),
+				 errmsg("snapbuild state file \"%s\" has wrong magic number: %u instead of %u",
+						path, ondisk->magic, SNAPBUILD_MAGIC)));
+
+	if (ondisk->version != SNAPBUILD_VERSION)
+		ereport(ERROR,
+				(errcode(ERRCODE_DATA_CORRUPTED),
+				 errmsg("snapbuild state file \"%s\" has unsupported version: %u instead of %u",
+						path, ondisk->version, SNAPBUILD_VERSION)));
+
+	INIT_CRC32C(checksum);
+	COMP_CRC32C(checksum,
+				((char *) ondisk) + SnapBuildOnDiskNotChecksummedSize,
+				SnapBuildOnDiskConstantSize - SnapBuildOnDiskNotChecksummedSize);
+
+	/* read SnapBuild */
+	SnapBuildRestoreContents(fd, (char *) &ondisk->builder, sizeof(SnapBuild), path);
+	COMP_CRC32C(checksum, &ondisk->builder, sizeof(SnapBuild));
+
+	ondisk->builder.context = context;
+
+	/* restore committed xacts information */
+	if (ondisk->builder.committed.xcnt > 0)
+	{
+		sz = sizeof(TransactionId) * ondisk->builder.committed.xcnt;
+		ondisk->builder.committed.xip = MemoryContextAllocZero(ondisk->builder.context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.committed.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.committed.xip, sz);
+	}
+
+	/* restore catalog modifying xacts information */
+	if (ondisk->builder.catchange.xcnt > 0)
+	{
+		sz = sizeof(TransactionId) * ondisk->builder.catchange.xcnt;
+		ondisk->builder.catchange.xip = MemoryContextAllocZero(ondisk->builder.context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.catchange.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.catchange.xip, sz);
+	}
+
+	if (CloseTransientFile(fd) != 0)
+		ereport(ERROR,
+				(errcode_for_file_access(),
+				 errmsg("could not close file \"%s\": %m", path)));
+
+	FIN_CRC32C(checksum);
+
+	/* verify checksum of what we've read */
+	if (!EQ_CRC32C(checksum, ondisk->checksum))
+		ereport(ERROR,
+				(errcode(ERRCODE_DATA_CORRUPTED),
+				 errmsg("checksum mismatch for snapbuild state file \"%s\": is %u, should be %u",
+						path, checksum, ondisk->checksum)));
+}
+
+/*
+ * Retrieve the logical snapshot file metadata.
+ */
+Datum
+pg_get_logical_snapshot_meta(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_META_COLS 3
+	SnapBuildOnDisk ondisk;
+	XLogRecPtr	lsn;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_META_COLS];
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_META_COLS];
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+	int			i = 0;
+
+	lsn = PG_GETARG_LSN(0);
+
+	sprintf(path, "%s/%X-%X.snap",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			LSN_FORMAT_ARGS(lsn));
+
+	ValidateAndRestoreSnapshotFile(lsn, &ondisk, path);
+
+	/* Build a tuple descriptor for our result type. */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	memset(nulls, 0, sizeof(nulls));
+
+	values[i++] = Int32GetDatum(ondisk.magic);
+	values[i++] = Int32GetDatum(ondisk.checksum);
+	values[i++] = Int32GetDatum(ondisk.version);
+
+	Assert(i == PG_GET_LOGICAL_SNAPSHOT_META_COLS);
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	MemoryContextReset(ondisk.builder.context);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_META_COLS
+}
+
+Datum
+pg_get_logical_snapshot_info(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_INFO_COLS 14
+	SnapBuildOnDisk ondisk;
+	XLogRecPtr	lsn;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS];
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS];
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+	int			i = 0;
+
+	lsn = PG_GETARG_LSN(0);
+
+	sprintf(path, "%s/%X-%X.snap",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			LSN_FORMAT_ARGS(lsn));
+
+	ValidateAndRestoreSnapshotFile(lsn, &ondisk, path);
+
+	/* Build a tuple descriptor for our result type. */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	memset(nulls, 0, sizeof(nulls));
+
+	values[i++] = Int16GetDatum(ondisk.builder.state);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.xmin);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.xmax);
+	values[i++] = LSNGetDatum(ondisk.builder.start_decoding_at);
+	values[i++] = LSNGetDatum(ondisk.builder.two_phase_at);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.initial_xmin_horizon);
+	values[i++] = BoolGetDatum(ondisk.builder.building_full_snapshot);
+	values[i++] = BoolGetDatum(ondisk.builder.in_slot_creation);
+	values[i++] = LSNGetDatum(ondisk.builder.last_serialized_snapshot);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.next_phase_at);
+	values[i++] = Int64GetDatum(ondisk.builder.committed.xcnt);
+
+	if (ondisk.builder.committed.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+		int			narrayelems = 0;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.committed.xcnt * sizeof(Datum));
+
+		for (; narrayelems < ondisk.builder.committed.xcnt; narrayelems++)
+			arrayelems[narrayelems] = Int64GetDatum((int64) ondisk.builder.committed.xip[narrayelems]);
+
+		values[i++] = PointerGetDatum(construct_array_builtin(arrayelems, narrayelems, INT8OID));
+	}
+	else
+		nulls[i++] = true;
+
+	values[i++] = Int64GetDatum(ondisk.builder.catchange.xcnt);
+
+	if (ondisk.builder.catchange.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+		int			narrayelems = 0;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.catchange.xcnt * sizeof(Datum));
+
+		for (; narrayelems < ondisk.builder.catchange.xcnt; narrayelems++)
+			arrayelems[narrayelems] = Int64GetDatum((int64) ondisk.builder.catchange.xip[narrayelems]);
+
+		values[i++] = PointerGetDatum(construct_array_builtin(arrayelems, narrayelems, INT8OID));
+	}
+	else
+		nulls[i++] = true;
+
+	Assert(i == PG_GET_LOGICAL_SNAPSHOT_INFO_COLS);
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	MemoryContextReset(ondisk.builder.context);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_INFO_COLS
+}
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.control b/contrib/pg_logicalinspect/pg_logicalinspect.control
new file mode 100644
index 0000000000..b4a70e57ba
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.control
@@ -0,0 +1,5 @@
+# pg_logicalinspect extension
+comment = 'functions to inspect logical decoding components'
+default_version = '1.0'
+module_pathname = '$libdir/pg_logicalinspect'
+relocatable = true
diff --git a/contrib/pg_logicalinspect/specs/logical_inspect.spec b/contrib/pg_logicalinspect/specs/logical_inspect.spec
new file mode 100644
index 0000000000..e11eb63615
--- /dev/null
+++ b/contrib/pg_logicalinspect/specs/logical_inspect.spec
@@ -0,0 +1,34 @@
+# Test the pg_logicalinspect functions: that needs some permutation to
+# ensure that we are creating multiple logical snapshots and that one of them
+# contains ongoing catalogs changes.
+setup
+{
+    DROP TABLE IF EXISTS tbl1;
+    CREATE TABLE tbl1 (val1 integer, val2 integer);
+	CREATE EXTENSION pg_logicalinspect;
+}
+
+teardown
+{
+    DROP TABLE tbl1;
+    SELECT 'stop' FROM pg_drop_replication_slot('isolation_slot');
+	DROP EXTENSION pg_logicalinspect;
+}
+
+session "s0"
+setup { SET synchronous_commit=on; }
+step "s0_init" { SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding'); }
+step "s0_begin" { BEGIN; }
+step "s0_savepoint" { SAVEPOINT sp1; }
+step "s0_truncate" { TRUNCATE tbl1; }
+step "s0_insert" { INSERT INTO tbl1 VALUES (1); }
+step "s0_commit" { COMMIT; }
+
+session "s1"
+setup { SET synchronous_commit=on; }
+step "s1_checkpoint" { CHECKPOINT; }
+step "s1_get_changes" { SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0'); }
+step "s1_get_logical_snapshot_meta" { SELECT COUNT((pg_get_logical_snapshot_meta(f.name::pg_lsn))) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f; }
+step "s1_get_logical_snapshot_info" { SELECT (pg_get_logical_snapshot_info(f.name::pg_lsn)).state,(pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_xip,1),(pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_xip,1) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f ORDER BY 2; }
+
+permutation "s0_init" "s0_begin" "s0_savepoint" "s0_truncate" "s1_checkpoint" "s1_get_changes" "s0_commit" "s0_begin" "s0_insert" "s1_checkpoint" "s1_get_changes" "s0_commit" "s1_get_changes" "s1_get_logical_snapshot_info" "s1_get_logical_snapshot_meta"
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index 44639a8dca..7c381949a5 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -154,6 +154,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
  &pgbuffercache;
  &pgcrypto;
  &pgfreespacemap;
+ &pglogicalinspect;
  &pgprewarm;
  &pgrowlocks;
  &pgstatstatements;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index a7ff5f8264..66e6dccd4c 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -143,6 +143,7 @@
 <!ENTITY pgbuffercache   SYSTEM "pgbuffercache.sgml">
 <!ENTITY pgcrypto        SYSTEM "pgcrypto.sgml">
 <!ENTITY pgfreespacemap  SYSTEM "pgfreespacemap.sgml">
+<!ENTITY pglogicalinspect  SYSTEM "pglogicalinspect.sgml">
 <!ENTITY pgprewarm       SYSTEM "pgprewarm.sgml">
 <!ENTITY pgrowlocks      SYSTEM "pgrowlocks.sgml">
 <!ENTITY pgstatstatements SYSTEM "pgstatstatements.sgml">
diff --git a/doc/src/sgml/pglogicalinspect.sgml b/doc/src/sgml/pglogicalinspect.sgml
new file mode 100644
index 0000000000..06c627df52
--- /dev/null
+++ b/doc/src/sgml/pglogicalinspect.sgml
@@ -0,0 +1,145 @@
+<!-- doc/src/sgml/pglogicalinspect.sgml -->
+
+<sect1 id="pglogicalinspect" xreflabel="pg_logicalinspect">
+ <title>pg_logicalinspect &mdash; logical decoding components inspection</title>
+
+ <indexterm zone="pglogicalinspect">
+  <primary>pg_logicalinspect</primary>
+ </indexterm>
+
+ <para>
+  The <filename>pg_logicalinspect</filename> module provides SQL functions
+  that allow you to inspect the contents of logical decoding components. It
+  allows the inspection of serialized logical snapshots of a running
+  <productname>PostgreSQL</productname> database cluster, which is useful
+  for debugging or educational purposes.
+ </para>
+
+ <note>
+  <para>
+   The <filename>pg_logicalinspect</filename> functions are called
+   using an LSN argument that can be extracted from the output name of the
+   <function>pg_ls_logicalsnapdir</function>() function.
+  </para>
+ </note>
+
+ <sect2 id="pglogicalinspect-funcs">
+  <title>General Functions</title>
+
+  <variablelist>
+   <varlistentry id="pglogicalinspect-funcs-pg-get-logical-snapshot-meta">
+    <term>
+     <function>pg_get_logical_snapshot_meta(in_lsn pg_lsn) returns record</function>
+    </term>
+
+    <listitem>
+     <para>
+      Gets logical snapshot metadata about a snapshot file that is located in
+      the server's <filename>pg_logical/snapshots</filename> directory.
+      The <replaceable>in_lsn</replaceable> argument can be extracted from the
+      snapshot file name.
+      For example:
+<screen>
+postgres=# SELECT * FROM pg_ls_logicalsnapdir();
+-[ RECORD 1 ]+-----------------------
+name         | 0-40796E18.snap
+size         | 152
+modification | 2024-08-14 16:36:32+00
+
+postgres=# SELECT * FROM pg_get_logical_snapshot_meta('0/40796E18');
+-[ RECORD 1 ]--------
+magic    | 1369563137
+checksum | 1028045905
+version  | 6
+
+postgres=# SELECT (pg_get_logical_snapshot_meta(f.name::pg_lsn)).*
+           FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name
+                 FROM pg_ls_logicalsnapdir()) AS f;
+-[ RECORD 1 ]--------
+magic    | 1369563137
+checksum | 1028045905
+version  | 6
+</screen>
+     </para>
+     <para>
+      If <replaceable>in_lsn</replaceable> does not match a snapshot file, the
+      function raises an error.
+     </para>
+    </listitem>
+   </varlistentry>
+
+   <varlistentry id="pglogicalinspect-funcs-pg-get-logical-snapshot-info">
+    <term>
+     <function>pg_get_logical_snapshot_info(in_lsn pg_lsn) returns record</function>
+    </term>
+
+    <listitem>
+     <para>
+      Gets logical snapshot information about a snapshot file that is located in
+      the <filename>pg_logical/snapshots</filename> directory.
+      The <replaceable>in_lsn</replaceable> argument can be extracted from the
+      snapshot file name.
+      For example:
+<screen>
+postgres=# SELECT * FROM pg_ls_logicalsnapdir();
+-[ RECORD 1 ]+-----------------------
+name         | 0-40796E18.snap
+size         | 152
+modification | 2024-08-14 16:36:32+00
+
+postgres=# SELECT * FROM pg_get_logical_snapshot_info('0/40796E18');
+-[ RECORD 1 ]------------+-----------
+state                    | 2
+xmin                     | 751
+xmax                     | 751
+start_decoding_at        | 0/40796AF8
+two_phase_at             | 0/40796AF8
+initial_xmin_horizon     | 0
+building_full_snapshot   | f
+in_slot_creation         | f
+last_serialized_snapshot | 0/0
+next_phase_at            | 0
+committed_count          | 0
+committed_xip            |
+catchange_count          | 2
+catchange_xip            | {751,752}
+
+postgres=# SELECT (pg_get_logical_snapshot_info(f.name::pg_lsn)).*
+           FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name
+                 FROM pg_ls_logicalsnapdir()) AS f;
+-[ RECORD 1 ]------------+-----------
+state                    | 2
+xmin                     | 751
+xmax                     | 751
+start_decoding_at        | 0/40796AF8
+two_phase_at             | 0/40796AF8
+initial_xmin_horizon     | 0
+building_full_snapshot   | f
+in_slot_creation         | f
+last_serialized_snapshot | 0/0
+next_phase_at            | 0
+committed_count          | 0
+committed_xip            |
+catchange_count          | 2
+catchange_xip            | {751,752}
+</screen>
+     </para>
+     <para>
+      If <replaceable>in_lsn</replaceable> does not match a snapshot file, the
+      function raises an error.
+     </para>
+    </listitem>
+   </varlistentry>
+
+  </variablelist>
+ </sect2>
+
+ <sect2 id="pglogicalinspect-author">
+  <title>Author</title>
+
+  <para>
+   Bertrand Drouvot <email>bertranddrouvot.pg@gmail.com</email>
+  </para>
+ </sect2>
+
+</sect1>
diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c
index 0450f94ba8..bb7def9440 100644
--- a/src/backend/replication/logical/snapbuild.c
+++ b/src/backend/replication/logical/snapbuild.c
@@ -134,6 +134,7 @@
 #include "replication/logical.h"
 #include "replication/reorderbuffer.h"
 #include "replication/snapbuild.h"
+#include "replication/snapbuild_internal.h"
 #include "storage/fd.h"
 #include "storage/lmgr.h"
 #include "storage/proc.h"
@@ -143,146 +144,6 @@
 #include "utils/memutils.h"
 #include "utils/snapmgr.h"
 #include "utils/snapshot.h"
-
-/*
- * This struct contains the current state of the snapshot building
- * machinery. Besides a forward declaration in the header, it is not exposed
- * to the public, so we can easily change its contents.
- */
-struct SnapBuild
-{
-	/* how far are we along building our first full snapshot */
-	SnapBuildState state;
-
-	/* private memory context used to allocate memory for this module. */
-	MemoryContext context;
-
-	/* all transactions < than this have committed/aborted */
-	TransactionId xmin;
-
-	/* all transactions >= than this are uncommitted */
-	TransactionId xmax;
-
-	/*
-	 * Don't replay commits from an LSN < this LSN. This can be set externally
-	 * but it will also be advanced (never retreat) from within snapbuild.c.
-	 */
-	XLogRecPtr	start_decoding_at;
-
-	/*
-	 * LSN at which two-phase decoding was enabled or LSN at which we found a
-	 * consistent point at the time of slot creation.
-	 *
-	 * The prepared transactions, that were skipped because previously
-	 * two-phase was not enabled or are not covered by initial snapshot, need
-	 * to be sent later along with commit prepared and they must be before
-	 * this point.
-	 */
-	XLogRecPtr	two_phase_at;
-
-	/*
-	 * Don't start decoding WAL until the "xl_running_xacts" information
-	 * indicates there are no running xids with an xid smaller than this.
-	 */
-	TransactionId initial_xmin_horizon;
-
-	/* Indicates if we are building full snapshot or just catalog one. */
-	bool		building_full_snapshot;
-
-	/*
-	 * Indicates if we are using the snapshot builder for the creation of a
-	 * logical replication slot. If it's true, the start point for decoding
-	 * changes is not determined yet. So we skip snapshot restores to properly
-	 * find the start point. See SnapBuildFindSnapshot() for details.
-	 */
-	bool		in_slot_creation;
-
-	/*
-	 * Snapshot that's valid to see the catalog state seen at this moment.
-	 */
-	Snapshot	snapshot;
-
-	/*
-	 * LSN of the last location we are sure a snapshot has been serialized to.
-	 */
-	XLogRecPtr	last_serialized_snapshot;
-
-	/*
-	 * The reorderbuffer we need to update with usable snapshots et al.
-	 */
-	ReorderBuffer *reorder;
-
-	/*
-	 * TransactionId at which the next phase of initial snapshot building will
-	 * happen. InvalidTransactionId if not known (i.e. SNAPBUILD_START), or
-	 * when no next phase necessary (SNAPBUILD_CONSISTENT).
-	 */
-	TransactionId next_phase_at;
-
-	/*
-	 * Array of transactions which could have catalog changes that committed
-	 * between xmin and xmax.
-	 */
-	struct
-	{
-		/* number of committed transactions */
-		size_t		xcnt;
-
-		/* available space for committed transactions */
-		size_t		xcnt_space;
-
-		/*
-		 * Until we reach a CONSISTENT state, we record commits of all
-		 * transactions, not just the catalog changing ones. Record when that
-		 * changes so we know we cannot export a snapshot safely anymore.
-		 */
-		bool		includes_all_transactions;
-
-		/*
-		 * Array of committed transactions that have modified the catalog.
-		 *
-		 * As this array is frequently modified we do *not* keep it in
-		 * xidComparator order. Instead we sort the array when building &
-		 * distributing a snapshot.
-		 *
-		 * TODO: It's unclear whether that reasoning has much merit. Every
-		 * time we add something here after becoming consistent will also
-		 * require distributing a snapshot. Storing them sorted would
-		 * potentially also make it easier to purge (but more complicated wrt
-		 * wraparound?). Should be improved if sorting while building the
-		 * snapshot shows up in profiles.
-		 */
-		TransactionId *xip;
-	}			committed;
-
-	/*
-	 * Array of transactions and subtransactions that had modified catalogs
-	 * and were running when the snapshot was serialized.
-	 *
-	 * We normally rely on some WAL record types such as HEAP2_NEW_CID to know
-	 * if the transaction has changed the catalog. But it could happen that
-	 * the logical decoding decodes only the commit record of the transaction
-	 * after restoring the previously serialized snapshot in which case we
-	 * will miss adding the xid to the snapshot and end up looking at the
-	 * catalogs with the wrong snapshot.
-	 *
-	 * Now to avoid the above problem, we serialize the transactions that had
-	 * modified the catalogs and are still running at the time of snapshot
-	 * serialization. We fill this array while restoring the snapshot and then
-	 * refer it while decoding commit to ensure if the xact has modified the
-	 * catalog. We discard this array when all the xids in the list become old
-	 * enough to matter. See SnapBuildPurgeOlderTxn for details.
-	 */
-	struct
-	{
-		/* number of transactions */
-		size_t		xcnt;
-
-		/* This array must be sorted in xidComparator order */
-		TransactionId *xip;
-	}			catchange;
-};
-
 /*
  * Starting a transaction -- which we need to do while exporting a snapshot --
  * removes knowledge about the previously used resowner, so we save it here.
@@ -312,7 +173,6 @@ static void SnapBuildWaitSnapshot(xl_running_xacts *running, TransactionId cutof
 /* serialization functions */
 static void SnapBuildSerialize(SnapBuild *builder, XLogRecPtr lsn);
 static bool SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn);
-static void SnapBuildRestoreContents(int fd, char *dest, Size size, const char *path);
 
 /*
  * Allocate a new snapshot builder.
@@ -1557,48 +1417,6 @@ SnapBuildWaitSnapshot(xl_running_xacts *running, TransactionId cutoff)
 	}
 }
 
-/* -----------------------------------
- * Snapshot serialization support
- * -----------------------------------
- */
-
-/*
- * We store current state of struct SnapBuild on disk in the following manner:
- *
- * struct SnapBuildOnDisk;
- * TransactionId * committed.xcnt; (*not xcnt_space*)
- * TransactionId * catchange.xcnt;
- *
- */
-typedef struct SnapBuildOnDisk
-{
-	/* first part of this struct needs to be version independent */
-
-	/* data not covered by checksum */
-	uint32		magic;
-	pg_crc32c	checksum;
-
-	/* data covered by checksum */
-
-	/* version, in case we want to support pg_upgrade */
-	uint32		version;
-	/* how large is the on disk data, excluding the constant sized part */
-	uint32		length;
-
-	/* version dependent part */
-	SnapBuild	builder;
-
-	/* variable amount of TransactionIds follows */
-} SnapBuildOnDisk;
-
-#define SnapBuildOnDiskConstantSize \
-	offsetof(SnapBuildOnDisk, builder)
-#define SnapBuildOnDiskNotChecksummedSize \
-	offsetof(SnapBuildOnDisk, version)
-
-#define SNAPBUILD_MAGIC 0x51A1E001
-#define SNAPBUILD_VERSION 6
-
 /*
  * Store/Load a snapshot from disk, depending on the snapshot builder's state.
  *
@@ -1859,6 +1677,10 @@ out:
 /*
  * Restore a snapshot into 'builder' if previously one has been stored at the
  * location indicated by 'lsn'. Returns true if successful, false otherwise.
+ *
+ * NOTE: For any code change or issue fix here, it is highly recommended to
+ * give a thought about doing the same in pg_logicalinspect contrib module
+ * as well.
  */
 static bool
 SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
@@ -2033,7 +1855,7 @@ snapshot_not_interesting:
 /*
  * Read the contents of the serialized snapshot to 'dest'.
  */
-static void
+void
 SnapBuildRestoreContents(int fd, char *dest, Size size, const char *path)
 {
 	int			readBytes;
diff --git a/src/include/port/pg_crc32c.h b/src/include/port/pg_crc32c.h
index 63c8e3a00b..cfc8c07944 100644
--- a/src/include/port/pg_crc32c.h
+++ b/src/include/port/pg_crc32c.h
@@ -47,7 +47,7 @@ typedef uint32 pg_crc32c;
 	((crc) = pg_comp_crc32c_sse42((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
 
 #elif defined(USE_ARMV8_CRC32C)
 /* Use ARMv8 CRC Extension instructions. */
@@ -56,7 +56,7 @@ extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t le
 	((crc) = pg_comp_crc32c_armv8((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
 
 #elif defined(USE_LOONGARCH_CRC32C)
 /* Use LoongArch CRCC instructions. */
@@ -65,7 +65,7 @@ extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t le
 	((crc) = pg_comp_crc32c_loongarch((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len);
 
 #elif defined(USE_SSE42_CRC32C_WITH_RUNTIME_CHECK) || defined(USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK)
 
@@ -77,14 +77,14 @@ extern pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_
 	((crc) = pg_comp_crc32c((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
-extern pg_crc32c (*pg_comp_crc32c) (pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c (*pg_comp_crc32c) (pg_crc32c crc, const void *data, size_t len);
 
 #ifdef USE_SSE42_CRC32C_WITH_RUNTIME_CHECK
-extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
 #endif
 #ifdef USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK
-extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
 #endif
 
 #else
@@ -103,7 +103,7 @@ extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t le
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 #endif
 
-extern pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
 
 #endif
 
diff --git a/src/include/replication/snapbuild.h b/src/include/replication/snapbuild.h
index caa5113ff8..dbb4bc2f4b 100644
--- a/src/include/replication/snapbuild.h
+++ b/src/include/replication/snapbuild.h
@@ -46,7 +46,7 @@ typedef enum
 	SNAPBUILD_CONSISTENT = 2,
 } SnapBuildState;
 
-/* forward declare so we don't have to expose the struct to the public */
+/* forward declare so we don't have to include snapbuild_internal.h */
 struct SnapBuild;
 typedef struct SnapBuild SnapBuild;
 
diff --git a/src/include/replication/snapbuild_internal.h b/src/include/replication/snapbuild_internal.h
new file mode 100644
index 0000000000..0e47ddfcb4
--- /dev/null
+++ b/src/include/replication/snapbuild_internal.h
@@ -0,0 +1,204 @@
+/*-------------------------------------------------------------------------
+ *
+ * snapbuild_internal.h
+ *    This file contains declarations for logical decoding utility
+ *    functions for internal use.
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * src/include/replication/snapbuild_internal.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef SNAPBUILD_INTERNAL_H
+#define SNAPBUILD_INTERNAL_H
+
+#include "port/pg_crc32c.h"
+#include "replication/reorderbuffer.h"
+#include "replication/snapbuild.h"
+
+/*
+ * This struct contains the current state of the snapshot building
+ * machinery. It is exposed to the public, so pay attention when changing its
+ * contents.
+ */
+typedef struct SnapBuild
+{
+	/* how far are we along building our first full snapshot */
+	SnapBuildState state;
+
+	/* private memory context used to allocate memory for this module. */
+	MemoryContext context;
+
+	/* all transactions < than this have committed/aborted */
+	TransactionId xmin;
+
+	/* all transactions >= than this are uncommitted */
+	TransactionId xmax;
+
+	/*
+	 * Don't replay commits from an LSN < this LSN. This can be set externally
+	 * but it will also be advanced (never retreat) from within snapbuild.c.
+	 */
+	XLogRecPtr	start_decoding_at;
+
+	/*
+	 * LSN at which two-phase decoding was enabled or LSN at which we found a
+	 * consistent point at the time of slot creation.
+	 *
+	 * The prepared transactions, that were skipped because previously
+	 * two-phase was not enabled or are not covered by initial snapshot, need
+	 * to be sent later along with commit prepared and they must be before
+	 * this point.
+	 */
+	XLogRecPtr	two_phase_at;
+
+	/*
+	 * Don't start decoding WAL until the "xl_running_xacts" information
+	 * indicates there are no running xids with an xid smaller than this.
+	 */
+	TransactionId initial_xmin_horizon;
+
+	/* Indicates if we are building full snapshot or just catalog one. */
+	bool		building_full_snapshot;
+
+	/*
+	 * Indicates if we are using the snapshot builder for the creation of a
+	 * logical replication slot. If it's true, the start point for decoding
+	 * changes is not determined yet. So we skip snapshot restores to properly
+	 * find the start point. See SnapBuildFindSnapshot() for details.
+	 */
+	bool		in_slot_creation;
+
+	/*
+	 * Snapshot that's valid to see the catalog state seen at this moment.
+	 */
+	Snapshot	snapshot;
+
+	/*
+	 * LSN of the last location we are sure a snapshot has been serialized to.
+	 */
+	XLogRecPtr	last_serialized_snapshot;
+
+	/*
+	 * The reorderbuffer we need to update with usable snapshots et al.
+	 */
+	ReorderBuffer *reorder;
+
+	/*
+	 * TransactionId at which the next phase of initial snapshot building will
+	 * happen. InvalidTransactionId if not known (i.e. SNAPBUILD_START), or
+	 * when no next phase necessary (SNAPBUILD_CONSISTENT).
+	 */
+	TransactionId next_phase_at;
+
+	/*
+	 * Array of transactions which could have catalog changes that committed
+	 * between xmin and xmax.
+	 */
+	struct
+	{
+		/* number of committed transactions */
+		size_t		xcnt;
+
+		/* available space for committed transactions */
+		size_t		xcnt_space;
+
+		/*
+		 * Until we reach a CONSISTENT state, we record commits of all
+		 * transactions, not just the catalog changing ones. Record when that
+		 * changes so we know we cannot export a snapshot safely anymore.
+		 */
+		bool		includes_all_transactions;
+
+		/*
+		 * Array of committed transactions that have modified the catalog.
+		 *
+		 * As this array is frequently modified we do *not* keep it in
+		 * xidComparator order. Instead we sort the array when building &
+		 * distributing a snapshot.
+		 *
+		 * TODO: It's unclear whether that reasoning has much merit. Every
+		 * time we add something here after becoming consistent will also
+		 * require distributing a snapshot. Storing them sorted would
+		 * potentially also make it easier to purge (but more complicated wrt
+		 * wraparound?). Should be improved if sorting while building the
+		 * snapshot shows up in profiles.
+		 */
+		TransactionId *xip;
+	}			committed;
+
+	/*
+	 * Array of transactions and subtransactions that had modified catalogs
+	 * and were running when the snapshot was serialized.
+	 *
+	 * We normally rely on some WAL record types such as HEAP2_NEW_CID to know
+	 * if the transaction has changed the catalog. But it could happen that
+	 * the logical decoding decodes only the commit record of the transaction
+	 * after restoring the previously serialized snapshot in which case we
+	 * will miss adding the xid to the snapshot and end up looking at the
+	 * catalogs with the wrong snapshot.
+	 *
+	 * Now to avoid the above problem, we serialize the transactions that had
+	 * modified the catalogs and are still running at the time of snapshot
+	 * serialization. We fill this array while restoring the snapshot and then
+	 * refer it while decoding commit to ensure if the xact has modified the
+	 * catalog. We discard this array when all the xids in the list become old
+	 * enough to matter. See SnapBuildPurgeOlderTxn for details.
+	 */
+	struct
+	{
+		/* number of transactions */
+		size_t		xcnt;
+
+		/* This array must be sorted in xidComparator order */
+		TransactionId *xip;
+	}			catchange;
+} SnapBuild;
+
+/* -----------------------------------
+ * Snapshot serialization support
+ * -----------------------------------
+ */
+
+/*
+ * We store current state of struct SnapBuild on disk in the following manner:
+ *
+ * struct SnapBuildOnDisk;
+ * TransactionId * committed.xcnt; (*not xcnt_space*)
+ * TransactionId * catchange.xcnt;
+ *
+ */
+typedef struct SnapBuildOnDisk
+{
+	/* first part of this struct needs to be version independent */
+
+	/* data not covered by checksum */
+	uint32		magic;
+	pg_crc32c	checksum;
+
+	/* data covered by checksum */
+
+	/* version, in case we want to support pg_upgrade */
+	uint32		version;
+	/* how large is the on disk data, excluding the constant sized part */
+	uint32		length;
+
+	/* version dependent part */
+	SnapBuild	builder;
+
+	/* variable amount of TransactionIds follows */
+} SnapBuildOnDisk;
+
+#define SnapBuildOnDiskConstantSize \
+	offsetof(SnapBuildOnDisk, builder)
+#define SnapBuildOnDiskNotChecksummedSize \
+	offsetof(SnapBuildOnDisk, version)
+
+#define SNAPBUILD_MAGIC 0x51A1E001
+#define SNAPBUILD_VERSION 6
+
+extern void SnapBuildRestoreContents(int fd, char *dest, Size size, const char *path);
+
+#endif							/* SNAPBUILD_INTERNAL_H */
-- 
2.34.1

#27Peter Smith
smithpb2250@gmail.com
In reply to: Bertrand Drouvot (#26)
1 attachment(s)
Re: Add contrib/pg_logicalsnapinspect

Thanks for the updated patch.

Here are a few more trivial comments for the patch v7-0001.

======

1.
Should the extension descriptions all be identical?

I noticed small variations:

contrib/pg_logicalinspect/Makefile
+PGFILEDESC = "pg_logicalinspect - functions to inspect logical
decoding components"

contrib/pg_logicalinspect/meson.build
+ '--FILEDESC', 'pg_logicalinspect - functions to inspect contents
of logical snapshots',])

contrib/pg_logicalinspect/pg_logicalinspect.control
+comment = 'functions to inspect logical decoding components'

======
.../expected/logical_inspect.out

2
+step s1_get_logical_snapshot_info: SELECT
(pg_get_logical_snapshot_info(f.name::pg_lsn)).state,(pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_xip,1),(pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_xip,1)
FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM
pg_ls_logicalsnapdir()) AS f ORDER BY 2;
+state|catchange_count|array_length|committed_count|array_length
+-----+---------------+------------+---------------+------------
+    2|              0|            |              2|           2
+    2|              2|           2|              0|
+(2 rows)
+

2a.
Would it be better to rearrange those columns so 'committed' stuff
comes before 'catchange' stuff, to make this table order consistent
with the structure/code?

~

2b.
Maybe those 2 'array_length' columns could have aliases to uniquely
identify them?
e.g. 'catchange_array_length' and 'committed_array_length'.

======
contrib/pg_logicalinspect/pg_logicalinspect.c

3.
+/*
+ * Validate the logical snapshot file.
+ */
+static void
+ValidateAndRestoreSnapshotFile(XLogRecPtr lsn, SnapBuildOnDisk *ondisk,
+    const char *path)

Since the name was updated then should the function comment also be
updated to include something like the SnapBuildRestoreContents
function comment? e.g. "Validate the logical snapshot file, and read
the contents of the serialized snapshot to 'ondisk'."

~~~

pg_get_logical_snapshot_info:

4.
nit - Add/remove some blank lines to help visually associate the array
counts with their arrays.

======
.../specs/logical_inspect.spec

5.
+setup
+{
+    DROP TABLE IF EXISTS tbl1;
+    CREATE TABLE tbl1 (val1 integer, val2 integer);
+ CREATE EXTENSION pg_logicalinspect;
+}
+
+teardown
+{
+    DROP TABLE tbl1;
+    SELECT 'stop' FROM pg_drop_replication_slot('isolation_slot');
+ DROP EXTENSION pg_logicalinspect;
+}

Different indentation for the CREATE/DROP EXTENSION?

======

The attached file shows the whitespace nit (#4)

======
Kind Regards,
Peter Smith.
Fujitsu Australia

Attachments:

PS_NITPICKS_v7.txttext/plain; charset=US-ASCII; name=PS_NITPICKS_v7.txtDownload
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.c b/contrib/pg_logicalinspect/pg_logicalinspect.c
index 185f36a..308c653 100644
--- a/contrib/pg_logicalinspect/pg_logicalinspect.c
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.c
@@ -205,8 +205,8 @@ pg_get_logical_snapshot_info(PG_FUNCTION_ARGS)
 	values[i++] = BoolGetDatum(ondisk.builder.in_slot_creation);
 	values[i++] = LSNGetDatum(ondisk.builder.last_serialized_snapshot);
 	values[i++] = TransactionIdGetDatum(ondisk.builder.next_phase_at);
-	values[i++] = Int64GetDatum(ondisk.builder.committed.xcnt);
 
+	values[i++] = Int64GetDatum(ondisk.builder.committed.xcnt);
 	if (ondisk.builder.committed.xcnt > 0)
 	{
 		Datum	   *arrayelems;
@@ -223,7 +223,6 @@ pg_get_logical_snapshot_info(PG_FUNCTION_ARGS)
 		nulls[i++] = true;
 
 	values[i++] = Int64GetDatum(ondisk.builder.catchange.xcnt);
-
 	if (ondisk.builder.catchange.xcnt > 0)
 	{
 		Datum	   *arrayelems;
#28Amit Kapila
amit.kapila16@gmail.com
In reply to: Bertrand Drouvot (#21)
Re: Add contrib/pg_logicalsnapinspect

On Tue, Sep 17, 2024 at 12:44 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

On Tue, Sep 17, 2024 at 10:18:35AM +0530, shveta malik wrote:

Thanks for addressing the comments. I have not started reviewing v4
yet, but here are few more comments on v3:

4)
Most of the output columns in pg_get_logical_snapshot_info() look
self-explanatory except 'state'. Should we have meaningful 'text' here
corresponding to SnapBuildState? Similar to what we do for
'invalidation_reason' in pg_replication_slots. (SlotInvalidationCauses
for ReplicationSlotInvalidationCause)

Yeah we could. I was not sure about that (and that was my first remark in [1])
, as the module is mainly for debugging purpose, I was thinking that the one
using it could refer to "snapbuild.h". Let's see what others think.

Displaying the 'text' for the state column makes it easy to
understand. So, +1 for doing it that way.

--
With Regards,
Amit Kapila.

#29shveta malik
shveta.malik@gmail.com
In reply to: Bertrand Drouvot (#26)
Re: Add contrib/pg_logicalsnapinspect

On Thu, Sep 19, 2024 at 12:17 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Hi,

Thanks for the review!

Thanks for the patch.

Should we include in the document who can execute these functions and
the required access permissions, similar to how it's done for
pgwalinspect, pg_ls_logicalmapdir(), and other such functions?

thanks
Shveta

#30Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: Peter Smith (#27)
1 attachment(s)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Thu, Sep 19, 2024 at 10:08:19AM +1000, Peter Smith wrote:

Thanks for the updated patch.

======
.../expected/logical_inspect.out

2
+step s1_get_logical_snapshot_info: SELECT
(pg_get_logical_snapshot_info(f.name::pg_lsn)).state,(pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_xip,1),(pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_xip,1)
FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM
pg_ls_logicalsnapdir()) AS f ORDER BY 2;
+state|catchange_count|array_length|committed_count|array_length
+-----+---------------+------------+---------------+------------
+    2|              0|            |              2|           2
+    2|              2|           2|              0|
+(2 rows)
+

2a.
Would it be better to rearrange those columns so 'committed' stuff
comes before 'catchange' stuff, to make this table order consistent
with the structure/code?

I'm not sure that's a good idea to create a "dependency" between the test output
and the code. I think that could be hard to "ensure" in the mid-long term.

Please find attached v8, that:

- takes care of your comments (except the one above)
- takes care of Shveta comment [1]/messages/by-id/CAJpy0uDJ65QHUZfww7n6TBZAGp-SP74P5U3fUorV+=baaRu6Dw@mail.gmail.com
- displays the 'text' for the state column (as confirmed by Amit [2]/messages/by-id/CAA4eK1JgW1o9wOTwgRJ9+bQkYcr3iRWAQHoL9eBC+rmoQoHZ=Q@mail.gmail.com): Note that
the enum -> text mapping is done in pg_logicalinspect.c and a comment has been
added in snapbuild.h (near the SnapBuildState definition). I thought it makes
more sense to do it that way instead of implementing the enum -> text mapping
in snapbuild.h (as the mapping is only used by the module).

[1]: /messages/by-id/CAJpy0uDJ65QHUZfww7n6TBZAGp-SP74P5U3fUorV+=baaRu6Dw@mail.gmail.com
[2]: /messages/by-id/CAA4eK1JgW1o9wOTwgRJ9+bQkYcr3iRWAQHoL9eBC+rmoQoHZ=Q@mail.gmail.com

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachments:

v8-0001-Add-contrib-pg_logicalinspect.patchtext/x-diff; charset=us-asciiDownload
From 42204dbf1b073e2ef0f7476979d5ebadb5201a5e Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Wed, 14 Aug 2024 08:46:05 +0000
Subject: [PATCH v8] Add contrib/pg_logicalinspect

Provides SQL functions that allow to inspect logical decoding components.

It currently allows to inspect the contents of serialized logical snapshots of
a running database cluster, which is useful for debugging or educational
purposes.
---
 contrib/Makefile                              |   1 +
 contrib/meson.build                           |   1 +
 contrib/pg_logicalinspect/.gitignore          |   4 +
 contrib/pg_logicalinspect/Makefile            |  31 ++
 .../expected/logical_inspect.out              |  52 ++++
 contrib/pg_logicalinspect/logicalinspect.conf |   1 +
 contrib/pg_logicalinspect/meson.build         |  39 +++
 .../pg_logicalinspect--1.0.sql                |  43 +++
 contrib/pg_logicalinspect/pg_logicalinspect.c | 265 ++++++++++++++++++
 .../pg_logicalinspect.control                 |   5 +
 .../specs/logical_inspect.spec                |  34 +++
 doc/src/sgml/contrib.sgml                     |   1 +
 doc/src/sgml/filelist.sgml                    |   1 +
 doc/src/sgml/pglogicalinspect.sgml            | 151 ++++++++++
 src/backend/replication/logical/snapbuild.c   | 190 +------------
 src/include/port/pg_crc32c.h                  |  16 +-
 src/include/replication/snapbuild.h           |   6 +-
 src/include/replication/snapbuild_internal.h  | 204 ++++++++++++++
 18 files changed, 852 insertions(+), 193 deletions(-)
   7.8% contrib/pg_logicalinspect/expected/
   5.7% contrib/pg_logicalinspect/specs/
  32.9% contrib/pg_logicalinspect/
  13.6% doc/src/sgml/
  16.9% src/backend/replication/logical/
   4.0% src/include/port/
  18.7% src/include/replication/

diff --git a/contrib/Makefile b/contrib/Makefile
index abd780f277..952855d9b6 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -32,6 +32,7 @@ SUBDIRS = \
 		passwordcheck	\
 		pg_buffercache	\
 		pg_freespacemap \
+		pg_logicalinspect \
 		pg_prewarm	\
 		pg_stat_statements \
 		pg_surgery	\
diff --git a/contrib/meson.build b/contrib/meson.build
index 14a8906865..159ff41555 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -46,6 +46,7 @@ subdir('passwordcheck')
 subdir('pg_buffercache')
 subdir('pgcrypto')
 subdir('pg_freespacemap')
+subdir('pg_logicalinspect')
 subdir('pg_prewarm')
 subdir('pgrowlocks')
 subdir('pg_stat_statements')
diff --git a/contrib/pg_logicalinspect/.gitignore b/contrib/pg_logicalinspect/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/contrib/pg_logicalinspect/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/contrib/pg_logicalinspect/Makefile b/contrib/pg_logicalinspect/Makefile
new file mode 100644
index 0000000000..55124514d4
--- /dev/null
+++ b/contrib/pg_logicalinspect/Makefile
@@ -0,0 +1,31 @@
+# contrib/pg_logicalinspect/Makefile
+
+MODULE_big = pg_logicalinspect
+OBJS = \
+	$(WIN32RES) \
+	pg_logicalinspect.o
+PGFILEDESC = "pg_logicalinspect - functions to inspect logical decoding components"
+
+EXTENSION = pg_logicalinspect
+DATA = pg_logicalinspect--1.0.sql
+
+EXTRA_INSTALL = contrib/test_decoding
+
+ISOLATION = logical_inspect
+
+ISOLATION_OPTS = --temp-config $(top_srcdir)/contrib/pg_logicalinspect/logicalinspect.conf
+
+# Disabled because these tests require "wal_level=logical", which
+# some installcheck users do not have (e.g. buildfarm clients).
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/pg_logicalinspect
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/pg_logicalinspect/expected/logical_inspect.out b/contrib/pg_logicalinspect/expected/logical_inspect.out
new file mode 100644
index 0000000000..08de273197
--- /dev/null
+++ b/contrib/pg_logicalinspect/expected/logical_inspect.out
@@ -0,0 +1,52 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s0_init s0_begin s0_savepoint s0_truncate s1_checkpoint s1_get_changes s0_commit s0_begin s0_insert s1_checkpoint s1_get_changes s0_commit s1_get_changes s1_get_logical_snapshot_info s1_get_logical_snapshot_meta
+step s0_init: SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding');
+?column?
+--------
+init    
+(1 row)
+
+step s0_begin: BEGIN;
+step s0_savepoint: SAVEPOINT sp1;
+step s0_truncate: TRUNCATE tbl1;
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data
+----
+(0 rows)
+
+step s0_commit: COMMIT;
+step s0_begin: BEGIN;
+step s0_insert: INSERT INTO tbl1 VALUES (1);
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                   
+---------------------------------------
+BEGIN                                  
+table public.tbl1: TRUNCATE: (no-flags)
+COMMIT                                 
+(3 rows)
+
+step s0_commit: COMMIT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                                         
+-------------------------------------------------------------
+BEGIN                                                        
+table public.tbl1: INSERT: val1[integer]:1 val2[integer]:null
+COMMIT                                                       
+(3 rows)
+
+step s1_get_logical_snapshot_info: SELECT (pg_get_logical_snapshot_info(f.name::pg_lsn)).state,(pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_xip,1) AS catchange_array_length,(pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_xip,1) AS committed_array_length FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f ORDER BY 2;
+state     |catchange_count|catchange_array_length|committed_count|committed_array_length
+----------+---------------+----------------------+---------------+----------------------
+consistent|              0|                      |              2|                     2
+consistent|              2|                     2|              0|                      
+(2 rows)
+
+step s1_get_logical_snapshot_meta: SELECT COUNT((pg_get_logical_snapshot_meta(f.name::pg_lsn))) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f;
+count
+-----
+    2
+(1 row)
+
diff --git a/contrib/pg_logicalinspect/logicalinspect.conf b/contrib/pg_logicalinspect/logicalinspect.conf
new file mode 100644
index 0000000000..e3d257315f
--- /dev/null
+++ b/contrib/pg_logicalinspect/logicalinspect.conf
@@ -0,0 +1 @@
+wal_level = logical
diff --git a/contrib/pg_logicalinspect/meson.build b/contrib/pg_logicalinspect/meson.build
new file mode 100644
index 0000000000..3ec635509b
--- /dev/null
+++ b/contrib/pg_logicalinspect/meson.build
@@ -0,0 +1,39 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+pg_logicalinspect_sources = files('pg_logicalinspect.c')
+
+if host_system == 'windows'
+  pg_logicalinspect_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+    '--NAME', 'pg_logicalinspect',
+    '--FILEDESC', 'pg_logicalinspect - functions to inspect logical decoding components',])
+endif
+
+pg_logicalinspect = shared_module('pg_logicalinspect',
+  pg_logicalinspect_sources,
+  kwargs: contrib_mod_args + {
+      'dependencies': contrib_mod_args['dependencies'],
+  },
+)
+contrib_targets += pg_logicalinspect
+
+install_data(
+  'pg_logicalinspect.control',
+  'pg_logicalinspect--1.0.sql',
+  kwargs: contrib_data_args,
+)
+
+tests += {
+  'name': 'pg_logicalinspect',
+  'sd': meson.current_source_dir(),
+  'bd': meson.current_build_dir(),
+  'isolation': {
+    'specs': [
+      'logical_inspect',
+    ],
+    'regress_args': [
+      '--temp-config', files('logicalinspect.conf'),
+    ],
+    # see above
+    'runningcheck': false,
+  },
+}
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql b/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
new file mode 100644
index 0000000000..760b0d53f4
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
@@ -0,0 +1,43 @@
+/* contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_logicalinspect" to load this file. \quit
+
+--
+-- pg_get_logical_snapshot_meta()
+--
+CREATE FUNCTION pg_get_logical_snapshot_meta(IN in_lsn pg_lsn,
+    OUT magic int4,
+    OUT checksum int4,
+    OUT version int4
+)
+AS 'MODULE_PATHNAME', 'pg_get_logical_snapshot_meta'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_logical_snapshot_meta(pg_lsn) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_logical_snapshot_meta(pg_lsn) TO pg_read_server_files;
+
+--
+-- pg_get_logical_snapshot_info()
+--
+CREATE FUNCTION pg_get_logical_snapshot_info(IN in_lsn pg_lsn,
+    OUT state text,
+    OUT xmin xid,
+    OUT xmax xid,
+    OUT start_decoding_at pg_lsn,
+    OUT two_phase_at pg_lsn,
+    OUT initial_xmin_horizon xid,
+    OUT building_full_snapshot boolean,
+    OUT in_slot_creation boolean,
+    OUT last_serialized_snapshot pg_lsn,
+    OUT next_phase_at xid,
+    OUT committed_count int8,
+    OUT committed_xip xid[],
+    OUT catchange_count int8,
+    OUT catchange_xip xid[]
+)
+AS 'MODULE_PATHNAME', 'pg_get_logical_snapshot_info'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_logical_snapshot_info(pg_lsn) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_logical_snapshot_info(pg_lsn) TO pg_read_server_files;
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.c b/contrib/pg_logicalinspect/pg_logicalinspect.c
new file mode 100644
index 0000000000..100b82fd46
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.c
@@ -0,0 +1,265 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_logicalinspect.c
+ *		  Functions to inspect contents of PostgreSQL logical snapshots
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ *		  contrib/pg_logicalinspect/pg_logicalinspect.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "funcapi.h"
+#include "replication/snapbuild_internal.h"
+#include "utils/array.h"
+#include "utils/builtins.h"
+#include "utils/pg_lsn.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_meta);
+PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_info);
+
+/*
+ * Lookup table for SnapBuildState.
+ */
+
+#define SNAPBUILD_STATE_INCR 1
+
+static const char *const SnapBuildStateDesc[] = {
+	[SNAPBUILD_START + SNAPBUILD_STATE_INCR] = "start",
+	[SNAPBUILD_BUILDING_SNAPSHOT + SNAPBUILD_STATE_INCR] = "building",
+	[SNAPBUILD_FULL_SNAPSHOT + SNAPBUILD_STATE_INCR] = "full",
+	[SNAPBUILD_CONSISTENT + SNAPBUILD_STATE_INCR] = "consistent",
+};
+
+/*
+ * NOTE: For any code change or issue fix here, it is highly recommended to
+ * give a thought about doing the same in SnapBuildRestore() as well.
+ */
+
+/*
+ * Validate the logical snapshot file and read its contents to 'ondisk'.
+ */
+static void
+ValidateAndRestoreSnapshotFile(XLogRecPtr lsn, SnapBuildOnDisk *ondisk,
+							   const char *path)
+{
+	int			fd;
+	Size		sz;
+	pg_crc32c	checksum;
+	MemoryContext context;
+
+	context = AllocSetContextCreate(CurrentMemoryContext,
+									"logicalsnapshot inspect context",
+									ALLOCSET_DEFAULT_SIZES);
+
+	fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
+
+	if (fd < 0 && errno == ENOENT)
+		ereport(ERROR,
+				errmsg("file \"%s\" does not exist", path));
+	else if (fd < 0)
+		ereport(ERROR,
+				(errcode_for_file_access(),
+				 errmsg("could not open file \"%s\": %m", path)));
+
+	/* ----
+	 * Make sure the snapshot had been stored safely to disk, that's normally
+	 * cheap.
+	 * Note that we do not need PANIC here, nobody will be able to use the
+	 * slot without fsyncing, and saving it won't succeed without an fsync()
+	 * either...
+	 * ----
+	 */
+	fsync_fname(path, false);
+	fsync_fname(PG_LOGICAL_SNAPSHOTS_DIR, true);
+
+	/* read statically sized portion of snapshot */
+	SnapBuildRestoreContents(fd, (char *) ondisk, SnapBuildOnDiskConstantSize, path);
+
+	if (ondisk->magic != SNAPBUILD_MAGIC)
+		ereport(ERROR,
+				(errcode(ERRCODE_DATA_CORRUPTED),
+				 errmsg("snapbuild state file \"%s\" has wrong magic number: %u instead of %u",
+						path, ondisk->magic, SNAPBUILD_MAGIC)));
+
+	if (ondisk->version != SNAPBUILD_VERSION)
+		ereport(ERROR,
+				(errcode(ERRCODE_DATA_CORRUPTED),
+				 errmsg("snapbuild state file \"%s\" has unsupported version: %u instead of %u",
+						path, ondisk->version, SNAPBUILD_VERSION)));
+
+	INIT_CRC32C(checksum);
+	COMP_CRC32C(checksum,
+				((char *) ondisk) + SnapBuildOnDiskNotChecksummedSize,
+				SnapBuildOnDiskConstantSize - SnapBuildOnDiskNotChecksummedSize);
+
+	/* read SnapBuild */
+	SnapBuildRestoreContents(fd, (char *) &ondisk->builder, sizeof(SnapBuild), path);
+	COMP_CRC32C(checksum, &ondisk->builder, sizeof(SnapBuild));
+
+	ondisk->builder.context = context;
+
+	/* restore committed xacts information */
+	if (ondisk->builder.committed.xcnt > 0)
+	{
+		sz = sizeof(TransactionId) * ondisk->builder.committed.xcnt;
+		ondisk->builder.committed.xip = MemoryContextAllocZero(ondisk->builder.context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.committed.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.committed.xip, sz);
+	}
+
+	/* restore catalog modifying xacts information */
+	if (ondisk->builder.catchange.xcnt > 0)
+	{
+		sz = sizeof(TransactionId) * ondisk->builder.catchange.xcnt;
+		ondisk->builder.catchange.xip = MemoryContextAllocZero(ondisk->builder.context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.catchange.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.catchange.xip, sz);
+	}
+
+	if (CloseTransientFile(fd) != 0)
+		ereport(ERROR,
+				(errcode_for_file_access(),
+				 errmsg("could not close file \"%s\": %m", path)));
+
+	FIN_CRC32C(checksum);
+
+	/* verify checksum of what we've read */
+	if (!EQ_CRC32C(checksum, ondisk->checksum))
+		ereport(ERROR,
+				(errcode(ERRCODE_DATA_CORRUPTED),
+				 errmsg("checksum mismatch for snapbuild state file \"%s\": is %u, should be %u",
+						path, checksum, ondisk->checksum)));
+}
+
+/*
+ * Retrieve the logical snapshot file metadata.
+ */
+Datum
+pg_get_logical_snapshot_meta(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_META_COLS 3
+	SnapBuildOnDisk ondisk;
+	XLogRecPtr	lsn;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_META_COLS];
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_META_COLS];
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+	int			i = 0;
+
+	lsn = PG_GETARG_LSN(0);
+
+	sprintf(path, "%s/%X-%X.snap",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			LSN_FORMAT_ARGS(lsn));
+
+	ValidateAndRestoreSnapshotFile(lsn, &ondisk, path);
+
+	/* Build a tuple descriptor for our result type. */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	memset(nulls, 0, sizeof(nulls));
+
+	values[i++] = Int32GetDatum(ondisk.magic);
+	values[i++] = Int32GetDatum(ondisk.checksum);
+	values[i++] = Int32GetDatum(ondisk.version);
+
+	Assert(i == PG_GET_LOGICAL_SNAPSHOT_META_COLS);
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	MemoryContextReset(ondisk.builder.context);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_META_COLS
+}
+
+Datum
+pg_get_logical_snapshot_info(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_INFO_COLS 14
+	SnapBuildOnDisk ondisk;
+	XLogRecPtr	lsn;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS];
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS];
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+	int			i = 0;
+
+	lsn = PG_GETARG_LSN(0);
+
+	sprintf(path, "%s/%X-%X.snap",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			LSN_FORMAT_ARGS(lsn));
+
+	ValidateAndRestoreSnapshotFile(lsn, &ondisk, path);
+
+	/* Build a tuple descriptor for our result type. */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	memset(nulls, 0, sizeof(nulls));
+
+	values[i++] = CStringGetTextDatum(SnapBuildStateDesc[ondisk.builder.state +
+														 SNAPBUILD_STATE_INCR]);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.xmin);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.xmax);
+	values[i++] = LSNGetDatum(ondisk.builder.start_decoding_at);
+	values[i++] = LSNGetDatum(ondisk.builder.two_phase_at);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.initial_xmin_horizon);
+	values[i++] = BoolGetDatum(ondisk.builder.building_full_snapshot);
+	values[i++] = BoolGetDatum(ondisk.builder.in_slot_creation);
+	values[i++] = LSNGetDatum(ondisk.builder.last_serialized_snapshot);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.next_phase_at);
+
+	values[i++] = Int64GetDatum(ondisk.builder.committed.xcnt);
+	if (ondisk.builder.committed.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+		int			narrayelems = 0;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.committed.xcnt * sizeof(Datum));
+
+		for (; narrayelems < ondisk.builder.committed.xcnt; narrayelems++)
+			arrayelems[narrayelems] = Int64GetDatum((int64) ondisk.builder.committed.xip[narrayelems]);
+
+		values[i++] = PointerGetDatum(construct_array_builtin(arrayelems, narrayelems, INT8OID));
+	}
+	else
+		nulls[i++] = true;
+
+	values[i++] = Int64GetDatum(ondisk.builder.catchange.xcnt);
+	if (ondisk.builder.catchange.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+		int			narrayelems = 0;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.catchange.xcnt * sizeof(Datum));
+
+		for (; narrayelems < ondisk.builder.catchange.xcnt; narrayelems++)
+			arrayelems[narrayelems] = Int64GetDatum((int64) ondisk.builder.catchange.xip[narrayelems]);
+
+		values[i++] = PointerGetDatum(construct_array_builtin(arrayelems, narrayelems, INT8OID));
+	}
+	else
+		nulls[i++] = true;
+
+	Assert(i == PG_GET_LOGICAL_SNAPSHOT_INFO_COLS);
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	MemoryContextReset(ondisk.builder.context);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_INFO_COLS
+}
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.control b/contrib/pg_logicalinspect/pg_logicalinspect.control
new file mode 100644
index 0000000000..b4a70e57ba
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.control
@@ -0,0 +1,5 @@
+# pg_logicalinspect extension
+comment = 'functions to inspect logical decoding components'
+default_version = '1.0'
+module_pathname = '$libdir/pg_logicalinspect'
+relocatable = true
diff --git a/contrib/pg_logicalinspect/specs/logical_inspect.spec b/contrib/pg_logicalinspect/specs/logical_inspect.spec
new file mode 100644
index 0000000000..47641163fe
--- /dev/null
+++ b/contrib/pg_logicalinspect/specs/logical_inspect.spec
@@ -0,0 +1,34 @@
+# Test the pg_logicalinspect functions: that needs some permutation to
+# ensure that we are creating multiple logical snapshots and that one of them
+# contains ongoing catalogs changes.
+setup
+{
+    DROP TABLE IF EXISTS tbl1;
+    CREATE TABLE tbl1 (val1 integer, val2 integer);
+    CREATE EXTENSION pg_logicalinspect;
+}
+
+teardown
+{
+    DROP TABLE tbl1;
+    SELECT 'stop' FROM pg_drop_replication_slot('isolation_slot');
+    DROP EXTENSION pg_logicalinspect;
+}
+
+session "s0"
+setup { SET synchronous_commit=on; }
+step "s0_init" { SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding'); }
+step "s0_begin" { BEGIN; }
+step "s0_savepoint" { SAVEPOINT sp1; }
+step "s0_truncate" { TRUNCATE tbl1; }
+step "s0_insert" { INSERT INTO tbl1 VALUES (1); }
+step "s0_commit" { COMMIT; }
+
+session "s1"
+setup { SET synchronous_commit=on; }
+step "s1_checkpoint" { CHECKPOINT; }
+step "s1_get_changes" { SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0'); }
+step "s1_get_logical_snapshot_meta" { SELECT COUNT((pg_get_logical_snapshot_meta(f.name::pg_lsn))) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f; }
+step "s1_get_logical_snapshot_info" { SELECT (pg_get_logical_snapshot_info(f.name::pg_lsn)).state,(pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_xip,1) AS catchange_array_length,(pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_xip,1) AS committed_array_length FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f ORDER BY 2; }
+
+permutation "s0_init" "s0_begin" "s0_savepoint" "s0_truncate" "s1_checkpoint" "s1_get_changes" "s0_commit" "s0_begin" "s0_insert" "s1_checkpoint" "s1_get_changes" "s0_commit" "s1_get_changes" "s1_get_logical_snapshot_info" "s1_get_logical_snapshot_meta"
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index 44639a8dca..7c381949a5 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -154,6 +154,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
  &pgbuffercache;
  &pgcrypto;
  &pgfreespacemap;
+ &pglogicalinspect;
  &pgprewarm;
  &pgrowlocks;
  &pgstatstatements;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index a7ff5f8264..66e6dccd4c 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -143,6 +143,7 @@
 <!ENTITY pgbuffercache   SYSTEM "pgbuffercache.sgml">
 <!ENTITY pgcrypto        SYSTEM "pgcrypto.sgml">
 <!ENTITY pgfreespacemap  SYSTEM "pgfreespacemap.sgml">
+<!ENTITY pglogicalinspect  SYSTEM "pglogicalinspect.sgml">
 <!ENTITY pgprewarm       SYSTEM "pgprewarm.sgml">
 <!ENTITY pgrowlocks      SYSTEM "pgrowlocks.sgml">
 <!ENTITY pgstatstatements SYSTEM "pgstatstatements.sgml">
diff --git a/doc/src/sgml/pglogicalinspect.sgml b/doc/src/sgml/pglogicalinspect.sgml
new file mode 100644
index 0000000000..f0b26637d0
--- /dev/null
+++ b/doc/src/sgml/pglogicalinspect.sgml
@@ -0,0 +1,151 @@
+<!-- doc/src/sgml/pglogicalinspect.sgml -->
+
+<sect1 id="pglogicalinspect" xreflabel="pg_logicalinspect">
+ <title>pg_logicalinspect &mdash; logical decoding components inspection</title>
+
+ <indexterm zone="pglogicalinspect">
+  <primary>pg_logicalinspect</primary>
+ </indexterm>
+
+ <para>
+  The <filename>pg_logicalinspect</filename> module provides SQL functions
+  that allow you to inspect the contents of logical decoding components. It
+  allows the inspection of serialized logical snapshots of a running
+  <productname>PostgreSQL</productname> database cluster, which is useful
+  for debugging or educational purposes.
+ </para>
+
+ <note>
+  <para>
+   The <filename>pg_logicalinspect</filename> functions are called
+   using an LSN argument that can be extracted from the output name of the
+   <function>pg_ls_logicalsnapdir</function>() function.
+  </para>
+ </note>
+
+ <para>
+  By default, use of these functions is restricted to superusers and members of
+  the <literal>pg_read_server_files</literal> role. Access may be granted by
+  superusers to others using <command>GRANT</command>.
+ </para>
+
+ <sect2 id="pglogicalinspect-funcs">
+  <title>General Functions</title>
+
+  <variablelist>
+   <varlistentry id="pglogicalinspect-funcs-pg-get-logical-snapshot-meta">
+    <term>
+     <function>pg_get_logical_snapshot_meta(in_lsn pg_lsn) returns record</function>
+    </term>
+
+    <listitem>
+     <para>
+      Gets logical snapshot metadata about a snapshot file that is located in
+      the server's <filename>pg_logical/snapshots</filename> directory.
+      The <replaceable>in_lsn</replaceable> argument can be extracted from the
+      snapshot file name.
+      For example:
+<screen>
+postgres=# SELECT * FROM pg_ls_logicalsnapdir();
+-[ RECORD 1 ]+-----------------------
+name         | 0-40796E18.snap
+size         | 152
+modification | 2024-08-14 16:36:32+00
+
+postgres=# SELECT * FROM pg_get_logical_snapshot_meta('0/40796E18');
+-[ RECORD 1 ]--------
+magic    | 1369563137
+checksum | 1028045905
+version  | 6
+
+postgres=# SELECT (pg_get_logical_snapshot_meta(f.name::pg_lsn)).*
+           FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name
+                 FROM pg_ls_logicalsnapdir()) AS f;
+-[ RECORD 1 ]--------
+magic    | 1369563137
+checksum | 1028045905
+version  | 6
+</screen>
+     </para>
+     <para>
+      If <replaceable>in_lsn</replaceable> does not match a snapshot file, the
+      function raises an error.
+     </para>
+    </listitem>
+   </varlistentry>
+
+   <varlistentry id="pglogicalinspect-funcs-pg-get-logical-snapshot-info">
+    <term>
+     <function>pg_get_logical_snapshot_info(in_lsn pg_lsn) returns record</function>
+    </term>
+
+    <listitem>
+     <para>
+      Gets logical snapshot information about a snapshot file that is located in
+      the server's <filename>pg_logical/snapshots</filename> directory.
+      The <replaceable>in_lsn</replaceable> argument can be extracted from the
+      snapshot file name.
+      For example:
+<screen>
+postgres=# SELECT * FROM pg_ls_logicalsnapdir();
+-[ RECORD 1 ]+-----------------------
+name         | 0-40796E18.snap
+size         | 152
+modification | 2024-08-14 16:36:32+00
+
+postgres=# SELECT * FROM pg_get_logical_snapshot_info('0/40796E18');
+-[ RECORD 1 ]------------+-----------
+state                    | consistent
+xmin                     | 751
+xmax                     | 751
+start_decoding_at        | 0/40796AF8
+two_phase_at             | 0/40796AF8
+initial_xmin_horizon     | 0
+building_full_snapshot   | f
+in_slot_creation         | f
+last_serialized_snapshot | 0/0
+next_phase_at            | 0
+committed_count          | 0
+committed_xip            |
+catchange_count          | 2
+catchange_xip            | {751,752}
+
+postgres=# SELECT (pg_get_logical_snapshot_info(f.name::pg_lsn)).*
+           FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name
+                 FROM pg_ls_logicalsnapdir()) AS f;
+-[ RECORD 1 ]------------+-----------
+state                    | consistent
+xmin                     | 751
+xmax                     | 751
+start_decoding_at        | 0/40796AF8
+two_phase_at             | 0/40796AF8
+initial_xmin_horizon     | 0
+building_full_snapshot   | f
+in_slot_creation         | f
+last_serialized_snapshot | 0/0
+next_phase_at            | 0
+committed_count          | 0
+committed_xip            |
+catchange_count          | 2
+catchange_xip            | {751,752}
+</screen>
+     </para>
+     <para>
+      If <replaceable>in_lsn</replaceable> does not match a snapshot file, the
+      function raises an error.
+     </para>
+    </listitem>
+   </varlistentry>
+
+  </variablelist>
+ </sect2>
+
+ <sect2 id="pglogicalinspect-author">
+  <title>Author</title>
+
+  <para>
+   Bertrand Drouvot <email>bertranddrouvot.pg@gmail.com</email>
+  </para>
+ </sect2>
+
+</sect1>
diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c
index 0450f94ba8..bb7def9440 100644
--- a/src/backend/replication/logical/snapbuild.c
+++ b/src/backend/replication/logical/snapbuild.c
@@ -134,6 +134,7 @@
 #include "replication/logical.h"
 #include "replication/reorderbuffer.h"
 #include "replication/snapbuild.h"
+#include "replication/snapbuild_internal.h"
 #include "storage/fd.h"
 #include "storage/lmgr.h"
 #include "storage/proc.h"
@@ -143,146 +144,6 @@
 #include "utils/memutils.h"
 #include "utils/snapmgr.h"
 #include "utils/snapshot.h"
-
-/*
- * This struct contains the current state of the snapshot building
- * machinery. Besides a forward declaration in the header, it is not exposed
- * to the public, so we can easily change its contents.
- */
-struct SnapBuild
-{
-	/* how far are we along building our first full snapshot */
-	SnapBuildState state;
-
-	/* private memory context used to allocate memory for this module. */
-	MemoryContext context;
-
-	/* all transactions < than this have committed/aborted */
-	TransactionId xmin;
-
-	/* all transactions >= than this are uncommitted */
-	TransactionId xmax;
-
-	/*
-	 * Don't replay commits from an LSN < this LSN. This can be set externally
-	 * but it will also be advanced (never retreat) from within snapbuild.c.
-	 */
-	XLogRecPtr	start_decoding_at;
-
-	/*
-	 * LSN at which two-phase decoding was enabled or LSN at which we found a
-	 * consistent point at the time of slot creation.
-	 *
-	 * The prepared transactions, that were skipped because previously
-	 * two-phase was not enabled or are not covered by initial snapshot, need
-	 * to be sent later along with commit prepared and they must be before
-	 * this point.
-	 */
-	XLogRecPtr	two_phase_at;
-
-	/*
-	 * Don't start decoding WAL until the "xl_running_xacts" information
-	 * indicates there are no running xids with an xid smaller than this.
-	 */
-	TransactionId initial_xmin_horizon;
-
-	/* Indicates if we are building full snapshot or just catalog one. */
-	bool		building_full_snapshot;
-
-	/*
-	 * Indicates if we are using the snapshot builder for the creation of a
-	 * logical replication slot. If it's true, the start point for decoding
-	 * changes is not determined yet. So we skip snapshot restores to properly
-	 * find the start point. See SnapBuildFindSnapshot() for details.
-	 */
-	bool		in_slot_creation;
-
-	/*
-	 * Snapshot that's valid to see the catalog state seen at this moment.
-	 */
-	Snapshot	snapshot;
-
-	/*
-	 * LSN of the last location we are sure a snapshot has been serialized to.
-	 */
-	XLogRecPtr	last_serialized_snapshot;
-
-	/*
-	 * The reorderbuffer we need to update with usable snapshots et al.
-	 */
-	ReorderBuffer *reorder;
-
-	/*
-	 * TransactionId at which the next phase of initial snapshot building will
-	 * happen. InvalidTransactionId if not known (i.e. SNAPBUILD_START), or
-	 * when no next phase necessary (SNAPBUILD_CONSISTENT).
-	 */
-	TransactionId next_phase_at;
-
-	/*
-	 * Array of transactions which could have catalog changes that committed
-	 * between xmin and xmax.
-	 */
-	struct
-	{
-		/* number of committed transactions */
-		size_t		xcnt;
-
-		/* available space for committed transactions */
-		size_t		xcnt_space;
-
-		/*
-		 * Until we reach a CONSISTENT state, we record commits of all
-		 * transactions, not just the catalog changing ones. Record when that
-		 * changes so we know we cannot export a snapshot safely anymore.
-		 */
-		bool		includes_all_transactions;
-
-		/*
-		 * Array of committed transactions that have modified the catalog.
-		 *
-		 * As this array is frequently modified we do *not* keep it in
-		 * xidComparator order. Instead we sort the array when building &
-		 * distributing a snapshot.
-		 *
-		 * TODO: It's unclear whether that reasoning has much merit. Every
-		 * time we add something here after becoming consistent will also
-		 * require distributing a snapshot. Storing them sorted would
-		 * potentially also make it easier to purge (but more complicated wrt
-		 * wraparound?). Should be improved if sorting while building the
-		 * snapshot shows up in profiles.
-		 */
-		TransactionId *xip;
-	}			committed;
-
-	/*
-	 * Array of transactions and subtransactions that had modified catalogs
-	 * and were running when the snapshot was serialized.
-	 *
-	 * We normally rely on some WAL record types such as HEAP2_NEW_CID to know
-	 * if the transaction has changed the catalog. But it could happen that
-	 * the logical decoding decodes only the commit record of the transaction
-	 * after restoring the previously serialized snapshot in which case we
-	 * will miss adding the xid to the snapshot and end up looking at the
-	 * catalogs with the wrong snapshot.
-	 *
-	 * Now to avoid the above problem, we serialize the transactions that had
-	 * modified the catalogs and are still running at the time of snapshot
-	 * serialization. We fill this array while restoring the snapshot and then
-	 * refer it while decoding commit to ensure if the xact has modified the
-	 * catalog. We discard this array when all the xids in the list become old
-	 * enough to matter. See SnapBuildPurgeOlderTxn for details.
-	 */
-	struct
-	{
-		/* number of transactions */
-		size_t		xcnt;
-
-		/* This array must be sorted in xidComparator order */
-		TransactionId *xip;
-	}			catchange;
-};
-
 /*
  * Starting a transaction -- which we need to do while exporting a snapshot --
  * removes knowledge about the previously used resowner, so we save it here.
@@ -312,7 +173,6 @@ static void SnapBuildWaitSnapshot(xl_running_xacts *running, TransactionId cutof
 /* serialization functions */
 static void SnapBuildSerialize(SnapBuild *builder, XLogRecPtr lsn);
 static bool SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn);
-static void SnapBuildRestoreContents(int fd, char *dest, Size size, const char *path);
 
 /*
  * Allocate a new snapshot builder.
@@ -1557,48 +1417,6 @@ SnapBuildWaitSnapshot(xl_running_xacts *running, TransactionId cutoff)
 	}
 }
 
-/* -----------------------------------
- * Snapshot serialization support
- * -----------------------------------
- */
-
-/*
- * We store current state of struct SnapBuild on disk in the following manner:
- *
- * struct SnapBuildOnDisk;
- * TransactionId * committed.xcnt; (*not xcnt_space*)
- * TransactionId * catchange.xcnt;
- *
- */
-typedef struct SnapBuildOnDisk
-{
-	/* first part of this struct needs to be version independent */
-
-	/* data not covered by checksum */
-	uint32		magic;
-	pg_crc32c	checksum;
-
-	/* data covered by checksum */
-
-	/* version, in case we want to support pg_upgrade */
-	uint32		version;
-	/* how large is the on disk data, excluding the constant sized part */
-	uint32		length;
-
-	/* version dependent part */
-	SnapBuild	builder;
-
-	/* variable amount of TransactionIds follows */
-} SnapBuildOnDisk;
-
-#define SnapBuildOnDiskConstantSize \
-	offsetof(SnapBuildOnDisk, builder)
-#define SnapBuildOnDiskNotChecksummedSize \
-	offsetof(SnapBuildOnDisk, version)
-
-#define SNAPBUILD_MAGIC 0x51A1E001
-#define SNAPBUILD_VERSION 6
-
 /*
  * Store/Load a snapshot from disk, depending on the snapshot builder's state.
  *
@@ -1859,6 +1677,10 @@ out:
 /*
  * Restore a snapshot into 'builder' if previously one has been stored at the
  * location indicated by 'lsn'. Returns true if successful, false otherwise.
+ *
+ * NOTE: For any code change or issue fix here, it is highly recommended to
+ * give a thought about doing the same in pg_logicalinspect contrib module
+ * as well.
  */
 static bool
 SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
@@ -2033,7 +1855,7 @@ snapshot_not_interesting:
 /*
  * Read the contents of the serialized snapshot to 'dest'.
  */
-static void
+void
 SnapBuildRestoreContents(int fd, char *dest, Size size, const char *path)
 {
 	int			readBytes;
diff --git a/src/include/port/pg_crc32c.h b/src/include/port/pg_crc32c.h
index 63c8e3a00b..cfc8c07944 100644
--- a/src/include/port/pg_crc32c.h
+++ b/src/include/port/pg_crc32c.h
@@ -47,7 +47,7 @@ typedef uint32 pg_crc32c;
 	((crc) = pg_comp_crc32c_sse42((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
 
 #elif defined(USE_ARMV8_CRC32C)
 /* Use ARMv8 CRC Extension instructions. */
@@ -56,7 +56,7 @@ extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t le
 	((crc) = pg_comp_crc32c_armv8((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
 
 #elif defined(USE_LOONGARCH_CRC32C)
 /* Use LoongArch CRCC instructions. */
@@ -65,7 +65,7 @@ extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t le
 	((crc) = pg_comp_crc32c_loongarch((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len);
 
 #elif defined(USE_SSE42_CRC32C_WITH_RUNTIME_CHECK) || defined(USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK)
 
@@ -77,14 +77,14 @@ extern pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_
 	((crc) = pg_comp_crc32c((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
-extern pg_crc32c (*pg_comp_crc32c) (pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c (*pg_comp_crc32c) (pg_crc32c crc, const void *data, size_t len);
 
 #ifdef USE_SSE42_CRC32C_WITH_RUNTIME_CHECK
-extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
 #endif
 #ifdef USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK
-extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
 #endif
 
 #else
@@ -103,7 +103,7 @@ extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t le
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 #endif
 
-extern pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
 
 #endif
 
diff --git a/src/include/replication/snapbuild.h b/src/include/replication/snapbuild.h
index caa5113ff8..78df2d1079 100644
--- a/src/include/replication/snapbuild.h
+++ b/src/include/replication/snapbuild.h
@@ -15,6 +15,10 @@
 #include "access/xlogdefs.h"
 #include "utils/snapmgr.h"
 
+/*
+ * Please keep SnapBuildStateDesc[] (located in the pg_logicalinspect module)
+ * updated should a change needs to be done in SnapBuildState.
+ */
 typedef enum
 {
 	/*
@@ -46,7 +50,7 @@ typedef enum
 	SNAPBUILD_CONSISTENT = 2,
 } SnapBuildState;
 
-/* forward declare so we don't have to expose the struct to the public */
+/* forward declare so we don't have to include snapbuild_internal.h */
 struct SnapBuild;
 typedef struct SnapBuild SnapBuild;
 
diff --git a/src/include/replication/snapbuild_internal.h b/src/include/replication/snapbuild_internal.h
new file mode 100644
index 0000000000..0e47ddfcb4
--- /dev/null
+++ b/src/include/replication/snapbuild_internal.h
@@ -0,0 +1,204 @@
+/*-------------------------------------------------------------------------
+ *
+ * snapbuild_internal.h
+ *    This file contains declarations for logical decoding utility
+ *    functions for internal use.
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * src/include/replication/snapbuild_internal.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef SNAPBUILD_INTERNAL_H
+#define SNAPBUILD_INTERNAL_H
+
+#include "port/pg_crc32c.h"
+#include "replication/reorderbuffer.h"
+#include "replication/snapbuild.h"
+
+/*
+ * This struct contains the current state of the snapshot building
+ * machinery. It is exposed to the public, so pay attention when changing its
+ * contents.
+ */
+typedef struct SnapBuild
+{
+	/* how far are we along building our first full snapshot */
+	SnapBuildState state;
+
+	/* private memory context used to allocate memory for this module. */
+	MemoryContext context;
+
+	/* all transactions < than this have committed/aborted */
+	TransactionId xmin;
+
+	/* all transactions >= than this are uncommitted */
+	TransactionId xmax;
+
+	/*
+	 * Don't replay commits from an LSN < this LSN. This can be set externally
+	 * but it will also be advanced (never retreat) from within snapbuild.c.
+	 */
+	XLogRecPtr	start_decoding_at;
+
+	/*
+	 * LSN at which two-phase decoding was enabled or LSN at which we found a
+	 * consistent point at the time of slot creation.
+	 *
+	 * The prepared transactions, that were skipped because previously
+	 * two-phase was not enabled or are not covered by initial snapshot, need
+	 * to be sent later along with commit prepared and they must be before
+	 * this point.
+	 */
+	XLogRecPtr	two_phase_at;
+
+	/*
+	 * Don't start decoding WAL until the "xl_running_xacts" information
+	 * indicates there are no running xids with an xid smaller than this.
+	 */
+	TransactionId initial_xmin_horizon;
+
+	/* Indicates if we are building full snapshot or just catalog one. */
+	bool		building_full_snapshot;
+
+	/*
+	 * Indicates if we are using the snapshot builder for the creation of a
+	 * logical replication slot. If it's true, the start point for decoding
+	 * changes is not determined yet. So we skip snapshot restores to properly
+	 * find the start point. See SnapBuildFindSnapshot() for details.
+	 */
+	bool		in_slot_creation;
+
+	/*
+	 * Snapshot that's valid to see the catalog state seen at this moment.
+	 */
+	Snapshot	snapshot;
+
+	/*
+	 * LSN of the last location we are sure a snapshot has been serialized to.
+	 */
+	XLogRecPtr	last_serialized_snapshot;
+
+	/*
+	 * The reorderbuffer we need to update with usable snapshots et al.
+	 */
+	ReorderBuffer *reorder;
+
+	/*
+	 * TransactionId at which the next phase of initial snapshot building will
+	 * happen. InvalidTransactionId if not known (i.e. SNAPBUILD_START), or
+	 * when no next phase necessary (SNAPBUILD_CONSISTENT).
+	 */
+	TransactionId next_phase_at;
+
+	/*
+	 * Array of transactions which could have catalog changes that committed
+	 * between xmin and xmax.
+	 */
+	struct
+	{
+		/* number of committed transactions */
+		size_t		xcnt;
+
+		/* available space for committed transactions */
+		size_t		xcnt_space;
+
+		/*
+		 * Until we reach a CONSISTENT state, we record commits of all
+		 * transactions, not just the catalog changing ones. Record when that
+		 * changes so we know we cannot export a snapshot safely anymore.
+		 */
+		bool		includes_all_transactions;
+
+		/*
+		 * Array of committed transactions that have modified the catalog.
+		 *
+		 * As this array is frequently modified we do *not* keep it in
+		 * xidComparator order. Instead we sort the array when building &
+		 * distributing a snapshot.
+		 *
+		 * TODO: It's unclear whether that reasoning has much merit. Every
+		 * time we add something here after becoming consistent will also
+		 * require distributing a snapshot. Storing them sorted would
+		 * potentially also make it easier to purge (but more complicated wrt
+		 * wraparound?). Should be improved if sorting while building the
+		 * snapshot shows up in profiles.
+		 */
+		TransactionId *xip;
+	}			committed;
+
+	/*
+	 * Array of transactions and subtransactions that had modified catalogs
+	 * and were running when the snapshot was serialized.
+	 *
+	 * We normally rely on some WAL record types such as HEAP2_NEW_CID to know
+	 * if the transaction has changed the catalog. But it could happen that
+	 * the logical decoding decodes only the commit record of the transaction
+	 * after restoring the previously serialized snapshot in which case we
+	 * will miss adding the xid to the snapshot and end up looking at the
+	 * catalogs with the wrong snapshot.
+	 *
+	 * Now to avoid the above problem, we serialize the transactions that had
+	 * modified the catalogs and are still running at the time of snapshot
+	 * serialization. We fill this array while restoring the snapshot and then
+	 * refer it while decoding commit to ensure if the xact has modified the
+	 * catalog. We discard this array when all the xids in the list become old
+	 * enough to matter. See SnapBuildPurgeOlderTxn for details.
+	 */
+	struct
+	{
+		/* number of transactions */
+		size_t		xcnt;
+
+		/* This array must be sorted in xidComparator order */
+		TransactionId *xip;
+	}			catchange;
+} SnapBuild;
+
+/* -----------------------------------
+ * Snapshot serialization support
+ * -----------------------------------
+ */
+
+/*
+ * We store current state of struct SnapBuild on disk in the following manner:
+ *
+ * struct SnapBuildOnDisk;
+ * TransactionId * committed.xcnt; (*not xcnt_space*)
+ * TransactionId * catchange.xcnt;
+ *
+ */
+typedef struct SnapBuildOnDisk
+{
+	/* first part of this struct needs to be version independent */
+
+	/* data not covered by checksum */
+	uint32		magic;
+	pg_crc32c	checksum;
+
+	/* data covered by checksum */
+
+	/* version, in case we want to support pg_upgrade */
+	uint32		version;
+	/* how large is the on disk data, excluding the constant sized part */
+	uint32		length;
+
+	/* version dependent part */
+	SnapBuild	builder;
+
+	/* variable amount of TransactionIds follows */
+} SnapBuildOnDisk;
+
+#define SnapBuildOnDiskConstantSize \
+	offsetof(SnapBuildOnDisk, builder)
+#define SnapBuildOnDiskNotChecksummedSize \
+	offsetof(SnapBuildOnDisk, version)
+
+#define SNAPBUILD_MAGIC 0x51A1E001
+#define SNAPBUILD_VERSION 6
+
+extern void SnapBuildRestoreContents(int fd, char *dest, Size size, const char *path);
+
+#endif							/* SNAPBUILD_INTERNAL_H */
-- 
2.34.1

#31Peter Smith
smithpb2250@gmail.com
In reply to: Bertrand Drouvot (#30)
1 attachment(s)
Re: Add contrib/pg_logicalsnapinspect

My review comments for v8-0001

======
contrib/pg_logicalinspect/pg_logicalinspect.c

1.
+/*
+ * Lookup table for SnapBuildState.
+ */
+
+#define SNAPBUILD_STATE_INCR 1
+
+static const char *const SnapBuildStateDesc[] = {
+ [SNAPBUILD_START + SNAPBUILD_STATE_INCR] = "start",
+ [SNAPBUILD_BUILDING_SNAPSHOT + SNAPBUILD_STATE_INCR] = "building",
+ [SNAPBUILD_FULL_SNAPSHOT + SNAPBUILD_STATE_INCR] = "full",
+ [SNAPBUILD_CONSISTENT + SNAPBUILD_STATE_INCR] = "consistent",
+};
+
+/*

nit - the SNAPBUILD_STATE_INCR made this code appear more complicated
than it is. Please take a look at the attachment for an alternative
implementation which includes an explanatory comment. YMMV. Feel free
to ignore it.

======
src/include/replication/snapbuild.h

2.
+ * Please keep SnapBuildStateDesc[] (located in the pg_logicalinspect module)
+ * updated should a change needs to be done in SnapBuildState.

nit - "...should a change needs to be done" -- the word "needs" is
incorrect here.

How about:
"...if a change needs to be made to SnapBuildState."
"...if a change is made to SnapBuildState."
"...if SnapBuildState is changed."

======
Kind Regards,
Peter Smith.
Fujitsu Australia

Attachments:

PS_NITPICKS_v8.txttext/plain; charset=US-ASCII; name=PS_NITPICKS_v8.txtDownload
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.c b/contrib/pg_logicalinspect/pg_logicalinspect.c
index 100b82f..2419df1 100644
--- a/contrib/pg_logicalinspect/pg_logicalinspect.c
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.c
@@ -24,19 +24,6 @@ PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_meta);
 PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_info);
 
 /*
- * Lookup table for SnapBuildState.
- */
-
-#define SNAPBUILD_STATE_INCR 1
-
-static const char *const SnapBuildStateDesc[] = {
-	[SNAPBUILD_START + SNAPBUILD_STATE_INCR] = "start",
-	[SNAPBUILD_BUILDING_SNAPSHOT + SNAPBUILD_STATE_INCR] = "building",
-	[SNAPBUILD_FULL_SNAPSHOT + SNAPBUILD_STATE_INCR] = "full",
-	[SNAPBUILD_CONSISTENT + SNAPBUILD_STATE_INCR] = "consistent",
-};
-
-/*
  * NOTE: For any code change or issue fix here, it is highly recommended to
  * give a thought about doing the same in SnapBuildRestore() as well.
  */
@@ -186,6 +173,16 @@ Datum
 pg_get_logical_snapshot_info(PG_FUNCTION_ARGS)
 {
 #define PG_GET_LOGICAL_SNAPSHOT_INFO_COLS 14
+	/*
+	 * Lookup table for SnapBuildState. The lookup index is offset by 1
+	 * because the consecutive SnapBuildState enum values start at -1.
+	 */
+	static const char *const SnapBuildStateDesc[] = {
+		[1 + SNAPBUILD_START] = "start",
+		[1 + SNAPBUILD_BUILDING_SNAPSHOT] = "building",
+		[1 + SNAPBUILD_FULL_SNAPSHOT] = "full",
+		[1 + SNAPBUILD_CONSISTENT] = "consistent",
+	};
 	SnapBuildOnDisk ondisk;
 	XLogRecPtr	lsn;
 	HeapTuple	tuple;
@@ -209,8 +206,7 @@ pg_get_logical_snapshot_info(PG_FUNCTION_ARGS)
 
 	memset(nulls, 0, sizeof(nulls));
 
-	values[i++] = CStringGetTextDatum(SnapBuildStateDesc[ondisk.builder.state +
-														 SNAPBUILD_STATE_INCR]);
+	values[i++] = CStringGetTextDatum(SnapBuildStateDesc[ondisk.builder.state + 1]);
 	values[i++] = TransactionIdGetDatum(ondisk.builder.xmin);
 	values[i++] = TransactionIdGetDatum(ondisk.builder.xmax);
 	values[i++] = LSNGetDatum(ondisk.builder.start_decoding_at);
diff --git a/src/include/replication/snapbuild.h b/src/include/replication/snapbuild.h
index 78df2d1..e844a89 100644
--- a/src/include/replication/snapbuild.h
+++ b/src/include/replication/snapbuild.h
@@ -17,7 +17,7 @@
 
 /*
  * Please keep SnapBuildStateDesc[] (located in the pg_logicalinspect module)
- * updated should a change needs to be done in SnapBuildState.
+ * updated if a change needs to be made to SnapBuildState.
  */
 typedef enum
 {
#32shveta malik
shveta.malik@gmail.com
In reply to: Bertrand Drouvot (#30)
Re: Add contrib/pg_logicalsnapinspect

On Fri, Sep 20, 2024 at 12:22 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Please find attached v8, that:

Thank You for the patch. In one of my tests, I noticed that I got
negative checksum:

postgres=# SELECT * FROM pg_get_logical_snapshot_meta('0/3481F20');
magic | checksum | version
------------+------------+---------
1369563137 | -266346460 | 6

But pg_crc32c is uint32. Is it because we are getting it as
Int32GetDatum(ondisk.checksum) in pg_get_logical_snapshot_meta()?
Instead should it be UInt32GetDatum?

Same goes for below:
values[i++] = Int32GetDatum(ondisk.magic);
values[i++] = Int32GetDatum(ondisk.magic);

We need to recheck the rest of the fields in the info() function as well.

thanks
Shveta

#33shveta malik
shveta.malik@gmail.com
In reply to: Peter Smith (#31)
Re: Add contrib/pg_logicalsnapinspect

On Mon, Sep 23, 2024 at 7:57 AM Peter Smith <smithpb2250@gmail.com> wrote:

My review comments for v8-0001

======
contrib/pg_logicalinspect/pg_logicalinspect.c

1.
+/*
+ * Lookup table for SnapBuildState.
+ */
+
+#define SNAPBUILD_STATE_INCR 1
+
+static const char *const SnapBuildStateDesc[] = {
+ [SNAPBUILD_START + SNAPBUILD_STATE_INCR] = "start",
+ [SNAPBUILD_BUILDING_SNAPSHOT + SNAPBUILD_STATE_INCR] = "building",
+ [SNAPBUILD_FULL_SNAPSHOT + SNAPBUILD_STATE_INCR] = "full",
+ [SNAPBUILD_CONSISTENT + SNAPBUILD_STATE_INCR] = "consistent",
+};
+
+/*

nit - the SNAPBUILD_STATE_INCR made this code appear more complicated
than it is.

I agree.

Please take a look at the attachment for an alternative
implementation which includes an explanatory comment. YMMV. Feel free
to ignore it.

+1. I find Peter's version with comments easier to understand.

thanks
Shveta

#34Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: Peter Smith (#31)
1 attachment(s)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Mon, Sep 23, 2024 at 12:27:27PM +1000, Peter Smith wrote:

My review comments for v8-0001

======
contrib/pg_logicalinspect/pg_logicalinspect.c

1.
+/*
+ * Lookup table for SnapBuildState.
+ */
+
+#define SNAPBUILD_STATE_INCR 1
+
+static const char *const SnapBuildStateDesc[] = {
+ [SNAPBUILD_START + SNAPBUILD_STATE_INCR] = "start",
+ [SNAPBUILD_BUILDING_SNAPSHOT + SNAPBUILD_STATE_INCR] = "building",
+ [SNAPBUILD_FULL_SNAPSHOT + SNAPBUILD_STATE_INCR] = "full",
+ [SNAPBUILD_CONSISTENT + SNAPBUILD_STATE_INCR] = "consistent",
+};
+
+/*

nit - the SNAPBUILD_STATE_INCR made this code appear more complicated
than it is. Please take a look at the attachment for an alternative
implementation which includes an explanatory comment. YMMV. Feel free
to ignore it.

Thanks for the feedback!

I like the commment, so added it in v9 attached. OTOH I think that's better
to keep SNAPBUILD_STATE_INCR as those "+1" are all linked and that would be
easy to miss the one in pg_get_logical_snapshot_info() should we change the
increment in the future.

======
src/include/replication/snapbuild.h

2.
+ * Please keep SnapBuildStateDesc[] (located in the pg_logicalinspect module)
+ * updated should a change needs to be done in SnapBuildState.

nit - "...should a change needs to be done" -- the word "needs" is
incorrect here.

How about:
"...if a change needs to be made to SnapBuildState."

Thanks, used this one in v9.

[1]: /messages/by-id/CAJpy0uCppUNdod4F3NaPpMCtrySdw1S0T1d8CA-2c4CX=ShMOQ@mail.gmail.com

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachments:

v9-0001-Add-contrib-pg_logicalinspect.patchtext/x-diff; charset=us-asciiDownload
From ae831c9042244f7da6a78d5e350d9c30672f1cd5 Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Wed, 14 Aug 2024 08:46:05 +0000
Subject: [PATCH v9] Add contrib/pg_logicalinspect

Provides SQL functions that allow to inspect logical decoding components.

It currently allows to inspect the contents of serialized logical snapshots of
a running database cluster, which is useful for debugging or educational
purposes.
---
 contrib/Makefile                              |   1 +
 contrib/meson.build                           |   1 +
 contrib/pg_logicalinspect/.gitignore          |   4 +
 contrib/pg_logicalinspect/Makefile            |  31 ++
 .../expected/logical_inspect.out              |  52 ++++
 contrib/pg_logicalinspect/logicalinspect.conf |   1 +
 contrib/pg_logicalinspect/meson.build         |  39 +++
 .../pg_logicalinspect--1.0.sql                |  43 +++
 contrib/pg_logicalinspect/pg_logicalinspect.c | 265 ++++++++++++++++++
 .../pg_logicalinspect.control                 |   5 +
 .../specs/logical_inspect.spec                |  34 +++
 doc/src/sgml/contrib.sgml                     |   1 +
 doc/src/sgml/filelist.sgml                    |   1 +
 doc/src/sgml/pglogicalinspect.sgml            | 151 ++++++++++
 src/backend/replication/logical/snapbuild.c   | 190 +------------
 src/include/port/pg_crc32c.h                  |  16 +-
 src/include/replication/snapbuild.h           |   6 +-
 src/include/replication/snapbuild_internal.h  | 204 ++++++++++++++
 18 files changed, 852 insertions(+), 193 deletions(-)
   7.8% contrib/pg_logicalinspect/expected/
   5.7% contrib/pg_logicalinspect/specs/
  33.1% contrib/pg_logicalinspect/
  13.6% doc/src/sgml/
  16.8% src/backend/replication/logical/
   4.0% src/include/port/
  18.6% src/include/replication/

diff --git a/contrib/Makefile b/contrib/Makefile
index abd780f277..952855d9b6 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -32,6 +32,7 @@ SUBDIRS = \
 		passwordcheck	\
 		pg_buffercache	\
 		pg_freespacemap \
+		pg_logicalinspect \
 		pg_prewarm	\
 		pg_stat_statements \
 		pg_surgery	\
diff --git a/contrib/meson.build b/contrib/meson.build
index 14a8906865..159ff41555 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -46,6 +46,7 @@ subdir('passwordcheck')
 subdir('pg_buffercache')
 subdir('pgcrypto')
 subdir('pg_freespacemap')
+subdir('pg_logicalinspect')
 subdir('pg_prewarm')
 subdir('pgrowlocks')
 subdir('pg_stat_statements')
diff --git a/contrib/pg_logicalinspect/.gitignore b/contrib/pg_logicalinspect/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/contrib/pg_logicalinspect/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/contrib/pg_logicalinspect/Makefile b/contrib/pg_logicalinspect/Makefile
new file mode 100644
index 0000000000..55124514d4
--- /dev/null
+++ b/contrib/pg_logicalinspect/Makefile
@@ -0,0 +1,31 @@
+# contrib/pg_logicalinspect/Makefile
+
+MODULE_big = pg_logicalinspect
+OBJS = \
+	$(WIN32RES) \
+	pg_logicalinspect.o
+PGFILEDESC = "pg_logicalinspect - functions to inspect logical decoding components"
+
+EXTENSION = pg_logicalinspect
+DATA = pg_logicalinspect--1.0.sql
+
+EXTRA_INSTALL = contrib/test_decoding
+
+ISOLATION = logical_inspect
+
+ISOLATION_OPTS = --temp-config $(top_srcdir)/contrib/pg_logicalinspect/logicalinspect.conf
+
+# Disabled because these tests require "wal_level=logical", which
+# some installcheck users do not have (e.g. buildfarm clients).
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/pg_logicalinspect
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/pg_logicalinspect/expected/logical_inspect.out b/contrib/pg_logicalinspect/expected/logical_inspect.out
new file mode 100644
index 0000000000..08de273197
--- /dev/null
+++ b/contrib/pg_logicalinspect/expected/logical_inspect.out
@@ -0,0 +1,52 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s0_init s0_begin s0_savepoint s0_truncate s1_checkpoint s1_get_changes s0_commit s0_begin s0_insert s1_checkpoint s1_get_changes s0_commit s1_get_changes s1_get_logical_snapshot_info s1_get_logical_snapshot_meta
+step s0_init: SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding');
+?column?
+--------
+init    
+(1 row)
+
+step s0_begin: BEGIN;
+step s0_savepoint: SAVEPOINT sp1;
+step s0_truncate: TRUNCATE tbl1;
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data
+----
+(0 rows)
+
+step s0_commit: COMMIT;
+step s0_begin: BEGIN;
+step s0_insert: INSERT INTO tbl1 VALUES (1);
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                   
+---------------------------------------
+BEGIN                                  
+table public.tbl1: TRUNCATE: (no-flags)
+COMMIT                                 
+(3 rows)
+
+step s0_commit: COMMIT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                                         
+-------------------------------------------------------------
+BEGIN                                                        
+table public.tbl1: INSERT: val1[integer]:1 val2[integer]:null
+COMMIT                                                       
+(3 rows)
+
+step s1_get_logical_snapshot_info: SELECT (pg_get_logical_snapshot_info(f.name::pg_lsn)).state,(pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_xip,1) AS catchange_array_length,(pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_xip,1) AS committed_array_length FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f ORDER BY 2;
+state     |catchange_count|catchange_array_length|committed_count|committed_array_length
+----------+---------------+----------------------+---------------+----------------------
+consistent|              0|                      |              2|                     2
+consistent|              2|                     2|              0|                      
+(2 rows)
+
+step s1_get_logical_snapshot_meta: SELECT COUNT((pg_get_logical_snapshot_meta(f.name::pg_lsn))) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f;
+count
+-----
+    2
+(1 row)
+
diff --git a/contrib/pg_logicalinspect/logicalinspect.conf b/contrib/pg_logicalinspect/logicalinspect.conf
new file mode 100644
index 0000000000..e3d257315f
--- /dev/null
+++ b/contrib/pg_logicalinspect/logicalinspect.conf
@@ -0,0 +1 @@
+wal_level = logical
diff --git a/contrib/pg_logicalinspect/meson.build b/contrib/pg_logicalinspect/meson.build
new file mode 100644
index 0000000000..3ec635509b
--- /dev/null
+++ b/contrib/pg_logicalinspect/meson.build
@@ -0,0 +1,39 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+pg_logicalinspect_sources = files('pg_logicalinspect.c')
+
+if host_system == 'windows'
+  pg_logicalinspect_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+    '--NAME', 'pg_logicalinspect',
+    '--FILEDESC', 'pg_logicalinspect - functions to inspect logical decoding components',])
+endif
+
+pg_logicalinspect = shared_module('pg_logicalinspect',
+  pg_logicalinspect_sources,
+  kwargs: contrib_mod_args + {
+      'dependencies': contrib_mod_args['dependencies'],
+  },
+)
+contrib_targets += pg_logicalinspect
+
+install_data(
+  'pg_logicalinspect.control',
+  'pg_logicalinspect--1.0.sql',
+  kwargs: contrib_data_args,
+)
+
+tests += {
+  'name': 'pg_logicalinspect',
+  'sd': meson.current_source_dir(),
+  'bd': meson.current_build_dir(),
+  'isolation': {
+    'specs': [
+      'logical_inspect',
+    ],
+    'regress_args': [
+      '--temp-config', files('logicalinspect.conf'),
+    ],
+    # see above
+    'runningcheck': false,
+  },
+}
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql b/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
new file mode 100644
index 0000000000..fc06e428ce
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
@@ -0,0 +1,43 @@
+/* contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_logicalinspect" to load this file. \quit
+
+--
+-- pg_get_logical_snapshot_meta()
+--
+CREATE FUNCTION pg_get_logical_snapshot_meta(IN in_lsn pg_lsn,
+    OUT magic int4,
+    OUT checksum int8,
+    OUT version int4
+)
+AS 'MODULE_PATHNAME', 'pg_get_logical_snapshot_meta'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_logical_snapshot_meta(pg_lsn) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_logical_snapshot_meta(pg_lsn) TO pg_read_server_files;
+
+--
+-- pg_get_logical_snapshot_info()
+--
+CREATE FUNCTION pg_get_logical_snapshot_info(IN in_lsn pg_lsn,
+    OUT state text,
+    OUT xmin xid,
+    OUT xmax xid,
+    OUT start_decoding_at pg_lsn,
+    OUT two_phase_at pg_lsn,
+    OUT initial_xmin_horizon xid,
+    OUT building_full_snapshot boolean,
+    OUT in_slot_creation boolean,
+    OUT last_serialized_snapshot pg_lsn,
+    OUT next_phase_at xid,
+    OUT committed_count int8,
+    OUT committed_xip xid[],
+    OUT catchange_count int8,
+    OUT catchange_xip xid[]
+)
+AS 'MODULE_PATHNAME', 'pg_get_logical_snapshot_info'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_logical_snapshot_info(pg_lsn) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_logical_snapshot_info(pg_lsn) TO pg_read_server_files;
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.c b/contrib/pg_logicalinspect/pg_logicalinspect.c
new file mode 100644
index 0000000000..770249425e
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.c
@@ -0,0 +1,265 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_logicalinspect.c
+ *		  Functions to inspect contents of PostgreSQL logical snapshots
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ *		  contrib/pg_logicalinspect/pg_logicalinspect.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "funcapi.h"
+#include "replication/snapbuild_internal.h"
+#include "utils/array.h"
+#include "utils/builtins.h"
+#include "utils/pg_lsn.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_meta);
+PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_info);
+
+/*
+ * Lookup table for SnapBuildState. The lookup index is offset by 1
+ * because the consecutive SnapBuildState enum values start at -1.
+ */
+#define SNAPBUILD_STATE_INCR 1
+
+static const char *const SnapBuildStateDesc[] = {
+	[SNAPBUILD_START + SNAPBUILD_STATE_INCR] = "start",
+	[SNAPBUILD_BUILDING_SNAPSHOT + SNAPBUILD_STATE_INCR] = "building",
+	[SNAPBUILD_FULL_SNAPSHOT + SNAPBUILD_STATE_INCR] = "full",
+	[SNAPBUILD_CONSISTENT + SNAPBUILD_STATE_INCR] = "consistent",
+};
+
+/*
+ * NOTE: For any code change or issue fix here, it is highly recommended to
+ * give a thought about doing the same in SnapBuildRestore() as well.
+ */
+
+/*
+ * Validate the logical snapshot file and read its contents to 'ondisk'.
+ */
+static void
+ValidateAndRestoreSnapshotFile(XLogRecPtr lsn, SnapBuildOnDisk *ondisk,
+							   const char *path)
+{
+	int			fd;
+	Size		sz;
+	pg_crc32c	checksum;
+	MemoryContext context;
+
+	context = AllocSetContextCreate(CurrentMemoryContext,
+									"logicalsnapshot inspect context",
+									ALLOCSET_DEFAULT_SIZES);
+
+	fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
+
+	if (fd < 0 && errno == ENOENT)
+		ereport(ERROR,
+				errmsg("file \"%s\" does not exist", path));
+	else if (fd < 0)
+		ereport(ERROR,
+				(errcode_for_file_access(),
+				 errmsg("could not open file \"%s\": %m", path)));
+
+	/* ----
+	 * Make sure the snapshot had been stored safely to disk, that's normally
+	 * cheap.
+	 * Note that we do not need PANIC here, nobody will be able to use the
+	 * slot without fsyncing, and saving it won't succeed without an fsync()
+	 * either...
+	 * ----
+	 */
+	fsync_fname(path, false);
+	fsync_fname(PG_LOGICAL_SNAPSHOTS_DIR, true);
+
+	/* read statically sized portion of snapshot */
+	SnapBuildRestoreContents(fd, (char *) ondisk, SnapBuildOnDiskConstantSize, path);
+
+	if (ondisk->magic != SNAPBUILD_MAGIC)
+		ereport(ERROR,
+				(errcode(ERRCODE_DATA_CORRUPTED),
+				 errmsg("snapbuild state file \"%s\" has wrong magic number: %u instead of %u",
+						path, ondisk->magic, SNAPBUILD_MAGIC)));
+
+	if (ondisk->version != SNAPBUILD_VERSION)
+		ereport(ERROR,
+				(errcode(ERRCODE_DATA_CORRUPTED),
+				 errmsg("snapbuild state file \"%s\" has unsupported version: %u instead of %u",
+						path, ondisk->version, SNAPBUILD_VERSION)));
+
+	INIT_CRC32C(checksum);
+	COMP_CRC32C(checksum,
+				((char *) ondisk) + SnapBuildOnDiskNotChecksummedSize,
+				SnapBuildOnDiskConstantSize - SnapBuildOnDiskNotChecksummedSize);
+
+	/* read SnapBuild */
+	SnapBuildRestoreContents(fd, (char *) &ondisk->builder, sizeof(SnapBuild), path);
+	COMP_CRC32C(checksum, &ondisk->builder, sizeof(SnapBuild));
+
+	ondisk->builder.context = context;
+
+	/* restore committed xacts information */
+	if (ondisk->builder.committed.xcnt > 0)
+	{
+		sz = sizeof(TransactionId) * ondisk->builder.committed.xcnt;
+		ondisk->builder.committed.xip = MemoryContextAllocZero(ondisk->builder.context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.committed.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.committed.xip, sz);
+	}
+
+	/* restore catalog modifying xacts information */
+	if (ondisk->builder.catchange.xcnt > 0)
+	{
+		sz = sizeof(TransactionId) * ondisk->builder.catchange.xcnt;
+		ondisk->builder.catchange.xip = MemoryContextAllocZero(ondisk->builder.context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.catchange.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.catchange.xip, sz);
+	}
+
+	if (CloseTransientFile(fd) != 0)
+		ereport(ERROR,
+				(errcode_for_file_access(),
+				 errmsg("could not close file \"%s\": %m", path)));
+
+	FIN_CRC32C(checksum);
+
+	/* verify checksum of what we've read */
+	if (!EQ_CRC32C(checksum, ondisk->checksum))
+		ereport(ERROR,
+				(errcode(ERRCODE_DATA_CORRUPTED),
+				 errmsg("checksum mismatch for snapbuild state file \"%s\": is %u, should be %u",
+						path, checksum, ondisk->checksum)));
+}
+
+/*
+ * Retrieve the logical snapshot file metadata.
+ */
+Datum
+pg_get_logical_snapshot_meta(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_META_COLS 3
+	SnapBuildOnDisk ondisk;
+	XLogRecPtr	lsn;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_META_COLS];
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_META_COLS];
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+	int			i = 0;
+
+	lsn = PG_GETARG_LSN(0);
+
+	sprintf(path, "%s/%X-%X.snap",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			LSN_FORMAT_ARGS(lsn));
+
+	ValidateAndRestoreSnapshotFile(lsn, &ondisk, path);
+
+	/* Build a tuple descriptor for our result type. */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	memset(nulls, 0, sizeof(nulls));
+
+	values[i++] = UInt32GetDatum(ondisk.magic);
+	values[i++] = Int64GetDatum((int64) ondisk.checksum);
+	values[i++] = UInt32GetDatum(ondisk.version);
+
+	Assert(i == PG_GET_LOGICAL_SNAPSHOT_META_COLS);
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	MemoryContextReset(ondisk.builder.context);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_META_COLS
+}
+
+Datum
+pg_get_logical_snapshot_info(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_INFO_COLS 14
+	SnapBuildOnDisk ondisk;
+	XLogRecPtr	lsn;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS];
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS];
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+	int			i = 0;
+
+	lsn = PG_GETARG_LSN(0);
+
+	sprintf(path, "%s/%X-%X.snap",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			LSN_FORMAT_ARGS(lsn));
+
+	ValidateAndRestoreSnapshotFile(lsn, &ondisk, path);
+
+	/* Build a tuple descriptor for our result type. */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	memset(nulls, 0, sizeof(nulls));
+
+	values[i++] = CStringGetTextDatum(SnapBuildStateDesc[ondisk.builder.state +
+														 SNAPBUILD_STATE_INCR]);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.xmin);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.xmax);
+	values[i++] = LSNGetDatum(ondisk.builder.start_decoding_at);
+	values[i++] = LSNGetDatum(ondisk.builder.two_phase_at);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.initial_xmin_horizon);
+	values[i++] = BoolGetDatum(ondisk.builder.building_full_snapshot);
+	values[i++] = BoolGetDatum(ondisk.builder.in_slot_creation);
+	values[i++] = LSNGetDatum(ondisk.builder.last_serialized_snapshot);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.next_phase_at);
+
+	values[i++] = Int64GetDatum(ondisk.builder.committed.xcnt);
+	if (ondisk.builder.committed.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+		int			narrayelems = 0;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.committed.xcnt * sizeof(Datum));
+
+		for (; narrayelems < ondisk.builder.committed.xcnt; narrayelems++)
+			arrayelems[narrayelems] = Int64GetDatum((int64) ondisk.builder.committed.xip[narrayelems]);
+
+		values[i++] = PointerGetDatum(construct_array_builtin(arrayelems, narrayelems, INT8OID));
+	}
+	else
+		nulls[i++] = true;
+
+	values[i++] = Int64GetDatum(ondisk.builder.catchange.xcnt);
+	if (ondisk.builder.catchange.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+		int			narrayelems = 0;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.catchange.xcnt * sizeof(Datum));
+
+		for (; narrayelems < ondisk.builder.catchange.xcnt; narrayelems++)
+			arrayelems[narrayelems] = Int64GetDatum((int64) ondisk.builder.catchange.xip[narrayelems]);
+
+		values[i++] = PointerGetDatum(construct_array_builtin(arrayelems, narrayelems, INT8OID));
+	}
+	else
+		nulls[i++] = true;
+
+	Assert(i == PG_GET_LOGICAL_SNAPSHOT_INFO_COLS);
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	MemoryContextReset(ondisk.builder.context);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_INFO_COLS
+}
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.control b/contrib/pg_logicalinspect/pg_logicalinspect.control
new file mode 100644
index 0000000000..b4a70e57ba
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.control
@@ -0,0 +1,5 @@
+# pg_logicalinspect extension
+comment = 'functions to inspect logical decoding components'
+default_version = '1.0'
+module_pathname = '$libdir/pg_logicalinspect'
+relocatable = true
diff --git a/contrib/pg_logicalinspect/specs/logical_inspect.spec b/contrib/pg_logicalinspect/specs/logical_inspect.spec
new file mode 100644
index 0000000000..47641163fe
--- /dev/null
+++ b/contrib/pg_logicalinspect/specs/logical_inspect.spec
@@ -0,0 +1,34 @@
+# Test the pg_logicalinspect functions: that needs some permutation to
+# ensure that we are creating multiple logical snapshots and that one of them
+# contains ongoing catalogs changes.
+setup
+{
+    DROP TABLE IF EXISTS tbl1;
+    CREATE TABLE tbl1 (val1 integer, val2 integer);
+    CREATE EXTENSION pg_logicalinspect;
+}
+
+teardown
+{
+    DROP TABLE tbl1;
+    SELECT 'stop' FROM pg_drop_replication_slot('isolation_slot');
+    DROP EXTENSION pg_logicalinspect;
+}
+
+session "s0"
+setup { SET synchronous_commit=on; }
+step "s0_init" { SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding'); }
+step "s0_begin" { BEGIN; }
+step "s0_savepoint" { SAVEPOINT sp1; }
+step "s0_truncate" { TRUNCATE tbl1; }
+step "s0_insert" { INSERT INTO tbl1 VALUES (1); }
+step "s0_commit" { COMMIT; }
+
+session "s1"
+setup { SET synchronous_commit=on; }
+step "s1_checkpoint" { CHECKPOINT; }
+step "s1_get_changes" { SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0'); }
+step "s1_get_logical_snapshot_meta" { SELECT COUNT((pg_get_logical_snapshot_meta(f.name::pg_lsn))) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f; }
+step "s1_get_logical_snapshot_info" { SELECT (pg_get_logical_snapshot_info(f.name::pg_lsn)).state,(pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_xip,1) AS catchange_array_length,(pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_xip,1) AS committed_array_length FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f ORDER BY 2; }
+
+permutation "s0_init" "s0_begin" "s0_savepoint" "s0_truncate" "s1_checkpoint" "s1_get_changes" "s0_commit" "s0_begin" "s0_insert" "s1_checkpoint" "s1_get_changes" "s0_commit" "s1_get_changes" "s1_get_logical_snapshot_info" "s1_get_logical_snapshot_meta"
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index 44639a8dca..7c381949a5 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -154,6 +154,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
  &pgbuffercache;
  &pgcrypto;
  &pgfreespacemap;
+ &pglogicalinspect;
  &pgprewarm;
  &pgrowlocks;
  &pgstatstatements;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index a7ff5f8264..66e6dccd4c 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -143,6 +143,7 @@
 <!ENTITY pgbuffercache   SYSTEM "pgbuffercache.sgml">
 <!ENTITY pgcrypto        SYSTEM "pgcrypto.sgml">
 <!ENTITY pgfreespacemap  SYSTEM "pgfreespacemap.sgml">
+<!ENTITY pglogicalinspect  SYSTEM "pglogicalinspect.sgml">
 <!ENTITY pgprewarm       SYSTEM "pgprewarm.sgml">
 <!ENTITY pgrowlocks      SYSTEM "pgrowlocks.sgml">
 <!ENTITY pgstatstatements SYSTEM "pgstatstatements.sgml">
diff --git a/doc/src/sgml/pglogicalinspect.sgml b/doc/src/sgml/pglogicalinspect.sgml
new file mode 100644
index 0000000000..f0b26637d0
--- /dev/null
+++ b/doc/src/sgml/pglogicalinspect.sgml
@@ -0,0 +1,151 @@
+<!-- doc/src/sgml/pglogicalinspect.sgml -->
+
+<sect1 id="pglogicalinspect" xreflabel="pg_logicalinspect">
+ <title>pg_logicalinspect &mdash; logical decoding components inspection</title>
+
+ <indexterm zone="pglogicalinspect">
+  <primary>pg_logicalinspect</primary>
+ </indexterm>
+
+ <para>
+  The <filename>pg_logicalinspect</filename> module provides SQL functions
+  that allow you to inspect the contents of logical decoding components. It
+  allows the inspection of serialized logical snapshots of a running
+  <productname>PostgreSQL</productname> database cluster, which is useful
+  for debugging or educational purposes.
+ </para>
+
+ <note>
+  <para>
+   The <filename>pg_logicalinspect</filename> functions are called
+   using an LSN argument that can be extracted from the output name of the
+   <function>pg_ls_logicalsnapdir</function>() function.
+  </para>
+ </note>
+
+ <para>
+  By default, use of these functions is restricted to superusers and members of
+  the <literal>pg_read_server_files</literal> role. Access may be granted by
+  superusers to others using <command>GRANT</command>.
+ </para>
+
+ <sect2 id="pglogicalinspect-funcs">
+  <title>General Functions</title>
+
+  <variablelist>
+   <varlistentry id="pglogicalinspect-funcs-pg-get-logical-snapshot-meta">
+    <term>
+     <function>pg_get_logical_snapshot_meta(in_lsn pg_lsn) returns record</function>
+    </term>
+
+    <listitem>
+     <para>
+      Gets logical snapshot metadata about a snapshot file that is located in
+      the server's <filename>pg_logical/snapshots</filename> directory.
+      The <replaceable>in_lsn</replaceable> argument can be extracted from the
+      snapshot file name.
+      For example:
+<screen>
+postgres=# SELECT * FROM pg_ls_logicalsnapdir();
+-[ RECORD 1 ]+-----------------------
+name         | 0-40796E18.snap
+size         | 152
+modification | 2024-08-14 16:36:32+00
+
+postgres=# SELECT * FROM pg_get_logical_snapshot_meta('0/40796E18');
+-[ RECORD 1 ]--------
+magic    | 1369563137
+checksum | 1028045905
+version  | 6
+
+postgres=# SELECT (pg_get_logical_snapshot_meta(f.name::pg_lsn)).*
+           FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name
+                 FROM pg_ls_logicalsnapdir()) AS f;
+-[ RECORD 1 ]--------
+magic    | 1369563137
+checksum | 1028045905
+version  | 6
+</screen>
+     </para>
+     <para>
+      If <replaceable>in_lsn</replaceable> does not match a snapshot file, the
+      function raises an error.
+     </para>
+    </listitem>
+   </varlistentry>
+
+   <varlistentry id="pglogicalinspect-funcs-pg-get-logical-snapshot-info">
+    <term>
+     <function>pg_get_logical_snapshot_info(in_lsn pg_lsn) returns record</function>
+    </term>
+
+    <listitem>
+     <para>
+      Gets logical snapshot information about a snapshot file that is located in
+      the server's <filename>pg_logical/snapshots</filename> directory.
+      The <replaceable>in_lsn</replaceable> argument can be extracted from the
+      snapshot file name.
+      For example:
+<screen>
+postgres=# SELECT * FROM pg_ls_logicalsnapdir();
+-[ RECORD 1 ]+-----------------------
+name         | 0-40796E18.snap
+size         | 152
+modification | 2024-08-14 16:36:32+00
+
+postgres=# SELECT * FROM pg_get_logical_snapshot_info('0/40796E18');
+-[ RECORD 1 ]------------+-----------
+state                    | consistent
+xmin                     | 751
+xmax                     | 751
+start_decoding_at        | 0/40796AF8
+two_phase_at             | 0/40796AF8
+initial_xmin_horizon     | 0
+building_full_snapshot   | f
+in_slot_creation         | f
+last_serialized_snapshot | 0/0
+next_phase_at            | 0
+committed_count          | 0
+committed_xip            |
+catchange_count          | 2
+catchange_xip            | {751,752}
+
+postgres=# SELECT (pg_get_logical_snapshot_info(f.name::pg_lsn)).*
+           FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name
+                 FROM pg_ls_logicalsnapdir()) AS f;
+-[ RECORD 1 ]------------+-----------
+state                    | consistent
+xmin                     | 751
+xmax                     | 751
+start_decoding_at        | 0/40796AF8
+two_phase_at             | 0/40796AF8
+initial_xmin_horizon     | 0
+building_full_snapshot   | f
+in_slot_creation         | f
+last_serialized_snapshot | 0/0
+next_phase_at            | 0
+committed_count          | 0
+committed_xip            |
+catchange_count          | 2
+catchange_xip            | {751,752}
+</screen>
+     </para>
+     <para>
+      If <replaceable>in_lsn</replaceable> does not match a snapshot file, the
+      function raises an error.
+     </para>
+    </listitem>
+   </varlistentry>
+
+  </variablelist>
+ </sect2>
+
+ <sect2 id="pglogicalinspect-author">
+  <title>Author</title>
+
+  <para>
+   Bertrand Drouvot <email>bertranddrouvot.pg@gmail.com</email>
+  </para>
+ </sect2>
+
+</sect1>
diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c
index 0450f94ba8..bb7def9440 100644
--- a/src/backend/replication/logical/snapbuild.c
+++ b/src/backend/replication/logical/snapbuild.c
@@ -134,6 +134,7 @@
 #include "replication/logical.h"
 #include "replication/reorderbuffer.h"
 #include "replication/snapbuild.h"
+#include "replication/snapbuild_internal.h"
 #include "storage/fd.h"
 #include "storage/lmgr.h"
 #include "storage/proc.h"
@@ -143,146 +144,6 @@
 #include "utils/memutils.h"
 #include "utils/snapmgr.h"
 #include "utils/snapshot.h"
-
-/*
- * This struct contains the current state of the snapshot building
- * machinery. Besides a forward declaration in the header, it is not exposed
- * to the public, so we can easily change its contents.
- */
-struct SnapBuild
-{
-	/* how far are we along building our first full snapshot */
-	SnapBuildState state;
-
-	/* private memory context used to allocate memory for this module. */
-	MemoryContext context;
-
-	/* all transactions < than this have committed/aborted */
-	TransactionId xmin;
-
-	/* all transactions >= than this are uncommitted */
-	TransactionId xmax;
-
-	/*
-	 * Don't replay commits from an LSN < this LSN. This can be set externally
-	 * but it will also be advanced (never retreat) from within snapbuild.c.
-	 */
-	XLogRecPtr	start_decoding_at;
-
-	/*
-	 * LSN at which two-phase decoding was enabled or LSN at which we found a
-	 * consistent point at the time of slot creation.
-	 *
-	 * The prepared transactions, that were skipped because previously
-	 * two-phase was not enabled or are not covered by initial snapshot, need
-	 * to be sent later along with commit prepared and they must be before
-	 * this point.
-	 */
-	XLogRecPtr	two_phase_at;
-
-	/*
-	 * Don't start decoding WAL until the "xl_running_xacts" information
-	 * indicates there are no running xids with an xid smaller than this.
-	 */
-	TransactionId initial_xmin_horizon;
-
-	/* Indicates if we are building full snapshot or just catalog one. */
-	bool		building_full_snapshot;
-
-	/*
-	 * Indicates if we are using the snapshot builder for the creation of a
-	 * logical replication slot. If it's true, the start point for decoding
-	 * changes is not determined yet. So we skip snapshot restores to properly
-	 * find the start point. See SnapBuildFindSnapshot() for details.
-	 */
-	bool		in_slot_creation;
-
-	/*
-	 * Snapshot that's valid to see the catalog state seen at this moment.
-	 */
-	Snapshot	snapshot;
-
-	/*
-	 * LSN of the last location we are sure a snapshot has been serialized to.
-	 */
-	XLogRecPtr	last_serialized_snapshot;
-
-	/*
-	 * The reorderbuffer we need to update with usable snapshots et al.
-	 */
-	ReorderBuffer *reorder;
-
-	/*
-	 * TransactionId at which the next phase of initial snapshot building will
-	 * happen. InvalidTransactionId if not known (i.e. SNAPBUILD_START), or
-	 * when no next phase necessary (SNAPBUILD_CONSISTENT).
-	 */
-	TransactionId next_phase_at;
-
-	/*
-	 * Array of transactions which could have catalog changes that committed
-	 * between xmin and xmax.
-	 */
-	struct
-	{
-		/* number of committed transactions */
-		size_t		xcnt;
-
-		/* available space for committed transactions */
-		size_t		xcnt_space;
-
-		/*
-		 * Until we reach a CONSISTENT state, we record commits of all
-		 * transactions, not just the catalog changing ones. Record when that
-		 * changes so we know we cannot export a snapshot safely anymore.
-		 */
-		bool		includes_all_transactions;
-
-		/*
-		 * Array of committed transactions that have modified the catalog.
-		 *
-		 * As this array is frequently modified we do *not* keep it in
-		 * xidComparator order. Instead we sort the array when building &
-		 * distributing a snapshot.
-		 *
-		 * TODO: It's unclear whether that reasoning has much merit. Every
-		 * time we add something here after becoming consistent will also
-		 * require distributing a snapshot. Storing them sorted would
-		 * potentially also make it easier to purge (but more complicated wrt
-		 * wraparound?). Should be improved if sorting while building the
-		 * snapshot shows up in profiles.
-		 */
-		TransactionId *xip;
-	}			committed;
-
-	/*
-	 * Array of transactions and subtransactions that had modified catalogs
-	 * and were running when the snapshot was serialized.
-	 *
-	 * We normally rely on some WAL record types such as HEAP2_NEW_CID to know
-	 * if the transaction has changed the catalog. But it could happen that
-	 * the logical decoding decodes only the commit record of the transaction
-	 * after restoring the previously serialized snapshot in which case we
-	 * will miss adding the xid to the snapshot and end up looking at the
-	 * catalogs with the wrong snapshot.
-	 *
-	 * Now to avoid the above problem, we serialize the transactions that had
-	 * modified the catalogs and are still running at the time of snapshot
-	 * serialization. We fill this array while restoring the snapshot and then
-	 * refer it while decoding commit to ensure if the xact has modified the
-	 * catalog. We discard this array when all the xids in the list become old
-	 * enough to matter. See SnapBuildPurgeOlderTxn for details.
-	 */
-	struct
-	{
-		/* number of transactions */
-		size_t		xcnt;
-
-		/* This array must be sorted in xidComparator order */
-		TransactionId *xip;
-	}			catchange;
-};
-
 /*
  * Starting a transaction -- which we need to do while exporting a snapshot --
  * removes knowledge about the previously used resowner, so we save it here.
@@ -312,7 +173,6 @@ static void SnapBuildWaitSnapshot(xl_running_xacts *running, TransactionId cutof
 /* serialization functions */
 static void SnapBuildSerialize(SnapBuild *builder, XLogRecPtr lsn);
 static bool SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn);
-static void SnapBuildRestoreContents(int fd, char *dest, Size size, const char *path);
 
 /*
  * Allocate a new snapshot builder.
@@ -1557,48 +1417,6 @@ SnapBuildWaitSnapshot(xl_running_xacts *running, TransactionId cutoff)
 	}
 }
 
-/* -----------------------------------
- * Snapshot serialization support
- * -----------------------------------
- */
-
-/*
- * We store current state of struct SnapBuild on disk in the following manner:
- *
- * struct SnapBuildOnDisk;
- * TransactionId * committed.xcnt; (*not xcnt_space*)
- * TransactionId * catchange.xcnt;
- *
- */
-typedef struct SnapBuildOnDisk
-{
-	/* first part of this struct needs to be version independent */
-
-	/* data not covered by checksum */
-	uint32		magic;
-	pg_crc32c	checksum;
-
-	/* data covered by checksum */
-
-	/* version, in case we want to support pg_upgrade */
-	uint32		version;
-	/* how large is the on disk data, excluding the constant sized part */
-	uint32		length;
-
-	/* version dependent part */
-	SnapBuild	builder;
-
-	/* variable amount of TransactionIds follows */
-} SnapBuildOnDisk;
-
-#define SnapBuildOnDiskConstantSize \
-	offsetof(SnapBuildOnDisk, builder)
-#define SnapBuildOnDiskNotChecksummedSize \
-	offsetof(SnapBuildOnDisk, version)
-
-#define SNAPBUILD_MAGIC 0x51A1E001
-#define SNAPBUILD_VERSION 6
-
 /*
  * Store/Load a snapshot from disk, depending on the snapshot builder's state.
  *
@@ -1859,6 +1677,10 @@ out:
 /*
  * Restore a snapshot into 'builder' if previously one has been stored at the
  * location indicated by 'lsn'. Returns true if successful, false otherwise.
+ *
+ * NOTE: For any code change or issue fix here, it is highly recommended to
+ * give a thought about doing the same in pg_logicalinspect contrib module
+ * as well.
  */
 static bool
 SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
@@ -2033,7 +1855,7 @@ snapshot_not_interesting:
 /*
  * Read the contents of the serialized snapshot to 'dest'.
  */
-static void
+void
 SnapBuildRestoreContents(int fd, char *dest, Size size, const char *path)
 {
 	int			readBytes;
diff --git a/src/include/port/pg_crc32c.h b/src/include/port/pg_crc32c.h
index 63c8e3a00b..cfc8c07944 100644
--- a/src/include/port/pg_crc32c.h
+++ b/src/include/port/pg_crc32c.h
@@ -47,7 +47,7 @@ typedef uint32 pg_crc32c;
 	((crc) = pg_comp_crc32c_sse42((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
 
 #elif defined(USE_ARMV8_CRC32C)
 /* Use ARMv8 CRC Extension instructions. */
@@ -56,7 +56,7 @@ extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t le
 	((crc) = pg_comp_crc32c_armv8((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
 
 #elif defined(USE_LOONGARCH_CRC32C)
 /* Use LoongArch CRCC instructions. */
@@ -65,7 +65,7 @@ extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t le
 	((crc) = pg_comp_crc32c_loongarch((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len);
 
 #elif defined(USE_SSE42_CRC32C_WITH_RUNTIME_CHECK) || defined(USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK)
 
@@ -77,14 +77,14 @@ extern pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_
 	((crc) = pg_comp_crc32c((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
-extern pg_crc32c (*pg_comp_crc32c) (pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c (*pg_comp_crc32c) (pg_crc32c crc, const void *data, size_t len);
 
 #ifdef USE_SSE42_CRC32C_WITH_RUNTIME_CHECK
-extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
 #endif
 #ifdef USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK
-extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
 #endif
 
 #else
@@ -103,7 +103,7 @@ extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t le
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 #endif
 
-extern pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
 
 #endif
 
diff --git a/src/include/replication/snapbuild.h b/src/include/replication/snapbuild.h
index caa5113ff8..e844a89882 100644
--- a/src/include/replication/snapbuild.h
+++ b/src/include/replication/snapbuild.h
@@ -15,6 +15,10 @@
 #include "access/xlogdefs.h"
 #include "utils/snapmgr.h"
 
+/*
+ * Please keep SnapBuildStateDesc[] (located in the pg_logicalinspect module)
+ * updated if a change needs to be made to SnapBuildState.
+ */
 typedef enum
 {
 	/*
@@ -46,7 +50,7 @@ typedef enum
 	SNAPBUILD_CONSISTENT = 2,
 } SnapBuildState;
 
-/* forward declare so we don't have to expose the struct to the public */
+/* forward declare so we don't have to include snapbuild_internal.h */
 struct SnapBuild;
 typedef struct SnapBuild SnapBuild;
 
diff --git a/src/include/replication/snapbuild_internal.h b/src/include/replication/snapbuild_internal.h
new file mode 100644
index 0000000000..0e47ddfcb4
--- /dev/null
+++ b/src/include/replication/snapbuild_internal.h
@@ -0,0 +1,204 @@
+/*-------------------------------------------------------------------------
+ *
+ * snapbuild_internal.h
+ *    This file contains declarations for logical decoding utility
+ *    functions for internal use.
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * src/include/replication/snapbuild_internal.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef SNAPBUILD_INTERNAL_H
+#define SNAPBUILD_INTERNAL_H
+
+#include "port/pg_crc32c.h"
+#include "replication/reorderbuffer.h"
+#include "replication/snapbuild.h"
+
+/*
+ * This struct contains the current state of the snapshot building
+ * machinery. It is exposed to the public, so pay attention when changing its
+ * contents.
+ */
+typedef struct SnapBuild
+{
+	/* how far are we along building our first full snapshot */
+	SnapBuildState state;
+
+	/* private memory context used to allocate memory for this module. */
+	MemoryContext context;
+
+	/* all transactions < than this have committed/aborted */
+	TransactionId xmin;
+
+	/* all transactions >= than this are uncommitted */
+	TransactionId xmax;
+
+	/*
+	 * Don't replay commits from an LSN < this LSN. This can be set externally
+	 * but it will also be advanced (never retreat) from within snapbuild.c.
+	 */
+	XLogRecPtr	start_decoding_at;
+
+	/*
+	 * LSN at which two-phase decoding was enabled or LSN at which we found a
+	 * consistent point at the time of slot creation.
+	 *
+	 * The prepared transactions, that were skipped because previously
+	 * two-phase was not enabled or are not covered by initial snapshot, need
+	 * to be sent later along with commit prepared and they must be before
+	 * this point.
+	 */
+	XLogRecPtr	two_phase_at;
+
+	/*
+	 * Don't start decoding WAL until the "xl_running_xacts" information
+	 * indicates there are no running xids with an xid smaller than this.
+	 */
+	TransactionId initial_xmin_horizon;
+
+	/* Indicates if we are building full snapshot or just catalog one. */
+	bool		building_full_snapshot;
+
+	/*
+	 * Indicates if we are using the snapshot builder for the creation of a
+	 * logical replication slot. If it's true, the start point for decoding
+	 * changes is not determined yet. So we skip snapshot restores to properly
+	 * find the start point. See SnapBuildFindSnapshot() for details.
+	 */
+	bool		in_slot_creation;
+
+	/*
+	 * Snapshot that's valid to see the catalog state seen at this moment.
+	 */
+	Snapshot	snapshot;
+
+	/*
+	 * LSN of the last location we are sure a snapshot has been serialized to.
+	 */
+	XLogRecPtr	last_serialized_snapshot;
+
+	/*
+	 * The reorderbuffer we need to update with usable snapshots et al.
+	 */
+	ReorderBuffer *reorder;
+
+	/*
+	 * TransactionId at which the next phase of initial snapshot building will
+	 * happen. InvalidTransactionId if not known (i.e. SNAPBUILD_START), or
+	 * when no next phase necessary (SNAPBUILD_CONSISTENT).
+	 */
+	TransactionId next_phase_at;
+
+	/*
+	 * Array of transactions which could have catalog changes that committed
+	 * between xmin and xmax.
+	 */
+	struct
+	{
+		/* number of committed transactions */
+		size_t		xcnt;
+
+		/* available space for committed transactions */
+		size_t		xcnt_space;
+
+		/*
+		 * Until we reach a CONSISTENT state, we record commits of all
+		 * transactions, not just the catalog changing ones. Record when that
+		 * changes so we know we cannot export a snapshot safely anymore.
+		 */
+		bool		includes_all_transactions;
+
+		/*
+		 * Array of committed transactions that have modified the catalog.
+		 *
+		 * As this array is frequently modified we do *not* keep it in
+		 * xidComparator order. Instead we sort the array when building &
+		 * distributing a snapshot.
+		 *
+		 * TODO: It's unclear whether that reasoning has much merit. Every
+		 * time we add something here after becoming consistent will also
+		 * require distributing a snapshot. Storing them sorted would
+		 * potentially also make it easier to purge (but more complicated wrt
+		 * wraparound?). Should be improved if sorting while building the
+		 * snapshot shows up in profiles.
+		 */
+		TransactionId *xip;
+	}			committed;
+
+	/*
+	 * Array of transactions and subtransactions that had modified catalogs
+	 * and were running when the snapshot was serialized.
+	 *
+	 * We normally rely on some WAL record types such as HEAP2_NEW_CID to know
+	 * if the transaction has changed the catalog. But it could happen that
+	 * the logical decoding decodes only the commit record of the transaction
+	 * after restoring the previously serialized snapshot in which case we
+	 * will miss adding the xid to the snapshot and end up looking at the
+	 * catalogs with the wrong snapshot.
+	 *
+	 * Now to avoid the above problem, we serialize the transactions that had
+	 * modified the catalogs and are still running at the time of snapshot
+	 * serialization. We fill this array while restoring the snapshot and then
+	 * refer it while decoding commit to ensure if the xact has modified the
+	 * catalog. We discard this array when all the xids in the list become old
+	 * enough to matter. See SnapBuildPurgeOlderTxn for details.
+	 */
+	struct
+	{
+		/* number of transactions */
+		size_t		xcnt;
+
+		/* This array must be sorted in xidComparator order */
+		TransactionId *xip;
+	}			catchange;
+} SnapBuild;
+
+/* -----------------------------------
+ * Snapshot serialization support
+ * -----------------------------------
+ */
+
+/*
+ * We store current state of struct SnapBuild on disk in the following manner:
+ *
+ * struct SnapBuildOnDisk;
+ * TransactionId * committed.xcnt; (*not xcnt_space*)
+ * TransactionId * catchange.xcnt;
+ *
+ */
+typedef struct SnapBuildOnDisk
+{
+	/* first part of this struct needs to be version independent */
+
+	/* data not covered by checksum */
+	uint32		magic;
+	pg_crc32c	checksum;
+
+	/* data covered by checksum */
+
+	/* version, in case we want to support pg_upgrade */
+	uint32		version;
+	/* how large is the on disk data, excluding the constant sized part */
+	uint32		length;
+
+	/* version dependent part */
+	SnapBuild	builder;
+
+	/* variable amount of TransactionIds follows */
+} SnapBuildOnDisk;
+
+#define SnapBuildOnDiskConstantSize \
+	offsetof(SnapBuildOnDisk, builder)
+#define SnapBuildOnDiskNotChecksummedSize \
+	offsetof(SnapBuildOnDisk, version)
+
+#define SNAPBUILD_MAGIC 0x51A1E001
+#define SNAPBUILD_VERSION 6
+
+extern void SnapBuildRestoreContents(int fd, char *dest, Size size, const char *path);
+
+#endif							/* SNAPBUILD_INTERNAL_H */
-- 
2.34.1

#35Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: shveta malik (#32)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Tue, Sep 24, 2024 at 09:15:31AM +0530, shveta malik wrote:

On Fri, Sep 20, 2024 at 12:22 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Please find attached v8, that:

Thank You for the patch. In one of my tests, I noticed that I got
negative checksum:

postgres=# SELECT * FROM pg_get_logical_snapshot_meta('0/3481F20');
magic | checksum | version
------------+------------+---------
1369563137 | -266346460 | 6

But pg_crc32c is uint32. Is it because we are getting it as
Int32GetDatum(ondisk.checksum) in pg_get_logical_snapshot_meta()?
Instead should it be UInt32GetDatum?

Thanks for the testing.

As the checksum could be > 2^31 - 1, then v9 (just shared up-thread) changes it
to an int8 in the pg_logicalinspect--1.0.sql file. So, to avoid CI failure on
the 32bit build, then v9 is using Int64GetDatum() instead of UInt32GetDatum().

Same goes for below:
values[i++] = Int32GetDatum(ondisk.magic);
values[i++] = Int32GetDatum(ondisk.magic);

The 2 others field (magic and version) are unlikely to be > 2^31 - 1, so v9 is
making use of UInt32GetDatum() and keep int4 in the sql file.

We need to recheck the rest of the fields in the info() function as well.

I think that the pg_get_logical_snapshot_info()'s fields are ok (I did spend some
time to debug CI failing on the 32bit build for some on them before submitting v1).

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

#36Peter Smith
smithpb2250@gmail.com
In reply to: Bertrand Drouvot (#34)
Re: Add contrib/pg_logicalsnapinspect

On Wed, Sep 25, 2024 at 2:51 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Hi,

On Mon, Sep 23, 2024 at 12:27:27PM +1000, Peter Smith wrote:

My review comments for v8-0001

======
contrib/pg_logicalinspect/pg_logicalinspect.c

1.
+/*
+ * Lookup table for SnapBuildState.
+ */
+
+#define SNAPBUILD_STATE_INCR 1
+
+static const char *const SnapBuildStateDesc[] = {
+ [SNAPBUILD_START + SNAPBUILD_STATE_INCR] = "start",
+ [SNAPBUILD_BUILDING_SNAPSHOT + SNAPBUILD_STATE_INCR] = "building",
+ [SNAPBUILD_FULL_SNAPSHOT + SNAPBUILD_STATE_INCR] = "full",
+ [SNAPBUILD_CONSISTENT + SNAPBUILD_STATE_INCR] = "consistent",
+};
+
+/*

nit - the SNAPBUILD_STATE_INCR made this code appear more complicated
than it is. Please take a look at the attachment for an alternative
implementation which includes an explanatory comment. YMMV. Feel free
to ignore it.

Thanks for the feedback!

I like the commment, so added it in v9 attached. OTOH I think that's better
to keep SNAPBUILD_STATE_INCR as those "+1" are all linked and that would be
easy to miss the one in pg_get_logical_snapshot_info() should we change the
increment in the future.

I see SNAPBUILD_STATE_INCR more as an "offset" (to get the lowest enum
value to be at lookup index [0]) than an "increment" (between the enum
values), so I'd be naming that differently. But, maybe I am straying
into just personal opinion instead of giving useful feedback, so let's
say I have no more review comments. Patch v9 looks OK to me.

======
Kind Regards,
Peter Smith.
Fujitsu Australia

#37shveta malik
shveta.malik@gmail.com
In reply to: Bertrand Drouvot (#35)
Re: Add contrib/pg_logicalsnapinspect

On Tue, Sep 24, 2024 at 10:23 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Hi,

On Tue, Sep 24, 2024 at 09:15:31AM +0530, shveta malik wrote:

On Fri, Sep 20, 2024 at 12:22 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Please find attached v8, that:

Thank You for the patch. In one of my tests, I noticed that I got
negative checksum:

postgres=# SELECT * FROM pg_get_logical_snapshot_meta('0/3481F20');
magic | checksum | version
------------+------------+---------
1369563137 | -266346460 | 6

But pg_crc32c is uint32. Is it because we are getting it as
Int32GetDatum(ondisk.checksum) in pg_get_logical_snapshot_meta()?
Instead should it be UInt32GetDatum?

Thanks for the testing.

As the checksum could be > 2^31 - 1, then v9 (just shared up-thread) changes it
to an int8 in the pg_logicalinspect--1.0.sql file. So, to avoid CI failure on
the 32bit build, then v9 is using Int64GetDatum() instead of UInt32GetDatum().

Okay, looks good,

Same goes for below:
values[i++] = Int32GetDatum(ondisk.magic);
values[i++] = Int32GetDatum(ondisk.magic);

The 2 others field (magic and version) are unlikely to be > 2^31 - 1, so v9 is
making use of UInt32GetDatum() and keep int4 in the sql file.

We need to recheck the rest of the fields in the info() function as well.

I think that the pg_get_logical_snapshot_info()'s fields are ok (I did spend some
time to debug CI failing on the 32bit build for some on them before submitting v1).

+ OUT catchange_xip xid[]

One question, what is xid datatype, is it too int8? Sorry, could not
find the correct doc. Since we are getting uint32 in Int64, this also
needs to be accordingly.

thanks
Shveta

#38Peter Eisentraut
peter@eisentraut.org
In reply to: Bertrand Drouvot (#34)
Re: Add contrib/pg_logicalsnapinspect

Is there a reason for this elaborate error handling:

+	fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
+
+	if (fd < 0 && errno == ENOENT)
+		ereport(ERROR,
+				errmsg("file \"%s\" does not exist", path));
+	else if (fd < 0)
+		ereport(ERROR,
+				(errcode_for_file_access(),
+				 errmsg("could not open file \"%s\": %m", path)));

Couldn't you just use the second branch for all errno's?

#39Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: Peter Eisentraut (#38)
1 attachment(s)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Wed, Sep 25, 2024 at 04:04:43PM +0200, Peter Eisentraut wrote:

Is there a reason for this elaborate error handling:

Thanks for looking at it!

+	fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
+
+	if (fd < 0 && errno == ENOENT)
+		ereport(ERROR,
+				errmsg("file \"%s\" does not exist", path));
+	else if (fd < 0)
+		ereport(ERROR,
+				(errcode_for_file_access(),
+				 errmsg("could not open file \"%s\": %m", path)));

Couldn't you just use the second branch for all errno's?

Yeah, I think it comes from copying/pasting from SnapBuildRestore() too "quickly".
v10 attached uses the second branch only.

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachments:

v10-0001-Add-contrib-pg_logicalinspect.patchtext/x-diff; charset=us-asciiDownload
From a093c768f1b3a70c80046742d7a264bee5d4d205 Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Wed, 14 Aug 2024 08:46:05 +0000
Subject: [PATCH v10] Add contrib/pg_logicalinspect

Provides SQL functions that allow to inspect logical decoding components.

It currently allows to inspect the contents of serialized logical snapshots of
a running database cluster, which is useful for debugging or educational
purposes.
---
 contrib/Makefile                              |   1 +
 contrib/meson.build                           |   1 +
 contrib/pg_logicalinspect/.gitignore          |   4 +
 contrib/pg_logicalinspect/Makefile            |  31 +++
 .../expected/logical_inspect.out              |  52 ++++
 contrib/pg_logicalinspect/logicalinspect.conf |   1 +
 contrib/pg_logicalinspect/meson.build         |  39 +++
 .../pg_logicalinspect--1.0.sql                |  43 +++
 contrib/pg_logicalinspect/pg_logicalinspect.c | 262 ++++++++++++++++++
 .../pg_logicalinspect.control                 |   5 +
 .../specs/logical_inspect.spec                |  34 +++
 doc/src/sgml/contrib.sgml                     |   1 +
 doc/src/sgml/filelist.sgml                    |   1 +
 doc/src/sgml/pglogicalinspect.sgml            | 151 ++++++++++
 src/backend/replication/logical/snapbuild.c   | 190 +------------
 src/include/port/pg_crc32c.h                  |  16 +-
 src/include/replication/snapbuild.h           |   6 +-
 src/include/replication/snapbuild_internal.h  | 204 ++++++++++++++
 18 files changed, 849 insertions(+), 193 deletions(-)
   7.8% contrib/pg_logicalinspect/expected/
   5.7% contrib/pg_logicalinspect/specs/
  32.9% contrib/pg_logicalinspect/
  13.6% doc/src/sgml/
  16.9% src/backend/replication/logical/
   4.0% src/include/port/
  18.7% src/include/replication/

diff --git a/contrib/Makefile b/contrib/Makefile
index abd780f277..952855d9b6 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -32,6 +32,7 @@ SUBDIRS = \
 		passwordcheck	\
 		pg_buffercache	\
 		pg_freespacemap \
+		pg_logicalinspect \
 		pg_prewarm	\
 		pg_stat_statements \
 		pg_surgery	\
diff --git a/contrib/meson.build b/contrib/meson.build
index 14a8906865..159ff41555 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -46,6 +46,7 @@ subdir('passwordcheck')
 subdir('pg_buffercache')
 subdir('pgcrypto')
 subdir('pg_freespacemap')
+subdir('pg_logicalinspect')
 subdir('pg_prewarm')
 subdir('pgrowlocks')
 subdir('pg_stat_statements')
diff --git a/contrib/pg_logicalinspect/.gitignore b/contrib/pg_logicalinspect/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/contrib/pg_logicalinspect/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/contrib/pg_logicalinspect/Makefile b/contrib/pg_logicalinspect/Makefile
new file mode 100644
index 0000000000..55124514d4
--- /dev/null
+++ b/contrib/pg_logicalinspect/Makefile
@@ -0,0 +1,31 @@
+# contrib/pg_logicalinspect/Makefile
+
+MODULE_big = pg_logicalinspect
+OBJS = \
+	$(WIN32RES) \
+	pg_logicalinspect.o
+PGFILEDESC = "pg_logicalinspect - functions to inspect logical decoding components"
+
+EXTENSION = pg_logicalinspect
+DATA = pg_logicalinspect--1.0.sql
+
+EXTRA_INSTALL = contrib/test_decoding
+
+ISOLATION = logical_inspect
+
+ISOLATION_OPTS = --temp-config $(top_srcdir)/contrib/pg_logicalinspect/logicalinspect.conf
+
+# Disabled because these tests require "wal_level=logical", which
+# some installcheck users do not have (e.g. buildfarm clients).
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/pg_logicalinspect
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/pg_logicalinspect/expected/logical_inspect.out b/contrib/pg_logicalinspect/expected/logical_inspect.out
new file mode 100644
index 0000000000..08de273197
--- /dev/null
+++ b/contrib/pg_logicalinspect/expected/logical_inspect.out
@@ -0,0 +1,52 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s0_init s0_begin s0_savepoint s0_truncate s1_checkpoint s1_get_changes s0_commit s0_begin s0_insert s1_checkpoint s1_get_changes s0_commit s1_get_changes s1_get_logical_snapshot_info s1_get_logical_snapshot_meta
+step s0_init: SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding');
+?column?
+--------
+init    
+(1 row)
+
+step s0_begin: BEGIN;
+step s0_savepoint: SAVEPOINT sp1;
+step s0_truncate: TRUNCATE tbl1;
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data
+----
+(0 rows)
+
+step s0_commit: COMMIT;
+step s0_begin: BEGIN;
+step s0_insert: INSERT INTO tbl1 VALUES (1);
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                   
+---------------------------------------
+BEGIN                                  
+table public.tbl1: TRUNCATE: (no-flags)
+COMMIT                                 
+(3 rows)
+
+step s0_commit: COMMIT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                                         
+-------------------------------------------------------------
+BEGIN                                                        
+table public.tbl1: INSERT: val1[integer]:1 val2[integer]:null
+COMMIT                                                       
+(3 rows)
+
+step s1_get_logical_snapshot_info: SELECT (pg_get_logical_snapshot_info(f.name::pg_lsn)).state,(pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_xip,1) AS catchange_array_length,(pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_xip,1) AS committed_array_length FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f ORDER BY 2;
+state     |catchange_count|catchange_array_length|committed_count|committed_array_length
+----------+---------------+----------------------+---------------+----------------------
+consistent|              0|                      |              2|                     2
+consistent|              2|                     2|              0|                      
+(2 rows)
+
+step s1_get_logical_snapshot_meta: SELECT COUNT((pg_get_logical_snapshot_meta(f.name::pg_lsn))) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f;
+count
+-----
+    2
+(1 row)
+
diff --git a/contrib/pg_logicalinspect/logicalinspect.conf b/contrib/pg_logicalinspect/logicalinspect.conf
new file mode 100644
index 0000000000..e3d257315f
--- /dev/null
+++ b/contrib/pg_logicalinspect/logicalinspect.conf
@@ -0,0 +1 @@
+wal_level = logical
diff --git a/contrib/pg_logicalinspect/meson.build b/contrib/pg_logicalinspect/meson.build
new file mode 100644
index 0000000000..3ec635509b
--- /dev/null
+++ b/contrib/pg_logicalinspect/meson.build
@@ -0,0 +1,39 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+pg_logicalinspect_sources = files('pg_logicalinspect.c')
+
+if host_system == 'windows'
+  pg_logicalinspect_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+    '--NAME', 'pg_logicalinspect',
+    '--FILEDESC', 'pg_logicalinspect - functions to inspect logical decoding components',])
+endif
+
+pg_logicalinspect = shared_module('pg_logicalinspect',
+  pg_logicalinspect_sources,
+  kwargs: contrib_mod_args + {
+      'dependencies': contrib_mod_args['dependencies'],
+  },
+)
+contrib_targets += pg_logicalinspect
+
+install_data(
+  'pg_logicalinspect.control',
+  'pg_logicalinspect--1.0.sql',
+  kwargs: contrib_data_args,
+)
+
+tests += {
+  'name': 'pg_logicalinspect',
+  'sd': meson.current_source_dir(),
+  'bd': meson.current_build_dir(),
+  'isolation': {
+    'specs': [
+      'logical_inspect',
+    ],
+    'regress_args': [
+      '--temp-config', files('logicalinspect.conf'),
+    ],
+    # see above
+    'runningcheck': false,
+  },
+}
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql b/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
new file mode 100644
index 0000000000..fc06e428ce
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
@@ -0,0 +1,43 @@
+/* contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_logicalinspect" to load this file. \quit
+
+--
+-- pg_get_logical_snapshot_meta()
+--
+CREATE FUNCTION pg_get_logical_snapshot_meta(IN in_lsn pg_lsn,
+    OUT magic int4,
+    OUT checksum int8,
+    OUT version int4
+)
+AS 'MODULE_PATHNAME', 'pg_get_logical_snapshot_meta'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_logical_snapshot_meta(pg_lsn) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_logical_snapshot_meta(pg_lsn) TO pg_read_server_files;
+
+--
+-- pg_get_logical_snapshot_info()
+--
+CREATE FUNCTION pg_get_logical_snapshot_info(IN in_lsn pg_lsn,
+    OUT state text,
+    OUT xmin xid,
+    OUT xmax xid,
+    OUT start_decoding_at pg_lsn,
+    OUT two_phase_at pg_lsn,
+    OUT initial_xmin_horizon xid,
+    OUT building_full_snapshot boolean,
+    OUT in_slot_creation boolean,
+    OUT last_serialized_snapshot pg_lsn,
+    OUT next_phase_at xid,
+    OUT committed_count int8,
+    OUT committed_xip xid[],
+    OUT catchange_count int8,
+    OUT catchange_xip xid[]
+)
+AS 'MODULE_PATHNAME', 'pg_get_logical_snapshot_info'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_logical_snapshot_info(pg_lsn) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_logical_snapshot_info(pg_lsn) TO pg_read_server_files;
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.c b/contrib/pg_logicalinspect/pg_logicalinspect.c
new file mode 100644
index 0000000000..4460f07a36
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.c
@@ -0,0 +1,262 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_logicalinspect.c
+ *		  Functions to inspect contents of PostgreSQL logical snapshots
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ *		  contrib/pg_logicalinspect/pg_logicalinspect.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "funcapi.h"
+#include "replication/snapbuild_internal.h"
+#include "utils/array.h"
+#include "utils/builtins.h"
+#include "utils/pg_lsn.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_meta);
+PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_info);
+
+/*
+ * Lookup table for SnapBuildState. The lookup index is offset by 1
+ * because the consecutive SnapBuildState enum values start at -1.
+ */
+#define SNAPBUILD_STATE_INCR 1
+
+static const char *const SnapBuildStateDesc[] = {
+	[SNAPBUILD_START + SNAPBUILD_STATE_INCR] = "start",
+	[SNAPBUILD_BUILDING_SNAPSHOT + SNAPBUILD_STATE_INCR] = "building",
+	[SNAPBUILD_FULL_SNAPSHOT + SNAPBUILD_STATE_INCR] = "full",
+	[SNAPBUILD_CONSISTENT + SNAPBUILD_STATE_INCR] = "consistent",
+};
+
+/*
+ * NOTE: For any code change or issue fix here, it is highly recommended to
+ * give a thought about doing the same in SnapBuildRestore() as well.
+ */
+
+/*
+ * Validate the logical snapshot file and read its contents to 'ondisk'.
+ */
+static void
+ValidateAndRestoreSnapshotFile(XLogRecPtr lsn, SnapBuildOnDisk *ondisk,
+							   const char *path)
+{
+	int			fd;
+	Size		sz;
+	pg_crc32c	checksum;
+	MemoryContext context;
+
+	context = AllocSetContextCreate(CurrentMemoryContext,
+									"logicalsnapshot inspect context",
+									ALLOCSET_DEFAULT_SIZES);
+
+	fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
+
+	if (fd < 0)
+		ereport(ERROR,
+				(errcode_for_file_access(),
+				 errmsg("could not open file \"%s\": %m", path)));
+
+	/* ----
+	 * Make sure the snapshot had been stored safely to disk, that's normally
+	 * cheap.
+	 * Note that we do not need PANIC here, nobody will be able to use the
+	 * slot without fsyncing, and saving it won't succeed without an fsync()
+	 * either...
+	 * ----
+	 */
+	fsync_fname(path, false);
+	fsync_fname(PG_LOGICAL_SNAPSHOTS_DIR, true);
+
+	/* read statically sized portion of snapshot */
+	SnapBuildRestoreContents(fd, (char *) ondisk, SnapBuildOnDiskConstantSize, path);
+
+	if (ondisk->magic != SNAPBUILD_MAGIC)
+		ereport(ERROR,
+				(errcode(ERRCODE_DATA_CORRUPTED),
+				 errmsg("snapbuild state file \"%s\" has wrong magic number: %u instead of %u",
+						path, ondisk->magic, SNAPBUILD_MAGIC)));
+
+	if (ondisk->version != SNAPBUILD_VERSION)
+		ereport(ERROR,
+				(errcode(ERRCODE_DATA_CORRUPTED),
+				 errmsg("snapbuild state file \"%s\" has unsupported version: %u instead of %u",
+						path, ondisk->version, SNAPBUILD_VERSION)));
+
+	INIT_CRC32C(checksum);
+	COMP_CRC32C(checksum,
+				((char *) ondisk) + SnapBuildOnDiskNotChecksummedSize,
+				SnapBuildOnDiskConstantSize - SnapBuildOnDiskNotChecksummedSize);
+
+	/* read SnapBuild */
+	SnapBuildRestoreContents(fd, (char *) &ondisk->builder, sizeof(SnapBuild), path);
+	COMP_CRC32C(checksum, &ondisk->builder, sizeof(SnapBuild));
+
+	ondisk->builder.context = context;
+
+	/* restore committed xacts information */
+	if (ondisk->builder.committed.xcnt > 0)
+	{
+		sz = sizeof(TransactionId) * ondisk->builder.committed.xcnt;
+		ondisk->builder.committed.xip = MemoryContextAllocZero(ondisk->builder.context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.committed.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.committed.xip, sz);
+	}
+
+	/* restore catalog modifying xacts information */
+	if (ondisk->builder.catchange.xcnt > 0)
+	{
+		sz = sizeof(TransactionId) * ondisk->builder.catchange.xcnt;
+		ondisk->builder.catchange.xip = MemoryContextAllocZero(ondisk->builder.context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.catchange.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.catchange.xip, sz);
+	}
+
+	if (CloseTransientFile(fd) != 0)
+		ereport(ERROR,
+				(errcode_for_file_access(),
+				 errmsg("could not close file \"%s\": %m", path)));
+
+	FIN_CRC32C(checksum);
+
+	/* verify checksum of what we've read */
+	if (!EQ_CRC32C(checksum, ondisk->checksum))
+		ereport(ERROR,
+				(errcode(ERRCODE_DATA_CORRUPTED),
+				 errmsg("checksum mismatch for snapbuild state file \"%s\": is %u, should be %u",
+						path, checksum, ondisk->checksum)));
+}
+
+/*
+ * Retrieve the logical snapshot file metadata.
+ */
+Datum
+pg_get_logical_snapshot_meta(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_META_COLS 3
+	SnapBuildOnDisk ondisk;
+	XLogRecPtr	lsn;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_META_COLS];
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_META_COLS];
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+	int			i = 0;
+
+	lsn = PG_GETARG_LSN(0);
+
+	sprintf(path, "%s/%X-%X.snap",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			LSN_FORMAT_ARGS(lsn));
+
+	ValidateAndRestoreSnapshotFile(lsn, &ondisk, path);
+
+	/* Build a tuple descriptor for our result type. */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	memset(nulls, 0, sizeof(nulls));
+
+	values[i++] = UInt32GetDatum(ondisk.magic);
+	values[i++] = Int64GetDatum((int64) ondisk.checksum);
+	values[i++] = UInt32GetDatum(ondisk.version);
+
+	Assert(i == PG_GET_LOGICAL_SNAPSHOT_META_COLS);
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	MemoryContextReset(ondisk.builder.context);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_META_COLS
+}
+
+Datum
+pg_get_logical_snapshot_info(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_INFO_COLS 14
+	SnapBuildOnDisk ondisk;
+	XLogRecPtr	lsn;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS];
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS];
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+	int			i = 0;
+
+	lsn = PG_GETARG_LSN(0);
+
+	sprintf(path, "%s/%X-%X.snap",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			LSN_FORMAT_ARGS(lsn));
+
+	ValidateAndRestoreSnapshotFile(lsn, &ondisk, path);
+
+	/* Build a tuple descriptor for our result type. */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	memset(nulls, 0, sizeof(nulls));
+
+	values[i++] = CStringGetTextDatum(SnapBuildStateDesc[ondisk.builder.state +
+														 SNAPBUILD_STATE_INCR]);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.xmin);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.xmax);
+	values[i++] = LSNGetDatum(ondisk.builder.start_decoding_at);
+	values[i++] = LSNGetDatum(ondisk.builder.two_phase_at);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.initial_xmin_horizon);
+	values[i++] = BoolGetDatum(ondisk.builder.building_full_snapshot);
+	values[i++] = BoolGetDatum(ondisk.builder.in_slot_creation);
+	values[i++] = LSNGetDatum(ondisk.builder.last_serialized_snapshot);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.next_phase_at);
+
+	values[i++] = Int64GetDatum(ondisk.builder.committed.xcnt);
+	if (ondisk.builder.committed.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+		int			narrayelems = 0;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.committed.xcnt * sizeof(Datum));
+
+		for (; narrayelems < ondisk.builder.committed.xcnt; narrayelems++)
+			arrayelems[narrayelems] = Int64GetDatum((int64) ondisk.builder.committed.xip[narrayelems]);
+
+		values[i++] = PointerGetDatum(construct_array_builtin(arrayelems, narrayelems, INT8OID));
+	}
+	else
+		nulls[i++] = true;
+
+	values[i++] = Int64GetDatum(ondisk.builder.catchange.xcnt);
+	if (ondisk.builder.catchange.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+		int			narrayelems = 0;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.catchange.xcnt * sizeof(Datum));
+
+		for (; narrayelems < ondisk.builder.catchange.xcnt; narrayelems++)
+			arrayelems[narrayelems] = Int64GetDatum((int64) ondisk.builder.catchange.xip[narrayelems]);
+
+		values[i++] = PointerGetDatum(construct_array_builtin(arrayelems, narrayelems, INT8OID));
+	}
+	else
+		nulls[i++] = true;
+
+	Assert(i == PG_GET_LOGICAL_SNAPSHOT_INFO_COLS);
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	MemoryContextReset(ondisk.builder.context);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_INFO_COLS
+}
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.control b/contrib/pg_logicalinspect/pg_logicalinspect.control
new file mode 100644
index 0000000000..b4a70e57ba
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.control
@@ -0,0 +1,5 @@
+# pg_logicalinspect extension
+comment = 'functions to inspect logical decoding components'
+default_version = '1.0'
+module_pathname = '$libdir/pg_logicalinspect'
+relocatable = true
diff --git a/contrib/pg_logicalinspect/specs/logical_inspect.spec b/contrib/pg_logicalinspect/specs/logical_inspect.spec
new file mode 100644
index 0000000000..47641163fe
--- /dev/null
+++ b/contrib/pg_logicalinspect/specs/logical_inspect.spec
@@ -0,0 +1,34 @@
+# Test the pg_logicalinspect functions: that needs some permutation to
+# ensure that we are creating multiple logical snapshots and that one of them
+# contains ongoing catalogs changes.
+setup
+{
+    DROP TABLE IF EXISTS tbl1;
+    CREATE TABLE tbl1 (val1 integer, val2 integer);
+    CREATE EXTENSION pg_logicalinspect;
+}
+
+teardown
+{
+    DROP TABLE tbl1;
+    SELECT 'stop' FROM pg_drop_replication_slot('isolation_slot');
+    DROP EXTENSION pg_logicalinspect;
+}
+
+session "s0"
+setup { SET synchronous_commit=on; }
+step "s0_init" { SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding'); }
+step "s0_begin" { BEGIN; }
+step "s0_savepoint" { SAVEPOINT sp1; }
+step "s0_truncate" { TRUNCATE tbl1; }
+step "s0_insert" { INSERT INTO tbl1 VALUES (1); }
+step "s0_commit" { COMMIT; }
+
+session "s1"
+setup { SET synchronous_commit=on; }
+step "s1_checkpoint" { CHECKPOINT; }
+step "s1_get_changes" { SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0'); }
+step "s1_get_logical_snapshot_meta" { SELECT COUNT((pg_get_logical_snapshot_meta(f.name::pg_lsn))) FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f; }
+step "s1_get_logical_snapshot_info" { SELECT (pg_get_logical_snapshot_info(f.name::pg_lsn)).state,(pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).catchange_xip,1) AS catchange_array_length,(pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_count,array_length((pg_get_logical_snapshot_info(f.name::pg_lsn)).committed_xip,1) AS committed_array_length FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name FROM pg_ls_logicalsnapdir()) AS f ORDER BY 2; }
+
+permutation "s0_init" "s0_begin" "s0_savepoint" "s0_truncate" "s1_checkpoint" "s1_get_changes" "s0_commit" "s0_begin" "s0_insert" "s1_checkpoint" "s1_get_changes" "s0_commit" "s1_get_changes" "s1_get_logical_snapshot_info" "s1_get_logical_snapshot_meta"
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index 44639a8dca..7c381949a5 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -154,6 +154,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
  &pgbuffercache;
  &pgcrypto;
  &pgfreespacemap;
+ &pglogicalinspect;
  &pgprewarm;
  &pgrowlocks;
  &pgstatstatements;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index a7ff5f8264..66e6dccd4c 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -143,6 +143,7 @@
 <!ENTITY pgbuffercache   SYSTEM "pgbuffercache.sgml">
 <!ENTITY pgcrypto        SYSTEM "pgcrypto.sgml">
 <!ENTITY pgfreespacemap  SYSTEM "pgfreespacemap.sgml">
+<!ENTITY pglogicalinspect  SYSTEM "pglogicalinspect.sgml">
 <!ENTITY pgprewarm       SYSTEM "pgprewarm.sgml">
 <!ENTITY pgrowlocks      SYSTEM "pgrowlocks.sgml">
 <!ENTITY pgstatstatements SYSTEM "pgstatstatements.sgml">
diff --git a/doc/src/sgml/pglogicalinspect.sgml b/doc/src/sgml/pglogicalinspect.sgml
new file mode 100644
index 0000000000..f0b26637d0
--- /dev/null
+++ b/doc/src/sgml/pglogicalinspect.sgml
@@ -0,0 +1,151 @@
+<!-- doc/src/sgml/pglogicalinspect.sgml -->
+
+<sect1 id="pglogicalinspect" xreflabel="pg_logicalinspect">
+ <title>pg_logicalinspect &mdash; logical decoding components inspection</title>
+
+ <indexterm zone="pglogicalinspect">
+  <primary>pg_logicalinspect</primary>
+ </indexterm>
+
+ <para>
+  The <filename>pg_logicalinspect</filename> module provides SQL functions
+  that allow you to inspect the contents of logical decoding components. It
+  allows the inspection of serialized logical snapshots of a running
+  <productname>PostgreSQL</productname> database cluster, which is useful
+  for debugging or educational purposes.
+ </para>
+
+ <note>
+  <para>
+   The <filename>pg_logicalinspect</filename> functions are called
+   using an LSN argument that can be extracted from the output name of the
+   <function>pg_ls_logicalsnapdir</function>() function.
+  </para>
+ </note>
+
+ <para>
+  By default, use of these functions is restricted to superusers and members of
+  the <literal>pg_read_server_files</literal> role. Access may be granted by
+  superusers to others using <command>GRANT</command>.
+ </para>
+
+ <sect2 id="pglogicalinspect-funcs">
+  <title>General Functions</title>
+
+  <variablelist>
+   <varlistentry id="pglogicalinspect-funcs-pg-get-logical-snapshot-meta">
+    <term>
+     <function>pg_get_logical_snapshot_meta(in_lsn pg_lsn) returns record</function>
+    </term>
+
+    <listitem>
+     <para>
+      Gets logical snapshot metadata about a snapshot file that is located in
+      the server's <filename>pg_logical/snapshots</filename> directory.
+      The <replaceable>in_lsn</replaceable> argument can be extracted from the
+      snapshot file name.
+      For example:
+<screen>
+postgres=# SELECT * FROM pg_ls_logicalsnapdir();
+-[ RECORD 1 ]+-----------------------
+name         | 0-40796E18.snap
+size         | 152
+modification | 2024-08-14 16:36:32+00
+
+postgres=# SELECT * FROM pg_get_logical_snapshot_meta('0/40796E18');
+-[ RECORD 1 ]--------
+magic    | 1369563137
+checksum | 1028045905
+version  | 6
+
+postgres=# SELECT (pg_get_logical_snapshot_meta(f.name::pg_lsn)).*
+           FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name
+                 FROM pg_ls_logicalsnapdir()) AS f;
+-[ RECORD 1 ]--------
+magic    | 1369563137
+checksum | 1028045905
+version  | 6
+</screen>
+     </para>
+     <para>
+      If <replaceable>in_lsn</replaceable> does not match a snapshot file, the
+      function raises an error.
+     </para>
+    </listitem>
+   </varlistentry>
+
+   <varlistentry id="pglogicalinspect-funcs-pg-get-logical-snapshot-info">
+    <term>
+     <function>pg_get_logical_snapshot_info(in_lsn pg_lsn) returns record</function>
+    </term>
+
+    <listitem>
+     <para>
+      Gets logical snapshot information about a snapshot file that is located in
+      the server's <filename>pg_logical/snapshots</filename> directory.
+      The <replaceable>in_lsn</replaceable> argument can be extracted from the
+      snapshot file name.
+      For example:
+<screen>
+postgres=# SELECT * FROM pg_ls_logicalsnapdir();
+-[ RECORD 1 ]+-----------------------
+name         | 0-40796E18.snap
+size         | 152
+modification | 2024-08-14 16:36:32+00
+
+postgres=# SELECT * FROM pg_get_logical_snapshot_info('0/40796E18');
+-[ RECORD 1 ]------------+-----------
+state                    | consistent
+xmin                     | 751
+xmax                     | 751
+start_decoding_at        | 0/40796AF8
+two_phase_at             | 0/40796AF8
+initial_xmin_horizon     | 0
+building_full_snapshot   | f
+in_slot_creation         | f
+last_serialized_snapshot | 0/0
+next_phase_at            | 0
+committed_count          | 0
+committed_xip            |
+catchange_count          | 2
+catchange_xip            | {751,752}
+
+postgres=# SELECT (pg_get_logical_snapshot_info(f.name::pg_lsn)).*
+           FROM (SELECT replace(replace(name,'.snap',''),'-','/') AS name
+                 FROM pg_ls_logicalsnapdir()) AS f;
+-[ RECORD 1 ]------------+-----------
+state                    | consistent
+xmin                     | 751
+xmax                     | 751
+start_decoding_at        | 0/40796AF8
+two_phase_at             | 0/40796AF8
+initial_xmin_horizon     | 0
+building_full_snapshot   | f
+in_slot_creation         | f
+last_serialized_snapshot | 0/0
+next_phase_at            | 0
+committed_count          | 0
+committed_xip            |
+catchange_count          | 2
+catchange_xip            | {751,752}
+</screen>
+     </para>
+     <para>
+      If <replaceable>in_lsn</replaceable> does not match a snapshot file, the
+      function raises an error.
+     </para>
+    </listitem>
+   </varlistentry>
+
+  </variablelist>
+ </sect2>
+
+ <sect2 id="pglogicalinspect-author">
+  <title>Author</title>
+
+  <para>
+   Bertrand Drouvot <email>bertranddrouvot.pg@gmail.com</email>
+  </para>
+ </sect2>
+
+</sect1>
diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c
index 0450f94ba8..bb7def9440 100644
--- a/src/backend/replication/logical/snapbuild.c
+++ b/src/backend/replication/logical/snapbuild.c
@@ -134,6 +134,7 @@
 #include "replication/logical.h"
 #include "replication/reorderbuffer.h"
 #include "replication/snapbuild.h"
+#include "replication/snapbuild_internal.h"
 #include "storage/fd.h"
 #include "storage/lmgr.h"
 #include "storage/proc.h"
@@ -143,146 +144,6 @@
 #include "utils/memutils.h"
 #include "utils/snapmgr.h"
 #include "utils/snapshot.h"
-
-/*
- * This struct contains the current state of the snapshot building
- * machinery. Besides a forward declaration in the header, it is not exposed
- * to the public, so we can easily change its contents.
- */
-struct SnapBuild
-{
-	/* how far are we along building our first full snapshot */
-	SnapBuildState state;
-
-	/* private memory context used to allocate memory for this module. */
-	MemoryContext context;
-
-	/* all transactions < than this have committed/aborted */
-	TransactionId xmin;
-
-	/* all transactions >= than this are uncommitted */
-	TransactionId xmax;
-
-	/*
-	 * Don't replay commits from an LSN < this LSN. This can be set externally
-	 * but it will also be advanced (never retreat) from within snapbuild.c.
-	 */
-	XLogRecPtr	start_decoding_at;
-
-	/*
-	 * LSN at which two-phase decoding was enabled or LSN at which we found a
-	 * consistent point at the time of slot creation.
-	 *
-	 * The prepared transactions, that were skipped because previously
-	 * two-phase was not enabled or are not covered by initial snapshot, need
-	 * to be sent later along with commit prepared and they must be before
-	 * this point.
-	 */
-	XLogRecPtr	two_phase_at;
-
-	/*
-	 * Don't start decoding WAL until the "xl_running_xacts" information
-	 * indicates there are no running xids with an xid smaller than this.
-	 */
-	TransactionId initial_xmin_horizon;
-
-	/* Indicates if we are building full snapshot or just catalog one. */
-	bool		building_full_snapshot;
-
-	/*
-	 * Indicates if we are using the snapshot builder for the creation of a
-	 * logical replication slot. If it's true, the start point for decoding
-	 * changes is not determined yet. So we skip snapshot restores to properly
-	 * find the start point. See SnapBuildFindSnapshot() for details.
-	 */
-	bool		in_slot_creation;
-
-	/*
-	 * Snapshot that's valid to see the catalog state seen at this moment.
-	 */
-	Snapshot	snapshot;
-
-	/*
-	 * LSN of the last location we are sure a snapshot has been serialized to.
-	 */
-	XLogRecPtr	last_serialized_snapshot;
-
-	/*
-	 * The reorderbuffer we need to update with usable snapshots et al.
-	 */
-	ReorderBuffer *reorder;
-
-	/*
-	 * TransactionId at which the next phase of initial snapshot building will
-	 * happen. InvalidTransactionId if not known (i.e. SNAPBUILD_START), or
-	 * when no next phase necessary (SNAPBUILD_CONSISTENT).
-	 */
-	TransactionId next_phase_at;
-
-	/*
-	 * Array of transactions which could have catalog changes that committed
-	 * between xmin and xmax.
-	 */
-	struct
-	{
-		/* number of committed transactions */
-		size_t		xcnt;
-
-		/* available space for committed transactions */
-		size_t		xcnt_space;
-
-		/*
-		 * Until we reach a CONSISTENT state, we record commits of all
-		 * transactions, not just the catalog changing ones. Record when that
-		 * changes so we know we cannot export a snapshot safely anymore.
-		 */
-		bool		includes_all_transactions;
-
-		/*
-		 * Array of committed transactions that have modified the catalog.
-		 *
-		 * As this array is frequently modified we do *not* keep it in
-		 * xidComparator order. Instead we sort the array when building &
-		 * distributing a snapshot.
-		 *
-		 * TODO: It's unclear whether that reasoning has much merit. Every
-		 * time we add something here after becoming consistent will also
-		 * require distributing a snapshot. Storing them sorted would
-		 * potentially also make it easier to purge (but more complicated wrt
-		 * wraparound?). Should be improved if sorting while building the
-		 * snapshot shows up in profiles.
-		 */
-		TransactionId *xip;
-	}			committed;
-
-	/*
-	 * Array of transactions and subtransactions that had modified catalogs
-	 * and were running when the snapshot was serialized.
-	 *
-	 * We normally rely on some WAL record types such as HEAP2_NEW_CID to know
-	 * if the transaction has changed the catalog. But it could happen that
-	 * the logical decoding decodes only the commit record of the transaction
-	 * after restoring the previously serialized snapshot in which case we
-	 * will miss adding the xid to the snapshot and end up looking at the
-	 * catalogs with the wrong snapshot.
-	 *
-	 * Now to avoid the above problem, we serialize the transactions that had
-	 * modified the catalogs and are still running at the time of snapshot
-	 * serialization. We fill this array while restoring the snapshot and then
-	 * refer it while decoding commit to ensure if the xact has modified the
-	 * catalog. We discard this array when all the xids in the list become old
-	 * enough to matter. See SnapBuildPurgeOlderTxn for details.
-	 */
-	struct
-	{
-		/* number of transactions */
-		size_t		xcnt;
-
-		/* This array must be sorted in xidComparator order */
-		TransactionId *xip;
-	}			catchange;
-};
-
 /*
  * Starting a transaction -- which we need to do while exporting a snapshot --
  * removes knowledge about the previously used resowner, so we save it here.
@@ -312,7 +173,6 @@ static void SnapBuildWaitSnapshot(xl_running_xacts *running, TransactionId cutof
 /* serialization functions */
 static void SnapBuildSerialize(SnapBuild *builder, XLogRecPtr lsn);
 static bool SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn);
-static void SnapBuildRestoreContents(int fd, char *dest, Size size, const char *path);
 
 /*
  * Allocate a new snapshot builder.
@@ -1557,48 +1417,6 @@ SnapBuildWaitSnapshot(xl_running_xacts *running, TransactionId cutoff)
 	}
 }
 
-/* -----------------------------------
- * Snapshot serialization support
- * -----------------------------------
- */
-
-/*
- * We store current state of struct SnapBuild on disk in the following manner:
- *
- * struct SnapBuildOnDisk;
- * TransactionId * committed.xcnt; (*not xcnt_space*)
- * TransactionId * catchange.xcnt;
- *
- */
-typedef struct SnapBuildOnDisk
-{
-	/* first part of this struct needs to be version independent */
-
-	/* data not covered by checksum */
-	uint32		magic;
-	pg_crc32c	checksum;
-
-	/* data covered by checksum */
-
-	/* version, in case we want to support pg_upgrade */
-	uint32		version;
-	/* how large is the on disk data, excluding the constant sized part */
-	uint32		length;
-
-	/* version dependent part */
-	SnapBuild	builder;
-
-	/* variable amount of TransactionIds follows */
-} SnapBuildOnDisk;
-
-#define SnapBuildOnDiskConstantSize \
-	offsetof(SnapBuildOnDisk, builder)
-#define SnapBuildOnDiskNotChecksummedSize \
-	offsetof(SnapBuildOnDisk, version)
-
-#define SNAPBUILD_MAGIC 0x51A1E001
-#define SNAPBUILD_VERSION 6
-
 /*
  * Store/Load a snapshot from disk, depending on the snapshot builder's state.
  *
@@ -1859,6 +1677,10 @@ out:
 /*
  * Restore a snapshot into 'builder' if previously one has been stored at the
  * location indicated by 'lsn'. Returns true if successful, false otherwise.
+ *
+ * NOTE: For any code change or issue fix here, it is highly recommended to
+ * give a thought about doing the same in pg_logicalinspect contrib module
+ * as well.
  */
 static bool
 SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
@@ -2033,7 +1855,7 @@ snapshot_not_interesting:
 /*
  * Read the contents of the serialized snapshot to 'dest'.
  */
-static void
+void
 SnapBuildRestoreContents(int fd, char *dest, Size size, const char *path)
 {
 	int			readBytes;
diff --git a/src/include/port/pg_crc32c.h b/src/include/port/pg_crc32c.h
index 63c8e3a00b..cfc8c07944 100644
--- a/src/include/port/pg_crc32c.h
+++ b/src/include/port/pg_crc32c.h
@@ -47,7 +47,7 @@ typedef uint32 pg_crc32c;
 	((crc) = pg_comp_crc32c_sse42((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
 
 #elif defined(USE_ARMV8_CRC32C)
 /* Use ARMv8 CRC Extension instructions. */
@@ -56,7 +56,7 @@ extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t le
 	((crc) = pg_comp_crc32c_armv8((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
 
 #elif defined(USE_LOONGARCH_CRC32C)
 /* Use LoongArch CRCC instructions. */
@@ -65,7 +65,7 @@ extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t le
 	((crc) = pg_comp_crc32c_loongarch((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len);
 
 #elif defined(USE_SSE42_CRC32C_WITH_RUNTIME_CHECK) || defined(USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK)
 
@@ -77,14 +77,14 @@ extern pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_
 	((crc) = pg_comp_crc32c((crc), (data), (len)))
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 
-extern pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
-extern pg_crc32c (*pg_comp_crc32c) (pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c (*pg_comp_crc32c) (pg_crc32c crc, const void *data, size_t len);
 
 #ifdef USE_SSE42_CRC32C_WITH_RUNTIME_CHECK
-extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);
 #endif
 #ifdef USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK
-extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
 #endif
 
 #else
@@ -103,7 +103,7 @@ extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t le
 #define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
 #endif
 
-extern pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
+extern PGDLLIMPORT pg_crc32c pg_comp_crc32c_sb8(pg_crc32c crc, const void *data, size_t len);
 
 #endif
 
diff --git a/src/include/replication/snapbuild.h b/src/include/replication/snapbuild.h
index caa5113ff8..e844a89882 100644
--- a/src/include/replication/snapbuild.h
+++ b/src/include/replication/snapbuild.h
@@ -15,6 +15,10 @@
 #include "access/xlogdefs.h"
 #include "utils/snapmgr.h"
 
+/*
+ * Please keep SnapBuildStateDesc[] (located in the pg_logicalinspect module)
+ * updated if a change needs to be made to SnapBuildState.
+ */
 typedef enum
 {
 	/*
@@ -46,7 +50,7 @@ typedef enum
 	SNAPBUILD_CONSISTENT = 2,
 } SnapBuildState;
 
-/* forward declare so we don't have to expose the struct to the public */
+/* forward declare so we don't have to include snapbuild_internal.h */
 struct SnapBuild;
 typedef struct SnapBuild SnapBuild;
 
diff --git a/src/include/replication/snapbuild_internal.h b/src/include/replication/snapbuild_internal.h
new file mode 100644
index 0000000000..0e47ddfcb4
--- /dev/null
+++ b/src/include/replication/snapbuild_internal.h
@@ -0,0 +1,204 @@
+/*-------------------------------------------------------------------------
+ *
+ * snapbuild_internal.h
+ *    This file contains declarations for logical decoding utility
+ *    functions for internal use.
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * src/include/replication/snapbuild_internal.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef SNAPBUILD_INTERNAL_H
+#define SNAPBUILD_INTERNAL_H
+
+#include "port/pg_crc32c.h"
+#include "replication/reorderbuffer.h"
+#include "replication/snapbuild.h"
+
+/*
+ * This struct contains the current state of the snapshot building
+ * machinery. It is exposed to the public, so pay attention when changing its
+ * contents.
+ */
+typedef struct SnapBuild
+{
+	/* how far are we along building our first full snapshot */
+	SnapBuildState state;
+
+	/* private memory context used to allocate memory for this module. */
+	MemoryContext context;
+
+	/* all transactions < than this have committed/aborted */
+	TransactionId xmin;
+
+	/* all transactions >= than this are uncommitted */
+	TransactionId xmax;
+
+	/*
+	 * Don't replay commits from an LSN < this LSN. This can be set externally
+	 * but it will also be advanced (never retreat) from within snapbuild.c.
+	 */
+	XLogRecPtr	start_decoding_at;
+
+	/*
+	 * LSN at which two-phase decoding was enabled or LSN at which we found a
+	 * consistent point at the time of slot creation.
+	 *
+	 * The prepared transactions, that were skipped because previously
+	 * two-phase was not enabled or are not covered by initial snapshot, need
+	 * to be sent later along with commit prepared and they must be before
+	 * this point.
+	 */
+	XLogRecPtr	two_phase_at;
+
+	/*
+	 * Don't start decoding WAL until the "xl_running_xacts" information
+	 * indicates there are no running xids with an xid smaller than this.
+	 */
+	TransactionId initial_xmin_horizon;
+
+	/* Indicates if we are building full snapshot or just catalog one. */
+	bool		building_full_snapshot;
+
+	/*
+	 * Indicates if we are using the snapshot builder for the creation of a
+	 * logical replication slot. If it's true, the start point for decoding
+	 * changes is not determined yet. So we skip snapshot restores to properly
+	 * find the start point. See SnapBuildFindSnapshot() for details.
+	 */
+	bool		in_slot_creation;
+
+	/*
+	 * Snapshot that's valid to see the catalog state seen at this moment.
+	 */
+	Snapshot	snapshot;
+
+	/*
+	 * LSN of the last location we are sure a snapshot has been serialized to.
+	 */
+	XLogRecPtr	last_serialized_snapshot;
+
+	/*
+	 * The reorderbuffer we need to update with usable snapshots et al.
+	 */
+	ReorderBuffer *reorder;
+
+	/*
+	 * TransactionId at which the next phase of initial snapshot building will
+	 * happen. InvalidTransactionId if not known (i.e. SNAPBUILD_START), or
+	 * when no next phase necessary (SNAPBUILD_CONSISTENT).
+	 */
+	TransactionId next_phase_at;
+
+	/*
+	 * Array of transactions which could have catalog changes that committed
+	 * between xmin and xmax.
+	 */
+	struct
+	{
+		/* number of committed transactions */
+		size_t		xcnt;
+
+		/* available space for committed transactions */
+		size_t		xcnt_space;
+
+		/*
+		 * Until we reach a CONSISTENT state, we record commits of all
+		 * transactions, not just the catalog changing ones. Record when that
+		 * changes so we know we cannot export a snapshot safely anymore.
+		 */
+		bool		includes_all_transactions;
+
+		/*
+		 * Array of committed transactions that have modified the catalog.
+		 *
+		 * As this array is frequently modified we do *not* keep it in
+		 * xidComparator order. Instead we sort the array when building &
+		 * distributing a snapshot.
+		 *
+		 * TODO: It's unclear whether that reasoning has much merit. Every
+		 * time we add something here after becoming consistent will also
+		 * require distributing a snapshot. Storing them sorted would
+		 * potentially also make it easier to purge (but more complicated wrt
+		 * wraparound?). Should be improved if sorting while building the
+		 * snapshot shows up in profiles.
+		 */
+		TransactionId *xip;
+	}			committed;
+
+	/*
+	 * Array of transactions and subtransactions that had modified catalogs
+	 * and were running when the snapshot was serialized.
+	 *
+	 * We normally rely on some WAL record types such as HEAP2_NEW_CID to know
+	 * if the transaction has changed the catalog. But it could happen that
+	 * the logical decoding decodes only the commit record of the transaction
+	 * after restoring the previously serialized snapshot in which case we
+	 * will miss adding the xid to the snapshot and end up looking at the
+	 * catalogs with the wrong snapshot.
+	 *
+	 * Now to avoid the above problem, we serialize the transactions that had
+	 * modified the catalogs and are still running at the time of snapshot
+	 * serialization. We fill this array while restoring the snapshot and then
+	 * refer it while decoding commit to ensure if the xact has modified the
+	 * catalog. We discard this array when all the xids in the list become old
+	 * enough to matter. See SnapBuildPurgeOlderTxn for details.
+	 */
+	struct
+	{
+		/* number of transactions */
+		size_t		xcnt;
+
+		/* This array must be sorted in xidComparator order */
+		TransactionId *xip;
+	}			catchange;
+} SnapBuild;
+
+/* -----------------------------------
+ * Snapshot serialization support
+ * -----------------------------------
+ */
+
+/*
+ * We store current state of struct SnapBuild on disk in the following manner:
+ *
+ * struct SnapBuildOnDisk;
+ * TransactionId * committed.xcnt; (*not xcnt_space*)
+ * TransactionId * catchange.xcnt;
+ *
+ */
+typedef struct SnapBuildOnDisk
+{
+	/* first part of this struct needs to be version independent */
+
+	/* data not covered by checksum */
+	uint32		magic;
+	pg_crc32c	checksum;
+
+	/* data covered by checksum */
+
+	/* version, in case we want to support pg_upgrade */
+	uint32		version;
+	/* how large is the on disk data, excluding the constant sized part */
+	uint32		length;
+
+	/* version dependent part */
+	SnapBuild	builder;
+
+	/* variable amount of TransactionIds follows */
+} SnapBuildOnDisk;
+
+#define SnapBuildOnDiskConstantSize \
+	offsetof(SnapBuildOnDisk, builder)
+#define SnapBuildOnDiskNotChecksummedSize \
+	offsetof(SnapBuildOnDisk, version)
+
+#define SNAPBUILD_MAGIC 0x51A1E001
+#define SNAPBUILD_VERSION 6
+
+extern void SnapBuildRestoreContents(int fd, char *dest, Size size, const char *path);
+
+#endif							/* SNAPBUILD_INTERNAL_H */
-- 
2.34.1

#40Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: shveta malik (#37)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Wed, Sep 25, 2024 at 11:23:17AM +0530, shveta malik wrote:

+ OUT catchange_xip xid[]

One question, what is xid datatype, is it too int8? Sorry, could not
find the correct doc.

I think that we can get the answer from pg_type:

postgres=# select typname,typlen from pg_type where typname = 'xid';
typname | typlen
---------+--------
xid | 4
(1 row)

Since we are getting uint32 in Int64, this also needs to be accordingly.

I think the way it is currently done is fine: we're dealing with TransactionId
(and not with FullTransactionId). So, the Int64GetDatum() output would still
stay in the "xid" range. Keeping xid in the .sql makes it clear that we are
dealing with transaction ID.

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

#41Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Bertrand Drouvot (#39)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Wed, Sep 25, 2024 at 10:29 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Hi,

On Wed, Sep 25, 2024 at 04:04:43PM +0200, Peter Eisentraut wrote:

Is there a reason for this elaborate error handling:

Thanks for looking at it!

+     fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
+
+     if (fd < 0 && errno == ENOENT)
+             ereport(ERROR,
+                             errmsg("file \"%s\" does not exist", path));
+     else if (fd < 0)
+             ereport(ERROR,
+                             (errcode_for_file_access(),
+                              errmsg("could not open file \"%s\": %m", path)));

Couldn't you just use the second branch for all errno's?

Yeah, I think it comes from copying/pasting from SnapBuildRestore() too "quickly".
v10 attached uses the second branch only.

I've reviewed v10 patch and here are some comments:

 +static void
 +ValidateAndRestoreSnapshotFile(XLogRecPtr lsn, SnapBuildOnDisk *ondisk,
 +                              const char *path)

This function and SnapBuildRestore() have duplicate codes. Can we have
a common function that reads the snapshot data from disk to
SnapBuildOnDisk, and have both ValidateAndRestoreSnapshotFile() and
SnapBuildRestore() call it?

---
+CREATE FUNCTION pg_get_logical_snapshot_meta(IN in_lsn pg_lsn,
(snip)
+CREATE FUNCTION pg_get_logical_snapshot_info(IN in_lsn pg_lsn,

Is there any reason why both functions take a pg_lsn value as an
argument? Given that the main usage would be to use these functions
with pg_ls_logicalsnapdir(), I think it would be easier for users if
these functions take a filename as a function argument. That way, we
can use these functions like:

select info.* from pg_ls_logicalsnapdir(),
pg_get_logical_snapshot_info(name) as info;

If there are use cases where specifying a LSN is easier, I think we
would have both types.

----
+static const char *const SnapBuildStateDesc[] = {
+        [SNAPBUILD_START + SNAPBUILD_STATE_INCR] = "start",
+        [SNAPBUILD_BUILDING_SNAPSHOT + SNAPBUILD_STATE_INCR] = "building",
+        [SNAPBUILD_FULL_SNAPSHOT + SNAPBUILD_STATE_INCR] = "full",
+        [SNAPBUILD_CONSISTENT + SNAPBUILD_STATE_INCR] = "consistent",
+};

I think that it'd be better to have a dedicated function that returns
a string representation of the given state by using a switch
statement. That way, we don't need SNAPBUILD_STATE_INCR and a compiler
warning would help realize a missing state if a new state is
introduced in the future. It needs a function call but I believe it
won't be a problem in this use case.

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com

#42shveta malik
shveta.malik@gmail.com
In reply to: Bertrand Drouvot (#40)
Re: Add contrib/pg_logicalsnapinspect

On Wed, Sep 25, 2024 at 11:01 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Hi,

On Wed, Sep 25, 2024 at 11:23:17AM +0530, shveta malik wrote:

+ OUT catchange_xip xid[]

One question, what is xid datatype, is it too int8? Sorry, could not
find the correct doc.

I think that we can get the answer from pg_type:

postgres=# select typname,typlen from pg_type where typname = 'xid';
typname | typlen
---------+--------
xid | 4
(1 row)

Since we are getting uint32 in Int64, this also needs to be accordingly.

I think the way it is currently done is fine: we're dealing with TransactionId
(and not with FullTransactionId). So, the Int64GetDatum() output would still
stay in the "xid" range. Keeping xid in the .sql makes it clear that we are
dealing with transaction ID.

Okay, got it. The 'xid' usage is fine then.

thanks
Shveta

#43Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: Masahiko Sawada (#41)
1 attachment(s)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Wed, Sep 25, 2024 at 05:04:18PM -0700, Masahiko Sawada wrote:

I've reviewed v10 patch and here are some comments:

Thanks for looking at it!

+static void
+ValidateAndRestoreSnapshotFile(XLogRecPtr lsn, SnapBuildOnDisk *ondisk,
+                              const char *path)

This function and SnapBuildRestore() have duplicate codes. Can we have
a common function that reads the snapshot data from disk to
SnapBuildOnDisk, and have both ValidateAndRestoreSnapshotFile() and
SnapBuildRestore() call it?

Right. I had in mind to remove the duplicate code while proposing the "in core"
functions version of this patch, see [1]/messages/by-id/ZtGKi5FdW+ky+0fV@ip-10-97-1-34.eu-west-3.compute.internal. Now that snapbuild_internal.h is there,
I do not see any reason not to remove the duplicate code.

That's done in v11 attached: ValidateAndRestoreSnapshotFile() has been modified
a bit and can now be used in the new module and in SnapBuildRestore().

Bonus points, as compared to v10, it allows to:

1. move the SnapBuildRestoreContents() declaration back from snapbuild_internal.h
to its original location (snapbuild.c)
2. same as above for the SnapBuildOnDiskConstantSize, SnapBuildOnDiskNotChecksummedSize,
SNAPBUILD_MAGIC and SNAPBUILD_VERSION definitions
3. remove the changes in pg_crc32c.h as the extras "PGDLLIMPORT" are not needed
anymore

---
+CREATE FUNCTION pg_get_logical_snapshot_meta(IN in_lsn pg_lsn,
(snip)
+CREATE FUNCTION pg_get_logical_snapshot_info(IN in_lsn pg_lsn,

Is there any reason why both functions take a pg_lsn value as an
argument? Given that the main usage would be to use these functions
with pg_ls_logicalsnapdir(), I think it would be easier for users if
these functions take a filename as a function argument. That way, we
can use these functions like:

select info.* from pg_ls_logicalsnapdir(),
pg_get_logical_snapshot_info(name) as info;

I think that makes sense. It also simplfies the tests and the examples, done
that way in v11.

If there are use cases where specifying a LSN is easier, I think we
would have both types.

----
+static const char *const SnapBuildStateDesc[] = {
+        [SNAPBUILD_START + SNAPBUILD_STATE_INCR] = "start",
+        [SNAPBUILD_BUILDING_SNAPSHOT + SNAPBUILD_STATE_INCR] = "building",
+        [SNAPBUILD_FULL_SNAPSHOT + SNAPBUILD_STATE_INCR] = "full",
+        [SNAPBUILD_CONSISTENT + SNAPBUILD_STATE_INCR] = "consistent",
+};

I think that it'd be better to have a dedicated function that returns
a string representation of the given state by using a switch
statement. That way, we don't need SNAPBUILD_STATE_INCR and a compiler
warning would help realize a missing state if a new state is
introduced in the future.

Yeah, that sounds reasonable. Done in v11: the switch does not handle "default"
so that [-Wswitch] would report a warning in case of enumeration value not
handled in the switch (v11 keeps a remark on top of the SnapBuildState enum
definition though).

It needs a function call but I believe it won't be a problem in this use case.

Agree.

[1]: /messages/by-id/ZtGKi5FdW+ky+0fV@ip-10-97-1-34.eu-west-3.compute.internal

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachments:

v11-0001-Add-contrib-pg_logicalinspect.patchtext/x-diff; charset=us-asciiDownload
From 52fbead00e079132881ec51a3f913cd784ccd356 Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Wed, 14 Aug 2024 08:46:05 +0000
Subject: [PATCH v11] Add contrib/pg_logicalinspect

Provides SQL functions that allow to inspect logical decoding components.

It currently allows to inspect the contents of serialized logical snapshots of
a running database cluster, which is useful for debugging or educational
purposes.
---
 contrib/Makefile                              |   1 +
 contrib/meson.build                           |   1 +
 contrib/pg_logicalinspect/.gitignore          |   4 +
 contrib/pg_logicalinspect/Makefile            |  31 ++
 .../expected/logical_inspect.out              |  52 ++++
 contrib/pg_logicalinspect/logicalinspect.conf |   1 +
 contrib/pg_logicalinspect/meson.build         |  39 +++
 .../pg_logicalinspect--1.0.sql                |  43 +++
 contrib/pg_logicalinspect/pg_logicalinspect.c | 199 +++++++++++++
 .../pg_logicalinspect.control                 |   5 +
 .../specs/logical_inspect.spec                |  34 +++
 doc/src/sgml/contrib.sgml                     |   1 +
 doc/src/sgml/filelist.sgml                    |   1 +
 doc/src/sgml/pglogicalinspect.sgml            | 150 ++++++++++
 src/backend/replication/logical/snapbuild.c   | 279 ++++--------------
 src/include/replication/snapbuild.h           |   6 +-
 src/include/replication/snapbuild_internal.h  | 197 +++++++++++++
 17 files changed, 823 insertions(+), 221 deletions(-)
   7.5% contrib/pg_logicalinspect/expected/
   5.2% contrib/pg_logicalinspect/specs/
  27.9% contrib/pg_logicalinspect/
  14.2% doc/src/sgml/
  25.0% src/backend/replication/logical/
  19.7% src/include/replication/

diff --git a/contrib/Makefile b/contrib/Makefile
index abd780f277..952855d9b6 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -32,6 +32,7 @@ SUBDIRS = \
 		passwordcheck	\
 		pg_buffercache	\
 		pg_freespacemap \
+		pg_logicalinspect \
 		pg_prewarm	\
 		pg_stat_statements \
 		pg_surgery	\
diff --git a/contrib/meson.build b/contrib/meson.build
index 14a8906865..159ff41555 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -46,6 +46,7 @@ subdir('passwordcheck')
 subdir('pg_buffercache')
 subdir('pgcrypto')
 subdir('pg_freespacemap')
+subdir('pg_logicalinspect')
 subdir('pg_prewarm')
 subdir('pgrowlocks')
 subdir('pg_stat_statements')
diff --git a/contrib/pg_logicalinspect/.gitignore b/contrib/pg_logicalinspect/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/contrib/pg_logicalinspect/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/contrib/pg_logicalinspect/Makefile b/contrib/pg_logicalinspect/Makefile
new file mode 100644
index 0000000000..55124514d4
--- /dev/null
+++ b/contrib/pg_logicalinspect/Makefile
@@ -0,0 +1,31 @@
+# contrib/pg_logicalinspect/Makefile
+
+MODULE_big = pg_logicalinspect
+OBJS = \
+	$(WIN32RES) \
+	pg_logicalinspect.o
+PGFILEDESC = "pg_logicalinspect - functions to inspect logical decoding components"
+
+EXTENSION = pg_logicalinspect
+DATA = pg_logicalinspect--1.0.sql
+
+EXTRA_INSTALL = contrib/test_decoding
+
+ISOLATION = logical_inspect
+
+ISOLATION_OPTS = --temp-config $(top_srcdir)/contrib/pg_logicalinspect/logicalinspect.conf
+
+# Disabled because these tests require "wal_level=logical", which
+# some installcheck users do not have (e.g. buildfarm clients).
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/pg_logicalinspect
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/pg_logicalinspect/expected/logical_inspect.out b/contrib/pg_logicalinspect/expected/logical_inspect.out
new file mode 100644
index 0000000000..219afd6a16
--- /dev/null
+++ b/contrib/pg_logicalinspect/expected/logical_inspect.out
@@ -0,0 +1,52 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s0_init s0_begin s0_savepoint s0_truncate s1_checkpoint s1_get_changes s0_commit s0_begin s0_insert s1_checkpoint s1_get_changes s0_commit s1_get_changes s1_get_logical_snapshot_info s1_get_logical_snapshot_meta
+step s0_init: SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding');
+?column?
+--------
+init    
+(1 row)
+
+step s0_begin: BEGIN;
+step s0_savepoint: SAVEPOINT sp1;
+step s0_truncate: TRUNCATE tbl1;
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data
+----
+(0 rows)
+
+step s0_commit: COMMIT;
+step s0_begin: BEGIN;
+step s0_insert: INSERT INTO tbl1 VALUES (1);
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                   
+---------------------------------------
+BEGIN                                  
+table public.tbl1: TRUNCATE: (no-flags)
+COMMIT                                 
+(3 rows)
+
+step s0_commit: COMMIT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                                         
+-------------------------------------------------------------
+BEGIN                                                        
+table public.tbl1: INSERT: val1[integer]:1 val2[integer]:null
+COMMIT                                                       
+(3 rows)
+
+step s1_get_logical_snapshot_info: SELECT info.state,info.catchange_count,array_length(info.catchange_xip,1) AS catchange_array_length,info.committed_count,array_length(info.committed_xip,1) AS committed_array_length FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info ORDER BY 2;
+state     |catchange_count|catchange_array_length|committed_count|committed_array_length
+----------+---------------+----------------------+---------------+----------------------
+consistent|              0|                      |              2|                     2
+consistent|              2|                     2|              0|                      
+(2 rows)
+
+step s1_get_logical_snapshot_meta: SELECT COUNT(meta.*) from pg_ls_logicalsnapdir(), pg_get_logical_snapshot_meta(name) as meta;
+count
+-----
+    2
+(1 row)
+
diff --git a/contrib/pg_logicalinspect/logicalinspect.conf b/contrib/pg_logicalinspect/logicalinspect.conf
new file mode 100644
index 0000000000..e3d257315f
--- /dev/null
+++ b/contrib/pg_logicalinspect/logicalinspect.conf
@@ -0,0 +1 @@
+wal_level = logical
diff --git a/contrib/pg_logicalinspect/meson.build b/contrib/pg_logicalinspect/meson.build
new file mode 100644
index 0000000000..3ec635509b
--- /dev/null
+++ b/contrib/pg_logicalinspect/meson.build
@@ -0,0 +1,39 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+pg_logicalinspect_sources = files('pg_logicalinspect.c')
+
+if host_system == 'windows'
+  pg_logicalinspect_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+    '--NAME', 'pg_logicalinspect',
+    '--FILEDESC', 'pg_logicalinspect - functions to inspect logical decoding components',])
+endif
+
+pg_logicalinspect = shared_module('pg_logicalinspect',
+  pg_logicalinspect_sources,
+  kwargs: contrib_mod_args + {
+      'dependencies': contrib_mod_args['dependencies'],
+  },
+)
+contrib_targets += pg_logicalinspect
+
+install_data(
+  'pg_logicalinspect.control',
+  'pg_logicalinspect--1.0.sql',
+  kwargs: contrib_data_args,
+)
+
+tests += {
+  'name': 'pg_logicalinspect',
+  'sd': meson.current_source_dir(),
+  'bd': meson.current_build_dir(),
+  'isolation': {
+    'specs': [
+      'logical_inspect',
+    ],
+    'regress_args': [
+      '--temp-config', files('logicalinspect.conf'),
+    ],
+    # see above
+    'runningcheck': false,
+  },
+}
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql b/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
new file mode 100644
index 0000000000..c773f6e458
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
@@ -0,0 +1,43 @@
+/* contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_logicalinspect" to load this file. \quit
+
+--
+-- pg_get_logical_snapshot_meta()
+--
+CREATE FUNCTION pg_get_logical_snapshot_meta(IN filename text,
+    OUT magic int4,
+    OUT checksum int8,
+    OUT version int4
+)
+AS 'MODULE_PATHNAME', 'pg_get_logical_snapshot_meta'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_logical_snapshot_meta(text) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_logical_snapshot_meta(text) TO pg_read_server_files;
+
+--
+-- pg_get_logical_snapshot_info()
+--
+CREATE FUNCTION pg_get_logical_snapshot_info(IN filename text,
+    OUT state text,
+    OUT xmin xid,
+    OUT xmax xid,
+    OUT start_decoding_at pg_lsn,
+    OUT two_phase_at pg_lsn,
+    OUT initial_xmin_horizon xid,
+    OUT building_full_snapshot boolean,
+    OUT in_slot_creation boolean,
+    OUT last_serialized_snapshot pg_lsn,
+    OUT next_phase_at xid,
+    OUT committed_count int8,
+    OUT committed_xip xid[],
+    OUT catchange_count int8,
+    OUT catchange_xip xid[]
+)
+AS 'MODULE_PATHNAME', 'pg_get_logical_snapshot_info'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_logical_snapshot_info(text) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_logical_snapshot_info(text) TO pg_read_server_files;
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.c b/contrib/pg_logicalinspect/pg_logicalinspect.c
new file mode 100644
index 0000000000..0e3e1f50fc
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.c
@@ -0,0 +1,199 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_logicalinspect.c
+ *		  Functions to inspect contents of PostgreSQL logical snapshots
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ *		  contrib/pg_logicalinspect/pg_logicalinspect.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "funcapi.h"
+#include "replication/snapbuild_internal.h"
+#include "utils/array.h"
+#include "utils/builtins.h"
+#include "utils/pg_lsn.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_meta);
+PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_info);
+
+/* Return the description of SnapBuildState */
+static const char *
+get_snapbuild_state_desc(SnapBuildState state)
+{
+	const char *stateDesc = "unknown state";
+
+	switch (state)
+	{
+		case SNAPBUILD_START:
+			stateDesc = "start";
+			break;
+		case SNAPBUILD_BUILDING_SNAPSHOT:
+			stateDesc = "building";
+			break;
+		case SNAPBUILD_FULL_SNAPSHOT:
+			stateDesc = "full";
+			break;
+		case SNAPBUILD_CONSISTENT:
+			stateDesc = "consistent";
+			break;
+	}
+
+	return stateDesc;
+}
+
+/*
+ * Retrieve the logical snapshot file metadata.
+ */
+Datum
+pg_get_logical_snapshot_meta(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_META_COLS 3
+	SnapBuildOnDisk ondisk;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_META_COLS];
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_META_COLS];
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+	MemoryContext context;
+	int			fd;
+	int			i = 0;
+	text	   *filename_t = PG_GETARG_TEXT_PP(0);
+
+	sprintf(path, "%s/%s",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			text_to_cstring(filename_t));
+
+	fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
+
+	if (fd < 0)
+		ereport(ERROR,
+				(errcode_for_file_access(),
+				 errmsg("could not open file \"%s\": %m", path)));
+
+	context = AllocSetContextCreate(CurrentMemoryContext,
+									"logicalsnapshot inspect context",
+									ALLOCSET_DEFAULT_SIZES);
+
+	/* Validate and restore the snapshot to 'ondisk' */
+	ValidateAndRestoreSnapshotFile(&ondisk, path, fd, context);
+
+	/* Build a tuple descriptor for our result type */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	memset(nulls, 0, sizeof(nulls));
+
+	values[i++] = UInt32GetDatum(ondisk.magic);
+	values[i++] = Int64GetDatum((int64) ondisk.checksum);
+	values[i++] = UInt32GetDatum(ondisk.version);
+
+	Assert(i == PG_GET_LOGICAL_SNAPSHOT_META_COLS);
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	MemoryContextReset(context);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_META_COLS
+}
+
+Datum
+pg_get_logical_snapshot_info(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_INFO_COLS 14
+	SnapBuildOnDisk ondisk;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS];
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS];
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+	MemoryContext context;
+	int			fd;
+	int			i = 0;
+	text	   *filename_t = PG_GETARG_TEXT_PP(0);
+
+	sprintf(path, "%s/%s",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			text_to_cstring(filename_t));
+
+	fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
+
+	if (fd < 0)
+		ereport(ERROR,
+				(errcode_for_file_access(),
+				 errmsg("could not open file \"%s\": %m", path)));
+
+	context = AllocSetContextCreate(CurrentMemoryContext,
+									"logicalsnapshot inspect context",
+									ALLOCSET_DEFAULT_SIZES);
+
+	/* Validate and restore the snapshot to 'ondisk' */
+	ValidateAndRestoreSnapshotFile(&ondisk, path, fd, context);
+
+	/* Build a tuple descriptor for our result type */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	memset(nulls, 0, sizeof(nulls));
+
+	values[i++] = CStringGetTextDatum(get_snapbuild_state_desc(ondisk.builder.state));
+	values[i++] = TransactionIdGetDatum(ondisk.builder.xmin);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.xmax);
+	values[i++] = LSNGetDatum(ondisk.builder.start_decoding_at);
+	values[i++] = LSNGetDatum(ondisk.builder.two_phase_at);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.initial_xmin_horizon);
+	values[i++] = BoolGetDatum(ondisk.builder.building_full_snapshot);
+	values[i++] = BoolGetDatum(ondisk.builder.in_slot_creation);
+	values[i++] = LSNGetDatum(ondisk.builder.last_serialized_snapshot);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.next_phase_at);
+
+	values[i++] = Int64GetDatum(ondisk.builder.committed.xcnt);
+	if (ondisk.builder.committed.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+		int			narrayelems = 0;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.committed.xcnt * sizeof(Datum));
+
+		for (; narrayelems < ondisk.builder.committed.xcnt; narrayelems++)
+			arrayelems[narrayelems] = Int64GetDatum((int64) ondisk.builder.committed.xip[narrayelems]);
+
+		values[i++] = PointerGetDatum(construct_array_builtin(arrayelems, narrayelems, INT8OID));
+	}
+	else
+		nulls[i++] = true;
+
+	values[i++] = Int64GetDatum(ondisk.builder.catchange.xcnt);
+	if (ondisk.builder.catchange.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+		int			narrayelems = 0;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.catchange.xcnt * sizeof(Datum));
+
+		for (; narrayelems < ondisk.builder.catchange.xcnt; narrayelems++)
+			arrayelems[narrayelems] = Int64GetDatum((int64) ondisk.builder.catchange.xip[narrayelems]);
+
+		values[i++] = PointerGetDatum(construct_array_builtin(arrayelems, narrayelems, INT8OID));
+	}
+	else
+		nulls[i++] = true;
+
+	Assert(i == PG_GET_LOGICAL_SNAPSHOT_INFO_COLS);
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	MemoryContextReset(context);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_INFO_COLS
+}
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.control b/contrib/pg_logicalinspect/pg_logicalinspect.control
new file mode 100644
index 0000000000..b4a70e57ba
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.control
@@ -0,0 +1,5 @@
+# pg_logicalinspect extension
+comment = 'functions to inspect logical decoding components'
+default_version = '1.0'
+module_pathname = '$libdir/pg_logicalinspect'
+relocatable = true
diff --git a/contrib/pg_logicalinspect/specs/logical_inspect.spec b/contrib/pg_logicalinspect/specs/logical_inspect.spec
new file mode 100644
index 0000000000..7dc3cd7504
--- /dev/null
+++ b/contrib/pg_logicalinspect/specs/logical_inspect.spec
@@ -0,0 +1,34 @@
+# Test the pg_logicalinspect functions: that needs some permutation to
+# ensure that we are creating multiple logical snapshots and that one of them
+# contains ongoing catalogs changes.
+setup
+{
+    DROP TABLE IF EXISTS tbl1;
+    CREATE TABLE tbl1 (val1 integer, val2 integer);
+    CREATE EXTENSION pg_logicalinspect;
+}
+
+teardown
+{
+    DROP TABLE tbl1;
+    SELECT 'stop' FROM pg_drop_replication_slot('isolation_slot');
+    DROP EXTENSION pg_logicalinspect;
+}
+
+session "s0"
+setup { SET synchronous_commit=on; }
+step "s0_init" { SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding'); }
+step "s0_begin" { BEGIN; }
+step "s0_savepoint" { SAVEPOINT sp1; }
+step "s0_truncate" { TRUNCATE tbl1; }
+step "s0_insert" { INSERT INTO tbl1 VALUES (1); }
+step "s0_commit" { COMMIT; }
+
+session "s1"
+setup { SET synchronous_commit=on; }
+step "s1_checkpoint" { CHECKPOINT; }
+step "s1_get_changes" { SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0'); }
+step "s1_get_logical_snapshot_meta" { SELECT COUNT(meta.*) from pg_ls_logicalsnapdir(), pg_get_logical_snapshot_meta(name) as meta;}
+step "s1_get_logical_snapshot_info" { SELECT info.state,info.catchange_count,array_length(info.catchange_xip,1) AS catchange_array_length,info.committed_count,array_length(info.committed_xip,1) AS committed_array_length FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info ORDER BY 2; }
+
+permutation "s0_init" "s0_begin" "s0_savepoint" "s0_truncate" "s1_checkpoint" "s1_get_changes" "s0_commit" "s0_begin" "s0_insert" "s1_checkpoint" "s1_get_changes" "s0_commit" "s1_get_changes" "s1_get_logical_snapshot_info" "s1_get_logical_snapshot_meta"
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index 44639a8dca..7c381949a5 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -154,6 +154,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
  &pgbuffercache;
  &pgcrypto;
  &pgfreespacemap;
+ &pglogicalinspect;
  &pgprewarm;
  &pgrowlocks;
  &pgstatstatements;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index a7ff5f8264..66e6dccd4c 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -143,6 +143,7 @@
 <!ENTITY pgbuffercache   SYSTEM "pgbuffercache.sgml">
 <!ENTITY pgcrypto        SYSTEM "pgcrypto.sgml">
 <!ENTITY pgfreespacemap  SYSTEM "pgfreespacemap.sgml">
+<!ENTITY pglogicalinspect  SYSTEM "pglogicalinspect.sgml">
 <!ENTITY pgprewarm       SYSTEM "pgprewarm.sgml">
 <!ENTITY pgrowlocks      SYSTEM "pgrowlocks.sgml">
 <!ENTITY pgstatstatements SYSTEM "pgstatstatements.sgml">
diff --git a/doc/src/sgml/pglogicalinspect.sgml b/doc/src/sgml/pglogicalinspect.sgml
new file mode 100644
index 0000000000..bb2cb0485a
--- /dev/null
+++ b/doc/src/sgml/pglogicalinspect.sgml
@@ -0,0 +1,150 @@
+<!-- doc/src/sgml/pglogicalinspect.sgml -->
+
+<sect1 id="pglogicalinspect" xreflabel="pg_logicalinspect">
+ <title>pg_logicalinspect &mdash; logical decoding components inspection</title>
+
+ <indexterm zone="pglogicalinspect">
+  <primary>pg_logicalinspect</primary>
+ </indexterm>
+
+ <para>
+  The <filename>pg_logicalinspect</filename> module provides SQL functions
+  that allow you to inspect the contents of logical decoding components. It
+  allows the inspection of serialized logical snapshots of a running
+  <productname>PostgreSQL</productname> database cluster, which is useful
+  for debugging or educational purposes.
+ </para>
+
+ <note>
+  <para>
+   The <filename>pg_logicalinspect</filename> functions are called
+   using a text argument that can be extracted from the output name of the
+   <function>pg_ls_logicalsnapdir</function>() function.
+  </para>
+ </note>
+
+ <para>
+  By default, use of these functions is restricted to superusers and members of
+  the <literal>pg_read_server_files</literal> role. Access may be granted by
+  superusers to others using <command>GRANT</command>.
+ </para>
+
+ <sect2 id="pglogicalinspect-funcs">
+  <title>General Functions</title>
+
+  <variablelist>
+   <varlistentry id="pglogicalinspect-funcs-pg-get-logical-snapshot-meta">
+    <term>
+     <function>pg_get_logical_snapshot_meta(filename text) returns record</function>
+    </term>
+
+    <listitem>
+     <para>
+      Gets logical snapshot metadata about a snapshot file that is located in
+      the server's <filename>pg_logical/snapshots</filename> directory.
+      The <replaceable>filename</replaceable> argument represents the snapshot
+      file name.
+      For example:
+<screen>
+postgres=# SELECT * FROM pg_ls_logicalsnapdir();
+-[ RECORD 1 ]+-----------------------
+name         | 0-40796E18.snap
+size         | 152
+modification | 2024-08-14 16:36:32+00
+
+postgres=# SELECT * FROM pg_get_logical_snapshot_meta('0-40796E18.snap');
+-[ RECORD 1 ]--------
+magic    | 1369563137
+checksum | 1028045905
+version  | 6
+
+postgres=# SELECT meta.* FROM pg_ls_logicalsnapdir(),
+pg_get_logical_snapshot_meta(name) AS meta;
+
+-[ RECORD 1 ]--------
+magic    | 1369563137
+checksum | 1028045905
+version  | 6
+</screen>
+     </para>
+     <para>
+      If <replaceable>filename</replaceable> does not match a snapshot file, the
+      function raises an error.
+     </para>
+    </listitem>
+   </varlistentry>
+
+   <varlistentry id="pglogicalinspect-funcs-pg-get-logical-snapshot-info">
+    <term>
+     <function>pg_get_logical_snapshot_info(filename text) returns record</function>
+    </term>
+
+    <listitem>
+     <para>
+      Gets logical snapshot information about a snapshot file that is located in
+      the server's <filename>pg_logical/snapshots</filename> directory.
+      The <replaceable>filename</replaceable> argument represents the snapshot
+      file name.
+      For example:
+<screen>
+postgres=# SELECT * FROM pg_ls_logicalsnapdir();
+-[ RECORD 1 ]+-----------------------
+name         | 0-40796E18.snap
+size         | 152
+modification | 2024-08-14 16:36:32+00
+
+postgres=# SELECT * FROM pg_get_logical_snapshot_info('0-40796E18.snap');
+-[ RECORD 1 ]------------+-----------
+state                    | consistent
+xmin                     | 751
+xmax                     | 751
+start_decoding_at        | 0/40796AF8
+two_phase_at             | 0/40796AF8
+initial_xmin_horizon     | 0
+building_full_snapshot   | f
+in_slot_creation         | f
+last_serialized_snapshot | 0/0
+next_phase_at            | 0
+committed_count          | 0
+committed_xip            |
+catchange_count          | 2
+catchange_xip            | {751,752}
+
+postgres=# SELECT info.* FROM pg_ls_logicalsnapdir(),
+pg_get_logical_snapshot_info(name) AS info;
+-[ RECORD 1 ]------------+-----------
+state                    | consistent
+xmin                     | 751
+xmax                     | 751
+start_decoding_at        | 0/40796AF8
+two_phase_at             | 0/40796AF8
+initial_xmin_horizon     | 0
+building_full_snapshot   | f
+in_slot_creation         | f
+last_serialized_snapshot | 0/0
+next_phase_at            | 0
+committed_count          | 0
+committed_xip            |
+catchange_count          | 2
+catchange_xip            | {751,752}
+</screen>
+     </para>
+     <para>
+      If <replaceable>filename</replaceable> does not match a snapshot file, the
+      function raises an error.
+     </para>
+    </listitem>
+   </varlistentry>
+
+  </variablelist>
+ </sect2>
+
+ <sect2 id="pglogicalinspect-author">
+  <title>Author</title>
+
+  <para>
+   Bertrand Drouvot <email>bertranddrouvot.pg@gmail.com</email>
+  </para>
+ </sect2>
+
+</sect1>
diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c
index 0450f94ba8..7a3b963a2f 100644
--- a/src/backend/replication/logical/snapbuild.c
+++ b/src/backend/replication/logical/snapbuild.c
@@ -134,6 +134,7 @@
 #include "replication/logical.h"
 #include "replication/reorderbuffer.h"
 #include "replication/snapbuild.h"
+#include "replication/snapbuild_internal.h"
 #include "storage/fd.h"
 #include "storage/lmgr.h"
 #include "storage/proc.h"
@@ -143,146 +144,6 @@
 #include "utils/memutils.h"
 #include "utils/snapmgr.h"
 #include "utils/snapshot.h"
-
-/*
- * This struct contains the current state of the snapshot building
- * machinery. Besides a forward declaration in the header, it is not exposed
- * to the public, so we can easily change its contents.
- */
-struct SnapBuild
-{
-	/* how far are we along building our first full snapshot */
-	SnapBuildState state;
-
-	/* private memory context used to allocate memory for this module. */
-	MemoryContext context;
-
-	/* all transactions < than this have committed/aborted */
-	TransactionId xmin;
-
-	/* all transactions >= than this are uncommitted */
-	TransactionId xmax;
-
-	/*
-	 * Don't replay commits from an LSN < this LSN. This can be set externally
-	 * but it will also be advanced (never retreat) from within snapbuild.c.
-	 */
-	XLogRecPtr	start_decoding_at;
-
-	/*
-	 * LSN at which two-phase decoding was enabled or LSN at which we found a
-	 * consistent point at the time of slot creation.
-	 *
-	 * The prepared transactions, that were skipped because previously
-	 * two-phase was not enabled or are not covered by initial snapshot, need
-	 * to be sent later along with commit prepared and they must be before
-	 * this point.
-	 */
-	XLogRecPtr	two_phase_at;
-
-	/*
-	 * Don't start decoding WAL until the "xl_running_xacts" information
-	 * indicates there are no running xids with an xid smaller than this.
-	 */
-	TransactionId initial_xmin_horizon;
-
-	/* Indicates if we are building full snapshot or just catalog one. */
-	bool		building_full_snapshot;
-
-	/*
-	 * Indicates if we are using the snapshot builder for the creation of a
-	 * logical replication slot. If it's true, the start point for decoding
-	 * changes is not determined yet. So we skip snapshot restores to properly
-	 * find the start point. See SnapBuildFindSnapshot() for details.
-	 */
-	bool		in_slot_creation;
-
-	/*
-	 * Snapshot that's valid to see the catalog state seen at this moment.
-	 */
-	Snapshot	snapshot;
-
-	/*
-	 * LSN of the last location we are sure a snapshot has been serialized to.
-	 */
-	XLogRecPtr	last_serialized_snapshot;
-
-	/*
-	 * The reorderbuffer we need to update with usable snapshots et al.
-	 */
-	ReorderBuffer *reorder;
-
-	/*
-	 * TransactionId at which the next phase of initial snapshot building will
-	 * happen. InvalidTransactionId if not known (i.e. SNAPBUILD_START), or
-	 * when no next phase necessary (SNAPBUILD_CONSISTENT).
-	 */
-	TransactionId next_phase_at;
-
-	/*
-	 * Array of transactions which could have catalog changes that committed
-	 * between xmin and xmax.
-	 */
-	struct
-	{
-		/* number of committed transactions */
-		size_t		xcnt;
-
-		/* available space for committed transactions */
-		size_t		xcnt_space;
-
-		/*
-		 * Until we reach a CONSISTENT state, we record commits of all
-		 * transactions, not just the catalog changing ones. Record when that
-		 * changes so we know we cannot export a snapshot safely anymore.
-		 */
-		bool		includes_all_transactions;
-
-		/*
-		 * Array of committed transactions that have modified the catalog.
-		 *
-		 * As this array is frequently modified we do *not* keep it in
-		 * xidComparator order. Instead we sort the array when building &
-		 * distributing a snapshot.
-		 *
-		 * TODO: It's unclear whether that reasoning has much merit. Every
-		 * time we add something here after becoming consistent will also
-		 * require distributing a snapshot. Storing them sorted would
-		 * potentially also make it easier to purge (but more complicated wrt
-		 * wraparound?). Should be improved if sorting while building the
-		 * snapshot shows up in profiles.
-		 */
-		TransactionId *xip;
-	}			committed;
-
-	/*
-	 * Array of transactions and subtransactions that had modified catalogs
-	 * and were running when the snapshot was serialized.
-	 *
-	 * We normally rely on some WAL record types such as HEAP2_NEW_CID to know
-	 * if the transaction has changed the catalog. But it could happen that
-	 * the logical decoding decodes only the commit record of the transaction
-	 * after restoring the previously serialized snapshot in which case we
-	 * will miss adding the xid to the snapshot and end up looking at the
-	 * catalogs with the wrong snapshot.
-	 *
-	 * Now to avoid the above problem, we serialize the transactions that had
-	 * modified the catalogs and are still running at the time of snapshot
-	 * serialization. We fill this array while restoring the snapshot and then
-	 * refer it while decoding commit to ensure if the xact has modified the
-	 * catalog. We discard this array when all the xids in the list become old
-	 * enough to matter. See SnapBuildPurgeOlderTxn for details.
-	 */
-	struct
-	{
-		/* number of transactions */
-		size_t		xcnt;
-
-		/* This array must be sorted in xidComparator order */
-		TransactionId *xip;
-	}			catchange;
-};
-
 /*
  * Starting a transaction -- which we need to do while exporting a snapshot --
  * removes knowledge about the previously used resowner, so we save it here.
@@ -1557,40 +1418,6 @@ SnapBuildWaitSnapshot(xl_running_xacts *running, TransactionId cutoff)
 	}
 }
 
-/* -----------------------------------
- * Snapshot serialization support
- * -----------------------------------
- */
-
-/*
- * We store current state of struct SnapBuild on disk in the following manner:
- *
- * struct SnapBuildOnDisk;
- * TransactionId * committed.xcnt; (*not xcnt_space*)
- * TransactionId * catchange.xcnt;
- *
- */
-typedef struct SnapBuildOnDisk
-{
-	/* first part of this struct needs to be version independent */
-
-	/* data not covered by checksum */
-	uint32		magic;
-	pg_crc32c	checksum;
-
-	/* data covered by checksum */
-
-	/* version, in case we want to support pg_upgrade */
-	uint32		version;
-	/* how large is the on disk data, excluding the constant sized part */
-	uint32		length;
-
-	/* version dependent part */
-	SnapBuild	builder;
-
-	/* variable amount of TransactionIds follows */
-} SnapBuildOnDisk;
-
 #define SnapBuildOnDiskConstantSize \
 	offsetof(SnapBuildOnDisk, builder)
 #define SnapBuildOnDiskNotChecksummedSize \
@@ -1857,34 +1684,14 @@ out:
 }
 
 /*
- * Restore a snapshot into 'builder' if previously one has been stored at the
- * location indicated by 'lsn'. Returns true if successful, false otherwise.
+ * Validate the logical snapshot file and read its contents to 'ondisk'.
  */
-static bool
-SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
+void
+ValidateAndRestoreSnapshotFile(SnapBuildOnDisk *ondisk, const char *path, int fd,
+							   MemoryContext context)
 {
-	SnapBuildOnDisk ondisk;
-	int			fd;
-	char		path[MAXPGPATH];
-	Size		sz;
 	pg_crc32c	checksum;
-
-	/* no point in loading a snapshot if we're already there */
-	if (builder->state == SNAPBUILD_CONSISTENT)
-		return false;
-
-	sprintf(path, "%s/%X-%X.snap",
-			PG_LOGICAL_SNAPSHOTS_DIR,
-			LSN_FORMAT_ARGS(lsn));
-
-	fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
-
-	if (fd < 0 && errno == ENOENT)
-		return false;
-	else if (fd < 0)
-		ereport(ERROR,
-				(errcode_for_file_access(),
-				 errmsg("could not open file \"%s\": %m", path)));
+	Size		sz;
 
 	/* ----
 	 * Make sure the snapshot had been stored safely to disk, that's normally
@@ -1897,47 +1704,46 @@ SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
 	fsync_fname(path, false);
 	fsync_fname(PG_LOGICAL_SNAPSHOTS_DIR, true);
 
-
 	/* read statically sized portion of snapshot */
-	SnapBuildRestoreContents(fd, (char *) &ondisk, SnapBuildOnDiskConstantSize, path);
+	SnapBuildRestoreContents(fd, (char *) ondisk, SnapBuildOnDiskConstantSize, path);
 
-	if (ondisk.magic != SNAPBUILD_MAGIC)
+	if (ondisk->magic != SNAPBUILD_MAGIC)
 		ereport(ERROR,
 				(errcode(ERRCODE_DATA_CORRUPTED),
 				 errmsg("snapbuild state file \"%s\" has wrong magic number: %u instead of %u",
-						path, ondisk.magic, SNAPBUILD_MAGIC)));
+						path, ondisk->magic, SNAPBUILD_MAGIC)));
 
-	if (ondisk.version != SNAPBUILD_VERSION)
+	if (ondisk->version != SNAPBUILD_VERSION)
 		ereport(ERROR,
 				(errcode(ERRCODE_DATA_CORRUPTED),
 				 errmsg("snapbuild state file \"%s\" has unsupported version: %u instead of %u",
-						path, ondisk.version, SNAPBUILD_VERSION)));
+						path, ondisk->version, SNAPBUILD_VERSION)));
 
 	INIT_CRC32C(checksum);
 	COMP_CRC32C(checksum,
-				((char *) &ondisk) + SnapBuildOnDiskNotChecksummedSize,
+				((char *) ondisk) + SnapBuildOnDiskNotChecksummedSize,
 				SnapBuildOnDiskConstantSize - SnapBuildOnDiskNotChecksummedSize);
 
 	/* read SnapBuild */
-	SnapBuildRestoreContents(fd, (char *) &ondisk.builder, sizeof(SnapBuild), path);
-	COMP_CRC32C(checksum, &ondisk.builder, sizeof(SnapBuild));
+	SnapBuildRestoreContents(fd, (char *) &ondisk->builder, sizeof(SnapBuild), path);
+	COMP_CRC32C(checksum, &ondisk->builder, sizeof(SnapBuild));
 
 	/* restore committed xacts information */
-	if (ondisk.builder.committed.xcnt > 0)
+	if (ondisk->builder.committed.xcnt > 0)
 	{
-		sz = sizeof(TransactionId) * ondisk.builder.committed.xcnt;
-		ondisk.builder.committed.xip = MemoryContextAllocZero(builder->context, sz);
-		SnapBuildRestoreContents(fd, (char *) ondisk.builder.committed.xip, sz, path);
-		COMP_CRC32C(checksum, ondisk.builder.committed.xip, sz);
+		sz = sizeof(TransactionId) * ondisk->builder.committed.xcnt;
+		ondisk->builder.committed.xip = MemoryContextAllocZero(context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.committed.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.committed.xip, sz);
 	}
 
 	/* restore catalog modifying xacts information */
-	if (ondisk.builder.catchange.xcnt > 0)
+	if (ondisk->builder.catchange.xcnt > 0)
 	{
-		sz = sizeof(TransactionId) * ondisk.builder.catchange.xcnt;
-		ondisk.builder.catchange.xip = MemoryContextAllocZero(builder->context, sz);
-		SnapBuildRestoreContents(fd, (char *) ondisk.builder.catchange.xip, sz, path);
-		COMP_CRC32C(checksum, ondisk.builder.catchange.xip, sz);
+		sz = sizeof(TransactionId) * ondisk->builder.catchange.xcnt;
+		ondisk->builder.catchange.xip = MemoryContextAllocZero(context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.catchange.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.catchange.xip, sz);
 	}
 
 	if (CloseTransientFile(fd) != 0)
@@ -1948,11 +1754,44 @@ SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
 	FIN_CRC32C(checksum);
 
 	/* verify checksum of what we've read */
-	if (!EQ_CRC32C(checksum, ondisk.checksum))
+	if (!EQ_CRC32C(checksum, ondisk->checksum))
 		ereport(ERROR,
 				(errcode(ERRCODE_DATA_CORRUPTED),
 				 errmsg("checksum mismatch for snapbuild state file \"%s\": is %u, should be %u",
-						path, checksum, ondisk.checksum)));
+						path, checksum, ondisk->checksum)));
+}
+
+/*
+ * Restore a snapshot into 'builder' if previously one has been stored at the
+ * location indicated by 'lsn'. Returns true if successful, false otherwise.
+ */
+static bool
+SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
+{
+	SnapBuildOnDisk ondisk;
+	int			fd;
+	char		path[MAXPGPATH];
+
+	/* no point in loading a snapshot if we're already there */
+	if (builder->state == SNAPBUILD_CONSISTENT)
+		return false;
+
+	sprintf(path, "%s/%X-%X.snap",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			LSN_FORMAT_ARGS(lsn));
+
+	fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
+
+	if (fd < 0 && errno == ENOENT)
+		return false;
+	else if (fd < 0)
+		ereport(ERROR,
+				(errcode_for_file_access(),
+				 errmsg("could not open file \"%s\": %m", path)));
+
+
+	/* validate and restore the snapshot to 'ondisk' */
+	ValidateAndRestoreSnapshotFile(&ondisk, path, fd, builder->context);
 
 	/*
 	 * ok, we now have a sensible snapshot here, figure out if it has more
diff --git a/src/include/replication/snapbuild.h b/src/include/replication/snapbuild.h
index caa5113ff8..3c1454df99 100644
--- a/src/include/replication/snapbuild.h
+++ b/src/include/replication/snapbuild.h
@@ -15,6 +15,10 @@
 #include "access/xlogdefs.h"
 #include "utils/snapmgr.h"
 
+/*
+ * Please keep get_snapbuild_state_desc() (located in the pg_logicalinspect
+ * module) updated if a change needs to be made to SnapBuildState.
+ */
 typedef enum
 {
 	/*
@@ -46,7 +50,7 @@ typedef enum
 	SNAPBUILD_CONSISTENT = 2,
 } SnapBuildState;
 
-/* forward declare so we don't have to expose the struct to the public */
+/* forward declare so we don't have to include snapbuild_internal.h */
 struct SnapBuild;
 typedef struct SnapBuild SnapBuild;
 
diff --git a/src/include/replication/snapbuild_internal.h b/src/include/replication/snapbuild_internal.h
new file mode 100644
index 0000000000..c3919beb95
--- /dev/null
+++ b/src/include/replication/snapbuild_internal.h
@@ -0,0 +1,197 @@
+/*-------------------------------------------------------------------------
+ *
+ * snapbuild_internal.h
+ *    This file contains declarations for logical decoding utility
+ *    functions for internal use.
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * src/include/replication/snapbuild_internal.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef SNAPBUILD_INTERNAL_H
+#define SNAPBUILD_INTERNAL_H
+
+#include "port/pg_crc32c.h"
+#include "replication/reorderbuffer.h"
+#include "replication/snapbuild.h"
+
+/*
+ * This struct contains the current state of the snapshot building
+ * machinery. It is exposed to the public, so pay attention when changing its
+ * contents.
+ */
+typedef struct SnapBuild
+{
+	/* how far are we along building our first full snapshot */
+	SnapBuildState state;
+
+	/* private memory context used to allocate memory for this module. */
+	MemoryContext context;
+
+	/* all transactions < than this have committed/aborted */
+	TransactionId xmin;
+
+	/* all transactions >= than this are uncommitted */
+	TransactionId xmax;
+
+	/*
+	 * Don't replay commits from an LSN < this LSN. This can be set externally
+	 * but it will also be advanced (never retreat) from within snapbuild.c.
+	 */
+	XLogRecPtr	start_decoding_at;
+
+	/*
+	 * LSN at which two-phase decoding was enabled or LSN at which we found a
+	 * consistent point at the time of slot creation.
+	 *
+	 * The prepared transactions, that were skipped because previously
+	 * two-phase was not enabled or are not covered by initial snapshot, need
+	 * to be sent later along with commit prepared and they must be before
+	 * this point.
+	 */
+	XLogRecPtr	two_phase_at;
+
+	/*
+	 * Don't start decoding WAL until the "xl_running_xacts" information
+	 * indicates there are no running xids with an xid smaller than this.
+	 */
+	TransactionId initial_xmin_horizon;
+
+	/* Indicates if we are building full snapshot or just catalog one. */
+	bool		building_full_snapshot;
+
+	/*
+	 * Indicates if we are using the snapshot builder for the creation of a
+	 * logical replication slot. If it's true, the start point for decoding
+	 * changes is not determined yet. So we skip snapshot restores to properly
+	 * find the start point. See SnapBuildFindSnapshot() for details.
+	 */
+	bool		in_slot_creation;
+
+	/*
+	 * Snapshot that's valid to see the catalog state seen at this moment.
+	 */
+	Snapshot	snapshot;
+
+	/*
+	 * LSN of the last location we are sure a snapshot has been serialized to.
+	 */
+	XLogRecPtr	last_serialized_snapshot;
+
+	/*
+	 * The reorderbuffer we need to update with usable snapshots et al.
+	 */
+	ReorderBuffer *reorder;
+
+	/*
+	 * TransactionId at which the next phase of initial snapshot building will
+	 * happen. InvalidTransactionId if not known (i.e. SNAPBUILD_START), or
+	 * when no next phase necessary (SNAPBUILD_CONSISTENT).
+	 */
+	TransactionId next_phase_at;
+
+	/*
+	 * Array of transactions which could have catalog changes that committed
+	 * between xmin and xmax.
+	 */
+	struct
+	{
+		/* number of committed transactions */
+		size_t		xcnt;
+
+		/* available space for committed transactions */
+		size_t		xcnt_space;
+
+		/*
+		 * Until we reach a CONSISTENT state, we record commits of all
+		 * transactions, not just the catalog changing ones. Record when that
+		 * changes so we know we cannot export a snapshot safely anymore.
+		 */
+		bool		includes_all_transactions;
+
+		/*
+		 * Array of committed transactions that have modified the catalog.
+		 *
+		 * As this array is frequently modified we do *not* keep it in
+		 * xidComparator order. Instead we sort the array when building &
+		 * distributing a snapshot.
+		 *
+		 * TODO: It's unclear whether that reasoning has much merit. Every
+		 * time we add something here after becoming consistent will also
+		 * require distributing a snapshot. Storing them sorted would
+		 * potentially also make it easier to purge (but more complicated wrt
+		 * wraparound?). Should be improved if sorting while building the
+		 * snapshot shows up in profiles.
+		 */
+		TransactionId *xip;
+	}			committed;
+
+	/*
+	 * Array of transactions and subtransactions that had modified catalogs
+	 * and were running when the snapshot was serialized.
+	 *
+	 * We normally rely on some WAL record types such as HEAP2_NEW_CID to know
+	 * if the transaction has changed the catalog. But it could happen that
+	 * the logical decoding decodes only the commit record of the transaction
+	 * after restoring the previously serialized snapshot in which case we
+	 * will miss adding the xid to the snapshot and end up looking at the
+	 * catalogs with the wrong snapshot.
+	 *
+	 * Now to avoid the above problem, we serialize the transactions that had
+	 * modified the catalogs and are still running at the time of snapshot
+	 * serialization. We fill this array while restoring the snapshot and then
+	 * refer it while decoding commit to ensure if the xact has modified the
+	 * catalog. We discard this array when all the xids in the list become old
+	 * enough to matter. See SnapBuildPurgeOlderTxn for details.
+	 */
+	struct
+	{
+		/* number of transactions */
+		size_t		xcnt;
+
+		/* This array must be sorted in xidComparator order */
+		TransactionId *xip;
+	}			catchange;
+} SnapBuild;
+
+/* -----------------------------------
+ * Snapshot serialization support
+ * -----------------------------------
+ */
+
+/*
+ * We store current state of struct SnapBuild on disk in the following manner:
+ *
+ * struct SnapBuildOnDisk;
+ * TransactionId * committed.xcnt; (*not xcnt_space*)
+ * TransactionId * catchange.xcnt;
+ *
+ */
+typedef struct SnapBuildOnDisk
+{
+	/* first part of this struct needs to be version independent */
+
+	/* data not covered by checksum */
+	uint32		magic;
+	pg_crc32c	checksum;
+
+	/* data covered by checksum */
+
+	/* version, in case we want to support pg_upgrade */
+	uint32		version;
+	/* how large is the on disk data, excluding the constant sized part */
+	uint32		length;
+
+	/* version dependent part */
+	SnapBuild	builder;
+
+	/* variable amount of TransactionIds follows */
+} SnapBuildOnDisk;
+
+extern void ValidateAndRestoreSnapshotFile(SnapBuildOnDisk *ondisk, const char *path,
+										   int fd, MemoryContext context);
+
+#endif							/* SNAPBUILD_INTERNAL_H */
-- 
2.34.1

#44Peter Smith
smithpb2250@gmail.com
In reply to: Bertrand Drouvot (#43)
1 attachment(s)
Re: Add contrib/pg_logicalsnapinspect

Hi, here are some review comments for patch v11.

======
contrib/pg_logicalinspect/specs/logical_inspect.spec

1.
nit - Add some missing spaces after commas (,) in the SQL.
nit - Also update the expected results accordingly

======
doc/src/sgml/pglogicalinspect.sgml

2.
+ <note>
+  <para>
+   The <filename>pg_logicalinspect</filename> functions are called
+   using a text argument that can be extracted from the output name of the
+   <function>pg_ls_logicalsnapdir</function>() function.
+  </para>
+ </note>

2a. wording

The wording "using a text argument that can be extracted" seems like a
hangover from the previous implementation; it does not even say what
that "text argument" means. Why not just say it is a snapshot
filename, something like below?

SUGGESTION:
The pg_logicalinspect functions are called passing a snapshot filename
to be inspected. For example, pass a name obtained from the
pg_ls_logicalsnapdir() function.

~

2b. formatting

nit - In the previous implementation the extraction of the LSN was
trickier, so this part was worthy of an SGML "NOTE". Now that it is
just a filename, I don't know if it needs to be a special note
anymore.

~~~

3.
+postgres=# SELECT meta.* FROM pg_ls_logicalsnapdir(),
+pg_get_logical_snapshot_meta(name) AS meta;
+
+-[ RECORD 1 ]--------
+magic    | 1369563137
+checksum | 1028045905
+version  | 6

3a.
If you are going to wrap the SQL across multiple lines like this, then
you should show the psql continuation prompt, so that the example
looks the same as what the user would see.

~

3b.
FYI, the output of that can return multiple records, which is
b.i) probably not what you intended to demonstrate
b.ii) not the same as what the example says

e.g., I got this:
test_pub=# SELECT meta.* FROM pg_ls_logicalsnapdir(),
test_pub-# pg_get_logical_snapshot_meta(name) AS meta;
-[ RECORD 1 ]--------
magic | 1369563137
checksum | 681884630
version | 6
-[ RECORD 2 ]--------
magic | 1369563137
checksum | 2213048308
version | 6
-[ RECORD 3 ]--------
magic | 1369563137
checksum | 3812680762
version | 6
-[ RECORD 4 ]--------
magic | 1369563137
checksum | 3759893001
version | 6

~~~

(Also those #3a, #3b comments apply to both examples)

======
src/backend/replication/logical/snapbuild.c

4.
- SnapBuild builder;
-
- /* variable amount of TransactionIds follows */
-} SnapBuildOnDisk;
-
#define SnapBuildOnDiskConstantSize \
offsetof(SnapBuildOnDisk, builder)
#define SnapBuildOnDiskNotChecksummedSize \

Is it better to try to keep those "Size" macros defined along with
wherever the SnapBuildOnDisk is defined? Otherwise, if the structure
is ever changed, adjusting the macros could be easily overlooked.

~~~

5.
ValidateAndRestoreSnapshotFile

nit - See [1]/messages/by-id/ZusgK0/B8F/1rqft@ip-10-97-1-34.eu-west-3.compute.internal #4 suggestion to declare 'sz' at scope where used. The
previous reason not to change this (e.g. "mainly inspired from
SnapBuildRestore") seems less relevant because now most lines of this
function have already been modified for other reasons.

~~~

6.
SnapBuildRestore:

+ if (fd < 0 && errno == ENOENT)
+ return false;
+ else if (fd < 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not open file \"%s\": %m", path)));

I think this code fragment looked like this before, and you only
relocated it, but it still seems a bit awkward to write this way.
Since so much else has changed, how about also improving this in
passing, like below:

if (fd < 0)
{
if (errno == ENOENT)
return false;

ereport(ERROR,
(errcode_for_file_access(),
errmsg("could not open file \"%s\": %m", path)));
}

======
[1]: /messages/by-id/ZusgK0/B8F/1rqft@ip-10-97-1-34.eu-west-3.compute.internal

Kind Regards,
Peter Smith.
Fujitsu Australia

Attachments:

PS_NITPICKS_20241008_v11.txttext/plain; charset=US-ASCII; name=PS_NITPICKS_20241008_v11.txtDownload
diff --git a/contrib/pg_logicalinspect/expected/logical_inspect.out b/contrib/pg_logicalinspect/expected/logical_inspect.out
index 219afd6..71dea4a 100644
--- a/contrib/pg_logicalinspect/expected/logical_inspect.out
+++ b/contrib/pg_logicalinspect/expected/logical_inspect.out
@@ -37,7 +37,7 @@ table public.tbl1: INSERT: val1[integer]:1 val2[integer]:null
 COMMIT                                                       
 (3 rows)
 
-step s1_get_logical_snapshot_info: SELECT info.state,info.catchange_count,array_length(info.catchange_xip,1) AS catchange_array_length,info.committed_count,array_length(info.committed_xip,1) AS committed_array_length FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info ORDER BY 2;
+step s1_get_logical_snapshot_info: SELECT info.state, info.catchange_count, array_length(info.catchange_xip, 1) AS catchange_array_length, info.committed_count, array_length(info.committed_xip, 1) AS committed_array_length FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info ORDER BY 2;
 state     |catchange_count|catchange_array_length|committed_count|committed_array_length
 ----------+---------------+----------------------+---------------+----------------------
 consistent|              0|                      |              2|                     2
diff --git a/contrib/pg_logicalinspect/specs/logical_inspect.spec b/contrib/pg_logicalinspect/specs/logical_inspect.spec
index 7dc3cd7..0a4d268 100644
--- a/contrib/pg_logicalinspect/specs/logical_inspect.spec
+++ b/contrib/pg_logicalinspect/specs/logical_inspect.spec
@@ -29,6 +29,6 @@ setup { SET synchronous_commit=on; }
 step "s1_checkpoint" { CHECKPOINT; }
 step "s1_get_changes" { SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0'); }
 step "s1_get_logical_snapshot_meta" { SELECT COUNT(meta.*) from pg_ls_logicalsnapdir(), pg_get_logical_snapshot_meta(name) as meta;}
-step "s1_get_logical_snapshot_info" { SELECT info.state,info.catchange_count,array_length(info.catchange_xip,1) AS catchange_array_length,info.committed_count,array_length(info.committed_xip,1) AS committed_array_length FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info ORDER BY 2; }
+step "s1_get_logical_snapshot_info" { SELECT info.state, info.catchange_count, array_length(info.catchange_xip, 1) AS catchange_array_length, info.committed_count, array_length(info.committed_xip, 1) AS committed_array_length FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info ORDER BY 2; }
 
 permutation "s0_init" "s0_begin" "s0_savepoint" "s0_truncate" "s1_checkpoint" "s1_get_changes" "s0_commit" "s0_begin" "s0_insert" "s1_checkpoint" "s1_get_changes" "s0_commit" "s1_get_changes" "s1_get_logical_snapshot_info" "s1_get_logical_snapshot_meta"
diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c
index 7a3b963..4b24718 100644
--- a/src/backend/replication/logical/snapbuild.c
+++ b/src/backend/replication/logical/snapbuild.c
@@ -1691,7 +1691,6 @@ ValidateAndRestoreSnapshotFile(SnapBuildOnDisk *ondisk, const char *path, int fd
 							   MemoryContext context)
 {
 	pg_crc32c	checksum;
-	Size		sz;
 
 	/* ----
 	 * Make sure the snapshot had been stored safely to disk, that's normally
@@ -1731,7 +1730,7 @@ ValidateAndRestoreSnapshotFile(SnapBuildOnDisk *ondisk, const char *path, int fd
 	/* restore committed xacts information */
 	if (ondisk->builder.committed.xcnt > 0)
 	{
-		sz = sizeof(TransactionId) * ondisk->builder.committed.xcnt;
+		Size sz = sizeof(TransactionId) * ondisk->builder.committed.xcnt;
 		ondisk->builder.committed.xip = MemoryContextAllocZero(context, sz);
 		SnapBuildRestoreContents(fd, (char *) ondisk->builder.committed.xip, sz, path);
 		COMP_CRC32C(checksum, ondisk->builder.committed.xip, sz);
@@ -1740,7 +1739,7 @@ ValidateAndRestoreSnapshotFile(SnapBuildOnDisk *ondisk, const char *path, int fd
 	/* restore catalog modifying xacts information */
 	if (ondisk->builder.catchange.xcnt > 0)
 	{
-		sz = sizeof(TransactionId) * ondisk->builder.catchange.xcnt;
+		Size sz = sizeof(TransactionId) * ondisk->builder.catchange.xcnt;
 		ondisk->builder.catchange.xip = MemoryContextAllocZero(context, sz);
 		SnapBuildRestoreContents(fd, (char *) ondisk->builder.catchange.xip, sz, path);
 		COMP_CRC32C(checksum, ondisk->builder.catchange.xip, sz);
@@ -1782,13 +1781,15 @@ SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
 
 	fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
 
-	if (fd < 0 && errno == ENOENT)
-		return false;
-	else if (fd < 0)
+	if (fd < 0)
+	{
+		if (errno == ENOENT)
+			return false;
+
 		ereport(ERROR,
 				(errcode_for_file_access(),
 				 errmsg("could not open file \"%s\": %m", path)));
-
+	}
 
 	/* validate and restore the snapshot to 'ondisk' */
 	ValidateAndRestoreSnapshotFile(&ondisk, path, fd, builder->context);
#45Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: Peter Smith (#44)
1 attachment(s)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Tue, Oct 08, 2024 at 04:25:29PM +1100, Peter Smith wrote:

Hi, here are some review comments for patch v11.

Thanks for looking at it!

======
contrib/pg_logicalinspect/specs/logical_inspect.spec

1.
nit - Add some missing spaces after commas (,) in the SQL.

Fine by me, done in v12 attached.

======
doc/src/sgml/pglogicalinspect.sgml

2.
+ <note>
+  <para>
+   The <filename>pg_logicalinspect</filename> functions are called
+   using a text argument that can be extracted from the output name of the
+   <function>pg_ls_logicalsnapdir</function>() function.
+  </para>
+ </note>

2a. wording

The wording "using a text argument that can be extracted" seems like a
hangover from the previous implementation; it does not even say what
that "text argument" means.

That's right (it's mentioned later on (for each function description) that
the argument represents the snapshot file name though).

Why not just say it is a snapshot
filename, something like below?

SUGGESTION:
The pg_logicalinspect functions are called passing a snapshot filename
to be inspected. For example, pass a name obtained from the
pg_ls_logicalsnapdir() function.

Yeah, I like it, but...

~

2b. formatting

nit - In the previous implementation the extraction of the LSN was
trickier, so this part was worthy of an SGML "NOTE". Now that it is
just a filename, I don't know if it needs to be a special note
anymore.

In fact, giving it more thoughts, I think we can just remove this part.
I don't see the extra value anymore and that's something that we may need to
remove depending on what will be added to this module in the future.

I think that having the argument explanation in each function description is
enough, done that way in v12.

~~~

3.
+postgres=# SELECT meta.* FROM pg_ls_logicalsnapdir(),
+pg_get_logical_snapshot_meta(name) AS meta;
+
+-[ RECORD 1 ]--------
+magic    | 1369563137
+checksum | 1028045905
+version  | 6

3a.
If you are going to wrap the SQL across multiple lines like this, then
you should show the psql continuation prompt, so that the example
looks the same as what the user would see.

I'm not sure about this one. If the user copy/paste the doc as it is then there
is no psql continuation prompt. If the user does not copy/paste the doc then he
might indeed see "something" else (but that's not surprising since he did not
copy/paste). FWIW, there is similar examples in pgstatstatements.sgml.

~

3b.
FYI, the output of that can return multiple records,

Yes, as the test in this patch does.

which is
b.i) probably not what you intended to demonstrate
b.ii) not the same as what the example says

e.g., I got this:
test_pub=# SELECT meta.* FROM pg_ls_logicalsnapdir(),
test_pub-# pg_get_logical_snapshot_meta(name) AS meta;
-[ RECORD 1 ]--------
magic | 1369563137
checksum | 681884630
version | 6
-[ RECORD 2 ]--------
magic | 1369563137
checksum | 2213048308
version | 6
-[ RECORD 3 ]--------
magic | 1369563137
checksum | 3812680762
version | 6
-[ RECORD 4 ]--------
magic | 1369563137
checksum | 3759893001
version | 6

I don't get the point here. The examples just show another way to use the functions,
the ouput is more "anecdotal" than anything else.

~~~

(Also those #3a, #3b comments apply to both examples)

======
src/backend/replication/logical/snapbuild.c

4.
- SnapBuild builder;
-
- /* variable amount of TransactionIds follows */
-} SnapBuildOnDisk;
-
#define SnapBuildOnDiskConstantSize \
offsetof(SnapBuildOnDisk, builder)
#define SnapBuildOnDiskNotChecksummedSize \

Is it better to try to keep those "Size" macros defined along with
wherever the SnapBuildOnDisk is defined? Otherwise, if the structure
is ever changed, adjusting the macros could be easily overlooked.

I think that the less we put in the snapbuild_internal.h the better. That said,
I think you have a good point so I added a comment around the SnapBuildOnDisk
definition instead in v12.

~~~

5.
ValidateAndRestoreSnapshotFile

nit - See [1] #4 suggestion to declare 'sz' at scope where used. The
previous reason not to change this (e.g. "mainly inspired from
SnapBuildRestore") seems less relevant because now most lines of this
function have already been modified for other reasons.

Right. I think that's a matter of taste and I do prefer to "only" do the
necessary changes that are linked to the feature the patch is implementing.

~~~

6.
SnapBuildRestore:

+ if (fd < 0 && errno == ENOENT)
+ return false;
+ else if (fd < 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not open file \"%s\": %m", path)));

I think this code fragment looked like this before, and you only
relocated it,

That's right.

but it still seems a bit awkward to write this way.
Since so much else has changed, how about also improving this in
passing, like below:

if (fd < 0)
{
if (errno == ENOENT)
return false;

ereport(ERROR,
(errcode_for_file_access(),
errmsg("could not open file \"%s\": %m", path)));
}

Same, I do prefer to "only" do the necessary changes that are linked to the
feature the patch is implementing (and why stop here, a similar change could be
made in logical/origin.c too for example).

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachments:

v12-0001-Add-contrib-pg_logicalinspect.patchtext/x-diff; charset=us-asciiDownload
From ea6e205761038924ddfe541498890d45f438aafc Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Wed, 14 Aug 2024 08:46:05 +0000
Subject: [PATCH v12] Add contrib/pg_logicalinspect

Provides SQL functions that allow to inspect logical decoding components.

It currently allows to inspect the contents of serialized logical snapshots of
a running database cluster, which is useful for debugging or educational
purposes.
---
 contrib/Makefile                              |   1 +
 contrib/meson.build                           |   1 +
 contrib/pg_logicalinspect/.gitignore          |   4 +
 contrib/pg_logicalinspect/Makefile            |  31 ++
 .../expected/logical_inspect.out              |  52 ++++
 contrib/pg_logicalinspect/logicalinspect.conf |   1 +
 contrib/pg_logicalinspect/meson.build         |  39 +++
 .../pg_logicalinspect--1.0.sql                |  43 +++
 contrib/pg_logicalinspect/pg_logicalinspect.c | 199 +++++++++++++
 .../pg_logicalinspect.control                 |   5 +
 .../specs/logical_inspect.spec                |  34 +++
 doc/src/sgml/contrib.sgml                     |   1 +
 doc/src/sgml/filelist.sgml                    |   1 +
 doc/src/sgml/pglogicalinspect.sgml            | 142 +++++++++
 src/backend/replication/logical/snapbuild.c   | 279 ++++--------------
 src/include/replication/snapbuild.h           |   6 +-
 src/include/replication/snapbuild_internal.h  | 199 +++++++++++++
 17 files changed, 817 insertions(+), 221 deletions(-)
   7.5% contrib/pg_logicalinspect/expected/
   5.3% contrib/pg_logicalinspect/specs/
  28.0% contrib/pg_logicalinspect/
  13.5% doc/src/sgml/
  25.1% src/backend/replication/logical/
  20.2% src/include/replication/

diff --git a/contrib/Makefile b/contrib/Makefile
index abd780f277..952855d9b6 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -32,6 +32,7 @@ SUBDIRS = \
 		passwordcheck	\
 		pg_buffercache	\
 		pg_freespacemap \
+		pg_logicalinspect \
 		pg_prewarm	\
 		pg_stat_statements \
 		pg_surgery	\
diff --git a/contrib/meson.build b/contrib/meson.build
index 14a8906865..159ff41555 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -46,6 +46,7 @@ subdir('passwordcheck')
 subdir('pg_buffercache')
 subdir('pgcrypto')
 subdir('pg_freespacemap')
+subdir('pg_logicalinspect')
 subdir('pg_prewarm')
 subdir('pgrowlocks')
 subdir('pg_stat_statements')
diff --git a/contrib/pg_logicalinspect/.gitignore b/contrib/pg_logicalinspect/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/contrib/pg_logicalinspect/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/contrib/pg_logicalinspect/Makefile b/contrib/pg_logicalinspect/Makefile
new file mode 100644
index 0000000000..55124514d4
--- /dev/null
+++ b/contrib/pg_logicalinspect/Makefile
@@ -0,0 +1,31 @@
+# contrib/pg_logicalinspect/Makefile
+
+MODULE_big = pg_logicalinspect
+OBJS = \
+	$(WIN32RES) \
+	pg_logicalinspect.o
+PGFILEDESC = "pg_logicalinspect - functions to inspect logical decoding components"
+
+EXTENSION = pg_logicalinspect
+DATA = pg_logicalinspect--1.0.sql
+
+EXTRA_INSTALL = contrib/test_decoding
+
+ISOLATION = logical_inspect
+
+ISOLATION_OPTS = --temp-config $(top_srcdir)/contrib/pg_logicalinspect/logicalinspect.conf
+
+# Disabled because these tests require "wal_level=logical", which
+# some installcheck users do not have (e.g. buildfarm clients).
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/pg_logicalinspect
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/pg_logicalinspect/expected/logical_inspect.out b/contrib/pg_logicalinspect/expected/logical_inspect.out
new file mode 100644
index 0000000000..d95efa4d1e
--- /dev/null
+++ b/contrib/pg_logicalinspect/expected/logical_inspect.out
@@ -0,0 +1,52 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s0_init s0_begin s0_savepoint s0_truncate s1_checkpoint s1_get_changes s0_commit s0_begin s0_insert s1_checkpoint s1_get_changes s0_commit s1_get_changes s1_get_logical_snapshot_info s1_get_logical_snapshot_meta
+step s0_init: SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding');
+?column?
+--------
+init    
+(1 row)
+
+step s0_begin: BEGIN;
+step s0_savepoint: SAVEPOINT sp1;
+step s0_truncate: TRUNCATE tbl1;
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data
+----
+(0 rows)
+
+step s0_commit: COMMIT;
+step s0_begin: BEGIN;
+step s0_insert: INSERT INTO tbl1 VALUES (1);
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                   
+---------------------------------------
+BEGIN                                  
+table public.tbl1: TRUNCATE: (no-flags)
+COMMIT                                 
+(3 rows)
+
+step s0_commit: COMMIT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                                         
+-------------------------------------------------------------
+BEGIN                                                        
+table public.tbl1: INSERT: val1[integer]:1 val2[integer]:null
+COMMIT                                                       
+(3 rows)
+
+step s1_get_logical_snapshot_info: SELECT info.state, info.catchange_count, array_length(info.catchange_xip,1) AS catchange_array_length, info.committed_count, array_length(info.committed_xip,1) AS committed_array_length FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info ORDER BY 2;
+state     |catchange_count|catchange_array_length|committed_count|committed_array_length
+----------+---------------+----------------------+---------------+----------------------
+consistent|              0|                      |              2|                     2
+consistent|              2|                     2|              0|                      
+(2 rows)
+
+step s1_get_logical_snapshot_meta: SELECT COUNT(meta.*) from pg_ls_logicalsnapdir(), pg_get_logical_snapshot_meta(name) as meta;
+count
+-----
+    2
+(1 row)
+
diff --git a/contrib/pg_logicalinspect/logicalinspect.conf b/contrib/pg_logicalinspect/logicalinspect.conf
new file mode 100644
index 0000000000..e3d257315f
--- /dev/null
+++ b/contrib/pg_logicalinspect/logicalinspect.conf
@@ -0,0 +1 @@
+wal_level = logical
diff --git a/contrib/pg_logicalinspect/meson.build b/contrib/pg_logicalinspect/meson.build
new file mode 100644
index 0000000000..3ec635509b
--- /dev/null
+++ b/contrib/pg_logicalinspect/meson.build
@@ -0,0 +1,39 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+pg_logicalinspect_sources = files('pg_logicalinspect.c')
+
+if host_system == 'windows'
+  pg_logicalinspect_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+    '--NAME', 'pg_logicalinspect',
+    '--FILEDESC', 'pg_logicalinspect - functions to inspect logical decoding components',])
+endif
+
+pg_logicalinspect = shared_module('pg_logicalinspect',
+  pg_logicalinspect_sources,
+  kwargs: contrib_mod_args + {
+      'dependencies': contrib_mod_args['dependencies'],
+  },
+)
+contrib_targets += pg_logicalinspect
+
+install_data(
+  'pg_logicalinspect.control',
+  'pg_logicalinspect--1.0.sql',
+  kwargs: contrib_data_args,
+)
+
+tests += {
+  'name': 'pg_logicalinspect',
+  'sd': meson.current_source_dir(),
+  'bd': meson.current_build_dir(),
+  'isolation': {
+    'specs': [
+      'logical_inspect',
+    ],
+    'regress_args': [
+      '--temp-config', files('logicalinspect.conf'),
+    ],
+    # see above
+    'runningcheck': false,
+  },
+}
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql b/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
new file mode 100644
index 0000000000..c773f6e458
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
@@ -0,0 +1,43 @@
+/* contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_logicalinspect" to load this file. \quit
+
+--
+-- pg_get_logical_snapshot_meta()
+--
+CREATE FUNCTION pg_get_logical_snapshot_meta(IN filename text,
+    OUT magic int4,
+    OUT checksum int8,
+    OUT version int4
+)
+AS 'MODULE_PATHNAME', 'pg_get_logical_snapshot_meta'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_logical_snapshot_meta(text) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_logical_snapshot_meta(text) TO pg_read_server_files;
+
+--
+-- pg_get_logical_snapshot_info()
+--
+CREATE FUNCTION pg_get_logical_snapshot_info(IN filename text,
+    OUT state text,
+    OUT xmin xid,
+    OUT xmax xid,
+    OUT start_decoding_at pg_lsn,
+    OUT two_phase_at pg_lsn,
+    OUT initial_xmin_horizon xid,
+    OUT building_full_snapshot boolean,
+    OUT in_slot_creation boolean,
+    OUT last_serialized_snapshot pg_lsn,
+    OUT next_phase_at xid,
+    OUT committed_count int8,
+    OUT committed_xip xid[],
+    OUT catchange_count int8,
+    OUT catchange_xip xid[]
+)
+AS 'MODULE_PATHNAME', 'pg_get_logical_snapshot_info'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_logical_snapshot_info(text) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_logical_snapshot_info(text) TO pg_read_server_files;
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.c b/contrib/pg_logicalinspect/pg_logicalinspect.c
new file mode 100644
index 0000000000..0e3e1f50fc
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.c
@@ -0,0 +1,199 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_logicalinspect.c
+ *		  Functions to inspect contents of PostgreSQL logical snapshots
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ *		  contrib/pg_logicalinspect/pg_logicalinspect.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "funcapi.h"
+#include "replication/snapbuild_internal.h"
+#include "utils/array.h"
+#include "utils/builtins.h"
+#include "utils/pg_lsn.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_meta);
+PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_info);
+
+/* Return the description of SnapBuildState */
+static const char *
+get_snapbuild_state_desc(SnapBuildState state)
+{
+	const char *stateDesc = "unknown state";
+
+	switch (state)
+	{
+		case SNAPBUILD_START:
+			stateDesc = "start";
+			break;
+		case SNAPBUILD_BUILDING_SNAPSHOT:
+			stateDesc = "building";
+			break;
+		case SNAPBUILD_FULL_SNAPSHOT:
+			stateDesc = "full";
+			break;
+		case SNAPBUILD_CONSISTENT:
+			stateDesc = "consistent";
+			break;
+	}
+
+	return stateDesc;
+}
+
+/*
+ * Retrieve the logical snapshot file metadata.
+ */
+Datum
+pg_get_logical_snapshot_meta(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_META_COLS 3
+	SnapBuildOnDisk ondisk;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_META_COLS];
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_META_COLS];
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+	MemoryContext context;
+	int			fd;
+	int			i = 0;
+	text	   *filename_t = PG_GETARG_TEXT_PP(0);
+
+	sprintf(path, "%s/%s",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			text_to_cstring(filename_t));
+
+	fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
+
+	if (fd < 0)
+		ereport(ERROR,
+				(errcode_for_file_access(),
+				 errmsg("could not open file \"%s\": %m", path)));
+
+	context = AllocSetContextCreate(CurrentMemoryContext,
+									"logicalsnapshot inspect context",
+									ALLOCSET_DEFAULT_SIZES);
+
+	/* Validate and restore the snapshot to 'ondisk' */
+	ValidateAndRestoreSnapshotFile(&ondisk, path, fd, context);
+
+	/* Build a tuple descriptor for our result type */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	memset(nulls, 0, sizeof(nulls));
+
+	values[i++] = UInt32GetDatum(ondisk.magic);
+	values[i++] = Int64GetDatum((int64) ondisk.checksum);
+	values[i++] = UInt32GetDatum(ondisk.version);
+
+	Assert(i == PG_GET_LOGICAL_SNAPSHOT_META_COLS);
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	MemoryContextReset(context);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_META_COLS
+}
+
+Datum
+pg_get_logical_snapshot_info(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_INFO_COLS 14
+	SnapBuildOnDisk ondisk;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS];
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS];
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+	MemoryContext context;
+	int			fd;
+	int			i = 0;
+	text	   *filename_t = PG_GETARG_TEXT_PP(0);
+
+	sprintf(path, "%s/%s",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			text_to_cstring(filename_t));
+
+	fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
+
+	if (fd < 0)
+		ereport(ERROR,
+				(errcode_for_file_access(),
+				 errmsg("could not open file \"%s\": %m", path)));
+
+	context = AllocSetContextCreate(CurrentMemoryContext,
+									"logicalsnapshot inspect context",
+									ALLOCSET_DEFAULT_SIZES);
+
+	/* Validate and restore the snapshot to 'ondisk' */
+	ValidateAndRestoreSnapshotFile(&ondisk, path, fd, context);
+
+	/* Build a tuple descriptor for our result type */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	memset(nulls, 0, sizeof(nulls));
+
+	values[i++] = CStringGetTextDatum(get_snapbuild_state_desc(ondisk.builder.state));
+	values[i++] = TransactionIdGetDatum(ondisk.builder.xmin);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.xmax);
+	values[i++] = LSNGetDatum(ondisk.builder.start_decoding_at);
+	values[i++] = LSNGetDatum(ondisk.builder.two_phase_at);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.initial_xmin_horizon);
+	values[i++] = BoolGetDatum(ondisk.builder.building_full_snapshot);
+	values[i++] = BoolGetDatum(ondisk.builder.in_slot_creation);
+	values[i++] = LSNGetDatum(ondisk.builder.last_serialized_snapshot);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.next_phase_at);
+
+	values[i++] = Int64GetDatum(ondisk.builder.committed.xcnt);
+	if (ondisk.builder.committed.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+		int			narrayelems = 0;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.committed.xcnt * sizeof(Datum));
+
+		for (; narrayelems < ondisk.builder.committed.xcnt; narrayelems++)
+			arrayelems[narrayelems] = Int64GetDatum((int64) ondisk.builder.committed.xip[narrayelems]);
+
+		values[i++] = PointerGetDatum(construct_array_builtin(arrayelems, narrayelems, INT8OID));
+	}
+	else
+		nulls[i++] = true;
+
+	values[i++] = Int64GetDatum(ondisk.builder.catchange.xcnt);
+	if (ondisk.builder.catchange.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+		int			narrayelems = 0;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.catchange.xcnt * sizeof(Datum));
+
+		for (; narrayelems < ondisk.builder.catchange.xcnt; narrayelems++)
+			arrayelems[narrayelems] = Int64GetDatum((int64) ondisk.builder.catchange.xip[narrayelems]);
+
+		values[i++] = PointerGetDatum(construct_array_builtin(arrayelems, narrayelems, INT8OID));
+	}
+	else
+		nulls[i++] = true;
+
+	Assert(i == PG_GET_LOGICAL_SNAPSHOT_INFO_COLS);
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	MemoryContextReset(context);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_INFO_COLS
+}
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.control b/contrib/pg_logicalinspect/pg_logicalinspect.control
new file mode 100644
index 0000000000..b4a70e57ba
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.control
@@ -0,0 +1,5 @@
+# pg_logicalinspect extension
+comment = 'functions to inspect logical decoding components'
+default_version = '1.0'
+module_pathname = '$libdir/pg_logicalinspect'
+relocatable = true
diff --git a/contrib/pg_logicalinspect/specs/logical_inspect.spec b/contrib/pg_logicalinspect/specs/logical_inspect.spec
new file mode 100644
index 0000000000..9851a6c18e
--- /dev/null
+++ b/contrib/pg_logicalinspect/specs/logical_inspect.spec
@@ -0,0 +1,34 @@
+# Test the pg_logicalinspect functions: that needs some permutation to
+# ensure that we are creating multiple logical snapshots and that one of them
+# contains ongoing catalogs changes.
+setup
+{
+    DROP TABLE IF EXISTS tbl1;
+    CREATE TABLE tbl1 (val1 integer, val2 integer);
+    CREATE EXTENSION pg_logicalinspect;
+}
+
+teardown
+{
+    DROP TABLE tbl1;
+    SELECT 'stop' FROM pg_drop_replication_slot('isolation_slot');
+    DROP EXTENSION pg_logicalinspect;
+}
+
+session "s0"
+setup { SET synchronous_commit=on; }
+step "s0_init" { SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding'); }
+step "s0_begin" { BEGIN; }
+step "s0_savepoint" { SAVEPOINT sp1; }
+step "s0_truncate" { TRUNCATE tbl1; }
+step "s0_insert" { INSERT INTO tbl1 VALUES (1); }
+step "s0_commit" { COMMIT; }
+
+session "s1"
+setup { SET synchronous_commit=on; }
+step "s1_checkpoint" { CHECKPOINT; }
+step "s1_get_changes" { SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0'); }
+step "s1_get_logical_snapshot_meta" { SELECT COUNT(meta.*) from pg_ls_logicalsnapdir(), pg_get_logical_snapshot_meta(name) as meta;}
+step "s1_get_logical_snapshot_info" { SELECT info.state, info.catchange_count, array_length(info.catchange_xip,1) AS catchange_array_length, info.committed_count, array_length(info.committed_xip,1) AS committed_array_length FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info ORDER BY 2; }
+
+permutation "s0_init" "s0_begin" "s0_savepoint" "s0_truncate" "s1_checkpoint" "s1_get_changes" "s0_commit" "s0_begin" "s0_insert" "s1_checkpoint" "s1_get_changes" "s0_commit" "s1_get_changes" "s1_get_logical_snapshot_info" "s1_get_logical_snapshot_meta"
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index 44639a8dca..7c381949a5 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -154,6 +154,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
  &pgbuffercache;
  &pgcrypto;
  &pgfreespacemap;
+ &pglogicalinspect;
  &pgprewarm;
  &pgrowlocks;
  &pgstatstatements;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index a7ff5f8264..66e6dccd4c 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -143,6 +143,7 @@
 <!ENTITY pgbuffercache   SYSTEM "pgbuffercache.sgml">
 <!ENTITY pgcrypto        SYSTEM "pgcrypto.sgml">
 <!ENTITY pgfreespacemap  SYSTEM "pgfreespacemap.sgml">
+<!ENTITY pglogicalinspect  SYSTEM "pglogicalinspect.sgml">
 <!ENTITY pgprewarm       SYSTEM "pgprewarm.sgml">
 <!ENTITY pgrowlocks      SYSTEM "pgrowlocks.sgml">
 <!ENTITY pgstatstatements SYSTEM "pgstatstatements.sgml">
diff --git a/doc/src/sgml/pglogicalinspect.sgml b/doc/src/sgml/pglogicalinspect.sgml
new file mode 100644
index 0000000000..e984979462
--- /dev/null
+++ b/doc/src/sgml/pglogicalinspect.sgml
@@ -0,0 +1,142 @@
+<!-- doc/src/sgml/pglogicalinspect.sgml -->
+
+<sect1 id="pglogicalinspect" xreflabel="pg_logicalinspect">
+ <title>pg_logicalinspect &mdash; logical decoding components inspection</title>
+
+ <indexterm zone="pglogicalinspect">
+  <primary>pg_logicalinspect</primary>
+ </indexterm>
+
+ <para>
+  The <filename>pg_logicalinspect</filename> module provides SQL functions
+  that allow you to inspect the contents of logical decoding components. It
+  allows the inspection of serialized logical snapshots of a running
+  <productname>PostgreSQL</productname> database cluster, which is useful
+  for debugging or educational purposes.
+ </para>
+
+ <para>
+  By default, use of these functions is restricted to superusers and members of
+  the <literal>pg_read_server_files</literal> role. Access may be granted by
+  superusers to others using <command>GRANT</command>.
+ </para>
+
+ <sect2 id="pglogicalinspect-funcs">
+  <title>General Functions</title>
+
+  <variablelist>
+   <varlistentry id="pglogicalinspect-funcs-pg-get-logical-snapshot-meta">
+    <term>
+     <function>pg_get_logical_snapshot_meta(filename text) returns record</function>
+    </term>
+
+    <listitem>
+     <para>
+      Gets logical snapshot metadata about a snapshot file that is located in
+      the server's <filename>pg_logical/snapshots</filename> directory.
+      The <replaceable>filename</replaceable> argument represents the snapshot
+      file name.
+      For example:
+<screen>
+postgres=# SELECT * FROM pg_ls_logicalsnapdir();
+-[ RECORD 1 ]+-----------------------
+name         | 0-40796E18.snap
+size         | 152
+modification | 2024-08-14 16:36:32+00
+
+postgres=# SELECT * FROM pg_get_logical_snapshot_meta('0-40796E18.snap');
+-[ RECORD 1 ]--------
+magic    | 1369563137
+checksum | 1028045905
+version  | 6
+
+postgres=# SELECT meta.* FROM pg_ls_logicalsnapdir(),
+pg_get_logical_snapshot_meta(name) AS meta;
+
+-[ RECORD 1 ]--------
+magic    | 1369563137
+checksum | 1028045905
+version  | 6
+</screen>
+     </para>
+     <para>
+      If <replaceable>filename</replaceable> does not match a snapshot file, the
+      function raises an error.
+     </para>
+    </listitem>
+   </varlistentry>
+
+   <varlistentry id="pglogicalinspect-funcs-pg-get-logical-snapshot-info">
+    <term>
+     <function>pg_get_logical_snapshot_info(filename text) returns record</function>
+    </term>
+
+    <listitem>
+     <para>
+      Gets logical snapshot information about a snapshot file that is located in
+      the server's <filename>pg_logical/snapshots</filename> directory.
+      The <replaceable>filename</replaceable> argument represents the snapshot
+      file name.
+      For example:
+<screen>
+postgres=# SELECT * FROM pg_ls_logicalsnapdir();
+-[ RECORD 1 ]+-----------------------
+name         | 0-40796E18.snap
+size         | 152
+modification | 2024-08-14 16:36:32+00
+
+postgres=# SELECT * FROM pg_get_logical_snapshot_info('0-40796E18.snap');
+-[ RECORD 1 ]------------+-----------
+state                    | consistent
+xmin                     | 751
+xmax                     | 751
+start_decoding_at        | 0/40796AF8
+two_phase_at             | 0/40796AF8
+initial_xmin_horizon     | 0
+building_full_snapshot   | f
+in_slot_creation         | f
+last_serialized_snapshot | 0/0
+next_phase_at            | 0
+committed_count          | 0
+committed_xip            |
+catchange_count          | 2
+catchange_xip            | {751,752}
+
+postgres=# SELECT info.* FROM pg_ls_logicalsnapdir(),
+pg_get_logical_snapshot_info(name) AS info;
+-[ RECORD 1 ]------------+-----------
+state                    | consistent
+xmin                     | 751
+xmax                     | 751
+start_decoding_at        | 0/40796AF8
+two_phase_at             | 0/40796AF8
+initial_xmin_horizon     | 0
+building_full_snapshot   | f
+in_slot_creation         | f
+last_serialized_snapshot | 0/0
+next_phase_at            | 0
+committed_count          | 0
+committed_xip            |
+catchange_count          | 2
+catchange_xip            | {751,752}
+</screen>
+     </para>
+     <para>
+      If <replaceable>filename</replaceable> does not match a snapshot file, the
+      function raises an error.
+     </para>
+    </listitem>
+   </varlistentry>
+
+  </variablelist>
+ </sect2>
+
+ <sect2 id="pglogicalinspect-author">
+  <title>Author</title>
+
+  <para>
+   Bertrand Drouvot <email>bertranddrouvot.pg@gmail.com</email>
+  </para>
+ </sect2>
+
+</sect1>
diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c
index 0450f94ba8..7a3b963a2f 100644
--- a/src/backend/replication/logical/snapbuild.c
+++ b/src/backend/replication/logical/snapbuild.c
@@ -134,6 +134,7 @@
 #include "replication/logical.h"
 #include "replication/reorderbuffer.h"
 #include "replication/snapbuild.h"
+#include "replication/snapbuild_internal.h"
 #include "storage/fd.h"
 #include "storage/lmgr.h"
 #include "storage/proc.h"
@@ -143,146 +144,6 @@
 #include "utils/memutils.h"
 #include "utils/snapmgr.h"
 #include "utils/snapshot.h"
-
-/*
- * This struct contains the current state of the snapshot building
- * machinery. Besides a forward declaration in the header, it is not exposed
- * to the public, so we can easily change its contents.
- */
-struct SnapBuild
-{
-	/* how far are we along building our first full snapshot */
-	SnapBuildState state;
-
-	/* private memory context used to allocate memory for this module. */
-	MemoryContext context;
-
-	/* all transactions < than this have committed/aborted */
-	TransactionId xmin;
-
-	/* all transactions >= than this are uncommitted */
-	TransactionId xmax;
-
-	/*
-	 * Don't replay commits from an LSN < this LSN. This can be set externally
-	 * but it will also be advanced (never retreat) from within snapbuild.c.
-	 */
-	XLogRecPtr	start_decoding_at;
-
-	/*
-	 * LSN at which two-phase decoding was enabled or LSN at which we found a
-	 * consistent point at the time of slot creation.
-	 *
-	 * The prepared transactions, that were skipped because previously
-	 * two-phase was not enabled or are not covered by initial snapshot, need
-	 * to be sent later along with commit prepared and they must be before
-	 * this point.
-	 */
-	XLogRecPtr	two_phase_at;
-
-	/*
-	 * Don't start decoding WAL until the "xl_running_xacts" information
-	 * indicates there are no running xids with an xid smaller than this.
-	 */
-	TransactionId initial_xmin_horizon;
-
-	/* Indicates if we are building full snapshot or just catalog one. */
-	bool		building_full_snapshot;
-
-	/*
-	 * Indicates if we are using the snapshot builder for the creation of a
-	 * logical replication slot. If it's true, the start point for decoding
-	 * changes is not determined yet. So we skip snapshot restores to properly
-	 * find the start point. See SnapBuildFindSnapshot() for details.
-	 */
-	bool		in_slot_creation;
-
-	/*
-	 * Snapshot that's valid to see the catalog state seen at this moment.
-	 */
-	Snapshot	snapshot;
-
-	/*
-	 * LSN of the last location we are sure a snapshot has been serialized to.
-	 */
-	XLogRecPtr	last_serialized_snapshot;
-
-	/*
-	 * The reorderbuffer we need to update with usable snapshots et al.
-	 */
-	ReorderBuffer *reorder;
-
-	/*
-	 * TransactionId at which the next phase of initial snapshot building will
-	 * happen. InvalidTransactionId if not known (i.e. SNAPBUILD_START), or
-	 * when no next phase necessary (SNAPBUILD_CONSISTENT).
-	 */
-	TransactionId next_phase_at;
-
-	/*
-	 * Array of transactions which could have catalog changes that committed
-	 * between xmin and xmax.
-	 */
-	struct
-	{
-		/* number of committed transactions */
-		size_t		xcnt;
-
-		/* available space for committed transactions */
-		size_t		xcnt_space;
-
-		/*
-		 * Until we reach a CONSISTENT state, we record commits of all
-		 * transactions, not just the catalog changing ones. Record when that
-		 * changes so we know we cannot export a snapshot safely anymore.
-		 */
-		bool		includes_all_transactions;
-
-		/*
-		 * Array of committed transactions that have modified the catalog.
-		 *
-		 * As this array is frequently modified we do *not* keep it in
-		 * xidComparator order. Instead we sort the array when building &
-		 * distributing a snapshot.
-		 *
-		 * TODO: It's unclear whether that reasoning has much merit. Every
-		 * time we add something here after becoming consistent will also
-		 * require distributing a snapshot. Storing them sorted would
-		 * potentially also make it easier to purge (but more complicated wrt
-		 * wraparound?). Should be improved if sorting while building the
-		 * snapshot shows up in profiles.
-		 */
-		TransactionId *xip;
-	}			committed;
-
-	/*
-	 * Array of transactions and subtransactions that had modified catalogs
-	 * and were running when the snapshot was serialized.
-	 *
-	 * We normally rely on some WAL record types such as HEAP2_NEW_CID to know
-	 * if the transaction has changed the catalog. But it could happen that
-	 * the logical decoding decodes only the commit record of the transaction
-	 * after restoring the previously serialized snapshot in which case we
-	 * will miss adding the xid to the snapshot and end up looking at the
-	 * catalogs with the wrong snapshot.
-	 *
-	 * Now to avoid the above problem, we serialize the transactions that had
-	 * modified the catalogs and are still running at the time of snapshot
-	 * serialization. We fill this array while restoring the snapshot and then
-	 * refer it while decoding commit to ensure if the xact has modified the
-	 * catalog. We discard this array when all the xids in the list become old
-	 * enough to matter. See SnapBuildPurgeOlderTxn for details.
-	 */
-	struct
-	{
-		/* number of transactions */
-		size_t		xcnt;
-
-		/* This array must be sorted in xidComparator order */
-		TransactionId *xip;
-	}			catchange;
-};
-
 /*
  * Starting a transaction -- which we need to do while exporting a snapshot --
  * removes knowledge about the previously used resowner, so we save it here.
@@ -1557,40 +1418,6 @@ SnapBuildWaitSnapshot(xl_running_xacts *running, TransactionId cutoff)
 	}
 }
 
-/* -----------------------------------
- * Snapshot serialization support
- * -----------------------------------
- */
-
-/*
- * We store current state of struct SnapBuild on disk in the following manner:
- *
- * struct SnapBuildOnDisk;
- * TransactionId * committed.xcnt; (*not xcnt_space*)
- * TransactionId * catchange.xcnt;
- *
- */
-typedef struct SnapBuildOnDisk
-{
-	/* first part of this struct needs to be version independent */
-
-	/* data not covered by checksum */
-	uint32		magic;
-	pg_crc32c	checksum;
-
-	/* data covered by checksum */
-
-	/* version, in case we want to support pg_upgrade */
-	uint32		version;
-	/* how large is the on disk data, excluding the constant sized part */
-	uint32		length;
-
-	/* version dependent part */
-	SnapBuild	builder;
-
-	/* variable amount of TransactionIds follows */
-} SnapBuildOnDisk;
-
 #define SnapBuildOnDiskConstantSize \
 	offsetof(SnapBuildOnDisk, builder)
 #define SnapBuildOnDiskNotChecksummedSize \
@@ -1857,34 +1684,14 @@ out:
 }
 
 /*
- * Restore a snapshot into 'builder' if previously one has been stored at the
- * location indicated by 'lsn'. Returns true if successful, false otherwise.
+ * Validate the logical snapshot file and read its contents to 'ondisk'.
  */
-static bool
-SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
+void
+ValidateAndRestoreSnapshotFile(SnapBuildOnDisk *ondisk, const char *path, int fd,
+							   MemoryContext context)
 {
-	SnapBuildOnDisk ondisk;
-	int			fd;
-	char		path[MAXPGPATH];
-	Size		sz;
 	pg_crc32c	checksum;
-
-	/* no point in loading a snapshot if we're already there */
-	if (builder->state == SNAPBUILD_CONSISTENT)
-		return false;
-
-	sprintf(path, "%s/%X-%X.snap",
-			PG_LOGICAL_SNAPSHOTS_DIR,
-			LSN_FORMAT_ARGS(lsn));
-
-	fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
-
-	if (fd < 0 && errno == ENOENT)
-		return false;
-	else if (fd < 0)
-		ereport(ERROR,
-				(errcode_for_file_access(),
-				 errmsg("could not open file \"%s\": %m", path)));
+	Size		sz;
 
 	/* ----
 	 * Make sure the snapshot had been stored safely to disk, that's normally
@@ -1897,47 +1704,46 @@ SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
 	fsync_fname(path, false);
 	fsync_fname(PG_LOGICAL_SNAPSHOTS_DIR, true);
 
-
 	/* read statically sized portion of snapshot */
-	SnapBuildRestoreContents(fd, (char *) &ondisk, SnapBuildOnDiskConstantSize, path);
+	SnapBuildRestoreContents(fd, (char *) ondisk, SnapBuildOnDiskConstantSize, path);
 
-	if (ondisk.magic != SNAPBUILD_MAGIC)
+	if (ondisk->magic != SNAPBUILD_MAGIC)
 		ereport(ERROR,
 				(errcode(ERRCODE_DATA_CORRUPTED),
 				 errmsg("snapbuild state file \"%s\" has wrong magic number: %u instead of %u",
-						path, ondisk.magic, SNAPBUILD_MAGIC)));
+						path, ondisk->magic, SNAPBUILD_MAGIC)));
 
-	if (ondisk.version != SNAPBUILD_VERSION)
+	if (ondisk->version != SNAPBUILD_VERSION)
 		ereport(ERROR,
 				(errcode(ERRCODE_DATA_CORRUPTED),
 				 errmsg("snapbuild state file \"%s\" has unsupported version: %u instead of %u",
-						path, ondisk.version, SNAPBUILD_VERSION)));
+						path, ondisk->version, SNAPBUILD_VERSION)));
 
 	INIT_CRC32C(checksum);
 	COMP_CRC32C(checksum,
-				((char *) &ondisk) + SnapBuildOnDiskNotChecksummedSize,
+				((char *) ondisk) + SnapBuildOnDiskNotChecksummedSize,
 				SnapBuildOnDiskConstantSize - SnapBuildOnDiskNotChecksummedSize);
 
 	/* read SnapBuild */
-	SnapBuildRestoreContents(fd, (char *) &ondisk.builder, sizeof(SnapBuild), path);
-	COMP_CRC32C(checksum, &ondisk.builder, sizeof(SnapBuild));
+	SnapBuildRestoreContents(fd, (char *) &ondisk->builder, sizeof(SnapBuild), path);
+	COMP_CRC32C(checksum, &ondisk->builder, sizeof(SnapBuild));
 
 	/* restore committed xacts information */
-	if (ondisk.builder.committed.xcnt > 0)
+	if (ondisk->builder.committed.xcnt > 0)
 	{
-		sz = sizeof(TransactionId) * ondisk.builder.committed.xcnt;
-		ondisk.builder.committed.xip = MemoryContextAllocZero(builder->context, sz);
-		SnapBuildRestoreContents(fd, (char *) ondisk.builder.committed.xip, sz, path);
-		COMP_CRC32C(checksum, ondisk.builder.committed.xip, sz);
+		sz = sizeof(TransactionId) * ondisk->builder.committed.xcnt;
+		ondisk->builder.committed.xip = MemoryContextAllocZero(context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.committed.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.committed.xip, sz);
 	}
 
 	/* restore catalog modifying xacts information */
-	if (ondisk.builder.catchange.xcnt > 0)
+	if (ondisk->builder.catchange.xcnt > 0)
 	{
-		sz = sizeof(TransactionId) * ondisk.builder.catchange.xcnt;
-		ondisk.builder.catchange.xip = MemoryContextAllocZero(builder->context, sz);
-		SnapBuildRestoreContents(fd, (char *) ondisk.builder.catchange.xip, sz, path);
-		COMP_CRC32C(checksum, ondisk.builder.catchange.xip, sz);
+		sz = sizeof(TransactionId) * ondisk->builder.catchange.xcnt;
+		ondisk->builder.catchange.xip = MemoryContextAllocZero(context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.catchange.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.catchange.xip, sz);
 	}
 
 	if (CloseTransientFile(fd) != 0)
@@ -1948,11 +1754,44 @@ SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
 	FIN_CRC32C(checksum);
 
 	/* verify checksum of what we've read */
-	if (!EQ_CRC32C(checksum, ondisk.checksum))
+	if (!EQ_CRC32C(checksum, ondisk->checksum))
 		ereport(ERROR,
 				(errcode(ERRCODE_DATA_CORRUPTED),
 				 errmsg("checksum mismatch for snapbuild state file \"%s\": is %u, should be %u",
-						path, checksum, ondisk.checksum)));
+						path, checksum, ondisk->checksum)));
+}
+
+/*
+ * Restore a snapshot into 'builder' if previously one has been stored at the
+ * location indicated by 'lsn'. Returns true if successful, false otherwise.
+ */
+static bool
+SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
+{
+	SnapBuildOnDisk ondisk;
+	int			fd;
+	char		path[MAXPGPATH];
+
+	/* no point in loading a snapshot if we're already there */
+	if (builder->state == SNAPBUILD_CONSISTENT)
+		return false;
+
+	sprintf(path, "%s/%X-%X.snap",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			LSN_FORMAT_ARGS(lsn));
+
+	fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
+
+	if (fd < 0 && errno == ENOENT)
+		return false;
+	else if (fd < 0)
+		ereport(ERROR,
+				(errcode_for_file_access(),
+				 errmsg("could not open file \"%s\": %m", path)));
+
+
+	/* validate and restore the snapshot to 'ondisk' */
+	ValidateAndRestoreSnapshotFile(&ondisk, path, fd, builder->context);
 
 	/*
 	 * ok, we now have a sensible snapshot here, figure out if it has more
diff --git a/src/include/replication/snapbuild.h b/src/include/replication/snapbuild.h
index caa5113ff8..3c1454df99 100644
--- a/src/include/replication/snapbuild.h
+++ b/src/include/replication/snapbuild.h
@@ -15,6 +15,10 @@
 #include "access/xlogdefs.h"
 #include "utils/snapmgr.h"
 
+/*
+ * Please keep get_snapbuild_state_desc() (located in the pg_logicalinspect
+ * module) updated if a change needs to be made to SnapBuildState.
+ */
 typedef enum
 {
 	/*
@@ -46,7 +50,7 @@ typedef enum
 	SNAPBUILD_CONSISTENT = 2,
 } SnapBuildState;
 
-/* forward declare so we don't have to expose the struct to the public */
+/* forward declare so we don't have to include snapbuild_internal.h */
 struct SnapBuild;
 typedef struct SnapBuild SnapBuild;
 
diff --git a/src/include/replication/snapbuild_internal.h b/src/include/replication/snapbuild_internal.h
new file mode 100644
index 0000000000..4791a90991
--- /dev/null
+++ b/src/include/replication/snapbuild_internal.h
@@ -0,0 +1,199 @@
+/*-------------------------------------------------------------------------
+ *
+ * snapbuild_internal.h
+ *    This file contains declarations for logical decoding utility
+ *    functions for internal use.
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * src/include/replication/snapbuild_internal.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef SNAPBUILD_INTERNAL_H
+#define SNAPBUILD_INTERNAL_H
+
+#include "port/pg_crc32c.h"
+#include "replication/reorderbuffer.h"
+#include "replication/snapbuild.h"
+
+/*
+ * This struct contains the current state of the snapshot building
+ * machinery. It is exposed to the public, so pay attention when changing its
+ * contents.
+ */
+typedef struct SnapBuild
+{
+	/* how far are we along building our first full snapshot */
+	SnapBuildState state;
+
+	/* private memory context used to allocate memory for this module. */
+	MemoryContext context;
+
+	/* all transactions < than this have committed/aborted */
+	TransactionId xmin;
+
+	/* all transactions >= than this are uncommitted */
+	TransactionId xmax;
+
+	/*
+	 * Don't replay commits from an LSN < this LSN. This can be set externally
+	 * but it will also be advanced (never retreat) from within snapbuild.c.
+	 */
+	XLogRecPtr	start_decoding_at;
+
+	/*
+	 * LSN at which two-phase decoding was enabled or LSN at which we found a
+	 * consistent point at the time of slot creation.
+	 *
+	 * The prepared transactions, that were skipped because previously
+	 * two-phase was not enabled or are not covered by initial snapshot, need
+	 * to be sent later along with commit prepared and they must be before
+	 * this point.
+	 */
+	XLogRecPtr	two_phase_at;
+
+	/*
+	 * Don't start decoding WAL until the "xl_running_xacts" information
+	 * indicates there are no running xids with an xid smaller than this.
+	 */
+	TransactionId initial_xmin_horizon;
+
+	/* Indicates if we are building full snapshot or just catalog one. */
+	bool		building_full_snapshot;
+
+	/*
+	 * Indicates if we are using the snapshot builder for the creation of a
+	 * logical replication slot. If it's true, the start point for decoding
+	 * changes is not determined yet. So we skip snapshot restores to properly
+	 * find the start point. See SnapBuildFindSnapshot() for details.
+	 */
+	bool		in_slot_creation;
+
+	/*
+	 * Snapshot that's valid to see the catalog state seen at this moment.
+	 */
+	Snapshot	snapshot;
+
+	/*
+	 * LSN of the last location we are sure a snapshot has been serialized to.
+	 */
+	XLogRecPtr	last_serialized_snapshot;
+
+	/*
+	 * The reorderbuffer we need to update with usable snapshots et al.
+	 */
+	ReorderBuffer *reorder;
+
+	/*
+	 * TransactionId at which the next phase of initial snapshot building will
+	 * happen. InvalidTransactionId if not known (i.e. SNAPBUILD_START), or
+	 * when no next phase necessary (SNAPBUILD_CONSISTENT).
+	 */
+	TransactionId next_phase_at;
+
+	/*
+	 * Array of transactions which could have catalog changes that committed
+	 * between xmin and xmax.
+	 */
+	struct
+	{
+		/* number of committed transactions */
+		size_t		xcnt;
+
+		/* available space for committed transactions */
+		size_t		xcnt_space;
+
+		/*
+		 * Until we reach a CONSISTENT state, we record commits of all
+		 * transactions, not just the catalog changing ones. Record when that
+		 * changes so we know we cannot export a snapshot safely anymore.
+		 */
+		bool		includes_all_transactions;
+
+		/*
+		 * Array of committed transactions that have modified the catalog.
+		 *
+		 * As this array is frequently modified we do *not* keep it in
+		 * xidComparator order. Instead we sort the array when building &
+		 * distributing a snapshot.
+		 *
+		 * TODO: It's unclear whether that reasoning has much merit. Every
+		 * time we add something here after becoming consistent will also
+		 * require distributing a snapshot. Storing them sorted would
+		 * potentially also make it easier to purge (but more complicated wrt
+		 * wraparound?). Should be improved if sorting while building the
+		 * snapshot shows up in profiles.
+		 */
+		TransactionId *xip;
+	}			committed;
+
+	/*
+	 * Array of transactions and subtransactions that had modified catalogs
+	 * and were running when the snapshot was serialized.
+	 *
+	 * We normally rely on some WAL record types such as HEAP2_NEW_CID to know
+	 * if the transaction has changed the catalog. But it could happen that
+	 * the logical decoding decodes only the commit record of the transaction
+	 * after restoring the previously serialized snapshot in which case we
+	 * will miss adding the xid to the snapshot and end up looking at the
+	 * catalogs with the wrong snapshot.
+	 *
+	 * Now to avoid the above problem, we serialize the transactions that had
+	 * modified the catalogs and are still running at the time of snapshot
+	 * serialization. We fill this array while restoring the snapshot and then
+	 * refer it while decoding commit to ensure if the xact has modified the
+	 * catalog. We discard this array when all the xids in the list become old
+	 * enough to matter. See SnapBuildPurgeOlderTxn for details.
+	 */
+	struct
+	{
+		/* number of transactions */
+		size_t		xcnt;
+
+		/* This array must be sorted in xidComparator order */
+		TransactionId *xip;
+	}			catchange;
+} SnapBuild;
+
+/* -----------------------------------
+ * Snapshot serialization support
+ * -----------------------------------
+ */
+
+/*
+ * We store current state of struct SnapBuild on disk in the following manner:
+ *
+ * struct SnapBuildOnDisk;
+ * TransactionId * committed.xcnt; (*not xcnt_space*)
+ * TransactionId * catchange.xcnt;
+ *
+ * Check if the SnapBuildOnDiskConstantSize and SnapBuildOnDiskNotChecksummedSize
+ * macros need to be updated when modifying the SnapBuildOnDisk struct.
+ */
+typedef struct SnapBuildOnDisk
+{
+	/* first part of this struct needs to be version independent */
+
+	/* data not covered by checksum */
+	uint32		magic;
+	pg_crc32c	checksum;
+
+	/* data covered by checksum */
+
+	/* version, in case we want to support pg_upgrade */
+	uint32		version;
+	/* how large is the on disk data, excluding the constant sized part */
+	uint32		length;
+
+	/* version dependent part */
+	SnapBuild	builder;
+
+	/* variable amount of TransactionIds follows */
+} SnapBuildOnDisk;
+
+extern void ValidateAndRestoreSnapshotFile(SnapBuildOnDisk *ondisk, const char *path,
+										   int fd, MemoryContext context);
+
+#endif							/* SNAPBUILD_INTERNAL_H */
-- 
2.34.1

#46Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Bertrand Drouvot (#45)
Re: Add contrib/pg_logicalsnapinspect

On Tue, Oct 8, 2024 at 9:25 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Hi,

On Tue, Oct 08, 2024 at 04:25:29PM +1100, Peter Smith wrote:

Hi, here are some review comments for patch v11.

Thanks for looking at it!

======
contrib/pg_logicalinspect/specs/logical_inspect.spec

1.
nit - Add some missing spaces after commas (,) in the SQL.

Fine by me, done in v12 attached.

======
doc/src/sgml/pglogicalinspect.sgml

2.
+ <note>
+  <para>
+   The <filename>pg_logicalinspect</filename> functions are called
+   using a text argument that can be extracted from the output name of the
+   <function>pg_ls_logicalsnapdir</function>() function.
+  </para>
+ </note>

2a. wording

The wording "using a text argument that can be extracted" seems like a
hangover from the previous implementation; it does not even say what
that "text argument" means.

That's right (it's mentioned later on (for each function description) that
the argument represents the snapshot file name though).

Why not just say it is a snapshot
filename, something like below?

SUGGESTION:
The pg_logicalinspect functions are called passing a snapshot filename
to be inspected. For example, pass a name obtained from the
pg_ls_logicalsnapdir() function.

Yeah, I like it, but...

~

2b. formatting

nit - In the previous implementation the extraction of the LSN was
trickier, so this part was worthy of an SGML "NOTE". Now that it is
just a filename, I don't know if it needs to be a special note
anymore.

In fact, giving it more thoughts, I think we can just remove this part.
I don't see the extra value anymore and that's something that we may need to
remove depending on what will be added to this module in the future.

I think that having the argument explanation in each function description is
enough, done that way in v12.

~~~

3.
+postgres=# SELECT meta.* FROM pg_ls_logicalsnapdir(),
+pg_get_logical_snapshot_meta(name) AS meta;
+
+-[ RECORD 1 ]--------
+magic    | 1369563137
+checksum | 1028045905
+version  | 6

3a.
If you are going to wrap the SQL across multiple lines like this, then
you should show the psql continuation prompt, so that the example
looks the same as what the user would see.

I'm not sure about this one. If the user copy/paste the doc as it is then there
is no psql continuation prompt. If the user does not copy/paste the doc then he
might indeed see "something" else (but that's not surprising since he did not
copy/paste). FWIW, there is similar examples in pgstatstatements.sgml.

~

3b.
FYI, the output of that can return multiple records,

Yes, as the test in this patch does.

which is
b.i) probably not what you intended to demonstrate
b.ii) not the same as what the example says

e.g., I got this:
test_pub=# SELECT meta.* FROM pg_ls_logicalsnapdir(),
test_pub-# pg_get_logical_snapshot_meta(name) AS meta;
-[ RECORD 1 ]--------
magic | 1369563137
checksum | 681884630
version | 6
-[ RECORD 2 ]--------
magic | 1369563137
checksum | 2213048308
version | 6
-[ RECORD 3 ]--------
magic | 1369563137
checksum | 3812680762
version | 6
-[ RECORD 4 ]--------
magic | 1369563137
checksum | 3759893001
version | 6

I don't get the point here. The examples just show another way to use the functions,
the ouput is more "anecdotal" than anything else.

~~~

(Also those #3a, #3b comments apply to both examples)

======
src/backend/replication/logical/snapbuild.c

4.
- SnapBuild builder;
-
- /* variable amount of TransactionIds follows */
-} SnapBuildOnDisk;
-
#define SnapBuildOnDiskConstantSize \
offsetof(SnapBuildOnDisk, builder)
#define SnapBuildOnDiskNotChecksummedSize \

Is it better to try to keep those "Size" macros defined along with
wherever the SnapBuildOnDisk is defined? Otherwise, if the structure
is ever changed, adjusting the macros could be easily overlooked.

I think that the less we put in the snapbuild_internal.h the better. That said,
I think you have a good point so I added a comment around the SnapBuildOnDisk
definition instead in v12.

~~~

5.
ValidateAndRestoreSnapshotFile

nit - See [1] #4 suggestion to declare 'sz' at scope where used. The
previous reason not to change this (e.g. "mainly inspired from
SnapBuildRestore") seems less relevant because now most lines of this
function have already been modified for other reasons.

Right. I think that's a matter of taste and I do prefer to "only" do the
necessary changes that are linked to the feature the patch is implementing.

~~~

6.
SnapBuildRestore:

+ if (fd < 0 && errno == ENOENT)
+ return false;
+ else if (fd < 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not open file \"%s\": %m", path)));

I think this code fragment looked like this before, and you only
relocated it,

That's right.

but it still seems a bit awkward to write this way.
Since so much else has changed, how about also improving this in
passing, like below:

if (fd < 0)
{
if (errno == ENOENT)
return false;

ereport(ERROR,
(errcode_for_file_access(),
errmsg("could not open file \"%s\": %m", path)));
}

Same, I do prefer to "only" do the necessary changes that are linked to the
feature the patch is implementing (and why stop here, a similar change could be
made in logical/origin.c too for example).

Thank you for updating the patch! I have some comments on v12 patch:

---
+       if (ondisk.builder.committed.xcnt > 0)
+       {
+               Datum      *arrayelems;
+               int                     narrayelems = 0;
+
+               arrayelems = (Datum *)
palloc(ondisk.builder.committed.xcnt * sizeof(Datum));
+
+               for (; narrayelems < ondisk.builder.committed.xcnt;
narrayelems++)
+                       arrayelems[narrayelems] =
Int64GetDatum((int64) ondisk.builder.committed.xip[narrayelems]);
+
+               values[i++] =
PointerGetDatum(construct_array_builtin(arrayelems, narrayelems,
INT8OID));
+       }

Since committed_xip and catchange_xip are xid[], we should use
TransactionIdGetDatum() and XIDOID instead.

I think that it would be cleaner if we pass
ondisk.builder.committed.xcnt instead of construct_array_builtin to
construct_array_buildin(). That is, we can rewrite it as follows:

for (int j = 0; j < ondisk.builder.committed.xcnt; j++)
arrayelems[j] = TransactionIdGetDatum(ondisk.builder.committed.xip[j]);

values[i++] = PointerGetDatum(construct_array_builtin(arrayelems,
ondisk.builder.committed.xcnt, XIDOID));

---
+# Test the pg_logicalinspect functions: that needs some permutation to
+# ensure that we are creating multiple logical snapshots and that one of them
+# contains ongoing catalogs changes.

If we use prepared transactions modifying catalog changes, can we
write the normal (i.e. not isolation check) tests? It would be easier
to write and add tests.

---
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/

If we need to use the isolation tests (see above comment), we need to
add both output_iso and tmp_check_iso as well.

---
+       tuple = heap_form_tuple(tupdesc, values, nulls);
+
+       MemoryContextReset(context);
+
+       PG_RETURN_DATUM(HeapTupleGetDatum(tuple));

I think we don't necessarily need to reset the memory context here.
Rather, I think we can just pass CurrentMemoryContext to
ValidateAndRestoreSnapshotFile() instead of passing the separate new
memory context.

---
+       fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
+
+       if (fd < 0)
+               ereport(ERROR,
+                               (errcode_for_file_access(),
+                                errmsg("could not open file \"%s\":
%m", path)));
+
+       context = AllocSetContextCreate(CurrentMemoryContext,
+
 "logicalsnapshot inspect context",
+
 ALLOCSET_DEFAULT_SIZES);
+
+       /* Validate and restore the snapshot to 'ondisk' */
+       ValidateAndRestoreSnapshotFile(&ondisk, path, fd, context);

It's a bit odd to me that this function opens a snapshot file and
passes both the file descriptor and file path. The file path is used
mostly only for error reporting in ValidateAndRestoreSnapshotFile(). I
guess it would be cleaner if we pass the file path to
ValidateAndRestoreSnapshotFile() which opens and validates the
snapshot file. Since SnapBuildRestore() wants to get false if the
specified file doesn't exist, we can also add missing_ok argument to
ValidateAndRestoreSnapshotFile(). That is, the function will be like:

void
ValidateAndRestoreSnapshotFile(SnapBuildOnDisk *ondisk, const char
*path, MemoryContext context, bool missing_ok)
{
:
fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);

if (fd < 0)
{
if (missing_ok && errno == ENOENT)
return false;
else
ereport(ERROR,
(errcode_for_file_access(),
errmsg("could not open file \"%s\": %m", path)));
}
:

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com

#47Peter Smith
smithpb2250@gmail.com
In reply to: Bertrand Drouvot (#45)
Re: Add contrib/pg_logicalsnapinspect

On Wed, Oct 9, 2024 at 3:25 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Hi,

On Tue, Oct 08, 2024 at 04:25:29PM +1100, Peter Smith wrote:

3.
+postgres=# SELECT meta.* FROM pg_ls_logicalsnapdir(),
+pg_get_logical_snapshot_meta(name) AS meta;
+
+-[ RECORD 1 ]--------
+magic    | 1369563137
+checksum | 1028045905
+version  | 6

3a.
If you are going to wrap the SQL across multiple lines like this, then
you should show the psql continuation prompt, so that the example
looks the same as what the user would see.

I'm not sure about this one. If the user copy/paste the doc as it is then there
is no psql continuation prompt. If the user does not copy/paste the doc then he
might indeed see "something" else (but that's not surprising since he did not
copy/paste). FWIW, there is similar examples in pgstatstatements.sgml.

~

3b.
FYI, the output of that can return multiple records,

Yes, as the test in this patch does.

which is
b.i) probably not what you intended to demonstrate
b.ii) not the same as what the example says

e.g., I got this:
test_pub=# SELECT meta.* FROM pg_ls_logicalsnapdir(),
test_pub-# pg_get_logical_snapshot_meta(name) AS meta;
-[ RECORD 1 ]--------
magic | 1369563137
checksum | 681884630
version | 6
-[ RECORD 2 ]--------
magic | 1369563137
checksum | 2213048308
version | 6
-[ RECORD 3 ]--------
magic | 1369563137
checksum | 3812680762
version | 6
-[ RECORD 4 ]--------
magic | 1369563137
checksum | 3759893001
version | 6

I don't get the point here. The examples just show another way to use the functions,
the ouput is more "anecdotal" than anything else.

I mistakenly thought the purpose of the third part of the example was
to give a shorthand way of doing the same as the first two parts --
so, using one SQL query instead of two.

But unless this SQL is modified also to output the name of the/each
snapfile then I'm not sure how these 3rd part examples are
particularly useful. e.g. Without an associated filename, all this
query will yield is a bunch of meta-data (or info-data) records but
you have no idea which snapshots they belong to.

How about doing this:
SELECT ss.name, info.* FROM pg_ls_logicalsnapdir() AS ss,
pg_get_logical_snapshot_info(ss.name) AS info;

======
Kind Regards,
Peter Smith.
Fujitsu Australia

#48Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: Peter Smith (#47)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Wed, Oct 09, 2024 at 11:41:44AM +1100, Peter Smith wrote:

On Wed, Oct 9, 2024 at 3:25 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

I don't get the point here. The examples just show another way to use the functions,
the ouput is more "anecdotal" than anything else.

How about doing this:
SELECT ss.name, info.* FROM pg_ls_logicalsnapdir() AS ss,
pg_get_logical_snapshot_info(ss.name) AS info;

Agree that it makes sense to add the snapshot file name, will add in v13, thanks!

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

#49Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: Masahiko Sawada (#46)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Tue, Oct 08, 2024 at 10:52:11AM -0700, Masahiko Sawada wrote:

On Tue, Oct 8, 2024 at 9:25 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Thank you for updating the patch! I have some comments on v12 patch:

Thanks for looking at it!

---
+       if (ondisk.builder.committed.xcnt > 0)
+       {
+               Datum      *arrayelems;
+               int                     narrayelems = 0;
+
+               arrayelems = (Datum *)
palloc(ondisk.builder.committed.xcnt * sizeof(Datum));
+
+               for (; narrayelems < ondisk.builder.committed.xcnt;
narrayelems++)
+                       arrayelems[narrayelems] =
Int64GetDatum((int64) ondisk.builder.committed.xip[narrayelems]);
+
+               values[i++] =
PointerGetDatum(construct_array_builtin(arrayelems, narrayelems,
INT8OID));
+       }

Since committed_xip and catchange_xip are xid[], we should use
TransactionIdGetDatum() and XIDOID instead.

I ended up using INT8OID because XIDOID is not part of the switch in
construct_array_builtin() and so leads to:

"
ERROR: type 28 not supported by construct_array_builtin()
"

One option could be (did not test it) to add this switch in construct_array_builtin():

+               case XIDOID:
+                       elmlen = sizeof(TransactionId);
+                       elmbyval = true;
+                       elmalign = TYPALIGN_INT;
+                       break;

I think that could make sense and would probably need a dedicated patch for that,
thoughts?

I think that it would be cleaner if we pass
ondisk.builder.committed.xcnt instead of construct_array_builtin to
construct_array_buildin(). That is, we can rewrite it as follows:

for (int j = 0; j < ondisk.builder.committed.xcnt; j++)
arrayelems[j] = TransactionIdGetDatum(ondisk.builder.committed.xip[j]);

values[i++] = PointerGetDatum(construct_array_builtin(arrayelems,
ondisk.builder.committed.xcnt, XIDOID));

Fine by me. Will do that in v13 with TransactionIdGetDatum/XIDOID or Int64GetDatum/INT8OID
once we decide what to do with the above remark linked to construct_array_builtin().

---
+# Test the pg_logicalinspect functions: that needs some permutation to
+# ensure that we are creating multiple logical snapshots and that one of them
+# contains ongoing catalogs changes.

If we use prepared transactions modifying catalog changes, can we
write the normal (i.e. not isolation check) tests? It would be easier
to write and add tests.

Not sure about this one. I think that the test is simple enough and mainly inspired
by what can be found in the test_decoding module.

We could still add "normal" (REGRESS) tests in the future should we add features
to the pg_logicalinspect module that would require new tests.

For example, test_decoding is using both kind of tests, what do you think?

---
+       tuple = heap_form_tuple(tupdesc, values, nulls);
+
+       MemoryContextReset(context);
+
+       PG_RETURN_DATUM(HeapTupleGetDatum(tuple));

I think we don't necessarily need to reset the memory context here.
Rather, I think we can just pass CurrentMemoryContext to
ValidateAndRestoreSnapshotFile() instead of passing the separate new
memory context.

Yeah, we should be in a short-lived memory context here (ExprContext or such),
so that's fine by me (will do in v13).

---
+       fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
+
+       if (fd < 0)
+               ereport(ERROR,
+                               (errcode_for_file_access(),
+                                errmsg("could not open file \"%s\":
%m", path)));
+
+       context = AllocSetContextCreate(CurrentMemoryContext,
+
"logicalsnapshot inspect context",
+
ALLOCSET_DEFAULT_SIZES);
+
+       /* Validate and restore the snapshot to 'ondisk' */
+       ValidateAndRestoreSnapshotFile(&ondisk, path, fd, context);

It's a bit odd to me that this function opens a snapshot file and
passes both the file descriptor and file path. The file path is used
mostly only for error reporting in ValidateAndRestoreSnapshotFile().

Right.

I guess it would be cleaner if we pass the file path to
ValidateAndRestoreSnapshotFile() which opens and validates the
snapshot file. Since SnapBuildRestore() wants to get false if the
specified file doesn't exist, we can also add missing_ok argument to
ValidateAndRestoreSnapshotFile(). That is, the function will be like:

void
ValidateAndRestoreSnapshotFile(SnapBuildOnDisk *ondisk, const char
*path, MemoryContext context, bool missing_ok)
{
:
fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);

if (fd < 0)
{
if (missing_ok && errno == ENOENT)
return false;
else
ereport(ERROR,
(errcode_for_file_access(),
errmsg("could not open file \"%s\": %m", path)));
}

Yeah, it makes sense to move the OpenTransientFile() call in
ValidateAndRestoreSnapshotFile(), will do in v13.

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

#50Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Bertrand Drouvot (#49)
Re: Add contrib/pg_logicalsnapinspect

On Wed, Oct 9, 2024 at 1:12 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Hi,

On Tue, Oct 08, 2024 at 10:52:11AM -0700, Masahiko Sawada wrote:

On Tue, Oct 8, 2024 at 9:25 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Thank you for updating the patch! I have some comments on v12 patch:

Thanks for looking at it!

---
+       if (ondisk.builder.committed.xcnt > 0)
+       {
+               Datum      *arrayelems;
+               int                     narrayelems = 0;
+
+               arrayelems = (Datum *)
palloc(ondisk.builder.committed.xcnt * sizeof(Datum));
+
+               for (; narrayelems < ondisk.builder.committed.xcnt;
narrayelems++)
+                       arrayelems[narrayelems] =
Int64GetDatum((int64) ondisk.builder.committed.xip[narrayelems]);
+
+               values[i++] =
PointerGetDatum(construct_array_builtin(arrayelems, narrayelems,
INT8OID));
+       }

Since committed_xip and catchange_xip are xid[], we should use
TransactionIdGetDatum() and XIDOID instead.

I ended up using INT8OID because XIDOID is not part of the switch in
construct_array_builtin() and so leads to:

"
ERROR: type 28 not supported by construct_array_builtin()
"

Thank you for pointing it out.

One option could be (did not test it) to add this switch in construct_array_builtin():

+               case XIDOID:
+                       elmlen = sizeof(TransactionId);
+                       elmbyval = true;
+                       elmalign = TYPALIGN_INT;
+                       break;

I think that could make sense and would probably need a dedicated patch for that,
thoughts?

Or can we use construct_array() instead?

---
+# Test the pg_logicalinspect functions: that needs some permutation to
+# ensure that we are creating multiple logical snapshots and that one of them
+# contains ongoing catalogs changes.

If we use prepared transactions modifying catalog changes, can we
write the normal (i.e. not isolation check) tests? It would be easier
to write and add tests.

Not sure about this one. I think that the test is simple enough and mainly inspired
by what can be found in the test_decoding module.

We could still add "normal" (REGRESS) tests in the future should we add features
to the pg_logicalinspect module that would require new tests.

For example, test_decoding is using both kind of tests, what do you think?

Fair point. I agree with you.

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com

#51Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: Masahiko Sawada (#50)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Wed, Oct 09, 2024 at 10:21:31AM -0700, Masahiko Sawada wrote:

On Wed, Oct 9, 2024 at 1:12 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

One option could be (did not test it) to add this switch in construct_array_builtin():

+               case XIDOID:
+                       elmlen = sizeof(TransactionId);
+                       elmbyval = true;
+                       elmalign = TYPALIGN_INT;
+                       break;

I think that could make sense and would probably need a dedicated patch for that,
thoughts?

Or can we use construct_array() instead?

I had a closer look to d746021de1 (which introduced construct_array_builtin())
and the hackers thread that lead to it [1]/messages/by-id/2914356f-9e5f-8c59-2995-5997fc48bcba@enterprisedb.com.

IIUC, the idea was to:

1. centralize the hardcoded knowledge that were in the calls to construct_array()
and deconstruct_array() for built-in types
2. notational simplification and bug-proofing

As XIDOID is a built-in type, I think that it would make sense to add it in
deconstruct_array_builtin()/construct_array_builtin().

I think the reason XIDOID has not been added in d746021de1 is that there were no
use case at that time (means no existing calls to construct_array()/deconstruct_array()
with hardcoded XIDOID related arguments).

One could say that we would just add 2 calls to construct_array() in the pg_logicalinspect
module but, for example, d746021de1 also took care of CSTRINGOID that had a single
call at that time:

$ git show d746021de1 | grep deconstruct_array_builtin | grep -c CSTRINGOID
1
$ git show d746021de1 | grep construct_array_builtin | grep -v deconstruct_array_builtin | grep -c CSTRINGOID
1

So I think that having construct_array_builtin()/deconstruct_array_builtin()
taking care of XIDOID is the way to go. If that makes sense to you then I'll
submit a dedicated patch for it, thoughts?

[1]: /messages/by-id/2914356f-9e5f-8c59-2995-5997fc48bcba@enterprisedb.com

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

#52Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Bertrand Drouvot (#51)
Re: Add contrib/pg_logicalsnapinspect

On Wed, Oct 9, 2024 at 8:32 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Hi,

On Wed, Oct 09, 2024 at 10:21:31AM -0700, Masahiko Sawada wrote:

On Wed, Oct 9, 2024 at 1:12 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

One option could be (did not test it) to add this switch in construct_array_builtin():

+               case XIDOID:
+                       elmlen = sizeof(TransactionId);
+                       elmbyval = true;
+                       elmalign = TYPALIGN_INT;
+                       break;

I think that could make sense and would probably need a dedicated patch for that,
thoughts?

Or can we use construct_array() instead?

I had a closer look to d746021de1 (which introduced construct_array_builtin())
and the hackers thread that lead to it [1].

IIUC, the idea was to:

1. centralize the hardcoded knowledge that were in the calls to construct_array()
and deconstruct_array() for built-in types
2. notational simplification and bug-proofing

As XIDOID is a built-in type, I think that it would make sense to add it in
deconstruct_array_builtin()/construct_array_builtin().

I think the reason XIDOID has not been added in d746021de1 is that there were no
use case at that time (means no existing calls to construct_array()/deconstruct_array()
with hardcoded XIDOID related arguments).

One could say that we would just add 2 calls to construct_array() in the pg_logicalinspect
module but, for example, d746021de1 also took care of CSTRINGOID that had a single
call at that time:

$ git show d746021de1 | grep deconstruct_array_builtin | grep -c CSTRINGOID
1
$ git show d746021de1 | grep construct_array_builtin | grep -v deconstruct_array_builtin | grep -c CSTRINGOID
1

So I think that having construct_array_builtin()/deconstruct_array_builtin()
taking care of XIDOID is the way to go. If that makes sense to you then I'll
submit a dedicated patch for it, thoughts?

Your explanation makes sense to me. I think it can be included in the
main pg_logicalinspect patch as this change is a part of it.

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com

#53Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: Masahiko Sawada (#52)
2 attachment(s)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Thu, Oct 10, 2024 at 12:05:10AM -0700, Masahiko Sawada wrote:

On Wed, Oct 9, 2024 at 8:32 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

So I think that having construct_array_builtin()/deconstruct_array_builtin()
taking care of XIDOID is the way to go. If that makes sense to you then I'll
submit a dedicated patch for it, thoughts?

Your explanation makes sense to me.

Thanks for sharing your thoughts.

I think it can be included in the main pg_logicalinspect patch as this change
is a part of it.

Okay, let's keep the discussion here. Please find attached v13 that takes care
of your previous remarks and Peter's one ([1]/messages/by-id/ZwY5vBI+R8Ky7yM5@ip-10-97-1-34.eu-west-3.compute.internal).

FYI, v13 is splitted into 2 sub-patches (0001 for the discussion related to
XIDOID and construct_array_builtin() and 0002 for the module itself).

FWIW, the elmbyval and elmalign values that are added in 0001 have been deduced
from:

postgres=# select typbyval, typalign from pg_type where typname = 'xid';
typbyval | typalign
----------+----------
t | i
(1 row)

[1]: /messages/by-id/ZwY5vBI+R8Ky7yM5@ip-10-97-1-34.eu-west-3.compute.internal

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachments:

v13-0001-Add-XIDOID-in-de-construct_array_builtin.patchtext/x-diff; charset=us-asciiDownload
From 6e43fffe9e2995c0183809550aadef505bf4693c Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Thu, 10 Oct 2024 04:09:53 +0000
Subject: [PATCH v13 1/2] Add XIDOID in [de]construct_array_builtin()

Using construct_array_builtin() for XIDOID is a new use case that is coming with
a new module (not added in the code tree yet).

d746021de1 (which introduced construct_array_builtin()) did not take care of
XIDOID because there were no use case at that time. Now that there is one, let's
add XIDOID.
---
 src/backend/utils/adt/arrayfuncs.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
 100.0% src/backend/utils/adt/

diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c
index e5c7e57a5d..8687fae359 100644
--- a/src/backend/utils/adt/arrayfuncs.c
+++ b/src/backend/utils/adt/arrayfuncs.c
@@ -3447,6 +3447,12 @@ construct_array_builtin(Datum *elems, int nelems, Oid elmtype)
 			elmalign = TYPALIGN_SHORT;
 			break;
 
+		case XIDOID:
+			elmlen = sizeof(TransactionId);
+			elmbyval = true;
+			elmalign = TYPALIGN_INT;
+			break;
+
 		default:
 			elog(ERROR, "type %u not supported by construct_array_builtin()", elmtype);
 			/* keep compiler quiet */
@@ -3734,6 +3740,12 @@ deconstruct_array_builtin(ArrayType *array,
 			elmalign = TYPALIGN_SHORT;
 			break;
 
+		case XIDOID:
+			elmlen = sizeof(TransactionId);
+			elmbyval = true;
+			elmalign = TYPALIGN_INT;
+			break;
+
 		default:
 			elog(ERROR, "type %u not supported by deconstruct_array_builtin()", elmtype);
 			/* keep compiler quiet */
-- 
2.34.1

v13-0002-Add-contrib-pg_logicalinspect.patchtext/x-diff; charset=us-asciiDownload
From 16d4bc443371a80a4d2617750cfa8b0f22fd6a89 Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Wed, 14 Aug 2024 08:46:05 +0000
Subject: [PATCH v13 2/2] Add contrib/pg_logicalinspect

Provides SQL functions that allow to inspect logical decoding components.

It currently allows to inspect the contents of serialized logical snapshots of
a running database cluster, which is useful for debugging or educational
purposes.
---
 contrib/Makefile                              |   1 +
 contrib/meson.build                           |   1 +
 contrib/pg_logicalinspect/.gitignore          |   4 +
 contrib/pg_logicalinspect/Makefile            |  31 ++
 .../expected/logical_inspect.out              |  52 ++++
 contrib/pg_logicalinspect/logicalinspect.conf |   1 +
 contrib/pg_logicalinspect/meson.build         |  39 +++
 .../pg_logicalinspect--1.0.sql                |  43 +++
 contrib/pg_logicalinspect/pg_logicalinspect.c | 171 +++++++++++
 .../pg_logicalinspect.control                 |   5 +
 .../specs/logical_inspect.spec                |  34 +++
 doc/src/sgml/contrib.sgml                     |   1 +
 doc/src/sgml/filelist.sgml                    |   1 +
 doc/src/sgml/pglogicalinspect.sgml            | 143 ++++++++++
 src/backend/replication/logical/snapbuild.c   | 270 ++++--------------
 src/include/replication/snapbuild.h           |   6 +-
 src/include/replication/snapbuild_internal.h  | 199 +++++++++++++
 17 files changed, 788 insertions(+), 214 deletions(-)
   7.7% contrib/pg_logicalinspect/expected/
   5.3% contrib/pg_logicalinspect/specs/
  26.1% contrib/pg_logicalinspect/
  14.0% doc/src/sgml/
  25.9% src/backend/replication/logical/
  20.6% src/include/replication/

diff --git a/contrib/Makefile b/contrib/Makefile
index abd780f277..952855d9b6 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -32,6 +32,7 @@ SUBDIRS = \
 		passwordcheck	\
 		pg_buffercache	\
 		pg_freespacemap \
+		pg_logicalinspect \
 		pg_prewarm	\
 		pg_stat_statements \
 		pg_surgery	\
diff --git a/contrib/meson.build b/contrib/meson.build
index 14a8906865..159ff41555 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -46,6 +46,7 @@ subdir('passwordcheck')
 subdir('pg_buffercache')
 subdir('pgcrypto')
 subdir('pg_freespacemap')
+subdir('pg_logicalinspect')
 subdir('pg_prewarm')
 subdir('pgrowlocks')
 subdir('pg_stat_statements')
diff --git a/contrib/pg_logicalinspect/.gitignore b/contrib/pg_logicalinspect/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/contrib/pg_logicalinspect/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/contrib/pg_logicalinspect/Makefile b/contrib/pg_logicalinspect/Makefile
new file mode 100644
index 0000000000..55124514d4
--- /dev/null
+++ b/contrib/pg_logicalinspect/Makefile
@@ -0,0 +1,31 @@
+# contrib/pg_logicalinspect/Makefile
+
+MODULE_big = pg_logicalinspect
+OBJS = \
+	$(WIN32RES) \
+	pg_logicalinspect.o
+PGFILEDESC = "pg_logicalinspect - functions to inspect logical decoding components"
+
+EXTENSION = pg_logicalinspect
+DATA = pg_logicalinspect--1.0.sql
+
+EXTRA_INSTALL = contrib/test_decoding
+
+ISOLATION = logical_inspect
+
+ISOLATION_OPTS = --temp-config $(top_srcdir)/contrib/pg_logicalinspect/logicalinspect.conf
+
+# Disabled because these tests require "wal_level=logical", which
+# some installcheck users do not have (e.g. buildfarm clients).
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/pg_logicalinspect
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/pg_logicalinspect/expected/logical_inspect.out b/contrib/pg_logicalinspect/expected/logical_inspect.out
new file mode 100644
index 0000000000..d95efa4d1e
--- /dev/null
+++ b/contrib/pg_logicalinspect/expected/logical_inspect.out
@@ -0,0 +1,52 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s0_init s0_begin s0_savepoint s0_truncate s1_checkpoint s1_get_changes s0_commit s0_begin s0_insert s1_checkpoint s1_get_changes s0_commit s1_get_changes s1_get_logical_snapshot_info s1_get_logical_snapshot_meta
+step s0_init: SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding');
+?column?
+--------
+init    
+(1 row)
+
+step s0_begin: BEGIN;
+step s0_savepoint: SAVEPOINT sp1;
+step s0_truncate: TRUNCATE tbl1;
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data
+----
+(0 rows)
+
+step s0_commit: COMMIT;
+step s0_begin: BEGIN;
+step s0_insert: INSERT INTO tbl1 VALUES (1);
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                   
+---------------------------------------
+BEGIN                                  
+table public.tbl1: TRUNCATE: (no-flags)
+COMMIT                                 
+(3 rows)
+
+step s0_commit: COMMIT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                                         
+-------------------------------------------------------------
+BEGIN                                                        
+table public.tbl1: INSERT: val1[integer]:1 val2[integer]:null
+COMMIT                                                       
+(3 rows)
+
+step s1_get_logical_snapshot_info: SELECT info.state, info.catchange_count, array_length(info.catchange_xip,1) AS catchange_array_length, info.committed_count, array_length(info.committed_xip,1) AS committed_array_length FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info ORDER BY 2;
+state     |catchange_count|catchange_array_length|committed_count|committed_array_length
+----------+---------------+----------------------+---------------+----------------------
+consistent|              0|                      |              2|                     2
+consistent|              2|                     2|              0|                      
+(2 rows)
+
+step s1_get_logical_snapshot_meta: SELECT COUNT(meta.*) from pg_ls_logicalsnapdir(), pg_get_logical_snapshot_meta(name) as meta;
+count
+-----
+    2
+(1 row)
+
diff --git a/contrib/pg_logicalinspect/logicalinspect.conf b/contrib/pg_logicalinspect/logicalinspect.conf
new file mode 100644
index 0000000000..e3d257315f
--- /dev/null
+++ b/contrib/pg_logicalinspect/logicalinspect.conf
@@ -0,0 +1 @@
+wal_level = logical
diff --git a/contrib/pg_logicalinspect/meson.build b/contrib/pg_logicalinspect/meson.build
new file mode 100644
index 0000000000..3ec635509b
--- /dev/null
+++ b/contrib/pg_logicalinspect/meson.build
@@ -0,0 +1,39 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+pg_logicalinspect_sources = files('pg_logicalinspect.c')
+
+if host_system == 'windows'
+  pg_logicalinspect_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+    '--NAME', 'pg_logicalinspect',
+    '--FILEDESC', 'pg_logicalinspect - functions to inspect logical decoding components',])
+endif
+
+pg_logicalinspect = shared_module('pg_logicalinspect',
+  pg_logicalinspect_sources,
+  kwargs: contrib_mod_args + {
+      'dependencies': contrib_mod_args['dependencies'],
+  },
+)
+contrib_targets += pg_logicalinspect
+
+install_data(
+  'pg_logicalinspect.control',
+  'pg_logicalinspect--1.0.sql',
+  kwargs: contrib_data_args,
+)
+
+tests += {
+  'name': 'pg_logicalinspect',
+  'sd': meson.current_source_dir(),
+  'bd': meson.current_build_dir(),
+  'isolation': {
+    'specs': [
+      'logical_inspect',
+    ],
+    'regress_args': [
+      '--temp-config', files('logicalinspect.conf'),
+    ],
+    # see above
+    'runningcheck': false,
+  },
+}
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql b/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
new file mode 100644
index 0000000000..c773f6e458
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
@@ -0,0 +1,43 @@
+/* contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_logicalinspect" to load this file. \quit
+
+--
+-- pg_get_logical_snapshot_meta()
+--
+CREATE FUNCTION pg_get_logical_snapshot_meta(IN filename text,
+    OUT magic int4,
+    OUT checksum int8,
+    OUT version int4
+)
+AS 'MODULE_PATHNAME', 'pg_get_logical_snapshot_meta'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_logical_snapshot_meta(text) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_logical_snapshot_meta(text) TO pg_read_server_files;
+
+--
+-- pg_get_logical_snapshot_info()
+--
+CREATE FUNCTION pg_get_logical_snapshot_info(IN filename text,
+    OUT state text,
+    OUT xmin xid,
+    OUT xmax xid,
+    OUT start_decoding_at pg_lsn,
+    OUT two_phase_at pg_lsn,
+    OUT initial_xmin_horizon xid,
+    OUT building_full_snapshot boolean,
+    OUT in_slot_creation boolean,
+    OUT last_serialized_snapshot pg_lsn,
+    OUT next_phase_at xid,
+    OUT committed_count int8,
+    OUT committed_xip xid[],
+    OUT catchange_count int8,
+    OUT catchange_xip xid[]
+)
+AS 'MODULE_PATHNAME', 'pg_get_logical_snapshot_info'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_logical_snapshot_info(text) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_logical_snapshot_info(text) TO pg_read_server_files;
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.c b/contrib/pg_logicalinspect/pg_logicalinspect.c
new file mode 100644
index 0000000000..a1c43f58cd
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.c
@@ -0,0 +1,171 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_logicalinspect.c
+ *		  Functions to inspect contents of PostgreSQL logical snapshots
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ *		  contrib/pg_logicalinspect/pg_logicalinspect.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "funcapi.h"
+#include "replication/snapbuild_internal.h"
+#include "utils/array.h"
+#include "utils/builtins.h"
+#include "utils/pg_lsn.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_meta);
+PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_info);
+
+/* Return the description of SnapBuildState */
+static const char *
+get_snapbuild_state_desc(SnapBuildState state)
+{
+	const char *stateDesc = "unknown state";
+
+	switch (state)
+	{
+		case SNAPBUILD_START:
+			stateDesc = "start";
+			break;
+		case SNAPBUILD_BUILDING_SNAPSHOT:
+			stateDesc = "building";
+			break;
+		case SNAPBUILD_FULL_SNAPSHOT:
+			stateDesc = "full";
+			break;
+		case SNAPBUILD_CONSISTENT:
+			stateDesc = "consistent";
+			break;
+	}
+
+	return stateDesc;
+}
+
+/*
+ * Retrieve the logical snapshot file metadata.
+ */
+Datum
+pg_get_logical_snapshot_meta(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_META_COLS 3
+	SnapBuildOnDisk ondisk;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_META_COLS];
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_META_COLS];
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+	int			i = 0;
+	text	   *filename_t = PG_GETARG_TEXT_PP(0);
+
+	sprintf(path, "%s/%s",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			text_to_cstring(filename_t));
+
+	/* Validate and restore the snapshot to 'ondisk' */
+	ValidateAndRestoreSnapshotFile(&ondisk, path, CurrentMemoryContext, false);
+
+	/* Build a tuple descriptor for our result type */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	memset(nulls, 0, sizeof(nulls));
+
+	values[i++] = UInt32GetDatum(ondisk.magic);
+	values[i++] = Int64GetDatum((int64) ondisk.checksum);
+	values[i++] = UInt32GetDatum(ondisk.version);
+
+	Assert(i == PG_GET_LOGICAL_SNAPSHOT_META_COLS);
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_META_COLS
+}
+
+Datum
+pg_get_logical_snapshot_info(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_INFO_COLS 14
+	SnapBuildOnDisk ondisk;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS];
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS];
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+	int			i = 0;
+	text	   *filename_t = PG_GETARG_TEXT_PP(0);
+
+	sprintf(path, "%s/%s",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			text_to_cstring(filename_t));
+
+	/* Validate and restore the snapshot to 'ondisk' */
+	ValidateAndRestoreSnapshotFile(&ondisk, path, CurrentMemoryContext, false);
+
+	/* Build a tuple descriptor for our result type */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	memset(nulls, 0, sizeof(nulls));
+
+	values[i++] = CStringGetTextDatum(get_snapbuild_state_desc(ondisk.builder.state));
+	values[i++] = TransactionIdGetDatum(ondisk.builder.xmin);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.xmax);
+	values[i++] = LSNGetDatum(ondisk.builder.start_decoding_at);
+	values[i++] = LSNGetDatum(ondisk.builder.two_phase_at);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.initial_xmin_horizon);
+	values[i++] = BoolGetDatum(ondisk.builder.building_full_snapshot);
+	values[i++] = BoolGetDatum(ondisk.builder.in_slot_creation);
+	values[i++] = LSNGetDatum(ondisk.builder.last_serialized_snapshot);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.next_phase_at);
+
+	values[i++] = Int64GetDatum(ondisk.builder.committed.xcnt);
+	if (ondisk.builder.committed.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.committed.xcnt * sizeof(Datum));
+
+		for (int j = 0; j < ondisk.builder.committed.xcnt; j++)
+			arrayelems[j] = TransactionIdGetDatum(ondisk.builder.committed.xip[j]);
+
+		values[i++] = PointerGetDatum(construct_array_builtin(arrayelems,
+															  ondisk.builder.committed.xcnt,
+															  XIDOID));
+	}
+	else
+		nulls[i++] = true;
+
+	values[i++] = Int64GetDatum(ondisk.builder.catchange.xcnt);
+	if (ondisk.builder.catchange.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.catchange.xcnt * sizeof(Datum));
+
+		for (int j = 0; j < ondisk.builder.catchange.xcnt; j++)
+			arrayelems[j] = TransactionIdGetDatum(ondisk.builder.catchange.xip[j]);
+
+		values[i++] = PointerGetDatum(construct_array_builtin(arrayelems,
+															  ondisk.builder.catchange.xcnt,
+															  XIDOID));
+	}
+	else
+		nulls[i++] = true;
+
+	Assert(i == PG_GET_LOGICAL_SNAPSHOT_INFO_COLS);
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_INFO_COLS
+}
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.control b/contrib/pg_logicalinspect/pg_logicalinspect.control
new file mode 100644
index 0000000000..b4a70e57ba
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.control
@@ -0,0 +1,5 @@
+# pg_logicalinspect extension
+comment = 'functions to inspect logical decoding components'
+default_version = '1.0'
+module_pathname = '$libdir/pg_logicalinspect'
+relocatable = true
diff --git a/contrib/pg_logicalinspect/specs/logical_inspect.spec b/contrib/pg_logicalinspect/specs/logical_inspect.spec
new file mode 100644
index 0000000000..9851a6c18e
--- /dev/null
+++ b/contrib/pg_logicalinspect/specs/logical_inspect.spec
@@ -0,0 +1,34 @@
+# Test the pg_logicalinspect functions: that needs some permutation to
+# ensure that we are creating multiple logical snapshots and that one of them
+# contains ongoing catalogs changes.
+setup
+{
+    DROP TABLE IF EXISTS tbl1;
+    CREATE TABLE tbl1 (val1 integer, val2 integer);
+    CREATE EXTENSION pg_logicalinspect;
+}
+
+teardown
+{
+    DROP TABLE tbl1;
+    SELECT 'stop' FROM pg_drop_replication_slot('isolation_slot');
+    DROP EXTENSION pg_logicalinspect;
+}
+
+session "s0"
+setup { SET synchronous_commit=on; }
+step "s0_init" { SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding'); }
+step "s0_begin" { BEGIN; }
+step "s0_savepoint" { SAVEPOINT sp1; }
+step "s0_truncate" { TRUNCATE tbl1; }
+step "s0_insert" { INSERT INTO tbl1 VALUES (1); }
+step "s0_commit" { COMMIT; }
+
+session "s1"
+setup { SET synchronous_commit=on; }
+step "s1_checkpoint" { CHECKPOINT; }
+step "s1_get_changes" { SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0'); }
+step "s1_get_logical_snapshot_meta" { SELECT COUNT(meta.*) from pg_ls_logicalsnapdir(), pg_get_logical_snapshot_meta(name) as meta;}
+step "s1_get_logical_snapshot_info" { SELECT info.state, info.catchange_count, array_length(info.catchange_xip,1) AS catchange_array_length, info.committed_count, array_length(info.committed_xip,1) AS committed_array_length FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info ORDER BY 2; }
+
+permutation "s0_init" "s0_begin" "s0_savepoint" "s0_truncate" "s1_checkpoint" "s1_get_changes" "s0_commit" "s0_begin" "s0_insert" "s1_checkpoint" "s1_get_changes" "s0_commit" "s1_get_changes" "s1_get_logical_snapshot_info" "s1_get_logical_snapshot_meta"
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index 44639a8dca..7c381949a5 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -154,6 +154,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
  &pgbuffercache;
  &pgcrypto;
  &pgfreespacemap;
+ &pglogicalinspect;
  &pgprewarm;
  &pgrowlocks;
  &pgstatstatements;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index a7ff5f8264..66e6dccd4c 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -143,6 +143,7 @@
 <!ENTITY pgbuffercache   SYSTEM "pgbuffercache.sgml">
 <!ENTITY pgcrypto        SYSTEM "pgcrypto.sgml">
 <!ENTITY pgfreespacemap  SYSTEM "pgfreespacemap.sgml">
+<!ENTITY pglogicalinspect  SYSTEM "pglogicalinspect.sgml">
 <!ENTITY pgprewarm       SYSTEM "pgprewarm.sgml">
 <!ENTITY pgrowlocks      SYSTEM "pgrowlocks.sgml">
 <!ENTITY pgstatstatements SYSTEM "pgstatstatements.sgml">
diff --git a/doc/src/sgml/pglogicalinspect.sgml b/doc/src/sgml/pglogicalinspect.sgml
new file mode 100644
index 0000000000..e0fac997b6
--- /dev/null
+++ b/doc/src/sgml/pglogicalinspect.sgml
@@ -0,0 +1,143 @@
+<!-- doc/src/sgml/pglogicalinspect.sgml -->
+
+<sect1 id="pglogicalinspect" xreflabel="pg_logicalinspect">
+ <title>pg_logicalinspect &mdash; logical decoding components inspection</title>
+
+ <indexterm zone="pglogicalinspect">
+  <primary>pg_logicalinspect</primary>
+ </indexterm>
+
+ <para>
+  The <filename>pg_logicalinspect</filename> module provides SQL functions
+  that allow you to inspect the contents of logical decoding components. It
+  allows the inspection of serialized logical snapshots of a running
+  <productname>PostgreSQL</productname> database cluster, which is useful
+  for debugging or educational purposes.
+ </para>
+
+ <para>
+  By default, use of these functions is restricted to superusers and members of
+  the <literal>pg_read_server_files</literal> role. Access may be granted by
+  superusers to others using <command>GRANT</command>.
+ </para>
+
+ <sect2 id="pglogicalinspect-funcs">
+  <title>General Functions</title>
+
+  <variablelist>
+   <varlistentry id="pglogicalinspect-funcs-pg-get-logical-snapshot-meta">
+    <term>
+     <function>pg_get_logical_snapshot_meta(filename text) returns record</function>
+    </term>
+
+    <listitem>
+     <para>
+      Gets logical snapshot metadata about a snapshot file that is located in
+      the server's <filename>pg_logical/snapshots</filename> directory.
+      The <replaceable>filename</replaceable> argument represents the snapshot
+      file name.
+      For example:
+<screen>
+postgres=# SELECT * FROM pg_ls_logicalsnapdir();
+-[ RECORD 1 ]+-----------------------
+name         | 0-40796E18.snap
+size         | 152
+modification | 2024-08-14 16:36:32+00
+
+postgres=# SELECT * FROM pg_get_logical_snapshot_meta('0-40796E18.snap');
+-[ RECORD 1 ]--------
+magic    | 1369563137
+checksum | 1028045905
+version  | 6
+
+postgres=# SELECT ss.name, meta.* FROM pg_ls_logicalsnapdir() AS ss,
+pg_get_logical_snapshot_meta(ss.name) AS meta;
+-[ RECORD 1 ]-------------
+name     | 0-40796E18.snap
+magic    | 1369563137
+checksum | 1028045905
+version  | 6
+</screen>
+     </para>
+     <para>
+      If <replaceable>filename</replaceable> does not match a snapshot file, the
+      function raises an error.
+     </para>
+    </listitem>
+   </varlistentry>
+
+   <varlistentry id="pglogicalinspect-funcs-pg-get-logical-snapshot-info">
+    <term>
+     <function>pg_get_logical_snapshot_info(filename text) returns record</function>
+    </term>
+
+    <listitem>
+     <para>
+      Gets logical snapshot information about a snapshot file that is located in
+      the server's <filename>pg_logical/snapshots</filename> directory.
+      The <replaceable>filename</replaceable> argument represents the snapshot
+      file name.
+      For example:
+<screen>
+postgres=# SELECT * FROM pg_ls_logicalsnapdir();
+-[ RECORD 1 ]+-----------------------
+name         | 0-40796E18.snap
+size         | 152
+modification | 2024-08-14 16:36:32+00
+
+postgres=# SELECT * FROM pg_get_logical_snapshot_info('0-40796E18.snap');
+-[ RECORD 1 ]------------+-----------
+state                    | consistent
+xmin                     | 751
+xmax                     | 751
+start_decoding_at        | 0/40796AF8
+two_phase_at             | 0/40796AF8
+initial_xmin_horizon     | 0
+building_full_snapshot   | f
+in_slot_creation         | f
+last_serialized_snapshot | 0/0
+next_phase_at            | 0
+committed_count          | 0
+committed_xip            |
+catchange_count          | 2
+catchange_xip            | {751,752}
+
+postgres=# SELECT ss.name, info.* FROM pg_ls_logicalsnapdir() AS ss,
+pg_get_logical_snapshot_info(ss.name) AS info;
+-[ RECORD 1 ]------------+----------------
+name                     | 0-40796E18.snap
+state                    | consistent
+xmin                     | 751
+xmax                     | 751
+start_decoding_at        | 0/40796AF8
+two_phase_at             | 0/40796AF8
+initial_xmin_horizon     | 0
+building_full_snapshot   | f
+in_slot_creation         | f
+last_serialized_snapshot | 0/0
+next_phase_at            | 0
+committed_count          | 0
+committed_xip            |
+catchange_count          | 2
+catchange_xip            | {751,752}
+</screen>
+     </para>
+     <para>
+      If <replaceable>filename</replaceable> does not match a snapshot file, the
+      function raises an error.
+     </para>
+    </listitem>
+   </varlistentry>
+
+  </variablelist>
+ </sect2>
+
+ <sect2 id="pglogicalinspect-author">
+  <title>Author</title>
+
+  <para>
+   Bertrand Drouvot <email>bertranddrouvot.pg@gmail.com</email>
+  </para>
+ </sect2>
+
+</sect1>
diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c
index 0450f94ba8..c197198b87 100644
--- a/src/backend/replication/logical/snapbuild.c
+++ b/src/backend/replication/logical/snapbuild.c
@@ -134,6 +134,7 @@
 #include "replication/logical.h"
 #include "replication/reorderbuffer.h"
 #include "replication/snapbuild.h"
+#include "replication/snapbuild_internal.h"
 #include "storage/fd.h"
 #include "storage/lmgr.h"
 #include "storage/proc.h"
@@ -143,146 +144,6 @@
 #include "utils/memutils.h"
 #include "utils/snapmgr.h"
 #include "utils/snapshot.h"
-
-/*
- * This struct contains the current state of the snapshot building
- * machinery. Besides a forward declaration in the header, it is not exposed
- * to the public, so we can easily change its contents.
- */
-struct SnapBuild
-{
-	/* how far are we along building our first full snapshot */
-	SnapBuildState state;
-
-	/* private memory context used to allocate memory for this module. */
-	MemoryContext context;
-
-	/* all transactions < than this have committed/aborted */
-	TransactionId xmin;
-
-	/* all transactions >= than this are uncommitted */
-	TransactionId xmax;
-
-	/*
-	 * Don't replay commits from an LSN < this LSN. This can be set externally
-	 * but it will also be advanced (never retreat) from within snapbuild.c.
-	 */
-	XLogRecPtr	start_decoding_at;
-
-	/*
-	 * LSN at which two-phase decoding was enabled or LSN at which we found a
-	 * consistent point at the time of slot creation.
-	 *
-	 * The prepared transactions, that were skipped because previously
-	 * two-phase was not enabled or are not covered by initial snapshot, need
-	 * to be sent later along with commit prepared and they must be before
-	 * this point.
-	 */
-	XLogRecPtr	two_phase_at;
-
-	/*
-	 * Don't start decoding WAL until the "xl_running_xacts" information
-	 * indicates there are no running xids with an xid smaller than this.
-	 */
-	TransactionId initial_xmin_horizon;
-
-	/* Indicates if we are building full snapshot or just catalog one. */
-	bool		building_full_snapshot;
-
-	/*
-	 * Indicates if we are using the snapshot builder for the creation of a
-	 * logical replication slot. If it's true, the start point for decoding
-	 * changes is not determined yet. So we skip snapshot restores to properly
-	 * find the start point. See SnapBuildFindSnapshot() for details.
-	 */
-	bool		in_slot_creation;
-
-	/*
-	 * Snapshot that's valid to see the catalog state seen at this moment.
-	 */
-	Snapshot	snapshot;
-
-	/*
-	 * LSN of the last location we are sure a snapshot has been serialized to.
-	 */
-	XLogRecPtr	last_serialized_snapshot;
-
-	/*
-	 * The reorderbuffer we need to update with usable snapshots et al.
-	 */
-	ReorderBuffer *reorder;
-
-	/*
-	 * TransactionId at which the next phase of initial snapshot building will
-	 * happen. InvalidTransactionId if not known (i.e. SNAPBUILD_START), or
-	 * when no next phase necessary (SNAPBUILD_CONSISTENT).
-	 */
-	TransactionId next_phase_at;
-
-	/*
-	 * Array of transactions which could have catalog changes that committed
-	 * between xmin and xmax.
-	 */
-	struct
-	{
-		/* number of committed transactions */
-		size_t		xcnt;
-
-		/* available space for committed transactions */
-		size_t		xcnt_space;
-
-		/*
-		 * Until we reach a CONSISTENT state, we record commits of all
-		 * transactions, not just the catalog changing ones. Record when that
-		 * changes so we know we cannot export a snapshot safely anymore.
-		 */
-		bool		includes_all_transactions;
-
-		/*
-		 * Array of committed transactions that have modified the catalog.
-		 *
-		 * As this array is frequently modified we do *not* keep it in
-		 * xidComparator order. Instead we sort the array when building &
-		 * distributing a snapshot.
-		 *
-		 * TODO: It's unclear whether that reasoning has much merit. Every
-		 * time we add something here after becoming consistent will also
-		 * require distributing a snapshot. Storing them sorted would
-		 * potentially also make it easier to purge (but more complicated wrt
-		 * wraparound?). Should be improved if sorting while building the
-		 * snapshot shows up in profiles.
-		 */
-		TransactionId *xip;
-	}			committed;
-
-	/*
-	 * Array of transactions and subtransactions that had modified catalogs
-	 * and were running when the snapshot was serialized.
-	 *
-	 * We normally rely on some WAL record types such as HEAP2_NEW_CID to know
-	 * if the transaction has changed the catalog. But it could happen that
-	 * the logical decoding decodes only the commit record of the transaction
-	 * after restoring the previously serialized snapshot in which case we
-	 * will miss adding the xid to the snapshot and end up looking at the
-	 * catalogs with the wrong snapshot.
-	 *
-	 * Now to avoid the above problem, we serialize the transactions that had
-	 * modified the catalogs and are still running at the time of snapshot
-	 * serialization. We fill this array while restoring the snapshot and then
-	 * refer it while decoding commit to ensure if the xact has modified the
-	 * catalog. We discard this array when all the xids in the list become old
-	 * enough to matter. See SnapBuildPurgeOlderTxn for details.
-	 */
-	struct
-	{
-		/* number of transactions */
-		size_t		xcnt;
-
-		/* This array must be sorted in xidComparator order */
-		TransactionId *xip;
-	}			catchange;
-};
-
 /*
  * Starting a transaction -- which we need to do while exporting a snapshot --
  * removes knowledge about the previously used resowner, so we save it here.
@@ -1557,40 +1418,6 @@ SnapBuildWaitSnapshot(xl_running_xacts *running, TransactionId cutoff)
 	}
 }
 
-/* -----------------------------------
- * Snapshot serialization support
- * -----------------------------------
- */
-
-/*
- * We store current state of struct SnapBuild on disk in the following manner:
- *
- * struct SnapBuildOnDisk;
- * TransactionId * committed.xcnt; (*not xcnt_space*)
- * TransactionId * catchange.xcnt;
- *
- */
-typedef struct SnapBuildOnDisk
-{
-	/* first part of this struct needs to be version independent */
-
-	/* data not covered by checksum */
-	uint32		magic;
-	pg_crc32c	checksum;
-
-	/* data covered by checksum */
-
-	/* version, in case we want to support pg_upgrade */
-	uint32		version;
-	/* how large is the on disk data, excluding the constant sized part */
-	uint32		length;
-
-	/* version dependent part */
-	SnapBuild	builder;
-
-	/* variable amount of TransactionIds follows */
-} SnapBuildOnDisk;
-
 #define SnapBuildOnDiskConstantSize \
 	offsetof(SnapBuildOnDisk, builder)
 #define SnapBuildOnDiskNotChecksummedSize \
@@ -1857,34 +1684,27 @@ out:
 }
 
 /*
- * Restore a snapshot into 'builder' if previously one has been stored at the
- * location indicated by 'lsn'. Returns true if successful, false otherwise.
+ * Validate the logical snapshot file and read its contents to 'ondisk'.
  */
-static bool
-SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
+bool
+ValidateAndRestoreSnapshotFile(SnapBuildOnDisk *ondisk, const char *path,
+							   MemoryContext context, bool missing_ok)
 {
-	SnapBuildOnDisk ondisk;
 	int			fd;
-	char		path[MAXPGPATH];
-	Size		sz;
 	pg_crc32c	checksum;
-
-	/* no point in loading a snapshot if we're already there */
-	if (builder->state == SNAPBUILD_CONSISTENT)
-		return false;
-
-	sprintf(path, "%s/%X-%X.snap",
-			PG_LOGICAL_SNAPSHOTS_DIR,
-			LSN_FORMAT_ARGS(lsn));
+	Size		sz;
 
 	fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
 
-	if (fd < 0 && errno == ENOENT)
-		return false;
-	else if (fd < 0)
+	if (fd < 0)
+	{
+		if (missing_ok && errno == ENOENT)
+			return false;
+
 		ereport(ERROR,
 				(errcode_for_file_access(),
 				 errmsg("could not open file \"%s\": %m", path)));
+	}
 
 	/* ----
 	 * Make sure the snapshot had been stored safely to disk, that's normally
@@ -1897,47 +1717,46 @@ SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
 	fsync_fname(path, false);
 	fsync_fname(PG_LOGICAL_SNAPSHOTS_DIR, true);
 
-
 	/* read statically sized portion of snapshot */
-	SnapBuildRestoreContents(fd, (char *) &ondisk, SnapBuildOnDiskConstantSize, path);
+	SnapBuildRestoreContents(fd, (char *) ondisk, SnapBuildOnDiskConstantSize, path);
 
-	if (ondisk.magic != SNAPBUILD_MAGIC)
+	if (ondisk->magic != SNAPBUILD_MAGIC)
 		ereport(ERROR,
 				(errcode(ERRCODE_DATA_CORRUPTED),
 				 errmsg("snapbuild state file \"%s\" has wrong magic number: %u instead of %u",
-						path, ondisk.magic, SNAPBUILD_MAGIC)));
+						path, ondisk->magic, SNAPBUILD_MAGIC)));
 
-	if (ondisk.version != SNAPBUILD_VERSION)
+	if (ondisk->version != SNAPBUILD_VERSION)
 		ereport(ERROR,
 				(errcode(ERRCODE_DATA_CORRUPTED),
 				 errmsg("snapbuild state file \"%s\" has unsupported version: %u instead of %u",
-						path, ondisk.version, SNAPBUILD_VERSION)));
+						path, ondisk->version, SNAPBUILD_VERSION)));
 
 	INIT_CRC32C(checksum);
 	COMP_CRC32C(checksum,
-				((char *) &ondisk) + SnapBuildOnDiskNotChecksummedSize,
+				((char *) ondisk) + SnapBuildOnDiskNotChecksummedSize,
 				SnapBuildOnDiskConstantSize - SnapBuildOnDiskNotChecksummedSize);
 
 	/* read SnapBuild */
-	SnapBuildRestoreContents(fd, (char *) &ondisk.builder, sizeof(SnapBuild), path);
-	COMP_CRC32C(checksum, &ondisk.builder, sizeof(SnapBuild));
+	SnapBuildRestoreContents(fd, (char *) &ondisk->builder, sizeof(SnapBuild), path);
+	COMP_CRC32C(checksum, &ondisk->builder, sizeof(SnapBuild));
 
 	/* restore committed xacts information */
-	if (ondisk.builder.committed.xcnt > 0)
+	if (ondisk->builder.committed.xcnt > 0)
 	{
-		sz = sizeof(TransactionId) * ondisk.builder.committed.xcnt;
-		ondisk.builder.committed.xip = MemoryContextAllocZero(builder->context, sz);
-		SnapBuildRestoreContents(fd, (char *) ondisk.builder.committed.xip, sz, path);
-		COMP_CRC32C(checksum, ondisk.builder.committed.xip, sz);
+		sz = sizeof(TransactionId) * ondisk->builder.committed.xcnt;
+		ondisk->builder.committed.xip = MemoryContextAllocZero(context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.committed.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.committed.xip, sz);
 	}
 
 	/* restore catalog modifying xacts information */
-	if (ondisk.builder.catchange.xcnt > 0)
+	if (ondisk->builder.catchange.xcnt > 0)
 	{
-		sz = sizeof(TransactionId) * ondisk.builder.catchange.xcnt;
-		ondisk.builder.catchange.xip = MemoryContextAllocZero(builder->context, sz);
-		SnapBuildRestoreContents(fd, (char *) ondisk.builder.catchange.xip, sz, path);
-		COMP_CRC32C(checksum, ondisk.builder.catchange.xip, sz);
+		sz = sizeof(TransactionId) * ondisk->builder.catchange.xcnt;
+		ondisk->builder.catchange.xip = MemoryContextAllocZero(context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.catchange.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.catchange.xip, sz);
 	}
 
 	if (CloseTransientFile(fd) != 0)
@@ -1948,11 +1767,36 @@ SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
 	FIN_CRC32C(checksum);
 
 	/* verify checksum of what we've read */
-	if (!EQ_CRC32C(checksum, ondisk.checksum))
+	if (!EQ_CRC32C(checksum, ondisk->checksum))
 		ereport(ERROR,
 				(errcode(ERRCODE_DATA_CORRUPTED),
 				 errmsg("checksum mismatch for snapbuild state file \"%s\": is %u, should be %u",
-						path, checksum, ondisk.checksum)));
+						path, checksum, ondisk->checksum)));
+
+	return true;
+}
+
+/*
+ * Restore a snapshot into 'builder' if previously one has been stored at the
+ * location indicated by 'lsn'. Returns true if successful, false otherwise.
+ */
+static bool
+SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
+{
+	SnapBuildOnDisk ondisk;
+	char		path[MAXPGPATH];
+
+	/* no point in loading a snapshot if we're already there */
+	if (builder->state == SNAPBUILD_CONSISTENT)
+		return false;
+
+	sprintf(path, "%s/%X-%X.snap",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			LSN_FORMAT_ARGS(lsn));
+
+	/* validate and restore the snapshot to 'ondisk' */
+	if (!ValidateAndRestoreSnapshotFile(&ondisk, path, builder->context, true))
+		return false;
 
 	/*
 	 * ok, we now have a sensible snapshot here, figure out if it has more
diff --git a/src/include/replication/snapbuild.h b/src/include/replication/snapbuild.h
index caa5113ff8..3c1454df99 100644
--- a/src/include/replication/snapbuild.h
+++ b/src/include/replication/snapbuild.h
@@ -15,6 +15,10 @@
 #include "access/xlogdefs.h"
 #include "utils/snapmgr.h"
 
+/*
+ * Please keep get_snapbuild_state_desc() (located in the pg_logicalinspect
+ * module) updated if a change needs to be made to SnapBuildState.
+ */
 typedef enum
 {
 	/*
@@ -46,7 +50,7 @@ typedef enum
 	SNAPBUILD_CONSISTENT = 2,
 } SnapBuildState;
 
-/* forward declare so we don't have to expose the struct to the public */
+/* forward declare so we don't have to include snapbuild_internal.h */
 struct SnapBuild;
 typedef struct SnapBuild SnapBuild;
 
diff --git a/src/include/replication/snapbuild_internal.h b/src/include/replication/snapbuild_internal.h
new file mode 100644
index 0000000000..ade75c94f1
--- /dev/null
+++ b/src/include/replication/snapbuild_internal.h
@@ -0,0 +1,199 @@
+/*-------------------------------------------------------------------------
+ *
+ * snapbuild_internal.h
+ *    This file contains declarations for logical decoding utility
+ *    functions for internal use.
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * src/include/replication/snapbuild_internal.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef SNAPBUILD_INTERNAL_H
+#define SNAPBUILD_INTERNAL_H
+
+#include "port/pg_crc32c.h"
+#include "replication/reorderbuffer.h"
+#include "replication/snapbuild.h"
+
+/*
+ * This struct contains the current state of the snapshot building
+ * machinery. It is exposed to the public, so pay attention when changing its
+ * contents.
+ */
+typedef struct SnapBuild
+{
+	/* how far are we along building our first full snapshot */
+	SnapBuildState state;
+
+	/* private memory context used to allocate memory for this module. */
+	MemoryContext context;
+
+	/* all transactions < than this have committed/aborted */
+	TransactionId xmin;
+
+	/* all transactions >= than this are uncommitted */
+	TransactionId xmax;
+
+	/*
+	 * Don't replay commits from an LSN < this LSN. This can be set externally
+	 * but it will also be advanced (never retreat) from within snapbuild.c.
+	 */
+	XLogRecPtr	start_decoding_at;
+
+	/*
+	 * LSN at which two-phase decoding was enabled or LSN at which we found a
+	 * consistent point at the time of slot creation.
+	 *
+	 * The prepared transactions, that were skipped because previously
+	 * two-phase was not enabled or are not covered by initial snapshot, need
+	 * to be sent later along with commit prepared and they must be before
+	 * this point.
+	 */
+	XLogRecPtr	two_phase_at;
+
+	/*
+	 * Don't start decoding WAL until the "xl_running_xacts" information
+	 * indicates there are no running xids with an xid smaller than this.
+	 */
+	TransactionId initial_xmin_horizon;
+
+	/* Indicates if we are building full snapshot or just catalog one. */
+	bool		building_full_snapshot;
+
+	/*
+	 * Indicates if we are using the snapshot builder for the creation of a
+	 * logical replication slot. If it's true, the start point for decoding
+	 * changes is not determined yet. So we skip snapshot restores to properly
+	 * find the start point. See SnapBuildFindSnapshot() for details.
+	 */
+	bool		in_slot_creation;
+
+	/*
+	 * Snapshot that's valid to see the catalog state seen at this moment.
+	 */
+	Snapshot	snapshot;
+
+	/*
+	 * LSN of the last location we are sure a snapshot has been serialized to.
+	 */
+	XLogRecPtr	last_serialized_snapshot;
+
+	/*
+	 * The reorderbuffer we need to update with usable snapshots et al.
+	 */
+	ReorderBuffer *reorder;
+
+	/*
+	 * TransactionId at which the next phase of initial snapshot building will
+	 * happen. InvalidTransactionId if not known (i.e. SNAPBUILD_START), or
+	 * when no next phase necessary (SNAPBUILD_CONSISTENT).
+	 */
+	TransactionId next_phase_at;
+
+	/*
+	 * Array of transactions which could have catalog changes that committed
+	 * between xmin and xmax.
+	 */
+	struct
+	{
+		/* number of committed transactions */
+		size_t		xcnt;
+
+		/* available space for committed transactions */
+		size_t		xcnt_space;
+
+		/*
+		 * Until we reach a CONSISTENT state, we record commits of all
+		 * transactions, not just the catalog changing ones. Record when that
+		 * changes so we know we cannot export a snapshot safely anymore.
+		 */
+		bool		includes_all_transactions;
+
+		/*
+		 * Array of committed transactions that have modified the catalog.
+		 *
+		 * As this array is frequently modified we do *not* keep it in
+		 * xidComparator order. Instead we sort the array when building &
+		 * distributing a snapshot.
+		 *
+		 * TODO: It's unclear whether that reasoning has much merit. Every
+		 * time we add something here after becoming consistent will also
+		 * require distributing a snapshot. Storing them sorted would
+		 * potentially also make it easier to purge (but more complicated wrt
+		 * wraparound?). Should be improved if sorting while building the
+		 * snapshot shows up in profiles.
+		 */
+		TransactionId *xip;
+	}			committed;
+
+	/*
+	 * Array of transactions and subtransactions that had modified catalogs
+	 * and were running when the snapshot was serialized.
+	 *
+	 * We normally rely on some WAL record types such as HEAP2_NEW_CID to know
+	 * if the transaction has changed the catalog. But it could happen that
+	 * the logical decoding decodes only the commit record of the transaction
+	 * after restoring the previously serialized snapshot in which case we
+	 * will miss adding the xid to the snapshot and end up looking at the
+	 * catalogs with the wrong snapshot.
+	 *
+	 * Now to avoid the above problem, we serialize the transactions that had
+	 * modified the catalogs and are still running at the time of snapshot
+	 * serialization. We fill this array while restoring the snapshot and then
+	 * refer it while decoding commit to ensure if the xact has modified the
+	 * catalog. We discard this array when all the xids in the list become old
+	 * enough to matter. See SnapBuildPurgeOlderTxn for details.
+	 */
+	struct
+	{
+		/* number of transactions */
+		size_t		xcnt;
+
+		/* This array must be sorted in xidComparator order */
+		TransactionId *xip;
+	}			catchange;
+} SnapBuild;
+
+/* -----------------------------------
+ * Snapshot serialization support
+ * -----------------------------------
+ */
+
+/*
+ * We store current state of struct SnapBuild on disk in the following manner:
+ *
+ * struct SnapBuildOnDisk;
+ * TransactionId * committed.xcnt; (*not xcnt_space*)
+ * TransactionId * catchange.xcnt;
+ *
+ * Check if the SnapBuildOnDiskConstantSize and SnapBuildOnDiskNotChecksummedSize
+ * macros need to be updated when modifying the SnapBuildOnDisk struct.
+ */
+typedef struct SnapBuildOnDisk
+{
+	/* first part of this struct needs to be version independent */
+
+	/* data not covered by checksum */
+	uint32		magic;
+	pg_crc32c	checksum;
+
+	/* data covered by checksum */
+
+	/* version, in case we want to support pg_upgrade */
+	uint32		version;
+	/* how large is the on disk data, excluding the constant sized part */
+	uint32		length;
+
+	/* version dependent part */
+	SnapBuild	builder;
+
+	/* variable amount of TransactionIds follows */
+} SnapBuildOnDisk;
+
+extern bool ValidateAndRestoreSnapshotFile(SnapBuildOnDisk *ondisk, const char *path,
+										   MemoryContext context, bool missing_ok);
+
+#endif							/* SNAPBUILD_INTERNAL_H */
-- 
2.34.1

#54Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Bertrand Drouvot (#53)
Re: Add contrib/pg_logicalsnapinspect

On Thu, Oct 10, 2024 at 6:10 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Hi,

On Thu, Oct 10, 2024 at 12:05:10AM -0700, Masahiko Sawada wrote:

On Wed, Oct 9, 2024 at 8:32 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

So I think that having construct_array_builtin()/deconstruct_array_builtin()
taking care of XIDOID is the way to go. If that makes sense to you then I'll
submit a dedicated patch for it, thoughts?

Your explanation makes sense to me.

Thanks for sharing your thoughts.

I think it can be included in the main pg_logicalinspect patch as this change
is a part of it.

Okay, let's keep the discussion here. Please find attached v13 that takes care
of your previous remarks and Peter's one ([1]).

FYI, v13 is splitted into 2 sub-patches (0001 for the discussion related to
XIDOID and construct_array_builtin() and 0002 for the module itself).

Thank you for updating the patch!

FWIW, the elmbyval and elmalign values that are added in 0001 have been deduced
from:

postgres=# select typbyval, typalign from pg_type where typname = 'xid';
typbyval | typalign
----------+----------
t | i
(1 row)

+1

The patches mostly look good to me. Here are some minor comments:

+       sprintf(path, "%s/%s",
+                       PG_LOGICAL_SNAPSHOTS_DIR,
+                       text_to_cstring(filename_t));
+
+       /* Validate and restore the snapshot to 'ondisk' */
+       ValidateAndRestoreSnapshotFile(&ondisk, path,
CurrentMemoryContext, false);
+
+       /* Build a tuple descriptor for our result type */
+       if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+               elog(ERROR, "return type must be a row type");
+
I think it would be better to check the result type before reading the
snapshot file.
---
+       values[i++] = Int64GetDatum((int64) ondisk.checksum);

Why is only checksum casted to int64? With that, it can show a
checksum value as a non-netagive integer but is it really necessary?
For instance, page_header() function in pageinspect shows a page
checksum as smallint.

---@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/

output_iso and tmp_check_iso should be added.

---
 /*
- * Restore a snapshot into 'builder' if previously one has been stored at the
- * location indicated by 'lsn'. Returns true if successful, false otherwise.
+ * Validate the logical snapshot file and read its contents to 'ondisk'.
  */
-static bool
-SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
+bool
+ValidateAndRestoreSnapshotFile(SnapBuildOnDisk *ondisk, const char *path,
+                              MemoryContext context, bool missing_ok)

I think it would be better to add some descriptions of the function
arguments, particularly context and missing_ok.

The names of all other functions in snapbuild.c have the "SnapBuild"
prefix. I think it's better to follow it. How about renaming it to
SnapBuildReadSnapshot(), SnapBuildRestoreSnapshot(), or something
along those lines?

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com

#55Peter Smith
smithpb2250@gmail.com
In reply to: Bertrand Drouvot (#53)
Re: Add contrib/pg_logicalsnapinspect

Hi, Here are a few comments for patch set v13*

//////////

Patch v13-0001

======
Commit message

1.1
/were no use case/was no use case/

~~~

1.2
It seemed a bit odd that the switch cases for
'construct_array_builtin' are not the same as those for
'deconstruct_array_builtin'.

For example, all these ones seem missing from deconstruct:
case INT4OID:
case INT8OID:
case NAMEOID:
case REGTYPEOID:

I know that has nothing to do with your patch, and I guess it does not
cause any problems otherwise there would be ERRORs. But, if you are to
follow this same current pattern, then perhaps you don't need to add
your new case for 'deconstruct_array_builtin', since AFAICT you are
never using it.

//////////

Patch v13-0002

======
pg_get_logical_snapshot_meta:

2.1
+ Datum values[PG_GET_LOGICAL_SNAPSHOT_META_COLS];
+ bool nulls[PG_GET_LOGICAL_SNAPSHOT_META_COLS];

FWIW, if you wanted to avoid a few lines you could initialise the
nulls array during the declaration.
bool nulls[PG_GET_LOGICAL_SNAPSHOT_META_COLS] = {0};

This seems a common pattern in other source code, and it replaces the
need for the subsequent memset.

~~~

pg_get_logical_snapshot_info:

2.2
+ Datum values[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS];
+ bool nulls[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS];

Ditto of #2.1. You could instead just initialise in the declaration like:
bool nulls[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS] = {0};

======
Kind Regards,
Peter Smith.
Fujitsu Australia

#56Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: Masahiko Sawada (#54)
2 attachment(s)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Thu, Oct 10, 2024 at 05:38:43PM -0700, Masahiko Sawada wrote:

On Thu, Oct 10, 2024 at 6:10 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

The patches mostly look good to me. Here are some minor comments:

Thanks for looking at it!

+       sprintf(path, "%s/%s",
+                       PG_LOGICAL_SNAPSHOTS_DIR,
+                       text_to_cstring(filename_t));
+
+       /* Validate and restore the snapshot to 'ondisk' */
+       ValidateAndRestoreSnapshotFile(&ondisk, path,
CurrentMemoryContext, false);
+
+       /* Build a tuple descriptor for our result type */
+       if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+               elog(ERROR, "return type must be a row type");
+
I think it would be better to check the result type before reading the
snapshot file.

Agree, done in v14.

---
+       values[i++] = Int64GetDatum((int64) ondisk.checksum);

Why is only checksum casted to int64? With that, it can show a
checksum value as a non-netagive integer but is it really necessary?
For instance, page_header() function in pageinspect shows a page
checksum as smallint.

Yeah, pd_checksum in PageHeaderData is uint16 while checksum in SnapBuildOnDisk
is pg_crc32c. The reason why it is casted to int64 is explained in [1]/messages/by-id/ZvLuhh5pzpIqolkW@ip-10-97-1-34.eu-west-3.compute.internal, does that
make sense to you?

---@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/

output_iso and tmp_check_iso should be added.

Yeah, done in v14.

---
/*
- * Restore a snapshot into 'builder' if previously one has been stored at the
- * location indicated by 'lsn'. Returns true if successful, false otherwise.
+ * Validate the logical snapshot file and read its contents to 'ondisk'.
*/
-static bool
-SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
+bool
+ValidateAndRestoreSnapshotFile(SnapBuildOnDisk *ondisk, const char *path,
+                              MemoryContext context, bool missing_ok)

I think it would be better to add some descriptions of the function
arguments, particularly context and missing_ok.

Thought about it but somehow managed to missed it. Thanks, done in v14.

The names of all other functions in snapbuild.c have the "SnapBuild"
prefix. I think it's better to follow it. How about renaming it to
SnapBuildReadSnapshot(), SnapBuildRestoreSnapshot(), or something
along those lines?

That makes sense. I opted for SnapBuildRestoreSnapshot() (as it's calling
SnapBuildRestoreContents()).

[1]: /messages/by-id/ZvLuhh5pzpIqolkW@ip-10-97-1-34.eu-west-3.compute.internal

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachments:

v14-0001-Add-XIDOID-in-construct_array_builtin.patchtext/x-diff; charset=us-asciiDownload
From 306d1d446773cf5d151fd2faa3b30df451c3dc1d Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Thu, 10 Oct 2024 04:09:53 +0000
Subject: [PATCH v14 1/2] Add XIDOID in construct_array_builtin()

Using construct_array_builtin() for XIDOID is a new use case that is coming with
a new module (not added in the code tree yet).

d746021de1 (which introduced construct_array_builtin()) did not take care of
XIDOID because there was no use case at that time. Now that there is one, let's
add XIDOID.
---
 src/backend/utils/adt/arrayfuncs.c | 6 ++++++
 1 file changed, 6 insertions(+)
 100.0% src/backend/utils/adt/

diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c
index e5c7e57a5d..41434279c5 100644
--- a/src/backend/utils/adt/arrayfuncs.c
+++ b/src/backend/utils/adt/arrayfuncs.c
@@ -3447,6 +3447,12 @@ construct_array_builtin(Datum *elems, int nelems, Oid elmtype)
 			elmalign = TYPALIGN_SHORT;
 			break;
 
+		case XIDOID:
+			elmlen = sizeof(TransactionId);
+			elmbyval = true;
+			elmalign = TYPALIGN_INT;
+			break;
+
 		default:
 			elog(ERROR, "type %u not supported by construct_array_builtin()", elmtype);
 			/* keep compiler quiet */
-- 
2.34.1

v14-0002-Add-contrib-pg_logicalinspect.patchtext/x-diff; charset=us-asciiDownload
From 420616a91db9967901aaa09825b970f0775a3405 Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Wed, 14 Aug 2024 08:46:05 +0000
Subject: [PATCH v14 2/2] Add contrib/pg_logicalinspect

Provides SQL functions that allow to inspect logical decoding components.

It currently allows to inspect the contents of serialized logical snapshots of
a running database cluster, which is useful for debugging or educational
purposes.
---
 contrib/Makefile                              |   1 +
 contrib/meson.build                           |   1 +
 contrib/pg_logicalinspect/.gitignore          |   6 +
 contrib/pg_logicalinspect/Makefile            |  31 ++
 .../expected/logical_inspect.out              |  52 ++++
 contrib/pg_logicalinspect/logicalinspect.conf |   1 +
 contrib/pg_logicalinspect/meson.build         |  39 +++
 .../pg_logicalinspect--1.0.sql                |  43 +++
 contrib/pg_logicalinspect/pg_logicalinspect.c | 167 +++++++++++
 .../pg_logicalinspect.control                 |   5 +
 .../specs/logical_inspect.spec                |  34 +++
 doc/src/sgml/contrib.sgml                     |   1 +
 doc/src/sgml/filelist.sgml                    |   1 +
 doc/src/sgml/pglogicalinspect.sgml            | 143 +++++++++
 src/backend/replication/logical/snapbuild.c   | 274 ++++--------------
 src/include/replication/snapbuild.h           |   6 +-
 src/include/replication/snapbuild_internal.h  | 199 +++++++++++++
 17 files changed, 790 insertions(+), 214 deletions(-)
   7.6% contrib/pg_logicalinspect/expected/
   5.3% contrib/pg_logicalinspect/specs/
  26.0% contrib/pg_logicalinspect/
  14.0% doc/src/sgml/
  26.2% src/backend/replication/logical/
  20.5% src/include/replication/

diff --git a/contrib/Makefile b/contrib/Makefile
index abd780f277..952855d9b6 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -32,6 +32,7 @@ SUBDIRS = \
 		passwordcheck	\
 		pg_buffercache	\
 		pg_freespacemap \
+		pg_logicalinspect \
 		pg_prewarm	\
 		pg_stat_statements \
 		pg_surgery	\
diff --git a/contrib/meson.build b/contrib/meson.build
index 14a8906865..159ff41555 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -46,6 +46,7 @@ subdir('passwordcheck')
 subdir('pg_buffercache')
 subdir('pgcrypto')
 subdir('pg_freespacemap')
+subdir('pg_logicalinspect')
 subdir('pg_prewarm')
 subdir('pgrowlocks')
 subdir('pg_stat_statements')
diff --git a/contrib/pg_logicalinspect/.gitignore b/contrib/pg_logicalinspect/.gitignore
new file mode 100644
index 0000000000..b4903eba65
--- /dev/null
+++ b/contrib/pg_logicalinspect/.gitignore
@@ -0,0 +1,6 @@
+# Generated subdirectories
+/log/
+/results/
+/output_iso/
+/tmp_check/
+/tmp_check_iso/
diff --git a/contrib/pg_logicalinspect/Makefile b/contrib/pg_logicalinspect/Makefile
new file mode 100644
index 0000000000..55124514d4
--- /dev/null
+++ b/contrib/pg_logicalinspect/Makefile
@@ -0,0 +1,31 @@
+# contrib/pg_logicalinspect/Makefile
+
+MODULE_big = pg_logicalinspect
+OBJS = \
+	$(WIN32RES) \
+	pg_logicalinspect.o
+PGFILEDESC = "pg_logicalinspect - functions to inspect logical decoding components"
+
+EXTENSION = pg_logicalinspect
+DATA = pg_logicalinspect--1.0.sql
+
+EXTRA_INSTALL = contrib/test_decoding
+
+ISOLATION = logical_inspect
+
+ISOLATION_OPTS = --temp-config $(top_srcdir)/contrib/pg_logicalinspect/logicalinspect.conf
+
+# Disabled because these tests require "wal_level=logical", which
+# some installcheck users do not have (e.g. buildfarm clients).
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/pg_logicalinspect
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/pg_logicalinspect/expected/logical_inspect.out b/contrib/pg_logicalinspect/expected/logical_inspect.out
new file mode 100644
index 0000000000..d95efa4d1e
--- /dev/null
+++ b/contrib/pg_logicalinspect/expected/logical_inspect.out
@@ -0,0 +1,52 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s0_init s0_begin s0_savepoint s0_truncate s1_checkpoint s1_get_changes s0_commit s0_begin s0_insert s1_checkpoint s1_get_changes s0_commit s1_get_changes s1_get_logical_snapshot_info s1_get_logical_snapshot_meta
+step s0_init: SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding');
+?column?
+--------
+init    
+(1 row)
+
+step s0_begin: BEGIN;
+step s0_savepoint: SAVEPOINT sp1;
+step s0_truncate: TRUNCATE tbl1;
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data
+----
+(0 rows)
+
+step s0_commit: COMMIT;
+step s0_begin: BEGIN;
+step s0_insert: INSERT INTO tbl1 VALUES (1);
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                   
+---------------------------------------
+BEGIN                                  
+table public.tbl1: TRUNCATE: (no-flags)
+COMMIT                                 
+(3 rows)
+
+step s0_commit: COMMIT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                                         
+-------------------------------------------------------------
+BEGIN                                                        
+table public.tbl1: INSERT: val1[integer]:1 val2[integer]:null
+COMMIT                                                       
+(3 rows)
+
+step s1_get_logical_snapshot_info: SELECT info.state, info.catchange_count, array_length(info.catchange_xip,1) AS catchange_array_length, info.committed_count, array_length(info.committed_xip,1) AS committed_array_length FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info ORDER BY 2;
+state     |catchange_count|catchange_array_length|committed_count|committed_array_length
+----------+---------------+----------------------+---------------+----------------------
+consistent|              0|                      |              2|                     2
+consistent|              2|                     2|              0|                      
+(2 rows)
+
+step s1_get_logical_snapshot_meta: SELECT COUNT(meta.*) from pg_ls_logicalsnapdir(), pg_get_logical_snapshot_meta(name) as meta;
+count
+-----
+    2
+(1 row)
+
diff --git a/contrib/pg_logicalinspect/logicalinspect.conf b/contrib/pg_logicalinspect/logicalinspect.conf
new file mode 100644
index 0000000000..e3d257315f
--- /dev/null
+++ b/contrib/pg_logicalinspect/logicalinspect.conf
@@ -0,0 +1 @@
+wal_level = logical
diff --git a/contrib/pg_logicalinspect/meson.build b/contrib/pg_logicalinspect/meson.build
new file mode 100644
index 0000000000..3ec635509b
--- /dev/null
+++ b/contrib/pg_logicalinspect/meson.build
@@ -0,0 +1,39 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+pg_logicalinspect_sources = files('pg_logicalinspect.c')
+
+if host_system == 'windows'
+  pg_logicalinspect_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+    '--NAME', 'pg_logicalinspect',
+    '--FILEDESC', 'pg_logicalinspect - functions to inspect logical decoding components',])
+endif
+
+pg_logicalinspect = shared_module('pg_logicalinspect',
+  pg_logicalinspect_sources,
+  kwargs: contrib_mod_args + {
+      'dependencies': contrib_mod_args['dependencies'],
+  },
+)
+contrib_targets += pg_logicalinspect
+
+install_data(
+  'pg_logicalinspect.control',
+  'pg_logicalinspect--1.0.sql',
+  kwargs: contrib_data_args,
+)
+
+tests += {
+  'name': 'pg_logicalinspect',
+  'sd': meson.current_source_dir(),
+  'bd': meson.current_build_dir(),
+  'isolation': {
+    'specs': [
+      'logical_inspect',
+    ],
+    'regress_args': [
+      '--temp-config', files('logicalinspect.conf'),
+    ],
+    # see above
+    'runningcheck': false,
+  },
+}
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql b/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
new file mode 100644
index 0000000000..c773f6e458
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
@@ -0,0 +1,43 @@
+/* contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_logicalinspect" to load this file. \quit
+
+--
+-- pg_get_logical_snapshot_meta()
+--
+CREATE FUNCTION pg_get_logical_snapshot_meta(IN filename text,
+    OUT magic int4,
+    OUT checksum int8,
+    OUT version int4
+)
+AS 'MODULE_PATHNAME', 'pg_get_logical_snapshot_meta'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_logical_snapshot_meta(text) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_logical_snapshot_meta(text) TO pg_read_server_files;
+
+--
+-- pg_get_logical_snapshot_info()
+--
+CREATE FUNCTION pg_get_logical_snapshot_info(IN filename text,
+    OUT state text,
+    OUT xmin xid,
+    OUT xmax xid,
+    OUT start_decoding_at pg_lsn,
+    OUT two_phase_at pg_lsn,
+    OUT initial_xmin_horizon xid,
+    OUT building_full_snapshot boolean,
+    OUT in_slot_creation boolean,
+    OUT last_serialized_snapshot pg_lsn,
+    OUT next_phase_at xid,
+    OUT committed_count int8,
+    OUT committed_xip xid[],
+    OUT catchange_count int8,
+    OUT catchange_xip xid[]
+)
+AS 'MODULE_PATHNAME', 'pg_get_logical_snapshot_info'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_logical_snapshot_info(text) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_logical_snapshot_info(text) TO pg_read_server_files;
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.c b/contrib/pg_logicalinspect/pg_logicalinspect.c
new file mode 100644
index 0000000000..fabfe2a10c
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.c
@@ -0,0 +1,167 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_logicalinspect.c
+ *		  Functions to inspect contents of PostgreSQL logical snapshots
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ *		  contrib/pg_logicalinspect/pg_logicalinspect.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "funcapi.h"
+#include "replication/snapbuild_internal.h"
+#include "utils/array.h"
+#include "utils/builtins.h"
+#include "utils/pg_lsn.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_meta);
+PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_info);
+
+/* Return the description of SnapBuildState */
+static const char *
+get_snapbuild_state_desc(SnapBuildState state)
+{
+	const char *stateDesc = "unknown state";
+
+	switch (state)
+	{
+		case SNAPBUILD_START:
+			stateDesc = "start";
+			break;
+		case SNAPBUILD_BUILDING_SNAPSHOT:
+			stateDesc = "building";
+			break;
+		case SNAPBUILD_FULL_SNAPSHOT:
+			stateDesc = "full";
+			break;
+		case SNAPBUILD_CONSISTENT:
+			stateDesc = "consistent";
+			break;
+	}
+
+	return stateDesc;
+}
+
+/*
+ * Retrieve the logical snapshot file metadata.
+ */
+Datum
+pg_get_logical_snapshot_meta(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_META_COLS 3
+	SnapBuildOnDisk ondisk;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_META_COLS] = {0};
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_META_COLS] = {0};
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+	int			i = 0;
+	text	   *filename_t = PG_GETARG_TEXT_PP(0);
+
+	sprintf(path, "%s/%s",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			text_to_cstring(filename_t));
+
+	/* Build a tuple descriptor for our result type */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	/* Validate and restore the snapshot to 'ondisk' */
+	SnapBuildRestoreSnapshot(&ondisk, path, CurrentMemoryContext, false);
+
+	values[i++] = UInt32GetDatum(ondisk.magic);
+	values[i++] = Int64GetDatum((int64) ondisk.checksum);
+	values[i++] = UInt32GetDatum(ondisk.version);
+
+	Assert(i == PG_GET_LOGICAL_SNAPSHOT_META_COLS);
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_META_COLS
+}
+
+Datum
+pg_get_logical_snapshot_info(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_INFO_COLS 14
+	SnapBuildOnDisk ondisk;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS] = {0};
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS] = {0};
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+	int			i = 0;
+	text	   *filename_t = PG_GETARG_TEXT_PP(0);
+
+	sprintf(path, "%s/%s",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			text_to_cstring(filename_t));
+
+	/* Build a tuple descriptor for our result type */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	/* Validate and restore the snapshot to 'ondisk' */
+	SnapBuildRestoreSnapshot(&ondisk, path, CurrentMemoryContext, false);
+
+	values[i++] = CStringGetTextDatum(get_snapbuild_state_desc(ondisk.builder.state));
+	values[i++] = TransactionIdGetDatum(ondisk.builder.xmin);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.xmax);
+	values[i++] = LSNGetDatum(ondisk.builder.start_decoding_at);
+	values[i++] = LSNGetDatum(ondisk.builder.two_phase_at);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.initial_xmin_horizon);
+	values[i++] = BoolGetDatum(ondisk.builder.building_full_snapshot);
+	values[i++] = BoolGetDatum(ondisk.builder.in_slot_creation);
+	values[i++] = LSNGetDatum(ondisk.builder.last_serialized_snapshot);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.next_phase_at);
+
+	values[i++] = Int64GetDatum(ondisk.builder.committed.xcnt);
+	if (ondisk.builder.committed.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.committed.xcnt * sizeof(Datum));
+
+		for (int j = 0; j < ondisk.builder.committed.xcnt; j++)
+			arrayelems[j] = TransactionIdGetDatum(ondisk.builder.committed.xip[j]);
+
+		values[i++] = PointerGetDatum(construct_array_builtin(arrayelems,
+															  ondisk.builder.committed.xcnt,
+															  XIDOID));
+	}
+	else
+		nulls[i++] = true;
+
+	values[i++] = Int64GetDatum(ondisk.builder.catchange.xcnt);
+	if (ondisk.builder.catchange.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.catchange.xcnt * sizeof(Datum));
+
+		for (int j = 0; j < ondisk.builder.catchange.xcnt; j++)
+			arrayelems[j] = TransactionIdGetDatum(ondisk.builder.catchange.xip[j]);
+
+		values[i++] = PointerGetDatum(construct_array_builtin(arrayelems,
+															  ondisk.builder.catchange.xcnt,
+															  XIDOID));
+	}
+	else
+		nulls[i++] = true;
+
+	Assert(i == PG_GET_LOGICAL_SNAPSHOT_INFO_COLS);
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_INFO_COLS
+}
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.control b/contrib/pg_logicalinspect/pg_logicalinspect.control
new file mode 100644
index 0000000000..b4a70e57ba
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.control
@@ -0,0 +1,5 @@
+# pg_logicalinspect extension
+comment = 'functions to inspect logical decoding components'
+default_version = '1.0'
+module_pathname = '$libdir/pg_logicalinspect'
+relocatable = true
diff --git a/contrib/pg_logicalinspect/specs/logical_inspect.spec b/contrib/pg_logicalinspect/specs/logical_inspect.spec
new file mode 100644
index 0000000000..9851a6c18e
--- /dev/null
+++ b/contrib/pg_logicalinspect/specs/logical_inspect.spec
@@ -0,0 +1,34 @@
+# Test the pg_logicalinspect functions: that needs some permutation to
+# ensure that we are creating multiple logical snapshots and that one of them
+# contains ongoing catalogs changes.
+setup
+{
+    DROP TABLE IF EXISTS tbl1;
+    CREATE TABLE tbl1 (val1 integer, val2 integer);
+    CREATE EXTENSION pg_logicalinspect;
+}
+
+teardown
+{
+    DROP TABLE tbl1;
+    SELECT 'stop' FROM pg_drop_replication_slot('isolation_slot');
+    DROP EXTENSION pg_logicalinspect;
+}
+
+session "s0"
+setup { SET synchronous_commit=on; }
+step "s0_init" { SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding'); }
+step "s0_begin" { BEGIN; }
+step "s0_savepoint" { SAVEPOINT sp1; }
+step "s0_truncate" { TRUNCATE tbl1; }
+step "s0_insert" { INSERT INTO tbl1 VALUES (1); }
+step "s0_commit" { COMMIT; }
+
+session "s1"
+setup { SET synchronous_commit=on; }
+step "s1_checkpoint" { CHECKPOINT; }
+step "s1_get_changes" { SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0'); }
+step "s1_get_logical_snapshot_meta" { SELECT COUNT(meta.*) from pg_ls_logicalsnapdir(), pg_get_logical_snapshot_meta(name) as meta;}
+step "s1_get_logical_snapshot_info" { SELECT info.state, info.catchange_count, array_length(info.catchange_xip,1) AS catchange_array_length, info.committed_count, array_length(info.committed_xip,1) AS committed_array_length FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info ORDER BY 2; }
+
+permutation "s0_init" "s0_begin" "s0_savepoint" "s0_truncate" "s1_checkpoint" "s1_get_changes" "s0_commit" "s0_begin" "s0_insert" "s1_checkpoint" "s1_get_changes" "s0_commit" "s1_get_changes" "s1_get_logical_snapshot_info" "s1_get_logical_snapshot_meta"
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index 44639a8dca..7c381949a5 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -154,6 +154,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
  &pgbuffercache;
  &pgcrypto;
  &pgfreespacemap;
+ &pglogicalinspect;
  &pgprewarm;
  &pgrowlocks;
  &pgstatstatements;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index a7ff5f8264..66e6dccd4c 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -143,6 +143,7 @@
 <!ENTITY pgbuffercache   SYSTEM "pgbuffercache.sgml">
 <!ENTITY pgcrypto        SYSTEM "pgcrypto.sgml">
 <!ENTITY pgfreespacemap  SYSTEM "pgfreespacemap.sgml">
+<!ENTITY pglogicalinspect  SYSTEM "pglogicalinspect.sgml">
 <!ENTITY pgprewarm       SYSTEM "pgprewarm.sgml">
 <!ENTITY pgrowlocks      SYSTEM "pgrowlocks.sgml">
 <!ENTITY pgstatstatements SYSTEM "pgstatstatements.sgml">
diff --git a/doc/src/sgml/pglogicalinspect.sgml b/doc/src/sgml/pglogicalinspect.sgml
new file mode 100644
index 0000000000..e0fac997b6
--- /dev/null
+++ b/doc/src/sgml/pglogicalinspect.sgml
@@ -0,0 +1,143 @@
+<!-- doc/src/sgml/pglogicalinspect.sgml -->
+
+<sect1 id="pglogicalinspect" xreflabel="pg_logicalinspect">
+ <title>pg_logicalinspect &mdash; logical decoding components inspection</title>
+
+ <indexterm zone="pglogicalinspect">
+  <primary>pg_logicalinspect</primary>
+ </indexterm>
+
+ <para>
+  The <filename>pg_logicalinspect</filename> module provides SQL functions
+  that allow you to inspect the contents of logical decoding components. It
+  allows the inspection of serialized logical snapshots of a running
+  <productname>PostgreSQL</productname> database cluster, which is useful
+  for debugging or educational purposes.
+ </para>
+
+ <para>
+  By default, use of these functions is restricted to superusers and members of
+  the <literal>pg_read_server_files</literal> role. Access may be granted by
+  superusers to others using <command>GRANT</command>.
+ </para>
+
+ <sect2 id="pglogicalinspect-funcs">
+  <title>General Functions</title>
+
+  <variablelist>
+   <varlistentry id="pglogicalinspect-funcs-pg-get-logical-snapshot-meta">
+    <term>
+     <function>pg_get_logical_snapshot_meta(filename text) returns record</function>
+    </term>
+
+    <listitem>
+     <para>
+      Gets logical snapshot metadata about a snapshot file that is located in
+      the server's <filename>pg_logical/snapshots</filename> directory.
+      The <replaceable>filename</replaceable> argument represents the snapshot
+      file name.
+      For example:
+<screen>
+postgres=# SELECT * FROM pg_ls_logicalsnapdir();
+-[ RECORD 1 ]+-----------------------
+name         | 0-40796E18.snap
+size         | 152
+modification | 2024-08-14 16:36:32+00
+
+postgres=# SELECT * FROM pg_get_logical_snapshot_meta('0-40796E18.snap');
+-[ RECORD 1 ]--------
+magic    | 1369563137
+checksum | 1028045905
+version  | 6
+
+postgres=# SELECT ss.name, meta.* FROM pg_ls_logicalsnapdir() AS ss,
+pg_get_logical_snapshot_meta(ss.name) AS meta;
+-[ RECORD 1 ]-------------
+name     | 0-40796E18.snap
+magic    | 1369563137
+checksum | 1028045905
+version  | 6
+</screen>
+     </para>
+     <para>
+      If <replaceable>filename</replaceable> does not match a snapshot file, the
+      function raises an error.
+     </para>
+    </listitem>
+   </varlistentry>
+
+   <varlistentry id="pglogicalinspect-funcs-pg-get-logical-snapshot-info">
+    <term>
+     <function>pg_get_logical_snapshot_info(filename text) returns record</function>
+    </term>
+
+    <listitem>
+     <para>
+      Gets logical snapshot information about a snapshot file that is located in
+      the server's <filename>pg_logical/snapshots</filename> directory.
+      The <replaceable>filename</replaceable> argument represents the snapshot
+      file name.
+      For example:
+<screen>
+postgres=# SELECT * FROM pg_ls_logicalsnapdir();
+-[ RECORD 1 ]+-----------------------
+name         | 0-40796E18.snap
+size         | 152
+modification | 2024-08-14 16:36:32+00
+
+postgres=# SELECT * FROM pg_get_logical_snapshot_info('0-40796E18.snap');
+-[ RECORD 1 ]------------+-----------
+state                    | consistent
+xmin                     | 751
+xmax                     | 751
+start_decoding_at        | 0/40796AF8
+two_phase_at             | 0/40796AF8
+initial_xmin_horizon     | 0
+building_full_snapshot   | f
+in_slot_creation         | f
+last_serialized_snapshot | 0/0
+next_phase_at            | 0
+committed_count          | 0
+committed_xip            |
+catchange_count          | 2
+catchange_xip            | {751,752}
+
+postgres=# SELECT ss.name, info.* FROM pg_ls_logicalsnapdir() AS ss,
+pg_get_logical_snapshot_info(ss.name) AS info;
+-[ RECORD 1 ]------------+----------------
+name                     | 0-40796E18.snap
+state                    | consistent
+xmin                     | 751
+xmax                     | 751
+start_decoding_at        | 0/40796AF8
+two_phase_at             | 0/40796AF8
+initial_xmin_horizon     | 0
+building_full_snapshot   | f
+in_slot_creation         | f
+last_serialized_snapshot | 0/0
+next_phase_at            | 0
+committed_count          | 0
+committed_xip            |
+catchange_count          | 2
+catchange_xip            | {751,752}
+</screen>
+     </para>
+     <para>
+      If <replaceable>filename</replaceable> does not match a snapshot file, the
+      function raises an error.
+     </para>
+    </listitem>
+   </varlistentry>
+
+  </variablelist>
+ </sect2>
+
+ <sect2 id="pglogicalinspect-author">
+  <title>Author</title>
+
+  <para>
+   Bertrand Drouvot <email>bertranddrouvot.pg@gmail.com</email>
+  </para>
+ </sect2>
+
+</sect1>
diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c
index 0450f94ba8..92fd57b77e 100644
--- a/src/backend/replication/logical/snapbuild.c
+++ b/src/backend/replication/logical/snapbuild.c
@@ -134,6 +134,7 @@
 #include "replication/logical.h"
 #include "replication/reorderbuffer.h"
 #include "replication/snapbuild.h"
+#include "replication/snapbuild_internal.h"
 #include "storage/fd.h"
 #include "storage/lmgr.h"
 #include "storage/proc.h"
@@ -143,146 +144,6 @@
 #include "utils/memutils.h"
 #include "utils/snapmgr.h"
 #include "utils/snapshot.h"
-
-/*
- * This struct contains the current state of the snapshot building
- * machinery. Besides a forward declaration in the header, it is not exposed
- * to the public, so we can easily change its contents.
- */
-struct SnapBuild
-{
-	/* how far are we along building our first full snapshot */
-	SnapBuildState state;
-
-	/* private memory context used to allocate memory for this module. */
-	MemoryContext context;
-
-	/* all transactions < than this have committed/aborted */
-	TransactionId xmin;
-
-	/* all transactions >= than this are uncommitted */
-	TransactionId xmax;
-
-	/*
-	 * Don't replay commits from an LSN < this LSN. This can be set externally
-	 * but it will also be advanced (never retreat) from within snapbuild.c.
-	 */
-	XLogRecPtr	start_decoding_at;
-
-	/*
-	 * LSN at which two-phase decoding was enabled or LSN at which we found a
-	 * consistent point at the time of slot creation.
-	 *
-	 * The prepared transactions, that were skipped because previously
-	 * two-phase was not enabled or are not covered by initial snapshot, need
-	 * to be sent later along with commit prepared and they must be before
-	 * this point.
-	 */
-	XLogRecPtr	two_phase_at;
-
-	/*
-	 * Don't start decoding WAL until the "xl_running_xacts" information
-	 * indicates there are no running xids with an xid smaller than this.
-	 */
-	TransactionId initial_xmin_horizon;
-
-	/* Indicates if we are building full snapshot or just catalog one. */
-	bool		building_full_snapshot;
-
-	/*
-	 * Indicates if we are using the snapshot builder for the creation of a
-	 * logical replication slot. If it's true, the start point for decoding
-	 * changes is not determined yet. So we skip snapshot restores to properly
-	 * find the start point. See SnapBuildFindSnapshot() for details.
-	 */
-	bool		in_slot_creation;
-
-	/*
-	 * Snapshot that's valid to see the catalog state seen at this moment.
-	 */
-	Snapshot	snapshot;
-
-	/*
-	 * LSN of the last location we are sure a snapshot has been serialized to.
-	 */
-	XLogRecPtr	last_serialized_snapshot;
-
-	/*
-	 * The reorderbuffer we need to update with usable snapshots et al.
-	 */
-	ReorderBuffer *reorder;
-
-	/*
-	 * TransactionId at which the next phase of initial snapshot building will
-	 * happen. InvalidTransactionId if not known (i.e. SNAPBUILD_START), or
-	 * when no next phase necessary (SNAPBUILD_CONSISTENT).
-	 */
-	TransactionId next_phase_at;
-
-	/*
-	 * Array of transactions which could have catalog changes that committed
-	 * between xmin and xmax.
-	 */
-	struct
-	{
-		/* number of committed transactions */
-		size_t		xcnt;
-
-		/* available space for committed transactions */
-		size_t		xcnt_space;
-
-		/*
-		 * Until we reach a CONSISTENT state, we record commits of all
-		 * transactions, not just the catalog changing ones. Record when that
-		 * changes so we know we cannot export a snapshot safely anymore.
-		 */
-		bool		includes_all_transactions;
-
-		/*
-		 * Array of committed transactions that have modified the catalog.
-		 *
-		 * As this array is frequently modified we do *not* keep it in
-		 * xidComparator order. Instead we sort the array when building &
-		 * distributing a snapshot.
-		 *
-		 * TODO: It's unclear whether that reasoning has much merit. Every
-		 * time we add something here after becoming consistent will also
-		 * require distributing a snapshot. Storing them sorted would
-		 * potentially also make it easier to purge (but more complicated wrt
-		 * wraparound?). Should be improved if sorting while building the
-		 * snapshot shows up in profiles.
-		 */
-		TransactionId *xip;
-	}			committed;
-
-	/*
-	 * Array of transactions and subtransactions that had modified catalogs
-	 * and were running when the snapshot was serialized.
-	 *
-	 * We normally rely on some WAL record types such as HEAP2_NEW_CID to know
-	 * if the transaction has changed the catalog. But it could happen that
-	 * the logical decoding decodes only the commit record of the transaction
-	 * after restoring the previously serialized snapshot in which case we
-	 * will miss adding the xid to the snapshot and end up looking at the
-	 * catalogs with the wrong snapshot.
-	 *
-	 * Now to avoid the above problem, we serialize the transactions that had
-	 * modified the catalogs and are still running at the time of snapshot
-	 * serialization. We fill this array while restoring the snapshot and then
-	 * refer it while decoding commit to ensure if the xact has modified the
-	 * catalog. We discard this array when all the xids in the list become old
-	 * enough to matter. See SnapBuildPurgeOlderTxn for details.
-	 */
-	struct
-	{
-		/* number of transactions */
-		size_t		xcnt;
-
-		/* This array must be sorted in xidComparator order */
-		TransactionId *xip;
-	}			catchange;
-};
-
 /*
  * Starting a transaction -- which we need to do while exporting a snapshot --
  * removes knowledge about the previously used resowner, so we save it here.
@@ -1557,40 +1418,6 @@ SnapBuildWaitSnapshot(xl_running_xacts *running, TransactionId cutoff)
 	}
 }
 
-/* -----------------------------------
- * Snapshot serialization support
- * -----------------------------------
- */
-
-/*
- * We store current state of struct SnapBuild on disk in the following manner:
- *
- * struct SnapBuildOnDisk;
- * TransactionId * committed.xcnt; (*not xcnt_space*)
- * TransactionId * catchange.xcnt;
- *
- */
-typedef struct SnapBuildOnDisk
-{
-	/* first part of this struct needs to be version independent */
-
-	/* data not covered by checksum */
-	uint32		magic;
-	pg_crc32c	checksum;
-
-	/* data covered by checksum */
-
-	/* version, in case we want to support pg_upgrade */
-	uint32		version;
-	/* how large is the on disk data, excluding the constant sized part */
-	uint32		length;
-
-	/* version dependent part */
-	SnapBuild	builder;
-
-	/* variable amount of TransactionIds follows */
-} SnapBuildOnDisk;
-
 #define SnapBuildOnDiskConstantSize \
 	offsetof(SnapBuildOnDisk, builder)
 #define SnapBuildOnDiskNotChecksummedSize \
@@ -1857,34 +1684,31 @@ out:
 }
 
 /*
- * Restore a snapshot into 'builder' if previously one has been stored at the
- * location indicated by 'lsn'. Returns true if successful, false otherwise.
+ * Restore the logical snapshot file contents to 'ondisk'.
+ *
+ * If 'missing_ok' is true, will not throw an error if the file is not found.
+ * 'context' is the memory context where the catalog modifying/committed xid
+ * will live.
  */
-static bool
-SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
+bool
+SnapBuildRestoreSnapshot(SnapBuildOnDisk *ondisk, const char *path,
+						 MemoryContext context, bool missing_ok)
 {
-	SnapBuildOnDisk ondisk;
 	int			fd;
-	char		path[MAXPGPATH];
-	Size		sz;
 	pg_crc32c	checksum;
-
-	/* no point in loading a snapshot if we're already there */
-	if (builder->state == SNAPBUILD_CONSISTENT)
-		return false;
-
-	sprintf(path, "%s/%X-%X.snap",
-			PG_LOGICAL_SNAPSHOTS_DIR,
-			LSN_FORMAT_ARGS(lsn));
+	Size		sz;
 
 	fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
 
-	if (fd < 0 && errno == ENOENT)
-		return false;
-	else if (fd < 0)
+	if (fd < 0)
+	{
+		if (missing_ok && errno == ENOENT)
+			return false;
+
 		ereport(ERROR,
 				(errcode_for_file_access(),
 				 errmsg("could not open file \"%s\": %m", path)));
+	}
 
 	/* ----
 	 * Make sure the snapshot had been stored safely to disk, that's normally
@@ -1897,47 +1721,46 @@ SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
 	fsync_fname(path, false);
 	fsync_fname(PG_LOGICAL_SNAPSHOTS_DIR, true);
 
-
 	/* read statically sized portion of snapshot */
-	SnapBuildRestoreContents(fd, (char *) &ondisk, SnapBuildOnDiskConstantSize, path);
+	SnapBuildRestoreContents(fd, (char *) ondisk, SnapBuildOnDiskConstantSize, path);
 
-	if (ondisk.magic != SNAPBUILD_MAGIC)
+	if (ondisk->magic != SNAPBUILD_MAGIC)
 		ereport(ERROR,
 				(errcode(ERRCODE_DATA_CORRUPTED),
 				 errmsg("snapbuild state file \"%s\" has wrong magic number: %u instead of %u",
-						path, ondisk.magic, SNAPBUILD_MAGIC)));
+						path, ondisk->magic, SNAPBUILD_MAGIC)));
 
-	if (ondisk.version != SNAPBUILD_VERSION)
+	if (ondisk->version != SNAPBUILD_VERSION)
 		ereport(ERROR,
 				(errcode(ERRCODE_DATA_CORRUPTED),
 				 errmsg("snapbuild state file \"%s\" has unsupported version: %u instead of %u",
-						path, ondisk.version, SNAPBUILD_VERSION)));
+						path, ondisk->version, SNAPBUILD_VERSION)));
 
 	INIT_CRC32C(checksum);
 	COMP_CRC32C(checksum,
-				((char *) &ondisk) + SnapBuildOnDiskNotChecksummedSize,
+				((char *) ondisk) + SnapBuildOnDiskNotChecksummedSize,
 				SnapBuildOnDiskConstantSize - SnapBuildOnDiskNotChecksummedSize);
 
 	/* read SnapBuild */
-	SnapBuildRestoreContents(fd, (char *) &ondisk.builder, sizeof(SnapBuild), path);
-	COMP_CRC32C(checksum, &ondisk.builder, sizeof(SnapBuild));
+	SnapBuildRestoreContents(fd, (char *) &ondisk->builder, sizeof(SnapBuild), path);
+	COMP_CRC32C(checksum, &ondisk->builder, sizeof(SnapBuild));
 
 	/* restore committed xacts information */
-	if (ondisk.builder.committed.xcnt > 0)
+	if (ondisk->builder.committed.xcnt > 0)
 	{
-		sz = sizeof(TransactionId) * ondisk.builder.committed.xcnt;
-		ondisk.builder.committed.xip = MemoryContextAllocZero(builder->context, sz);
-		SnapBuildRestoreContents(fd, (char *) ondisk.builder.committed.xip, sz, path);
-		COMP_CRC32C(checksum, ondisk.builder.committed.xip, sz);
+		sz = sizeof(TransactionId) * ondisk->builder.committed.xcnt;
+		ondisk->builder.committed.xip = MemoryContextAllocZero(context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.committed.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.committed.xip, sz);
 	}
 
 	/* restore catalog modifying xacts information */
-	if (ondisk.builder.catchange.xcnt > 0)
+	if (ondisk->builder.catchange.xcnt > 0)
 	{
-		sz = sizeof(TransactionId) * ondisk.builder.catchange.xcnt;
-		ondisk.builder.catchange.xip = MemoryContextAllocZero(builder->context, sz);
-		SnapBuildRestoreContents(fd, (char *) ondisk.builder.catchange.xip, sz, path);
-		COMP_CRC32C(checksum, ondisk.builder.catchange.xip, sz);
+		sz = sizeof(TransactionId) * ondisk->builder.catchange.xcnt;
+		ondisk->builder.catchange.xip = MemoryContextAllocZero(context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.catchange.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.catchange.xip, sz);
 	}
 
 	if (CloseTransientFile(fd) != 0)
@@ -1948,11 +1771,36 @@ SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
 	FIN_CRC32C(checksum);
 
 	/* verify checksum of what we've read */
-	if (!EQ_CRC32C(checksum, ondisk.checksum))
+	if (!EQ_CRC32C(checksum, ondisk->checksum))
 		ereport(ERROR,
 				(errcode(ERRCODE_DATA_CORRUPTED),
 				 errmsg("checksum mismatch for snapbuild state file \"%s\": is %u, should be %u",
-						path, checksum, ondisk.checksum)));
+						path, checksum, ondisk->checksum)));
+
+	return true;
+}
+
+/*
+ * Restore a snapshot into 'builder' if previously one has been stored at the
+ * location indicated by 'lsn'. Returns true if successful, false otherwise.
+ */
+static bool
+SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
+{
+	SnapBuildOnDisk ondisk;
+	char		path[MAXPGPATH];
+
+	/* no point in loading a snapshot if we're already there */
+	if (builder->state == SNAPBUILD_CONSISTENT)
+		return false;
+
+	sprintf(path, "%s/%X-%X.snap",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			LSN_FORMAT_ARGS(lsn));
+
+	/* validate and restore the snapshot to 'ondisk' */
+	if (!SnapBuildRestoreSnapshot(&ondisk, path, builder->context, true))
+		return false;
 
 	/*
 	 * ok, we now have a sensible snapshot here, figure out if it has more
diff --git a/src/include/replication/snapbuild.h b/src/include/replication/snapbuild.h
index caa5113ff8..3c1454df99 100644
--- a/src/include/replication/snapbuild.h
+++ b/src/include/replication/snapbuild.h
@@ -15,6 +15,10 @@
 #include "access/xlogdefs.h"
 #include "utils/snapmgr.h"
 
+/*
+ * Please keep get_snapbuild_state_desc() (located in the pg_logicalinspect
+ * module) updated if a change needs to be made to SnapBuildState.
+ */
 typedef enum
 {
 	/*
@@ -46,7 +50,7 @@ typedef enum
 	SNAPBUILD_CONSISTENT = 2,
 } SnapBuildState;
 
-/* forward declare so we don't have to expose the struct to the public */
+/* forward declare so we don't have to include snapbuild_internal.h */
 struct SnapBuild;
 typedef struct SnapBuild SnapBuild;
 
diff --git a/src/include/replication/snapbuild_internal.h b/src/include/replication/snapbuild_internal.h
new file mode 100644
index 0000000000..7134b48b96
--- /dev/null
+++ b/src/include/replication/snapbuild_internal.h
@@ -0,0 +1,199 @@
+/*-------------------------------------------------------------------------
+ *
+ * snapbuild_internal.h
+ *    This file contains declarations for logical decoding utility
+ *    functions for internal use.
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * src/include/replication/snapbuild_internal.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef SNAPBUILD_INTERNAL_H
+#define SNAPBUILD_INTERNAL_H
+
+#include "port/pg_crc32c.h"
+#include "replication/reorderbuffer.h"
+#include "replication/snapbuild.h"
+
+/*
+ * This struct contains the current state of the snapshot building
+ * machinery. It is exposed to the public, so pay attention when changing its
+ * contents.
+ */
+typedef struct SnapBuild
+{
+	/* how far are we along building our first full snapshot */
+	SnapBuildState state;
+
+	/* private memory context used to allocate memory for this module. */
+	MemoryContext context;
+
+	/* all transactions < than this have committed/aborted */
+	TransactionId xmin;
+
+	/* all transactions >= than this are uncommitted */
+	TransactionId xmax;
+
+	/*
+	 * Don't replay commits from an LSN < this LSN. This can be set externally
+	 * but it will also be advanced (never retreat) from within snapbuild.c.
+	 */
+	XLogRecPtr	start_decoding_at;
+
+	/*
+	 * LSN at which two-phase decoding was enabled or LSN at which we found a
+	 * consistent point at the time of slot creation.
+	 *
+	 * The prepared transactions, that were skipped because previously
+	 * two-phase was not enabled or are not covered by initial snapshot, need
+	 * to be sent later along with commit prepared and they must be before
+	 * this point.
+	 */
+	XLogRecPtr	two_phase_at;
+
+	/*
+	 * Don't start decoding WAL until the "xl_running_xacts" information
+	 * indicates there are no running xids with an xid smaller than this.
+	 */
+	TransactionId initial_xmin_horizon;
+
+	/* Indicates if we are building full snapshot or just catalog one. */
+	bool		building_full_snapshot;
+
+	/*
+	 * Indicates if we are using the snapshot builder for the creation of a
+	 * logical replication slot. If it's true, the start point for decoding
+	 * changes is not determined yet. So we skip snapshot restores to properly
+	 * find the start point. See SnapBuildFindSnapshot() for details.
+	 */
+	bool		in_slot_creation;
+
+	/*
+	 * Snapshot that's valid to see the catalog state seen at this moment.
+	 */
+	Snapshot	snapshot;
+
+	/*
+	 * LSN of the last location we are sure a snapshot has been serialized to.
+	 */
+	XLogRecPtr	last_serialized_snapshot;
+
+	/*
+	 * The reorderbuffer we need to update with usable snapshots et al.
+	 */
+	ReorderBuffer *reorder;
+
+	/*
+	 * TransactionId at which the next phase of initial snapshot building will
+	 * happen. InvalidTransactionId if not known (i.e. SNAPBUILD_START), or
+	 * when no next phase necessary (SNAPBUILD_CONSISTENT).
+	 */
+	TransactionId next_phase_at;
+
+	/*
+	 * Array of transactions which could have catalog changes that committed
+	 * between xmin and xmax.
+	 */
+	struct
+	{
+		/* number of committed transactions */
+		size_t		xcnt;
+
+		/* available space for committed transactions */
+		size_t		xcnt_space;
+
+		/*
+		 * Until we reach a CONSISTENT state, we record commits of all
+		 * transactions, not just the catalog changing ones. Record when that
+		 * changes so we know we cannot export a snapshot safely anymore.
+		 */
+		bool		includes_all_transactions;
+
+		/*
+		 * Array of committed transactions that have modified the catalog.
+		 *
+		 * As this array is frequently modified we do *not* keep it in
+		 * xidComparator order. Instead we sort the array when building &
+		 * distributing a snapshot.
+		 *
+		 * TODO: It's unclear whether that reasoning has much merit. Every
+		 * time we add something here after becoming consistent will also
+		 * require distributing a snapshot. Storing them sorted would
+		 * potentially also make it easier to purge (but more complicated wrt
+		 * wraparound?). Should be improved if sorting while building the
+		 * snapshot shows up in profiles.
+		 */
+		TransactionId *xip;
+	}			committed;
+
+	/*
+	 * Array of transactions and subtransactions that had modified catalogs
+	 * and were running when the snapshot was serialized.
+	 *
+	 * We normally rely on some WAL record types such as HEAP2_NEW_CID to know
+	 * if the transaction has changed the catalog. But it could happen that
+	 * the logical decoding decodes only the commit record of the transaction
+	 * after restoring the previously serialized snapshot in which case we
+	 * will miss adding the xid to the snapshot and end up looking at the
+	 * catalogs with the wrong snapshot.
+	 *
+	 * Now to avoid the above problem, we serialize the transactions that had
+	 * modified the catalogs and are still running at the time of snapshot
+	 * serialization. We fill this array while restoring the snapshot and then
+	 * refer it while decoding commit to ensure if the xact has modified the
+	 * catalog. We discard this array when all the xids in the list become old
+	 * enough to matter. See SnapBuildPurgeOlderTxn for details.
+	 */
+	struct
+	{
+		/* number of transactions */
+		size_t		xcnt;
+
+		/* This array must be sorted in xidComparator order */
+		TransactionId *xip;
+	}			catchange;
+} SnapBuild;
+
+/* -----------------------------------
+ * Snapshot serialization support
+ * -----------------------------------
+ */
+
+/*
+ * We store current state of struct SnapBuild on disk in the following manner:
+ *
+ * struct SnapBuildOnDisk;
+ * TransactionId * committed.xcnt; (*not xcnt_space*)
+ * TransactionId * catchange.xcnt;
+ *
+ * Check if the SnapBuildOnDiskConstantSize and SnapBuildOnDiskNotChecksummedSize
+ * macros need to be updated when modifying the SnapBuildOnDisk struct.
+ */
+typedef struct SnapBuildOnDisk
+{
+	/* first part of this struct needs to be version independent */
+
+	/* data not covered by checksum */
+	uint32		magic;
+	pg_crc32c	checksum;
+
+	/* data covered by checksum */
+
+	/* version, in case we want to support pg_upgrade */
+	uint32		version;
+	/* how large is the on disk data, excluding the constant sized part */
+	uint32		length;
+
+	/* version dependent part */
+	SnapBuild	builder;
+
+	/* variable amount of TransactionIds follows */
+} SnapBuildOnDisk;
+
+extern bool SnapBuildRestoreSnapshot(SnapBuildOnDisk *ondisk, const char *path,
+									 MemoryContext context, bool missing_ok);
+
+#endif							/* SNAPBUILD_INTERNAL_H */
-- 
2.34.1

#57Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: Peter Smith (#55)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Fri, Oct 11, 2024 at 12:59:33PM +1100, Peter Smith wrote:

Hi, Here are a few comments for patch set v13*

Thanks for looking at it.

//////////

Patch v13-0001

======
Commit message

1.1
/were no use case/was no use case/

Updated in v14 just shared up-thread.

~~~

1.2
It seemed a bit odd that the switch cases for
'construct_array_builtin' are not the same as those for
'deconstruct_array_builtin'.

For example, all these ones seem missing from deconstruct:
case INT4OID:
case INT8OID:
case NAMEOID:
case REGTYPEOID:

I know that has nothing to do with your patch, and I guess it does not
cause any problems otherwise there would be ERRORs. But, if you are to
follow this same current pattern, then perhaps you don't need to add
your new case for 'deconstruct_array_builtin', since AFAICT you are
never using it.

That's right. Strict pairing between deconstruct_array_builtin() and
construct_array_builtin() is not required, let's remove this extra switch in
deconstruct_array_builtin() for code consistency (done in v14).

//////////

Patch v13-0002

======
pg_get_logical_snapshot_meta:

2.1
+ Datum values[PG_GET_LOGICAL_SNAPSHOT_META_COLS];
+ bool nulls[PG_GET_LOGICAL_SNAPSHOT_META_COLS];

FWIW, if you wanted to avoid a few lines you could initialise the
nulls array during the declaration.
bool nulls[PG_GET_LOGICAL_SNAPSHOT_META_COLS] = {0};

This seems a common pattern in other source code, and it replaces the
need for the subsequent memset.

Okay, fine by me, let's do it for the "values" too in passing (this seems also
a common pattern), in v14.

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

#58Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Bertrand Drouvot (#56)
Re: Add contrib/pg_logicalsnapinspect

On Fri, Oct 11, 2024 at 6:15 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Hi,

On Thu, Oct 10, 2024 at 05:38:43PM -0700, Masahiko Sawada wrote:

On Thu, Oct 10, 2024 at 6:10 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

The patches mostly look good to me. Here are some minor comments:

Thanks for looking at it!

+       sprintf(path, "%s/%s",
+                       PG_LOGICAL_SNAPSHOTS_DIR,
+                       text_to_cstring(filename_t));
+
+       /* Validate and restore the snapshot to 'ondisk' */
+       ValidateAndRestoreSnapshotFile(&ondisk, path,
CurrentMemoryContext, false);
+
+       /* Build a tuple descriptor for our result type */
+       if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+               elog(ERROR, "return type must be a row type");
+
I think it would be better to check the result type before reading the
snapshot file.

Agree, done in v14.

---
+       values[i++] = Int64GetDatum((int64) ondisk.checksum);

Why is only checksum casted to int64? With that, it can show a
checksum value as a non-netagive integer but is it really necessary?
For instance, page_header() function in pageinspect shows a page
checksum as smallint.

Yeah, pd_checksum in PageHeaderData is uint16 while checksum in SnapBuildOnDisk
is pg_crc32c. The reason why it is casted to int64 is explained in [1], does that
make sense to you?

In the email, you said:

As the checksum could be > 2^31 - 1, then v9 (just shared up-thread) changes it
to an int8 in the pg_logicalinspect--1.0.sql file. So, to avoid CI failure on
the 32bit build, then v9 is using Int64GetDatum() instead of UInt32GetDatum().

I'm fine with using Int64GetDatum() for checksum.

Same goes for below:
values[i++] = Int32GetDatum(ondisk.magic);
values[i++] = Int32GetDatum(ondisk.magic);

The 2 others field (magic and version) are unlikely to be > 2^31 - 1, so v9 is
making use of UInt32GetDatum() and keep int4 in the sql file.

While I agree that these two fields are unlikely to be > 2^31 - 1, I'm
concerned a bit about an inconsistency that the patch uses
Int64GetDatum also for both ondisk.builder.committed.xcnt and
ondisk.builder.catchange.xcnt.

I have a minor comment:

+ <sect2 id="pglogicalinspect-funcs">
+ <title>General Functions</title>

If we use "General Functions" here it sounds like there are other
functions for specific purposes in pg_logicalinspect module. How about
using "Functions" instead?

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com

#59Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Masahiko Sawada (#58)
3 attachment(s)
Re: Add contrib/pg_logicalsnapinspect

On Fri, Oct 11, 2024 at 11:15 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:

On Fri, Oct 11, 2024 at 6:15 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Hi,

On Thu, Oct 10, 2024 at 05:38:43PM -0700, Masahiko Sawada wrote:

On Thu, Oct 10, 2024 at 6:10 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

The patches mostly look good to me. Here are some minor comments:

Thanks for looking at it!

+       sprintf(path, "%s/%s",
+                       PG_LOGICAL_SNAPSHOTS_DIR,
+                       text_to_cstring(filename_t));
+
+       /* Validate and restore the snapshot to 'ondisk' */
+       ValidateAndRestoreSnapshotFile(&ondisk, path,
CurrentMemoryContext, false);
+
+       /* Build a tuple descriptor for our result type */
+       if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+               elog(ERROR, "return type must be a row type");
+
I think it would be better to check the result type before reading the
snapshot file.

Agree, done in v14.

---
+       values[i++] = Int64GetDatum((int64) ondisk.checksum);

Why is only checksum casted to int64? With that, it can show a
checksum value as a non-netagive integer but is it really necessary?
For instance, page_header() function in pageinspect shows a page
checksum as smallint.

Yeah, pd_checksum in PageHeaderData is uint16 while checksum in SnapBuildOnDisk
is pg_crc32c. The reason why it is casted to int64 is explained in [1], does that
make sense to you?

In the email, you said:

As the checksum could be > 2^31 - 1, then v9 (just shared up-thread) changes it
to an int8 in the pg_logicalinspect--1.0.sql file. So, to avoid CI failure on
the 32bit build, then v9 is using Int64GetDatum() instead of UInt32GetDatum().

I'm fine with using Int64GetDatum() for checksum.

Same goes for below:
values[i++] = Int32GetDatum(ondisk.magic);
values[i++] = Int32GetDatum(ondisk.magic);

The 2 others field (magic and version) are unlikely to be > 2^31 - 1, so v9 is
making use of UInt32GetDatum() and keep int4 in the sql file.

While I agree that these two fields are unlikely to be > 2^31 - 1, I'm
concerned a bit about an inconsistency that the patch uses
Int64GetDatum also for both ondisk.builder.committed.xcnt and
ondisk.builder.catchange.xcnt.

I have a minor comment:

+ <sect2 id="pglogicalinspect-funcs">
+ <title>General Functions</title>

If we use "General Functions" here it sounds like there are other
functions for specific purposes in pg_logicalinspect module. How about
using "Functions" instead?

To elaborate further, pageinspect has a "General Functions" section,
which makes sense to me as it has other AM-type specific functions. On
the other hand, pg_logicalinspect has SQL functions only for one
logical replication component. So I think it makes sense to use
"Function" instead. pg_walinspect also has the sole section "General
Function" but I personally think that "Function" is more appropriate
like other modules does.

BTW I think that adding snapshot_internal.h could be a separate patch.
That makes the main pg_logicalinspect patch cleaner.

I've attached updated patches with some minor changes, and done the patch split.

The 0001 patch just moves both SnapBuild and SnapBuildOnDisk structs
to snapshot_internal.h. The 0002 is the main pg_logicalinspect patch.
I've merged the previous Add-XIDOID-in-construct_array_builtin.patch
to the main patch. The minor_change.patch.txt is the difference I
added on top of v14 patch set.

I'm going to push them early next week, barring any objections and
further comments.

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com

Attachments:

minor_change.patch.txttext/plain; charset=US-ASCII; name=minor_change.patch.txtDownload
commit 483f5a402b6f650d02e8f8668bd50b17680f2805
Author: Masahiko Sawada <sawada.mshk@gmail.com>
Date:   Fri Oct 11 15:52:49 2024 -0700

    minor changes

diff --git a/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql b/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
index c773f6e458..8f7f947cbb 100644
--- a/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
+++ b/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
@@ -31,9 +31,9 @@ CREATE FUNCTION pg_get_logical_snapshot_info(IN filename text,
     OUT in_slot_creation boolean,
     OUT last_serialized_snapshot pg_lsn,
     OUT next_phase_at xid,
-    OUT committed_count int8,
+    OUT committed_count int4,
     OUT committed_xip xid[],
-    OUT catchange_count int8,
+    OUT catchange_count int4,
     OUT catchange_xip xid[]
 )
 AS 'MODULE_PATHNAME', 'pg_get_logical_snapshot_info'
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.c b/contrib/pg_logicalinspect/pg_logicalinspect.c
index fabfe2a10c..790c64d6fa 100644
--- a/contrib/pg_logicalinspect/pg_logicalinspect.c
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.c
@@ -123,7 +123,7 @@ pg_get_logical_snapshot_info(PG_FUNCTION_ARGS)
 	values[i++] = LSNGetDatum(ondisk.builder.last_serialized_snapshot);
 	values[i++] = TransactionIdGetDatum(ondisk.builder.next_phase_at);
 
-	values[i++] = Int64GetDatum(ondisk.builder.committed.xcnt);
+	values[i++] = UInt32GetDatum(ondisk.builder.committed.xcnt);
 	if (ondisk.builder.committed.xcnt > 0)
 	{
 		Datum	   *arrayelems;
@@ -140,7 +140,7 @@ pg_get_logical_snapshot_info(PG_FUNCTION_ARGS)
 	else
 		nulls[i++] = true;
 
-	values[i++] = Int64GetDatum(ondisk.builder.catchange.xcnt);
+	values[i++] = UInt32GetDatum(ondisk.builder.catchange.xcnt);
 	if (ondisk.builder.catchange.xcnt > 0)
 	{
 		Datum	   *arrayelems;
diff --git a/doc/src/sgml/pglogicalinspect.sgml b/doc/src/sgml/pglogicalinspect.sgml
index e0fac997b6..4b111f9611 100644
--- a/doc/src/sgml/pglogicalinspect.sgml
+++ b/doc/src/sgml/pglogicalinspect.sgml
@@ -22,7 +22,7 @@
  </para>
 
  <sect2 id="pglogicalinspect-funcs">
-  <title>General Functions</title>
+  <title>Functions</title>
 
   <variablelist>
    <varlistentry id="pglogicalinspect-funcs-pg-get-logical-snapshot-meta">
v15-0001-Move-SnapBuild-and-SnapBuildOnDisk-structs-to-sn.patchapplication/octet-stream; name=v15-0001-Move-SnapBuild-and-SnapBuildOnDisk-structs-to-sn.patchDownload
From 0c58f9984a3848e8b51f1438de32cfaa7e0db438 Mon Sep 17 00:00:00 2001
From: Masahiko Sawada <sawada.mshk@gmail.com>
Date: Fri, 11 Oct 2024 16:23:14 -0700
Subject: [PATCH v15 1/2] Move SnapBuild and SnapBuildOnDisk structs to
 snapshot_internal.h

This commit moves the definitions of the SnapBuild and SnapBuildOnDisk
structs, related to logical snapshots, to the snapshot_internal.h
file. This change allows external tools, such as
pg_logicalinspect (with an upcoming patch), to access and utilize the
contents of logical snapshots.

Author: Bertrand Drouvot
Reviewed-by: Amit Kapila, Shveta Malik, Peter Smith
Discussion: https://postgr.es/m/ZscuZ92uGh3wm4tW%40ip-10-97-1-34.eu-west-3.compute.internal
---
 src/backend/replication/logical/snapbuild.c  | 175 +----------------
 src/include/replication/snapbuild.h          |   2 +-
 src/include/replication/snapbuild_internal.h | 196 +++++++++++++++++++
 3 files changed, 198 insertions(+), 175 deletions(-)
 create mode 100644 src/include/replication/snapbuild_internal.h

diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c
index 0450f94ba8..b9df8c0a02 100644
--- a/src/backend/replication/logical/snapbuild.c
+++ b/src/backend/replication/logical/snapbuild.c
@@ -134,6 +134,7 @@
 #include "replication/logical.h"
 #include "replication/reorderbuffer.h"
 #include "replication/snapbuild.h"
+#include "replication/snapbuild_internal.h"
 #include "storage/fd.h"
 #include "storage/lmgr.h"
 #include "storage/proc.h"
@@ -143,146 +144,6 @@
 #include "utils/memutils.h"
 #include "utils/snapmgr.h"
 #include "utils/snapshot.h"
-
-/*
- * This struct contains the current state of the snapshot building
- * machinery. Besides a forward declaration in the header, it is not exposed
- * to the public, so we can easily change its contents.
- */
-struct SnapBuild
-{
-	/* how far are we along building our first full snapshot */
-	SnapBuildState state;
-
-	/* private memory context used to allocate memory for this module. */
-	MemoryContext context;
-
-	/* all transactions < than this have committed/aborted */
-	TransactionId xmin;
-
-	/* all transactions >= than this are uncommitted */
-	TransactionId xmax;
-
-	/*
-	 * Don't replay commits from an LSN < this LSN. This can be set externally
-	 * but it will also be advanced (never retreat) from within snapbuild.c.
-	 */
-	XLogRecPtr	start_decoding_at;
-
-	/*
-	 * LSN at which two-phase decoding was enabled or LSN at which we found a
-	 * consistent point at the time of slot creation.
-	 *
-	 * The prepared transactions, that were skipped because previously
-	 * two-phase was not enabled or are not covered by initial snapshot, need
-	 * to be sent later along with commit prepared and they must be before
-	 * this point.
-	 */
-	XLogRecPtr	two_phase_at;
-
-	/*
-	 * Don't start decoding WAL until the "xl_running_xacts" information
-	 * indicates there are no running xids with an xid smaller than this.
-	 */
-	TransactionId initial_xmin_horizon;
-
-	/* Indicates if we are building full snapshot or just catalog one. */
-	bool		building_full_snapshot;
-
-	/*
-	 * Indicates if we are using the snapshot builder for the creation of a
-	 * logical replication slot. If it's true, the start point for decoding
-	 * changes is not determined yet. So we skip snapshot restores to properly
-	 * find the start point. See SnapBuildFindSnapshot() for details.
-	 */
-	bool		in_slot_creation;
-
-	/*
-	 * Snapshot that's valid to see the catalog state seen at this moment.
-	 */
-	Snapshot	snapshot;
-
-	/*
-	 * LSN of the last location we are sure a snapshot has been serialized to.
-	 */
-	XLogRecPtr	last_serialized_snapshot;
-
-	/*
-	 * The reorderbuffer we need to update with usable snapshots et al.
-	 */
-	ReorderBuffer *reorder;
-
-	/*
-	 * TransactionId at which the next phase of initial snapshot building will
-	 * happen. InvalidTransactionId if not known (i.e. SNAPBUILD_START), or
-	 * when no next phase necessary (SNAPBUILD_CONSISTENT).
-	 */
-	TransactionId next_phase_at;
-
-	/*
-	 * Array of transactions which could have catalog changes that committed
-	 * between xmin and xmax.
-	 */
-	struct
-	{
-		/* number of committed transactions */
-		size_t		xcnt;
-
-		/* available space for committed transactions */
-		size_t		xcnt_space;
-
-		/*
-		 * Until we reach a CONSISTENT state, we record commits of all
-		 * transactions, not just the catalog changing ones. Record when that
-		 * changes so we know we cannot export a snapshot safely anymore.
-		 */
-		bool		includes_all_transactions;
-
-		/*
-		 * Array of committed transactions that have modified the catalog.
-		 *
-		 * As this array is frequently modified we do *not* keep it in
-		 * xidComparator order. Instead we sort the array when building &
-		 * distributing a snapshot.
-		 *
-		 * TODO: It's unclear whether that reasoning has much merit. Every
-		 * time we add something here after becoming consistent will also
-		 * require distributing a snapshot. Storing them sorted would
-		 * potentially also make it easier to purge (but more complicated wrt
-		 * wraparound?). Should be improved if sorting while building the
-		 * snapshot shows up in profiles.
-		 */
-		TransactionId *xip;
-	}			committed;
-
-	/*
-	 * Array of transactions and subtransactions that had modified catalogs
-	 * and were running when the snapshot was serialized.
-	 *
-	 * We normally rely on some WAL record types such as HEAP2_NEW_CID to know
-	 * if the transaction has changed the catalog. But it could happen that
-	 * the logical decoding decodes only the commit record of the transaction
-	 * after restoring the previously serialized snapshot in which case we
-	 * will miss adding the xid to the snapshot and end up looking at the
-	 * catalogs with the wrong snapshot.
-	 *
-	 * Now to avoid the above problem, we serialize the transactions that had
-	 * modified the catalogs and are still running at the time of snapshot
-	 * serialization. We fill this array while restoring the snapshot and then
-	 * refer it while decoding commit to ensure if the xact has modified the
-	 * catalog. We discard this array when all the xids in the list become old
-	 * enough to matter. See SnapBuildPurgeOlderTxn for details.
-	 */
-	struct
-	{
-		/* number of transactions */
-		size_t		xcnt;
-
-		/* This array must be sorted in xidComparator order */
-		TransactionId *xip;
-	}			catchange;
-};
-
 /*
  * Starting a transaction -- which we need to do while exporting a snapshot --
  * removes knowledge about the previously used resowner, so we save it here.
@@ -1557,40 +1418,6 @@ SnapBuildWaitSnapshot(xl_running_xacts *running, TransactionId cutoff)
 	}
 }
 
-/* -----------------------------------
- * Snapshot serialization support
- * -----------------------------------
- */
-
-/*
- * We store current state of struct SnapBuild on disk in the following manner:
- *
- * struct SnapBuildOnDisk;
- * TransactionId * committed.xcnt; (*not xcnt_space*)
- * TransactionId * catchange.xcnt;
- *
- */
-typedef struct SnapBuildOnDisk
-{
-	/* first part of this struct needs to be version independent */
-
-	/* data not covered by checksum */
-	uint32		magic;
-	pg_crc32c	checksum;
-
-	/* data covered by checksum */
-
-	/* version, in case we want to support pg_upgrade */
-	uint32		version;
-	/* how large is the on disk data, excluding the constant sized part */
-	uint32		length;
-
-	/* version dependent part */
-	SnapBuild	builder;
-
-	/* variable amount of TransactionIds follows */
-} SnapBuildOnDisk;
-
 #define SnapBuildOnDiskConstantSize \
 	offsetof(SnapBuildOnDisk, builder)
 #define SnapBuildOnDiskNotChecksummedSize \
diff --git a/src/include/replication/snapbuild.h b/src/include/replication/snapbuild.h
index caa5113ff8..dbb4bc2f4b 100644
--- a/src/include/replication/snapbuild.h
+++ b/src/include/replication/snapbuild.h
@@ -46,7 +46,7 @@ typedef enum
 	SNAPBUILD_CONSISTENT = 2,
 } SnapBuildState;
 
-/* forward declare so we don't have to expose the struct to the public */
+/* forward declare so we don't have to include snapbuild_internal.h */
 struct SnapBuild;
 typedef struct SnapBuild SnapBuild;
 
diff --git a/src/include/replication/snapbuild_internal.h b/src/include/replication/snapbuild_internal.h
new file mode 100644
index 0000000000..03719ccf2a
--- /dev/null
+++ b/src/include/replication/snapbuild_internal.h
@@ -0,0 +1,196 @@
+/*-------------------------------------------------------------------------
+ *
+ * snapbuild_internal.h
+ *    This file contains declarations for logical decoding utility
+ *    functions for internal use.
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * src/include/replication/snapbuild_internal.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef SNAPBUILD_INTERNAL_H
+#define SNAPBUILD_INTERNAL_H
+
+#include "port/pg_crc32c.h"
+#include "replication/reorderbuffer.h"
+#include "replication/snapbuild.h"
+
+/*
+ * This struct contains the current state of the snapshot building
+ * machinery. It is exposed to the public, so pay attention when changing its
+ * contents.
+ */
+typedef struct SnapBuild
+{
+	/* how far are we along building our first full snapshot */
+	SnapBuildState state;
+
+	/* private memory context used to allocate memory for this module. */
+	MemoryContext context;
+
+	/* all transactions < than this have committed/aborted */
+	TransactionId xmin;
+
+	/* all transactions >= than this are uncommitted */
+	TransactionId xmax;
+
+	/*
+	 * Don't replay commits from an LSN < this LSN. This can be set externally
+	 * but it will also be advanced (never retreat) from within snapbuild.c.
+	 */
+	XLogRecPtr	start_decoding_at;
+
+	/*
+	 * LSN at which two-phase decoding was enabled or LSN at which we found a
+	 * consistent point at the time of slot creation.
+	 *
+	 * The prepared transactions, that were skipped because previously
+	 * two-phase was not enabled or are not covered by initial snapshot, need
+	 * to be sent later along with commit prepared and they must be before
+	 * this point.
+	 */
+	XLogRecPtr	two_phase_at;
+
+	/*
+	 * Don't start decoding WAL until the "xl_running_xacts" information
+	 * indicates there are no running xids with an xid smaller than this.
+	 */
+	TransactionId initial_xmin_horizon;
+
+	/* Indicates if we are building full snapshot or just catalog one. */
+	bool		building_full_snapshot;
+
+	/*
+	 * Indicates if we are using the snapshot builder for the creation of a
+	 * logical replication slot. If it's true, the start point for decoding
+	 * changes is not determined yet. So we skip snapshot restores to properly
+	 * find the start point. See SnapBuildFindSnapshot() for details.
+	 */
+	bool		in_slot_creation;
+
+	/*
+	 * Snapshot that's valid to see the catalog state seen at this moment.
+	 */
+	Snapshot	snapshot;
+
+	/*
+	 * LSN of the last location we are sure a snapshot has been serialized to.
+	 */
+	XLogRecPtr	last_serialized_snapshot;
+
+	/*
+	 * The reorderbuffer we need to update with usable snapshots et al.
+	 */
+	ReorderBuffer *reorder;
+
+	/*
+	 * TransactionId at which the next phase of initial snapshot building will
+	 * happen. InvalidTransactionId if not known (i.e. SNAPBUILD_START), or
+	 * when no next phase necessary (SNAPBUILD_CONSISTENT).
+	 */
+	TransactionId next_phase_at;
+
+	/*
+	 * Array of transactions which could have catalog changes that committed
+	 * between xmin and xmax.
+	 */
+	struct
+	{
+		/* number of committed transactions */
+		size_t		xcnt;
+
+		/* available space for committed transactions */
+		size_t		xcnt_space;
+
+		/*
+		 * Until we reach a CONSISTENT state, we record commits of all
+		 * transactions, not just the catalog changing ones. Record when that
+		 * changes so we know we cannot export a snapshot safely anymore.
+		 */
+		bool		includes_all_transactions;
+
+		/*
+		 * Array of committed transactions that have modified the catalog.
+		 *
+		 * As this array is frequently modified we do *not* keep it in
+		 * xidComparator order. Instead we sort the array when building &
+		 * distributing a snapshot.
+		 *
+		 * TODO: It's unclear whether that reasoning has much merit. Every
+		 * time we add something here after becoming consistent will also
+		 * require distributing a snapshot. Storing them sorted would
+		 * potentially also make it easier to purge (but more complicated wrt
+		 * wraparound?). Should be improved if sorting while building the
+		 * snapshot shows up in profiles.
+		 */
+		TransactionId *xip;
+	}			committed;
+
+	/*
+	 * Array of transactions and subtransactions that had modified catalogs
+	 * and were running when the snapshot was serialized.
+	 *
+	 * We normally rely on some WAL record types such as HEAP2_NEW_CID to know
+	 * if the transaction has changed the catalog. But it could happen that
+	 * the logical decoding decodes only the commit record of the transaction
+	 * after restoring the previously serialized snapshot in which case we
+	 * will miss adding the xid to the snapshot and end up looking at the
+	 * catalogs with the wrong snapshot.
+	 *
+	 * Now to avoid the above problem, we serialize the transactions that had
+	 * modified the catalogs and are still running at the time of snapshot
+	 * serialization. We fill this array while restoring the snapshot and then
+	 * refer it while decoding commit to ensure if the xact has modified the
+	 * catalog. We discard this array when all the xids in the list become old
+	 * enough to matter. See SnapBuildPurgeOlderTxn for details.
+	 */
+	struct
+	{
+		/* number of transactions */
+		size_t		xcnt;
+
+		/* This array must be sorted in xidComparator order */
+		TransactionId *xip;
+	}			catchange;
+} SnapBuild;
+
+/* -----------------------------------
+ * Snapshot serialization support
+ * -----------------------------------
+ */
+
+/*
+ * We store current state of struct SnapBuild on disk in the following manner:
+ *
+ * struct SnapBuildOnDisk;
+ * TransactionId * committed.xcnt; (*not xcnt_space*)
+ * TransactionId * catchange.xcnt;
+ *
+ * Check if the SnapBuildOnDiskConstantSize and SnapBuildOnDiskNotChecksummedSize
+ * macros need to be updated when modifying the SnapBuildOnDisk struct.
+ */
+typedef struct SnapBuildOnDisk
+{
+	/* first part of this struct needs to be version independent */
+
+	/* data not covered by checksum */
+	uint32		magic;
+	pg_crc32c	checksum;
+
+	/* data covered by checksum */
+
+	/* version, in case we want to support pg_upgrade */
+	uint32		version;
+	/* how large is the on disk data, excluding the constant sized part */
+	uint32		length;
+
+	/* version dependent part */
+	SnapBuild	builder;
+
+	/* variable amount of TransactionIds follows */
+} SnapBuildOnDisk;
+
+#endif							/* SNAPBUILD_INTERNAL_H */
-- 
2.39.3

v15-0002-Add-contrib-pg_logicalinspect.patchapplication/octet-stream; name=v15-0002-Add-contrib-pg_logicalinspect.patchDownload
From 5f6bc116ef46d1e32a0728db3e3b91ba80729201 Mon Sep 17 00:00:00 2001
From: Masahiko Sawada <sawada.mshk@gmail.com>
Date: Fri, 11 Oct 2024 16:24:14 -0700
Subject: [PATCH v15 2/2] Add contrib/pg_logicalinspect.

This module provides SQL functions that allow to inspect logical
decoding components.

It currently allows to inspect the contents of serialized logical
snapshots of a running database cluster, which is useful for debugging
or educational purposes.

Author: Bertrand Drouvot
Reviewed-by: Amit Kapila, Shveta Malik, Peter Smith, Peter Eisentraut
Reviewed-by: David G. Johnston
Discussion: https://postgr.es/m/ZscuZ92uGh3wm4tW%40ip-10-97-1-34.eu-west-3.compute.internal
---
 contrib/Makefile                              |   1 +
 contrib/meson.build                           |   1 +
 contrib/pg_logicalinspect/.gitignore          |   6 +
 contrib/pg_logicalinspect/Makefile            |  31 ++++
 .../expected/logical_inspect.out              |  52 ++++++
 contrib/pg_logicalinspect/logicalinspect.conf |   1 +
 contrib/pg_logicalinspect/meson.build         |  39 ++++
 .../pg_logicalinspect--1.0.sql                |  43 +++++
 contrib/pg_logicalinspect/pg_logicalinspect.c | 167 ++++++++++++++++++
 .../pg_logicalinspect.control                 |   5 +
 .../specs/logical_inspect.spec                |  34 ++++
 doc/src/sgml/contrib.sgml                     |   1 +
 doc/src/sgml/filelist.sgml                    |   1 +
 doc/src/sgml/pglogicalinspect.sgml            | 143 +++++++++++++++
 src/backend/replication/logical/snapbuild.c   |  99 +++++++----
 src/backend/utils/adt/arrayfuncs.c            |   6 +
 src/include/replication/snapbuild.h           |   4 +
 src/include/replication/snapbuild_internal.h  |   3 +
 18 files changed, 598 insertions(+), 39 deletions(-)
 create mode 100644 contrib/pg_logicalinspect/.gitignore
 create mode 100644 contrib/pg_logicalinspect/Makefile
 create mode 100644 contrib/pg_logicalinspect/expected/logical_inspect.out
 create mode 100644 contrib/pg_logicalinspect/logicalinspect.conf
 create mode 100644 contrib/pg_logicalinspect/meson.build
 create mode 100644 contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
 create mode 100644 contrib/pg_logicalinspect/pg_logicalinspect.c
 create mode 100644 contrib/pg_logicalinspect/pg_logicalinspect.control
 create mode 100644 contrib/pg_logicalinspect/specs/logical_inspect.spec
 create mode 100644 doc/src/sgml/pglogicalinspect.sgml

diff --git a/contrib/Makefile b/contrib/Makefile
index abd780f277..952855d9b6 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -32,6 +32,7 @@ SUBDIRS = \
 		passwordcheck	\
 		pg_buffercache	\
 		pg_freespacemap \
+		pg_logicalinspect \
 		pg_prewarm	\
 		pg_stat_statements \
 		pg_surgery	\
diff --git a/contrib/meson.build b/contrib/meson.build
index 14a8906865..159ff41555 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -46,6 +46,7 @@ subdir('passwordcheck')
 subdir('pg_buffercache')
 subdir('pgcrypto')
 subdir('pg_freespacemap')
+subdir('pg_logicalinspect')
 subdir('pg_prewarm')
 subdir('pgrowlocks')
 subdir('pg_stat_statements')
diff --git a/contrib/pg_logicalinspect/.gitignore b/contrib/pg_logicalinspect/.gitignore
new file mode 100644
index 0000000000..b4903eba65
--- /dev/null
+++ b/contrib/pg_logicalinspect/.gitignore
@@ -0,0 +1,6 @@
+# Generated subdirectories
+/log/
+/results/
+/output_iso/
+/tmp_check/
+/tmp_check_iso/
diff --git a/contrib/pg_logicalinspect/Makefile b/contrib/pg_logicalinspect/Makefile
new file mode 100644
index 0000000000..55124514d4
--- /dev/null
+++ b/contrib/pg_logicalinspect/Makefile
@@ -0,0 +1,31 @@
+# contrib/pg_logicalinspect/Makefile
+
+MODULE_big = pg_logicalinspect
+OBJS = \
+	$(WIN32RES) \
+	pg_logicalinspect.o
+PGFILEDESC = "pg_logicalinspect - functions to inspect logical decoding components"
+
+EXTENSION = pg_logicalinspect
+DATA = pg_logicalinspect--1.0.sql
+
+EXTRA_INSTALL = contrib/test_decoding
+
+ISOLATION = logical_inspect
+
+ISOLATION_OPTS = --temp-config $(top_srcdir)/contrib/pg_logicalinspect/logicalinspect.conf
+
+# Disabled because these tests require "wal_level=logical", which
+# some installcheck users do not have (e.g. buildfarm clients).
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/pg_logicalinspect
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/pg_logicalinspect/expected/logical_inspect.out b/contrib/pg_logicalinspect/expected/logical_inspect.out
new file mode 100644
index 0000000000..d95efa4d1e
--- /dev/null
+++ b/contrib/pg_logicalinspect/expected/logical_inspect.out
@@ -0,0 +1,52 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s0_init s0_begin s0_savepoint s0_truncate s1_checkpoint s1_get_changes s0_commit s0_begin s0_insert s1_checkpoint s1_get_changes s0_commit s1_get_changes s1_get_logical_snapshot_info s1_get_logical_snapshot_meta
+step s0_init: SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding');
+?column?
+--------
+init    
+(1 row)
+
+step s0_begin: BEGIN;
+step s0_savepoint: SAVEPOINT sp1;
+step s0_truncate: TRUNCATE tbl1;
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data
+----
+(0 rows)
+
+step s0_commit: COMMIT;
+step s0_begin: BEGIN;
+step s0_insert: INSERT INTO tbl1 VALUES (1);
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                   
+---------------------------------------
+BEGIN                                  
+table public.tbl1: TRUNCATE: (no-flags)
+COMMIT                                 
+(3 rows)
+
+step s0_commit: COMMIT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                                         
+-------------------------------------------------------------
+BEGIN                                                        
+table public.tbl1: INSERT: val1[integer]:1 val2[integer]:null
+COMMIT                                                       
+(3 rows)
+
+step s1_get_logical_snapshot_info: SELECT info.state, info.catchange_count, array_length(info.catchange_xip,1) AS catchange_array_length, info.committed_count, array_length(info.committed_xip,1) AS committed_array_length FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info ORDER BY 2;
+state     |catchange_count|catchange_array_length|committed_count|committed_array_length
+----------+---------------+----------------------+---------------+----------------------
+consistent|              0|                      |              2|                     2
+consistent|              2|                     2|              0|                      
+(2 rows)
+
+step s1_get_logical_snapshot_meta: SELECT COUNT(meta.*) from pg_ls_logicalsnapdir(), pg_get_logical_snapshot_meta(name) as meta;
+count
+-----
+    2
+(1 row)
+
diff --git a/contrib/pg_logicalinspect/logicalinspect.conf b/contrib/pg_logicalinspect/logicalinspect.conf
new file mode 100644
index 0000000000..e3d257315f
--- /dev/null
+++ b/contrib/pg_logicalinspect/logicalinspect.conf
@@ -0,0 +1 @@
+wal_level = logical
diff --git a/contrib/pg_logicalinspect/meson.build b/contrib/pg_logicalinspect/meson.build
new file mode 100644
index 0000000000..3ec635509b
--- /dev/null
+++ b/contrib/pg_logicalinspect/meson.build
@@ -0,0 +1,39 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+pg_logicalinspect_sources = files('pg_logicalinspect.c')
+
+if host_system == 'windows'
+  pg_logicalinspect_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+    '--NAME', 'pg_logicalinspect',
+    '--FILEDESC', 'pg_logicalinspect - functions to inspect logical decoding components',])
+endif
+
+pg_logicalinspect = shared_module('pg_logicalinspect',
+  pg_logicalinspect_sources,
+  kwargs: contrib_mod_args + {
+      'dependencies': contrib_mod_args['dependencies'],
+  },
+)
+contrib_targets += pg_logicalinspect
+
+install_data(
+  'pg_logicalinspect.control',
+  'pg_logicalinspect--1.0.sql',
+  kwargs: contrib_data_args,
+)
+
+tests += {
+  'name': 'pg_logicalinspect',
+  'sd': meson.current_source_dir(),
+  'bd': meson.current_build_dir(),
+  'isolation': {
+    'specs': [
+      'logical_inspect',
+    ],
+    'regress_args': [
+      '--temp-config', files('logicalinspect.conf'),
+    ],
+    # see above
+    'runningcheck': false,
+  },
+}
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql b/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
new file mode 100644
index 0000000000..8f7f947cbb
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
@@ -0,0 +1,43 @@
+/* contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_logicalinspect" to load this file. \quit
+
+--
+-- pg_get_logical_snapshot_meta()
+--
+CREATE FUNCTION pg_get_logical_snapshot_meta(IN filename text,
+    OUT magic int4,
+    OUT checksum int8,
+    OUT version int4
+)
+AS 'MODULE_PATHNAME', 'pg_get_logical_snapshot_meta'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_logical_snapshot_meta(text) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_logical_snapshot_meta(text) TO pg_read_server_files;
+
+--
+-- pg_get_logical_snapshot_info()
+--
+CREATE FUNCTION pg_get_logical_snapshot_info(IN filename text,
+    OUT state text,
+    OUT xmin xid,
+    OUT xmax xid,
+    OUT start_decoding_at pg_lsn,
+    OUT two_phase_at pg_lsn,
+    OUT initial_xmin_horizon xid,
+    OUT building_full_snapshot boolean,
+    OUT in_slot_creation boolean,
+    OUT last_serialized_snapshot pg_lsn,
+    OUT next_phase_at xid,
+    OUT committed_count int4,
+    OUT committed_xip xid[],
+    OUT catchange_count int4,
+    OUT catchange_xip xid[]
+)
+AS 'MODULE_PATHNAME', 'pg_get_logical_snapshot_info'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_logical_snapshot_info(text) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_logical_snapshot_info(text) TO pg_read_server_files;
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.c b/contrib/pg_logicalinspect/pg_logicalinspect.c
new file mode 100644
index 0000000000..790c64d6fa
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.c
@@ -0,0 +1,167 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_logicalinspect.c
+ *		  Functions to inspect contents of PostgreSQL logical snapshots
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ *		  contrib/pg_logicalinspect/pg_logicalinspect.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "funcapi.h"
+#include "replication/snapbuild_internal.h"
+#include "utils/array.h"
+#include "utils/builtins.h"
+#include "utils/pg_lsn.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_meta);
+PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_info);
+
+/* Return the description of SnapBuildState */
+static const char *
+get_snapbuild_state_desc(SnapBuildState state)
+{
+	const char *stateDesc = "unknown state";
+
+	switch (state)
+	{
+		case SNAPBUILD_START:
+			stateDesc = "start";
+			break;
+		case SNAPBUILD_BUILDING_SNAPSHOT:
+			stateDesc = "building";
+			break;
+		case SNAPBUILD_FULL_SNAPSHOT:
+			stateDesc = "full";
+			break;
+		case SNAPBUILD_CONSISTENT:
+			stateDesc = "consistent";
+			break;
+	}
+
+	return stateDesc;
+}
+
+/*
+ * Retrieve the logical snapshot file metadata.
+ */
+Datum
+pg_get_logical_snapshot_meta(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_META_COLS 3
+	SnapBuildOnDisk ondisk;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_META_COLS] = {0};
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_META_COLS] = {0};
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+	int			i = 0;
+	text	   *filename_t = PG_GETARG_TEXT_PP(0);
+
+	sprintf(path, "%s/%s",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			text_to_cstring(filename_t));
+
+	/* Build a tuple descriptor for our result type */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	/* Validate and restore the snapshot to 'ondisk' */
+	SnapBuildRestoreSnapshot(&ondisk, path, CurrentMemoryContext, false);
+
+	values[i++] = UInt32GetDatum(ondisk.magic);
+	values[i++] = Int64GetDatum((int64) ondisk.checksum);
+	values[i++] = UInt32GetDatum(ondisk.version);
+
+	Assert(i == PG_GET_LOGICAL_SNAPSHOT_META_COLS);
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_META_COLS
+}
+
+Datum
+pg_get_logical_snapshot_info(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_INFO_COLS 14
+	SnapBuildOnDisk ondisk;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS] = {0};
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS] = {0};
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+	int			i = 0;
+	text	   *filename_t = PG_GETARG_TEXT_PP(0);
+
+	sprintf(path, "%s/%s",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			text_to_cstring(filename_t));
+
+	/* Build a tuple descriptor for our result type */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	/* Validate and restore the snapshot to 'ondisk' */
+	SnapBuildRestoreSnapshot(&ondisk, path, CurrentMemoryContext, false);
+
+	values[i++] = CStringGetTextDatum(get_snapbuild_state_desc(ondisk.builder.state));
+	values[i++] = TransactionIdGetDatum(ondisk.builder.xmin);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.xmax);
+	values[i++] = LSNGetDatum(ondisk.builder.start_decoding_at);
+	values[i++] = LSNGetDatum(ondisk.builder.two_phase_at);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.initial_xmin_horizon);
+	values[i++] = BoolGetDatum(ondisk.builder.building_full_snapshot);
+	values[i++] = BoolGetDatum(ondisk.builder.in_slot_creation);
+	values[i++] = LSNGetDatum(ondisk.builder.last_serialized_snapshot);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.next_phase_at);
+
+	values[i++] = UInt32GetDatum(ondisk.builder.committed.xcnt);
+	if (ondisk.builder.committed.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.committed.xcnt * sizeof(Datum));
+
+		for (int j = 0; j < ondisk.builder.committed.xcnt; j++)
+			arrayelems[j] = TransactionIdGetDatum(ondisk.builder.committed.xip[j]);
+
+		values[i++] = PointerGetDatum(construct_array_builtin(arrayelems,
+															  ondisk.builder.committed.xcnt,
+															  XIDOID));
+	}
+	else
+		nulls[i++] = true;
+
+	values[i++] = UInt32GetDatum(ondisk.builder.catchange.xcnt);
+	if (ondisk.builder.catchange.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.catchange.xcnt * sizeof(Datum));
+
+		for (int j = 0; j < ondisk.builder.catchange.xcnt; j++)
+			arrayelems[j] = TransactionIdGetDatum(ondisk.builder.catchange.xip[j]);
+
+		values[i++] = PointerGetDatum(construct_array_builtin(arrayelems,
+															  ondisk.builder.catchange.xcnt,
+															  XIDOID));
+	}
+	else
+		nulls[i++] = true;
+
+	Assert(i == PG_GET_LOGICAL_SNAPSHOT_INFO_COLS);
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_INFO_COLS
+}
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.control b/contrib/pg_logicalinspect/pg_logicalinspect.control
new file mode 100644
index 0000000000..b4a70e57ba
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.control
@@ -0,0 +1,5 @@
+# pg_logicalinspect extension
+comment = 'functions to inspect logical decoding components'
+default_version = '1.0'
+module_pathname = '$libdir/pg_logicalinspect'
+relocatable = true
diff --git a/contrib/pg_logicalinspect/specs/logical_inspect.spec b/contrib/pg_logicalinspect/specs/logical_inspect.spec
new file mode 100644
index 0000000000..9851a6c18e
--- /dev/null
+++ b/contrib/pg_logicalinspect/specs/logical_inspect.spec
@@ -0,0 +1,34 @@
+# Test the pg_logicalinspect functions: that needs some permutation to
+# ensure that we are creating multiple logical snapshots and that one of them
+# contains ongoing catalogs changes.
+setup
+{
+    DROP TABLE IF EXISTS tbl1;
+    CREATE TABLE tbl1 (val1 integer, val2 integer);
+    CREATE EXTENSION pg_logicalinspect;
+}
+
+teardown
+{
+    DROP TABLE tbl1;
+    SELECT 'stop' FROM pg_drop_replication_slot('isolation_slot');
+    DROP EXTENSION pg_logicalinspect;
+}
+
+session "s0"
+setup { SET synchronous_commit=on; }
+step "s0_init" { SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding'); }
+step "s0_begin" { BEGIN; }
+step "s0_savepoint" { SAVEPOINT sp1; }
+step "s0_truncate" { TRUNCATE tbl1; }
+step "s0_insert" { INSERT INTO tbl1 VALUES (1); }
+step "s0_commit" { COMMIT; }
+
+session "s1"
+setup { SET synchronous_commit=on; }
+step "s1_checkpoint" { CHECKPOINT; }
+step "s1_get_changes" { SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0'); }
+step "s1_get_logical_snapshot_meta" { SELECT COUNT(meta.*) from pg_ls_logicalsnapdir(), pg_get_logical_snapshot_meta(name) as meta;}
+step "s1_get_logical_snapshot_info" { SELECT info.state, info.catchange_count, array_length(info.catchange_xip,1) AS catchange_array_length, info.committed_count, array_length(info.committed_xip,1) AS committed_array_length FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info ORDER BY 2; }
+
+permutation "s0_init" "s0_begin" "s0_savepoint" "s0_truncate" "s1_checkpoint" "s1_get_changes" "s0_commit" "s0_begin" "s0_insert" "s1_checkpoint" "s1_get_changes" "s0_commit" "s1_get_changes" "s1_get_logical_snapshot_info" "s1_get_logical_snapshot_meta"
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index 44639a8dca..7c381949a5 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -154,6 +154,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
  &pgbuffercache;
  &pgcrypto;
  &pgfreespacemap;
+ &pglogicalinspect;
  &pgprewarm;
  &pgrowlocks;
  &pgstatstatements;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index a7ff5f8264..66e6dccd4c 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -143,6 +143,7 @@
 <!ENTITY pgbuffercache   SYSTEM "pgbuffercache.sgml">
 <!ENTITY pgcrypto        SYSTEM "pgcrypto.sgml">
 <!ENTITY pgfreespacemap  SYSTEM "pgfreespacemap.sgml">
+<!ENTITY pglogicalinspect  SYSTEM "pglogicalinspect.sgml">
 <!ENTITY pgprewarm       SYSTEM "pgprewarm.sgml">
 <!ENTITY pgrowlocks      SYSTEM "pgrowlocks.sgml">
 <!ENTITY pgstatstatements SYSTEM "pgstatstatements.sgml">
diff --git a/doc/src/sgml/pglogicalinspect.sgml b/doc/src/sgml/pglogicalinspect.sgml
new file mode 100644
index 0000000000..4b111f9611
--- /dev/null
+++ b/doc/src/sgml/pglogicalinspect.sgml
@@ -0,0 +1,143 @@
+<!-- doc/src/sgml/pglogicalinspect.sgml -->
+
+<sect1 id="pglogicalinspect" xreflabel="pg_logicalinspect">
+ <title>pg_logicalinspect &mdash; logical decoding components inspection</title>
+
+ <indexterm zone="pglogicalinspect">
+  <primary>pg_logicalinspect</primary>
+ </indexterm>
+
+ <para>
+  The <filename>pg_logicalinspect</filename> module provides SQL functions
+  that allow you to inspect the contents of logical decoding components. It
+  allows the inspection of serialized logical snapshots of a running
+  <productname>PostgreSQL</productname> database cluster, which is useful
+  for debugging or educational purposes.
+ </para>
+
+ <para>
+  By default, use of these functions is restricted to superusers and members of
+  the <literal>pg_read_server_files</literal> role. Access may be granted by
+  superusers to others using <command>GRANT</command>.
+ </para>
+
+ <sect2 id="pglogicalinspect-funcs">
+  <title>Functions</title>
+
+  <variablelist>
+   <varlistentry id="pglogicalinspect-funcs-pg-get-logical-snapshot-meta">
+    <term>
+     <function>pg_get_logical_snapshot_meta(filename text) returns record</function>
+    </term>
+
+    <listitem>
+     <para>
+      Gets logical snapshot metadata about a snapshot file that is located in
+      the server's <filename>pg_logical/snapshots</filename> directory.
+      The <replaceable>filename</replaceable> argument represents the snapshot
+      file name.
+      For example:
+<screen>
+postgres=# SELECT * FROM pg_ls_logicalsnapdir();
+-[ RECORD 1 ]+-----------------------
+name         | 0-40796E18.snap
+size         | 152
+modification | 2024-08-14 16:36:32+00
+
+postgres=# SELECT * FROM pg_get_logical_snapshot_meta('0-40796E18.snap');
+-[ RECORD 1 ]--------
+magic    | 1369563137
+checksum | 1028045905
+version  | 6
+
+postgres=# SELECT ss.name, meta.* FROM pg_ls_logicalsnapdir() AS ss,
+pg_get_logical_snapshot_meta(ss.name) AS meta;
+-[ RECORD 1 ]-------------
+name     | 0-40796E18.snap
+magic    | 1369563137
+checksum | 1028045905
+version  | 6
+</screen>
+     </para>
+     <para>
+      If <replaceable>filename</replaceable> does not match a snapshot file, the
+      function raises an error.
+     </para>
+    </listitem>
+   </varlistentry>
+
+   <varlistentry id="pglogicalinspect-funcs-pg-get-logical-snapshot-info">
+    <term>
+     <function>pg_get_logical_snapshot_info(filename text) returns record</function>
+    </term>
+
+    <listitem>
+     <para>
+      Gets logical snapshot information about a snapshot file that is located in
+      the server's <filename>pg_logical/snapshots</filename> directory.
+      The <replaceable>filename</replaceable> argument represents the snapshot
+      file name.
+      For example:
+<screen>
+postgres=# SELECT * FROM pg_ls_logicalsnapdir();
+-[ RECORD 1 ]+-----------------------
+name         | 0-40796E18.snap
+size         | 152
+modification | 2024-08-14 16:36:32+00
+
+postgres=# SELECT * FROM pg_get_logical_snapshot_info('0-40796E18.snap');
+-[ RECORD 1 ]------------+-----------
+state                    | consistent
+xmin                     | 751
+xmax                     | 751
+start_decoding_at        | 0/40796AF8
+two_phase_at             | 0/40796AF8
+initial_xmin_horizon     | 0
+building_full_snapshot   | f
+in_slot_creation         | f
+last_serialized_snapshot | 0/0
+next_phase_at            | 0
+committed_count          | 0
+committed_xip            |
+catchange_count          | 2
+catchange_xip            | {751,752}
+
+postgres=# SELECT ss.name, info.* FROM pg_ls_logicalsnapdir() AS ss,
+pg_get_logical_snapshot_info(ss.name) AS info;
+-[ RECORD 1 ]------------+----------------
+name                     | 0-40796E18.snap
+state                    | consistent
+xmin                     | 751
+xmax                     | 751
+start_decoding_at        | 0/40796AF8
+two_phase_at             | 0/40796AF8
+initial_xmin_horizon     | 0
+building_full_snapshot   | f
+in_slot_creation         | f
+last_serialized_snapshot | 0/0
+next_phase_at            | 0
+committed_count          | 0
+committed_xip            |
+catchange_count          | 2
+catchange_xip            | {751,752}
+</screen>
+     </para>
+     <para>
+      If <replaceable>filename</replaceable> does not match a snapshot file, the
+      function raises an error.
+     </para>
+    </listitem>
+   </varlistentry>
+
+  </variablelist>
+ </sect2>
+
+ <sect2 id="pglogicalinspect-author">
+  <title>Author</title>
+
+  <para>
+   Bertrand Drouvot <email>bertranddrouvot.pg@gmail.com</email>
+  </para>
+ </sect2>
+
+</sect1>
diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c
index b9df8c0a02..92fd57b77e 100644
--- a/src/backend/replication/logical/snapbuild.c
+++ b/src/backend/replication/logical/snapbuild.c
@@ -1684,34 +1684,31 @@ out:
 }
 
 /*
- * Restore a snapshot into 'builder' if previously one has been stored at the
- * location indicated by 'lsn'. Returns true if successful, false otherwise.
+ * Restore the logical snapshot file contents to 'ondisk'.
+ *
+ * If 'missing_ok' is true, will not throw an error if the file is not found.
+ * 'context' is the memory context where the catalog modifying/committed xid
+ * will live.
  */
-static bool
-SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
+bool
+SnapBuildRestoreSnapshot(SnapBuildOnDisk *ondisk, const char *path,
+						 MemoryContext context, bool missing_ok)
 {
-	SnapBuildOnDisk ondisk;
 	int			fd;
-	char		path[MAXPGPATH];
-	Size		sz;
 	pg_crc32c	checksum;
-
-	/* no point in loading a snapshot if we're already there */
-	if (builder->state == SNAPBUILD_CONSISTENT)
-		return false;
-
-	sprintf(path, "%s/%X-%X.snap",
-			PG_LOGICAL_SNAPSHOTS_DIR,
-			LSN_FORMAT_ARGS(lsn));
+	Size		sz;
 
 	fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
 
-	if (fd < 0 && errno == ENOENT)
-		return false;
-	else if (fd < 0)
+	if (fd < 0)
+	{
+		if (missing_ok && errno == ENOENT)
+			return false;
+
 		ereport(ERROR,
 				(errcode_for_file_access(),
 				 errmsg("could not open file \"%s\": %m", path)));
+	}
 
 	/* ----
 	 * Make sure the snapshot had been stored safely to disk, that's normally
@@ -1724,47 +1721,46 @@ SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
 	fsync_fname(path, false);
 	fsync_fname(PG_LOGICAL_SNAPSHOTS_DIR, true);
 
-
 	/* read statically sized portion of snapshot */
-	SnapBuildRestoreContents(fd, (char *) &ondisk, SnapBuildOnDiskConstantSize, path);
+	SnapBuildRestoreContents(fd, (char *) ondisk, SnapBuildOnDiskConstantSize, path);
 
-	if (ondisk.magic != SNAPBUILD_MAGIC)
+	if (ondisk->magic != SNAPBUILD_MAGIC)
 		ereport(ERROR,
 				(errcode(ERRCODE_DATA_CORRUPTED),
 				 errmsg("snapbuild state file \"%s\" has wrong magic number: %u instead of %u",
-						path, ondisk.magic, SNAPBUILD_MAGIC)));
+						path, ondisk->magic, SNAPBUILD_MAGIC)));
 
-	if (ondisk.version != SNAPBUILD_VERSION)
+	if (ondisk->version != SNAPBUILD_VERSION)
 		ereport(ERROR,
 				(errcode(ERRCODE_DATA_CORRUPTED),
 				 errmsg("snapbuild state file \"%s\" has unsupported version: %u instead of %u",
-						path, ondisk.version, SNAPBUILD_VERSION)));
+						path, ondisk->version, SNAPBUILD_VERSION)));
 
 	INIT_CRC32C(checksum);
 	COMP_CRC32C(checksum,
-				((char *) &ondisk) + SnapBuildOnDiskNotChecksummedSize,
+				((char *) ondisk) + SnapBuildOnDiskNotChecksummedSize,
 				SnapBuildOnDiskConstantSize - SnapBuildOnDiskNotChecksummedSize);
 
 	/* read SnapBuild */
-	SnapBuildRestoreContents(fd, (char *) &ondisk.builder, sizeof(SnapBuild), path);
-	COMP_CRC32C(checksum, &ondisk.builder, sizeof(SnapBuild));
+	SnapBuildRestoreContents(fd, (char *) &ondisk->builder, sizeof(SnapBuild), path);
+	COMP_CRC32C(checksum, &ondisk->builder, sizeof(SnapBuild));
 
 	/* restore committed xacts information */
-	if (ondisk.builder.committed.xcnt > 0)
+	if (ondisk->builder.committed.xcnt > 0)
 	{
-		sz = sizeof(TransactionId) * ondisk.builder.committed.xcnt;
-		ondisk.builder.committed.xip = MemoryContextAllocZero(builder->context, sz);
-		SnapBuildRestoreContents(fd, (char *) ondisk.builder.committed.xip, sz, path);
-		COMP_CRC32C(checksum, ondisk.builder.committed.xip, sz);
+		sz = sizeof(TransactionId) * ondisk->builder.committed.xcnt;
+		ondisk->builder.committed.xip = MemoryContextAllocZero(context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.committed.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.committed.xip, sz);
 	}
 
 	/* restore catalog modifying xacts information */
-	if (ondisk.builder.catchange.xcnt > 0)
+	if (ondisk->builder.catchange.xcnt > 0)
 	{
-		sz = sizeof(TransactionId) * ondisk.builder.catchange.xcnt;
-		ondisk.builder.catchange.xip = MemoryContextAllocZero(builder->context, sz);
-		SnapBuildRestoreContents(fd, (char *) ondisk.builder.catchange.xip, sz, path);
-		COMP_CRC32C(checksum, ondisk.builder.catchange.xip, sz);
+		sz = sizeof(TransactionId) * ondisk->builder.catchange.xcnt;
+		ondisk->builder.catchange.xip = MemoryContextAllocZero(context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.catchange.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.catchange.xip, sz);
 	}
 
 	if (CloseTransientFile(fd) != 0)
@@ -1775,11 +1771,36 @@ SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
 	FIN_CRC32C(checksum);
 
 	/* verify checksum of what we've read */
-	if (!EQ_CRC32C(checksum, ondisk.checksum))
+	if (!EQ_CRC32C(checksum, ondisk->checksum))
 		ereport(ERROR,
 				(errcode(ERRCODE_DATA_CORRUPTED),
 				 errmsg("checksum mismatch for snapbuild state file \"%s\": is %u, should be %u",
-						path, checksum, ondisk.checksum)));
+						path, checksum, ondisk->checksum)));
+
+	return true;
+}
+
+/*
+ * Restore a snapshot into 'builder' if previously one has been stored at the
+ * location indicated by 'lsn'. Returns true if successful, false otherwise.
+ */
+static bool
+SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
+{
+	SnapBuildOnDisk ondisk;
+	char		path[MAXPGPATH];
+
+	/* no point in loading a snapshot if we're already there */
+	if (builder->state == SNAPBUILD_CONSISTENT)
+		return false;
+
+	sprintf(path, "%s/%X-%X.snap",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			LSN_FORMAT_ARGS(lsn));
+
+	/* validate and restore the snapshot to 'ondisk' */
+	if (!SnapBuildRestoreSnapshot(&ondisk, path, builder->context, true))
+		return false;
 
 	/*
 	 * ok, we now have a sensible snapshot here, figure out if it has more
diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c
index e5c7e57a5d..41434279c5 100644
--- a/src/backend/utils/adt/arrayfuncs.c
+++ b/src/backend/utils/adt/arrayfuncs.c
@@ -3447,6 +3447,12 @@ construct_array_builtin(Datum *elems, int nelems, Oid elmtype)
 			elmalign = TYPALIGN_SHORT;
 			break;
 
+		case XIDOID:
+			elmlen = sizeof(TransactionId);
+			elmbyval = true;
+			elmalign = TYPALIGN_INT;
+			break;
+
 		default:
 			elog(ERROR, "type %u not supported by construct_array_builtin()", elmtype);
 			/* keep compiler quiet */
diff --git a/src/include/replication/snapbuild.h b/src/include/replication/snapbuild.h
index dbb4bc2f4b..3c1454df99 100644
--- a/src/include/replication/snapbuild.h
+++ b/src/include/replication/snapbuild.h
@@ -15,6 +15,10 @@
 #include "access/xlogdefs.h"
 #include "utils/snapmgr.h"
 
+/*
+ * Please keep get_snapbuild_state_desc() (located in the pg_logicalinspect
+ * module) updated if a change needs to be made to SnapBuildState.
+ */
 typedef enum
 {
 	/*
diff --git a/src/include/replication/snapbuild_internal.h b/src/include/replication/snapbuild_internal.h
index 03719ccf2a..7134b48b96 100644
--- a/src/include/replication/snapbuild_internal.h
+++ b/src/include/replication/snapbuild_internal.h
@@ -193,4 +193,7 @@ typedef struct SnapBuildOnDisk
 	/* variable amount of TransactionIds follows */
 } SnapBuildOnDisk;
 
+extern bool SnapBuildRestoreSnapshot(SnapBuildOnDisk *ondisk, const char *path,
+									 MemoryContext context, bool missing_ok);
+
 #endif							/* SNAPBUILD_INTERNAL_H */
-- 
2.39.3

#60Peter Smith
smithpb2250@gmail.com
In reply to: Masahiko Sawada (#59)
Re: Add contrib/pg_logicalsnapinspect

Here are some minor review comments for v15-0002.

======
contrib/pg_logicalinspect/pg_logicalinspect.c

1.
+pg_get_logical_snapshot_meta(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_META_COLS 3
+ SnapBuildOnDisk ondisk;
+ HeapTuple tuple;
+ Datum values[PG_GET_LOGICAL_SNAPSHOT_META_COLS] = {0};
+ bool nulls[PG_GET_LOGICAL_SNAPSHOT_META_COLS] = {0};
+ TupleDesc tupdesc;
+ char path[MAXPGPATH];
+ int i = 0;
+ text    *filename_t = PG_GETARG_TEXT_PP(0);
+
+ sprintf(path, "%s/%s",
+ PG_LOGICAL_SNAPSHOTS_DIR,
+ text_to_cstring(filename_t));
+
+ /* Build a tuple descriptor for our result type */
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
+
+ /* Validate and restore the snapshot to 'ondisk' */
+ SnapBuildRestoreSnapshot(&ondisk, path, CurrentMemoryContext, false);

The sprintf should be deferred. Could you do it after the ERROR check?

~~~

2.
+pg_get_logical_snapshot_info(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_INFO_COLS 14
+ SnapBuildOnDisk ondisk;
+ HeapTuple tuple;
+ Datum values[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS] = {0};
+ bool nulls[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS] = {0};
+ TupleDesc tupdesc;
+ char path[MAXPGPATH];
+ int i = 0;
+ text    *filename_t = PG_GETARG_TEXT_PP(0);
+
+ sprintf(path, "%s/%s",
+ PG_LOGICAL_SNAPSHOTS_DIR,
+ text_to_cstring(filename_t));
+
+ /* Build a tuple descriptor for our result type */
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");

Ditto #1. The sprintf should be deferred. Could you do it after the ERROR check?

======
src/backend/replication/logical/snapbuild.c

3.
 /*
- * Restore a snapshot into 'builder' if previously one has been stored at the
- * location indicated by 'lsn'. Returns true if successful, false otherwise.
+ * Restore the logical snapshot file contents to 'ondisk'.
+ *
+ * If 'missing_ok' is true, will not throw an error if the file is not found.
+ * 'context' is the memory context where the catalog modifying/committed xid
+ * will live.
  */
-static bool
-SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
+bool
+SnapBuildRestoreSnapshot(SnapBuildOnDisk *ondisk, const char *path,
+ MemoryContext context, bool missing_ok)

nit - I think it's better to describe parameters in the same order
that they are declared. Also, include a 'path' description, so it is
not the only one omitted.

SUGGESTION:
'path' - snapshot file path.
'context' - memory context where the catalog modifying/committed xid will live.
‘missing_ok’ – when true, don't throw an error if the file is not found.

======
Kind Regards,
Peter Smith.
Fujitsu Australia

#61Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: Masahiko Sawada (#59)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Fri, Oct 11, 2024 at 04:48:26PM -0700, Masahiko Sawada wrote:

On Fri, Oct 11, 2024 at 11:15 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:

On Fri, Oct 11, 2024 at 6:15 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Hi,

On Thu, Oct 10, 2024 at 05:38:43PM -0700, Masahiko Sawada wrote:

On Thu, Oct 10, 2024 at 6:10 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

The patches mostly look good to me. Here are some minor comments:

Thanks for looking at it!

+       sprintf(path, "%s/%s",
+                       PG_LOGICAL_SNAPSHOTS_DIR,
+                       text_to_cstring(filename_t));
+
+       /* Validate and restore the snapshot to 'ondisk' */
+       ValidateAndRestoreSnapshotFile(&ondisk, path,
CurrentMemoryContext, false);
+
+       /* Build a tuple descriptor for our result type */
+       if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+               elog(ERROR, "return type must be a row type");
+
I think it would be better to check the result type before reading the
snapshot file.

Agree, done in v14.

---
+       values[i++] = Int64GetDatum((int64) ondisk.checksum);

Why is only checksum casted to int64? With that, it can show a
checksum value as a non-netagive integer but is it really necessary?
For instance, page_header() function in pageinspect shows a page
checksum as smallint.

Yeah, pd_checksum in PageHeaderData is uint16 while checksum in SnapBuildOnDisk
is pg_crc32c. The reason why it is casted to int64 is explained in [1], does that
make sense to you?

In the email, you said:

As the checksum could be > 2^31 - 1, then v9 (just shared up-thread) changes it
to an int8 in the pg_logicalinspect--1.0.sql file. So, to avoid CI failure on
the 32bit build, then v9 is using Int64GetDatum() instead of UInt32GetDatum().

I'm fine with using Int64GetDatum() for checksum.

Same goes for below:
values[i++] = Int32GetDatum(ondisk.magic);
values[i++] = Int32GetDatum(ondisk.magic);

The 2 others field (magic and version) are unlikely to be > 2^31 - 1, so v9 is
making use of UInt32GetDatum() and keep int4 in the sql file.

While I agree that these two fields are unlikely to be > 2^31 - 1, I'm
concerned a bit about an inconsistency that the patch uses
Int64GetDatum also for both ondisk.builder.committed.xcnt and
ondisk.builder.catchange.xcnt.

Thanks for the feedback. That makes sense and I agree with the proposal done
in v15.

I have a minor comment:

+ <sect2 id="pglogicalinspect-funcs">
+ <title>General Functions</title>

If we use "General Functions" here it sounds like there are other
functions for specific purposes in pg_logicalinspect module. How about
using "Functions" instead?

To elaborate further, pageinspect has a "General Functions" section,
which makes sense to me as it has other AM-type specific functions. On
the other hand, pg_logicalinspect has SQL functions only for one
logical replication component. So I think it makes sense to use
"Function" instead. pg_walinspect also has the sole section "General
Function"

Yeah, I used it as a "template".

but I personally think that "Function" is more appropriate
like other modules does.

I do agree.

BTW I think that adding snapshot_internal.h could be a separate patch.
That makes the main pg_logicalinspect patch cleaner.

Agree.

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

#62Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: Peter Smith (#60)
2 attachment(s)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Mon, Oct 14, 2024 at 09:57:22AM +1100, Peter Smith wrote:

Here are some minor review comments for v15-0002.

======
contrib/pg_logicalinspect/pg_logicalinspect.c

1.
+pg_get_logical_snapshot_meta(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_META_COLS 3
+ SnapBuildOnDisk ondisk;
+ HeapTuple tuple;
+ Datum values[PG_GET_LOGICAL_SNAPSHOT_META_COLS] = {0};
+ bool nulls[PG_GET_LOGICAL_SNAPSHOT_META_COLS] = {0};
+ TupleDesc tupdesc;
+ char path[MAXPGPATH];
+ int i = 0;
+ text    *filename_t = PG_GETARG_TEXT_PP(0);
+
+ sprintf(path, "%s/%s",
+ PG_LOGICAL_SNAPSHOTS_DIR,
+ text_to_cstring(filename_t));
+
+ /* Build a tuple descriptor for our result type */
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
+
+ /* Validate and restore the snapshot to 'ondisk' */
+ SnapBuildRestoreSnapshot(&ondisk, path, CurrentMemoryContext, false);

The sprintf should be deferred. Could you do it after the ERROR check?

I think that makes sense, done in v16 attached.

======
src/backend/replication/logical/snapbuild.c

3.
/*
- * Restore a snapshot into 'builder' if previously one has been stored at the
- * location indicated by 'lsn'. Returns true if successful, false otherwise.
+ * Restore the logical snapshot file contents to 'ondisk'.
+ *
+ * If 'missing_ok' is true, will not throw an error if the file is not found.
+ * 'context' is the memory context where the catalog modifying/committed xid
+ * will live.
*/
-static bool
-SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
+bool
+SnapBuildRestoreSnapshot(SnapBuildOnDisk *ondisk, const char *path,
+ MemoryContext context, bool missing_ok)

nit - I think it's better to describe parameters in the same order
that they are declared.

Done in v16.

Also, include a 'path' description, so it is
not the only one omitted.

I don't think that's worth it as self explanatory IMHO.

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachments:

v16-0001-Move-SnapBuild-and-SnapBuildOnDisk-structs-to-sn.patchtext/x-diff; charset=us-asciiDownload
From ef5997544afd9f414648e5ccae6c0d08c5ab66fa Mon Sep 17 00:00:00 2001
From: Masahiko Sawada <sawada.mshk@gmail.com>
Date: Fri, 11 Oct 2024 16:23:14 -0700
Subject: [PATCH v16 1/2] Move SnapBuild and SnapBuildOnDisk structs to
 snapshot_internal.h

This commit moves the definitions of the SnapBuild and SnapBuildOnDisk
structs, related to logical snapshots, to the snapshot_internal.h
file. This change allows external tools, such as
pg_logicalinspect (with an upcoming patch), to access and utilize the
contents of logical snapshots.

Author: Bertrand Drouvot
Reviewed-by: Amit Kapila, Shveta Malik, Peter Smith
Discussion: https://postgr.es/m/ZscuZ92uGh3wm4tW%40ip-10-97-1-34.eu-west-3.compute.internal
---
 src/backend/replication/logical/snapbuild.c  | 175 +----------------
 src/include/replication/snapbuild.h          |   2 +-
 src/include/replication/snapbuild_internal.h | 196 +++++++++++++++++++
 3 files changed, 198 insertions(+), 175 deletions(-)
  46.4% src/backend/replication/logical/
  53.5% src/include/replication/

diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c
index 0450f94ba8..b9df8c0a02 100644
--- a/src/backend/replication/logical/snapbuild.c
+++ b/src/backend/replication/logical/snapbuild.c
@@ -134,6 +134,7 @@
 #include "replication/logical.h"
 #include "replication/reorderbuffer.h"
 #include "replication/snapbuild.h"
+#include "replication/snapbuild_internal.h"
 #include "storage/fd.h"
 #include "storage/lmgr.h"
 #include "storage/proc.h"
@@ -143,146 +144,6 @@
 #include "utils/memutils.h"
 #include "utils/snapmgr.h"
 #include "utils/snapshot.h"
-
-/*
- * This struct contains the current state of the snapshot building
- * machinery. Besides a forward declaration in the header, it is not exposed
- * to the public, so we can easily change its contents.
- */
-struct SnapBuild
-{
-	/* how far are we along building our first full snapshot */
-	SnapBuildState state;
-
-	/* private memory context used to allocate memory for this module. */
-	MemoryContext context;
-
-	/* all transactions < than this have committed/aborted */
-	TransactionId xmin;
-
-	/* all transactions >= than this are uncommitted */
-	TransactionId xmax;
-
-	/*
-	 * Don't replay commits from an LSN < this LSN. This can be set externally
-	 * but it will also be advanced (never retreat) from within snapbuild.c.
-	 */
-	XLogRecPtr	start_decoding_at;
-
-	/*
-	 * LSN at which two-phase decoding was enabled or LSN at which we found a
-	 * consistent point at the time of slot creation.
-	 *
-	 * The prepared transactions, that were skipped because previously
-	 * two-phase was not enabled or are not covered by initial snapshot, need
-	 * to be sent later along with commit prepared and they must be before
-	 * this point.
-	 */
-	XLogRecPtr	two_phase_at;
-
-	/*
-	 * Don't start decoding WAL until the "xl_running_xacts" information
-	 * indicates there are no running xids with an xid smaller than this.
-	 */
-	TransactionId initial_xmin_horizon;
-
-	/* Indicates if we are building full snapshot or just catalog one. */
-	bool		building_full_snapshot;
-
-	/*
-	 * Indicates if we are using the snapshot builder for the creation of a
-	 * logical replication slot. If it's true, the start point for decoding
-	 * changes is not determined yet. So we skip snapshot restores to properly
-	 * find the start point. See SnapBuildFindSnapshot() for details.
-	 */
-	bool		in_slot_creation;
-
-	/*
-	 * Snapshot that's valid to see the catalog state seen at this moment.
-	 */
-	Snapshot	snapshot;
-
-	/*
-	 * LSN of the last location we are sure a snapshot has been serialized to.
-	 */
-	XLogRecPtr	last_serialized_snapshot;
-
-	/*
-	 * The reorderbuffer we need to update with usable snapshots et al.
-	 */
-	ReorderBuffer *reorder;
-
-	/*
-	 * TransactionId at which the next phase of initial snapshot building will
-	 * happen. InvalidTransactionId if not known (i.e. SNAPBUILD_START), or
-	 * when no next phase necessary (SNAPBUILD_CONSISTENT).
-	 */
-	TransactionId next_phase_at;
-
-	/*
-	 * Array of transactions which could have catalog changes that committed
-	 * between xmin and xmax.
-	 */
-	struct
-	{
-		/* number of committed transactions */
-		size_t		xcnt;
-
-		/* available space for committed transactions */
-		size_t		xcnt_space;
-
-		/*
-		 * Until we reach a CONSISTENT state, we record commits of all
-		 * transactions, not just the catalog changing ones. Record when that
-		 * changes so we know we cannot export a snapshot safely anymore.
-		 */
-		bool		includes_all_transactions;
-
-		/*
-		 * Array of committed transactions that have modified the catalog.
-		 *
-		 * As this array is frequently modified we do *not* keep it in
-		 * xidComparator order. Instead we sort the array when building &
-		 * distributing a snapshot.
-		 *
-		 * TODO: It's unclear whether that reasoning has much merit. Every
-		 * time we add something here after becoming consistent will also
-		 * require distributing a snapshot. Storing them sorted would
-		 * potentially also make it easier to purge (but more complicated wrt
-		 * wraparound?). Should be improved if sorting while building the
-		 * snapshot shows up in profiles.
-		 */
-		TransactionId *xip;
-	}			committed;
-
-	/*
-	 * Array of transactions and subtransactions that had modified catalogs
-	 * and were running when the snapshot was serialized.
-	 *
-	 * We normally rely on some WAL record types such as HEAP2_NEW_CID to know
-	 * if the transaction has changed the catalog. But it could happen that
-	 * the logical decoding decodes only the commit record of the transaction
-	 * after restoring the previously serialized snapshot in which case we
-	 * will miss adding the xid to the snapshot and end up looking at the
-	 * catalogs with the wrong snapshot.
-	 *
-	 * Now to avoid the above problem, we serialize the transactions that had
-	 * modified the catalogs and are still running at the time of snapshot
-	 * serialization. We fill this array while restoring the snapshot and then
-	 * refer it while decoding commit to ensure if the xact has modified the
-	 * catalog. We discard this array when all the xids in the list become old
-	 * enough to matter. See SnapBuildPurgeOlderTxn for details.
-	 */
-	struct
-	{
-		/* number of transactions */
-		size_t		xcnt;
-
-		/* This array must be sorted in xidComparator order */
-		TransactionId *xip;
-	}			catchange;
-};
-
 /*
  * Starting a transaction -- which we need to do while exporting a snapshot --
  * removes knowledge about the previously used resowner, so we save it here.
@@ -1557,40 +1418,6 @@ SnapBuildWaitSnapshot(xl_running_xacts *running, TransactionId cutoff)
 	}
 }
 
-/* -----------------------------------
- * Snapshot serialization support
- * -----------------------------------
- */
-
-/*
- * We store current state of struct SnapBuild on disk in the following manner:
- *
- * struct SnapBuildOnDisk;
- * TransactionId * committed.xcnt; (*not xcnt_space*)
- * TransactionId * catchange.xcnt;
- *
- */
-typedef struct SnapBuildOnDisk
-{
-	/* first part of this struct needs to be version independent */
-
-	/* data not covered by checksum */
-	uint32		magic;
-	pg_crc32c	checksum;
-
-	/* data covered by checksum */
-
-	/* version, in case we want to support pg_upgrade */
-	uint32		version;
-	/* how large is the on disk data, excluding the constant sized part */
-	uint32		length;
-
-	/* version dependent part */
-	SnapBuild	builder;
-
-	/* variable amount of TransactionIds follows */
-} SnapBuildOnDisk;
-
 #define SnapBuildOnDiskConstantSize \
 	offsetof(SnapBuildOnDisk, builder)
 #define SnapBuildOnDiskNotChecksummedSize \
diff --git a/src/include/replication/snapbuild.h b/src/include/replication/snapbuild.h
index caa5113ff8..dbb4bc2f4b 100644
--- a/src/include/replication/snapbuild.h
+++ b/src/include/replication/snapbuild.h
@@ -46,7 +46,7 @@ typedef enum
 	SNAPBUILD_CONSISTENT = 2,
 } SnapBuildState;
 
-/* forward declare so we don't have to expose the struct to the public */
+/* forward declare so we don't have to include snapbuild_internal.h */
 struct SnapBuild;
 typedef struct SnapBuild SnapBuild;
 
diff --git a/src/include/replication/snapbuild_internal.h b/src/include/replication/snapbuild_internal.h
new file mode 100644
index 0000000000..03719ccf2a
--- /dev/null
+++ b/src/include/replication/snapbuild_internal.h
@@ -0,0 +1,196 @@
+/*-------------------------------------------------------------------------
+ *
+ * snapbuild_internal.h
+ *    This file contains declarations for logical decoding utility
+ *    functions for internal use.
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * src/include/replication/snapbuild_internal.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef SNAPBUILD_INTERNAL_H
+#define SNAPBUILD_INTERNAL_H
+
+#include "port/pg_crc32c.h"
+#include "replication/reorderbuffer.h"
+#include "replication/snapbuild.h"
+
+/*
+ * This struct contains the current state of the snapshot building
+ * machinery. It is exposed to the public, so pay attention when changing its
+ * contents.
+ */
+typedef struct SnapBuild
+{
+	/* how far are we along building our first full snapshot */
+	SnapBuildState state;
+
+	/* private memory context used to allocate memory for this module. */
+	MemoryContext context;
+
+	/* all transactions < than this have committed/aborted */
+	TransactionId xmin;
+
+	/* all transactions >= than this are uncommitted */
+	TransactionId xmax;
+
+	/*
+	 * Don't replay commits from an LSN < this LSN. This can be set externally
+	 * but it will also be advanced (never retreat) from within snapbuild.c.
+	 */
+	XLogRecPtr	start_decoding_at;
+
+	/*
+	 * LSN at which two-phase decoding was enabled or LSN at which we found a
+	 * consistent point at the time of slot creation.
+	 *
+	 * The prepared transactions, that were skipped because previously
+	 * two-phase was not enabled or are not covered by initial snapshot, need
+	 * to be sent later along with commit prepared and they must be before
+	 * this point.
+	 */
+	XLogRecPtr	two_phase_at;
+
+	/*
+	 * Don't start decoding WAL until the "xl_running_xacts" information
+	 * indicates there are no running xids with an xid smaller than this.
+	 */
+	TransactionId initial_xmin_horizon;
+
+	/* Indicates if we are building full snapshot or just catalog one. */
+	bool		building_full_snapshot;
+
+	/*
+	 * Indicates if we are using the snapshot builder for the creation of a
+	 * logical replication slot. If it's true, the start point for decoding
+	 * changes is not determined yet. So we skip snapshot restores to properly
+	 * find the start point. See SnapBuildFindSnapshot() for details.
+	 */
+	bool		in_slot_creation;
+
+	/*
+	 * Snapshot that's valid to see the catalog state seen at this moment.
+	 */
+	Snapshot	snapshot;
+
+	/*
+	 * LSN of the last location we are sure a snapshot has been serialized to.
+	 */
+	XLogRecPtr	last_serialized_snapshot;
+
+	/*
+	 * The reorderbuffer we need to update with usable snapshots et al.
+	 */
+	ReorderBuffer *reorder;
+
+	/*
+	 * TransactionId at which the next phase of initial snapshot building will
+	 * happen. InvalidTransactionId if not known (i.e. SNAPBUILD_START), or
+	 * when no next phase necessary (SNAPBUILD_CONSISTENT).
+	 */
+	TransactionId next_phase_at;
+
+	/*
+	 * Array of transactions which could have catalog changes that committed
+	 * between xmin and xmax.
+	 */
+	struct
+	{
+		/* number of committed transactions */
+		size_t		xcnt;
+
+		/* available space for committed transactions */
+		size_t		xcnt_space;
+
+		/*
+		 * Until we reach a CONSISTENT state, we record commits of all
+		 * transactions, not just the catalog changing ones. Record when that
+		 * changes so we know we cannot export a snapshot safely anymore.
+		 */
+		bool		includes_all_transactions;
+
+		/*
+		 * Array of committed transactions that have modified the catalog.
+		 *
+		 * As this array is frequently modified we do *not* keep it in
+		 * xidComparator order. Instead we sort the array when building &
+		 * distributing a snapshot.
+		 *
+		 * TODO: It's unclear whether that reasoning has much merit. Every
+		 * time we add something here after becoming consistent will also
+		 * require distributing a snapshot. Storing them sorted would
+		 * potentially also make it easier to purge (but more complicated wrt
+		 * wraparound?). Should be improved if sorting while building the
+		 * snapshot shows up in profiles.
+		 */
+		TransactionId *xip;
+	}			committed;
+
+	/*
+	 * Array of transactions and subtransactions that had modified catalogs
+	 * and were running when the snapshot was serialized.
+	 *
+	 * We normally rely on some WAL record types such as HEAP2_NEW_CID to know
+	 * if the transaction has changed the catalog. But it could happen that
+	 * the logical decoding decodes only the commit record of the transaction
+	 * after restoring the previously serialized snapshot in which case we
+	 * will miss adding the xid to the snapshot and end up looking at the
+	 * catalogs with the wrong snapshot.
+	 *
+	 * Now to avoid the above problem, we serialize the transactions that had
+	 * modified the catalogs and are still running at the time of snapshot
+	 * serialization. We fill this array while restoring the snapshot and then
+	 * refer it while decoding commit to ensure if the xact has modified the
+	 * catalog. We discard this array when all the xids in the list become old
+	 * enough to matter. See SnapBuildPurgeOlderTxn for details.
+	 */
+	struct
+	{
+		/* number of transactions */
+		size_t		xcnt;
+
+		/* This array must be sorted in xidComparator order */
+		TransactionId *xip;
+	}			catchange;
+} SnapBuild;
+
+/* -----------------------------------
+ * Snapshot serialization support
+ * -----------------------------------
+ */
+
+/*
+ * We store current state of struct SnapBuild on disk in the following manner:
+ *
+ * struct SnapBuildOnDisk;
+ * TransactionId * committed.xcnt; (*not xcnt_space*)
+ * TransactionId * catchange.xcnt;
+ *
+ * Check if the SnapBuildOnDiskConstantSize and SnapBuildOnDiskNotChecksummedSize
+ * macros need to be updated when modifying the SnapBuildOnDisk struct.
+ */
+typedef struct SnapBuildOnDisk
+{
+	/* first part of this struct needs to be version independent */
+
+	/* data not covered by checksum */
+	uint32		magic;
+	pg_crc32c	checksum;
+
+	/* data covered by checksum */
+
+	/* version, in case we want to support pg_upgrade */
+	uint32		version;
+	/* how large is the on disk data, excluding the constant sized part */
+	uint32		length;
+
+	/* version dependent part */
+	SnapBuild	builder;
+
+	/* variable amount of TransactionIds follows */
+} SnapBuildOnDisk;
+
+#endif							/* SNAPBUILD_INTERNAL_H */
-- 
2.34.1

v16-0002-Add-contrib-pg_logicalinspect.patchtext/x-diff; charset=us-asciiDownload
From 9f9e2ed520d0e9315dc16521f2f7d1e26fc50cb7 Mon Sep 17 00:00:00 2001
From: Masahiko Sawada <sawada.mshk@gmail.com>
Date: Fri, 11 Oct 2024 16:24:14 -0700
Subject: [PATCH v16 2/2] Add contrib/pg_logicalinspect.

This module provides SQL functions that allow to inspect logical
decoding components.

It currently allows to inspect the contents of serialized logical
snapshots of a running database cluster, which is useful for debugging
or educational purposes.

Author: Bertrand Drouvot
Reviewed-by: Amit Kapila, Shveta Malik, Peter Smith, Peter Eisentraut
Reviewed-by: David G. Johnston
Discussion: https://postgr.es/m/ZscuZ92uGh3wm4tW%40ip-10-97-1-34.eu-west-3.compute.internal
---
 contrib/Makefile                              |   1 +
 contrib/meson.build                           |   1 +
 contrib/pg_logicalinspect/.gitignore          |   6 +
 contrib/pg_logicalinspect/Makefile            |  31 ++++
 .../expected/logical_inspect.out              |  52 ++++++
 contrib/pg_logicalinspect/logicalinspect.conf |   1 +
 contrib/pg_logicalinspect/meson.build         |  39 ++++
 .../pg_logicalinspect--1.0.sql                |  43 +++++
 contrib/pg_logicalinspect/pg_logicalinspect.c | 167 ++++++++++++++++++
 .../pg_logicalinspect.control                 |   5 +
 .../specs/logical_inspect.spec                |  34 ++++
 doc/src/sgml/contrib.sgml                     |   1 +
 doc/src/sgml/filelist.sgml                    |   1 +
 doc/src/sgml/pglogicalinspect.sgml            | 143 +++++++++++++++
 src/backend/replication/logical/snapbuild.c   |  99 +++++++----
 src/backend/utils/adt/arrayfuncs.c            |   6 +
 src/include/replication/snapbuild.h           |   4 +
 src/include/replication/snapbuild_internal.h  |   3 +
 18 files changed, 598 insertions(+), 39 deletions(-)
  12.0% contrib/pg_logicalinspect/expected/
   8.4% contrib/pg_logicalinspect/specs/
  40.8% contrib/pg_logicalinspect/
  21.9% doc/src/sgml/
  14.5% src/backend/replication/logical/

diff --git a/contrib/Makefile b/contrib/Makefile
index abd780f277..952855d9b6 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -32,6 +32,7 @@ SUBDIRS = \
 		passwordcheck	\
 		pg_buffercache	\
 		pg_freespacemap \
+		pg_logicalinspect \
 		pg_prewarm	\
 		pg_stat_statements \
 		pg_surgery	\
diff --git a/contrib/meson.build b/contrib/meson.build
index 14a8906865..159ff41555 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -46,6 +46,7 @@ subdir('passwordcheck')
 subdir('pg_buffercache')
 subdir('pgcrypto')
 subdir('pg_freespacemap')
+subdir('pg_logicalinspect')
 subdir('pg_prewarm')
 subdir('pgrowlocks')
 subdir('pg_stat_statements')
diff --git a/contrib/pg_logicalinspect/.gitignore b/contrib/pg_logicalinspect/.gitignore
new file mode 100644
index 0000000000..b4903eba65
--- /dev/null
+++ b/contrib/pg_logicalinspect/.gitignore
@@ -0,0 +1,6 @@
+# Generated subdirectories
+/log/
+/results/
+/output_iso/
+/tmp_check/
+/tmp_check_iso/
diff --git a/contrib/pg_logicalinspect/Makefile b/contrib/pg_logicalinspect/Makefile
new file mode 100644
index 0000000000..55124514d4
--- /dev/null
+++ b/contrib/pg_logicalinspect/Makefile
@@ -0,0 +1,31 @@
+# contrib/pg_logicalinspect/Makefile
+
+MODULE_big = pg_logicalinspect
+OBJS = \
+	$(WIN32RES) \
+	pg_logicalinspect.o
+PGFILEDESC = "pg_logicalinspect - functions to inspect logical decoding components"
+
+EXTENSION = pg_logicalinspect
+DATA = pg_logicalinspect--1.0.sql
+
+EXTRA_INSTALL = contrib/test_decoding
+
+ISOLATION = logical_inspect
+
+ISOLATION_OPTS = --temp-config $(top_srcdir)/contrib/pg_logicalinspect/logicalinspect.conf
+
+# Disabled because these tests require "wal_level=logical", which
+# some installcheck users do not have (e.g. buildfarm clients).
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/pg_logicalinspect
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/pg_logicalinspect/expected/logical_inspect.out b/contrib/pg_logicalinspect/expected/logical_inspect.out
new file mode 100644
index 0000000000..d95efa4d1e
--- /dev/null
+++ b/contrib/pg_logicalinspect/expected/logical_inspect.out
@@ -0,0 +1,52 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s0_init s0_begin s0_savepoint s0_truncate s1_checkpoint s1_get_changes s0_commit s0_begin s0_insert s1_checkpoint s1_get_changes s0_commit s1_get_changes s1_get_logical_snapshot_info s1_get_logical_snapshot_meta
+step s0_init: SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding');
+?column?
+--------
+init    
+(1 row)
+
+step s0_begin: BEGIN;
+step s0_savepoint: SAVEPOINT sp1;
+step s0_truncate: TRUNCATE tbl1;
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data
+----
+(0 rows)
+
+step s0_commit: COMMIT;
+step s0_begin: BEGIN;
+step s0_insert: INSERT INTO tbl1 VALUES (1);
+step s1_checkpoint: CHECKPOINT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                   
+---------------------------------------
+BEGIN                                  
+table public.tbl1: TRUNCATE: (no-flags)
+COMMIT                                 
+(3 rows)
+
+step s0_commit: COMMIT;
+step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
+data                                                         
+-------------------------------------------------------------
+BEGIN                                                        
+table public.tbl1: INSERT: val1[integer]:1 val2[integer]:null
+COMMIT                                                       
+(3 rows)
+
+step s1_get_logical_snapshot_info: SELECT info.state, info.catchange_count, array_length(info.catchange_xip,1) AS catchange_array_length, info.committed_count, array_length(info.committed_xip,1) AS committed_array_length FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info ORDER BY 2;
+state     |catchange_count|catchange_array_length|committed_count|committed_array_length
+----------+---------------+----------------------+---------------+----------------------
+consistent|              0|                      |              2|                     2
+consistent|              2|                     2|              0|                      
+(2 rows)
+
+step s1_get_logical_snapshot_meta: SELECT COUNT(meta.*) from pg_ls_logicalsnapdir(), pg_get_logical_snapshot_meta(name) as meta;
+count
+-----
+    2
+(1 row)
+
diff --git a/contrib/pg_logicalinspect/logicalinspect.conf b/contrib/pg_logicalinspect/logicalinspect.conf
new file mode 100644
index 0000000000..e3d257315f
--- /dev/null
+++ b/contrib/pg_logicalinspect/logicalinspect.conf
@@ -0,0 +1 @@
+wal_level = logical
diff --git a/contrib/pg_logicalinspect/meson.build b/contrib/pg_logicalinspect/meson.build
new file mode 100644
index 0000000000..3ec635509b
--- /dev/null
+++ b/contrib/pg_logicalinspect/meson.build
@@ -0,0 +1,39 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+pg_logicalinspect_sources = files('pg_logicalinspect.c')
+
+if host_system == 'windows'
+  pg_logicalinspect_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+    '--NAME', 'pg_logicalinspect',
+    '--FILEDESC', 'pg_logicalinspect - functions to inspect logical decoding components',])
+endif
+
+pg_logicalinspect = shared_module('pg_logicalinspect',
+  pg_logicalinspect_sources,
+  kwargs: contrib_mod_args + {
+      'dependencies': contrib_mod_args['dependencies'],
+  },
+)
+contrib_targets += pg_logicalinspect
+
+install_data(
+  'pg_logicalinspect.control',
+  'pg_logicalinspect--1.0.sql',
+  kwargs: contrib_data_args,
+)
+
+tests += {
+  'name': 'pg_logicalinspect',
+  'sd': meson.current_source_dir(),
+  'bd': meson.current_build_dir(),
+  'isolation': {
+    'specs': [
+      'logical_inspect',
+    ],
+    'regress_args': [
+      '--temp-config', files('logicalinspect.conf'),
+    ],
+    # see above
+    'runningcheck': false,
+  },
+}
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql b/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
new file mode 100644
index 0000000000..8f7f947cbb
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql
@@ -0,0 +1,43 @@
+/* contrib/pg_logicalinspect/pg_logicalinspect--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_logicalinspect" to load this file. \quit
+
+--
+-- pg_get_logical_snapshot_meta()
+--
+CREATE FUNCTION pg_get_logical_snapshot_meta(IN filename text,
+    OUT magic int4,
+    OUT checksum int8,
+    OUT version int4
+)
+AS 'MODULE_PATHNAME', 'pg_get_logical_snapshot_meta'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_logical_snapshot_meta(text) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_logical_snapshot_meta(text) TO pg_read_server_files;
+
+--
+-- pg_get_logical_snapshot_info()
+--
+CREATE FUNCTION pg_get_logical_snapshot_info(IN filename text,
+    OUT state text,
+    OUT xmin xid,
+    OUT xmax xid,
+    OUT start_decoding_at pg_lsn,
+    OUT two_phase_at pg_lsn,
+    OUT initial_xmin_horizon xid,
+    OUT building_full_snapshot boolean,
+    OUT in_slot_creation boolean,
+    OUT last_serialized_snapshot pg_lsn,
+    OUT next_phase_at xid,
+    OUT committed_count int4,
+    OUT committed_xip xid[],
+    OUT catchange_count int4,
+    OUT catchange_xip xid[]
+)
+AS 'MODULE_PATHNAME', 'pg_get_logical_snapshot_info'
+LANGUAGE C STRICT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_logical_snapshot_info(text) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_logical_snapshot_info(text) TO pg_read_server_files;
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.c b/contrib/pg_logicalinspect/pg_logicalinspect.c
new file mode 100644
index 0000000000..675760e686
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.c
@@ -0,0 +1,167 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_logicalinspect.c
+ *		  Functions to inspect contents of PostgreSQL logical snapshots
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ *		  contrib/pg_logicalinspect/pg_logicalinspect.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "funcapi.h"
+#include "replication/snapbuild_internal.h"
+#include "utils/array.h"
+#include "utils/builtins.h"
+#include "utils/pg_lsn.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_meta);
+PG_FUNCTION_INFO_V1(pg_get_logical_snapshot_info);
+
+/* Return the description of SnapBuildState */
+static const char *
+get_snapbuild_state_desc(SnapBuildState state)
+{
+	const char *stateDesc = "unknown state";
+
+	switch (state)
+	{
+		case SNAPBUILD_START:
+			stateDesc = "start";
+			break;
+		case SNAPBUILD_BUILDING_SNAPSHOT:
+			stateDesc = "building";
+			break;
+		case SNAPBUILD_FULL_SNAPSHOT:
+			stateDesc = "full";
+			break;
+		case SNAPBUILD_CONSISTENT:
+			stateDesc = "consistent";
+			break;
+	}
+
+	return stateDesc;
+}
+
+/*
+ * Retrieve the logical snapshot file metadata.
+ */
+Datum
+pg_get_logical_snapshot_meta(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_META_COLS 3
+	SnapBuildOnDisk ondisk;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_META_COLS] = {0};
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_META_COLS] = {0};
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+	int			i = 0;
+	text	   *filename_t = PG_GETARG_TEXT_PP(0);
+
+	/* Build a tuple descriptor for our result type */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	sprintf(path, "%s/%s",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			text_to_cstring(filename_t));
+
+	/* Validate and restore the snapshot to 'ondisk' */
+	SnapBuildRestoreSnapshot(&ondisk, path, CurrentMemoryContext, false);
+
+	values[i++] = UInt32GetDatum(ondisk.magic);
+	values[i++] = Int64GetDatum((int64) ondisk.checksum);
+	values[i++] = UInt32GetDatum(ondisk.version);
+
+	Assert(i == PG_GET_LOGICAL_SNAPSHOT_META_COLS);
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_META_COLS
+}
+
+Datum
+pg_get_logical_snapshot_info(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_INFO_COLS 14
+	SnapBuildOnDisk ondisk;
+	HeapTuple	tuple;
+	Datum		values[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS] = {0};
+	bool		nulls[PG_GET_LOGICAL_SNAPSHOT_INFO_COLS] = {0};
+	TupleDesc	tupdesc;
+	char		path[MAXPGPATH];
+	int			i = 0;
+	text	   *filename_t = PG_GETARG_TEXT_PP(0);
+
+	/* Build a tuple descriptor for our result type */
+	if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+		elog(ERROR, "return type must be a row type");
+
+	sprintf(path, "%s/%s",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			text_to_cstring(filename_t));
+
+	/* Validate and restore the snapshot to 'ondisk' */
+	SnapBuildRestoreSnapshot(&ondisk, path, CurrentMemoryContext, false);
+
+	values[i++] = CStringGetTextDatum(get_snapbuild_state_desc(ondisk.builder.state));
+	values[i++] = TransactionIdGetDatum(ondisk.builder.xmin);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.xmax);
+	values[i++] = LSNGetDatum(ondisk.builder.start_decoding_at);
+	values[i++] = LSNGetDatum(ondisk.builder.two_phase_at);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.initial_xmin_horizon);
+	values[i++] = BoolGetDatum(ondisk.builder.building_full_snapshot);
+	values[i++] = BoolGetDatum(ondisk.builder.in_slot_creation);
+	values[i++] = LSNGetDatum(ondisk.builder.last_serialized_snapshot);
+	values[i++] = TransactionIdGetDatum(ondisk.builder.next_phase_at);
+
+	values[i++] = UInt32GetDatum(ondisk.builder.committed.xcnt);
+	if (ondisk.builder.committed.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.committed.xcnt * sizeof(Datum));
+
+		for (int j = 0; j < ondisk.builder.committed.xcnt; j++)
+			arrayelems[j] = TransactionIdGetDatum(ondisk.builder.committed.xip[j]);
+
+		values[i++] = PointerGetDatum(construct_array_builtin(arrayelems,
+															  ondisk.builder.committed.xcnt,
+															  XIDOID));
+	}
+	else
+		nulls[i++] = true;
+
+	values[i++] = UInt32GetDatum(ondisk.builder.catchange.xcnt);
+	if (ondisk.builder.catchange.xcnt > 0)
+	{
+		Datum	   *arrayelems;
+
+		arrayelems = (Datum *) palloc(ondisk.builder.catchange.xcnt * sizeof(Datum));
+
+		for (int j = 0; j < ondisk.builder.catchange.xcnt; j++)
+			arrayelems[j] = TransactionIdGetDatum(ondisk.builder.catchange.xip[j]);
+
+		values[i++] = PointerGetDatum(construct_array_builtin(arrayelems,
+															  ondisk.builder.catchange.xcnt,
+															  XIDOID));
+	}
+	else
+		nulls[i++] = true;
+
+	Assert(i == PG_GET_LOGICAL_SNAPSHOT_INFO_COLS);
+
+	tuple = heap_form_tuple(tupdesc, values, nulls);
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+
+#undef PG_GET_LOGICAL_SNAPSHOT_INFO_COLS
+}
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.control b/contrib/pg_logicalinspect/pg_logicalinspect.control
new file mode 100644
index 0000000000..b4a70e57ba
--- /dev/null
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.control
@@ -0,0 +1,5 @@
+# pg_logicalinspect extension
+comment = 'functions to inspect logical decoding components'
+default_version = '1.0'
+module_pathname = '$libdir/pg_logicalinspect'
+relocatable = true
diff --git a/contrib/pg_logicalinspect/specs/logical_inspect.spec b/contrib/pg_logicalinspect/specs/logical_inspect.spec
new file mode 100644
index 0000000000..9851a6c18e
--- /dev/null
+++ b/contrib/pg_logicalinspect/specs/logical_inspect.spec
@@ -0,0 +1,34 @@
+# Test the pg_logicalinspect functions: that needs some permutation to
+# ensure that we are creating multiple logical snapshots and that one of them
+# contains ongoing catalogs changes.
+setup
+{
+    DROP TABLE IF EXISTS tbl1;
+    CREATE TABLE tbl1 (val1 integer, val2 integer);
+    CREATE EXTENSION pg_logicalinspect;
+}
+
+teardown
+{
+    DROP TABLE tbl1;
+    SELECT 'stop' FROM pg_drop_replication_slot('isolation_slot');
+    DROP EXTENSION pg_logicalinspect;
+}
+
+session "s0"
+setup { SET synchronous_commit=on; }
+step "s0_init" { SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding'); }
+step "s0_begin" { BEGIN; }
+step "s0_savepoint" { SAVEPOINT sp1; }
+step "s0_truncate" { TRUNCATE tbl1; }
+step "s0_insert" { INSERT INTO tbl1 VALUES (1); }
+step "s0_commit" { COMMIT; }
+
+session "s1"
+setup { SET synchronous_commit=on; }
+step "s1_checkpoint" { CHECKPOINT; }
+step "s1_get_changes" { SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0'); }
+step "s1_get_logical_snapshot_meta" { SELECT COUNT(meta.*) from pg_ls_logicalsnapdir(), pg_get_logical_snapshot_meta(name) as meta;}
+step "s1_get_logical_snapshot_info" { SELECT info.state, info.catchange_count, array_length(info.catchange_xip,1) AS catchange_array_length, info.committed_count, array_length(info.committed_xip,1) AS committed_array_length FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info ORDER BY 2; }
+
+permutation "s0_init" "s0_begin" "s0_savepoint" "s0_truncate" "s1_checkpoint" "s1_get_changes" "s0_commit" "s0_begin" "s0_insert" "s1_checkpoint" "s1_get_changes" "s0_commit" "s1_get_changes" "s1_get_logical_snapshot_info" "s1_get_logical_snapshot_meta"
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index 44639a8dca..7c381949a5 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -154,6 +154,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
  &pgbuffercache;
  &pgcrypto;
  &pgfreespacemap;
+ &pglogicalinspect;
  &pgprewarm;
  &pgrowlocks;
  &pgstatstatements;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index a7ff5f8264..66e6dccd4c 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -143,6 +143,7 @@
 <!ENTITY pgbuffercache   SYSTEM "pgbuffercache.sgml">
 <!ENTITY pgcrypto        SYSTEM "pgcrypto.sgml">
 <!ENTITY pgfreespacemap  SYSTEM "pgfreespacemap.sgml">
+<!ENTITY pglogicalinspect  SYSTEM "pglogicalinspect.sgml">
 <!ENTITY pgprewarm       SYSTEM "pgprewarm.sgml">
 <!ENTITY pgrowlocks      SYSTEM "pgrowlocks.sgml">
 <!ENTITY pgstatstatements SYSTEM "pgstatstatements.sgml">
diff --git a/doc/src/sgml/pglogicalinspect.sgml b/doc/src/sgml/pglogicalinspect.sgml
new file mode 100644
index 0000000000..4b111f9611
--- /dev/null
+++ b/doc/src/sgml/pglogicalinspect.sgml
@@ -0,0 +1,143 @@
+<!-- doc/src/sgml/pglogicalinspect.sgml -->
+
+<sect1 id="pglogicalinspect" xreflabel="pg_logicalinspect">
+ <title>pg_logicalinspect &mdash; logical decoding components inspection</title>
+
+ <indexterm zone="pglogicalinspect">
+  <primary>pg_logicalinspect</primary>
+ </indexterm>
+
+ <para>
+  The <filename>pg_logicalinspect</filename> module provides SQL functions
+  that allow you to inspect the contents of logical decoding components. It
+  allows the inspection of serialized logical snapshots of a running
+  <productname>PostgreSQL</productname> database cluster, which is useful
+  for debugging or educational purposes.
+ </para>
+
+ <para>
+  By default, use of these functions is restricted to superusers and members of
+  the <literal>pg_read_server_files</literal> role. Access may be granted by
+  superusers to others using <command>GRANT</command>.
+ </para>
+
+ <sect2 id="pglogicalinspect-funcs">
+  <title>Functions</title>
+
+  <variablelist>
+   <varlistentry id="pglogicalinspect-funcs-pg-get-logical-snapshot-meta">
+    <term>
+     <function>pg_get_logical_snapshot_meta(filename text) returns record</function>
+    </term>
+
+    <listitem>
+     <para>
+      Gets logical snapshot metadata about a snapshot file that is located in
+      the server's <filename>pg_logical/snapshots</filename> directory.
+      The <replaceable>filename</replaceable> argument represents the snapshot
+      file name.
+      For example:
+<screen>
+postgres=# SELECT * FROM pg_ls_logicalsnapdir();
+-[ RECORD 1 ]+-----------------------
+name         | 0-40796E18.snap
+size         | 152
+modification | 2024-08-14 16:36:32+00
+
+postgres=# SELECT * FROM pg_get_logical_snapshot_meta('0-40796E18.snap');
+-[ RECORD 1 ]--------
+magic    | 1369563137
+checksum | 1028045905
+version  | 6
+
+postgres=# SELECT ss.name, meta.* FROM pg_ls_logicalsnapdir() AS ss,
+pg_get_logical_snapshot_meta(ss.name) AS meta;
+-[ RECORD 1 ]-------------
+name     | 0-40796E18.snap
+magic    | 1369563137
+checksum | 1028045905
+version  | 6
+</screen>
+     </para>
+     <para>
+      If <replaceable>filename</replaceable> does not match a snapshot file, the
+      function raises an error.
+     </para>
+    </listitem>
+   </varlistentry>
+
+   <varlistentry id="pglogicalinspect-funcs-pg-get-logical-snapshot-info">
+    <term>
+     <function>pg_get_logical_snapshot_info(filename text) returns record</function>
+    </term>
+
+    <listitem>
+     <para>
+      Gets logical snapshot information about a snapshot file that is located in
+      the server's <filename>pg_logical/snapshots</filename> directory.
+      The <replaceable>filename</replaceable> argument represents the snapshot
+      file name.
+      For example:
+<screen>
+postgres=# SELECT * FROM pg_ls_logicalsnapdir();
+-[ RECORD 1 ]+-----------------------
+name         | 0-40796E18.snap
+size         | 152
+modification | 2024-08-14 16:36:32+00
+
+postgres=# SELECT * FROM pg_get_logical_snapshot_info('0-40796E18.snap');
+-[ RECORD 1 ]------------+-----------
+state                    | consistent
+xmin                     | 751
+xmax                     | 751
+start_decoding_at        | 0/40796AF8
+two_phase_at             | 0/40796AF8
+initial_xmin_horizon     | 0
+building_full_snapshot   | f
+in_slot_creation         | f
+last_serialized_snapshot | 0/0
+next_phase_at            | 0
+committed_count          | 0
+committed_xip            |
+catchange_count          | 2
+catchange_xip            | {751,752}
+
+postgres=# SELECT ss.name, info.* FROM pg_ls_logicalsnapdir() AS ss,
+pg_get_logical_snapshot_info(ss.name) AS info;
+-[ RECORD 1 ]------------+----------------
+name                     | 0-40796E18.snap
+state                    | consistent
+xmin                     | 751
+xmax                     | 751
+start_decoding_at        | 0/40796AF8
+two_phase_at             | 0/40796AF8
+initial_xmin_horizon     | 0
+building_full_snapshot   | f
+in_slot_creation         | f
+last_serialized_snapshot | 0/0
+next_phase_at            | 0
+committed_count          | 0
+committed_xip            |
+catchange_count          | 2
+catchange_xip            | {751,752}
+</screen>
+     </para>
+     <para>
+      If <replaceable>filename</replaceable> does not match a snapshot file, the
+      function raises an error.
+     </para>
+    </listitem>
+   </varlistentry>
+
+  </variablelist>
+ </sect2>
+
+ <sect2 id="pglogicalinspect-author">
+  <title>Author</title>
+
+  <para>
+   Bertrand Drouvot <email>bertranddrouvot.pg@gmail.com</email>
+  </para>
+ </sect2>
+
+</sect1>
diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c
index b9df8c0a02..a6a4da3266 100644
--- a/src/backend/replication/logical/snapbuild.c
+++ b/src/backend/replication/logical/snapbuild.c
@@ -1684,34 +1684,31 @@ out:
 }
 
 /*
- * Restore a snapshot into 'builder' if previously one has been stored at the
- * location indicated by 'lsn'. Returns true if successful, false otherwise.
+ * Restore the logical snapshot file contents to 'ondisk'.
+ *
+ * 'context' is the memory context where the catalog modifying/committed xid
+ * will live.
+ * If 'missing_ok' is true, will not throw an error if the file is not found.
  */
-static bool
-SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
+bool
+SnapBuildRestoreSnapshot(SnapBuildOnDisk *ondisk, const char *path,
+						 MemoryContext context, bool missing_ok)
 {
-	SnapBuildOnDisk ondisk;
 	int			fd;
-	char		path[MAXPGPATH];
-	Size		sz;
 	pg_crc32c	checksum;
-
-	/* no point in loading a snapshot if we're already there */
-	if (builder->state == SNAPBUILD_CONSISTENT)
-		return false;
-
-	sprintf(path, "%s/%X-%X.snap",
-			PG_LOGICAL_SNAPSHOTS_DIR,
-			LSN_FORMAT_ARGS(lsn));
+	Size		sz;
 
 	fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
 
-	if (fd < 0 && errno == ENOENT)
-		return false;
-	else if (fd < 0)
+	if (fd < 0)
+	{
+		if (missing_ok && errno == ENOENT)
+			return false;
+
 		ereport(ERROR,
 				(errcode_for_file_access(),
 				 errmsg("could not open file \"%s\": %m", path)));
+	}
 
 	/* ----
 	 * Make sure the snapshot had been stored safely to disk, that's normally
@@ -1724,47 +1721,46 @@ SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
 	fsync_fname(path, false);
 	fsync_fname(PG_LOGICAL_SNAPSHOTS_DIR, true);
 
-
 	/* read statically sized portion of snapshot */
-	SnapBuildRestoreContents(fd, (char *) &ondisk, SnapBuildOnDiskConstantSize, path);
+	SnapBuildRestoreContents(fd, (char *) ondisk, SnapBuildOnDiskConstantSize, path);
 
-	if (ondisk.magic != SNAPBUILD_MAGIC)
+	if (ondisk->magic != SNAPBUILD_MAGIC)
 		ereport(ERROR,
 				(errcode(ERRCODE_DATA_CORRUPTED),
 				 errmsg("snapbuild state file \"%s\" has wrong magic number: %u instead of %u",
-						path, ondisk.magic, SNAPBUILD_MAGIC)));
+						path, ondisk->magic, SNAPBUILD_MAGIC)));
 
-	if (ondisk.version != SNAPBUILD_VERSION)
+	if (ondisk->version != SNAPBUILD_VERSION)
 		ereport(ERROR,
 				(errcode(ERRCODE_DATA_CORRUPTED),
 				 errmsg("snapbuild state file \"%s\" has unsupported version: %u instead of %u",
-						path, ondisk.version, SNAPBUILD_VERSION)));
+						path, ondisk->version, SNAPBUILD_VERSION)));
 
 	INIT_CRC32C(checksum);
 	COMP_CRC32C(checksum,
-				((char *) &ondisk) + SnapBuildOnDiskNotChecksummedSize,
+				((char *) ondisk) + SnapBuildOnDiskNotChecksummedSize,
 				SnapBuildOnDiskConstantSize - SnapBuildOnDiskNotChecksummedSize);
 
 	/* read SnapBuild */
-	SnapBuildRestoreContents(fd, (char *) &ondisk.builder, sizeof(SnapBuild), path);
-	COMP_CRC32C(checksum, &ondisk.builder, sizeof(SnapBuild));
+	SnapBuildRestoreContents(fd, (char *) &ondisk->builder, sizeof(SnapBuild), path);
+	COMP_CRC32C(checksum, &ondisk->builder, sizeof(SnapBuild));
 
 	/* restore committed xacts information */
-	if (ondisk.builder.committed.xcnt > 0)
+	if (ondisk->builder.committed.xcnt > 0)
 	{
-		sz = sizeof(TransactionId) * ondisk.builder.committed.xcnt;
-		ondisk.builder.committed.xip = MemoryContextAllocZero(builder->context, sz);
-		SnapBuildRestoreContents(fd, (char *) ondisk.builder.committed.xip, sz, path);
-		COMP_CRC32C(checksum, ondisk.builder.committed.xip, sz);
+		sz = sizeof(TransactionId) * ondisk->builder.committed.xcnt;
+		ondisk->builder.committed.xip = MemoryContextAllocZero(context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.committed.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.committed.xip, sz);
 	}
 
 	/* restore catalog modifying xacts information */
-	if (ondisk.builder.catchange.xcnt > 0)
+	if (ondisk->builder.catchange.xcnt > 0)
 	{
-		sz = sizeof(TransactionId) * ondisk.builder.catchange.xcnt;
-		ondisk.builder.catchange.xip = MemoryContextAllocZero(builder->context, sz);
-		SnapBuildRestoreContents(fd, (char *) ondisk.builder.catchange.xip, sz, path);
-		COMP_CRC32C(checksum, ondisk.builder.catchange.xip, sz);
+		sz = sizeof(TransactionId) * ondisk->builder.catchange.xcnt;
+		ondisk->builder.catchange.xip = MemoryContextAllocZero(context, sz);
+		SnapBuildRestoreContents(fd, (char *) ondisk->builder.catchange.xip, sz, path);
+		COMP_CRC32C(checksum, ondisk->builder.catchange.xip, sz);
 	}
 
 	if (CloseTransientFile(fd) != 0)
@@ -1775,11 +1771,36 @@ SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
 	FIN_CRC32C(checksum);
 
 	/* verify checksum of what we've read */
-	if (!EQ_CRC32C(checksum, ondisk.checksum))
+	if (!EQ_CRC32C(checksum, ondisk->checksum))
 		ereport(ERROR,
 				(errcode(ERRCODE_DATA_CORRUPTED),
 				 errmsg("checksum mismatch for snapbuild state file \"%s\": is %u, should be %u",
-						path, checksum, ondisk.checksum)));
+						path, checksum, ondisk->checksum)));
+
+	return true;
+}
+
+/*
+ * Restore a snapshot into 'builder' if previously one has been stored at the
+ * location indicated by 'lsn'. Returns true if successful, false otherwise.
+ */
+static bool
+SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
+{
+	SnapBuildOnDisk ondisk;
+	char		path[MAXPGPATH];
+
+	/* no point in loading a snapshot if we're already there */
+	if (builder->state == SNAPBUILD_CONSISTENT)
+		return false;
+
+	sprintf(path, "%s/%X-%X.snap",
+			PG_LOGICAL_SNAPSHOTS_DIR,
+			LSN_FORMAT_ARGS(lsn));
+
+	/* validate and restore the snapshot to 'ondisk' */
+	if (!SnapBuildRestoreSnapshot(&ondisk, path, builder->context, true))
+		return false;
 
 	/*
 	 * ok, we now have a sensible snapshot here, figure out if it has more
diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c
index e5c7e57a5d..41434279c5 100644
--- a/src/backend/utils/adt/arrayfuncs.c
+++ b/src/backend/utils/adt/arrayfuncs.c
@@ -3447,6 +3447,12 @@ construct_array_builtin(Datum *elems, int nelems, Oid elmtype)
 			elmalign = TYPALIGN_SHORT;
 			break;
 
+		case XIDOID:
+			elmlen = sizeof(TransactionId);
+			elmbyval = true;
+			elmalign = TYPALIGN_INT;
+			break;
+
 		default:
 			elog(ERROR, "type %u not supported by construct_array_builtin()", elmtype);
 			/* keep compiler quiet */
diff --git a/src/include/replication/snapbuild.h b/src/include/replication/snapbuild.h
index dbb4bc2f4b..3c1454df99 100644
--- a/src/include/replication/snapbuild.h
+++ b/src/include/replication/snapbuild.h
@@ -15,6 +15,10 @@
 #include "access/xlogdefs.h"
 #include "utils/snapmgr.h"
 
+/*
+ * Please keep get_snapbuild_state_desc() (located in the pg_logicalinspect
+ * module) updated if a change needs to be made to SnapBuildState.
+ */
 typedef enum
 {
 	/*
diff --git a/src/include/replication/snapbuild_internal.h b/src/include/replication/snapbuild_internal.h
index 03719ccf2a..7134b48b96 100644
--- a/src/include/replication/snapbuild_internal.h
+++ b/src/include/replication/snapbuild_internal.h
@@ -193,4 +193,7 @@ typedef struct SnapBuildOnDisk
 	/* variable amount of TransactionIds follows */
 } SnapBuildOnDisk;
 
+extern bool SnapBuildRestoreSnapshot(SnapBuildOnDisk *ondisk, const char *path,
+									 MemoryContext context, bool missing_ok);
+
 #endif							/* SNAPBUILD_INTERNAL_H */
-- 
2.34.1

#63Peter Smith
smithpb2250@gmail.com
In reply to: Bertrand Drouvot (#62)
Re: Add contrib/pg_logicalsnapinspect

FYI - Although I did not re-apply/test the latest patchset v16*, by
visual inspection of the minor v15/v16 diffs it looks good to me.

======
Kind Regards,
Peter Smith.
Fujitsu Australia

#64Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Bertrand Drouvot (#62)
Re: Add contrib/pg_logicalsnapinspect

On Sun, Oct 13, 2024 at 11:23 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Hi,

On Mon, Oct 14, 2024 at 09:57:22AM +1100, Peter Smith wrote:

Here are some minor review comments for v15-0002.

======
contrib/pg_logicalinspect/pg_logicalinspect.c

1.
+pg_get_logical_snapshot_meta(PG_FUNCTION_ARGS)
+{
+#define PG_GET_LOGICAL_SNAPSHOT_META_COLS 3
+ SnapBuildOnDisk ondisk;
+ HeapTuple tuple;
+ Datum values[PG_GET_LOGICAL_SNAPSHOT_META_COLS] = {0};
+ bool nulls[PG_GET_LOGICAL_SNAPSHOT_META_COLS] = {0};
+ TupleDesc tupdesc;
+ char path[MAXPGPATH];
+ int i = 0;
+ text    *filename_t = PG_GETARG_TEXT_PP(0);
+
+ sprintf(path, "%s/%s",
+ PG_LOGICAL_SNAPSHOTS_DIR,
+ text_to_cstring(filename_t));
+
+ /* Build a tuple descriptor for our result type */
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
+
+ /* Validate and restore the snapshot to 'ondisk' */
+ SnapBuildRestoreSnapshot(&ondisk, path, CurrentMemoryContext, false);

The sprintf should be deferred. Could you do it after the ERROR check?

I think that makes sense, done in v16 attached.

======
src/backend/replication/logical/snapbuild.c

3.
/*
- * Restore a snapshot into 'builder' if previously one has been stored at the
- * location indicated by 'lsn'. Returns true if successful, false otherwise.
+ * Restore the logical snapshot file contents to 'ondisk'.
+ *
+ * If 'missing_ok' is true, will not throw an error if the file is not found.
+ * 'context' is the memory context where the catalog modifying/committed xid
+ * will live.
*/
-static bool
-SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
+bool
+SnapBuildRestoreSnapshot(SnapBuildOnDisk *ondisk, const char *path,
+ MemoryContext context, bool missing_ok)

nit - I think it's better to describe parameters in the same order
that they are declared.

Done in v16.

Also, include a 'path' description, so it is
not the only one omitted.

I don't think that's worth it as self explanatory IMHO.

Thank you for updating the patches!

I fixed a compiler warning by -Wtypedef-redefinition related to the
declaration of SnapBuild struct, then pushed both patches.

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com

#65Andres Freund
andres@anarazel.de
In reply to: Masahiko Sawada (#64)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On 2024-10-14 18:08:10 -0700, Masahiko Sawada wrote:

I fixed a compiler warning by -Wtypedef-redefinition related to the
declaration of SnapBuild struct, then pushed both patches.

This just failed on skink (valgrind buildfarm animal):
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink&amp;dt=2025-03-04%2017%3A35%3A01

In the last months (not sure quite how long) only the main regression tests
were running under valgrind. I fixed that, and in one of the runs since then
the above regression failure was triggered.

diff -U3 /home/bf/bf-build/skink-master/HEAD/pgsql/contrib/pg_logicalinspect/expected/logical_inspect.out /home/bf/bf-build/skink-master/HEAD/pgsql.build/testrun/pg_logicalinspect/isolation/results/logical_inspect.out
--- /home/bf/bf-build/skink-master/HEAD/pgsql/contrib/pg_logicalinspect/expected/logical_inspect.out	2024-10-15 01:07:04.632684683 +0000
+++ /home/bf/bf-build/skink-master/HEAD/pgsql.build/testrun/pg_logicalinspect/isolation/results/logical_inspect.out	2025-03-04 18:49:34.659306138 +0000
@@ -42,11 +42,12 @@
 ----------+---------------+----------------------+---------------+----------------------
 consistent|              0|                      |              2|                     2
 consistent|              2|                     2|              0|
-(2 rows)
+consistent|              2|                     2|              0|
+(3 rows)
 step s1_get_logical_snapshot_meta: SELECT COUNT(meta.*) from pg_ls_logicalsnapdir(), pg_get_logical_snapshot_meta(name) as meta;
 count
 -----
-    2
+    3
 (1 row)

Greetings,

Andres Freund

#66Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Andres Freund (#65)
Re: Add contrib/pg_logicalsnapinspect

On Tue, Mar 4, 2025 at 1:56 PM Andres Freund <andres@anarazel.de> wrote:

Hi,

On 2024-10-14 18:08:10 -0700, Masahiko Sawada wrote:

I fixed a compiler warning by -Wtypedef-redefinition related to the
declaration of SnapBuild struct, then pushed both patches.

This just failed on skink (valgrind buildfarm animal):
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink&amp;dt=2025-03-04%2017%3A35%3A01

In the last months (not sure quite how long) only the main regression tests
were running under valgrind. I fixed that, and in one of the runs since then
the above regression failure was triggered.

diff -U3 /home/bf/bf-build/skink-master/HEAD/pgsql/contrib/pg_logicalinspect/expected/logical_inspect.out /home/bf/bf-build/skink-master/HEAD/pgsql.build/testrun/pg_logicalinspect/isolation/results/logical_inspect.out
--- /home/bf/bf-build/skink-master/HEAD/pgsql/contrib/pg_logicalinspect/expected/logical_inspect.out    2024-10-15 01:07:04.632684683 +0000
+++ /home/bf/bf-build/skink-master/HEAD/pgsql.build/testrun/pg_logicalinspect/isolation/results/logical_inspect.out     2025-03-04 18:49:34.659306138 +0000
@@ -42,11 +42,12 @@
----------+---------------+----------------------+---------------+----------------------
consistent|              0|                      |              2|                     2
consistent|              2|                     2|              0|
-(2 rows)
+consistent|              2|                     2|              0|
+(3 rows)
step s1_get_logical_snapshot_meta: SELECT COUNT(meta.*) from pg_ls_logicalsnapdir(), pg_get_logical_snapshot_meta(name) as meta;
count
-----
-    2
+    3
(1 row)

Thank you for the report.

It seems that bgwriter wrote another RUNNING_XACTS record during the
test, making the logical decoding write an extra snapshot on the disk.

One way to stabilize the regression test would be that we check if
there is a serialized snapshot that has expected number of catchange
transactions and number of committed transactions, instead of dumping
all serialized snapshots.

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com

#67Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: Masahiko Sawada (#66)
1 attachment(s)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Tue, Mar 04, 2025 at 10:25:57PM -0800, Masahiko Sawada wrote:

On Tue, Mar 4, 2025 at 1:56 PM Andres Freund <andres@anarazel.de> wrote:

Thank you for the report.

+1

It seems that bgwriter wrote another RUNNING_XACTS record during the
test, making the logical decoding write an extra snapshot on the disk.

Yup.

One way to stabilize the regression test would be that we check if
there is a serialized snapshot that has expected number of catchange
transactions and number of committed transactions, instead of dumping
all serialized snapshots.

Agree, PFA a patch doing so.

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachments:

v1-0001-Modify-pg_logicalinspect-isolation-test.patchtext/x-diff; charset=us-asciiDownload
From d590d3184d4345908f1b033aa8a5d19cf98e88ba Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Wed, 5 Mar 2025 07:06:58 +0000
Subject: [PATCH v1] Modify pg_logicalinspect isolation test

The previous version was relying on the fact that the test produces exactly
2 snapshots on disk, while in fact it can produce more. Changing the test knowing
that at least 2 snapshots are generated.

Per buildfarm member skink.
---
 .../expected/logical_inspect.out              | 29 +++++++++++--------
 .../specs/logical_inspect.spec                |  7 +++--
 2 files changed, 21 insertions(+), 15 deletions(-)
  57.3% contrib/pg_logicalinspect/expected/
  42.6% contrib/pg_logicalinspect/specs/

diff --git a/contrib/pg_logicalinspect/expected/logical_inspect.out b/contrib/pg_logicalinspect/expected/logical_inspect.out
index d95efa4d1e5..ecba05b9ab1 100644
--- a/contrib/pg_logicalinspect/expected/logical_inspect.out
+++ b/contrib/pg_logicalinspect/expected/logical_inspect.out
@@ -1,6 +1,6 @@
 Parsed test spec with 2 sessions
 
-starting permutation: s0_init s0_begin s0_savepoint s0_truncate s1_checkpoint s1_get_changes s0_commit s0_begin s0_insert s1_checkpoint s1_get_changes s0_commit s1_get_changes s1_get_logical_snapshot_info s1_get_logical_snapshot_meta
+starting permutation: s0_init s0_begin s0_savepoint s0_truncate s1_checkpoint s1_get_changes s0_commit s0_begin s0_insert s1_checkpoint s1_get_changes s0_commit s1_get_changes s1_get_logical_snapshot_info_catchange s1_get_logical_snapshot_info_committed s1_get_logical_snapshot_meta
 step s0_init: SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding');
 ?column?
 --------
@@ -37,16 +37,21 @@ table public.tbl1: INSERT: val1[integer]:1 val2[integer]:null
 COMMIT                                                       
 (3 rows)
 
-step s1_get_logical_snapshot_info: SELECT info.state, info.catchange_count, array_length(info.catchange_xip,1) AS catchange_array_length, info.committed_count, array_length(info.committed_xip,1) AS committed_array_length FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info ORDER BY 2;
-state     |catchange_count|catchange_array_length|committed_count|committed_array_length
-----------+---------------+----------------------+---------------+----------------------
-consistent|              0|                      |              2|                     2
-consistent|              2|                     2|              0|                      
-(2 rows)
-
-step s1_get_logical_snapshot_meta: SELECT COUNT(meta.*) from pg_ls_logicalsnapdir(), pg_get_logical_snapshot_meta(name) as meta;
-count
------
-    2
+step s1_get_logical_snapshot_info_catchange: SELECT count(*) > 0 as has_catchange FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info where info.catchange_count = 2 and array_length(info.catchange_xip,1) = 2 and info.committed_count = 0;
+has_catchange
+-------------
+t            
+(1 row)
+
+step s1_get_logical_snapshot_info_committed: SELECT count(*) > 0 as has_committed FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info where info.committed_count = 2 and array_length(info.committed_xip,1) = 2 and info.catchange_count = 0;
+has_committed
+-------------
+t            
+(1 row)
+
+step s1_get_logical_snapshot_meta: SELECT COUNT(meta.*) > 1 AS has_meta from pg_ls_logicalsnapdir(), pg_get_logical_snapshot_meta(name) as meta;
+has_meta
+--------
+t       
 (1 row)
 
diff --git a/contrib/pg_logicalinspect/specs/logical_inspect.spec b/contrib/pg_logicalinspect/specs/logical_inspect.spec
index 9851a6c18e4..673d2f5ed0a 100644
--- a/contrib/pg_logicalinspect/specs/logical_inspect.spec
+++ b/contrib/pg_logicalinspect/specs/logical_inspect.spec
@@ -28,7 +28,8 @@ session "s1"
 setup { SET synchronous_commit=on; }
 step "s1_checkpoint" { CHECKPOINT; }
 step "s1_get_changes" { SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0'); }
-step "s1_get_logical_snapshot_meta" { SELECT COUNT(meta.*) from pg_ls_logicalsnapdir(), pg_get_logical_snapshot_meta(name) as meta;}
-step "s1_get_logical_snapshot_info" { SELECT info.state, info.catchange_count, array_length(info.catchange_xip,1) AS catchange_array_length, info.committed_count, array_length(info.committed_xip,1) AS committed_array_length FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info ORDER BY 2; }
+step "s1_get_logical_snapshot_meta" { SELECT COUNT(meta.*) > 1 AS has_meta from pg_ls_logicalsnapdir(), pg_get_logical_snapshot_meta(name) as meta; }
+step "s1_get_logical_snapshot_info_catchange" { SELECT count(*) > 0 as has_catchange FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info where info.catchange_count = 2 and array_length(info.catchange_xip,1) = 2 and info.committed_count = 0; }
+step "s1_get_logical_snapshot_info_committed" { SELECT count(*) > 0 as has_committed FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info where info.committed_count = 2 and array_length(info.committed_xip,1) = 2 and info.catchange_count = 0; }
 
-permutation "s0_init" "s0_begin" "s0_savepoint" "s0_truncate" "s1_checkpoint" "s1_get_changes" "s0_commit" "s0_begin" "s0_insert" "s1_checkpoint" "s1_get_changes" "s0_commit" "s1_get_changes" "s1_get_logical_snapshot_info" "s1_get_logical_snapshot_meta"
+permutation "s0_init" "s0_begin" "s0_savepoint" "s0_truncate" "s1_checkpoint" "s1_get_changes" "s0_commit" "s0_begin" "s0_insert" "s1_checkpoint" "s1_get_changes" "s0_commit" "s1_get_changes" "s1_get_logical_snapshot_info_catchange" "s1_get_logical_snapshot_info_committed" "s1_get_logical_snapshot_meta"
-- 
2.34.1

#68Amit Kapila
amit.kapila16@gmail.com
In reply to: Bertrand Drouvot (#67)
Re: Add contrib/pg_logicalsnapinspect

On Wed, Mar 5, 2025 at 12:47 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Hi,

On Tue, Mar 04, 2025 at 10:25:57PM -0800, Masahiko Sawada wrote:

On Tue, Mar 4, 2025 at 1:56 PM Andres Freund <andres@anarazel.de> wrote:

Thank you for the report.

+1

It seems that bgwriter wrote another RUNNING_XACTS record during the
test, making the logical decoding write an extra snapshot on the disk.

Yup.

One way to stabilize the regression test would be that we check if
there is a serialized snapshot that has expected number of catchange
transactions and number of committed transactions, instead of dumping
all serialized snapshots.

Agree, PFA a patch doing so.

It would be better if you could add a few comments atop the
permutation line to explain the working of the test. We have it for
other decoding-related tests. See
test_decoding/specs/subxact_without_top.spec for reference.

--
With Regards,
Amit Kapila.

#69Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: Amit Kapila (#68)
1 attachment(s)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Wed, Mar 05, 2025 at 02:42:15PM +0530, Amit Kapila wrote:

On Wed, Mar 5, 2025 at 12:47 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Agree, PFA a patch doing so.

It would be better if you could add a few comments atop the
permutation line to explain the working of the test.

yeah makes sense. Done in the attached, and bonus point I realized that the
test could be simplified (so, removing useless steps in passing).

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachments:

v2-0001-Modify-pg_logicalinspect-isolation-test.patchtext/x-diff; charset=us-asciiDownload
From 6154e847e4e5c2c7eb816d4302a4704d5a690954 Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Wed, 5 Mar 2025 07:06:58 +0000
Subject: [PATCH v2] Modify pg_logicalinspect isolation test

The previous version was relying on the fact that the test produces exactly
2 snapshots on disk, while in fact it can produce more. Changing the test knowing
that at least 2 snapshots are generated.

In passing, removing useless steps and adding some comments.

Per buildfarm member skink.
---
 .../expected/logical_inspect.out              | 36 ++++++++-----------
 .../specs/logical_inspect.spec                | 11 +++---
 2 files changed, 22 insertions(+), 25 deletions(-)
  58.4% contrib/pg_logicalinspect/expected/
  41.5% contrib/pg_logicalinspect/specs/

diff --git a/contrib/pg_logicalinspect/expected/logical_inspect.out b/contrib/pg_logicalinspect/expected/logical_inspect.out
index d95efa4d1e5..b7d01dbb68b 100644
--- a/contrib/pg_logicalinspect/expected/logical_inspect.out
+++ b/contrib/pg_logicalinspect/expected/logical_inspect.out
@@ -1,6 +1,6 @@
 Parsed test spec with 2 sessions
 
-starting permutation: s0_init s0_begin s0_savepoint s0_truncate s1_checkpoint s1_get_changes s0_commit s0_begin s0_insert s1_checkpoint s1_get_changes s0_commit s1_get_changes s1_get_logical_snapshot_info s1_get_logical_snapshot_meta
+starting permutation: s0_init s0_begin s0_savepoint s0_truncate s1_checkpoint s1_get_changes s0_commit s1_checkpoint s1_get_changes s1_get_logical_snapshot_info_catchange s1_get_logical_snapshot_info_committed s1_get_logical_snapshot_meta
 step s0_init: SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding');
 ?column?
 --------
@@ -17,8 +17,6 @@ data
 (0 rows)
 
 step s0_commit: COMMIT;
-step s0_begin: BEGIN;
-step s0_insert: INSERT INTO tbl1 VALUES (1);
 step s1_checkpoint: CHECKPOINT;
 step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
 data                                   
@@ -28,25 +26,21 @@ table public.tbl1: TRUNCATE: (no-flags)
 COMMIT                                 
 (3 rows)
 
-step s0_commit: COMMIT;
-step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
-data                                                         
--------------------------------------------------------------
-BEGIN                                                        
-table public.tbl1: INSERT: val1[integer]:1 val2[integer]:null
-COMMIT                                                       
-(3 rows)
+step s1_get_logical_snapshot_info_catchange: SELECT count(*) > 0 as has_catchange FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info where info.catchange_count = 2 and array_length(info.catchange_xip,1) = 2 and info.committed_count = 0;
+has_catchange
+-------------
+t            
+(1 row)
 
-step s1_get_logical_snapshot_info: SELECT info.state, info.catchange_count, array_length(info.catchange_xip,1) AS catchange_array_length, info.committed_count, array_length(info.committed_xip,1) AS committed_array_length FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info ORDER BY 2;
-state     |catchange_count|catchange_array_length|committed_count|committed_array_length
-----------+---------------+----------------------+---------------+----------------------
-consistent|              0|                      |              2|                     2
-consistent|              2|                     2|              0|                      
-(2 rows)
+step s1_get_logical_snapshot_info_committed: SELECT count(*) > 0 as has_committed FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info where info.committed_count = 2 and array_length(info.committed_xip,1) = 2 and info.catchange_count = 0;
+has_committed
+-------------
+t            
+(1 row)
 
-step s1_get_logical_snapshot_meta: SELECT COUNT(meta.*) from pg_ls_logicalsnapdir(), pg_get_logical_snapshot_meta(name) as meta;
-count
------
-    2
+step s1_get_logical_snapshot_meta: SELECT COUNT(meta.*) > 1 AS has_meta from pg_ls_logicalsnapdir(), pg_get_logical_snapshot_meta(name) as meta;
+has_meta
+--------
+t       
 (1 row)
 
diff --git a/contrib/pg_logicalinspect/specs/logical_inspect.spec b/contrib/pg_logicalinspect/specs/logical_inspect.spec
index 9851a6c18e4..631daf5db6c 100644
--- a/contrib/pg_logicalinspect/specs/logical_inspect.spec
+++ b/contrib/pg_logicalinspect/specs/logical_inspect.spec
@@ -21,14 +21,17 @@ step "s0_init" { SELECT 'init' FROM pg_create_logical_replication_slot('isolatio
 step "s0_begin" { BEGIN; }
 step "s0_savepoint" { SAVEPOINT sp1; }
 step "s0_truncate" { TRUNCATE tbl1; }
-step "s0_insert" { INSERT INTO tbl1 VALUES (1); }
 step "s0_commit" { COMMIT; }
 
 session "s1"
 setup { SET synchronous_commit=on; }
 step "s1_checkpoint" { CHECKPOINT; }
 step "s1_get_changes" { SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0'); }
-step "s1_get_logical_snapshot_meta" { SELECT COUNT(meta.*) from pg_ls_logicalsnapdir(), pg_get_logical_snapshot_meta(name) as meta;}
-step "s1_get_logical_snapshot_info" { SELECT info.state, info.catchange_count, array_length(info.catchange_xip,1) AS catchange_array_length, info.committed_count, array_length(info.committed_xip,1) AS committed_array_length FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info ORDER BY 2; }
+step "s1_get_logical_snapshot_meta" { SELECT COUNT(meta.*) > 1 AS has_meta from pg_ls_logicalsnapdir(), pg_get_logical_snapshot_meta(name) as meta; }
+step "s1_get_logical_snapshot_info_catchange" { SELECT count(*) > 0 as has_catchange FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info where info.catchange_count = 2 and array_length(info.catchange_xip,1) = 2 and info.committed_count = 0; }
+step "s1_get_logical_snapshot_info_committed" { SELECT count(*) > 0 as has_committed FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info where info.committed_count = 2 and array_length(info.committed_xip,1) = 2 and info.catchange_count = 0; }
 
-permutation "s0_init" "s0_begin" "s0_savepoint" "s0_truncate" "s1_checkpoint" "s1_get_changes" "s0_commit" "s0_begin" "s0_insert" "s1_checkpoint" "s1_get_changes" "s0_commit" "s1_get_changes" "s1_get_logical_snapshot_info" "s1_get_logical_snapshot_meta"
+# The first get_changes produces (at least) one snapshot that contains 2 catchanges
+# (the truncate and its parent transaction). The second get_changes produces one
+# snapshot that contains the 2 transactions above as committed.
+permutation "s0_init" "s0_begin" "s0_savepoint" "s0_truncate" "s1_checkpoint" "s1_get_changes" "s0_commit" "s1_checkpoint" "s1_get_changes" "s1_get_logical_snapshot_info_catchange" "s1_get_logical_snapshot_info_committed" "s1_get_logical_snapshot_meta"
-- 
2.34.1

#70Tom Lane
tgl@sss.pgh.pa.us
In reply to: Bertrand Drouvot (#69)
Re: Add contrib/pg_logicalsnapinspect

Bertrand Drouvot <bertranddrouvot.pg@gmail.com> writes:

yeah makes sense. Done in the attached, and bonus point I realized that the
test could be simplified (so, removing useless steps in passing).

Just a side note: tayra showed two instances of this failure today
[1]: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=tayra&amp;dt=2025-03-05%2021%3A36%3A40
else recently that would make this more probable?

regards, tom lane

[1]: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=tayra&amp;dt=2025-03-05%2021%3A36%3A40
[2]: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=tayra&amp;dt=2025-03-05%2013%3A42%3A17

#71Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Tom Lane (#70)
Re: Add contrib/pg_logicalsnapinspect

On Wed, Mar 5, 2025 at 3:10 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:

Bertrand Drouvot <bertranddrouvot.pg@gmail.com> writes:

yeah makes sense. Done in the attached, and bonus point I realized that the
test could be simplified (so, removing useless steps in passing).

Just a side note: tayra showed two instances of this failure today
[1][2]. That's not using valgrind. I wonder if we changed something
else recently that would make this more probable?

I've observed the third failure. I read through recent commits but
have no idea what commit made this more probable. Comparing other
tests on the success case[1]https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=tayra&amp;dt=2025-03-05%2001%3A22%3A07 and failure case[2]https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=tayra&amp;dt=2025-03-05%2013%3A42%3A17, it seems that tayra
were slow overall. For instance, the 'build' and 'check' were 00:00:19
vs. 00:02:42 and 00:02:42 vs. 00:19:17, respectively. I'm not sure
what caused tayra to be slower overall recently.

Regards,

[1]: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=tayra&amp;dt=2025-03-05%2001%3A22%3A07
[2]: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=tayra&amp;dt=2025-03-05%2013%3A42%3A17

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com

#72Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: Masahiko Sawada (#71)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Wed, Mar 05, 2025 at 11:28:23PM -0800, Masahiko Sawada wrote:

On Wed, Mar 5, 2025 at 3:10 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:

Bertrand Drouvot <bertranddrouvot.pg@gmail.com> writes:

yeah makes sense. Done in the attached, and bonus point I realized that the
test could be simplified (so, removing useless steps in passing).

Just a side note: tayra showed two instances of this failure today
[1][2]. That's not using valgrind.

Thanks for the report!

I wonder if we changed something

else recently that would make this more probable?

I've observed the third failure.

I also did a "slow" test with the code tree at 7cdfeee320e and I can observe
the same "issue".

I'm not sure
what caused tayra to be slower overall recently.

yeah, tayra being slower is what make the test failure more probable. I'm also
not sure as to why.

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

#73Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Bertrand Drouvot (#69)
Re: Add contrib/pg_logicalsnapinspect

On Wed, Mar 5, 2025 at 4:05 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Hi,

On Wed, Mar 05, 2025 at 02:42:15PM +0530, Amit Kapila wrote:

On Wed, Mar 5, 2025 at 12:47 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Agree, PFA a patch doing so.

It would be better if you could add a few comments atop the
permutation line to explain the working of the test.

yeah makes sense. Done in the attached, and bonus point I realized that the
test could be simplified (so, removing useless steps in passing).

Thank you for the patch.

The new simplified test case can be pretty-formatted as:

init
begin
savepoint
truncate
checkpoint-1
get_changes-1
commit
checkpoint-2
get_changes-2
info_catchange check
info_committed check
meta check

IIUC if another checkpoint happens between get_change-2 and the
subsequent checks, the first snapshot would be removed during the
checkpoint, resulting in a test failure. I think we could check the
snapshot files while one transaction keeps open. The more simplified
test case would be:

init
begin
savepoint
insert(cat-change)
begin
insert(cat-change)
commit
checkpoint
get_changes
info_catchange check
info_committed check
meta check
commit

In this test case, we would have at least one serialized snapshot that
has both cat-changes and committed txns. What do you think?

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com

#74Amit Kapila
amit.kapila16@gmail.com
In reply to: Masahiko Sawada (#73)
Re: Add contrib/pg_logicalsnapinspect

On Fri, Mar 7, 2025 at 3:19 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:

On Wed, Mar 5, 2025 at 4:05 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Hi,

On Wed, Mar 05, 2025 at 02:42:15PM +0530, Amit Kapila wrote:

On Wed, Mar 5, 2025 at 12:47 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Agree, PFA a patch doing so.

It would be better if you could add a few comments atop the
permutation line to explain the working of the test.

yeah makes sense. Done in the attached, and bonus point I realized that the
test could be simplified (so, removing useless steps in passing).

Thank you for the patch.

The new simplified test case can be pretty-formatted as:

init
begin
savepoint
truncate
checkpoint-1
get_changes-1
commit
checkpoint-2
get_changes-2
info_catchange check
info_committed check
meta check

IIUC if another checkpoint happens between get_change-2 and the
subsequent checks, the first snapshot would be removed during the
checkpoint, resulting in a test failure. I think we could check the
snapshot files while one transaction keeps open. The more simplified
test case would be:

init
begin
savepoint
insert(cat-change)
begin
insert(cat-change)
commit
checkpoint
get_changes
info_catchange check
info_committed check
meta check
commit

In this test case, we would have at least one serialized snapshot that
has both cat-changes and committed txns. What do you think?

Your proposed change in the test sounds better than what we have now
but I think we should also avoid autovacuum to perform analyze as that
may add additional counts. For test_decoding, we keep
autovacuum_naptime = 1d in logical.conf file, we can either use the
same here or simply keep autovacuum off.

--
With Regards,
Amit Kapila.

#75Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: Amit Kapila (#74)
1 attachment(s)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Fri, Mar 07, 2025 at 10:26:23AM +0530, Amit Kapila wrote:

On Fri, Mar 7, 2025 at 3:19 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:

On Wed, Mar 5, 2025 at 4:05 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Hi,

On Wed, Mar 05, 2025 at 02:42:15PM +0530, Amit Kapila wrote:

On Wed, Mar 5, 2025 at 12:47 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Agree, PFA a patch doing so.

It would be better if you could add a few comments atop the
permutation line to explain the working of the test.

yeah makes sense. Done in the attached, and bonus point I realized that the
test could be simplified (so, removing useless steps in passing).

Thank you for the patch.

The new simplified test case can be pretty-formatted as:

init
begin
savepoint
truncate
checkpoint-1
get_changes-1
commit
checkpoint-2
get_changes-2
info_catchange check
info_committed check
meta check

Yes.

IIUC if another checkpoint happens between get_change-2 and the
subsequent checks, the first snapshot would be removed during the
checkpoint, resulting in a test failure.

Good catch! Yeah you're right, thanks!

I think we could check the

snapshot files while one transaction keeps open. The more simplified
test case would be:

init
begin
savepoint
insert(cat-change)
begin
insert(cat-change)
commit
checkpoint
get_changes
info_catchange check
info_committed check
meta check
commit

In this test case, we would have at least one serialized snapshot that
has both cat-changes and committed txns. What do you think?

Indeed, I think that would prevent snapshots to be removed.

The attached ends up doing:

init
begin
savepoint
truncate table1
create table table2
checkpoint
get_changes
info check
meta check
commit

As the 2 ongoing catalog changes and the committed catalog change are part of the
same snapshot, then I grouped the catchanges and committed changes checks in the
same "info check".

Your proposed change in the test sounds better than what we have now
but I think we should also avoid autovacuum to perform analyze as that
may add additional counts. For test_decoding, we keep
autovacuum_naptime = 1d in logical.conf file, we can either use the
same here or simply keep autovacuum off.

When writing the attached, I initially added extra paranoia in the tests by
using ">=", does that also address your autovacuum concern?

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachments:

v3-0001-Modify-pg_logicalinspect-isolation-test.patchtext/x-diff; charset=us-asciiDownload
From 415d707187070df88e11e9bb1059309ab102953e Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Wed, 5 Mar 2025 07:06:58 +0000
Subject: [PATCH v3] Modify pg_logicalinspect isolation test

The previous version was relying on the fact that the test produces exactly
2 snapshots on disk, while in fact it can produce more. Changing the test knowing
that at least 2 snapshots are generated.

In passing, removing useless steps and adding some comments.

Per buildfarm member skink.
---
 .../expected/logical_inspect.out              | 44 +++++--------------
 .../specs/logical_inspect.spec                | 18 +++++---
 2 files changed, 24 insertions(+), 38 deletions(-)
  57.1% contrib/pg_logicalinspect/expected/
  42.8% contrib/pg_logicalinspect/specs/

diff --git a/contrib/pg_logicalinspect/expected/logical_inspect.out b/contrib/pg_logicalinspect/expected/logical_inspect.out
index d95efa4d1e5..c86711e471d 100644
--- a/contrib/pg_logicalinspect/expected/logical_inspect.out
+++ b/contrib/pg_logicalinspect/expected/logical_inspect.out
@@ -1,6 +1,6 @@
 Parsed test spec with 2 sessions
 
-starting permutation: s0_init s0_begin s0_savepoint s0_truncate s1_checkpoint s1_get_changes s0_commit s0_begin s0_insert s1_checkpoint s1_get_changes s0_commit s1_get_changes s1_get_logical_snapshot_info s1_get_logical_snapshot_meta
+starting permutation: s0_init s0_begin s0_savepoint s0_truncate s1_create_table s1_checkpoint s1_get_changes s1_get_logical_snapshot_info s1_get_logical_snapshot_meta s0_commit
 step s0_init: SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding');
 ?column?
 --------
@@ -10,43 +10,23 @@ init
 step s0_begin: BEGIN;
 step s0_savepoint: SAVEPOINT sp1;
 step s0_truncate: TRUNCATE tbl1;
+step s1_create_table: CREATE TABLE tbl2 (val1 integer, val2 integer);
 step s1_checkpoint: CHECKPOINT;
 step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
 data
 ----
 (0 rows)
 
-step s0_commit: COMMIT;
-step s0_begin: BEGIN;
-step s0_insert: INSERT INTO tbl1 VALUES (1);
-step s1_checkpoint: CHECKPOINT;
-step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
-data                                   
----------------------------------------
-BEGIN                                  
-table public.tbl1: TRUNCATE: (no-flags)
-COMMIT                                 
-(3 rows)
-
-step s0_commit: COMMIT;
-step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
-data                                                         
--------------------------------------------------------------
-BEGIN                                                        
-table public.tbl1: INSERT: val1[integer]:1 val2[integer]:null
-COMMIT                                                       
-(3 rows)
-
-step s1_get_logical_snapshot_info: SELECT info.state, info.catchange_count, array_length(info.catchange_xip,1) AS catchange_array_length, info.committed_count, array_length(info.committed_xip,1) AS committed_array_length FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info ORDER BY 2;
-state     |catchange_count|catchange_array_length|committed_count|committed_array_length
-----------+---------------+----------------------+---------------+----------------------
-consistent|              0|                      |              2|                     2
-consistent|              2|                     2|              0|                      
-(2 rows)
+step s1_get_logical_snapshot_info: SELECT count(*) > 0 as has_info FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info where info.catchange_count >= 2 and array_length(info.catchange_xip,1) >= 2 and info.committed_count >= 1 and array_length(info.committed_xip,1) >= 1;
+has_info
+--------
+t       
+(1 row)
 
-step s1_get_logical_snapshot_meta: SELECT COUNT(meta.*) from pg_ls_logicalsnapdir(), pg_get_logical_snapshot_meta(name) as meta;
-count
------
-    2
+step s1_get_logical_snapshot_meta: SELECT COUNT(meta.*) > 0 AS has_meta from pg_ls_logicalsnapdir(), pg_get_logical_snapshot_meta(name) as meta;
+has_meta
+--------
+t       
 (1 row)
 
+step s0_commit: COMMIT;
diff --git a/contrib/pg_logicalinspect/specs/logical_inspect.spec b/contrib/pg_logicalinspect/specs/logical_inspect.spec
index 9851a6c18e4..a6bc58ae955 100644
--- a/contrib/pg_logicalinspect/specs/logical_inspect.spec
+++ b/contrib/pg_logicalinspect/specs/logical_inspect.spec
@@ -1,6 +1,6 @@
 # Test the pg_logicalinspect functions: that needs some permutation to
-# ensure that we are creating multiple logical snapshots and that one of them
-# contains ongoing catalogs changes.
+# ensure that we are creating at least one snapshot that contains ongoing and
+# committed catalogs changes.
 setup
 {
     DROP TABLE IF EXISTS tbl1;
@@ -11,6 +11,7 @@ setup
 teardown
 {
     DROP TABLE tbl1;
+    DROP TABLE tbl2;
     SELECT 'stop' FROM pg_drop_replication_slot('isolation_slot');
     DROP EXTENSION pg_logicalinspect;
 }
@@ -21,14 +22,19 @@ step "s0_init" { SELECT 'init' FROM pg_create_logical_replication_slot('isolatio
 step "s0_begin" { BEGIN; }
 step "s0_savepoint" { SAVEPOINT sp1; }
 step "s0_truncate" { TRUNCATE tbl1; }
-step "s0_insert" { INSERT INTO tbl1 VALUES (1); }
 step "s0_commit" { COMMIT; }
 
 session "s1"
 setup { SET synchronous_commit=on; }
 step "s1_checkpoint" { CHECKPOINT; }
+step "s1_create_table" { CREATE TABLE tbl2 (val1 integer, val2 integer); }
 step "s1_get_changes" { SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0'); }
-step "s1_get_logical_snapshot_meta" { SELECT COUNT(meta.*) from pg_ls_logicalsnapdir(), pg_get_logical_snapshot_meta(name) as meta;}
-step "s1_get_logical_snapshot_info" { SELECT info.state, info.catchange_count, array_length(info.catchange_xip,1) AS catchange_array_length, info.committed_count, array_length(info.committed_xip,1) AS committed_array_length FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info ORDER BY 2; }
+step "s1_get_logical_snapshot_meta" { SELECT COUNT(meta.*) > 0 AS has_meta from pg_ls_logicalsnapdir(), pg_get_logical_snapshot_meta(name) as meta; }
+step "s1_get_logical_snapshot_info" { SELECT count(*) > 0 as has_info FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info where info.catchange_count >= 2 and array_length(info.catchange_xip,1) >= 2 and info.committed_count >= 1 and array_length(info.committed_xip,1) >= 1; }
 
-permutation "s0_init" "s0_begin" "s0_savepoint" "s0_truncate" "s1_checkpoint" "s1_get_changes" "s0_commit" "s0_begin" "s0_insert" "s1_checkpoint" "s1_get_changes" "s0_commit" "s1_get_changes" "s1_get_logical_snapshot_info" "s1_get_logical_snapshot_meta"
+# s0 does not commit until the end of the test. This is needed to ensure that
+# a checkpoint will not remove any snapshots. s0 produces 2 ongoing catalog changes
+# (the truncate and its parent transaction). s1 produces a committed catalog change.
+# So that the get_changes produces (at least) one snapshot that contains 2
+# ongoing catalog changes and the committed catalog change.
+permutation "s0_init" "s0_begin" "s0_savepoint" "s0_truncate" "s1_create_table" "s1_checkpoint" "s1_get_changes" "s1_get_logical_snapshot_info" "s1_get_logical_snapshot_meta" "s0_commit"
-- 
2.34.1

#76Amit Kapila
amit.kapila16@gmail.com
In reply to: Bertrand Drouvot (#75)
Re: Add contrib/pg_logicalsnapinspect

On Fri, Mar 7, 2025 at 4:12 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

On Fri, Mar 07, 2025 at 10:26:23AM +0530, Amit Kapila wrote:

Your proposed change in the test sounds better than what we have now
but I think we should also avoid autovacuum to perform analyze as that
may add additional counts. For test_decoding, we keep
autovacuum_naptime = 1d in logical.conf file, we can either use the
same here or simply keep autovacuum off.

When writing the attached, I initially added extra paranoia in the tests by
using ">=", does that also address your autovacuum concern?

Yes, that will address the autovacuum concern.

--
With Regards,
Amit Kapila.

#77Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Bertrand Drouvot (#75)
1 attachment(s)
Re: Add contrib/pg_logicalsnapinspect

On Fri, Mar 7, 2025 at 2:42 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Hi,

On Fri, Mar 07, 2025 at 10:26:23AM +0530, Amit Kapila wrote:

On Fri, Mar 7, 2025 at 3:19 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:

On Wed, Mar 5, 2025 at 4:05 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Hi,

On Wed, Mar 05, 2025 at 02:42:15PM +0530, Amit Kapila wrote:

On Wed, Mar 5, 2025 at 12:47 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Agree, PFA a patch doing so.

It would be better if you could add a few comments atop the
permutation line to explain the working of the test.

yeah makes sense. Done in the attached, and bonus point I realized that the
test could be simplified (so, removing useless steps in passing).

Thank you for the patch.

The new simplified test case can be pretty-formatted as:

init
begin
savepoint
truncate
checkpoint-1
get_changes-1
commit
checkpoint-2
get_changes-2
info_catchange check
info_committed check
meta check

Yes.

IIUC if another checkpoint happens between get_change-2 and the
subsequent checks, the first snapshot would be removed during the
checkpoint, resulting in a test failure.

Good catch! Yeah you're right, thanks!

I think we could check the

snapshot files while one transaction keeps open. The more simplified
test case would be:

init
begin
savepoint
insert(cat-change)
begin
insert(cat-change)
commit
checkpoint
get_changes
info_catchange check
info_committed check
meta check
commit

In this test case, we would have at least one serialized snapshot that
has both cat-changes and committed txns. What do you think?

Indeed, I think that would prevent snapshots to be removed.

The attached ends up doing:

init
begin
savepoint
truncate table1
create table table2
checkpoint
get_changes
info check
meta check
commit

As the 2 ongoing catalog changes and the committed catalog change are part of the
same snapshot, then I grouped the catchanges and committed changes checks in the
same "info check".

Your proposed change in the test sounds better than what we have now
but I think we should also avoid autovacuum to perform analyze as that
may add additional counts. For test_decoding, we keep
autovacuum_naptime = 1d in logical.conf file, we can either use the
same here or simply keep autovacuum off.

When writing the attached, I initially added extra paranoia in the tests by
using ">=", does that also address your autovacuum concern?

Thank you for updating the patch. It looks mostly good to me. I've
made some cosmetic changes and attached the updated version.

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com

Attachments:

v4-0001-pg_logicalinspect-Stabilize-isolation-tests.patchapplication/octet-stream; name=v4-0001-pg_logicalinspect-Stabilize-isolation-tests.patchDownload
From c15a40c256e19c462e5e0ee8fdf868af25fa4fae Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Wed, 5 Mar 2025 07:06:58 +0000
Subject: [PATCH v4] pg_logicalinspect: Stabilize isolation tests.

The previous isolation tests did not account for the possibility that
the background writer or the checkpointer could write a RUNNING_XACTS
record, which could cause logical decoding to produce more logical
snapshots than expected.

This commit modifies the isolation tests to verify that at least one
logical snapshot contains the expected number of committed or ongoing
catalog-change transactions.

Per buildfarm member skink.

Reported-by: Andres Freund <andres@anarazel.de>
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/5qbxud4pvnvmtuoi7weiizm5hmumxaeohx4vztfhrwlfhyz6rj@buh4435mllwo
---
 .../expected/logical_inspect.out              | 44 +++++--------------
 .../specs/logical_inspect.spec                | 24 +++++++---
 2 files changed, 30 insertions(+), 38 deletions(-)

diff --git a/contrib/pg_logicalinspect/expected/logical_inspect.out b/contrib/pg_logicalinspect/expected/logical_inspect.out
index d95efa4d1e5..b343d3ad733 100644
--- a/contrib/pg_logicalinspect/expected/logical_inspect.out
+++ b/contrib/pg_logicalinspect/expected/logical_inspect.out
@@ -1,6 +1,6 @@
 Parsed test spec with 2 sessions
 
-starting permutation: s0_init s0_begin s0_savepoint s0_truncate s1_checkpoint s1_get_changes s0_commit s0_begin s0_insert s1_checkpoint s1_get_changes s0_commit s1_get_changes s1_get_logical_snapshot_info s1_get_logical_snapshot_meta
+starting permutation: s0_init s0_begin s0_savepoint s0_truncate s1_create_table s1_checkpoint s1_get_changes s1_check_snapshot_info s1_check_snapshot_meta s0_commit
 step s0_init: SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'test_decoding');
 ?column?
 --------
@@ -10,43 +10,23 @@ init
 step s0_begin: BEGIN;
 step s0_savepoint: SAVEPOINT sp1;
 step s0_truncate: TRUNCATE tbl1;
+step s1_create_table: CREATE TABLE tbl2 (val1 integer, val2 integer);
 step s1_checkpoint: CHECKPOINT;
 step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
 data
 ----
 (0 rows)
 
-step s0_commit: COMMIT;
-step s0_begin: BEGIN;
-step s0_insert: INSERT INTO tbl1 VALUES (1);
-step s1_checkpoint: CHECKPOINT;
-step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
-data                                   
----------------------------------------
-BEGIN                                  
-table public.tbl1: TRUNCATE: (no-flags)
-COMMIT                                 
-(3 rows)
-
-step s0_commit: COMMIT;
-step s1_get_changes: SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0');
-data                                                         
--------------------------------------------------------------
-BEGIN                                                        
-table public.tbl1: INSERT: val1[integer]:1 val2[integer]:null
-COMMIT                                                       
-(3 rows)
-
-step s1_get_logical_snapshot_info: SELECT info.state, info.catchange_count, array_length(info.catchange_xip,1) AS catchange_array_length, info.committed_count, array_length(info.committed_xip,1) AS committed_array_length FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info ORDER BY 2;
-state     |catchange_count|catchange_array_length|committed_count|committed_array_length
-----------+---------------+----------------------+---------------+----------------------
-consistent|              0|                      |              2|                     2
-consistent|              2|                     2|              0|                      
-(2 rows)
+step s1_check_snapshot_info: SELECT count(*) > 0 as has_info FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info where info.catchange_count >= 2 and array_length(info.catchange_xip,1) >= 2 and info.committed_count >= 1 and array_length(info.committed_xip,1) >= 1;
+has_info
+--------
+t       
+(1 row)
 
-step s1_get_logical_snapshot_meta: SELECT COUNT(meta.*) from pg_ls_logicalsnapdir(), pg_get_logical_snapshot_meta(name) as meta;
-count
------
-    2
+step s1_check_snapshot_meta: SELECT count(meta.*) > 0 AS has_meta from pg_ls_logicalsnapdir(), pg_get_logical_snapshot_meta(name) as meta;
+has_meta
+--------
+t       
 (1 row)
 
+step s0_commit: COMMIT;
diff --git a/contrib/pg_logicalinspect/specs/logical_inspect.spec b/contrib/pg_logicalinspect/specs/logical_inspect.spec
index 9851a6c18e4..26b2db10f3e 100644
--- a/contrib/pg_logicalinspect/specs/logical_inspect.spec
+++ b/contrib/pg_logicalinspect/specs/logical_inspect.spec
@@ -1,9 +1,10 @@
 # Test the pg_logicalinspect functions: that needs some permutation to
-# ensure that we are creating multiple logical snapshots and that one of them
-# contains ongoing catalogs changes.
+# ensure that we are creating at least one snapshot that contains ongoing and
+# committed catalogs changes.
 setup
 {
     DROP TABLE IF EXISTS tbl1;
+    DROP TABLE IF EXISTS tbl2;
     CREATE TABLE tbl1 (val1 integer, val2 integer);
     CREATE EXTENSION pg_logicalinspect;
 }
@@ -11,6 +12,7 @@ setup
 teardown
 {
     DROP TABLE tbl1;
+    DROP TABLE tbl2;
     SELECT 'stop' FROM pg_drop_replication_slot('isolation_slot');
     DROP EXTENSION pg_logicalinspect;
 }
@@ -21,14 +23,24 @@ step "s0_init" { SELECT 'init' FROM pg_create_logical_replication_slot('isolatio
 step "s0_begin" { BEGIN; }
 step "s0_savepoint" { SAVEPOINT sp1; }
 step "s0_truncate" { TRUNCATE tbl1; }
-step "s0_insert" { INSERT INTO tbl1 VALUES (1); }
 step "s0_commit" { COMMIT; }
 
 session "s1"
 setup { SET synchronous_commit=on; }
 step "s1_checkpoint" { CHECKPOINT; }
+step "s1_create_table" { CREATE TABLE tbl2 (val1 integer, val2 integer); }
 step "s1_get_changes" { SELECT data FROM pg_logical_slot_get_changes('isolation_slot', NULL, NULL, 'skip-empty-xacts', '1', 'include-xids', '0'); }
-step "s1_get_logical_snapshot_meta" { SELECT COUNT(meta.*) from pg_ls_logicalsnapdir(), pg_get_logical_snapshot_meta(name) as meta;}
-step "s1_get_logical_snapshot_info" { SELECT info.state, info.catchange_count, array_length(info.catchange_xip,1) AS catchange_array_length, info.committed_count, array_length(info.committed_xip,1) AS committed_array_length FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info ORDER BY 2; }
+step "s1_check_snapshot_meta" { SELECT count(meta.*) > 0 AS has_meta from pg_ls_logicalsnapdir(), pg_get_logical_snapshot_meta(name) as meta; }
+step "s1_check_snapshot_info" { SELECT count(*) > 0 as has_info FROM pg_ls_logicalsnapdir(), pg_get_logical_snapshot_info(name) AS info where info.catchange_count >= 2 and array_length(info.catchange_xip,1) >= 2 and info.committed_count >= 1 and array_length(info.committed_xip,1) >= 1; }
 
-permutation "s0_init" "s0_begin" "s0_savepoint" "s0_truncate" "s1_checkpoint" "s1_get_changes" "s0_commit" "s0_begin" "s0_insert" "s1_checkpoint" "s1_get_changes" "s0_commit" "s1_get_changes" "s1_get_logical_snapshot_info" "s1_get_logical_snapshot_meta"
+
+# Both s0 and s1 execute catalog-change transactions. When "s1_get_changes" is
+# executed, s0's transaction is still in progress, while s1's transaction has
+# already completed. Consequently, the logical decoding produces a snapshot at
+# the point where a RUNNING_XACTS record is generated by "s1_checkpoint".
+# This snapshot contains both two ongoing catalog-change transactions (from s0's
+# top-level and sub transactions) and one completed transaction (from s1).
+# "s1_check_snapshot_info" verifies whether the logical snapshot contains at
+# least the expected number of transactions, accounting for potential
+# additional catalog changes that may occur due to concurrent autoanalyze.
+permutation "s0_init" "s0_begin" "s0_savepoint" "s0_truncate" "s1_create_table" "s1_checkpoint" "s1_get_changes" "s1_check_snapshot_info" "s1_check_snapshot_meta" "s0_commit"
-- 
2.43.5

#78Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: Masahiko Sawada (#77)
Re: Add contrib/pg_logicalsnapinspect

Hi,

On Fri, Mar 07, 2025 at 12:09:35PM -0800, Masahiko Sawada wrote:

Thank you for updating the patch. It looks mostly good to me. I've
made some cosmetic changes and attached the updated version.

LGTM, thanks!

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

#79Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Bertrand Drouvot (#78)
Re: Add contrib/pg_logicalsnapinspect

On Fri, Mar 7, 2025 at 11:58 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Hi,

On Fri, Mar 07, 2025 at 12:09:35PM -0800, Masahiko Sawada wrote:

Thank you for updating the patch. It looks mostly good to me. I've
made some cosmetic changes and attached the updated version.

LGTM, thanks!

Pushed.

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com

#80Euler Taveira
euler@eulerto.com
In reply to: Masahiko Sawada (#79)
Re: Add contrib/pg_logicalsnapinspect

On Tue, Mar 11, 2025, at 7:34 PM, Masahiko Sawada wrote:

Pushed.

pgindent is saying this commit included some extra tabs.

git diff
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.c b/contrib/pg_logicalinspect/pg_logicalinspect.c
index ff6c682679f..5a44718bea8 100644
--- a/contrib/pg_logicalinspect/pg_logicalinspect.c
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.c
@@ -86,7 +86,7 @@ parse_error:
    ereport(ERROR,
            errmsg("invalid snapshot file name \"%s\"", filename));
-   return InvalidXLogRecPtr;                   /* keep compiler quiet */
+   return InvalidXLogRecPtr;   /* keep compiler quiet */
} 

--
Euler Taveira
EDB https://www.enterprisedb.com/

#81Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Euler Taveira (#80)
Re: Add contrib/pg_logicalsnapinspect

On Thu, Mar 13, 2025 at 6:20 PM Euler Taveira <euler@eulerto.com> wrote:

On Tue, Mar 11, 2025, at 7:34 PM, Masahiko Sawada wrote:

Pushed.

pgindent is saying this commit included some extra tabs.

git diff
diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.c b/contrib/pg_logicalinspect/pg_logicalinspect.c
index ff6c682679f..5a44718bea8 100644
--- a/contrib/pg_logicalinspect/pg_logicalinspect.c
+++ b/contrib/pg_logicalinspect/pg_logicalinspect.c
@@ -86,7 +86,7 @@ parse_error:
ereport(ERROR,
errmsg("invalid snapshot file name \"%s\"", filename));
-   return InvalidXLogRecPtr;                   /* keep compiler quiet */
+   return InvalidXLogRecPtr;   /* keep compiler quiet */
}

Yes, David fixed it in commit b955df44340.

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com