Combine pg_walinspect till_end_of_wal functions with others
Started by Bharath Rupireddyalmost 3 years ago35 messages
Hi,
In a recent discussion [1]/messages/by-id/CALj2ACV-WBN=EUgUPyYOGitp+rn163vMnQd=HcWrnKt-uqFYFA@mail.gmail.com, Michael Paquier asked if we can combine
pg_walinspect till_end_of_wal functions with other functions
pg_get_wal_records_info and pg_get_wal_stats. The code currently looks
much duplicated and the number of functions that pg_walinspect exposes
to the users is bloated. The point was that the till_end_of_wal
functions determine the end LSN and everything else that they do is
the same as their counterpart functions. Well, the idea then was to
keep things simple, not clutter the APIs, have better and consistent
user-inputted end_lsn validations at the cost of usability and code
redundancy. However, now I tend to agree with the feedback received.
I'm attaching a patch doing the $subject with the following behavior:
1. If start_lsn is NULL, error out/return NULL.
2. If end_lsn isn't specified, default to NULL, then determine the end_lsn.
3. If end_lsn is specified as NULL, then determine the end_lsn.
4. If end_lsn is specified as non-NULL, then determine if it is
greater than start_lsn if yes, go ahead do the job, otherwise error
out.
Another idea is to convert till_end_of_wal flavors to SQL-only
functions and remove the c code from pg_walinspect.c. However, I
prefer $subject and completely remove till_end_of_wal flavors for
better usability in the long term.
Thoughts?
[1]: /messages/by-id/CALj2ACV-WBN=EUgUPyYOGitp+rn163vMnQd=HcWrnKt-uqFYFA@mail.gmail.com
--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
Attachments:
v1-0001-Reduce-pg_walinspect-s-functions-without-losing-f.patchapplication/x-patch; name=v1-0001-Reduce-pg_walinspect-s-functions-without-losing-f.patchDownload
From 156e1d9148af79b905095b627e2db7919e8489ca Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Sun, 26 Feb 2023 13:34:33 +0000
Subject: [PATCH v1] Reduce pg_walinspect's functions without losing
functionality
---
contrib/pg_walinspect/Makefile | 6 +-
.../pg_walinspect/expected/oldextversions.out | 64 +++++++
.../pg_walinspect/expected/pg_walinspect.out | 32 +++-
contrib/pg_walinspect/meson.build | 4 +-
.../pg_walinspect/pg_walinspect--1.1--1.2.sql | 78 ++++++++
contrib/pg_walinspect/pg_walinspect.c | 174 +++++++++---------
contrib/pg_walinspect/pg_walinspect.control | 2 +-
contrib/pg_walinspect/sql/oldextversions.sql | 27 +++
contrib/pg_walinspect/sql/pg_walinspect.sql | 28 ++-
doc/src/sgml/pgwalinspect.sgml | 54 ++----
10 files changed, 336 insertions(+), 133 deletions(-)
create mode 100644 contrib/pg_walinspect/expected/oldextversions.out
create mode 100644 contrib/pg_walinspect/pg_walinspect--1.1--1.2.sql
create mode 100644 contrib/pg_walinspect/sql/oldextversions.sql
diff --git a/contrib/pg_walinspect/Makefile b/contrib/pg_walinspect/Makefile
index 7033878a79..9527a0b268 100644
--- a/contrib/pg_walinspect/Makefile
+++ b/contrib/pg_walinspect/Makefile
@@ -7,9 +7,11 @@ OBJS = \
PGFILEDESC = "pg_walinspect - functions to inspect contents of PostgreSQL Write-Ahead Log"
EXTENSION = pg_walinspect
-DATA = pg_walinspect--1.0.sql pg_walinspect--1.0--1.1.sql
+DATA = pg_walinspect--1.0.sql \
+ pg_walinspect--1.0--1.1.sql \
+ pg_walinspect--1.1--1.2.sql
-REGRESS = pg_walinspect
+REGRESS = pg_walinspect oldextversions
REGRESS_OPTS = --temp-config $(top_srcdir)/contrib/pg_walinspect/walinspect.conf
diff --git a/contrib/pg_walinspect/expected/oldextversions.out b/contrib/pg_walinspect/expected/oldextversions.out
new file mode 100644
index 0000000000..65963c77f4
--- /dev/null
+++ b/contrib/pg_walinspect/expected/oldextversions.out
@@ -0,0 +1,64 @@
+-- test old extension version entry points
+DROP EXTENSION pg_walinspect;
+CREATE EXTENSION pg_walinspect VERSION '1.1';
+-- New function introduced in 1.1
+SELECT pg_get_functiondef('pg_get_wal_fpi_info'::regproc);
+ pg_get_functiondef
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ CREATE OR REPLACE FUNCTION public.pg_get_wal_fpi_info(start_lsn pg_lsn, end_lsn pg_lsn, OUT lsn pg_lsn, OUT reltablespace oid, OUT reldatabase oid, OUT relfilenode oid, OUT relblocknumber bigint, OUT forkname text, OUT fpi bytea)+
+ RETURNS SETOF record +
+ LANGUAGE c +
+ PARALLEL SAFE STRICT +
+ AS '$libdir/pg_walinspect', $function$pg_get_wal_fpi_info$function$ +
+
+(1 row)
+
+-- Move to new version 1.2
+ALTER EXTENSION pg_walinspect UPDATE TO '1.2';
+-- Redefined function
+SELECT pg_get_functiondef('pg_get_wal_records_info'::regproc);
+ pg_get_functiondef
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ CREATE OR REPLACE FUNCTION public.pg_get_wal_records_info(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL::pg_lsn, OUT start_lsn pg_lsn, OUT end_lsn pg_lsn, OUT prev_lsn pg_lsn, OUT xid xid, OUT resource_manager text, OUT record_type text, OUT record_length integer, OUT main_data_length integer, OUT fpi_length integer, OUT description text, OUT block_ref text)+
+ RETURNS SETOF record +
+ LANGUAGE c +
+ PARALLEL SAFE +
+ AS '$libdir/pg_walinspect', $function$pg_get_wal_records_info$function$ +
+
+(1 row)
+
+-- Removed function
+SELECT pg_get_functiondef('pg_get_wal_records_info_till_end_of_wal'::regproc);
+ERROR: function "pg_get_wal_records_info_till_end_of_wal" does not exist
+LINE 1: SELECT pg_get_functiondef('pg_get_wal_records_info_till_end_...
+ ^
+-- Redefined function
+SELECT pg_get_functiondef('pg_get_wal_stats'::regproc);
+ pg_get_functiondef
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ CREATE OR REPLACE FUNCTION public.pg_get_wal_stats(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL::pg_lsn, per_record boolean DEFAULT false, OUT "resource_manager/record_type" text, OUT count bigint, OUT count_percentage double precision, OUT record_size bigint, OUT record_size_percentage double precision, OUT fpi_size bigint, OUT fpi_size_percentage double precision, OUT combined_size bigint, OUT combined_size_percentage double precision)+
+ RETURNS SETOF record +
+ LANGUAGE c +
+ PARALLEL SAFE +
+ AS '$libdir/pg_walinspect', $function$pg_get_wal_stats$function$ +
+
+(1 row)
+
+-- Removed function
+SELECT pg_get_functiondef('pg_get_wal_stats_till_end_of_wal'::regproc);
+ERROR: function "pg_get_wal_stats_till_end_of_wal" does not exist
+LINE 1: SELECT pg_get_functiondef('pg_get_wal_stats_till_end_of_wal'...
+ ^
+-- Redefined function
+SELECT pg_get_functiondef('pg_get_wal_fpi_info'::regproc);
+ pg_get_functiondef
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ CREATE OR REPLACE FUNCTION public.pg_get_wal_fpi_info(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL::pg_lsn, OUT lsn pg_lsn, OUT reltablespace oid, OUT reldatabase oid, OUT relfilenode oid, OUT relblocknumber bigint, OUT forkname text, OUT fpi bytea)+
+ RETURNS SETOF record +
+ LANGUAGE c +
+ PARALLEL SAFE +
+ AS '$libdir/pg_walinspect', $function$pg_get_wal_fpi_info$function$ +
+
+(1 row)
+
+DROP EXTENSION pg_walinspect;
diff --git a/contrib/pg_walinspect/expected/pg_walinspect.out b/contrib/pg_walinspect/expected/pg_walinspect.out
index 9bcb05354e..54284d22de 100644
--- a/contrib/pg_walinspect/expected/pg_walinspect.out
+++ b/contrib/pg_walinspect/expected/pg_walinspect.out
@@ -14,10 +14,30 @@ INSERT INTO sample_tbl SELECT * FROM generate_series(3, 4);
-- ===================================================================
-- Tests for input validation
-- ===================================================================
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info('0/0'); -- ERROR
+ERROR: WAL start LSN cannot be invalid
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn2', :'wal_lsn1'); -- ERROR
ERROR: WAL start LSN must be less than end LSN
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(NULL, NULL); -- ERROR
+ERROR: WAL start LSN cannot be null
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', '0/0'); -- ERROR
+ERROR: WAL end LSN cannot be invalid
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats('0/0'); -- ERROR
+ERROR: WAL start LSN cannot be invalid
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn2', :'wal_lsn1'); -- ERROR
ERROR: WAL start LSN must be less than end LSN
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(NULL, NULL); -- ERROR
+ERROR: WAL start LSN cannot be null
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', '0/0'); -- ERROR
+ERROR: WAL end LSN cannot be invalid
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_fpi_info('0/0'); -- ERROR
+ERROR: WAL start LSN cannot be invalid
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_fpi_info(:'wal_lsn2', :'wal_lsn1'); -- ERROR
+ERROR: WAL start LSN must be less than end LSN
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_fpi_info(NULL, NULL); -- ERROR
+ERROR: WAL start LSN cannot be null
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_fpi_info(:'wal_lsn1', '0/0'); -- ERROR
+ERROR: WAL end LSN cannot be invalid
-- ===================================================================
-- Tests for all function executions
-- ===================================================================
@@ -33,7 +53,7 @@ SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2'
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info_till_end_of_wal(:'wal_lsn1');
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1');
ok
----
t
@@ -45,7 +65,7 @@ SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats_till_end_of_wal(:'wal_lsn1');
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1');
ok
----
t
@@ -90,6 +110,14 @@ SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_fpi_info(:'wal_lsn3', :'wal_lsn4')
t
(1 row)
+-- Check if we get FPI from WAL record.
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_fpi_info(:'wal_lsn3')
+ WHERE relfilenode = :'sample_tbl_oid';
+ ok
+----
+ t
+(1 row)
+
-- ===================================================================
-- Tests for permissions
-- ===================================================================
diff --git a/contrib/pg_walinspect/meson.build b/contrib/pg_walinspect/meson.build
index bf7b79b1b7..cecf041b5b 100644
--- a/contrib/pg_walinspect/meson.build
+++ b/contrib/pg_walinspect/meson.build
@@ -20,6 +20,7 @@ install_data(
'pg_walinspect.control',
'pg_walinspect--1.0.sql',
'pg_walinspect--1.0--1.1.sql',
+ 'pg_walinspect--1.1--1.2.sql',
kwargs: contrib_data_args,
)
@@ -30,10 +31,11 @@ tests += {
'regress': {
'sql': [
'pg_walinspect',
+ 'oldextversions',
],
# Disabled because these tests require "wal_level=replica", which
# some runningcheck users do not have (e.g. buildfarm clients).
'regress_args': ['--temp-config', files('walinspect.conf')],
'runningcheck': false,
},
-}
+}
\ No newline at end of file
diff --git a/contrib/pg_walinspect/pg_walinspect--1.1--1.2.sql b/contrib/pg_walinspect/pg_walinspect--1.1--1.2.sql
new file mode 100644
index 0000000000..bd03ca4ba5
--- /dev/null
+++ b/contrib/pg_walinspect/pg_walinspect--1.1--1.2.sql
@@ -0,0 +1,78 @@
+/* contrib/pg_walinspect/pg_walinspect--1.1--1.2.sql */
+
+-- complain if script is sourced in psql, rather than via ALTER EXTENSION
+\echo Use "ALTER EXTENSION pg_walinspect UPDATE TO '1.2'" to load this file. \quit
+
+/* Drop unneeded functions and redefine needed ones */
+DROP FUNCTION pg_get_wal_records_info(pg_lsn, pg_lsn);
+DROP FUNCTION pg_get_wal_records_info_till_end_of_wal(pg_lsn);
+DROP FUNCTION pg_get_wal_stats(pg_lsn, pg_lsn, boolean);
+DROP FUNCTION pg_get_wal_stats_till_end_of_wal(pg_lsn, boolean);
+DROP FUNCTION pg_get_wal_fpi_info(pg_lsn, pg_lsn);
+
+--
+-- pg_get_wal_records_info()
+--
+CREATE FUNCTION pg_get_wal_records_info(IN start_lsn pg_lsn,
+ IN end_lsn pg_lsn DEFAULT NULL,
+ OUT start_lsn pg_lsn,
+ OUT end_lsn pg_lsn,
+ OUT prev_lsn pg_lsn,
+ OUT xid xid,
+ OUT resource_manager text,
+ OUT record_type text,
+ OUT record_length int4,
+ OUT main_data_length int4,
+ OUT fpi_length int4,
+ OUT description text,
+ OUT block_ref text
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_get_wal_records_info'
+LANGUAGE C CALLED ON NULL INPUT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_wal_records_info(pg_lsn, pg_lsn) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_wal_records_info(pg_lsn, pg_lsn) TO pg_read_server_files;
+
+--
+-- pg_get_wal_stats()
+--
+CREATE FUNCTION pg_get_wal_stats(IN start_lsn pg_lsn,
+ IN end_lsn pg_lsn DEFAULT NULL,
+ IN per_record boolean DEFAULT false,
+ OUT "resource_manager/record_type" text,
+ OUT count int8,
+ OUT count_percentage float8,
+ OUT record_size int8,
+ OUT record_size_percentage float8,
+ OUT fpi_size int8,
+ OUT fpi_size_percentage float8,
+ OUT combined_size int8,
+ OUT combined_size_percentage float8
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_get_wal_stats'
+LANGUAGE C CALLED ON NULL INPUT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_wal_stats(pg_lsn, pg_lsn, boolean) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_wal_stats(pg_lsn, pg_lsn, boolean) TO pg_read_server_files;
+
+--
+-- pg_get_wal_fpi_info()
+--
+CREATE FUNCTION pg_get_wal_fpi_info(IN start_lsn pg_lsn,
+ IN end_lsn pg_lsn DEFAULT NULL,
+ OUT lsn pg_lsn,
+ OUT reltablespace oid,
+ OUT reldatabase oid,
+ OUT relfilenode oid,
+ OUT relblocknumber int8,
+ OUT forkname text,
+ OUT fpi bytea
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_get_wal_fpi_info'
+LANGUAGE C CALLED ON NULL INPUT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_wal_fpi_info(pg_lsn, pg_lsn) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_wal_fpi_info(pg_lsn, pg_lsn) TO pg_read_server_files;
diff --git a/contrib/pg_walinspect/pg_walinspect.c b/contrib/pg_walinspect/pg_walinspect.c
index b7b0a805ee..7d57a8b062 100644
--- a/contrib/pg_walinspect/pg_walinspect.c
+++ b/contrib/pg_walinspect/pg_walinspect.c
@@ -37,13 +37,13 @@ PG_FUNCTION_INFO_V1(pg_get_wal_records_info_till_end_of_wal);
PG_FUNCTION_INFO_V1(pg_get_wal_stats);
PG_FUNCTION_INFO_V1(pg_get_wal_stats_till_end_of_wal);
-static bool IsFutureLSN(XLogRecPtr lsn, XLogRecPtr *curr_lsn);
+static XLogRecPtr GetCurrentLSN(void);
static XLogReaderState *InitXLogReaderState(XLogRecPtr lsn);
static XLogRecord *ReadNextXLogRecord(XLogReaderState *xlogreader);
static void GetWALRecordInfo(XLogReaderState *record, Datum *values,
bool *nulls, uint32 ncols);
-static XLogRecPtr ValidateInputLSNs(bool till_end_of_wal,
- XLogRecPtr start_lsn, XLogRecPtr end_lsn);
+static void ValidateInputLSNs(FunctionCallInfo fcinfo, XLogRecPtr *start_lsn,
+ XLogRecPtr *end_lsn);
static void GetWALRecordsInfo(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
XLogRecPtr end_lsn);
static void GetXLogSummaryStats(XLogStats *stats, ReturnSetInfo *rsinfo,
@@ -59,27 +59,25 @@ static void GetWalStats(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
static void GetWALFPIInfo(FunctionCallInfo fcinfo, XLogReaderState *record);
/*
- * Check if the given LSN is in future. Also, return the LSN up to which the
- * server has WAL.
+ * Return the LSN up to which the server has WAL.
*/
-static bool
-IsFutureLSN(XLogRecPtr lsn, XLogRecPtr *curr_lsn)
+static XLogRecPtr
+GetCurrentLSN(void)
{
+ XLogRecPtr curr_lsn;
+
/*
* We determine the current LSN of the server similar to how page_read
* callback read_local_xlog_page_no_wait does.
*/
if (!RecoveryInProgress())
- *curr_lsn = GetFlushRecPtr(NULL);
+ curr_lsn = GetFlushRecPtr(NULL);
else
- *curr_lsn = GetXLogReplayRecPtr(NULL);
-
- Assert(!XLogRecPtrIsInvalid(*curr_lsn));
+ curr_lsn = GetXLogReplayRecPtr(NULL);
- if (lsn >= *curr_lsn)
- return true;
+ Assert(!XLogRecPtrIsInvalid(curr_lsn));
- return false;
+ return curr_lsn;
}
/*
@@ -295,6 +293,11 @@ GetWALFPIInfo(FunctionCallInfo fcinfo, XLogReaderState *record)
* records between start and end LSNs. Decompression is applied to the
* blocks, if necessary.
*
+ * This function determines end WAL LSN if it is not specified or specified as
+ * NULL. In such cases, the end WAL LSN is assigned with the last flushed WAL
+ * LSN when not in recovery or the last replayed WAL record LSN when in
+ * recovery.
+ *
* This function emits an error if a future start or end WAL LSN i.e. WAL LSN
* the database system doesn't know about is specified.
*/
@@ -307,11 +310,7 @@ pg_get_wal_fpi_info(PG_FUNCTION_ARGS)
MemoryContext old_cxt;
MemoryContext tmp_cxt;
- start_lsn = PG_GETARG_LSN(0);
- end_lsn = PG_GETARG_LSN(1);
-
- end_lsn = ValidateInputLSNs(false, start_lsn, end_lsn);
-
+ ValidateInputLSNs(fcinfo, &start_lsn, &end_lsn);
InitMaterializedSRF(fcinfo, 0);
xlogreader = InitXLogReaderState(start_lsn);
@@ -363,7 +362,9 @@ pg_get_wal_record_info(PG_FUNCTION_ARGS)
lsn = PG_GETARG_LSN(0);
- if (IsFutureLSN(lsn, &curr_lsn))
+ curr_lsn = GetCurrentLSN();
+
+ if (lsn >= curr_lsn)
{
/*
* GetFlushRecPtr or GetXLogReplayRecPtr gives "end+1" LSN of the last
@@ -402,44 +403,64 @@ pg_get_wal_record_info(PG_FUNCTION_ARGS)
}
/*
- * Validate the input LSNs and compute end LSN for till_end_of_wal versions.
+ * Validate input LSNs and return start_lsn and end_lsn. Also, compute end LSN
+ * in case it is not specified.
*/
-static XLogRecPtr
-ValidateInputLSNs(bool till_end_of_wal, XLogRecPtr start_lsn,
- XLogRecPtr end_lsn)
+static void
+ValidateInputLSNs(FunctionCallInfo fcinfo, XLogRecPtr *start_lsn,
+ XLogRecPtr *end_lsn)
{
- XLogRecPtr curr_lsn;
+ XLogRecPtr c_lsn;
+ XLogRecPtr s_lsn;
+ XLogRecPtr e_lsn;
- if (IsFutureLSN(start_lsn, &curr_lsn))
- {
- /*
- * GetFlushRecPtr or GetXLogReplayRecPtr gives "end+1" LSN of the last
- * record flushed or replayed respectively. But let's use the LSN up
- * to "end" in user facing message.
- */
+ if (PG_ARGISNULL(0))
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("cannot accept future start LSN"),
- errdetail("Last known WAL LSN on the database system is at %X/%X.",
- LSN_FORMAT_ARGS(curr_lsn))));
- }
+ errmsg("WAL start LSN cannot be null")));
- if (till_end_of_wal)
- end_lsn = curr_lsn;
+ s_lsn = PG_GETARG_LSN(0);
- if (end_lsn > curr_lsn)
+ if (XLogRecPtrIsInvalid(s_lsn))
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("cannot accept future end LSN"),
- errdetail("Last known WAL LSN on the database system is at %X/%X.",
- LSN_FORMAT_ARGS(curr_lsn))));
+ errmsg("WAL start LSN cannot be invalid")));
- if (start_lsn >= end_lsn)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("WAL start LSN must be less than end LSN")));
+ c_lsn = GetCurrentLSN();
+
+ if (PG_ARGISNULL(1))
+ e_lsn = c_lsn;
+ else
+ {
+ e_lsn = PG_GETARG_LSN(1);
+
+ if (XLogRecPtrIsInvalid(e_lsn))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("WAL end LSN cannot be invalid")));
- return end_lsn;
+ if (e_lsn > c_lsn)
+ {
+ /*
+ * GetFlushRecPtr or GetXLogReplayRecPtr gives "end+1" LSN of the
+ * last record flushed or replayed respectively. But let's use the
+ * LSN up to "end" in user facing message.
+ */
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("cannot accept future end LSN"),
+ errdetail("Last known WAL LSN on the database system is at %X/%X.",
+ LSN_FORMAT_ARGS(c_lsn))));
+ }
+
+ if (s_lsn >= e_lsn)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("WAL start LSN must be less than end LSN")));
+ }
+
+ *start_lsn = s_lsn;
+ *end_lsn = e_lsn;
}
/*
@@ -494,6 +515,11 @@ GetWALRecordsInfo(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
/*
* Get info and data of all WAL records between start LSN and end LSN.
*
+ * This function determines end WAL LSN if it is not specified or specified as
+ * NULL. In such cases, the end WAL LSN is assigned with the last flushed WAL
+ * LSN when not in recovery or the last replayed WAL record LSN when in
+ * recovery.
+ *
* This function emits an error if a future start or end WAL LSN i.e. WAL LSN
* the database system doesn't know about is specified.
*/
@@ -503,35 +529,23 @@ pg_get_wal_records_info(PG_FUNCTION_ARGS)
XLogRecPtr start_lsn;
XLogRecPtr end_lsn;
- start_lsn = PG_GETARG_LSN(0);
- end_lsn = PG_GETARG_LSN(1);
-
- end_lsn = ValidateInputLSNs(false, start_lsn, end_lsn);
-
+ ValidateInputLSNs(fcinfo, &start_lsn, &end_lsn);
GetWALRecordsInfo(fcinfo, start_lsn, end_lsn);
PG_RETURN_VOID();
}
/*
- * Get info and data of all WAL records from start LSN till end of WAL.
+ * NB: This function does nothing and stays here for backward compatibility.
+ * Without it, the extension fails to install.
*
- * This function emits an error if a future start i.e. WAL LSN the database
- * system doesn't know about is specified.
+ * Try using pg_get_wal_records_info() for the same till_end_of_wal
+ * functionaility.
*/
Datum
pg_get_wal_records_info_till_end_of_wal(PG_FUNCTION_ARGS)
{
- XLogRecPtr start_lsn;
- XLogRecPtr end_lsn = InvalidXLogRecPtr;
-
- start_lsn = PG_GETARG_LSN(0);
-
- end_lsn = ValidateInputLSNs(true, start_lsn, end_lsn);
-
- GetWALRecordsInfo(fcinfo, start_lsn, end_lsn);
-
- PG_RETURN_VOID();
+ PG_RETURN_NULL();
}
/*
@@ -732,6 +746,11 @@ GetWalStats(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
/*
* Get stats of all WAL records between start LSN and end LSN.
*
+ * This function determines end WAL LSN if it is not specified or specified as
+ * NULL. In such cases, the end WAL LSN is assigned with the last flushed WAL
+ * LSN when not in recovery or the last replayed WAL record LSN when in
+ * recovery.
+ *
* This function emits an error if a future start or end WAL LSN i.e. WAL LSN
* the database system doesn't know about is specified.
*/
@@ -742,36 +761,23 @@ pg_get_wal_stats(PG_FUNCTION_ARGS)
XLogRecPtr end_lsn;
bool stats_per_record;
- start_lsn = PG_GETARG_LSN(0);
- end_lsn = PG_GETARG_LSN(1);
+ ValidateInputLSNs(fcinfo, &start_lsn, &end_lsn);
stats_per_record = PG_GETARG_BOOL(2);
- end_lsn = ValidateInputLSNs(false, start_lsn, end_lsn);
-
GetWalStats(fcinfo, start_lsn, end_lsn, stats_per_record);
PG_RETURN_VOID();
}
/*
- * Get stats of all WAL records from start LSN till end of WAL.
+ * NB: This function does nothing and stays here for backward compatibility.
+ * Without it, the extension fails to install.
*
- * This function emits an error if a future start i.e. WAL LSN the database
- * system doesn't know about is specified.
+ * Try using pg_get_wal_records_info() for the same till_end_of_wal
+ * functionaility.
*/
Datum
pg_get_wal_stats_till_end_of_wal(PG_FUNCTION_ARGS)
{
- XLogRecPtr start_lsn;
- XLogRecPtr end_lsn = InvalidXLogRecPtr;
- bool stats_per_record;
-
- start_lsn = PG_GETARG_LSN(0);
- stats_per_record = PG_GETARG_BOOL(1);
-
- end_lsn = ValidateInputLSNs(true, start_lsn, end_lsn);
-
- GetWalStats(fcinfo, start_lsn, end_lsn, stats_per_record);
-
- PG_RETURN_VOID();
+ PG_RETURN_NULL();
}
diff --git a/contrib/pg_walinspect/pg_walinspect.control b/contrib/pg_walinspect/pg_walinspect.control
index efa3cb2cfe..5f574b865b 100644
--- a/contrib/pg_walinspect/pg_walinspect.control
+++ b/contrib/pg_walinspect/pg_walinspect.control
@@ -1,5 +1,5 @@
# pg_walinspect extension
comment = 'functions to inspect contents of PostgreSQL Write-Ahead Log'
-default_version = '1.1'
+default_version = '1.2'
module_pathname = '$libdir/pg_walinspect'
relocatable = true
diff --git a/contrib/pg_walinspect/sql/oldextversions.sql b/contrib/pg_walinspect/sql/oldextversions.sql
new file mode 100644
index 0000000000..ead4997d37
--- /dev/null
+++ b/contrib/pg_walinspect/sql/oldextversions.sql
@@ -0,0 +1,27 @@
+-- test old extension version entry points
+
+DROP EXTENSION pg_walinspect;
+CREATE EXTENSION pg_walinspect VERSION '1.1';
+
+-- New function introduced in 1.1
+SELECT pg_get_functiondef('pg_get_wal_fpi_info'::regproc);
+
+-- Move to new version 1.2
+ALTER EXTENSION pg_walinspect UPDATE TO '1.2';
+
+-- Redefined function
+SELECT pg_get_functiondef('pg_get_wal_records_info'::regproc);
+
+-- Removed function
+SELECT pg_get_functiondef('pg_get_wal_records_info_till_end_of_wal'::regproc);
+
+-- Redefined function
+SELECT pg_get_functiondef('pg_get_wal_stats'::regproc);
+
+-- Removed function
+SELECT pg_get_functiondef('pg_get_wal_stats_till_end_of_wal'::regproc);
+
+-- Redefined function
+SELECT pg_get_functiondef('pg_get_wal_fpi_info'::regproc);
+
+DROP EXTENSION pg_walinspect;
diff --git a/contrib/pg_walinspect/sql/pg_walinspect.sql b/contrib/pg_walinspect/sql/pg_walinspect.sql
index 849201a1f8..64fd95f387 100644
--- a/contrib/pg_walinspect/sql/pg_walinspect.sql
+++ b/contrib/pg_walinspect/sql/pg_walinspect.sql
@@ -17,10 +17,30 @@ INSERT INTO sample_tbl SELECT * FROM generate_series(3, 4);
-- Tests for input validation
-- ===================================================================
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info('0/0'); -- ERROR
+
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn2', :'wal_lsn1'); -- ERROR
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(NULL, NULL); -- ERROR
+
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', '0/0'); -- ERROR
+
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats('0/0'); -- ERROR
+
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn2', :'wal_lsn1'); -- ERROR
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(NULL, NULL); -- ERROR
+
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', '0/0'); -- ERROR
+
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_fpi_info('0/0'); -- ERROR
+
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_fpi_info(:'wal_lsn2', :'wal_lsn1'); -- ERROR
+
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_fpi_info(NULL, NULL); -- ERROR
+
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_fpi_info(:'wal_lsn1', '0/0'); -- ERROR
+
-- ===================================================================
-- Tests for all function executions
-- ===================================================================
@@ -29,11 +49,11 @@ SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info_till_end_of_wal(:'wal_lsn1');
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1');
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats_till_end_of_wal(:'wal_lsn1');
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1');
-- ===================================================================
-- Test for filtering out WAL records of a particular table
@@ -68,6 +88,10 @@ SELECT pg_current_wal_lsn() AS wal_lsn4 \gset
SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_fpi_info(:'wal_lsn3', :'wal_lsn4')
WHERE relfilenode = :'sample_tbl_oid';
+-- Check if we get FPI from WAL record.
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_fpi_info(:'wal_lsn3')
+ WHERE relfilenode = :'sample_tbl_oid';
+
-- ===================================================================
-- Tests for permissions
-- ===================================================================
diff --git a/doc/src/sgml/pgwalinspect.sgml b/doc/src/sgml/pgwalinspect.sgml
index 22677e54f2..1b3626df6d 100644
--- a/doc/src/sgml/pgwalinspect.sgml
+++ b/doc/src/sgml/pgwalinspect.sgml
@@ -85,7 +85,7 @@ block_ref | blkref #0: rel 1663/5/60221 fork main blk 2
<varlistentry id="pgwalinspect-funcs-pg-get-wal-records-info">
<term>
<function>
- pg_get_wal_records_info(start_lsn pg_lsn, end_lsn pg_lsn)
+ pg_get_wal_records_info(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL)
returns setof record
</function>
</term>
@@ -94,9 +94,10 @@ block_ref | blkref #0: rel 1663/5/60221 fork main blk 2
<para>
Gets information of all the valid WAL records between
<replaceable>start_lsn</replaceable> and <replaceable>end_lsn</replaceable>.
- Returns one row per WAL record. If <replaceable>start_lsn</replaceable>
- or <replaceable>end_lsn</replaceable> are not yet available, the
- function will raise an error. For example:
+ Returns one row per WAL record. If <replaceable>end_lsn</replaceable>
+ isn't specified, it returns information till the end of WAL.
+ If <replaceable>start_lsn</replaceable> or <replaceable>end_lsn</replaceable>
+ are not yet available, the function will raise an error. For example:
<screen>
postgres=# SELECT * FROM pg_get_wal_records_info('0/1E913618', '0/1E913740') LIMIT 1;
-[ RECORD 1 ]----+--------------------------------------------------------------
@@ -116,27 +117,10 @@ block_ref |
</listitem>
</varlistentry>
- <varlistentry id="pgwalinspect-funcs-pg-get-wal-records-info-till-end-of-wal">
- <term>
- <function>
- pg_get_wal_records_info_till_end_of_wal(start_lsn pg_lsn)
- returns setof record
- </function>
- </term>
-
- <listitem>
- <para>
- This function is the same as <function>pg_get_wal_records_info()</function>,
- except that it gets information of all the valid WAL records from
- <replaceable>start_lsn</replaceable> till the end of WAL.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry id="pgwalinspect-funcs-pg-get-wal-stats">
<term>
<function>
- pg_get_wal_stats(start_lsn pg_lsn, end_lsn pg_lsn, per_record boolean DEFAULT false)
+ pg_get_wal_stats(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL, per_record boolean DEFAULT false)
returns setof record
</function>
</term>
@@ -149,7 +133,8 @@ block_ref |
<replaceable>resource_manager</replaceable> type. When
<replaceable>per_record</replaceable> is set to <literal>true</literal>,
it returns one row per <replaceable>record_type</replaceable>.
- If <replaceable>start_lsn</replaceable>
+ If <replaceable>end_lsn</replaceable> isn't specified, it returns
+ information till the end of WAL. If <replaceable>start_lsn</replaceable>
or <replaceable>end_lsn</replaceable> are not yet available, the
function will raise an error. For example:
<screen>
@@ -171,28 +156,14 @@ combined_size_percentage | 2.8634072910530795
</listitem>
</varlistentry>
- <varlistentry id="pgwalinspect-funcs-pg-get-wal-stats-till-end-of-wal">
+ <varlistentry>
<term>
<function>
- pg_get_wal_stats_till_end_of_wal(start_lsn pg_lsn, end_lsn pg_lsn, per_record boolean DEFAULT false)
+ pg_get_wal_fpi_info(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL)
returns setof record
</function>
</term>
- <listitem>
- <para>
- This function is the same as <function>pg_get_wal_stats()</function>,
- except that it gets statistics of all the valid WAL records from
- <replaceable>start_lsn</replaceable> till end of WAL.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term>
- <function>pg_get_wal_fpi_info(start_lsn pg_lsn, end_lsn pg_lsn) returns setof record</function>
- </term>
-
<listitem>
<para>
Gets a copy of full page images as <type>bytea</type> values (after
@@ -200,8 +171,9 @@ combined_size_percentage | 2.8634072910530795
with all the valid WAL records between
<replaceable>start_lsn</replaceable> and
<replaceable>end_lsn</replaceable>. Returns one row per full page image.
- If <replaceable>start_lsn</replaceable> or
- <replaceable>end_lsn</replaceable> are not yet available, the function
+ If <replaceable>end_lsn</replaceable> isn't specified, it returns
+ information till the end of WAL. If <replaceable>start_lsn</replaceable>
+ or <replaceable>end_lsn</replaceable> are not yet available, the function
will raise an error. For example:
<screen>
postgres=# SELECT lsn, reltablespace, reldatabase, relfilenode, relblocknumber,
--
2.34.1
Re: Combine pg_walinspect till_end_of_wal functions with others
On Wed, Mar 1, 2023 at 1:00 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:
Hi,
In a recent discussion [1], Michael Paquier asked if we can combine
pg_walinspect till_end_of_wal functions with other functions
pg_get_wal_records_info and pg_get_wal_stats. The code currently looks
much duplicated and the number of functions that pg_walinspect exposes
to the users is bloated. The point was that the till_end_of_wal
functions determine the end LSN and everything else that they do is
the same as their counterpart functions. Well, the idea then was to
keep things simple, not clutter the APIs, have better and consistent
user-inputted end_lsn validations at the cost of usability and code
redundancy. However, now I tend to agree with the feedback received.I'm attaching a patch doing the $subject with the following behavior:
1. If start_lsn is NULL, error out/return NULL.
2. If end_lsn isn't specified, default to NULL, then determine the end_lsn.
3. If end_lsn is specified as NULL, then determine the end_lsn.
4. If end_lsn is specified as non-NULL, then determine if it is
greater than start_lsn if yes, go ahead do the job, otherwise error
out.Another idea is to convert till_end_of_wal flavors to SQL-only
functions and remove the c code from pg_walinspect.c. However, I
prefer $subject and completely remove till_end_of_wal flavors for
better usability in the long term.Thoughts?
[1] /messages/by-id/CALj2ACV-WBN=EUgUPyYOGitp+rn163vMnQd=HcWrnKt-uqFYFA@mail.gmail.com
Needed a rebase due to 019f8624664dbf1e25e2bd721c7e99822812d109.
Attaching v2 patch. Sorry for the noise.
--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
Attachments:
v2-0001-Reduce-pg_walinspect-s-functions-without-losing-f.patchapplication/x-patch; name=v2-0001-Reduce-pg_walinspect-s-functions-without-losing-f.patchDownload
From ef5d69fff02913217834c36f93a213d4a9292938 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Wed, 1 Mar 2023 09:17:58 +0000
Subject: [PATCH v2] Reduce pg_walinspect's functions without losing
functionality
---
contrib/pg_walinspect/Makefile | 6 +-
.../pg_walinspect/expected/oldextversions.out | 64 +++++++
.../pg_walinspect/expected/pg_walinspect.out | 32 +++-
contrib/pg_walinspect/meson.build | 4 +-
.../pg_walinspect/pg_walinspect--1.1--1.2.sql | 78 ++++++++
contrib/pg_walinspect/pg_walinspect.c | 174 +++++++++---------
contrib/pg_walinspect/pg_walinspect.control | 2 +-
contrib/pg_walinspect/sql/oldextversions.sql | 27 +++
contrib/pg_walinspect/sql/pg_walinspect.sql | 28 ++-
doc/src/sgml/pgwalinspect.sgml | 54 ++----
10 files changed, 336 insertions(+), 133 deletions(-)
create mode 100644 contrib/pg_walinspect/expected/oldextversions.out
create mode 100644 contrib/pg_walinspect/pg_walinspect--1.1--1.2.sql
create mode 100644 contrib/pg_walinspect/sql/oldextversions.sql
diff --git a/contrib/pg_walinspect/Makefile b/contrib/pg_walinspect/Makefile
index 7033878a79..9527a0b268 100644
--- a/contrib/pg_walinspect/Makefile
+++ b/contrib/pg_walinspect/Makefile
@@ -7,9 +7,11 @@ OBJS = \
PGFILEDESC = "pg_walinspect - functions to inspect contents of PostgreSQL Write-Ahead Log"
EXTENSION = pg_walinspect
-DATA = pg_walinspect--1.0.sql pg_walinspect--1.0--1.1.sql
+DATA = pg_walinspect--1.0.sql \
+ pg_walinspect--1.0--1.1.sql \
+ pg_walinspect--1.1--1.2.sql
-REGRESS = pg_walinspect
+REGRESS = pg_walinspect oldextversions
REGRESS_OPTS = --temp-config $(top_srcdir)/contrib/pg_walinspect/walinspect.conf
diff --git a/contrib/pg_walinspect/expected/oldextversions.out b/contrib/pg_walinspect/expected/oldextversions.out
new file mode 100644
index 0000000000..65963c77f4
--- /dev/null
+++ b/contrib/pg_walinspect/expected/oldextversions.out
@@ -0,0 +1,64 @@
+-- test old extension version entry points
+DROP EXTENSION pg_walinspect;
+CREATE EXTENSION pg_walinspect VERSION '1.1';
+-- New function introduced in 1.1
+SELECT pg_get_functiondef('pg_get_wal_fpi_info'::regproc);
+ pg_get_functiondef
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ CREATE OR REPLACE FUNCTION public.pg_get_wal_fpi_info(start_lsn pg_lsn, end_lsn pg_lsn, OUT lsn pg_lsn, OUT reltablespace oid, OUT reldatabase oid, OUT relfilenode oid, OUT relblocknumber bigint, OUT forkname text, OUT fpi bytea)+
+ RETURNS SETOF record +
+ LANGUAGE c +
+ PARALLEL SAFE STRICT +
+ AS '$libdir/pg_walinspect', $function$pg_get_wal_fpi_info$function$ +
+
+(1 row)
+
+-- Move to new version 1.2
+ALTER EXTENSION pg_walinspect UPDATE TO '1.2';
+-- Redefined function
+SELECT pg_get_functiondef('pg_get_wal_records_info'::regproc);
+ pg_get_functiondef
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ CREATE OR REPLACE FUNCTION public.pg_get_wal_records_info(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL::pg_lsn, OUT start_lsn pg_lsn, OUT end_lsn pg_lsn, OUT prev_lsn pg_lsn, OUT xid xid, OUT resource_manager text, OUT record_type text, OUT record_length integer, OUT main_data_length integer, OUT fpi_length integer, OUT description text, OUT block_ref text)+
+ RETURNS SETOF record +
+ LANGUAGE c +
+ PARALLEL SAFE +
+ AS '$libdir/pg_walinspect', $function$pg_get_wal_records_info$function$ +
+
+(1 row)
+
+-- Removed function
+SELECT pg_get_functiondef('pg_get_wal_records_info_till_end_of_wal'::regproc);
+ERROR: function "pg_get_wal_records_info_till_end_of_wal" does not exist
+LINE 1: SELECT pg_get_functiondef('pg_get_wal_records_info_till_end_...
+ ^
+-- Redefined function
+SELECT pg_get_functiondef('pg_get_wal_stats'::regproc);
+ pg_get_functiondef
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ CREATE OR REPLACE FUNCTION public.pg_get_wal_stats(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL::pg_lsn, per_record boolean DEFAULT false, OUT "resource_manager/record_type" text, OUT count bigint, OUT count_percentage double precision, OUT record_size bigint, OUT record_size_percentage double precision, OUT fpi_size bigint, OUT fpi_size_percentage double precision, OUT combined_size bigint, OUT combined_size_percentage double precision)+
+ RETURNS SETOF record +
+ LANGUAGE c +
+ PARALLEL SAFE +
+ AS '$libdir/pg_walinspect', $function$pg_get_wal_stats$function$ +
+
+(1 row)
+
+-- Removed function
+SELECT pg_get_functiondef('pg_get_wal_stats_till_end_of_wal'::regproc);
+ERROR: function "pg_get_wal_stats_till_end_of_wal" does not exist
+LINE 1: SELECT pg_get_functiondef('pg_get_wal_stats_till_end_of_wal'...
+ ^
+-- Redefined function
+SELECT pg_get_functiondef('pg_get_wal_fpi_info'::regproc);
+ pg_get_functiondef
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ CREATE OR REPLACE FUNCTION public.pg_get_wal_fpi_info(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL::pg_lsn, OUT lsn pg_lsn, OUT reltablespace oid, OUT reldatabase oid, OUT relfilenode oid, OUT relblocknumber bigint, OUT forkname text, OUT fpi bytea)+
+ RETURNS SETOF record +
+ LANGUAGE c +
+ PARALLEL SAFE +
+ AS '$libdir/pg_walinspect', $function$pg_get_wal_fpi_info$function$ +
+
+(1 row)
+
+DROP EXTENSION pg_walinspect;
diff --git a/contrib/pg_walinspect/expected/pg_walinspect.out b/contrib/pg_walinspect/expected/pg_walinspect.out
index 9bcb05354e..54284d22de 100644
--- a/contrib/pg_walinspect/expected/pg_walinspect.out
+++ b/contrib/pg_walinspect/expected/pg_walinspect.out
@@ -14,10 +14,30 @@ INSERT INTO sample_tbl SELECT * FROM generate_series(3, 4);
-- ===================================================================
-- Tests for input validation
-- ===================================================================
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info('0/0'); -- ERROR
+ERROR: WAL start LSN cannot be invalid
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn2', :'wal_lsn1'); -- ERROR
ERROR: WAL start LSN must be less than end LSN
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(NULL, NULL); -- ERROR
+ERROR: WAL start LSN cannot be null
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', '0/0'); -- ERROR
+ERROR: WAL end LSN cannot be invalid
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats('0/0'); -- ERROR
+ERROR: WAL start LSN cannot be invalid
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn2', :'wal_lsn1'); -- ERROR
ERROR: WAL start LSN must be less than end LSN
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(NULL, NULL); -- ERROR
+ERROR: WAL start LSN cannot be null
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', '0/0'); -- ERROR
+ERROR: WAL end LSN cannot be invalid
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_fpi_info('0/0'); -- ERROR
+ERROR: WAL start LSN cannot be invalid
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_fpi_info(:'wal_lsn2', :'wal_lsn1'); -- ERROR
+ERROR: WAL start LSN must be less than end LSN
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_fpi_info(NULL, NULL); -- ERROR
+ERROR: WAL start LSN cannot be null
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_fpi_info(:'wal_lsn1', '0/0'); -- ERROR
+ERROR: WAL end LSN cannot be invalid
-- ===================================================================
-- Tests for all function executions
-- ===================================================================
@@ -33,7 +53,7 @@ SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2'
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info_till_end_of_wal(:'wal_lsn1');
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1');
ok
----
t
@@ -45,7 +65,7 @@ SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats_till_end_of_wal(:'wal_lsn1');
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1');
ok
----
t
@@ -90,6 +110,14 @@ SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_fpi_info(:'wal_lsn3', :'wal_lsn4')
t
(1 row)
+-- Check if we get FPI from WAL record.
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_fpi_info(:'wal_lsn3')
+ WHERE relfilenode = :'sample_tbl_oid';
+ ok
+----
+ t
+(1 row)
+
-- ===================================================================
-- Tests for permissions
-- ===================================================================
diff --git a/contrib/pg_walinspect/meson.build b/contrib/pg_walinspect/meson.build
index bf7b79b1b7..cecf041b5b 100644
--- a/contrib/pg_walinspect/meson.build
+++ b/contrib/pg_walinspect/meson.build
@@ -20,6 +20,7 @@ install_data(
'pg_walinspect.control',
'pg_walinspect--1.0.sql',
'pg_walinspect--1.0--1.1.sql',
+ 'pg_walinspect--1.1--1.2.sql',
kwargs: contrib_data_args,
)
@@ -30,10 +31,11 @@ tests += {
'regress': {
'sql': [
'pg_walinspect',
+ 'oldextversions',
],
# Disabled because these tests require "wal_level=replica", which
# some runningcheck users do not have (e.g. buildfarm clients).
'regress_args': ['--temp-config', files('walinspect.conf')],
'runningcheck': false,
},
-}
+}
\ No newline at end of file
diff --git a/contrib/pg_walinspect/pg_walinspect--1.1--1.2.sql b/contrib/pg_walinspect/pg_walinspect--1.1--1.2.sql
new file mode 100644
index 0000000000..bd03ca4ba5
--- /dev/null
+++ b/contrib/pg_walinspect/pg_walinspect--1.1--1.2.sql
@@ -0,0 +1,78 @@
+/* contrib/pg_walinspect/pg_walinspect--1.1--1.2.sql */
+
+-- complain if script is sourced in psql, rather than via ALTER EXTENSION
+\echo Use "ALTER EXTENSION pg_walinspect UPDATE TO '1.2'" to load this file. \quit
+
+/* Drop unneeded functions and redefine needed ones */
+DROP FUNCTION pg_get_wal_records_info(pg_lsn, pg_lsn);
+DROP FUNCTION pg_get_wal_records_info_till_end_of_wal(pg_lsn);
+DROP FUNCTION pg_get_wal_stats(pg_lsn, pg_lsn, boolean);
+DROP FUNCTION pg_get_wal_stats_till_end_of_wal(pg_lsn, boolean);
+DROP FUNCTION pg_get_wal_fpi_info(pg_lsn, pg_lsn);
+
+--
+-- pg_get_wal_records_info()
+--
+CREATE FUNCTION pg_get_wal_records_info(IN start_lsn pg_lsn,
+ IN end_lsn pg_lsn DEFAULT NULL,
+ OUT start_lsn pg_lsn,
+ OUT end_lsn pg_lsn,
+ OUT prev_lsn pg_lsn,
+ OUT xid xid,
+ OUT resource_manager text,
+ OUT record_type text,
+ OUT record_length int4,
+ OUT main_data_length int4,
+ OUT fpi_length int4,
+ OUT description text,
+ OUT block_ref text
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_get_wal_records_info'
+LANGUAGE C CALLED ON NULL INPUT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_wal_records_info(pg_lsn, pg_lsn) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_wal_records_info(pg_lsn, pg_lsn) TO pg_read_server_files;
+
+--
+-- pg_get_wal_stats()
+--
+CREATE FUNCTION pg_get_wal_stats(IN start_lsn pg_lsn,
+ IN end_lsn pg_lsn DEFAULT NULL,
+ IN per_record boolean DEFAULT false,
+ OUT "resource_manager/record_type" text,
+ OUT count int8,
+ OUT count_percentage float8,
+ OUT record_size int8,
+ OUT record_size_percentage float8,
+ OUT fpi_size int8,
+ OUT fpi_size_percentage float8,
+ OUT combined_size int8,
+ OUT combined_size_percentage float8
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_get_wal_stats'
+LANGUAGE C CALLED ON NULL INPUT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_wal_stats(pg_lsn, pg_lsn, boolean) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_wal_stats(pg_lsn, pg_lsn, boolean) TO pg_read_server_files;
+
+--
+-- pg_get_wal_fpi_info()
+--
+CREATE FUNCTION pg_get_wal_fpi_info(IN start_lsn pg_lsn,
+ IN end_lsn pg_lsn DEFAULT NULL,
+ OUT lsn pg_lsn,
+ OUT reltablespace oid,
+ OUT reldatabase oid,
+ OUT relfilenode oid,
+ OUT relblocknumber int8,
+ OUT forkname text,
+ OUT fpi bytea
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_get_wal_fpi_info'
+LANGUAGE C CALLED ON NULL INPUT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_wal_fpi_info(pg_lsn, pg_lsn) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_wal_fpi_info(pg_lsn, pg_lsn) TO pg_read_server_files;
diff --git a/contrib/pg_walinspect/pg_walinspect.c b/contrib/pg_walinspect/pg_walinspect.c
index b7b0a805ee..7d57a8b062 100644
--- a/contrib/pg_walinspect/pg_walinspect.c
+++ b/contrib/pg_walinspect/pg_walinspect.c
@@ -37,13 +37,13 @@ PG_FUNCTION_INFO_V1(pg_get_wal_records_info_till_end_of_wal);
PG_FUNCTION_INFO_V1(pg_get_wal_stats);
PG_FUNCTION_INFO_V1(pg_get_wal_stats_till_end_of_wal);
-static bool IsFutureLSN(XLogRecPtr lsn, XLogRecPtr *curr_lsn);
+static XLogRecPtr GetCurrentLSN(void);
static XLogReaderState *InitXLogReaderState(XLogRecPtr lsn);
static XLogRecord *ReadNextXLogRecord(XLogReaderState *xlogreader);
static void GetWALRecordInfo(XLogReaderState *record, Datum *values,
bool *nulls, uint32 ncols);
-static XLogRecPtr ValidateInputLSNs(bool till_end_of_wal,
- XLogRecPtr start_lsn, XLogRecPtr end_lsn);
+static void ValidateInputLSNs(FunctionCallInfo fcinfo, XLogRecPtr *start_lsn,
+ XLogRecPtr *end_lsn);
static void GetWALRecordsInfo(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
XLogRecPtr end_lsn);
static void GetXLogSummaryStats(XLogStats *stats, ReturnSetInfo *rsinfo,
@@ -59,27 +59,25 @@ static void GetWalStats(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
static void GetWALFPIInfo(FunctionCallInfo fcinfo, XLogReaderState *record);
/*
- * Check if the given LSN is in future. Also, return the LSN up to which the
- * server has WAL.
+ * Return the LSN up to which the server has WAL.
*/
-static bool
-IsFutureLSN(XLogRecPtr lsn, XLogRecPtr *curr_lsn)
+static XLogRecPtr
+GetCurrentLSN(void)
{
+ XLogRecPtr curr_lsn;
+
/*
* We determine the current LSN of the server similar to how page_read
* callback read_local_xlog_page_no_wait does.
*/
if (!RecoveryInProgress())
- *curr_lsn = GetFlushRecPtr(NULL);
+ curr_lsn = GetFlushRecPtr(NULL);
else
- *curr_lsn = GetXLogReplayRecPtr(NULL);
-
- Assert(!XLogRecPtrIsInvalid(*curr_lsn));
+ curr_lsn = GetXLogReplayRecPtr(NULL);
- if (lsn >= *curr_lsn)
- return true;
+ Assert(!XLogRecPtrIsInvalid(curr_lsn));
- return false;
+ return curr_lsn;
}
/*
@@ -295,6 +293,11 @@ GetWALFPIInfo(FunctionCallInfo fcinfo, XLogReaderState *record)
* records between start and end LSNs. Decompression is applied to the
* blocks, if necessary.
*
+ * This function determines end WAL LSN if it is not specified or specified as
+ * NULL. In such cases, the end WAL LSN is assigned with the last flushed WAL
+ * LSN when not in recovery or the last replayed WAL record LSN when in
+ * recovery.
+ *
* This function emits an error if a future start or end WAL LSN i.e. WAL LSN
* the database system doesn't know about is specified.
*/
@@ -307,11 +310,7 @@ pg_get_wal_fpi_info(PG_FUNCTION_ARGS)
MemoryContext old_cxt;
MemoryContext tmp_cxt;
- start_lsn = PG_GETARG_LSN(0);
- end_lsn = PG_GETARG_LSN(1);
-
- end_lsn = ValidateInputLSNs(false, start_lsn, end_lsn);
-
+ ValidateInputLSNs(fcinfo, &start_lsn, &end_lsn);
InitMaterializedSRF(fcinfo, 0);
xlogreader = InitXLogReaderState(start_lsn);
@@ -363,7 +362,9 @@ pg_get_wal_record_info(PG_FUNCTION_ARGS)
lsn = PG_GETARG_LSN(0);
- if (IsFutureLSN(lsn, &curr_lsn))
+ curr_lsn = GetCurrentLSN();
+
+ if (lsn >= curr_lsn)
{
/*
* GetFlushRecPtr or GetXLogReplayRecPtr gives "end+1" LSN of the last
@@ -402,44 +403,64 @@ pg_get_wal_record_info(PG_FUNCTION_ARGS)
}
/*
- * Validate the input LSNs and compute end LSN for till_end_of_wal versions.
+ * Validate input LSNs and return start_lsn and end_lsn. Also, compute end LSN
+ * in case it is not specified.
*/
-static XLogRecPtr
-ValidateInputLSNs(bool till_end_of_wal, XLogRecPtr start_lsn,
- XLogRecPtr end_lsn)
+static void
+ValidateInputLSNs(FunctionCallInfo fcinfo, XLogRecPtr *start_lsn,
+ XLogRecPtr *end_lsn)
{
- XLogRecPtr curr_lsn;
+ XLogRecPtr c_lsn;
+ XLogRecPtr s_lsn;
+ XLogRecPtr e_lsn;
- if (IsFutureLSN(start_lsn, &curr_lsn))
- {
- /*
- * GetFlushRecPtr or GetXLogReplayRecPtr gives "end+1" LSN of the last
- * record flushed or replayed respectively. But let's use the LSN up
- * to "end" in user facing message.
- */
+ if (PG_ARGISNULL(0))
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("cannot accept future start LSN"),
- errdetail("Last known WAL LSN on the database system is at %X/%X.",
- LSN_FORMAT_ARGS(curr_lsn))));
- }
+ errmsg("WAL start LSN cannot be null")));
- if (till_end_of_wal)
- end_lsn = curr_lsn;
+ s_lsn = PG_GETARG_LSN(0);
- if (end_lsn > curr_lsn)
+ if (XLogRecPtrIsInvalid(s_lsn))
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("cannot accept future end LSN"),
- errdetail("Last known WAL LSN on the database system is at %X/%X.",
- LSN_FORMAT_ARGS(curr_lsn))));
+ errmsg("WAL start LSN cannot be invalid")));
- if (start_lsn >= end_lsn)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("WAL start LSN must be less than end LSN")));
+ c_lsn = GetCurrentLSN();
+
+ if (PG_ARGISNULL(1))
+ e_lsn = c_lsn;
+ else
+ {
+ e_lsn = PG_GETARG_LSN(1);
+
+ if (XLogRecPtrIsInvalid(e_lsn))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("WAL end LSN cannot be invalid")));
- return end_lsn;
+ if (e_lsn > c_lsn)
+ {
+ /*
+ * GetFlushRecPtr or GetXLogReplayRecPtr gives "end+1" LSN of the
+ * last record flushed or replayed respectively. But let's use the
+ * LSN up to "end" in user facing message.
+ */
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("cannot accept future end LSN"),
+ errdetail("Last known WAL LSN on the database system is at %X/%X.",
+ LSN_FORMAT_ARGS(c_lsn))));
+ }
+
+ if (s_lsn >= e_lsn)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("WAL start LSN must be less than end LSN")));
+ }
+
+ *start_lsn = s_lsn;
+ *end_lsn = e_lsn;
}
/*
@@ -494,6 +515,11 @@ GetWALRecordsInfo(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
/*
* Get info and data of all WAL records between start LSN and end LSN.
*
+ * This function determines end WAL LSN if it is not specified or specified as
+ * NULL. In such cases, the end WAL LSN is assigned with the last flushed WAL
+ * LSN when not in recovery or the last replayed WAL record LSN when in
+ * recovery.
+ *
* This function emits an error if a future start or end WAL LSN i.e. WAL LSN
* the database system doesn't know about is specified.
*/
@@ -503,35 +529,23 @@ pg_get_wal_records_info(PG_FUNCTION_ARGS)
XLogRecPtr start_lsn;
XLogRecPtr end_lsn;
- start_lsn = PG_GETARG_LSN(0);
- end_lsn = PG_GETARG_LSN(1);
-
- end_lsn = ValidateInputLSNs(false, start_lsn, end_lsn);
-
+ ValidateInputLSNs(fcinfo, &start_lsn, &end_lsn);
GetWALRecordsInfo(fcinfo, start_lsn, end_lsn);
PG_RETURN_VOID();
}
/*
- * Get info and data of all WAL records from start LSN till end of WAL.
+ * NB: This function does nothing and stays here for backward compatibility.
+ * Without it, the extension fails to install.
*
- * This function emits an error if a future start i.e. WAL LSN the database
- * system doesn't know about is specified.
+ * Try using pg_get_wal_records_info() for the same till_end_of_wal
+ * functionaility.
*/
Datum
pg_get_wal_records_info_till_end_of_wal(PG_FUNCTION_ARGS)
{
- XLogRecPtr start_lsn;
- XLogRecPtr end_lsn = InvalidXLogRecPtr;
-
- start_lsn = PG_GETARG_LSN(0);
-
- end_lsn = ValidateInputLSNs(true, start_lsn, end_lsn);
-
- GetWALRecordsInfo(fcinfo, start_lsn, end_lsn);
-
- PG_RETURN_VOID();
+ PG_RETURN_NULL();
}
/*
@@ -732,6 +746,11 @@ GetWalStats(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
/*
* Get stats of all WAL records between start LSN and end LSN.
*
+ * This function determines end WAL LSN if it is not specified or specified as
+ * NULL. In such cases, the end WAL LSN is assigned with the last flushed WAL
+ * LSN when not in recovery or the last replayed WAL record LSN when in
+ * recovery.
+ *
* This function emits an error if a future start or end WAL LSN i.e. WAL LSN
* the database system doesn't know about is specified.
*/
@@ -742,36 +761,23 @@ pg_get_wal_stats(PG_FUNCTION_ARGS)
XLogRecPtr end_lsn;
bool stats_per_record;
- start_lsn = PG_GETARG_LSN(0);
- end_lsn = PG_GETARG_LSN(1);
+ ValidateInputLSNs(fcinfo, &start_lsn, &end_lsn);
stats_per_record = PG_GETARG_BOOL(2);
- end_lsn = ValidateInputLSNs(false, start_lsn, end_lsn);
-
GetWalStats(fcinfo, start_lsn, end_lsn, stats_per_record);
PG_RETURN_VOID();
}
/*
- * Get stats of all WAL records from start LSN till end of WAL.
+ * NB: This function does nothing and stays here for backward compatibility.
+ * Without it, the extension fails to install.
*
- * This function emits an error if a future start i.e. WAL LSN the database
- * system doesn't know about is specified.
+ * Try using pg_get_wal_records_info() for the same till_end_of_wal
+ * functionaility.
*/
Datum
pg_get_wal_stats_till_end_of_wal(PG_FUNCTION_ARGS)
{
- XLogRecPtr start_lsn;
- XLogRecPtr end_lsn = InvalidXLogRecPtr;
- bool stats_per_record;
-
- start_lsn = PG_GETARG_LSN(0);
- stats_per_record = PG_GETARG_BOOL(1);
-
- end_lsn = ValidateInputLSNs(true, start_lsn, end_lsn);
-
- GetWalStats(fcinfo, start_lsn, end_lsn, stats_per_record);
-
- PG_RETURN_VOID();
+ PG_RETURN_NULL();
}
diff --git a/contrib/pg_walinspect/pg_walinspect.control b/contrib/pg_walinspect/pg_walinspect.control
index efa3cb2cfe..5f574b865b 100644
--- a/contrib/pg_walinspect/pg_walinspect.control
+++ b/contrib/pg_walinspect/pg_walinspect.control
@@ -1,5 +1,5 @@
# pg_walinspect extension
comment = 'functions to inspect contents of PostgreSQL Write-Ahead Log'
-default_version = '1.1'
+default_version = '1.2'
module_pathname = '$libdir/pg_walinspect'
relocatable = true
diff --git a/contrib/pg_walinspect/sql/oldextversions.sql b/contrib/pg_walinspect/sql/oldextversions.sql
new file mode 100644
index 0000000000..ead4997d37
--- /dev/null
+++ b/contrib/pg_walinspect/sql/oldextversions.sql
@@ -0,0 +1,27 @@
+-- test old extension version entry points
+
+DROP EXTENSION pg_walinspect;
+CREATE EXTENSION pg_walinspect VERSION '1.1';
+
+-- New function introduced in 1.1
+SELECT pg_get_functiondef('pg_get_wal_fpi_info'::regproc);
+
+-- Move to new version 1.2
+ALTER EXTENSION pg_walinspect UPDATE TO '1.2';
+
+-- Redefined function
+SELECT pg_get_functiondef('pg_get_wal_records_info'::regproc);
+
+-- Removed function
+SELECT pg_get_functiondef('pg_get_wal_records_info_till_end_of_wal'::regproc);
+
+-- Redefined function
+SELECT pg_get_functiondef('pg_get_wal_stats'::regproc);
+
+-- Removed function
+SELECT pg_get_functiondef('pg_get_wal_stats_till_end_of_wal'::regproc);
+
+-- Redefined function
+SELECT pg_get_functiondef('pg_get_wal_fpi_info'::regproc);
+
+DROP EXTENSION pg_walinspect;
diff --git a/contrib/pg_walinspect/sql/pg_walinspect.sql b/contrib/pg_walinspect/sql/pg_walinspect.sql
index 849201a1f8..64fd95f387 100644
--- a/contrib/pg_walinspect/sql/pg_walinspect.sql
+++ b/contrib/pg_walinspect/sql/pg_walinspect.sql
@@ -17,10 +17,30 @@ INSERT INTO sample_tbl SELECT * FROM generate_series(3, 4);
-- Tests for input validation
-- ===================================================================
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info('0/0'); -- ERROR
+
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn2', :'wal_lsn1'); -- ERROR
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(NULL, NULL); -- ERROR
+
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', '0/0'); -- ERROR
+
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats('0/0'); -- ERROR
+
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn2', :'wal_lsn1'); -- ERROR
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(NULL, NULL); -- ERROR
+
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', '0/0'); -- ERROR
+
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_fpi_info('0/0'); -- ERROR
+
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_fpi_info(:'wal_lsn2', :'wal_lsn1'); -- ERROR
+
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_fpi_info(NULL, NULL); -- ERROR
+
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_fpi_info(:'wal_lsn1', '0/0'); -- ERROR
+
-- ===================================================================
-- Tests for all function executions
-- ===================================================================
@@ -29,11 +49,11 @@ SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info_till_end_of_wal(:'wal_lsn1');
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1');
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats_till_end_of_wal(:'wal_lsn1');
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1');
-- ===================================================================
-- Test for filtering out WAL records of a particular table
@@ -68,6 +88,10 @@ SELECT pg_current_wal_lsn() AS wal_lsn4 \gset
SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_fpi_info(:'wal_lsn3', :'wal_lsn4')
WHERE relfilenode = :'sample_tbl_oid';
+-- Check if we get FPI from WAL record.
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_fpi_info(:'wal_lsn3')
+ WHERE relfilenode = :'sample_tbl_oid';
+
-- ===================================================================
-- Tests for permissions
-- ===================================================================
diff --git a/doc/src/sgml/pgwalinspect.sgml b/doc/src/sgml/pgwalinspect.sgml
index 3d7cdb95cc..78b8eb36fd 100644
--- a/doc/src/sgml/pgwalinspect.sgml
+++ b/doc/src/sgml/pgwalinspect.sgml
@@ -85,7 +85,7 @@ block_ref | blkref #0: rel 1663/5/60221 fork main blk 2
<varlistentry id="pgwalinspect-funcs-pg-get-wal-records-info">
<term>
<function>
- pg_get_wal_records_info(start_lsn pg_lsn, end_lsn pg_lsn)
+ pg_get_wal_records_info(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL)
returns setof record
</function>
</term>
@@ -94,9 +94,10 @@ block_ref | blkref #0: rel 1663/5/60221 fork main blk 2
<para>
Gets information of all the valid WAL records between
<replaceable>start_lsn</replaceable> and <replaceable>end_lsn</replaceable>.
- Returns one row per WAL record. If <replaceable>start_lsn</replaceable>
- or <replaceable>end_lsn</replaceable> are not yet available, the
- function will raise an error. For example:
+ Returns one row per WAL record. If <replaceable>end_lsn</replaceable>
+ isn't specified, it returns information till the end of WAL.
+ If <replaceable>start_lsn</replaceable> or <replaceable>end_lsn</replaceable>
+ are not yet available, the function will raise an error. For example:
<screen>
postgres=# SELECT * FROM pg_get_wal_records_info('0/1E913618', '0/1E913740') LIMIT 1;
-[ RECORD 1 ]----+--------------------------------------------------------------
@@ -116,27 +117,10 @@ block_ref |
</listitem>
</varlistentry>
- <varlistentry id="pgwalinspect-funcs-pg-get-wal-records-info-till-end-of-wal">
- <term>
- <function>
- pg_get_wal_records_info_till_end_of_wal(start_lsn pg_lsn)
- returns setof record
- </function>
- </term>
-
- <listitem>
- <para>
- This function is the same as <function>pg_get_wal_records_info()</function>,
- except that it gets information of all the valid WAL records from
- <replaceable>start_lsn</replaceable> till the end of WAL.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry id="pgwalinspect-funcs-pg-get-wal-stats">
<term>
<function>
- pg_get_wal_stats(start_lsn pg_lsn, end_lsn pg_lsn, per_record boolean DEFAULT false)
+ pg_get_wal_stats(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL, per_record boolean DEFAULT false)
returns setof record
</function>
</term>
@@ -149,7 +133,8 @@ block_ref |
<replaceable>resource_manager</replaceable> type. When
<replaceable>per_record</replaceable> is set to <literal>true</literal>,
it returns one row per <replaceable>record_type</replaceable>.
- If <replaceable>start_lsn</replaceable>
+ If <replaceable>end_lsn</replaceable> isn't specified, it returns
+ information till the end of WAL. If <replaceable>start_lsn</replaceable>
or <replaceable>end_lsn</replaceable> are not yet available, the
function will raise an error. For example:
<screen>
@@ -171,28 +156,14 @@ combined_size_percentage | 2.8634072910530795
</listitem>
</varlistentry>
- <varlistentry id="pgwalinspect-funcs-pg-get-wal-stats-till-end-of-wal">
+ <varlistentry>
<term>
<function>
- pg_get_wal_stats_till_end_of_wal(start_lsn pg_lsn, per_record boolean DEFAULT false)
+ pg_get_wal_fpi_info(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL)
returns setof record
</function>
</term>
- <listitem>
- <para>
- This function is the same as <function>pg_get_wal_stats()</function>,
- except that it gets statistics of all the valid WAL records from
- <replaceable>start_lsn</replaceable> till end of WAL.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term>
- <function>pg_get_wal_fpi_info(start_lsn pg_lsn, end_lsn pg_lsn) returns setof record</function>
- </term>
-
<listitem>
<para>
Gets a copy of full page images as <type>bytea</type> values (after
@@ -200,8 +171,9 @@ combined_size_percentage | 2.8634072910530795
with all the valid WAL records between
<replaceable>start_lsn</replaceable> and
<replaceable>end_lsn</replaceable>. Returns one row per full page image.
- If <replaceable>start_lsn</replaceable> or
- <replaceable>end_lsn</replaceable> are not yet available, the function
+ If <replaceable>end_lsn</replaceable> isn't specified, it returns
+ information till the end of WAL. If <replaceable>start_lsn</replaceable>
+ or <replaceable>end_lsn</replaceable> are not yet available, the function
will raise an error. For example:
<screen>
postgres=# SELECT lsn, reltablespace, reldatabase, relfilenode, relblocknumber,
--
2.34.1
Re: Combine pg_walinspect till_end_of_wal functions with others
On Wed, Mar 01, 2023 at 08:30:00PM +0530, Bharath Rupireddy wrote:
On Wed, Mar 1, 2023 at 1:00 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:In a recent discussion [1], Michael Paquier asked if we can combine
pg_walinspect till_end_of_wal functions with other functions
pg_get_wal_records_info and pg_get_wal_stats. The code currently looks
much duplicated and the number of functions that pg_walinspect exposes
to the users is bloated. The point was that the till_end_of_wal
functions determine the end LSN and everything else that they do is
the same as their counterpart functions. Well, the idea then was to
keep things simple, not clutter the APIs, have better and consistent
user-inputted end_lsn validations at the cost of usability and code
redundancy. However, now I tend to agree with the feedback received.
+1, especially since I really don't like the use of "till" in the function
names.
I'm attaching a patch doing the $subject with the following behavior:
1. If start_lsn is NULL, error out/return NULL.
Maybe naive and unrelated question, but is that really helpful? If for some
reason I want to see information about *all available WAL*, I have to manually
dig for a suitable LSN. The same action with pg_waldump is easier as I just
need to use the oldest available WAL that's present on disk.
Another idea is to convert till_end_of_wal flavors to SQL-only
functions and remove the c code from pg_walinspect.c. However, I
prefer $subject and completely remove till_end_of_wal flavors for
better usability in the long term.
I agree that using default arguments is a way better API.
Nitpicking:
Maybe we could group the kept unused exported C function at the end of the
file?
Also:
/*
- * Get info and data of all WAL records from start LSN till end of WAL.
+ * NB: This function does nothing and stays here for backward compatibility.
+ * Without it, the extension fails to install.
*
- * This function emits an error if a future start i.e. WAL LSN the database
- * system doesn't know about is specified.
+ * Try using pg_get_wal_records_info() for the same till_end_of_wal
+ * functionaility.
*/
Datum
pg_get_wal_records_info_till_end_of_wal(PG_FUNCTION_ARGS)
{
- XLogRecPtr start_lsn;
- XLogRecPtr end_lsn = InvalidXLogRecPtr;
-
- start_lsn = PG_GETARG_LSN(0);
-
- end_lsn = ValidateInputLSNs(true, start_lsn, end_lsn);
-
- GetWALRecordsInfo(fcinfo, start_lsn, end_lsn);
-
- PG_RETURN_VOID();
+ PG_RETURN_NULL();
}I don't like much this chunk (same for the other kept function). Apart from
the obvious typo in "functionaility", I don't think that the comment is really
accurate.
Also, are we actually helping users if we simply return NULL there? It's quite
possible that people will start to use the new shared lib while still having
the 1.1 SQL definition of the extension installed. In that case, they will
simply retrieve a NULL row and may spend some time wondering why until they
eventually realize that their only option is to upgrade the extension first and
then use another function. Why not make their life easier and explicity raise
a suitable error at the SQL level if users try to use those functions?
Re: Combine pg_walinspect till_end_of_wal functions with others
On Mon, Mar 6, 2023 at 2:22 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
I'm attaching a patch doing the $subject with the following behavior:
1. If start_lsn is NULL, error out/return NULL.Maybe naive and unrelated question, but is that really helpful? If for some
reason I want to see information about *all available WAL*, I have to manually
dig for a suitable LSN. The same action with pg_waldump is easier as I just
need to use the oldest available WAL that's present on disk.
Are you saying that the pg_walinspect functions should figure out the
oldest available WAL file and LSN, and start from there if start_lsn
specified as NULL or invalid? Note that pg_waldump requires either
explicit startlsn and/or startseg (WAL file name), it can't search for
the oldest WAL file available and start from there automatically.
If the user wants to figure it out, they can do something like below:
ostgres=# select * from pg_ls_waldir() order by name;
name | size | modification
--------------------------+----------+------------------------
000000010000000000000001 | 16777216 | 2023-03-06 14:54:55+00
000000010000000000000002 | 16777216 | 2023-03-06 14:54:55+00
If we try to make these functions figure out the oldest WAl file and
start from there, then it'll unnecessarily complicate the APIs and
functions. If we still think we need a better function for the users
to figure out the oldest WAL file, perhaps, add a SQL-only
view/function to pg_walinspect that returns "select name from
pg_ls_waldir() order by name limit 1;", but honestly, that's so
trivial.
Another idea is to convert till_end_of_wal flavors to SQL-only
functions and remove the c code from pg_walinspect.c. However, I
prefer $subject and completely remove till_end_of_wal flavors for
better usability in the long term.I agree that using default arguments is a way better API.
Thanks. Yes, that's true.
Nitpicking:
Maybe we could group the kept unused exported C function at the end of the
file?
Will do.
Also:
/* - * Get info and data of all WAL records from start LSN till end of WAL. + * NB: This function does nothing and stays here for backward compatibility. + * Without it, the extension fails to install. * - * This function emits an error if a future start i.e. WAL LSN the database - * system doesn't know about is specified. + * Try using pg_get_wal_records_info() for the same till_end_of_wal + * functionaility.I don't like much this chunk (same for the other kept function). Apart from
the obvious typo in "functionaility", I don't think that the comment is really
accurate.
Can you be more specific what's inaccurate about the comment?
Also, are we actually helping users if we simply return NULL there? It's quite
possible that people will start to use the new shared lib while still having
the 1.1 SQL definition of the extension installed. In that case, they will
simply retrieve a NULL row and may spend some time wondering why until they
eventually realize that their only option is to upgrade the extension first and
then use another function. Why not make their life easier and explicity raise
a suitable error at the SQL level if users try to use those functions?
I thought about it initially, but wanted to avoid more errors. An
error would make them use the new version easily. I will change it
that way.
--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
Re: Combine pg_walinspect till_end_of_wal functions with others
On Mon, 6 Mar 2023 at 16:06, Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:
If we try to make these functions figure out the oldest WAl file and
start from there, then it'll unnecessarily complicate the APIs and
functions. If we still think we need a better function for the users
to figure out the oldest WAL file, perhaps, add a SQL-only
view/function to pg_walinspect that returns "select name from
pg_ls_waldir() order by name limit 1;", but honestly, that's so
trivial.
That "order by name limit 1" has subtle bugs when you're working on a
system that has experienced timeline switches. It is entirely possible
that the first file (as sorted by the default collation) is not the
first record you can inspect, or even in your timeline's history.
Kind regards,
Matthias van de Meent
Re: Combine pg_walinspect till_end_of_wal functions with others
On Mon, Mar 6, 2023 at 8:52 PM Matthias van de Meent
<boekewurm+postgres@gmail.com> wrote:
On Mon, 6 Mar 2023 at 16:06, Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:If we try to make these functions figure out the oldest WAl file and
start from there, then it'll unnecessarily complicate the APIs and
functions. If we still think we need a better function for the users
to figure out the oldest WAL file, perhaps, add a SQL-only
view/function to pg_walinspect that returns "select name from
pg_ls_waldir() order by name limit 1;", but honestly, that's so
trivial.That "order by name limit 1" has subtle bugs when you're working on a
system that has experienced timeline switches. It is entirely possible
that the first file (as sorted by the default collation) is not the
first record you can inspect, or even in your timeline's history.
Hm. Note that pg_walinspect currently searches WAL on insertion
timeline; it doesn't care about the older timelines. The idea of
making it look at WAL on an older timeline was discussed, but for the
sake of simplicity we kept the functions simple. If needed, I can try
adding the timeline as input parameters to all the functions (with
default -1 meaning current insertion timeline; if specified, look for
WAL on that timeline).
Are you saying that a pg_walinspect function that traverses the pg_wal
directory and figures out the old valid WAL on a given timeline is
still useful? Or make the functions look for older WAL if start_lsn is
given as NULL or invalid?
--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
Re: Combine pg_walinspect till_end_of_wal functions with others
On Mon, 6 Mar 2023 at 16:37, Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:
On Mon, Mar 6, 2023 at 8:52 PM Matthias van de Meent
<boekewurm+postgres@gmail.com> wrote:On Mon, 6 Mar 2023 at 16:06, Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:If we try to make these functions figure out the oldest WAl file and
start from there, then it'll unnecessarily complicate the APIs and
functions. If we still think we need a better function for the users
to figure out the oldest WAL file, perhaps, add a SQL-only
view/function to pg_walinspect that returns "select name from
pg_ls_waldir() order by name limit 1;", but honestly, that's so
trivial.That "order by name limit 1" has subtle bugs when you're working on a
system that has experienced timeline switches. It is entirely possible
that the first file (as sorted by the default collation) is not the
first record you can inspect, or even in your timeline's history.Hm. Note that pg_walinspect currently searches WAL on insertion
timeline; it doesn't care about the older timelines. The idea of
making it look at WAL on an older timeline was discussed, but for the
sake of simplicity we kept the functions simple. If needed, I can try
adding the timeline as input parameters to all the functions (with
default -1 meaning current insertion timeline; if specified, look for
WAL on that timeline).Are you saying that a pg_walinspect function that traverses the pg_wal
directory and figures out the old valid WAL on a given timeline is
still useful? Or make the functions look for older WAL if start_lsn is
given as NULL or invalid?
The specific comment I made was only regarding the following issue: An
instance may still have WAL segments from before the latest timeline
switch. These segments may have a higher LSN and lower timeline number
than your current running timeline+LSN (because of e.g. pg_rewind).
This will then result in unwanted behaviour when you sort the segments
numerically/alphabetically and then assume that the first file's LSN
is valid (or available) in your current timeline.
That is why "order by name limit 1" isn't a good solution, and that's
what I was commenting on: you need to parse the timeline hierarchy to
determine which timelines you can use which WAL segments of.
To answer your question on whether I'd like us to traverse timeline
switches: Yes, I'd really like it if we were able to decode the
current timeline's hierarchical WAL of a PG instance in one go, from
the start at (iirc) 0x10000 all the way to the current LSN, assuming
the segments are available.
Kind regards,
Matthias van de Meent
Re: Combine pg_walinspect till_end_of_wal functions with others
On Mon, Mar 06, 2023 at 08:36:17PM +0530, Bharath Rupireddy wrote:
On Mon, Mar 6, 2023 at 2:22 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
Also:
/* - * Get info and data of all WAL records from start LSN till end of WAL. + * NB: This function does nothing and stays here for backward compatibility. + * Without it, the extension fails to install. * - * This function emits an error if a future start i.e. WAL LSN the database - * system doesn't know about is specified. + * Try using pg_get_wal_records_info() for the same till_end_of_wal + * functionaility.I don't like much this chunk (same for the other kept function). Apart from
the obvious typo in "functionaility", I don't think that the comment is really
accurate.Can you be more specific what's inaccurate about the comment?
It's problematic to install the extension if we rely on upgrade scripts only.
We could also provide a pg_walinspect--1.2.sql file and it would just work, and
that may have been a good idea if there wasn't also the problem of people still
having the version 1.1 locally installed, as we don't want them to see random
failures like "could not find function ... in file ...", or keeping the ability
to install the former 1.1 version (with those functions bypassed).
Re: Combine pg_walinspect till_end_of_wal functions with others
On Tue, Mar 07, 2023 at 09:13:46AM +0800, Julien Rouhaud wrote:
It's problematic to install the extension if we rely on upgrade scripts only.
We could also provide a pg_walinspect--1.2.sql file and it would just work, and
that may have been a good idea if there wasn't also the problem of people still
having the version 1.1 locally installed, as we don't want them to see random
failures like "could not find function ... in file ...", or keeping the ability
to install the former 1.1 version (with those functions bypassed).
Why would we need a 1.2? HEAD is the only branch with pg_walinspect
1.1, and it has not been released yet.
--
Michael
Re: Combine pg_walinspect till_end_of_wal functions with others
On Tue, 7 Mar 2023, 12:36 Michael Paquier, <michael@paquier.xyz> wrote:
On Tue, Mar 07, 2023 at 09:13:46AM +0800, Julien Rouhaud wrote:
It's problematic to install the extension if we rely on upgrade scripts
only.
We could also provide a pg_walinspect--1.2.sql file and it would just
work, and
that may have been a good idea if there wasn't also the problem of
people still
having the version 1.1 locally installed, as we don't want them to see
random
failures like "could not find function ... in file ...", or keeping the
ability
to install the former 1.1 version (with those functions bypassed).
Why would we need a 1.2? HEAD is the only branch with pg_walinspect
1.1, and it has not been released yet.
ah right I should have checked. but the same ABI compatibility concern
still exists for version 1.0 of the extension.
Show quoted text
Re: Combine pg_walinspect till_end_of_wal functions with others
On Tue, Mar 07, 2023 at 12:42:20PM +0800, Julien Rouhaud wrote:
ah right I should have checked. but the same ABI compatibility concern
still exists for version 1.0 of the extension.
Yes, we'd better make sure that the past code is able to run, at
least. Now I am not really convinced that we have the need to enforce
an error with the new code even if 1.0 is still installed, so as it is
possible to remove all the traces of the code that triggers errors if
an end LSN is higher than the current insert LSN for primaries or
replayed LSN for standbys.
--
Michael
Re: Combine pg_walinspect till_end_of_wal functions with others
On Tue, Mar 07, 2023 at 01:56:24PM +0900, Michael Paquier wrote:
On Tue, Mar 07, 2023 at 12:42:20PM +0800, Julien Rouhaud wrote:
ah right I should have checked. but the same ABI compatibility concern
still exists for version 1.0 of the extension.Yes, we'd better make sure that the past code is able to run, at
least. Now I am not really convinced that we have the need to enforce
an error with the new code even if 1.0 is still installed,
So keep this "deprecated" C function working, as it would only be a few lines
of code?
so as it is
possible to remove all the traces of the code that triggers errors if
an end LSN is higher than the current insert LSN for primaries or
replayed LSN for standbys.
+1 for that
Re: Combine pg_walinspect till_end_of_wal functions with others
On Tue, Mar 7, 2023 at 11:17 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Tue, Mar 07, 2023 at 01:56:24PM +0900, Michael Paquier wrote:
On Tue, Mar 07, 2023 at 12:42:20PM +0800, Julien Rouhaud wrote:
ah right I should have checked. but the same ABI compatibility concern
still exists for version 1.0 of the extension.Yes, we'd better make sure that the past code is able to run, at
least. Now I am not really convinced that we have the need to enforce
an error with the new code even if 1.0 is still installed,So keep this "deprecated" C function working, as it would only be a few lines
of code?so as it is
possible to remove all the traces of the code that triggers errors if
an end LSN is higher than the current insert LSN for primaries or
replayed LSN for standbys.+1 for that
I understand that we want to keep till_end_of_wal functions defined
around in .c file so that if someone does CREATE EXTENSION
pg_walinspect WITH VERSION '1.0'; on the latest extension shared
library (with 1.1 version), the till_end_of_wal functions should work
for them.
Also, I noticed that there's some improvement needed for the input
validations, especially for the end_lsn.
Here I'm with the v3 patch addressing the above comments. Please
review it further.
1. When start_lsn is NULL or invalid ('0/0'), emit an error. There was
a comment on the functions automatically determining start_lsn to be
the oldest WAL LSN. I'm not implementing this change now, as it
requires extra work to traverse the pg_wal directory. I'm planning to
do it in the next set of improvements where I'm planning to make the
functions timeline-aware, introduce functions for inspecting
wal_buffers and so on.
2. When end_lsn is NULL or invalid ('0/0') IOW end_lsn is not
specified, deduce end_lsn to be the current flush LSN when not in
recovery, current replayed LSN when in recovery. This is the main
change that avoids till_end_of_wal functions in version 1.1.
3. When end_lsn is specified but greater than or equal to the
start_lsn, return NULL. Given the above review comments on more errors
being reported, I chose to return NULL for better usability.
4. When end_lsn is specified but less than the start_lsn, get
info/stats up until end_lsn.
5. Retained pg_get_wal_records_info_till_end_of_wal and
pg_get_wal_stats_till_end_of_wal for backward compatibility.
6. Piggybacked these functions and behaviour under the new HEAD-only
extension version 1.1 introduced recently, instead of bumping to 1.2.
When PG16 is out, users will have 1.1 with all of these new
functionality.
7. Added tests to verify the extension update path in
oldextversions.sql similar to other extensions'. (suggested by Michael
Paquier).
8. Added a note in the pg_walinspect documentation about removal of
pg_get_wal_records_info_till_end_of_wal and
pg_get_wal_stats_till_end_of_wal in version 1.1 and how the other
functions can be used to achieve the same functionality and how these
till_end_of_wal functions can work if extension is installed
explicitly with version 1.0.
9. Refactored the tests according to the new behaviours.
--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
Attachments:
v3-0001-Rework-pg_walinspect-functions.patchapplication/x-patch; name=v3-0001-Rework-pg_walinspect-functions.patchDownload
From 9d63a01bc634bbb899ce9f4645bf677bcbaec998 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Wed, 8 Mar 2023 07:39:36 +0000
Subject: [PATCH v3] Rework pg_walinspect functions
---
contrib/pg_walinspect/Makefile | 2 +-
.../pg_walinspect/expected/oldextversions.out | 64 +++++
.../pg_walinspect/expected/pg_walinspect.out | 87 ++++++-
contrib/pg_walinspect/meson.build | 1 +
.../pg_walinspect/pg_walinspect--1.0--1.1.sql | 57 ++++-
contrib/pg_walinspect/pg_walinspect.c | 235 +++++++++++-------
contrib/pg_walinspect/sql/oldextversions.sql | 27 ++
contrib/pg_walinspect/sql/pg_walinspect.sql | 51 +++-
doc/src/sgml/pgwalinspect.sgml | 71 ++----
9 files changed, 442 insertions(+), 153 deletions(-)
create mode 100644 contrib/pg_walinspect/expected/oldextversions.out
create mode 100644 contrib/pg_walinspect/sql/oldextversions.sql
diff --git a/contrib/pg_walinspect/Makefile b/contrib/pg_walinspect/Makefile
index 7033878a79..22090f7716 100644
--- a/contrib/pg_walinspect/Makefile
+++ b/contrib/pg_walinspect/Makefile
@@ -9,7 +9,7 @@ PGFILEDESC = "pg_walinspect - functions to inspect contents of PostgreSQL Write-
EXTENSION = pg_walinspect
DATA = pg_walinspect--1.0.sql pg_walinspect--1.0--1.1.sql
-REGRESS = pg_walinspect
+REGRESS = pg_walinspect oldextversions
REGRESS_OPTS = --temp-config $(top_srcdir)/contrib/pg_walinspect/walinspect.conf
diff --git a/contrib/pg_walinspect/expected/oldextversions.out b/contrib/pg_walinspect/expected/oldextversions.out
new file mode 100644
index 0000000000..c3ebcc2b49
--- /dev/null
+++ b/contrib/pg_walinspect/expected/oldextversions.out
@@ -0,0 +1,64 @@
+-- test old extension version entry points
+DROP EXTENSION pg_walinspect;
+CREATE EXTENSION pg_walinspect WITH VERSION '1.0';
+-- Move to new version 1.1
+ALTER EXTENSION pg_walinspect UPDATE TO '1.1';
+-- New function introduced in 1.1
+SELECT pg_get_functiondef('pg_get_wal_fpi_info'::regproc);
+ pg_get_functiondef
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ CREATE OR REPLACE FUNCTION public.pg_get_wal_fpi_info(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL::pg_lsn, OUT lsn pg_lsn, OUT reltablespace oid, OUT reldatabase oid, OUT relfilenode oid, OUT relblocknumber bigint, OUT forkname text, OUT fpi bytea)+
+ RETURNS SETOF record +
+ LANGUAGE c +
+ PARALLEL SAFE +
+ AS '$libdir/pg_walinspect', $function$pg_get_wal_fpi_info$function$ +
+
+(1 row)
+
+-- Redefined function
+SELECT pg_get_functiondef('pg_get_wal_records_info'::regproc);
+ pg_get_functiondef
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ CREATE OR REPLACE FUNCTION public.pg_get_wal_records_info(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL::pg_lsn, OUT start_lsn pg_lsn, OUT end_lsn pg_lsn, OUT prev_lsn pg_lsn, OUT xid xid, OUT resource_manager text, OUT record_type text, OUT record_length integer, OUT main_data_length integer, OUT fpi_length integer, OUT description text, OUT block_ref text)+
+ RETURNS SETOF record +
+ LANGUAGE c +
+ PARALLEL SAFE +
+ AS '$libdir/pg_walinspect', $function$pg_get_wal_records_info$function$ +
+
+(1 row)
+
+-- Removed function
+SELECT pg_get_functiondef('pg_get_wal_records_info_till_end_of_wal'::regproc);
+ERROR: function "pg_get_wal_records_info_till_end_of_wal" does not exist
+LINE 1: SELECT pg_get_functiondef('pg_get_wal_records_info_till_end_...
+ ^
+-- Redefined function
+SELECT pg_get_functiondef('pg_get_wal_stats'::regproc);
+ pg_get_functiondef
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ CREATE OR REPLACE FUNCTION public.pg_get_wal_stats(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL::pg_lsn, per_record boolean DEFAULT false, OUT "resource_manager/record_type" text, OUT count bigint, OUT count_percentage double precision, OUT record_size bigint, OUT record_size_percentage double precision, OUT fpi_size bigint, OUT fpi_size_percentage double precision, OUT combined_size bigint, OUT combined_size_percentage double precision)+
+ RETURNS SETOF record +
+ LANGUAGE c +
+ PARALLEL SAFE +
+ AS '$libdir/pg_walinspect', $function$pg_get_wal_stats$function$ +
+
+(1 row)
+
+-- Removed function
+SELECT pg_get_functiondef('pg_get_wal_stats_till_end_of_wal'::regproc);
+ERROR: function "pg_get_wal_stats_till_end_of_wal" does not exist
+LINE 1: SELECT pg_get_functiondef('pg_get_wal_stats_till_end_of_wal'...
+ ^
+-- Redefined function
+SELECT pg_get_functiondef('pg_get_wal_fpi_info'::regproc);
+ pg_get_functiondef
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ CREATE OR REPLACE FUNCTION public.pg_get_wal_fpi_info(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL::pg_lsn, OUT lsn pg_lsn, OUT reltablespace oid, OUT reldatabase oid, OUT relfilenode oid, OUT relblocknumber bigint, OUT forkname text, OUT fpi bytea)+
+ RETURNS SETOF record +
+ LANGUAGE c +
+ PARALLEL SAFE +
+ AS '$libdir/pg_walinspect', $function$pg_get_wal_fpi_info$function$ +
+
+(1 row)
+
+DROP EXTENSION pg_walinspect;
diff --git a/contrib/pg_walinspect/expected/pg_walinspect.out b/contrib/pg_walinspect/expected/pg_walinspect.out
index 9bcb05354e..71dcd8cfbb 100644
--- a/contrib/pg_walinspect/expected/pg_walinspect.out
+++ b/contrib/pg_walinspect/expected/pg_walinspect.out
@@ -14,38 +14,99 @@ INSERT INTO sample_tbl SELECT * FROM generate_series(3, 4);
-- ===================================================================
-- Tests for input validation
-- ===================================================================
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn2', :'wal_lsn1'); -- ERROR
-ERROR: WAL start LSN must be less than end LSN
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn2', :'wal_lsn1'); -- ERROR
-ERROR: WAL start LSN must be less than end LSN
+-- Invalid input LSNs
+SELECT * FROM pg_get_wal_record_info('0/0'); -- ERROR
+ERROR: invalid input LSN
+SELECT * FROM pg_get_wal_records_info('0/0'); -- ERROR
+ERROR: invalid start LSN
+SELECT * FROM pg_get_wal_stats('0/0'); -- ERROR
+ERROR: invalid start LSN
+SELECT * FROM pg_get_wal_fpi_info('0/0'); -- ERROR
+ERROR: invalid start LSN
+-- Start LSN NULL
+SELECT * FROM pg_get_wal_records_info(NULL); -- ERROR
+ERROR: invalid start LSN
+SELECT * FROM pg_get_wal_stats(NULL); -- ERROR
+ERROR: invalid start LSN
+SELECT * FROM pg_get_wal_fpi_info(NULL); -- ERROR
+ERROR: invalid start LSN
+-- Start LSN > End LSN
+SELECT * FROM pg_get_wal_records_info(:'wal_lsn2', :'wal_lsn1'); -- NULL
+ start_lsn | end_lsn | prev_lsn | xid | resource_manager | record_type | record_length | main_data_length | fpi_length | description | block_ref
+-----------+---------+----------+-----+------------------+-------------+---------------+------------------+------------+-------------+-----------
+ | | | | | | | | | |
+(1 row)
+
+SELECT * FROM pg_get_wal_stats(:'wal_lsn2', :'wal_lsn1'); -- NULL
+ resource_manager/record_type | count | count_percentage | record_size | record_size_percentage | fpi_size | fpi_size_percentage | combined_size | combined_size_percentage
+------------------------------+-------+------------------+-------------+------------------------+----------+---------------------+---------------+--------------------------
+ | | | | | | | |
+(1 row)
+
+SELECT * FROM pg_get_wal_fpi_info(:'wal_lsn2', :'wal_lsn1'); -- NULL
+ lsn | reltablespace | reldatabase | relfilenode | relblocknumber | forkname | fpi
+-----+---------------+-------------+-------------+----------------+----------+-----
+ | | | | | |
+(1 row)
+
-- ===================================================================
-- Tests for all function executions
-- ===================================================================
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
+ ok
+----
+ t
+(1 row)
+
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
+ ok
+----
+ t
+(1 row)
+
+-- Get info till end of WAL
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1');
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
+-- Get info till end of WAL
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', '0/0');
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info_till_end_of_wal(:'wal_lsn1');
+-- Get info till end of WAL
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', NULL);
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats_till_end_of_wal(:'wal_lsn1');
+-- Get stats till end of WAL
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1');
+ ok
+----
+ t
+(1 row)
+
+-- Get stats till end of WAL
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', '0/0');
+ ok
+----
+ t
+(1 row)
+
+-- Get stats till end of WAL
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', NULL);
ok
----
t
@@ -90,6 +151,14 @@ SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_fpi_info(:'wal_lsn3', :'wal_lsn4')
t
(1 row)
+-- Check till end of WAL if we get FPI from WAL record.
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_fpi_info(:'wal_lsn3')
+ WHERE relfilenode = :'sample_tbl_oid';
+ ok
+----
+ t
+(1 row)
+
-- ===================================================================
-- Tests for permissions
-- ===================================================================
diff --git a/contrib/pg_walinspect/meson.build b/contrib/pg_walinspect/meson.build
index bf7b79b1b7..80059f6119 100644
--- a/contrib/pg_walinspect/meson.build
+++ b/contrib/pg_walinspect/meson.build
@@ -30,6 +30,7 @@ tests += {
'regress': {
'sql': [
'pg_walinspect',
+ 'oldextversions',
],
# Disabled because these tests require "wal_level=replica", which
# some runningcheck users do not have (e.g. buildfarm clients).
diff --git a/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql b/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql
index 1e9e1e6115..219a088eb9 100644
--- a/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql
+++ b/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql
@@ -3,11 +3,64 @@
-- complain if script is sourced in psql, rather than via ALTER EXTENSION
\echo Use "ALTER EXTENSION pg_walinspect UPDATE TO '1.1'" to load this file. \quit
+/* Drop the unneeded functions and redefine needed ones */
+DROP FUNCTION pg_get_wal_records_info(pg_lsn, pg_lsn);
+DROP FUNCTION pg_get_wal_records_info_till_end_of_wal(pg_lsn);
+DROP FUNCTION pg_get_wal_stats(pg_lsn, pg_lsn, boolean);
+DROP FUNCTION pg_get_wal_stats_till_end_of_wal(pg_lsn, boolean);
+
+--
+-- pg_get_wal_records_info()
+--
+CREATE FUNCTION pg_get_wal_records_info(IN start_lsn pg_lsn,
+ IN end_lsn pg_lsn DEFAULT NULL,
+ OUT start_lsn pg_lsn,
+ OUT end_lsn pg_lsn,
+ OUT prev_lsn pg_lsn,
+ OUT xid xid,
+ OUT resource_manager text,
+ OUT record_type text,
+ OUT record_length int4,
+ OUT main_data_length int4,
+ OUT fpi_length int4,
+ OUT description text,
+ OUT block_ref text
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_get_wal_records_info'
+LANGUAGE C CALLED ON NULL INPUT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_wal_records_info(pg_lsn, pg_lsn) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_wal_records_info(pg_lsn, pg_lsn) TO pg_read_server_files;
+
+--
+-- pg_get_wal_stats()
+--
+CREATE FUNCTION pg_get_wal_stats(IN start_lsn pg_lsn,
+ IN end_lsn pg_lsn DEFAULT NULL,
+ IN per_record boolean DEFAULT false,
+ OUT "resource_manager/record_type" text,
+ OUT count int8,
+ OUT count_percentage float8,
+ OUT record_size int8,
+ OUT record_size_percentage float8,
+ OUT fpi_size int8,
+ OUT fpi_size_percentage float8,
+ OUT combined_size int8,
+ OUT combined_size_percentage float8
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_get_wal_stats'
+LANGUAGE C CALLED ON NULL INPUT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_wal_stats(pg_lsn, pg_lsn, boolean) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_wal_stats(pg_lsn, pg_lsn, boolean) TO pg_read_server_files;
+
--
-- pg_get_wal_fpi_info()
--
CREATE FUNCTION pg_get_wal_fpi_info(IN start_lsn pg_lsn,
- IN end_lsn pg_lsn,
+ IN end_lsn pg_lsn DEFAULT NULL,
OUT lsn pg_lsn,
OUT reltablespace oid,
OUT reldatabase oid,
@@ -18,7 +71,7 @@ CREATE FUNCTION pg_get_wal_fpi_info(IN start_lsn pg_lsn,
)
RETURNS SETOF record
AS 'MODULE_PATHNAME', 'pg_get_wal_fpi_info'
-LANGUAGE C STRICT PARALLEL SAFE;
+LANGUAGE C CALLED ON NULL INPUT PARALLEL SAFE;
REVOKE EXECUTE ON FUNCTION pg_get_wal_fpi_info(pg_lsn, pg_lsn) FROM PUBLIC;
GRANT EXECUTE ON FUNCTION pg_get_wal_fpi_info(pg_lsn, pg_lsn) TO pg_read_server_files;
diff --git a/contrib/pg_walinspect/pg_walinspect.c b/contrib/pg_walinspect/pg_walinspect.c
index b7b0a805ee..4451e3e557 100644
--- a/contrib/pg_walinspect/pg_walinspect.c
+++ b/contrib/pg_walinspect/pg_walinspect.c
@@ -37,13 +37,13 @@ PG_FUNCTION_INFO_V1(pg_get_wal_records_info_till_end_of_wal);
PG_FUNCTION_INFO_V1(pg_get_wal_stats);
PG_FUNCTION_INFO_V1(pg_get_wal_stats_till_end_of_wal);
-static bool IsFutureLSN(XLogRecPtr lsn, XLogRecPtr *curr_lsn);
+static XLogRecPtr GetCurrentLSN(void);
static XLogReaderState *InitXLogReaderState(XLogRecPtr lsn);
static XLogRecord *ReadNextXLogRecord(XLogReaderState *xlogreader);
static void GetWALRecordInfo(XLogReaderState *record, Datum *values,
bool *nulls, uint32 ncols);
-static XLogRecPtr ValidateInputLSNs(bool till_end_of_wal,
- XLogRecPtr start_lsn, XLogRecPtr end_lsn);
+static void GetInputLSNs(FunctionCallInfo fcinfo, XLogRecPtr *start_lsn,
+ XLogRecPtr *end_lsn);
static void GetWALRecordsInfo(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
XLogRecPtr end_lsn);
static void GetXLogSummaryStats(XLogStats *stats, ReturnSetInfo *rsinfo,
@@ -59,27 +59,25 @@ static void GetWalStats(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
static void GetWALFPIInfo(FunctionCallInfo fcinfo, XLogReaderState *record);
/*
- * Check if the given LSN is in future. Also, return the LSN up to which the
- * server has WAL.
+ * Return the LSN up to which the server has WAL.
*/
-static bool
-IsFutureLSN(XLogRecPtr lsn, XLogRecPtr *curr_lsn)
+static XLogRecPtr
+GetCurrentLSN(void)
{
+ XLogRecPtr curr_lsn;
+
/*
* We determine the current LSN of the server similar to how page_read
* callback read_local_xlog_page_no_wait does.
*/
if (!RecoveryInProgress())
- *curr_lsn = GetFlushRecPtr(NULL);
+ curr_lsn = GetFlushRecPtr(NULL);
else
- *curr_lsn = GetXLogReplayRecPtr(NULL);
+ curr_lsn = GetXLogReplayRecPtr(NULL);
- Assert(!XLogRecPtrIsInvalid(*curr_lsn));
+ Assert(!XLogRecPtrIsInvalid(curr_lsn));
- if (lsn >= *curr_lsn)
- return true;
-
- return false;
+ return curr_lsn;
}
/*
@@ -295,8 +293,14 @@ GetWALFPIInfo(FunctionCallInfo fcinfo, XLogReaderState *record)
* records between start and end LSNs. Decompression is applied to the
* blocks, if necessary.
*
- * This function emits an error if a future start or end WAL LSN i.e. WAL LSN
- * the database system doesn't know about is specified.
+ * This function determines end LSN if it is not specified or specified as NULL
+ * or invalid. In such cases, the end LSN is assigned with the last flushed LSN
+ * when not in recovery or the last replayed LSN when in recovery.
+ *
+ * This function emits an error if start LSN is invalid or in future i.e. LSN
+ * the database system doesn't know about.
+ *
+ * This function returns NULL, when start LSN is past the end LSN.
*/
Datum
pg_get_wal_fpi_info(PG_FUNCTION_ARGS)
@@ -307,10 +311,14 @@ pg_get_wal_fpi_info(PG_FUNCTION_ARGS)
MemoryContext old_cxt;
MemoryContext tmp_cxt;
- start_lsn = PG_GETARG_LSN(0);
- end_lsn = PG_GETARG_LSN(1);
+ GetInputLSNs(fcinfo, &start_lsn, &end_lsn);
- end_lsn = ValidateInputLSNs(false, start_lsn, end_lsn);
+ /*
+ * When start LSN is past the end LSN, let's return NULL instead of raising
+ * an error for better usability of the functions.
+ */
+ if (start_lsn >= end_lsn)
+ PG_RETURN_NULL();
InitMaterializedSRF(fcinfo, 0);
@@ -345,8 +353,8 @@ pg_get_wal_fpi_info(PG_FUNCTION_ARGS)
/*
* Get WAL record info.
*
- * This function emits an error if a future WAL LSN i.e. WAL LSN the database
- * system doesn't know about is specified.
+ * This function emits an error if input LSN is invalid or in future i.e. LSN
+ * the database system doesn't know about.
*/
Datum
pg_get_wal_record_info(PG_FUNCTION_ARGS)
@@ -363,7 +371,14 @@ pg_get_wal_record_info(PG_FUNCTION_ARGS)
lsn = PG_GETARG_LSN(0);
- if (IsFutureLSN(lsn, &curr_lsn))
+ if (XLogRecPtrIsInvalid(lsn))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid input LSN")));
+
+ curr_lsn = GetCurrentLSN();
+
+ if (lsn >= curr_lsn)
{
/*
* GetFlushRecPtr or GetXLogReplayRecPtr gives "end+1" LSN of the last
@@ -402,44 +417,36 @@ pg_get_wal_record_info(PG_FUNCTION_ARGS)
}
/*
- * Validate the input LSNs and compute end LSN for till_end_of_wal versions.
+ * Get start LSN and get/deduce end LSN.
*/
-static XLogRecPtr
-ValidateInputLSNs(bool till_end_of_wal, XLogRecPtr start_lsn,
- XLogRecPtr end_lsn)
+static void
+GetInputLSNs(FunctionCallInfo fcinfo, XLogRecPtr *start_lsn,
+ XLogRecPtr *end_lsn)
{
XLogRecPtr curr_lsn;
- if (IsFutureLSN(start_lsn, &curr_lsn))
- {
- /*
- * GetFlushRecPtr or GetXLogReplayRecPtr gives "end+1" LSN of the last
- * record flushed or replayed respectively. But let's use the LSN up
- * to "end" in user facing message.
- */
+ if (PG_ARGISNULL(0) ||
+ ((*start_lsn = PG_GETARG_LSN(0)) == InvalidXLogRecPtr))
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("cannot accept future start LSN"),
- errdetail("Last known WAL LSN on the database system is at %X/%X.",
- LSN_FORMAT_ARGS(curr_lsn))));
- }
-
- if (till_end_of_wal)
- end_lsn = curr_lsn;
+ errmsg("invalid start LSN")));
- if (end_lsn > curr_lsn)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("cannot accept future end LSN"),
- errdetail("Last known WAL LSN on the database system is at %X/%X.",
- LSN_FORMAT_ARGS(curr_lsn))));
+ curr_lsn = GetCurrentLSN();
- if (start_lsn >= end_lsn)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("WAL start LSN must be less than end LSN")));
+ if (PG_ARGISNULL(1))
+ *end_lsn = curr_lsn;
+ else
+ {
+ *end_lsn = PG_GETARG_LSN(1);
- return end_lsn;
+ /*
+ * Adjust end LSN to what the system knows at this point instead of
+ * raising errors for better usability of the functions.
+ */
+ if (XLogRecPtrIsInvalid(*end_lsn) ||
+ *end_lsn > curr_lsn)
+ *end_lsn = curr_lsn;
+ }
}
/*
@@ -494,8 +501,14 @@ GetWALRecordsInfo(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
/*
* Get info and data of all WAL records between start LSN and end LSN.
*
- * This function emits an error if a future start or end WAL LSN i.e. WAL LSN
- * the database system doesn't know about is specified.
+ * This function determines end LSN if it is not specified or specified as NULL
+ * or invalid. In such cases, the end LSN is assigned with the last flushed LSN
+ * when not in recovery or the last replayed LSN when in recovery.
+ *
+ * This function emits an error if start LSN is invalid or in future i.e. LSN
+ * the database system doesn't know about.
+ *
+ * This function returns NULL, when start LSN is past the end LSN.
*/
Datum
pg_get_wal_records_info(PG_FUNCTION_ARGS)
@@ -503,31 +516,14 @@ pg_get_wal_records_info(PG_FUNCTION_ARGS)
XLogRecPtr start_lsn;
XLogRecPtr end_lsn;
- start_lsn = PG_GETARG_LSN(0);
- end_lsn = PG_GETARG_LSN(1);
-
- end_lsn = ValidateInputLSNs(false, start_lsn, end_lsn);
-
- GetWALRecordsInfo(fcinfo, start_lsn, end_lsn);
-
- PG_RETURN_VOID();
-}
-
-/*
- * Get info and data of all WAL records from start LSN till end of WAL.
- *
- * This function emits an error if a future start i.e. WAL LSN the database
- * system doesn't know about is specified.
- */
-Datum
-pg_get_wal_records_info_till_end_of_wal(PG_FUNCTION_ARGS)
-{
- XLogRecPtr start_lsn;
- XLogRecPtr end_lsn = InvalidXLogRecPtr;
-
- start_lsn = PG_GETARG_LSN(0);
+ GetInputLSNs(fcinfo, &start_lsn, &end_lsn);
- end_lsn = ValidateInputLSNs(true, start_lsn, end_lsn);
+ /*
+ * When start LSN is past the end LSN, let's return NULL instead of raising
+ * an error for better usability of the functions.
+ */
+ if (start_lsn >= end_lsn)
+ PG_RETURN_NULL();
GetWALRecordsInfo(fcinfo, start_lsn, end_lsn);
@@ -732,8 +728,14 @@ GetWalStats(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
/*
* Get stats of all WAL records between start LSN and end LSN.
*
- * This function emits an error if a future start or end WAL LSN i.e. WAL LSN
- * the database system doesn't know about is specified.
+ * This function determines end LSN if it is not specified or specified as NULL
+ * or invalid. In such cases, the end LSN is assigned with the last flushed LSN
+ * when not in recovery or the last replayed LSN when in recovery.
+ *
+ * This function emits an error if start LSN is invalid or in future i.e. LSN
+ * the database system doesn't know about.
+ *
+ * This function returns NULL, when start LSN is past the end LSN.
*/
Datum
pg_get_wal_stats(PG_FUNCTION_ARGS)
@@ -742,11 +744,16 @@ pg_get_wal_stats(PG_FUNCTION_ARGS)
XLogRecPtr end_lsn;
bool stats_per_record;
- start_lsn = PG_GETARG_LSN(0);
- end_lsn = PG_GETARG_LSN(1);
- stats_per_record = PG_GETARG_BOOL(2);
+ GetInputLSNs(fcinfo, &start_lsn, &end_lsn);
+
+ /*
+ * When start LSN is past the end LSN, let's return NULL instead of raising
+ * an error for better usability of the functions.
+ */
+ if (start_lsn >= end_lsn)
+ PG_RETURN_NULL();
- end_lsn = ValidateInputLSNs(false, start_lsn, end_lsn);
+ stats_per_record = PG_GETARG_BOOL(2);
GetWalStats(fcinfo, start_lsn, end_lsn, stats_per_record);
@@ -754,22 +761,70 @@ pg_get_wal_stats(PG_FUNCTION_ARGS)
}
/*
- * Get stats of all WAL records from start LSN till end of WAL.
+ * NB: Following till_end_of_wal functions have been removed in newer versions
+ * of extension. However, we keep them around for backward compatibility. Which
+ * means, these functions work if someone explicitly installs the older
+ * extension version (using CREATE EXTENSION pg_walinspect WITH VERSION '1.0';)
+ * containing them.
*
- * This function emits an error if a future start i.e. WAL LSN the database
- * system doesn't know about is specified.
+ * If definitions of these functions are removed completely, the extension
+ * fails to install.
+ *
+ * In newer versions, one can use pg_get_wal_records_info()/pg_get_wal_stats()
+ * for the same till_end_of_wal functionality.
*/
+
+Datum
+pg_get_wal_records_info_till_end_of_wal(PG_FUNCTION_ARGS)
+{
+ XLogRecPtr start_lsn;
+ XLogRecPtr end_lsn;
+
+ if (PG_ARGISNULL(0) ||
+ ((start_lsn = PG_GETARG_LSN(0)) == InvalidXLogRecPtr))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid start LSN")));
+
+ /* Let's compute end LSN ourselves. */
+ end_lsn = GetCurrentLSN();
+
+ /*
+ * When start LSN is past the end LSN, let's return NULL instead of raising
+ * an error for better usability of the functions.
+ */
+ if (start_lsn >= end_lsn)
+ PG_RETURN_NULL();
+
+ GetWALRecordsInfo(fcinfo, start_lsn, end_lsn);
+
+ PG_RETURN_VOID();
+}
+
Datum
pg_get_wal_stats_till_end_of_wal(PG_FUNCTION_ARGS)
{
XLogRecPtr start_lsn;
- XLogRecPtr end_lsn = InvalidXLogRecPtr;
+ XLogRecPtr end_lsn;
bool stats_per_record;
- start_lsn = PG_GETARG_LSN(0);
- stats_per_record = PG_GETARG_BOOL(1);
+ if (PG_ARGISNULL(0) ||
+ ((start_lsn = PG_GETARG_LSN(0)) == InvalidXLogRecPtr))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid start LSN")));
+
+ /* Let's compute end LSN ourselves. */
+ end_lsn = GetCurrentLSN();
+
+ /*
+ * When start LSN is past the end LSN, let's return NULL instead of raising
+ * an error for better usability of the functions.
+ */
+ if (start_lsn >= end_lsn)
+ PG_RETURN_NULL();
- end_lsn = ValidateInputLSNs(true, start_lsn, end_lsn);
+ stats_per_record = PG_GETARG_BOOL(2);
GetWalStats(fcinfo, start_lsn, end_lsn, stats_per_record);
diff --git a/contrib/pg_walinspect/sql/oldextversions.sql b/contrib/pg_walinspect/sql/oldextversions.sql
new file mode 100644
index 0000000000..2c4f484177
--- /dev/null
+++ b/contrib/pg_walinspect/sql/oldextversions.sql
@@ -0,0 +1,27 @@
+-- test old extension version entry points
+
+DROP EXTENSION pg_walinspect;
+CREATE EXTENSION pg_walinspect WITH VERSION '1.0';
+
+-- Move to new version 1.1
+ALTER EXTENSION pg_walinspect UPDATE TO '1.1';
+
+-- New function introduced in 1.1
+SELECT pg_get_functiondef('pg_get_wal_fpi_info'::regproc);
+
+-- Redefined function
+SELECT pg_get_functiondef('pg_get_wal_records_info'::regproc);
+
+-- Removed function
+SELECT pg_get_functiondef('pg_get_wal_records_info_till_end_of_wal'::regproc);
+
+-- Redefined function
+SELECT pg_get_functiondef('pg_get_wal_stats'::regproc);
+
+-- Removed function
+SELECT pg_get_functiondef('pg_get_wal_stats_till_end_of_wal'::regproc);
+
+-- Redefined function
+SELECT pg_get_functiondef('pg_get_wal_fpi_info'::regproc);
+
+DROP EXTENSION pg_walinspect;
diff --git a/contrib/pg_walinspect/sql/pg_walinspect.sql b/contrib/pg_walinspect/sql/pg_walinspect.sql
index 849201a1f8..1d73782388 100644
--- a/contrib/pg_walinspect/sql/pg_walinspect.sql
+++ b/contrib/pg_walinspect/sql/pg_walinspect.sql
@@ -17,23 +17,56 @@ INSERT INTO sample_tbl SELECT * FROM generate_series(3, 4);
-- Tests for input validation
-- ===================================================================
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn2', :'wal_lsn1'); -- ERROR
+-- Invalid input LSNs
+SELECT * FROM pg_get_wal_record_info('0/0'); -- ERROR
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn2', :'wal_lsn1'); -- ERROR
+SELECT * FROM pg_get_wal_records_info('0/0'); -- ERROR
+
+SELECT * FROM pg_get_wal_stats('0/0'); -- ERROR
+
+SELECT * FROM pg_get_wal_fpi_info('0/0'); -- ERROR
+
+-- Start LSN NULL
+SELECT * FROM pg_get_wal_records_info(NULL); -- ERROR
+
+SELECT * FROM pg_get_wal_stats(NULL); -- ERROR
+
+SELECT * FROM pg_get_wal_fpi_info(NULL); -- ERROR
+
+-- Start LSN > End LSN
+SELECT * FROM pg_get_wal_records_info(:'wal_lsn2', :'wal_lsn1'); -- NULL
+
+SELECT * FROM pg_get_wal_stats(:'wal_lsn2', :'wal_lsn1'); -- NULL
+
+SELECT * FROM pg_get_wal_fpi_info(:'wal_lsn2', :'wal_lsn1'); -- NULL
-- ===================================================================
-- Tests for all function executions
-- ===================================================================
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
+
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
+
+-- Get info till end of WAL
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1');
+
+-- Get info till end of WAL
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', '0/0');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
+-- Get info till end of WAL
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', NULL);
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info_till_end_of_wal(:'wal_lsn1');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
+-- Get stats till end of WAL
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats_till_end_of_wal(:'wal_lsn1');
+-- Get stats till end of WAL
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', '0/0');
+
+-- Get stats till end of WAL
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', NULL);
-- ===================================================================
-- Test for filtering out WAL records of a particular table
@@ -68,6 +101,10 @@ SELECT pg_current_wal_lsn() AS wal_lsn4 \gset
SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_fpi_info(:'wal_lsn3', :'wal_lsn4')
WHERE relfilenode = :'sample_tbl_oid';
+-- Check till end of WAL if we get FPI from WAL record.
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_fpi_info(:'wal_lsn3')
+ WHERE relfilenode = :'sample_tbl_oid';
+
-- ===================================================================
-- Tests for permissions
-- ===================================================================
diff --git a/doc/src/sgml/pgwalinspect.sgml b/doc/src/sgml/pgwalinspect.sgml
index 3d7cdb95cc..86ce4381d5 100644
--- a/doc/src/sgml/pgwalinspect.sgml
+++ b/doc/src/sgml/pgwalinspect.sgml
@@ -61,6 +61,7 @@
Gets WAL record information of a given LSN. If the given LSN isn't
at the start of a WAL record, it gives the information of the next
available valid WAL record; or an error if no such record is found.
+ If given LSN is not yet available, the function will raise an error.
For example, usage of the function is as
follows:
<screen>
@@ -85,7 +86,7 @@ block_ref | blkref #0: rel 1663/5/60221 fork main blk 2
<varlistentry id="pgwalinspect-funcs-pg-get-wal-records-info">
<term>
<function>
- pg_get_wal_records_info(start_lsn pg_lsn, end_lsn pg_lsn)
+ pg_get_wal_records_info(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL)
returns setof record
</function>
</term>
@@ -94,8 +95,9 @@ block_ref | blkref #0: rel 1663/5/60221 fork main blk 2
<para>
Gets information of all the valid WAL records between
<replaceable>start_lsn</replaceable> and <replaceable>end_lsn</replaceable>.
- Returns one row per WAL record. If <replaceable>start_lsn</replaceable>
- or <replaceable>end_lsn</replaceable> are not yet available, the
+ Returns one row per WAL record. If <replaceable>end_lsn</replaceable>
+ isn't specified, it returns information till the end of WAL.
+ If <replaceable>start_lsn</replaceable> is not yet available, the
function will raise an error. For example:
<screen>
postgres=# SELECT * FROM pg_get_wal_records_info('0/1E913618', '0/1E913740') LIMIT 1;
@@ -116,27 +118,10 @@ block_ref |
</listitem>
</varlistentry>
- <varlistentry id="pgwalinspect-funcs-pg-get-wal-records-info-till-end-of-wal">
- <term>
- <function>
- pg_get_wal_records_info_till_end_of_wal(start_lsn pg_lsn)
- returns setof record
- </function>
- </term>
-
- <listitem>
- <para>
- This function is the same as <function>pg_get_wal_records_info()</function>,
- except that it gets information of all the valid WAL records from
- <replaceable>start_lsn</replaceable> till the end of WAL.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry id="pgwalinspect-funcs-pg-get-wal-stats">
<term>
<function>
- pg_get_wal_stats(start_lsn pg_lsn, end_lsn pg_lsn, per_record boolean DEFAULT false)
+ pg_get_wal_stats(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL, per_record boolean DEFAULT false)
returns setof record
</function>
</term>
@@ -149,9 +134,9 @@ block_ref |
<replaceable>resource_manager</replaceable> type. When
<replaceable>per_record</replaceable> is set to <literal>true</literal>,
it returns one row per <replaceable>record_type</replaceable>.
- If <replaceable>start_lsn</replaceable>
- or <replaceable>end_lsn</replaceable> are not yet available, the
- function will raise an error. For example:
+ If <replaceable>end_lsn</replaceable> isn't specified, it returns
+ information till the end of WAL. If <replaceable>start_lsn</replaceable>
+ is not yet available, the function will raise an error. For example:
<screen>
postgres=# SELECT * FROM pg_get_wal_stats('0/1E847D00', '0/1E84F500')
WHERE count > 0 LIMIT 1 AND
@@ -171,28 +156,14 @@ combined_size_percentage | 2.8634072910530795
</listitem>
</varlistentry>
- <varlistentry id="pgwalinspect-funcs-pg-get-wal-stats-till-end-of-wal">
+ <varlistentry>
<term>
<function>
- pg_get_wal_stats_till_end_of_wal(start_lsn pg_lsn, per_record boolean DEFAULT false)
+ pg_get_wal_fpi_info(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL)
returns setof record
</function>
</term>
- <listitem>
- <para>
- This function is the same as <function>pg_get_wal_stats()</function>,
- except that it gets statistics of all the valid WAL records from
- <replaceable>start_lsn</replaceable> till end of WAL.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term>
- <function>pg_get_wal_fpi_info(start_lsn pg_lsn, end_lsn pg_lsn) returns setof record</function>
- </term>
-
<listitem>
<para>
Gets a copy of full page images as <type>bytea</type> values (after
@@ -200,9 +171,9 @@ combined_size_percentage | 2.8634072910530795
with all the valid WAL records between
<replaceable>start_lsn</replaceable> and
<replaceable>end_lsn</replaceable>. Returns one row per full page image.
- If <replaceable>start_lsn</replaceable> or
- <replaceable>end_lsn</replaceable> are not yet available, the function
- will raise an error. For example:
+ If <replaceable>end_lsn</replaceable> isn't specified, it returns
+ information till the end of WAL. If <replaceable>start_lsn</replaceable>
+ is not yet available, the function will raise an error. For example:
<screen>
postgres=# SELECT lsn, reltablespace, reldatabase, relfilenode, relblocknumber,
forkname, substring(fpi for 24) as fpi_trimmed
@@ -219,8 +190,20 @@ fpi_trimmed | \x00000000b89e660100000000a003c0030020042000000000
</para>
</listitem>
</varlistentry>
-
</variablelist>
+
+ <note>
+ <para>
+ Note that <function>pg_get_wal_records_info_till_end_of_wal</function> and
+ <function>pg_get_wal_stats_till_end_of_wal</function> functions have been
+ removed in the <filename>pg_walinspect</filename> version
+ <literal>1.1</literal>. The same functionality can be achieved with
+ <function>pg_get_wal_records_info</function> and
+ <function>pg_get_wal_stats</function> functions. However,
+ <function>till_end_of_wal</function> functions will still work if the
+ extension is installed explicitly with version <literal>1.0</literal>.
+ </para>
+ </note>
</sect2>
<sect2 id="pgwalinspect-author">
--
2.34.1
Re: Combine pg_walinspect till_end_of_wal functions with others
On Tue, Mar 07, 2023 at 01:47:01PM +0800, Julien Rouhaud wrote:
So keep this "deprecated" C function working, as it would only be a few lines
of code?
Yes, I guess that this would be the final picture, moving forward I'd
like to think that we should just remove the SQL declaration of the
till_end_of_wal() functions to keep a clean interface.
--
Michael
Re: Combine pg_walinspect till_end_of_wal functions with others
On Wed, Mar 08, 2023 at 01:40:46PM +0530, Bharath Rupireddy wrote:
1. When start_lsn is NULL or invalid ('0/0'), emit an error. There was
a comment on the functions automatically determining start_lsn to be
the oldest WAL LSN. I'm not implementing this change now, as it
requires extra work to traverse the pg_wal directory. I'm planning to
do it in the next set of improvements where I'm planning to make the
functions timeline-aware, introduce functions for inspecting
wal_buffers and so on.[.. long description ..]
9. Refactored the tests according to the new behaviours.
Hmm. I think this patch ought to have a result simpler than what's
proposed here.
First, do we really have to begin marking the functions as non-STRICT
to abide with the treatment of NULL as a special value? The part that
I've found personally the most annoying with these functions is that
an incorrect bound leads to a random failure, particularly when such
queries are used for monitoring. I would simplify the whole with two
simple rules, as of:
- Keeping all the functions strict.
- When end_lsn is a LSN in the future of the current LSN inserted or
replayed, adjust its value to be the exactly GetXLogReplayRecPtr() or
GetFlushRecPtr(). This way, monitoring tools can use a value ahead,
at will.
- Failing if start_lsn > end_lsn.
- Failing if start_lsn refers to a position older than what exists is
still fine by me.
I would also choose to remove
pg_get_wal_records_info_till_end_of_wal() from the SQL interface in
1.1 to limit the confusion arount it, but keep a few lines of code so
as we are still able to use it when pg_walinspect 1.0 is the version
enabled with CREATE EXTENSION.
In short, pg_get_wal_records_info_till_end_of_wal() should be able to
use exactly the same code as pg_get_wal_records_info(), still you need
to keep *two* functions for their prosrc with PG_FUNCTION_ARGS as
arguments so as 1.0 would work when dropped in place. The result, it
seems to me, mostly comes to simplify ValidateInputLSNs() and remove
its till_end_of_wal argument.
+-- Removed function
+SELECT pg_get_functiondef('pg_get_wal_records_info_till_end_of_wal'::regproc);
+ERROR: function "pg_get_wal_records_info_till_end_of_wal" does not exist
+LINE 1: SELECT pg_get_functiondef('pg_get_wal_records_info_till_end_...It seems to me that you should just replace all that and anything
depending on pg_get_functiondef() with a \dx+ pg_walinspect, that
would list all the objects part of the extension for the specific
version you want to test. Not sure that there is a need to list the
full function definitions, either. That just bloats the tests.
I think, however, that it is critical to test in oldextversions.out
the *executions* of the functions, so as we make sure that they don't
crash. The patch is missing that.
+-- Invalid input LSNs
+SELECT * FROM pg_get_wal_record_info('0/0'); -- ERROR
+ERROR: invalid input LSN
--
MichaelRe: Combine pg_walinspect till_end_of_wal functions with others
On Wed, Mar 8, 2023 at 1:40 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:
On Tue, Mar 7, 2023 at 11:17 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
Here I'm with the v3 patch addressing the above comments. Please
review it further.
Needed a rebase. v4 patch is attached. I'll address the latest review
comments in a bit.
--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
Attachments:
v4-0001-Rework-pg_walinspect-functions.patchapplication/octet-stream; name=v4-0001-Rework-pg_walinspect-functions.patchDownload
From ecd578361158f4c6d90a1d983c2a2e055945c638 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Fri, 10 Mar 2023 05:07:18 +0000
Subject: [PATCH v4] Rework pg_walinspect functions
---
contrib/pg_walinspect/Makefile | 2 +-
.../pg_walinspect/expected/oldextversions.out | 64 +++++
.../pg_walinspect/expected/pg_walinspect.out | 85 ++++++-
contrib/pg_walinspect/meson.build | 1 +
.../pg_walinspect/pg_walinspect--1.0--1.1.sql | 57 ++++-
contrib/pg_walinspect/pg_walinspect.c | 235 +++++++++++-------
contrib/pg_walinspect/sql/oldextversions.sql | 27 ++
contrib/pg_walinspect/sql/pg_walinspect.sql | 70 +++---
doc/src/sgml/pgwalinspect.sgml | 73 +++---
9 files changed, 428 insertions(+), 186 deletions(-)
create mode 100644 contrib/pg_walinspect/expected/oldextversions.out
create mode 100644 contrib/pg_walinspect/sql/oldextversions.sql
diff --git a/contrib/pg_walinspect/Makefile b/contrib/pg_walinspect/Makefile
index 7033878a79..22090f7716 100644
--- a/contrib/pg_walinspect/Makefile
+++ b/contrib/pg_walinspect/Makefile
@@ -9,7 +9,7 @@ PGFILEDESC = "pg_walinspect - functions to inspect contents of PostgreSQL Write-
EXTENSION = pg_walinspect
DATA = pg_walinspect--1.0.sql pg_walinspect--1.0--1.1.sql
-REGRESS = pg_walinspect
+REGRESS = pg_walinspect oldextversions
REGRESS_OPTS = --temp-config $(top_srcdir)/contrib/pg_walinspect/walinspect.conf
diff --git a/contrib/pg_walinspect/expected/oldextversions.out b/contrib/pg_walinspect/expected/oldextversions.out
new file mode 100644
index 0000000000..0ac82dfeb1
--- /dev/null
+++ b/contrib/pg_walinspect/expected/oldextversions.out
@@ -0,0 +1,64 @@
+-- test old extension version entry points
+DROP EXTENSION pg_walinspect;
+CREATE EXTENSION pg_walinspect WITH VERSION '1.0';
+-- Move to new version 1.1
+ALTER EXTENSION pg_walinspect UPDATE TO '1.1';
+-- New function introduced in 1.1
+SELECT pg_get_functiondef('pg_get_wal_block_info'::regproc);
+ pg_get_functiondef
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ CREATE OR REPLACE FUNCTION public.pg_get_wal_block_info(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL::pg_lsn, OUT lsn pg_lsn, OUT blockid smallint, OUT reltablespace oid, OUT reldatabase oid, OUT relfilenode oid, OUT relblocknumber bigint, OUT forkname text, OUT blockdata bytea, OUT fpi bytea, OUT fpilen integer, OUT fpiinfo text[])+
+ RETURNS SETOF record +
+ LANGUAGE c +
+ PARALLEL SAFE +
+ AS '$libdir/pg_walinspect', $function$pg_get_wal_block_info$function$ +
+
+(1 row)
+
+-- Redefined function
+SELECT pg_get_functiondef('pg_get_wal_records_info'::regproc);
+ pg_get_functiondef
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ CREATE OR REPLACE FUNCTION public.pg_get_wal_records_info(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL::pg_lsn, OUT start_lsn pg_lsn, OUT end_lsn pg_lsn, OUT prev_lsn pg_lsn, OUT xid xid, OUT resource_manager text, OUT record_type text, OUT record_length integer, OUT main_data_length integer, OUT fpi_length integer, OUT description text, OUT block_ref text)+
+ RETURNS SETOF record +
+ LANGUAGE c +
+ PARALLEL SAFE +
+ AS '$libdir/pg_walinspect', $function$pg_get_wal_records_info$function$ +
+
+(1 row)
+
+-- Removed function
+SELECT pg_get_functiondef('pg_get_wal_records_info_till_end_of_wal'::regproc);
+ERROR: function "pg_get_wal_records_info_till_end_of_wal" does not exist
+LINE 1: SELECT pg_get_functiondef('pg_get_wal_records_info_till_end_...
+ ^
+-- Redefined function
+SELECT pg_get_functiondef('pg_get_wal_stats'::regproc);
+ pg_get_functiondef
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ CREATE OR REPLACE FUNCTION public.pg_get_wal_stats(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL::pg_lsn, per_record boolean DEFAULT false, OUT "resource_manager/record_type" text, OUT count bigint, OUT count_percentage double precision, OUT record_size bigint, OUT record_size_percentage double precision, OUT fpi_size bigint, OUT fpi_size_percentage double precision, OUT combined_size bigint, OUT combined_size_percentage double precision)+
+ RETURNS SETOF record +
+ LANGUAGE c +
+ PARALLEL SAFE +
+ AS '$libdir/pg_walinspect', $function$pg_get_wal_stats$function$ +
+
+(1 row)
+
+-- Removed function
+SELECT pg_get_functiondef('pg_get_wal_stats_till_end_of_wal'::regproc);
+ERROR: function "pg_get_wal_stats_till_end_of_wal" does not exist
+LINE 1: SELECT pg_get_functiondef('pg_get_wal_stats_till_end_of_wal'...
+ ^
+-- Redefined function
+SELECT pg_get_functiondef('pg_get_wal_block_info'::regproc);
+ pg_get_functiondef
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ CREATE OR REPLACE FUNCTION public.pg_get_wal_block_info(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL::pg_lsn, OUT lsn pg_lsn, OUT blockid smallint, OUT reltablespace oid, OUT reldatabase oid, OUT relfilenode oid, OUT relblocknumber bigint, OUT forkname text, OUT blockdata bytea, OUT fpi bytea, OUT fpilen integer, OUT fpiinfo text[])+
+ RETURNS SETOF record +
+ LANGUAGE c +
+ PARALLEL SAFE +
+ AS '$libdir/pg_walinspect', $function$pg_get_wal_block_info$function$ +
+
+(1 row)
+
+DROP EXTENSION pg_walinspect;
diff --git a/contrib/pg_walinspect/expected/pg_walinspect.out b/contrib/pg_walinspect/expected/pg_walinspect.out
index e0eb7ca08f..31ef22a2cc 100644
--- a/contrib/pg_walinspect/expected/pg_walinspect.out
+++ b/contrib/pg_walinspect/expected/pg_walinspect.out
@@ -14,38 +14,89 @@ INSERT INTO sample_tbl SELECT * FROM generate_series(3, 4);
-- ===================================================================
-- Tests for input validation
-- ===================================================================
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn2', :'wal_lsn1'); -- ERROR
-ERROR: WAL start LSN must be less than end LSN
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn2', :'wal_lsn1'); -- ERROR
-ERROR: WAL start LSN must be less than end LSN
+-- Invalid input LSNs
+SELECT * FROM pg_get_wal_record_info('0/0'); -- ERROR
+ERROR: invalid input LSN
+SELECT * FROM pg_get_wal_records_info('0/0'); -- ERROR
+ERROR: invalid start LSN
+SELECT * FROM pg_get_wal_stats('0/0'); -- ERROR
+ERROR: invalid start LSN
+SELECT * FROM pg_get_wal_block_info('0/0'); -- ERROR
+ERROR: invalid start LSN
+-- Start LSN NULL
+SELECT * FROM pg_get_wal_records_info(NULL); -- ERROR
+ERROR: invalid start LSN
+SELECT * FROM pg_get_wal_stats(NULL); -- ERROR
+ERROR: invalid start LSN
+SELECT * FROM pg_get_wal_block_info(NULL); -- ERROR
+ERROR: invalid start LSN
+-- Start LSN > End LSN
+SELECT * FROM pg_get_wal_records_info(:'wal_lsn2', :'wal_lsn1'); -- NULL
+ start_lsn | end_lsn | prev_lsn | xid | resource_manager | record_type | record_length | main_data_length | fpi_length | description | block_ref
+-----------+---------+----------+-----+------------------+-------------+---------------+------------------+------------+-------------+-----------
+ | | | | | | | | | |
+(1 row)
+
+SELECT * FROM pg_get_wal_stats(:'wal_lsn2', :'wal_lsn1'); -- NULL
+ resource_manager/record_type | count | count_percentage | record_size | record_size_percentage | fpi_size | fpi_size_percentage | combined_size | combined_size_percentage
+------------------------------+-------+------------------+-------------+------------------------+----------+---------------------+---------------+--------------------------
+ | | | | | | | |
+(1 row)
+
+SELECT * FROM pg_get_wal_block_info(:'wal_lsn2', :'wal_lsn1'); -- NULL
+ lsn | blockid | reltablespace | reldatabase | relfilenode | relblocknumber | forkname | blockdata | fpi | fpilen | fpiinfo
+-----+---------+---------------+-------------+-------------+----------------+----------+-----------+-----+--------+---------
+ | | | | | | | | | |
+(1 row)
+
-- ===================================================================
-- Tests for all function executions
-- ===================================================================
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
+ ok
+----
+ t
+(1 row)
+
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
+ ok
+----
+ t
+(1 row)
+
+-- Get info till end of WAL
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1');
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', '0/0');
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info_till_end_of_wal(:'wal_lsn1');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', NULL);
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
+-- Get stats till end of WAL
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1');
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats_till_end_of_wal(:'wal_lsn1');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', '0/0');
+ ok
+----
+ t
+(1 row)
+
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', NULL);
ok
----
t
@@ -88,6 +139,14 @@ SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn3', :'wal_lsn4')
t
(1 row)
+-- Check till end of WAL if we get block data from WAL record.
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn3')
+ WHERE relfilenode = :'sample_tbl_oid';
+ ok
+----
+ t
+(1 row)
+
-- Force full-page image on the next update.
SELECT pg_current_wal_lsn() AS wal_lsn5 \gset
CHECKPOINT;
@@ -101,6 +160,14 @@ SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn5', :'wal_lsn6')
t
(1 row)
+-- Check till end of WAL if we get FPI from WAL record.
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn5')
+ WHERE relfilenode = :'sample_tbl_oid';
+ ok
+----
+ t
+(1 row)
+
-- ===================================================================
-- Tests for permissions
-- ===================================================================
diff --git a/contrib/pg_walinspect/meson.build b/contrib/pg_walinspect/meson.build
index bf7b79b1b7..80059f6119 100644
--- a/contrib/pg_walinspect/meson.build
+++ b/contrib/pg_walinspect/meson.build
@@ -30,6 +30,7 @@ tests += {
'regress': {
'sql': [
'pg_walinspect',
+ 'oldextversions',
],
# Disabled because these tests require "wal_level=replica", which
# some runningcheck users do not have (e.g. buildfarm clients).
diff --git a/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql b/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql
index e674ef25aa..8d9b82ead2 100644
--- a/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql
+++ b/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql
@@ -3,11 +3,64 @@
-- complain if script is sourced in psql, rather than via ALTER EXTENSION
\echo Use "ALTER EXTENSION pg_walinspect UPDATE TO '1.1'" to load this file. \quit
+/* Drop the unneeded functions and redefine needed ones */
+DROP FUNCTION pg_get_wal_records_info(pg_lsn, pg_lsn);
+DROP FUNCTION pg_get_wal_records_info_till_end_of_wal(pg_lsn);
+DROP FUNCTION pg_get_wal_stats(pg_lsn, pg_lsn, boolean);
+DROP FUNCTION pg_get_wal_stats_till_end_of_wal(pg_lsn, boolean);
+
+--
+-- pg_get_wal_records_info()
+--
+CREATE FUNCTION pg_get_wal_records_info(IN start_lsn pg_lsn,
+ IN end_lsn pg_lsn DEFAULT NULL,
+ OUT start_lsn pg_lsn,
+ OUT end_lsn pg_lsn,
+ OUT prev_lsn pg_lsn,
+ OUT xid xid,
+ OUT resource_manager text,
+ OUT record_type text,
+ OUT record_length int4,
+ OUT main_data_length int4,
+ OUT fpi_length int4,
+ OUT description text,
+ OUT block_ref text
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_get_wal_records_info'
+LANGUAGE C CALLED ON NULL INPUT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_wal_records_info(pg_lsn, pg_lsn) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_wal_records_info(pg_lsn, pg_lsn) TO pg_read_server_files;
+
+--
+-- pg_get_wal_stats()
+--
+CREATE FUNCTION pg_get_wal_stats(IN start_lsn pg_lsn,
+ IN end_lsn pg_lsn DEFAULT NULL,
+ IN per_record boolean DEFAULT false,
+ OUT "resource_manager/record_type" text,
+ OUT count int8,
+ OUT count_percentage float8,
+ OUT record_size int8,
+ OUT record_size_percentage float8,
+ OUT fpi_size int8,
+ OUT fpi_size_percentage float8,
+ OUT combined_size int8,
+ OUT combined_size_percentage float8
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_get_wal_stats'
+LANGUAGE C CALLED ON NULL INPUT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_wal_stats(pg_lsn, pg_lsn, boolean) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_wal_stats(pg_lsn, pg_lsn, boolean) TO pg_read_server_files;
+
--
-- pg_get_wal_block_info()
--
CREATE FUNCTION pg_get_wal_block_info(IN start_lsn pg_lsn,
- IN end_lsn pg_lsn,
+ IN end_lsn pg_lsn DEFAULT NULL,
OUT lsn pg_lsn,
OUT blockid int2,
OUT reltablespace oid,
@@ -22,7 +75,7 @@ CREATE FUNCTION pg_get_wal_block_info(IN start_lsn pg_lsn,
)
RETURNS SETOF record
AS 'MODULE_PATHNAME', 'pg_get_wal_block_info'
-LANGUAGE C STRICT PARALLEL SAFE;
+LANGUAGE C CALLED ON NULL INPUT PARALLEL SAFE;
REVOKE EXECUTE ON FUNCTION pg_get_wal_block_info(pg_lsn, pg_lsn) FROM PUBLIC;
GRANT EXECUTE ON FUNCTION pg_get_wal_block_info(pg_lsn, pg_lsn) TO pg_read_server_files;
diff --git a/contrib/pg_walinspect/pg_walinspect.c b/contrib/pg_walinspect/pg_walinspect.c
index ee88dc4992..4a8974ac1f 100644
--- a/contrib/pg_walinspect/pg_walinspect.c
+++ b/contrib/pg_walinspect/pg_walinspect.c
@@ -38,13 +38,13 @@ PG_FUNCTION_INFO_V1(pg_get_wal_records_info_till_end_of_wal);
PG_FUNCTION_INFO_V1(pg_get_wal_stats);
PG_FUNCTION_INFO_V1(pg_get_wal_stats_till_end_of_wal);
-static bool IsFutureLSN(XLogRecPtr lsn, XLogRecPtr *curr_lsn);
+static XLogRecPtr GetCurrentLSN(void);
static XLogReaderState *InitXLogReaderState(XLogRecPtr lsn);
static XLogRecord *ReadNextXLogRecord(XLogReaderState *xlogreader);
static void GetWALRecordInfo(XLogReaderState *record, Datum *values,
bool *nulls, uint32 ncols);
-static XLogRecPtr ValidateInputLSNs(bool till_end_of_wal,
- XLogRecPtr start_lsn, XLogRecPtr end_lsn);
+static void GetInputLSNs(FunctionCallInfo fcinfo, XLogRecPtr *start_lsn,
+ XLogRecPtr *end_lsn);
static void GetWALRecordsInfo(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
XLogRecPtr end_lsn);
static void GetXLogSummaryStats(XLogStats *stats, ReturnSetInfo *rsinfo,
@@ -60,27 +60,25 @@ static void GetWalStats(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
static void GetWALBlockInfo(FunctionCallInfo fcinfo, XLogReaderState *record);
/*
- * Check if the given LSN is in future. Also, return the LSN up to which the
- * server has WAL.
+ * Return the LSN up to which the server has WAL.
*/
-static bool
-IsFutureLSN(XLogRecPtr lsn, XLogRecPtr *curr_lsn)
+static XLogRecPtr
+GetCurrentLSN(void)
{
+ XLogRecPtr curr_lsn;
+
/*
* We determine the current LSN of the server similar to how page_read
* callback read_local_xlog_page_no_wait does.
*/
if (!RecoveryInProgress())
- *curr_lsn = GetFlushRecPtr(NULL);
+ curr_lsn = GetFlushRecPtr(NULL);
else
- *curr_lsn = GetXLogReplayRecPtr(NULL);
+ curr_lsn = GetXLogReplayRecPtr(NULL);
- Assert(!XLogRecPtrIsInvalid(*curr_lsn));
+ Assert(!XLogRecPtrIsInvalid(curr_lsn));
- if (lsn >= *curr_lsn)
- return true;
-
- return false;
+ return curr_lsn;
}
/*
@@ -355,8 +353,14 @@ GetWALBlockInfo(FunctionCallInfo fcinfo, XLogReaderState *record)
* to a record. Decompression is applied to the full page images, if
* necessary.
*
- * This function emits an error if a future start or end WAL LSN i.e. WAL LSN
- * the database system doesn't know about is specified.
+ * This function determines end LSN if it is not specified or specified as NULL
+ * or invalid. In such cases, the end LSN is assigned with the last flushed LSN
+ * when not in recovery or the last replayed LSN when in recovery.
+ *
+ * This function emits an error if start LSN is invalid or in future i.e. LSN
+ * the database system doesn't know about.
+ *
+ * This function returns NULL, when start LSN is past the end LSN.
*/
Datum
pg_get_wal_block_info(PG_FUNCTION_ARGS)
@@ -367,10 +371,14 @@ pg_get_wal_block_info(PG_FUNCTION_ARGS)
MemoryContext old_cxt;
MemoryContext tmp_cxt;
- start_lsn = PG_GETARG_LSN(0);
- end_lsn = PG_GETARG_LSN(1);
+ GetInputLSNs(fcinfo, &start_lsn, &end_lsn);
- end_lsn = ValidateInputLSNs(false, start_lsn, end_lsn);
+ /*
+ * When start LSN is past the end LSN, let's return NULL instead of raising
+ * an error for better usability of the functions.
+ */
+ if (start_lsn >= end_lsn)
+ PG_RETURN_NULL();
InitMaterializedSRF(fcinfo, 0);
@@ -405,8 +413,8 @@ pg_get_wal_block_info(PG_FUNCTION_ARGS)
/*
* Get WAL record info.
*
- * This function emits an error if a future WAL LSN i.e. WAL LSN the database
- * system doesn't know about is specified.
+ * This function emits an error if input LSN is invalid or in future i.e. LSN
+ * the database system doesn't know about.
*/
Datum
pg_get_wal_record_info(PG_FUNCTION_ARGS)
@@ -423,7 +431,14 @@ pg_get_wal_record_info(PG_FUNCTION_ARGS)
lsn = PG_GETARG_LSN(0);
- if (IsFutureLSN(lsn, &curr_lsn))
+ if (XLogRecPtrIsInvalid(lsn))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid input LSN")));
+
+ curr_lsn = GetCurrentLSN();
+
+ if (lsn >= curr_lsn)
{
/*
* GetFlushRecPtr or GetXLogReplayRecPtr gives "end+1" LSN of the last
@@ -462,44 +477,36 @@ pg_get_wal_record_info(PG_FUNCTION_ARGS)
}
/*
- * Validate the input LSNs and compute end LSN for till_end_of_wal versions.
+ * Get start LSN and get/deduce end LSN.
*/
-static XLogRecPtr
-ValidateInputLSNs(bool till_end_of_wal, XLogRecPtr start_lsn,
- XLogRecPtr end_lsn)
+static void
+GetInputLSNs(FunctionCallInfo fcinfo, XLogRecPtr *start_lsn,
+ XLogRecPtr *end_lsn)
{
XLogRecPtr curr_lsn;
- if (IsFutureLSN(start_lsn, &curr_lsn))
- {
- /*
- * GetFlushRecPtr or GetXLogReplayRecPtr gives "end+1" LSN of the last
- * record flushed or replayed respectively. But let's use the LSN up
- * to "end" in user facing message.
- */
+ if (PG_ARGISNULL(0) ||
+ ((*start_lsn = PG_GETARG_LSN(0)) == InvalidXLogRecPtr))
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("cannot accept future start LSN"),
- errdetail("Last known WAL LSN on the database system is at %X/%X.",
- LSN_FORMAT_ARGS(curr_lsn))));
- }
-
- if (till_end_of_wal)
- end_lsn = curr_lsn;
+ errmsg("invalid start LSN")));
- if (end_lsn > curr_lsn)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("cannot accept future end LSN"),
- errdetail("Last known WAL LSN on the database system is at %X/%X.",
- LSN_FORMAT_ARGS(curr_lsn))));
+ curr_lsn = GetCurrentLSN();
- if (start_lsn >= end_lsn)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("WAL start LSN must be less than end LSN")));
+ if (PG_ARGISNULL(1))
+ *end_lsn = curr_lsn;
+ else
+ {
+ *end_lsn = PG_GETARG_LSN(1);
- return end_lsn;
+ /*
+ * Adjust end LSN to what the system knows at this point instead of
+ * raising errors for better usability of the functions.
+ */
+ if (XLogRecPtrIsInvalid(*end_lsn) ||
+ *end_lsn > curr_lsn)
+ *end_lsn = curr_lsn;
+ }
}
/*
@@ -554,8 +561,14 @@ GetWALRecordsInfo(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
/*
* Get info and data of all WAL records between start LSN and end LSN.
*
- * This function emits an error if a future start or end WAL LSN i.e. WAL LSN
- * the database system doesn't know about is specified.
+ * This function determines end LSN if it is not specified or specified as NULL
+ * or invalid. In such cases, the end LSN is assigned with the last flushed LSN
+ * when not in recovery or the last replayed LSN when in recovery.
+ *
+ * This function emits an error if start LSN is invalid or in future i.e. LSN
+ * the database system doesn't know about.
+ *
+ * This function returns NULL, when start LSN is past the end LSN.
*/
Datum
pg_get_wal_records_info(PG_FUNCTION_ARGS)
@@ -563,31 +576,14 @@ pg_get_wal_records_info(PG_FUNCTION_ARGS)
XLogRecPtr start_lsn;
XLogRecPtr end_lsn;
- start_lsn = PG_GETARG_LSN(0);
- end_lsn = PG_GETARG_LSN(1);
-
- end_lsn = ValidateInputLSNs(false, start_lsn, end_lsn);
-
- GetWALRecordsInfo(fcinfo, start_lsn, end_lsn);
-
- PG_RETURN_VOID();
-}
-
-/*
- * Get info and data of all WAL records from start LSN till end of WAL.
- *
- * This function emits an error if a future start i.e. WAL LSN the database
- * system doesn't know about is specified.
- */
-Datum
-pg_get_wal_records_info_till_end_of_wal(PG_FUNCTION_ARGS)
-{
- XLogRecPtr start_lsn;
- XLogRecPtr end_lsn = InvalidXLogRecPtr;
-
- start_lsn = PG_GETARG_LSN(0);
+ GetInputLSNs(fcinfo, &start_lsn, &end_lsn);
- end_lsn = ValidateInputLSNs(true, start_lsn, end_lsn);
+ /*
+ * When start LSN is past the end LSN, let's return NULL instead of raising
+ * an error for better usability of the functions.
+ */
+ if (start_lsn >= end_lsn)
+ PG_RETURN_NULL();
GetWALRecordsInfo(fcinfo, start_lsn, end_lsn);
@@ -792,8 +788,14 @@ GetWalStats(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
/*
* Get stats of all WAL records between start LSN and end LSN.
*
- * This function emits an error if a future start or end WAL LSN i.e. WAL LSN
- * the database system doesn't know about is specified.
+ * This function determines end LSN if it is not specified or specified as NULL
+ * or invalid. In such cases, the end LSN is assigned with the last flushed LSN
+ * when not in recovery or the last replayed LSN when in recovery.
+ *
+ * This function emits an error if start LSN is invalid or in future i.e. LSN
+ * the database system doesn't know about.
+ *
+ * This function returns NULL, when start LSN is past the end LSN.
*/
Datum
pg_get_wal_stats(PG_FUNCTION_ARGS)
@@ -802,11 +804,16 @@ pg_get_wal_stats(PG_FUNCTION_ARGS)
XLogRecPtr end_lsn;
bool stats_per_record;
- start_lsn = PG_GETARG_LSN(0);
- end_lsn = PG_GETARG_LSN(1);
- stats_per_record = PG_GETARG_BOOL(2);
+ GetInputLSNs(fcinfo, &start_lsn, &end_lsn);
+
+ /*
+ * When start LSN is past the end LSN, let's return NULL instead of raising
+ * an error for better usability of the functions.
+ */
+ if (start_lsn >= end_lsn)
+ PG_RETURN_NULL();
- end_lsn = ValidateInputLSNs(false, start_lsn, end_lsn);
+ stats_per_record = PG_GETARG_BOOL(2);
GetWalStats(fcinfo, start_lsn, end_lsn, stats_per_record);
@@ -814,22 +821,70 @@ pg_get_wal_stats(PG_FUNCTION_ARGS)
}
/*
- * Get stats of all WAL records from start LSN till end of WAL.
+ * NB: Following till_end_of_wal functions have been removed in newer versions
+ * of extension. However, we keep them around for backward compatibility. Which
+ * means, these functions work if someone explicitly installs the older
+ * extension version (using CREATE EXTENSION pg_walinspect WITH VERSION '1.0';)
+ * containing them.
*
- * This function emits an error if a future start i.e. WAL LSN the database
- * system doesn't know about is specified.
+ * If definitions of these functions are removed completely, the extension
+ * fails to install.
+ *
+ * In newer versions, one can use pg_get_wal_records_info()/pg_get_wal_stats()
+ * for the same till_end_of_wal functionality.
*/
+
+Datum
+pg_get_wal_records_info_till_end_of_wal(PG_FUNCTION_ARGS)
+{
+ XLogRecPtr start_lsn;
+ XLogRecPtr end_lsn;
+
+ if (PG_ARGISNULL(0) ||
+ ((start_lsn = PG_GETARG_LSN(0)) == InvalidXLogRecPtr))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid start LSN")));
+
+ /* Let's compute end LSN ourselves. */
+ end_lsn = GetCurrentLSN();
+
+ /*
+ * When start LSN is past the end LSN, let's return NULL instead of raising
+ * an error for better usability of the functions.
+ */
+ if (start_lsn >= end_lsn)
+ PG_RETURN_NULL();
+
+ GetWALRecordsInfo(fcinfo, start_lsn, end_lsn);
+
+ PG_RETURN_VOID();
+}
+
Datum
pg_get_wal_stats_till_end_of_wal(PG_FUNCTION_ARGS)
{
XLogRecPtr start_lsn;
- XLogRecPtr end_lsn = InvalidXLogRecPtr;
+ XLogRecPtr end_lsn;
bool stats_per_record;
- start_lsn = PG_GETARG_LSN(0);
- stats_per_record = PG_GETARG_BOOL(1);
+ if (PG_ARGISNULL(0) ||
+ ((start_lsn = PG_GETARG_LSN(0)) == InvalidXLogRecPtr))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid start LSN")));
+
+ /* Let's compute end LSN ourselves. */
+ end_lsn = GetCurrentLSN();
+
+ /*
+ * When start LSN is past the end LSN, let's return NULL instead of raising
+ * an error for better usability of the functions.
+ */
+ if (start_lsn >= end_lsn)
+ PG_RETURN_NULL();
- end_lsn = ValidateInputLSNs(true, start_lsn, end_lsn);
+ stats_per_record = PG_GETARG_BOOL(2);
GetWalStats(fcinfo, start_lsn, end_lsn, stats_per_record);
diff --git a/contrib/pg_walinspect/sql/oldextversions.sql b/contrib/pg_walinspect/sql/oldextversions.sql
new file mode 100644
index 0000000000..3ccea0cdb2
--- /dev/null
+++ b/contrib/pg_walinspect/sql/oldextversions.sql
@@ -0,0 +1,27 @@
+-- test old extension version entry points
+
+DROP EXTENSION pg_walinspect;
+CREATE EXTENSION pg_walinspect WITH VERSION '1.0';
+
+-- Move to new version 1.1
+ALTER EXTENSION pg_walinspect UPDATE TO '1.1';
+
+-- New function introduced in 1.1
+SELECT pg_get_functiondef('pg_get_wal_block_info'::regproc);
+
+-- Redefined function
+SELECT pg_get_functiondef('pg_get_wal_records_info'::regproc);
+
+-- Removed function
+SELECT pg_get_functiondef('pg_get_wal_records_info_till_end_of_wal'::regproc);
+
+-- Redefined function
+SELECT pg_get_functiondef('pg_get_wal_stats'::regproc);
+
+-- Removed function
+SELECT pg_get_functiondef('pg_get_wal_stats_till_end_of_wal'::regproc);
+
+-- Redefined function
+SELECT pg_get_functiondef('pg_get_wal_block_info'::regproc);
+
+DROP EXTENSION pg_walinspect;
diff --git a/contrib/pg_walinspect/sql/pg_walinspect.sql b/contrib/pg_walinspect/sql/pg_walinspect.sql
index 01a120f398..602aa9c7ae 100644
--- a/contrib/pg_walinspect/sql/pg_walinspect.sql
+++ b/contrib/pg_walinspect/sql/pg_walinspect.sql
@@ -4,43 +4,50 @@ CREATE EXTENSION pg_walinspect;
SELECT 'init' FROM pg_create_physical_replication_slot('regress_pg_walinspect_slot', true, false);
CREATE TABLE sample_tbl(col1 int, col2 int);
-
SELECT pg_current_wal_lsn() AS wal_lsn1 \gset
-
INSERT INTO sample_tbl SELECT * FROM generate_series(1, 2);
-
SELECT pg_current_wal_lsn() AS wal_lsn2 \gset
-
INSERT INTO sample_tbl SELECT * FROM generate_series(3, 4);
-- ===================================================================
-- Tests for input validation
-- ===================================================================
-
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn2', :'wal_lsn1'); -- ERROR
-
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn2', :'wal_lsn1'); -- ERROR
+-- Invalid input LSNs
+SELECT * FROM pg_get_wal_record_info('0/0'); -- ERROR
+SELECT * FROM pg_get_wal_records_info('0/0'); -- ERROR
+SELECT * FROM pg_get_wal_stats('0/0'); -- ERROR
+SELECT * FROM pg_get_wal_block_info('0/0'); -- ERROR
+
+-- Start LSN NULL
+SELECT * FROM pg_get_wal_records_info(NULL); -- ERROR
+SELECT * FROM pg_get_wal_stats(NULL); -- ERROR
+SELECT * FROM pg_get_wal_block_info(NULL); -- ERROR
+
+-- Start LSN > End LSN
+SELECT * FROM pg_get_wal_records_info(:'wal_lsn2', :'wal_lsn1'); -- NULL
+SELECT * FROM pg_get_wal_stats(:'wal_lsn2', :'wal_lsn1'); -- NULL
+SELECT * FROM pg_get_wal_block_info(:'wal_lsn2', :'wal_lsn1'); -- NULL
-- ===================================================================
-- Tests for all function executions
-- ===================================================================
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
-
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
-
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info_till_end_of_wal(:'wal_lsn1');
+-- Get info till end of WAL
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', '0/0');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', NULL);
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
-
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats_till_end_of_wal(:'wal_lsn1');
+-- Get stats till end of WAL
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', '0/0');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', NULL);
-- ===================================================================
-- Test for filtering out WAL records of a particular table
-- ===================================================================
-
SELECT oid AS sample_tbl_oid FROM pg_class WHERE relname = 'sample_tbl' \gset
-
SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2')
WHERE block_ref LIKE concat('%', :'sample_tbl_oid', '%') AND resource_manager = 'Heap';
@@ -48,14 +55,12 @@ SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2'
-- Test for filtering out WAL records based on resource_manager and
-- record_type
-- ===================================================================
-
SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2')
WHERE resource_manager = 'Heap' AND record_type = 'INSERT';
-- ===================================================================
-- Tests to get block information from WAL record
-- ===================================================================
-
-- Update table to generate some block data
SELECT pg_current_wal_lsn() AS wal_lsn3 \gset
UPDATE sample_tbl SET col1 = col1 + 1 WHERE col1 = 1;
@@ -63,6 +68,9 @@ SELECT pg_current_wal_lsn() AS wal_lsn4 \gset
-- Check if we get block data from WAL record.
SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn3', :'wal_lsn4')
WHERE relfilenode = :'sample_tbl_oid' AND blockdata IS NOT NULL;
+-- Check till end of WAL if we get block data from WAL record.
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn3')
+ WHERE relfilenode = :'sample_tbl_oid';
-- Force full-page image on the next update.
SELECT pg_current_wal_lsn() AS wal_lsn5 \gset
@@ -72,6 +80,9 @@ SELECT pg_current_wal_lsn() AS wal_lsn6 \gset
-- Check if we get FPI from WAL record.
SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn5', :'wal_lsn6')
WHERE relfilenode = :'sample_tbl_oid' AND fpi IS NOT NULL;
+-- Check till end of WAL if we get FPI from WAL record.
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn5')
+ WHERE relfilenode = :'sample_tbl_oid';
-- ===================================================================
-- Tests for permissions
@@ -80,29 +91,22 @@ CREATE ROLE regress_pg_walinspect;
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_record_info(pg_lsn)', 'EXECUTE'); -- no
-
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_records_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- no
-
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_stats(pg_lsn, pg_lsn, boolean) ', 'EXECUTE'); -- no
-
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_block_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- no
-- Functions accessible by users with role pg_read_server_files
-
GRANT pg_read_server_files TO regress_pg_walinspect;
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_record_info(pg_lsn)', 'EXECUTE'); -- yes
-
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_records_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- yes
-
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_stats(pg_lsn, pg_lsn, boolean) ', 'EXECUTE'); -- yes
-
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_block_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- yes
@@ -111,46 +115,34 @@ REVOKE pg_read_server_files FROM regress_pg_walinspect;
-- Superuser can grant execute to other users
GRANT EXECUTE ON FUNCTION pg_get_wal_record_info(pg_lsn)
TO regress_pg_walinspect;
-
GRANT EXECUTE ON FUNCTION pg_get_wal_records_info(pg_lsn, pg_lsn)
TO regress_pg_walinspect;
-
GRANT EXECUTE ON FUNCTION pg_get_wal_stats(pg_lsn, pg_lsn, boolean)
TO regress_pg_walinspect;
-
GRANT EXECUTE ON FUNCTION pg_get_wal_block_info(pg_lsn, pg_lsn)
TO regress_pg_walinspect;
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_record_info(pg_lsn)', 'EXECUTE'); -- yes
-
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_records_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- yes
-
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_stats(pg_lsn, pg_lsn, boolean) ', 'EXECUTE'); -- yes
-
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_block_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- yes
REVOKE EXECUTE ON FUNCTION pg_get_wal_record_info(pg_lsn)
FROM regress_pg_walinspect;
-
REVOKE EXECUTE ON FUNCTION pg_get_wal_records_info(pg_lsn, pg_lsn)
FROM regress_pg_walinspect;
-
REVOKE EXECUTE ON FUNCTION pg_get_wal_stats(pg_lsn, pg_lsn, boolean)
FROM regress_pg_walinspect;
-
REVOKE EXECUTE ON FUNCTION pg_get_wal_block_info(pg_lsn, pg_lsn)
FROM regress_pg_walinspect;
-- ===================================================================
-- Clean up
-- ===================================================================
-
DROP ROLE regress_pg_walinspect;
-
SELECT pg_drop_replication_slot('regress_pg_walinspect_slot');
-
DROP TABLE sample_tbl;
diff --git a/doc/src/sgml/pgwalinspect.sgml b/doc/src/sgml/pgwalinspect.sgml
index 3b19863dce..4ecbd32ba1 100644
--- a/doc/src/sgml/pgwalinspect.sgml
+++ b/doc/src/sgml/pgwalinspect.sgml
@@ -61,6 +61,7 @@
Gets WAL record information of a given LSN. If the given LSN isn't
at the start of a WAL record, it gives the information of the next
available valid WAL record; or an error if no such record is found.
+ If given LSN is not yet available, the function will raise an error.
For example, usage of the function is as
follows:
<screen>
@@ -85,7 +86,7 @@ block_ref | blkref #0: rel 1663/5/60221 fork main blk 2
<varlistentry id="pgwalinspect-funcs-pg-get-wal-records-info">
<term>
<function>
- pg_get_wal_records_info(start_lsn pg_lsn, end_lsn pg_lsn)
+ pg_get_wal_records_info(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL)
returns setof record
</function>
</term>
@@ -95,8 +96,9 @@ block_ref | blkref #0: rel 1663/5/60221 fork main blk 2
Gets information of all the valid WAL records between
<replaceable>start_lsn</replaceable> and <replaceable>end_lsn</replaceable>.
Returns one row per WAL record. If <replaceable>start_lsn</replaceable>
- or <replaceable>end_lsn</replaceable> are not yet available, the
- function will raise an error. For example:
+ is not yet available, it will raise an error. If <replaceable>end_lsn</replaceable>
+ isn't specified, it returns information till the end of WAL. For example,
+ usage of the function is as follows:
<screen>
postgres=# SELECT * FROM pg_get_wal_records_info('0/1E913618', '0/1E913740') LIMIT 1;
-[ RECORD 1 ]----+--------------------------------------------------------------
@@ -116,27 +118,10 @@ block_ref |
</listitem>
</varlistentry>
- <varlistentry id="pgwalinspect-funcs-pg-get-wal-records-info-till-end-of-wal">
- <term>
- <function>
- pg_get_wal_records_info_till_end_of_wal(start_lsn pg_lsn)
- returns setof record
- </function>
- </term>
-
- <listitem>
- <para>
- This function is the same as <function>pg_get_wal_records_info()</function>,
- except that it gets information of all the valid WAL records from
- <replaceable>start_lsn</replaceable> till the end of WAL.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry id="pgwalinspect-funcs-pg-get-wal-stats">
<term>
<function>
- pg_get_wal_stats(start_lsn pg_lsn, end_lsn pg_lsn, per_record boolean DEFAULT false)
+ pg_get_wal_stats(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL, per_record boolean DEFAULT false)
returns setof record
</function>
</term>
@@ -149,9 +134,10 @@ block_ref |
<replaceable>resource_manager</replaceable> type. When
<replaceable>per_record</replaceable> is set to <literal>true</literal>,
it returns one row per <replaceable>record_type</replaceable>.
- If <replaceable>start_lsn</replaceable>
- or <replaceable>end_lsn</replaceable> are not yet available, the
- function will raise an error. For example:
+ If <replaceable>start_lsn</replaceable> is not yet available, it will
+ raise an error. If <replaceable>end_lsn</replaceable> isn't specified, it
+ returns information till the end of WAL. For example, usage of the
+ function is as follows:
<screen>
postgres=# SELECT * FROM pg_get_wal_stats('0/1E847D00', '0/1E84F500')
WHERE count > 0 LIMIT 1 AND
@@ -171,23 +157,6 @@ combined_size_percentage | 2.8634072910530795
</listitem>
</varlistentry>
- <varlistentry id="pgwalinspect-funcs-pg-get-wal-stats-till-end-of-wal">
- <term>
- <function>
- pg_get_wal_stats_till_end_of_wal(start_lsn pg_lsn, per_record boolean DEFAULT false)
- returns setof record
- </function>
- </term>
-
- <listitem>
- <para>
- This function is the same as <function>pg_get_wal_stats()</function>,
- except that it gets statistics of all the valid WAL records from
- <replaceable>start_lsn</replaceable> till end of WAL.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term>
<function>pg_get_wal_block_info(start_lsn pg_lsn, end_lsn pg_lsn) returns setof record</function>
@@ -202,9 +171,10 @@ combined_size_percentage | 2.8634072910530795
and their information associated with all the valid WAL records between
<replaceable>start_lsn</replaceable> and
<replaceable>end_lsn</replaceable>. Returns one row per block registered
- in a WAL record. If <replaceable>start_lsn</replaceable> or
- <replaceable>end_lsn</replaceable> are not yet available, the function
- will raise an error. For example:
+ in a WAL record. If <replaceable>start_lsn</replaceable> is not yet
+ available, it will raise an error. If <replaceable>end_lsn</replaceable>
+ isn't specified, it returns information till the end of WAL. For example,
+ usage of the function is as follows:
<screen>
postgres=# SELECT lsn, blockid, reltablespace, reldatabase, relfilenode,
relblocknumber, forkname,
@@ -227,8 +197,21 @@ fpiinfo | {HAS_HOLE,APPLY}
</para>
</listitem>
</varlistentry>
-
</variablelist>
+
+ <note>
+ <para>
+ Note that <function>pg_get_wal_records_info_till_end_of_wal</function> and
+ <function>pg_get_wal_stats_till_end_of_wal</function> functions have been
+ removed in the <filename>pg_walinspect</filename> version
+ <literal>1.1</literal>. The same functionality can be achieved with
+ <function>pg_get_wal_records_info</function> and
+ <function>pg_get_wal_stats</function> functions. However,
+ <function>till_end_of_wal</function> functions will still work if the
+ extension is installed explicitly with version <literal>1.0</literal>.
+ </para>
+ </note>
+
</sect2>
<sect2 id="pgwalinspect-author">
--
2.34.1
Re: Combine pg_walinspect till_end_of_wal functions with others
On Fri, Mar 10, 2023 at 12:24 PM Michael Paquier <michael@paquier.xyz> wrote:
On Wed, Mar 08, 2023 at 01:40:46PM +0530, Bharath Rupireddy wrote:
1. When start_lsn is NULL or invalid ('0/0'), emit an error. There was
a comment on the functions automatically determining start_lsn to be
the oldest WAL LSN. I'm not implementing this change now, as it
requires extra work to traverse the pg_wal directory. I'm planning to
do it in the next set of improvements where I'm planning to make the
functions timeline-aware, introduce functions for inspecting
wal_buffers and so on.[.. long description ..]
9. Refactored the tests according to the new behaviours.
Hmm. I think this patch ought to have a result simpler than what's
proposed here.First, do we really have to begin marking the functions as non-STRICT
to abide with the treatment of NULL as a special value? The part that
I've found personally the most annoying with these functions is that
an incorrect bound leads to a random failure, particularly when such
queries are used for monitoring.
As long as we provide a sensible default value (so I guess '0/0' to
mean "no upper bound") and that we therefore don't have to manually
specify an upper bound if we don't want one I'm fine with keeping the
functions marked as STRICT.
I would simplify the whole with two
simple rules, as of:
- Keeping all the functions strict.
- When end_lsn is a LSN in the future of the current LSN inserted or
replayed, adjust its value to be the exactly GetXLogReplayRecPtr() or
GetFlushRecPtr(). This way, monitoring tools can use a value ahead,
at will.
- Failing if start_lsn > end_lsn.
- Failing if start_lsn refers to a position older than what exists is
still fine by me.
+1
I would also choose to remove
pg_get_wal_records_info_till_end_of_wal() from the SQL interface in
1.1 to limit the confusion arount it, but keep a few lines of code so
as we are still able to use it when pg_walinspect 1.0 is the version
enabled with CREATE EXTENSION.
Yeah the SQL function should be removed no matter what.
Re: Combine pg_walinspect till_end_of_wal functions with others
On Fri, Mar 10, 2023 at 04:04:15PM +0800, Julien Rouhaud wrote:
As long as we provide a sensible default value (so I guess '0/0' to
mean "no upper bound") and that we therefore don't have to manually
specify an upper bound if we don't want one I'm fine with keeping the
functions marked as STRICT.
FWIW, using also InvalidXLogRecPtr as a shortcut to say "Don't fail,
just do the job" is fine by me. Something like a FFF/FFFFFFFF should
just mean the same on a fresh cluster, still it gets risky the longer
the WAL is generated.
--
Michael
Re: Combine pg_walinspect till_end_of_wal functions with others
On Fri, 10 Mar 2023, 16:14 Michael Paquier, <michael@paquier.xyz> wrote:
On Fri, Mar 10, 2023 at 04:04:15PM +0800, Julien Rouhaud wrote:
As long as we provide a sensible default value (so I guess '0/0' to
mean "no upper bound") and that we therefore don't have to manually
specify an upper bound if we don't want one I'm fine with keeping the
functions marked as STRICT.FWIW, using also InvalidXLogRecPtr as a shortcut to say "Don't fail,
just do the job" is fine by me.
isn't '0/0' the same as InvalidXLogRecPtr? but my point is that we
shouldn't require to spell it explicitly, just rely on the default value.
Something like a FFF/FFFFFFFF should
just mean the same on a fresh cluster, still it gets risky the longer
the WAL is generated.
yeah, it would be handy to accept 'infinity' in that context.
Show quoted text
Re: Combine pg_walinspect till_end_of_wal functions with others
On Fri, Mar 10, 2023 at 04:37:23PM +0800, Julien Rouhaud wrote:
isn't '0/0' the same as InvalidXLogRecPtr? but my point is that we
shouldn't require to spell it explicitly, just rely on the default value.
Perhaps. Still the addition of a DEFAULT to the function definitions
and its value looks like a second patch to me. The first should just
lift the bound restrictions currently in place while cleaning up the
till_* functions.
--
Michael
Re: Combine pg_walinspect till_end_of_wal functions with others
On Fri, Mar 10, 2023 at 9:54 AM Michael Paquier <michael@paquier.xyz> wrote:
Hmm. I think this patch ought to have a result simpler than what's
proposed here.First, do we really have to begin marking the functions as non-STRICT
to abide with the treatment of NULL as a special value? The part that
I've found personally the most annoying with these functions is that
an incorrect bound leads to a random failure, particularly when such
queries are used for monitoring. I would simplify the whole with two
simple rules, as of:
- Keeping all the functions strict.
- When end_lsn is a LSN in the future of the current LSN inserted or
replayed, adjust its value to be the exactly GetXLogReplayRecPtr() or
GetFlushRecPtr(). This way, monitoring tools can use a value ahead,
at will.
- Failing if start_lsn > end_lsn.
- Failing if start_lsn refers to a position older than what exists is
still fine by me.
Done this way in the attached v5 patch.
I would also choose to remove
pg_get_wal_records_info_till_end_of_wal() from the SQL interface in
1.1 to limit the confusion arount it, but keep a few lines of code so
as we are still able to use it when pg_walinspect 1.0 is the version
enabled with CREATE EXTENSION.In short, pg_get_wal_records_info_till_end_of_wal() should be able to
use exactly the same code as pg_get_wal_records_info(), still you need
to keep *two* functions for their prosrc with PG_FUNCTION_ARGS as
arguments so as 1.0 would work when dropped in place. The result, it
seems to me, mostly comes to simplify ValidateInputLSNs() and remove
its till_end_of_wal argument.
This has already been taken care of in the previous patches, e.g. v3
and v4 and so in the latest v5 patch.
+-- Removed function +SELECT pg_get_functiondef('pg_get_wal_records_info_till_end_of_wal'::regproc); +ERROR: function "pg_get_wal_records_info_till_end_of_wal" does not exist +LINE 1: SELECT pg_get_functiondef('pg_get_wal_records_info_till_end_...It seems to me that you should just replace all that and anything
depending on pg_get_functiondef() with a \dx+ pg_walinspect, that
would list all the objects part of the extension for the specific
version you want to test. Not sure that there is a need to list the
full function definitions, either. That just bloats the tests.
Agreed and used \dx+. One can anyways look at the function definitions
and compare for knowing what's changed.
I think, however, that it is critical to test in oldextversions.out
the *executions* of the functions, so as we make sure that they don't
crash. The patch is missing that.
You mean, we need to test the till_end_of_wal functions that were
removed in the latest version 1.1 but they must work if the extension
is installed with 1.0? If yes, I now added them.
+-- Invalid input LSNs +SELECT * FROM pg_get_wal_record_info('0/0'); -- ERROR +ERROR: invalid input LSN
Removed InvalidRecPtr checks for input/start LSN because anyways the
functions will fail with ERROR: could not read WAL at LSN 0/0.
Any comments on the attached v5 patch?
--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
Attachments:
v5-0001-Rework-pg_walinspect-functions.patchapplication/x-patch; name=v5-0001-Rework-pg_walinspect-functions.patchDownload
From 63360cfa8c2a3141a1360a966bc7e210eb57810b Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Fri, 10 Mar 2023 10:54:20 +0000
Subject: [PATCH v5] Rework pg_walinspect functions
---
contrib/pg_walinspect/Makefile | 2 +-
.../pg_walinspect/expected/oldextversions.out | 55 +++++
.../pg_walinspect/expected/pg_walinspect.out | 60 ++++-
contrib/pg_walinspect/meson.build | 1 +
.../pg_walinspect/pg_walinspect--1.0--1.1.sql | 4 +
contrib/pg_walinspect/pg_walinspect.c | 213 ++++++++----------
contrib/pg_walinspect/sql/oldextversions.sql | 29 +++
contrib/pg_walinspect/sql/pg_walinspect.sql | 74 +++---
doc/src/sgml/pgwalinspect.sgml | 88 ++++----
9 files changed, 307 insertions(+), 219 deletions(-)
create mode 100644 contrib/pg_walinspect/expected/oldextversions.out
create mode 100644 contrib/pg_walinspect/sql/oldextversions.sql
diff --git a/contrib/pg_walinspect/Makefile b/contrib/pg_walinspect/Makefile
index 7033878a79..22090f7716 100644
--- a/contrib/pg_walinspect/Makefile
+++ b/contrib/pg_walinspect/Makefile
@@ -9,7 +9,7 @@ PGFILEDESC = "pg_walinspect - functions to inspect contents of PostgreSQL Write-
EXTENSION = pg_walinspect
DATA = pg_walinspect--1.0.sql pg_walinspect--1.0--1.1.sql
-REGRESS = pg_walinspect
+REGRESS = pg_walinspect oldextversions
REGRESS_OPTS = --temp-config $(top_srcdir)/contrib/pg_walinspect/walinspect.conf
diff --git a/contrib/pg_walinspect/expected/oldextversions.out b/contrib/pg_walinspect/expected/oldextversions.out
new file mode 100644
index 0000000000..4ad85392f0
--- /dev/null
+++ b/contrib/pg_walinspect/expected/oldextversions.out
@@ -0,0 +1,55 @@
+-- test old extension version entry points
+DROP EXTENSION pg_walinspect;
+CREATE EXTENSION pg_walinspect WITH VERSION '1.0';
+-- List what version 1.0 contains
+\dx+ pg_walinspect
+ Objects in extension "pg_walinspect"
+ Object description
+-----------------------------------------------------------
+ function pg_get_wal_record_info(pg_lsn)
+ function pg_get_wal_records_info(pg_lsn,pg_lsn)
+ function pg_get_wal_records_info_till_end_of_wal(pg_lsn)
+ function pg_get_wal_stats(pg_lsn,pg_lsn,boolean)
+ function pg_get_wal_stats_till_end_of_wal(pg_lsn,boolean)
+(5 rows)
+
+-- ===================================================================
+-- Tests to check if the removed functions work when older version of
+-- extension is explicitly installed.
+-- ===================================================================
+-- Make sure checkpoints don't interfere with the test.
+SELECT 'init' FROM pg_create_physical_replication_slot('regress_pg_walinspect_slot', true, false);
+ ?column?
+----------
+ init
+(1 row)
+
+CREATE TABLE sample_tbl(col1 int, col2 int);
+SELECT pg_current_wal_lsn() AS wal_lsn1 \gset
+INSERT INTO sample_tbl SELECT * FROM generate_series(1, 2);
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info_till_end_of_wal(:'wal_lsn1');
+ ok
+----
+ t
+(1 row)
+
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats_till_end_of_wal(:'wal_lsn1');
+ ok
+----
+ t
+(1 row)
+
+-- Move to new version 1.1
+ALTER EXTENSION pg_walinspect UPDATE TO '1.1';
+-- List what version 1.1 contains
+\dx+ pg_walinspect
+ Objects in extension "pg_walinspect"
+ Object description
+--------------------------------------------------
+ function pg_get_wal_block_info(pg_lsn,pg_lsn)
+ function pg_get_wal_record_info(pg_lsn)
+ function pg_get_wal_records_info(pg_lsn,pg_lsn)
+ function pg_get_wal_stats(pg_lsn,pg_lsn,boolean)
+(4 rows)
+
+DROP EXTENSION pg_walinspect;
diff --git a/contrib/pg_walinspect/expected/pg_walinspect.out b/contrib/pg_walinspect/expected/pg_walinspect.out
index e0eb7ca08f..d3ddc2409c 100644
--- a/contrib/pg_walinspect/expected/pg_walinspect.out
+++ b/contrib/pg_walinspect/expected/pg_walinspect.out
@@ -1,11 +1,12 @@
CREATE EXTENSION pg_walinspect;
--- Make sure checkpoints don't interfere with the test.
+-- Make sure checkpoints don't interfere with the test
SELECT 'init' FROM pg_create_physical_replication_slot('regress_pg_walinspect_slot', true, false);
?column?
----------
init
(1 row)
+-- Generate WAL and captures LSNs as needed
CREATE TABLE sample_tbl(col1 int, col2 int);
SELECT pg_current_wal_lsn() AS wal_lsn1 \gset
INSERT INTO sample_tbl SELECT * FROM generate_series(1, 2);
@@ -14,38 +15,55 @@ INSERT INTO sample_tbl SELECT * FROM generate_series(3, 4);
-- ===================================================================
-- Tests for input validation
-- ===================================================================
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn2', :'wal_lsn1'); -- ERROR
+-- Invalid input LSN or LSN that server doesn't have WAL for
+SELECT * FROM pg_get_wal_record_info('0/0'); -- ERROR
+ERROR: could not read WAL at LSN 0/0
+-- Invalid start LSN or start LSN that server doesn't have WAL for
+SELECT * FROM pg_get_wal_records_info('0/0', :'wal_lsn1'); -- ERROR
+ERROR: could not read WAL at LSN 0/0
+SELECT * FROM pg_get_wal_stats('0/0', :'wal_lsn1'); -- ERROR
+ERROR: could not read WAL at LSN 0/0
+SELECT * FROM pg_get_wal_block_info('0/0', :'wal_lsn1'); -- ERROR
+ERROR: could not read WAL at LSN 0/0
+-- Start LSN >= End LSN
+SELECT * FROM pg_get_wal_records_info(:'wal_lsn2', :'wal_lsn1'); -- ERROR
ERROR: WAL start LSN must be less than end LSN
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn2', :'wal_lsn1'); -- ERROR
+SELECT * FROM pg_get_wal_stats(:'wal_lsn2', :'wal_lsn1'); -- ERROR
+ERROR: WAL start LSN must be less than end LSN
+SELECT * FROM pg_get_wal_block_info(:'wal_lsn2', :'wal_lsn1'); -- ERROR
ERROR: WAL start LSN must be less than end LSN
-- ===================================================================
-- Tests for all function executions
-- ===================================================================
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info_till_end_of_wal(:'wal_lsn1');
+-- Get info till end of WAL. End LSN is way higher, adjust it to the latest LSN
+-- that server knows.
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats_till_end_of_wal(:'wal_lsn1');
+-- Get info till end of WAL. End LSN is way higher, adjust it to the latest LSN
+-- that server knows.
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
ok
----
t
@@ -64,7 +82,7 @@ SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2'
-- ===================================================================
-- Test for filtering out WAL records based on resource_manager and
--- record_type
+-- record_type.
-- ===================================================================
SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2')
WHERE resource_manager = 'Heap' AND record_type = 'INSERT';
@@ -80,7 +98,7 @@ SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2'
SELECT pg_current_wal_lsn() AS wal_lsn3 \gset
UPDATE sample_tbl SET col1 = col1 + 1 WHERE col1 = 1;
SELECT pg_current_wal_lsn() AS wal_lsn4 \gset
--- Check if we get block data from WAL record.
+-- Check if we get block data from WAL record
SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn3', :'wal_lsn4')
WHERE relfilenode = :'sample_tbl_oid' AND blockdata IS NOT NULL;
ok
@@ -88,12 +106,21 @@ SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn3', :'wal_lsn4')
t
(1 row)
--- Force full-page image on the next update.
+-- Check till end of WAL if we get block data from WAL record. End LSN is way
+-- higher, adjust it to the latest LSN that server knows.
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn3', 'FFFFFFFF/FFFFFFFF')
+ WHERE relfilenode = :'sample_tbl_oid';
+ ok
+----
+ t
+(1 row)
+
+-- Force full-page image on the next update
SELECT pg_current_wal_lsn() AS wal_lsn5 \gset
CHECKPOINT;
UPDATE sample_tbl SET col1 = col1 + 1 WHERE col1 = 2;
SELECT pg_current_wal_lsn() AS wal_lsn6 \gset
--- Check if we get FPI from WAL record.
+-- Check if we get FPI from WAL record
SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn5', :'wal_lsn6')
WHERE relfilenode = :'sample_tbl_oid' AND fpi IS NOT NULL;
ok
@@ -101,6 +128,15 @@ SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn5', :'wal_lsn6')
t
(1 row)
+-- Check till end of WAL if we get FPI from WAL record. End LSN is way higher,
+-- adjust it to the latest LSN that server knoows.
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn5', 'FFFFFFFF/FFFFFFFF')
+ WHERE relfilenode = :'sample_tbl_oid';
+ ok
+----
+ t
+(1 row)
+
-- ===================================================================
-- Tests for permissions
-- ===================================================================
diff --git a/contrib/pg_walinspect/meson.build b/contrib/pg_walinspect/meson.build
index bf7b79b1b7..80059f6119 100644
--- a/contrib/pg_walinspect/meson.build
+++ b/contrib/pg_walinspect/meson.build
@@ -30,6 +30,7 @@ tests += {
'regress': {
'sql': [
'pg_walinspect',
+ 'oldextversions',
],
# Disabled because these tests require "wal_level=replica", which
# some runningcheck users do not have (e.g. buildfarm clients).
diff --git a/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql b/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql
index e674ef25aa..a9c9ddda0e 100644
--- a/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql
+++ b/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql
@@ -3,6 +3,10 @@
-- complain if script is sourced in psql, rather than via ALTER EXTENSION
\echo Use "ALTER EXTENSION pg_walinspect UPDATE TO '1.1'" to load this file. \quit
+/* Drop the unneeded functions */
+DROP FUNCTION pg_get_wal_records_info_till_end_of_wal(pg_lsn);
+DROP FUNCTION pg_get_wal_stats_till_end_of_wal(pg_lsn, boolean);
+
--
-- pg_get_wal_block_info()
--
diff --git a/contrib/pg_walinspect/pg_walinspect.c b/contrib/pg_walinspect/pg_walinspect.c
index ee88dc4992..f604112623 100644
--- a/contrib/pg_walinspect/pg_walinspect.c
+++ b/contrib/pg_walinspect/pg_walinspect.c
@@ -38,15 +38,14 @@ PG_FUNCTION_INFO_V1(pg_get_wal_records_info_till_end_of_wal);
PG_FUNCTION_INFO_V1(pg_get_wal_stats);
PG_FUNCTION_INFO_V1(pg_get_wal_stats_till_end_of_wal);
-static bool IsFutureLSN(XLogRecPtr lsn, XLogRecPtr *curr_lsn);
+static XLogRecPtr GetCurrentLSN(void);
static XLogReaderState *InitXLogReaderState(XLogRecPtr lsn);
static XLogRecord *ReadNextXLogRecord(XLogReaderState *xlogreader);
static void GetWALRecordInfo(XLogReaderState *record, Datum *values,
bool *nulls, uint32 ncols);
-static XLogRecPtr ValidateInputLSNs(bool till_end_of_wal,
- XLogRecPtr start_lsn, XLogRecPtr end_lsn);
-static void GetWALRecordsInfo(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
- XLogRecPtr end_lsn);
+static void GetInputLSNs(FunctionCallInfo fcinfo, XLogRecPtr *start_lsn,
+ XLogRecPtr *end_lsn, bool till_end_of_wal);
+static void GetWALRecordsInfo(FunctionCallInfo fcinfo, bool till_end_of_wal);
static void GetXLogSummaryStats(XLogStats *stats, ReturnSetInfo *rsinfo,
Datum *values, bool *nulls, uint32 ncols,
bool stats_per_record);
@@ -55,32 +54,29 @@ static void FillXLogStatsRow(const char *name, uint64 n, uint64 total_count,
uint64 fpi_len, uint64 total_fpi_len,
uint64 tot_len, uint64 total_len,
Datum *values, bool *nulls, uint32 ncols);
-static void GetWalStats(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
- XLogRecPtr end_lsn, bool stats_per_record);
+static void GetWalStats(FunctionCallInfo fcinfo, bool till_end_of_wal);
static void GetWALBlockInfo(FunctionCallInfo fcinfo, XLogReaderState *record);
/*
- * Check if the given LSN is in future. Also, return the LSN up to which the
- * server has WAL.
+ * Return the LSN up to which the server has WAL.
*/
-static bool
-IsFutureLSN(XLogRecPtr lsn, XLogRecPtr *curr_lsn)
+static XLogRecPtr
+GetCurrentLSN(void)
{
+ XLogRecPtr curr_lsn;
+
/*
* We determine the current LSN of the server similar to how page_read
* callback read_local_xlog_page_no_wait does.
*/
if (!RecoveryInProgress())
- *curr_lsn = GetFlushRecPtr(NULL);
+ curr_lsn = GetFlushRecPtr(NULL);
else
- *curr_lsn = GetXLogReplayRecPtr(NULL);
+ curr_lsn = GetXLogReplayRecPtr(NULL);
- Assert(!XLogRecPtrIsInvalid(*curr_lsn));
+ Assert(!XLogRecPtrIsInvalid(curr_lsn));
- if (lsn >= *curr_lsn)
- return true;
-
- return false;
+ return curr_lsn;
}
/*
@@ -355,8 +351,13 @@ GetWALBlockInfo(FunctionCallInfo fcinfo, XLogReaderState *record)
* to a record. Decompression is applied to the full page images, if
* necessary.
*
- * This function emits an error if a future start or end WAL LSN i.e. WAL LSN
- * the database system doesn't know about is specified.
+ * This function emits an error if the start LSN is in future i.e. LSN the
+ * database system doesn't know about.
+ *
+ * This function adjusts future end LSN to the last flushed LSN when not in
+ * recovery or the last replayed LSN when in recovery.
+ *
+ * This function emits an error, if start LSN is past the end LSN.
*/
Datum
pg_get_wal_block_info(PG_FUNCTION_ARGS)
@@ -367,10 +368,7 @@ pg_get_wal_block_info(PG_FUNCTION_ARGS)
MemoryContext old_cxt;
MemoryContext tmp_cxt;
- start_lsn = PG_GETARG_LSN(0);
- end_lsn = PG_GETARG_LSN(1);
-
- end_lsn = ValidateInputLSNs(false, start_lsn, end_lsn);
+ GetInputLSNs(fcinfo, &start_lsn, &end_lsn, false);
InitMaterializedSRF(fcinfo, 0);
@@ -405,8 +403,8 @@ pg_get_wal_block_info(PG_FUNCTION_ARGS)
/*
* Get WAL record info.
*
- * This function emits an error if a future WAL LSN i.e. WAL LSN the database
- * system doesn't know about is specified.
+ * This function emits an error if input LSN is in future i.e. LSN the database
+ * system doesn't know about.
*/
Datum
pg_get_wal_record_info(PG_FUNCTION_ARGS)
@@ -422,20 +420,14 @@ pg_get_wal_record_info(PG_FUNCTION_ARGS)
HeapTuple tuple;
lsn = PG_GETARG_LSN(0);
+ curr_lsn = GetCurrentLSN();
- if (IsFutureLSN(lsn, &curr_lsn))
- {
- /*
- * GetFlushRecPtr or GetXLogReplayRecPtr gives "end+1" LSN of the last
- * record flushed or replayed respectively. But let's use the LSN up
- * to "end" in user facing message.
- */
+ if (lsn >= curr_lsn)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("cannot accept future input LSN"),
errdetail("Last known WAL LSN on the database system is at %X/%X.",
LSN_FORMAT_ARGS(curr_lsn))));
- }
/* Build a tuple descriptor for our result type. */
if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
@@ -462,54 +454,54 @@ pg_get_wal_record_info(PG_FUNCTION_ARGS)
}
/*
- * Validate the input LSNs and compute end LSN for till_end_of_wal versions.
+ * Get start and end LSN. Adjust end LSN if needed.
*/
-static XLogRecPtr
-ValidateInputLSNs(bool till_end_of_wal, XLogRecPtr start_lsn,
- XLogRecPtr end_lsn)
+static void
+GetInputLSNs(FunctionCallInfo fcinfo, XLogRecPtr *start_lsn,
+ XLogRecPtr *end_lsn, bool till_end_of_wal)
{
XLogRecPtr curr_lsn;
- if (IsFutureLSN(start_lsn, &curr_lsn))
- {
- /*
- * GetFlushRecPtr or GetXLogReplayRecPtr gives "end+1" LSN of the last
- * record flushed or replayed respectively. But let's use the LSN up
- * to "end" in user facing message.
- */
+ *start_lsn = PG_GETARG_LSN(0);
+ curr_lsn = GetCurrentLSN();
+
+ if (*start_lsn > curr_lsn)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("cannot accept future start LSN"),
errdetail("Last known WAL LSN on the database system is at %X/%X.",
LSN_FORMAT_ARGS(curr_lsn))));
- }
if (till_end_of_wal)
- end_lsn = curr_lsn;
+ *end_lsn = curr_lsn;
+ else
+ {
+ *end_lsn = PG_GETARG_LSN(1);
- if (end_lsn > curr_lsn)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("cannot accept future end LSN"),
- errdetail("Last known WAL LSN on the database system is at %X/%X.",
- LSN_FORMAT_ARGS(curr_lsn))));
+ /*
+ * Adjust end LSN to what the system knows at this point instead of
+ * raising errors. This helps to achieve till end of WAL functionality.
+ */
+ if (*end_lsn > curr_lsn)
+ *end_lsn = curr_lsn;
+ }
- if (start_lsn >= end_lsn)
+ /* Return false, when start LSN is past the end LSN. */
+ if (*start_lsn >= *end_lsn)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("WAL start LSN must be less than end LSN")));
-
- return end_lsn;
}
/*
* Get info and data of all WAL records between start LSN and end LSN.
*/
static void
-GetWALRecordsInfo(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
- XLogRecPtr end_lsn)
+GetWALRecordsInfo(FunctionCallInfo fcinfo, bool till_end_of_wal)
{
#define PG_GET_WAL_RECORDS_INFO_COLS 11
+ XLogRecPtr start_lsn;
+ XLogRecPtr end_lsn;
XLogReaderState *xlogreader;
ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
Datum values[PG_GET_WAL_RECORDS_INFO_COLS] = {0};
@@ -517,6 +509,8 @@ GetWALRecordsInfo(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
MemoryContext old_cxt;
MemoryContext tmp_cxt;
+ GetInputLSNs(fcinfo, &start_lsn, &end_lsn, till_end_of_wal);
+
InitMaterializedSRF(fcinfo, 0);
xlogreader = InitXLogReaderState(start_lsn);
@@ -554,42 +548,18 @@ GetWALRecordsInfo(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
/*
* Get info and data of all WAL records between start LSN and end LSN.
*
- * This function emits an error if a future start or end WAL LSN i.e. WAL LSN
- * the database system doesn't know about is specified.
- */
-Datum
-pg_get_wal_records_info(PG_FUNCTION_ARGS)
-{
- XLogRecPtr start_lsn;
- XLogRecPtr end_lsn;
-
- start_lsn = PG_GETARG_LSN(0);
- end_lsn = PG_GETARG_LSN(1);
-
- end_lsn = ValidateInputLSNs(false, start_lsn, end_lsn);
-
- GetWALRecordsInfo(fcinfo, start_lsn, end_lsn);
-
- PG_RETURN_VOID();
-}
-
-/*
- * Get info and data of all WAL records from start LSN till end of WAL.
+ * This function emits an error if the start LSN is in future i.e. LSN the
+ * database system doesn't know about.
+ *
+ * This function adjusts future end LSN to the last flushed LSN when not in
+ * recovery or the last replayed LSN when in recovery.
*
- * This function emits an error if a future start i.e. WAL LSN the database
- * system doesn't know about is specified.
+ * This function emits an error, if start LSN is past the end LSN.
*/
Datum
-pg_get_wal_records_info_till_end_of_wal(PG_FUNCTION_ARGS)
+pg_get_wal_records_info(PG_FUNCTION_ARGS)
{
- XLogRecPtr start_lsn;
- XLogRecPtr end_lsn = InvalidXLogRecPtr;
-
- start_lsn = PG_GETARG_LSN(0);
-
- end_lsn = ValidateInputLSNs(true, start_lsn, end_lsn);
-
- GetWALRecordsInfo(fcinfo, start_lsn, end_lsn);
+ GetWALRecordsInfo(fcinfo, false);
PG_RETURN_VOID();
}
@@ -757,16 +727,22 @@ GetXLogSummaryStats(XLogStats *stats, ReturnSetInfo *rsinfo,
* Get WAL stats between start LSN and end LSN.
*/
static void
-GetWalStats(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
- XLogRecPtr end_lsn, bool stats_per_record)
+GetWalStats(FunctionCallInfo fcinfo, bool till_end_of_wal)
{
#define PG_GET_WAL_STATS_COLS 9
+ XLogRecPtr start_lsn;
+ XLogRecPtr end_lsn;
+ bool stats_per_record;
XLogReaderState *xlogreader;
XLogStats stats = {0};
ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
Datum values[PG_GET_WAL_STATS_COLS] = {0};
bool nulls[PG_GET_WAL_STATS_COLS] = {0};
+ GetInputLSNs(fcinfo, &start_lsn, &end_lsn, till_end_of_wal);
+
+ stats_per_record = PG_GETARG_BOOL(2);
+
InitMaterializedSRF(fcinfo, 0);
xlogreader = InitXLogReaderState(start_lsn);
@@ -792,46 +768,49 @@ GetWalStats(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
/*
* Get stats of all WAL records between start LSN and end LSN.
*
- * This function emits an error if a future start or end WAL LSN i.e. WAL LSN
- * the database system doesn't know about is specified.
+ * This function emits an error if the start LSN is in future i.e. LSN the
+ * database system doesn't know about.
+ *
+ * This function adjusts future end LSN to the last flushed LSN when not in
+ * recovery or the last replayed LSN when in recovery.
+ *
+ * This function emits an error, if start LSN is past the end LSN.
*/
Datum
pg_get_wal_stats(PG_FUNCTION_ARGS)
{
- XLogRecPtr start_lsn;
- XLogRecPtr end_lsn;
- bool stats_per_record;
-
- start_lsn = PG_GETARG_LSN(0);
- end_lsn = PG_GETARG_LSN(1);
- stats_per_record = PG_GETARG_BOOL(2);
-
- end_lsn = ValidateInputLSNs(false, start_lsn, end_lsn);
-
- GetWalStats(fcinfo, start_lsn, end_lsn, stats_per_record);
+ GetWalStats(fcinfo, false);
PG_RETURN_VOID();
}
/*
- * Get stats of all WAL records from start LSN till end of WAL.
+ * NB: Following till_end_of_wal functions have been removed in newer versions
+ * of extension. However, we keep them around for backward compatibility. Which
+ * means, these functions work if someone explicitly installs the older
+ * extension version (using CREATE EXTENSION pg_walinspect WITH VERSION '1.0';)
+ * containing them.
*
- * This function emits an error if a future start i.e. WAL LSN the database
- * system doesn't know about is specified.
+ * If definitions of these functions are removed completely, the extension
+ * fails to install. XXX: We might consider removing them eventually after
+ * enough extension versions are out and we do away with version '1.0'
+ * completely.
+ *
+ * In newer versions, one can use pg_get_wal_records_info()/pg_get_wal_stats()
+ * for the same till_end_of_wal functionality.
*/
Datum
-pg_get_wal_stats_till_end_of_wal(PG_FUNCTION_ARGS)
+pg_get_wal_records_info_till_end_of_wal(PG_FUNCTION_ARGS)
{
- XLogRecPtr start_lsn;
- XLogRecPtr end_lsn = InvalidXLogRecPtr;
- bool stats_per_record;
+ GetWALRecordsInfo(fcinfo, true);
- start_lsn = PG_GETARG_LSN(0);
- stats_per_record = PG_GETARG_BOOL(1);
-
- end_lsn = ValidateInputLSNs(true, start_lsn, end_lsn);
+ PG_RETURN_VOID();
+}
- GetWalStats(fcinfo, start_lsn, end_lsn, stats_per_record);
+Datum
+pg_get_wal_stats_till_end_of_wal(PG_FUNCTION_ARGS)
+{
+ GetWalStats(fcinfo, true);
PG_RETURN_VOID();
}
diff --git a/contrib/pg_walinspect/sql/oldextversions.sql b/contrib/pg_walinspect/sql/oldextversions.sql
new file mode 100644
index 0000000000..79a382acb4
--- /dev/null
+++ b/contrib/pg_walinspect/sql/oldextversions.sql
@@ -0,0 +1,29 @@
+-- test old extension version entry points
+
+DROP EXTENSION pg_walinspect;
+CREATE EXTENSION pg_walinspect WITH VERSION '1.0';
+
+-- List what version 1.0 contains
+\dx+ pg_walinspect
+
+-- ===================================================================
+-- Tests to check if the removed functions work when older version of
+-- extension is explicitly installed.
+-- ===================================================================
+-- Make sure checkpoints don't interfere with the test.
+SELECT 'init' FROM pg_create_physical_replication_slot('regress_pg_walinspect_slot', true, false);
+
+CREATE TABLE sample_tbl(col1 int, col2 int);
+SELECT pg_current_wal_lsn() AS wal_lsn1 \gset
+INSERT INTO sample_tbl SELECT * FROM generate_series(1, 2);
+
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info_till_end_of_wal(:'wal_lsn1');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats_till_end_of_wal(:'wal_lsn1');
+
+-- Move to new version 1.1
+ALTER EXTENSION pg_walinspect UPDATE TO '1.1';
+
+-- List what version 1.1 contains
+\dx+ pg_walinspect
+
+DROP EXTENSION pg_walinspect;
diff --git a/contrib/pg_walinspect/sql/pg_walinspect.sql b/contrib/pg_walinspect/sql/pg_walinspect.sql
index 01a120f398..8f2cdd09f9 100644
--- a/contrib/pg_walinspect/sql/pg_walinspect.sql
+++ b/contrib/pg_walinspect/sql/pg_walinspect.sql
@@ -1,77 +1,88 @@
CREATE EXTENSION pg_walinspect;
--- Make sure checkpoints don't interfere with the test.
+-- Make sure checkpoints don't interfere with the test
SELECT 'init' FROM pg_create_physical_replication_slot('regress_pg_walinspect_slot', true, false);
+-- Generate WAL and captures LSNs as needed
CREATE TABLE sample_tbl(col1 int, col2 int);
-
SELECT pg_current_wal_lsn() AS wal_lsn1 \gset
-
INSERT INTO sample_tbl SELECT * FROM generate_series(1, 2);
-
SELECT pg_current_wal_lsn() AS wal_lsn2 \gset
-
INSERT INTO sample_tbl SELECT * FROM generate_series(3, 4);
-- ===================================================================
-- Tests for input validation
-- ===================================================================
+-- Invalid input LSN or LSN that server doesn't have WAL for
+SELECT * FROM pg_get_wal_record_info('0/0'); -- ERROR
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn2', :'wal_lsn1'); -- ERROR
+-- Invalid start LSN or start LSN that server doesn't have WAL for
+SELECT * FROM pg_get_wal_records_info('0/0', :'wal_lsn1'); -- ERROR
+SELECT * FROM pg_get_wal_stats('0/0', :'wal_lsn1'); -- ERROR
+SELECT * FROM pg_get_wal_block_info('0/0', :'wal_lsn1'); -- ERROR
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn2', :'wal_lsn1'); -- ERROR
+-- Start LSN >= End LSN
+SELECT * FROM pg_get_wal_records_info(:'wal_lsn2', :'wal_lsn1'); -- ERROR
+SELECT * FROM pg_get_wal_stats(:'wal_lsn2', :'wal_lsn1'); -- ERROR
+SELECT * FROM pg_get_wal_block_info(:'wal_lsn2', :'wal_lsn1'); -- ERROR
-- ===================================================================
-- Tests for all function executions
-- ===================================================================
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
+-- Get info till end of WAL. End LSN is way higher, adjust it to the latest LSN
+-- that server knows.
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info_till_end_of_wal(:'wal_lsn1');
-
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
-
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats_till_end_of_wal(:'wal_lsn1');
+-- Get info till end of WAL. End LSN is way higher, adjust it to the latest LSN
+-- that server knows.
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
-- ===================================================================
-- Test for filtering out WAL records of a particular table
-- ===================================================================
-
SELECT oid AS sample_tbl_oid FROM pg_class WHERE relname = 'sample_tbl' \gset
-
SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2')
WHERE block_ref LIKE concat('%', :'sample_tbl_oid', '%') AND resource_manager = 'Heap';
-- ===================================================================
-- Test for filtering out WAL records based on resource_manager and
--- record_type
+-- record_type.
-- ===================================================================
-
SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2')
WHERE resource_manager = 'Heap' AND record_type = 'INSERT';
-- ===================================================================
-- Tests to get block information from WAL record
-- ===================================================================
-
-- Update table to generate some block data
SELECT pg_current_wal_lsn() AS wal_lsn3 \gset
UPDATE sample_tbl SET col1 = col1 + 1 WHERE col1 = 1;
SELECT pg_current_wal_lsn() AS wal_lsn4 \gset
--- Check if we get block data from WAL record.
+-- Check if we get block data from WAL record
SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn3', :'wal_lsn4')
WHERE relfilenode = :'sample_tbl_oid' AND blockdata IS NOT NULL;
+-- Check till end of WAL if we get block data from WAL record. End LSN is way
+-- higher, adjust it to the latest LSN that server knows.
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn3', 'FFFFFFFF/FFFFFFFF')
+ WHERE relfilenode = :'sample_tbl_oid';
--- Force full-page image on the next update.
+-- Force full-page image on the next update
SELECT pg_current_wal_lsn() AS wal_lsn5 \gset
CHECKPOINT;
UPDATE sample_tbl SET col1 = col1 + 1 WHERE col1 = 2;
SELECT pg_current_wal_lsn() AS wal_lsn6 \gset
--- Check if we get FPI from WAL record.
+-- Check if we get FPI from WAL record
SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn5', :'wal_lsn6')
WHERE relfilenode = :'sample_tbl_oid' AND fpi IS NOT NULL;
+-- Check till end of WAL if we get FPI from WAL record. End LSN is way higher,
+-- adjust it to the latest LSN that server knoows.
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn5', 'FFFFFFFF/FFFFFFFF')
+ WHERE relfilenode = :'sample_tbl_oid';
-- ===================================================================
-- Tests for permissions
@@ -80,29 +91,22 @@ CREATE ROLE regress_pg_walinspect;
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_record_info(pg_lsn)', 'EXECUTE'); -- no
-
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_records_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- no
-
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_stats(pg_lsn, pg_lsn, boolean) ', 'EXECUTE'); -- no
-
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_block_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- no
-- Functions accessible by users with role pg_read_server_files
-
GRANT pg_read_server_files TO regress_pg_walinspect;
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_record_info(pg_lsn)', 'EXECUTE'); -- yes
-
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_records_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- yes
-
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_stats(pg_lsn, pg_lsn, boolean) ', 'EXECUTE'); -- yes
-
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_block_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- yes
@@ -111,46 +115,34 @@ REVOKE pg_read_server_files FROM regress_pg_walinspect;
-- Superuser can grant execute to other users
GRANT EXECUTE ON FUNCTION pg_get_wal_record_info(pg_lsn)
TO regress_pg_walinspect;
-
GRANT EXECUTE ON FUNCTION pg_get_wal_records_info(pg_lsn, pg_lsn)
TO regress_pg_walinspect;
-
GRANT EXECUTE ON FUNCTION pg_get_wal_stats(pg_lsn, pg_lsn, boolean)
TO regress_pg_walinspect;
-
GRANT EXECUTE ON FUNCTION pg_get_wal_block_info(pg_lsn, pg_lsn)
TO regress_pg_walinspect;
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_record_info(pg_lsn)', 'EXECUTE'); -- yes
-
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_records_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- yes
-
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_stats(pg_lsn, pg_lsn, boolean) ', 'EXECUTE'); -- yes
-
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_block_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- yes
REVOKE EXECUTE ON FUNCTION pg_get_wal_record_info(pg_lsn)
FROM regress_pg_walinspect;
-
REVOKE EXECUTE ON FUNCTION pg_get_wal_records_info(pg_lsn, pg_lsn)
FROM regress_pg_walinspect;
-
REVOKE EXECUTE ON FUNCTION pg_get_wal_stats(pg_lsn, pg_lsn, boolean)
FROM regress_pg_walinspect;
-
REVOKE EXECUTE ON FUNCTION pg_get_wal_block_info(pg_lsn, pg_lsn)
FROM regress_pg_walinspect;
-- ===================================================================
-- Clean up
-- ===================================================================
-
DROP ROLE regress_pg_walinspect;
-
SELECT pg_drop_replication_slot('regress_pg_walinspect_slot');
-
DROP TABLE sample_tbl;
diff --git a/doc/src/sgml/pgwalinspect.sgml b/doc/src/sgml/pgwalinspect.sgml
index 3b19863dce..d9aac121d5 100644
--- a/doc/src/sgml/pgwalinspect.sgml
+++ b/doc/src/sgml/pgwalinspect.sgml
@@ -61,8 +61,8 @@
Gets WAL record information of a given LSN. If the given LSN isn't
at the start of a WAL record, it gives the information of the next
available valid WAL record; or an error if no such record is found.
- For example, usage of the function is as
- follows:
+ It will raise an error, if given LSN is in future (i.e. the LSN server
+ doesn't know about). For example, usage of the function is as follows:
<screen>
postgres=# SELECT * FROM pg_get_wal_record_info('0/1E826E98');
-[ RECORD 1 ]----+----------------------------------------------------
@@ -85,7 +85,7 @@ block_ref | blkref #0: rel 1663/5/60221 fork main blk 2
<varlistentry id="pgwalinspect-funcs-pg-get-wal-records-info">
<term>
<function>
- pg_get_wal_records_info(start_lsn pg_lsn, end_lsn pg_lsn)
+ pg_get_wal_records_info(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL)
returns setof record
</function>
</term>
@@ -94,9 +94,13 @@ block_ref | blkref #0: rel 1663/5/60221 fork main blk 2
<para>
Gets information of all the valid WAL records between
<replaceable>start_lsn</replaceable> and <replaceable>end_lsn</replaceable>.
- Returns one row per WAL record. If <replaceable>start_lsn</replaceable>
- or <replaceable>end_lsn</replaceable> are not yet available, the
- function will raise an error. For example:
+ Returns one row per WAL record. If a future <replaceable>end_lsn</replaceable>
+ (i.e. the LSN server doesn't know about) is specified, it returns
+ informaton till end of WAL. It will raise an error, if the server
+ doesn't have WAL available at given <replaceable>start_lsn</replaceable>
+ or if the <replaceable>start_lsn</replaceable> is in future or is past
+ the <replaceable>end_lsn</replaceable>. For example, usage of the
+ function is as follows:
<screen>
postgres=# SELECT * FROM pg_get_wal_records_info('0/1E913618', '0/1E913740') LIMIT 1;
-[ RECORD 1 ]----+--------------------------------------------------------------
@@ -116,27 +120,10 @@ block_ref |
</listitem>
</varlistentry>
- <varlistentry id="pgwalinspect-funcs-pg-get-wal-records-info-till-end-of-wal">
- <term>
- <function>
- pg_get_wal_records_info_till_end_of_wal(start_lsn pg_lsn)
- returns setof record
- </function>
- </term>
-
- <listitem>
- <para>
- This function is the same as <function>pg_get_wal_records_info()</function>,
- except that it gets information of all the valid WAL records from
- <replaceable>start_lsn</replaceable> till the end of WAL.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry id="pgwalinspect-funcs-pg-get-wal-stats">
<term>
<function>
- pg_get_wal_stats(start_lsn pg_lsn, end_lsn pg_lsn, per_record boolean DEFAULT false)
+ pg_get_wal_stats(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL, per_record boolean DEFAULT false)
returns setof record
</function>
</term>
@@ -149,9 +136,13 @@ block_ref |
<replaceable>resource_manager</replaceable> type. When
<replaceable>per_record</replaceable> is set to <literal>true</literal>,
it returns one row per <replaceable>record_type</replaceable>.
- If <replaceable>start_lsn</replaceable>
- or <replaceable>end_lsn</replaceable> are not yet available, the
- function will raise an error. For example:
+ If a future <replaceable>end_lsn</replaceable> (i.e. the LSN server
+ doesn't know about) is specified, it returns stats till end of WAL. It
+ will raise an error, if the server doesn't have WAL available at given
+ <replaceable>start_lsn</replaceable> or if the
+ <replaceable>start_lsn</replaceable> is in future or is past the
+ <replaceable>end_lsn</replaceable>. For example, usage of the function is
+ as follows:
<screen>
postgres=# SELECT * FROM pg_get_wal_stats('0/1E847D00', '0/1E84F500')
WHERE count > 0 LIMIT 1 AND
@@ -171,23 +162,6 @@ combined_size_percentage | 2.8634072910530795
</listitem>
</varlistentry>
- <varlistentry id="pgwalinspect-funcs-pg-get-wal-stats-till-end-of-wal">
- <term>
- <function>
- pg_get_wal_stats_till_end_of_wal(start_lsn pg_lsn, per_record boolean DEFAULT false)
- returns setof record
- </function>
- </term>
-
- <listitem>
- <para>
- This function is the same as <function>pg_get_wal_stats()</function>,
- except that it gets statistics of all the valid WAL records from
- <replaceable>start_lsn</replaceable> till end of WAL.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term>
<function>pg_get_wal_block_info(start_lsn pg_lsn, end_lsn pg_lsn) returns setof record</function>
@@ -202,9 +176,13 @@ combined_size_percentage | 2.8634072910530795
and their information associated with all the valid WAL records between
<replaceable>start_lsn</replaceable> and
<replaceable>end_lsn</replaceable>. Returns one row per block registered
- in a WAL record. If <replaceable>start_lsn</replaceable> or
- <replaceable>end_lsn</replaceable> are not yet available, the function
- will raise an error. For example:
+ in a WAL record. If a future <replaceable>end_lsn</replaceable>
+ (i.e. the LSN server doesn't know about) is specified, it returns
+ information till end of WAL. It will raise an error, if the server
+ doesn't have WAL available at given <replaceable>start_lsn</replaceable>
+ or if the <replaceable>start_lsn</replaceable> is in future or is past
+ the <replaceable>end_lsn</replaceable>. For example, usage of the
+ function is as follows:
<screen>
postgres=# SELECT lsn, blockid, reltablespace, reldatabase, relfilenode,
relblocknumber, forkname,
@@ -227,8 +205,22 @@ fpiinfo | {HAS_HOLE,APPLY}
</para>
</listitem>
</varlistentry>
-
</variablelist>
+
+ <note>
+ <para>
+ Note that <function>pg_get_wal_records_info_till_end_of_wal</function> and
+ <function>pg_get_wal_stats_till_end_of_wal</function> functions have been
+ removed in the <filename>pg_walinspect</filename> version
+ <literal>1.1</literal>. The same functionality can be achieved with
+ <function>pg_get_wal_records_info</function> and
+ <function>pg_get_wal_stats</function> functions by specifying a future
+ <replaceable>end_lsn</replaceable>. However, <function>till_end_of_wal</function>
+ functions will still work if the extension is installed explicitly with
+ version <literal>1.0</literal>.
+ </para>
+ </note>
+
</sect2>
<sect2 id="pgwalinspect-author">
--
2.34.1
Re: Combine pg_walinspect till_end_of_wal functions with others
On Fri, Mar 10, 2023 at 04:45:06PM +0530, Bharath Rupireddy wrote:
Any comments on the attached v5 patch?
I have reviewed the patch, and found it pretty messy. The tests
should have been divided into their own patch, I think. This is
rather straight-forward once the six functions have their checks
grouped together. The result was pretty good, so I have begun by
applying that as 1f282c2. This also includes most of the refinements
you have proposed for the whitespaces in the tests. Note that we were
missing a few spots with the bound checks for the six functions, so
now the coverage should be full.
After that comes the rest of the patch, and I have found a couple of
mistakes.
- pg_get_wal_records_info(start_lsn pg_lsn, end_lsn pg_lsn)
+ pg_get_wal_records_info(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL)
returns setof record
[...]
- pg_get_wal_stats(start_lsn pg_lsn, end_lsn pg_lsn, per_record boolean DEFAULT false)
+ pg_get_wal_stats(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL, per_record boolean DEFAULT false)This part of the documentation is now incorrect.
+-- Make sure checkpoints don't interfere with the test.
+SELECT 'init' FROM pg_create_physical_replication_slot('regress_pg_walinspect_slot', true, false);Adding a physical slot is better for stability of course, but the test
also has to drop it or installcheck would cause an existing cluster to
have that still around. The third argument could be true, as well, so
as we'd use a temporary slot.
- If <replaceable>start_lsn</replaceable>
- or <replaceable>end_lsn</replaceable> are not yet available, the
- function will raise an error. For example:
+ If a future <replaceable>end_lsn</replaceable> (i.e. the LSN server
+ doesn't know about) is specified, it returns stats till end of WAL. It
+ will raise an error, if the server doesn't have WAL available at given
+ <replaceable>start_lsn</replaceable> or if the
+ <replaceable>start_lsn</replaceable> is in future or is past the
+ <replaceable>end_lsn</replaceable>. For example, usage of the function is
+ as follows:Hmm. I would simplify that, and just mention that an error is raised
when the start LSN is available, without caring about the rest (valid
end LSN being past the current insert LSN, and error if start > end,
the second being obvious).
+ <note>
+ <para>
+ Note that <function>pg_get_wal_records_info_till_end_of_wal</function> and
+ <function>pg_get_wal_stats_till_end_of_wal</function> functions have been
+ removed in the <filename>pg_walinspect</filename> version
+ <literal>1.1</literal>. The same functionality can be achieved with
+ <function>pg_get_wal_records_info</function> and
+ <function>pg_get_wal_stats</function> functions by specifying a future
+ <replaceable>end_lsn</replaceable>. However, <function>till_end_of_wal</function>
+ functions will still work if the extension is installed explicitly with
+ version <literal>1.0</literal>.
+ </para>
+ </note>Not convinced that this is necessary.
+ GetInputLSNs(fcinfo, &start_lsn, &end_lsn, till_end_of_wal);
+
+ stats_per_record = PG_GETARG_BOOL(2);This code in GetWalStats() is incorrect.
pg_get_wal_stats_till_end_of_wal() has a stats_per_record, but as
*second* argument, so this would be broken.
+ GetInputLSNs(fcinfo, &start_lsn, &end_lsn, till_end_of_wal);
Coming from the last point, I think that this interface is confusing,
and actually incorrect. From what I can see, we should be doing what
~15 has by grepping the argument values within the main function
calls, and just pass them down to the internal routines GetWalStats()
and GetWALRecordsInfo().
-static bool
-IsFutureLSN(XLogRecPtr lsn, XLogRecPtr *curr_lsn)
+static XLogRecPtr
+GetCurrentLSN(void)This wrapper is actually a good idea.
At the end, I am finishing with the attached. ValidateInputLSNs()
ought to be called, IMO, when the caller of the SQL functions can
directly specify an end_lsn. This means that there is no point to do
this check in the two till_end_* functions. This has as cost two
extra checks to make sure that the start_lsn is not higher than the
current LSN, but I am fine to live with that. It seemed rather
natural to me to let ValidateInputLSNs() do a refresh of the end_lsn
if it sees that it is higher than the current LSN. And if you look
closely, you will see that we only call *once* GetCurrentLSN() for
each function call, so the maths are more precise.
I have cleaned up the comments of the modules, while on it, as there
was not much value in copy-pasting how a function fails while there is
a centralized validation code path. The tests for the till_end()
functions have been moved to the test path where we install 1.0.
With all these cleanups done, there is less code than at the
beginning, which comes from the docs, so the current code does not
change in size:
7 files changed, 173 insertions(+), 206 deletions(-)
--
Michael
Attachments:
v6-0001-Rework-pg_walinspect-functions.patchtext/x-diff; charset=us-asciiDownload
From 74b873e5d61173719ec1961adbc30d711b63f79b Mon Sep 17 00:00:00 2001
From: Michael Paquier <michael@paquier.xyz>
Date: Mon, 13 Mar 2023 15:51:51 +0900
Subject: [PATCH v6] Rework pg_walinspect functions
---
doc/src/sgml/pgwalinspect.sgml | 53 +----
.../pg_walinspect/expected/oldextversions.out | 45 +++-
.../pg_walinspect/expected/pg_walinspect.out | 42 ++--
.../pg_walinspect/pg_walinspect--1.0--1.1.sql | 4 +
contrib/pg_walinspect/pg_walinspect.c | 211 +++++++-----------
contrib/pg_walinspect/sql/oldextversions.sql | 18 ++
contrib/pg_walinspect/sql/pg_walinspect.sql | 6 +-
7 files changed, 173 insertions(+), 206 deletions(-)
diff --git a/doc/src/sgml/pgwalinspect.sgml b/doc/src/sgml/pgwalinspect.sgml
index 3b19863dce..4f7b8f8f19 100644
--- a/doc/src/sgml/pgwalinspect.sgml
+++ b/doc/src/sgml/pgwalinspect.sgml
@@ -94,9 +94,9 @@ block_ref | blkref #0: rel 1663/5/60221 fork main blk 2
<para>
Gets information of all the valid WAL records between
<replaceable>start_lsn</replaceable> and <replaceable>end_lsn</replaceable>.
- Returns one row per WAL record. If <replaceable>start_lsn</replaceable>
- or <replaceable>end_lsn</replaceable> are not yet available, the
- function will raise an error. For example:
+ Returns one row per WAL record. The function raises an error if
+ <replaceable>start_lsn</replaceable> is not available. For example, usage
+ of the function is as follows:
<screen>
postgres=# SELECT * FROM pg_get_wal_records_info('0/1E913618', '0/1E913740') LIMIT 1;
-[ RECORD 1 ]----+--------------------------------------------------------------
@@ -116,23 +116,6 @@ block_ref |
</listitem>
</varlistentry>
- <varlistentry id="pgwalinspect-funcs-pg-get-wal-records-info-till-end-of-wal">
- <term>
- <function>
- pg_get_wal_records_info_till_end_of_wal(start_lsn pg_lsn)
- returns setof record
- </function>
- </term>
-
- <listitem>
- <para>
- This function is the same as <function>pg_get_wal_records_info()</function>,
- except that it gets information of all the valid WAL records from
- <replaceable>start_lsn</replaceable> till the end of WAL.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry id="pgwalinspect-funcs-pg-get-wal-stats">
<term>
<function>
@@ -148,10 +131,9 @@ block_ref |
<replaceable>end_lsn</replaceable>. By default, it returns one row per
<replaceable>resource_manager</replaceable> type. When
<replaceable>per_record</replaceable> is set to <literal>true</literal>,
- it returns one row per <replaceable>record_type</replaceable>.
- If <replaceable>start_lsn</replaceable>
- or <replaceable>end_lsn</replaceable> are not yet available, the
- function will raise an error. For example:
+ it returns one row per <replaceable>record_type</replaceable>. An error
+ is raised if <replaceable>start_lsn</replaceable> is not available. For
+ example, usage of the function is as follows:
<screen>
postgres=# SELECT * FROM pg_get_wal_stats('0/1E847D00', '0/1E84F500')
WHERE count > 0 LIMIT 1 AND
@@ -171,23 +153,6 @@ combined_size_percentage | 2.8634072910530795
</listitem>
</varlistentry>
- <varlistentry id="pgwalinspect-funcs-pg-get-wal-stats-till-end-of-wal">
- <term>
- <function>
- pg_get_wal_stats_till_end_of_wal(start_lsn pg_lsn, per_record boolean DEFAULT false)
- returns setof record
- </function>
- </term>
-
- <listitem>
- <para>
- This function is the same as <function>pg_get_wal_stats()</function>,
- except that it gets statistics of all the valid WAL records from
- <replaceable>start_lsn</replaceable> till end of WAL.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term>
<function>pg_get_wal_block_info(start_lsn pg_lsn, end_lsn pg_lsn) returns setof record</function>
@@ -202,9 +167,9 @@ combined_size_percentage | 2.8634072910530795
and their information associated with all the valid WAL records between
<replaceable>start_lsn</replaceable> and
<replaceable>end_lsn</replaceable>. Returns one row per block registered
- in a WAL record. If <replaceable>start_lsn</replaceable> or
- <replaceable>end_lsn</replaceable> are not yet available, the function
- will raise an error. For example:
+ in a WAL record. An error is raised if
+ <replaceable>start_lsn</replaceable> is not available. For example, usage
+ of the function is as follows:
<screen>
postgres=# SELECT lsn, blockid, reltablespace, reldatabase, relfilenode,
relblocknumber, forkname,
diff --git a/contrib/pg_walinspect/expected/oldextversions.out b/contrib/pg_walinspect/expected/oldextversions.out
index 0db3538693..c35626cd3c 100644
--- a/contrib/pg_walinspect/expected/oldextversions.out
+++ b/contrib/pg_walinspect/expected/oldextversions.out
@@ -1,5 +1,7 @@
-- test old extension version entry points
CREATE EXTENSION pg_walinspect WITH VERSION '1.0';
+-- Mask DETAIL messages as these could refer to current LSN positions.
+\set VERBOSITY terse
-- List what version 1.0 contains
\dx+ pg_walinspect
Objects in extension "pg_walinspect"
@@ -12,19 +14,50 @@ CREATE EXTENSION pg_walinspect WITH VERSION '1.0';
function pg_get_wal_stats_till_end_of_wal(pg_lsn,boolean)
(5 rows)
+-- Make sure checkpoints don't interfere with the test.
+SELECT 'init' FROM pg_create_physical_replication_slot('regress_pg_walinspect_slot', true, false);
+ ?column?
+----------
+ init
+(1 row)
+
+CREATE TABLE sample_tbl(col1 int, col2 int);
+SELECT pg_current_wal_lsn() AS wal_lsn1 \gset
+INSERT INTO sample_tbl SELECT * FROM generate_series(1, 2);
+-- Check bounds for these past functions.
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info_till_end_of_wal(:'wal_lsn1');
+ ok
+----
+ t
+(1 row)
+
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats_till_end_of_wal(:'wal_lsn1');
+ ok
+----
+ t
+(1 row)
+
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info_till_end_of_wal('FFFFFFFF/FFFFFFFF');
+ERROR: WAL start LSN must be less than current LSN
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats_till_end_of_wal('FFFFFFFF/FFFFFFFF');
+ERROR: WAL start LSN must be less than current LSN
-- Move to new version 1.1
ALTER EXTENSION pg_walinspect UPDATE TO '1.1';
-- List what version 1.1 contains
\dx+ pg_walinspect
- Objects in extension "pg_walinspect"
- Object description
------------------------------------------------------------
+ Objects in extension "pg_walinspect"
+ Object description
+--------------------------------------------------
function pg_get_wal_block_info(pg_lsn,pg_lsn)
function pg_get_wal_record_info(pg_lsn)
function pg_get_wal_records_info(pg_lsn,pg_lsn)
- function pg_get_wal_records_info_till_end_of_wal(pg_lsn)
function pg_get_wal_stats(pg_lsn,pg_lsn,boolean)
- function pg_get_wal_stats_till_end_of_wal(pg_lsn,boolean)
-(6 rows)
+(4 rows)
+
+SELECT pg_drop_replication_slot('regress_pg_walinspect_slot');
+ pg_drop_replication_slot
+--------------------------
+
+(1 row)
DROP EXTENSION pg_walinspect;
diff --git a/contrib/pg_walinspect/expected/pg_walinspect.out b/contrib/pg_walinspect/expected/pg_walinspect.out
index c9fb781055..1ab6f6556d 100644
--- a/contrib/pg_walinspect/expected/pg_walinspect.out
+++ b/contrib/pg_walinspect/expected/pg_walinspect.out
@@ -37,24 +37,32 @@ ERROR: WAL start LSN must be less than end LSN
-- LSNs with the highest value possible.
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_record_info('FFFFFFFF/FFFFFFFF');
ERROR: cannot accept future input LSN
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info_till_end_of_wal('FFFFFFFF/FFFFFFFF');
-ERROR: cannot accept future start LSN
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats_till_end_of_wal('FFFFFFFF/FFFFFFFF');
-ERROR: cannot accept future start LSN
--- failures with end LSNs
+-- Success with end LSNs.
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
-ERROR: cannot accept future end LSN
+ ok
+----
+ t
+(1 row)
+
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
-ERROR: cannot accept future end LSN
+ ok
+----
+ t
+(1 row)
+
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_block_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
-ERROR: cannot accept future end LSN
+ ok
+----
+ t
+(1 row)
+
-- failures with start LSNs
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
-ERROR: cannot accept future start LSN
+ERROR: WAL start LSN must be less than current LSN
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
-ERROR: cannot accept future start LSN
+ERROR: WAL start LSN must be less than current LSN
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_block_info('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
-ERROR: cannot accept future start LSN
+ERROR: WAL start LSN must be less than current LSN
-- ===================================================================
-- Tests for all function executions
-- ===================================================================
@@ -64,18 +72,6 @@ SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info_till_end_of_wal(:'wal_lsn1');
- ok
-----
- t
-(1 row)
-
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats_till_end_of_wal(:'wal_lsn1');
- ok
-----
- t
-(1 row)
-
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
ok
----
diff --git a/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql b/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql
index e674ef25aa..586c3b4467 100644
--- a/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql
+++ b/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql
@@ -3,6 +3,10 @@
-- complain if script is sourced in psql, rather than via ALTER EXTENSION
\echo Use "ALTER EXTENSION pg_walinspect UPDATE TO '1.1'" to load this file. \quit
+-- Unsupported functions after 1.1.
+DROP FUNCTION pg_get_wal_records_info_till_end_of_wal(pg_lsn);
+DROP FUNCTION pg_get_wal_stats_till_end_of_wal(pg_lsn, boolean);
+
--
-- pg_get_wal_block_info()
--
diff --git a/contrib/pg_walinspect/pg_walinspect.c b/contrib/pg_walinspect/pg_walinspect.c
index ee88dc4992..7ab28458c4 100644
--- a/contrib/pg_walinspect/pg_walinspect.c
+++ b/contrib/pg_walinspect/pg_walinspect.c
@@ -38,14 +38,14 @@ PG_FUNCTION_INFO_V1(pg_get_wal_records_info_till_end_of_wal);
PG_FUNCTION_INFO_V1(pg_get_wal_stats);
PG_FUNCTION_INFO_V1(pg_get_wal_stats_till_end_of_wal);
-static bool IsFutureLSN(XLogRecPtr lsn, XLogRecPtr *curr_lsn);
+static void ValidateInputLSNs(XLogRecPtr start_lsn, XLogRecPtr *end_lsn);
+static XLogRecPtr GetCurrentLSN(void);
static XLogReaderState *InitXLogReaderState(XLogRecPtr lsn);
static XLogRecord *ReadNextXLogRecord(XLogReaderState *xlogreader);
static void GetWALRecordInfo(XLogReaderState *record, Datum *values,
bool *nulls, uint32 ncols);
-static XLogRecPtr ValidateInputLSNs(bool till_end_of_wal,
- XLogRecPtr start_lsn, XLogRecPtr end_lsn);
-static void GetWALRecordsInfo(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
+static void GetWALRecordsInfo(FunctionCallInfo fcinfo,
+ XLogRecPtr start_lsn,
XLogRecPtr end_lsn);
static void GetXLogSummaryStats(XLogStats *stats, ReturnSetInfo *rsinfo,
Datum *values, bool *nulls, uint32 ncols,
@@ -55,32 +55,32 @@ static void FillXLogStatsRow(const char *name, uint64 n, uint64 total_count,
uint64 fpi_len, uint64 total_fpi_len,
uint64 tot_len, uint64 total_len,
Datum *values, bool *nulls, uint32 ncols);
-static void GetWalStats(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
- XLogRecPtr end_lsn, bool stats_per_record);
+static void GetWalStats(FunctionCallInfo fcinfo,
+ XLogRecPtr start_lsn,
+ XLogRecPtr end_lsn,
+ bool stats_per_record);
static void GetWALBlockInfo(FunctionCallInfo fcinfo, XLogReaderState *record);
/*
- * Check if the given LSN is in future. Also, return the LSN up to which the
- * server has WAL.
+ * Return the LSN up to which the server has WAL.
*/
-static bool
-IsFutureLSN(XLogRecPtr lsn, XLogRecPtr *curr_lsn)
+static XLogRecPtr
+GetCurrentLSN(void)
{
+ XLogRecPtr curr_lsn;
+
/*
* We determine the current LSN of the server similar to how page_read
* callback read_local_xlog_page_no_wait does.
*/
if (!RecoveryInProgress())
- *curr_lsn = GetFlushRecPtr(NULL);
+ curr_lsn = GetFlushRecPtr(NULL);
else
- *curr_lsn = GetXLogReplayRecPtr(NULL);
+ curr_lsn = GetXLogReplayRecPtr(NULL);
- Assert(!XLogRecPtrIsInvalid(*curr_lsn));
+ Assert(!XLogRecPtrIsInvalid(curr_lsn));
- if (lsn >= *curr_lsn)
- return true;
-
- return false;
+ return curr_lsn;
}
/*
@@ -354,23 +354,17 @@ GetWALBlockInfo(FunctionCallInfo fcinfo, XLogReaderState *record)
* their relation information, and the data saved in each block associated
* to a record. Decompression is applied to the full page images, if
* necessary.
- *
- * This function emits an error if a future start or end WAL LSN i.e. WAL LSN
- * the database system doesn't know about is specified.
*/
Datum
pg_get_wal_block_info(PG_FUNCTION_ARGS)
{
- XLogRecPtr start_lsn;
- XLogRecPtr end_lsn;
+ XLogRecPtr start_lsn = PG_GETARG_LSN(0);
+ XLogRecPtr end_lsn = PG_GETARG_LSN(1);
XLogReaderState *xlogreader;
MemoryContext old_cxt;
MemoryContext tmp_cxt;
- start_lsn = PG_GETARG_LSN(0);
- end_lsn = PG_GETARG_LSN(1);
-
- end_lsn = ValidateInputLSNs(false, start_lsn, end_lsn);
+ ValidateInputLSNs(start_lsn, &end_lsn);
InitMaterializedSRF(fcinfo, 0);
@@ -404,9 +398,6 @@ pg_get_wal_block_info(PG_FUNCTION_ARGS)
/*
* Get WAL record info.
- *
- * This function emits an error if a future WAL LSN i.e. WAL LSN the database
- * system doesn't know about is specified.
*/
Datum
pg_get_wal_record_info(PG_FUNCTION_ARGS)
@@ -422,20 +413,14 @@ pg_get_wal_record_info(PG_FUNCTION_ARGS)
HeapTuple tuple;
lsn = PG_GETARG_LSN(0);
+ curr_lsn = GetCurrentLSN();
- if (IsFutureLSN(lsn, &curr_lsn))
- {
- /*
- * GetFlushRecPtr or GetXLogReplayRecPtr gives "end+1" LSN of the last
- * record flushed or replayed respectively. But let's use the LSN up
- * to "end" in user facing message.
- */
+ if (lsn >= curr_lsn)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("cannot accept future input LSN"),
errdetail("Last known WAL LSN on the database system is at %X/%X.",
LSN_FORMAT_ARGS(curr_lsn))));
- }
/* Build a tuple descriptor for our result type. */
if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
@@ -462,44 +447,28 @@ pg_get_wal_record_info(PG_FUNCTION_ARGS)
}
/*
- * Validate the input LSNs and compute end LSN for till_end_of_wal versions.
+ * Validate start and end LSNs coming from the function inputs.
+ *
+ * If end_lsn is found to be higher than the current LSN reported by the
+ * cluster, use the current LSN as the upper bound.
*/
-static XLogRecPtr
-ValidateInputLSNs(bool till_end_of_wal, XLogRecPtr start_lsn,
- XLogRecPtr end_lsn)
+static void
+ValidateInputLSNs(XLogRecPtr start_lsn, XLogRecPtr *end_lsn)
{
- XLogRecPtr curr_lsn;
+ XLogRecPtr curr_lsn = GetCurrentLSN();
- if (IsFutureLSN(start_lsn, &curr_lsn))
- {
- /*
- * GetFlushRecPtr or GetXLogReplayRecPtr gives "end+1" LSN of the last
- * record flushed or replayed respectively. But let's use the LSN up
- * to "end" in user facing message.
- */
+ if (start_lsn > curr_lsn)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("cannot accept future start LSN"),
- errdetail("Last known WAL LSN on the database system is at %X/%X.",
- LSN_FORMAT_ARGS(curr_lsn))));
- }
+ errmsg("WAL start LSN must be less than current LSN")));
- if (till_end_of_wal)
- end_lsn = curr_lsn;
-
- if (end_lsn > curr_lsn)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("cannot accept future end LSN"),
- errdetail("Last known WAL LSN on the database system is at %X/%X.",
- LSN_FORMAT_ARGS(curr_lsn))));
-
- if (start_lsn >= end_lsn)
+ if (start_lsn > *end_lsn)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("WAL start LSN must be less than end LSN")));
- return end_lsn;
+ if (*end_lsn > curr_lsn)
+ *end_lsn = curr_lsn;
}
/*
@@ -517,6 +486,8 @@ GetWALRecordsInfo(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
MemoryContext old_cxt;
MemoryContext tmp_cxt;
+ Assert(start_lsn <= end_lsn);
+
InitMaterializedSRF(fcinfo, 0);
xlogreader = InitXLogReaderState(start_lsn);
@@ -552,43 +523,17 @@ GetWALRecordsInfo(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
}
/*
- * Get info and data of all WAL records between start LSN and end LSN.
+ * pg_get_wal_records_info
*
- * This function emits an error if a future start or end WAL LSN i.e. WAL LSN
- * the database system doesn't know about is specified.
+ * Get info and data of all WAL records between start LSN and end LSN.
*/
Datum
pg_get_wal_records_info(PG_FUNCTION_ARGS)
{
- XLogRecPtr start_lsn;
- XLogRecPtr end_lsn;
-
- start_lsn = PG_GETARG_LSN(0);
- end_lsn = PG_GETARG_LSN(1);
-
- end_lsn = ValidateInputLSNs(false, start_lsn, end_lsn);
-
- GetWALRecordsInfo(fcinfo, start_lsn, end_lsn);
-
- PG_RETURN_VOID();
-}
-
-/*
- * Get info and data of all WAL records from start LSN till end of WAL.
- *
- * This function emits an error if a future start i.e. WAL LSN the database
- * system doesn't know about is specified.
- */
-Datum
-pg_get_wal_records_info_till_end_of_wal(PG_FUNCTION_ARGS)
-{
- XLogRecPtr start_lsn;
- XLogRecPtr end_lsn = InvalidXLogRecPtr;
-
- start_lsn = PG_GETARG_LSN(0);
-
- end_lsn = ValidateInputLSNs(true, start_lsn, end_lsn);
+ XLogRecPtr start_lsn = PG_GETARG_LSN(0);
+ XLogRecPtr end_lsn = PG_GETARG_LSN(1);
+ ValidateInputLSNs(start_lsn, &end_lsn);
GetWALRecordsInfo(fcinfo, start_lsn, end_lsn);
PG_RETURN_VOID();
@@ -648,13 +593,13 @@ GetXLogSummaryStats(XLogStats *stats, ReturnSetInfo *rsinfo,
Datum *values, bool *nulls, uint32 ncols,
bool stats_per_record)
{
- MemoryContext old_cxt;
- MemoryContext tmp_cxt;
- uint64 total_count = 0;
- uint64 total_rec_len = 0;
- uint64 total_fpi_len = 0;
- uint64 total_len = 0;
- int ri;
+ MemoryContext old_cxt;
+ MemoryContext tmp_cxt;
+ uint64 total_count = 0;
+ uint64 total_rec_len = 0;
+ uint64 total_fpi_len = 0;
+ uint64 total_len = 0;
+ int ri;
/*
* Each row shows its percentages of the total, so make a first pass to
@@ -757,8 +702,8 @@ GetXLogSummaryStats(XLogStats *stats, ReturnSetInfo *rsinfo,
* Get WAL stats between start LSN and end LSN.
*/
static void
-GetWalStats(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
- XLogRecPtr end_lsn, bool stats_per_record)
+GetWalStats(FunctionCallInfo fcinfo, XLogRecPtr start_lsn, XLogRecPtr end_lsn,
+ bool stats_per_record)
{
#define PG_GET_WAL_STATS_COLS 9
XLogReaderState *xlogreader;
@@ -767,6 +712,8 @@ GetWalStats(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
Datum values[PG_GET_WAL_STATS_COLS] = {0};
bool nulls[PG_GET_WAL_STATS_COLS] = {0};
+ Assert(start_lsn <= end_lsn);
+
InitMaterializedSRF(fcinfo, 0);
xlogreader = InitXLogReaderState(start_lsn);
@@ -790,46 +737,54 @@ GetWalStats(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
}
/*
- * Get stats of all WAL records between start LSN and end LSN.
+ * pg_get_wal_stats
*
- * This function emits an error if a future start or end WAL LSN i.e. WAL LSN
- * the database system doesn't know about is specified.
+ * Get stats of all WAL records between start LSN and end LSN.
*/
Datum
pg_get_wal_stats(PG_FUNCTION_ARGS)
{
- XLogRecPtr start_lsn;
- XLogRecPtr end_lsn;
- bool stats_per_record;
-
- start_lsn = PG_GETARG_LSN(0);
- end_lsn = PG_GETARG_LSN(1);
- stats_per_record = PG_GETARG_BOOL(2);
-
- end_lsn = ValidateInputLSNs(false, start_lsn, end_lsn);
+ XLogRecPtr start_lsn = PG_GETARG_LSN(0);
+ XLogRecPtr end_lsn = PG_GETARG_LSN(1);
+ bool stats_per_record = PG_GETARG_BOOL(2);
+ ValidateInputLSNs(start_lsn, &end_lsn);
GetWalStats(fcinfo, start_lsn, end_lsn, stats_per_record);
PG_RETURN_VOID();
}
/*
- * Get stats of all WAL records from start LSN till end of WAL.
- *
- * This function emits an error if a future start i.e. WAL LSN the database
- * system doesn't know about is specified.
+ * The following functions have been removed in newer versions in 1.1, but
+ * they are kept around for compatibility.
*/
+Datum
+pg_get_wal_records_info_till_end_of_wal(PG_FUNCTION_ARGS)
+{
+ XLogRecPtr start_lsn = PG_GETARG_LSN(0);
+ XLogRecPtr end_lsn = GetCurrentLSN();
+
+ if (start_lsn > end_lsn)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("WAL start LSN must be less than current LSN")));
+
+ GetWALRecordsInfo(fcinfo, start_lsn, end_lsn);
+
+ PG_RETURN_VOID();
+}
+
Datum
pg_get_wal_stats_till_end_of_wal(PG_FUNCTION_ARGS)
{
- XLogRecPtr start_lsn;
- XLogRecPtr end_lsn = InvalidXLogRecPtr;
- bool stats_per_record;
+ XLogRecPtr start_lsn = PG_GETARG_LSN(0);
+ XLogRecPtr end_lsn = GetCurrentLSN();
+ bool stats_per_record = PG_GETARG_BOOL(1);
- start_lsn = PG_GETARG_LSN(0);
- stats_per_record = PG_GETARG_BOOL(1);
-
- end_lsn = ValidateInputLSNs(true, start_lsn, end_lsn);
+ if (start_lsn > end_lsn)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("WAL start LSN must be less than current LSN")));
GetWalStats(fcinfo, start_lsn, end_lsn, stats_per_record);
diff --git a/contrib/pg_walinspect/sql/oldextversions.sql b/contrib/pg_walinspect/sql/oldextversions.sql
index f448094add..39e926a096 100644
--- a/contrib/pg_walinspect/sql/oldextversions.sql
+++ b/contrib/pg_walinspect/sql/oldextversions.sql
@@ -2,13 +2,31 @@
CREATE EXTENSION pg_walinspect WITH VERSION '1.0';
+-- Mask DETAIL messages as these could refer to current LSN positions.
+\set VERBOSITY terse
+
-- List what version 1.0 contains
\dx+ pg_walinspect
+-- Make sure checkpoints don't interfere with the test.
+SELECT 'init' FROM pg_create_physical_replication_slot('regress_pg_walinspect_slot', true, false);
+
+CREATE TABLE sample_tbl(col1 int, col2 int);
+SELECT pg_current_wal_lsn() AS wal_lsn1 \gset
+INSERT INTO sample_tbl SELECT * FROM generate_series(1, 2);
+
+-- Check bounds for these past functions.
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info_till_end_of_wal(:'wal_lsn1');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats_till_end_of_wal(:'wal_lsn1');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info_till_end_of_wal('FFFFFFFF/FFFFFFFF');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats_till_end_of_wal('FFFFFFFF/FFFFFFFF');
+
-- Move to new version 1.1
ALTER EXTENSION pg_walinspect UPDATE TO '1.1';
-- List what version 1.1 contains
\dx+ pg_walinspect
+SELECT pg_drop_replication_slot('regress_pg_walinspect_slot');
+
DROP EXTENSION pg_walinspect;
diff --git a/contrib/pg_walinspect/sql/pg_walinspect.sql b/contrib/pg_walinspect/sql/pg_walinspect.sql
index 766ed6b87d..ba4d17df6c 100644
--- a/contrib/pg_walinspect/sql/pg_walinspect.sql
+++ b/contrib/pg_walinspect/sql/pg_walinspect.sql
@@ -33,9 +33,7 @@ SELECT * FROM pg_get_wal_block_info(:'wal_lsn2', :'wal_lsn1');
-- LSNs with the highest value possible.
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_record_info('FFFFFFFF/FFFFFFFF');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info_till_end_of_wal('FFFFFFFF/FFFFFFFF');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats_till_end_of_wal('FFFFFFFF/FFFFFFFF');
--- failures with end LSNs
+-- Success with end LSNs.
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_block_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
@@ -49,8 +47,6 @@ SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_block_info('FFFFFFFF/FFFFFFFE', 'FFFF
-- ===================================================================
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info_till_end_of_wal(:'wal_lsn1');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats_till_end_of_wal(:'wal_lsn1');
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_block_info(:'wal_lsn1', :'wal_lsn2');
--
2.39.2
Re: Combine pg_walinspect till_end_of_wal functions with others
On Mon, Mar 13, 2023 at 12:26 PM Michael Paquier <michael@paquier.xyz> wrote:
On Fri, Mar 10, 2023 at 04:45:06PM +0530, Bharath Rupireddy wrote:
After that comes the rest of the patch, and I have found a couple of
mistakes.- pg_get_wal_records_info(start_lsn pg_lsn, end_lsn pg_lsn) + pg_get_wal_records_info(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL) returns setof record [...] - pg_get_wal_stats(start_lsn pg_lsn, end_lsn pg_lsn, per_record boolean DEFAULT false) + pg_get_wal_stats(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL, per_record boolean DEFAULT false)This part of the documentation is now incorrect.
Oh, yeah. Thanks for fixing it.
+-- Make sure checkpoints don't interfere with the test. +SELECT 'init' FROM pg_create_physical_replication_slot('regress_pg_walinspect_slot', true, false);Adding a physical slot is better for stability of course, but the test
also has to drop it or installcheck would cause an existing cluster to
have that still around. The third argument could be true, as well, so
as we'd use a temporary slot.
# Disabled because these tests require "wal_level=replica", which
# some installcheck users do not have (e.g. buildfarm clients).
NO_INSTALLCHECK = 1
pg_walinspect can't be run under installcheck. I don't think dropping
the slot at the end is needed, it's unnecessary. I saw
oldextversions.sql using the same replication slot name, well no
problem, but I changed it to a unique name.
Hmm. I would simplify that, and just mention that an error is raised
when the start LSN is available, without caring about the rest (valid
end LSN being past the current insert LSN, and error if start > end,
the second being obvious).
Okay.
+ <note> + <para> + Note that <function>pg_get_wal_records_info_till_end_of_wal</function> and + <function>pg_get_wal_stats_till_end_of_wal</function> functions have been + removed in the <filename>pg_walinspect</filename> version + <literal>1.1</literal>. The same functionality can be achieved with + <function>pg_get_wal_records_info</function> and + <function>pg_get_wal_stats</function> functions by specifying a future + <replaceable>end_lsn</replaceable>. However, <function>till_end_of_wal</function> + functions will still work if the extension is installed explicitly with + version <literal>1.0</literal>. + </para> + </note>Not convinced that this is necessary.
As hackers we know that these functions have been removed and how to
achieve till_end_of_wal with the other functions. I noticed that
you've removed my changes (see below) from the docs that were saying
how to get info/stats till_end_of_wal. That leaves end users confused
as to how they can achieve till_end_of_wal functionality. All users
can't look for commit history/message but they can easily read the
docs. I prefer to have the following (did so in the attached v7) and
get rid of the above note if you don't feel strongly about it.
+ If a future <replaceable>end_lsn</replaceable>
+ (i.e. the LSN server doesn't know about) is specified, it returns
+ informaton till end of WAL.+ GetInputLSNs(fcinfo, &start_lsn, &end_lsn, till_end_of_wal); + + stats_per_record = PG_GETARG_BOOL(2);This code in GetWalStats() is incorrect.
pg_get_wal_stats_till_end_of_wal() has a stats_per_record, but as
*second* argument, so this would be broken.
Oh, yeah. Thanks for fixing it.
+ GetInputLSNs(fcinfo, &start_lsn, &end_lsn, till_end_of_wal);
Coming from the last point, I think that this interface is confusing,
and actually incorrect. From what I can see, we should be doing what
~15 has by grepping the argument values within the main function
calls, and just pass them down to the internal routines GetWalStats()
and GetWALRecordsInfo().
Hm, what you have in v6 works for me.
At the end, I am finishing with the attached. ValidateInputLSNs()
ought to be called, IMO, when the caller of the SQL functions can
directly specify an end_lsn. This means that there is no point to do
this check in the two till_end_* functions. This has as cost two
extra checks to make sure that the start_lsn is not higher than the
current LSN, but I am fine to live with that. It seemed rather
natural to me to let ValidateInputLSNs() do a refresh of the end_lsn
if it sees that it is higher than the current LSN. And if you look
closely, you will see that we only call *once* GetCurrentLSN() for
each function call, so the maths are more precise.I have cleaned up the comments of the modules, while on it, as there
was not much value in copy-pasting how a function fails while there is
a centralized validation code path. The tests for the till_end()
functions have been moved to the test path where we install 1.0.
I have some comments and fixed them in the attached v7 patch:
1.
+ * pg_get_wal_records_info
*
+ * pg_get_wal_stats
*
I think you wanted to be consistent with function comments with
function names atop, but missed adding for all functions. Actually, I
don't have a strong opinion on these changes as they unnecessarily
bloat the changes, so I removed them.
2.
+ if (start_lsn > curr_lsn)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("cannot accept future start LSN"),
- errdetail("Last known WAL LSN on the database system
is at %X/%X.",
- LSN_FORMAT_ARGS(curr_lsn))));
- }
+ errmsg("WAL start LSN must be less than current LSN")));I don't like this inconsistency much, especially when
pg_get_wal_record_info emits "cannot accept future input LSN" with the
current LSN details (this current LSN will give a bit more information
to the user). Also, let's be consistent across returning NULLs if
input LSN/start LSN equal to the current LSN. I've done these changes
in the attached v7 patch.
3. I wanted COUNT(*) >= 0 for successful function execution to be
COUNT(*) >= 1 so that we will check for at least the functions
returning 1 record. And failures to be SELECT * FROM. This was my
intention but I don't see that in this patch or in the previous
test-refactoring commit. I added that in the attached v7 patch again.
Also, made test comments conssitent.
--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
Attachments:
v7-0001-Rework-pg_walinspect-functions.patchapplication/x-patch; name=v7-0001-Rework-pg_walinspect-functions.patchDownload
From 188a199d1c1ed6dd12021030c592751d2cadc2fb Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Mon, 13 Mar 2023 10:06:16 +0000
Subject: [PATCH v7] Rework pg_walinspect functions
---
.../pg_walinspect/expected/oldextversions.out | 41 +++-
.../pg_walinspect/expected/pg_walinspect.out | 66 +++---
.../pg_walinspect/pg_walinspect--1.0--1.1.sql | 4 +
contrib/pg_walinspect/pg_walinspect.c | 215 +++++++-----------
contrib/pg_walinspect/sql/oldextversions.sql | 19 ++
contrib/pg_walinspect/sql/pg_walinspect.sql | 43 ++--
doc/src/sgml/pgwalinspect.sgml | 58 ++---
7 files changed, 205 insertions(+), 241 deletions(-)
diff --git a/contrib/pg_walinspect/expected/oldextversions.out b/contrib/pg_walinspect/expected/oldextversions.out
index 0db3538693..fb453feec5 100644
--- a/contrib/pg_walinspect/expected/oldextversions.out
+++ b/contrib/pg_walinspect/expected/oldextversions.out
@@ -1,5 +1,7 @@
-- test old extension version entry points
CREATE EXTENSION pg_walinspect WITH VERSION '1.0';
+-- Mask DETAIL messages as these could refer to current LSN positions
+\set VERBOSITY terse
-- List what version 1.0 contains
\dx+ pg_walinspect
Objects in extension "pg_walinspect"
@@ -12,19 +14,46 @@ CREATE EXTENSION pg_walinspect WITH VERSION '1.0';
function pg_get_wal_stats_till_end_of_wal(pg_lsn,boolean)
(5 rows)
+-- Make sure checkpoints don't interfere with the test
+SELECT 'init' FROM pg_create_physical_replication_slot('regress_pg_walinspect_slot_old_ext_ver', true, false);
+ ?column?
+----------
+ init
+(1 row)
+
+CREATE TABLE sample_tbl(col1 int, col2 int);
+SELECT pg_current_wal_lsn() AS wal_lsn1 \gset
+INSERT INTO sample_tbl SELECT * FROM generate_series(1, 2);
+-- Tests for the past functions
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info_till_end_of_wal(:'wal_lsn1');
+ ok
+----
+ t
+(1 row)
+
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats_till_end_of_wal(:'wal_lsn1');
+ ok
+----
+ t
+(1 row)
+
+-- Failures with start LSNs
+SELECT * FROM pg_get_wal_records_info_till_end_of_wal('FFFFFFFF/FFFFFFFF');
+ERROR: WAL start LSN must be less than current LSN
+SELECT * FROM pg_get_wal_stats_till_end_of_wal('FFFFFFFF/FFFFFFFF');
+ERROR: WAL start LSN must be less than current LSN
-- Move to new version 1.1
ALTER EXTENSION pg_walinspect UPDATE TO '1.1';
-- List what version 1.1 contains
\dx+ pg_walinspect
- Objects in extension "pg_walinspect"
- Object description
------------------------------------------------------------
+ Objects in extension "pg_walinspect"
+ Object description
+--------------------------------------------------
function pg_get_wal_block_info(pg_lsn,pg_lsn)
function pg_get_wal_record_info(pg_lsn)
function pg_get_wal_records_info(pg_lsn,pg_lsn)
- function pg_get_wal_records_info_till_end_of_wal(pg_lsn)
function pg_get_wal_stats(pg_lsn,pg_lsn,boolean)
- function pg_get_wal_stats_till_end_of_wal(pg_lsn,boolean)
-(6 rows)
+(4 rows)
+-- Clean up
DROP EXTENSION pg_walinspect;
diff --git a/contrib/pg_walinspect/expected/pg_walinspect.out b/contrib/pg_walinspect/expected/pg_walinspect.out
index c9fb781055..2042ac365e 100644
--- a/contrib/pg_walinspect/expected/pg_walinspect.out
+++ b/contrib/pg_walinspect/expected/pg_walinspect.out
@@ -9,7 +9,7 @@ SELECT 'init' FROM pg_create_physical_replication_slot('regress_pg_walinspect_sl
(1 row)
CREATE TABLE sample_tbl(col1 int, col2 int);
--- Save some LSNs for comparisons
+-- Save some LSNs for comparisons.
SELECT pg_current_wal_lsn() AS wal_lsn1 \gset
INSERT INTO sample_tbl SELECT * FROM generate_series(1, 2);
SELECT pg_current_wal_lsn() AS wal_lsn2 \gset
@@ -35,60 +35,56 @@ ERROR: WAL start LSN must be less than end LSN
SELECT * FROM pg_get_wal_block_info(:'wal_lsn2', :'wal_lsn1');
ERROR: WAL start LSN must be less than end LSN
-- LSNs with the highest value possible.
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_record_info('FFFFFFFF/FFFFFFFF');
-ERROR: cannot accept future input LSN
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info_till_end_of_wal('FFFFFFFF/FFFFFFFF');
-ERROR: cannot accept future start LSN
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats_till_end_of_wal('FFFFFFFF/FFFFFFFF');
-ERROR: cannot accept future start LSN
--- failures with end LSNs
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
-ERROR: cannot accept future end LSN
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
-ERROR: cannot accept future end LSN
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_block_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
-ERROR: cannot accept future end LSN
--- failures with start LSNs
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
-ERROR: cannot accept future start LSN
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
-ERROR: cannot accept future start LSN
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_block_info('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
-ERROR: cannot accept future start LSN
--- ===================================================================
--- Tests for all function executions
--- ===================================================================
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
+SELECT * FROM pg_get_wal_record_info('FFFFFFFF/FFFFFFFF');
+ERROR: WAL input LSN must be less than current LSN
+-- Failures with start LSNs.
+SELECT * FROM pg_get_wal_records_info('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
+ERROR: WAL start LSN must be less than current LSN
+SELECT * FROM pg_get_wal_stats('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
+ERROR: WAL start LSN must be less than current LSN
+SELECT * FROM pg_get_wal_block_info('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
+ERROR: WAL start LSN must be less than current LSN
+-- Success with end LSNs.
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info_till_end_of_wal(:'wal_lsn1');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats_till_end_of_wal(:'wal_lsn1');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
+-- ===================================================================
+-- Tests for all function executions
+-- ===================================================================
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_block_info(:'wal_lsn1', :'wal_lsn2');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
+ ok
+----
+ t
+(1 row)
+
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn1', :'wal_lsn2');
ok
----
t
@@ -119,7 +115,7 @@ SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2'
-- ===================================================================
-- Tests to get block information from WAL record
-- ===================================================================
--- Update table to generate some block data
+-- Update table to generate some block data.
SELECT pg_current_wal_lsn() AS wal_lsn3 \gset
UPDATE sample_tbl SET col1 = col1 + 1 WHERE col1 = 1;
SELECT pg_current_wal_lsn() AS wal_lsn4 \gset
@@ -176,7 +172,7 @@ SELECT has_function_privilege('regress_pg_walinspect',
f
(1 row)
--- Functions accessible by users with role pg_read_server_files
+-- Functions accessible by users with role pg_read_server_files.
GRANT pg_read_server_files TO regress_pg_walinspect;
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_record_info(pg_lsn)', 'EXECUTE'); -- yes
@@ -256,11 +252,5 @@ REVOKE EXECUTE ON FUNCTION pg_get_wal_block_info(pg_lsn, pg_lsn)
-- Clean up
-- ===================================================================
DROP ROLE regress_pg_walinspect;
-SELECT pg_drop_replication_slot('regress_pg_walinspect_slot');
- pg_drop_replication_slot
---------------------------
-
-(1 row)
-
DROP TABLE sample_tbl;
DROP EXTENSION pg_walinspect;
diff --git a/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql b/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql
index e674ef25aa..586c3b4467 100644
--- a/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql
+++ b/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql
@@ -3,6 +3,10 @@
-- complain if script is sourced in psql, rather than via ALTER EXTENSION
\echo Use "ALTER EXTENSION pg_walinspect UPDATE TO '1.1'" to load this file. \quit
+-- Unsupported functions after 1.1.
+DROP FUNCTION pg_get_wal_records_info_till_end_of_wal(pg_lsn);
+DROP FUNCTION pg_get_wal_stats_till_end_of_wal(pg_lsn, boolean);
+
--
-- pg_get_wal_block_info()
--
diff --git a/contrib/pg_walinspect/pg_walinspect.c b/contrib/pg_walinspect/pg_walinspect.c
index ee88dc4992..3b3215daf5 100644
--- a/contrib/pg_walinspect/pg_walinspect.c
+++ b/contrib/pg_walinspect/pg_walinspect.c
@@ -38,14 +38,14 @@ PG_FUNCTION_INFO_V1(pg_get_wal_records_info_till_end_of_wal);
PG_FUNCTION_INFO_V1(pg_get_wal_stats);
PG_FUNCTION_INFO_V1(pg_get_wal_stats_till_end_of_wal);
-static bool IsFutureLSN(XLogRecPtr lsn, XLogRecPtr *curr_lsn);
+static void ValidateInputLSNs(XLogRecPtr start_lsn, XLogRecPtr *end_lsn);
+static XLogRecPtr GetCurrentLSN(void);
static XLogReaderState *InitXLogReaderState(XLogRecPtr lsn);
static XLogRecord *ReadNextXLogRecord(XLogReaderState *xlogreader);
static void GetWALRecordInfo(XLogReaderState *record, Datum *values,
bool *nulls, uint32 ncols);
-static XLogRecPtr ValidateInputLSNs(bool till_end_of_wal,
- XLogRecPtr start_lsn, XLogRecPtr end_lsn);
-static void GetWALRecordsInfo(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
+static void GetWALRecordsInfo(FunctionCallInfo fcinfo,
+ XLogRecPtr start_lsn,
XLogRecPtr end_lsn);
static void GetXLogSummaryStats(XLogStats *stats, ReturnSetInfo *rsinfo,
Datum *values, bool *nulls, uint32 ncols,
@@ -55,32 +55,32 @@ static void FillXLogStatsRow(const char *name, uint64 n, uint64 total_count,
uint64 fpi_len, uint64 total_fpi_len,
uint64 tot_len, uint64 total_len,
Datum *values, bool *nulls, uint32 ncols);
-static void GetWalStats(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
- XLogRecPtr end_lsn, bool stats_per_record);
+static void GetWalStats(FunctionCallInfo fcinfo,
+ XLogRecPtr start_lsn,
+ XLogRecPtr end_lsn,
+ bool stats_per_record);
static void GetWALBlockInfo(FunctionCallInfo fcinfo, XLogReaderState *record);
/*
- * Check if the given LSN is in future. Also, return the LSN up to which the
- * server has WAL.
+ * Return the LSN up to which the server has WAL.
*/
-static bool
-IsFutureLSN(XLogRecPtr lsn, XLogRecPtr *curr_lsn)
+static XLogRecPtr
+GetCurrentLSN(void)
{
+ XLogRecPtr curr_lsn;
+
/*
* We determine the current LSN of the server similar to how page_read
* callback read_local_xlog_page_no_wait does.
*/
if (!RecoveryInProgress())
- *curr_lsn = GetFlushRecPtr(NULL);
+ curr_lsn = GetFlushRecPtr(NULL);
else
- *curr_lsn = GetXLogReplayRecPtr(NULL);
-
- Assert(!XLogRecPtrIsInvalid(*curr_lsn));
+ curr_lsn = GetXLogReplayRecPtr(NULL);
- if (lsn >= *curr_lsn)
- return true;
+ Assert(!XLogRecPtrIsInvalid(curr_lsn));
- return false;
+ return curr_lsn;
}
/*
@@ -354,23 +354,17 @@ GetWALBlockInfo(FunctionCallInfo fcinfo, XLogReaderState *record)
* their relation information, and the data saved in each block associated
* to a record. Decompression is applied to the full page images, if
* necessary.
- *
- * This function emits an error if a future start or end WAL LSN i.e. WAL LSN
- * the database system doesn't know about is specified.
*/
Datum
pg_get_wal_block_info(PG_FUNCTION_ARGS)
{
- XLogRecPtr start_lsn;
- XLogRecPtr end_lsn;
+ XLogRecPtr start_lsn = PG_GETARG_LSN(0);
+ XLogRecPtr end_lsn = PG_GETARG_LSN(1);
XLogReaderState *xlogreader;
MemoryContext old_cxt;
MemoryContext tmp_cxt;
- start_lsn = PG_GETARG_LSN(0);
- end_lsn = PG_GETARG_LSN(1);
-
- end_lsn = ValidateInputLSNs(false, start_lsn, end_lsn);
+ ValidateInputLSNs(start_lsn, &end_lsn);
InitMaterializedSRF(fcinfo, 0);
@@ -404,9 +398,6 @@ pg_get_wal_block_info(PG_FUNCTION_ARGS)
/*
* Get WAL record info.
- *
- * This function emits an error if a future WAL LSN i.e. WAL LSN the database
- * system doesn't know about is specified.
*/
Datum
pg_get_wal_record_info(PG_FUNCTION_ARGS)
@@ -422,20 +413,14 @@ pg_get_wal_record_info(PG_FUNCTION_ARGS)
HeapTuple tuple;
lsn = PG_GETARG_LSN(0);
+ curr_lsn = GetCurrentLSN();
- if (IsFutureLSN(lsn, &curr_lsn))
- {
- /*
- * GetFlushRecPtr or GetXLogReplayRecPtr gives "end+1" LSN of the last
- * record flushed or replayed respectively. But let's use the LSN up
- * to "end" in user facing message.
- */
+ if (lsn > curr_lsn)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("cannot accept future input LSN"),
- errdetail("Last known WAL LSN on the database system is at %X/%X.",
+ errmsg("WAL input LSN must be less than current LSN"),
+ errdetail("Current WAL LSN on the database system is at %X/%X.",
LSN_FORMAT_ARGS(curr_lsn))));
- }
/* Build a tuple descriptor for our result type. */
if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
@@ -462,44 +447,30 @@ pg_get_wal_record_info(PG_FUNCTION_ARGS)
}
/*
- * Validate the input LSNs and compute end LSN for till_end_of_wal versions.
+ * Validate start and end LSNs coming from the function inputs.
+ *
+ * If end_lsn is found to be higher than the current LSN reported by the
+ * cluster, use the current LSN as the upper bound.
*/
-static XLogRecPtr
-ValidateInputLSNs(bool till_end_of_wal, XLogRecPtr start_lsn,
- XLogRecPtr end_lsn)
+static void
+ValidateInputLSNs(XLogRecPtr start_lsn, XLogRecPtr *end_lsn)
{
- XLogRecPtr curr_lsn;
+ XLogRecPtr curr_lsn = GetCurrentLSN();
- if (IsFutureLSN(start_lsn, &curr_lsn))
- {
- /*
- * GetFlushRecPtr or GetXLogReplayRecPtr gives "end+1" LSN of the last
- * record flushed or replayed respectively. But let's use the LSN up
- * to "end" in user facing message.
- */
+ if (start_lsn > curr_lsn)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("cannot accept future start LSN"),
- errdetail("Last known WAL LSN on the database system is at %X/%X.",
+ errmsg("WAL start LSN must be less than current LSN"),
+ errdetail("Current WAL LSN on the database system is at %X/%X.",
LSN_FORMAT_ARGS(curr_lsn))));
- }
- if (till_end_of_wal)
- end_lsn = curr_lsn;
-
- if (end_lsn > curr_lsn)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("cannot accept future end LSN"),
- errdetail("Last known WAL LSN on the database system is at %X/%X.",
- LSN_FORMAT_ARGS(curr_lsn))));
-
- if (start_lsn >= end_lsn)
+ if (start_lsn > *end_lsn)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("WAL start LSN must be less than end LSN")));
- return end_lsn;
+ if (*end_lsn > curr_lsn)
+ *end_lsn = curr_lsn;
}
/*
@@ -517,6 +488,8 @@ GetWALRecordsInfo(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
MemoryContext old_cxt;
MemoryContext tmp_cxt;
+ Assert(start_lsn <= end_lsn);
+
InitMaterializedSRF(fcinfo, 0);
xlogreader = InitXLogReaderState(start_lsn);
@@ -553,42 +526,14 @@ GetWALRecordsInfo(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
/*
* Get info and data of all WAL records between start LSN and end LSN.
- *
- * This function emits an error if a future start or end WAL LSN i.e. WAL LSN
- * the database system doesn't know about is specified.
*/
Datum
pg_get_wal_records_info(PG_FUNCTION_ARGS)
{
- XLogRecPtr start_lsn;
- XLogRecPtr end_lsn;
-
- start_lsn = PG_GETARG_LSN(0);
- end_lsn = PG_GETARG_LSN(1);
-
- end_lsn = ValidateInputLSNs(false, start_lsn, end_lsn);
-
- GetWALRecordsInfo(fcinfo, start_lsn, end_lsn);
-
- PG_RETURN_VOID();
-}
-
-/*
- * Get info and data of all WAL records from start LSN till end of WAL.
- *
- * This function emits an error if a future start i.e. WAL LSN the database
- * system doesn't know about is specified.
- */
-Datum
-pg_get_wal_records_info_till_end_of_wal(PG_FUNCTION_ARGS)
-{
- XLogRecPtr start_lsn;
- XLogRecPtr end_lsn = InvalidXLogRecPtr;
-
- start_lsn = PG_GETARG_LSN(0);
-
- end_lsn = ValidateInputLSNs(true, start_lsn, end_lsn);
+ XLogRecPtr start_lsn = PG_GETARG_LSN(0);
+ XLogRecPtr end_lsn = PG_GETARG_LSN(1);
+ ValidateInputLSNs(start_lsn, &end_lsn);
GetWALRecordsInfo(fcinfo, start_lsn, end_lsn);
PG_RETURN_VOID();
@@ -648,13 +593,13 @@ GetXLogSummaryStats(XLogStats *stats, ReturnSetInfo *rsinfo,
Datum *values, bool *nulls, uint32 ncols,
bool stats_per_record)
{
- MemoryContext old_cxt;
- MemoryContext tmp_cxt;
- uint64 total_count = 0;
- uint64 total_rec_len = 0;
- uint64 total_fpi_len = 0;
- uint64 total_len = 0;
- int ri;
+ MemoryContext old_cxt;
+ MemoryContext tmp_cxt;
+ uint64 total_count = 0;
+ uint64 total_rec_len = 0;
+ uint64 total_fpi_len = 0;
+ uint64 total_len = 0;
+ int ri;
/*
* Each row shows its percentages of the total, so make a first pass to
@@ -757,8 +702,8 @@ GetXLogSummaryStats(XLogStats *stats, ReturnSetInfo *rsinfo,
* Get WAL stats between start LSN and end LSN.
*/
static void
-GetWalStats(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
- XLogRecPtr end_lsn, bool stats_per_record)
+GetWalStats(FunctionCallInfo fcinfo, XLogRecPtr start_lsn, XLogRecPtr end_lsn,
+ bool stats_per_record)
{
#define PG_GET_WAL_STATS_COLS 9
XLogReaderState *xlogreader;
@@ -767,6 +712,8 @@ GetWalStats(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
Datum values[PG_GET_WAL_STATS_COLS] = {0};
bool nulls[PG_GET_WAL_STATS_COLS] = {0};
+ Assert(start_lsn <= end_lsn);
+
InitMaterializedSRF(fcinfo, 0);
xlogreader = InitXLogReaderState(start_lsn);
@@ -791,45 +738,55 @@ GetWalStats(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
/*
* Get stats of all WAL records between start LSN and end LSN.
- *
- * This function emits an error if a future start or end WAL LSN i.e. WAL LSN
- * the database system doesn't know about is specified.
*/
Datum
pg_get_wal_stats(PG_FUNCTION_ARGS)
{
- XLogRecPtr start_lsn;
- XLogRecPtr end_lsn;
- bool stats_per_record;
-
- start_lsn = PG_GETARG_LSN(0);
- end_lsn = PG_GETARG_LSN(1);
- stats_per_record = PG_GETARG_BOOL(2);
-
- end_lsn = ValidateInputLSNs(false, start_lsn, end_lsn);
+ XLogRecPtr start_lsn = PG_GETARG_LSN(0);
+ XLogRecPtr end_lsn = PG_GETARG_LSN(1);
+ bool stats_per_record = PG_GETARG_BOOL(2);
+ ValidateInputLSNs(start_lsn, &end_lsn);
GetWalStats(fcinfo, start_lsn, end_lsn, stats_per_record);
PG_RETURN_VOID();
}
/*
- * Get stats of all WAL records from start LSN till end of WAL.
- *
- * This function emits an error if a future start i.e. WAL LSN the database
- * system doesn't know about is specified.
+ * The following functions have been removed in newer versions in 1.1, but
+ * they are kept around for compatibility.
*/
Datum
-pg_get_wal_stats_till_end_of_wal(PG_FUNCTION_ARGS)
+pg_get_wal_records_info_till_end_of_wal(PG_FUNCTION_ARGS)
{
- XLogRecPtr start_lsn;
- XLogRecPtr end_lsn = InvalidXLogRecPtr;
- bool stats_per_record;
+ XLogRecPtr start_lsn = PG_GETARG_LSN(0);
+ XLogRecPtr end_lsn = GetCurrentLSN();
+
+ if (start_lsn > end_lsn)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("WAL start LSN must be less than current LSN"),
+ errdetail("Current WAL LSN on the database system is at %X/%X.",
+ LSN_FORMAT_ARGS(end_lsn))));
- start_lsn = PG_GETARG_LSN(0);
- stats_per_record = PG_GETARG_BOOL(1);
+ GetWALRecordsInfo(fcinfo, start_lsn, end_lsn);
- end_lsn = ValidateInputLSNs(true, start_lsn, end_lsn);
+ PG_RETURN_VOID();
+}
+
+Datum
+pg_get_wal_stats_till_end_of_wal(PG_FUNCTION_ARGS)
+{
+ XLogRecPtr start_lsn = PG_GETARG_LSN(0);
+ XLogRecPtr end_lsn = GetCurrentLSN();
+ bool stats_per_record = PG_GETARG_BOOL(1);
+
+ if (start_lsn > end_lsn)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("WAL start LSN must be less than current LSN"),
+ errdetail("Current WAL LSN on the database system is at %X/%X.",
+ LSN_FORMAT_ARGS(end_lsn))));
GetWalStats(fcinfo, start_lsn, end_lsn, stats_per_record);
diff --git a/contrib/pg_walinspect/sql/oldextversions.sql b/contrib/pg_walinspect/sql/oldextversions.sql
index f448094add..bd4523d0e6 100644
--- a/contrib/pg_walinspect/sql/oldextversions.sql
+++ b/contrib/pg_walinspect/sql/oldextversions.sql
@@ -2,13 +2,32 @@
CREATE EXTENSION pg_walinspect WITH VERSION '1.0';
+-- Mask DETAIL messages as these could refer to current LSN positions
+\set VERBOSITY terse
+
-- List what version 1.0 contains
\dx+ pg_walinspect
+-- Make sure checkpoints don't interfere with the test
+SELECT 'init' FROM pg_create_physical_replication_slot('regress_pg_walinspect_slot_old_ext_ver', true, false);
+
+CREATE TABLE sample_tbl(col1 int, col2 int);
+SELECT pg_current_wal_lsn() AS wal_lsn1 \gset
+INSERT INTO sample_tbl SELECT * FROM generate_series(1, 2);
+
+-- Tests for the past functions
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info_till_end_of_wal(:'wal_lsn1');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats_till_end_of_wal(:'wal_lsn1');
+
+-- Failures with start LSNs
+SELECT * FROM pg_get_wal_records_info_till_end_of_wal('FFFFFFFF/FFFFFFFF');
+SELECT * FROM pg_get_wal_stats_till_end_of_wal('FFFFFFFF/FFFFFFFF');
+
-- Move to new version 1.1
ALTER EXTENSION pg_walinspect UPDATE TO '1.1';
-- List what version 1.1 contains
\dx+ pg_walinspect
+-- Clean up
DROP EXTENSION pg_walinspect;
diff --git a/contrib/pg_walinspect/sql/pg_walinspect.sql b/contrib/pg_walinspect/sql/pg_walinspect.sql
index 766ed6b87d..e9c2a4d1ee 100644
--- a/contrib/pg_walinspect/sql/pg_walinspect.sql
+++ b/contrib/pg_walinspect/sql/pg_walinspect.sql
@@ -8,7 +8,7 @@ SELECT 'init' FROM pg_create_physical_replication_slot('regress_pg_walinspect_sl
CREATE TABLE sample_tbl(col1 int, col2 int);
--- Save some LSNs for comparisons
+-- Save some LSNs for comparisons.
SELECT pg_current_wal_lsn() AS wal_lsn1 \gset
INSERT INTO sample_tbl SELECT * FROM generate_series(1, 2);
SELECT pg_current_wal_lsn() AS wal_lsn2 \gset
@@ -32,28 +32,26 @@ SELECT * FROM pg_get_wal_stats(:'wal_lsn2', :'wal_lsn1');
SELECT * FROM pg_get_wal_block_info(:'wal_lsn2', :'wal_lsn1');
-- LSNs with the highest value possible.
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_record_info('FFFFFFFF/FFFFFFFF');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info_till_end_of_wal('FFFFFFFF/FFFFFFFF');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats_till_end_of_wal('FFFFFFFF/FFFFFFFF');
--- failures with end LSNs
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_block_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
--- failures with start LSNs
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_block_info('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
+SELECT * FROM pg_get_wal_record_info('FFFFFFFF/FFFFFFFF');
+
+-- Failures with start LSNs.
+SELECT * FROM pg_get_wal_records_info('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
+SELECT * FROM pg_get_wal_stats('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
+SELECT * FROM pg_get_wal_block_info('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
+
+-- Success with end LSNs.
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
-- ===================================================================
-- Tests for all function executions
-- ===================================================================
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info_till_end_of_wal(:'wal_lsn1');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats_till_end_of_wal(:'wal_lsn1');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_block_info(:'wal_lsn1', :'wal_lsn2');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn1', :'wal_lsn2');
-- ===================================================================
-- Test for filtering out WAL records of a particular table
@@ -76,7 +74,7 @@ SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2'
-- Tests to get block information from WAL record
-- ===================================================================
--- Update table to generate some block data
+-- Update table to generate some block data.
SELECT pg_current_wal_lsn() AS wal_lsn3 \gset
UPDATE sample_tbl SET col1 = col1 + 1 WHERE col1 = 1;
SELECT pg_current_wal_lsn() AS wal_lsn4 \gset
@@ -107,9 +105,9 @@ SELECT has_function_privilege('regress_pg_walinspect',
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_block_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- no
--- Functions accessible by users with role pg_read_server_files
-
+-- Functions accessible by users with role pg_read_server_files.
GRANT pg_read_server_files TO regress_pg_walinspect;
+
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_record_info(pg_lsn)', 'EXECUTE'); -- yes
SELECT has_function_privilege('regress_pg_walinspect',
@@ -154,8 +152,5 @@ REVOKE EXECUTE ON FUNCTION pg_get_wal_block_info(pg_lsn, pg_lsn)
-- ===================================================================
DROP ROLE regress_pg_walinspect;
-
-SELECT pg_drop_replication_slot('regress_pg_walinspect_slot');
-
DROP TABLE sample_tbl;
DROP EXTENSION pg_walinspect;
diff --git a/doc/src/sgml/pgwalinspect.sgml b/doc/src/sgml/pgwalinspect.sgml
index 3b19863dce..15e1f6d5ef 100644
--- a/doc/src/sgml/pgwalinspect.sgml
+++ b/doc/src/sgml/pgwalinspect.sgml
@@ -94,9 +94,11 @@ block_ref | blkref #0: rel 1663/5/60221 fork main blk 2
<para>
Gets information of all the valid WAL records between
<replaceable>start_lsn</replaceable> and <replaceable>end_lsn</replaceable>.
- Returns one row per WAL record. If <replaceable>start_lsn</replaceable>
- or <replaceable>end_lsn</replaceable> are not yet available, the
- function will raise an error. For example:
+ Returns one row per WAL record. If a future <replaceable>end_lsn</replaceable>
+ (i.e. the LSN server doesn't know about) is specified, it returns
+ informaton till end of WAL. The function raises an error if
+ <replaceable>start_lsn</replaceable> is not available. For example, usage
+ of the function is as follows:
<screen>
postgres=# SELECT * FROM pg_get_wal_records_info('0/1E913618', '0/1E913740') LIMIT 1;
-[ RECORD 1 ]----+--------------------------------------------------------------
@@ -116,23 +118,6 @@ block_ref |
</listitem>
</varlistentry>
- <varlistentry id="pgwalinspect-funcs-pg-get-wal-records-info-till-end-of-wal">
- <term>
- <function>
- pg_get_wal_records_info_till_end_of_wal(start_lsn pg_lsn)
- returns setof record
- </function>
- </term>
-
- <listitem>
- <para>
- This function is the same as <function>pg_get_wal_records_info()</function>,
- except that it gets information of all the valid WAL records from
- <replaceable>start_lsn</replaceable> till the end of WAL.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry id="pgwalinspect-funcs-pg-get-wal-stats">
<term>
<function>
@@ -148,10 +133,11 @@ block_ref |
<replaceable>end_lsn</replaceable>. By default, it returns one row per
<replaceable>resource_manager</replaceable> type. When
<replaceable>per_record</replaceable> is set to <literal>true</literal>,
- it returns one row per <replaceable>record_type</replaceable>.
- If <replaceable>start_lsn</replaceable>
- or <replaceable>end_lsn</replaceable> are not yet available, the
- function will raise an error. For example:
+ it returns one row per <replaceable>record_type</replaceable>. If a
+ future <replaceable>end_lsn</replaceable> (i.e. the LSN server doesn't
+ know about) is specified, it returns statistics till end of WAL. An error
+ is raised if <replaceable>start_lsn</replaceable> is not available. For
+ example, usage of the function is as follows:
<screen>
postgres=# SELECT * FROM pg_get_wal_stats('0/1E847D00', '0/1E84F500')
WHERE count > 0 LIMIT 1 AND
@@ -171,23 +157,6 @@ combined_size_percentage | 2.8634072910530795
</listitem>
</varlistentry>
- <varlistentry id="pgwalinspect-funcs-pg-get-wal-stats-till-end-of-wal">
- <term>
- <function>
- pg_get_wal_stats_till_end_of_wal(start_lsn pg_lsn, per_record boolean DEFAULT false)
- returns setof record
- </function>
- </term>
-
- <listitem>
- <para>
- This function is the same as <function>pg_get_wal_stats()</function>,
- except that it gets statistics of all the valid WAL records from
- <replaceable>start_lsn</replaceable> till end of WAL.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term>
<function>pg_get_wal_block_info(start_lsn pg_lsn, end_lsn pg_lsn) returns setof record</function>
@@ -202,9 +171,10 @@ combined_size_percentage | 2.8634072910530795
and their information associated with all the valid WAL records between
<replaceable>start_lsn</replaceable> and
<replaceable>end_lsn</replaceable>. Returns one row per block registered
- in a WAL record. If <replaceable>start_lsn</replaceable> or
- <replaceable>end_lsn</replaceable> are not yet available, the function
- will raise an error. For example:
+ in a WAL record. If a future <replaceable>end_lsn</replaceable> (i.e. the
+ LSN server doesn't know about) is specified, it returns statistics till
+ end of WAL. An error is raised if <replaceable>start_lsn</replaceable> is
+ not available. For example, usage of the function is as follows:
<screen>
postgres=# SELECT lsn, blockid, reltablespace, reldatabase, relfilenode,
relblocknumber, forkname,
--
2.34.1
Re: Combine pg_walinspect till_end_of_wal functions with others
On Mon, Mar 13, 2023 at 03:53:37PM +0530, Bharath Rupireddy wrote:
On Mon, Mar 13, 2023 at 12:26 PM Michael Paquier <michael@paquier.xyz> wrote:
+-- Make sure checkpoints don't interfere with the test. +SELECT 'init' FROM pg_create_physical_replication_slot('regress_pg_walinspect_slot', true, false);Adding a physical slot is better for stability of course, but the test
also has to drop it or installcheck would cause an existing cluster to
have that still around. The third argument could be true, as well, so
as we'd use a temporary slot.# Disabled because these tests require "wal_level=replica", which
# some installcheck users do not have (e.g. buildfarm clients).
NO_INSTALLCHECK = 1pg_walinspect can't be run under installcheck. I don't think dropping
the slot at the end is needed, it's unnecessary. I saw
oldextversions.sql using the same replication slot name, well no
problem, but I changed it to a unique name.
-SELECT pg_drop_replication_slot('regress_pg_walinspect_slot');
-
+-- Clean upIn my opinion, it is an incorrect practice to assume that nobody will
ever run these tests on a running instance. FWIW, I have managed
QE/QA flows in the past that did exactly that. I cannot say for
already-deployed clusters that could be used for production still I
don't feel comfortable with the idea to assume that nobody would do
ever that, and calls of pg_drop_replication_slot() are not a
bottleneck. So let's be clean and drop these slots to keep the tests
self-contained. pg_walinspect in REL_15_STABLE gets that right, IMV,
and that's no different from the role cleanup, as one example.
As hackers we know that these functions have been removed and how to
achieve till_end_of_wal with the other functions. I noticed that
you've removed my changes (see below) from the docs that were saying
how to get info/stats till_end_of_wal. That leaves end users confused
as to how they can achieve till_end_of_wal functionality. All users
can't look for commit history/message but they can easily read the
docs. I prefer to have the following (did so in the attached v7) and
get rid of the above note if you don't feel strongly about it.+ If a future <replaceable>end_lsn</replaceable> + (i.e. the LSN server doesn't know about) is specified, it returns + informaton till end of WAL.
FWIW, I don't see a strong need for that, because this documents a
behavior where we would not fail. And FWIW, it just feel natural to
me because the process stops the scan up to where it can. In short,
it should be enough for the docs to mention the error patterns,
nothing else.
I have some comments and fixed them in the attached v7 patch:
1.
+ * pg_get_wal_records_info
*
+ * pg_get_wal_stats
*
I think you wanted to be consistent with function comments with
function names atop, but missed adding for all functions. Actually, I
don't have a strong opinion on these changes as they unnecessarily
bloat the changes, so I removed them.
Either is fine if you feel strongly on this one, I am just used to
doing that.
2. + if (start_lsn > curr_lsn) ereport(ERROR, (errcode(ERRCODE_INVALID_PARAMETER_VALUE), - errmsg("cannot accept future start LSN"), - errdetail("Last known WAL LSN on the database system is at %X/%X.", - LSN_FORMAT_ARGS(curr_lsn)))); - } + errmsg("WAL start LSN must be less than current LSN")));I don't like this inconsistency much, especially when
pg_get_wal_record_info emits "cannot accept future input LSN" with the
current LSN details (this current LSN will give a bit more information
to the user). Also, let's be consistent across returning NULLs if
input LSN/start LSN equal to the current LSN. I've done these changes
in the attached v7 patch.
No arguments against that, consistency is good.
3. I wanted COUNT(*) >= 0 for successful function execution to be
COUNT(*) >= 1 so that we will check for at least the functions
returning 1 record. And failures to be SELECT * FROM. This was my
intention but I don't see that in this patch or in the previous
test-refactoring commit. I added that in the attached v7 patch again.
Also, made test comments conssitent.
Noticed that as well, still it feels to me that these had better be
separated from the rest, and be in their own patch, perhaps *after*
the main patch discussed on this thread, or just moved into their own
threads. If a commit finishes with a list of bullet points referring
to a list of completely different things than the subject, there may
be a problem. In this v7, we have:
- Change the behavior of the functions for end LSNs, tweaking the
tests to do so.
- Adjust more comments and formats in the tests.
- Adjust some tests to be pickier with detection of generated WAL
records.
- Remove the drop slot calls.
But what we need to care most here is the first point.
I am not arguing that none of that should not be changed, but it
should not be inside a patch that slightly tweaks the behaviors of
some existing functions. First, it creates a lot of noise in the
diffs, making it harder for anybody reading this change to find the
core of what's happening. Second, it increases the odds of mistakes
and bugs (if a revert is done, the work to-be-done gets greater at the
end). When it comes to this patch, the changes should only involve
the calls of till_end_of_wal() being moved around from
pg_walinspect.sql to oldextversions.sql. If you look at v6, the tests
are only focusing on this part, and nothing else.
--
Michael
Re: Combine pg_walinspect till_end_of_wal functions with others
On Tue, Mar 14, 2023 at 5:02 AM Michael Paquier <michael@paquier.xyz> wrote:
So let's be clean and drop these slots to keep the tests
self-contained. pg_walinspect in REL_15_STABLE gets that right, IMV,
and that's no different from the role cleanup, as one example.
Hm, added replication slot drop back.
As hackers we know that these functions have been removed and how to
achieve till_end_of_wal with the other functions. I noticed that
you've removed my changes (see below) from the docs that were saying
how to get info/stats till_end_of_wal. That leaves end users confused
as to how they can achieve till_end_of_wal functionality. All users
can't look for commit history/message but they can easily read the
docs. I prefer to have the following (did so in the attached v7) and
get rid of the above note if you don't feel strongly about it.+ If a future <replaceable>end_lsn</replaceable> + (i.e. the LSN server doesn't know about) is specified, it returns + informaton till end of WAL.FWIW, I don't see a strong need for that, because this documents a
behavior where we would not fail. And FWIW, it just feel natural to
me because the process stops the scan up to where it can. In short,
it should be enough for the docs to mention the error patterns,
nothing else.
My thoughts are simple here - how would one (an end user, not me and
not you) figure out how to get info/stats till the end of WAL? I'm
sure it would be difficult to find that out without looking at the
code or commit history. Search for till end of WAL behaviour with new
version will be more given the 1.0 version has explicit functions to
do that. IMO, there's no harm in being explicit in how to achieve till
end of WAL functionality around in the docs.
3. I wanted COUNT(*) >= 0 for successful function execution to be
COUNT(*) >= 1 so that we will check for at least the functions
returning 1 record. And failures to be SELECT * FROM. This was my
intention but I don't see that in this patch or in the previous
test-refactoring commit. I added that in the attached v7 patch again.
Also, made test comments conssitent.Noticed that as well, still it feels to me that these had better be
separated from the rest, and be in their own patch, perhaps *after*
the main patch discussed on this thread, or just moved into their own
threads. If a commit finishes with a list of bullet points referring
to a list of completely different things than the subject, there may
be a problem. In this v7, we have:
- Change the behavior of the functions for end LSNs, tweaking the
tests to do so.
- Adjust more comments and formats in the tests.
- Adjust some tests to be pickier with detection of generated WAL
records.
- Remove the drop slot calls.
But what we need to care most here is the first point.
I get it. I divided the patches to 0001 and 0002 with 0001 focussing
on the change of behaviour around future end LSNs, dropping till end
of WAL functions and tests tweakings related to it. 0002 has all other
tests tidy up things.
Please find the attached v8 patch set for further review.
--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
Attachments:
v8-0001-Rework-pg_walinspect-functions.patchapplication/octet-stream; name=v8-0001-Rework-pg_walinspect-functions.patchDownload
From c82516444da3d00e990055d8da26ba728cb0ca26 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Tue, 14 Mar 2023 04:31:26 +0000
Subject: [PATCH v8] Rework pg_walinspect functions
---
.../pg_walinspect/expected/oldextversions.out | 47 +++-
.../pg_walinspect/expected/pg_walinspect.out | 42 ++--
.../pg_walinspect/pg_walinspect--1.0--1.1.sql | 4 +
contrib/pg_walinspect/pg_walinspect.c | 215 +++++++-----------
contrib/pg_walinspect/sql/oldextversions.sql | 21 ++
contrib/pg_walinspect/sql/pg_walinspect.sql | 6 +-
doc/src/sgml/pgwalinspect.sgml | 58 ++---
7 files changed, 186 insertions(+), 207 deletions(-)
diff --git a/contrib/pg_walinspect/expected/oldextversions.out b/contrib/pg_walinspect/expected/oldextversions.out
index 0db3538693..9195ab312f 100644
--- a/contrib/pg_walinspect/expected/oldextversions.out
+++ b/contrib/pg_walinspect/expected/oldextversions.out
@@ -1,5 +1,7 @@
-- test old extension version entry points
CREATE EXTENSION pg_walinspect WITH VERSION '1.0';
+-- Mask DETAIL messages as these could refer to current LSN positions
+\set VERBOSITY terse
-- List what version 1.0 contains
\dx+ pg_walinspect
Objects in extension "pg_walinspect"
@@ -12,19 +14,52 @@ CREATE EXTENSION pg_walinspect WITH VERSION '1.0';
function pg_get_wal_stats_till_end_of_wal(pg_lsn,boolean)
(5 rows)
+-- Make sure checkpoints don't interfere with the test
+SELECT 'init' FROM pg_create_physical_replication_slot('regress_pg_walinspect_slot_old_ext_ver', true, false);
+ ?column?
+----------
+ init
+(1 row)
+
+CREATE TABLE sample_tbl(col1 int, col2 int);
+SELECT pg_current_wal_lsn() AS wal_lsn1 \gset
+INSERT INTO sample_tbl SELECT * FROM generate_series(1, 2);
+-- Tests for the past functions
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info_till_end_of_wal(:'wal_lsn1');
+ ok
+----
+ t
+(1 row)
+
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats_till_end_of_wal(:'wal_lsn1');
+ ok
+----
+ t
+(1 row)
+
+-- Failures with start LSNs
+SELECT * FROM pg_get_wal_records_info_till_end_of_wal('FFFFFFFF/FFFFFFFF');
+ERROR: WAL start LSN must be less than current LSN
+SELECT * FROM pg_get_wal_stats_till_end_of_wal('FFFFFFFF/FFFFFFFF');
+ERROR: WAL start LSN must be less than current LSN
-- Move to new version 1.1
ALTER EXTENSION pg_walinspect UPDATE TO '1.1';
-- List what version 1.1 contains
\dx+ pg_walinspect
- Objects in extension "pg_walinspect"
- Object description
------------------------------------------------------------
+ Objects in extension "pg_walinspect"
+ Object description
+--------------------------------------------------
function pg_get_wal_block_info(pg_lsn,pg_lsn)
function pg_get_wal_record_info(pg_lsn)
function pg_get_wal_records_info(pg_lsn,pg_lsn)
- function pg_get_wal_records_info_till_end_of_wal(pg_lsn)
function pg_get_wal_stats(pg_lsn,pg_lsn,boolean)
- function pg_get_wal_stats_till_end_of_wal(pg_lsn,boolean)
-(6 rows)
+(4 rows)
+-- Clean up
DROP EXTENSION pg_walinspect;
+SELECT pg_drop_replication_slot('regress_pg_walinspect_slot_old_ext_ver');
+ pg_drop_replication_slot
+--------------------------
+
+(1 row)
+
diff --git a/contrib/pg_walinspect/expected/pg_walinspect.out b/contrib/pg_walinspect/expected/pg_walinspect.out
index c9fb781055..1d4397812a 100644
--- a/contrib/pg_walinspect/expected/pg_walinspect.out
+++ b/contrib/pg_walinspect/expected/pg_walinspect.out
@@ -36,41 +36,37 @@ SELECT * FROM pg_get_wal_block_info(:'wal_lsn2', :'wal_lsn1');
ERROR: WAL start LSN must be less than end LSN
-- LSNs with the highest value possible.
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_record_info('FFFFFFFF/FFFFFFFF');
-ERROR: cannot accept future input LSN
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info_till_end_of_wal('FFFFFFFF/FFFFFFFF');
-ERROR: cannot accept future start LSN
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats_till_end_of_wal('FFFFFFFF/FFFFFFFF');
-ERROR: cannot accept future start LSN
--- failures with end LSNs
+ERROR: WAL input LSN must be less than current LSN
+-- successes with end LSNs
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
-ERROR: cannot accept future end LSN
+ ok
+----
+ t
+(1 row)
+
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
-ERROR: cannot accept future end LSN
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_block_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
-ERROR: cannot accept future end LSN
--- failures with start LSNs
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
-ERROR: cannot accept future start LSN
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
-ERROR: cannot accept future start LSN
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_block_info('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
-ERROR: cannot accept future start LSN
--- ===================================================================
--- Tests for all function executions
--- ===================================================================
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info_till_end_of_wal(:'wal_lsn1');
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_block_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats_till_end_of_wal(:'wal_lsn1');
+-- failures with start LSNs
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
+ERROR: WAL start LSN must be less than current LSN
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
+ERROR: WAL start LSN must be less than current LSN
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_block_info('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
+ERROR: WAL start LSN must be less than current LSN
+-- ===================================================================
+-- Tests for all function executions
+-- ===================================================================
+SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
ok
----
t
diff --git a/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql b/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql
index e674ef25aa..586c3b4467 100644
--- a/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql
+++ b/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql
@@ -3,6 +3,10 @@
-- complain if script is sourced in psql, rather than via ALTER EXTENSION
\echo Use "ALTER EXTENSION pg_walinspect UPDATE TO '1.1'" to load this file. \quit
+-- Unsupported functions after 1.1.
+DROP FUNCTION pg_get_wal_records_info_till_end_of_wal(pg_lsn);
+DROP FUNCTION pg_get_wal_stats_till_end_of_wal(pg_lsn, boolean);
+
--
-- pg_get_wal_block_info()
--
diff --git a/contrib/pg_walinspect/pg_walinspect.c b/contrib/pg_walinspect/pg_walinspect.c
index ee88dc4992..3b3215daf5 100644
--- a/contrib/pg_walinspect/pg_walinspect.c
+++ b/contrib/pg_walinspect/pg_walinspect.c
@@ -38,14 +38,14 @@ PG_FUNCTION_INFO_V1(pg_get_wal_records_info_till_end_of_wal);
PG_FUNCTION_INFO_V1(pg_get_wal_stats);
PG_FUNCTION_INFO_V1(pg_get_wal_stats_till_end_of_wal);
-static bool IsFutureLSN(XLogRecPtr lsn, XLogRecPtr *curr_lsn);
+static void ValidateInputLSNs(XLogRecPtr start_lsn, XLogRecPtr *end_lsn);
+static XLogRecPtr GetCurrentLSN(void);
static XLogReaderState *InitXLogReaderState(XLogRecPtr lsn);
static XLogRecord *ReadNextXLogRecord(XLogReaderState *xlogreader);
static void GetWALRecordInfo(XLogReaderState *record, Datum *values,
bool *nulls, uint32 ncols);
-static XLogRecPtr ValidateInputLSNs(bool till_end_of_wal,
- XLogRecPtr start_lsn, XLogRecPtr end_lsn);
-static void GetWALRecordsInfo(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
+static void GetWALRecordsInfo(FunctionCallInfo fcinfo,
+ XLogRecPtr start_lsn,
XLogRecPtr end_lsn);
static void GetXLogSummaryStats(XLogStats *stats, ReturnSetInfo *rsinfo,
Datum *values, bool *nulls, uint32 ncols,
@@ -55,32 +55,32 @@ static void FillXLogStatsRow(const char *name, uint64 n, uint64 total_count,
uint64 fpi_len, uint64 total_fpi_len,
uint64 tot_len, uint64 total_len,
Datum *values, bool *nulls, uint32 ncols);
-static void GetWalStats(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
- XLogRecPtr end_lsn, bool stats_per_record);
+static void GetWalStats(FunctionCallInfo fcinfo,
+ XLogRecPtr start_lsn,
+ XLogRecPtr end_lsn,
+ bool stats_per_record);
static void GetWALBlockInfo(FunctionCallInfo fcinfo, XLogReaderState *record);
/*
- * Check if the given LSN is in future. Also, return the LSN up to which the
- * server has WAL.
+ * Return the LSN up to which the server has WAL.
*/
-static bool
-IsFutureLSN(XLogRecPtr lsn, XLogRecPtr *curr_lsn)
+static XLogRecPtr
+GetCurrentLSN(void)
{
+ XLogRecPtr curr_lsn;
+
/*
* We determine the current LSN of the server similar to how page_read
* callback read_local_xlog_page_no_wait does.
*/
if (!RecoveryInProgress())
- *curr_lsn = GetFlushRecPtr(NULL);
+ curr_lsn = GetFlushRecPtr(NULL);
else
- *curr_lsn = GetXLogReplayRecPtr(NULL);
-
- Assert(!XLogRecPtrIsInvalid(*curr_lsn));
+ curr_lsn = GetXLogReplayRecPtr(NULL);
- if (lsn >= *curr_lsn)
- return true;
+ Assert(!XLogRecPtrIsInvalid(curr_lsn));
- return false;
+ return curr_lsn;
}
/*
@@ -354,23 +354,17 @@ GetWALBlockInfo(FunctionCallInfo fcinfo, XLogReaderState *record)
* their relation information, and the data saved in each block associated
* to a record. Decompression is applied to the full page images, if
* necessary.
- *
- * This function emits an error if a future start or end WAL LSN i.e. WAL LSN
- * the database system doesn't know about is specified.
*/
Datum
pg_get_wal_block_info(PG_FUNCTION_ARGS)
{
- XLogRecPtr start_lsn;
- XLogRecPtr end_lsn;
+ XLogRecPtr start_lsn = PG_GETARG_LSN(0);
+ XLogRecPtr end_lsn = PG_GETARG_LSN(1);
XLogReaderState *xlogreader;
MemoryContext old_cxt;
MemoryContext tmp_cxt;
- start_lsn = PG_GETARG_LSN(0);
- end_lsn = PG_GETARG_LSN(1);
-
- end_lsn = ValidateInputLSNs(false, start_lsn, end_lsn);
+ ValidateInputLSNs(start_lsn, &end_lsn);
InitMaterializedSRF(fcinfo, 0);
@@ -404,9 +398,6 @@ pg_get_wal_block_info(PG_FUNCTION_ARGS)
/*
* Get WAL record info.
- *
- * This function emits an error if a future WAL LSN i.e. WAL LSN the database
- * system doesn't know about is specified.
*/
Datum
pg_get_wal_record_info(PG_FUNCTION_ARGS)
@@ -422,20 +413,14 @@ pg_get_wal_record_info(PG_FUNCTION_ARGS)
HeapTuple tuple;
lsn = PG_GETARG_LSN(0);
+ curr_lsn = GetCurrentLSN();
- if (IsFutureLSN(lsn, &curr_lsn))
- {
- /*
- * GetFlushRecPtr or GetXLogReplayRecPtr gives "end+1" LSN of the last
- * record flushed or replayed respectively. But let's use the LSN up
- * to "end" in user facing message.
- */
+ if (lsn > curr_lsn)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("cannot accept future input LSN"),
- errdetail("Last known WAL LSN on the database system is at %X/%X.",
+ errmsg("WAL input LSN must be less than current LSN"),
+ errdetail("Current WAL LSN on the database system is at %X/%X.",
LSN_FORMAT_ARGS(curr_lsn))));
- }
/* Build a tuple descriptor for our result type. */
if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
@@ -462,44 +447,30 @@ pg_get_wal_record_info(PG_FUNCTION_ARGS)
}
/*
- * Validate the input LSNs and compute end LSN for till_end_of_wal versions.
+ * Validate start and end LSNs coming from the function inputs.
+ *
+ * If end_lsn is found to be higher than the current LSN reported by the
+ * cluster, use the current LSN as the upper bound.
*/
-static XLogRecPtr
-ValidateInputLSNs(bool till_end_of_wal, XLogRecPtr start_lsn,
- XLogRecPtr end_lsn)
+static void
+ValidateInputLSNs(XLogRecPtr start_lsn, XLogRecPtr *end_lsn)
{
- XLogRecPtr curr_lsn;
+ XLogRecPtr curr_lsn = GetCurrentLSN();
- if (IsFutureLSN(start_lsn, &curr_lsn))
- {
- /*
- * GetFlushRecPtr or GetXLogReplayRecPtr gives "end+1" LSN of the last
- * record flushed or replayed respectively. But let's use the LSN up
- * to "end" in user facing message.
- */
+ if (start_lsn > curr_lsn)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("cannot accept future start LSN"),
- errdetail("Last known WAL LSN on the database system is at %X/%X.",
+ errmsg("WAL start LSN must be less than current LSN"),
+ errdetail("Current WAL LSN on the database system is at %X/%X.",
LSN_FORMAT_ARGS(curr_lsn))));
- }
- if (till_end_of_wal)
- end_lsn = curr_lsn;
-
- if (end_lsn > curr_lsn)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("cannot accept future end LSN"),
- errdetail("Last known WAL LSN on the database system is at %X/%X.",
- LSN_FORMAT_ARGS(curr_lsn))));
-
- if (start_lsn >= end_lsn)
+ if (start_lsn > *end_lsn)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("WAL start LSN must be less than end LSN")));
- return end_lsn;
+ if (*end_lsn > curr_lsn)
+ *end_lsn = curr_lsn;
}
/*
@@ -517,6 +488,8 @@ GetWALRecordsInfo(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
MemoryContext old_cxt;
MemoryContext tmp_cxt;
+ Assert(start_lsn <= end_lsn);
+
InitMaterializedSRF(fcinfo, 0);
xlogreader = InitXLogReaderState(start_lsn);
@@ -553,42 +526,14 @@ GetWALRecordsInfo(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
/*
* Get info and data of all WAL records between start LSN and end LSN.
- *
- * This function emits an error if a future start or end WAL LSN i.e. WAL LSN
- * the database system doesn't know about is specified.
*/
Datum
pg_get_wal_records_info(PG_FUNCTION_ARGS)
{
- XLogRecPtr start_lsn;
- XLogRecPtr end_lsn;
-
- start_lsn = PG_GETARG_LSN(0);
- end_lsn = PG_GETARG_LSN(1);
-
- end_lsn = ValidateInputLSNs(false, start_lsn, end_lsn);
-
- GetWALRecordsInfo(fcinfo, start_lsn, end_lsn);
-
- PG_RETURN_VOID();
-}
-
-/*
- * Get info and data of all WAL records from start LSN till end of WAL.
- *
- * This function emits an error if a future start i.e. WAL LSN the database
- * system doesn't know about is specified.
- */
-Datum
-pg_get_wal_records_info_till_end_of_wal(PG_FUNCTION_ARGS)
-{
- XLogRecPtr start_lsn;
- XLogRecPtr end_lsn = InvalidXLogRecPtr;
-
- start_lsn = PG_GETARG_LSN(0);
-
- end_lsn = ValidateInputLSNs(true, start_lsn, end_lsn);
+ XLogRecPtr start_lsn = PG_GETARG_LSN(0);
+ XLogRecPtr end_lsn = PG_GETARG_LSN(1);
+ ValidateInputLSNs(start_lsn, &end_lsn);
GetWALRecordsInfo(fcinfo, start_lsn, end_lsn);
PG_RETURN_VOID();
@@ -648,13 +593,13 @@ GetXLogSummaryStats(XLogStats *stats, ReturnSetInfo *rsinfo,
Datum *values, bool *nulls, uint32 ncols,
bool stats_per_record)
{
- MemoryContext old_cxt;
- MemoryContext tmp_cxt;
- uint64 total_count = 0;
- uint64 total_rec_len = 0;
- uint64 total_fpi_len = 0;
- uint64 total_len = 0;
- int ri;
+ MemoryContext old_cxt;
+ MemoryContext tmp_cxt;
+ uint64 total_count = 0;
+ uint64 total_rec_len = 0;
+ uint64 total_fpi_len = 0;
+ uint64 total_len = 0;
+ int ri;
/*
* Each row shows its percentages of the total, so make a first pass to
@@ -757,8 +702,8 @@ GetXLogSummaryStats(XLogStats *stats, ReturnSetInfo *rsinfo,
* Get WAL stats between start LSN and end LSN.
*/
static void
-GetWalStats(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
- XLogRecPtr end_lsn, bool stats_per_record)
+GetWalStats(FunctionCallInfo fcinfo, XLogRecPtr start_lsn, XLogRecPtr end_lsn,
+ bool stats_per_record)
{
#define PG_GET_WAL_STATS_COLS 9
XLogReaderState *xlogreader;
@@ -767,6 +712,8 @@ GetWalStats(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
Datum values[PG_GET_WAL_STATS_COLS] = {0};
bool nulls[PG_GET_WAL_STATS_COLS] = {0};
+ Assert(start_lsn <= end_lsn);
+
InitMaterializedSRF(fcinfo, 0);
xlogreader = InitXLogReaderState(start_lsn);
@@ -791,45 +738,55 @@ GetWalStats(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
/*
* Get stats of all WAL records between start LSN and end LSN.
- *
- * This function emits an error if a future start or end WAL LSN i.e. WAL LSN
- * the database system doesn't know about is specified.
*/
Datum
pg_get_wal_stats(PG_FUNCTION_ARGS)
{
- XLogRecPtr start_lsn;
- XLogRecPtr end_lsn;
- bool stats_per_record;
-
- start_lsn = PG_GETARG_LSN(0);
- end_lsn = PG_GETARG_LSN(1);
- stats_per_record = PG_GETARG_BOOL(2);
-
- end_lsn = ValidateInputLSNs(false, start_lsn, end_lsn);
+ XLogRecPtr start_lsn = PG_GETARG_LSN(0);
+ XLogRecPtr end_lsn = PG_GETARG_LSN(1);
+ bool stats_per_record = PG_GETARG_BOOL(2);
+ ValidateInputLSNs(start_lsn, &end_lsn);
GetWalStats(fcinfo, start_lsn, end_lsn, stats_per_record);
PG_RETURN_VOID();
}
/*
- * Get stats of all WAL records from start LSN till end of WAL.
- *
- * This function emits an error if a future start i.e. WAL LSN the database
- * system doesn't know about is specified.
+ * The following functions have been removed in newer versions in 1.1, but
+ * they are kept around for compatibility.
*/
Datum
-pg_get_wal_stats_till_end_of_wal(PG_FUNCTION_ARGS)
+pg_get_wal_records_info_till_end_of_wal(PG_FUNCTION_ARGS)
{
- XLogRecPtr start_lsn;
- XLogRecPtr end_lsn = InvalidXLogRecPtr;
- bool stats_per_record;
+ XLogRecPtr start_lsn = PG_GETARG_LSN(0);
+ XLogRecPtr end_lsn = GetCurrentLSN();
+
+ if (start_lsn > end_lsn)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("WAL start LSN must be less than current LSN"),
+ errdetail("Current WAL LSN on the database system is at %X/%X.",
+ LSN_FORMAT_ARGS(end_lsn))));
- start_lsn = PG_GETARG_LSN(0);
- stats_per_record = PG_GETARG_BOOL(1);
+ GetWALRecordsInfo(fcinfo, start_lsn, end_lsn);
- end_lsn = ValidateInputLSNs(true, start_lsn, end_lsn);
+ PG_RETURN_VOID();
+}
+
+Datum
+pg_get_wal_stats_till_end_of_wal(PG_FUNCTION_ARGS)
+{
+ XLogRecPtr start_lsn = PG_GETARG_LSN(0);
+ XLogRecPtr end_lsn = GetCurrentLSN();
+ bool stats_per_record = PG_GETARG_BOOL(1);
+
+ if (start_lsn > end_lsn)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("WAL start LSN must be less than current LSN"),
+ errdetail("Current WAL LSN on the database system is at %X/%X.",
+ LSN_FORMAT_ARGS(end_lsn))));
GetWalStats(fcinfo, start_lsn, end_lsn, stats_per_record);
diff --git a/contrib/pg_walinspect/sql/oldextversions.sql b/contrib/pg_walinspect/sql/oldextversions.sql
index f448094add..3037a193de 100644
--- a/contrib/pg_walinspect/sql/oldextversions.sql
+++ b/contrib/pg_walinspect/sql/oldextversions.sql
@@ -2,13 +2,34 @@
CREATE EXTENSION pg_walinspect WITH VERSION '1.0';
+-- Mask DETAIL messages as these could refer to current LSN positions
+\set VERBOSITY terse
+
-- List what version 1.0 contains
\dx+ pg_walinspect
+-- Make sure checkpoints don't interfere with the test
+SELECT 'init' FROM pg_create_physical_replication_slot('regress_pg_walinspect_slot_old_ext_ver', true, false);
+
+CREATE TABLE sample_tbl(col1 int, col2 int);
+SELECT pg_current_wal_lsn() AS wal_lsn1 \gset
+INSERT INTO sample_tbl SELECT * FROM generate_series(1, 2);
+
+-- Tests for the past functions
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info_till_end_of_wal(:'wal_lsn1');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats_till_end_of_wal(:'wal_lsn1');
+
+-- Failures with start LSNs
+SELECT * FROM pg_get_wal_records_info_till_end_of_wal('FFFFFFFF/FFFFFFFF');
+SELECT * FROM pg_get_wal_stats_till_end_of_wal('FFFFFFFF/FFFFFFFF');
+
-- Move to new version 1.1
ALTER EXTENSION pg_walinspect UPDATE TO '1.1';
-- List what version 1.1 contains
\dx+ pg_walinspect
+-- Clean up
DROP EXTENSION pg_walinspect;
+
+SELECT pg_drop_replication_slot('regress_pg_walinspect_slot_old_ext_ver');
diff --git a/contrib/pg_walinspect/sql/pg_walinspect.sql b/contrib/pg_walinspect/sql/pg_walinspect.sql
index 766ed6b87d..27ee1d90b5 100644
--- a/contrib/pg_walinspect/sql/pg_walinspect.sql
+++ b/contrib/pg_walinspect/sql/pg_walinspect.sql
@@ -33,9 +33,7 @@ SELECT * FROM pg_get_wal_block_info(:'wal_lsn2', :'wal_lsn1');
-- LSNs with the highest value possible.
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_record_info('FFFFFFFF/FFFFFFFF');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info_till_end_of_wal('FFFFFFFF/FFFFFFFF');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats_till_end_of_wal('FFFFFFFF/FFFFFFFF');
--- failures with end LSNs
+-- successes with end LSNs
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_block_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
@@ -49,8 +47,6 @@ SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_block_info('FFFFFFFF/FFFFFFFE', 'FFFF
-- ===================================================================
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info_till_end_of_wal(:'wal_lsn1');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats_till_end_of_wal(:'wal_lsn1');
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_block_info(:'wal_lsn1', :'wal_lsn2');
diff --git a/doc/src/sgml/pgwalinspect.sgml b/doc/src/sgml/pgwalinspect.sgml
index 3b19863dce..5f371f1a1c 100644
--- a/doc/src/sgml/pgwalinspect.sgml
+++ b/doc/src/sgml/pgwalinspect.sgml
@@ -94,9 +94,11 @@ block_ref | blkref #0: rel 1663/5/60221 fork main blk 2
<para>
Gets information of all the valid WAL records between
<replaceable>start_lsn</replaceable> and <replaceable>end_lsn</replaceable>.
- Returns one row per WAL record. If <replaceable>start_lsn</replaceable>
- or <replaceable>end_lsn</replaceable> are not yet available, the
- function will raise an error. For example:
+ Returns one row per WAL record. If a future <replaceable>end_lsn</replaceable>
+ (i.e. the LSN server doesn't know about) is specified, it returns
+ information till end of WAL. The function raises an error if
+ <replaceable>start_lsn</replaceable> is not available. For example, usage
+ of the function is as follows:
<screen>
postgres=# SELECT * FROM pg_get_wal_records_info('0/1E913618', '0/1E913740') LIMIT 1;
-[ RECORD 1 ]----+--------------------------------------------------------------
@@ -116,23 +118,6 @@ block_ref |
</listitem>
</varlistentry>
- <varlistentry id="pgwalinspect-funcs-pg-get-wal-records-info-till-end-of-wal">
- <term>
- <function>
- pg_get_wal_records_info_till_end_of_wal(start_lsn pg_lsn)
- returns setof record
- </function>
- </term>
-
- <listitem>
- <para>
- This function is the same as <function>pg_get_wal_records_info()</function>,
- except that it gets information of all the valid WAL records from
- <replaceable>start_lsn</replaceable> till the end of WAL.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry id="pgwalinspect-funcs-pg-get-wal-stats">
<term>
<function>
@@ -148,10 +133,11 @@ block_ref |
<replaceable>end_lsn</replaceable>. By default, it returns one row per
<replaceable>resource_manager</replaceable> type. When
<replaceable>per_record</replaceable> is set to <literal>true</literal>,
- it returns one row per <replaceable>record_type</replaceable>.
- If <replaceable>start_lsn</replaceable>
- or <replaceable>end_lsn</replaceable> are not yet available, the
- function will raise an error. For example:
+ it returns one row per <replaceable>record_type</replaceable>. If a
+ future <replaceable>end_lsn</replaceable> (i.e. the LSN server doesn't
+ know about) is specified, it returns statistics till end of WAL. An error
+ is raised if <replaceable>start_lsn</replaceable> is not available. For
+ example, usage of the function is as follows:
<screen>
postgres=# SELECT * FROM pg_get_wal_stats('0/1E847D00', '0/1E84F500')
WHERE count > 0 LIMIT 1 AND
@@ -171,23 +157,6 @@ combined_size_percentage | 2.8634072910530795
</listitem>
</varlistentry>
- <varlistentry id="pgwalinspect-funcs-pg-get-wal-stats-till-end-of-wal">
- <term>
- <function>
- pg_get_wal_stats_till_end_of_wal(start_lsn pg_lsn, per_record boolean DEFAULT false)
- returns setof record
- </function>
- </term>
-
- <listitem>
- <para>
- This function is the same as <function>pg_get_wal_stats()</function>,
- except that it gets statistics of all the valid WAL records from
- <replaceable>start_lsn</replaceable> till end of WAL.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term>
<function>pg_get_wal_block_info(start_lsn pg_lsn, end_lsn pg_lsn) returns setof record</function>
@@ -202,9 +171,10 @@ combined_size_percentage | 2.8634072910530795
and their information associated with all the valid WAL records between
<replaceable>start_lsn</replaceable> and
<replaceable>end_lsn</replaceable>. Returns one row per block registered
- in a WAL record. If <replaceable>start_lsn</replaceable> or
- <replaceable>end_lsn</replaceable> are not yet available, the function
- will raise an error. For example:
+ in a WAL record. If a future <replaceable>end_lsn</replaceable> (i.e. the
+ LSN server doesn't know about) is specified, it returns information till
+ end of WAL. An error is raised if <replaceable>start_lsn</replaceable> is
+ not available. For example, usage of the function is as follows:
<screen>
postgres=# SELECT lsn, blockid, reltablespace, reldatabase, relfilenode,
relblocknumber, forkname,
--
2.34.1
v8-0002-Tweak-pg_walinspect-tests-to-be-consistent.patchapplication/octet-stream; name=v8-0002-Tweak-pg_walinspect-tests-to-be-consistent.patchDownload
From d58a3e4a6eaaba39e658e20464f61c96b1a09890 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Tue, 14 Mar 2023 04:42:59 +0000
Subject: [PATCH v8] Tweak pg_walinspect tests to be consistent
---
.../pg_walinspect/expected/oldextversions.out | 18 ++++----
.../pg_walinspect/expected/pg_walinspect.out | 41 +++++++++---------
contrib/pg_walinspect/sql/oldextversions.sql | 18 ++++----
contrib/pg_walinspect/sql/pg_walinspect.sql | 42 ++++++++++---------
4 files changed, 60 insertions(+), 59 deletions(-)
diff --git a/contrib/pg_walinspect/expected/oldextversions.out b/contrib/pg_walinspect/expected/oldextversions.out
index 9195ab312f..026ca07108 100644
--- a/contrib/pg_walinspect/expected/oldextversions.out
+++ b/contrib/pg_walinspect/expected/oldextversions.out
@@ -1,8 +1,8 @@
--- test old extension version entry points
+-- Test old extension version entry points.
CREATE EXTENSION pg_walinspect WITH VERSION '1.0';
--- Mask DETAIL messages as these could refer to current LSN positions
+-- Mask DETAIL messages as these could refer to current LSN positions.
\set VERBOSITY terse
--- List what version 1.0 contains
+-- List what version 1.0 contains.
\dx+ pg_walinspect
Objects in extension "pg_walinspect"
Object description
@@ -14,7 +14,7 @@ CREATE EXTENSION pg_walinspect WITH VERSION '1.0';
function pg_get_wal_stats_till_end_of_wal(pg_lsn,boolean)
(5 rows)
--- Make sure checkpoints don't interfere with the test
+-- Make sure checkpoints don't interfere with the test.
SELECT 'init' FROM pg_create_physical_replication_slot('regress_pg_walinspect_slot_old_ext_ver', true, false);
?column?
----------
@@ -24,7 +24,7 @@ SELECT 'init' FROM pg_create_physical_replication_slot('regress_pg_walinspect_sl
CREATE TABLE sample_tbl(col1 int, col2 int);
SELECT pg_current_wal_lsn() AS wal_lsn1 \gset
INSERT INTO sample_tbl SELECT * FROM generate_series(1, 2);
--- Tests for the past functions
+-- Tests for the past functions.
SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info_till_end_of_wal(:'wal_lsn1');
ok
----
@@ -37,14 +37,14 @@ SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats_till_end_of_wal(:'wal_lsn1');
t
(1 row)
--- Failures with start LSNs
+-- Failure with highest possible start LSNs.
SELECT * FROM pg_get_wal_records_info_till_end_of_wal('FFFFFFFF/FFFFFFFF');
ERROR: WAL start LSN must be less than current LSN
SELECT * FROM pg_get_wal_stats_till_end_of_wal('FFFFFFFF/FFFFFFFF');
ERROR: WAL start LSN must be less than current LSN
--- Move to new version 1.1
+-- Move to new version 1.1.
ALTER EXTENSION pg_walinspect UPDATE TO '1.1';
--- List what version 1.1 contains
+-- List what version 1.1 contains.
\dx+ pg_walinspect
Objects in extension "pg_walinspect"
Object description
@@ -55,7 +55,7 @@ ALTER EXTENSION pg_walinspect UPDATE TO '1.1';
function pg_get_wal_stats(pg_lsn,pg_lsn,boolean)
(4 rows)
--- Clean up
+-- Clean up.
DROP EXTENSION pg_walinspect;
SELECT pg_drop_replication_slot('regress_pg_walinspect_slot_old_ext_ver');
pg_drop_replication_slot
diff --git a/contrib/pg_walinspect/expected/pg_walinspect.out b/contrib/pg_walinspect/expected/pg_walinspect.out
index 1d4397812a..710d1a2859 100644
--- a/contrib/pg_walinspect/expected/pg_walinspect.out
+++ b/contrib/pg_walinspect/expected/pg_walinspect.out
@@ -9,7 +9,6 @@ SELECT 'init' FROM pg_create_physical_replication_slot('regress_pg_walinspect_sl
(1 row)
CREATE TABLE sample_tbl(col1 int, col2 int);
--- Save some LSNs for comparisons
SELECT pg_current_wal_lsn() AS wal_lsn1 \gset
INSERT INTO sample_tbl SELECT * FROM generate_series(1, 2);
SELECT pg_current_wal_lsn() AS wal_lsn2 \gset
@@ -34,57 +33,57 @@ SELECT * FROM pg_get_wal_stats(:'wal_lsn2', :'wal_lsn1');
ERROR: WAL start LSN must be less than end LSN
SELECT * FROM pg_get_wal_block_info(:'wal_lsn2', :'wal_lsn1');
ERROR: WAL start LSN must be less than end LSN
--- LSNs with the highest value possible.
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_record_info('FFFFFFFF/FFFFFFFF');
+-- Failure with highest possible input LSN.
+SELECT * FROM pg_get_wal_record_info('FFFFFFFF/FFFFFFFF');
ERROR: WAL input LSN must be less than current LSN
--- successes with end LSNs
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
+-- Failures with highest possible start LSNs.
+SELECT * FROM pg_get_wal_records_info('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
+ERROR: WAL start LSN must be less than current LSN
+SELECT * FROM pg_get_wal_stats('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
+ERROR: WAL start LSN must be less than current LSN
+SELECT * FROM pg_get_wal_block_info('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
+ERROR: WAL start LSN must be less than current LSN
+-- Successes with highest possible end LSNs.
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_block_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
ok
----
t
(1 row)
--- failures with start LSNs
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
-ERROR: WAL start LSN must be less than current LSN
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
-ERROR: WAL start LSN must be less than current LSN
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_block_info('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
-ERROR: WAL start LSN must be less than current LSN
-- ===================================================================
-- Tests for all function executions
-- ===================================================================
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_block_info(:'wal_lsn1', :'wal_lsn2');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn1', :'wal_lsn2');
ok
----
t
@@ -115,7 +114,7 @@ SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2'
-- ===================================================================
-- Tests to get block information from WAL record
-- ===================================================================
--- Update table to generate some block data
+-- Update table to generate some block data.
SELECT pg_current_wal_lsn() AS wal_lsn3 \gset
UPDATE sample_tbl SET col1 = col1 + 1 WHERE col1 = 1;
SELECT pg_current_wal_lsn() AS wal_lsn4 \gset
@@ -172,7 +171,7 @@ SELECT has_function_privilege('regress_pg_walinspect',
f
(1 row)
--- Functions accessible by users with role pg_read_server_files
+-- Functions accessible by users with role pg_read_server_files.
GRANT pg_read_server_files TO regress_pg_walinspect;
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_record_info(pg_lsn)', 'EXECUTE'); -- yes
@@ -203,7 +202,7 @@ SELECT has_function_privilege('regress_pg_walinspect',
(1 row)
REVOKE pg_read_server_files FROM regress_pg_walinspect;
--- Superuser can grant execute to other users
+-- Superuser can grant execute to other users.
GRANT EXECUTE ON FUNCTION pg_get_wal_record_info(pg_lsn)
TO regress_pg_walinspect;
GRANT EXECUTE ON FUNCTION pg_get_wal_records_info(pg_lsn, pg_lsn)
diff --git a/contrib/pg_walinspect/sql/oldextversions.sql b/contrib/pg_walinspect/sql/oldextversions.sql
index 3037a193de..70ee4516ad 100644
--- a/contrib/pg_walinspect/sql/oldextversions.sql
+++ b/contrib/pg_walinspect/sql/oldextversions.sql
@@ -1,35 +1,35 @@
--- test old extension version entry points
+-- Test old extension version entry points.
CREATE EXTENSION pg_walinspect WITH VERSION '1.0';
--- Mask DETAIL messages as these could refer to current LSN positions
+-- Mask DETAIL messages as these could refer to current LSN positions.
\set VERBOSITY terse
--- List what version 1.0 contains
+-- List what version 1.0 contains.
\dx+ pg_walinspect
--- Make sure checkpoints don't interfere with the test
+-- Make sure checkpoints don't interfere with the test.
SELECT 'init' FROM pg_create_physical_replication_slot('regress_pg_walinspect_slot_old_ext_ver', true, false);
CREATE TABLE sample_tbl(col1 int, col2 int);
SELECT pg_current_wal_lsn() AS wal_lsn1 \gset
INSERT INTO sample_tbl SELECT * FROM generate_series(1, 2);
--- Tests for the past functions
+-- Tests for the past functions.
SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info_till_end_of_wal(:'wal_lsn1');
SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats_till_end_of_wal(:'wal_lsn1');
--- Failures with start LSNs
+-- Failure with highest possible start LSNs.
SELECT * FROM pg_get_wal_records_info_till_end_of_wal('FFFFFFFF/FFFFFFFF');
SELECT * FROM pg_get_wal_stats_till_end_of_wal('FFFFFFFF/FFFFFFFF');
--- Move to new version 1.1
+-- Move to new version 1.1.
ALTER EXTENSION pg_walinspect UPDATE TO '1.1';
--- List what version 1.1 contains
+-- List what version 1.1 contains.
\dx+ pg_walinspect
--- Clean up
+-- Clean up.
DROP EXTENSION pg_walinspect;
SELECT pg_drop_replication_slot('regress_pg_walinspect_slot_old_ext_ver');
diff --git a/contrib/pg_walinspect/sql/pg_walinspect.sql b/contrib/pg_walinspect/sql/pg_walinspect.sql
index 27ee1d90b5..863dc33710 100644
--- a/contrib/pg_walinspect/sql/pg_walinspect.sql
+++ b/contrib/pg_walinspect/sql/pg_walinspect.sql
@@ -7,8 +7,6 @@ CREATE EXTENSION pg_walinspect;
SELECT 'init' FROM pg_create_physical_replication_slot('regress_pg_walinspect_slot', true, false);
CREATE TABLE sample_tbl(col1 int, col2 int);
-
--- Save some LSNs for comparisons
SELECT pg_current_wal_lsn() AS wal_lsn1 \gset
INSERT INTO sample_tbl SELECT * FROM generate_series(1, 2);
SELECT pg_current_wal_lsn() AS wal_lsn2 \gset
@@ -31,25 +29,27 @@ SELECT * FROM pg_get_wal_records_info(:'wal_lsn2', :'wal_lsn1');
SELECT * FROM pg_get_wal_stats(:'wal_lsn2', :'wal_lsn1');
SELECT * FROM pg_get_wal_block_info(:'wal_lsn2', :'wal_lsn1');
--- LSNs with the highest value possible.
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_record_info('FFFFFFFF/FFFFFFFF');
--- successes with end LSNs
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_block_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
--- failures with start LSNs
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_block_info('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
+-- Failure with highest possible input LSN.
+SELECT * FROM pg_get_wal_record_info('FFFFFFFF/FFFFFFFF');
+
+-- Failures with highest possible start LSNs.
+SELECT * FROM pg_get_wal_records_info('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
+SELECT * FROM pg_get_wal_stats('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
+SELECT * FROM pg_get_wal_block_info('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
+
+-- Successes with highest possible end LSNs.
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
-- ===================================================================
-- Tests for all function executions
-- ===================================================================
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_block_info(:'wal_lsn1', :'wal_lsn2');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn1', :'wal_lsn2');
-- ===================================================================
-- Test for filtering out WAL records of a particular table
@@ -72,10 +72,11 @@ SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2'
-- Tests to get block information from WAL record
-- ===================================================================
--- Update table to generate some block data
+-- Update table to generate some block data.
SELECT pg_current_wal_lsn() AS wal_lsn3 \gset
UPDATE sample_tbl SET col1 = col1 + 1 WHERE col1 = 1;
SELECT pg_current_wal_lsn() AS wal_lsn4 \gset
+
-- Check if we get block data from WAL record.
SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn3', :'wal_lsn4')
WHERE relfilenode = :'sample_tbl_oid' AND blockdata IS NOT NULL;
@@ -85,6 +86,7 @@ SELECT pg_current_wal_lsn() AS wal_lsn5 \gset
CHECKPOINT;
UPDATE sample_tbl SET col1 = col1 + 1 WHERE col1 = 2;
SELECT pg_current_wal_lsn() AS wal_lsn6 \gset
+
-- Check if we get FPI from WAL record.
SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn5', :'wal_lsn6')
WHERE relfilenode = :'sample_tbl_oid' AND fpi IS NOT NULL;
@@ -103,9 +105,9 @@ SELECT has_function_privilege('regress_pg_walinspect',
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_block_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- no
--- Functions accessible by users with role pg_read_server_files
-
+-- Functions accessible by users with role pg_read_server_files.
GRANT pg_read_server_files TO regress_pg_walinspect;
+
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_record_info(pg_lsn)', 'EXECUTE'); -- yes
SELECT has_function_privilege('regress_pg_walinspect',
@@ -117,7 +119,7 @@ SELECT has_function_privilege('regress_pg_walinspect',
REVOKE pg_read_server_files FROM regress_pg_walinspect;
--- Superuser can grant execute to other users
+-- Superuser can grant execute to other users.
GRANT EXECUTE ON FUNCTION pg_get_wal_record_info(pg_lsn)
TO regress_pg_walinspect;
GRANT EXECUTE ON FUNCTION pg_get_wal_records_info(pg_lsn, pg_lsn)
--
2.34.1
Re: Combine pg_walinspect till_end_of_wal functions with others
Hi,
I just rebased a patch over
commit 1f282c24e46
Author: Michael Paquier <michael@paquier.xyz>
Date: 2023-03-13 13:03:29 +0900
Refactor and improve tests of pg_walinspect
and got a test failure:
https://cirrus-ci.com/task/5693041982308352
https://api.cirrus-ci.com/v1/artifact/task/5693041982308352/testrun/build/testrun/pg_walinspect/regress/regression.diffs
diff -w -U3 C:/cirrus/contrib/pg_walinspect/expected/oldextversions.out C:/cirrus/build/testrun/pg_walinspect/regress/results/oldextversions.out
--- C:/cirrus/contrib/pg_walinspect/expected/oldextversions.out 2023-03-14 21:19:01.399716500 +0000
+++ C:/cirrus/build/testrun/pg_walinspect/regress/results/oldextversions.out 2023-03-14 21:26:27.504876700 +0000
@@ -8,10 +8,10 @@
Object description
-----------------------------------------------------------
function pg_get_wal_record_info(pg_lsn)
- function pg_get_wal_records_info(pg_lsn,pg_lsn)
function pg_get_wal_records_info_till_end_of_wal(pg_lsn)
- function pg_get_wal_stats(pg_lsn,pg_lsn,boolean)
+ function pg_get_wal_records_info(pg_lsn,pg_lsn)
function pg_get_wal_stats_till_end_of_wal(pg_lsn,boolean)
+ function pg_get_wal_stats(pg_lsn,pg_lsn,boolean)
(5 rows)-- Make sure checkpoints don't interfere with the test.
Looks like it's missing an ORDER BY.
Greetings,
Andres Freund
Re: Combine pg_walinspect till_end_of_wal functions with others
On Tue, Mar 14, 2023 at 02:54:40PM -0700, Andres Freund wrote:
Object description ----------------------------------------------------------- function pg_get_wal_record_info(pg_lsn) - function pg_get_wal_records_info(pg_lsn,pg_lsn) function pg_get_wal_records_info_till_end_of_wal(pg_lsn) - function pg_get_wal_stats(pg_lsn,pg_lsn,boolean) + function pg_get_wal_records_info(pg_lsn,pg_lsn) function pg_get_wal_stats_till_end_of_wal(pg_lsn,boolean) + function pg_get_wal_stats(pg_lsn,pg_lsn,boolean) (5 rows)-- Make sure checkpoints don't interfere with the test.
Looks like it's missing an ORDER BY.
Interesting. This is "\dx+ pg_walinspect".
listOneExtensionContents() uses pg_describe_object() for that, and
there is already an ORDER BY based on it. I would not have expected
this part to be that much sensitive. Is this using a specific ICU
collation, because this is a side-effect of switching ICU as the
default in initdb?
As a solution, this could use pg_identify_object(classid, objid, 0) in
the ORDER BY clause to enforce a better ordering of the objects dealt
with as it decomposes the object name and the object type. That
should be enough, I assume, as it looks to be parenthesis vs
underscore that switch the order.
--
Michael
Re: Combine pg_walinspect till_end_of_wal functions with others
On Tue, Mar 14, 2023 at 10:35:43AM +0530, Bharath Rupireddy wrote:
My thoughts are simple here - how would one (an end user, not me and
not you) figure out how to get info/stats till the end of WAL? I'm
sure it would be difficult to find that out without looking at the
code or commit history. Search for till end of WAL behaviour with new
version will be more given the 1.0 version has explicit functions to
do that. IMO, there's no harm in being explicit in how to achieve till
end of WAL functionality around in the docs.
Okay. I have kept these notes, but tweaked the wording to be a bit
cleaner, replacing the term "till" by "until". To my surprise while
studying this point, "till" is a term older than "until" in English
literacy, but it is rarely used.
I get it. I divided the patches to 0001 and 0002 with 0001 focussing
on the change of behaviour around future end LSNs, dropping till end
of WAL functions and tests tweakings related to it. 0002 has all other
tests tidy up things.Please find the attached v8 patch set for further review.
The tests of 0001 were still too complex IMO. The changes can be much
simpler as it requires only to move the till_end_of_wal() calls from
pg_walinspect.sql to oldextversions.sql. Nothing more.
--
Michael
Re: Combine pg_walinspect till_end_of_wal functions with others
Hi,
On 2023-03-15 09:56:10 +0900, Michael Paquier wrote:
On Tue, Mar 14, 2023 at 02:54:40PM -0700, Andres Freund wrote:
Object description ----------------------------------------------------------- function pg_get_wal_record_info(pg_lsn) - function pg_get_wal_records_info(pg_lsn,pg_lsn) function pg_get_wal_records_info_till_end_of_wal(pg_lsn) - function pg_get_wal_stats(pg_lsn,pg_lsn,boolean) + function pg_get_wal_records_info(pg_lsn,pg_lsn) function pg_get_wal_stats_till_end_of_wal(pg_lsn,boolean) + function pg_get_wal_stats(pg_lsn,pg_lsn,boolean) (5 rows)-- Make sure checkpoints don't interfere with the test.
Looks like it's missing an ORDER BY.
Interesting. This is "\dx+ pg_walinspect".
listOneExtensionContents() uses pg_describe_object() for that, and
there is already an ORDER BY based on it. I would not have expected
this part to be that much sensitive. Is this using a specific ICU
collation, because this is a side-effect of switching ICU as the
default in initdb?
It's using ICU, but not a specific collation. The build I linked to is WIP
hackery to add ICU support to windows CI. Here's the initdb output:
https://api.cirrus-ci.com/v1/artifact/task/6288336663347200/testrun/build/testrun/pg_walinspect/regress/log/initdb.log
The database cluster will be initialized with this locale configuration:
provider: icu
ICU locale: en_US
LC_COLLATE: English_United States.1252
LC_CTYPE: English_United States.1252
LC_MESSAGES: English_United States.1252
LC_MONETARY: English_United States.1252
LC_NUMERIC: English_United States.1252
LC_TIME: English_United States.1252
The default database encoding has accordingly been set to "WIN1252".
The default text search configuration will be set to "english".
For comparison, here's a recent CI run (which also failed on windows, but for
unrelated reasons), without ICU:
https://api.cirrus-ci.com/v1/artifact/task/6478925920993280/testrun/build/testrun/pg_walinspect/regress/log/initdb.log
The database cluster will be initialized with locale "English_United States.1252".
The default database encoding has accordingly been set to "WIN1252".
The default text search configuration will be set to "english".
As a solution, this could use pg_identify_object(classid, objid, 0) in
the ORDER BY clause to enforce a better ordering of the objects dealt
with as it decomposes the object name and the object type. That
should be enough, I assume, as it looks to be parenthesis vs
underscore that switch the order.
Greetings,
Andres Freund
Re: Combine pg_walinspect till_end_of_wal functions with others
On Tue, Mar 14, 2023 at 07:05:20PM -0700, Andres Freund wrote:
It's using ICU, but not a specific collation. The build I linked to is WIP
hackery to add ICU support to windows CI. Here's the initdb output:
https://api.cirrus-ci.com/v1/artifact/task/6288336663347200/testrun/build/testrun/pg_walinspect/regress/log/initdb.log
Hmm. Thanks. At the end, I think that I would be tempted to just
remove this \dx query and move on. I did not anticipate that this
ordering would be that much sensitive, and the solution of using a
COLLATE C at the end of a describe.c query does not sound much
appealing, either.
--
Michael
Re: Combine pg_walinspect till_end_of_wal functions with others
On Wed, Mar 15, 2023 at 12:27 PM Michael Paquier <michael@paquier.xyz> wrote:
On Tue, Mar 14, 2023 at 07:05:20PM -0700, Andres Freund wrote:
It's using ICU, but not a specific collation. The build I linked to is WIP
hackery to add ICU support to windows CI. Here's the initdb output:
https://api.cirrus-ci.com/v1/artifact/task/6288336663347200/testrun/build/testrun/pg_walinspect/regress/log/initdb.logHmm. Thanks. At the end, I think that I would be tempted to just
remove this \dx query and move on. I did not anticipate that this
ordering would be that much sensitive, and the solution of using a
COLLATE C at the end of a describe.c query does not sound much
appealing, either.
-1 for removing \dx+ for pg_walinspect version 1.0, because we wanted
to show the diff of functions along with testing the upgrade path in
the oldextversions.sql. Therefore, I prefer something like [1]diff --git a/contrib/pg_walinspect/sql/oldextversions.sql b/contrib/pg_walinspect/sql/oldextversions.sql index 258a009888..32a059c72d 100644 --- a/contrib/pg_walinspect/sql/oldextversions.sql +++ b/contrib/pg_walinspect/sql/oldextversions.sql @@ -5,8 +5,17 @@ CREATE EXTENSION pg_walinspect WITH VERSION '1.0'; -- Mask DETAIL messages as these could refer to current LSN positions. \set VERBOSITY terse:
[1]
diff --git a/contrib/pg_walinspect/sql/oldextversions.sql
b/contrib/pg_walinspect/sql/oldextversions.sql
index 258a009888..32a059c72d 100644
--- a/contrib/pg_walinspect/sql/oldextversions.sql
+++ b/contrib/pg_walinspect/sql/oldextversions.sql
@@ -5,8 +5,17 @@ CREATE EXTENSION pg_walinspect WITH VERSION '1.0';
-- Mask DETAIL messages as these could refer to current LSN positions.
\set VERBOSITY terse+-- \dx+ will give locale-sensitive results, so we can't use it here.
+CREATE VIEW list_pg_walinspect_objects AS
+ SELECT pg_describe_object(classid, objid, 0) AS "Object description"
+ FROM pg_depend
+ WHERE refclassid = 'pg_extension'::regclass AND
+ refobjid = (SELECT oid FROM pg_extension WHERE
extname = 'pg_walinspect') AND
+ deptype = 'e'
+ ORDER BY pg_describe_object(classid, objid, 0) COLLATE "C";
+
-- List what version 1.0 contains
-\dx+ pg_walinspect
+SELECT * FROM list_pg_walinspect_objects; -- Make sure checkpoints don't interfere with the test.
SELECT 'init' FROM
pg_create_physical_replication_slot('regress_pg_walinspect_slot',
true, false);
@@ -25,7 +34,7 @@ SELECT COUNT(*) >= 1 AS ok FROM
pg_get_wal_stats_till_end_of_wal('FFFFFFFF/FFFFF
ALTER EXTENSION pg_walinspect UPDATE TO '1.1'; -- List what version 1.1 contains
-\dx+ pg_walinspect
+SELECT * FROM list_pg_walinspect_objects;SELECT pg_drop_replication_slot('regress_pg_walinspect_slot');
--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
Re: Combine pg_walinspect till_end_of_wal functions with others
On Wed, Mar 15, 2023 at 12:40:17PM +0530, Bharath Rupireddy wrote:
-1 for removing \dx+ for pg_walinspect version 1.0, because we wanted
to show the diff of functions along with testing the upgrade path in
the oldextversions.sql. Therefore, I prefer something like [1]:
This is a duplicate of what describe.c uses, with a COLLATE clause.
The main goal was to have a simple check, so I'd still stand by the
simplest choice and move on.
--
Michael
Re: Combine pg_walinspect till_end_of_wal functions with others
On Wed, Mar 15, 2023 at 06:50:01PM +0900, Michael Paquier wrote:
This is a duplicate of what describe.c uses, with a COLLATE clause.
The main goal was to have a simple check, so I'd still stand by the
simplest choice and move on.
Please note that I have done something about that with e643a31 by
replacing the problematic \dx with a SELECT query, but left the second
one as it should not be a problem.
--
Michael
Re: Combine pg_walinspect till_end_of_wal functions with others
On Thu, Mar 16, 2023 at 12:48 PM Michael Paquier <michael@paquier.xyz> wrote:
On Wed, Mar 15, 2023 at 06:50:01PM +0900, Michael Paquier wrote:
This is a duplicate of what describe.c uses, with a COLLATE clause.
The main goal was to have a simple check, so I'd still stand by the
simplest choice and move on.Please note that I have done something about that with e643a31 by
replacing the problematic \dx with a SELECT query, but left the second
one as it should not be a problem.
Thanks.
FWIW, I rebased the tests tweaking patch and attached it here as v9.
This should keep the pg_walinspect tests consistent across comments,
spaces, new lines and using count(*) >= 1 for all successful function
executions. Thoughts?
--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
Attachments:
v9-0001-Tweak-pg_walinspect-tests-to-be-consistent.patchapplication/octet-stream; name=v9-0001-Tweak-pg_walinspect-tests-to-be-consistent.patchDownload
From 08afdff06be1ff65682d73e487e09ddeb3f6592c Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Thu, 16 Mar 2023 07:44:42 +0000
Subject: [PATCH v9] Tweak pg_walinspect tests to be consistent
---
.../pg_walinspect/expected/oldextversions.out | 14 ++++---
.../pg_walinspect/expected/pg_walinspect.out | 35 ++++++++--------
contrib/pg_walinspect/sql/oldextversions.sql | 15 ++++---
contrib/pg_walinspect/sql/pg_walinspect.sql | 40 +++++++++----------
4 files changed, 54 insertions(+), 50 deletions(-)
diff --git a/contrib/pg_walinspect/expected/oldextversions.out b/contrib/pg_walinspect/expected/oldextversions.out
index da18914d92..a0bd51b780 100644
--- a/contrib/pg_walinspect/expected/oldextversions.out
+++ b/contrib/pg_walinspect/expected/oldextversions.out
@@ -1,4 +1,4 @@
--- test old extension version entry points
+-- Test old extension version entry points.
CREATE EXTENSION pg_walinspect WITH VERSION '1.0';
-- Mask DETAIL messages as these could refer to current LSN positions.
\set VERBOSITY terse
@@ -28,7 +28,7 @@ SELECT 'init' FROM pg_create_physical_replication_slot('regress_pg_walinspect_sl
CREATE TABLE sample_tbl(col1 int, col2 int);
SELECT pg_current_wal_lsn() AS wal_lsn1 \gset
INSERT INTO sample_tbl SELECT * FROM generate_series(1, 2);
--- Check bounds for these past functions.
+-- Tests for the past functions.
SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info_till_end_of_wal(:'wal_lsn1');
ok
----
@@ -41,13 +41,14 @@ SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats_till_end_of_wal(:'wal_lsn1');
t
(1 row)
-SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info_till_end_of_wal('FFFFFFFF/FFFFFFFF');
+-- Failure with highest possible start LSNs.
+SELECT * FROM pg_get_wal_records_info_till_end_of_wal('FFFFFFFF/FFFFFFFF');
ERROR: WAL start LSN must be less than current LSN
-SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats_till_end_of_wal('FFFFFFFF/FFFFFFFF');
+SELECT * FROM pg_get_wal_stats_till_end_of_wal('FFFFFFFF/FFFFFFFF');
ERROR: WAL start LSN must be less than current LSN
--- Move to new version 1.1
+-- Move to new version 1.1.
ALTER EXTENSION pg_walinspect UPDATE TO '1.1';
--- List what version 1.1 contains
+-- List what version 1.1 contains.
\dx+ pg_walinspect
Objects in extension "pg_walinspect"
Object description
@@ -58,6 +59,7 @@ ALTER EXTENSION pg_walinspect UPDATE TO '1.1';
function pg_get_wal_stats(pg_lsn,pg_lsn,boolean)
(4 rows)
+-- Clean up.
SELECT pg_drop_replication_slot('regress_pg_walinspect_slot');
pg_drop_replication_slot
--------------------------
diff --git a/contrib/pg_walinspect/expected/pg_walinspect.out b/contrib/pg_walinspect/expected/pg_walinspect.out
index e3b31f15e3..ec62728ee8 100644
--- a/contrib/pg_walinspect/expected/pg_walinspect.out
+++ b/contrib/pg_walinspect/expected/pg_walinspect.out
@@ -9,7 +9,6 @@ SELECT 'init' FROM pg_create_physical_replication_slot('regress_pg_walinspect_sl
(1 row)
CREATE TABLE sample_tbl(col1 int, col2 int);
--- Save some LSNs for comparisons
SELECT pg_current_wal_lsn() AS wal_lsn1 \gset
INSERT INTO sample_tbl SELECT * FROM generate_series(1, 2);
SELECT pg_current_wal_lsn() AS wal_lsn2 \gset
@@ -34,57 +33,57 @@ SELECT * FROM pg_get_wal_stats(:'wal_lsn2', :'wal_lsn1');
ERROR: WAL start LSN must be less than end LSN
SELECT * FROM pg_get_wal_block_info(:'wal_lsn2', :'wal_lsn1');
ERROR: WAL start LSN must be less than end LSN
--- LSNs with the highest value possible.
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_record_info('FFFFFFFF/FFFFFFFF');
+-- Failure with highest possible input LSN.
+SELECT * FROM pg_get_wal_record_info('FFFFFFFF/FFFFFFFF');
ERROR: WAL input LSN must be less than current LSN
--- Success with end LSNs.
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
+-- Successes with highest possible end LSNs.
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_block_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
ok
----
t
(1 row)
--- failures with start LSNs
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
+-- Failures with highest possible start LSNs.
+SELECT * FROM pg_get_wal_records_info('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
ERROR: WAL start LSN must be less than current LSN
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
+SELECT * FROM pg_get_wal_stats('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
ERROR: WAL start LSN must be less than current LSN
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_block_info('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
+SELECT * FROM pg_get_wal_block_info('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
ERROR: WAL start LSN must be less than current LSN
-- ===================================================================
-- Tests for all function executions
-- ===================================================================
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_block_info(:'wal_lsn1', :'wal_lsn2');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn1', :'wal_lsn2');
ok
----
t
@@ -115,7 +114,7 @@ SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2'
-- ===================================================================
-- Tests to get block information from WAL record
-- ===================================================================
--- Update table to generate some block data
+-- Update table to generate some block data.
SELECT pg_current_wal_lsn() AS wal_lsn3 \gset
UPDATE sample_tbl SET col1 = col1 + 1 WHERE col1 = 1;
SELECT pg_current_wal_lsn() AS wal_lsn4 \gset
@@ -172,7 +171,7 @@ SELECT has_function_privilege('regress_pg_walinspect',
f
(1 row)
--- Functions accessible by users with role pg_read_server_files
+-- Functions accessible by users with role pg_read_server_files.
GRANT pg_read_server_files TO regress_pg_walinspect;
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_record_info(pg_lsn)', 'EXECUTE'); -- yes
@@ -203,7 +202,7 @@ SELECT has_function_privilege('regress_pg_walinspect',
(1 row)
REVOKE pg_read_server_files FROM regress_pg_walinspect;
--- Superuser can grant execute to other users
+-- Superuser can grant execute to other users.
GRANT EXECUTE ON FUNCTION pg_get_wal_record_info(pg_lsn)
TO regress_pg_walinspect;
GRANT EXECUTE ON FUNCTION pg_get_wal_records_info(pg_lsn, pg_lsn)
diff --git a/contrib/pg_walinspect/sql/oldextversions.sql b/contrib/pg_walinspect/sql/oldextversions.sql
index 1f8307816e..c61fb1fe71 100644
--- a/contrib/pg_walinspect/sql/oldextversions.sql
+++ b/contrib/pg_walinspect/sql/oldextversions.sql
@@ -1,4 +1,4 @@
--- test old extension version entry points
+-- Test old extension version entry points.
CREATE EXTENSION pg_walinspect WITH VERSION '1.0';
@@ -20,18 +20,21 @@ CREATE TABLE sample_tbl(col1 int, col2 int);
SELECT pg_current_wal_lsn() AS wal_lsn1 \gset
INSERT INTO sample_tbl SELECT * FROM generate_series(1, 2);
--- Check bounds for these past functions.
+-- Tests for the past functions.
SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info_till_end_of_wal(:'wal_lsn1');
SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats_till_end_of_wal(:'wal_lsn1');
-SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info_till_end_of_wal('FFFFFFFF/FFFFFFFF');
-SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats_till_end_of_wal('FFFFFFFF/FFFFFFFF');
--- Move to new version 1.1
+-- Failure with highest possible start LSNs.
+SELECT * FROM pg_get_wal_records_info_till_end_of_wal('FFFFFFFF/FFFFFFFF');
+SELECT * FROM pg_get_wal_stats_till_end_of_wal('FFFFFFFF/FFFFFFFF');
+
+-- Move to new version 1.1.
ALTER EXTENSION pg_walinspect UPDATE TO '1.1';
--- List what version 1.1 contains
+-- List what version 1.1 contains.
\dx+ pg_walinspect
+-- Clean up.
SELECT pg_drop_replication_slot('regress_pg_walinspect_slot');
DROP TABLE sample_tbl;
diff --git a/contrib/pg_walinspect/sql/pg_walinspect.sql b/contrib/pg_walinspect/sql/pg_walinspect.sql
index ba4d17df6c..6d0b637354 100644
--- a/contrib/pg_walinspect/sql/pg_walinspect.sql
+++ b/contrib/pg_walinspect/sql/pg_walinspect.sql
@@ -7,8 +7,6 @@ CREATE EXTENSION pg_walinspect;
SELECT 'init' FROM pg_create_physical_replication_slot('regress_pg_walinspect_slot', true, false);
CREATE TABLE sample_tbl(col1 int, col2 int);
-
--- Save some LSNs for comparisons
SELECT pg_current_wal_lsn() AS wal_lsn1 \gset
INSERT INTO sample_tbl SELECT * FROM generate_series(1, 2);
SELECT pg_current_wal_lsn() AS wal_lsn2 \gset
@@ -31,25 +29,27 @@ SELECT * FROM pg_get_wal_records_info(:'wal_lsn2', :'wal_lsn1');
SELECT * FROM pg_get_wal_stats(:'wal_lsn2', :'wal_lsn1');
SELECT * FROM pg_get_wal_block_info(:'wal_lsn2', :'wal_lsn1');
--- LSNs with the highest value possible.
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_record_info('FFFFFFFF/FFFFFFFF');
--- Success with end LSNs.
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_block_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
--- failures with start LSNs
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_block_info('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
+-- Failure with highest possible input LSN.
+SELECT * FROM pg_get_wal_record_info('FFFFFFFF/FFFFFFFF');
+
+-- Successes with highest possible end LSNs.
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn1', 'FFFFFFFF/FFFFFFFF');
+
+-- Failures with highest possible start LSNs.
+SELECT * FROM pg_get_wal_records_info('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
+SELECT * FROM pg_get_wal_stats('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
+SELECT * FROM pg_get_wal_block_info('FFFFFFFF/FFFFFFFE', 'FFFFFFFF/FFFFFFFF');
-- ===================================================================
-- Tests for all function executions
-- ===================================================================
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_block_info(:'wal_lsn1', :'wal_lsn2');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn1', :'wal_lsn2');
-- ===================================================================
-- Test for filtering out WAL records of a particular table
@@ -72,7 +72,7 @@ SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2'
-- Tests to get block information from WAL record
-- ===================================================================
--- Update table to generate some block data
+-- Update table to generate some block data.
SELECT pg_current_wal_lsn() AS wal_lsn3 \gset
UPDATE sample_tbl SET col1 = col1 + 1 WHERE col1 = 1;
SELECT pg_current_wal_lsn() AS wal_lsn4 \gset
@@ -103,9 +103,9 @@ SELECT has_function_privilege('regress_pg_walinspect',
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_block_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- no
--- Functions accessible by users with role pg_read_server_files
-
+-- Functions accessible by users with role pg_read_server_files.
GRANT pg_read_server_files TO regress_pg_walinspect;
+
SELECT has_function_privilege('regress_pg_walinspect',
'pg_get_wal_record_info(pg_lsn)', 'EXECUTE'); -- yes
SELECT has_function_privilege('regress_pg_walinspect',
@@ -117,7 +117,7 @@ SELECT has_function_privilege('regress_pg_walinspect',
REVOKE pg_read_server_files FROM regress_pg_walinspect;
--- Superuser can grant execute to other users
+-- Superuser can grant execute to other users.
GRANT EXECUTE ON FUNCTION pg_get_wal_record_info(pg_lsn)
TO regress_pg_walinspect;
GRANT EXECUTE ON FUNCTION pg_get_wal_records_info(pg_lsn, pg_lsn)
--
2.34.1
Re: Combine pg_walinspect till_end_of_wal functions with others
On Thu, Mar 16, 2023 at 01:17:59PM +0530, Bharath Rupireddy wrote:
FWIW, I rebased the tests tweaking patch and attached it here as v9.
This should keep the pg_walinspect tests consistent across comments,
spaces, new lines and using count(*) >= 1 for all successful function
executions. Thoughts?
Mostly OK by me, so applied after tweaking a few tiny things. The
rewrites of the queries where we should have more than one record and
the removal of count() for the failure cases have been kept as
proposed, as are most of the comments.
--
Michael