From 592ba15c35bb16e55b0bb0a7e7bdbb6dd4e08a0b Mon Sep 17 00:00:00 2001 From: James Coleman Date: Sun, 8 Mar 2020 16:39:45 -0400 Subject: [PATCH v3] Improve pg_rewind explanation and warnings The pg_rewind docs currently assert that the state of the target's data directory after rewind is equivalent to the source's data directory. But that isn't quite true both because the base state is further back in time and because the target's data directory will include the current state on the source of any copied blocks. Additionally the state isn't equal to a copy of the source data directory; it's equivalent to a base backup of the source. The How It Works section now: - Includes details about how the backup label file is created. - Is updated to include WAL segments and new relation files in the list of files copied wholesale from the source. Finally, document clearly the state of the cluster after the operation and also the operation sequencing dangers caused by copying configuration files from the source. --- doc/src/sgml/ref/pg_rewind.sgml | 87 ++++++++++++++++++++------------- 1 file changed, 54 insertions(+), 33 deletions(-) diff --git a/doc/src/sgml/ref/pg_rewind.sgml b/doc/src/sgml/ref/pg_rewind.sgml index 42d29edd4e..bc6f0009cc 100644 --- a/doc/src/sgml/ref/pg_rewind.sgml +++ b/doc/src/sgml/ref/pg_rewind.sgml @@ -48,14 +48,16 @@ PostgreSQL documentation - The result is equivalent to replacing the target data directory with the - source one. Only changed blocks from relation files are copied; - all other files are copied in full, including configuration files. The - advantage of pg_rewind over taking a new base backup, or - tools like rsync, is that pg_rewind does - not require reading through unchanged blocks in the cluster. This makes - it a lot faster when the database is large and only a small - fraction of blocks differ between the clusters. + After a successful rewind and subsequent WAL replay, the target data + directory is equivalent to a base backup of the source data directory. While + only changed blocks from existing relation files are copied; all other files + are copied in full, including new relation files, configuration files, and WAL + segments. The advantage of pg_rewind over taking a + new base backup, or tools like rsync, is that + pg_rewind does not require comparing or copying + unchanged relation blocks in the cluster. As such the rewind operation is + significantly faster than other approaches when the database is large and + only a small fraction of blocks differ between the clusters. @@ -77,16 +79,18 @@ PostgreSQL documentation - When the target server is started for the first time after running - pg_rewind, it will go into recovery mode and replay all - WAL generated in the source server after the point of divergence. - If some of the WAL was no longer available in the source server when - pg_rewind was run, and therefore could not be copied by the - pg_rewind session, it must be made available when the - target server is started. This can be done by creating a - recovery.signal file in the target data directory - and configuring suitable - in postgresql.conf. + After running pg_rewind the data directory is + not immediately in a consistent state. However + pg_rewind configures the control file so that when + the target server is started again it will enter recovery mode and replay all + WAL generated in the source server after the point of divergence. If some of + the WAL was no longer available in the source server when + pg_rewind was run, and therefore could not be + copied by the pg_rewind session, it must be made + available when the target server is started. This can be done by creating a + recovery.signal file in the target data directory and + configuring suitable in + postgresql.conf. @@ -105,6 +109,15 @@ PostgreSQL documentation recovered. In such a case, taking a new fresh backup is recommended. + + Because pg_rewind copies configuration files + entirely from the source, correcting recovery configuration options before + restarting the server is necessary if you intend to re-introduce the target + as a replica of the source. If you restart the server after the rewind + operation has finished but without configuring recovery, the target will + again diverge from the primary. + + pg_rewind will fail immediately if it finds files it cannot write directly to. This can happen for example when @@ -326,34 +339,42 @@ GRANT EXECUTE ON function pg_catalog.pg_read_binary_file(text, bigint, bigint, b Copy all those changed blocks from the source cluster to the target cluster, either using direct file system access () or SQL (). + The relation files are now to their state at the last checkpoint completed + prior to the point at which the WAL timelines of the source and target + diverged plus the current state on the source of any blocks changed on the + target after that divergence. - Copy all other files such as pg_xact and - configuration files from the source cluster to the target cluster - (everything except the relation files). Similarly to base backups, - the contents of the directories pg_dynshmem/, + Copy all other files, including new relation files, WAL segments, + pg_xact, and configuration files from the source + cluster to the target cluster. Similarly to base backups, the contents + of the directories pg_dynshmem/, pg_notify/, pg_replslot/, pg_serial/, pg_snapshots/, - pg_stat_tmp/, and - pg_subtrans/ are omitted from the data copied - from the source cluster. Any file or directory beginning with - pgsql_tmp is omitted, as well as are + pg_stat_tmp/, and pg_subtrans/ + are omitted from the data copied from the source cluster. The files backup_label, tablespace_map, pg_internal.init, - postmaster.opts and - postmaster.pid. + postmaster.opts, and + postmaster.pid, as well as any file or directory + beginning with pgsql_tmp, are omitted. + + + + + Create a backup label file to begin WAL replay at the checkpoint created + at failover and a minimum consistency LSN using + pg_current_wal_insert_lsn(), when using a live source + and the last checkpoint LSN, when using a stopped source. - Apply the WAL from the source cluster, starting from the checkpoint - created at failover. (Strictly speaking, pg_rewind - doesn't apply the WAL, it just creates a backup label file that - makes PostgreSQL start by replaying all WAL from - that checkpoint forward.) + On restart, PostgreSQL replays the required WAL + resulting in a consistent data directory state. base-commit: 691e8b2e1889d61df47ae76601fa9db6cbac6f1c -- 2.17.1