Using base backup exclusion filters to reduce data transferred with pg_rewind
Hi all,
Many threads have touched $subject:
/messages/by-id/CAN-RpxDPE4baiMMJ6TLd6AiUvrG=YrC05tGxrgp4aUutH9j5TQ@mail.gmail.com
/messages/by-id/7c50423.5ad0.15e8b308b2f.Coremail.chjischj@163.com
/messages/by-id/1516736993.5599.4.camel@cybertec.at
Thread [2] is a bit different as it discusses with WAL segments data
which is useless at the end, still it aimed at reducing the amount of
data transferred during a rewind. I am not tackling this problem in the
patch set of this thread. This can be discussed separately.
Attached is a patch set which implements what I have mentioned a couple
of times in those threads by having pg_rewind reuse the same exclusion
filtering rules as for base backups, as after a rewind a node enters in
recovery and a bunch of data which is copied during the rewind finishes
in the void. There has been as well many complains about the need to
remove all the time replication slot data manually after a rewind, so I
believe that this makes the user experience much easier with the tool.
Something useful is replication slot data getting filtered out.
In order to reach this point, I have been hacking the backend code and
finished with quite a bit of shuffling around how system paths are
hardcoded in many places, like pg_replslot, pg_wal, etc. Well you can
think here about all the paths hardcoded in initdb.c. So I have
introduced a couple of things:
- src/include/pg_paths.h, a new header which gathers the set of system
file and directory names. With this facility in place, the backend code
loses knowledge of hardcoded system-related paths, including things like
"base", "global", "pg_tblspc", "base/pg_control", etc. A fun
consequence of that refactoring is that it is possible to just change
pg_paths.h and have change the system paths of a PostgreSQL instance
with one-liners. For example you could change "base/" to "foo/". This
can make PostgreSQL more malleable for forks. It would be more simple
to just hardcode more the paths but I think that this would not be
manageable in the long-term, especially if similar logics spread more.
- src/include/replication/basebackup_paths.h, which extracts the exclude
rules now in basebackup.c into a header which can be consumed by both
frontends and backends. This is useful for any backup tools.
- pg_rewind update to ensure that the filters are working correctly.
So the patch set attached is made of the following:
- 0001, which refactors all hardcoded system paths into pg_paths.h.
This modifies only initdb.c and basebackup.c to ease reviews.
- 0002 spreads the path changes and the use of pg_paths.h across the
core code.
- 0003 moves the last set of definitions with backup_label,
tablespace_map and pg_internal.init.
- 0004 creates basebackup_paths.h, this can be consumed by pg_rewind.
- 0005 makes the changes for pg_rewind.
0001~0003 can be merged together, I have just done a split to ease
reviews.
I am adding that to the next CF.
Thanks,
--
Michael
Attachments:
0001-Refactor-path-definitions-into-a-single-header-file-.patchtext/plain; charset=us-asciiDownload
From 6698d430ffb6bd2e33aba0b21dc2a9358cf6328c Mon Sep 17 00:00:00 2001
From: Michael Paquier <michael@paquier.xyz>
Date: Mon, 5 Feb 2018 13:03:38 +0900
Subject: [PATCH 1/5] Refactor path definitions into a single header file,
pg_paths.h
The definition of all internal paths of PostgreSQL are spread across the
code base, making tracing of those definitions difficult to begin with,
and resulting in a lot of code paths to be hardcoded. While this has no
real practical consequences in practice, this makes the creation of
lists manipulating those paths less maintainable as as things stand the
already-hardcoded paths need to be copied around more. In order to
improve things for the long-term, move all those definitions into a
single header file.
This commit does a first step by switching basebackup.c and initdb.c to
use them. An upcoming commit will make all the definitions across the
code base use this new header more consistently.
---
src/backend/postmaster/postmaster.c | 1 +
src/backend/postmaster/syslogger.c | 1 +
src/backend/replication/basebackup.c | 16 ++--
src/backend/storage/ipc/dsm.c | 1 +
src/backend/storage/ipc/dsm_impl.c | 1 +
src/backend/storage/lmgr/predicate.c | 3 +-
src/backend/utils/adt/misc.c | 1 +
src/bin/initdb/initdb.c | 45 ++++++------
src/include/access/xlog_internal.h | 7 +-
src/include/pg_paths.h | 137 +++++++++++++++++++++++++++++++++++
src/include/pgstat.h | 3 -
src/include/postmaster/syslogger.h | 7 --
src/include/storage/dsm_impl.h | 9 ---
src/include/utils/guc.h | 7 --
14 files changed, 176 insertions(+), 63 deletions(-)
create mode 100644 src/include/pg_paths.h
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index f3ddf828bb..66d80914a0 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -105,6 +105,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pg_getopt.h"
+#include "pg_paths.h"
#include "pgstat.h"
#include "port/pg_bswap.h"
#include "postmaster/autovacuum.h"
diff --git a/src/backend/postmaster/syslogger.c b/src/backend/postmaster/syslogger.c
index f70eea37df..c8770f6c61 100644
--- a/src/backend/postmaster/syslogger.c
+++ b/src/backend/postmaster/syslogger.c
@@ -35,6 +35,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "nodes/pg_list.h"
+#include "pg_paths.h"
#include "pgstat.h"
#include "pgtime.h"
#include "postmaster/fork_process.h"
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index dd7ad64862..63ab81f837 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -117,25 +117,25 @@ static const char *excludeDirContents[] =
* even if the intention is to restore to another master. See backup.sgml
* for a more detailed description.
*/
- "pg_replslot",
+ PG_REPLSLOT_DIR,
/* Contents removed on startup, see dsm_cleanup_for_mmap(). */
PG_DYNSHMEM_DIR,
/* Contents removed on startup, see AsyncShmemInit(). */
- "pg_notify",
+ PG_NOTIFY_DIR,
/*
* Old contents are loaded for possible debugging but are not required for
* normal operation, see OldSerXidInit().
*/
- "pg_serial",
+ PG_SERIAL_DIR,
/* Contents removed on startup, see DeleteAllExportedSnapshotFiles(). */
- "pg_snapshots",
+ PG_SNAPSHOTS_DIR,
/* Contents zeroed on startup, see StartupSUBTRANS(). */
- "pg_subtrans",
+ PG_SUBTRANS_DIR,
/* end of list */
NULL
@@ -147,7 +147,7 @@ static const char *excludeDirContents[] =
static const char *excludeFiles[] =
{
/* Skip auto conf temporary file. */
- PG_AUTOCONF_FILENAME ".tmp",
+ PG_AUTOCONF_FILENAME_TMP,
/* Skip current log file temporary file */
LOG_METAINFO_DATAFILE_TMP,
@@ -164,8 +164,8 @@ static const char *excludeFiles[] =
BACKUP_LABEL_FILE,
TABLESPACE_MAP,
- "postmaster.pid",
- "postmaster.opts",
+ POSTMASTER_PID_FILE,
+ POSTMASTER_OPTS_FILE,
/* end of list */
NULL
diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c
index f1f75b73f5..a0d69aa0d8 100644
--- a/src/backend/storage/ipc/dsm.c
+++ b/src/backend/storage/ipc/dsm.c
@@ -35,6 +35,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pg_paths.h"
#include "storage/dsm.h"
#include "storage/ipc.h"
#include "storage/lwlock.h"
diff --git a/src/backend/storage/ipc/dsm_impl.c b/src/backend/storage/ipc/dsm_impl.c
index 67e76b98fe..a2bb1e2288 100644
--- a/src/backend/storage/ipc/dsm_impl.c
+++ b/src/backend/storage/ipc/dsm_impl.c
@@ -62,6 +62,7 @@
#endif
#include "pgstat.h"
+#include "pg_paths.h"
#include "portability/mem.h"
#include "storage/dsm_impl.h"
#include "storage/fd.h"
diff --git a/src/backend/storage/lmgr/predicate.c b/src/backend/storage/lmgr/predicate.c
index d1ff2b1edc..731ed66251 100644
--- a/src/backend/storage/lmgr/predicate.c
+++ b/src/backend/storage/lmgr/predicate.c
@@ -194,6 +194,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "miscadmin.h"
+#include "pg_paths.h"
#include "pgstat.h"
#include "storage/bufmgr.h"
#include "storage/predicate.h"
@@ -806,7 +807,7 @@ OldSerXidInit(void)
*/
OldSerXidSlruCtl->PagePrecedes = OldSerXidPagePrecedesLogically;
SimpleLruInit(OldSerXidSlruCtl, "oldserxid",
- NUM_OLDSERXID_BUFFERS, 0, OldSerXidLock, "pg_serial",
+ NUM_OLDSERXID_BUFFERS, 0, OldSerXidLock, PG_SERIAL_DIR,
LWTRANCHE_OLDSERXID_BUFFERS);
/* Override default assumption that writes should be fsync'd */
OldSerXidSlruCtl->do_fsync = false;
diff --git a/src/backend/utils/adt/misc.c b/src/backend/utils/adt/misc.c
index 2e1e020c4b..9f98c03173 100644
--- a/src/backend/utils/adt/misc.c
+++ b/src/backend/utils/adt/misc.c
@@ -30,6 +30,7 @@
#include "common/keywords.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "pg_paths.h"
#include "pgstat.h"
#include "parser/scansup.h"
#include "postmaster/syslogger.h"
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index 2efd3b75b1..528fdf7f57 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -72,6 +72,7 @@
#include "getopt_long.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
+#include "pg_paths.h"
/* Ideally this would be in a .h file, but it hardly seems worth the trouble */
@@ -199,28 +200,28 @@ static const char *boot_options = "-F";
static const char *backend_options = "--single -F -O -j -c search_path=pg_catalog -c exit_on_error=true";
static const char *const subdirs[] = {
- "global",
- "pg_wal/archive_status",
- "pg_commit_ts",
- "pg_dynshmem",
- "pg_notify",
- "pg_serial",
- "pg_snapshots",
- "pg_subtrans",
- "pg_twophase",
- "pg_multixact",
- "pg_multixact/members",
- "pg_multixact/offsets",
- "base",
- "base/1",
- "pg_replslot",
- "pg_tblspc",
- "pg_stat",
- "pg_stat_tmp",
- "pg_xact",
- "pg_logical",
- "pg_logical/snapshots",
- "pg_logical/mappings"
+ PG_GLOBAL_DIR,
+ XLOG_ARCHIVE_STATUS_DIR,
+ PG_COMMIT_TS_DIR,
+ PG_DYNSHMEM_DIR,
+ PG_NOTIFY_DIR,
+ PG_SERIAL_DIR,
+ PG_SNAPSHOTS_DIR,
+ PG_SUBTRANS_DIR,
+ PG_TWOPHASE_DIR,
+ PG_MULTIXACT_DIR,
+ PG_MULTIXACT_MEMBERS_DIR,
+ PG_MULTIXACT_OFFSETS_DIR,
+ PG_BASE_DIR,
+ PG_BASE_DIR "/1",
+ PG_REPLSLOT_DIR,
+ PG_TABLESPACE_DIR,
+ PG_STAT_DIR,
+ PG_STAT_TMP_DIR,
+ PG_XACT_DIR,
+ PG_LOGICAL_DIR,
+ PG_LOGICAL_SNAPSHOTS_DIR,
+ PG_LOGICAL_MAPPINGS_DIR
};
diff --git a/src/include/access/xlog_internal.h b/src/include/access/xlog_internal.h
index a5c074642f..4ab8be61b5 100644
--- a/src/include/access/xlog_internal.h
+++ b/src/include/access/xlog_internal.h
@@ -23,6 +23,7 @@
#include "access/xlogreader.h"
#include "datatype/timestamp.h"
#include "lib/stringinfo.h"
+#include "pg_paths.h"
#include "pgtime.h"
#include "storage/block.h"
#include "storage/relfilenode.h"
@@ -137,12 +138,6 @@ typedef XLogLongPageHeaderData *XLogLongPageHeader;
#define XRecOffIsValid(xlrp) \
((xlrp) % XLOG_BLCKSZ >= SizeOfXLogShortPHD)
-/*
- * The XLog directory and control file (relative to $PGDATA)
- */
-#define XLOGDIR "pg_wal"
-#define XLOG_CONTROL_FILE "global/pg_control"
-
/*
* These macros encapsulate knowledge about the exact layout of XLog file
* names, timeline history file names, and archive-status file names.
diff --git a/src/include/pg_paths.h b/src/include/pg_paths.h
new file mode 100644
index 0000000000..a4746e75e4
--- /dev/null
+++ b/src/include/pg_paths.h
@@ -0,0 +1,137 @@
+/* ----------
+ * pg_paths.h
+ *
+ * Definitions for paths internal to PostgreSQL instances. This can
+ * be used by both frontend and backend.
+ *
+ * Copyright (c) 2018, PostgreSQL Global Development Group
+ *
+ * src/include/pg_paths.h
+ * ----------
+ */
+
+#ifndef PG_PATHS_H
+#define PG_PATHS_H
+
+/*
+ * Name of files saving meta-data information about the log
+ * files currently in use by the syslogger
+ */
+#define LOG_METAINFO_DATAFILE "current_logfiles"
+#define LOG_METAINFO_DATAFILE_TMP LOG_METAINFO_DATAFILE ".tmp"
+
+/*
+ * A file recording the command-line options the server was
+ * last started with.
+ */
+#define POSTMASTER_OPTS_FILE "postmaster.opts"
+
+/*
+ * Lock file recording data about the current postmaster process, like
+ * its PID.
+ */
+#define POSTMASTER_PID_FILE "postmaster.pid"
+
+/*
+ * Automatic configuration file name for ALTER SYSTEM.
+ * This file will be used to store values of configuration parameters
+ * set by ALTER SYSTEM command.
+ */
+#define PG_AUTOCONF_FILENAME "postgresql.auto.conf"
+#define PG_AUTOCONF_FILENAME_TMP PG_AUTOCONF_FILENAME ".tmp"
+
+/*
+ * Directory containing transaction commit timestamp data.
+ */
+#define PG_COMMIT_TS_DIR "pg_commit_ts"
+
+/*
+ * Directory for on-disk state of dynamic shared memory segments.
+ *
+ * This is used by all implementations for crash recovery and by the mmap
+ * implementation for storage.
+ */
+#define PG_DYNSHMEM_DIR "pg_dynshmem"
+#define PG_DYNSHMEM_MMAP_FILE_PREFIX "mmap."
+
+/*
+ * Directory containing cluster-wide tables.
+ */
+#define PG_GLOBAL_DIR "global"
+
+/*
+ * Directory containing per-database subdirectories
+ */
+#define PG_BASE_DIR "base"
+
+/*
+ * WAL directory, archive status directory and control file.
+ */
+#define XLOGDIR "pg_wal"
+#define XLOG_ARCHIVE_STATUS_DIR XLOGDIR "/archive_status"
+#define XLOG_CONTROL_FILE PG_GLOBAL_DIR "/pg_control"
+
+/*
+ * Directory containing status data for logical decoding.
+ */
+#define PG_LOGICAL_DIR "pg_logical"
+#define PG_LOGICAL_MAPPINGS_DIR PG_LOGICAL_DIR "/mappings"
+#define PG_LOGICAL_SNAPSHOTS_DIR PG_LOGICAL_DIR "/snapshots"
+
+/*
+ * Directories containing multitransaction status data (used for shared
+ * row locks).
+ */
+#define PG_MULTIXACT_DIR "pg_multixact"
+#define PG_MULTIXACT_MEMBERS_DIR PG_MULTIXACT_DIR "/members"
+#define PG_MULTIXACT_OFFSETS_DIR PG_MULTIXACT_DIR "/offsets"
+
+/*
+ * Directory containing LISTEN/NOTIFY status data.
+ */
+#define PG_NOTIFY_DIR "pg_notify"
+
+/*
+ * Directory containing replication slot data.
+ */
+#define PG_REPLSLOT_DIR "pg_replslot"
+
+/*
+ * Default directories to store permanent and temporary statistics
+ * data in. Used by the statistics collector.
+ */
+#define PG_STAT_DIR "pg_stat"
+#define PG_STAT_TMP_DIR "pg_stat_tmp"
+
+/*
+ * Directory containing information about committed serializable
+ * transactions.
+ */
+#define PG_SERIAL_DIR "pg_serial"
+
+/*
+ * Directory containing exported snapshots dara.
+ */
+#define PG_SNAPSHOTS_DIR "pg_snapshots"
+
+/*
+ * Directory containing subtransaction status data.
+ */
+#define PG_SUBTRANS_DIR "pg_subtrans"
+
+/*
+ * Directory containing symbolic links to tablespaces.
+ */
+#define PG_TABLESPACE_DIR "pg_tblspc"
+
+/*
+ * Directory containing state files for prepared transactions.
+ */
+#define PG_TWOPHASE_DIR "pg_twophase"
+
+/*
+ * Directory containing transaction commit status data.
+ */
+#define PG_XACT_DIR "pg_xact"
+
+#endif /* PG_PATHS_H */
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index be2f59239b..7ae4b9097e 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -30,9 +30,6 @@
#define PGSTAT_STAT_PERMANENT_FILENAME "pg_stat/global.stat"
#define PGSTAT_STAT_PERMANENT_TMPFILE "pg_stat/global.tmp"
-/* Default directory to store temporary statistics data in */
-#define PG_STAT_TMP_DIR "pg_stat_tmp"
-
/* Values for track_functions GUC variable --- order is significant! */
typedef enum TrackFunctionsLevel
{
diff --git a/src/include/postmaster/syslogger.h b/src/include/postmaster/syslogger.h
index b35fadc1bd..00cc7a3023 100644
--- a/src/include/postmaster/syslogger.h
+++ b/src/include/postmaster/syslogger.h
@@ -87,11 +87,4 @@ extern void write_syslogger_file(const char *buffer, int count, int dest);
extern void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
#endif
-/*
- * Name of files saving meta-data information about the log
- * files currently in use by the syslogger
- */
-#define LOG_METAINFO_DATAFILE "current_logfiles"
-#define LOG_METAINFO_DATAFILE_TMP LOG_METAINFO_DATAFILE ".tmp"
-
#endif /* _SYSLOGGER_H */
diff --git a/src/include/storage/dsm_impl.h b/src/include/storage/dsm_impl.h
index 0e5730f7c5..f13b46b6c3 100644
--- a/src/include/storage/dsm_impl.h
+++ b/src/include/storage/dsm_impl.h
@@ -42,15 +42,6 @@
/* GUC. */
extern int dynamic_shared_memory_type;
-/*
- * Directory for on-disk state.
- *
- * This is used by all implementations for crash recovery and by the mmap
- * implementation for storage.
- */
-#define PG_DYNSHMEM_DIR "pg_dynshmem"
-#define PG_DYNSHMEM_MMAP_FILE_PREFIX "mmap."
-
/* A "name" for a dynamic shared memory segment. */
typedef uint32 dsm_handle;
diff --git a/src/include/utils/guc.h b/src/include/utils/guc.h
index 77daa5a539..8bb5ba9198 100644
--- a/src/include/utils/guc.h
+++ b/src/include/utils/guc.h
@@ -26,13 +26,6 @@
#define MAX_KILOBYTES (INT_MAX / 1024)
#endif
-/*
- * Automatic configuration file name for ALTER SYSTEM.
- * This file will be used to store values of configuration parameters
- * set by ALTER SYSTEM command.
- */
-#define PG_AUTOCONF_FILENAME "postgresql.auto.conf"
-
/*
* Certain options can only be set at certain times. The rules are
* like this:
--
2.16.1
0002-Replace-all-system-paths-hardcoded-with-data-from-pg.patchtext/plain; charset=us-asciiDownload
From 18e643219da62fae62e312a840aa35873f73bcc9 Mon Sep 17 00:00:00 2001
From: Michael Paquier <michael@paquier.xyz>
Date: Mon, 5 Feb 2018 14:28:07 +0900
Subject: [PATCH 2/5] Replace all system paths hardcoded with data from
pg_paths.h
This unifies the interface where system paths are defined and located,
making it easier to use such paths in frontend tools which aim at
manipulating them or a list of them.
An upcoming patch will externalize the list of paths used by base
backups for the exclude lists, which will then be used by pg_rewind to
do the same.
---
contrib/pg_stat_statements/pg_stat_statements.c | 1 +
src/backend/access/heap/rewriteheap.c | 12 +++++----
src/backend/access/transam/clog.c | 7 ++---
src/backend/access/transam/commit_ts.c | 7 ++---
src/backend/access/transam/multixact.c | 5 ++--
src/backend/access/transam/subtrans.c | 3 ++-
src/backend/access/transam/twophase.c | 14 ++++------
src/backend/access/transam/xlog.c | 9 ++++---
src/backend/commands/async.c | 3 ++-
src/backend/commands/tablespace.c | 5 ++--
src/backend/postmaster/postmaster.c | 10 +++----
src/backend/replication/basebackup.c | 14 +++++-----
src/backend/replication/logical/origin.c | 7 ++---
src/backend/replication/logical/reorderbuffer.c | 25 +++++++++--------
src/backend/replication/logical/snapbuild.c | 25 ++++++++++-------
src/backend/replication/slot.c | 36 +++++++++++++------------
src/backend/storage/file/fd.c | 33 ++++++++++++-----------
src/backend/storage/file/reinit.c | 9 ++++---
src/backend/utils/adt/dbsize.c | 13 ++++-----
src/backend/utils/adt/misc.c | 7 ++---
src/backend/utils/cache/relcache.c | 23 +++++++++-------
src/backend/utils/cache/relmapper.c | 7 +++--
src/backend/utils/init/miscinit.c | 3 ++-
src/backend/utils/time/snapmgr.c | 22 +++++++--------
src/bin/initdb/initdb.c | 6 ++---
src/bin/pg_ctl/pg_ctl.c | 7 +++--
src/bin/pg_resetwal/pg_resetwal.c | 7 ++---
src/bin/pg_rewind/copy_fetch.c | 5 ++--
src/bin/pg_rewind/filemap.c | 19 ++++++-------
src/bin/pg_rewind/pg_rewind.c | 6 ++---
src/bin/pg_upgrade/controldata.c | 13 ++++++---
src/bin/pg_upgrade/exec.c | 16 ++++++-----
src/bin/pg_upgrade/option.c | 5 ++--
src/bin/pg_upgrade/pg_upgrade.c | 7 ++---
src/common/controldata_utils.c | 4 ++-
src/common/relpath.c | 28 ++++++++++---------
36 files changed, 235 insertions(+), 188 deletions(-)
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 928673498a..e251a3ee48 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -71,6 +71,7 @@
#include "parser/parsetree.h"
#include "parser/scanner.h"
#include "parser/scansup.h"
+#include "pg_paths.h"
#include "pgstat.h"
#include "storage/fd.h"
#include "storage/ipc.h"
diff --git a/src/backend/access/heap/rewriteheap.c b/src/backend/access/heap/rewriteheap.c
index 7d466c2588..dd3657f5e2 100644
--- a/src/backend/access/heap/rewriteheap.c
+++ b/src/backend/access/heap/rewriteheap.c
@@ -119,6 +119,7 @@
#include "lib/ilist.h"
+#include "pg_paths.h"
#include "pgstat.h"
#include "replication/logical.h"
@@ -1005,7 +1006,7 @@ logical_rewrite_log_mapping(RewriteState state, TransactionId xid,
dboid = MyDatabaseId;
snprintf(path, MAXPGPATH,
- "pg_logical/mappings/" LOGICAL_REWRITE_FORMAT,
+ PG_LOGICAL_MAPPINGS_DIR "/" LOGICAL_REWRITE_FORMAT,
dboid, relid,
(uint32) (state->rs_begin_lsn >> 32),
(uint32) state->rs_begin_lsn,
@@ -1128,7 +1129,7 @@ heap_xlog_logical_rewrite(XLogReaderState *r)
xlrec = (xl_heap_rewrite_mapping *) XLogRecGetData(r);
snprintf(path, MAXPGPATH,
- "pg_logical/mappings/" LOGICAL_REWRITE_FORMAT,
+ PG_LOGICAL_MAPPINGS_DIR "/" LOGICAL_REWRITE_FORMAT,
xlrec->mapped_db, xlrec->mapped_rel,
(uint32) (xlrec->start_lsn >> 32),
(uint32) xlrec->start_lsn,
@@ -1219,8 +1220,8 @@ CheckPointLogicalRewriteHeap(void)
if (cutoff != InvalidXLogRecPtr && redo < cutoff)
cutoff = redo;
- mappings_dir = AllocateDir("pg_logical/mappings");
- while ((mapping_de = ReadDir(mappings_dir, "pg_logical/mappings")) != NULL)
+ mappings_dir = AllocateDir(PG_LOGICAL_MAPPINGS_DIR);
+ while ((mapping_de = ReadDir(mappings_dir, PG_LOGICAL_MAPPINGS_DIR)) != NULL)
{
struct stat statbuf;
Oid dboid;
@@ -1235,7 +1236,8 @@ CheckPointLogicalRewriteHeap(void)
strcmp(mapping_de->d_name, "..") == 0)
continue;
- snprintf(path, sizeof(path), "pg_logical/mappings/%s", mapping_de->d_name);
+ snprintf(path, sizeof(path), "%s/%s", PG_LOGICAL_MAPPINGS_DIR,
+ mapping_de->d_name);
if (lstat(path, &statbuf) == 0 && !S_ISREG(statbuf.st_mode))
continue;
diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c
index 8b7ff5b0c2..bf4db22a24 100644
--- a/src/backend/access/transam/clog.c
+++ b/src/backend/access/transam/clog.c
@@ -40,6 +40,7 @@
#include "access/xlogutils.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "pg_paths.h"
#include "pg_trace.h"
#include "storage/proc.h"
@@ -699,7 +700,7 @@ CLOGShmemInit(void)
{
ClogCtl->PagePrecedes = CLOGPagePrecedes;
SimpleLruInit(ClogCtl, "clog", CLOGShmemBuffers(), CLOG_LSNS_PER_PAGE,
- CLogControlLock, "pg_xact", LWTRANCHE_CLOG_BUFFERS);
+ CLogControlLock, PG_XACT_DIR, LWTRANCHE_CLOG_BUFFERS);
}
/*
@@ -830,7 +831,7 @@ ShutdownCLOG(void)
* fsync pg_xact to ensure that any files flushed previously are durably
* on disk.
*/
- fsync_fname("pg_xact", true);
+ fsync_fname(PG_XACT_DIR, true);
TRACE_POSTGRESQL_CLOG_CHECKPOINT_DONE(false);
}
@@ -849,7 +850,7 @@ CheckPointCLOG(void)
* fsync pg_xact to ensure that any files flushed previously are durably
* on disk.
*/
- fsync_fname("pg_xact", true);
+ fsync_fname(PG_XACT_DIR, true);
TRACE_POSTGRESQL_CLOG_CHECKPOINT_DONE(true);
}
diff --git a/src/backend/access/transam/commit_ts.c b/src/backend/access/transam/commit_ts.c
index 04a15e4e29..5d0e4d0091 100644
--- a/src/backend/access/transam/commit_ts.c
+++ b/src/backend/access/transam/commit_ts.c
@@ -31,6 +31,7 @@
#include "catalog/pg_type.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "pg_paths.h"
#include "pg_trace.h"
#include "storage/shmem.h"
#include "utils/builtins.h"
@@ -493,7 +494,7 @@ CommitTsShmemInit(void)
CommitTsCtl->PagePrecedes = CommitTsPagePrecedes;
SimpleLruInit(CommitTsCtl, "commit_timestamp", CommitTsShmemBuffers(), 0,
- CommitTsControlLock, "pg_commit_ts",
+ CommitTsControlLock, PG_COMMIT_TS_DIR,
LWTRANCHE_COMMITTS_BUFFERS);
commitTsShared = ShmemInitStruct("CommitTs shared",
@@ -751,7 +752,7 @@ ShutdownCommitTs(void)
* fsync pg_commit_ts to ensure that any files flushed previously are
* durably on disk.
*/
- fsync_fname("pg_commit_ts", true);
+ fsync_fname(PG_COMMIT_TS_DIR, true);
}
/*
@@ -767,7 +768,7 @@ CheckPointCommitTs(void)
* fsync pg_commit_ts to ensure that any files flushed previously are
* durably on disk.
*/
- fsync_fname("pg_commit_ts", true);
+ fsync_fname(PG_COMMIT_TS_DIR, true);
}
/*
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index 6d6f2e3016..a158b4947c 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -81,6 +81,7 @@
#include "funcapi.h"
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pg_paths.h"
#include "pg_trace.h"
#include "postmaster/autovacuum.h"
#include "storage/lmgr.h"
@@ -1828,11 +1829,11 @@ MultiXactShmemInit(void)
SimpleLruInit(MultiXactOffsetCtl,
"multixact_offset", NUM_MXACTOFFSET_BUFFERS, 0,
- MultiXactOffsetControlLock, "pg_multixact/offsets",
+ MultiXactOffsetControlLock, PG_MULTIXACT_OFFSETS_DIR,
LWTRANCHE_MXACTOFFSET_BUFFERS);
SimpleLruInit(MultiXactMemberCtl,
"multixact_member", NUM_MXACTMEMBER_BUFFERS, 0,
- MultiXactMemberControlLock, "pg_multixact/members",
+ MultiXactMemberControlLock, PG_MULTIXACT_MEMBERS_DIR,
LWTRANCHE_MXACTMEMBER_BUFFERS);
/* Initialize our shared state struct */
diff --git a/src/backend/access/transam/subtrans.c b/src/backend/access/transam/subtrans.c
index 4faa21f5ae..977d052217 100644
--- a/src/backend/access/transam/subtrans.c
+++ b/src/backend/access/transam/subtrans.c
@@ -31,6 +31,7 @@
#include "access/slru.h"
#include "access/subtrans.h"
#include "access/transam.h"
+#include "pg_paths.h"
#include "pg_trace.h"
#include "utils/snapmgr.h"
@@ -192,7 +193,7 @@ SUBTRANSShmemInit(void)
{
SubTransCtl->PagePrecedes = SubTransPagePrecedes;
SimpleLruInit(SubTransCtl, "subtrans", NUM_SUBTRANS_BUFFERS, 0,
- SubtransControlLock, "pg_subtrans",
+ SubtransControlLock, PG_SUBTRANS_DIR,
LWTRANCHE_SUBTRANS_BUFFERS);
/* Override default assumption that writes should be fsync'd */
SubTransCtl->do_fsync = false;
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index c479c4881b..89e816429f 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -91,6 +91,7 @@
#include "catalog/storage.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "pg_paths.h"
#include "pg_trace.h"
#include "pgstat.h"
#include "replication/origin.h"
@@ -108,11 +109,6 @@
#include "utils/timestamp.h"
-/*
- * Directory where Two-phase commit files reside within PGDATA
- */
-#define TWOPHASE_DIR "pg_twophase"
-
/* GUC variable, can't be changed after startup */
int max_prepared_xacts = 0;
@@ -877,7 +873,7 @@ TwoPhaseGetDummyProc(TransactionId xid)
/************************************************************************/
#define TwoPhaseFilePath(path, xid) \
- snprintf(path, MAXPGPATH, TWOPHASE_DIR "/%08X", xid)
+ snprintf(path, MAXPGPATH, PG_TWOPHASE_DIR "/%08X", xid)
/*
* 2PC state file format:
@@ -1707,7 +1703,7 @@ CheckPointTwoPhase(XLogRecPtr redo_horizon)
* removals need to be made persistent as well as any files newly created
* previously since the last checkpoint.
*/
- fsync_fname(TWOPHASE_DIR, true);
+ fsync_fname(PG_TWOPHASE_DIR, true);
TRACE_POSTGRESQL_TWOPHASE_CHECKPOINT_DONE();
@@ -1736,8 +1732,8 @@ restoreTwoPhaseData(void)
struct dirent *clde;
LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
- cldir = AllocateDir(TWOPHASE_DIR);
- while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
+ cldir = AllocateDir(PG_TWOPHASE_DIR);
+ while ((clde = ReadDir(cldir, PG_TWOPHASE_DIR)) != NULL)
{
if (strlen(clde->d_name) == 8 &&
strspn(clde->d_name, "0123456789ABCDEF") == 8)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index e42b828edf..08b655913f 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6481,7 +6481,7 @@ StartupXLOG(void)
tablespaceinfo *ti = lfirst(lc);
char *linkloc;
- linkloc = psprintf("pg_tblspc/%s", ti->oid);
+ linkloc = psprintf(PG_TABLESPACE_DIR "/%s", ti->oid);
/*
* Remove the existing symlink if any and Create the symlink
@@ -10429,8 +10429,8 @@ do_pg_start_backup(const char *backupidstr, bool fast, TimeLineID *starttli_p,
datadirpathlen = strlen(DataDir);
/* Collect information about all tablespaces */
- tblspcdir = AllocateDir("pg_tblspc");
- while ((de = ReadDir(tblspcdir, "pg_tblspc")) != NULL)
+ tblspcdir = AllocateDir(PG_TABLESPACE_DIR);
+ while ((de = ReadDir(tblspcdir, PG_TABLESPACE_DIR)) != NULL)
{
char fullpath[MAXPGPATH + 10];
char linkpath[MAXPGPATH];
@@ -10443,7 +10443,8 @@ do_pg_start_backup(const char *backupidstr, bool fast, TimeLineID *starttli_p,
if (strcmp(de->d_name, ".") == 0 || strcmp(de->d_name, "..") == 0)
continue;
- snprintf(fullpath, sizeof(fullpath), "pg_tblspc/%s", de->d_name);
+ snprintf(fullpath, sizeof(fullpath), PG_TABLESPACE_DIR "/%s",
+ de->d_name);
#if defined(HAVE_READLINK) || defined(WIN32)
rllen = readlink(fullpath, linkpath, sizeof(linkpath));
diff --git a/src/backend/commands/async.c b/src/backend/commands/async.c
index ee7c6d41b4..52bbb28dc8 100644
--- a/src/backend/commands/async.c
+++ b/src/backend/commands/async.c
@@ -127,6 +127,7 @@
#include "libpq/libpq.h"
#include "libpq/pqformat.h"
#include "miscadmin.h"
+#include "pg_paths.h"
#include "storage/ipc.h"
#include "storage/lmgr.h"
#include "storage/proc.h"
@@ -480,7 +481,7 @@ AsyncShmemInit(void)
*/
AsyncCtl->PagePrecedes = asyncQueuePagePrecedes;
SimpleLruInit(AsyncCtl, "async", NUM_ASYNC_BUFFERS, 0,
- AsyncCtlLock, "pg_notify", LWTRANCHE_ASYNC_BUFFERS);
+ AsyncCtlLock, PG_NOTIFY_DIR, LWTRANCHE_ASYNC_BUFFERS);
/* Override default assumption that writes should be fsync'd */
AsyncCtl->do_fsync = false;
diff --git a/src/backend/commands/tablespace.c b/src/backend/commands/tablespace.c
index 5c450caa4e..2894a62b4b 100644
--- a/src/backend/commands/tablespace.c
+++ b/src/backend/commands/tablespace.c
@@ -69,6 +69,7 @@
#include "commands/tablecmds.h"
#include "commands/tablespace.h"
#include "miscadmin.h"
+#include "pg_paths.h"
#include "postmaster/bgwriter.h"
#include "storage/fd.h"
#include "storage/lmgr.h"
@@ -566,7 +567,7 @@ create_tablespace_directories(const char *location, const Oid tablespaceoid)
char *location_with_version_dir;
struct stat st;
- linkloc = psprintf("pg_tblspc/%u", tablespaceoid);
+ linkloc = psprintf(PG_TABLESPACE_DIR "/%u", tablespaceoid);
location_with_version_dir = psprintf("%s/%s", location,
TABLESPACE_VERSION_DIRECTORY);
@@ -667,7 +668,7 @@ destroy_tablespace_directories(Oid tablespaceoid, bool redo)
char *subfile;
struct stat st;
- linkloc_with_version_dir = psprintf("pg_tblspc/%u/%s", tablespaceoid,
+ linkloc_with_version_dir = psprintf(PG_TABLESPACE_DIR "/%u/%s", tablespaceoid,
TABLESPACE_VERSION_DIRECTORY);
/*
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 66d80914a0..cf39bc1c91 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -1549,7 +1549,7 @@ checkDataDir(void)
/* Look for PG_VERSION before looking for pg_control */
ValidatePgVersion(DataDir);
- snprintf(path, sizeof(path), "%s/global/pg_control", DataDir);
+ snprintf(path, sizeof(path), "%s/%s", DataDir, XLOG_CONTROL_FILE);
fp = AllocateFile(path, PG_BINARY_R);
if (fp == NULL)
@@ -5541,11 +5541,9 @@ CreateOptsFile(int argc, char *argv[], char *fullprogname)
FILE *fp;
int i;
-#define OPTS_FILE "postmaster.opts"
-
- if ((fp = fopen(OPTS_FILE, "w")) == NULL)
+ if ((fp = fopen(POSTMASTER_OPTS_FILE, "w")) == NULL)
{
- elog(LOG, "could not create file \"%s\": %m", OPTS_FILE);
+ elog(LOG, "could not create file \"%s\": %m", POSTMASTER_OPTS_FILE);
return false;
}
@@ -5556,7 +5554,7 @@ CreateOptsFile(int argc, char *argv[], char *fullprogname)
if (fclose(fp))
{
- elog(LOG, "could not write file \"%s\": %m", OPTS_FILE);
+ elog(LOG, "could not write file \"%s\": %m", POSTMASTER_OPTS_FILE);
return false;
}
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index 63ab81f837..2e764ab293 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -367,8 +367,8 @@ perform_base_backup(basebackup_options *opt)
XLByteToPrevSeg(endptr, endsegno, wal_segment_size);
XLogFileName(lastoff, ThisTimeLineID, endsegno, wal_segment_size);
- dir = AllocateDir("pg_wal");
- while ((de = ReadDir(dir, "pg_wal")) != NULL)
+ dir = AllocateDir(XLOGDIR);
+ while ((de = ReadDir(dir, XLOGDIR)) != NULL)
{
/* Does it look like a WAL segment, and is it in the range? */
if (IsXLogFileName(de->d_name) &&
@@ -1011,7 +1011,7 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
snprintf(pathbuf, sizeof(pathbuf), "%s/%s", path, de->d_name);
/* Skip pg_control here to back up it last */
- if (strcmp(pathbuf, "./global/pg_control") == 0)
+ if (strcmp(pathbuf, "./" XLOG_CONTROL_FILE) == 0)
continue;
if (lstat(pathbuf, &statbuf) != 0)
@@ -1058,7 +1058,7 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
* WAL archive anyway. But include it as an empty directory anyway, so
* we get permissions right.
*/
- if (strcmp(pathbuf, "./pg_wal") == 0)
+ if (strcmp(pathbuf, "./" XLOGDIR) == 0)
{
/* If pg_wal is a symlink, write it as a directory anyway */
size += _tarWriteDir(pathbuf, basepathlen, &statbuf, sizeonly);
@@ -1067,14 +1067,14 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
* Also send archive_status directory (by hackishly reusing
* statbuf from above ...).
*/
- size += _tarWriteHeader("./pg_wal/archive_status", NULL, &statbuf,
+ size += _tarWriteHeader("./" XLOG_ARCHIVE_STATUS_DIR, NULL, &statbuf,
sizeonly);
continue; /* don't recurse into pg_wal */
}
/* Allow symbolic links in pg_tblspc only */
- if (strcmp(path, "./pg_tblspc") == 0 &&
+ if (strcmp(path, "./" PG_TABLESPACE_DIR) == 0 &&
#ifndef WIN32
S_ISLNK(statbuf.st_mode)
#else
@@ -1151,7 +1151,7 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
/*
* skip sending directories inside pg_tblspc, if not required.
*/
- if (strcmp(pathbuf, "./pg_tblspc") == 0 && !sendtblspclinks)
+ if (strcmp(pathbuf, "./" PG_TABLESPACE_DIR) == 0 && !sendtblspclinks)
skip_this_dir = true;
if (!skip_this_dir)
diff --git a/src/backend/replication/logical/origin.c b/src/backend/replication/logical/origin.c
index 5cc9a955d7..87a0a339ae 100644
--- a/src/backend/replication/logical/origin.c
+++ b/src/backend/replication/logical/origin.c
@@ -83,6 +83,7 @@
#include "replication/origin.h"
#include "replication/logical.h"
+#include "pg_paths.h"
#include "pgstat.h"
#include "storage/fd.h"
#include "storage/ipc.h"
@@ -544,8 +545,8 @@ ReplicationOriginShmemInit(void)
void
CheckPointReplicationOrigin(void)
{
- const char *tmppath = "pg_logical/replorigin_checkpoint.tmp";
- const char *path = "pg_logical/replorigin_checkpoint";
+ const char *tmppath = PG_LOGICAL_DIR "/replorigin_checkpoint.tmp";
+ const char *path = PG_LOGICAL_DIR "/replorigin_checkpoint";
int tmpfd;
int i;
uint32 magic = REPLICATION_STATE_MAGIC;
@@ -657,7 +658,7 @@ CheckPointReplicationOrigin(void)
void
StartupReplicationOrigin(void)
{
- const char *path = "pg_logical/replorigin_checkpoint";
+ const char *path = PG_LOGICAL_DIR "/replorigin_checkpoint";
int fd;
int readBytes;
uint32 magic = REPLICATION_STATE_MAGIC;
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index c72a611a39..b706b757a7 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -2070,7 +2070,8 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
* No need to care about TLIs here, only used during a single run,
* so each LSN only maps to a specific WAL record.
*/
- sprintf(path, "pg_replslot/%s/xid-%u-lsn-%X-%X.snap",
+ sprintf(path, "%s/%s/xid-%u-lsn-%X-%X.snap",
+ PG_REPLSLOT_DIR,
NameStr(MyReplicationSlot->data.name), txn->xid,
(uint32) (recptr >> 32), (uint32) recptr);
@@ -2316,7 +2317,8 @@ ReorderBufferRestoreChanges(ReorderBuffer *rb, ReorderBufferTXN *txn,
* No need to care about TLIs here, only used during a single run,
* so each LSN only maps to a specific WAL record.
*/
- sprintf(path, "pg_replslot/%s/xid-%u-lsn-%X-%X.snap",
+ sprintf(path, "%s/%s/xid-%u-lsn-%X-%X.snap",
+ PG_REPLSLOT_DIR,
NameStr(MyReplicationSlot->data.name), txn->xid,
(uint32) (recptr >> 32), (uint32) recptr);
@@ -2558,7 +2560,8 @@ ReorderBufferRestoreCleanup(ReorderBuffer *rb, ReorderBufferTXN *txn)
XLogSegNoOffsetToRecPtr(cur, 0, recptr, wal_segment_size);
- sprintf(path, "pg_replslot/%s/xid-%u-lsn-%X-%X.snap",
+ sprintf(path, "%s/%s/xid-%u-lsn-%X-%X.snap",
+ PG_REPLSLOT_DIR,
NameStr(MyReplicationSlot->data.name), txn->xid,
(uint32) (recptr >> 32), (uint32) recptr);
if (unlink(path) != 0 && errno != ENOENT)
@@ -2581,8 +2584,8 @@ StartupReorderBuffer(void)
DIR *spill_dir;
struct dirent *spill_de;
- logical_dir = AllocateDir("pg_replslot");
- while ((logical_de = ReadDir(logical_dir, "pg_replslot")) != NULL)
+ logical_dir = AllocateDir(PG_REPLSLOT_DIR);
+ while ((logical_de = ReadDir(logical_dir, PG_REPLSLOT_DIR)) != NULL)
{
struct stat statbuf;
char path[MAXPGPATH * 2 + 12];
@@ -2599,7 +2602,7 @@ StartupReorderBuffer(void)
* ok, has to be a surviving logical slot, iterate and delete
* everything starting with xid-*
*/
- sprintf(path, "pg_replslot/%s", logical_de->d_name);
+ sprintf(path, "%s/%s", PG_REPLSLOT_DIR, logical_de->d_name);
/* we're only creating directories here, skip if it's not our's */
if (lstat(path, &statbuf) == 0 && !S_ISDIR(statbuf.st_mode))
@@ -2615,8 +2618,8 @@ StartupReorderBuffer(void)
/* only look at names that can be ours */
if (strncmp(spill_de->d_name, "xid", 3) == 0)
{
- sprintf(path, "pg_replslot/%s/%s", logical_de->d_name,
- spill_de->d_name);
+ sprintf(path, "%s/%s/%s", PG_REPLSLOT_DIR,
+ logical_de->d_name, spill_de->d_name);
if (unlink(path) != 0)
ereport(PANIC,
@@ -3008,7 +3011,7 @@ ApplyLogicalMappingFile(HTAB *tuplecid_data, Oid relid, const char *fname)
int readBytes;
LogicalRewriteMappingData map;
- sprintf(path, "pg_logical/mappings/%s", fname);
+ sprintf(path, "%s/%s", PG_LOGICAL_MAPPINGS_DIR, fname);
fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
if (fd < 0)
ereport(ERROR,
@@ -3131,8 +3134,8 @@ UpdateLogicalMappings(HTAB *tuplecid_data, Oid relid, Snapshot snapshot)
size_t off;
Oid dboid = IsSharedRelation(relid) ? InvalidOid : MyDatabaseId;
- mapping_dir = AllocateDir("pg_logical/mappings");
- while ((mapping_de = ReadDir(mapping_dir, "pg_logical/mappings")) != NULL)
+ mapping_dir = AllocateDir(PG_LOGICAL_MAPPINGS_DIR);
+ while ((mapping_de = ReadDir(mapping_dir, PG_LOGICAL_MAPPINGS_DIR)) != NULL)
{
Oid f_dboid;
Oid f_relid;
diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c
index 4123cdebcf..bc913305db 100644
--- a/src/backend/replication/logical/snapbuild.c
+++ b/src/backend/replication/logical/snapbuild.c
@@ -126,6 +126,7 @@
#include "access/transam.h"
#include "access/xact.h"
+#include "pg_paths.h"
#include "pgstat.h"
#include "replication/logical.h"
@@ -1503,7 +1504,8 @@ SnapBuildSerialize(SnapBuild *builder, XLogRecPtr lsn)
* unless the user used pg_resetwal or similar. If a user did so, there's
* no hope continuing to decode anyway.
*/
- sprintf(path, "pg_logical/snapshots/%X-%X.snap",
+ sprintf(path, "%s/%X-%X.snap",
+ PG_LOGICAL_SNAPSHOTS_DIR,
(uint32) (lsn >> 32), (uint32) lsn);
/*
@@ -1529,7 +1531,7 @@ SnapBuildSerialize(SnapBuild *builder, XLogRecPtr lsn)
* be safely on disk.
*/
fsync_fname(path, false);
- fsync_fname("pg_logical/snapshots", true);
+ fsync_fname(PG_LOGICAL_SNAPSHOTS_DIR, true);
builder->last_serialized_snapshot = lsn;
goto out;
@@ -1545,7 +1547,8 @@ SnapBuildSerialize(SnapBuild *builder, XLogRecPtr lsn)
elog(DEBUG1, "serializing snapshot to %s", path);
/* to make sure only we will write to this tempfile, include pid */
- sprintf(tmppath, "pg_logical/snapshots/%X-%X.snap.%u.tmp",
+ sprintf(tmppath, "%s/%X-%X.snap.%u.tmp",
+ PG_LOGICAL_SNAPSHOTS_DIR,
(uint32) (lsn >> 32), (uint32) lsn, MyProcPid);
/*
@@ -1631,7 +1634,7 @@ SnapBuildSerialize(SnapBuild *builder, XLogRecPtr lsn)
pgstat_report_wait_end();
CloseTransientFile(fd);
- fsync_fname("pg_logical/snapshots", true);
+ fsync_fname(PG_LOGICAL_SNAPSHOTS_DIR, true);
/*
* We may overwrite the work from some other backend, but that's ok, our
@@ -1647,7 +1650,7 @@ SnapBuildSerialize(SnapBuild *builder, XLogRecPtr lsn)
/* make sure we persist */
fsync_fname(path, false);
- fsync_fname("pg_logical/snapshots", true);
+ fsync_fname(PG_LOGICAL_SNAPSHOTS_DIR, true);
/*
* Now there's no way we can loose the dumped state anymore, remember this
@@ -1678,7 +1681,8 @@ SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
if (builder->state == SNAPBUILD_CONSISTENT)
return false;
- sprintf(path, "pg_logical/snapshots/%X-%X.snap",
+ sprintf(path, "%s/%X-%X.snap",
+ PG_LOGICAL_SNAPSHOTS_DIR,
(uint32) (lsn >> 32), (uint32) lsn);
fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
@@ -1699,7 +1703,7 @@ SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
* ----
*/
fsync_fname(path, false);
- fsync_fname("pg_logical/snapshots", true);
+ fsync_fname(PG_LOGICAL_SNAPSHOTS_DIR, true);
/* read statically sized portion of snapshot */
@@ -1880,8 +1884,8 @@ CheckPointSnapBuild(void)
if (redo < cutoff)
cutoff = redo;
- snap_dir = AllocateDir("pg_logical/snapshots");
- while ((snap_de = ReadDir(snap_dir, "pg_logical/snapshots")) != NULL)
+ snap_dir = AllocateDir(PG_LOGICAL_SNAPSHOTS_DIR);
+ while ((snap_de = ReadDir(snap_dir, PG_LOGICAL_SNAPSHOTS_DIR)) != NULL)
{
uint32 hi;
uint32 lo;
@@ -1892,7 +1896,8 @@ CheckPointSnapBuild(void)
strcmp(snap_de->d_name, "..") == 0)
continue;
- snprintf(path, sizeof(path), "pg_logical/snapshots/%s", snap_de->d_name);
+ snprintf(path, sizeof(path), "%s/%s", PG_LOGICAL_SNAPSHOTS_DIR,
+ snap_de->d_name);
if (lstat(path, &statbuf) == 0 && !S_ISREG(statbuf.st_mode))
{
diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c
index fc9ef22b0b..c64641a1ae 100644
--- a/src/backend/replication/slot.c
+++ b/src/backend/replication/slot.c
@@ -549,8 +549,8 @@ ReplicationSlotDropPtr(ReplicationSlot *slot)
LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE);
/* Generate pathnames. */
- sprintf(path, "pg_replslot/%s", NameStr(slot->data.name));
- sprintf(tmppath, "pg_replslot/%s.tmp", NameStr(slot->data.name));
+ sprintf(path, "%s/%s", PG_REPLSLOT_DIR, NameStr(slot->data.name));
+ sprintf(tmppath, "%s/%s.tmp", PG_REPLSLOT_DIR, NameStr(slot->data.name));
/*
* Rename the slot directory on disk, so that we'll no longer recognize
@@ -571,7 +571,7 @@ ReplicationSlotDropPtr(ReplicationSlot *slot)
*/
START_CRIT_SECTION();
fsync_fname(tmppath, true);
- fsync_fname("pg_replslot", true);
+ fsync_fname(PG_REPLSLOT_DIR, true);
END_CRIT_SECTION();
}
else
@@ -641,7 +641,8 @@ ReplicationSlotSave(void)
Assert(MyReplicationSlot != NULL);
- sprintf(path, "pg_replslot/%s", NameStr(MyReplicationSlot->data.name));
+ sprintf(path, "%s/%s", PG_REPLSLOT_DIR,
+ NameStr(MyReplicationSlot->data.name));
SaveSlotToPath(MyReplicationSlot, path, ERROR);
}
@@ -1076,7 +1077,7 @@ CheckPointReplicationSlots(void)
continue;
/* save the slot to disk, locking is handled in SaveSlotToPath() */
- sprintf(path, "pg_replslot/%s", NameStr(s->data.name));
+ sprintf(path, "%s/%s", PG_REPLSLOT_DIR, NameStr(s->data.name));
SaveSlotToPath(s, path, LOG);
}
LWLockRelease(ReplicationSlotAllocationLock);
@@ -1095,8 +1096,8 @@ StartupReplicationSlots(void)
elog(DEBUG1, "starting up replication slots");
/* restore all slots by iterating over all on-disk entries */
- replication_dir = AllocateDir("pg_replslot");
- while ((replication_de = ReadDir(replication_dir, "pg_replslot")) != NULL)
+ replication_dir = AllocateDir(PG_REPLSLOT_DIR);
+ while ((replication_de = ReadDir(replication_dir, PG_REPLSLOT_DIR)) != NULL)
{
struct stat statbuf;
char path[MAXPGPATH + 12];
@@ -1105,7 +1106,8 @@ StartupReplicationSlots(void)
strcmp(replication_de->d_name, "..") == 0)
continue;
- snprintf(path, sizeof(path), "pg_replslot/%s", replication_de->d_name);
+ snprintf(path, sizeof(path), "%s/%s", PG_REPLSLOT_DIR,
+ replication_de->d_name);
/* we're only creating directories here, skip if it's not our's */
if (lstat(path, &statbuf) == 0 && !S_ISDIR(statbuf.st_mode))
@@ -1121,7 +1123,7 @@ StartupReplicationSlots(void)
errmsg("could not remove directory \"%s\"", path)));
continue;
}
- fsync_fname("pg_replslot", true);
+ fsync_fname(PG_REPLSLOT_DIR, true);
continue;
}
@@ -1159,8 +1161,8 @@ CreateSlotOnDisk(ReplicationSlot *slot)
* takes out the lock, if we'd take the lock here, we'd deadlock.
*/
- sprintf(path, "pg_replslot/%s", NameStr(slot->data.name));
- sprintf(tmppath, "pg_replslot/%s.tmp", NameStr(slot->data.name));
+ sprintf(path, "%s/%s", PG_REPLSLOT_DIR, NameStr(slot->data.name));
+ sprintf(tmppath, "%s/%s.tmp", PG_REPLSLOT_DIR, NameStr(slot->data.name));
/*
* It's just barely possible that some previous effort to create or drop a
@@ -1198,7 +1200,7 @@ CreateSlotOnDisk(ReplicationSlot *slot)
START_CRIT_SECTION();
fsync_fname(path, true);
- fsync_fname("pg_replslot", true);
+ fsync_fname(PG_REPLSLOT_DIR, true);
END_CRIT_SECTION();
}
@@ -1309,7 +1311,7 @@ SaveSlotToPath(ReplicationSlot *slot, const char *dir, int elevel)
fsync_fname(path, false);
fsync_fname(dir, true);
- fsync_fname("pg_replslot", true);
+ fsync_fname(PG_REPLSLOT_DIR, true);
END_CRIT_SECTION();
@@ -1342,13 +1344,13 @@ RestoreSlotFromDisk(const char *name)
/* no need to lock here, no concurrent access allowed yet */
/* delete temp file if it exists */
- sprintf(path, "pg_replslot/%s/state.tmp", name);
+ sprintf(path, "%s/%s/state.tmp", PG_REPLSLOT_DIR, name);
if (unlink(path) < 0 && errno != ENOENT)
ereport(PANIC,
(errcode_for_file_access(),
errmsg("could not remove file \"%s\": %m", path)));
- sprintf(path, "pg_replslot/%s/state", name);
+ sprintf(path, "%s/%s/state", PG_REPLSLOT_DIR, name);
elog(DEBUG1, "restoring replication slot from \"%s\"", path);
@@ -1459,7 +1461,7 @@ RestoreSlotFromDisk(const char *name)
*/
if (cp.slotdata.persistency != RS_PERSISTENT)
{
- sprintf(path, "pg_replslot/%s", name);
+ sprintf(path, "%s/%s", PG_REPLSLOT_DIR, name);
if (!rmtree(path, true))
{
@@ -1467,7 +1469,7 @@ RestoreSlotFromDisk(const char *name)
(errcode_for_file_access(),
errmsg("could not remove directory \"%s\"", path)));
}
- fsync_fname("pg_replslot", true);
+ fsync_fname(PG_REPLSLOT_DIR, true);
return;
}
diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c
index 2a18e94ff4..45888e958f 100644
--- a/src/backend/storage/file/fd.c
+++ b/src/backend/storage/file/fd.c
@@ -84,6 +84,7 @@
#include "access/xlog.h"
#include "catalog/catalog.h"
#include "catalog/pg_tablespace.h"
+#include "pg_paths.h"
#include "pgstat.h"
#include "portability/mem.h"
#include "storage/fd.h"
@@ -1560,11 +1561,12 @@ TempTablespacePath(char *path, Oid tablespace)
if (tablespace == InvalidOid ||
tablespace == DEFAULTTABLESPACE_OID ||
tablespace == GLOBALTABLESPACE_OID)
- snprintf(path, MAXPGPATH, "base/%s", PG_TEMP_FILES_DIR);
+ snprintf(path, MAXPGPATH, "%s/%s",
+ PG_BASE_DIR, PG_TEMP_FILES_DIR);
else
{
/* All other tablespaces are accessed via symlinks */
- snprintf(path, MAXPGPATH, "pg_tblspc/%u/%s/%s",
+ snprintf(path, MAXPGPATH, PG_TABLESPACE_DIR "/%u/%s/%s",
tablespace, TABLESPACE_VERSION_DIRECTORY,
PG_TEMP_FILES_DIR);
}
@@ -3020,26 +3022,27 @@ RemovePgTempFiles(void)
/*
* First process temp files in pg_default ($PGDATA/base)
*/
- snprintf(temp_path, sizeof(temp_path), "base/%s", PG_TEMP_FILES_DIR);
+ snprintf(temp_path, sizeof(temp_path), "%s/%s",
+ PG_BASE_DIR, PG_TEMP_FILES_DIR);
RemovePgTempFilesInDir(temp_path, true, false);
- RemovePgTempRelationFiles("base");
+ RemovePgTempRelationFiles(PG_BASE_DIR);
/*
* Cycle through temp directories for all non-default tablespaces.
*/
- spc_dir = AllocateDir("pg_tblspc");
+ spc_dir = AllocateDir(PG_TABLESPACE_DIR);
- while ((spc_de = ReadDirExtended(spc_dir, "pg_tblspc", LOG)) != NULL)
+ while ((spc_de = ReadDirExtended(spc_dir, PG_TABLESPACE_DIR, LOG)) != NULL)
{
if (strcmp(spc_de->d_name, ".") == 0 ||
strcmp(spc_de->d_name, "..") == 0)
continue;
- snprintf(temp_path, sizeof(temp_path), "pg_tblspc/%s/%s/%s",
+ snprintf(temp_path, sizeof(temp_path), PG_TABLESPACE_DIR "/%s/%s/%s",
spc_de->d_name, TABLESPACE_VERSION_DIRECTORY, PG_TEMP_FILES_DIR);
RemovePgTempFilesInDir(temp_path, true, false);
- snprintf(temp_path, sizeof(temp_path), "pg_tblspc/%s/%s",
+ snprintf(temp_path, sizeof(temp_path), PG_TABLESPACE_DIR "/%s/%s",
spc_de->d_name, TABLESPACE_VERSION_DIRECTORY);
RemovePgTempRelationFiles(temp_path);
}
@@ -3278,16 +3281,16 @@ SyncDataDirectory(void)
{
struct stat st;
- if (lstat("pg_wal", &st) < 0)
+ if (lstat(XLOGDIR, &st) < 0)
ereport(LOG,
(errcode_for_file_access(),
errmsg("could not stat file \"%s\": %m",
- "pg_wal")));
+ XLOGDIR)));
else if (S_ISLNK(st.st_mode))
xlog_is_symlink = true;
}
#else
- if (pgwin32_is_junction("pg_wal"))
+ if (pgwin32_is_junction(XLOGDIR))
xlog_is_symlink = true;
#endif
@@ -3299,8 +3302,8 @@ SyncDataDirectory(void)
#ifdef PG_FLUSH_DATA_WORKS
walkdir(".", pre_sync_fname, false, DEBUG1);
if (xlog_is_symlink)
- walkdir("pg_wal", pre_sync_fname, false, DEBUG1);
- walkdir("pg_tblspc", pre_sync_fname, true, DEBUG1);
+ walkdir(XLOGDIR, pre_sync_fname, false, DEBUG1);
+ walkdir(PG_TABLESPACE_DIR, pre_sync_fname, true, DEBUG1);
#endif
/*
@@ -3314,8 +3317,8 @@ SyncDataDirectory(void)
*/
walkdir(".", datadir_fsync_fname, false, LOG);
if (xlog_is_symlink)
- walkdir("pg_wal", datadir_fsync_fname, false, LOG);
- walkdir("pg_tblspc", datadir_fsync_fname, true, LOG);
+ walkdir(XLOGDIR, datadir_fsync_fname, false, LOG);
+ walkdir(PG_TABLESPACE_DIR, datadir_fsync_fname, true, LOG);
}
/*
diff --git a/src/backend/storage/file/reinit.c b/src/backend/storage/file/reinit.c
index 92363ae6ad..69e9c7bfc6 100644
--- a/src/backend/storage/file/reinit.c
+++ b/src/backend/storage/file/reinit.c
@@ -18,6 +18,7 @@
#include "catalog/catalog.h"
#include "common/relpath.h"
+#include "pg_paths.h"
#include "storage/copydir.h"
#include "storage/fd.h"
#include "storage/reinit.h"
@@ -71,20 +72,20 @@ ResetUnloggedRelations(int op)
/*
* First process unlogged files in pg_default ($PGDATA/base)
*/
- ResetUnloggedRelationsInTablespaceDir("base", op);
+ ResetUnloggedRelationsInTablespaceDir(PG_BASE_DIR, op);
/*
* Cycle through directories for all non-default tablespaces.
*/
- spc_dir = AllocateDir("pg_tblspc");
+ spc_dir = AllocateDir(PG_TABLESPACE_DIR);
- while ((spc_de = ReadDir(spc_dir, "pg_tblspc")) != NULL)
+ while ((spc_de = ReadDir(spc_dir, PG_TABLESPACE_DIR)) != NULL)
{
if (strcmp(spc_de->d_name, ".") == 0 ||
strcmp(spc_de->d_name, "..") == 0)
continue;
- snprintf(temp_path, sizeof(temp_path), "pg_tblspc/%s/%s",
+ snprintf(temp_path, sizeof(temp_path), PG_TABLESPACE_DIR "/%s/%s",
spc_de->d_name, TABLESPACE_VERSION_DIRECTORY);
ResetUnloggedRelationsInTablespaceDir(temp_path, op);
}
diff --git a/src/backend/utils/adt/dbsize.c b/src/backend/utils/adt/dbsize.c
index 834a10485f..ee3cece8d7 100644
--- a/src/backend/utils/adt/dbsize.c
+++ b/src/backend/utils/adt/dbsize.c
@@ -22,6 +22,7 @@
#include "commands/dbcommands.h"
#include "commands/tablespace.h"
#include "miscadmin.h"
+#include "pg_paths.h"
#include "storage/fd.h"
#include "utils/acl.h"
#include "utils/builtins.h"
@@ -104,11 +105,11 @@ calculate_database_size(Oid dbOid)
/* Shared storage in pg_global is not counted */
/* Include pg_default storage */
- snprintf(pathname, sizeof(pathname), "base/%u", dbOid);
+ snprintf(pathname, sizeof(pathname), PG_BASE_DIR "/%u", dbOid);
totalsize = db_dir_size(pathname);
/* Scan the non-default tablespaces */
- snprintf(dirpath, MAXPGPATH, "pg_tblspc");
+ snprintf(dirpath, MAXPGPATH, PG_TABLESPACE_DIR);
dirdesc = AllocateDir(dirpath);
while ((direntry = ReadDir(dirdesc, dirpath)) != NULL)
@@ -119,7 +120,7 @@ calculate_database_size(Oid dbOid)
strcmp(direntry->d_name, "..") == 0)
continue;
- snprintf(pathname, sizeof(pathname), "pg_tblspc/%s/%s/%u",
+ snprintf(pathname, sizeof(pathname), PG_TABLESPACE_DIR "/%s/%s/%u",
direntry->d_name, TABLESPACE_VERSION_DIRECTORY, dbOid);
totalsize += db_dir_size(pathname);
}
@@ -188,11 +189,11 @@ calculate_tablespace_size(Oid tblspcOid)
}
if (tblspcOid == DEFAULTTABLESPACE_OID)
- snprintf(tblspcPath, MAXPGPATH, "base");
+ snprintf(tblspcPath, MAXPGPATH, PG_BASE_DIR);
else if (tblspcOid == GLOBALTABLESPACE_OID)
- snprintf(tblspcPath, MAXPGPATH, "global");
+ snprintf(tblspcPath, MAXPGPATH, PG_GLOBAL_DIR);
else
- snprintf(tblspcPath, MAXPGPATH, "pg_tblspc/%u/%s", tblspcOid,
+ snprintf(tblspcPath, MAXPGPATH, PG_TABLESPACE_DIR "/%u/%s", tblspcOid,
TABLESPACE_VERSION_DIRECTORY);
dirdesc = AllocateDir(tblspcPath);
diff --git a/src/backend/utils/adt/misc.c b/src/backend/utils/adt/misc.c
index 9f98c03173..41a2407b6a 100644
--- a/src/backend/utils/adt/misc.c
+++ b/src/backend/utils/adt/misc.c
@@ -396,9 +396,9 @@ pg_tablespace_databases(PG_FUNCTION_ARGS)
else
{
if (tablespaceOid == DEFAULTTABLESPACE_OID)
- fctx->location = psprintf("base");
+ fctx->location = psprintf(PG_BASE_DIR);
else
- fctx->location = psprintf("pg_tblspc/%u/%s", tablespaceOid,
+ fctx->location = psprintf(PG_TABLESPACE_DIR "/%u/%s", tablespaceOid,
TABLESPACE_VERSION_DIRECTORY);
fctx->dirdesc = AllocateDir(fctx->location);
@@ -484,7 +484,8 @@ pg_tablespace_location(PG_FUNCTION_ARGS)
* Find the location of the tablespace by reading the symbolic link that
* is in pg_tblspc/<oid>.
*/
- snprintf(sourcepath, sizeof(sourcepath), "pg_tblspc/%u", tablespaceOid);
+ snprintf(sourcepath, sizeof(sourcepath), PG_TABLESPACE_DIR "/%u",
+ tablespaceOid);
rllen = readlink(sourcepath, targetpath, sizeof(targetpath));
if (rllen < 0)
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index c081b88b73..a186aed835 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -71,6 +71,7 @@
#include "optimizer/clauses.h"
#include "optimizer/prep.h"
#include "optimizer/var.h"
+#include "pg_paths.h"
#include "rewrite/rewriteDefine.h"
#include "rewrite/rowsecurity.h"
#include "storage/lmgr.h"
@@ -5351,7 +5352,8 @@ load_relcache_init_file(bool shared)
int i;
if (shared)
- snprintf(initfilename, sizeof(initfilename), "global/%s",
+ snprintf(initfilename, sizeof(initfilename), "%s/%s",
+ PG_GLOBAL_DIR,
RELCACHE_INIT_FILENAME);
else
snprintf(initfilename, sizeof(initfilename), "%s/%s",
@@ -5750,9 +5752,11 @@ write_relcache_init_file(bool shared)
*/
if (shared)
{
- snprintf(tempfilename, sizeof(tempfilename), "global/%s.%d",
+ snprintf(tempfilename, sizeof(tempfilename), "%s/%s.%d",
+ PG_GLOBAL_DIR,
RELCACHE_INIT_FILENAME, MyProcPid);
- snprintf(finalfilename, sizeof(finalfilename), "global/%s",
+ snprintf(finalfilename, sizeof(finalfilename), "%s/%s",
+ PG_GLOBAL_DIR,
RELCACHE_INIT_FILENAME);
}
else
@@ -6073,7 +6077,6 @@ RelationCacheInitFilePostInvalidate(void)
void
RelationCacheInitFileRemove(void)
{
- const char *tblspcdir = "pg_tblspc";
DIR *dir;
struct dirent *de;
char path[MAXPGPATH + 10 + sizeof(TABLESPACE_VERSION_DIRECTORY)];
@@ -6082,23 +6085,25 @@ RelationCacheInitFileRemove(void)
* We zap the shared cache file too. In theory it can't get out of sync
* enough to be a problem, but in data-corruption cases, who knows ...
*/
- snprintf(path, sizeof(path), "global/%s",
+ snprintf(path, sizeof(path), "%s/%s",
+ PG_GLOBAL_DIR,
RELCACHE_INIT_FILENAME);
unlink_initfile(path);
/* Scan everything in the default tablespace */
- RelationCacheInitFileRemoveInDir("base");
+ RelationCacheInitFileRemoveInDir(PG_BASE_DIR);
/* Scan the tablespace link directory to find non-default tablespaces */
- dir = AllocateDir(tblspcdir);
+ dir = AllocateDir(PG_TABLESPACE_DIR);
- while ((de = ReadDirExtended(dir, tblspcdir, LOG)) != NULL)
+ while ((de = ReadDirExtended(dir, PG_TABLESPACE_DIR, LOG)) != NULL)
{
if (strspn(de->d_name, "0123456789") == strlen(de->d_name))
{
/* Scan the tablespace dir for per-database dirs */
snprintf(path, sizeof(path), "%s/%s/%s",
- tblspcdir, de->d_name, TABLESPACE_VERSION_DIRECTORY);
+ PG_TABLESPACE_DIR, de->d_name,
+ TABLESPACE_VERSION_DIRECTORY);
RelationCacheInitFileRemoveInDir(path);
}
}
diff --git a/src/backend/utils/cache/relmapper.c b/src/backend/utils/cache/relmapper.c
index 99d095f2df..479082f285 100644
--- a/src/backend/utils/cache/relmapper.c
+++ b/src/backend/utils/cache/relmapper.c
@@ -50,6 +50,7 @@
#include "catalog/pg_tablespace.h"
#include "catalog/storage.h"
#include "miscadmin.h"
+#include "pg_paths.h"
#include "pgstat.h"
#include "storage/fd.h"
#include "storage/lwlock.h"
@@ -632,7 +633,8 @@ load_relmap_file(bool shared)
if (shared)
{
- snprintf(mapfilename, sizeof(mapfilename), "global/%s",
+ snprintf(mapfilename, sizeof(mapfilename), "%s/%s",
+ PG_GLOBAL_DIR,
RELMAPPER_FILENAME);
map = &shared_map;
}
@@ -733,7 +735,8 @@ write_relmap_file(bool shared, RelMapFile *newmap,
*/
if (shared)
{
- snprintf(mapfilename, sizeof(mapfilename), "global/%s",
+ snprintf(mapfilename, sizeof(mapfilename), "%s/%s",
+ PG_GLOBAL_DIR,
RELMAPPER_FILENAME);
realmap = &shared_map;
}
diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c
index 87ed7d3f71..4d1dd06507 100644
--- a/src/backend/utils/init/miscinit.c
+++ b/src/backend/utils/init/miscinit.c
@@ -35,6 +35,7 @@
#include "libpq/libpq.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
+#include "pg_paths.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
#include "postmaster/postmaster.h"
@@ -52,7 +53,7 @@
#include "utils/varlena.h"
-#define DIRECTORY_LOCK_FILE "postmaster.pid"
+#define DIRECTORY_LOCK_FILE POSTMASTER_PID_FILE
ProcessingMode Mode = InitProcessing;
diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c
index e58c69dbd7..6c3571f3b5 100644
--- a/src/backend/utils/time/snapmgr.c
+++ b/src/backend/utils/time/snapmgr.c
@@ -54,6 +54,7 @@
#include "catalog/catalog.h"
#include "lib/pairingheap.h"
#include "miscadmin.h"
+#include "pg_paths.h"
#include "storage/predicate.h"
#include "storage/proc.h"
#include "storage/procarray.h"
@@ -209,9 +210,6 @@ bool FirstSnapshotSet = false;
*/
static Snapshot FirstXactSnapshot = NULL;
-/* Define pathname of exported-snapshot files */
-#define SNAPSHOT_EXPORT_DIR "pg_snapshots"
-
/* Structure holding info about exported snapshot. */
typedef struct ExportedSnapshot
{
@@ -1210,7 +1208,7 @@ ExportSnapshot(Snapshot snapshot)
* Generate file path for the snapshot. We start numbering of snapshots
* inside the transaction from 1.
*/
- snprintf(path, sizeof(path), SNAPSHOT_EXPORT_DIR "/%08X-%08X-%d",
+ snprintf(path, sizeof(path), PG_SNAPSHOTS_DIR "/%08X-%08X-%d",
MyProc->backendId, MyProc->lxid, list_length(exportedSnapshots) + 1);
/*
@@ -1321,10 +1319,10 @@ ExportSnapshot(Snapshot snapshot)
/*
* The basename of the file is what we return from pg_export_snapshot().
* It's already in path in a textual format and we know that the path
- * starts with SNAPSHOT_EXPORT_DIR. Skip over the prefix and the slash
+ * starts with PG_SNAPSHOTS_DIR. Skip over the prefix and the slash
* and pstrdup it so as not to return the address of a local variable.
*/
- return pstrdup(path + strlen(SNAPSHOT_EXPORT_DIR) + 1);
+ return pstrdup(path + strlen(PG_SNAPSHOTS_DIR) + 1);
}
/*
@@ -1423,7 +1421,7 @@ parseVxidFromText(const char *prefix, char **s, const char *filename,
/*
* ImportSnapshot
* Import a previously exported snapshot. The argument should be a
- * filename in SNAPSHOT_EXPORT_DIR. Load the snapshot from that file.
+ * filename in PG_SNAPSHOTS_DIR. Load the snapshot from that file.
* This is called by "SET TRANSACTION SNAPSHOT 'foo'".
*/
void
@@ -1474,7 +1472,7 @@ ImportSnapshot(const char *idstr)
errmsg("invalid snapshot identifier: \"%s\"", idstr)));
/* OK, read the file */
- snprintf(path, MAXPGPATH, SNAPSHOT_EXPORT_DIR "/%s", idstr);
+ snprintf(path, MAXPGPATH, PG_SNAPSHOTS_DIR "/%s", idstr);
f = AllocateFile(path, PG_BINARY_R);
if (!f)
@@ -1615,7 +1613,7 @@ XactHasExportedSnapshots(void)
void
DeleteAllExportedSnapshotFiles(void)
{
- char buf[MAXPGPATH + sizeof(SNAPSHOT_EXPORT_DIR)];
+ char buf[MAXPGPATH + sizeof(PG_SNAPSHOTS_DIR)];
DIR *s_dir;
struct dirent *s_de;
@@ -1624,15 +1622,15 @@ DeleteAllExportedSnapshotFiles(void)
* LOG level. Since we're running in the startup process, ERROR level
* would prevent database start, and it's not important enough for that.
*/
- s_dir = AllocateDir(SNAPSHOT_EXPORT_DIR);
+ s_dir = AllocateDir(PG_SNAPSHOTS_DIR);
- while ((s_de = ReadDirExtended(s_dir, SNAPSHOT_EXPORT_DIR, LOG)) != NULL)
+ while ((s_de = ReadDirExtended(s_dir, PG_SNAPSHOTS_DIR, LOG)) != NULL)
{
if (strcmp(s_de->d_name, ".") == 0 ||
strcmp(s_de->d_name, "..") == 0)
continue;
- snprintf(buf, sizeof(buf), SNAPSHOT_EXPORT_DIR "/%s", s_de->d_name);
+ snprintf(buf, sizeof(buf), PG_SNAPSHOTS_DIR "/%s", s_de->d_name);
if (unlink(buf) != 0)
ereport(LOG,
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index 528fdf7f57..8a0601da66 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1188,7 +1188,7 @@ setup_config(void)
autoconflines[1] = pg_strdup("# It will be overwritten by the ALTER SYSTEM command.\n");
autoconflines[2] = NULL;
- sprintf(path, "%s/postgresql.auto.conf", pg_data);
+ sprintf(path, "%s/%s", pg_data, PG_AUTOCONF_FILENAME);
writefile(path, autoconflines);
if (chmod(path, S_IRUSR | S_IWUSR) != 0)
@@ -2756,7 +2756,7 @@ create_xlog_or_symlink(void)
char *subdirloc;
/* form name of the place for the subdirectory or symlink */
- subdirloc = psprintf("%s/pg_wal", pg_data);
+ subdirloc = psprintf("%s/%s", pg_data, XLOGDIR);
if (xlog_dir)
{
@@ -2931,7 +2931,7 @@ initialize_data_directory(void)
/*
* Make the per-database PG_VERSION for template1 only after init'ing it
*/
- write_version_file("base/1");
+ write_version_file(PG_BASE_DIR "/1");
/*
* Create the stuff we don't need to use bootstrap mode for, using a
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 9bc830b085..7dc15b13b0 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -26,6 +26,7 @@
#include "catalog/pg_control.h"
#include "common/controldata_utils.h"
#include "getopt_long.h"
+#include "pg_paths.h"
#include "utils/pidfile.h"
#ifdef WIN32 /* on Unix, we don't need libpq */
@@ -2401,9 +2402,11 @@ main(int argc, char **argv)
if (pg_data)
{
- snprintf(postopts_file, MAXPGPATH, "%s/postmaster.opts", pg_data);
+ snprintf(postopts_file, MAXPGPATH, "%s/%s", pg_data,
+ POSTMASTER_OPTS_FILE);
snprintf(version_file, MAXPGPATH, "%s/PG_VERSION", pg_data);
- snprintf(pid_file, MAXPGPATH, "%s/postmaster.pid", pg_data);
+ snprintf(pid_file, MAXPGPATH, "%s/%s", pg_data,
+ POSTMASTER_PID_FILE);
snprintf(backup_file, MAXPGPATH, "%s/backup_label", pg_data);
}
diff --git a/src/bin/pg_resetwal/pg_resetwal.c b/src/bin/pg_resetwal/pg_resetwal.c
index a132cf2e9f..673683a42e 100644
--- a/src/bin/pg_resetwal/pg_resetwal.c
+++ b/src/bin/pg_resetwal/pg_resetwal.c
@@ -55,6 +55,7 @@
#include "common/restricted_token.h"
#include "storage/large_object.h"
#include "pg_getopt.h"
+#include "pg_paths.h"
static ControlFileData ControlFile; /* pg_control values */
@@ -334,12 +335,12 @@ main(int argc, char *argv[])
* Check for a postmaster lock file --- if there is one, refuse to
* proceed, on grounds we might be interfering with a live installation.
*/
- if ((fd = open("postmaster.pid", O_RDONLY, 0)) < 0)
+ if ((fd = open(POSTMASTER_PID_FILE, O_RDONLY, 0)) < 0)
{
if (errno != ENOENT)
{
fprintf(stderr, _("%s: could not open file \"%s\" for reading: %s\n"),
- progname, "postmaster.pid", strerror(errno));
+ progname, POSTMASTER_PID_FILE, strerror(errno));
exit(1);
}
}
@@ -347,7 +348,7 @@ main(int argc, char *argv[])
{
fprintf(stderr, _("%s: lock file \"%s\" exists\n"
"Is a server running? If not, delete the lock file and try again.\n"),
- progname, "postmaster.pid");
+ progname, POSTMASTER_PID_FILE);
exit(1);
}
diff --git a/src/bin/pg_rewind/copy_fetch.c b/src/bin/pg_rewind/copy_fetch.c
index 04db409675..3a44da3502 100644
--- a/src/bin/pg_rewind/copy_fetch.c
+++ b/src/bin/pg_rewind/copy_fetch.c
@@ -22,6 +22,7 @@
#include "pg_rewind.h"
#include "catalog/catalog.h"
+#include "pg_paths.h"
static void recurse_dir(const char *datadir, const char *path,
process_file_callback_t callback);
@@ -131,8 +132,8 @@ recurse_dir(const char *datadir, const char *parentpath,
* to process all the tablespaces. We also follow a symlink if
* it's for pg_wal. Symlinks elsewhere are ignored.
*/
- if ((parentpath && strcmp(parentpath, "pg_tblspc") == 0) ||
- strcmp(path, "pg_wal") == 0)
+ if ((parentpath && strcmp(parentpath, PG_TABLESPACE_DIR) == 0) ||
+ strcmp(path, XLOGDIR) == 0)
recurse_dir(datadir, path, callback);
#else
pg_fatal("\"%s\" is a symbolic link, but symbolic links are not supported on this platform\n",
diff --git a/src/bin/pg_rewind/filemap.c b/src/bin/pg_rewind/filemap.c
index 19da1ee7e0..1c462c1041 100644
--- a/src/bin/pg_rewind/filemap.c
+++ b/src/bin/pg_rewind/filemap.c
@@ -20,6 +20,7 @@
#include "common/string.h"
#include "catalog/pg_tablespace.h"
+#include "pg_paths.h"
#include "storage/fd.h"
filemap_t *filemap = NULL;
@@ -73,8 +74,8 @@ process_source_file(const char *path, file_type_t type, size_t newsize,
/*
* Completely ignore some special files in source and destination.
*/
- if (strcmp(path, "postmaster.pid") == 0 ||
- strcmp(path, "postmaster.opts") == 0)
+ if (strcmp(path, POSTMASTER_PID_FILE) == 0 ||
+ strcmp(path, POSTMASTER_OPTS_FILE) == 0)
return;
/*
@@ -82,7 +83,7 @@ process_source_file(const char *path, file_type_t type, size_t newsize,
* don't want to mess with the symlink itself, nor complain if it's a
* symlink in source but not in target or vice versa.
*/
- if (strcmp(path, "pg_wal") == 0 && type == FILE_TYPE_SYMLINK)
+ if (strcmp(path, XLOGDIR) == 0 && type == FILE_TYPE_SYMLINK)
type = FILE_TYPE_DIRECTORY;
/*
@@ -119,7 +120,7 @@ process_source_file(const char *path, file_type_t type, size_t newsize,
switch (type)
{
case FILE_TYPE_DIRECTORY:
- if (exists && !S_ISDIR(statbuf.st_mode) && strcmp(path, "pg_wal") != 0)
+ if (exists && !S_ISDIR(statbuf.st_mode) && strcmp(path, XLOGDIR) != 0)
{
/* it's a directory in source, but not in target. Strange.. */
pg_fatal("\"%s\" is not a directory\n", localpath);
@@ -288,14 +289,14 @@ process_target_file(const char *path, file_type_t type, size_t oldsize,
/*
* Completely ignore some special files
*/
- if (strcmp(path, "postmaster.pid") == 0 ||
- strcmp(path, "postmaster.opts") == 0)
+ if (strcmp(path, POSTMASTER_PID_FILE) == 0 ||
+ strcmp(path, POSTMASTER_OPTS_FILE) == 0)
return;
/*
* Like in process_source_file, pretend that xlog is always a directory.
*/
- if (strcmp(path, "pg_wal") == 0 && type == FILE_TYPE_SYMLINK)
+ if (strcmp(path, XLOGDIR) == 0 && type == FILE_TYPE_SYMLINK)
type = FILE_TYPE_DIRECTORY;
key.path = (char *) path;
@@ -585,7 +586,7 @@ isRelDataFile(const char *path)
segNo = 0;
matched = false;
- nmatch = sscanf(path, "global/%u.%u", &rnode.relNode, &segNo);
+ nmatch = sscanf(path, PG_GLOBAL_DIR "/%u.%u", &rnode.relNode, &segNo);
if (nmatch == 1 || nmatch == 2)
{
rnode.spcNode = GLOBALTABLESPACE_OID;
@@ -594,7 +595,7 @@ isRelDataFile(const char *path)
}
else
{
- nmatch = sscanf(path, "base/%u/%u.%u",
+ nmatch = sscanf(path, PG_BASE_DIR "/%u/%u.%u",
&rnode.dbNode, &rnode.relNode, &segNo);
if (nmatch == 2 || nmatch == 3)
{
diff --git a/src/bin/pg_rewind/pg_rewind.c b/src/bin/pg_rewind/pg_rewind.c
index 72ab2f8d5e..56aabfd4af 100644
--- a/src/bin/pg_rewind/pg_rewind.c
+++ b/src/bin/pg_rewind/pg_rewind.c
@@ -210,11 +210,11 @@ main(int argc, char **argv)
* Ok, we have all the options and we're ready to start. Read in all the
* information we need from both clusters.
*/
- buffer = slurpFile(datadir_target, "global/pg_control", &size);
+ buffer = slurpFile(datadir_target, XLOG_CONTROL_FILE, &size);
digestControlFile(&ControlFile_target, buffer, size);
pg_free(buffer);
- buffer = fetchFile("global/pg_control", &size);
+ buffer = fetchFile(XLOG_CONTROL_FILE, &size);
digestControlFile(&ControlFile_source, buffer, size);
pg_free(buffer);
@@ -675,7 +675,7 @@ updateControlFile(ControlFileData *ControlFile)
memset(buffer, 0, PG_CONTROL_FILE_SIZE);
memcpy(buffer, ControlFile, sizeof(ControlFileData));
- open_target_file("global/pg_control", false);
+ open_target_file(XLOG_CONTROL_FILE, false);
write_target_range(buffer, 0, PG_CONTROL_FILE_SIZE);
diff --git a/src/bin/pg_upgrade/controldata.c b/src/bin/pg_upgrade/controldata.c
index 0fe98a550e..7dce5ecd14 100644
--- a/src/bin/pg_upgrade/controldata.c
+++ b/src/bin/pg_upgrade/controldata.c
@@ -13,6 +13,8 @@
#include <ctype.h>
+#include "pg_paths.h"
+
/*
* get_control_data()
*
@@ -614,15 +616,18 @@ disable_old_cluster(void)
/* rename pg_control so old server cannot be accidentally started */
prep_status("Adding \".old\" suffix to old global/pg_control");
- snprintf(old_path, sizeof(old_path), "%s/global/pg_control", old_cluster.pgdata);
- snprintf(new_path, sizeof(new_path), "%s/global/pg_control.old", old_cluster.pgdata);
+ snprintf(old_path, sizeof(old_path), "%s/%s", old_cluster.pgdata,
+ XLOG_CONTROL_FILE);
+ snprintf(new_path, sizeof(new_path), "%s/%s.old", old_cluster.pgdata,
+ XLOG_CONTROL_FILE);
if (pg_mv_file(old_path, new_path) != 0)
pg_fatal("Unable to rename %s to %s.\n", old_path, new_path);
check_ok();
pg_log(PG_REPORT, "\n"
"If you want to start the old cluster, you will need to remove\n"
- "the \".old\" suffix from %s/global/pg_control.old.\n"
+ "the \".old\" suffix from %s/%s.old.\n"
"Because \"link\" mode was used, the old cluster cannot be safely\n"
- "started once the new cluster has been started.\n\n", old_cluster.pgdata);
+ "started once the new cluster has been started.\n\n",
+ old_cluster.pgdata, XLOG_CONTROL_FILE);
}
diff --git a/src/bin/pg_upgrade/exec.c b/src/bin/pg_upgrade/exec.c
index 9122e2769e..4138455466 100644
--- a/src/bin/pg_upgrade/exec.c
+++ b/src/bin/pg_upgrade/exec.c
@@ -11,6 +11,7 @@
#include <fcntl.h>
+#include "pg_paths.h"
#include "pg_upgrade.h"
static void check_data_dir(ClusterInfo *cluster);
@@ -219,7 +220,8 @@ pid_lock_file_exists(const char *datadir)
char path[MAXPGPATH];
int fd;
- snprintf(path, sizeof(path), "%s/postmaster.pid", datadir);
+ snprintf(path, sizeof(path), "%s/%s", datadir,
+ POSTMASTER_PID_FILE);
if ((fd = open(path, O_RDONLY, 0)) < 0)
{
@@ -331,12 +333,12 @@ check_data_dir(ClusterInfo *cluster)
cluster->major_version = get_major_server_version(cluster);
check_single_dir(pg_data, "");
- check_single_dir(pg_data, "base");
- check_single_dir(pg_data, "global");
- check_single_dir(pg_data, "pg_multixact");
- check_single_dir(pg_data, "pg_subtrans");
- check_single_dir(pg_data, "pg_tblspc");
- check_single_dir(pg_data, "pg_twophase");
+ check_single_dir(pg_data, PG_BASE_DIR);
+ check_single_dir(pg_data, PG_GLOBAL_DIR);
+ check_single_dir(pg_data, PG_MULTIXACT_DIR);
+ check_single_dir(pg_data, PG_SUBTRANS_DIR);
+ check_single_dir(pg_data, PG_TABLESPACE_DIR);
+ check_single_dir(pg_data, PG_TWOPHASE_DIR);
/* pg_xlog has been renamed to pg_wal in v10 */
if (GET_MAJOR_VERSION(cluster->major_version) < 1000)
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index 9dbc9225a6..6d96c788b3 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -15,6 +15,7 @@
#endif
#include "getopt_long.h"
+#include "pg_paths.h"
#include "utils/pidfile.h"
#include "pg_upgrade.h"
@@ -474,8 +475,8 @@ get_sock_dir(ClusterInfo *cluster, bool live_check)
FILE *fp;
int lineno;
- snprintf(filename, sizeof(filename), "%s/postmaster.pid",
- cluster->pgdata);
+ snprintf(filename, sizeof(filename), "%s/%s",
+ cluster->pgdata, POSTMASTER_PID_FILE);
if ((fp = fopen(filename, "r")) == NULL)
pg_fatal("could not open file \"%s\": %s\n",
filename, strerror(errno));
diff --git a/src/bin/pg_upgrade/pg_upgrade.c b/src/bin/pg_upgrade/pg_upgrade.c
index a67e484a85..065aaaaded 100644
--- a/src/bin/pg_upgrade/pg_upgrade.c
+++ b/src/bin/pg_upgrade/pg_upgrade.c
@@ -40,6 +40,7 @@
#include "catalog/pg_class.h"
#include "common/restricted_token.h"
#include "fe_utils/string_utils.h"
+#include "pg_paths.h"
#ifdef HAVE_LANGINFO_H
#include <langinfo.h>
@@ -442,8 +443,8 @@ copy_xact_xlog_xid(void)
if (old_cluster.controldata.cat_ver >= MULTIXACT_FORMATCHANGE_CAT_VER &&
new_cluster.controldata.cat_ver >= MULTIXACT_FORMATCHANGE_CAT_VER)
{
- copy_subdir_files("pg_multixact/offsets", "pg_multixact/offsets");
- copy_subdir_files("pg_multixact/members", "pg_multixact/members");
+ copy_subdir_files(PG_MULTIXACT_OFFSETS_DIR, PG_MULTIXACT_OFFSETS_DIR);
+ copy_subdir_files(PG_MULTIXACT_MEMBERS_DIR, PG_MULTIXACT_MEMBERS_DIR);
prep_status("Setting next multixact ID and offset for new cluster");
@@ -467,7 +468,7 @@ copy_xact_xlog_xid(void)
* the new multi-xid value. "members" starts at zero so no need to
* remove it.
*/
- remove_new_subdir("pg_multixact/offsets", false);
+ remove_new_subdir(PG_MULTIXACT_OFFSETS_DIR, false);
prep_status("Setting oldest multixact ID in new cluster");
diff --git a/src/common/controldata_utils.c b/src/common/controldata_utils.c
index f12a188856..a93a567615 100644
--- a/src/common/controldata_utils.c
+++ b/src/common/controldata_utils.c
@@ -26,6 +26,7 @@
#include "catalog/pg_control.h"
#include "common/controldata_utils.h"
+#include "pg_paths.h"
#include "port/pg_crc32c.h"
/*
@@ -48,7 +49,8 @@ get_controlfile(const char *DataDir, const char *progname, bool *crc_ok_p)
AssertArg(crc_ok_p);
ControlFile = palloc(sizeof(ControlFileData));
- snprintf(ControlFilePath, MAXPGPATH, "%s/global/pg_control", DataDir);
+ snprintf(ControlFilePath, MAXPGPATH, "%s/%s", DataDir,
+ XLOG_CONTROL_FILE);
if ((fd = open(ControlFilePath, O_RDONLY | PG_BINARY, 0)) == -1)
#ifndef FRONTEND
diff --git a/src/common/relpath.c b/src/common/relpath.c
index d98050c590..e34aa4c2d1 100644
--- a/src/common/relpath.c
+++ b/src/common/relpath.c
@@ -21,6 +21,7 @@
#include "catalog/catalog.h"
#include "catalog/pg_tablespace.h"
#include "common/relpath.h"
+#include "pg_paths.h"
#include "storage/backendid.h"
@@ -111,17 +112,17 @@ GetDatabasePath(Oid dbNode, Oid spcNode)
{
/* Shared system relations live in {datadir}/global */
Assert(dbNode == 0);
- return pstrdup("global");
+ return pstrdup(PG_GLOBAL_DIR);
}
else if (spcNode == DEFAULTTABLESPACE_OID)
{
/* The default tablespace is {datadir}/base */
- return psprintf("base/%u", dbNode);
+ return psprintf(PG_BASE_DIR "/%u", dbNode);
}
else
{
/* All other tablespaces are accessed via symlinks */
- return psprintf("pg_tblspc/%u/%s/%u",
+ return psprintf(PG_TABLESPACE_DIR "/%u/%s/%u",
spcNode, TABLESPACE_VERSION_DIRECTORY, dbNode);
}
}
@@ -147,10 +148,11 @@ GetRelationPath(Oid dbNode, Oid spcNode, Oid relNode,
Assert(dbNode == 0);
Assert(backendId == InvalidBackendId);
if (forkNumber != MAIN_FORKNUM)
- path = psprintf("global/%u_%s",
+ path = psprintf("%s/%u_%s",
+ PG_GLOBAL_DIR,
relNode, forkNames[forkNumber]);
else
- path = psprintf("global/%u", relNode);
+ path = psprintf("%s/%u", PG_GLOBAL_DIR, relNode);
}
else if (spcNode == DEFAULTTABLESPACE_OID)
{
@@ -158,22 +160,22 @@ GetRelationPath(Oid dbNode, Oid spcNode, Oid relNode,
if (backendId == InvalidBackendId)
{
if (forkNumber != MAIN_FORKNUM)
- path = psprintf("base/%u/%u_%s",
- dbNode, relNode,
+ path = psprintf("%s/%u/%u_%s",
+ PG_BASE_DIR, dbNode, relNode,
forkNames[forkNumber]);
else
- path = psprintf("base/%u/%u",
- dbNode, relNode);
+ path = psprintf("%s/%u/%u",
+ PG_BASE_DIR, dbNode, relNode);
}
else
{
if (forkNumber != MAIN_FORKNUM)
- path = psprintf("base/%u/t%d_%u_%s",
- dbNode, backendId, relNode,
+ path = psprintf("%s/%u/t%d_%u_%s",
+ PG_BASE_DIR, dbNode, backendId, relNode,
forkNames[forkNumber]);
else
- path = psprintf("base/%u/t%d_%u",
- dbNode, backendId, relNode);
+ path = psprintf("%s/%u/t%d_%u",
+ PG_BASE_DIR, dbNode, backendId, relNode);
}
}
else
--
2.16.1
0003-Add-backup_label-pg_internal.init-and-tablespace_map.patchtext/plain; charset=us-asciiDownload
From b01fbf41f6d9d3b1fe211c8ea2a929ce36d1e658 Mon Sep 17 00:00:00 2001
From: Michael Paquier <michael@paquier.xyz>
Date: Mon, 5 Feb 2018 15:01:42 +0900
Subject: [PATCH 3/5] Add backup_label, pg_internal.init and tablespace_map to
pg_paths.h
Those are going to be used in the list of files to be excluded from base
backups which is made available to the frontends.
---
src/bin/pg_ctl/pg_ctl.c | 3 ++-
src/bin/pg_rewind/pg_rewind.c | 2 +-
src/include/access/xlog.h | 7 -------
src/include/pg_paths.h | 17 +++++++++++++++++
src/include/utils/relcache.h | 5 -----
5 files changed, 20 insertions(+), 14 deletions(-)
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 7dc15b13b0..7f9106a527 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -2407,7 +2407,8 @@ main(int argc, char **argv)
snprintf(version_file, MAXPGPATH, "%s/PG_VERSION", pg_data);
snprintf(pid_file, MAXPGPATH, "%s/%s", pg_data,
POSTMASTER_PID_FILE);
- snprintf(backup_file, MAXPGPATH, "%s/backup_label", pg_data);
+ snprintf(backup_file, MAXPGPATH, "%s/%s", pg_data,
+ BACKUP_LABEL_FILE);
}
switch (ctl_command)
diff --git a/src/bin/pg_rewind/pg_rewind.c b/src/bin/pg_rewind/pg_rewind.c
index 56aabfd4af..bbd514f6f1 100644
--- a/src/bin/pg_rewind/pg_rewind.c
+++ b/src/bin/pg_rewind/pg_rewind.c
@@ -597,7 +597,7 @@ createBackupLabel(XLogRecPtr startpoint, TimeLineID starttli, XLogRecPtr checkpo
pg_fatal("backup label buffer too small\n"); /* shouldn't happen */
/* TODO: move old file out of the way, if any. */
- open_target_file("backup_label", true); /* BACKUP_LABEL_FILE */
+ open_target_file(BACKUP_LABEL_FILE, true);
write_target_range(buf, 0, len);
close_target_file();
}
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 421ba6d775..ca4cbc811a 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -318,11 +318,4 @@ extern XLogRecPtr do_pg_stop_backup(char *labelfile, bool waitforarchive,
extern void do_pg_abort_backup(void);
extern SessionBackupState get_backup_status(void);
-/* File path names (all relative to $PGDATA) */
-#define BACKUP_LABEL_FILE "backup_label"
-#define BACKUP_LABEL_OLD "backup_label.old"
-
-#define TABLESPACE_MAP "tablespace_map"
-#define TABLESPACE_MAP_OLD "tablespace_map.old"
-
#endif /* XLOG_H */
diff --git a/src/include/pg_paths.h b/src/include/pg_paths.h
index a4746e75e4..f033f8f390 100644
--- a/src/include/pg_paths.h
+++ b/src/include/pg_paths.h
@@ -13,6 +13,23 @@
#ifndef PG_PATHS_H
#define PG_PATHS_H
+/*
+ * Backup label file names used in base backups.
+ */
+#define BACKUP_LABEL_FILE "backup_label"
+#define BACKUP_LABEL_OLD "backup_label.old"
+
+/*
+ * Tablespace map files used in base backups.
+ */
+#define TABLESPACE_MAP "tablespace_map"
+#define TABLESPACE_MAP_OLD "tablespace_map.old"
+
+/*
+ * Name of relcache init file(s), used to speed up backend startup
+ */
+#define RELCACHE_INIT_FILENAME "pg_internal.init"
+
/*
* Name of files saving meta-data information about the log
* files currently in use by the syslogger
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 8a546aba28..7a884cb26a 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -18,11 +18,6 @@
#include "nodes/bitmapset.h"
-/*
- * Name of relcache init file(s), used to speed up backend startup
- */
-#define RELCACHE_INIT_FILENAME "pg_internal.init"
-
typedef struct RelationData *Relation;
/* ----------------
--
2.16.1
0004-Move-base-backup-filter-lists-into-their-own-header-.patchtext/plain; charset=us-asciiDownload
From 7aaafd2631386bb2bd7e628334a8409fbb648402 Mon Sep 17 00:00:00 2001
From: Michael Paquier <michael@paquier.xyz>
Date: Mon, 5 Feb 2018 14:43:10 +0900
Subject: [PATCH 4/5] Move base backup filter lists into their own header file
This list is for now consumed only by the backend-side base backup code,
which is used to filter out a set of paths and/or files which cannot be
included in base backup data sent to clients.
A follow-up patch will make use of that in pg_rewind itself.
---
src/backend/replication/basebackup.c | 82 ++-------------------------
src/include/replication/basebackup_paths.h | 89 ++++++++++++++++++++++++++++++
2 files changed, 94 insertions(+), 77 deletions(-)
create mode 100644 src/include/replication/basebackup_paths.h
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index 2e764ab293..2ea2306cfa 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -28,6 +28,7 @@
#include "pgstat.h"
#include "postmaster/syslogger.h"
#include "replication/basebackup.h"
+#include "replication/basebackup_paths.h"
#include "replication/walsender.h"
#include "replication/walsender_private.h"
#include "storage/dsm_impl.h"
@@ -98,79 +99,6 @@ static TimeOffset elapsed_min_unit;
/* The last check of the transfer rate. */
static TimestampTz throttled_last;
-/*
- * The contents of these directories are removed or recreated during server
- * start so they are not included in backups. The directories themselves are
- * kept and included as empty to preserve access permissions.
- */
-static const char *excludeDirContents[] =
-{
- /*
- * Skip temporary statistics files. PG_STAT_TMP_DIR must be skipped even
- * when stats_temp_directory is set because PGSS_TEXT_FILE is always
- * created there.
- */
- PG_STAT_TMP_DIR,
-
- /*
- * It is generally not useful to backup the contents of this directory
- * even if the intention is to restore to another master. See backup.sgml
- * for a more detailed description.
- */
- PG_REPLSLOT_DIR,
-
- /* Contents removed on startup, see dsm_cleanup_for_mmap(). */
- PG_DYNSHMEM_DIR,
-
- /* Contents removed on startup, see AsyncShmemInit(). */
- PG_NOTIFY_DIR,
-
- /*
- * Old contents are loaded for possible debugging but are not required for
- * normal operation, see OldSerXidInit().
- */
- PG_SERIAL_DIR,
-
- /* Contents removed on startup, see DeleteAllExportedSnapshotFiles(). */
- PG_SNAPSHOTS_DIR,
-
- /* Contents zeroed on startup, see StartupSUBTRANS(). */
- PG_SUBTRANS_DIR,
-
- /* end of list */
- NULL
-};
-
-/*
- * List of files excluded from backups.
- */
-static const char *excludeFiles[] =
-{
- /* Skip auto conf temporary file. */
- PG_AUTOCONF_FILENAME_TMP,
-
- /* Skip current log file temporary file */
- LOG_METAINFO_DATAFILE_TMP,
-
- /* Skip relation cache because it is rebuilt on startup */
- RELCACHE_INIT_FILENAME,
-
- /*
- * If there's a backup_label or tablespace_map file, it belongs to a
- * backup started by the user with pg_start_backup(). It is *not* correct
- * for this backup. Our backup_label/tablespace_map is injected into the
- * tar separately.
- */
- BACKUP_LABEL_FILE,
- TABLESPACE_MAP,
-
- POSTMASTER_PID_FILE,
- POSTMASTER_OPTS_FILE,
-
- /* end of list */
- NULL
-};
-
/*
* Called when ERROR or FATAL happens in perform_base_backup() after
* we have started the backup - make sure we end it!
@@ -995,9 +923,9 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
/* Scan for files that should be excluded */
excludeFound = false;
- for (excludeIdx = 0; excludeFiles[excludeIdx] != NULL; excludeIdx++)
+ for (excludeIdx = 0; backupExcludeFiles[excludeIdx] != NULL; excludeIdx++)
{
- if (strcmp(de->d_name, excludeFiles[excludeIdx]) == 0)
+ if (strcmp(de->d_name, backupExcludeFiles[excludeIdx]) == 0)
{
elog(DEBUG1, "file \"%s\" excluded from backup", de->d_name);
excludeFound = true;
@@ -1028,9 +956,9 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
/* Scan for directories whose contents should be excluded */
excludeFound = false;
- for (excludeIdx = 0; excludeDirContents[excludeIdx] != NULL; excludeIdx++)
+ for (excludeIdx = 0; backupExcludeDirs[excludeIdx] != NULL; excludeIdx++)
{
- if (strcmp(de->d_name, excludeDirContents[excludeIdx]) == 0)
+ if (strcmp(de->d_name, backupExcludeDirs[excludeIdx]) == 0)
{
elog(DEBUG1, "contents of directory \"%s\" excluded from backup", de->d_name);
size += _tarWriteDir(pathbuf, basepathlen, &statbuf, sizeonly);
diff --git a/src/include/replication/basebackup_paths.h b/src/include/replication/basebackup_paths.h
new file mode 100644
index 0000000000..64c9f20333
--- /dev/null
+++ b/src/include/replication/basebackup_paths.h
@@ -0,0 +1,89 @@
+/*-------------------------------------------------------------------------
+ *
+ * basebackup_paths.h
+ * Filter lists when working on base backups. Can be used by both
+ * frontends and backends.
+ *
+ * Portions Copyright (c) 2018, PostgreSQL Global Development Group
+ *
+ * src/include/replication/basebackup_paths.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef _BASEBACKUP_PATHS_H
+#define _BASEBACKUP_PATHS_H
+
+/*
+ * The contents of these directories are removed or recreated during server
+ * start so they are not included in backups. The directories themselves are
+ * kept and included as empty to preserve access permissions.
+ */
+static const char *backupExcludeDirs[] =
+{
+ /*
+ * Skip temporary statistics files. PG_STAT_TMP_DIR must be skipped even
+ * when stats_temp_directory is set because PGSS_TEXT_FILE is always
+ * created there.
+ */
+ PG_STAT_TMP_DIR,
+
+ /*
+ * It is generally not useful to backup the contents of this directory
+ * even if the intention is to restore to another master. See backup.sgml
+ * for a more detailed description.
+ */
+ PG_REPLSLOT_DIR,
+
+ /* Contents removed on startup, see dsm_cleanup_for_mmap(). */
+ PG_DYNSHMEM_DIR,
+
+ /* Contents removed on startup, see AsyncShmemInit(). */
+ PG_NOTIFY_DIR,
+
+ /*
+ * Old contents are loaded for possible debugging but are not required for
+ * normal operation, see OldSerXidInit().
+ */
+ PG_SERIAL_DIR,
+
+ /* Contents removed on startup, see DeleteAllExportedSnapshotFiles(). */
+ PG_SNAPSHOTS_DIR,
+
+ /* Contents zeroed on startup, see StartupSUBTRANS(). */
+ PG_SUBTRANS_DIR,
+
+ /* end of list */
+ NULL
+};
+
+/*
+ * List of files excluded from base backups.
+ */
+static const char *backupExcludeFiles[] =
+{
+ /* Skip auto conf temporary file. */
+ PG_AUTOCONF_FILENAME_TMP,
+
+ /* Skip current log file temporary file */
+ LOG_METAINFO_DATAFILE_TMP,
+
+ /* Skip relation cache because it is rebuilt on startup */
+ RELCACHE_INIT_FILENAME,
+
+ /*
+ * If there's a backup_label or tablespace_map file, it belongs to a
+ * backup started by the user with pg_start_backup(). It is *not* correct
+ * for this backup. Our backup_label/tablespace_map is injected into the
+ * tar separately.
+ */
+ BACKUP_LABEL_FILE,
+ TABLESPACE_MAP,
+
+ POSTMASTER_PID_FILE,
+ POSTMASTER_OPTS_FILE,
+
+ /* end of list */
+ NULL
+};
+
+#endif /* _BASEBACKUP_PATHS_H */
--
2.16.1
0005-Use-filtering-list-of-base-backups-in-pg_rewind-to-e.patchtext/plain; charset=us-asciiDownload
From 9ac82aebd2de8d3821633fcbc0838bd872f2e423 Mon Sep 17 00:00:00 2001
From: Michael Paquier <michael@paquier.xyz>
Date: Mon, 5 Feb 2018 15:48:35 +0900
Subject: [PATCH 5/5] Use filtering list of base backups in pg_rewind to
exclude more content
After being rewound, a standby to-be-recycled needs to perform recovery
from the last checkpoint where WAL forked after a promotion, which leads
it to automatically remove some files which may have been copied from
the source cluster. Make use of the same filtering list as base backups
to find out what is this data and then remove it. This reduces the
amount of data transferred during a rewind without changing the
usefulness of the operation. This takes advantage of the
newly-introduced basebackup_paths.h to allow pg_rewind to have access to
this data.
Documentation is updated to take into account what is filtered out.
---
doc/src/sgml/ref/pg_rewind.sgml | 12 ++++++++-
src/bin/pg_rewind/filemap.c | 60 +++++++++++++++++++++++++++++++++++------
2 files changed, 63 insertions(+), 9 deletions(-)
diff --git a/doc/src/sgml/ref/pg_rewind.sgml b/doc/src/sgml/ref/pg_rewind.sgml
index 8e49249826..ccd0ae505e 100644
--- a/doc/src/sgml/ref/pg_rewind.sgml
+++ b/doc/src/sgml/ref/pg_rewind.sgml
@@ -231,7 +231,17 @@ PostgreSQL documentation
<para>
Copy all other files such as <filename>pg_xact</filename> and
configuration files from the source cluster to the target cluster
- (everything except the relation files).
+ (everything except the relation files). Similarly to base backups,
+ the contents of the directories <filename>pg_dynshmem/</filename>,
+ <filename>pg_notify/</filename>, <filename>pg_replslot/</filename>,
+ <filename>pg_serial/</filename>, <filename>pg_snapshots/</filename>,
+ <filename>pg_stat_tmp/</filename>, and
+ <filename>pg_subtrans/</filename> are omitted from the data copied
+ from the source cluster. Any file or directory beginning with
+ <filename>pgsql_tmp</filename> is omitted, as well as are
+ <filename>pg_internal.init</filename>,
+ <filename>postmaster.opts</filename> and
+ <filename>postmaster.pid</filename>.
</para>
</step>
<step>
diff --git a/src/bin/pg_rewind/filemap.c b/src/bin/pg_rewind/filemap.c
index 1c462c1041..0a6426e6c2 100644
--- a/src/bin/pg_rewind/filemap.c
+++ b/src/bin/pg_rewind/filemap.c
@@ -21,6 +21,7 @@
#include "common/string.h"
#include "catalog/pg_tablespace.h"
#include "pg_paths.h"
+#include "replication/basebackup_paths.h"
#include "storage/fd.h"
filemap_t *filemap = NULL;
@@ -68,15 +69,37 @@ process_source_file(const char *path, file_type_t type, size_t newsize,
file_action_t action = FILE_ACTION_NONE;
size_t oldsize = 0;
file_entry_t *entry;
+ int excludeIdx;
Assert(map->array == NULL);
/*
- * Completely ignore some special files in source and destination.
+ * Completely ignore some special files in source and destination. This
+ * filters willingly any files matching an entry in the list of files to
+ * filter out.
*/
- if (strcmp(path, POSTMASTER_PID_FILE) == 0 ||
- strcmp(path, POSTMASTER_OPTS_FILE) == 0)
- return;
+ for (excludeIdx = 0; backupExcludeFiles[excludeIdx] != NULL; excludeIdx++)
+ {
+ if (strstr(path, backupExcludeFiles[excludeIdx]) != NULL)
+ {
+ pg_log(PG_DEBUG, "file \"%s\" excluded from source file list\n",
+ path);
+ return;
+ }
+ }
+
+ /*
+ * ... And ignore some directories.
+ */
+ for (excludeIdx = 0; backupExcludeDirs[excludeIdx] != NULL; excludeIdx++)
+ {
+ if (strcmp(path, backupExcludeDirs[excludeIdx]) == 0)
+ {
+ pg_log(PG_DEBUG, "directory \"%s\" excluded from source file list\n",
+ path);
+ return;
+ }
+ }
/*
* Pretend that pg_wal is a directory, even if it's really a symlink. We
@@ -259,6 +282,7 @@ process_target_file(const char *path, file_type_t type, size_t oldsize,
file_entry_t *key_ptr;
filemap_t *map = filemap;
file_entry_t *entry;
+ int excludeIdx;
snprintf(localpath, sizeof(localpath), "%s/%s", datadir_target, path);
if (lstat(localpath, &statbuf) < 0)
@@ -287,11 +311,31 @@ process_target_file(const char *path, file_type_t type, size_t oldsize,
}
/*
- * Completely ignore some special files
+ * Completely ignore some special files. This filters willingly any files
+ * matching an entry in the list of files to filter out.
*/
- if (strcmp(path, POSTMASTER_PID_FILE) == 0 ||
- strcmp(path, POSTMASTER_OPTS_FILE) == 0)
- return;
+ for (excludeIdx = 0; backupExcludeFiles[excludeIdx] != NULL; excludeIdx++)
+ {
+ if (strstr(path, backupExcludeFiles[excludeIdx]) != NULL)
+ {
+ pg_log(PG_DEBUG, "file \"%s\" excluded from target file list\n",
+ path);
+ return;
+ }
+ }
+
+ /*
+ * ... And ignore some directories.
+ */
+ for (excludeIdx = 0; backupExcludeDirs[excludeIdx] != NULL; excludeIdx++)
+ {
+ if (strcmp(path, backupExcludeDirs[excludeIdx]) == 0)
+ {
+ pg_log(PG_DEBUG, "directory \"%s\" excluded from target file list\n",
+ path);
+ return;
+ }
+ }
/*
* Like in process_source_file, pretend that xlog is always a directory.
--
2.16.1
05.02.2018 10:10, Michael Paquier:
So the patch set attached is made of the following:
- 0001, which refactors all hardcoded system paths into pg_paths.h.
This modifies only initdb.c and basebackup.c to ease reviews.
- 0002 spreads the path changes and the use of pg_paths.h across the
core code.
- 0003 moves the last set of definitions with backup_label,
tablespace_map and pg_internal.init.
- 0004 creates basebackup_paths.h, this can be consumed by pg_rewind.
- 0005 makes the changes for pg_rewind.
Thank you for this set of patches.
This refactoring makes code way more convenient to read and change.
Due to some merge conflicts, patch 0002 was not applying clearly.
So I attach the updated version.
I also noticed a couple of rows that were not updated, and wrote a patch
0006,
which contains just minor changes and can be applied on top of any patch
after 0003.
Since these patches contain mostly cosmetic changes and do not break
anything,
I think it's fine to mark this thread as Ready For Committer without
long discussion.
--
Anastasia Lubennikova
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Attachments:
0001-Refactor-path-definitions-into-a-single-header-file-.patchtext/x-patch; name=0001-Refactor-path-definitions-into-a-single-header-file-.patchDownload
From 6698d430ffb6bd2e33aba0b21dc2a9358cf6328c Mon Sep 17 00:00:00 2001
From: Michael Paquier <michael@paquier.xyz>
Date: Mon, 5 Feb 2018 13:03:38 +0900
Subject: [PATCH 1/5] Refactor path definitions into a single header file,
pg_paths.h
The definition of all internal paths of PostgreSQL are spread across the
code base, making tracing of those definitions difficult to begin with,
and resulting in a lot of code paths to be hardcoded. While this has no
real practical consequences in practice, this makes the creation of
lists manipulating those paths less maintainable as as things stand the
already-hardcoded paths need to be copied around more. In order to
improve things for the long-term, move all those definitions into a
single header file.
This commit does a first step by switching basebackup.c and initdb.c to
use them. An upcoming commit will make all the definitions across the
code base use this new header more consistently.
---
src/backend/postmaster/postmaster.c | 1 +
src/backend/postmaster/syslogger.c | 1 +
src/backend/replication/basebackup.c | 16 ++--
src/backend/storage/ipc/dsm.c | 1 +
src/backend/storage/ipc/dsm_impl.c | 1 +
src/backend/storage/lmgr/predicate.c | 3 +-
src/backend/utils/adt/misc.c | 1 +
src/bin/initdb/initdb.c | 45 ++++++------
src/include/access/xlog_internal.h | 7 +-
src/include/pg_paths.h | 137 +++++++++++++++++++++++++++++++++++
src/include/pgstat.h | 3 -
src/include/postmaster/syslogger.h | 7 --
src/include/storage/dsm_impl.h | 9 ---
src/include/utils/guc.h | 7 --
14 files changed, 176 insertions(+), 63 deletions(-)
create mode 100644 src/include/pg_paths.h
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index f3ddf828bb..66d80914a0 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -105,6 +105,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pg_getopt.h"
+#include "pg_paths.h"
#include "pgstat.h"
#include "port/pg_bswap.h"
#include "postmaster/autovacuum.h"
diff --git a/src/backend/postmaster/syslogger.c b/src/backend/postmaster/syslogger.c
index f70eea37df..c8770f6c61 100644
--- a/src/backend/postmaster/syslogger.c
+++ b/src/backend/postmaster/syslogger.c
@@ -35,6 +35,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "nodes/pg_list.h"
+#include "pg_paths.h"
#include "pgstat.h"
#include "pgtime.h"
#include "postmaster/fork_process.h"
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index dd7ad64862..63ab81f837 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -117,25 +117,25 @@ static const char *excludeDirContents[] =
* even if the intention is to restore to another master. See backup.sgml
* for a more detailed description.
*/
- "pg_replslot",
+ PG_REPLSLOT_DIR,
/* Contents removed on startup, see dsm_cleanup_for_mmap(). */
PG_DYNSHMEM_DIR,
/* Contents removed on startup, see AsyncShmemInit(). */
- "pg_notify",
+ PG_NOTIFY_DIR,
/*
* Old contents are loaded for possible debugging but are not required for
* normal operation, see OldSerXidInit().
*/
- "pg_serial",
+ PG_SERIAL_DIR,
/* Contents removed on startup, see DeleteAllExportedSnapshotFiles(). */
- "pg_snapshots",
+ PG_SNAPSHOTS_DIR,
/* Contents zeroed on startup, see StartupSUBTRANS(). */
- "pg_subtrans",
+ PG_SUBTRANS_DIR,
/* end of list */
NULL
@@ -147,7 +147,7 @@ static const char *excludeDirContents[] =
static const char *excludeFiles[] =
{
/* Skip auto conf temporary file. */
- PG_AUTOCONF_FILENAME ".tmp",
+ PG_AUTOCONF_FILENAME_TMP,
/* Skip current log file temporary file */
LOG_METAINFO_DATAFILE_TMP,
@@ -164,8 +164,8 @@ static const char *excludeFiles[] =
BACKUP_LABEL_FILE,
TABLESPACE_MAP,
- "postmaster.pid",
- "postmaster.opts",
+ POSTMASTER_PID_FILE,
+ POSTMASTER_OPTS_FILE,
/* end of list */
NULL
diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c
index f1f75b73f5..a0d69aa0d8 100644
--- a/src/backend/storage/ipc/dsm.c
+++ b/src/backend/storage/ipc/dsm.c
@@ -35,6 +35,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pg_paths.h"
#include "storage/dsm.h"
#include "storage/ipc.h"
#include "storage/lwlock.h"
diff --git a/src/backend/storage/ipc/dsm_impl.c b/src/backend/storage/ipc/dsm_impl.c
index 67e76b98fe..a2bb1e2288 100644
--- a/src/backend/storage/ipc/dsm_impl.c
+++ b/src/backend/storage/ipc/dsm_impl.c
@@ -62,6 +62,7 @@
#endif
#include "pgstat.h"
+#include "pg_paths.h"
#include "portability/mem.h"
#include "storage/dsm_impl.h"
#include "storage/fd.h"
diff --git a/src/backend/storage/lmgr/predicate.c b/src/backend/storage/lmgr/predicate.c
index d1ff2b1edc..731ed66251 100644
--- a/src/backend/storage/lmgr/predicate.c
+++ b/src/backend/storage/lmgr/predicate.c
@@ -194,6 +194,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "miscadmin.h"
+#include "pg_paths.h"
#include "pgstat.h"
#include "storage/bufmgr.h"
#include "storage/predicate.h"
@@ -806,7 +807,7 @@ OldSerXidInit(void)
*/
OldSerXidSlruCtl->PagePrecedes = OldSerXidPagePrecedesLogically;
SimpleLruInit(OldSerXidSlruCtl, "oldserxid",
- NUM_OLDSERXID_BUFFERS, 0, OldSerXidLock, "pg_serial",
+ NUM_OLDSERXID_BUFFERS, 0, OldSerXidLock, PG_SERIAL_DIR,
LWTRANCHE_OLDSERXID_BUFFERS);
/* Override default assumption that writes should be fsync'd */
OldSerXidSlruCtl->do_fsync = false;
diff --git a/src/backend/utils/adt/misc.c b/src/backend/utils/adt/misc.c
index 2e1e020c4b..9f98c03173 100644
--- a/src/backend/utils/adt/misc.c
+++ b/src/backend/utils/adt/misc.c
@@ -30,6 +30,7 @@
#include "common/keywords.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "pg_paths.h"
#include "pgstat.h"
#include "parser/scansup.h"
#include "postmaster/syslogger.h"
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index 2efd3b75b1..528fdf7f57 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -72,6 +72,7 @@
#include "getopt_long.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
+#include "pg_paths.h"
/* Ideally this would be in a .h file, but it hardly seems worth the trouble */
@@ -199,28 +200,28 @@ static const char *boot_options = "-F";
static const char *backend_options = "--single -F -O -j -c search_path=pg_catalog -c exit_on_error=true";
static const char *const subdirs[] = {
- "global",
- "pg_wal/archive_status",
- "pg_commit_ts",
- "pg_dynshmem",
- "pg_notify",
- "pg_serial",
- "pg_snapshots",
- "pg_subtrans",
- "pg_twophase",
- "pg_multixact",
- "pg_multixact/members",
- "pg_multixact/offsets",
- "base",
- "base/1",
- "pg_replslot",
- "pg_tblspc",
- "pg_stat",
- "pg_stat_tmp",
- "pg_xact",
- "pg_logical",
- "pg_logical/snapshots",
- "pg_logical/mappings"
+ PG_GLOBAL_DIR,
+ XLOG_ARCHIVE_STATUS_DIR,
+ PG_COMMIT_TS_DIR,
+ PG_DYNSHMEM_DIR,
+ PG_NOTIFY_DIR,
+ PG_SERIAL_DIR,
+ PG_SNAPSHOTS_DIR,
+ PG_SUBTRANS_DIR,
+ PG_TWOPHASE_DIR,
+ PG_MULTIXACT_DIR,
+ PG_MULTIXACT_MEMBERS_DIR,
+ PG_MULTIXACT_OFFSETS_DIR,
+ PG_BASE_DIR,
+ PG_BASE_DIR "/1",
+ PG_REPLSLOT_DIR,
+ PG_TABLESPACE_DIR,
+ PG_STAT_DIR,
+ PG_STAT_TMP_DIR,
+ PG_XACT_DIR,
+ PG_LOGICAL_DIR,
+ PG_LOGICAL_SNAPSHOTS_DIR,
+ PG_LOGICAL_MAPPINGS_DIR
};
diff --git a/src/include/access/xlog_internal.h b/src/include/access/xlog_internal.h
index a5c074642f..4ab8be61b5 100644
--- a/src/include/access/xlog_internal.h
+++ b/src/include/access/xlog_internal.h
@@ -23,6 +23,7 @@
#include "access/xlogreader.h"
#include "datatype/timestamp.h"
#include "lib/stringinfo.h"
+#include "pg_paths.h"
#include "pgtime.h"
#include "storage/block.h"
#include "storage/relfilenode.h"
@@ -137,12 +138,6 @@ typedef XLogLongPageHeaderData *XLogLongPageHeader;
#define XRecOffIsValid(xlrp) \
((xlrp) % XLOG_BLCKSZ >= SizeOfXLogShortPHD)
-/*
- * The XLog directory and control file (relative to $PGDATA)
- */
-#define XLOGDIR "pg_wal"
-#define XLOG_CONTROL_FILE "global/pg_control"
-
/*
* These macros encapsulate knowledge about the exact layout of XLog file
* names, timeline history file names, and archive-status file names.
diff --git a/src/include/pg_paths.h b/src/include/pg_paths.h
new file mode 100644
index 0000000000..a4746e75e4
--- /dev/null
+++ b/src/include/pg_paths.h
@@ -0,0 +1,137 @@
+/* ----------
+ * pg_paths.h
+ *
+ * Definitions for paths internal to PostgreSQL instances. This can
+ * be used by both frontend and backend.
+ *
+ * Copyright (c) 2018, PostgreSQL Global Development Group
+ *
+ * src/include/pg_paths.h
+ * ----------
+ */
+
+#ifndef PG_PATHS_H
+#define PG_PATHS_H
+
+/*
+ * Name of files saving meta-data information about the log
+ * files currently in use by the syslogger
+ */
+#define LOG_METAINFO_DATAFILE "current_logfiles"
+#define LOG_METAINFO_DATAFILE_TMP LOG_METAINFO_DATAFILE ".tmp"
+
+/*
+ * A file recording the command-line options the server was
+ * last started with.
+ */
+#define POSTMASTER_OPTS_FILE "postmaster.opts"
+
+/*
+ * Lock file recording data about the current postmaster process, like
+ * its PID.
+ */
+#define POSTMASTER_PID_FILE "postmaster.pid"
+
+/*
+ * Automatic configuration file name for ALTER SYSTEM.
+ * This file will be used to store values of configuration parameters
+ * set by ALTER SYSTEM command.
+ */
+#define PG_AUTOCONF_FILENAME "postgresql.auto.conf"
+#define PG_AUTOCONF_FILENAME_TMP PG_AUTOCONF_FILENAME ".tmp"
+
+/*
+ * Directory containing transaction commit timestamp data.
+ */
+#define PG_COMMIT_TS_DIR "pg_commit_ts"
+
+/*
+ * Directory for on-disk state of dynamic shared memory segments.
+ *
+ * This is used by all implementations for crash recovery and by the mmap
+ * implementation for storage.
+ */
+#define PG_DYNSHMEM_DIR "pg_dynshmem"
+#define PG_DYNSHMEM_MMAP_FILE_PREFIX "mmap."
+
+/*
+ * Directory containing cluster-wide tables.
+ */
+#define PG_GLOBAL_DIR "global"
+
+/*
+ * Directory containing per-database subdirectories
+ */
+#define PG_BASE_DIR "base"
+
+/*
+ * WAL directory, archive status directory and control file.
+ */
+#define XLOGDIR "pg_wal"
+#define XLOG_ARCHIVE_STATUS_DIR XLOGDIR "/archive_status"
+#define XLOG_CONTROL_FILE PG_GLOBAL_DIR "/pg_control"
+
+/*
+ * Directory containing status data for logical decoding.
+ */
+#define PG_LOGICAL_DIR "pg_logical"
+#define PG_LOGICAL_MAPPINGS_DIR PG_LOGICAL_DIR "/mappings"
+#define PG_LOGICAL_SNAPSHOTS_DIR PG_LOGICAL_DIR "/snapshots"
+
+/*
+ * Directories containing multitransaction status data (used for shared
+ * row locks).
+ */
+#define PG_MULTIXACT_DIR "pg_multixact"
+#define PG_MULTIXACT_MEMBERS_DIR PG_MULTIXACT_DIR "/members"
+#define PG_MULTIXACT_OFFSETS_DIR PG_MULTIXACT_DIR "/offsets"
+
+/*
+ * Directory containing LISTEN/NOTIFY status data.
+ */
+#define PG_NOTIFY_DIR "pg_notify"
+
+/*
+ * Directory containing replication slot data.
+ */
+#define PG_REPLSLOT_DIR "pg_replslot"
+
+/*
+ * Default directories to store permanent and temporary statistics
+ * data in. Used by the statistics collector.
+ */
+#define PG_STAT_DIR "pg_stat"
+#define PG_STAT_TMP_DIR "pg_stat_tmp"
+
+/*
+ * Directory containing information about committed serializable
+ * transactions.
+ */
+#define PG_SERIAL_DIR "pg_serial"
+
+/*
+ * Directory containing exported snapshots dara.
+ */
+#define PG_SNAPSHOTS_DIR "pg_snapshots"
+
+/*
+ * Directory containing subtransaction status data.
+ */
+#define PG_SUBTRANS_DIR "pg_subtrans"
+
+/*
+ * Directory containing symbolic links to tablespaces.
+ */
+#define PG_TABLESPACE_DIR "pg_tblspc"
+
+/*
+ * Directory containing state files for prepared transactions.
+ */
+#define PG_TWOPHASE_DIR "pg_twophase"
+
+/*
+ * Directory containing transaction commit status data.
+ */
+#define PG_XACT_DIR "pg_xact"
+
+#endif /* PG_PATHS_H */
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index be2f59239b..7ae4b9097e 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -30,9 +30,6 @@
#define PGSTAT_STAT_PERMANENT_FILENAME "pg_stat/global.stat"
#define PGSTAT_STAT_PERMANENT_TMPFILE "pg_stat/global.tmp"
-/* Default directory to store temporary statistics data in */
-#define PG_STAT_TMP_DIR "pg_stat_tmp"
-
/* Values for track_functions GUC variable --- order is significant! */
typedef enum TrackFunctionsLevel
{
diff --git a/src/include/postmaster/syslogger.h b/src/include/postmaster/syslogger.h
index b35fadc1bd..00cc7a3023 100644
--- a/src/include/postmaster/syslogger.h
+++ b/src/include/postmaster/syslogger.h
@@ -87,11 +87,4 @@ extern void write_syslogger_file(const char *buffer, int count, int dest);
extern void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
#endif
-/*
- * Name of files saving meta-data information about the log
- * files currently in use by the syslogger
- */
-#define LOG_METAINFO_DATAFILE "current_logfiles"
-#define LOG_METAINFO_DATAFILE_TMP LOG_METAINFO_DATAFILE ".tmp"
-
#endif /* _SYSLOGGER_H */
diff --git a/src/include/storage/dsm_impl.h b/src/include/storage/dsm_impl.h
index 0e5730f7c5..f13b46b6c3 100644
--- a/src/include/storage/dsm_impl.h
+++ b/src/include/storage/dsm_impl.h
@@ -42,15 +42,6 @@
/* GUC. */
extern int dynamic_shared_memory_type;
-/*
- * Directory for on-disk state.
- *
- * This is used by all implementations for crash recovery and by the mmap
- * implementation for storage.
- */
-#define PG_DYNSHMEM_DIR "pg_dynshmem"
-#define PG_DYNSHMEM_MMAP_FILE_PREFIX "mmap."
-
/* A "name" for a dynamic shared memory segment. */
typedef uint32 dsm_handle;
diff --git a/src/include/utils/guc.h b/src/include/utils/guc.h
index 77daa5a539..8bb5ba9198 100644
--- a/src/include/utils/guc.h
+++ b/src/include/utils/guc.h
@@ -26,13 +26,6 @@
#define MAX_KILOBYTES (INT_MAX / 1024)
#endif
-/*
- * Automatic configuration file name for ALTER SYSTEM.
- * This file will be used to store values of configuration parameters
- * set by ALTER SYSTEM command.
- */
-#define PG_AUTOCONF_FILENAME "postgresql.auto.conf"
-
/*
* Certain options can only be set at certain times. The rules are
* like this:
--
2.16.1
0002-v2-Replace-all-system-paths-hardcoded-with-data-from-pg.patchtext/x-patch; name=0002-v2-Replace-all-system-paths-hardcoded-with-data-from-pg.patchDownload
commit 6ce93994d1d70d80431115473933fd70ea882a5c
Author: Anastasia <a.lubennikova@postgrespro.ru>
Date: Tue Mar 13 12:20:54 2018 +0300
Subject: [PATCH 2/5] Replace all system paths hardcoded with data from
pg_paths.h
This unifies the interface where system paths are defined and located,
making it easier to use such paths in frontend tools which aim at
manipulating them or a list of them.
An upcoming patch will externalize the list of paths used by base
backups for the exclude lists, which will then be used by pg_rewind to
do the same.
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 9286734..e251a3e 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -71,6 +71,7 @@
#include "parser/parsetree.h"
#include "parser/scanner.h"
#include "parser/scansup.h"
+#include "pg_paths.h"
#include "pgstat.h"
#include "storage/fd.h"
#include "storage/ipc.h"
diff --git a/src/backend/access/heap/rewriteheap.c b/src/backend/access/heap/rewriteheap.c
index 7d466c2..dd3657f 100644
--- a/src/backend/access/heap/rewriteheap.c
+++ b/src/backend/access/heap/rewriteheap.c
@@ -119,6 +119,7 @@
#include "lib/ilist.h"
+#include "pg_paths.h"
#include "pgstat.h"
#include "replication/logical.h"
@@ -1005,7 +1006,7 @@ logical_rewrite_log_mapping(RewriteState state, TransactionId xid,
dboid = MyDatabaseId;
snprintf(path, MAXPGPATH,
- "pg_logical/mappings/" LOGICAL_REWRITE_FORMAT,
+ PG_LOGICAL_MAPPINGS_DIR "/" LOGICAL_REWRITE_FORMAT,
dboid, relid,
(uint32) (state->rs_begin_lsn >> 32),
(uint32) state->rs_begin_lsn,
@@ -1128,7 +1129,7 @@ heap_xlog_logical_rewrite(XLogReaderState *r)
xlrec = (xl_heap_rewrite_mapping *) XLogRecGetData(r);
snprintf(path, MAXPGPATH,
- "pg_logical/mappings/" LOGICAL_REWRITE_FORMAT,
+ PG_LOGICAL_MAPPINGS_DIR "/" LOGICAL_REWRITE_FORMAT,
xlrec->mapped_db, xlrec->mapped_rel,
(uint32) (xlrec->start_lsn >> 32),
(uint32) xlrec->start_lsn,
@@ -1219,8 +1220,8 @@ CheckPointLogicalRewriteHeap(void)
if (cutoff != InvalidXLogRecPtr && redo < cutoff)
cutoff = redo;
- mappings_dir = AllocateDir("pg_logical/mappings");
- while ((mapping_de = ReadDir(mappings_dir, "pg_logical/mappings")) != NULL)
+ mappings_dir = AllocateDir(PG_LOGICAL_MAPPINGS_DIR);
+ while ((mapping_de = ReadDir(mappings_dir, PG_LOGICAL_MAPPINGS_DIR)) != NULL)
{
struct stat statbuf;
Oid dboid;
@@ -1235,7 +1236,8 @@ CheckPointLogicalRewriteHeap(void)
strcmp(mapping_de->d_name, "..") == 0)
continue;
- snprintf(path, sizeof(path), "pg_logical/mappings/%s", mapping_de->d_name);
+ snprintf(path, sizeof(path), "%s/%s", PG_LOGICAL_MAPPINGS_DIR,
+ mapping_de->d_name);
if (lstat(path, &statbuf) == 0 && !S_ISREG(statbuf.st_mode))
continue;
diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c
index 8b7ff5b..bf4db22 100644
--- a/src/backend/access/transam/clog.c
+++ b/src/backend/access/transam/clog.c
@@ -40,6 +40,7 @@
#include "access/xlogutils.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "pg_paths.h"
#include "pg_trace.h"
#include "storage/proc.h"
@@ -699,7 +700,7 @@ CLOGShmemInit(void)
{
ClogCtl->PagePrecedes = CLOGPagePrecedes;
SimpleLruInit(ClogCtl, "clog", CLOGShmemBuffers(), CLOG_LSNS_PER_PAGE,
- CLogControlLock, "pg_xact", LWTRANCHE_CLOG_BUFFERS);
+ CLogControlLock, PG_XACT_DIR, LWTRANCHE_CLOG_BUFFERS);
}
/*
@@ -830,7 +831,7 @@ ShutdownCLOG(void)
* fsync pg_xact to ensure that any files flushed previously are durably
* on disk.
*/
- fsync_fname("pg_xact", true);
+ fsync_fname(PG_XACT_DIR, true);
TRACE_POSTGRESQL_CLOG_CHECKPOINT_DONE(false);
}
@@ -849,7 +850,7 @@ CheckPointCLOG(void)
* fsync pg_xact to ensure that any files flushed previously are durably
* on disk.
*/
- fsync_fname("pg_xact", true);
+ fsync_fname(PG_XACT_DIR, true);
TRACE_POSTGRESQL_CLOG_CHECKPOINT_DONE(true);
}
diff --git a/src/backend/access/transam/commit_ts.c b/src/backend/access/transam/commit_ts.c
index 04a15e4..5d0e4d0 100644
--- a/src/backend/access/transam/commit_ts.c
+++ b/src/backend/access/transam/commit_ts.c
@@ -31,6 +31,7 @@
#include "catalog/pg_type.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "pg_paths.h"
#include "pg_trace.h"
#include "storage/shmem.h"
#include "utils/builtins.h"
@@ -493,7 +494,7 @@ CommitTsShmemInit(void)
CommitTsCtl->PagePrecedes = CommitTsPagePrecedes;
SimpleLruInit(CommitTsCtl, "commit_timestamp", CommitTsShmemBuffers(), 0,
- CommitTsControlLock, "pg_commit_ts",
+ CommitTsControlLock, PG_COMMIT_TS_DIR,
LWTRANCHE_COMMITTS_BUFFERS);
commitTsShared = ShmemInitStruct("CommitTs shared",
@@ -751,7 +752,7 @@ ShutdownCommitTs(void)
* fsync pg_commit_ts to ensure that any files flushed previously are
* durably on disk.
*/
- fsync_fname("pg_commit_ts", true);
+ fsync_fname(PG_COMMIT_TS_DIR, true);
}
/*
@@ -767,7 +768,7 @@ CheckPointCommitTs(void)
* fsync pg_commit_ts to ensure that any files flushed previously are
* durably on disk.
*/
- fsync_fname("pg_commit_ts", true);
+ fsync_fname(PG_COMMIT_TS_DIR, true);
}
/*
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index 6d6f2e3..a158b49 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -81,6 +81,7 @@
#include "funcapi.h"
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pg_paths.h"
#include "pg_trace.h"
#include "postmaster/autovacuum.h"
#include "storage/lmgr.h"
@@ -1828,11 +1829,11 @@ MultiXactShmemInit(void)
SimpleLruInit(MultiXactOffsetCtl,
"multixact_offset", NUM_MXACTOFFSET_BUFFERS, 0,
- MultiXactOffsetControlLock, "pg_multixact/offsets",
+ MultiXactOffsetControlLock, PG_MULTIXACT_OFFSETS_DIR,
LWTRANCHE_MXACTOFFSET_BUFFERS);
SimpleLruInit(MultiXactMemberCtl,
"multixact_member", NUM_MXACTMEMBER_BUFFERS, 0,
- MultiXactMemberControlLock, "pg_multixact/members",
+ MultiXactMemberControlLock, PG_MULTIXACT_MEMBERS_DIR,
LWTRANCHE_MXACTMEMBER_BUFFERS);
/* Initialize our shared state struct */
diff --git a/src/backend/access/transam/subtrans.c b/src/backend/access/transam/subtrans.c
index 4faa21f..977d052 100644
--- a/src/backend/access/transam/subtrans.c
+++ b/src/backend/access/transam/subtrans.c
@@ -31,6 +31,7 @@
#include "access/slru.h"
#include "access/subtrans.h"
#include "access/transam.h"
+#include "pg_paths.h"
#include "pg_trace.h"
#include "utils/snapmgr.h"
@@ -192,7 +193,7 @@ SUBTRANSShmemInit(void)
{
SubTransCtl->PagePrecedes = SubTransPagePrecedes;
SimpleLruInit(SubTransCtl, "subtrans", NUM_SUBTRANS_BUFFERS, 0,
- SubtransControlLock, "pg_subtrans",
+ SubtransControlLock, PG_SUBTRANS_DIR,
LWTRANCHE_SUBTRANS_BUFFERS);
/* Override default assumption that writes should be fsync'd */
SubTransCtl->do_fsync = false;
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index c479c48..89e8164 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -91,6 +91,7 @@
#include "catalog/storage.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "pg_paths.h"
#include "pg_trace.h"
#include "pgstat.h"
#include "replication/origin.h"
@@ -108,11 +109,6 @@
#include "utils/timestamp.h"
-/*
- * Directory where Two-phase commit files reside within PGDATA
- */
-#define TWOPHASE_DIR "pg_twophase"
-
/* GUC variable, can't be changed after startup */
int max_prepared_xacts = 0;
@@ -877,7 +873,7 @@ TwoPhaseGetDummyProc(TransactionId xid)
/************************************************************************/
#define TwoPhaseFilePath(path, xid) \
- snprintf(path, MAXPGPATH, TWOPHASE_DIR "/%08X", xid)
+ snprintf(path, MAXPGPATH, PG_TWOPHASE_DIR "/%08X", xid)
/*
* 2PC state file format:
@@ -1707,7 +1703,7 @@ CheckPointTwoPhase(XLogRecPtr redo_horizon)
* removals need to be made persistent as well as any files newly created
* previously since the last checkpoint.
*/
- fsync_fname(TWOPHASE_DIR, true);
+ fsync_fname(PG_TWOPHASE_DIR, true);
TRACE_POSTGRESQL_TWOPHASE_CHECKPOINT_DONE();
@@ -1736,8 +1732,8 @@ restoreTwoPhaseData(void)
struct dirent *clde;
LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
- cldir = AllocateDir(TWOPHASE_DIR);
- while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
+ cldir = AllocateDir(PG_TWOPHASE_DIR);
+ while ((clde = ReadDir(cldir, PG_TWOPHASE_DIR)) != NULL)
{
if (strlen(clde->d_name) == 8 &&
strspn(clde->d_name, "0123456789ABCDEF") == 8)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 47a6c4d..0598212 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6481,7 +6481,7 @@ StartupXLOG(void)
tablespaceinfo *ti = lfirst(lc);
char *linkloc;
- linkloc = psprintf("pg_tblspc/%s", ti->oid);
+ linkloc = psprintf(PG_TABLESPACE_DIR "/%s", ti->oid);
/*
* Remove the existing symlink if any and Create the symlink
@@ -10426,8 +10426,8 @@ do_pg_start_backup(const char *backupidstr, bool fast, TimeLineID *starttli_p,
datadirpathlen = strlen(DataDir);
/* Collect information about all tablespaces */
- tblspcdir = AllocateDir("pg_tblspc");
- while ((de = ReadDir(tblspcdir, "pg_tblspc")) != NULL)
+ tblspcdir = AllocateDir(PG_TABLESPACE_DIR);
+ while ((de = ReadDir(tblspcdir, PG_TABLESPACE_DIR)) != NULL)
{
char fullpath[MAXPGPATH + 10];
char linkpath[MAXPGPATH];
@@ -10440,7 +10440,8 @@ do_pg_start_backup(const char *backupidstr, bool fast, TimeLineID *starttli_p,
if (strcmp(de->d_name, ".") == 0 || strcmp(de->d_name, "..") == 0)
continue;
- snprintf(fullpath, sizeof(fullpath), "pg_tblspc/%s", de->d_name);
+ snprintf(fullpath, sizeof(fullpath), PG_TABLESPACE_DIR "/%s",
+ de->d_name);
#if defined(HAVE_READLINK) || defined(WIN32)
rllen = readlink(fullpath, linkpath, sizeof(linkpath));
diff --git a/src/backend/commands/async.c b/src/backend/commands/async.c
index ee7c6d4..52bbb28 100644
--- a/src/backend/commands/async.c
+++ b/src/backend/commands/async.c
@@ -127,6 +127,7 @@
#include "libpq/libpq.h"
#include "libpq/pqformat.h"
#include "miscadmin.h"
+#include "pg_paths.h"
#include "storage/ipc.h"
#include "storage/lmgr.h"
#include "storage/proc.h"
@@ -480,7 +481,7 @@ AsyncShmemInit(void)
*/
AsyncCtl->PagePrecedes = asyncQueuePagePrecedes;
SimpleLruInit(AsyncCtl, "async", NUM_ASYNC_BUFFERS, 0,
- AsyncCtlLock, "pg_notify", LWTRANCHE_ASYNC_BUFFERS);
+ AsyncCtlLock, PG_NOTIFY_DIR, LWTRANCHE_ASYNC_BUFFERS);
/* Override default assumption that writes should be fsync'd */
AsyncCtl->do_fsync = false;
diff --git a/src/backend/commands/tablespace.c b/src/backend/commands/tablespace.c
index 5c450ca..2894a62 100644
--- a/src/backend/commands/tablespace.c
+++ b/src/backend/commands/tablespace.c
@@ -69,6 +69,7 @@
#include "commands/tablecmds.h"
#include "commands/tablespace.h"
#include "miscadmin.h"
+#include "pg_paths.h"
#include "postmaster/bgwriter.h"
#include "storage/fd.h"
#include "storage/lmgr.h"
@@ -566,7 +567,7 @@ create_tablespace_directories(const char *location, const Oid tablespaceoid)
char *location_with_version_dir;
struct stat st;
- linkloc = psprintf("pg_tblspc/%u", tablespaceoid);
+ linkloc = psprintf(PG_TABLESPACE_DIR "/%u", tablespaceoid);
location_with_version_dir = psprintf("%s/%s", location,
TABLESPACE_VERSION_DIRECTORY);
@@ -667,7 +668,7 @@ destroy_tablespace_directories(Oid tablespaceoid, bool redo)
char *subfile;
struct stat st;
- linkloc_with_version_dir = psprintf("pg_tblspc/%u/%s", tablespaceoid,
+ linkloc_with_version_dir = psprintf(PG_TABLESPACE_DIR "/%u/%s", tablespaceoid,
TABLESPACE_VERSION_DIRECTORY);
/*
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 2093d87..e2d9d82 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -1546,7 +1546,7 @@ checkDataDir(void)
/* Look for PG_VERSION before looking for pg_control */
ValidatePgVersion(DataDir);
- snprintf(path, sizeof(path), "%s/global/pg_control", DataDir);
+ snprintf(path, sizeof(path), "%s/%s", DataDir, XLOG_CONTROL_FILE);
fp = AllocateFile(path, PG_BINARY_R);
if (fp == NULL)
@@ -5538,11 +5538,9 @@ CreateOptsFile(int argc, char *argv[], char *fullprogname)
FILE *fp;
int i;
-#define OPTS_FILE "postmaster.opts"
-
- if ((fp = fopen(OPTS_FILE, "w")) == NULL)
+ if ((fp = fopen(POSTMASTER_OPTS_FILE, "w")) == NULL)
{
- elog(LOG, "could not create file \"%s\": %m", OPTS_FILE);
+ elog(LOG, "could not create file \"%s\": %m", POSTMASTER_OPTS_FILE);
return false;
}
@@ -5553,7 +5551,7 @@ CreateOptsFile(int argc, char *argv[], char *fullprogname)
if (fclose(fp))
{
- elog(LOG, "could not write file \"%s\": %m", OPTS_FILE);
+ elog(LOG, "could not write file \"%s\": %m", POSTMASTER_OPTS_FILE);
return false;
}
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index a23c999..a395175 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -366,8 +366,8 @@ perform_base_backup(basebackup_options *opt)
XLByteToPrevSeg(endptr, endsegno, wal_segment_size);
XLogFileName(lastoff, ThisTimeLineID, endsegno, wal_segment_size);
- dir = AllocateDir("pg_wal");
- while ((de = ReadDir(dir, "pg_wal")) != NULL)
+ dir = AllocateDir(XLOGDIR);
+ while ((de = ReadDir(dir, XLOGDIR)) != NULL)
{
/* Does it look like a WAL segment, and is it in the range? */
if (IsXLogFileName(de->d_name) &&
@@ -1010,7 +1010,7 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
snprintf(pathbuf, sizeof(pathbuf), "%s/%s", path, de->d_name);
/* Skip pg_control here to back up it last */
- if (strcmp(pathbuf, "./global/pg_control") == 0)
+ if (strcmp(pathbuf, "./" XLOG_CONTROL_FILE) == 0)
continue;
if (lstat(pathbuf, &statbuf) != 0)
@@ -1057,7 +1057,7 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
* WAL archive anyway. But include it as an empty directory anyway, so
* we get permissions right.
*/
- if (strcmp(pathbuf, "./pg_wal") == 0)
+ if (strcmp(pathbuf, "./" XLOGDIR) == 0)
{
/* If pg_wal is a symlink, write it as a directory anyway */
size += _tarWriteDir(pathbuf, basepathlen, &statbuf, sizeonly);
@@ -1066,14 +1066,14 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
* Also send archive_status directory (by hackishly reusing
* statbuf from above ...).
*/
- size += _tarWriteHeader("./pg_wal/archive_status", NULL, &statbuf,
+ size += _tarWriteHeader("./" XLOG_ARCHIVE_STATUS_DIR, NULL, &statbuf,
sizeonly);
continue; /* don't recurse into pg_wal */
}
/* Allow symbolic links in pg_tblspc only */
- if (strcmp(path, "./pg_tblspc") == 0 &&
+ if (strcmp(path, "./" PG_TABLESPACE_DIR) == 0 &&
#ifndef WIN32
S_ISLNK(statbuf.st_mode)
#else
@@ -1150,7 +1150,7 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
/*
* skip sending directories inside pg_tblspc, if not required.
*/
- if (strcmp(pathbuf, "./pg_tblspc") == 0 && !sendtblspclinks)
+ if (strcmp(pathbuf, "./" PG_TABLESPACE_DIR) == 0 && !sendtblspclinks)
skip_this_dir = true;
if (!skip_this_dir)
diff --git a/src/backend/replication/logical/origin.c b/src/backend/replication/logical/origin.c
index 963878a..e2a2a3b 100644
--- a/src/backend/replication/logical/origin.c
+++ b/src/backend/replication/logical/origin.c
@@ -83,6 +83,7 @@
#include "replication/origin.h"
#include "replication/logical.h"
+#include "pg_paths.h"
#include "pgstat.h"
#include "storage/fd.h"
#include "storage/ipc.h"
@@ -544,8 +545,8 @@ ReplicationOriginShmemInit(void)
void
CheckPointReplicationOrigin(void)
{
- const char *tmppath = "pg_logical/replorigin_checkpoint.tmp";
- const char *path = "pg_logical/replorigin_checkpoint";
+ const char *tmppath = PG_LOGICAL_DIR "/replorigin_checkpoint.tmp";
+ const char *path = PG_LOGICAL_DIR "/replorigin_checkpoint";
int tmpfd;
int i;
uint32 magic = REPLICATION_STATE_MAGIC;
@@ -657,7 +658,7 @@ CheckPointReplicationOrigin(void)
void
StartupReplicationOrigin(void)
{
- const char *path = "pg_logical/replorigin_checkpoint";
+ const char *path = PG_LOGICAL_DIR "/replorigin_checkpoint";
int fd;
int readBytes;
uint32 magic = REPLICATION_STATE_MAGIC;
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 7612cf5..8713a44 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -2581,7 +2581,7 @@ ReorderBufferCleanupSerializedTXNs(const char *slotname)
struct stat statbuf;
char path[MAXPGPATH * 2 + 12];
- sprintf(path, "pg_replslot/%s", slotname);
+ sprintf(path, "%s/%s", PG_REPLSLOT_DIR, slotname);
/* we're only handling directories here, skip if it's not ours */
if (lstat(path, &statbuf) == 0 && !S_ISDIR(statbuf.st_mode))
@@ -2594,14 +2594,14 @@ ReorderBufferCleanupSerializedTXNs(const char *slotname)
if (strncmp(spill_de->d_name, "xid", 3) == 0)
{
snprintf(path, sizeof(path),
- "pg_replslot/%s/%s", slotname,
+ "%s/%s/%s",PG_REPLSLOT_DIR, slotname,
spill_de->d_name);
if (unlink(path) != 0)
ereport(ERROR,
(errcode_for_file_access(),
- errmsg("could not remove file \"%s\" during removal of pg_replslot/%s/*.xid: %m",
- path, slotname)));
+ errmsg("could not remove file \"%s\" during removal of %s/%s/*.xid: %m",
+ PG_REPLSLOT_DIR, path, slotname)));
}
}
FreeDir(spill_dir);
@@ -2620,8 +2620,8 @@ ReorderBufferSerializedPath(char *path, ReplicationSlot *slot, TransactionId xid
XLogSegNoOffsetToRecPtr(segno, 0, recptr, wal_segment_size);
- snprintf(path, MAXPGPATH, "pg_replslot/%s/xid-%u-lsn-%X-%X.snap",
- NameStr(MyReplicationSlot->data.name),
+ snprintf(path, MAXPGPATH, "%s/%s/xid-%u-lsn-%X-%X.snap",
+ PG_REPLSLOT_DIR, NameStr(MyReplicationSlot->data.name),
xid,
(uint32) (recptr >> 32), (uint32) recptr);
}
@@ -2636,8 +2636,8 @@ StartupReorderBuffer(void)
DIR *logical_dir;
struct dirent *logical_de;
- logical_dir = AllocateDir("pg_replslot");
- while ((logical_de = ReadDir(logical_dir, "pg_replslot")) != NULL)
+ logical_dir = AllocateDir(PG_REPLSLOT_DIR);
+ while ((logical_de = ReadDir(logical_dir, PG_REPLSLOT_DIR)) != NULL)
{
if (strcmp(logical_de->d_name, ".") == 0 ||
strcmp(logical_de->d_name, "..") == 0)
@@ -3034,7 +3034,7 @@ ApplyLogicalMappingFile(HTAB *tuplecid_data, Oid relid, const char *fname)
int readBytes;
LogicalRewriteMappingData map;
- sprintf(path, "pg_logical/mappings/%s", fname);
+ sprintf(path, "%s/%s", PG_LOGICAL_MAPPINGS_DIR, fname);
fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
if (fd < 0)
ereport(ERROR,
@@ -3157,8 +3157,8 @@ UpdateLogicalMappings(HTAB *tuplecid_data, Oid relid, Snapshot snapshot)
size_t off;
Oid dboid = IsSharedRelation(relid) ? InvalidOid : MyDatabaseId;
- mapping_dir = AllocateDir("pg_logical/mappings");
- while ((mapping_de = ReadDir(mapping_dir, "pg_logical/mappings")) != NULL)
+ mapping_dir = AllocateDir(PG_LOGICAL_MAPPINGS_DIR);
+ while ((mapping_de = ReadDir(mapping_dir, PG_LOGICAL_MAPPINGS_DIR)) != NULL)
{
Oid f_dboid;
Oid f_relid;
diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c
index 4123cde..bc91330 100644
--- a/src/backend/replication/logical/snapbuild.c
+++ b/src/backend/replication/logical/snapbuild.c
@@ -126,6 +126,7 @@
#include "access/transam.h"
#include "access/xact.h"
+#include "pg_paths.h"
#include "pgstat.h"
#include "replication/logical.h"
@@ -1503,7 +1504,8 @@ SnapBuildSerialize(SnapBuild *builder, XLogRecPtr lsn)
* unless the user used pg_resetwal or similar. If a user did so, there's
* no hope continuing to decode anyway.
*/
- sprintf(path, "pg_logical/snapshots/%X-%X.snap",
+ sprintf(path, "%s/%X-%X.snap",
+ PG_LOGICAL_SNAPSHOTS_DIR,
(uint32) (lsn >> 32), (uint32) lsn);
/*
@@ -1529,7 +1531,7 @@ SnapBuildSerialize(SnapBuild *builder, XLogRecPtr lsn)
* be safely on disk.
*/
fsync_fname(path, false);
- fsync_fname("pg_logical/snapshots", true);
+ fsync_fname(PG_LOGICAL_SNAPSHOTS_DIR, true);
builder->last_serialized_snapshot = lsn;
goto out;
@@ -1545,7 +1547,8 @@ SnapBuildSerialize(SnapBuild *builder, XLogRecPtr lsn)
elog(DEBUG1, "serializing snapshot to %s", path);
/* to make sure only we will write to this tempfile, include pid */
- sprintf(tmppath, "pg_logical/snapshots/%X-%X.snap.%u.tmp",
+ sprintf(tmppath, "%s/%X-%X.snap.%u.tmp",
+ PG_LOGICAL_SNAPSHOTS_DIR,
(uint32) (lsn >> 32), (uint32) lsn, MyProcPid);
/*
@@ -1631,7 +1634,7 @@ SnapBuildSerialize(SnapBuild *builder, XLogRecPtr lsn)
pgstat_report_wait_end();
CloseTransientFile(fd);
- fsync_fname("pg_logical/snapshots", true);
+ fsync_fname(PG_LOGICAL_SNAPSHOTS_DIR, true);
/*
* We may overwrite the work from some other backend, but that's ok, our
@@ -1647,7 +1650,7 @@ SnapBuildSerialize(SnapBuild *builder, XLogRecPtr lsn)
/* make sure we persist */
fsync_fname(path, false);
- fsync_fname("pg_logical/snapshots", true);
+ fsync_fname(PG_LOGICAL_SNAPSHOTS_DIR, true);
/*
* Now there's no way we can loose the dumped state anymore, remember this
@@ -1678,7 +1681,8 @@ SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
if (builder->state == SNAPBUILD_CONSISTENT)
return false;
- sprintf(path, "pg_logical/snapshots/%X-%X.snap",
+ sprintf(path, "%s/%X-%X.snap",
+ PG_LOGICAL_SNAPSHOTS_DIR,
(uint32) (lsn >> 32), (uint32) lsn);
fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
@@ -1699,7 +1703,7 @@ SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
* ----
*/
fsync_fname(path, false);
- fsync_fname("pg_logical/snapshots", true);
+ fsync_fname(PG_LOGICAL_SNAPSHOTS_DIR, true);
/* read statically sized portion of snapshot */
@@ -1880,8 +1884,8 @@ CheckPointSnapBuild(void)
if (redo < cutoff)
cutoff = redo;
- snap_dir = AllocateDir("pg_logical/snapshots");
- while ((snap_de = ReadDir(snap_dir, "pg_logical/snapshots")) != NULL)
+ snap_dir = AllocateDir(PG_LOGICAL_SNAPSHOTS_DIR);
+ while ((snap_de = ReadDir(snap_dir, PG_LOGICAL_SNAPSHOTS_DIR)) != NULL)
{
uint32 hi;
uint32 lo;
@@ -1892,7 +1896,8 @@ CheckPointSnapBuild(void)
strcmp(snap_de->d_name, "..") == 0)
continue;
- snprintf(path, sizeof(path), "pg_logical/snapshots/%s", snap_de->d_name);
+ snprintf(path, sizeof(path), "%s/%s", PG_LOGICAL_SNAPSHOTS_DIR,
+ snap_de->d_name);
if (lstat(path, &statbuf) == 0 && !S_ISREG(statbuf.st_mode))
{
diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c
index fc9ef22..c64641a 100644
--- a/src/backend/replication/slot.c
+++ b/src/backend/replication/slot.c
@@ -549,8 +549,8 @@ ReplicationSlotDropPtr(ReplicationSlot *slot)
LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE);
/* Generate pathnames. */
- sprintf(path, "pg_replslot/%s", NameStr(slot->data.name));
- sprintf(tmppath, "pg_replslot/%s.tmp", NameStr(slot->data.name));
+ sprintf(path, "%s/%s", PG_REPLSLOT_DIR, NameStr(slot->data.name));
+ sprintf(tmppath, "%s/%s.tmp", PG_REPLSLOT_DIR, NameStr(slot->data.name));
/*
* Rename the slot directory on disk, so that we'll no longer recognize
@@ -571,7 +571,7 @@ ReplicationSlotDropPtr(ReplicationSlot *slot)
*/
START_CRIT_SECTION();
fsync_fname(tmppath, true);
- fsync_fname("pg_replslot", true);
+ fsync_fname(PG_REPLSLOT_DIR, true);
END_CRIT_SECTION();
}
else
@@ -641,7 +641,8 @@ ReplicationSlotSave(void)
Assert(MyReplicationSlot != NULL);
- sprintf(path, "pg_replslot/%s", NameStr(MyReplicationSlot->data.name));
+ sprintf(path, "%s/%s", PG_REPLSLOT_DIR,
+ NameStr(MyReplicationSlot->data.name));
SaveSlotToPath(MyReplicationSlot, path, ERROR);
}
@@ -1076,7 +1077,7 @@ CheckPointReplicationSlots(void)
continue;
/* save the slot to disk, locking is handled in SaveSlotToPath() */
- sprintf(path, "pg_replslot/%s", NameStr(s->data.name));
+ sprintf(path, "%s/%s", PG_REPLSLOT_DIR, NameStr(s->data.name));
SaveSlotToPath(s, path, LOG);
}
LWLockRelease(ReplicationSlotAllocationLock);
@@ -1095,8 +1096,8 @@ StartupReplicationSlots(void)
elog(DEBUG1, "starting up replication slots");
/* restore all slots by iterating over all on-disk entries */
- replication_dir = AllocateDir("pg_replslot");
- while ((replication_de = ReadDir(replication_dir, "pg_replslot")) != NULL)
+ replication_dir = AllocateDir(PG_REPLSLOT_DIR);
+ while ((replication_de = ReadDir(replication_dir, PG_REPLSLOT_DIR)) != NULL)
{
struct stat statbuf;
char path[MAXPGPATH + 12];
@@ -1105,7 +1106,8 @@ StartupReplicationSlots(void)
strcmp(replication_de->d_name, "..") == 0)
continue;
- snprintf(path, sizeof(path), "pg_replslot/%s", replication_de->d_name);
+ snprintf(path, sizeof(path), "%s/%s", PG_REPLSLOT_DIR,
+ replication_de->d_name);
/* we're only creating directories here, skip if it's not our's */
if (lstat(path, &statbuf) == 0 && !S_ISDIR(statbuf.st_mode))
@@ -1121,7 +1123,7 @@ StartupReplicationSlots(void)
errmsg("could not remove directory \"%s\"", path)));
continue;
}
- fsync_fname("pg_replslot", true);
+ fsync_fname(PG_REPLSLOT_DIR, true);
continue;
}
@@ -1159,8 +1161,8 @@ CreateSlotOnDisk(ReplicationSlot *slot)
* takes out the lock, if we'd take the lock here, we'd deadlock.
*/
- sprintf(path, "pg_replslot/%s", NameStr(slot->data.name));
- sprintf(tmppath, "pg_replslot/%s.tmp", NameStr(slot->data.name));
+ sprintf(path, "%s/%s", PG_REPLSLOT_DIR, NameStr(slot->data.name));
+ sprintf(tmppath, "%s/%s.tmp", PG_REPLSLOT_DIR, NameStr(slot->data.name));
/*
* It's just barely possible that some previous effort to create or drop a
@@ -1198,7 +1200,7 @@ CreateSlotOnDisk(ReplicationSlot *slot)
START_CRIT_SECTION();
fsync_fname(path, true);
- fsync_fname("pg_replslot", true);
+ fsync_fname(PG_REPLSLOT_DIR, true);
END_CRIT_SECTION();
}
@@ -1309,7 +1311,7 @@ SaveSlotToPath(ReplicationSlot *slot, const char *dir, int elevel)
fsync_fname(path, false);
fsync_fname(dir, true);
- fsync_fname("pg_replslot", true);
+ fsync_fname(PG_REPLSLOT_DIR, true);
END_CRIT_SECTION();
@@ -1342,13 +1344,13 @@ RestoreSlotFromDisk(const char *name)
/* no need to lock here, no concurrent access allowed yet */
/* delete temp file if it exists */
- sprintf(path, "pg_replslot/%s/state.tmp", name);
+ sprintf(path, "%s/%s/state.tmp", PG_REPLSLOT_DIR, name);
if (unlink(path) < 0 && errno != ENOENT)
ereport(PANIC,
(errcode_for_file_access(),
errmsg("could not remove file \"%s\": %m", path)));
- sprintf(path, "pg_replslot/%s/state", name);
+ sprintf(path, "%s/%s/state", PG_REPLSLOT_DIR, name);
elog(DEBUG1, "restoring replication slot from \"%s\"", path);
@@ -1459,7 +1461,7 @@ RestoreSlotFromDisk(const char *name)
*/
if (cp.slotdata.persistency != RS_PERSISTENT)
{
- sprintf(path, "pg_replslot/%s", name);
+ sprintf(path, "%s/%s", PG_REPLSLOT_DIR, name);
if (!rmtree(path, true))
{
@@ -1467,7 +1469,7 @@ RestoreSlotFromDisk(const char *name)
(errcode_for_file_access(),
errmsg("could not remove directory \"%s\"", path)));
}
- fsync_fname("pg_replslot", true);
+ fsync_fname(PG_REPLSLOT_DIR, true);
return;
}
diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c
index 2a18e94..45888e9 100644
--- a/src/backend/storage/file/fd.c
+++ b/src/backend/storage/file/fd.c
@@ -84,6 +84,7 @@
#include "access/xlog.h"
#include "catalog/catalog.h"
#include "catalog/pg_tablespace.h"
+#include "pg_paths.h"
#include "pgstat.h"
#include "portability/mem.h"
#include "storage/fd.h"
@@ -1560,11 +1561,12 @@ TempTablespacePath(char *path, Oid tablespace)
if (tablespace == InvalidOid ||
tablespace == DEFAULTTABLESPACE_OID ||
tablespace == GLOBALTABLESPACE_OID)
- snprintf(path, MAXPGPATH, "base/%s", PG_TEMP_FILES_DIR);
+ snprintf(path, MAXPGPATH, "%s/%s",
+ PG_BASE_DIR, PG_TEMP_FILES_DIR);
else
{
/* All other tablespaces are accessed via symlinks */
- snprintf(path, MAXPGPATH, "pg_tblspc/%u/%s/%s",
+ snprintf(path, MAXPGPATH, PG_TABLESPACE_DIR "/%u/%s/%s",
tablespace, TABLESPACE_VERSION_DIRECTORY,
PG_TEMP_FILES_DIR);
}
@@ -3020,26 +3022,27 @@ RemovePgTempFiles(void)
/*
* First process temp files in pg_default ($PGDATA/base)
*/
- snprintf(temp_path, sizeof(temp_path), "base/%s", PG_TEMP_FILES_DIR);
+ snprintf(temp_path, sizeof(temp_path), "%s/%s",
+ PG_BASE_DIR, PG_TEMP_FILES_DIR);
RemovePgTempFilesInDir(temp_path, true, false);
- RemovePgTempRelationFiles("base");
+ RemovePgTempRelationFiles(PG_BASE_DIR);
/*
* Cycle through temp directories for all non-default tablespaces.
*/
- spc_dir = AllocateDir("pg_tblspc");
+ spc_dir = AllocateDir(PG_TABLESPACE_DIR);
- while ((spc_de = ReadDirExtended(spc_dir, "pg_tblspc", LOG)) != NULL)
+ while ((spc_de = ReadDirExtended(spc_dir, PG_TABLESPACE_DIR, LOG)) != NULL)
{
if (strcmp(spc_de->d_name, ".") == 0 ||
strcmp(spc_de->d_name, "..") == 0)
continue;
- snprintf(temp_path, sizeof(temp_path), "pg_tblspc/%s/%s/%s",
+ snprintf(temp_path, sizeof(temp_path), PG_TABLESPACE_DIR "/%s/%s/%s",
spc_de->d_name, TABLESPACE_VERSION_DIRECTORY, PG_TEMP_FILES_DIR);
RemovePgTempFilesInDir(temp_path, true, false);
- snprintf(temp_path, sizeof(temp_path), "pg_tblspc/%s/%s",
+ snprintf(temp_path, sizeof(temp_path), PG_TABLESPACE_DIR "/%s/%s",
spc_de->d_name, TABLESPACE_VERSION_DIRECTORY);
RemovePgTempRelationFiles(temp_path);
}
@@ -3278,16 +3281,16 @@ SyncDataDirectory(void)
{
struct stat st;
- if (lstat("pg_wal", &st) < 0)
+ if (lstat(XLOGDIR, &st) < 0)
ereport(LOG,
(errcode_for_file_access(),
errmsg("could not stat file \"%s\": %m",
- "pg_wal")));
+ XLOGDIR)));
else if (S_ISLNK(st.st_mode))
xlog_is_symlink = true;
}
#else
- if (pgwin32_is_junction("pg_wal"))
+ if (pgwin32_is_junction(XLOGDIR))
xlog_is_symlink = true;
#endif
@@ -3299,8 +3302,8 @@ SyncDataDirectory(void)
#ifdef PG_FLUSH_DATA_WORKS
walkdir(".", pre_sync_fname, false, DEBUG1);
if (xlog_is_symlink)
- walkdir("pg_wal", pre_sync_fname, false, DEBUG1);
- walkdir("pg_tblspc", pre_sync_fname, true, DEBUG1);
+ walkdir(XLOGDIR, pre_sync_fname, false, DEBUG1);
+ walkdir(PG_TABLESPACE_DIR, pre_sync_fname, true, DEBUG1);
#endif
/*
@@ -3314,8 +3317,8 @@ SyncDataDirectory(void)
*/
walkdir(".", datadir_fsync_fname, false, LOG);
if (xlog_is_symlink)
- walkdir("pg_wal", datadir_fsync_fname, false, LOG);
- walkdir("pg_tblspc", datadir_fsync_fname, true, LOG);
+ walkdir(XLOGDIR, datadir_fsync_fname, false, LOG);
+ walkdir(PG_TABLESPACE_DIR, datadir_fsync_fname, true, LOG);
}
/*
diff --git a/src/backend/storage/file/reinit.c b/src/backend/storage/file/reinit.c
index 92363ae..69e9c7b 100644
--- a/src/backend/storage/file/reinit.c
+++ b/src/backend/storage/file/reinit.c
@@ -18,6 +18,7 @@
#include "catalog/catalog.h"
#include "common/relpath.h"
+#include "pg_paths.h"
#include "storage/copydir.h"
#include "storage/fd.h"
#include "storage/reinit.h"
@@ -71,20 +72,20 @@ ResetUnloggedRelations(int op)
/*
* First process unlogged files in pg_default ($PGDATA/base)
*/
- ResetUnloggedRelationsInTablespaceDir("base", op);
+ ResetUnloggedRelationsInTablespaceDir(PG_BASE_DIR, op);
/*
* Cycle through directories for all non-default tablespaces.
*/
- spc_dir = AllocateDir("pg_tblspc");
+ spc_dir = AllocateDir(PG_TABLESPACE_DIR);
- while ((spc_de = ReadDir(spc_dir, "pg_tblspc")) != NULL)
+ while ((spc_de = ReadDir(spc_dir, PG_TABLESPACE_DIR)) != NULL)
{
if (strcmp(spc_de->d_name, ".") == 0 ||
strcmp(spc_de->d_name, "..") == 0)
continue;
- snprintf(temp_path, sizeof(temp_path), "pg_tblspc/%s/%s",
+ snprintf(temp_path, sizeof(temp_path), PG_TABLESPACE_DIR "/%s/%s",
spc_de->d_name, TABLESPACE_VERSION_DIRECTORY);
ResetUnloggedRelationsInTablespaceDir(temp_path, op);
}
diff --git a/src/backend/utils/adt/dbsize.c b/src/backend/utils/adt/dbsize.c
index 834a104..ee3cece 100644
--- a/src/backend/utils/adt/dbsize.c
+++ b/src/backend/utils/adt/dbsize.c
@@ -22,6 +22,7 @@
#include "commands/dbcommands.h"
#include "commands/tablespace.h"
#include "miscadmin.h"
+#include "pg_paths.h"
#include "storage/fd.h"
#include "utils/acl.h"
#include "utils/builtins.h"
@@ -104,11 +105,11 @@ calculate_database_size(Oid dbOid)
/* Shared storage in pg_global is not counted */
/* Include pg_default storage */
- snprintf(pathname, sizeof(pathname), "base/%u", dbOid);
+ snprintf(pathname, sizeof(pathname), PG_BASE_DIR "/%u", dbOid);
totalsize = db_dir_size(pathname);
/* Scan the non-default tablespaces */
- snprintf(dirpath, MAXPGPATH, "pg_tblspc");
+ snprintf(dirpath, MAXPGPATH, PG_TABLESPACE_DIR);
dirdesc = AllocateDir(dirpath);
while ((direntry = ReadDir(dirdesc, dirpath)) != NULL)
@@ -119,7 +120,7 @@ calculate_database_size(Oid dbOid)
strcmp(direntry->d_name, "..") == 0)
continue;
- snprintf(pathname, sizeof(pathname), "pg_tblspc/%s/%s/%u",
+ snprintf(pathname, sizeof(pathname), PG_TABLESPACE_DIR "/%s/%s/%u",
direntry->d_name, TABLESPACE_VERSION_DIRECTORY, dbOid);
totalsize += db_dir_size(pathname);
}
@@ -188,11 +189,11 @@ calculate_tablespace_size(Oid tblspcOid)
}
if (tblspcOid == DEFAULTTABLESPACE_OID)
- snprintf(tblspcPath, MAXPGPATH, "base");
+ snprintf(tblspcPath, MAXPGPATH, PG_BASE_DIR);
else if (tblspcOid == GLOBALTABLESPACE_OID)
- snprintf(tblspcPath, MAXPGPATH, "global");
+ snprintf(tblspcPath, MAXPGPATH, PG_GLOBAL_DIR);
else
- snprintf(tblspcPath, MAXPGPATH, "pg_tblspc/%u/%s", tblspcOid,
+ snprintf(tblspcPath, MAXPGPATH, PG_TABLESPACE_DIR "/%u/%s", tblspcOid,
TABLESPACE_VERSION_DIRECTORY);
dirdesc = AllocateDir(tblspcPath);
diff --git a/src/backend/utils/adt/misc.c b/src/backend/utils/adt/misc.c
index 9f98c03..41a2407 100644
--- a/src/backend/utils/adt/misc.c
+++ b/src/backend/utils/adt/misc.c
@@ -396,9 +396,9 @@ pg_tablespace_databases(PG_FUNCTION_ARGS)
else
{
if (tablespaceOid == DEFAULTTABLESPACE_OID)
- fctx->location = psprintf("base");
+ fctx->location = psprintf(PG_BASE_DIR);
else
- fctx->location = psprintf("pg_tblspc/%u/%s", tablespaceOid,
+ fctx->location = psprintf(PG_TABLESPACE_DIR "/%u/%s", tablespaceOid,
TABLESPACE_VERSION_DIRECTORY);
fctx->dirdesc = AllocateDir(fctx->location);
@@ -484,7 +484,8 @@ pg_tablespace_location(PG_FUNCTION_ARGS)
* Find the location of the tablespace by reading the symbolic link that
* is in pg_tblspc/<oid>.
*/
- snprintf(sourcepath, sizeof(sourcepath), "pg_tblspc/%u", tablespaceOid);
+ snprintf(sourcepath, sizeof(sourcepath), PG_TABLESPACE_DIR "/%u",
+ tablespaceOid);
rllen = readlink(sourcepath, targetpath, sizeof(targetpath));
if (rllen < 0)
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 6ab4db2..f3d932f 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -71,6 +71,7 @@
#include "optimizer/clauses.h"
#include "optimizer/prep.h"
#include "optimizer/var.h"
+#include "pg_paths.h"
#include "rewrite/rewriteDefine.h"
#include "rewrite/rowsecurity.h"
#include "storage/lmgr.h"
@@ -5352,7 +5353,8 @@ load_relcache_init_file(bool shared)
int i;
if (shared)
- snprintf(initfilename, sizeof(initfilename), "global/%s",
+ snprintf(initfilename, sizeof(initfilename), "%s/%s",
+ PG_GLOBAL_DIR,
RELCACHE_INIT_FILENAME);
else
snprintf(initfilename, sizeof(initfilename), "%s/%s",
@@ -5751,9 +5753,11 @@ write_relcache_init_file(bool shared)
*/
if (shared)
{
- snprintf(tempfilename, sizeof(tempfilename), "global/%s.%d",
+ snprintf(tempfilename, sizeof(tempfilename), "%s/%s.%d",
+ PG_GLOBAL_DIR,
RELCACHE_INIT_FILENAME, MyProcPid);
- snprintf(finalfilename, sizeof(finalfilename), "global/%s",
+ snprintf(finalfilename, sizeof(finalfilename), "%s/%s",
+ PG_GLOBAL_DIR,
RELCACHE_INIT_FILENAME);
}
else
@@ -6074,7 +6078,6 @@ RelationCacheInitFilePostInvalidate(void)
void
RelationCacheInitFileRemove(void)
{
- const char *tblspcdir = "pg_tblspc";
DIR *dir;
struct dirent *de;
char path[MAXPGPATH + 10 + sizeof(TABLESPACE_VERSION_DIRECTORY)];
@@ -6083,23 +6086,25 @@ RelationCacheInitFileRemove(void)
* We zap the shared cache file too. In theory it can't get out of sync
* enough to be a problem, but in data-corruption cases, who knows ...
*/
- snprintf(path, sizeof(path), "global/%s",
+ snprintf(path, sizeof(path), "%s/%s",
+ PG_GLOBAL_DIR,
RELCACHE_INIT_FILENAME);
unlink_initfile(path);
/* Scan everything in the default tablespace */
- RelationCacheInitFileRemoveInDir("base");
+ RelationCacheInitFileRemoveInDir(PG_BASE_DIR);
/* Scan the tablespace link directory to find non-default tablespaces */
- dir = AllocateDir(tblspcdir);
+ dir = AllocateDir(PG_TABLESPACE_DIR);
- while ((de = ReadDirExtended(dir, tblspcdir, LOG)) != NULL)
+ while ((de = ReadDirExtended(dir, PG_TABLESPACE_DIR, LOG)) != NULL)
{
if (strspn(de->d_name, "0123456789") == strlen(de->d_name))
{
/* Scan the tablespace dir for per-database dirs */
snprintf(path, sizeof(path), "%s/%s/%s",
- tblspcdir, de->d_name, TABLESPACE_VERSION_DIRECTORY);
+ PG_TABLESPACE_DIR, de->d_name,
+ TABLESPACE_VERSION_DIRECTORY);
RelationCacheInitFileRemoveInDir(path);
}
}
diff --git a/src/backend/utils/cache/relmapper.c b/src/backend/utils/cache/relmapper.c
index 99d095f..479082f 100644
--- a/src/backend/utils/cache/relmapper.c
+++ b/src/backend/utils/cache/relmapper.c
@@ -50,6 +50,7 @@
#include "catalog/pg_tablespace.h"
#include "catalog/storage.h"
#include "miscadmin.h"
+#include "pg_paths.h"
#include "pgstat.h"
#include "storage/fd.h"
#include "storage/lwlock.h"
@@ -632,7 +633,8 @@ load_relmap_file(bool shared)
if (shared)
{
- snprintf(mapfilename, sizeof(mapfilename), "global/%s",
+ snprintf(mapfilename, sizeof(mapfilename), "%s/%s",
+ PG_GLOBAL_DIR,
RELMAPPER_FILENAME);
map = &shared_map;
}
@@ -733,7 +735,8 @@ write_relmap_file(bool shared, RelMapFile *newmap,
*/
if (shared)
{
- snprintf(mapfilename, sizeof(mapfilename), "global/%s",
+ snprintf(mapfilename, sizeof(mapfilename), "%s/%s",
+ PG_GLOBAL_DIR,
RELMAPPER_FILENAME);
realmap = &shared_map;
}
diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c
index 87ed7d3..4d1dd06 100644
--- a/src/backend/utils/init/miscinit.c
+++ b/src/backend/utils/init/miscinit.c
@@ -35,6 +35,7 @@
#include "libpq/libpq.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
+#include "pg_paths.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
#include "postmaster/postmaster.h"
@@ -52,7 +53,7 @@
#include "utils/varlena.h"
-#define DIRECTORY_LOCK_FILE "postmaster.pid"
+#define DIRECTORY_LOCK_FILE POSTMASTER_PID_FILE
ProcessingMode Mode = InitProcessing;
diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c
index e58c69d..6c3571f 100644
--- a/src/backend/utils/time/snapmgr.c
+++ b/src/backend/utils/time/snapmgr.c
@@ -54,6 +54,7 @@
#include "catalog/catalog.h"
#include "lib/pairingheap.h"
#include "miscadmin.h"
+#include "pg_paths.h"
#include "storage/predicate.h"
#include "storage/proc.h"
#include "storage/procarray.h"
@@ -209,9 +210,6 @@ bool FirstSnapshotSet = false;
*/
static Snapshot FirstXactSnapshot = NULL;
-/* Define pathname of exported-snapshot files */
-#define SNAPSHOT_EXPORT_DIR "pg_snapshots"
-
/* Structure holding info about exported snapshot. */
typedef struct ExportedSnapshot
{
@@ -1210,7 +1208,7 @@ ExportSnapshot(Snapshot snapshot)
* Generate file path for the snapshot. We start numbering of snapshots
* inside the transaction from 1.
*/
- snprintf(path, sizeof(path), SNAPSHOT_EXPORT_DIR "/%08X-%08X-%d",
+ snprintf(path, sizeof(path), PG_SNAPSHOTS_DIR "/%08X-%08X-%d",
MyProc->backendId, MyProc->lxid, list_length(exportedSnapshots) + 1);
/*
@@ -1321,10 +1319,10 @@ ExportSnapshot(Snapshot snapshot)
/*
* The basename of the file is what we return from pg_export_snapshot().
* It's already in path in a textual format and we know that the path
- * starts with SNAPSHOT_EXPORT_DIR. Skip over the prefix and the slash
+ * starts with PG_SNAPSHOTS_DIR. Skip over the prefix and the slash
* and pstrdup it so as not to return the address of a local variable.
*/
- return pstrdup(path + strlen(SNAPSHOT_EXPORT_DIR) + 1);
+ return pstrdup(path + strlen(PG_SNAPSHOTS_DIR) + 1);
}
/*
@@ -1423,7 +1421,7 @@ parseVxidFromText(const char *prefix, char **s, const char *filename,
/*
* ImportSnapshot
* Import a previously exported snapshot. The argument should be a
- * filename in SNAPSHOT_EXPORT_DIR. Load the snapshot from that file.
+ * filename in PG_SNAPSHOTS_DIR. Load the snapshot from that file.
* This is called by "SET TRANSACTION SNAPSHOT 'foo'".
*/
void
@@ -1474,7 +1472,7 @@ ImportSnapshot(const char *idstr)
errmsg("invalid snapshot identifier: \"%s\"", idstr)));
/* OK, read the file */
- snprintf(path, MAXPGPATH, SNAPSHOT_EXPORT_DIR "/%s", idstr);
+ snprintf(path, MAXPGPATH, PG_SNAPSHOTS_DIR "/%s", idstr);
f = AllocateFile(path, PG_BINARY_R);
if (!f)
@@ -1615,7 +1613,7 @@ XactHasExportedSnapshots(void)
void
DeleteAllExportedSnapshotFiles(void)
{
- char buf[MAXPGPATH + sizeof(SNAPSHOT_EXPORT_DIR)];
+ char buf[MAXPGPATH + sizeof(PG_SNAPSHOTS_DIR)];
DIR *s_dir;
struct dirent *s_de;
@@ -1624,15 +1622,15 @@ DeleteAllExportedSnapshotFiles(void)
* LOG level. Since we're running in the startup process, ERROR level
* would prevent database start, and it's not important enough for that.
*/
- s_dir = AllocateDir(SNAPSHOT_EXPORT_DIR);
+ s_dir = AllocateDir(PG_SNAPSHOTS_DIR);
- while ((s_de = ReadDirExtended(s_dir, SNAPSHOT_EXPORT_DIR, LOG)) != NULL)
+ while ((s_de = ReadDirExtended(s_dir, PG_SNAPSHOTS_DIR, LOG)) != NULL)
{
if (strcmp(s_de->d_name, ".") == 0 ||
strcmp(s_de->d_name, "..") == 0)
continue;
- snprintf(buf, sizeof(buf), SNAPSHOT_EXPORT_DIR "/%s", s_de->d_name);
+ snprintf(buf, sizeof(buf), PG_SNAPSHOTS_DIR "/%s", s_de->d_name);
if (unlink(buf) != 0)
ereport(LOG,
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index 0d75bd7..6b061c9 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1188,7 +1188,7 @@ setup_config(void)
autoconflines[1] = pg_strdup("# It will be overwritten by the ALTER SYSTEM command.\n");
autoconflines[2] = NULL;
- sprintf(path, "%s/postgresql.auto.conf", pg_data);
+ sprintf(path, "%s/%s", pg_data, PG_AUTOCONF_FILENAME);
writefile(path, autoconflines);
if (chmod(path, S_IRUSR | S_IWUSR) != 0)
@@ -2756,7 +2756,7 @@ create_xlog_or_symlink(void)
char *subdirloc;
/* form name of the place for the subdirectory or symlink */
- subdirloc = psprintf("%s/pg_wal", pg_data);
+ subdirloc = psprintf("%s/%s", pg_data, XLOGDIR);
if (xlog_dir)
{
@@ -2931,7 +2931,7 @@ initialize_data_directory(void)
/*
* Make the per-database PG_VERSION for template1 only after init'ing it
*/
- write_version_file("base/1");
+ write_version_file(PG_BASE_DIR "/1");
/*
* Create the stuff we don't need to use bootstrap mode for, using a
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 9bc830b0..7dc15b1 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -26,6 +26,7 @@
#include "catalog/pg_control.h"
#include "common/controldata_utils.h"
#include "getopt_long.h"
+#include "pg_paths.h"
#include "utils/pidfile.h"
#ifdef WIN32 /* on Unix, we don't need libpq */
@@ -2401,9 +2402,11 @@ main(int argc, char **argv)
if (pg_data)
{
- snprintf(postopts_file, MAXPGPATH, "%s/postmaster.opts", pg_data);
+ snprintf(postopts_file, MAXPGPATH, "%s/%s", pg_data,
+ POSTMASTER_OPTS_FILE);
snprintf(version_file, MAXPGPATH, "%s/PG_VERSION", pg_data);
- snprintf(pid_file, MAXPGPATH, "%s/postmaster.pid", pg_data);
+ snprintf(pid_file, MAXPGPATH, "%s/%s", pg_data,
+ POSTMASTER_PID_FILE);
snprintf(backup_file, MAXPGPATH, "%s/backup_label", pg_data);
}
diff --git a/src/bin/pg_resetwal/pg_resetwal.c b/src/bin/pg_resetwal/pg_resetwal.c
index a132cf2..673683a 100644
--- a/src/bin/pg_resetwal/pg_resetwal.c
+++ b/src/bin/pg_resetwal/pg_resetwal.c
@@ -55,6 +55,7 @@
#include "common/restricted_token.h"
#include "storage/large_object.h"
#include "pg_getopt.h"
+#include "pg_paths.h"
static ControlFileData ControlFile; /* pg_control values */
@@ -334,12 +335,12 @@ main(int argc, char *argv[])
* Check for a postmaster lock file --- if there is one, refuse to
* proceed, on grounds we might be interfering with a live installation.
*/
- if ((fd = open("postmaster.pid", O_RDONLY, 0)) < 0)
+ if ((fd = open(POSTMASTER_PID_FILE, O_RDONLY, 0)) < 0)
{
if (errno != ENOENT)
{
fprintf(stderr, _("%s: could not open file \"%s\" for reading: %s\n"),
- progname, "postmaster.pid", strerror(errno));
+ progname, POSTMASTER_PID_FILE, strerror(errno));
exit(1);
}
}
@@ -347,7 +348,7 @@ main(int argc, char *argv[])
{
fprintf(stderr, _("%s: lock file \"%s\" exists\n"
"Is a server running? If not, delete the lock file and try again.\n"),
- progname, "postmaster.pid");
+ progname, POSTMASTER_PID_FILE);
exit(1);
}
diff --git a/src/bin/pg_rewind/copy_fetch.c b/src/bin/pg_rewind/copy_fetch.c
index 04db409..3a44da3 100644
--- a/src/bin/pg_rewind/copy_fetch.c
+++ b/src/bin/pg_rewind/copy_fetch.c
@@ -22,6 +22,7 @@
#include "pg_rewind.h"
#include "catalog/catalog.h"
+#include "pg_paths.h"
static void recurse_dir(const char *datadir, const char *path,
process_file_callback_t callback);
@@ -131,8 +132,8 @@ recurse_dir(const char *datadir, const char *parentpath,
* to process all the tablespaces. We also follow a symlink if
* it's for pg_wal. Symlinks elsewhere are ignored.
*/
- if ((parentpath && strcmp(parentpath, "pg_tblspc") == 0) ||
- strcmp(path, "pg_wal") == 0)
+ if ((parentpath && strcmp(parentpath, PG_TABLESPACE_DIR) == 0) ||
+ strcmp(path, XLOGDIR) == 0)
recurse_dir(datadir, path, callback);
#else
pg_fatal("\"%s\" is a symbolic link, but symbolic links are not supported on this platform\n",
diff --git a/src/bin/pg_rewind/filemap.c b/src/bin/pg_rewind/filemap.c
index 6122f17..d34f032 100644
--- a/src/bin/pg_rewind/filemap.c
+++ b/src/bin/pg_rewind/filemap.c
@@ -21,6 +21,7 @@
#include "common/string.h"
#include "catalog/catalog.h"
#include "catalog/pg_tablespace.h"
+#include "pg_paths.h"
#include "storage/fd.h"
filemap_t *filemap = NULL;
@@ -74,8 +75,8 @@ process_source_file(const char *path, file_type_t type, size_t newsize,
/*
* Completely ignore some special files in source and destination.
*/
- if (strcmp(path, "postmaster.pid") == 0 ||
- strcmp(path, "postmaster.opts") == 0)
+ if (strcmp(path, POSTMASTER_PID_FILE) == 0 ||
+ strcmp(path, POSTMASTER_OPTS_FILE) == 0)
return;
/*
@@ -83,7 +84,7 @@ process_source_file(const char *path, file_type_t type, size_t newsize,
* don't want to mess with the symlink itself, nor complain if it's a
* symlink in source but not in target or vice versa.
*/
- if (strcmp(path, "pg_wal") == 0 && type == FILE_TYPE_SYMLINK)
+ if (strcmp(path, XLOGDIR) == 0 && type == FILE_TYPE_SYMLINK)
type = FILE_TYPE_DIRECTORY;
/*
@@ -120,7 +121,7 @@ process_source_file(const char *path, file_type_t type, size_t newsize,
switch (type)
{
case FILE_TYPE_DIRECTORY:
- if (exists && !S_ISDIR(statbuf.st_mode) && strcmp(path, "pg_wal") != 0)
+ if (exists && !S_ISDIR(statbuf.st_mode) && strcmp(path, XLOGDIR) != 0)
{
/* it's a directory in source, but not in target. Strange.. */
pg_fatal("\"%s\" is not a directory\n", localpath);
@@ -289,14 +290,14 @@ process_target_file(const char *path, file_type_t type, size_t oldsize,
/*
* Completely ignore some special files
*/
- if (strcmp(path, "postmaster.pid") == 0 ||
- strcmp(path, "postmaster.opts") == 0)
+ if (strcmp(path, POSTMASTER_PID_FILE) == 0 ||
+ strcmp(path, POSTMASTER_OPTS_FILE) == 0)
return;
/*
* Like in process_source_file, pretend that xlog is always a directory.
*/
- if (strcmp(path, "pg_wal") == 0 && type == FILE_TYPE_SYMLINK)
+ if (strcmp(path, XLOGDIR) == 0 && type == FILE_TYPE_SYMLINK)
type = FILE_TYPE_DIRECTORY;
key.path = (char *) path;
@@ -585,7 +586,7 @@ isRelDataFile(const char *path)
segNo = 0;
matched = false;
- nmatch = sscanf(path, "global/%u.%u", &rnode.relNode, &segNo);
+ nmatch = sscanf(path, PG_GLOBAL_DIR "/%u.%u", &rnode.relNode, &segNo);
if (nmatch == 1 || nmatch == 2)
{
rnode.spcNode = GLOBALTABLESPACE_OID;
@@ -594,7 +595,7 @@ isRelDataFile(const char *path)
}
else
{
- nmatch = sscanf(path, "base/%u/%u.%u",
+ nmatch = sscanf(path, PG_BASE_DIR "/%u/%u.%u",
&rnode.dbNode, &rnode.relNode, &segNo);
if (nmatch == 2 || nmatch == 3)
{
diff --git a/src/bin/pg_rewind/pg_rewind.c b/src/bin/pg_rewind/pg_rewind.c
index 72ab2f8..56aabfd 100644
--- a/src/bin/pg_rewind/pg_rewind.c
+++ b/src/bin/pg_rewind/pg_rewind.c
@@ -210,11 +210,11 @@ main(int argc, char **argv)
* Ok, we have all the options and we're ready to start. Read in all the
* information we need from both clusters.
*/
- buffer = slurpFile(datadir_target, "global/pg_control", &size);
+ buffer = slurpFile(datadir_target, XLOG_CONTROL_FILE, &size);
digestControlFile(&ControlFile_target, buffer, size);
pg_free(buffer);
- buffer = fetchFile("global/pg_control", &size);
+ buffer = fetchFile(XLOG_CONTROL_FILE, &size);
digestControlFile(&ControlFile_source, buffer, size);
pg_free(buffer);
@@ -675,7 +675,7 @@ updateControlFile(ControlFileData *ControlFile)
memset(buffer, 0, PG_CONTROL_FILE_SIZE);
memcpy(buffer, ControlFile, sizeof(ControlFileData));
- open_target_file("global/pg_control", false);
+ open_target_file(XLOG_CONTROL_FILE, false);
write_target_range(buffer, 0, PG_CONTROL_FILE_SIZE);
diff --git a/src/bin/pg_upgrade/controldata.c b/src/bin/pg_upgrade/controldata.c
index 0fe98a5..7dce5ec 100644
--- a/src/bin/pg_upgrade/controldata.c
+++ b/src/bin/pg_upgrade/controldata.c
@@ -13,6 +13,8 @@
#include <ctype.h>
+#include "pg_paths.h"
+
/*
* get_control_data()
*
@@ -614,15 +616,18 @@ disable_old_cluster(void)
/* rename pg_control so old server cannot be accidentally started */
prep_status("Adding \".old\" suffix to old global/pg_control");
- snprintf(old_path, sizeof(old_path), "%s/global/pg_control", old_cluster.pgdata);
- snprintf(new_path, sizeof(new_path), "%s/global/pg_control.old", old_cluster.pgdata);
+ snprintf(old_path, sizeof(old_path), "%s/%s", old_cluster.pgdata,
+ XLOG_CONTROL_FILE);
+ snprintf(new_path, sizeof(new_path), "%s/%s.old", old_cluster.pgdata,
+ XLOG_CONTROL_FILE);
if (pg_mv_file(old_path, new_path) != 0)
pg_fatal("Unable to rename %s to %s.\n", old_path, new_path);
check_ok();
pg_log(PG_REPORT, "\n"
"If you want to start the old cluster, you will need to remove\n"
- "the \".old\" suffix from %s/global/pg_control.old.\n"
+ "the \".old\" suffix from %s/%s.old.\n"
"Because \"link\" mode was used, the old cluster cannot be safely\n"
- "started once the new cluster has been started.\n\n", old_cluster.pgdata);
+ "started once the new cluster has been started.\n\n",
+ old_cluster.pgdata, XLOG_CONTROL_FILE);
}
diff --git a/src/bin/pg_upgrade/exec.c b/src/bin/pg_upgrade/exec.c
index 9122e27..4138455 100644
--- a/src/bin/pg_upgrade/exec.c
+++ b/src/bin/pg_upgrade/exec.c
@@ -11,6 +11,7 @@
#include <fcntl.h>
+#include "pg_paths.h"
#include "pg_upgrade.h"
static void check_data_dir(ClusterInfo *cluster);
@@ -219,7 +220,8 @@ pid_lock_file_exists(const char *datadir)
char path[MAXPGPATH];
int fd;
- snprintf(path, sizeof(path), "%s/postmaster.pid", datadir);
+ snprintf(path, sizeof(path), "%s/%s", datadir,
+ POSTMASTER_PID_FILE);
if ((fd = open(path, O_RDONLY, 0)) < 0)
{
@@ -331,12 +333,12 @@ check_data_dir(ClusterInfo *cluster)
cluster->major_version = get_major_server_version(cluster);
check_single_dir(pg_data, "");
- check_single_dir(pg_data, "base");
- check_single_dir(pg_data, "global");
- check_single_dir(pg_data, "pg_multixact");
- check_single_dir(pg_data, "pg_subtrans");
- check_single_dir(pg_data, "pg_tblspc");
- check_single_dir(pg_data, "pg_twophase");
+ check_single_dir(pg_data, PG_BASE_DIR);
+ check_single_dir(pg_data, PG_GLOBAL_DIR);
+ check_single_dir(pg_data, PG_MULTIXACT_DIR);
+ check_single_dir(pg_data, PG_SUBTRANS_DIR);
+ check_single_dir(pg_data, PG_TABLESPACE_DIR);
+ check_single_dir(pg_data, PG_TWOPHASE_DIR);
/* pg_xlog has been renamed to pg_wal in v10 */
if (GET_MAJOR_VERSION(cluster->major_version) < 1000)
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index 9dbc922..6d96c78 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -15,6 +15,7 @@
#endif
#include "getopt_long.h"
+#include "pg_paths.h"
#include "utils/pidfile.h"
#include "pg_upgrade.h"
@@ -474,8 +475,8 @@ get_sock_dir(ClusterInfo *cluster, bool live_check)
FILE *fp;
int lineno;
- snprintf(filename, sizeof(filename), "%s/postmaster.pid",
- cluster->pgdata);
+ snprintf(filename, sizeof(filename), "%s/%s",
+ cluster->pgdata, POSTMASTER_PID_FILE);
if ((fp = fopen(filename, "r")) == NULL)
pg_fatal("could not open file \"%s\": %s\n",
filename, strerror(errno));
diff --git a/src/bin/pg_upgrade/pg_upgrade.c b/src/bin/pg_upgrade/pg_upgrade.c
index d124127..f11d772 100644
--- a/src/bin/pg_upgrade/pg_upgrade.c
+++ b/src/bin/pg_upgrade/pg_upgrade.c
@@ -40,6 +40,7 @@
#include "catalog/pg_class.h"
#include "common/restricted_token.h"
#include "fe_utils/string_utils.h"
+#include "pg_paths.h"
#ifdef HAVE_LANGINFO_H
#include <langinfo.h>
@@ -483,8 +484,8 @@ copy_xact_xlog_xid(void)
if (old_cluster.controldata.cat_ver >= MULTIXACT_FORMATCHANGE_CAT_VER &&
new_cluster.controldata.cat_ver >= MULTIXACT_FORMATCHANGE_CAT_VER)
{
- copy_subdir_files("pg_multixact/offsets", "pg_multixact/offsets");
- copy_subdir_files("pg_multixact/members", "pg_multixact/members");
+ copy_subdir_files(PG_MULTIXACT_OFFSETS_DIR, PG_MULTIXACT_OFFSETS_DIR);
+ copy_subdir_files(PG_MULTIXACT_MEMBERS_DIR, PG_MULTIXACT_MEMBERS_DIR);
prep_status("Setting next multixact ID and offset for new cluster");
@@ -508,7 +509,7 @@ copy_xact_xlog_xid(void)
* the new multi-xid value. "members" starts at zero so no need to
* remove it.
*/
- remove_new_subdir("pg_multixact/offsets", false);
+ remove_new_subdir(PG_MULTIXACT_OFFSETS_DIR, false);
prep_status("Setting oldest multixact ID in new cluster");
diff --git a/src/common/controldata_utils.c b/src/common/controldata_utils.c
index f12a188..a93a567 100644
--- a/src/common/controldata_utils.c
+++ b/src/common/controldata_utils.c
@@ -26,6 +26,7 @@
#include "catalog/pg_control.h"
#include "common/controldata_utils.h"
+#include "pg_paths.h"
#include "port/pg_crc32c.h"
/*
@@ -48,7 +49,8 @@ get_controlfile(const char *DataDir, const char *progname, bool *crc_ok_p)
AssertArg(crc_ok_p);
ControlFile = palloc(sizeof(ControlFileData));
- snprintf(ControlFilePath, MAXPGPATH, "%s/global/pg_control", DataDir);
+ snprintf(ControlFilePath, MAXPGPATH, "%s/%s", DataDir,
+ XLOG_CONTROL_FILE);
if ((fd = open(ControlFilePath, O_RDONLY | PG_BINARY, 0)) == -1)
#ifndef FRONTEND
diff --git a/src/common/relpath.c b/src/common/relpath.c
index d98050c..e34aa4c 100644
--- a/src/common/relpath.c
+++ b/src/common/relpath.c
@@ -21,6 +21,7 @@
#include "catalog/catalog.h"
#include "catalog/pg_tablespace.h"
#include "common/relpath.h"
+#include "pg_paths.h"
#include "storage/backendid.h"
@@ -111,17 +112,17 @@ GetDatabasePath(Oid dbNode, Oid spcNode)
{
/* Shared system relations live in {datadir}/global */
Assert(dbNode == 0);
- return pstrdup("global");
+ return pstrdup(PG_GLOBAL_DIR);
}
else if (spcNode == DEFAULTTABLESPACE_OID)
{
/* The default tablespace is {datadir}/base */
- return psprintf("base/%u", dbNode);
+ return psprintf(PG_BASE_DIR "/%u", dbNode);
}
else
{
/* All other tablespaces are accessed via symlinks */
- return psprintf("pg_tblspc/%u/%s/%u",
+ return psprintf(PG_TABLESPACE_DIR "/%u/%s/%u",
spcNode, TABLESPACE_VERSION_DIRECTORY, dbNode);
}
}
@@ -147,10 +148,11 @@ GetRelationPath(Oid dbNode, Oid spcNode, Oid relNode,
Assert(dbNode == 0);
Assert(backendId == InvalidBackendId);
if (forkNumber != MAIN_FORKNUM)
- path = psprintf("global/%u_%s",
+ path = psprintf("%s/%u_%s",
+ PG_GLOBAL_DIR,
relNode, forkNames[forkNumber]);
else
- path = psprintf("global/%u", relNode);
+ path = psprintf("%s/%u", PG_GLOBAL_DIR, relNode);
}
else if (spcNode == DEFAULTTABLESPACE_OID)
{
@@ -158,22 +160,22 @@ GetRelationPath(Oid dbNode, Oid spcNode, Oid relNode,
if (backendId == InvalidBackendId)
{
if (forkNumber != MAIN_FORKNUM)
- path = psprintf("base/%u/%u_%s",
- dbNode, relNode,
+ path = psprintf("%s/%u/%u_%s",
+ PG_BASE_DIR, dbNode, relNode,
forkNames[forkNumber]);
else
- path = psprintf("base/%u/%u",
- dbNode, relNode);
+ path = psprintf("%s/%u/%u",
+ PG_BASE_DIR, dbNode, relNode);
}
else
{
if (forkNumber != MAIN_FORKNUM)
- path = psprintf("base/%u/t%d_%u_%s",
- dbNode, backendId, relNode,
+ path = psprintf("%s/%u/t%d_%u_%s",
+ PG_BASE_DIR, dbNode, backendId, relNode,
forkNames[forkNumber]);
else
- path = psprintf("base/%u/t%d_%u",
- dbNode, backendId, relNode);
+ path = psprintf("%s/%u/t%d_%u",
+ PG_BASE_DIR, dbNode, backendId, relNode);
}
}
else
0003-Add-backup_label-pg_internal.init-and-tablespace_map.patchtext/x-patch; name=0003-Add-backup_label-pg_internal.init-and-tablespace_map.patchDownload
From b01fbf41f6d9d3b1fe211c8ea2a929ce36d1e658 Mon Sep 17 00:00:00 2001
From: Michael Paquier <michael@paquier.xyz>
Date: Mon, 5 Feb 2018 15:01:42 +0900
Subject: [PATCH 3/5] Add backup_label, pg_internal.init and tablespace_map to
pg_paths.h
Those are going to be used in the list of files to be excluded from base
backups which is made available to the frontends.
---
src/bin/pg_ctl/pg_ctl.c | 3 ++-
src/bin/pg_rewind/pg_rewind.c | 2 +-
src/include/access/xlog.h | 7 -------
src/include/pg_paths.h | 17 +++++++++++++++++
src/include/utils/relcache.h | 5 -----
5 files changed, 20 insertions(+), 14 deletions(-)
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 7dc15b13b0..7f9106a527 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -2407,7 +2407,8 @@ main(int argc, char **argv)
snprintf(version_file, MAXPGPATH, "%s/PG_VERSION", pg_data);
snprintf(pid_file, MAXPGPATH, "%s/%s", pg_data,
POSTMASTER_PID_FILE);
- snprintf(backup_file, MAXPGPATH, "%s/backup_label", pg_data);
+ snprintf(backup_file, MAXPGPATH, "%s/%s", pg_data,
+ BACKUP_LABEL_FILE);
}
switch (ctl_command)
diff --git a/src/bin/pg_rewind/pg_rewind.c b/src/bin/pg_rewind/pg_rewind.c
index 56aabfd4af..bbd514f6f1 100644
--- a/src/bin/pg_rewind/pg_rewind.c
+++ b/src/bin/pg_rewind/pg_rewind.c
@@ -597,7 +597,7 @@ createBackupLabel(XLogRecPtr startpoint, TimeLineID starttli, XLogRecPtr checkpo
pg_fatal("backup label buffer too small\n"); /* shouldn't happen */
/* TODO: move old file out of the way, if any. */
- open_target_file("backup_label", true); /* BACKUP_LABEL_FILE */
+ open_target_file(BACKUP_LABEL_FILE, true);
write_target_range(buf, 0, len);
close_target_file();
}
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 421ba6d775..ca4cbc811a 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -318,11 +318,4 @@ extern XLogRecPtr do_pg_stop_backup(char *labelfile, bool waitforarchive,
extern void do_pg_abort_backup(void);
extern SessionBackupState get_backup_status(void);
-/* File path names (all relative to $PGDATA) */
-#define BACKUP_LABEL_FILE "backup_label"
-#define BACKUP_LABEL_OLD "backup_label.old"
-
-#define TABLESPACE_MAP "tablespace_map"
-#define TABLESPACE_MAP_OLD "tablespace_map.old"
-
#endif /* XLOG_H */
diff --git a/src/include/pg_paths.h b/src/include/pg_paths.h
index a4746e75e4..f033f8f390 100644
--- a/src/include/pg_paths.h
+++ b/src/include/pg_paths.h
@@ -13,6 +13,23 @@
#ifndef PG_PATHS_H
#define PG_PATHS_H
+/*
+ * Backup label file names used in base backups.
+ */
+#define BACKUP_LABEL_FILE "backup_label"
+#define BACKUP_LABEL_OLD "backup_label.old"
+
+/*
+ * Tablespace map files used in base backups.
+ */
+#define TABLESPACE_MAP "tablespace_map"
+#define TABLESPACE_MAP_OLD "tablespace_map.old"
+
+/*
+ * Name of relcache init file(s), used to speed up backend startup
+ */
+#define RELCACHE_INIT_FILENAME "pg_internal.init"
+
/*
* Name of files saving meta-data information about the log
* files currently in use by the syslogger
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 8a546aba28..7a884cb26a 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -18,11 +18,6 @@
#include "nodes/bitmapset.h"
-/*
- * Name of relcache init file(s), used to speed up backend startup
- */
-#define RELCACHE_INIT_FILENAME "pg_internal.init"
-
typedef struct RelationData *Relation;
/* ----------------
--
2.16.1
0004-Move-base-backup-filter-lists-into-their-own-header-.patchtext/x-patch; name=0004-Move-base-backup-filter-lists-into-their-own-header-.patchDownload
From 7aaafd2631386bb2bd7e628334a8409fbb648402 Mon Sep 17 00:00:00 2001
From: Michael Paquier <michael@paquier.xyz>
Date: Mon, 5 Feb 2018 14:43:10 +0900
Subject: [PATCH 4/5] Move base backup filter lists into their own header file
This list is for now consumed only by the backend-side base backup code,
which is used to filter out a set of paths and/or files which cannot be
included in base backup data sent to clients.
A follow-up patch will make use of that in pg_rewind itself.
---
src/backend/replication/basebackup.c | 82 ++-------------------------
src/include/replication/basebackup_paths.h | 89 ++++++++++++++++++++++++++++++
2 files changed, 94 insertions(+), 77 deletions(-)
create mode 100644 src/include/replication/basebackup_paths.h
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index 2e764ab293..2ea2306cfa 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -28,6 +28,7 @@
#include "pgstat.h"
#include "postmaster/syslogger.h"
#include "replication/basebackup.h"
+#include "replication/basebackup_paths.h"
#include "replication/walsender.h"
#include "replication/walsender_private.h"
#include "storage/dsm_impl.h"
@@ -98,79 +99,6 @@ static TimeOffset elapsed_min_unit;
/* The last check of the transfer rate. */
static TimestampTz throttled_last;
-/*
- * The contents of these directories are removed or recreated during server
- * start so they are not included in backups. The directories themselves are
- * kept and included as empty to preserve access permissions.
- */
-static const char *excludeDirContents[] =
-{
- /*
- * Skip temporary statistics files. PG_STAT_TMP_DIR must be skipped even
- * when stats_temp_directory is set because PGSS_TEXT_FILE is always
- * created there.
- */
- PG_STAT_TMP_DIR,
-
- /*
- * It is generally not useful to backup the contents of this directory
- * even if the intention is to restore to another master. See backup.sgml
- * for a more detailed description.
- */
- PG_REPLSLOT_DIR,
-
- /* Contents removed on startup, see dsm_cleanup_for_mmap(). */
- PG_DYNSHMEM_DIR,
-
- /* Contents removed on startup, see AsyncShmemInit(). */
- PG_NOTIFY_DIR,
-
- /*
- * Old contents are loaded for possible debugging but are not required for
- * normal operation, see OldSerXidInit().
- */
- PG_SERIAL_DIR,
-
- /* Contents removed on startup, see DeleteAllExportedSnapshotFiles(). */
- PG_SNAPSHOTS_DIR,
-
- /* Contents zeroed on startup, see StartupSUBTRANS(). */
- PG_SUBTRANS_DIR,
-
- /* end of list */
- NULL
-};
-
-/*
- * List of files excluded from backups.
- */
-static const char *excludeFiles[] =
-{
- /* Skip auto conf temporary file. */
- PG_AUTOCONF_FILENAME_TMP,
-
- /* Skip current log file temporary file */
- LOG_METAINFO_DATAFILE_TMP,
-
- /* Skip relation cache because it is rebuilt on startup */
- RELCACHE_INIT_FILENAME,
-
- /*
- * If there's a backup_label or tablespace_map file, it belongs to a
- * backup started by the user with pg_start_backup(). It is *not* correct
- * for this backup. Our backup_label/tablespace_map is injected into the
- * tar separately.
- */
- BACKUP_LABEL_FILE,
- TABLESPACE_MAP,
-
- POSTMASTER_PID_FILE,
- POSTMASTER_OPTS_FILE,
-
- /* end of list */
- NULL
-};
-
/*
* Called when ERROR or FATAL happens in perform_base_backup() after
* we have started the backup - make sure we end it!
@@ -995,9 +923,9 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
/* Scan for files that should be excluded */
excludeFound = false;
- for (excludeIdx = 0; excludeFiles[excludeIdx] != NULL; excludeIdx++)
+ for (excludeIdx = 0; backupExcludeFiles[excludeIdx] != NULL; excludeIdx++)
{
- if (strcmp(de->d_name, excludeFiles[excludeIdx]) == 0)
+ if (strcmp(de->d_name, backupExcludeFiles[excludeIdx]) == 0)
{
elog(DEBUG1, "file \"%s\" excluded from backup", de->d_name);
excludeFound = true;
@@ -1028,9 +956,9 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
/* Scan for directories whose contents should be excluded */
excludeFound = false;
- for (excludeIdx = 0; excludeDirContents[excludeIdx] != NULL; excludeIdx++)
+ for (excludeIdx = 0; backupExcludeDirs[excludeIdx] != NULL; excludeIdx++)
{
- if (strcmp(de->d_name, excludeDirContents[excludeIdx]) == 0)
+ if (strcmp(de->d_name, backupExcludeDirs[excludeIdx]) == 0)
{
elog(DEBUG1, "contents of directory \"%s\" excluded from backup", de->d_name);
size += _tarWriteDir(pathbuf, basepathlen, &statbuf, sizeonly);
diff --git a/src/include/replication/basebackup_paths.h b/src/include/replication/basebackup_paths.h
new file mode 100644
index 0000000000..64c9f20333
--- /dev/null
+++ b/src/include/replication/basebackup_paths.h
@@ -0,0 +1,89 @@
+/*-------------------------------------------------------------------------
+ *
+ * basebackup_paths.h
+ * Filter lists when working on base backups. Can be used by both
+ * frontends and backends.
+ *
+ * Portions Copyright (c) 2018, PostgreSQL Global Development Group
+ *
+ * src/include/replication/basebackup_paths.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef _BASEBACKUP_PATHS_H
+#define _BASEBACKUP_PATHS_H
+
+/*
+ * The contents of these directories are removed or recreated during server
+ * start so they are not included in backups. The directories themselves are
+ * kept and included as empty to preserve access permissions.
+ */
+static const char *backupExcludeDirs[] =
+{
+ /*
+ * Skip temporary statistics files. PG_STAT_TMP_DIR must be skipped even
+ * when stats_temp_directory is set because PGSS_TEXT_FILE is always
+ * created there.
+ */
+ PG_STAT_TMP_DIR,
+
+ /*
+ * It is generally not useful to backup the contents of this directory
+ * even if the intention is to restore to another master. See backup.sgml
+ * for a more detailed description.
+ */
+ PG_REPLSLOT_DIR,
+
+ /* Contents removed on startup, see dsm_cleanup_for_mmap(). */
+ PG_DYNSHMEM_DIR,
+
+ /* Contents removed on startup, see AsyncShmemInit(). */
+ PG_NOTIFY_DIR,
+
+ /*
+ * Old contents are loaded for possible debugging but are not required for
+ * normal operation, see OldSerXidInit().
+ */
+ PG_SERIAL_DIR,
+
+ /* Contents removed on startup, see DeleteAllExportedSnapshotFiles(). */
+ PG_SNAPSHOTS_DIR,
+
+ /* Contents zeroed on startup, see StartupSUBTRANS(). */
+ PG_SUBTRANS_DIR,
+
+ /* end of list */
+ NULL
+};
+
+/*
+ * List of files excluded from base backups.
+ */
+static const char *backupExcludeFiles[] =
+{
+ /* Skip auto conf temporary file. */
+ PG_AUTOCONF_FILENAME_TMP,
+
+ /* Skip current log file temporary file */
+ LOG_METAINFO_DATAFILE_TMP,
+
+ /* Skip relation cache because it is rebuilt on startup */
+ RELCACHE_INIT_FILENAME,
+
+ /*
+ * If there's a backup_label or tablespace_map file, it belongs to a
+ * backup started by the user with pg_start_backup(). It is *not* correct
+ * for this backup. Our backup_label/tablespace_map is injected into the
+ * tar separately.
+ */
+ BACKUP_LABEL_FILE,
+ TABLESPACE_MAP,
+
+ POSTMASTER_PID_FILE,
+ POSTMASTER_OPTS_FILE,
+
+ /* end of list */
+ NULL
+};
+
+#endif /* _BASEBACKUP_PATHS_H */
--
2.16.1
0005-Use-filtering-list-of-base-backups-in-pg_rewind-to-e.patchtext/x-patch; name=0005-Use-filtering-list-of-base-backups-in-pg_rewind-to-e.patchDownload
From 9ac82aebd2de8d3821633fcbc0838bd872f2e423 Mon Sep 17 00:00:00 2001
From: Michael Paquier <michael@paquier.xyz>
Date: Mon, 5 Feb 2018 15:48:35 +0900
Subject: [PATCH 5/5] Use filtering list of base backups in pg_rewind to
exclude more content
After being rewound, a standby to-be-recycled needs to perform recovery
from the last checkpoint where WAL forked after a promotion, which leads
it to automatically remove some files which may have been copied from
the source cluster. Make use of the same filtering list as base backups
to find out what is this data and then remove it. This reduces the
amount of data transferred during a rewind without changing the
usefulness of the operation. This takes advantage of the
newly-introduced basebackup_paths.h to allow pg_rewind to have access to
this data.
Documentation is updated to take into account what is filtered out.
---
doc/src/sgml/ref/pg_rewind.sgml | 12 ++++++++-
src/bin/pg_rewind/filemap.c | 60 +++++++++++++++++++++++++++++++++++------
2 files changed, 63 insertions(+), 9 deletions(-)
diff --git a/doc/src/sgml/ref/pg_rewind.sgml b/doc/src/sgml/ref/pg_rewind.sgml
index 8e49249826..ccd0ae505e 100644
--- a/doc/src/sgml/ref/pg_rewind.sgml
+++ b/doc/src/sgml/ref/pg_rewind.sgml
@@ -231,7 +231,17 @@ PostgreSQL documentation
<para>
Copy all other files such as <filename>pg_xact</filename> and
configuration files from the source cluster to the target cluster
- (everything except the relation files).
+ (everything except the relation files). Similarly to base backups,
+ the contents of the directories <filename>pg_dynshmem/</filename>,
+ <filename>pg_notify/</filename>, <filename>pg_replslot/</filename>,
+ <filename>pg_serial/</filename>, <filename>pg_snapshots/</filename>,
+ <filename>pg_stat_tmp/</filename>, and
+ <filename>pg_subtrans/</filename> are omitted from the data copied
+ from the source cluster. Any file or directory beginning with
+ <filename>pgsql_tmp</filename> is omitted, as well as are
+ <filename>pg_internal.init</filename>,
+ <filename>postmaster.opts</filename> and
+ <filename>postmaster.pid</filename>.
</para>
</step>
<step>
diff --git a/src/bin/pg_rewind/filemap.c b/src/bin/pg_rewind/filemap.c
index 1c462c1041..0a6426e6c2 100644
--- a/src/bin/pg_rewind/filemap.c
+++ b/src/bin/pg_rewind/filemap.c
@@ -21,6 +21,7 @@
#include "common/string.h"
#include "catalog/pg_tablespace.h"
#include "pg_paths.h"
+#include "replication/basebackup_paths.h"
#include "storage/fd.h"
filemap_t *filemap = NULL;
@@ -68,15 +69,37 @@ process_source_file(const char *path, file_type_t type, size_t newsize,
file_action_t action = FILE_ACTION_NONE;
size_t oldsize = 0;
file_entry_t *entry;
+ int excludeIdx;
Assert(map->array == NULL);
/*
- * Completely ignore some special files in source and destination.
+ * Completely ignore some special files in source and destination. This
+ * filters willingly any files matching an entry in the list of files to
+ * filter out.
*/
- if (strcmp(path, POSTMASTER_PID_FILE) == 0 ||
- strcmp(path, POSTMASTER_OPTS_FILE) == 0)
- return;
+ for (excludeIdx = 0; backupExcludeFiles[excludeIdx] != NULL; excludeIdx++)
+ {
+ if (strstr(path, backupExcludeFiles[excludeIdx]) != NULL)
+ {
+ pg_log(PG_DEBUG, "file \"%s\" excluded from source file list\n",
+ path);
+ return;
+ }
+ }
+
+ /*
+ * ... And ignore some directories.
+ */
+ for (excludeIdx = 0; backupExcludeDirs[excludeIdx] != NULL; excludeIdx++)
+ {
+ if (strcmp(path, backupExcludeDirs[excludeIdx]) == 0)
+ {
+ pg_log(PG_DEBUG, "directory \"%s\" excluded from source file list\n",
+ path);
+ return;
+ }
+ }
/*
* Pretend that pg_wal is a directory, even if it's really a symlink. We
@@ -259,6 +282,7 @@ process_target_file(const char *path, file_type_t type, size_t oldsize,
file_entry_t *key_ptr;
filemap_t *map = filemap;
file_entry_t *entry;
+ int excludeIdx;
snprintf(localpath, sizeof(localpath), "%s/%s", datadir_target, path);
if (lstat(localpath, &statbuf) < 0)
@@ -287,11 +311,31 @@ process_target_file(const char *path, file_type_t type, size_t oldsize,
}
/*
- * Completely ignore some special files
+ * Completely ignore some special files. This filters willingly any files
+ * matching an entry in the list of files to filter out.
*/
- if (strcmp(path, POSTMASTER_PID_FILE) == 0 ||
- strcmp(path, POSTMASTER_OPTS_FILE) == 0)
- return;
+ for (excludeIdx = 0; backupExcludeFiles[excludeIdx] != NULL; excludeIdx++)
+ {
+ if (strstr(path, backupExcludeFiles[excludeIdx]) != NULL)
+ {
+ pg_log(PG_DEBUG, "file \"%s\" excluded from target file list\n",
+ path);
+ return;
+ }
+ }
+
+ /*
+ * ... And ignore some directories.
+ */
+ for (excludeIdx = 0; backupExcludeDirs[excludeIdx] != NULL; excludeIdx++)
+ {
+ if (strcmp(path, backupExcludeDirs[excludeIdx]) == 0)
+ {
+ pg_log(PG_DEBUG, "directory \"%s\" excluded from target file list\n",
+ path);
+ return;
+ }
+ }
/*
* Like in process_source_file, pretend that xlog is always a directory.
--
2.16.1
0006-Minor-renames.patchtext/x-patch; name=0006-Minor-renames.patchDownload
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 0598212..a7b64bd 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6460,14 +6460,16 @@ StartupXLOG(void)
if (!ReadRecord(xlogreader, checkPoint.redo, LOG, false))
ereport(FATAL,
(errmsg("could not find redo location referenced by checkpoint record"),
- errhint("If you are not restoring from a backup, try removing the file \"%s/backup_label\".", DataDir)));
+ errhint("If you are not restoring from a backup, try removing the file \"%s/%s\".",
+ DataDir, BACKUP_LABEL_FILE)));
}
}
else
{
ereport(FATAL,
(errmsg("could not locate required checkpoint record"),
- errhint("If you are not restoring from a backup, try removing the file \"%s/backup_label\".", DataDir)));
+ errhint("If you are not restoring from a backup, try removing the file \"%s/%s\".",
+ DataDir, BACKUP_LABEL_FILE)));
wasShutdown = false; /* keep compiler quiet */
}
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index 6a202fb..22dcbaa 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -590,7 +590,7 @@ parse_basebackup_options(List *options, basebackup_options *opt)
opt->maxrate = (uint32) maxrate;
o_maxrate = true;
}
- else if (strcmp(defel->defname, "tablespace_map") == 0)
+ else if (strcmp(defel->defname, TABLESPACE_MAP) == 0)
{
if (o_tablespace_map)
ereport(ERROR,
On Tue, Mar 13, 2018 at 01:16:27PM +0300, Anastasia Lubennikova wrote:
Since these patches contain mostly cosmetic changes and do not break
anything, I think it's fine to mark this thread as Ready For Committer
without long discussion.
Thanks Anastasia for the review. The refactoring is quite intuitive I
think, still that's perhaps a bit too much intrusive. Extra opinions
about that are welcome.
As I read again the patch set, please note that I am not much happy yet
about the part for the handling of temporary files when applying the
filters in pg_rewind and the lack inconsistency for file filters and
directory filters...
--
Michael
On Tue, Mar 13, 2018 at 9:48 PM, Michael Paquier <michael@paquier.xyz> wrote:
On Tue, Mar 13, 2018 at 01:16:27PM +0300, Anastasia Lubennikova wrote:
Since these patches contain mostly cosmetic changes and do not break
anything, I think it's fine to mark this thread as Ready For Committer
without long discussion.Thanks Anastasia for the review. The refactoring is quite intuitive I
think, still that's perhaps a bit too much intrusive. Extra opinions
about that are welcome.
Personally it looks very intrusive, so I'm feeling inclined to push
the changes without that refactoring.
The 0005 patch doesn't seem to be right. The patch changes process_source_file()
so that it excludes some directories like pg_notify from the filemap. However,
after that, process_source_file() is called for the files under those "excluded"
directories and then they are not excluded. For example, pg_notify/0000 file is
unexpectedly included in the filemap and copied by pg_rewind.
This problem happens because recurse_dir() has the following code and
ISTM you forgot to take into account it.
else if (S_ISDIR(fst.st_mode))
{
callback(path, FILE_TYPE_DIRECTORY, 0, NULL);
/* recurse to handle subdirectories */
recurse_dir(datadir, path, callback);
}
Regards,
--
Fujii Masao
On Fri, Mar 23, 2018 at 01:38:38AM +0900, Fujii Masao wrote:
Personally it looks very intrusive, so I'm feeling inclined to push
the changes without that refactoring.
Okay. Just moving the list of items from basebackup.c to a dedicated
header is not sufficient though as things like RELCACHE_INIT_FILENAME as
declared in headers which are backend-only. The same applies to
LOG_METAINFO_DATAFILE_TMP, PG_AUTOCONF_FILENAME, PG_STAT_TMP_DIR and
PG_DYNSHMEM_DIR.
BACKUP_LABEL_FILE and TABLESPACE_MAP can be included though via xlog.h.
So what are you looking for? I see a couple of options:
1) The inclusive refactoring, which you are discarding.
2) A dedicated header, but some of the now-not-hardcoded values will
need to be so. That's the list I am giving above.
3) A copy of the list from basebackup.c to src/bin/pg_rewind/.
I would guess that you are looking for 2), but I am not sure if you
imply that 3) would be acceptable or not.
--
Michael
Greetings,
* Michael Paquier (michael@paquier.xyz) wrote:
On Fri, Mar 23, 2018 at 01:38:38AM +0900, Fujii Masao wrote:
Personally it looks very intrusive, so I'm feeling inclined to push
the changes without that refactoring.
I've been reading over the first couple of posted patches and mulling
over the changes proposed. They certainly touch a lot of places but
they're pretty straight-forward changes and so I'm not really sure I'd
call them all that intrusive.
I don't completely buy off on the argument that having these #define's
would make it easier for forks (we've had quite a few folks fork PG, but
how many of them have actually changed "base"?) and I'm on the fence
about if these will make our lives simpler down the road when it comes
to changing the directory names (if we changed "pg_multixact/members" to
be "pg_multixact/processes" or some such, I imagine we'd also go through
and change PG_MULTIXACT_MEMBERS_DIR to be PG_MULTIXACT_PROCESSES_DIR
anyway, so this doesn't result in much improvement there), but as it
relates to new tool development and such, there's some value here
because the compiler is going to complain if you say
PG_COMMIT_TX_DIR and there is no such #define, whereas a similar mistake
with a string literal of "pg_commit_tx" might end up getting missed and
that would be unfortunate.
Therefore, on the whole, I'm +1 on these changes, but I'd argue a bit
about some of the choices made:
- Let's have them all be PG_WHATEVER_DIR/FILE
- Use WAL instead of XLOG (I thought we agreed new code would..?)
- Remove extraneous #define's (most of these were, but
DIRECTORY_LOCK_FILE was kept..? Let's use PG_POSTMASTER_PID_FILE
throughout instead, with a comment that it's also used as a lock
file).
Might be nice to go back and modify other pre-existing #define's to use
that form also, but perhaps not that big of a deal. I would have that
be independent from this, in any case.
Okay. Just moving the list of items from basebackup.c to a dedicated
header is not sufficient though as things like RELCACHE_INIT_FILENAME as
declared in headers which are backend-only. The same applies to
LOG_METAINFO_DATAFILE_TMP, PG_AUTOCONF_FILENAME, PG_STAT_TMP_DIR and
PG_DYNSHMEM_DIR.BACKUP_LABEL_FILE and TABLESPACE_MAP can be included though via xlog.h.
So what are you looking for? I see a couple of options:
1) The inclusive refactoring, which you are discarding.
2) A dedicated header, but some of the now-not-hardcoded values will
need to be so. That's the list I am giving above.
3) A copy of the list from basebackup.c to src/bin/pg_rewind/.I would guess that you are looking for 2), but I am not sure if you
imply that 3) would be acceptable or not.
Yeah, neither 2 or 3 really appeals to me. Option 1 does touch a number
of places but in a pretty straight-forward way- and if there's a typo
there, the compiler is likely to complain, so it seems like the risk is
relatively low.
Thanks!
Stephen
Stephen Frost <sfrost@snowman.net> writes:
I don't completely buy off on the argument that having these #define's
would make it easier for forks (we've had quite a few folks fork PG, but
how many of them have actually changed "base"?) and I'm on the fence
about if these will make our lives simpler down the road when it comes
to changing the directory names
I am distressed that nobody, apparently, is putting any weight on the
back-patching pain that will result from widespread replacement of path
names with macros. I don't buy that either we or anyone else will need
to change these names in future, so I see pain and effectively no gain.
Furthermore, I think it's completely silly to claim that this sort of
thing is a gain in readability or understandability:
- path = psprintf("base/%u/t%d_%u",
- dbNode, backendId, relNode);
+ path = psprintf("%s/%u/t%d_%u",
+ PG_BASE_DIR, dbNode, backendId, relNode);
For my money it's a loss on both points. The extra level of indirection
is just obscuring what's actually happening and putting extra cognitive
load on the reader.
We have better things to spend our time on.
regards, tom lane
On Sat, Mar 24, 2018 at 11:14:34PM -0400, Tom Lane wrote:
Stephen Frost <sfrost@snowman.net> writes:
I don't completely buy off on the argument that having these #define's
would make it easier for forks (we've had quite a few folks fork PG, but
how many of them have actually changed "base"?) and I'm on the fence
about if these will make our lives simpler down the road when it comes
to changing the directory namesI am distressed that nobody, apparently, is putting any weight on the
back-patching pain that will result from widespread replacement of path
names with macros. I don't buy that either we or anyone else will need
to change these names in future, so I see pain and effectively no
gain.
That's actually something I worry about as well (as the author!), which
is why I qualify the changes as intrusive. At the end, I think that I
would be tempted to just do #3, aka to keep a copy of the filter list in
pg_rewind code while hardcoding a minimum of names and mention in both
basebackup.c and pg_rewind code to not forget to update the filter list
if necessary. New paths in the data folder are not added on a monthly
basis either, and not all of them can be filtered out so that's easy to
maintain.
--
Michael
Greetings,
* Michael Paquier (michael@paquier.xyz) wrote:
On Sat, Mar 24, 2018 at 11:14:34PM -0400, Tom Lane wrote:
Stephen Frost <sfrost@snowman.net> writes:
I don't completely buy off on the argument that having these #define's
would make it easier for forks (we've had quite a few folks fork PG, but
how many of them have actually changed "base"?) and I'm on the fence
about if these will make our lives simpler down the road when it comes
to changing the directory namesI am distressed that nobody, apparently, is putting any weight on the
back-patching pain that will result from widespread replacement of path
names with macros. I don't buy that either we or anyone else will need
to change these names in future, so I see pain and effectively no
gain.
While I agree that some consideration should be given to the impact a
change has on back-patching, I continue to be of the opinion that this
would be a good change, for the reasons which I outlined up-thread.
That said, I don't hold that position very strongly.
That's actually something I worry about as well (as the author!), which
is why I qualify the changes as intrusive. At the end, I think that I
would be tempted to just do #3, aka to keep a copy of the filter list in
pg_rewind code while hardcoding a minimum of names and mention in both
basebackup.c and pg_rewind code to not forget to update the filter list
if necessary. New paths in the data folder are not added on a monthly
basis either, and not all of them can be filtered out so that's easy to
maintain.
Intrusive, at least from my viewpoint, is more about how the code is
changed around and/or refactored than about the impact it has on
back-patching. These are pretty mechanical changes, after all.
I'm not particularly thrilled with the idea of having two independent
lists of hard-coded paths to maintain, even with comments in both places
saying to update the other list.
Thanks!
Stephen
On Sun, Mar 25, 2018 at 5:06 PM, Michael Paquier <michael@paquier.xyz> wrote:
On Sat, Mar 24, 2018 at 11:14:34PM -0400, Tom Lane wrote:
Stephen Frost <sfrost@snowman.net> writes:
I don't completely buy off on the argument that having these #define's
would make it easier for forks (we've had quite a few folks fork PG, but
how many of them have actually changed "base"?) and I'm on the fence
about if these will make our lives simpler down the road when it comes
to changing the directory namesI am distressed that nobody, apparently, is putting any weight on the
back-patching pain that will result from widespread replacement of path
names with macros. I don't buy that either we or anyone else will need
to change these names in future, so I see pain and effectively no
gain.That's actually something I worry about as well (as the author!), which
is why I qualify the changes as intrusive. At the end, I think that I
would be tempted to just do #3, aka to keep a copy of the filter list in
pg_rewind code while hardcoding a minimum of names and mention in both
basebackup.c and pg_rewind code to not forget to update the filter list
if necessary.
+1. It's better for us to focus on the code change of the fillter on pg_rewind
rather than such "refactoring". As I told upthread, the previous patch has the
problem where the files which should be skipped are not skipped. ISTM that,
in pg_rewind, the fillter should be triggered in recurse_dir() not
process_source_file().
BTW what should pg_rewind do when it finds the directory which should be
skipped, in the source directory? In your patch, pg_rewind just tries to skip
that directory at all. But isn't this right behavior? If that directory doesn't
exist in the target directory(though I'm not sure if this situation is really
possible), I'm thinking that pg_rewind should create that "empty" directory
in the target. No?
Regards,
--
Fujii Masao
On Tue, Mar 27, 2018 at 01:32:41AM +0900, Fujii Masao wrote:
+1. It's better for us to focus on the code change of the fillter on pg_rewind
rather than such "refactoring".
(filter takes one 'l', not two)
Okay. I had my mind mostly focused on how to shape the exclusion list
and get it shared between the base backup and pg_rewind, so let's move
on with the duplicated list for now. I did not put much efforts into
the pg_rewind portion to be honest.
As I told upthread, the previous patch has the
problem where the files which should be skipped are not skipped. ISTM that,
in pg_rewind, the filter should be triggered in recurse_dir() not
process_source_file().
If you put that into recurse_dir you completely ignore the case where
changes are fetched by libpq. Doing the filtering when processing the
file map has the advantage to take care of both the local and remote
cases, which is why I am doing it there. So you would just get half of
the cake and not the whole of it.
BTW what should pg_rewind do when it finds the directory which should be
skipped, in the source directory? In your patch, pg_rewind just tries to skip
that directory at all. But isn't this right behavior? If that directory doesn't
exist in the target directory (though I'm not sure if this situation is really
possible), I'm thinking that pg_rewind should create that "empty" directory
in the target. No?
I am not exactly sure what you are coming up with here. The target
server should have the same basic directory mapping as the source as the
target has been initialized normally with initdb or a base backup from
another node, so checking for the *contents* of directories is enough
and keeps the code more simple, as the exclude filter entries are based
on elements inherent to PostgreSQL internals. Please note as well that
if a non-system directory is present on the source but not the target
then it would get created on the target.
At the end I have finished with the attached. I have taken the decision
to not include as well xlog.h in pg_rewind to avoid having to drag a lot
of backend-only headers like pg_resetwal does, which I prefer avoid as
that's only hardcoding values for "backup_label" and "tablespace_map".
This applies filters based on directory contents, so by running the
regression tests you can see entries like the following ones:
entry "postmaster.opts" excluded from source file list
entry "pg_subtrans/0000" excluded from source file list
entry "pg_notify/0000" excluded from source file list
entry "base/12360/pg_internal.init" excluded from source file list
entry "backup_label.old" excluded from source file list
entry "global/pg_internal.init" excluded from source file list
entry "postmaster.opts" excluded from target file list
entry "pg_subtrans/0000" excluded from target file list
entry "pg_notify/0000" excluded from target file list
entry "base/12360/pg_internal.init" excluded from target file list
entry "global/pg_internal.init" excluded from target file list
Processing the filemap list on the target also matters in my opinion.
When at recovery, all the previous files will be wiped out, and we
should not remove either things like postmaster.pid as those are around
to prevent corruption problems.
Thanks,
--
Michael
Attachments:
0001-Add-exclude-list-similar-to-base-backups-in-pg_rewin.patchtext/x-diff; charset=us-asciiDownload
From ba4502c9711c7b7933a5ef2186e895c0ae6b4616 Mon Sep 17 00:00:00 2001
From: Michael Paquier <michael@paquier.xyz>
Date: Mon, 5 Feb 2018 15:48:35 +0900
Subject: [PATCH] Add exclude list similar to base backups in pg_rewind
After being rewound, a standby to-be-recycled needs to perform recovery
from the last checkpoint where WAL forked after a promotion, which leads
it to automatically remove some files which may have been copied from
the source cluster. This makes use of the same filtering list as base
backups to find out what is this data and then remove it. This reduces
the amount of data transferred during a rewind without changing the
usefulness of the operation.
Documentation is updated to take into account what is filtered out.
---
doc/src/sgml/ref/pg_rewind.sgml | 14 +++-
src/backend/replication/basebackup.c | 3 +
src/bin/pg_rewind/filemap.c | 144 ++++++++++++++++++++++++++++++++---
3 files changed, 148 insertions(+), 13 deletions(-)
diff --git a/doc/src/sgml/ref/pg_rewind.sgml b/doc/src/sgml/ref/pg_rewind.sgml
index 8e49249826..520d843f0e 100644
--- a/doc/src/sgml/ref/pg_rewind.sgml
+++ b/doc/src/sgml/ref/pg_rewind.sgml
@@ -231,7 +231,19 @@ PostgreSQL documentation
<para>
Copy all other files such as <filename>pg_xact</filename> and
configuration files from the source cluster to the target cluster
- (everything except the relation files).
+ (everything except the relation files). Similarly to base backups,
+ the contents of the directories <filename>pg_dynshmem/</filename>,
+ <filename>pg_notify/</filename>, <filename>pg_replslot/</filename>,
+ <filename>pg_serial/</filename>, <filename>pg_snapshots/</filename>,
+ <filename>pg_stat_tmp/</filename>, and
+ <filename>pg_subtrans/</filename> are omitted from the data copied
+ from the source cluster. Any file or directory beginning with
+ <filename>pgsql_tmp</filename> is omitted, as well as are
+ <filename>backup_label</filename>,
+ <filename>tablespace_map</filename>,
+ <filename>pg_internal.init</filename>,
+ <filename>postmaster.opts</filename> and
+ <filename>postmaster.pid</filename>.
</para>
</step>
<step>
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index e4c45c5025..902a553ffa 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -103,6 +103,9 @@ static TimestampTz throttled_last;
* The contents of these directories are removed or recreated during server
* start so they are not included in backups. The directories themselves are
* kept and included as empty to preserve access permissions.
+ *
+ * Note: this list should be kept in sync with the filter lists in pg_rewind's
+ * filemap.c.
*/
static const char *excludeDirContents[] =
{
diff --git a/src/bin/pg_rewind/filemap.c b/src/bin/pg_rewind/filemap.c
index 6122f177fe..67b01cbdfe 100644
--- a/src/bin/pg_rewind/filemap.c
+++ b/src/bin/pg_rewind/filemap.c
@@ -31,6 +31,83 @@ static char *datasegpath(RelFileNode rnode, ForkNumber forknum,
static int path_cmp(const void *a, const void *b);
static int final_filemap_cmp(const void *a, const void *b);
static void filemap_list_to_array(filemap_t *map);
+static bool is_path_excluded(const char *path, const char *type);
+
+/*
+ * The contents of these directories are removed or recreated during server
+ * start so they are not included in data processed by pg_rewind.
+ *
+ * Note: those lists should be kept in sync with what basebackup.c provides.
+ * Some of the values, contrary to what basebackup.c uses, are hardcoded as
+ * they are defined in backend-only headers. So this list is maintained
+ * with a best effort in mind.
+ */
+static const char *excludeDirContents[] =
+{
+ /*
+ * Skip temporary statistics files. PG_STAT_TMP_DIR must be skipped even
+ * when stats_temp_directory is set because PGSS_TEXT_FILE is always
+ * created there.
+ */
+ "pg_stat_tmp", /* defined as PG_STAT_TMP_DIR */
+
+ /*
+ * It is generally not useful to backup the contents of this directory
+ * even if the intention is to restore to another master. See backup.sgml
+ * for a more detailed description.
+ */
+ "pg_replslot",
+
+ /* Contents removed on startup, see dsm_cleanup_for_mmap(). */
+ "pg_dynshmem", /* defined as PG_DYNSHMEM_DIR */
+
+ /* Contents removed on startup, see AsyncShmemInit(). */
+ "pg_notify",
+
+ /*
+ * Old contents are loaded for possible debugging but are not required for
+ * normal operation, see OldSerXidInit().
+ */
+ "pg_serial",
+
+ /* Contents removed on startup, see DeleteAllExportedSnapshotFiles(). */
+ "pg_snapshots",
+
+ /* Contents zeroed on startup, see StartupSUBTRANS(). */
+ "pg_subtrans",
+
+ /* end of list */
+ NULL
+};
+
+/*
+ * List of files excluded from filemap processing.
+ */
+static const char *excludeFiles[] =
+{
+ /* Skip auto conf temporary file. */
+ "postgresql.auto.conf.tmp", /* defined as PG_AUTOCONF_FILENAME */
+
+ /* Skip current log file temporary file */
+ "current_logfiles.tmp", /* defined as LOG_METAINFO_DATAFILE_TMP */
+
+ /* Skip relation cache because it is rebuilt on startup */
+ "pg_internal.init", /* defined as RELCACHE_INIT_FILENAME */
+
+ /*
+ * If there's a backup_label or tablespace_map file, it belongs to a
+ * backup started by the user with pg_start_backup(). It is *not* correct
+ * for this backup. Our backup_label is written later on separately.
+ */
+ "backup_label", /* defined as BACKUP_LABEL_FILE */
+ "tablespace_map", /* defined as TABLESPACE_MAP */
+
+ "postmaster.pid",
+ "postmaster.opts",
+
+ /* end of list */
+ NULL
+};
/*
* Create a new file map (stored in the global pointer "filemap").
@@ -71,11 +148,8 @@ process_source_file(const char *path, file_type_t type, size_t newsize,
Assert(map->array == NULL);
- /*
- * Completely ignore some special files in source and destination.
- */
- if (strcmp(path, "postmaster.pid") == 0 ||
- strcmp(path, "postmaster.opts") == 0)
+ /* ignore any path matching the exclusion filters */
+ if (is_path_excluded(path, "source"))
return;
/*
@@ -260,6 +334,14 @@ process_target_file(const char *path, file_type_t type, size_t oldsize,
filemap_t *map = filemap;
file_entry_t *entry;
+ /*
+ * Ignore any path matching the exclusion filters. This is not actually
+ * mandatory for target files, but this does not hurt and let's be
+ * consistent with the source processing.
+ */
+ if (is_path_excluded(path, "target"))
+ return;
+
snprintf(localpath, sizeof(localpath), "%s/%s", datadir_target, path);
if (lstat(localpath, &statbuf) < 0)
{
@@ -286,13 +368,6 @@ process_target_file(const char *path, file_type_t type, size_t oldsize,
qsort(map->array, map->narray, sizeof(file_entry_t *), path_cmp);
}
- /*
- * Completely ignore some special files
- */
- if (strcmp(path, "postmaster.pid") == 0 ||
- strcmp(path, "postmaster.opts") == 0)
- return;
-
/*
* Like in process_source_file, pretend that xlog is always a directory.
*/
@@ -412,6 +487,51 @@ process_block_change(ForkNumber forknum, RelFileNode rnode, BlockNumber blkno)
}
}
+/*
+ * Completely ignore some special files in source and destination. This
+ * filters willingly any files matching an entry in the list of files to
+ * filter out. The input path is a relative path from a data directory,
+ * so comparisons need to compromise with that and do not check for
+ * slashes before the compared names. This does not matter in practice
+ * as files listed in the exclusion list do not have names mapping with
+ * existing PostgreSQL internal paths.
+ */
+static bool
+is_path_excluded(const char *path, const char *type)
+{
+ char localpath[MAXPGPATH];
+ int excludeIdx;
+
+ /* check individual files... */
+ for (excludeIdx = 0; excludeFiles[excludeIdx] != NULL; excludeIdx++)
+ {
+ if (strstr(path, excludeFiles[excludeIdx]) != NULL)
+ {
+ pg_log(PG_DEBUG, "entry \"%s\" excluded from %s file list\n",
+ path, type);
+ return true;
+ }
+ }
+
+ /*
+ * ... And check some directories. Note that this includes any contents
+ * within the directories themselves.
+ */
+ for (excludeIdx = 0; excludeDirContents[excludeIdx] != NULL; excludeIdx++)
+ {
+ snprintf(localpath, sizeof(localpath), "%s/",
+ excludeDirContents[excludeIdx]);
+ if (strstr(path, localpath) != NULL)
+ {
+ pg_log(PG_DEBUG, "entry \"%s\" excluded from %s file list\n",
+ path, type);
+ return true;
+ }
+ }
+
+ return false;
+}
+
/*
* Convert the linked list of entries in map->first/last to the array,
* map->array.
--
2.16.3
On Sat, Mar 24, 2018 at 09:12:09PM -0400, Stephen Frost wrote:
Yeah, neither 2 or 3 really appeals to me. Option 1 does touch a number
of places but in a pretty straight-forward way- and if there's a typo
there, the compiler is likely to complain, so it seems like the risk is
relatively low.
One example of place which can be easily consolidated is pg_wal whose
definition is in xlog_internal.h. And there are a couple of other
places which could be consolidated without major refactoring like what I
proposed initially on this thread. I would suggest to focus on this
effort on a separate thread later on.
--
Michael
On Tue, Mar 27, 2018 at 10:55 AM, Michael Paquier <michael@paquier.xyz> wrote:
On Tue, Mar 27, 2018 at 01:32:41AM +0900, Fujii Masao wrote:
+1. It's better for us to focus on the code change of the fillter on pg_rewind
rather than such "refactoring".(filter takes one 'l', not two)
Okay. I had my mind mostly focused on how to shape the exclusion list
and get it shared between the base backup and pg_rewind, so let's move
on with the duplicated list for now. I did not put much efforts into
the pg_rewind portion to be honest.As I told upthread, the previous patch has the
problem where the files which should be skipped are not skipped. ISTM that,
in pg_rewind, the filter should be triggered in recurse_dir() not
process_source_file().If you put that into recurse_dir you completely ignore the case where
changes are fetched by libpq. Doing the filtering when processing the
file map has the advantage to take care of both the local and remote
cases, which is why I am doing it there.
OK.
BTW what should pg_rewind do when it finds the directory which should be
skipped, in the source directory? In your patch, pg_rewind just tries to skip
that directory at all. But isn't this right behavior? If that directory doesn't
exist in the target directory (though I'm not sure if this situation is really
possible), I'm thinking that pg_rewind should create that "empty" directory
in the target. No?I am not exactly sure what you are coming up with here. The target
server should have the same basic directory mapping as the source as the
target has been initialized normally with initdb or a base backup from
another node, so checking for the *contents* of directories is enough
and keeps the code more simple, as the exclude filter entries are based
on elements inherent to PostgreSQL internals. Please note as well that
if a non-system directory is present on the source but not the target
then it would get created on the target.At the end I have finished with the attached.
Thanks for the patch!
+ snprintf(localpath, sizeof(localpath), "%s/",
+ excludeDirContents[excludeIdx]);
+ if (strstr(path, localpath) != NULL)
This code is almost ok in practice, but using the check of
"strstr(path, localpath) == path" is more robust here?
+ for (excludeIdx = 0; excludeFiles[excludeIdx] != NULL; excludeIdx++)
+ {
+ if (strstr(path, excludeFiles[excludeIdx]) != NULL)
Using the following code instead is more robust?
This original code is almost ok in practice, though.
filename = last_dir_separator(path);
if (filename == NULL)
filename = path;
else
filename++;
if (strcmp(filename, excludeFiles[excludeIdx]) == 0)
+ (everything except the relation files). Similarly to base backups,
+ the contents of the directories <filename>pg_dynshmem/</filename>,
+ <filename>pg_notify/</filename>, <filename>pg_replslot/</filename>,
+ <filename>pg_serial/</filename>, <filename>pg_snapshots/</filename>,
+ <filename>pg_stat_tmp/</filename>, and
+ <filename>pg_subtrans/</filename> are omitted from the data copied
+ from the source cluster. Any file or directory beginning with
+ <filename>pgsql_tmp</filename> is omitted, as well as are
+ <filename>backup_label</filename>,
+ <filename>tablespace_map</filename>,
+ <filename>pg_internal.init</filename>,
+ <filename>postmaster.opts</filename> and
+ <filename>postmaster.pid</filename>.
I don't think this description is necessary. The doc for pg_basebackup
also doesn't seem to have this kind of description.
So attached is the patch that I updated based on your patch and
am thinking to apply.
Regards,
--
Fujii Masao
Attachments:
add_exclude_list_similar_to_base_backup_v2_fujii.patchapplication/octet-stream; name=add_exclude_list_similar_to_base_backup_v2_fujii.patchDownload
*** a/src/backend/replication/basebackup.c
--- b/src/backend/replication/basebackup.c
***************
*** 103,108 **** static TimestampTz throttled_last;
--- 103,111 ----
* The contents of these directories are removed or recreated during server
* start so they are not included in backups. The directories themselves are
* kept and included as empty to preserve access permissions.
+ *
+ * Note: this list should be kept in sync with the filter lists in pg_rewind's
+ * filemap.c.
*/
static const char *excludeDirContents[] =
{
*** a/src/bin/pg_rewind/filemap.c
--- b/src/bin/pg_rewind/filemap.c
***************
*** 31,36 **** static char *datasegpath(RelFileNode rnode, ForkNumber forknum,
--- 31,113 ----
static int path_cmp(const void *a, const void *b);
static int final_filemap_cmp(const void *a, const void *b);
static void filemap_list_to_array(filemap_t *map);
+ static bool check_file_excluded(const char *path, const char *type);
+
+ /*
+ * The contents of these directories are removed or recreated during server
+ * start so they are not included in data processed by pg_rewind.
+ *
+ * Note: those lists should be kept in sync with what basebackup.c provides.
+ * Some of the values, contrary to what basebackup.c uses, are hardcoded as
+ * they are defined in backend-only headers. So this list is maintained
+ * with a best effort in mind.
+ */
+ static const char *excludeDirContents[] =
+ {
+ /*
+ * Skip temporary statistics files. PG_STAT_TMP_DIR must be skipped even
+ * when stats_temp_directory is set because PGSS_TEXT_FILE is always
+ * created there.
+ */
+ "pg_stat_tmp", /* defined as PG_STAT_TMP_DIR */
+
+ /*
+ * It is generally not useful to backup the contents of this directory
+ * even if the intention is to restore to another master. See backup.sgml
+ * for a more detailed description.
+ */
+ "pg_replslot",
+
+ /* Contents removed on startup, see dsm_cleanup_for_mmap(). */
+ "pg_dynshmem", /* defined as PG_DYNSHMEM_DIR */
+
+ /* Contents removed on startup, see AsyncShmemInit(). */
+ "pg_notify",
+
+ /*
+ * Old contents are loaded for possible debugging but are not required for
+ * normal operation, see OldSerXidInit().
+ */
+ "pg_serial",
+
+ /* Contents removed on startup, see DeleteAllExportedSnapshotFiles(). */
+ "pg_snapshots",
+
+ /* Contents zeroed on startup, see StartupSUBTRANS(). */
+ "pg_subtrans",
+
+ /* end of list */
+ NULL
+ };
+
+ /*
+ * List of files excluded from filemap processing.
+ */
+ static const char *excludeFiles[] =
+ {
+ /* Skip auto conf temporary file. */
+ "postgresql.auto.conf.tmp", /* defined as PG_AUTOCONF_FILENAME */
+
+ /* Skip current log file temporary file */
+ "current_logfiles.tmp", /* defined as LOG_METAINFO_DATAFILE_TMP */
+
+ /* Skip relation cache because it is rebuilt on startup */
+ "pg_internal.init", /* defined as RELCACHE_INIT_FILENAME */
+
+ /*
+ * If there's a backup_label or tablespace_map file, it belongs to a
+ * backup started by the user with pg_start_backup(). It is *not* correct
+ * for this backup. Our backup_label is written later on separately.
+ */
+ "backup_label", /* defined as BACKUP_LABEL_FILE */
+ "tablespace_map", /* defined as TABLESPACE_MAP */
+
+ "postmaster.pid",
+ "postmaster.opts",
+
+ /* end of list */
+ NULL
+ };
/*
* Create a new file map (stored in the global pointer "filemap").
***************
*** 71,81 **** process_source_file(const char *path, file_type_t type, size_t newsize,
Assert(map->array == NULL);
! /*
! * Completely ignore some special files in source and destination.
! */
! if (strcmp(path, "postmaster.pid") == 0 ||
! strcmp(path, "postmaster.opts") == 0)
return;
/*
--- 148,155 ----
Assert(map->array == NULL);
! /* ignore any path matching the exclusion filters */
! if (check_file_excluded(path, "source"))
return;
/*
***************
*** 260,265 **** process_target_file(const char *path, file_type_t type, size_t oldsize,
--- 334,347 ----
filemap_t *map = filemap;
file_entry_t *entry;
+ /*
+ * Ignore any path matching the exclusion filters. This is not actually
+ * mandatory for target files, but this does not hurt and let's be
+ * consistent with the source processing.
+ */
+ if (check_file_excluded(path, "target"))
+ return;
+
snprintf(localpath, sizeof(localpath), "%s/%s", datadir_target, path);
if (lstat(localpath, &statbuf) < 0)
{
***************
*** 287,299 **** process_target_file(const char *path, file_type_t type, size_t oldsize,
}
/*
- * Completely ignore some special files
- */
- if (strcmp(path, "postmaster.pid") == 0 ||
- strcmp(path, "postmaster.opts") == 0)
- return;
-
- /*
* Like in process_source_file, pretend that xlog is always a directory.
*/
if (strcmp(path, "pg_wal") == 0 && type == FILE_TYPE_SYMLINK)
--- 369,374 ----
***************
*** 413,418 **** process_block_change(ForkNumber forknum, RelFileNode rnode, BlockNumber blkno)
--- 488,538 ----
}
/*
+ * Is this the path of file that pg_rewind can skip copying?
+ */
+ static bool
+ check_file_excluded(const char *path, const char *type)
+ {
+ char localpath[MAXPGPATH];
+ int excludeIdx;
+ const char *filename;
+
+ /* check individual files... */
+ for (excludeIdx = 0; excludeFiles[excludeIdx] != NULL; excludeIdx++)
+ {
+ filename = last_dir_separator(path);
+ if (filename == NULL)
+ filename = path;
+ else
+ filename++;
+ if (strcmp(filename, excludeFiles[excludeIdx]) == 0)
+ {
+ pg_log(PG_DEBUG, "entry \"%s\" excluded from %s file list\n",
+ path, type);
+ return true;
+ }
+ }
+
+ /*
+ * ... And check some directories. Note that this includes any contents
+ * within the directories themselves.
+ */
+ for (excludeIdx = 0; excludeDirContents[excludeIdx] != NULL; excludeIdx++)
+ {
+ snprintf(localpath, sizeof(localpath), "%s/",
+ excludeDirContents[excludeIdx]);
+ if (strstr(path, localpath) == path)
+ {
+ pg_log(PG_DEBUG, "entry \"%s\" excluded from %s file list\n",
+ path, type);
+ return true;
+ }
+ }
+
+ return false;
+ }
+
+ /*
* Convert the linked list of entries in map->first/last to the array,
* map->array.
*/
On Wed, Mar 28, 2018 at 04:13:25AM +0900, Fujii Masao wrote:
This code is almost ok in practice, but using the check of
"strstr(path, localpath) == path" is more robust here?
No problems with that either.
Using the following code instead is more robust?
This original code is almost ok in practice, though.filename = last_dir_separator(path);
if (filename == NULL)
filename = path;
else
filename++;
if (strcmp(filename, excludeFiles[excludeIdx]) == 0)
Indeed, using last_dir_separator is a better idea for files. I was
looking for something like that actually.
+ (everything except the relation files). Similarly to base backups, + the contents of the directories <filename>pg_dynshmem/</filename>, + <filename>pg_notify/</filename>, <filename>pg_replslot/</filename>, + <filename>pg_serial/</filename>, <filename>pg_snapshots/</filename>, + <filename>pg_stat_tmp/</filename>, and + <filename>pg_subtrans/</filename> are omitted from the data copied + from the source cluster. Any file or directory beginning with + <filename>pgsql_tmp</filename> is omitted, as well as are + <filename>backup_label</filename>, + <filename>tablespace_map</filename>, + <filename>pg_internal.init</filename>, + <filename>postmaster.opts</filename> and + <filename>postmaster.pid</filename>.I don't think this description is necessary. The doc for pg_basebackup
also doesn't seem to have this kind of description.
Those are listed in backup.sgml. And I really think that we should at
least document that the same type of exclusion filters as base backups
are used in pg_rewind. If you don't want to include the whole list,
then let's use a reference to backup-lowlevel-base-backup-data.
So attached is the patch that I updated based on your patch and
am thinking to apply.
Thanks. Except for the documentation part, I am OK for the changes
proposed.
--
Michael
On Wed, Mar 28, 2018 at 7:54 AM, Michael Paquier <michael@paquier.xyz> wrote:
On Wed, Mar 28, 2018 at 04:13:25AM +0900, Fujii Masao wrote:
This code is almost ok in practice, but using the check of
"strstr(path, localpath) == path" is more robust here?No problems with that either.
Using the following code instead is more robust?
This original code is almost ok in practice, though.filename = last_dir_separator(path);
if (filename == NULL)
filename = path;
else
filename++;
if (strcmp(filename, excludeFiles[excludeIdx]) == 0)Indeed, using last_dir_separator is a better idea for files. I was
looking for something like that actually.+ (everything except the relation files). Similarly to base backups, + the contents of the directories <filename>pg_dynshmem/</filename>, + <filename>pg_notify/</filename>, <filename>pg_replslot/</filename>, + <filename>pg_serial/</filename>, <filename>pg_snapshots/</filename>, + <filename>pg_stat_tmp/</filename>, and + <filename>pg_subtrans/</filename> are omitted from the data copied + from the source cluster. Any file or directory beginning with + <filename>pgsql_tmp</filename> is omitted, as well as are + <filename>backup_label</filename>, + <filename>tablespace_map</filename>, + <filename>pg_internal.init</filename>, + <filename>postmaster.opts</filename> and + <filename>postmaster.pid</filename>.I don't think this description is necessary. The doc for pg_basebackup
also doesn't seem to have this kind of description.Those are listed in backup.sgml. And I really think that we should at
least document that the same type of exclusion filters as base backups
are used in pg_rewind.
Okay, I revived that change in the doc.
So attached is the patch that I updated based on your patch and
am thinking to apply.Thanks. Except for the documentation part, I am OK for the changes
proposed.
Committed. Thanks!
Regards,
--
Fujii Masao