[Patch] ALTER SYSTEM READ ONLY

From b7bc7cb9c417cdb7a1607237642ac2f61fe4ba74 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Tue, 2 Jun 2020 00:45:20 -0400
Subject: [PATCH v2 6/6] Documentation - WIP

TODOs:

1] TBC, the section for pg_is_in_readonly() function, right now it is under "Recovery Information Functions"
2] Documentation regarding ALTER SYSTEM READ/WRITE
---
 src/backend/access/transam/README | 59 ++++++++++++++++++++++++++++---
 src/backend/storage/page/README   | 13 +++----
 2 files changed, 61 insertions(+), 11 deletions(-)

diff --git a/src/backend/access/transam/README b/src/backend/access/transam/README
index eb9aac5fd39..f62929f1660 100644
--- a/src/backend/access/transam/README
+++ b/src/backend/access/transam/README
@@ -433,8 +433,8 @@ to be modified.
 2. START_CRIT_SECTION()  (Any error during the next three steps must cause a
 PANIC because the shared buffers will contain unlogged changes, which we
 have to ensure don't get to disk.  Obviously, you should check conditions
-such as whether there's enough free space on the page before you start the
-critical section.)
+such as whether there's WAL write permission and enough free space on the page
+before you start the critical section.)
 
 3. Apply the required changes to the shared buffer(s).
 
@@ -477,6 +477,54 @@ with the incomplete-split flag set, it will finish the interrupted split by
 inserting the key to the parent, before proceeding.
 
 
+Read only system state
+----------------------
+
+The system state when it is not currently possible to insert write-ahead log
+records, either because the system is still in recovery or because the system
+forced to WAL prohibited by executing ALTER SYSTEM READ ONLY.  We have a
+lower-level defense in XLogBeginInsert() and elsewhere to stop us from modifying
+data during recovery when !XLogInsertAllowed(), but if XLogBeginInsert() is
+inside the critical section we must not depend on it to report an error.
+Otherwise, it will cause PANIC as mentioned previously.
+
+We do not reach the point where we try to write WAL during recovery but ALTER
+SYSTEM READ ONLY can be executed anytime by the user to stop WAL writing.  Any
+backends which receive read-only system state transition barrier interrupt need
+to stop WAL writing immediately.  For barrier absorption the backed(s) will kill
+the running transaction which has valid XID indicates that the transaction has
+performed and/or planning WAL write.  The transaction which doesn't acquire
+valid XID yet or operation such VACUUM or CONCURRENT CREATE INDEX which not
+necessary have valid XID for WAL will not be prevented while barrier processing,
+and those might hit the error from XLogBeginInsert() while trying to write WAL
+in read only system state.  To prevent such error from XLogBeginInsert() inside
+the critical section the WAL write permission has to check before
+START_CRIT_SECTION().
+
+To enforce the practice to check WAL permission before entering into critical
+section for the WAL write, we have added an assert check flag that indicates
+permission has been checked before calling XLogBeginInsert().  If not,
+XLogBeginInsert() will have assertion failure.  WAL permission check is not
+mandatory if the XLogBeginInsert() is not inside the critical section where
+throwing the error is acceptable.  To get permission check flag set either
+CheckWALPermitted(), AssertWALPermitted_HaveXID(), or AssertWALPermitted()
+should be called before START_CRIT_SECTION().  This flag automatically resets
+while exiting from the critical section.  The rule to place either of permission
+check routines will be:
+
+	The places where WAL write operation in critical can be expected without
+	having valid XID (e.g vacuum) need to protect by CheckWALPermitted(), so
+	that error can be reported outside before critical section.
+
+	The places where INSERT and UPDATE are expected which are never happened
+	without valid XID can be checked using AssertWALPermitted_HaveXID.  So that
+	non-assert build will not have the checking overhead.
+
+	The places we know that we cannot be reached in the read-only state and may
+	or may not have XID, but need to ensure the permission has been checked on
+	assert enabled build should use AssertWALPermitted().
+
+
 Constructing a WAL record
 -------------------------
 
@@ -522,7 +570,8 @@ Details of the API functions:
 
 void XLogBeginInsert(void)
 
-    Must be called before XLogRegisterBuffer and XLogRegisterData.
+	Must be called before XLogRegisterBuffer and XLogRegisterData.  WAL
+	permission must be check before calling it in a critical section.
 
 void XLogResetInsertion(void)
 
@@ -630,8 +679,8 @@ If the buffer is clean and checksums are in use then
 MarkBufferDirtyHint() inserts an XLOG_FPI record to ensure that we
 take a full page image that includes the hint. We do this to avoid
 a partial page write, when we write the dirtied page. WAL is not
-written during recovery, so we simply skip dirtying blocks because
-of hints when in recovery.
+written while in read only (i.e. during recovery or in WAL prohibit state), so
+we simply skip dirtying blocks because of hints when in read only.
 
 If you do decide to optimise away a WAL record, then any calls to
 MarkBufferDirty() must be replaced by MarkBufferDirtyHint(),
diff --git a/src/backend/storage/page/README b/src/backend/storage/page/README
index 4e45bd92abc..e5a32e53649 100644
--- a/src/backend/storage/page/README
+++ b/src/backend/storage/page/README
@@ -56,9 +56,10 @@ WAL is a fatal error and prevents further recovery, whereas a checksum failure
 on a normal data block is a hard error but not a critical one for the server,
 even if it is a very bad thing for the user.
 
-New WAL records cannot be written during recovery, so hint bits set during
-recovery must not dirty the page if the buffer is not already dirty, when
-checksums are enabled.  Systems in Hot-Standby mode may benefit from hint bits
-being set, but with checksums enabled, a page cannot be dirtied after setting a
-hint bit (due to the torn page risk). So, it must wait for full-page images
-containing the hint bit updates to arrive from the master.
+New WAL records cannot be written during recovery or while in WAL prohibit
+state, so hint bits set during read only system state must not dirty the page if
+the buffer is not already dirty, when checksums are enabled.  Systems in
+Hot-Standby mode may benefit from hint bits being set, but with checksums
+enabled, a page cannot be dirtied after setting a hint bit (due to the torn page
+risk). So, it must wait for full-page images containing the hint bit updates to
+arrive from the master.
-- 
2.18.0

From bf257bcfd0a5c404209521249991b63dcc79a712 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 19 Jun 2020 06:29:36 -0400
Subject: [PATCH v2 3/6] Implement ALTER SYSTEM READ ONLY using global barrier.

Implementation:

 1. When a user tried to change server state to WAL-Prohibited using
    ALTER SYSTEM READ ONLY command; AlterSystemSetWALProhibitState() will emit
    PROCSIGNAL_BARRIER_WAL_PROHIBIT_STATE_CHANGE barrier and will wait until the
    barrier has been absorbed by all the backends.

 2. When a backend receives the WAL-Prohibited barrier, at that moment if
    it is already in a transaction and the transaction already assigned XID,
    then the backend will be killed by throwing FATAL(XXX: need more discussion
    on this)

 3. Otherwise, if that backend running transaction which yet to get XID
    assigned we don't need to do anything special, simply call
    ResetLocalXLogInsertAllowed() so that any future WAL insert in will check
    XLogInsertAllowed() first which set ready only state appropriately.

 4. A new transaction (from existing or new backend) starts as a read-only
    transaction.

 5. Auxiliary processes like autovacuum launcher, background writer,
    checkpointer and  walwriter will don't do anything in WAL-Prohibited
    server state until someone wakes us up. E.g. a backend might later on
    request us to put the system back to read-write.

 6. At shutdown in WAL-Prohibited mode, we'll skip shutdown checkpoint
    and xlog rotation. Starting up again will perform crash recovery(XXX:
    need some discussion on this as well)

 7. ALTER SYSTEM READ ONLY/WRITE is restricted on standby server.

 8. Only super user can toggle WAL-Prohibit state.

 9. Add system_is_read_only GUC show the system state -- will true when system
    is wal prohibited or in recovery.
---
 src/backend/access/transam/Makefile      |  1 +
 src/backend/access/transam/walprohibit.c | 81 ++++++++++++++++++++++++
 src/backend/access/transam/xact.c        | 49 ++++++++------
 src/backend/access/transam/xlog.c        | 72 ++++++++++++++++++---
 src/backend/postmaster/autovacuum.c      |  4 ++
 src/backend/postmaster/bgwriter.c        |  2 +-
 src/backend/postmaster/checkpointer.c    | 12 ++++
 src/backend/storage/ipc/procsignal.c     | 26 ++------
 src/backend/tcop/utility.c               | 14 +---
 src/backend/utils/misc/guc.c             | 26 ++++++++
 src/include/access/walprohibit.h         | 21 ++++++
 src/include/access/xlog.h                |  3 +
 src/include/storage/procsignal.h         |  7 +-
 13 files changed, 246 insertions(+), 72 deletions(-)
 create mode 100644 src/backend/access/transam/walprohibit.c
 create mode 100644 src/include/access/walprohibit.h

diff --git a/src/backend/access/transam/Makefile b/src/backend/access/transam/Makefile
index 595e02de722..b5322a69954 100644
--- a/src/backend/access/transam/Makefile
+++ b/src/backend/access/transam/Makefile
@@ -26,6 +26,7 @@ OBJS = \
 	twophase.o \
 	twophase_rmgr.o \
 	varsup.o \
+	walprohibit.o \
 	xact.o \
 	xlog.o \
 	xlogarchive.o \
diff --git a/src/backend/access/transam/walprohibit.c b/src/backend/access/transam/walprohibit.c
new file mode 100644
index 00000000000..df97596ddf9
--- /dev/null
+++ b/src/backend/access/transam/walprohibit.c
@@ -0,0 +1,81 @@
+/*-------------------------------------------------------------------------
+ *
+ * walprohibit.c
+ * 		PostgreSQL write-ahead log prohibit states
+ *
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/backend/access/transam/walprohibit.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/walprohibit.h"
+#include "postmaster/bgwriter.h"
+#include "storage/procsignal.h"
+
+/*
+ * ProcessBarrierWALProhibit()
+ *
+ * Handle WAL prohibit state change request.
+ */
+bool
+ProcessBarrierWALProhibit(void)
+{
+	/*
+	 * Kill off any transactions that have an XID *before* allowing the system
+	 * to go WAL prohibit state.
+	 */
+	if (FullTransactionIdIsValid(GetTopFullTransactionIdIfAny()))
+	{
+		/*
+		 * XXX: Kill off the whole session by throwing FATAL instead of killing
+		 * transaction by throwing ERROR due to following reasons that need be
+		 * thought:
+		 *
+		 * 1. Due to some presents challenges with the wire protocol, we could
+		 * not simply kill of idle transaction.
+		 *
+		 * 2. If we are here in subtransaction then the ERROR will kill the
+		 * current subtransaction only. In the case of invalidations, that
+		 * might be good enough, but for XID assignment it's not, because
+		 * assigning an XID to a subtransaction also causes higher
+		 * sub-transaction levels and the parent transaction to get XIDs.
+		 */
+		ereport(FATAL,
+				(errcode(ERRCODE_ACTIVE_SQL_TRANSACTION),
+				 errmsg("system is now read only"),
+				 errhint("Cannot continue a transaction if it has performed writes while system is read only.")));
+	}
+
+	/* Return to "check" state */
+	ResetLocalXLogInsertAllowed();
+
+	return true;
+}
+
+/*
+ * AlterSystemSetWALProhibitState
+ *
+ * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
+ */
+void
+AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
+{
+	if (!superuser())
+		ereport(ERROR,
+				(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+				 errmsg("must be superuser to execute ALTER SYSTEM command")));
+
+	/* Alter WAL prohibit state not allowed during recovery */
+	PreventCommandDuringRecovery("ALTER SYSTEM");
+
+	/* Yet to add ALTER SYTEM READ WRITE support */
+	if (!stmt->WALProhibited)
+		elog(ERROR, "XXX: Yet to implement");
+
+	MakeReadOnlyXLOG();
+	WaitForProcSignalBarrier(EmitProcSignalBarrier(PROCSIGNAL_BARRIER_WALPROHIBIT));
+}
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 905dc7d8d3b..bf699f19fe8 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -1937,23 +1937,28 @@ StartTransaction(void)
 	Assert(s->prevSecContext == 0);
 
 	/*
-	 * Make sure we've reset xact state variables
+	 * Reset xact state variables.
 	 *
-	 * If recovery is still in progress, mark this transaction as read-only.
-	 * We have lower level defences in XLogInsert and elsewhere to stop us
-	 * from modifying data during recovery, but this gives the normal
-	 * indication to the user that the transaction is read-only.
-	 */
-	if (RecoveryInProgress())
-	{
-		s->startedInRecovery = true;
-		XactReadOnly = true;
-	}
-	else
-	{
-		s->startedInRecovery = false;
-		XactReadOnly = DefaultXactReadOnly;
-	}
+	 * If it is not currently possible to insert write-ahead log records,
+	 * either because we are still in recovery or because ALTER SYSTEM READ
+	 * ONLY has been executed, force this to be a read-only transaction.
+	 * We have lower level defences in XLogBeginInsert() and elsewhere to stop
+	 * us from modifying data during recovery when !XLogInsertAllowed(), but
+	 * this gives the normal indication to the user that the transaction is
+	 * read-only.
+	 *
+	 * On the other hand, we only need to set the startedInRecovery flag when
+	 * the transaction started during recovery, and not when WAL is otherwise
+	 * prohibited. This information is used by RelationGetIndexScan() to
+	 * decide whether to permit (1) relying on existing killed-tuple markings
+	 * and (2) further killing of index tuples. Even when WAL is prohibited
+	 * on the master, it's still the master, so the former is OK; and since
+	 * killing index tuples doesn't generate WAL, the latter is also OK.
+	 * See comments in RelationGetIndexScan() and MarkBufferDirtyHint().
+	 */
+	XactReadOnly = DefaultXactReadOnly || !XLogInsertAllowed();
+	s->startedInRecovery = RecoveryInProgress();
+
 	XactDeferrable = DefaultXactDeferrable;
 	XactIsoLevel = DefaultXactIsoLevel;
 	forceSyncCommit = false;
@@ -4875,9 +4880,11 @@ CommitSubTransaction(void)
 	/*
 	 * We need to restore the upper transaction's read-only state, in case the
 	 * upper is read-write while the child is read-only; GUC will incorrectly
-	 * think it should leave the child state in place.
+	 * think it should leave the child state in place.  Note that the upper
+	 * transaction will be a force to ready-only irrespective of its previous
+	 * status if the server state is WAL prohibited.
 	 */
-	XactReadOnly = s->prevXactReadOnly;
+	XactReadOnly = s->prevXactReadOnly || !XLogInsertAllowed();
 
 	CurrentResourceOwner = s->parent->curTransactionOwner;
 	CurTransactionResourceOwner = s->parent->curTransactionOwner;
@@ -5033,9 +5040,11 @@ AbortSubTransaction(void)
 	/*
 	 * Restore the upper transaction's read-only state, too.  This should be
 	 * redundant with GUC's cleanup but we may as well do it for consistency
-	 * with the commit case.
+	 * with the commit case.  Note that the upper transaction will be a force
+	 * to ready-only irrespective of its previous status if the server state is
+	 * WAL prohibited.
 	 */
-	XactReadOnly = s->prevXactReadOnly;
+	XactReadOnly = s->prevXactReadOnly || !XLogInsertAllowed();
 
 	RESUME_INTERRUPTS();
 }
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index a1256a103b6..ffa25b9d9b2 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -245,9 +245,10 @@ static bool LocalPromoteIsTriggered = false;
  *		0: unconditionally not allowed to insert XLOG
  *		-1: must check RecoveryInProgress(); disallow until it is false
  * Most processes start with -1 and transition to 1 after seeing that recovery
- * is not in progress.  But we can also force the value for special cases.
- * The coding in XLogInsertAllowed() depends on the first two of these states
- * being numerically the same as bool true and false.
+ * is not in progress or the server state is not a WAL prohibited state.  But
+ * we can also force the value for special cases.  The coding in
+ * XLogInsertAllowed() depends on the first two of these states being
+ * numerically the same as bool true and false.
  */
 static int	LocalXLogInsertAllowed = -1;
 
@@ -659,6 +660,12 @@ typedef struct XLogCtlData
 	 */
 	RecoveryState SharedRecoveryState;
 
+	/*
+	 * WALProhibited indicates if we have stopped allowing WAL writes.
+	 * Protected by info_lck.
+	 */
+	bool		WALProhibited;
+
 	/*
 	 * SharedHotStandbyActive indicates if we allow hot standby queries to be
 	 * run.  Protected by info_lck.
@@ -7959,6 +7966,25 @@ StartupXLOG(void)
 		RequestCheckpoint(CHECKPOINT_FORCE);
 }
 
+void
+MakeReadOnlyXLOG(void)
+{
+	SpinLockAcquire(&XLogCtl->info_lck);
+	XLogCtl->WALProhibited = true;
+	SpinLockRelease(&XLogCtl->info_lck);
+}
+
+/*
+ * Is the system still in WAL prohibited state?
+ */
+bool
+IsWALProhibited(void)
+{
+	volatile XLogCtlData *xlogctl = XLogCtl;
+
+	return xlogctl->WALProhibited;
+}
+
 /*
  * Checks if recovery has reached a consistent state. When consistency is
  * reached and we have a valid starting standby snapshot, tell postmaster
@@ -8174,9 +8200,9 @@ HotStandbyActiveInReplay(void)
 /*
  * Is this process allowed to insert new WAL records?
  *
- * Ordinarily this is essentially equivalent to !RecoveryInProgress().
- * But we also have provisions for forcing the result "true" or "false"
- * within specific processes regardless of the global state.
+ * Ordinarily this is essentially equivalent to !RecoveryInProgress() and
+ * !IsWALProhibited().  But we also have provisions for forcing the result
+ * "true" or "false" within specific processes regardless of the global state.
  */
 bool
 XLogInsertAllowed(void)
@@ -8190,14 +8216,25 @@ XLogInsertAllowed(void)
 		return (bool) LocalXLogInsertAllowed;
 
 	/*
-	 * Else, must check to see if we're still in recovery.
+	 * Else, must check to see if we're still in recovery
 	 */
 	if (RecoveryInProgress())
 		return false;
 
+	/* Or, in WAL prohibited state */
+	if (IsWALProhibited())
+	{
+		/*
+		 * Set it to "unconditionally false" to avoid checking until it gets
+		 * reset.
+		 */
+		LocalXLogInsertAllowed = 0;
+		return false;
+	}
+
 	/*
-	 * On exit from recovery, reset to "unconditionally true", since there is
-	 * no need to keep checking.
+	 * On exit from recovery or WAL prohibited state, reset to "unconditionally
+	 * true", since there is no need to keep checking.
 	 */
 	LocalXLogInsertAllowed = 1;
 	return true;
@@ -8213,12 +8250,20 @@ static void
 LocalSetXLogInsertAllowed(void)
 {
 	Assert(LocalXLogInsertAllowed == -1);
+	Assert(!IsWALProhibited());
+
 	LocalXLogInsertAllowed = 1;
 
 	/* Initialize as RecoveryInProgress() would do when switching state */
 	InitXLOGAccess();
 }
 
+void
+ResetLocalXLogInsertAllowed(void)
+{
+	LocalXLogInsertAllowed = -1;
+}
+
 /*
  * Subroutine to try to fetch and validate a prior checkpoint record.
  *
@@ -8509,7 +8554,10 @@ ShutdownXLOG(int code, Datum arg)
 
 	if (RecoveryInProgress())
 		CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
-	else
+	/*
+	 * Can't perform checkpoint or xlog rotation without writing WAL.
+	 */
+	else if (XLogInsertAllowed())
 	{
 		/*
 		 * If archiving is enabled, rotate the last XLOG file so that all the
@@ -8522,6 +8570,10 @@ ShutdownXLOG(int code, Datum arg)
 
 		CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
 	}
+	else
+		ereport(LOG,
+			   (errmsg("skipping shutdown checkpoint because the system is read only")));
+
 	ShutdownCLOG();
 	ShutdownCommitTs();
 	ShutdownSUBTRANS();
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 9c7d4b0c60e..f83f86994db 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -652,6 +652,10 @@ AutoVacLauncherMain(int argc, char *argv[])
 
 		HandleAutoVacLauncherInterrupts();
 
+		/* If the server is read only just go back to sleep. */
+		if (!XLogInsertAllowed())
+			continue;
+
 		/*
 		 * a worker finished, or postmaster signaled failure to start a worker
 		 */
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index 069e27e427f..6c6ff7dc3af 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -268,7 +268,7 @@ BackgroundWriterMain(void)
 		 * Checkpointer, when active, is barely ever in its mainloop and thus
 		 * makes it hard to log regularly.
 		 */
-		if (XLogStandbyInfoActive() && !RecoveryInProgress())
+		if (XLogStandbyInfoActive() && XLogInsertAllowed())
 		{
 			TimestampTz timeout = 0;
 			TimestampTz now = GetCurrentTimestamp();
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b80..5e5e56d4eec 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -342,6 +342,18 @@ CheckpointerMain(void)
 		AbsorbSyncRequests();
 		HandleCheckpointerInterrupts();
 
+		/*
+		 * If the server is in WAL-Prohibited state then don't do anything until
+		 * someone wakes us up. E.g. a backend might later on request us to put
+		 * the system back to read-write.
+		 */
+		if (IsWALProhibited())
+		{
+			(void) WaitLatch(MyLatch, WL_LATCH_SET | WL_EXIT_ON_PM_DEATH, -1,
+							 WAIT_EVENT_CHECKPOINTER_MAIN);
+			continue;
+		}
+
 		/*
 		 * Detect a pending checkpoint request by checking whether the flags
 		 * word in shared memory is nonzero.  We shouldn't need to acquire the
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 13648887187..b973727a580 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -18,6 +18,7 @@
 #include <unistd.h>
 
 #include "access/parallel.h"
+#include "access/walprohibit.h"
 #include "commands/async.h"
 #include "miscadmin.h"
 #include "pgstat.h"
@@ -96,7 +97,6 @@ static volatile ProcSignalSlot *MyProcSignalSlot = NULL;
 
 static bool CheckProcSignal(ProcSignalReason reason);
 static void CleanupProcSignalState(int status, Datum arg);
-static bool ProcessBarrierPlaceholder(void);
 
 /*
  * ProcSignalShmemSize
@@ -510,9 +510,9 @@ ProcessProcSignalBarrier(void)
 			 * unconditionally, but it's more efficient to call only the ones
 			 * that might need us to do something based on the flags.
 			 */
-			if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_PLACEHOLDER)
-				&& ProcessBarrierPlaceholder())
-				BARRIER_CLEAR_BIT(flags, PROCSIGNAL_BARRIER_PLACEHOLDER);
+			if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_WALPROHIBIT)
+				&& ProcessBarrierWALProhibit())
+				BARRIER_CLEAR_BIT(flags, PROCSIGNAL_BARRIER_WALPROHIBIT);
 		}
 		PG_CATCH();
 		{
@@ -554,24 +554,6 @@ ProcessProcSignalBarrier(void)
 	pg_atomic_write_u64(&MyProcSignalSlot->pss_barrierGeneration, shared_gen);
 }
 
-static bool
-ProcessBarrierPlaceholder(void)
-{
-	/*
-	 * XXX. This is just a placeholder until the first real user of this
-	 * machinery gets committed. Rename PROCSIGNAL_BARRIER_PLACEHOLDER to
-	 * PROCSIGNAL_BARRIER_SOMETHING_ELSE where SOMETHING_ELSE is something
-	 * appropriately descriptive. Get rid of this function and instead have
-	 * ProcessBarrierSomethingElse. Most likely, that function should live in
-	 * the file pertaining to that subsystem, rather than here.
-	 *
-	 * The return value should be 'true' if the barrier was successfully
-	 * absorbed and 'false' if not. Note that returning 'false' can lead to
-	 * very frequent retries, so try hard to make that an uncommon case.
-	 */
-	return true;
-}
-
 /*
  * CheckProcSignal - check to see if a particular reason has been
  * signaled, and clear the signal flag.  Should be called after receiving
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 900088a2209..2767cf18c68 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -19,6 +19,7 @@
 #include "access/htup_details.h"
 #include "access/reloptions.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "catalog/catalog.h"
@@ -85,7 +86,6 @@ static void ProcessUtilitySlow(ParseState *pstate,
 							   DestReceiver *dest,
 							   QueryCompletion *qc);
 static void ExecDropStmt(DropStmt *stmt, bool isTopLevel);
-static void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
 
 /*
  * CommandIsReadOnly: is an executable query read-only?
@@ -3644,15 +3644,3 @@ GetCommandLogLevel(Node *parsetree)
 
 	return lev;
 }
-
-/*
- * AlterSystemSetWALProhibitState
- *
- * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
- */
-static void
-AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
-{
-	/* some code */
-	elog(INFO, "AlterSystemSetWALProhibitState() called");
-}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 75fc6f11d6a..58b56eac21f 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -221,6 +221,7 @@ static bool check_recovery_target_lsn(char **newval, void **extra, GucSource sou
 static void assign_recovery_target_lsn(const char *newval, void *extra);
 static bool check_primary_slot_name(char **newval, void **extra, GucSource source);
 static bool check_default_with_oids(bool *newval, void **extra, GucSource source);
+static const char *show_system_is_read_only(void);
 
 /* Private functions in guc-file.l that need to be called from guc.c */
 static ConfigVariable *ProcessConfigFileInternal(GucContext context,
@@ -610,6 +611,7 @@ static char *recovery_target_string;
 static char *recovery_target_xid_string;
 static char *recovery_target_name_string;
 static char *recovery_target_lsn_string;
+static bool system_is_read_only;
 
 
 /* should be static, but commands/variable.c needs to get at this */
@@ -2041,6 +2043,18 @@ static struct config_bool ConfigureNamesBool[] =
 		NULL, NULL, NULL
 	},
 
+	{
+		/* Not for general use */
+		{"system_is_read_only", PGC_INTERNAL, WAL,
+			gettext_noop("Shows whether the system is read only."),
+			NULL,
+			GUC_NO_RESET_ALL | GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE
+		},
+		&system_is_read_only,
+		false,
+		NULL, NULL, show_system_is_read_only
+	},
+
 	/* End-of-list marker */
 	{
 		{NULL, 0, 0, NULL, NULL}, NULL, false, NULL, NULL, NULL
@@ -11998,4 +12012,16 @@ check_default_with_oids(bool *newval, void **extra, GucSource source)
 	return true;
 }
 
+/*
+ * NB: The return string should be the same as the _ShowOption() for boolean
+ * type.
+ */
+ static const char *
+ show_system_is_read_only(void)
+{
+	if (!XLogInsertAllowed())
+		return "on";
+	return "off";
+}
+
 #include "guc-file.c"
diff --git a/src/include/access/walprohibit.h b/src/include/access/walprohibit.h
new file mode 100644
index 00000000000..619c33cd780
--- /dev/null
+++ b/src/include/access/walprohibit.h
@@ -0,0 +1,21 @@
+/*
+ * walprohibit.h
+ *
+ * PostgreSQL write-ahead log prohibit states
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/include/access/walprohibit.h
+ */
+#ifndef WALPROHIBIT_H
+#define WALPROHIBIT_H
+
+#include "access/xact.h"
+#include "access/xlog.h"
+#include "miscadmin.h"
+#include "nodes/parsenodes.h"
+
+extern bool ProcessBarrierWALProhibit(void);
+extern void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
+
+#endif		/* WALPROHIBIT_H */
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 347a38f57cf..6e98e27b2c8 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -298,11 +298,13 @@ extern const char *xlog_identify(uint8 info);
 
 extern void issue_xlog_fsync(int fd, XLogSegNo segno);
 
+extern bool IsWALProhibited(void);
 extern bool RecoveryInProgress(void);
 extern RecoveryState GetRecoveryState(void);
 extern bool HotStandbyActive(void);
 extern bool HotStandbyActiveInReplay(void);
 extern bool XLogInsertAllowed(void);
+extern void ResetLocalXLogInsertAllowed(void);
 extern void GetXLogReceiptTime(TimestampTz *rtime, bool *fromStream);
 extern XLogRecPtr GetXLogReplayRecPtr(TimeLineID *replayTLI);
 extern XLogRecPtr GetXLogInsertRecPtr(void);
@@ -322,6 +324,7 @@ extern void XLOGShmemInit(void);
 extern void BootStrapXLOG(void);
 extern void LocalProcessControlFile(bool reset);
 extern void StartupXLOG(void);
+extern void MakeReadOnlyXLOG(void);
 extern void ShutdownXLOG(int code, Datum arg);
 extern void InitXLOGAccess(void);
 extern void CreateCheckPoint(int flags);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f38..bae06202b4a 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -48,12 +48,7 @@ typedef enum
 
 typedef enum
 {
-	/*
-	 * XXX. PROCSIGNAL_BARRIER_PLACEHOLDER should be replaced when the first
-	 * real user of the ProcSignalBarrier mechanism is added. It's just here
-	 * for now because we can't have an empty enum.
-	 */
-	PROCSIGNAL_BARRIER_PLACEHOLDER = 0
+	PROCSIGNAL_BARRIER_WALPROHIBIT = 0
 } ProcSignalBarrierType;
 
 /*
-- 
2.18.0

From f301f644428afed7c886f5bf13451829b3e13144 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 19 Jun 2020 06:27:53 -0400
Subject: [PATCH v2 1/6] Allow error or refusal while absorbing barriers.

Patch by Robert Haas
---
 src/backend/storage/ipc/procsignal.c | 75 +++++++++++++++++++++++-----
 1 file changed, 63 insertions(+), 12 deletions(-)

diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ece..13648887187 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -87,12 +87,16 @@ typedef struct
 #define BARRIER_SHOULD_CHECK(flags, type) \
 	(((flags) & (((uint32) 1) << (uint32) (type))) != 0)
 
+/* Clear the relevant type bit from the flags. */
+#define BARRIER_CLEAR_BIT(flags, type) \
+	((flags) &= ~(((uint32) 1) << (uint32) (type)))
+
 static ProcSignalHeader *ProcSignal = NULL;
 static volatile ProcSignalSlot *MyProcSignalSlot = NULL;
 
 static bool CheckProcSignal(ProcSignalReason reason);
 static void CleanupProcSignalState(int status, Datum arg);
-static void ProcessBarrierPlaceholder(void);
+static bool ProcessBarrierPlaceholder(void);
 
 /*
  * ProcSignalShmemSize
@@ -486,17 +490,59 @@ ProcessProcSignalBarrier(void)
 	flags = pg_atomic_exchange_u32(&MyProcSignalSlot->pss_barrierCheckMask, 0);
 
 	/*
-	 * Process each type of barrier. It's important that nothing we call from
-	 * here throws an error, because pss_barrierCheckMask has already been
-	 * cleared. If we jumped out of here before processing all barrier types,
-	 * then we'd forget about the need to do so later.
-	 *
-	 * NB: It ought to be OK to call the barrier-processing functions
-	 * unconditionally, but it's more efficient to call only the ones that
-	 * might need us to do something based on the flags.
+	 * If there are no flags set, then we can skip doing any real work.
+	 * Otherwise, establish a PG_TRY block, so that we don't lose track of
+	 * which types of barrier processing are needed if an ERROR occurs.
 	 */
-	if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_PLACEHOLDER))
-		ProcessBarrierPlaceholder();
+	if (flags != 0)
+	{
+		PG_TRY();
+		{
+			/*
+			 * Process each type of barrier. The barrier-processing functions
+			 * should normally return true, but may return false if the barrier
+			 * can't be absorbed at the current time. This should be rare,
+			 * because it's pretty expensive.  Every single
+			 * CHECK_FOR_INTERRUPTS() will return here until we manage to
+			 * absorb the barrier, and that cost will add up in a hurry.
+			 *
+			 * NB: It ought to be OK to call the barrier-processing functions
+			 * unconditionally, but it's more efficient to call only the ones
+			 * that might need us to do something based on the flags.
+			 */
+			if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_PLACEHOLDER)
+				&& ProcessBarrierPlaceholder())
+				BARRIER_CLEAR_BIT(flags, PROCSIGNAL_BARRIER_PLACEHOLDER);
+		}
+		PG_CATCH();
+		{
+			/*
+			 * If an ERROR occurred, add any flags that weren't yet handled
+			 * back into pss_barrierCheckMask, and reset the global variables
+			 * so that we try again the next time we check for interrupts.
+			 */
+			pg_atomic_fetch_or_u32(&MyProcSignalSlot->pss_barrierCheckMask,
+								   flags);
+			ProcSignalBarrierPending = true;
+			InterruptPending = true;
+
+			PG_RE_THROW();
+		}
+		PG_END_TRY();
+
+		/*
+		 * If some barrier was not successfully absorbed, we will have to try
+		 * again later.
+		 */
+		if (flags != 0)
+		{
+			pg_atomic_fetch_or_u32(&MyProcSignalSlot->pss_barrierCheckMask,
+								   flags);
+			ProcSignalBarrierPending = true;
+			InterruptPending = true;
+			return;
+		}
+	}
 
 	/*
 	 * State changes related to all types of barriers that might have been
@@ -508,7 +554,7 @@ ProcessProcSignalBarrier(void)
 	pg_atomic_write_u64(&MyProcSignalSlot->pss_barrierGeneration, shared_gen);
 }
 
-static void
+static bool
 ProcessBarrierPlaceholder(void)
 {
 	/*
@@ -518,7 +564,12 @@ ProcessBarrierPlaceholder(void)
 	 * appropriately descriptive. Get rid of this function and instead have
 	 * ProcessBarrierSomethingElse. Most likely, that function should live in
 	 * the file pertaining to that subsystem, rather than here.
+	 *
+	 * The return value should be 'true' if the barrier was successfully
+	 * absorbed and 'false' if not. Note that returning 'false' can lead to
+	 * very frequent retries, so try hard to make that an uncommon case.
 	 */
+	return true;
 }
 
 /*
-- 
2.18.0

From 15ec36c9fd9659fe8a497ef59719b5a87ca8014b Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Tue, 16 Jun 2020 06:35:41 -0400
Subject: [PATCH v2 5/6] Error or Assert before START_CRIT_SECTION for WAL
 write

Based on the following criteria added the Assert or the Error when system is
prohibited:

 - Added ERROR for the function which can be reachable without valid XID in case
   of VACUUM or CONCURRENT CREATE INDEX.  For that added common static inline
   function CheckWALPermitted().
 - Added ASSERT for the function which cannot be reached without valid XID;
   Assert to ensure XID validation. For that added AssertWALPermitted_HaveXID().

To enforce the rule to have aforesaid assert or error check before entering a
critical section for WAL write, a new walpermit_checked_state assert only flag
is added. If these check is missing then XLogBeginInsert() will have an
assertion if it is in critical section.

If we are not doing WAL insert inside the critical section then the above
checking is not necessary, we can rely on XLogBeginInsert() for that check &
report an error.
---
 src/backend/access/brin/brin.c            |  4 ++
 src/backend/access/brin/brin_pageops.c    | 16 ++++++++
 src/backend/access/brin/brin_revmap.c     |  8 ++++
 src/backend/access/gin/ginbtree.c         | 17 ++++++--
 src/backend/access/gin/gindatapage.c      | 14 ++++++-
 src/backend/access/gin/ginfast.c          |  8 ++++
 src/backend/access/gin/gininsert.c        |  4 ++
 src/backend/access/gin/ginutil.c          | 10 ++++-
 src/backend/access/gin/ginvacuum.c        |  9 +++++
 src/backend/access/gist/gist.c            | 16 ++++++++
 src/backend/access/gist/gistvacuum.c      |  9 +++++
 src/backend/access/hash/hash.c            | 13 ++++++
 src/backend/access/hash/hashinsert.c      |  8 ++++
 src/backend/access/hash/hashovfl.c        | 14 +++++++
 src/backend/access/hash/hashpage.c        | 13 ++++++
 src/backend/access/heap/heapam.c          | 32 +++++++++++++++
 src/backend/access/heap/pruneheap.c       |  7 +++-
 src/backend/access/heap/vacuumlazy.c      | 13 ++++++
 src/backend/access/heap/visibilitymap.c   | 20 ++++++++-
 src/backend/access/nbtree/nbtdedup.c      |  4 ++
 src/backend/access/nbtree/nbtinsert.c     | 13 +++++-
 src/backend/access/nbtree/nbtpage.c       | 24 +++++++++++
 src/backend/access/spgist/spgdoinsert.c   | 19 +++++++++
 src/backend/access/spgist/spgvacuum.c     | 13 ++++++
 src/backend/access/transam/multixact.c    |  6 ++-
 src/backend/access/transam/twophase.c     | 10 +++++
 src/backend/access/transam/varsup.c       |  4 ++
 src/backend/access/transam/walprohibit.c  | 10 +++++
 src/backend/access/transam/xact.c         |  7 ++++
 src/backend/access/transam/xlog.c         | 27 +++++++++----
 src/backend/access/transam/xloginsert.c   | 13 +++++-
 src/backend/commands/sequence.c           | 16 ++++++++
 src/backend/commands/variable.c           |  9 +++--
 src/backend/postmaster/checkpointer.c     |  4 ++
 src/backend/storage/buffer/bufmgr.c       | 10 +++--
 src/backend/storage/freespace/freespace.c | 11 ++++-
 src/backend/storage/lmgr/lock.c           |  6 +--
 src/backend/utils/cache/relmapper.c       |  4 ++
 src/include/access/walprohibit.h          | 49 ++++++++++++++++++++++-
 src/include/miscadmin.h                   | 27 +++++++++++++
 40 files changed, 490 insertions(+), 31 deletions(-)

diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index 7db3ae5ee0c..ef002a51773 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -24,6 +24,7 @@
 #include "access/relscan.h"
 #include "access/table.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/index.h"
 #include "catalog/pg_am.h"
@@ -758,6 +759,9 @@ brinbuildempty(Relation index)
 		ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermitted_HaveXID();
+
 	/* Initialize and xlog metabuffer. */
 	START_CRIT_SECTION();
 	brin_metapage_init(BufferGetPage(metabuf), BrinGetPagesPerRange(index),
diff --git a/src/backend/access/brin/brin_pageops.c b/src/backend/access/brin/brin_pageops.c
index 87de0b855b5..197e1213137 100644
--- a/src/backend/access/brin/brin_pageops.c
+++ b/src/backend/access/brin/brin_pageops.c
@@ -14,6 +14,7 @@
 #include "access/brin_pageops.h"
 #include "access/brin_revmap.h"
 #include "access/brin_xlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -176,6 +177,10 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 	if (((BrinPageFlags(oldpage) & BRIN_EVACUATE_PAGE) == 0) &&
 		brin_can_do_samepage_update(oldbuf, origsz, newsz))
 	{
+		/* Can reach here from VACUUM, so need not have an XID */
+		if (RelationNeedsWAL(idxrel))
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 		if (!PageIndexTupleOverwrite(oldpage, oldoff, (Item) unconstify(BrinTuple *, newtup), newsz))
 			elog(ERROR, "failed to replace BRIN tuple");
@@ -240,6 +245,10 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 
 		revmapbuf = brinLockRevmapPageForUpdate(revmap, heapBlk);
 
+		/* Can reach here from VACUUM, so need not have an XID */
+		if (RelationNeedsWAL(idxrel))
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		/*
@@ -405,6 +414,10 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	page = BufferGetPage(*buffer);
 	blk = BufferGetBlockNumber(*buffer);
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(idxrel))
+		CheckWALPermitted();
+
 	/* Execute the actual insertion */
 	START_CRIT_SECTION();
 	if (extended)
@@ -881,6 +894,9 @@ brin_initialize_empty_new_buffer(Relation idxrel, Buffer buffer)
 			   "brin_initialize_empty_new_buffer: initializing blank page %u",
 			   BufferGetBlockNumber(buffer)));
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 	page = BufferGetPage(buffer);
 	brin_page_init(page, BRIN_PAGETYPE_REGULAR);
diff --git a/src/backend/access/brin/brin_revmap.c b/src/backend/access/brin/brin_revmap.c
index 9c4b3e22021..80b6e826ae7 100644
--- a/src/backend/access/brin/brin_revmap.c
+++ b/src/backend/access/brin/brin_revmap.c
@@ -26,6 +26,7 @@
 #include "access/brin_tuple.h"
 #include "access/brin_xlog.h"
 #include "access/rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -405,6 +406,10 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 				(errmsg("leftover placeholder tuple detected in BRIN index \"%s\", deleting",
 						RelationGetRelationName(idxrel))));
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(idxrel))
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	ItemPointerSetInvalid(&invalidIptr);
@@ -614,6 +619,9 @@ revmap_physical_extend(BrinRevmap *revmap)
 		return;
 	}
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	/*
 	 * Ok, we have now locked the metapage and the target block. Re-initialize
 	 * the target block as a revmap page, and update the metapage.
diff --git a/src/backend/access/gin/ginbtree.c b/src/backend/access/gin/ginbtree.c
index 8d08b05f515..1b835b3000b 100644
--- a/src/backend/access/gin/ginbtree.c
+++ b/src/backend/access/gin/ginbtree.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/predicate.h"
@@ -333,6 +334,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 {
 	Page		page = BufferGetPage(stack->buffer);
 	bool		result;
+	bool		needwal;
 	GinPlaceToPageRC rc;
 	uint16		xlflags = 0;
 	Page		childpage = NULL;
@@ -378,6 +380,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 								 insertdata, updateblkno,
 								 &ptp_workspace,
 								 &newlpage, &newrpage);
+	needwal = RelationNeedsWAL(btree->index) && !btree->isBuild;
 
 	if (rc == GPTP_NO_WORK)
 	{
@@ -386,10 +389,14 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 	}
 	else if (rc == GPTP_INSERT)
 	{
+		/* Can reach here from VACUUM, so need not have an XID */
+		if (needwal)
+			CheckWALPermitted();
+
 		/* It will fit, perform the insertion */
 		START_CRIT_SECTION();
 
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogBeginInsert();
 			XLogRegisterBuffer(0, stack->buffer, REGBUF_STANDARD);
@@ -410,7 +417,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 			MarkBufferDirty(childbuf);
 		}
 
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 			ginxlogInsert xlrec;
@@ -548,6 +555,10 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 			}
 		}
 
+		/* Can reach here from VACUUM, so need not have an XID */
+		if (needwal)
+			CheckWALPermitted();
+
 		/*
 		 * OK, we have the new contents of the left page in a temporary copy
 		 * now (newlpage), and likewise for the new contents of the
@@ -588,7 +599,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 		}
 
 		/* write WAL record */
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gin/gindatapage.c b/src/backend/access/gin/gindatapage.c
index 7a2690e97f2..226cb3ce44b 100644
--- a/src/backend/access/gin/gindatapage.c
+++ b/src/backend/access/gin/gindatapage.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "lib/ilist.h"
 #include "miscadmin.h"
@@ -836,7 +837,11 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 		}
 
 		if (RelationNeedsWAL(indexrel))
+		{
+			/* Can reach here from VACUUM, so need not have an XID */
+			CheckWALPermitted();
 			computeLeafRecompressWALData(leaf);
+		}
 
 		/* Apply changes to page */
 		START_CRIT_SECTION();
@@ -1777,6 +1782,7 @@ createPostingTree(Relation index, ItemPointerData *items, uint32 nitems,
 	int			nrootitems;
 	int			rootsize;
 	bool		is_build = (buildStats != NULL);
+	bool		needwal;
 
 	/* Construct the new root page in memory first. */
 	tmppage = (Page) palloc(BLCKSZ);
@@ -1825,12 +1831,18 @@ createPostingTree(Relation index, ItemPointerData *items, uint32 nitems,
 	 */
 	PredicateLockPageSplit(index, BufferGetBlockNumber(entrybuffer), blkno);
 
+	needwal = RelationNeedsWAL(index) && !is_build;
+
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	PageRestoreTempPage(tmppage, page);
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index) && !is_build)
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogCreatePostingTree data;
diff --git a/src/backend/access/gin/ginfast.c b/src/backend/access/gin/ginfast.c
index 2e41b34d8d5..d7781de7674 100644
--- a/src/backend/access/gin/ginfast.c
+++ b/src/backend/access/gin/ginfast.c
@@ -20,6 +20,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
 #include "catalog/pg_am.h"
@@ -68,6 +69,9 @@ writeListPage(Relation index, Buffer buffer,
 	PGAlignedBlock workspace;
 	char	   *ptr;
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	START_CRIT_SECTION();
 
 	GinInitBuffer(buffer, GIN_LIST);
@@ -587,7 +591,11 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 		 * critical section.
 		 */
 		if (RelationNeedsWAL(index))
+		{
+			/* Can reach here from VACUUM, so need not have an XID */
+			CheckWALPermitted();
 			XLogEnsureRecordSpace(data.ndeleted, 0);
+		}
 
 		START_CRIT_SECTION();
 
diff --git a/src/backend/access/gin/gininsert.c b/src/backend/access/gin/gininsert.c
index 77433dc8a41..d957aa6e582 100644
--- a/src/backend/access/gin/gininsert.c
+++ b/src/backend/access/gin/gininsert.c
@@ -17,6 +17,7 @@
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/index.h"
 #include "miscadmin.h"
@@ -447,6 +448,9 @@ ginbuildempty(Relation index)
 		ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(RootBuffer, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermitted_HaveXID();
+
 	/* Initialize and xlog metabuffer and root buffer. */
 	START_CRIT_SECTION();
 	GinInitMetabuffer(MetaBuffer);
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index a400f1fedbc..938089238da 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -17,6 +17,7 @@
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
 #include "access/reloptions.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/pg_collation.h"
 #include "catalog/pg_type.h"
@@ -658,12 +659,19 @@ ginUpdateStats(Relation index, const GinStatsData *stats, bool is_build)
 	Buffer		metabuffer;
 	Page		metapage;
 	GinMetaPageData *metadata;
+	bool		needwal;
 
 	metabuffer = ReadBuffer(index, GIN_METAPAGE_BLKNO);
 	LockBuffer(metabuffer, GIN_EXCLUSIVE);
 	metapage = BufferGetPage(metabuffer);
 	metadata = GinPageGetMeta(metapage);
 
+	needwal = RelationNeedsWAL(index) && !is_build;
+
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	metadata->nTotalPages = stats->nTotalPages;
@@ -683,7 +691,7 @@ ginUpdateStats(Relation index, const GinStatsData *stats, bool is_build)
 
 	MarkBufferDirty(metabuffer);
 
-	if (RelationNeedsWAL(index) && !is_build)
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogUpdateMeta data;
diff --git a/src/backend/access/gin/ginvacuum.c b/src/backend/access/gin/ginvacuum.c
index 8ae4fd95a7b..36a884af597 100644
--- a/src/backend/access/gin/ginvacuum.c
+++ b/src/backend/access/gin/ginvacuum.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "commands/vacuum.h"
 #include "miscadmin.h"
@@ -159,6 +160,10 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	 */
 	PredicateLockPageCombine(gvs->index, deleteBlkno, rightlink);
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(gvs->index))
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* Unlink the page by changing left sibling's rightlink */
@@ -650,6 +655,10 @@ ginbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 
 		if (resPage)
 		{
+			/* Can reach here from VACUUM, so need not have an XID */
+			if (RelationNeedsWAL(gvs.index))
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 			PageRestoreTempPage(resPage, page);
 			MarkBufferDirty(buffer);
diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c
index 79fe6eb8d62..8f6b15d8ee4 100644
--- a/src/backend/access/gist/gist.c
+++ b/src/backend/access/gist/gist.c
@@ -16,6 +16,7 @@
 
 #include "access/gist_private.h"
 #include "access/gistscan.h"
+#include "access/walprohibit.h"
 #include "catalog/pg_collation.h"
 #include "commands/vacuum.h"
 #include "miscadmin.h"
@@ -134,6 +135,9 @@ gistbuildempty(Relation index)
 	buffer = ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermitted_HaveXID();
+
 	/* Initialize and xlog buffer */
 	START_CRIT_SECTION();
 	GISTInitBuffer(buffer, F_LEAF);
@@ -467,6 +471,10 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 		if (!is_build && RelationNeedsWAL(rel))
 			XLogEnsureRecordSpace(npage, 1 + npage * 2);
 
+		/* Can reach here from VACUUM, so need not have an XID */
+		if (RelationNeedsWAL(rel))
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		/*
@@ -525,6 +533,10 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 	}
 	else
 	{
+		/* Can reach here from VACUUM, so need not have an XID */
+		if (RelationNeedsWAL(rel))
+			CheckWALPermitted();
+
 		/*
 		 * Enough space.  We always get here if ntup==0.
 		 */
@@ -1665,6 +1677,10 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 
 	if (ndeletable > 0)
 	{
+		/* Can reach here from VACUUM, so need not have an XID */
+		if (RelationNeedsWAL(rel))
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		PageIndexMultiDelete(page, deletable, ndeletable);
diff --git a/src/backend/access/gist/gistvacuum.c b/src/backend/access/gist/gistvacuum.c
index a9c616c7724..ccf9bc0c214 100644
--- a/src/backend/access/gist/gistvacuum.c
+++ b/src/backend/access/gist/gistvacuum.c
@@ -17,6 +17,7 @@
 #include "access/genam.h"
 #include "access/gist_private.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "commands/vacuum.h"
 #include "lib/integerset.h"
 #include "miscadmin.h"
@@ -341,6 +342,10 @@ restart:
 		 */
 		if (ntodelete > 0)
 		{
+			/* Can reach here from VACUUM, so need not have an XID */
+			if (RelationNeedsWAL(rel))
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 
 			MarkBufferDirty(buffer);
@@ -634,6 +639,10 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	 */
 	txid = ReadNextFullTransactionId();
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(info->index))
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* mark the page as deleted */
diff --git a/src/backend/access/hash/hash.c b/src/backend/access/hash/hash.c
index 3ec6d528e77..1d3f4c92f19 100644
--- a/src/backend/access/hash/hash.c
+++ b/src/backend/access/hash/hash.c
@@ -22,6 +22,7 @@
 #include "access/hash_xlog.h"
 #include "access/relscan.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "catalog/index.h"
 #include "commands/progress.h"
 #include "commands/vacuum.h"
@@ -572,6 +573,10 @@ loop_top:
 		goto loop_top;
 	}
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(rel))
+		CheckWALPermitted();
+
 	/* Okay, we're really done.  Update tuple count in metapage. */
 	START_CRIT_SECTION();
 
@@ -787,6 +792,10 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		 */
 		if (ndeletable > 0)
 		{
+			/* Can reach here from VACUUM, so need not have an XID */
+			if (RelationNeedsWAL(rel))
+				CheckWALPermitted();
+
 			/* No ereport(ERROR) until changes are logged */
 			START_CRIT_SECTION();
 
@@ -882,6 +891,10 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		page = BufferGetPage(bucket_buf);
 		bucket_opaque = (HashPageOpaque) PageGetSpecialPointer(page);
 
+		/* Can reach here from VACUUM, so need not have an XID */
+		if (RelationNeedsWAL(rel))
+			CheckWALPermitted();
+
 		/* No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c
index 2ebe671967b..360e30456fe 100644
--- a/src/backend/access/hash/hashinsert.c
+++ b/src/backend/access/hash/hashinsert.c
@@ -17,6 +17,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "storage/buf_internals.h"
 #include "storage/lwlock.h"
@@ -193,6 +194,9 @@ restart_insert:
 	 */
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	/* Do the update.  No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -370,6 +374,10 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 		 */
 		LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+		/* Can reach here from VACUUM, so need not have an XID */
+		if (RelationNeedsWAL(rel))
+			CheckWALPermitted();
+
 		/* No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
diff --git a/src/backend/access/hash/hashovfl.c b/src/backend/access/hash/hashovfl.c
index 00f0a940116..5abba14899e 100644
--- a/src/backend/access/hash/hashovfl.c
+++ b/src/backend/access/hash/hashovfl.c
@@ -19,6 +19,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "utils/rel.h"
 
@@ -312,6 +313,9 @@ _hash_addovflpage(Relation rel, Buffer metabuf, Buffer buf, bool retain_pin)
 
 found:
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	/*
 	 * Do the update.  No ereport(ERROR) until changes are logged. We want to
 	 * log the changes for bitmap page and overflow page together to avoid
@@ -577,6 +581,10 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	if (RelationNeedsWAL(rel))
 		XLogEnsureRecordSpace(HASH_XLOG_FREE_OVFL_BUFS, 4 + nitups);
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(rel))
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -929,7 +937,13 @@ readpage:
 					 * WAL for that.
 					 */
 					if (RelationNeedsWAL(rel))
+					{
+						/*
+						 * Can reach here from VACUUM, so need not have an XID
+						 */
+						CheckWALPermitted();
 						XLogEnsureRecordSpace(0, 3 + nitups);
+					}
 
 					START_CRIT_SECTION();
 
diff --git a/src/backend/access/hash/hashpage.c b/src/backend/access/hash/hashpage.c
index a664ecf494a..faad58297d2 100644
--- a/src/backend/access/hash/hashpage.c
+++ b/src/backend/access/hash/hashpage.c
@@ -30,6 +30,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "port/pg_bitutils.h"
 #include "storage/lmgr.h"
@@ -816,6 +817,9 @@ restart_expand:
 		goto fail;
 	}
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	/*
 	 * Since we are scribbling on the pages in the shared buffers, establish a
 	 * critical section.  Any failure in this next code leaves us with a big
@@ -1172,6 +1176,9 @@ _hash_splitbucket(Relation rel,
 
 				if (PageGetFreeSpaceForMultipleTuples(npage, nitups + 1) < (all_tups_size + itemsz))
 				{
+					/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+					AssertWALPermitted_HaveXID();
+
 					/*
 					 * Change the shared buffer state in critical section,
 					 * otherwise any error could make it unrecoverable.
@@ -1223,6 +1230,9 @@ _hash_splitbucket(Relation rel,
 		/* Exit loop if no more overflow pages in old bucket */
 		if (!BlockNumberIsValid(oblkno))
 		{
+			/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+			AssertWALPermitted_HaveXID();
+
 			/*
 			 * Change the shared buffer state in critical section, otherwise
 			 * any error could make it unrecoverable.
@@ -1269,6 +1279,9 @@ _hash_splitbucket(Relation rel,
 	npage = BufferGetPage(bucket_nbuf);
 	nopaque = (HashPageOpaque) PageGetSpecialPointer(npage);
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	START_CRIT_SECTION();
 
 	oopaque->hasho_flag &= ~LH_BUCKET_BEING_SPLIT;
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 537913d1bb3..c52200463a4 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -46,6 +46,7 @@
 #include "access/transam.h"
 #include "access/valid.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1870,6 +1871,9 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
 	 */
 	CheckForSerializableConflictIn(relation, NULL, InvalidBlockNumber);
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -2143,6 +2147,9 @@ heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples,
 										   &vmbuffer, NULL);
 		page = BufferGetPage(buffer);
 
+		/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+		AssertWALPermitted_HaveXID();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -2661,6 +2668,9 @@ l1:
 							  xid, LockTupleExclusive, true,
 							  &new_xmax, &new_infomask, &new_infomask2);
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -3413,6 +3423,9 @@ l2:
 
 		Assert(HEAP_XMAX_IS_LOCKED_ONLY(infomask_lock_old_tuple));
 
+		/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+		AssertWALPermitted_HaveXID();
+
 		START_CRIT_SECTION();
 
 		/* Clear obsolete visibility flags ... */
@@ -3586,6 +3599,9 @@ l2:
 										   bms_overlap(modified_attrs, id_attrs),
 										   &old_key_copied);
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -4519,6 +4535,9 @@ failed:
 							  GetCurrentTransactionId(), mode, false,
 							  &xid, &new_infomask, &new_infomask2);
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -5310,6 +5329,9 @@ l4:
 								VISIBILITYMAP_ALL_FROZEN))
 			cleared_all_frozen = true;
 
+		/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+		AssertWALPermitted_HaveXID();
+
 		START_CRIT_SECTION();
 
 		/* ... and set them */
@@ -5468,6 +5490,9 @@ heap_finish_speculative(Relation relation, ItemPointer tid)
 	StaticAssertStmt(MaxOffsetNumber < SpecTokenOffsetNumber,
 					 "invalid speculative token constant");
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -5576,6 +5601,9 @@ heap_abort_speculative(Relation relation, ItemPointer tid)
 		elog(ERROR, "attempted to kill a non-speculative tuple");
 	Assert(!HeapTupleHeaderIsHeapOnly(tp.t_data));
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	/*
 	 * No need to check for serializable conflicts here.  There is never a
 	 * need for a combocid, either.  No need to extract replica identity, or
@@ -5722,6 +5750,10 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	if (oldlen != newlen || htup->t_hoff != tuple->t_data->t_hoff)
 		elog(ERROR, "wrong tuple length");
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(relation))
+		CheckWALPermitted();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index 1794cfd8d9a..e7fcfb02864 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -18,6 +18,7 @@
 #include "access/heapam_xlog.h"
 #include "access/htup_details.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "catalog/catalog.h"
 #include "miscadmin.h"
@@ -81,7 +82,7 @@ heap_page_prune_opt(Relation relation, Buffer buffer)
 	 * clean the page. The master will likely issue a cleaning WAL record soon
 	 * anyway, so this is no particular loss.
 	 */
-	if (RecoveryInProgress())
+	if (!XLogInsertAllowed())
 		return;
 
 	/*
@@ -225,6 +226,10 @@ heap_page_prune(Relation relation, Buffer buffer, TransactionId OldestXmin,
 									 &prstate);
 	}
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(relation))
+		CheckWALPermitted();
+
 	/* Any error while applying the changes is critical */
 	START_CRIT_SECTION();
 
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 3bef0e124ba..3613b7a88d6 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -59,6 +59,7 @@
 #include "access/parallel.h"
 #include "access/transam.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "catalog/storage.h"
@@ -1195,6 +1196,10 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 			 */
 			if (!PageIsAllVisible(page))
 			{
+				/* Can reach here from VACUUM, so need not have an XID */
+				if (RelationNeedsWAL(onerel))
+					CheckWALPermitted();
+
 				START_CRIT_SECTION();
 
 				/* mark buffer dirty before writing a WAL record */
@@ -1463,6 +1468,10 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 		 */
 		if (nfrozen > 0)
 		{
+			/* Can reach here from VACUUM, so need not have an XID */
+			if (RelationNeedsWAL(onerel))
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 
 			MarkBufferDirty(buf);
@@ -1914,6 +1923,10 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
 							 blkno, NULL);
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(onerel))
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	for (; tupindex < dead_tuples->num_tuples; tupindex++)
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 0a51678c40d..30d1d6f34c7 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -88,6 +88,7 @@
 
 #include "access/heapam_xlog.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "miscadmin.h"
 #include "port/pg_bitutils.h"
@@ -270,6 +271,16 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 	map = (uint8 *) PageGetContents(page);
 	LockBuffer(vmBuf, BUFFER_LOCK_EXCLUSIVE);
 
+	/*
+	 * Can reach here from VACUUM or from startup process, so need not have an
+	 * XID.
+	 *
+	 * Recovery in the startup process never have wal prohibit state, skip
+	 * permission check if reach here in the startup process.
+	 */
+	if (RelationNeedsWAL(rel))
+		InRecovery ? AssertWALPermitted() : CheckWALPermitted();
+
 	if (flags != (map[mapByte] >> mapOffset & VISIBILITYMAP_VALID_BITS))
 	{
 		START_CRIT_SECTION();
@@ -476,6 +487,7 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		Buffer		mapBuffer;
 		Page		page;
 		char	   *map;
+		bool		needwal;
 
 		newnblocks = truncBlock + 1;
 
@@ -489,8 +501,14 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		page = BufferGetPage(mapBuffer);
 		map = PageGetContents(page);
 
+		needwal = (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded());
+
 		LockBuffer(mapBuffer, BUFFER_LOCK_EXCLUSIVE);
 
+		/* Can reach here from VACUUM, so need not have an XID */
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -518,7 +536,7 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		 * during recovery.
 		 */
 		MarkBufferDirty(mapBuffer);
-		if (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded())
+		if (needwal)
 			log_newpage_buffer(mapBuffer, false);
 
 		END_CRIT_SECTION();
diff --git a/src/backend/access/nbtree/nbtdedup.c b/src/backend/access/nbtree/nbtdedup.c
index f6be865b17e..a471a4b7ff5 100644
--- a/src/backend/access/nbtree/nbtdedup.c
+++ b/src/backend/access/nbtree/nbtdedup.c
@@ -16,6 +16,7 @@
 
 #include "access/nbtree.h"
 #include "access/nbtxlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "utils/rel.h"
 
@@ -271,6 +272,9 @@ _bt_dedup_one_page(Relation rel, Buffer buf, Relation heapRel,
 		nopaque->btpo_flags &= ~BTP_HAS_GARBAGE;
 	}
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	START_CRIT_SECTION();
 
 	PageRestoreTempPage(newpage, page);
diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c
index 55fe16bd4e1..b88ec09a397 100644
--- a/src/backend/access/nbtree/nbtinsert.c
+++ b/src/backend/access/nbtree/nbtinsert.c
@@ -19,6 +19,7 @@
 #include "access/nbtxlog.h"
 #include "access/tableam.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/lmgr.h"
@@ -1245,6 +1246,9 @@ _bt_insertonpg(Relation rel,
 			}
 		}
 
+		/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+		AssertWALPermitted_HaveXID();
+
 		/* Do the update.  No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -1899,13 +1903,17 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf,
 			ropaque->btpo_flags |= BTP_SPLIT_END;
 	}
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	/*
 	 * Right sibling is locked, new siblings are prepared, but original page
 	 * is not updated yet.
 	 *
 	 * NO EREPORT(ERROR) till right sibling is updated.  We can get away with
 	 * not starting the critical section till here because we haven't been
-	 * scribbling on the original page yet; see comments above.
+	 * scribbling on the original page yet; see the comments above for grabbing
+	 * the right sibling.
 	 */
 	START_CRIT_SECTION();
 
@@ -2468,6 +2476,9 @@ _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf)
 	right_item = CopyIndexTuple(item);
 	BTreeTupleSetDownLink(right_item, rbkno);
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	/* NO EREPORT(ERROR) from here till newroot op is logged */
 	START_CRIT_SECTION();
 
diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c
index 75628e0eb98..09b45fbb559 100644
--- a/src/backend/access/nbtree/nbtpage.c
+++ b/src/backend/access/nbtree/nbtpage.c
@@ -26,6 +26,7 @@
 #include "access/nbtxlog.h"
 #include "access/tableam.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
@@ -201,6 +202,10 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	LockBuffer(metabuf, BUFFER_LOCK_UNLOCK);
 	LockBuffer(metabuf, BT_WRITE);
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(rel))
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* upgrade meta-page if needed */
@@ -376,6 +381,10 @@ _bt_getroot(Relation rel, int access)
 		/* Get raw page pointer for metapage */
 		metapg = BufferGetPage(metabuf);
 
+		/* Can reach here from VACUUM, so need not have an XID */
+		if (RelationNeedsWAL(rel))
+			CheckWALPermitted();
+
 		/* NO ELOG(ERROR) till meta is updated */
 		START_CRIT_SECTION();
 
@@ -1068,6 +1077,10 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 		}
 	}
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(rel))
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -1195,6 +1208,9 @@ _bt_delitems_delete(Relation rel, Buffer buf,
 		latestRemovedXid =
 			_bt_xid_horizon(rel, heapRel, page, deletable, ndeletable);
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -1812,6 +1828,10 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	 */
 	PredicateLockPageCombine(rel, leafblkno, leafrightsib);
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(rel))
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -2168,6 +2188,10 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 	 * Here we begin doing the deletion.
 	 */
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(rel))
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
diff --git a/src/backend/access/spgist/spgdoinsert.c b/src/backend/access/spgist/spgdoinsert.c
index 934d65b89f2..003b5e80f21 100644
--- a/src/backend/access/spgist/spgdoinsert.c
+++ b/src/backend/access/spgist/spgdoinsert.c
@@ -18,6 +18,7 @@
 #include "access/genam.h"
 #include "access/spgist_private.h"
 #include "access/spgxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -214,6 +215,9 @@ addLeafTuple(Relation index, SpGistState *state, SpGistLeafTuple leafTuple,
 	xlrec.offnumParent = InvalidOffsetNumber;
 	xlrec.nodeI = 0;
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	START_CRIT_SECTION();
 
 	if (current->offnum == InvalidOffsetNumber ||
@@ -458,6 +462,9 @@ moveLeafs(Relation index, SpGistState *state,
 
 	leafdata = leafptr = palloc(size);
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	START_CRIT_SECTION();
 
 	/* copy all the old tuples to new page, unless they're dead */
@@ -1109,6 +1116,9 @@ doPickSplit(Relation index, SpGistState *state,
 
 	leafdata = leafptr = (char *) palloc(totalLeafSizes);
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	/* Here we begin making the changes to the target pages */
 	START_CRIT_SECTION();
 
@@ -1517,6 +1527,9 @@ spgAddNodeAction(Relation index, SpGistState *state,
 	if (PageGetExactFreeSpace(current->page) >=
 		newInnerTuple->size - innerTuple->size)
 	{
+		/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+		AssertWALPermitted_HaveXID();
+
 		/*
 		 * We can replace the inner tuple by new version in-place
 		 */
@@ -1603,6 +1616,9 @@ spgAddNodeAction(Relation index, SpGistState *state,
 		else
 			xlrec.parentBlk = 2;
 
+		/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+		AssertWALPermitted_HaveXID();
+
 		START_CRIT_SECTION();
 
 		/* insert new ... */
@@ -1788,6 +1804,9 @@ spgSplitNodeAction(Relation index, SpGistState *state,
 									&xlrec.newPage);
 	}
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c
index bd98707f3c0..39bace9e490 100644
--- a/src/backend/access/spgist/spgvacuum.c
+++ b/src/backend/access/spgist/spgvacuum.c
@@ -19,6 +19,7 @@
 #include "access/spgist_private.h"
 #include "access/spgxlog.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/storage_xlog.h"
 #include "commands/vacuum.h"
@@ -323,6 +324,10 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 	if (nDeletable != xlrec.nDead + xlrec.nPlaceholder + xlrec.nMove)
 		elog(ERROR, "inconsistent counts of deletable tuples");
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(index))
+		CheckWALPermitted();
+
 	/* Do the updates */
 	START_CRIT_SECTION();
 
@@ -447,6 +452,10 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 	if (xlrec.nDelete == 0)
 		return;					/* nothing more to do */
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(index))
+		CheckWALPermitted();
+
 	/* Do the update */
 	START_CRIT_SECTION();
 
@@ -505,6 +514,10 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer)
 	xlrec.nToPlaceholder = 0;
 	xlrec.newestRedirectXid = InvalidTransactionId;
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(index))
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index ce84dac0c40..2b7b2ccad31 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -73,6 +73,7 @@
 #include "access/transam.h"
 #include "access/twophase.h"
 #include "access/twophase_rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1143,6 +1144,9 @@ GetNewMultiXactId(int nmembers, MultiXactOffset *offset)
 
 	ExtendMultiXactMember(nextOffset, nmembers);
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	CheckWALPermitted();
+
 	/*
 	 * Critical section from here until caller has written the data into the
 	 * just-reserved SLRU space; we don't want to error out with a partly
@@ -2942,7 +2946,7 @@ TruncateMultiXact(MultiXactId newOldestMulti, Oid newOldestMultiDB)
 	mxtruncinfo trunc;
 	MultiXactId earliest;
 
-	Assert(!RecoveryInProgress());
+	Assert(XLogInsertAllowed());
 	Assert(MultiXactState->finishedStartup);
 
 	/*
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 9b2e59bf0ec..0fa01f241f3 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -82,6 +82,7 @@
 #include "access/transam.h"
 #include "access/twophase.h"
 #include "access/twophase_rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1112,6 +1113,9 @@ EndPrepare(GlobalTransaction gxact)
 	 */
 	XLogEnsureRecordSpace(0, records.num_chunks);
 
+	/* Recording transaction prepares, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	START_CRIT_SECTION();
 
 	MyProc->delayChkpt = true;
@@ -2204,6 +2208,9 @@ RecordTransactionCommitPrepared(TransactionId xid,
 	replorigin = (replorigin_session_origin != InvalidRepOriginId &&
 				  replorigin_session_origin != DoNotReplicateId);
 
+	/* COMMIT PREPARED need not have an XID */
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* See notes in RecordTransactionCommit */
@@ -2294,6 +2301,9 @@ RecordTransactionAbortPrepared(TransactionId xid,
 		elog(PANIC, "cannot abort transaction %u, it was already committed",
 			 xid);
 
+	/* ROLLBACK PREPARED need not have an XID */
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
diff --git a/src/backend/access/transam/varsup.c b/src/backend/access/transam/varsup.c
index e14b53bf9e3..365de44321d 100644
--- a/src/backend/access/transam/varsup.c
+++ b/src/backend/access/transam/varsup.c
@@ -17,6 +17,7 @@
 #include "access/commit_ts.h"
 #include "access/subtrans.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "commands/dbcommands.h"
@@ -73,6 +74,9 @@ GetNewTransactionId(bool isSubXact)
 	if (RecoveryInProgress())
 		elog(ERROR, "cannot assign TransactionIds during recovery");
 
+	/* Cannot assign transaction id in read-only mode */
+	CheckWALPermitted();
+
 	LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
 
 	full_xid = ShmemVariableCache->nextFullXid;
diff --git a/src/backend/access/transam/walprohibit.c b/src/backend/access/transam/walprohibit.c
index a8cda2fafbc..896f0917cef 100644
--- a/src/backend/access/transam/walprohibit.c
+++ b/src/backend/access/transam/walprohibit.c
@@ -16,6 +16,16 @@
 #include "postmaster/bgwriter.h"
 #include "storage/procsignal.h"
 
+/*
+ * Assert flag to enforce WAL insert permission check rule before starting a
+ * critical section for the WAL writes.  For this, either of CheckWALPermitted,
+ * AssertWALPermitted_HaveXID, or AssertWALPermitted must be called before
+ * starting the critical section.
+ */
+#ifdef USE_ASSERT_CHECKING
+WALPermitCheckState walpermit_checked_state = WALPERMIT_UNCHECKED;
+#endif
+
 /*
  * ProcessBarrierWALProhibit()
  *
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index bf699f19fe8..28cccfa3de5 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -26,6 +26,7 @@
 #include "access/subtrans.h"
 #include "access/transam.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1292,6 +1293,9 @@ RecordTransactionCommit(void)
 		/* Tell bufmgr and smgr to prepare for commit */
 		BufmgrCommit();
 
+		/* We'll be reaching here with valid XID only. */
+		AssertWALPermitted_HaveXID();
+
 		/*
 		 * Mark ourselves as within our "commit critical section".  This
 		 * forces any concurrent checkpoint to wait until we've updated
@@ -1652,6 +1656,9 @@ RecordTransactionAbort(bool isSubXact)
 		elog(PANIC, "cannot abort transaction %u, it was already committed",
 			 xid);
 
+	/* We'll be reaching here with valid XID only. */
+	AssertWALPermitted_HaveXID();
+
 	/* Fetch the data we need for the abort record */
 	nrels = smgrGetPendingDeletes(false, &rels);
 	nchildren = xactGetCommittedChildren(&children);
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 9eb4bf413eb..5e8512bdfca 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -1024,7 +1024,7 @@ XLogInsertRecord(XLogRecData *rdata,
 
 	/* cross-check on whether we should be here or not */
 	if (!XLogInsertAllowed())
-		elog(ERROR, "cannot make new WAL entries during recovery");
+		elog(ERROR, "cannot make new WAL entries now");
 
 	/*----------
 	 *
@@ -2859,9 +2859,11 @@ XLogFlush(XLogRecPtr record)
 	 * trying to flush the WAL, we should update minRecoveryPoint instead. We
 	 * test XLogInsertAllowed(), not InRecovery, because we need checkpointer
 	 * to act this way too, and because when it tries to write the
-	 * end-of-recovery checkpoint, it should indeed flush.
+	 * end-of-recovery checkpoint, it should indeed flush. Also, WAL prohibit
+	 * state should not restrict WAL flushing. Otherwise, the dirty buffer
+	 * cannot be evicted until WAL has been flushed up to the buffer's LSN.
 	 */
-	if (!XLogInsertAllowed())
+	if (!XLogInsertAllowed() && !IsWALProhibited())
 	{
 		UpdateMinRecoveryPoint(record, false);
 		return;
@@ -8832,6 +8834,8 @@ CreateCheckPoint(int flags)
 	/* sanity check */
 	if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
 		elog(ERROR, "can't create a checkpoint during recovery");
+	if (!XLogInsertAllowed() && !RecoveryInProgress())
+		elog(ERROR, "can't create a checkpoint while system is read only ");
 
 	/*
 	 * Initialize InitXLogInsert working areas before entering the critical
@@ -8861,6 +8865,8 @@ CreateCheckPoint(int flags)
 	MemSet(&CheckpointStats, 0, sizeof(CheckpointStats));
 	CheckpointStats.ckpt_start_t = GetCurrentTimestamp();
 
+	AssertWALPermitted();
+
 	/*
 	 * Use a critical section to force system panic if we have trouble.
 	 */
@@ -9089,6 +9095,8 @@ CreateCheckPoint(int flags)
 	if (!shutdown && XLogStandbyInfoActive())
 		LogStandbySnapshot();
 
+	AssertWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -9246,6 +9254,8 @@ CreateEndOfRecoveryRecord(void)
 
 	LocalSetXLogInsertAllowed();
 
+	AssertWALPermitted();
+
 	START_CRIT_SECTION();
 
 	XLogBeginInsert();
@@ -9879,7 +9889,7 @@ void
 UpdateFullPageWrites(void)
 {
 	XLogCtlInsert *Insert = &XLogCtl->Insert;
-	bool		recoveryInProgress;
+	bool		WALInsertAllowed;
 
 	/*
 	 * Do nothing if full_page_writes has not been changed.
@@ -9893,10 +9903,10 @@ UpdateFullPageWrites(void)
 
 	/*
 	 * Perform this outside critical section so that the WAL insert
-	 * initialization done by RecoveryInProgress() doesn't trigger an
+	 * initialization done by XLogInsertAllowed() doesn't trigger an
 	 * assertion failure.
 	 */
-	recoveryInProgress = RecoveryInProgress();
+	WALInsertAllowed = XLogInsertAllowed();
 
 	START_CRIT_SECTION();
 
@@ -9918,8 +9928,11 @@ UpdateFullPageWrites(void)
 	 * Write an XLOG_FPW_CHANGE record. This allows us to keep track of
 	 * full_page_writes during archive recovery, if required.
 	 */
-	if (XLogStandbyInfoActive() && !recoveryInProgress)
+	if (XLogStandbyInfoActive() && WALInsertAllowed)
 	{
+		/* Assured that WAL permission has been checked */
+		AssertWALPermitted();
+
 		XLogBeginInsert();
 		XLogRegisterData((char *) (&fullPageWrites), sizeof(bool));
 
diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c
index b21679f09eb..d69f6ca427a 100644
--- a/src/backend/access/transam/xloginsert.c
+++ b/src/backend/access/transam/xloginsert.c
@@ -19,6 +19,7 @@
 
 #include "postgres.h"
 
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xlog_internal.h"
@@ -124,9 +125,14 @@ XLogBeginInsert(void)
 	Assert(mainrdata_last == (XLogRecData *) &mainrdata_head);
 	Assert(mainrdata_len == 0);
 
+	/*
+	 * WAL permission must have checked before entering the critical section.
+	 * Otherwise, WAL prohibited error will force system panic.
+	 */
+	Assert(walpermit_checked_state != WALPERMIT_UNCHECKED || !CritSectionCount);
+
 	/* cross-check on whether we should be here or not */
-	if (!XLogInsertAllowed())
-		elog(ERROR, "cannot make new WAL entries during recovery");
+	CheckWALPermitted();
 
 	if (begininsert_called)
 		elog(ERROR, "XLogBeginInsert was already called");
@@ -204,6 +210,9 @@ XLogResetInsertion(void)
 	mainrdata_last = (XLogRecData *) &mainrdata_head;
 	curinsert_flags = 0;
 	begininsert_called = false;
+
+	/* Reset walpermit_checked flag */
+	RESET_WALPERMIT_CHECKED_STATE();
 }
 
 /*
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 6aab73bfd44..f961178b358 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -20,6 +20,7 @@
 #include "access/relation.h"
 #include "access/table.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -378,8 +379,13 @@ fill_seq_with_data(Relation rel, HeapTuple tuple)
 
 	/* check the comment above nextval_internal()'s equivalent call. */
 	if (RelationNeedsWAL(rel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermitted_HaveXID();
+	}
+
 	START_CRIT_SECTION();
 
 	MarkBufferDirty(buf);
@@ -766,8 +772,13 @@ nextval_internal(Oid relid, bool check_permissions)
 	 * (Have to do that here, so we're outside the critical section)
 	 */
 	if (logit && RelationNeedsWAL(seqrel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermitted_HaveXID();
+	}
+
 	/* ready to change the on-disk (or really, in-buffer) tuple */
 	START_CRIT_SECTION();
 
@@ -977,8 +988,13 @@ do_setval(Oid relid, int64 next, bool iscalled)
 
 	/* check the comment above nextval_internal()'s equivalent call. */
 	if (RelationNeedsWAL(seqrel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermitted_HaveXID();
+	}
+
 	/* ready to change the on-disk (or really, in-buffer) tuple */
 	START_CRIT_SECTION();
 
diff --git a/src/backend/commands/variable.c b/src/backend/commands/variable.c
index 484f7ea2c0e..acc34d2ad7c 100644
--- a/src/backend/commands/variable.c
+++ b/src/backend/commands/variable.c
@@ -501,11 +501,14 @@ check_transaction_read_only(bool *newval, void **extra, GucSource source)
 			GUC_check_errmsg("transaction read-write mode must be set before any query");
 			return false;
 		}
-		/* Can't go to r/w mode while recovery is still active */
-		if (RecoveryInProgress())
+		/*
+		 * Can't go to r/w mode while recovery is still active or while in WAL
+		 * prohibit state
+		 */
+		if (!XLogInsertAllowed())
 		{
 			GUC_check_errcode(ERRCODE_FEATURE_NOT_SUPPORTED);
-			GUC_check_errmsg("cannot set transaction read-write mode during recovery");
+			GUC_check_errmsg("cannot set transaction read-write mode while system is read only");
 			return false;
 		}
 	}
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index bbe62b73a08..7e3882a055c 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -934,6 +934,10 @@ RequestCheckpoint(int flags)
 	int			old_failed,
 				old_started;
 
+	/* The checkpoint is allowed in recovery but not in WAL prohibit state */
+	if (!RecoveryInProgress())
+		CheckWALPermitted();
+
 	/*
 	 * If in a standalone backend, just do it ourselves.
 	 */
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 29c920800a6..ba74ddcd249 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -3603,13 +3603,15 @@ MarkBufferDirtyHint(Buffer buffer, bool buffer_std)
 		{
 			/*
 			 * If we must not write WAL, due to a relfilenode-specific
-			 * condition or being in recovery, don't dirty the page.  We can
-			 * set the hint, just not dirty the page as a result so the hint
-			 * is lost when we evict the page or shutdown.
+			 * condition or in general, don't dirty the page.  We can
+			 * set the hint, but must not dirty the page as a result, lest
+			 * we trigger WAL generation. Unless the page is dirtied again
+			 * later, the hint will be lost when the page is evicted, or at
+			 * shutdown.
 			 *
 			 * See src/backend/storage/page/README for longer discussion.
 			 */
-			if (RecoveryInProgress() ||
+			if (!XLogInsertAllowed() ||
 				RelFileNodeSkippingWAL(bufHdr->tag.rnode))
 				return;
 
diff --git a/src/backend/storage/freespace/freespace.c b/src/backend/storage/freespace/freespace.c
index 95a21f6cc38..5faa69fabb9 100644
--- a/src/backend/storage/freespace/freespace.c
+++ b/src/backend/storage/freespace/freespace.c
@@ -24,6 +24,7 @@
 #include "postgres.h"
 
 #include "access/htup_details.h"
+#include "access/walprohibit.h"
 #include "access/xlogutils.h"
 #include "miscadmin.h"
 #include "storage/freespace.h"
@@ -285,12 +286,20 @@ FreeSpaceMapPrepareTruncateRel(Relation rel, BlockNumber nblocks)
 	 */
 	if (first_removed_slot > 0)
 	{
+		bool needwal;
+
 		buf = fsm_readbuf(rel, first_removed_address, false);
 		if (!BufferIsValid(buf))
 			return InvalidBlockNumber;	/* nothing to do; the FSM was already
 										 * smaller */
 		LockBuffer(buf, BUFFER_LOCK_EXCLUSIVE);
 
+		needwal = (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded());
+
+		/* Can reach here from VACUUM, so need not have an XID */
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -305,7 +314,7 @@ FreeSpaceMapPrepareTruncateRel(Relation rel, BlockNumber nblocks)
 		 * during recovery.
 		 */
 		MarkBufferDirty(buf);
-		if (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded())
+		if (needwal)
 			log_newpage_buffer(buf, false);
 
 		END_CRIT_SECTION();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 95989ce79bd..212312d5ae5 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -794,15 +794,15 @@ LockAcquireExtended(const LOCKTAG *locktag,
 	if (lockmode <= 0 || lockmode > lockMethodTable->numLockModes)
 		elog(ERROR, "unrecognized lock mode: %d", lockmode);
 
-	if (RecoveryInProgress() && !InRecovery &&
+	if (!XLogInsertAllowed() && !InRecovery &&
 		(locktag->locktag_type == LOCKTAG_OBJECT ||
 		 locktag->locktag_type == LOCKTAG_RELATION) &&
 		lockmode > RowExclusiveLock)
 		ereport(ERROR,
 				(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
-				 errmsg("cannot acquire lock mode %s on database objects while recovery is in progress",
+				 errmsg("cannot acquire lock mode %s on database objects while system is read only",
 						lockMethodTable->lockModeNames[lockmode]),
-				 errhint("Only RowExclusiveLock or less can be acquired on database objects during recovery.")));
+				 errhint("Only RowExclusiveLock or less can be acquired on database objects while system is read only")));
 
 #ifdef LOCK_DEBUG
 	if (LOCK_DEBUG_ENABLED(locktag))
diff --git a/src/backend/utils/cache/relmapper.c b/src/backend/utils/cache/relmapper.c
index 671fbb0ed5c..ec48073bbf1 100644
--- a/src/backend/utils/cache/relmapper.c
+++ b/src/backend/utils/cache/relmapper.c
@@ -43,6 +43,7 @@
 #include <sys/stat.h>
 #include <unistd.h>
 
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -841,6 +842,9 @@ write_relmap_file(bool shared, RelMapFile *newmap,
 		xl_relmap_update xlrec;
 		XLogRecPtr	lsn;
 
+		/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+		AssertWALPermitted_HaveXID();
+
 		/* now errors are fatal ... */
 		START_CRIT_SECTION();
 
diff --git a/src/include/access/walprohibit.h b/src/include/access/walprohibit.h
index 163fe0d2fce..1adcfc571d6 100644
--- a/src/include/access/walprohibit.h
+++ b/src/include/access/walprohibit.h
@@ -19,8 +19,8 @@ extern bool ProcessBarrierWALProhibit(void);
 extern void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
 
 /* WAL Prohibit States */
-#define	WALPROHIBIT_STATE_READ_WRITE		0x0000
-#define	WALPROHIBIT_STATE_READ_ONLY			0x0001
+#define	WALPROHIBIT_STATE_READ_WRITE		0x0000	/* WAL permitted */
+#define	WALPROHIBIT_STATE_READ_ONLY			0x0001	/* WAL prohibited */
 
 /*
  * The bit is used in state transition from one state to another.  When this
@@ -29,4 +29,49 @@ extern void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
  */
 #define WALPROHIBIT_TRANSITION_IN_PROGRESS	0x0002
 
+/* Never reaches when WAL is prohibited. */
+static inline void
+AssertWALPermitted(void)
+{
+	/*
+	 * Recovery in the startup process never is in wal prohibited state.
+	 */
+	Assert(InRecovery || XLogInsertAllowed());
+
+#ifdef USE_ASSERT_CHECKING
+	walpermit_checked_state = WALPERMIT_CHECKED;
+#endif
+}
+
+/*
+ * XID-bearing transactions are killed off by "ALTER SYSTEM READ ONLY", so any
+ * part of the code that can only be reached with an XID assigned is never
+ * reached when WAL is prohibited.
+ */
+static inline void
+AssertWALPermitted_HaveXID(void)
+{
+	Assert(FullTransactionIdIsValid(GetTopFullTransactionIdIfAny()));
+	AssertWALPermitted();
+}
+
+/*
+ * In opposite to the above assertion if a transaction doesn't have valid XID
+ * then it won't be killed while changing the system state to WAL prohibited.
+ * Therefore, we need to explicitly error out before entering into the critical
+ * section.
+ */
+static inline void
+CheckWALPermitted(void)
+{
+	if (!XLogInsertAllowed())
+		ereport(ERROR,
+				(errcode(ERRCODE_READ_ONLY_SQL_TRANSACTION),
+				 errmsg("system is now read only")));
+
+#ifdef USE_ASSERT_CHECKING
+	walpermit_checked_state = WALPERMIT_CHECKED;
+#endif
+}
+
 #endif		/* WALPROHIBIT_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 18bc8a7b904..63459305383 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -94,12 +94,37 @@ extern PGDLLIMPORT volatile uint32 CritSectionCount;
 /* in tcop/postgres.c */
 extern void ProcessInterrupts(void);
 
+#ifdef USE_ASSERT_CHECKING
+typedef enum
+{
+	WALPERMIT_UNCHECKED,
+	WALPERMIT_CHECKED,
+	WALPERMIT_CHECKED_AND_USED
+} WALPermitCheckState;
+
+/* in access/walprohibit.c */
+extern WALPermitCheckState walpermit_checked_state;
+
+/*
+ * Reset walpermit_checked flag when no longer in the critical section.
+ * Otherwise, marked checked and used.
+ */
+#define RESET_WALPERMIT_CHECKED_STATE() \
+do { \
+	walpermit_checked_state = CritSectionCount ? \
+	WALPERMIT_CHECKED_AND_USED : WALPERMIT_UNCHECKED; \
+} while(0)
+#else
+#define RESET_WALPERMIT_CHECKED_STATE() ((void) 0)
+#endif
+
 #ifndef WIN32
 
 #define CHECK_FOR_INTERRUPTS() \
 do { \
 	if (unlikely(InterruptPending)) \
 		ProcessInterrupts(); \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 #else							/* WIN32 */
 
@@ -109,6 +134,7 @@ do { \
 		pgwin32_dispatch_queued_signals(); \
 	if (unlikely(InterruptPending)) \
 		ProcessInterrupts(); \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 #endif							/* WIN32 */
 
@@ -135,6 +161,7 @@ do { \
 do { \
 	Assert(CritSectionCount > 0); \
 	CritSectionCount--; \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 
 
-- 
2.18.0

From bc61aef9b1749d503ab9c020f83d48c357f80122 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 27 Mar 2020 05:05:38 -0400
Subject: [PATCH v2 2/6] Add alter system read only/write syntax

Note that syntax doesn't have any implementation.
---
 src/backend/nodes/copyfuncs.c    | 12 ++++++++++++
 src/backend/nodes/equalfuncs.c   |  9 +++++++++
 src/backend/parser/gram.y        | 13 +++++++++++++
 src/backend/tcop/utility.c       | 20 ++++++++++++++++++++
 src/bin/psql/tab-complete.c      |  6 ++++--
 src/include/nodes/nodes.h        |  1 +
 src/include/nodes/parsenodes.h   | 10 ++++++++++
 src/tools/pgindent/typedefs.list |  1 +
 8 files changed, 70 insertions(+), 2 deletions(-)

diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index d8cf87e6d08..19aa6a2f88b 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -4020,6 +4020,15 @@ _copyAlterSystemStmt(const AlterSystemStmt *from)
 	return newnode;
 }
 
+static AlterSystemWALProhibitState *
+_copyAlterSystemWALProhibitState(const AlterSystemWALProhibitState *from)
+{
+	AlterSystemWALProhibitState *newnode = makeNode(AlterSystemWALProhibitState);
+
+	COPY_SCALAR_FIELD(WALProhibited);
+	return newnode;
+}
+
 static CreateSeqStmt *
 _copyCreateSeqStmt(const CreateSeqStmt *from)
 {
@@ -5406,6 +5415,9 @@ copyObjectImpl(const void *from)
 		case T_AlterSystemStmt:
 			retval = _copyAlterSystemStmt(from);
 			break;
+		case T_AlterSystemWALProhibitState:
+			retval = _copyAlterSystemWALProhibitState(from);
+			break;
 		case T_CreateSeqStmt:
 			retval = _copyCreateSeqStmt(from);
 			break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 627b026b195..01cedb38115 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1769,6 +1769,12 @@ _equalAlterSystemStmt(const AlterSystemStmt *a, const AlterSystemStmt *b)
 	return true;
 }
 
+static bool
+_equalAlterSystemWALProhibitState(const AlterSystemWALProhibitState *a, const AlterSystemWALProhibitState *b)
+{
+	COMPARE_SCALAR_FIELD(WALProhibited);
+	return true;
+}
 
 static bool
 _equalCreateSeqStmt(const CreateSeqStmt *a, const CreateSeqStmt *b)
@@ -3458,6 +3464,9 @@ equal(const void *a, const void *b)
 		case T_AlterSystemStmt:
 			retval = _equalAlterSystemStmt(a, b);
 			break;
+		case T_AlterSystemWALProhibitState:
+			retval = _equalAlterSystemWALProhibitState(a, b);
+			break;
 		case T_CreateSeqStmt:
 			retval = _equalCreateSeqStmt(a, b);
 			break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index e669d75a5af..f97bd6f658e 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -480,6 +480,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
 
 %type <vsetstmt> generic_set set_rest set_rest_more generic_reset reset_rest
 				 SetResetClause FunctionSetResetClause
+%type <boolean> system_readonly_state
 
 %type <node>	TableElement TypedTableElement ConstraintElem TableFuncElement
 %type <node>	columnDef columnOptions
@@ -10173,8 +10174,20 @@ AlterSystemStmt:
 					n->setstmt = $4;
 					$$ = (Node *)n;
 				}
+			| ALTER SYSTEM_P system_readonly_state
+				{
+					AlterSystemWALProhibitState *n = makeNode(AlterSystemWALProhibitState);
+					n->WALProhibited = $3;
+					$$ = (Node *)n;
+				}
 		;
 
+system_readonly_state:
+			 READ ONLY
+					{ $$ = true; }
+			| READ WRITE
+					{ $$ = false; }
+		;
 
 /*****************************************************************************
  *
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 97cbaa3072b..900088a2209 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -85,6 +85,7 @@ static void ProcessUtilitySlow(ParseState *pstate,
 							   DestReceiver *dest,
 							   QueryCompletion *qc);
 static void ExecDropStmt(DropStmt *stmt, bool isTopLevel);
+static void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
 
 /*
  * CommandIsReadOnly: is an executable query read-only?
@@ -219,6 +220,7 @@ ClassifyUtilityCommandAsReadOnly(Node *parsetree)
 				return COMMAND_IS_NOT_READ_ONLY;
 			}
 
+		case T_AlterSystemWALProhibitState:
 		case T_AlterSystemStmt:
 			{
 				/*
@@ -834,6 +836,11 @@ standard_ProcessUtility(PlannedStmt *pstmt,
 			AlterSystemSetConfigFile((AlterSystemStmt *) parsetree);
 			break;
 
+		case T_AlterSystemWALProhibitState:
+			PreventInTransactionBlock(isTopLevel, "ALTER SYSTEM");
+			AlterSystemSetWALProhibitState((AlterSystemWALProhibitState *) parsetree);
+			break;
+
 		case T_VariableSetStmt:
 			ExecSetVariableStmt((VariableSetStmt *) parsetree, isTopLevel);
 			break;
@@ -2772,6 +2779,7 @@ CreateCommandTag(Node *parsetree)
 			tag = CMDTAG_REFRESH_MATERIALIZED_VIEW;
 			break;
 
+		case T_AlterSystemWALProhibitState:
 		case T_AlterSystemStmt:
 			tag = CMDTAG_ALTER_SYSTEM;
 			break;
@@ -3636,3 +3644,15 @@ GetCommandLogLevel(Node *parsetree)
 
 	return lev;
 }
+
+/*
+ * AlterSystemSetWALProhibitState
+ *
+ * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
+ */
+static void
+AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
+{
+	/* some code */
+	elog(INFO, "AlterSystemSetWALProhibitState() called");
+}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index eb018854a5c..d586cf74816 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1858,9 +1858,11 @@ psql_completion(const char *text, int start, int end)
 	/* ALTER SERVER <name> VERSION <version> */
 	else if (Matches("ALTER", "SERVER", MatchAny, "VERSION", MatchAny))
 		COMPLETE_WITH("OPTIONS");
-	/* ALTER SYSTEM SET, RESET, RESET ALL */
+	/* ALTER SYSTEM READ, SET, RESET, RESET ALL */
 	else if (Matches("ALTER", "SYSTEM"))
-		COMPLETE_WITH("SET", "RESET");
+		COMPLETE_WITH("SET", "RESET", "READ");
+	else if (Matches("ALTER", "SYSTEM", "READ"))
+		COMPLETE_WITH("ONLY", "WRITE");
 	else if (Matches("ALTER", "SYSTEM", "SET|RESET"))
 		COMPLETE_WITH_QUERY(Query_for_list_of_alter_system_set_vars);
 	else if (Matches("ALTER", "SYSTEM", "SET", MatchAny))
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 381d84b4e4f..17d6942c734 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -412,6 +412,7 @@ typedef enum NodeTag
 	T_RefreshMatViewStmt,
 	T_ReplicaIdentityStmt,
 	T_AlterSystemStmt,
+	T_AlterSystemWALProhibitState,
 	T_CreatePolicyStmt,
 	T_AlterPolicyStmt,
 	T_CreateTransformStmt,
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 5e1ffafb91b..636654bb450 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -3194,6 +3194,16 @@ typedef struct AlterSystemStmt
 	VariableSetStmt *setstmt;	/* SET subcommand */
 } AlterSystemStmt;
 
+/* ----------------------
+ *		Alter System Read Statement
+ * ----------------------
+ */
+typedef struct AlterSystemWALProhibitState
+{
+	NodeTag		type;
+	bool		WALProhibited;
+} AlterSystemWALProhibitState;
+
 /* ----------------------
  *		Cluster Statement (support pbrown's cluster index implementation)
  * ----------------------
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index c65a55257dd..eb48b29828e 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -87,6 +87,7 @@ AlterStatsStmt
 AlterSubscriptionStmt
 AlterSubscriptionType
 AlterSystemStmt
+AlterSystemWALProhibitState
 AlterTSConfigType
 AlterTSConfigurationStmt
 AlterTSDictionaryStmt
-- 
2.18.0

From c31d946ff7e0531c68993d9ada89dcb54da07add Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 15 May 2020 06:39:43 -0400
Subject: [PATCH v2 4/6] Use checkpointer to make system READ-ONLY or
 READ-WRITE

Till the previous commit, the backend used to do this, but now the backend
requests checkpointer to do it. Checkpointer, noticing that the current state
is has WALPROHIBIT_TRANSITION_IN_PROGRESS flag set, does the barrier request,
and then acknowledges back to the backend who requested the state change.

Note that this commit also enables ALTER SYSTEM READ WRITE support and make WAL
prohibited state persistent across the system restarts.
---
 src/backend/access/transam/walprohibit.c |  26 ++++-
 src/backend/access/transam/xlog.c        |  77 +++++++++++++--
 src/backend/postmaster/checkpointer.c    | 116 +++++++++++++++++++++--
 src/backend/postmaster/pgstat.c          |   3 +
 src/bin/pg_controldata/pg_controldata.c  |   2 +
 src/include/access/walprohibit.h         |  11 +++
 src/include/access/xlog.h                |   3 +-
 src/include/catalog/pg_control.h         |   3 +
 src/include/pgstat.h                     |   1 +
 src/include/postmaster/bgwriter.h        |   2 +
 10 files changed, 222 insertions(+), 22 deletions(-)

diff --git a/src/backend/access/transam/walprohibit.c b/src/backend/access/transam/walprohibit.c
index df97596ddf9..a8cda2fafbc 100644
--- a/src/backend/access/transam/walprohibit.c
+++ b/src/backend/access/transam/walprohibit.c
@@ -30,6 +30,8 @@ ProcessBarrierWALProhibit(void)
 	 */
 	if (FullTransactionIdIsValid(GetTopFullTransactionIdIfAny()))
 	{
+		Assert(GetWALProhibitState() & WALPROHIBIT_STATE_READ_ONLY);
+
 		/*
 		 * XXX: Kill off the whole session by throwing FATAL instead of killing
 		 * transaction by throwing ERROR due to following reasons that need be
@@ -64,6 +66,8 @@ ProcessBarrierWALProhibit(void)
 void
 AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
 {
+	uint32			state;
+
 	if (!superuser())
 		ereport(ERROR,
 				(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
@@ -72,10 +76,22 @@ AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
 	/* Alter WAL prohibit state not allowed during recovery */
 	PreventCommandDuringRecovery("ALTER SYSTEM");
 
-	/* Yet to add ALTER SYTEM READ WRITE support */
-	if (!stmt->WALProhibited)
-		elog(ERROR, "XXX: Yet to implement");
+	/* Requested state */
+	state = stmt->WALProhibited ?
+		WALPROHIBIT_STATE_READ_ONLY : WALPROHIBIT_STATE_READ_WRITE;
+
+	/*
+	 * Since we yet to convey this WAL prohibit state to all backend mark it
+	 * in-progress.
+	 */
+	state |= WALPROHIBIT_TRANSITION_IN_PROGRESS;
+
+	if (!SetWALProhibitState(state))
+		return; /* server is already in the desired state */
 
-	MakeReadOnlyXLOG();
-	WaitForProcSignalBarrier(EmitProcSignalBarrier(PROCSIGNAL_BARRIER_WALPROHIBIT));
+	/*
+	 * Signal the checkpointer to do the actual state transition, and wait for
+	 * the state change to occur.
+	 */
+	WALProhibitRequest();
 }
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index ffa25b9d9b2..9eb4bf413eb 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -31,6 +31,7 @@
 #include "access/timeline.h"
 #include "access/transam.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog_internal.h"
 #include "access/xlogarchive.h"
@@ -661,10 +662,10 @@ typedef struct XLogCtlData
 	RecoveryState SharedRecoveryState;
 
 	/*
-	 * WALProhibited indicates if we have stopped allowing WAL writes.
+	 * SharedWALProhibitState indicates current WAL prohibit state.
 	 * Protected by info_lck.
 	 */
-	bool		WALProhibited;
+	uint32		SharedWALProhibitState;
 
 	/*
 	 * SharedHotStandbyActive indicates if we allow hot standby queries to be
@@ -7710,6 +7711,15 @@ StartupXLOG(void)
 	XLogCtl->LogwrtRqst.Write = EndOfLog;
 	XLogCtl->LogwrtRqst.Flush = EndOfLog;
 
+	/*
+	 * Before enabling WAL insertion update WAL prohibit state in shared memory
+	 * that will decide the further WAL insert should be allowed or not.
+	 */
+	SpinLockAcquire(&XLogCtl->info_lck);
+	XLogCtl->SharedWALProhibitState = ControlFile->wal_prohibited ?
+		WALPROHIBIT_STATE_READ_ONLY : WALPROHIBIT_STATE_READ_WRITE;
+	SpinLockRelease(&XLogCtl->info_lck);
+
 	/*
 	 * Update full_page_writes in shared memory and write an XLOG_FPW_CHANGE
 	 * record before resource manager writes cleanup WAL records or checkpoint
@@ -7720,7 +7730,15 @@ StartupXLOG(void)
 	UpdateFullPageWrites();
 	LocalXLogInsertAllowed = -1;
 
-	if (InRecovery)
+	/*
+	 * Skip end-of-recovery checkpoint if the system is in WAL prohibited state.
+	 */
+	if (ControlFile->wal_prohibited && InRecovery)
+	{
+		ereport(LOG,
+				(errmsg("skipping startup checkpoint because the system is read only")));
+	}
+	else if (InRecovery)
 	{
 		/*
 		 * Perform a checkpoint to update all our recovery activity to disk.
@@ -7966,12 +7984,54 @@ StartupXLOG(void)
 		RequestCheckpoint(CHECKPOINT_FORCE);
 }
 
-void
-MakeReadOnlyXLOG(void)
+/* Atomically return the current server WAL prohibited state */
+uint32
+GetWALProhibitState(void)
 {
+	uint32		state;
+
 	SpinLockAcquire(&XLogCtl->info_lck);
-	XLogCtl->WALProhibited = true;
+	state = XLogCtl->SharedWALProhibitState;
 	SpinLockRelease(&XLogCtl->info_lck);
+
+	return state;
+}
+
+/*
+ * SetWALProhibitState: Change current wal prohibit state to the input state.
+ *
+ * If the server is already completely moved to the requested WAL prohibit
+ * state, or if the desired state is same as the current state, return false,
+ * indicating that the server state did not change. Else return true.
+ */
+bool
+SetWALProhibitState(uint32 new_state)
+{
+	uint32		cur_state;
+
+	cur_state = GetWALProhibitState();
+
+	if (new_state == cur_state ||
+		new_state == (cur_state | WALPROHIBIT_TRANSITION_IN_PROGRESS))
+		return false;
+
+	/* Update new state in share memory */
+	SpinLockAcquire(&XLogCtl->info_lck);
+	XLogCtl->SharedWALProhibitState = new_state;
+	SpinLockRelease(&XLogCtl->info_lck);
+
+	/* Update control file if it is the final state */
+	if (!(new_state & WALPROHIBIT_TRANSITION_IN_PROGRESS))
+	{
+		bool	wal_prohibited = (new_state & WALPROHIBIT_STATE_READ_ONLY) != 0;
+
+		LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+		ControlFile->wal_prohibited = wal_prohibited;
+		UpdateControlFile();
+		LWLockRelease(ControlFileLock);
+	}
+
+	return true;
 }
 
 /*
@@ -7980,9 +8040,7 @@ MakeReadOnlyXLOG(void)
 bool
 IsWALProhibited(void)
 {
-	volatile XLogCtlData *xlogctl = XLogCtl;
-
-	return xlogctl->WALProhibited;
+	return (GetWALProhibitState() & WALPROHIBIT_STATE_READ_ONLY) != 0;
 }
 
 /*
@@ -8250,7 +8308,6 @@ static void
 LocalSetXLogInsertAllowed(void)
 {
 	Assert(LocalXLogInsertAllowed == -1);
-	Assert(!IsWALProhibited());
 
 	LocalXLogInsertAllowed = 1;
 
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 5e5e56d4eec..bbe62b73a08 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -39,6 +39,7 @@
 #include <sys/time.h>
 #include <time.h>
 
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xlog_internal.h"
 #include "libpq/pqsignal.h"
@@ -127,6 +128,8 @@ typedef struct
 	ConditionVariable start_cv; /* signaled when ckpt_started advances */
 	ConditionVariable done_cv;	/* signaled when ckpt_done advances */
 
+	ConditionVariable readonly_cv; /* signaled when ckpt_started advances */
+
 	uint32		num_backend_writes; /* counts user backend buffer writes */
 	uint32		num_backend_fsync;	/* counts user backend fsync calls */
 
@@ -168,6 +171,7 @@ static bool IsCheckpointOnSchedule(double progress);
 static bool ImmediateCheckpointRequested(void);
 static bool CompactCheckpointerRequestQueue(void);
 static void UpdateSharedMemoryConfig(void);
+static void performWALProhibitStateChange(uint32 wal_state);
 
 /* Signal handlers */
 static void ReqCheckpointHandler(SIGNAL_ARGS);
@@ -332,6 +336,7 @@ CheckpointerMain(void)
 		pg_time_t	now;
 		int			elapsed_secs;
 		int			cur_timeout;
+		uint32		wal_state;
 
 		/* Clear any already-pending wakeups */
 		ResetLatch(MyLatch);
@@ -342,18 +347,28 @@ CheckpointerMain(void)
 		AbsorbSyncRequests();
 		HandleCheckpointerInterrupts();
 
-		/*
-		 * If the server is in WAL-Prohibited state then don't do anything until
-		 * someone wakes us up. E.g. a backend might later on request us to put
-		 * the system back to read-write.
-		 */
-		if (IsWALProhibited())
+		wal_state = GetWALProhibitState();
+
+		if (wal_state & WALPROHIBIT_TRANSITION_IN_PROGRESS)
+		{
+			/* Complete WAL prohibit state change request */
+			performWALProhibitStateChange(wal_state);
+			continue;
+		}
+		else if (wal_state & WALPROHIBIT_STATE_READ_ONLY)
 		{
+			/*
+			 * Don't do anything until someone wakes us up.  For example a
+			 * backend might later on request us to put the system back to
+			 * read-write wal prohibit sate.
+			 */
 			(void) WaitLatch(MyLatch, WL_LATCH_SET | WL_EXIT_ON_PM_DEATH, -1,
 							 WAIT_EVENT_CHECKPOINTER_MAIN);
 			continue;
 		}
 
+		Assert(wal_state == WALPROHIBIT_STATE_READ_WRITE);
+
 		/*
 		 * Detect a pending checkpoint request by checking whether the flags
 		 * word in shared memory is nonzero.  We shouldn't need to acquire the
@@ -891,6 +906,7 @@ CheckpointerShmemInit(void)
 		CheckpointerShmem->max_requests = NBuffers;
 		ConditionVariableInit(&CheckpointerShmem->start_cv);
 		ConditionVariableInit(&CheckpointerShmem->done_cv);
+		ConditionVariableInit(&CheckpointerShmem->readonly_cv);
 	}
 }
 
@@ -1121,6 +1137,94 @@ ForwardSyncRequest(const FileTag *ftag, SyncRequestType type)
 	return true;
 }
 
+/*
+ * WALProhibitedRequest: Request checkpointer to make the WALProhibitState to
+ * read-only.
+ */
+void
+WALProhibitRequest(void)
+{
+	/* Must not be called from checkpointer */
+	Assert(!AmCheckpointerProcess());
+	Assert(GetWALProhibitState() & WALPROHIBIT_TRANSITION_IN_PROGRESS);
+
+	/*
+	 * If in a standalone backend, just do it ourselves.
+	 */
+	if (!IsPostmasterEnvironment)
+	{
+		performWALProhibitStateChange(GetWALProhibitState());
+		return;
+	}
+
+	if (CheckpointerShmem->checkpointer_pid == 0)
+		elog(ERROR, "checkpointer is not running");
+
+	if (kill(CheckpointerShmem->checkpointer_pid, SIGINT) != 0)
+		elog(ERROR, "could not signal checkpointer: %m");
+
+	/* Wait for the state to change to read-only */
+	ConditionVariablePrepareToSleep(&CheckpointerShmem->readonly_cv);
+	for (;;)
+	{
+		/*  We'll be done once in-progress flag bit is cleared */
+		if (!(GetWALProhibitState() & WALPROHIBIT_TRANSITION_IN_PROGRESS))
+			break;
+
+		elog(DEBUG1, "WALProhibitRequest: Waiting for checkpointer");
+		ConditionVariableSleep(&CheckpointerShmem->readonly_cv,
+							   WAIT_EVENT_SYSTEM_WALPROHIBIT_STATE_CHANGE);
+	}
+	ConditionVariableCancelSleep();
+	elog(DEBUG1, "Done WALProhibitRequest");
+}
+
+/*
+ * performWALProhibitStateChange: checkpointer will call this to complete
+ * the requested WAL prohibit state transition.
+ */
+static void
+performWALProhibitStateChange(uint32 wal_state)
+{
+	uint64		barrierGeneration;
+
+	/*
+	 * Must be called from checkpointer. Otherwise, it must be single-user
+	 * backend.
+	 */
+	Assert(AmCheckpointerProcess() || !IsPostmasterEnvironment);
+	Assert(wal_state & WALPROHIBIT_TRANSITION_IN_PROGRESS);
+
+	/*
+	 * WAL prohibit state change is initiated. We need to complete the state
+	 * transition by setting requested WAL prohibit state in all backends.
+	 */
+	elog(DEBUG1, "waiting for backends to adopt requested WAL prohibit state");
+
+	/* Emit global barrier */
+	barrierGeneration = EmitProcSignalBarrier(PROCSIGNAL_BARRIER_WALPROHIBIT);
+	WaitForProcSignalBarrier(barrierGeneration);
+
+	/* And flush all writes. */
+	XLogFlush(GetXLogWriteRecPtr());
+
+	/* Set final state by clearing in-progress flag bit */
+	if (SetWALProhibitState(wal_state & ~(WALPROHIBIT_TRANSITION_IN_PROGRESS)))
+	{
+		if ((wal_state & WALPROHIBIT_STATE_READ_ONLY) != 0)
+			ereport(LOG, (errmsg("system is now read only")));
+		else
+		{
+			/* Request checkpoint */
+			RequestCheckpoint(CHECKPOINT_IMMEDIATE);
+			ereport(LOG, (errmsg("system is now read write")));
+		}
+	}
+
+	/* Wake up the backend who requested the state change */
+	ConditionVariableBroadcast(&CheckpointerShmem->readonly_cv);
+}
+
 /*
  * CompactCheckpointerRequestQueue
  *		Remove duplicates from the request queue to avoid backend fsyncs.
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index c022597bc09..2b8b65f5628 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -4057,6 +4057,9 @@ pgstat_get_wait_io(WaitEventIO w)
 		case WAIT_EVENT_REPLICATION_SLOT_WRITE:
 			event_name = "ReplicationSlotWrite";
 			break;
+		case WAIT_EVENT_SYSTEM_WALPROHIBIT_STATE_CHANGE:
+			event_name = "SystemWALProhibitStateChange";
+			break;
 		case WAIT_EVENT_SLRU_FLUSH_SYNC:
 			event_name = "SLRUFlushSync";
 			break;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index e73639df744..9594df76946 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -290,6 +290,8 @@ main(int argc, char *argv[])
 		   (uint32) ControlFile->backupEndPoint);
 	printf(_("End-of-backup record required:        %s\n"),
 		   ControlFile->backupEndRequired ? _("yes") : _("no"));
+	printf(_("WAL write prohibited:                 %s\n"),
+		   ControlFile->wal_prohibited ? _("yes") : _("no"));
 	printf(_("wal_level setting:                    %s\n"),
 		   wal_level_str(ControlFile->wal_level));
 	printf(_("wal_log_hints setting:                %s\n"),
diff --git a/src/include/access/walprohibit.h b/src/include/access/walprohibit.h
index 619c33cd780..163fe0d2fce 100644
--- a/src/include/access/walprohibit.h
+++ b/src/include/access/walprohibit.h
@@ -18,4 +18,15 @@
 extern bool ProcessBarrierWALProhibit(void);
 extern void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
 
+/* WAL Prohibit States */
+#define	WALPROHIBIT_STATE_READ_WRITE		0x0000
+#define	WALPROHIBIT_STATE_READ_ONLY			0x0001
+
+/*
+ * The bit is used in state transition from one state to another.  When this
+ * bit is set then the state indicated by the 0th position bit is yet to
+ * confirmed.
+ */
+#define WALPROHIBIT_TRANSITION_IN_PROGRESS	0x0002
+
 #endif		/* WALPROHIBIT_H */
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 6e98e27b2c8..6555242e3ee 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -324,7 +324,8 @@ extern void XLOGShmemInit(void);
 extern void BootStrapXLOG(void);
 extern void LocalProcessControlFile(bool reset);
 extern void StartupXLOG(void);
-extern void MakeReadOnlyXLOG(void);
+extern uint32 GetWALProhibitState(void);
+extern bool SetWALProhibitState(uint32 new_state);
 extern void ShutdownXLOG(int code, Datum arg);
 extern void InitXLOGAccess(void);
 extern void CreateCheckPoint(int flags);
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index de5670e5382..b32c7723275 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -182,6 +182,9 @@ typedef struct ControlFileData
 	int			max_locks_per_xact;
 	bool		track_commit_timestamp;
 
+	/* WAL prohibited determines if the WAL insert is allowed or not. */
+	bool		wal_prohibited;
+
 	/*
 	 * This data is used to check for hardware-architecture compatibility of
 	 * the database and the backend executable.  We need not check endianness
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 13872013823..780c59f3e48 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -955,6 +955,7 @@ typedef enum
 	WAIT_EVENT_REPLICATION_SLOT_RESTORE_SYNC,
 	WAIT_EVENT_REPLICATION_SLOT_SYNC,
 	WAIT_EVENT_REPLICATION_SLOT_WRITE,
+	WAIT_EVENT_SYSTEM_WALPROHIBIT_STATE_CHANGE,
 	WAIT_EVENT_SLRU_FLUSH_SYNC,
 	WAIT_EVENT_SLRU_READ,
 	WAIT_EVENT_SLRU_SYNC,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e6..e8271b49f6d 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -35,6 +35,8 @@ extern void CheckpointWriteDelay(int flags, double progress);
 
 extern bool ForwardSyncRequest(const FileTag *ftag, SyncRequestType type);
 
+extern void WALProhibitRequest(void);
+
 extern void AbsorbSyncRequests(void);
 
 extern Size CheckpointerShmemSize(void);
-- 
2.18.0

From 94b2f05c121ea4bb3198dbfcc34bd521f5902acc Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 19 Jun 2020 06:27:53 -0400
Subject: [PATCH v3 1/6] Allow error or refusal while absorbing barriers.

Patch by Robert Haas
---
 src/backend/storage/ipc/procsignal.c | 75 +++++++++++++++++++++++-----
 1 file changed, 63 insertions(+), 12 deletions(-)

diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ece..13648887187 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -87,12 +87,16 @@ typedef struct
 #define BARRIER_SHOULD_CHECK(flags, type) \
 	(((flags) & (((uint32) 1) << (uint32) (type))) != 0)
 
+/* Clear the relevant type bit from the flags. */
+#define BARRIER_CLEAR_BIT(flags, type) \
+	((flags) &= ~(((uint32) 1) << (uint32) (type)))
+
 static ProcSignalHeader *ProcSignal = NULL;
 static volatile ProcSignalSlot *MyProcSignalSlot = NULL;
 
 static bool CheckProcSignal(ProcSignalReason reason);
 static void CleanupProcSignalState(int status, Datum arg);
-static void ProcessBarrierPlaceholder(void);
+static bool ProcessBarrierPlaceholder(void);
 
 /*
  * ProcSignalShmemSize
@@ -486,17 +490,59 @@ ProcessProcSignalBarrier(void)
 	flags = pg_atomic_exchange_u32(&MyProcSignalSlot->pss_barrierCheckMask, 0);
 
 	/*
-	 * Process each type of barrier. It's important that nothing we call from
-	 * here throws an error, because pss_barrierCheckMask has already been
-	 * cleared. If we jumped out of here before processing all barrier types,
-	 * then we'd forget about the need to do so later.
-	 *
-	 * NB: It ought to be OK to call the barrier-processing functions
-	 * unconditionally, but it's more efficient to call only the ones that
-	 * might need us to do something based on the flags.
+	 * If there are no flags set, then we can skip doing any real work.
+	 * Otherwise, establish a PG_TRY block, so that we don't lose track of
+	 * which types of barrier processing are needed if an ERROR occurs.
 	 */
-	if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_PLACEHOLDER))
-		ProcessBarrierPlaceholder();
+	if (flags != 0)
+	{
+		PG_TRY();
+		{
+			/*
+			 * Process each type of barrier. The barrier-processing functions
+			 * should normally return true, but may return false if the barrier
+			 * can't be absorbed at the current time. This should be rare,
+			 * because it's pretty expensive.  Every single
+			 * CHECK_FOR_INTERRUPTS() will return here until we manage to
+			 * absorb the barrier, and that cost will add up in a hurry.
+			 *
+			 * NB: It ought to be OK to call the barrier-processing functions
+			 * unconditionally, but it's more efficient to call only the ones
+			 * that might need us to do something based on the flags.
+			 */
+			if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_PLACEHOLDER)
+				&& ProcessBarrierPlaceholder())
+				BARRIER_CLEAR_BIT(flags, PROCSIGNAL_BARRIER_PLACEHOLDER);
+		}
+		PG_CATCH();
+		{
+			/*
+			 * If an ERROR occurred, add any flags that weren't yet handled
+			 * back into pss_barrierCheckMask, and reset the global variables
+			 * so that we try again the next time we check for interrupts.
+			 */
+			pg_atomic_fetch_or_u32(&MyProcSignalSlot->pss_barrierCheckMask,
+								   flags);
+			ProcSignalBarrierPending = true;
+			InterruptPending = true;
+
+			PG_RE_THROW();
+		}
+		PG_END_TRY();
+
+		/*
+		 * If some barrier was not successfully absorbed, we will have to try
+		 * again later.
+		 */
+		if (flags != 0)
+		{
+			pg_atomic_fetch_or_u32(&MyProcSignalSlot->pss_barrierCheckMask,
+								   flags);
+			ProcSignalBarrierPending = true;
+			InterruptPending = true;
+			return;
+		}
+	}
 
 	/*
 	 * State changes related to all types of barriers that might have been
@@ -508,7 +554,7 @@ ProcessProcSignalBarrier(void)
 	pg_atomic_write_u64(&MyProcSignalSlot->pss_barrierGeneration, shared_gen);
 }
 
-static void
+static bool
 ProcessBarrierPlaceholder(void)
 {
 	/*
@@ -518,7 +564,12 @@ ProcessBarrierPlaceholder(void)
 	 * appropriately descriptive. Get rid of this function and instead have
 	 * ProcessBarrierSomethingElse. Most likely, that function should live in
 	 * the file pertaining to that subsystem, rather than here.
+	 *
+	 * The return value should be 'true' if the barrier was successfully
+	 * absorbed and 'false' if not. Note that returning 'false' can lead to
+	 * very frequent retries, so try hard to make that an uncommon case.
 	 */
+	return true;
 }
 
 /*
-- 
2.22.0

From 5600adc647bd729e4074ecf13e97b9f297e9d5c6 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 15 May 2020 06:39:43 -0400
Subject: [PATCH v3 4/6] Use checkpointer to make system READ-ONLY or
 READ-WRITE

Till the previous commit, the backend used to do this, but now the backend
requests checkpointer to do it. Checkpointer, noticing that the current state
is has WALPROHIBIT_TRANSITION_IN_PROGRESS flag set, does the barrier request,
and then acknowledges back to the backend who requested the state change.

Note that this commit also enables ALTER SYSTEM READ WRITE support and make WAL
prohibited state persistent across the system restarts.
---
 src/backend/access/transam/walprohibit.c |  26 ++++-
 src/backend/access/transam/xlog.c        |  77 +++++++++++++--
 src/backend/postmaster/checkpointer.c    | 116 +++++++++++++++++++++--
 src/backend/postmaster/pgstat.c          |   3 +
 src/bin/pg_controldata/pg_controldata.c  |   2 +
 src/include/access/walprohibit.h         |  11 +++
 src/include/access/xlog.h                |   3 +-
 src/include/catalog/pg_control.h         |   3 +
 src/include/pgstat.h                     |   1 +
 src/include/postmaster/bgwriter.h        |   2 +
 10 files changed, 222 insertions(+), 22 deletions(-)

diff --git a/src/backend/access/transam/walprohibit.c b/src/backend/access/transam/walprohibit.c
index df97596ddf9..a8cda2fafbc 100644
--- a/src/backend/access/transam/walprohibit.c
+++ b/src/backend/access/transam/walprohibit.c
@@ -30,6 +30,8 @@ ProcessBarrierWALProhibit(void)
 	 */
 	if (FullTransactionIdIsValid(GetTopFullTransactionIdIfAny()))
 	{
+		Assert(GetWALProhibitState() & WALPROHIBIT_STATE_READ_ONLY);
+
 		/*
 		 * XXX: Kill off the whole session by throwing FATAL instead of killing
 		 * transaction by throwing ERROR due to following reasons that need be
@@ -64,6 +66,8 @@ ProcessBarrierWALProhibit(void)
 void
 AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
 {
+	uint32			state;
+
 	if (!superuser())
 		ereport(ERROR,
 				(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
@@ -72,10 +76,22 @@ AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
 	/* Alter WAL prohibit state not allowed during recovery */
 	PreventCommandDuringRecovery("ALTER SYSTEM");
 
-	/* Yet to add ALTER SYTEM READ WRITE support */
-	if (!stmt->WALProhibited)
-		elog(ERROR, "XXX: Yet to implement");
+	/* Requested state */
+	state = stmt->WALProhibited ?
+		WALPROHIBIT_STATE_READ_ONLY : WALPROHIBIT_STATE_READ_WRITE;
+
+	/*
+	 * Since we yet to convey this WAL prohibit state to all backend mark it
+	 * in-progress.
+	 */
+	state |= WALPROHIBIT_TRANSITION_IN_PROGRESS;
+
+	if (!SetWALProhibitState(state))
+		return; /* server is already in the desired state */
 
-	MakeReadOnlyXLOG();
-	WaitForProcSignalBarrier(EmitProcSignalBarrier(PROCSIGNAL_BARRIER_WALPROHIBIT));
+	/*
+	 * Signal the checkpointer to do the actual state transition, and wait for
+	 * the state change to occur.
+	 */
+	WALProhibitRequest();
 }
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index c2d6d19716c..64cc347caa2 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -31,6 +31,7 @@
 #include "access/timeline.h"
 #include "access/transam.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog_internal.h"
 #include "access/xlogarchive.h"
@@ -662,10 +663,10 @@ typedef struct XLogCtlData
 	RecoveryState SharedRecoveryState;
 
 	/*
-	 * WALProhibited indicates if we have stopped allowing WAL writes.
+	 * SharedWALProhibitState indicates current WAL prohibit state.
 	 * Protected by info_lck.
 	 */
-	bool		WALProhibited;
+	uint32		SharedWALProhibitState;
 
 	/*
 	 * SharedHotStandbyActive indicates if we allow hot standby queries to be
@@ -7713,6 +7714,15 @@ StartupXLOG(void)
 	XLogCtl->LogwrtRqst.Write = EndOfLog;
 	XLogCtl->LogwrtRqst.Flush = EndOfLog;
 
+	/*
+	 * Before enabling WAL insertion update WAL prohibit state in shared memory
+	 * that will decide the further WAL insert should be allowed or not.
+	 */
+	SpinLockAcquire(&XLogCtl->info_lck);
+	XLogCtl->SharedWALProhibitState = ControlFile->wal_prohibited ?
+		WALPROHIBIT_STATE_READ_ONLY : WALPROHIBIT_STATE_READ_WRITE;
+	SpinLockRelease(&XLogCtl->info_lck);
+
 	/*
 	 * Update full_page_writes in shared memory and write an XLOG_FPW_CHANGE
 	 * record before resource manager writes cleanup WAL records or checkpoint
@@ -7723,7 +7733,15 @@ StartupXLOG(void)
 	UpdateFullPageWrites();
 	LocalXLogInsertAllowed = -1;
 
-	if (InRecovery)
+	/*
+	 * Skip end-of-recovery checkpoint if the system is in WAL prohibited state.
+	 */
+	if (ControlFile->wal_prohibited && InRecovery)
+	{
+		ereport(LOG,
+				(errmsg("skipping startup checkpoint because the system is read only")));
+	}
+	else if (InRecovery)
 	{
 		/*
 		 * Perform a checkpoint to update all our recovery activity to disk.
@@ -7969,12 +7987,54 @@ StartupXLOG(void)
 		RequestCheckpoint(CHECKPOINT_FORCE);
 }
 
-void
-MakeReadOnlyXLOG(void)
+/* Atomically return the current server WAL prohibited state */
+uint32
+GetWALProhibitState(void)
 {
+	uint32		state;
+
 	SpinLockAcquire(&XLogCtl->info_lck);
-	XLogCtl->WALProhibited = true;
+	state = XLogCtl->SharedWALProhibitState;
 	SpinLockRelease(&XLogCtl->info_lck);
+
+	return state;
+}
+
+/*
+ * SetWALProhibitState: Change current wal prohibit state to the input state.
+ *
+ * If the server is already completely moved to the requested WAL prohibit
+ * state, or if the desired state is same as the current state, return false,
+ * indicating that the server state did not change. Else return true.
+ */
+bool
+SetWALProhibitState(uint32 new_state)
+{
+	uint32		cur_state;
+
+	cur_state = GetWALProhibitState();
+
+	if (new_state == cur_state ||
+		new_state == (cur_state | WALPROHIBIT_TRANSITION_IN_PROGRESS))
+		return false;
+
+	/* Update new state in share memory */
+	SpinLockAcquire(&XLogCtl->info_lck);
+	XLogCtl->SharedWALProhibitState = new_state;
+	SpinLockRelease(&XLogCtl->info_lck);
+
+	/* Update control file if it is the final state */
+	if (!(new_state & WALPROHIBIT_TRANSITION_IN_PROGRESS))
+	{
+		bool	wal_prohibited = (new_state & WALPROHIBIT_STATE_READ_ONLY) != 0;
+
+		LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+		ControlFile->wal_prohibited = wal_prohibited;
+		UpdateControlFile();
+		LWLockRelease(ControlFileLock);
+	}
+
+	return true;
 }
 
 /*
@@ -7983,9 +8043,7 @@ MakeReadOnlyXLOG(void)
 bool
 IsWALProhibited(void)
 {
-	volatile XLogCtlData *xlogctl = XLogCtl;
-
-	return xlogctl->WALProhibited;
+	return (GetWALProhibitState() & WALPROHIBIT_STATE_READ_ONLY) != 0;
 }
 
 /*
@@ -8253,7 +8311,6 @@ static void
 LocalSetXLogInsertAllowed(void)
 {
 	Assert(LocalXLogInsertAllowed == -1);
-	Assert(!IsWALProhibited());
 
 	LocalXLogInsertAllowed = 1;
 
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 5e5e56d4eec..bbe62b73a08 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -39,6 +39,7 @@
 #include <sys/time.h>
 #include <time.h>
 
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xlog_internal.h"
 #include "libpq/pqsignal.h"
@@ -127,6 +128,8 @@ typedef struct
 	ConditionVariable start_cv; /* signaled when ckpt_started advances */
 	ConditionVariable done_cv;	/* signaled when ckpt_done advances */
 
+	ConditionVariable readonly_cv; /* signaled when ckpt_started advances */
+
 	uint32		num_backend_writes; /* counts user backend buffer writes */
 	uint32		num_backend_fsync;	/* counts user backend fsync calls */
 
@@ -168,6 +171,7 @@ static bool IsCheckpointOnSchedule(double progress);
 static bool ImmediateCheckpointRequested(void);
 static bool CompactCheckpointerRequestQueue(void);
 static void UpdateSharedMemoryConfig(void);
+static void performWALProhibitStateChange(uint32 wal_state);
 
 /* Signal handlers */
 static void ReqCheckpointHandler(SIGNAL_ARGS);
@@ -332,6 +336,7 @@ CheckpointerMain(void)
 		pg_time_t	now;
 		int			elapsed_secs;
 		int			cur_timeout;
+		uint32		wal_state;
 
 		/* Clear any already-pending wakeups */
 		ResetLatch(MyLatch);
@@ -342,18 +347,28 @@ CheckpointerMain(void)
 		AbsorbSyncRequests();
 		HandleCheckpointerInterrupts();
 
-		/*
-		 * If the server is in WAL-Prohibited state then don't do anything until
-		 * someone wakes us up. E.g. a backend might later on request us to put
-		 * the system back to read-write.
-		 */
-		if (IsWALProhibited())
+		wal_state = GetWALProhibitState();
+
+		if (wal_state & WALPROHIBIT_TRANSITION_IN_PROGRESS)
+		{
+			/* Complete WAL prohibit state change request */
+			performWALProhibitStateChange(wal_state);
+			continue;
+		}
+		else if (wal_state & WALPROHIBIT_STATE_READ_ONLY)
 		{
+			/*
+			 * Don't do anything until someone wakes us up.  For example a
+			 * backend might later on request us to put the system back to
+			 * read-write wal prohibit sate.
+			 */
 			(void) WaitLatch(MyLatch, WL_LATCH_SET | WL_EXIT_ON_PM_DEATH, -1,
 							 WAIT_EVENT_CHECKPOINTER_MAIN);
 			continue;
 		}
 
+		Assert(wal_state == WALPROHIBIT_STATE_READ_WRITE);
+
 		/*
 		 * Detect a pending checkpoint request by checking whether the flags
 		 * word in shared memory is nonzero.  We shouldn't need to acquire the
@@ -891,6 +906,7 @@ CheckpointerShmemInit(void)
 		CheckpointerShmem->max_requests = NBuffers;
 		ConditionVariableInit(&CheckpointerShmem->start_cv);
 		ConditionVariableInit(&CheckpointerShmem->done_cv);
+		ConditionVariableInit(&CheckpointerShmem->readonly_cv);
 	}
 }
 
@@ -1121,6 +1137,94 @@ ForwardSyncRequest(const FileTag *ftag, SyncRequestType type)
 	return true;
 }
 
+/*
+ * WALProhibitedRequest: Request checkpointer to make the WALProhibitState to
+ * read-only.
+ */
+void
+WALProhibitRequest(void)
+{
+	/* Must not be called from checkpointer */
+	Assert(!AmCheckpointerProcess());
+	Assert(GetWALProhibitState() & WALPROHIBIT_TRANSITION_IN_PROGRESS);
+
+	/*
+	 * If in a standalone backend, just do it ourselves.
+	 */
+	if (!IsPostmasterEnvironment)
+	{
+		performWALProhibitStateChange(GetWALProhibitState());
+		return;
+	}
+
+	if (CheckpointerShmem->checkpointer_pid == 0)
+		elog(ERROR, "checkpointer is not running");
+
+	if (kill(CheckpointerShmem->checkpointer_pid, SIGINT) != 0)
+		elog(ERROR, "could not signal checkpointer: %m");
+
+	/* Wait for the state to change to read-only */
+	ConditionVariablePrepareToSleep(&CheckpointerShmem->readonly_cv);
+	for (;;)
+	{
+		/*  We'll be done once in-progress flag bit is cleared */
+		if (!(GetWALProhibitState() & WALPROHIBIT_TRANSITION_IN_PROGRESS))
+			break;
+
+		elog(DEBUG1, "WALProhibitRequest: Waiting for checkpointer");
+		ConditionVariableSleep(&CheckpointerShmem->readonly_cv,
+							   WAIT_EVENT_SYSTEM_WALPROHIBIT_STATE_CHANGE);
+	}
+	ConditionVariableCancelSleep();
+	elog(DEBUG1, "Done WALProhibitRequest");
+}
+
+/*
+ * performWALProhibitStateChange: checkpointer will call this to complete
+ * the requested WAL prohibit state transition.
+ */
+static void
+performWALProhibitStateChange(uint32 wal_state)
+{
+	uint64		barrierGeneration;
+
+	/*
+	 * Must be called from checkpointer. Otherwise, it must be single-user
+	 * backend.
+	 */
+	Assert(AmCheckpointerProcess() || !IsPostmasterEnvironment);
+	Assert(wal_state & WALPROHIBIT_TRANSITION_IN_PROGRESS);
+
+	/*
+	 * WAL prohibit state change is initiated. We need to complete the state
+	 * transition by setting requested WAL prohibit state in all backends.
+	 */
+	elog(DEBUG1, "waiting for backends to adopt requested WAL prohibit state");
+
+	/* Emit global barrier */
+	barrierGeneration = EmitProcSignalBarrier(PROCSIGNAL_BARRIER_WALPROHIBIT);
+	WaitForProcSignalBarrier(barrierGeneration);
+
+	/* And flush all writes. */
+	XLogFlush(GetXLogWriteRecPtr());
+
+	/* Set final state by clearing in-progress flag bit */
+	if (SetWALProhibitState(wal_state & ~(WALPROHIBIT_TRANSITION_IN_PROGRESS)))
+	{
+		if ((wal_state & WALPROHIBIT_STATE_READ_ONLY) != 0)
+			ereport(LOG, (errmsg("system is now read only")));
+		else
+		{
+			/* Request checkpoint */
+			RequestCheckpoint(CHECKPOINT_IMMEDIATE);
+			ereport(LOG, (errmsg("system is now read write")));
+		}
+	}
+
+	/* Wake up the backend who requested the state change */
+	ConditionVariableBroadcast(&CheckpointerShmem->readonly_cv);
+}
+
 /*
  * CompactCheckpointerRequestQueue
  *		Remove duplicates from the request queue to avoid backend fsyncs.
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 88992c2da2c..0eb40a86b52 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -4057,6 +4057,9 @@ pgstat_get_wait_io(WaitEventIO w)
 		case WAIT_EVENT_REPLICATION_SLOT_WRITE:
 			event_name = "ReplicationSlotWrite";
 			break;
+		case WAIT_EVENT_SYSTEM_WALPROHIBIT_STATE_CHANGE:
+			event_name = "SystemWALProhibitStateChange";
+			break;
 		case WAIT_EVENT_SLRU_FLUSH_SYNC:
 			event_name = "SLRUFlushSync";
 			break;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index e73639df744..9594df76946 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -290,6 +290,8 @@ main(int argc, char *argv[])
 		   (uint32) ControlFile->backupEndPoint);
 	printf(_("End-of-backup record required:        %s\n"),
 		   ControlFile->backupEndRequired ? _("yes") : _("no"));
+	printf(_("WAL write prohibited:                 %s\n"),
+		   ControlFile->wal_prohibited ? _("yes") : _("no"));
 	printf(_("wal_level setting:                    %s\n"),
 		   wal_level_str(ControlFile->wal_level));
 	printf(_("wal_log_hints setting:                %s\n"),
diff --git a/src/include/access/walprohibit.h b/src/include/access/walprohibit.h
index 619c33cd780..163fe0d2fce 100644
--- a/src/include/access/walprohibit.h
+++ b/src/include/access/walprohibit.h
@@ -18,4 +18,15 @@
 extern bool ProcessBarrierWALProhibit(void);
 extern void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
 
+/* WAL Prohibit States */
+#define	WALPROHIBIT_STATE_READ_WRITE		0x0000
+#define	WALPROHIBIT_STATE_READ_ONLY			0x0001
+
+/*
+ * The bit is used in state transition from one state to another.  When this
+ * bit is set then the state indicated by the 0th position bit is yet to
+ * confirmed.
+ */
+#define WALPROHIBIT_TRANSITION_IN_PROGRESS	0x0002
+
 #endif		/* WALPROHIBIT_H */
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 3578da2f420..3f9e96cd18e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -326,7 +326,8 @@ extern void XLOGShmemInit(void);
 extern void BootStrapXLOG(void);
 extern void LocalProcessControlFile(bool reset);
 extern void StartupXLOG(void);
-extern void MakeReadOnlyXLOG(void);
+extern uint32 GetWALProhibitState(void);
+extern bool SetWALProhibitState(uint32 new_state);
 extern void ShutdownXLOG(int code, Datum arg);
 extern void InitXLOGAccess(void);
 extern void CreateCheckPoint(int flags);
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index de5670e5382..b32c7723275 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -182,6 +182,9 @@ typedef struct ControlFileData
 	int			max_locks_per_xact;
 	bool		track_commit_timestamp;
 
+	/* WAL prohibited determines if the WAL insert is allowed or not. */
+	bool		wal_prohibited;
+
 	/*
 	 * This data is used to check for hardware-architecture compatibility of
 	 * the database and the backend executable.  We need not check endianness
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 13872013823..780c59f3e48 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -955,6 +955,7 @@ typedef enum
 	WAIT_EVENT_REPLICATION_SLOT_RESTORE_SYNC,
 	WAIT_EVENT_REPLICATION_SLOT_SYNC,
 	WAIT_EVENT_REPLICATION_SLOT_WRITE,
+	WAIT_EVENT_SYSTEM_WALPROHIBIT_STATE_CHANGE,
 	WAIT_EVENT_SLRU_FLUSH_SYNC,
 	WAIT_EVENT_SLRU_READ,
 	WAIT_EVENT_SLRU_SYNC,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e6..e8271b49f6d 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -35,6 +35,8 @@ extern void CheckpointWriteDelay(int flags, double progress);
 
 extern bool ForwardSyncRequest(const FileTag *ftag, SyncRequestType type);
 
+extern void WALProhibitRequest(void);
+
 extern void AbsorbSyncRequests(void);
 
 extern Size CheckpointerShmemSize(void);
-- 
2.22.0

From f0188a48723b1ae7372bcc6a344ed7868fdc40fb Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 27 Mar 2020 05:05:38 -0400
Subject: [PATCH v3 2/6] Add alter system read only/write syntax

Note that syntax doesn't have any implementation.
---
 src/backend/nodes/copyfuncs.c    | 12 ++++++++++++
 src/backend/nodes/equalfuncs.c   |  9 +++++++++
 src/backend/parser/gram.y        | 13 +++++++++++++
 src/backend/tcop/utility.c       | 20 ++++++++++++++++++++
 src/bin/psql/tab-complete.c      |  6 ++++--
 src/include/nodes/nodes.h        |  1 +
 src/include/nodes/parsenodes.h   | 10 ++++++++++
 src/tools/pgindent/typedefs.list |  1 +
 8 files changed, 70 insertions(+), 2 deletions(-)

diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 89c409de664..ba3393b8ccf 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -4020,6 +4020,15 @@ _copyAlterSystemStmt(const AlterSystemStmt *from)
 	return newnode;
 }
 
+static AlterSystemWALProhibitState *
+_copyAlterSystemWALProhibitState(const AlterSystemWALProhibitState *from)
+{
+	AlterSystemWALProhibitState *newnode = makeNode(AlterSystemWALProhibitState);
+
+	COPY_SCALAR_FIELD(WALProhibited);
+	return newnode;
+}
+
 static CreateSeqStmt *
 _copyCreateSeqStmt(const CreateSeqStmt *from)
 {
@@ -5406,6 +5415,9 @@ copyObjectImpl(const void *from)
 		case T_AlterSystemStmt:
 			retval = _copyAlterSystemStmt(from);
 			break;
+		case T_AlterSystemWALProhibitState:
+			retval = _copyAlterSystemWALProhibitState(from);
+			break;
 		case T_CreateSeqStmt:
 			retval = _copyCreateSeqStmt(from);
 			break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e3f33c40be5..b09bff458af 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1769,6 +1769,12 @@ _equalAlterSystemStmt(const AlterSystemStmt *a, const AlterSystemStmt *b)
 	return true;
 }
 
+static bool
+_equalAlterSystemWALProhibitState(const AlterSystemWALProhibitState *a, const AlterSystemWALProhibitState *b)
+{
+	COMPARE_SCALAR_FIELD(WALProhibited);
+	return true;
+}
 
 static bool
 _equalCreateSeqStmt(const CreateSeqStmt *a, const CreateSeqStmt *b)
@@ -3458,6 +3464,9 @@ equal(const void *a, const void *b)
 		case T_AlterSystemStmt:
 			retval = _equalAlterSystemStmt(a, b);
 			break;
+		case T_AlterSystemWALProhibitState:
+			retval = _equalAlterSystemWALProhibitState(a, b);
+			break;
 		case T_CreateSeqStmt:
 			retval = _equalCreateSeqStmt(a, b);
 			break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index dbb47d49829..6090d18ec61 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -479,6 +479,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
 
 %type <vsetstmt> generic_set set_rest set_rest_more generic_reset reset_rest
 				 SetResetClause FunctionSetResetClause
+%type <boolean> system_readonly_state
 
 %type <node>	TableElement TypedTableElement ConstraintElem TableFuncElement
 %type <node>	columnDef columnOptions
@@ -10172,8 +10173,20 @@ AlterSystemStmt:
 					n->setstmt = $4;
 					$$ = (Node *)n;
 				}
+			| ALTER SYSTEM_P system_readonly_state
+				{
+					AlterSystemWALProhibitState *n = makeNode(AlterSystemWALProhibitState);
+					n->WALProhibited = $3;
+					$$ = (Node *)n;
+				}
 		;
 
+system_readonly_state:
+			 READ ONLY
+					{ $$ = true; }
+			| READ WRITE
+					{ $$ = false; }
+		;
 
 /*****************************************************************************
  *
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 9b0c376c8cb..7af96c77082 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -85,6 +85,7 @@ static void ProcessUtilitySlow(ParseState *pstate,
 							   DestReceiver *dest,
 							   QueryCompletion *qc);
 static void ExecDropStmt(DropStmt *stmt, bool isTopLevel);
+static void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
 
 /*
  * CommandIsReadOnly: is an executable query read-only?
@@ -219,6 +220,7 @@ ClassifyUtilityCommandAsReadOnly(Node *parsetree)
 				return COMMAND_IS_NOT_READ_ONLY;
 			}
 
+		case T_AlterSystemWALProhibitState:
 		case T_AlterSystemStmt:
 			{
 				/*
@@ -834,6 +836,11 @@ standard_ProcessUtility(PlannedStmt *pstmt,
 			AlterSystemSetConfigFile((AlterSystemStmt *) parsetree);
 			break;
 
+		case T_AlterSystemWALProhibitState:
+			PreventInTransactionBlock(isTopLevel, "ALTER SYSTEM");
+			AlterSystemSetWALProhibitState((AlterSystemWALProhibitState *) parsetree);
+			break;
+
 		case T_VariableSetStmt:
 			ExecSetVariableStmt((VariableSetStmt *) parsetree, isTopLevel);
 			break;
@@ -2772,6 +2779,7 @@ CreateCommandTag(Node *parsetree)
 			tag = CMDTAG_REFRESH_MATERIALIZED_VIEW;
 			break;
 
+		case T_AlterSystemWALProhibitState:
 		case T_AlterSystemStmt:
 			tag = CMDTAG_ALTER_SYSTEM;
 			break;
@@ -3636,3 +3644,15 @@ GetCommandLogLevel(Node *parsetree)
 
 	return lev;
 }
+
+/*
+ * AlterSystemSetWALProhibitState
+ *
+ * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
+ */
+static void
+AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
+{
+	/* some code */
+	elog(INFO, "AlterSystemSetWALProhibitState() called");
+}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index eb018854a5c..d586cf74816 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1858,9 +1858,11 @@ psql_completion(const char *text, int start, int end)
 	/* ALTER SERVER <name> VERSION <version> */
 	else if (Matches("ALTER", "SERVER", MatchAny, "VERSION", MatchAny))
 		COMPLETE_WITH("OPTIONS");
-	/* ALTER SYSTEM SET, RESET, RESET ALL */
+	/* ALTER SYSTEM READ, SET, RESET, RESET ALL */
 	else if (Matches("ALTER", "SYSTEM"))
-		COMPLETE_WITH("SET", "RESET");
+		COMPLETE_WITH("SET", "RESET", "READ");
+	else if (Matches("ALTER", "SYSTEM", "READ"))
+		COMPLETE_WITH("ONLY", "WRITE");
 	else if (Matches("ALTER", "SYSTEM", "SET|RESET"))
 		COMPLETE_WITH_QUERY(Query_for_list_of_alter_system_set_vars);
 	else if (Matches("ALTER", "SYSTEM", "SET", MatchAny))
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 381d84b4e4f..17d6942c734 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -412,6 +412,7 @@ typedef enum NodeTag
 	T_RefreshMatViewStmt,
 	T_ReplicaIdentityStmt,
 	T_AlterSystemStmt,
+	T_AlterSystemWALProhibitState,
 	T_CreatePolicyStmt,
 	T_AlterPolicyStmt,
 	T_CreateTransformStmt,
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 151bcdb7ef5..f2c1ae8e3fe 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -3194,6 +3194,16 @@ typedef struct AlterSystemStmt
 	VariableSetStmt *setstmt;	/* SET subcommand */
 } AlterSystemStmt;
 
+/* ----------------------
+ *		Alter System Read Statement
+ * ----------------------
+ */
+typedef struct AlterSystemWALProhibitState
+{
+	NodeTag		type;
+	bool		WALProhibited;
+} AlterSystemWALProhibitState;
+
 /* ----------------------
  *		Cluster Statement (support pbrown's cluster index implementation)
  * ----------------------
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 7eaaad1e140..247bdf1bacc 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -87,6 +87,7 @@ AlterStatsStmt
 AlterSubscriptionStmt
 AlterSubscriptionType
 AlterSystemStmt
+AlterSystemWALProhibitState
 AlterTSConfigType
 AlterTSConfigurationStmt
 AlterTSDictionaryStmt
-- 
2.22.0

From 2c5db7db70d4cebebf574fbc47db7fbf7c440be1 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 19 Jun 2020 06:29:36 -0400
Subject: [PATCH v3 3/6] Implement ALTER SYSTEM READ ONLY using global barrier.

Implementation:

 1. When a user tried to change server state to WAL-Prohibited using
    ALTER SYSTEM READ ONLY command; AlterSystemSetWALProhibitState() will emit
    PROCSIGNAL_BARRIER_WAL_PROHIBIT_STATE_CHANGE barrier and will wait until the
    barrier has been absorbed by all the backends.

 2. When a backend receives the WAL-Prohibited barrier, at that moment if
    it is already in a transaction and the transaction already assigned XID,
    then the backend will be killed by throwing FATAL(XXX: need more discussion
    on this)

 3. Otherwise, if that backend running transaction which yet to get XID
    assigned we don't need to do anything special, simply call
    ResetLocalXLogInsertAllowed() so that any future WAL insert in will check
    XLogInsertAllowed() first which set ready only state appropriately.

 4. A new transaction (from existing or new backend) starts as a read-only
    transaction.

 5. Auxiliary processes like autovacuum launcher, background writer,
    checkpointer and  walwriter will don't do anything in WAL-Prohibited
    server state until someone wakes us up. E.g. a backend might later on
    request us to put the system back to read-write.

 6. At shutdown in WAL-Prohibited mode, we'll skip shutdown checkpoint
    and xlog rotation. Starting up again will perform crash recovery(XXX:
    need some discussion on this as well)

 7. ALTER SYSTEM READ ONLY/WRITE is restricted on standby server.

 8. Only super user can toggle WAL-Prohibit state.

 9. Add system_is_read_only GUC show the system state -- will true when system
    is wal prohibited or in recovery.
---
 src/backend/access/transam/Makefile      |  1 +
 src/backend/access/transam/walprohibit.c | 81 ++++++++++++++++++++++++
 src/backend/access/transam/xact.c        | 49 ++++++++------
 src/backend/access/transam/xlog.c        | 72 ++++++++++++++++++---
 src/backend/postmaster/autovacuum.c      |  4 ++
 src/backend/postmaster/bgwriter.c        |  2 +-
 src/backend/postmaster/checkpointer.c    | 12 ++++
 src/backend/storage/ipc/procsignal.c     | 26 ++------
 src/backend/tcop/utility.c               | 14 +---
 src/backend/utils/misc/guc.c             | 26 ++++++++
 src/include/access/walprohibit.h         | 21 ++++++
 src/include/access/xlog.h                |  3 +
 src/include/storage/procsignal.h         |  7 +-
 13 files changed, 246 insertions(+), 72 deletions(-)
 create mode 100644 src/backend/access/transam/walprohibit.c
 create mode 100644 src/include/access/walprohibit.h

diff --git a/src/backend/access/transam/Makefile b/src/backend/access/transam/Makefile
index 595e02de722..b5322a69954 100644
--- a/src/backend/access/transam/Makefile
+++ b/src/backend/access/transam/Makefile
@@ -26,6 +26,7 @@ OBJS = \
 	twophase.o \
 	twophase_rmgr.o \
 	varsup.o \
+	walprohibit.o \
 	xact.o \
 	xlog.o \
 	xlogarchive.o \
diff --git a/src/backend/access/transam/walprohibit.c b/src/backend/access/transam/walprohibit.c
new file mode 100644
index 00000000000..df97596ddf9
--- /dev/null
+++ b/src/backend/access/transam/walprohibit.c
@@ -0,0 +1,81 @@
+/*-------------------------------------------------------------------------
+ *
+ * walprohibit.c
+ * 		PostgreSQL write-ahead log prohibit states
+ *
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/backend/access/transam/walprohibit.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/walprohibit.h"
+#include "postmaster/bgwriter.h"
+#include "storage/procsignal.h"
+
+/*
+ * ProcessBarrierWALProhibit()
+ *
+ * Handle WAL prohibit state change request.
+ */
+bool
+ProcessBarrierWALProhibit(void)
+{
+	/*
+	 * Kill off any transactions that have an XID *before* allowing the system
+	 * to go WAL prohibit state.
+	 */
+	if (FullTransactionIdIsValid(GetTopFullTransactionIdIfAny()))
+	{
+		/*
+		 * XXX: Kill off the whole session by throwing FATAL instead of killing
+		 * transaction by throwing ERROR due to following reasons that need be
+		 * thought:
+		 *
+		 * 1. Due to some presents challenges with the wire protocol, we could
+		 * not simply kill of idle transaction.
+		 *
+		 * 2. If we are here in subtransaction then the ERROR will kill the
+		 * current subtransaction only. In the case of invalidations, that
+		 * might be good enough, but for XID assignment it's not, because
+		 * assigning an XID to a subtransaction also causes higher
+		 * sub-transaction levels and the parent transaction to get XIDs.
+		 */
+		ereport(FATAL,
+				(errcode(ERRCODE_ACTIVE_SQL_TRANSACTION),
+				 errmsg("system is now read only"),
+				 errhint("Cannot continue a transaction if it has performed writes while system is read only.")));
+	}
+
+	/* Return to "check" state */
+	ResetLocalXLogInsertAllowed();
+
+	return true;
+}
+
+/*
+ * AlterSystemSetWALProhibitState
+ *
+ * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
+ */
+void
+AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
+{
+	if (!superuser())
+		ereport(ERROR,
+				(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+				 errmsg("must be superuser to execute ALTER SYSTEM command")));
+
+	/* Alter WAL prohibit state not allowed during recovery */
+	PreventCommandDuringRecovery("ALTER SYSTEM");
+
+	/* Yet to add ALTER SYTEM READ WRITE support */
+	if (!stmt->WALProhibited)
+		elog(ERROR, "XXX: Yet to implement");
+
+	MakeReadOnlyXLOG();
+	WaitForProcSignalBarrier(EmitProcSignalBarrier(PROCSIGNAL_BARRIER_WALPROHIBIT));
+}
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index b3ee7fa7ea0..4fbfcdbb965 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -1937,23 +1937,28 @@ StartTransaction(void)
 	Assert(s->prevSecContext == 0);
 
 	/*
-	 * Make sure we've reset xact state variables
+	 * Reset xact state variables.
 	 *
-	 * If recovery is still in progress, mark this transaction as read-only.
-	 * We have lower level defences in XLogInsert and elsewhere to stop us
-	 * from modifying data during recovery, but this gives the normal
-	 * indication to the user that the transaction is read-only.
-	 */
-	if (RecoveryInProgress())
-	{
-		s->startedInRecovery = true;
-		XactReadOnly = true;
-	}
-	else
-	{
-		s->startedInRecovery = false;
-		XactReadOnly = DefaultXactReadOnly;
-	}
+	 * If it is not currently possible to insert write-ahead log records,
+	 * either because we are still in recovery or because ALTER SYSTEM READ
+	 * ONLY has been executed, force this to be a read-only transaction.
+	 * We have lower level defences in XLogBeginInsert() and elsewhere to stop
+	 * us from modifying data during recovery when !XLogInsertAllowed(), but
+	 * this gives the normal indication to the user that the transaction is
+	 * read-only.
+	 *
+	 * On the other hand, we only need to set the startedInRecovery flag when
+	 * the transaction started during recovery, and not when WAL is otherwise
+	 * prohibited. This information is used by RelationGetIndexScan() to
+	 * decide whether to permit (1) relying on existing killed-tuple markings
+	 * and (2) further killing of index tuples. Even when WAL is prohibited
+	 * on the master, it's still the master, so the former is OK; and since
+	 * killing index tuples doesn't generate WAL, the latter is also OK.
+	 * See comments in RelationGetIndexScan() and MarkBufferDirtyHint().
+	 */
+	XactReadOnly = DefaultXactReadOnly || !XLogInsertAllowed();
+	s->startedInRecovery = RecoveryInProgress();
+
 	XactDeferrable = DefaultXactDeferrable;
 	XactIsoLevel = DefaultXactIsoLevel;
 	forceSyncCommit = false;
@@ -4875,9 +4880,11 @@ CommitSubTransaction(void)
 	/*
 	 * We need to restore the upper transaction's read-only state, in case the
 	 * upper is read-write while the child is read-only; GUC will incorrectly
-	 * think it should leave the child state in place.
+	 * think it should leave the child state in place.  Note that the upper
+	 * transaction will be a force to ready-only irrespective of its previous
+	 * status if the server state is WAL prohibited.
 	 */
-	XactReadOnly = s->prevXactReadOnly;
+	XactReadOnly = s->prevXactReadOnly || !XLogInsertAllowed();
 
 	CurrentResourceOwner = s->parent->curTransactionOwner;
 	CurTransactionResourceOwner = s->parent->curTransactionOwner;
@@ -5033,9 +5040,11 @@ AbortSubTransaction(void)
 	/*
 	 * Restore the upper transaction's read-only state, too.  This should be
 	 * redundant with GUC's cleanup but we may as well do it for consistency
-	 * with the commit case.
+	 * with the commit case.  Note that the upper transaction will be a force
+	 * to ready-only irrespective of its previous status if the server state is
+	 * WAL prohibited.
 	 */
-	XactReadOnly = s->prevXactReadOnly;
+	XactReadOnly = s->prevXactReadOnly || !XLogInsertAllowed();
 
 	RESUME_INTERRUPTS();
 }
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 0a97b1d37fb..c2d6d19716c 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -245,9 +245,10 @@ static bool LocalPromoteIsTriggered = false;
  *		0: unconditionally not allowed to insert XLOG
  *		-1: must check RecoveryInProgress(); disallow until it is false
  * Most processes start with -1 and transition to 1 after seeing that recovery
- * is not in progress.  But we can also force the value for special cases.
- * The coding in XLogInsertAllowed() depends on the first two of these states
- * being numerically the same as bool true and false.
+ * is not in progress or the server state is not a WAL prohibited state.  But
+ * we can also force the value for special cases.  The coding in
+ * XLogInsertAllowed() depends on the first two of these states being
+ * numerically the same as bool true and false.
  */
 static int	LocalXLogInsertAllowed = -1;
 
@@ -660,6 +661,12 @@ typedef struct XLogCtlData
 	 */
 	RecoveryState SharedRecoveryState;
 
+	/*
+	 * WALProhibited indicates if we have stopped allowing WAL writes.
+	 * Protected by info_lck.
+	 */
+	bool		WALProhibited;
+
 	/*
 	 * SharedHotStandbyActive indicates if we allow hot standby queries to be
 	 * run.  Protected by info_lck.
@@ -7962,6 +7969,25 @@ StartupXLOG(void)
 		RequestCheckpoint(CHECKPOINT_FORCE);
 }
 
+void
+MakeReadOnlyXLOG(void)
+{
+	SpinLockAcquire(&XLogCtl->info_lck);
+	XLogCtl->WALProhibited = true;
+	SpinLockRelease(&XLogCtl->info_lck);
+}
+
+/*
+ * Is the system still in WAL prohibited state?
+ */
+bool
+IsWALProhibited(void)
+{
+	volatile XLogCtlData *xlogctl = XLogCtl;
+
+	return xlogctl->WALProhibited;
+}
+
 /*
  * Checks if recovery has reached a consistent state. When consistency is
  * reached and we have a valid starting standby snapshot, tell postmaster
@@ -8177,9 +8203,9 @@ HotStandbyActiveInReplay(void)
 /*
  * Is this process allowed to insert new WAL records?
  *
- * Ordinarily this is essentially equivalent to !RecoveryInProgress().
- * But we also have provisions for forcing the result "true" or "false"
- * within specific processes regardless of the global state.
+ * Ordinarily this is essentially equivalent to !RecoveryInProgress() and
+ * !IsWALProhibited().  But we also have provisions for forcing the result
+ * "true" or "false" within specific processes regardless of the global state.
  */
 bool
 XLogInsertAllowed(void)
@@ -8193,14 +8219,25 @@ XLogInsertAllowed(void)
 		return (bool) LocalXLogInsertAllowed;
 
 	/*
-	 * Else, must check to see if we're still in recovery.
+	 * Else, must check to see if we're still in recovery
 	 */
 	if (RecoveryInProgress())
 		return false;
 
+	/* Or, in WAL prohibited state */
+	if (IsWALProhibited())
+	{
+		/*
+		 * Set it to "unconditionally false" to avoid checking until it gets
+		 * reset.
+		 */
+		LocalXLogInsertAllowed = 0;
+		return false;
+	}
+
 	/*
-	 * On exit from recovery, reset to "unconditionally true", since there is
-	 * no need to keep checking.
+	 * On exit from recovery or WAL prohibited state, reset to "unconditionally
+	 * true", since there is no need to keep checking.
 	 */
 	LocalXLogInsertAllowed = 1;
 	return true;
@@ -8216,12 +8253,20 @@ static void
 LocalSetXLogInsertAllowed(void)
 {
 	Assert(LocalXLogInsertAllowed == -1);
+	Assert(!IsWALProhibited());
+
 	LocalXLogInsertAllowed = 1;
 
 	/* Initialize as RecoveryInProgress() would do when switching state */
 	InitXLOGAccess();
 }
 
+void
+ResetLocalXLogInsertAllowed(void)
+{
+	LocalXLogInsertAllowed = -1;
+}
+
 /*
  * Subroutine to try to fetch and validate a prior checkpoint record.
  *
@@ -8513,7 +8558,10 @@ ShutdownXLOG(int code, Datum arg)
 
 	if (RecoveryInProgress())
 		CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
-	else
+	/*
+	 * Can't perform checkpoint or xlog rotation without writing WAL.
+	 */
+	else if (XLogInsertAllowed())
 	{
 		/*
 		 * If archiving is enabled, rotate the last XLOG file so that all the
@@ -8526,6 +8574,10 @@ ShutdownXLOG(int code, Datum arg)
 
 		CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
 	}
+	else
+		ereport(LOG,
+			   (errmsg("skipping shutdown checkpoint because the system is read only")));
+
 	ShutdownCLOG();
 	ShutdownCommitTs();
 	ShutdownSUBTRANS();
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 9c7d4b0c60e..f83f86994db 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -652,6 +652,10 @@ AutoVacLauncherMain(int argc, char *argv[])
 
 		HandleAutoVacLauncherInterrupts();
 
+		/* If the server is read only just go back to sleep. */
+		if (!XLogInsertAllowed())
+			continue;
+
 		/*
 		 * a worker finished, or postmaster signaled failure to start a worker
 		 */
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index 069e27e427f..6c6ff7dc3af 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -268,7 +268,7 @@ BackgroundWriterMain(void)
 		 * Checkpointer, when active, is barely ever in its mainloop and thus
 		 * makes it hard to log regularly.
 		 */
-		if (XLogStandbyInfoActive() && !RecoveryInProgress())
+		if (XLogStandbyInfoActive() && XLogInsertAllowed())
 		{
 			TimestampTz timeout = 0;
 			TimestampTz now = GetCurrentTimestamp();
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b80..5e5e56d4eec 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -342,6 +342,18 @@ CheckpointerMain(void)
 		AbsorbSyncRequests();
 		HandleCheckpointerInterrupts();
 
+		/*
+		 * If the server is in WAL-Prohibited state then don't do anything until
+		 * someone wakes us up. E.g. a backend might later on request us to put
+		 * the system back to read-write.
+		 */
+		if (IsWALProhibited())
+		{
+			(void) WaitLatch(MyLatch, WL_LATCH_SET | WL_EXIT_ON_PM_DEATH, -1,
+							 WAIT_EVENT_CHECKPOINTER_MAIN);
+			continue;
+		}
+
 		/*
 		 * Detect a pending checkpoint request by checking whether the flags
 		 * word in shared memory is nonzero.  We shouldn't need to acquire the
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 13648887187..b973727a580 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -18,6 +18,7 @@
 #include <unistd.h>
 
 #include "access/parallel.h"
+#include "access/walprohibit.h"
 #include "commands/async.h"
 #include "miscadmin.h"
 #include "pgstat.h"
@@ -96,7 +97,6 @@ static volatile ProcSignalSlot *MyProcSignalSlot = NULL;
 
 static bool CheckProcSignal(ProcSignalReason reason);
 static void CleanupProcSignalState(int status, Datum arg);
-static bool ProcessBarrierPlaceholder(void);
 
 /*
  * ProcSignalShmemSize
@@ -510,9 +510,9 @@ ProcessProcSignalBarrier(void)
 			 * unconditionally, but it's more efficient to call only the ones
 			 * that might need us to do something based on the flags.
 			 */
-			if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_PLACEHOLDER)
-				&& ProcessBarrierPlaceholder())
-				BARRIER_CLEAR_BIT(flags, PROCSIGNAL_BARRIER_PLACEHOLDER);
+			if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_WALPROHIBIT)
+				&& ProcessBarrierWALProhibit())
+				BARRIER_CLEAR_BIT(flags, PROCSIGNAL_BARRIER_WALPROHIBIT);
 		}
 		PG_CATCH();
 		{
@@ -554,24 +554,6 @@ ProcessProcSignalBarrier(void)
 	pg_atomic_write_u64(&MyProcSignalSlot->pss_barrierGeneration, shared_gen);
 }
 
-static bool
-ProcessBarrierPlaceholder(void)
-{
-	/*
-	 * XXX. This is just a placeholder until the first real user of this
-	 * machinery gets committed. Rename PROCSIGNAL_BARRIER_PLACEHOLDER to
-	 * PROCSIGNAL_BARRIER_SOMETHING_ELSE where SOMETHING_ELSE is something
-	 * appropriately descriptive. Get rid of this function and instead have
-	 * ProcessBarrierSomethingElse. Most likely, that function should live in
-	 * the file pertaining to that subsystem, rather than here.
-	 *
-	 * The return value should be 'true' if the barrier was successfully
-	 * absorbed and 'false' if not. Note that returning 'false' can lead to
-	 * very frequent retries, so try hard to make that an uncommon case.
-	 */
-	return true;
-}
-
 /*
  * CheckProcSignal - check to see if a particular reason has been
  * signaled, and clear the signal flag.  Should be called after receiving
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 7af96c77082..d6411e4f3e9 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -19,6 +19,7 @@
 #include "access/htup_details.h"
 #include "access/reloptions.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "catalog/catalog.h"
@@ -85,7 +86,6 @@ static void ProcessUtilitySlow(ParseState *pstate,
 							   DestReceiver *dest,
 							   QueryCompletion *qc);
 static void ExecDropStmt(DropStmt *stmt, bool isTopLevel);
-static void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
 
 /*
  * CommandIsReadOnly: is an executable query read-only?
@@ -3644,15 +3644,3 @@ GetCommandLogLevel(Node *parsetree)
 
 	return lev;
 }
-
-/*
- * AlterSystemSetWALProhibitState
- *
- * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
- */
-static void
-AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
-{
-	/* some code */
-	elog(INFO, "AlterSystemSetWALProhibitState() called");
-}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 031ca0327f0..3384523fe48 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -221,6 +221,7 @@ static bool check_recovery_target_lsn(char **newval, void **extra, GucSource sou
 static void assign_recovery_target_lsn(const char *newval, void *extra);
 static bool check_primary_slot_name(char **newval, void **extra, GucSource source);
 static bool check_default_with_oids(bool *newval, void **extra, GucSource source);
+static const char *show_system_is_read_only(void);
 
 /* Private functions in guc-file.l that need to be called from guc.c */
 static ConfigVariable *ProcessConfigFileInternal(GucContext context,
@@ -610,6 +611,7 @@ static char *recovery_target_string;
 static char *recovery_target_xid_string;
 static char *recovery_target_name_string;
 static char *recovery_target_lsn_string;
+static bool system_is_read_only;
 
 
 /* should be static, but commands/variable.c needs to get at this */
@@ -2041,6 +2043,18 @@ static struct config_bool ConfigureNamesBool[] =
 		NULL, NULL, NULL
 	},
 
+	{
+		/* Not for general use */
+		{"system_is_read_only", PGC_INTERNAL, WAL,
+			gettext_noop("Shows whether the system is read only."),
+			NULL,
+			GUC_NO_RESET_ALL | GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE
+		},
+		&system_is_read_only,
+		false,
+		NULL, NULL, show_system_is_read_only
+	},
+
 	/* End-of-list marker */
 	{
 		{NULL, 0, 0, NULL, NULL}, NULL, false, NULL, NULL, NULL
@@ -11998,4 +12012,16 @@ check_default_with_oids(bool *newval, void **extra, GucSource source)
 	return true;
 }
 
+/*
+ * NB: The return string should be the same as the _ShowOption() for boolean
+ * type.
+ */
+ static const char *
+ show_system_is_read_only(void)
+{
+	if (!XLogInsertAllowed())
+		return "on";
+	return "off";
+}
+
 #include "guc-file.c"
diff --git a/src/include/access/walprohibit.h b/src/include/access/walprohibit.h
new file mode 100644
index 00000000000..619c33cd780
--- /dev/null
+++ b/src/include/access/walprohibit.h
@@ -0,0 +1,21 @@
+/*
+ * walprohibit.h
+ *
+ * PostgreSQL write-ahead log prohibit states
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/include/access/walprohibit.h
+ */
+#ifndef WALPROHIBIT_H
+#define WALPROHIBIT_H
+
+#include "access/xact.h"
+#include "access/xlog.h"
+#include "miscadmin.h"
+#include "nodes/parsenodes.h"
+
+extern bool ProcessBarrierWALProhibit(void);
+extern void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
+
+#endif		/* WALPROHIBIT_H */
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 5b143348879..3578da2f420 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -300,11 +300,13 @@ extern const char *xlog_identify(uint8 info);
 
 extern void issue_xlog_fsync(int fd, XLogSegNo segno);
 
+extern bool IsWALProhibited(void);
 extern bool RecoveryInProgress(void);
 extern RecoveryState GetRecoveryState(void);
 extern bool HotStandbyActive(void);
 extern bool HotStandbyActiveInReplay(void);
 extern bool XLogInsertAllowed(void);
+extern void ResetLocalXLogInsertAllowed(void);
 extern void GetXLogReceiptTime(TimestampTz *rtime, bool *fromStream);
 extern XLogRecPtr GetXLogReplayRecPtr(TimeLineID *replayTLI);
 extern XLogRecPtr GetXLogInsertRecPtr(void);
@@ -324,6 +326,7 @@ extern void XLOGShmemInit(void);
 extern void BootStrapXLOG(void);
 extern void LocalProcessControlFile(bool reset);
 extern void StartupXLOG(void);
+extern void MakeReadOnlyXLOG(void);
 extern void ShutdownXLOG(int code, Datum arg);
 extern void InitXLOGAccess(void);
 extern void CreateCheckPoint(int flags);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f38..bae06202b4a 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -48,12 +48,7 @@ typedef enum
 
 typedef enum
 {
-	/*
-	 * XXX. PROCSIGNAL_BARRIER_PLACEHOLDER should be replaced when the first
-	 * real user of the ProcSignalBarrier mechanism is added. It's just here
-	 * for now because we can't have an empty enum.
-	 */
-	PROCSIGNAL_BARRIER_PLACEHOLDER = 0
+	PROCSIGNAL_BARRIER_WALPROHIBIT = 0
 } ProcSignalBarrierType;
 
 /*
-- 
2.22.0

From 0b7426fc4708cc0e4ad333da3b35e473658bba28 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Tue, 14 Jul 2020 02:10:55 -0400
Subject: [PATCH v3 5/6] Error or Assert before START_CRIT_SECTION for WAL
 write

Based on the following criteria added the Assert or the Error when system is
prohibited:

 - Added ERROR for the function which can be reachable without valid XID in case
   of VACUUM or CONCURRENT CREATE INDEX.  For that added common static inline
   function CheckWALPermitted().
 - Added ASSERT for the function which cannot be reached without valid XID;
   Assert to ensure XID validation. For that added AssertWALPermitted_HaveXID().

To enforce the rule to have aforesaid assert or error check before entering a
critical section for WAL write, a new walpermit_checked_state assert only flag
is added. If these check is missing then XLogBeginInsert() will have an
assertion if it is in critical section.

If we are not doing WAL insert inside the critical section then the above
checking is not necessary, we can rely on XLogBeginInsert() for that check &
report an error.
---
 src/backend/access/brin/brin.c            |  4 ++
 src/backend/access/brin/brin_pageops.c    | 16 ++++++++
 src/backend/access/brin/brin_revmap.c     |  8 ++++
 src/backend/access/gin/ginbtree.c         | 17 ++++++--
 src/backend/access/gin/gindatapage.c      | 14 ++++++-
 src/backend/access/gin/ginfast.c          |  8 ++++
 src/backend/access/gin/gininsert.c        |  4 ++
 src/backend/access/gin/ginutil.c          | 10 ++++-
 src/backend/access/gin/ginvacuum.c        |  9 +++++
 src/backend/access/gist/gist.c            | 16 ++++++++
 src/backend/access/gist/gistvacuum.c      |  9 +++++
 src/backend/access/hash/hash.c            | 13 ++++++
 src/backend/access/hash/hashinsert.c      |  8 ++++
 src/backend/access/hash/hashovfl.c        | 14 +++++++
 src/backend/access/hash/hashpage.c        | 13 ++++++
 src/backend/access/heap/heapam.c          | 32 +++++++++++++++
 src/backend/access/heap/pruneheap.c       |  9 ++++-
 src/backend/access/heap/vacuumlazy.c      | 13 ++++++
 src/backend/access/heap/visibilitymap.c   | 20 ++++++++-
 src/backend/access/nbtree/nbtdedup.c      |  4 ++
 src/backend/access/nbtree/nbtinsert.c     | 13 +++++-
 src/backend/access/nbtree/nbtpage.c       | 24 +++++++++++
 src/backend/access/spgist/spgdoinsert.c   | 19 +++++++++
 src/backend/access/spgist/spgvacuum.c     | 13 ++++++
 src/backend/access/transam/multixact.c    |  6 ++-
 src/backend/access/transam/twophase.c     | 10 +++++
 src/backend/access/transam/varsup.c       |  4 ++
 src/backend/access/transam/walprohibit.c  | 10 +++++
 src/backend/access/transam/xact.c         |  7 ++++
 src/backend/access/transam/xlog.c         | 27 +++++++++----
 src/backend/access/transam/xloginsert.c   | 13 +++++-
 src/backend/commands/sequence.c           | 16 ++++++++
 src/backend/commands/variable.c           |  9 +++--
 src/backend/postmaster/checkpointer.c     |  4 ++
 src/backend/storage/buffer/bufmgr.c       | 10 +++--
 src/backend/storage/freespace/freespace.c | 11 ++++-
 src/backend/storage/lmgr/lock.c           |  6 +--
 src/backend/utils/cache/relmapper.c       |  4 ++
 src/include/access/walprohibit.h          | 49 ++++++++++++++++++++++-
 src/include/miscadmin.h                   | 27 +++++++++++++
 40 files changed, 491 insertions(+), 32 deletions(-)

diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index 7db3ae5ee0c..ef002a51773 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -24,6 +24,7 @@
 #include "access/relscan.h"
 #include "access/table.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/index.h"
 #include "catalog/pg_am.h"
@@ -758,6 +759,9 @@ brinbuildempty(Relation index)
 		ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermitted_HaveXID();
+
 	/* Initialize and xlog metabuffer. */
 	START_CRIT_SECTION();
 	brin_metapage_init(BufferGetPage(metabuf), BrinGetPagesPerRange(index),
diff --git a/src/backend/access/brin/brin_pageops.c b/src/backend/access/brin/brin_pageops.c
index 87de0b855b5..197e1213137 100644
--- a/src/backend/access/brin/brin_pageops.c
+++ b/src/backend/access/brin/brin_pageops.c
@@ -14,6 +14,7 @@
 #include "access/brin_pageops.h"
 #include "access/brin_revmap.h"
 #include "access/brin_xlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -176,6 +177,10 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 	if (((BrinPageFlags(oldpage) & BRIN_EVACUATE_PAGE) == 0) &&
 		brin_can_do_samepage_update(oldbuf, origsz, newsz))
 	{
+		/* Can reach here from VACUUM, so need not have an XID */
+		if (RelationNeedsWAL(idxrel))
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 		if (!PageIndexTupleOverwrite(oldpage, oldoff, (Item) unconstify(BrinTuple *, newtup), newsz))
 			elog(ERROR, "failed to replace BRIN tuple");
@@ -240,6 +245,10 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 
 		revmapbuf = brinLockRevmapPageForUpdate(revmap, heapBlk);
 
+		/* Can reach here from VACUUM, so need not have an XID */
+		if (RelationNeedsWAL(idxrel))
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		/*
@@ -405,6 +414,10 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	page = BufferGetPage(*buffer);
 	blk = BufferGetBlockNumber(*buffer);
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(idxrel))
+		CheckWALPermitted();
+
 	/* Execute the actual insertion */
 	START_CRIT_SECTION();
 	if (extended)
@@ -881,6 +894,9 @@ brin_initialize_empty_new_buffer(Relation idxrel, Buffer buffer)
 			   "brin_initialize_empty_new_buffer: initializing blank page %u",
 			   BufferGetBlockNumber(buffer)));
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 	page = BufferGetPage(buffer);
 	brin_page_init(page, BRIN_PAGETYPE_REGULAR);
diff --git a/src/backend/access/brin/brin_revmap.c b/src/backend/access/brin/brin_revmap.c
index e8b8308f82e..b2d286404a2 100644
--- a/src/backend/access/brin/brin_revmap.c
+++ b/src/backend/access/brin/brin_revmap.c
@@ -26,6 +26,7 @@
 #include "access/brin_tuple.h"
 #include "access/brin_xlog.h"
 #include "access/rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -397,6 +398,10 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 	 * crashed or aborted summarization; remove them silently.
 	 */
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(idxrel))
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	ItemPointerSetInvalid(&invalidIptr);
@@ -606,6 +611,9 @@ revmap_physical_extend(BrinRevmap *revmap)
 		return;
 	}
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	/*
 	 * Ok, we have now locked the metapage and the target block. Re-initialize
 	 * the target block as a revmap page, and update the metapage.
diff --git a/src/backend/access/gin/ginbtree.c b/src/backend/access/gin/ginbtree.c
index 8d08b05f515..1b835b3000b 100644
--- a/src/backend/access/gin/ginbtree.c
+++ b/src/backend/access/gin/ginbtree.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/predicate.h"
@@ -333,6 +334,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 {
 	Page		page = BufferGetPage(stack->buffer);
 	bool		result;
+	bool		needwal;
 	GinPlaceToPageRC rc;
 	uint16		xlflags = 0;
 	Page		childpage = NULL;
@@ -378,6 +380,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 								 insertdata, updateblkno,
 								 &ptp_workspace,
 								 &newlpage, &newrpage);
+	needwal = RelationNeedsWAL(btree->index) && !btree->isBuild;
 
 	if (rc == GPTP_NO_WORK)
 	{
@@ -386,10 +389,14 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 	}
 	else if (rc == GPTP_INSERT)
 	{
+		/* Can reach here from VACUUM, so need not have an XID */
+		if (needwal)
+			CheckWALPermitted();
+
 		/* It will fit, perform the insertion */
 		START_CRIT_SECTION();
 
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogBeginInsert();
 			XLogRegisterBuffer(0, stack->buffer, REGBUF_STANDARD);
@@ -410,7 +417,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 			MarkBufferDirty(childbuf);
 		}
 
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 			ginxlogInsert xlrec;
@@ -548,6 +555,10 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 			}
 		}
 
+		/* Can reach here from VACUUM, so need not have an XID */
+		if (needwal)
+			CheckWALPermitted();
+
 		/*
 		 * OK, we have the new contents of the left page in a temporary copy
 		 * now (newlpage), and likewise for the new contents of the
@@ -588,7 +599,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 		}
 
 		/* write WAL record */
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gin/gindatapage.c b/src/backend/access/gin/gindatapage.c
index 7a2690e97f2..226cb3ce44b 100644
--- a/src/backend/access/gin/gindatapage.c
+++ b/src/backend/access/gin/gindatapage.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "lib/ilist.h"
 #include "miscadmin.h"
@@ -836,7 +837,11 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 		}
 
 		if (RelationNeedsWAL(indexrel))
+		{
+			/* Can reach here from VACUUM, so need not have an XID */
+			CheckWALPermitted();
 			computeLeafRecompressWALData(leaf);
+		}
 
 		/* Apply changes to page */
 		START_CRIT_SECTION();
@@ -1777,6 +1782,7 @@ createPostingTree(Relation index, ItemPointerData *items, uint32 nitems,
 	int			nrootitems;
 	int			rootsize;
 	bool		is_build = (buildStats != NULL);
+	bool		needwal;
 
 	/* Construct the new root page in memory first. */
 	tmppage = (Page) palloc(BLCKSZ);
@@ -1825,12 +1831,18 @@ createPostingTree(Relation index, ItemPointerData *items, uint32 nitems,
 	 */
 	PredicateLockPageSplit(index, BufferGetBlockNumber(entrybuffer), blkno);
 
+	needwal = RelationNeedsWAL(index) && !is_build;
+
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	PageRestoreTempPage(tmppage, page);
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index) && !is_build)
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogCreatePostingTree data;
diff --git a/src/backend/access/gin/ginfast.c b/src/backend/access/gin/ginfast.c
index 2e41b34d8d5..d7781de7674 100644
--- a/src/backend/access/gin/ginfast.c
+++ b/src/backend/access/gin/ginfast.c
@@ -20,6 +20,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
 #include "catalog/pg_am.h"
@@ -68,6 +69,9 @@ writeListPage(Relation index, Buffer buffer,
 	PGAlignedBlock workspace;
 	char	   *ptr;
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	START_CRIT_SECTION();
 
 	GinInitBuffer(buffer, GIN_LIST);
@@ -587,7 +591,11 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 		 * critical section.
 		 */
 		if (RelationNeedsWAL(index))
+		{
+			/* Can reach here from VACUUM, so need not have an XID */
+			CheckWALPermitted();
 			XLogEnsureRecordSpace(data.ndeleted, 0);
+		}
 
 		START_CRIT_SECTION();
 
diff --git a/src/backend/access/gin/gininsert.c b/src/backend/access/gin/gininsert.c
index 77433dc8a41..d957aa6e582 100644
--- a/src/backend/access/gin/gininsert.c
+++ b/src/backend/access/gin/gininsert.c
@@ -17,6 +17,7 @@
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/index.h"
 #include "miscadmin.h"
@@ -447,6 +448,9 @@ ginbuildempty(Relation index)
 		ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(RootBuffer, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermitted_HaveXID();
+
 	/* Initialize and xlog metabuffer and root buffer. */
 	START_CRIT_SECTION();
 	GinInitMetabuffer(MetaBuffer);
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index a400f1fedbc..938089238da 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -17,6 +17,7 @@
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
 #include "access/reloptions.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/pg_collation.h"
 #include "catalog/pg_type.h"
@@ -658,12 +659,19 @@ ginUpdateStats(Relation index, const GinStatsData *stats, bool is_build)
 	Buffer		metabuffer;
 	Page		metapage;
 	GinMetaPageData *metadata;
+	bool		needwal;
 
 	metabuffer = ReadBuffer(index, GIN_METAPAGE_BLKNO);
 	LockBuffer(metabuffer, GIN_EXCLUSIVE);
 	metapage = BufferGetPage(metabuffer);
 	metadata = GinPageGetMeta(metapage);
 
+	needwal = RelationNeedsWAL(index) && !is_build;
+
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	metadata->nTotalPages = stats->nTotalPages;
@@ -683,7 +691,7 @@ ginUpdateStats(Relation index, const GinStatsData *stats, bool is_build)
 
 	MarkBufferDirty(metabuffer);
 
-	if (RelationNeedsWAL(index) && !is_build)
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogUpdateMeta data;
diff --git a/src/backend/access/gin/ginvacuum.c b/src/backend/access/gin/ginvacuum.c
index 8ae4fd95a7b..36a884af597 100644
--- a/src/backend/access/gin/ginvacuum.c
+++ b/src/backend/access/gin/ginvacuum.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "commands/vacuum.h"
 #include "miscadmin.h"
@@ -159,6 +160,10 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	 */
 	PredicateLockPageCombine(gvs->index, deleteBlkno, rightlink);
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(gvs->index))
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* Unlink the page by changing left sibling's rightlink */
@@ -650,6 +655,10 @@ ginbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 
 		if (resPage)
 		{
+			/* Can reach here from VACUUM, so need not have an XID */
+			if (RelationNeedsWAL(gvs.index))
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 			PageRestoreTempPage(resPage, page);
 			MarkBufferDirty(buffer);
diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c
index 79fe6eb8d62..8f6b15d8ee4 100644
--- a/src/backend/access/gist/gist.c
+++ b/src/backend/access/gist/gist.c
@@ -16,6 +16,7 @@
 
 #include "access/gist_private.h"
 #include "access/gistscan.h"
+#include "access/walprohibit.h"
 #include "catalog/pg_collation.h"
 #include "commands/vacuum.h"
 #include "miscadmin.h"
@@ -134,6 +135,9 @@ gistbuildempty(Relation index)
 	buffer = ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermitted_HaveXID();
+
 	/* Initialize and xlog buffer */
 	START_CRIT_SECTION();
 	GISTInitBuffer(buffer, F_LEAF);
@@ -467,6 +471,10 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 		if (!is_build && RelationNeedsWAL(rel))
 			XLogEnsureRecordSpace(npage, 1 + npage * 2);
 
+		/* Can reach here from VACUUM, so need not have an XID */
+		if (RelationNeedsWAL(rel))
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		/*
@@ -525,6 +533,10 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 	}
 	else
 	{
+		/* Can reach here from VACUUM, so need not have an XID */
+		if (RelationNeedsWAL(rel))
+			CheckWALPermitted();
+
 		/*
 		 * Enough space.  We always get here if ntup==0.
 		 */
@@ -1665,6 +1677,10 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 
 	if (ndeletable > 0)
 	{
+		/* Can reach here from VACUUM, so need not have an XID */
+		if (RelationNeedsWAL(rel))
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		PageIndexMultiDelete(page, deletable, ndeletable);
diff --git a/src/backend/access/gist/gistvacuum.c b/src/backend/access/gist/gistvacuum.c
index a9c616c7724..ccf9bc0c214 100644
--- a/src/backend/access/gist/gistvacuum.c
+++ b/src/backend/access/gist/gistvacuum.c
@@ -17,6 +17,7 @@
 #include "access/genam.h"
 #include "access/gist_private.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "commands/vacuum.h"
 #include "lib/integerset.h"
 #include "miscadmin.h"
@@ -341,6 +342,10 @@ restart:
 		 */
 		if (ntodelete > 0)
 		{
+			/* Can reach here from VACUUM, so need not have an XID */
+			if (RelationNeedsWAL(rel))
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 
 			MarkBufferDirty(buffer);
@@ -634,6 +639,10 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	 */
 	txid = ReadNextFullTransactionId();
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(info->index))
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* mark the page as deleted */
diff --git a/src/backend/access/hash/hash.c b/src/backend/access/hash/hash.c
index 3ec6d528e77..1d3f4c92f19 100644
--- a/src/backend/access/hash/hash.c
+++ b/src/backend/access/hash/hash.c
@@ -22,6 +22,7 @@
 #include "access/hash_xlog.h"
 #include "access/relscan.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "catalog/index.h"
 #include "commands/progress.h"
 #include "commands/vacuum.h"
@@ -572,6 +573,10 @@ loop_top:
 		goto loop_top;
 	}
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(rel))
+		CheckWALPermitted();
+
 	/* Okay, we're really done.  Update tuple count in metapage. */
 	START_CRIT_SECTION();
 
@@ -787,6 +792,10 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		 */
 		if (ndeletable > 0)
 		{
+			/* Can reach here from VACUUM, so need not have an XID */
+			if (RelationNeedsWAL(rel))
+				CheckWALPermitted();
+
 			/* No ereport(ERROR) until changes are logged */
 			START_CRIT_SECTION();
 
@@ -882,6 +891,10 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		page = BufferGetPage(bucket_buf);
 		bucket_opaque = (HashPageOpaque) PageGetSpecialPointer(page);
 
+		/* Can reach here from VACUUM, so need not have an XID */
+		if (RelationNeedsWAL(rel))
+			CheckWALPermitted();
+
 		/* No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c
index 2ebe671967b..360e30456fe 100644
--- a/src/backend/access/hash/hashinsert.c
+++ b/src/backend/access/hash/hashinsert.c
@@ -17,6 +17,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "storage/buf_internals.h"
 #include "storage/lwlock.h"
@@ -193,6 +194,9 @@ restart_insert:
 	 */
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	/* Do the update.  No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -370,6 +374,10 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 		 */
 		LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+		/* Can reach here from VACUUM, so need not have an XID */
+		if (RelationNeedsWAL(rel))
+			CheckWALPermitted();
+
 		/* No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
diff --git a/src/backend/access/hash/hashovfl.c b/src/backend/access/hash/hashovfl.c
index 00f0a940116..5abba14899e 100644
--- a/src/backend/access/hash/hashovfl.c
+++ b/src/backend/access/hash/hashovfl.c
@@ -19,6 +19,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "utils/rel.h"
 
@@ -312,6 +313,9 @@ _hash_addovflpage(Relation rel, Buffer metabuf, Buffer buf, bool retain_pin)
 
 found:
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	/*
 	 * Do the update.  No ereport(ERROR) until changes are logged. We want to
 	 * log the changes for bitmap page and overflow page together to avoid
@@ -577,6 +581,10 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	if (RelationNeedsWAL(rel))
 		XLogEnsureRecordSpace(HASH_XLOG_FREE_OVFL_BUFS, 4 + nitups);
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(rel))
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -929,7 +937,13 @@ readpage:
 					 * WAL for that.
 					 */
 					if (RelationNeedsWAL(rel))
+					{
+						/*
+						 * Can reach here from VACUUM, so need not have an XID
+						 */
+						CheckWALPermitted();
 						XLogEnsureRecordSpace(0, 3 + nitups);
+					}
 
 					START_CRIT_SECTION();
 
diff --git a/src/backend/access/hash/hashpage.c b/src/backend/access/hash/hashpage.c
index a664ecf494a..faad58297d2 100644
--- a/src/backend/access/hash/hashpage.c
+++ b/src/backend/access/hash/hashpage.c
@@ -30,6 +30,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "port/pg_bitutils.h"
 #include "storage/lmgr.h"
@@ -816,6 +817,9 @@ restart_expand:
 		goto fail;
 	}
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	/*
 	 * Since we are scribbling on the pages in the shared buffers, establish a
 	 * critical section.  Any failure in this next code leaves us with a big
@@ -1172,6 +1176,9 @@ _hash_splitbucket(Relation rel,
 
 				if (PageGetFreeSpaceForMultipleTuples(npage, nitups + 1) < (all_tups_size + itemsz))
 				{
+					/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+					AssertWALPermitted_HaveXID();
+
 					/*
 					 * Change the shared buffer state in critical section,
 					 * otherwise any error could make it unrecoverable.
@@ -1223,6 +1230,9 @@ _hash_splitbucket(Relation rel,
 		/* Exit loop if no more overflow pages in old bucket */
 		if (!BlockNumberIsValid(oblkno))
 		{
+			/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+			AssertWALPermitted_HaveXID();
+
 			/*
 			 * Change the shared buffer state in critical section, otherwise
 			 * any error could make it unrecoverable.
@@ -1269,6 +1279,9 @@ _hash_splitbucket(Relation rel,
 	npage = BufferGetPage(bucket_nbuf);
 	nopaque = (HashPageOpaque) PageGetSpecialPointer(npage);
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	START_CRIT_SECTION();
 
 	oopaque->hasho_flag &= ~LH_BUCKET_BEING_SPLIT;
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index d881f4cd46a..f8cafc378ff 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -46,6 +46,7 @@
 #include "access/transam.h"
 #include "access/valid.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1870,6 +1871,9 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
 	 */
 	CheckForSerializableConflictIn(relation, NULL, InvalidBlockNumber);
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -2143,6 +2147,9 @@ heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples,
 										   &vmbuffer, NULL);
 		page = BufferGetPage(buffer);
 
+		/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+		AssertWALPermitted_HaveXID();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -2661,6 +2668,9 @@ l1:
 							  xid, LockTupleExclusive, true,
 							  &new_xmax, &new_infomask, &new_infomask2);
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -3413,6 +3423,9 @@ l2:
 
 		Assert(HEAP_XMAX_IS_LOCKED_ONLY(infomask_lock_old_tuple));
 
+		/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+		AssertWALPermitted_HaveXID();
+
 		START_CRIT_SECTION();
 
 		/* Clear obsolete visibility flags ... */
@@ -3586,6 +3599,9 @@ l2:
 										   bms_overlap(modified_attrs, id_attrs),
 										   &old_key_copied);
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -4519,6 +4535,9 @@ failed:
 							  GetCurrentTransactionId(), mode, false,
 							  &xid, &new_infomask, &new_infomask2);
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -5310,6 +5329,9 @@ l4:
 								VISIBILITYMAP_ALL_FROZEN))
 			cleared_all_frozen = true;
 
+		/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+		AssertWALPermitted_HaveXID();
+
 		START_CRIT_SECTION();
 
 		/* ... and set them */
@@ -5468,6 +5490,9 @@ heap_finish_speculative(Relation relation, ItemPointer tid)
 	StaticAssertStmt(MaxOffsetNumber < SpecTokenOffsetNumber,
 					 "invalid speculative token constant");
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -5576,6 +5601,9 @@ heap_abort_speculative(Relation relation, ItemPointer tid)
 		elog(ERROR, "attempted to kill a non-speculative tuple");
 	Assert(!HeapTupleHeaderIsHeapOnly(tp.t_data));
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	/*
 	 * No need to check for serializable conflicts here.  There is never a
 	 * need for a combocid, either.  No need to extract replica identity, or
@@ -5722,6 +5750,10 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	if (oldlen != newlen || htup->t_hoff != tuple->t_data->t_hoff)
 		elog(ERROR, "wrong tuple length");
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(relation))
+		CheckWALPermitted();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index 256df4de105..90f43cbcb9b 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -18,6 +18,7 @@
 #include "access/heapam_xlog.h"
 #include "access/htup_details.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "catalog/catalog.h"
 #include "miscadmin.h"
@@ -77,11 +78,11 @@ heap_page_prune_opt(Relation relation, Buffer buffer)
 	TransactionId OldestXmin;
 
 	/*
-	 * We can't write WAL in recovery mode, so there's no point trying to
+	 * We can't write WAL during read-only mode, so there's no point trying to
 	 * clean the page. The primary will likely issue a cleaning WAL record soon
 	 * anyway, so this is no particular loss.
 	 */
-	if (RecoveryInProgress())
+	if (!XLogInsertAllowed())
 		return;
 
 	/*
@@ -225,6 +226,10 @@ heap_page_prune(Relation relation, Buffer buffer, TransactionId OldestXmin,
 									 &prstate);
 	}
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(relation))
+		CheckWALPermitted();
+
 	/* Any error while applying the changes is critical */
 	START_CRIT_SECTION();
 
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 1bbc4598f75..2e5202d8068 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -59,6 +59,7 @@
 #include "access/parallel.h"
 #include "access/transam.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "catalog/storage.h"
@@ -1203,6 +1204,10 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 			 */
 			if (!PageIsAllVisible(page))
 			{
+				/* Can reach here from VACUUM, so need not have an XID */
+				if (RelationNeedsWAL(onerel))
+					CheckWALPermitted();
+
 				START_CRIT_SECTION();
 
 				/* mark buffer dirty before writing a WAL record */
@@ -1471,6 +1476,10 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 		 */
 		if (nfrozen > 0)
 		{
+			/* Can reach here from VACUUM, so need not have an XID */
+			if (RelationNeedsWAL(onerel))
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 
 			MarkBufferDirty(buf);
@@ -1917,6 +1926,10 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	update_vacuum_error_info(vacrelstats, &saved_err_info, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
 							 blkno);
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(onerel))
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	for (; tupindex < dead_tuples->num_tuples; tupindex++)
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 0a51678c40d..30d1d6f34c7 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -88,6 +88,7 @@
 
 #include "access/heapam_xlog.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "miscadmin.h"
 #include "port/pg_bitutils.h"
@@ -270,6 +271,16 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 	map = (uint8 *) PageGetContents(page);
 	LockBuffer(vmBuf, BUFFER_LOCK_EXCLUSIVE);
 
+	/*
+	 * Can reach here from VACUUM or from startup process, so need not have an
+	 * XID.
+	 *
+	 * Recovery in the startup process never have wal prohibit state, skip
+	 * permission check if reach here in the startup process.
+	 */
+	if (RelationNeedsWAL(rel))
+		InRecovery ? AssertWALPermitted() : CheckWALPermitted();
+
 	if (flags != (map[mapByte] >> mapOffset & VISIBILITYMAP_VALID_BITS))
 	{
 		START_CRIT_SECTION();
@@ -476,6 +487,7 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		Buffer		mapBuffer;
 		Page		page;
 		char	   *map;
+		bool		needwal;
 
 		newnblocks = truncBlock + 1;
 
@@ -489,8 +501,14 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		page = BufferGetPage(mapBuffer);
 		map = PageGetContents(page);
 
+		needwal = (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded());
+
 		LockBuffer(mapBuffer, BUFFER_LOCK_EXCLUSIVE);
 
+		/* Can reach here from VACUUM, so need not have an XID */
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -518,7 +536,7 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		 * during recovery.
 		 */
 		MarkBufferDirty(mapBuffer);
-		if (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded())
+		if (needwal)
 			log_newpage_buffer(mapBuffer, false);
 
 		END_CRIT_SECTION();
diff --git a/src/backend/access/nbtree/nbtdedup.c b/src/backend/access/nbtree/nbtdedup.c
index f6be865b17e..a471a4b7ff5 100644
--- a/src/backend/access/nbtree/nbtdedup.c
+++ b/src/backend/access/nbtree/nbtdedup.c
@@ -16,6 +16,7 @@
 
 #include "access/nbtree.h"
 #include "access/nbtxlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "utils/rel.h"
 
@@ -271,6 +272,9 @@ _bt_dedup_one_page(Relation rel, Buffer buf, Relation heapRel,
 		nopaque->btpo_flags &= ~BTP_HAS_GARBAGE;
 	}
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	START_CRIT_SECTION();
 
 	PageRestoreTempPage(newpage, page);
diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c
index b86c122763e..3527c9f4183 100644
--- a/src/backend/access/nbtree/nbtinsert.c
+++ b/src/backend/access/nbtree/nbtinsert.c
@@ -19,6 +19,7 @@
 #include "access/nbtxlog.h"
 #include "access/tableam.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/lmgr.h"
@@ -1246,6 +1247,9 @@ _bt_insertonpg(Relation rel,
 			}
 		}
 
+		/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+		AssertWALPermitted_HaveXID();
+
 		/* Do the update.  No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -1900,13 +1904,17 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf,
 			ropaque->btpo_flags |= BTP_SPLIT_END;
 	}
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	/*
 	 * Right sibling is locked, new siblings are prepared, but original page
 	 * is not updated yet.
 	 *
 	 * NO EREPORT(ERROR) till right sibling is updated.  We can get away with
 	 * not starting the critical section till here because we haven't been
-	 * scribbling on the original page yet; see comments above.
+	 * scribbling on the original page yet; see the comments above for grabbing
+	 * the right sibling.
 	 */
 	START_CRIT_SECTION();
 
@@ -2469,6 +2477,9 @@ _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf)
 	right_item = CopyIndexTuple(item);
 	BTreeTupleSetDownLink(right_item, rbkno);
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	/* NO EREPORT(ERROR) from here till newroot op is logged */
 	START_CRIT_SECTION();
 
diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c
index 75628e0eb98..09b45fbb559 100644
--- a/src/backend/access/nbtree/nbtpage.c
+++ b/src/backend/access/nbtree/nbtpage.c
@@ -26,6 +26,7 @@
 #include "access/nbtxlog.h"
 #include "access/tableam.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
@@ -201,6 +202,10 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	LockBuffer(metabuf, BUFFER_LOCK_UNLOCK);
 	LockBuffer(metabuf, BT_WRITE);
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(rel))
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* upgrade meta-page if needed */
@@ -376,6 +381,10 @@ _bt_getroot(Relation rel, int access)
 		/* Get raw page pointer for metapage */
 		metapg = BufferGetPage(metabuf);
 
+		/* Can reach here from VACUUM, so need not have an XID */
+		if (RelationNeedsWAL(rel))
+			CheckWALPermitted();
+
 		/* NO ELOG(ERROR) till meta is updated */
 		START_CRIT_SECTION();
 
@@ -1068,6 +1077,10 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 		}
 	}
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(rel))
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -1195,6 +1208,9 @@ _bt_delitems_delete(Relation rel, Buffer buf,
 		latestRemovedXid =
 			_bt_xid_horizon(rel, heapRel, page, deletable, ndeletable);
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -1812,6 +1828,10 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	 */
 	PredicateLockPageCombine(rel, leafblkno, leafrightsib);
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(rel))
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -2168,6 +2188,10 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 	 * Here we begin doing the deletion.
 	 */
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(rel))
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
diff --git a/src/backend/access/spgist/spgdoinsert.c b/src/backend/access/spgist/spgdoinsert.c
index 934d65b89f2..003b5e80f21 100644
--- a/src/backend/access/spgist/spgdoinsert.c
+++ b/src/backend/access/spgist/spgdoinsert.c
@@ -18,6 +18,7 @@
 #include "access/genam.h"
 #include "access/spgist_private.h"
 #include "access/spgxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -214,6 +215,9 @@ addLeafTuple(Relation index, SpGistState *state, SpGistLeafTuple leafTuple,
 	xlrec.offnumParent = InvalidOffsetNumber;
 	xlrec.nodeI = 0;
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	START_CRIT_SECTION();
 
 	if (current->offnum == InvalidOffsetNumber ||
@@ -458,6 +462,9 @@ moveLeafs(Relation index, SpGistState *state,
 
 	leafdata = leafptr = palloc(size);
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	START_CRIT_SECTION();
 
 	/* copy all the old tuples to new page, unless they're dead */
@@ -1109,6 +1116,9 @@ doPickSplit(Relation index, SpGistState *state,
 
 	leafdata = leafptr = (char *) palloc(totalLeafSizes);
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	/* Here we begin making the changes to the target pages */
 	START_CRIT_SECTION();
 
@@ -1517,6 +1527,9 @@ spgAddNodeAction(Relation index, SpGistState *state,
 	if (PageGetExactFreeSpace(current->page) >=
 		newInnerTuple->size - innerTuple->size)
 	{
+		/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+		AssertWALPermitted_HaveXID();
+
 		/*
 		 * We can replace the inner tuple by new version in-place
 		 */
@@ -1603,6 +1616,9 @@ spgAddNodeAction(Relation index, SpGistState *state,
 		else
 			xlrec.parentBlk = 2;
 
+		/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+		AssertWALPermitted_HaveXID();
+
 		START_CRIT_SECTION();
 
 		/* insert new ... */
@@ -1788,6 +1804,9 @@ spgSplitNodeAction(Relation index, SpGistState *state,
 									&xlrec.newPage);
 	}
 
+	/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c
index bd98707f3c0..39bace9e490 100644
--- a/src/backend/access/spgist/spgvacuum.c
+++ b/src/backend/access/spgist/spgvacuum.c
@@ -19,6 +19,7 @@
 #include "access/spgist_private.h"
 #include "access/spgxlog.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/storage_xlog.h"
 #include "commands/vacuum.h"
@@ -323,6 +324,10 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 	if (nDeletable != xlrec.nDead + xlrec.nPlaceholder + xlrec.nMove)
 		elog(ERROR, "inconsistent counts of deletable tuples");
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(index))
+		CheckWALPermitted();
+
 	/* Do the updates */
 	START_CRIT_SECTION();
 
@@ -447,6 +452,10 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 	if (xlrec.nDelete == 0)
 		return;					/* nothing more to do */
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(index))
+		CheckWALPermitted();
+
 	/* Do the update */
 	START_CRIT_SECTION();
 
@@ -505,6 +514,10 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer)
 	xlrec.nToPlaceholder = 0;
 	xlrec.newestRedirectXid = InvalidTransactionId;
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	if (RelationNeedsWAL(index))
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index ce84dac0c40..2b7b2ccad31 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -73,6 +73,7 @@
 #include "access/transam.h"
 #include "access/twophase.h"
 #include "access/twophase_rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1143,6 +1144,9 @@ GetNewMultiXactId(int nmembers, MultiXactOffset *offset)
 
 	ExtendMultiXactMember(nextOffset, nmembers);
 
+	/* Can reach here from VACUUM, so need not have an XID */
+	CheckWALPermitted();
+
 	/*
 	 * Critical section from here until caller has written the data into the
 	 * just-reserved SLRU space; we don't want to error out with a partly
@@ -2942,7 +2946,7 @@ TruncateMultiXact(MultiXactId newOldestMulti, Oid newOldestMultiDB)
 	mxtruncinfo trunc;
 	MultiXactId earliest;
 
-	Assert(!RecoveryInProgress());
+	Assert(XLogInsertAllowed());
 	Assert(MultiXactState->finishedStartup);
 
 	/*
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 9b2e59bf0ec..0fa01f241f3 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -82,6 +82,7 @@
 #include "access/transam.h"
 #include "access/twophase.h"
 #include "access/twophase_rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1112,6 +1113,9 @@ EndPrepare(GlobalTransaction gxact)
 	 */
 	XLogEnsureRecordSpace(0, records.num_chunks);
 
+	/* Recording transaction prepares, so we'll have an XID */
+	AssertWALPermitted_HaveXID();
+
 	START_CRIT_SECTION();
 
 	MyProc->delayChkpt = true;
@@ -2204,6 +2208,9 @@ RecordTransactionCommitPrepared(TransactionId xid,
 	replorigin = (replorigin_session_origin != InvalidRepOriginId &&
 				  replorigin_session_origin != DoNotReplicateId);
 
+	/* COMMIT PREPARED need not have an XID */
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* See notes in RecordTransactionCommit */
@@ -2294,6 +2301,9 @@ RecordTransactionAbortPrepared(TransactionId xid,
 		elog(PANIC, "cannot abort transaction %u, it was already committed",
 			 xid);
 
+	/* ROLLBACK PREPARED need not have an XID */
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
diff --git a/src/backend/access/transam/varsup.c b/src/backend/access/transam/varsup.c
index e14b53bf9e3..365de44321d 100644
--- a/src/backend/access/transam/varsup.c
+++ b/src/backend/access/transam/varsup.c
@@ -17,6 +17,7 @@
 #include "access/commit_ts.h"
 #include "access/subtrans.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "commands/dbcommands.h"
@@ -73,6 +74,9 @@ GetNewTransactionId(bool isSubXact)
 	if (RecoveryInProgress())
 		elog(ERROR, "cannot assign TransactionIds during recovery");
 
+	/* Cannot assign transaction id in read-only mode */
+	CheckWALPermitted();
+
 	LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
 
 	full_xid = ShmemVariableCache->nextFullXid;
diff --git a/src/backend/access/transam/walprohibit.c b/src/backend/access/transam/walprohibit.c
index a8cda2fafbc..896f0917cef 100644
--- a/src/backend/access/transam/walprohibit.c
+++ b/src/backend/access/transam/walprohibit.c
@@ -16,6 +16,16 @@
 #include "postmaster/bgwriter.h"
 #include "storage/procsignal.h"
 
+/*
+ * Assert flag to enforce WAL insert permission check rule before starting a
+ * critical section for the WAL writes.  For this, either of CheckWALPermitted,
+ * AssertWALPermitted_HaveXID, or AssertWALPermitted must be called before
+ * starting the critical section.
+ */
+#ifdef USE_ASSERT_CHECKING
+WALPermitCheckState walpermit_checked_state = WALPERMIT_UNCHECKED;
+#endif
+
 /*
  * ProcessBarrierWALProhibit()
  *
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 4fbfcdbb965..02830cbf85d 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -26,6 +26,7 @@
 #include "access/subtrans.h"
 #include "access/transam.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1292,6 +1293,9 @@ RecordTransactionCommit(void)
 		/* Tell bufmgr and smgr to prepare for commit */
 		BufmgrCommit();
 
+		/* We'll be reaching here with valid XID only. */
+		AssertWALPermitted_HaveXID();
+
 		/*
 		 * Mark ourselves as within our "commit critical section".  This
 		 * forces any concurrent checkpoint to wait until we've updated
@@ -1652,6 +1656,9 @@ RecordTransactionAbort(bool isSubXact)
 		elog(PANIC, "cannot abort transaction %u, it was already committed",
 			 xid);
 
+	/* We'll be reaching here with valid XID only. */
+	AssertWALPermitted_HaveXID();
+
 	/* Fetch the data we need for the abort record */
 	nrels = smgrGetPendingDeletes(false, &rels);
 	nchildren = xactGetCommittedChildren(&children);
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 64cc347caa2..d5d184b3a50 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -1027,7 +1027,7 @@ XLogInsertRecord(XLogRecData *rdata,
 
 	/* cross-check on whether we should be here or not */
 	if (!XLogInsertAllowed())
-		elog(ERROR, "cannot make new WAL entries during recovery");
+		elog(ERROR, "cannot make new WAL entries now");
 
 	/*----------
 	 *
@@ -2862,9 +2862,11 @@ XLogFlush(XLogRecPtr record)
 	 * trying to flush the WAL, we should update minRecoveryPoint instead. We
 	 * test XLogInsertAllowed(), not InRecovery, because we need checkpointer
 	 * to act this way too, and because when it tries to write the
-	 * end-of-recovery checkpoint, it should indeed flush.
+	 * end-of-recovery checkpoint, it should indeed flush. Also, WAL prohibit
+	 * state should not restrict WAL flushing. Otherwise, the dirty buffer
+	 * cannot be evicted until WAL has been flushed up to the buffer's LSN.
 	 */
-	if (!XLogInsertAllowed())
+	if (!XLogInsertAllowed() && !IsWALProhibited())
 	{
 		UpdateMinRecoveryPoint(record, false);
 		return;
@@ -8836,6 +8838,8 @@ CreateCheckPoint(int flags)
 	/* sanity check */
 	if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
 		elog(ERROR, "can't create a checkpoint during recovery");
+	if (!XLogInsertAllowed() && !RecoveryInProgress())
+		elog(ERROR, "can't create a checkpoint while system is read only ");
 
 	/*
 	 * Initialize InitXLogInsert working areas before entering the critical
@@ -8865,6 +8869,8 @@ CreateCheckPoint(int flags)
 	MemSet(&CheckpointStats, 0, sizeof(CheckpointStats));
 	CheckpointStats.ckpt_start_t = GetCurrentTimestamp();
 
+	AssertWALPermitted();
+
 	/*
 	 * Use a critical section to force system panic if we have trouble.
 	 */
@@ -9093,6 +9099,8 @@ CreateCheckPoint(int flags)
 	if (!shutdown && XLogStandbyInfoActive())
 		LogStandbySnapshot();
 
+	AssertWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -9250,6 +9258,8 @@ CreateEndOfRecoveryRecord(void)
 
 	LocalSetXLogInsertAllowed();
 
+	AssertWALPermitted();
+
 	START_CRIT_SECTION();
 
 	XLogBeginInsert();
@@ -9890,7 +9900,7 @@ void
 UpdateFullPageWrites(void)
 {
 	XLogCtlInsert *Insert = &XLogCtl->Insert;
-	bool		recoveryInProgress;
+	bool		WALInsertAllowed;
 
 	/*
 	 * Do nothing if full_page_writes has not been changed.
@@ -9904,10 +9914,10 @@ UpdateFullPageWrites(void)
 
 	/*
 	 * Perform this outside critical section so that the WAL insert
-	 * initialization done by RecoveryInProgress() doesn't trigger an
+	 * initialization done by XLogInsertAllowed() doesn't trigger an
 	 * assertion failure.
 	 */
-	recoveryInProgress = RecoveryInProgress();
+	WALInsertAllowed = XLogInsertAllowed();
 
 	START_CRIT_SECTION();
 
@@ -9929,8 +9939,11 @@ UpdateFullPageWrites(void)
 	 * Write an XLOG_FPW_CHANGE record. This allows us to keep track of
 	 * full_page_writes during archive recovery, if required.
 	 */
-	if (XLogStandbyInfoActive() && !recoveryInProgress)
+	if (XLogStandbyInfoActive() && WALInsertAllowed)
 	{
+		/* Assured that WAL permission has been checked */
+		AssertWALPermitted();
+
 		XLogBeginInsert();
 		XLogRegisterData((char *) (&fullPageWrites), sizeof(bool));
 
diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c
index b21679f09eb..d69f6ca427a 100644
--- a/src/backend/access/transam/xloginsert.c
+++ b/src/backend/access/transam/xloginsert.c
@@ -19,6 +19,7 @@
 
 #include "postgres.h"
 
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xlog_internal.h"
@@ -124,9 +125,14 @@ XLogBeginInsert(void)
 	Assert(mainrdata_last == (XLogRecData *) &mainrdata_head);
 	Assert(mainrdata_len == 0);
 
+	/*
+	 * WAL permission must have checked before entering the critical section.
+	 * Otherwise, WAL prohibited error will force system panic.
+	 */
+	Assert(walpermit_checked_state != WALPERMIT_UNCHECKED || !CritSectionCount);
+
 	/* cross-check on whether we should be here or not */
-	if (!XLogInsertAllowed())
-		elog(ERROR, "cannot make new WAL entries during recovery");
+	CheckWALPermitted();
 
 	if (begininsert_called)
 		elog(ERROR, "XLogBeginInsert was already called");
@@ -204,6 +210,9 @@ XLogResetInsertion(void)
 	mainrdata_last = (XLogRecData *) &mainrdata_head;
 	curinsert_flags = 0;
 	begininsert_called = false;
+
+	/* Reset walpermit_checked flag */
+	RESET_WALPERMIT_CHECKED_STATE();
 }
 
 /*
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 6aab73bfd44..f961178b358 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -20,6 +20,7 @@
 #include "access/relation.h"
 #include "access/table.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -378,8 +379,13 @@ fill_seq_with_data(Relation rel, HeapTuple tuple)
 
 	/* check the comment above nextval_internal()'s equivalent call. */
 	if (RelationNeedsWAL(rel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermitted_HaveXID();
+	}
+
 	START_CRIT_SECTION();
 
 	MarkBufferDirty(buf);
@@ -766,8 +772,13 @@ nextval_internal(Oid relid, bool check_permissions)
 	 * (Have to do that here, so we're outside the critical section)
 	 */
 	if (logit && RelationNeedsWAL(seqrel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermitted_HaveXID();
+	}
+
 	/* ready to change the on-disk (or really, in-buffer) tuple */
 	START_CRIT_SECTION();
 
@@ -977,8 +988,13 @@ do_setval(Oid relid, int64 next, bool iscalled)
 
 	/* check the comment above nextval_internal()'s equivalent call. */
 	if (RelationNeedsWAL(seqrel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermitted_HaveXID();
+	}
+
 	/* ready to change the on-disk (or really, in-buffer) tuple */
 	START_CRIT_SECTION();
 
diff --git a/src/backend/commands/variable.c b/src/backend/commands/variable.c
index 484f7ea2c0e..acc34d2ad7c 100644
--- a/src/backend/commands/variable.c
+++ b/src/backend/commands/variable.c
@@ -501,11 +501,14 @@ check_transaction_read_only(bool *newval, void **extra, GucSource source)
 			GUC_check_errmsg("transaction read-write mode must be set before any query");
 			return false;
 		}
-		/* Can't go to r/w mode while recovery is still active */
-		if (RecoveryInProgress())
+		/*
+		 * Can't go to r/w mode while recovery is still active or while in WAL
+		 * prohibit state
+		 */
+		if (!XLogInsertAllowed())
 		{
 			GUC_check_errcode(ERRCODE_FEATURE_NOT_SUPPORTED);
-			GUC_check_errmsg("cannot set transaction read-write mode during recovery");
+			GUC_check_errmsg("cannot set transaction read-write mode while system is read only");
 			return false;
 		}
 	}
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index bbe62b73a08..7e3882a055c 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -934,6 +934,10 @@ RequestCheckpoint(int flags)
 	int			old_failed,
 				old_started;
 
+	/* The checkpoint is allowed in recovery but not in WAL prohibit state */
+	if (!RecoveryInProgress())
+		CheckWALPermitted();
+
 	/*
 	 * If in a standalone backend, just do it ourselves.
 	 */
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 29c920800a6..ba74ddcd249 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -3603,13 +3603,15 @@ MarkBufferDirtyHint(Buffer buffer, bool buffer_std)
 		{
 			/*
 			 * If we must not write WAL, due to a relfilenode-specific
-			 * condition or being in recovery, don't dirty the page.  We can
-			 * set the hint, just not dirty the page as a result so the hint
-			 * is lost when we evict the page or shutdown.
+			 * condition or in general, don't dirty the page.  We can
+			 * set the hint, but must not dirty the page as a result, lest
+			 * we trigger WAL generation. Unless the page is dirtied again
+			 * later, the hint will be lost when the page is evicted, or at
+			 * shutdown.
 			 *
 			 * See src/backend/storage/page/README for longer discussion.
 			 */
-			if (RecoveryInProgress() ||
+			if (!XLogInsertAllowed() ||
 				RelFileNodeSkippingWAL(bufHdr->tag.rnode))
 				return;
 
diff --git a/src/backend/storage/freespace/freespace.c b/src/backend/storage/freespace/freespace.c
index 95a21f6cc38..5faa69fabb9 100644
--- a/src/backend/storage/freespace/freespace.c
+++ b/src/backend/storage/freespace/freespace.c
@@ -24,6 +24,7 @@
 #include "postgres.h"
 
 #include "access/htup_details.h"
+#include "access/walprohibit.h"
 #include "access/xlogutils.h"
 #include "miscadmin.h"
 #include "storage/freespace.h"
@@ -285,12 +286,20 @@ FreeSpaceMapPrepareTruncateRel(Relation rel, BlockNumber nblocks)
 	 */
 	if (first_removed_slot > 0)
 	{
+		bool needwal;
+
 		buf = fsm_readbuf(rel, first_removed_address, false);
 		if (!BufferIsValid(buf))
 			return InvalidBlockNumber;	/* nothing to do; the FSM was already
 										 * smaller */
 		LockBuffer(buf, BUFFER_LOCK_EXCLUSIVE);
 
+		needwal = (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded());
+
+		/* Can reach here from VACUUM, so need not have an XID */
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -305,7 +314,7 @@ FreeSpaceMapPrepareTruncateRel(Relation rel, BlockNumber nblocks)
 		 * during recovery.
 		 */
 		MarkBufferDirty(buf);
-		if (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded())
+		if (needwal)
 			log_newpage_buffer(buf, false);
 
 		END_CRIT_SECTION();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 95989ce79bd..212312d5ae5 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -794,15 +794,15 @@ LockAcquireExtended(const LOCKTAG *locktag,
 	if (lockmode <= 0 || lockmode > lockMethodTable->numLockModes)
 		elog(ERROR, "unrecognized lock mode: %d", lockmode);
 
-	if (RecoveryInProgress() && !InRecovery &&
+	if (!XLogInsertAllowed() && !InRecovery &&
 		(locktag->locktag_type == LOCKTAG_OBJECT ||
 		 locktag->locktag_type == LOCKTAG_RELATION) &&
 		lockmode > RowExclusiveLock)
 		ereport(ERROR,
 				(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
-				 errmsg("cannot acquire lock mode %s on database objects while recovery is in progress",
+				 errmsg("cannot acquire lock mode %s on database objects while system is read only",
 						lockMethodTable->lockModeNames[lockmode]),
-				 errhint("Only RowExclusiveLock or less can be acquired on database objects during recovery.")));
+				 errhint("Only RowExclusiveLock or less can be acquired on database objects while system is read only")));
 
 #ifdef LOCK_DEBUG
 	if (LOCK_DEBUG_ENABLED(locktag))
diff --git a/src/backend/utils/cache/relmapper.c b/src/backend/utils/cache/relmapper.c
index 671fbb0ed5c..ec48073bbf1 100644
--- a/src/backend/utils/cache/relmapper.c
+++ b/src/backend/utils/cache/relmapper.c
@@ -43,6 +43,7 @@
 #include <sys/stat.h>
 #include <unistd.h>
 
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -841,6 +842,9 @@ write_relmap_file(bool shared, RelMapFile *newmap,
 		xl_relmap_update xlrec;
 		XLogRecPtr	lsn;
 
+		/* Must be performing an INSERT or UPDATE, so we'll have an XID */
+		AssertWALPermitted_HaveXID();
+
 		/* now errors are fatal ... */
 		START_CRIT_SECTION();
 
diff --git a/src/include/access/walprohibit.h b/src/include/access/walprohibit.h
index 163fe0d2fce..1adcfc571d6 100644
--- a/src/include/access/walprohibit.h
+++ b/src/include/access/walprohibit.h
@@ -19,8 +19,8 @@ extern bool ProcessBarrierWALProhibit(void);
 extern void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
 
 /* WAL Prohibit States */
-#define	WALPROHIBIT_STATE_READ_WRITE		0x0000
-#define	WALPROHIBIT_STATE_READ_ONLY			0x0001
+#define	WALPROHIBIT_STATE_READ_WRITE		0x0000	/* WAL permitted */
+#define	WALPROHIBIT_STATE_READ_ONLY			0x0001	/* WAL prohibited */
 
 /*
  * The bit is used in state transition from one state to another.  When this
@@ -29,4 +29,49 @@ extern void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
  */
 #define WALPROHIBIT_TRANSITION_IN_PROGRESS	0x0002
 
+/* Never reaches when WAL is prohibited. */
+static inline void
+AssertWALPermitted(void)
+{
+	/*
+	 * Recovery in the startup process never is in wal prohibited state.
+	 */
+	Assert(InRecovery || XLogInsertAllowed());
+
+#ifdef USE_ASSERT_CHECKING
+	walpermit_checked_state = WALPERMIT_CHECKED;
+#endif
+}
+
+/*
+ * XID-bearing transactions are killed off by "ALTER SYSTEM READ ONLY", so any
+ * part of the code that can only be reached with an XID assigned is never
+ * reached when WAL is prohibited.
+ */
+static inline void
+AssertWALPermitted_HaveXID(void)
+{
+	Assert(FullTransactionIdIsValid(GetTopFullTransactionIdIfAny()));
+	AssertWALPermitted();
+}
+
+/*
+ * In opposite to the above assertion if a transaction doesn't have valid XID
+ * then it won't be killed while changing the system state to WAL prohibited.
+ * Therefore, we need to explicitly error out before entering into the critical
+ * section.
+ */
+static inline void
+CheckWALPermitted(void)
+{
+	if (!XLogInsertAllowed())
+		ereport(ERROR,
+				(errcode(ERRCODE_READ_ONLY_SQL_TRANSACTION),
+				 errmsg("system is now read only")));
+
+#ifdef USE_ASSERT_CHECKING
+	walpermit_checked_state = WALPERMIT_CHECKED;
+#endif
+}
+
 #endif		/* WALPROHIBIT_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 18bc8a7b904..63459305383 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -94,12 +94,37 @@ extern PGDLLIMPORT volatile uint32 CritSectionCount;
 /* in tcop/postgres.c */
 extern void ProcessInterrupts(void);
 
+#ifdef USE_ASSERT_CHECKING
+typedef enum
+{
+	WALPERMIT_UNCHECKED,
+	WALPERMIT_CHECKED,
+	WALPERMIT_CHECKED_AND_USED
+} WALPermitCheckState;
+
+/* in access/walprohibit.c */
+extern WALPermitCheckState walpermit_checked_state;
+
+/*
+ * Reset walpermit_checked flag when no longer in the critical section.
+ * Otherwise, marked checked and used.
+ */
+#define RESET_WALPERMIT_CHECKED_STATE() \
+do { \
+	walpermit_checked_state = CritSectionCount ? \
+	WALPERMIT_CHECKED_AND_USED : WALPERMIT_UNCHECKED; \
+} while(0)
+#else
+#define RESET_WALPERMIT_CHECKED_STATE() ((void) 0)
+#endif
+
 #ifndef WIN32
 
 #define CHECK_FOR_INTERRUPTS() \
 do { \
 	if (unlikely(InterruptPending)) \
 		ProcessInterrupts(); \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 #else							/* WIN32 */
 
@@ -109,6 +134,7 @@ do { \
 		pgwin32_dispatch_queued_signals(); \
 	if (unlikely(InterruptPending)) \
 		ProcessInterrupts(); \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 #endif							/* WIN32 */
 
@@ -135,6 +161,7 @@ do { \
 do { \
 	Assert(CritSectionCount > 0); \
 	CritSectionCount--; \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 
 
-- 
2.22.0

From 75d3fcea66b1f6acee65bfd8506fef67c2152f62 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Tue, 14 Jul 2020 02:30:44 -0400
Subject: [PATCH v3 6/6] Documentation - WIP

TODOs:

1] TBC, the section for pg_is_in_readonly() function, right now it is under "Recovery Information Functions"
2] Documentation regarding ALTER SYSTEM READ/WRITE
---
 src/backend/access/transam/README | 59 ++++++++++++++++++++++++++++---
 src/backend/storage/page/README   | 12 +++----
 2 files changed, 60 insertions(+), 11 deletions(-)

diff --git a/src/backend/access/transam/README b/src/backend/access/transam/README
index eb9aac5fd39..f62929f1660 100644
--- a/src/backend/access/transam/README
+++ b/src/backend/access/transam/README
@@ -433,8 +433,8 @@ to be modified.
 2. START_CRIT_SECTION()  (Any error during the next three steps must cause a
 PANIC because the shared buffers will contain unlogged changes, which we
 have to ensure don't get to disk.  Obviously, you should check conditions
-such as whether there's enough free space on the page before you start the
-critical section.)
+such as whether there's WAL write permission and enough free space on the page
+before you start the critical section.)
 
 3. Apply the required changes to the shared buffer(s).
 
@@ -477,6 +477,54 @@ with the incomplete-split flag set, it will finish the interrupted split by
 inserting the key to the parent, before proceeding.
 
 
+Read only system state
+----------------------
+
+The system state when it is not currently possible to insert write-ahead log
+records, either because the system is still in recovery or because the system
+forced to WAL prohibited by executing ALTER SYSTEM READ ONLY.  We have a
+lower-level defense in XLogBeginInsert() and elsewhere to stop us from modifying
+data during recovery when !XLogInsertAllowed(), but if XLogBeginInsert() is
+inside the critical section we must not depend on it to report an error.
+Otherwise, it will cause PANIC as mentioned previously.
+
+We do not reach the point where we try to write WAL during recovery but ALTER
+SYSTEM READ ONLY can be executed anytime by the user to stop WAL writing.  Any
+backends which receive read-only system state transition barrier interrupt need
+to stop WAL writing immediately.  For barrier absorption the backed(s) will kill
+the running transaction which has valid XID indicates that the transaction has
+performed and/or planning WAL write.  The transaction which doesn't acquire
+valid XID yet or operation such VACUUM or CONCURRENT CREATE INDEX which not
+necessary have valid XID for WAL will not be prevented while barrier processing,
+and those might hit the error from XLogBeginInsert() while trying to write WAL
+in read only system state.  To prevent such error from XLogBeginInsert() inside
+the critical section the WAL write permission has to check before
+START_CRIT_SECTION().
+
+To enforce the practice to check WAL permission before entering into critical
+section for the WAL write, we have added an assert check flag that indicates
+permission has been checked before calling XLogBeginInsert().  If not,
+XLogBeginInsert() will have assertion failure.  WAL permission check is not
+mandatory if the XLogBeginInsert() is not inside the critical section where
+throwing the error is acceptable.  To get permission check flag set either
+CheckWALPermitted(), AssertWALPermitted_HaveXID(), or AssertWALPermitted()
+should be called before START_CRIT_SECTION().  This flag automatically resets
+while exiting from the critical section.  The rule to place either of permission
+check routines will be:
+
+	The places where WAL write operation in critical can be expected without
+	having valid XID (e.g vacuum) need to protect by CheckWALPermitted(), so
+	that error can be reported outside before critical section.
+
+	The places where INSERT and UPDATE are expected which are never happened
+	without valid XID can be checked using AssertWALPermitted_HaveXID.  So that
+	non-assert build will not have the checking overhead.
+
+	The places we know that we cannot be reached in the read-only state and may
+	or may not have XID, but need to ensure the permission has been checked on
+	assert enabled build should use AssertWALPermitted().
+
+
 Constructing a WAL record
 -------------------------
 
@@ -522,7 +570,8 @@ Details of the API functions:
 
 void XLogBeginInsert(void)
 
-    Must be called before XLogRegisterBuffer and XLogRegisterData.
+	Must be called before XLogRegisterBuffer and XLogRegisterData.  WAL
+	permission must be check before calling it in a critical section.
 
 void XLogResetInsertion(void)
 
@@ -630,8 +679,8 @@ If the buffer is clean and checksums are in use then
 MarkBufferDirtyHint() inserts an XLOG_FPI record to ensure that we
 take a full page image that includes the hint. We do this to avoid
 a partial page write, when we write the dirtied page. WAL is not
-written during recovery, so we simply skip dirtying blocks because
-of hints when in recovery.
+written while in read only (i.e. during recovery or in WAL prohibit state), so
+we simply skip dirtying blocks because of hints when in read only.
 
 If you do decide to optimise away a WAL record, then any calls to
 MarkBufferDirty() must be replaced by MarkBufferDirtyHint(),
diff --git a/src/backend/storage/page/README b/src/backend/storage/page/README
index e30d7ac59ad..15f0bb4b7b5 100644
--- a/src/backend/storage/page/README
+++ b/src/backend/storage/page/README
@@ -56,9 +56,9 @@ WAL is a fatal error and prevents further recovery, whereas a checksum failure
 on a normal data block is a hard error but not a critical one for the server,
 even if it is a very bad thing for the user.
 
-New WAL records cannot be written during recovery, so hint bits set during
-recovery must not dirty the page if the buffer is not already dirty, when
-checksums are enabled.  Systems in Hot-Standby mode may benefit from hint bits
-being set, but with checksums enabled, a page cannot be dirtied after setting a
-hint bit (due to the torn page risk). So, it must wait for full-page images
-containing the hint bit updates to arrive from the primary.
+New WAL records cannot be written during recovery or or while in WAL prohibit
+state, so hint bits set during recovery must not dirty the page if the buffer is
+not already dirty, when checksums are enabled.  Systems in Hot-Standby mode may
+benefit from hint bits being set, but with checksums enabled, a page cannot be
+dirtied after setting a hint bit (due to the torn page risk). So, it must wait
+for full-page images containing the hint bit updates to arrive from the primary.
-- 
2.22.0

From c8d1f8abbc521442de2852c9d693e0a7a8477f7d Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 19 Jun 2020 06:27:53 -0400
Subject: [PATCH v4 1/5] Allow error or refusal while absorbing barriers.

Patch by Robert Haas
---
 src/backend/storage/ipc/procsignal.c | 75 +++++++++++++++++++++++-----
 1 file changed, 63 insertions(+), 12 deletions(-)

diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ece..13648887187 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -87,12 +87,16 @@ typedef struct
 #define BARRIER_SHOULD_CHECK(flags, type) \
 	(((flags) & (((uint32) 1) << (uint32) (type))) != 0)
 
+/* Clear the relevant type bit from the flags. */
+#define BARRIER_CLEAR_BIT(flags, type) \
+	((flags) &= ~(((uint32) 1) << (uint32) (type)))
+
 static ProcSignalHeader *ProcSignal = NULL;
 static volatile ProcSignalSlot *MyProcSignalSlot = NULL;
 
 static bool CheckProcSignal(ProcSignalReason reason);
 static void CleanupProcSignalState(int status, Datum arg);
-static void ProcessBarrierPlaceholder(void);
+static bool ProcessBarrierPlaceholder(void);
 
 /*
  * ProcSignalShmemSize
@@ -486,17 +490,59 @@ ProcessProcSignalBarrier(void)
 	flags = pg_atomic_exchange_u32(&MyProcSignalSlot->pss_barrierCheckMask, 0);
 
 	/*
-	 * Process each type of barrier. It's important that nothing we call from
-	 * here throws an error, because pss_barrierCheckMask has already been
-	 * cleared. If we jumped out of here before processing all barrier types,
-	 * then we'd forget about the need to do so later.
-	 *
-	 * NB: It ought to be OK to call the barrier-processing functions
-	 * unconditionally, but it's more efficient to call only the ones that
-	 * might need us to do something based on the flags.
+	 * If there are no flags set, then we can skip doing any real work.
+	 * Otherwise, establish a PG_TRY block, so that we don't lose track of
+	 * which types of barrier processing are needed if an ERROR occurs.
 	 */
-	if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_PLACEHOLDER))
-		ProcessBarrierPlaceholder();
+	if (flags != 0)
+	{
+		PG_TRY();
+		{
+			/*
+			 * Process each type of barrier. The barrier-processing functions
+			 * should normally return true, but may return false if the barrier
+			 * can't be absorbed at the current time. This should be rare,
+			 * because it's pretty expensive.  Every single
+			 * CHECK_FOR_INTERRUPTS() will return here until we manage to
+			 * absorb the barrier, and that cost will add up in a hurry.
+			 *
+			 * NB: It ought to be OK to call the barrier-processing functions
+			 * unconditionally, but it's more efficient to call only the ones
+			 * that might need us to do something based on the flags.
+			 */
+			if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_PLACEHOLDER)
+				&& ProcessBarrierPlaceholder())
+				BARRIER_CLEAR_BIT(flags, PROCSIGNAL_BARRIER_PLACEHOLDER);
+		}
+		PG_CATCH();
+		{
+			/*
+			 * If an ERROR occurred, add any flags that weren't yet handled
+			 * back into pss_barrierCheckMask, and reset the global variables
+			 * so that we try again the next time we check for interrupts.
+			 */
+			pg_atomic_fetch_or_u32(&MyProcSignalSlot->pss_barrierCheckMask,
+								   flags);
+			ProcSignalBarrierPending = true;
+			InterruptPending = true;
+
+			PG_RE_THROW();
+		}
+		PG_END_TRY();
+
+		/*
+		 * If some barrier was not successfully absorbed, we will have to try
+		 * again later.
+		 */
+		if (flags != 0)
+		{
+			pg_atomic_fetch_or_u32(&MyProcSignalSlot->pss_barrierCheckMask,
+								   flags);
+			ProcSignalBarrierPending = true;
+			InterruptPending = true;
+			return;
+		}
+	}
 
 	/*
 	 * State changes related to all types of barriers that might have been
@@ -508,7 +554,7 @@ ProcessProcSignalBarrier(void)
 	pg_atomic_write_u64(&MyProcSignalSlot->pss_barrierGeneration, shared_gen);
 }
 
-static void
+static bool
 ProcessBarrierPlaceholder(void)
 {
 	/*
@@ -518,7 +564,12 @@ ProcessBarrierPlaceholder(void)
 	 * appropriately descriptive. Get rid of this function and instead have
 	 * ProcessBarrierSomethingElse. Most likely, that function should live in
 	 * the file pertaining to that subsystem, rather than here.
+	 *
+	 * The return value should be 'true' if the barrier was successfully
+	 * absorbed and 'false' if not. Note that returning 'false' can lead to
+	 * very frequent retries, so try hard to make that an uncommon case.
 	 */
+	return true;
 }
 
 /*
-- 
2.22.0

From 92bc6df6f0aff4274c9511c0afd94d2759be82a5 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 27 Mar 2020 05:05:38 -0400
Subject: [PATCH v4 2/5] Add alter system read only/write syntax

Note that syntax doesn't have any implementation.
---
 src/backend/nodes/copyfuncs.c    | 12 ++++++++++++
 src/backend/nodes/equalfuncs.c   |  9 +++++++++
 src/backend/nodes/outfuncs.c     | 12 ++++++++++++
 src/backend/nodes/readfuncs.c    | 15 +++++++++++++++
 src/backend/parser/gram.y        | 13 +++++++++++++
 src/backend/tcop/utility.c       | 20 ++++++++++++++++++++
 src/bin/psql/tab-complete.c      |  6 ++++--
 src/include/nodes/nodes.h        |  1 +
 src/include/nodes/parsenodes.h   | 10 ++++++++++
 src/tools/pgindent/typedefs.list |  1 +
 10 files changed, 97 insertions(+), 2 deletions(-)

diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 89c409de664..ba3393b8ccf 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -4020,6 +4020,15 @@ _copyAlterSystemStmt(const AlterSystemStmt *from)
 	return newnode;
 }
 
+static AlterSystemWALProhibitState *
+_copyAlterSystemWALProhibitState(const AlterSystemWALProhibitState *from)
+{
+	AlterSystemWALProhibitState *newnode = makeNode(AlterSystemWALProhibitState);
+
+	COPY_SCALAR_FIELD(WALProhibited);
+	return newnode;
+}
+
 static CreateSeqStmt *
 _copyCreateSeqStmt(const CreateSeqStmt *from)
 {
@@ -5406,6 +5415,9 @@ copyObjectImpl(const void *from)
 		case T_AlterSystemStmt:
 			retval = _copyAlterSystemStmt(from);
 			break;
+		case T_AlterSystemWALProhibitState:
+			retval = _copyAlterSystemWALProhibitState(from);
+			break;
 		case T_CreateSeqStmt:
 			retval = _copyCreateSeqStmt(from);
 			break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e3f33c40be5..b09bff458af 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1769,6 +1769,12 @@ _equalAlterSystemStmt(const AlterSystemStmt *a, const AlterSystemStmt *b)
 	return true;
 }
 
+static bool
+_equalAlterSystemWALProhibitState(const AlterSystemWALProhibitState *a, const AlterSystemWALProhibitState *b)
+{
+	COMPARE_SCALAR_FIELD(WALProhibited);
+	return true;
+}
 
 static bool
 _equalCreateSeqStmt(const CreateSeqStmt *a, const CreateSeqStmt *b)
@@ -3458,6 +3464,9 @@ equal(const void *a, const void *b)
 		case T_AlterSystemStmt:
 			retval = _equalAlterSystemStmt(a, b);
 			break;
+		case T_AlterSystemWALProhibitState:
+			retval = _equalAlterSystemWALProhibitState(a, b);
+			break;
 		case T_CreateSeqStmt:
 			retval = _equalCreateSeqStmt(a, b);
 			break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index e2f177515da..37f297f39a5 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -1358,6 +1358,15 @@ _outAlternativeSubPlan(StringInfo str, const AlternativeSubPlan *node)
 	WRITE_NODE_FIELD(subplans);
 }
 
+static void
+_outAlterSystemWALProhibitState(StringInfo str,
+								const AlterSystemWALProhibitState *node)
+{
+	WRITE_NODE_TYPE("ALTERSYSTEMWALPROHIBITSTATE");
+
+	WRITE_BOOL_FIELD(WALProhibited);
+}
+
 static void
 _outFieldSelect(StringInfo str, const FieldSelect *node)
 {
@@ -3914,6 +3923,9 @@ outNode(StringInfo str, const void *obj)
 			case T_AlternativeSubPlan:
 				_outAlternativeSubPlan(str, obj);
 				break;
+			case T_AlterSystemWALProhibitState:
+				_outAlterSystemWALProhibitState(str, obj);
+				break;
 			case T_FieldSelect:
 				_outFieldSelect(str, obj);
 				break;
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 42050ab7195..0ac826d3c2f 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -2552,6 +2552,19 @@ _readAlternativeSubPlan(void)
 	READ_DONE();
 }
 
+/*
+ * _readAlterSystemWALProhibitState
+ */
+static AlterSystemWALProhibitState *
+_readAlterSystemWALProhibitState(void)
+{
+	READ_LOCALS(AlterSystemWALProhibitState);
+
+	READ_BOOL_FIELD(WALProhibited);
+
+	READ_DONE();
+}
+
 /*
  * _readExtensibleNode
  */
@@ -2874,6 +2887,8 @@ parseNodeString(void)
 		return_value = _readSubPlan();
 	else if (MATCH("ALTERNATIVESUBPLAN", 18))
 		return_value = _readAlternativeSubPlan();
+	else if (MATCH("ALTERSYSTEMWALPROHIBITSTATE", 27))
+		return_value = _readAlterSystemWALProhibitState();
 	else if (MATCH("EXTENSIBLENODE", 14))
 		return_value = _readExtensibleNode();
 	else if (MATCH("PARTITIONBOUNDSPEC", 18))
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index dbb47d49829..6090d18ec61 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -479,6 +479,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
 
 %type <vsetstmt> generic_set set_rest set_rest_more generic_reset reset_rest
 				 SetResetClause FunctionSetResetClause
+%type <boolean> system_readonly_state
 
 %type <node>	TableElement TypedTableElement ConstraintElem TableFuncElement
 %type <node>	columnDef columnOptions
@@ -10172,8 +10173,20 @@ AlterSystemStmt:
 					n->setstmt = $4;
 					$$ = (Node *)n;
 				}
+			| ALTER SYSTEM_P system_readonly_state
+				{
+					AlterSystemWALProhibitState *n = makeNode(AlterSystemWALProhibitState);
+					n->WALProhibited = $3;
+					$$ = (Node *)n;
+				}
 		;
 
+system_readonly_state:
+			 READ ONLY
+					{ $$ = true; }
+			| READ WRITE
+					{ $$ = false; }
+		;
 
 /*****************************************************************************
  *
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 9b0c376c8cb..7af96c77082 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -85,6 +85,7 @@ static void ProcessUtilitySlow(ParseState *pstate,
 							   DestReceiver *dest,
 							   QueryCompletion *qc);
 static void ExecDropStmt(DropStmt *stmt, bool isTopLevel);
+static void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
 
 /*
  * CommandIsReadOnly: is an executable query read-only?
@@ -219,6 +220,7 @@ ClassifyUtilityCommandAsReadOnly(Node *parsetree)
 				return COMMAND_IS_NOT_READ_ONLY;
 			}
 
+		case T_AlterSystemWALProhibitState:
 		case T_AlterSystemStmt:
 			{
 				/*
@@ -834,6 +836,11 @@ standard_ProcessUtility(PlannedStmt *pstmt,
 			AlterSystemSetConfigFile((AlterSystemStmt *) parsetree);
 			break;
 
+		case T_AlterSystemWALProhibitState:
+			PreventInTransactionBlock(isTopLevel, "ALTER SYSTEM");
+			AlterSystemSetWALProhibitState((AlterSystemWALProhibitState *) parsetree);
+			break;
+
 		case T_VariableSetStmt:
 			ExecSetVariableStmt((VariableSetStmt *) parsetree, isTopLevel);
 			break;
@@ -2772,6 +2779,7 @@ CreateCommandTag(Node *parsetree)
 			tag = CMDTAG_REFRESH_MATERIALIZED_VIEW;
 			break;
 
+		case T_AlterSystemWALProhibitState:
 		case T_AlterSystemStmt:
 			tag = CMDTAG_ALTER_SYSTEM;
 			break;
@@ -3636,3 +3644,15 @@ GetCommandLogLevel(Node *parsetree)
 
 	return lev;
 }
+
+/*
+ * AlterSystemSetWALProhibitState
+ *
+ * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
+ */
+static void
+AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
+{
+	/* some code */
+	elog(INFO, "AlterSystemSetWALProhibitState() called");
+}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 8b735476ade..65d1487f80f 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1858,9 +1858,11 @@ psql_completion(const char *text, int start, int end)
 	/* ALTER SERVER <name> VERSION <version> */
 	else if (Matches("ALTER", "SERVER", MatchAny, "VERSION", MatchAny))
 		COMPLETE_WITH("OPTIONS");
-	/* ALTER SYSTEM SET, RESET, RESET ALL */
+	/* ALTER SYSTEM READ, SET, RESET, RESET ALL */
 	else if (Matches("ALTER", "SYSTEM"))
-		COMPLETE_WITH("SET", "RESET");
+		COMPLETE_WITH("SET", "RESET", "READ");
+	else if (Matches("ALTER", "SYSTEM", "READ"))
+		COMPLETE_WITH("ONLY", "WRITE");
 	else if (Matches("ALTER", "SYSTEM", "SET|RESET"))
 		COMPLETE_WITH_QUERY(Query_for_list_of_alter_system_set_vars);
 	else if (Matches("ALTER", "SYSTEM", "SET", MatchAny))
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 381d84b4e4f..17d6942c734 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -412,6 +412,7 @@ typedef enum NodeTag
 	T_RefreshMatViewStmt,
 	T_ReplicaIdentityStmt,
 	T_AlterSystemStmt,
+	T_AlterSystemWALProhibitState,
 	T_CreatePolicyStmt,
 	T_AlterPolicyStmt,
 	T_CreateTransformStmt,
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 151bcdb7ef5..f2c1ae8e3fe 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -3194,6 +3194,16 @@ typedef struct AlterSystemStmt
 	VariableSetStmt *setstmt;	/* SET subcommand */
 } AlterSystemStmt;
 
+/* ----------------------
+ *		Alter System Read Statement
+ * ----------------------
+ */
+typedef struct AlterSystemWALProhibitState
+{
+	NodeTag		type;
+	bool		WALProhibited;
+} AlterSystemWALProhibitState;
+
 /* ----------------------
  *		Cluster Statement (support pbrown's cluster index implementation)
  * ----------------------
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 7eaaad1e140..247bdf1bacc 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -87,6 +87,7 @@ AlterStatsStmt
 AlterSubscriptionStmt
 AlterSubscriptionType
 AlterSystemStmt
+AlterSystemWALProhibitState
 AlterTSConfigType
 AlterTSConfigurationStmt
 AlterTSDictionaryStmt
-- 
2.22.0

From 15aabceff6dcffce60300623fed48ce3a2fced25 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Mon, 27 Jul 2020 02:13:36 -0400
Subject: [PATCH v4 4/5] Error or Assert before START_CRIT_SECTION for WAL
 write

Based on the following criteria added the Assert or the Error when system is
prohibited:

 - Added ERROR for the function which can be reachable without valid XID in case
   of VACUUM or CONCURRENT CREATE INDEX.  For that added common static inline
   function CheckWALPermitted().
 - Added ASSERT for the function which cannot be reached without valid XID;
   Assert to ensure XID validation. For that added AssertWALPermitted_HaveXID().

To enforce the rule to have aforesaid assert or error check before entering a
critical section for WAL write, a new walpermit_checked_state assert only flag
is added. If these check is missing then XLogBeginInsert() will have an
assertion if it is in critical section.

If we are not doing WAL insert inside the critical section then the above
checking is not necessary, we can rely on XLogBeginInsert() for that check &
report an error.
---
 src/backend/access/brin/brin.c            |  4 ++
 src/backend/access/brin/brin_pageops.c    | 21 ++++++++--
 src/backend/access/brin/brin_revmap.c     | 10 ++++-
 src/backend/access/gin/ginbtree.c         | 15 +++++--
 src/backend/access/gin/gindatapage.c      | 18 ++++++--
 src/backend/access/gin/ginfast.c          | 11 ++++-
 src/backend/access/gin/gininsert.c        |  4 ++
 src/backend/access/gin/ginutil.c          |  9 +++-
 src/backend/access/gin/ginvacuum.c        | 11 ++++-
 src/backend/access/gist/gist.c            | 25 +++++++++---
 src/backend/access/gist/gistvacuum.c      | 13 +++++-
 src/backend/access/hash/hash.c            | 19 +++++++--
 src/backend/access/hash/hashinsert.c      |  9 +++-
 src/backend/access/hash/hashovfl.c        | 22 ++++++++--
 src/backend/access/hash/hashpage.c        |  9 ++++
 src/backend/access/heap/heapam.c          | 26 +++++++++++-
 src/backend/access/heap/pruneheap.c       | 12 ++++--
 src/backend/access/heap/vacuumlazy.c      | 18 ++++++--
 src/backend/access/heap/visibilitymap.c   | 22 +++++++++-
 src/backend/access/nbtree/nbtdedup.c      |  3 ++
 src/backend/access/nbtree/nbtinsert.c     | 10 ++++-
 src/backend/access/nbtree/nbtpage.c       | 39 +++++++++++++++---
 src/backend/access/spgist/spgdoinsert.c   | 13 ++++++
 src/backend/access/spgist/spgvacuum.c     | 21 ++++++++--
 src/backend/access/transam/multixact.c    |  5 ++-
 src/backend/access/transam/twophase.c     |  9 ++++
 src/backend/access/transam/varsup.c       |  4 ++
 src/backend/access/transam/walprohibit.c  | 10 +++++
 src/backend/access/transam/xact.c         |  6 +++
 src/backend/access/transam/xlog.c         | 27 ++++++++----
 src/backend/access/transam/xloginsert.c   | 13 +++++-
 src/backend/commands/sequence.c           | 16 ++++++++
 src/backend/commands/variable.c           |  9 ++--
 src/backend/postmaster/checkpointer.c     |  4 ++
 src/backend/storage/buffer/bufmgr.c       | 10 +++--
 src/backend/storage/freespace/freespace.c | 10 ++++-
 src/backend/storage/lmgr/lock.c           |  6 +--
 src/backend/utils/cache/relmapper.c       |  3 ++
 src/include/access/walprohibit.h          | 50 ++++++++++++++++++++++-
 src/include/miscadmin.h                   | 27 ++++++++++++
 40 files changed, 502 insertions(+), 71 deletions(-)

diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index 7db3ae5ee0c..3ea57bbb4bd 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -24,6 +24,7 @@
 #include "access/relscan.h"
 #include "access/table.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/index.h"
 #include "catalog/pg_am.h"
@@ -758,6 +759,9 @@ brinbuildempty(Relation index)
 		ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermittedHaveXID();
+
 	/* Initialize and xlog metabuffer. */
 	START_CRIT_SECTION();
 	brin_metapage_init(BufferGetPage(metabuf), BrinGetPagesPerRange(index),
diff --git a/src/backend/access/brin/brin_pageops.c b/src/backend/access/brin/brin_pageops.c
index 87de0b855b5..8b377a679ab 100644
--- a/src/backend/access/brin/brin_pageops.c
+++ b/src/backend/access/brin/brin_pageops.c
@@ -14,6 +14,7 @@
 #include "access/brin_pageops.h"
 #include "access/brin_revmap.h"
 #include "access/brin_xlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -65,6 +66,7 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 	Buffer		newbuf;
 	BlockNumber newblk = InvalidBlockNumber;
 	bool		extended;
+	bool		needwal = RelationNeedsWAL(idxrel);
 
 	Assert(newsz == MAXALIGN(newsz));
 
@@ -176,13 +178,16 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 	if (((BrinPageFlags(oldpage) & BRIN_EVACUATE_PAGE) == 0) &&
 		brin_can_do_samepage_update(oldbuf, origsz, newsz))
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 		if (!PageIndexTupleOverwrite(oldpage, oldoff, (Item) unconstify(BrinTuple *, newtup), newsz))
 			elog(ERROR, "failed to replace BRIN tuple");
 		MarkBufferDirty(oldbuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(idxrel))
+		if (needwal)
 		{
 			xl_brin_samepage_update xlrec;
 			XLogRecPtr	recptr;
@@ -240,6 +245,9 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 
 		revmapbuf = brinLockRevmapPageForUpdate(revmap, heapBlk);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		/*
@@ -267,7 +275,7 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 		MarkBufferDirty(revmapbuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(idxrel))
+		if (needwal)
 		{
 			xl_brin_update xlrec;
 			XLogRecPtr	recptr;
@@ -351,6 +359,7 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	Buffer		revmapbuf;
 	ItemPointerData tid;
 	bool		extended;
+	bool		needwal;
 
 	Assert(itemsz == MAXALIGN(itemsz));
 
@@ -405,6 +414,10 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	page = BufferGetPage(*buffer);
 	blk = BufferGetBlockNumber(*buffer);
 
+	needwal = RelationNeedsWAL(idxrel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Execute the actual insertion */
 	START_CRIT_SECTION();
 	if (extended)
@@ -424,7 +437,7 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	MarkBufferDirty(revmapbuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(idxrel))
+	if (needwal)
 	{
 		xl_brin_insert xlrec;
 		XLogRecPtr	recptr;
@@ -881,6 +894,8 @@ brin_initialize_empty_new_buffer(Relation idxrel, Buffer buffer)
 			   "brin_initialize_empty_new_buffer: initializing blank page %u",
 			   BufferGetBlockNumber(buffer)));
 
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 	page = BufferGetPage(buffer);
 	brin_page_init(page, BRIN_PAGETYPE_REGULAR);
diff --git a/src/backend/access/brin/brin_revmap.c b/src/backend/access/brin/brin_revmap.c
index e8b8308f82e..242c61b1f1e 100644
--- a/src/backend/access/brin/brin_revmap.c
+++ b/src/backend/access/brin/brin_revmap.c
@@ -26,6 +26,7 @@
 #include "access/brin_tuple.h"
 #include "access/brin_xlog.h"
 #include "access/rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -333,6 +334,7 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 	OffsetNumber revmapOffset;
 	OffsetNumber regOffset;
 	ItemId		lp;
+	bool		needwal;
 
 	revmap = brinRevmapInitialize(idxrel, &pagesPerRange, NULL);
 
@@ -397,6 +399,10 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 	 * crashed or aborted summarization; remove them silently.
 	 */
 
+	needwal = RelationNeedsWAL(idxrel);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	ItemPointerSetInvalid(&invalidIptr);
@@ -408,7 +414,7 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 	MarkBufferDirty(regBuf);
 	MarkBufferDirty(revmapBuf);
 
-	if (RelationNeedsWAL(idxrel))
+	if (needwal)
 	{
 		xl_brin_desummarize xlrec;
 		XLogRecPtr	recptr;
@@ -606,6 +612,8 @@ revmap_physical_extend(BrinRevmap *revmap)
 		return;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Ok, we have now locked the metapage and the target block. Re-initialize
 	 * the target block as a revmap page, and update the metapage.
diff --git a/src/backend/access/gin/ginbtree.c b/src/backend/access/gin/ginbtree.c
index 8d08b05f515..0d9997463b4 100644
--- a/src/backend/access/gin/ginbtree.c
+++ b/src/backend/access/gin/ginbtree.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/predicate.h"
@@ -333,6 +334,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 {
 	Page		page = BufferGetPage(stack->buffer);
 	bool		result;
+	bool		needwal;
 	GinPlaceToPageRC rc;
 	uint16		xlflags = 0;
 	Page		childpage = NULL;
@@ -378,6 +380,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 								 insertdata, updateblkno,
 								 &ptp_workspace,
 								 &newlpage, &newrpage);
+	needwal = RelationNeedsWAL(btree->index) && !btree->isBuild;
 
 	if (rc == GPTP_NO_WORK)
 	{
@@ -386,10 +389,13 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 	}
 	else if (rc == GPTP_INSERT)
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		/* It will fit, perform the insertion */
 		START_CRIT_SECTION();
 
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogBeginInsert();
 			XLogRegisterBuffer(0, stack->buffer, REGBUF_STANDARD);
@@ -410,7 +416,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 			MarkBufferDirty(childbuf);
 		}
 
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 			ginxlogInsert xlrec;
@@ -548,6 +554,9 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 			}
 		}
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/*
 		 * OK, we have the new contents of the left page in a temporary copy
 		 * now (newlpage), and likewise for the new contents of the
@@ -588,7 +597,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 		}
 
 		/* write WAL record */
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gin/gindatapage.c b/src/backend/access/gin/gindatapage.c
index 7a2690e97f2..0abc5990100 100644
--- a/src/backend/access/gin/gindatapage.c
+++ b/src/backend/access/gin/gindatapage.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "lib/ilist.h"
 #include "miscadmin.h"
@@ -811,6 +812,7 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 	if (removedsomething)
 	{
 		bool		modified;
+		bool		needwal;
 
 		/*
 		 * Make sure we have a palloc'd copy of all segments, after the first
@@ -835,8 +837,12 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 			}
 		}
 
-		if (RelationNeedsWAL(indexrel))
+		needwal = RelationNeedsWAL(indexrel);
+		if (needwal)
+		{
+			CheckWALPermitted();
 			computeLeafRecompressWALData(leaf);
+		}
 
 		/* Apply changes to page */
 		START_CRIT_SECTION();
@@ -845,7 +851,7 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 
 		MarkBufferDirty(buffer);
 
-		if (RelationNeedsWAL(indexrel))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
@@ -1777,6 +1783,7 @@ createPostingTree(Relation index, ItemPointerData *items, uint32 nitems,
 	int			nrootitems;
 	int			rootsize;
 	bool		is_build = (buildStats != NULL);
+	bool		needwal;
 
 	/* Construct the new root page in memory first. */
 	tmppage = (Page) palloc(BLCKSZ);
@@ -1825,12 +1832,17 @@ createPostingTree(Relation index, ItemPointerData *items, uint32 nitems,
 	 */
 	PredicateLockPageSplit(index, BufferGetBlockNumber(entrybuffer), blkno);
 
+	needwal = RelationNeedsWAL(index) && !is_build;
+
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	PageRestoreTempPage(tmppage, page);
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index) && !is_build)
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogCreatePostingTree data;
diff --git a/src/backend/access/gin/ginfast.c b/src/backend/access/gin/ginfast.c
index 2e41b34d8d5..b8c2a993408 100644
--- a/src/backend/access/gin/ginfast.c
+++ b/src/backend/access/gin/ginfast.c
@@ -20,6 +20,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
 #include "catalog/pg_am.h"
@@ -68,6 +69,8 @@ writeListPage(Relation index, Buffer buffer,
 	PGAlignedBlock workspace;
 	char	   *ptr;
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	GinInitBuffer(buffer, GIN_LIST);
@@ -548,6 +551,7 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 	Page		metapage;
 	GinMetaPageData *metadata;
 	BlockNumber blknoToDelete;
+	bool		needwal = RelationNeedsWAL(index);
 
 	metapage = BufferGetPage(metabuffer);
 	metadata = GinPageGetMeta(metapage);
@@ -586,8 +590,11 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 		 * prepare the XLogInsert machinery for that before entering the
 		 * critical section.
 		 */
-		if (RelationNeedsWAL(index))
+		if (needwal)
+		{
+			CheckWALPermitted();
 			XLogEnsureRecordSpace(data.ndeleted, 0);
+		}
 
 		START_CRIT_SECTION();
 
@@ -625,7 +632,7 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 			MarkBufferDirty(buffers[i]);
 		}
 
-		if (RelationNeedsWAL(index))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gin/gininsert.c b/src/backend/access/gin/gininsert.c
index 77433dc8a41..989d82ffcaf 100644
--- a/src/backend/access/gin/gininsert.c
+++ b/src/backend/access/gin/gininsert.c
@@ -17,6 +17,7 @@
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/index.h"
 #include "miscadmin.h"
@@ -447,6 +448,9 @@ ginbuildempty(Relation index)
 		ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(RootBuffer, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermittedHaveXID();
+
 	/* Initialize and xlog metabuffer and root buffer. */
 	START_CRIT_SECTION();
 	GinInitMetabuffer(MetaBuffer);
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index a400f1fedbc..1a7d777cc42 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -17,6 +17,7 @@
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
 #include "access/reloptions.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/pg_collation.h"
 #include "catalog/pg_type.h"
@@ -658,12 +659,18 @@ ginUpdateStats(Relation index, const GinStatsData *stats, bool is_build)
 	Buffer		metabuffer;
 	Page		metapage;
 	GinMetaPageData *metadata;
+	bool		needwal;
 
 	metabuffer = ReadBuffer(index, GIN_METAPAGE_BLKNO);
 	LockBuffer(metabuffer, GIN_EXCLUSIVE);
 	metapage = BufferGetPage(metabuffer);
 	metadata = GinPageGetMeta(metapage);
 
+	needwal = RelationNeedsWAL(index) && !is_build;
+
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	metadata->nTotalPages = stats->nTotalPages;
@@ -683,7 +690,7 @@ ginUpdateStats(Relation index, const GinStatsData *stats, bool is_build)
 
 	MarkBufferDirty(metabuffer);
 
-	if (RelationNeedsWAL(index) && !is_build)
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogUpdateMeta data;
diff --git a/src/backend/access/gin/ginvacuum.c b/src/backend/access/gin/ginvacuum.c
index 8ae4fd95a7b..a8443e73cd5 100644
--- a/src/backend/access/gin/ginvacuum.c
+++ b/src/backend/access/gin/ginvacuum.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "commands/vacuum.h"
 #include "miscadmin.h"
@@ -136,6 +137,7 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	Page		page,
 				parentPage;
 	BlockNumber rightlink;
+	bool		needwal;
 
 	/*
 	 * This function MUST be called only if someone of parent pages hold
@@ -159,6 +161,10 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	 */
 	PredicateLockPageCombine(gvs->index, deleteBlkno, rightlink);
 
+	needwal = RelationNeedsWAL(gvs->index);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* Unlink the page by changing left sibling's rightlink */
@@ -195,7 +201,7 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	MarkBufferDirty(lBuffer);
 	MarkBufferDirty(dBuffer);
 
-	if (RelationNeedsWAL(gvs->index))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogDeletePage data;
@@ -650,6 +656,9 @@ ginbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 
 		if (resPage)
 		{
+			if (RelationNeedsWAL(gvs.index))
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 			PageRestoreTempPage(resPage, page);
 			MarkBufferDirty(buffer);
diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c
index 79fe6eb8d62..fca136b0d41 100644
--- a/src/backend/access/gist/gist.c
+++ b/src/backend/access/gist/gist.c
@@ -16,6 +16,7 @@
 
 #include "access/gist_private.h"
 #include "access/gistscan.h"
+#include "access/walprohibit.h"
 #include "catalog/pg_collation.h"
 #include "commands/vacuum.h"
 #include "miscadmin.h"
@@ -134,6 +135,9 @@ gistbuildempty(Relation index)
 	buffer = ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermittedHaveXID();
+
 	/* Initialize and xlog buffer */
 	START_CRIT_SECTION();
 	GISTInitBuffer(buffer, F_LEAF);
@@ -233,6 +237,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 	XLogRecPtr	recptr;
 	int			i;
 	bool		is_split;
+	bool		needwal = RelationNeedsWAL(rel);
 
 	/*
 	 * Refuse to modify a page that's incompletely split. This should not
@@ -464,9 +469,12 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 		 * insertion for that. NB: The number of pages and data segments
 		 * specified here must match the calculations in gistXLogSplit()!
 		 */
-		if (!is_build && RelationNeedsWAL(rel))
+		if (!is_build && needwal)
 			XLogEnsureRecordSpace(npage, 1 + npage * 2);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		/*
@@ -499,7 +507,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 			recptr = GistBuildLSN;
 		else
 		{
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 				recptr = gistXLogSplit(is_leaf,
 									   dist, oldrlink, oldnsn, leftchildbuf,
 									   markfollowright);
@@ -525,6 +533,9 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 	}
 	else
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		/*
 		 * Enough space.  We always get here if ntup==0.
 		 */
@@ -566,7 +577,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 			recptr = GistBuildLSN;
 		else
 		{
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 			{
 				OffsetNumber ndeloffs = 0,
 							deloffs[1];
@@ -1640,6 +1651,7 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 	OffsetNumber offnum,
 				maxoff;
 	TransactionId latestRemovedXid = InvalidTransactionId;
+	bool		needwal = RelationNeedsWAL(rel);
 
 	Assert(GistPageIsLeaf(page));
 
@@ -1658,13 +1670,16 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 			deletable[ndeletable++] = offnum;
 	}
 
-	if (XLogStandbyInfoActive() && RelationNeedsWAL(rel))
+	if (XLogStandbyInfoActive() && needwal)
 		latestRemovedXid =
 			index_compute_xid_horizon_for_tuples(rel, heapRel, buffer,
 												 deletable, ndeletable);
 
 	if (ndeletable > 0)
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		PageIndexMultiDelete(page, deletable, ndeletable);
@@ -1681,7 +1696,7 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 		MarkBufferDirty(buffer);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gist/gistvacuum.c b/src/backend/access/gist/gistvacuum.c
index a9c616c7724..bbb3ebb19ad 100644
--- a/src/backend/access/gist/gistvacuum.c
+++ b/src/backend/access/gist/gistvacuum.c
@@ -17,6 +17,7 @@
 #include "access/genam.h"
 #include "access/gist_private.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "commands/vacuum.h"
 #include "lib/integerset.h"
 #include "miscadmin.h"
@@ -260,6 +261,7 @@ gistvacuumpage(GistVacState *vstate, BlockNumber blkno, BlockNumber orig_blkno)
 	Buffer		buffer;
 	Page		page;
 	BlockNumber recurse_to;
+	bool		needwal = RelationNeedsWAL(rel);
 
 restart:
 	recurse_to = InvalidBlockNumber;
@@ -341,6 +343,9 @@ restart:
 		 */
 		if (ntodelete > 0)
 		{
+			if (needwal)
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 
 			MarkBufferDirty(buffer);
@@ -348,7 +353,7 @@ restart:
 			PageIndexMultiDelete(page, todelete, ntodelete);
 			GistMarkTuplesDeleted(page);
 
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 			{
 				XLogRecPtr	recptr;
 
@@ -580,6 +585,7 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	IndexTuple	idxtuple;
 	XLogRecPtr	recptr;
 	FullTransactionId txid;
+	bool		needwal = RelationNeedsWAL(info->index);
 
 	/*
 	 * Check that the leaf is still empty and deletable.
@@ -634,6 +640,9 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	 */
 	txid = ReadNextFullTransactionId();
 
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* mark the page as deleted */
@@ -645,7 +654,7 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	MarkBufferDirty(parentBuffer);
 	PageIndexTupleDelete(parentPage, downlink);
 
-	if (RelationNeedsWAL(info->index))
+	if (needwal)
 		recptr = gistXLogPageDelete(leafBuffer, txid, parentBuffer, downlink);
 	else
 		recptr = gistGetFakeLSN(info->index);
diff --git a/src/backend/access/hash/hash.c b/src/backend/access/hash/hash.c
index 3ec6d528e77..b7707f90aa1 100644
--- a/src/backend/access/hash/hash.c
+++ b/src/backend/access/hash/hash.c
@@ -22,6 +22,7 @@
 #include "access/hash_xlog.h"
 #include "access/relscan.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "catalog/index.h"
 #include "commands/progress.h"
 #include "commands/vacuum.h"
@@ -466,6 +467,7 @@ hashbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	Buffer		metabuf = InvalidBuffer;
 	HashMetaPage metap;
 	HashMetaPage cachedmetap;
+	bool		needwal;
 
 	tuples_removed = 0;
 	num_index_tuples = 0;
@@ -572,6 +574,10 @@ loop_top:
 		goto loop_top;
 	}
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Okay, we're really done.  Update tuple count in metapage. */
 	START_CRIT_SECTION();
 
@@ -602,7 +608,7 @@ loop_top:
 	MarkBufferDirty(metabuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_hash_update_meta_page xlrec;
 		XLogRecPtr	recptr;
@@ -689,6 +695,7 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 	Buffer		buf;
 	Bucket		new_bucket PG_USED_FOR_ASSERTS_ONLY = InvalidBucket;
 	bool		bucket_dirty = false;
+	bool		needwal = RelationNeedsWAL(rel);
 
 	blkno = bucket_blkno;
 	buf = bucket_buf;
@@ -787,6 +794,9 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		 */
 		if (ndeletable > 0)
 		{
+			if (needwal)
+				CheckWALPermitted();
+
 			/* No ereport(ERROR) until changes are logged */
 			START_CRIT_SECTION();
 
@@ -808,7 +818,7 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 			MarkBufferDirty(buf);
 
 			/* XLOG stuff */
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 			{
 				xl_hash_delete xlrec;
 				XLogRecPtr	recptr;
@@ -882,6 +892,9 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		page = BufferGetPage(bucket_buf);
 		bucket_opaque = (HashPageOpaque) PageGetSpecialPointer(page);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -889,7 +902,7 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		MarkBufferDirty(bucket_buf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c
index 2ebe671967b..2eab69efa91 100644
--- a/src/backend/access/hash/hashinsert.c
+++ b/src/backend/access/hash/hashinsert.c
@@ -17,6 +17,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "storage/buf_internals.h"
 #include "storage/lwlock.h"
@@ -193,6 +194,8 @@ restart_insert:
 	 */
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	AssertWALPermittedHaveXID();
+
 	/* Do the update.  No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -360,6 +363,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 	if (ndeletable > 0)
 	{
 		TransactionId latestRemovedXid;
+		bool		needwal = RelationNeedsWAL(rel);
 
 		latestRemovedXid =
 			index_compute_xid_horizon_for_tuples(rel, hrel, buf,
@@ -370,6 +374,9 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 		 */
 		LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -393,7 +400,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 		MarkBufferDirty(metabuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			xl_hash_vacuum_one_page xlrec;
 			XLogRecPtr	recptr;
diff --git a/src/backend/access/hash/hashovfl.c b/src/backend/access/hash/hashovfl.c
index 00f0a940116..e7c5dd3e3ce 100644
--- a/src/backend/access/hash/hashovfl.c
+++ b/src/backend/access/hash/hashovfl.c
@@ -19,6 +19,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "utils/rel.h"
 
@@ -312,6 +313,8 @@ _hash_addovflpage(Relation rel, Buffer metabuf, Buffer buf, bool retain_pin)
 
 found:
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Do the update.  No ereport(ERROR) until changes are logged. We want to
 	 * log the changes for bitmap page and overflow page together to avoid
@@ -510,6 +513,7 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	Buffer		prevbuf = InvalidBuffer;
 	Buffer		nextbuf = InvalidBuffer;
 	bool		update_metap = false;
+	bool		needwal;
 
 	/* Get information from the doomed page */
 	_hash_checkpage(rel, ovflbuf, LH_OVERFLOW_PAGE);
@@ -573,10 +577,15 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	/* Get write-lock on metapage to update firstfree */
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	needwal = RelationNeedsWAL(rel);
+
 	/* This operation needs to log multiple tuples, prepare WAL for that */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 		XLogEnsureRecordSpace(HASH_XLOG_FREE_OVFL_BUFS, 4 + nitups);
 
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -641,7 +650,7 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	}
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_hash_squeeze_page xlrec;
 		XLogRecPtr	recptr;
@@ -922,14 +931,19 @@ readpage:
 
 				if (nitups > 0)
 				{
+					bool		needwal = RelationNeedsWAL(rel);
+
 					Assert(nitups == ndeletable);
 
 					/*
 					 * This operation needs to log multiple tuples, prepare
 					 * WAL for that.
 					 */
-					if (RelationNeedsWAL(rel))
+					if (needwal)
+					{
+						CheckWALPermitted();
 						XLogEnsureRecordSpace(0, 3 + nitups);
+					}
 
 					START_CRIT_SECTION();
 
@@ -947,7 +961,7 @@ readpage:
 					MarkBufferDirty(rbuf);
 
 					/* XLOG stuff */
-					if (RelationNeedsWAL(rel))
+					if (needwal)
 					{
 						XLogRecPtr	recptr;
 						xl_hash_move_page_contents xlrec;
diff --git a/src/backend/access/hash/hashpage.c b/src/backend/access/hash/hashpage.c
index a664ecf494a..55a867dd375 100644
--- a/src/backend/access/hash/hashpage.c
+++ b/src/backend/access/hash/hashpage.c
@@ -30,6 +30,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "port/pg_bitutils.h"
 #include "storage/lmgr.h"
@@ -816,6 +817,8 @@ restart_expand:
 		goto fail;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Since we are scribbling on the pages in the shared buffers, establish a
 	 * critical section.  Any failure in this next code leaves us with a big
@@ -1172,6 +1175,8 @@ _hash_splitbucket(Relation rel,
 
 				if (PageGetFreeSpaceForMultipleTuples(npage, nitups + 1) < (all_tups_size + itemsz))
 				{
+					AssertWALPermittedHaveXID();
+
 					/*
 					 * Change the shared buffer state in critical section,
 					 * otherwise any error could make it unrecoverable.
@@ -1223,6 +1228,8 @@ _hash_splitbucket(Relation rel,
 		/* Exit loop if no more overflow pages in old bucket */
 		if (!BlockNumberIsValid(oblkno))
 		{
+			AssertWALPermittedHaveXID();
+
 			/*
 			 * Change the shared buffer state in critical section, otherwise
 			 * any error could make it unrecoverable.
@@ -1269,6 +1276,8 @@ _hash_splitbucket(Relation rel,
 	npage = BufferGetPage(bucket_nbuf);
 	nopaque = (HashPageOpaque) PageGetSpecialPointer(npage);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	oopaque->hasho_flag &= ~LH_BUCKET_BEING_SPLIT;
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 8df2716de46..cccd0ca891d 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -47,6 +47,7 @@
 #include "access/transam.h"
 #include "access/valid.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1881,6 +1882,8 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
 	 */
 	CheckForSerializableConflictIn(relation, NULL, InvalidBlockNumber);
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -2154,6 +2157,8 @@ heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples,
 										   &vmbuffer, NULL);
 		page = BufferGetPage(buffer);
 
+		AssertWALPermittedHaveXID();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -2672,6 +2677,8 @@ l1:
 							  xid, LockTupleExclusive, true,
 							  &new_xmax, &new_infomask, &new_infomask2);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -3424,6 +3431,8 @@ l2:
 
 		Assert(HEAP_XMAX_IS_LOCKED_ONLY(infomask_lock_old_tuple));
 
+		AssertWALPermittedHaveXID();
+
 		START_CRIT_SECTION();
 
 		/* Clear obsolete visibility flags ... */
@@ -3597,6 +3606,8 @@ l2:
 										   bms_overlap(modified_attrs, id_attrs),
 										   &old_key_copied);
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -4530,6 +4541,8 @@ failed:
 							  GetCurrentTransactionId(), mode, false,
 							  &xid, &new_infomask, &new_infomask2);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -5321,6 +5334,8 @@ l4:
 								VISIBILITYMAP_ALL_FROZEN))
 			cleared_all_frozen = true;
 
+		AssertWALPermittedHaveXID();
+
 		START_CRIT_SECTION();
 
 		/* ... and set them */
@@ -5479,6 +5494,8 @@ heap_finish_speculative(Relation relation, ItemPointer tid)
 	StaticAssertStmt(MaxOffsetNumber < SpecTokenOffsetNumber,
 					 "invalid speculative token constant");
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -5587,6 +5604,8 @@ heap_abort_speculative(Relation relation, ItemPointer tid)
 		elog(ERROR, "attempted to kill a non-speculative tuple");
 	Assert(!HeapTupleHeaderIsHeapOnly(tp.t_data));
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * No need to check for serializable conflicts here.  There is never a
 	 * need for a combocid, either.  No need to extract replica identity, or
@@ -5703,6 +5722,7 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	HeapTupleHeader htup;
 	uint32		oldlen;
 	uint32		newlen;
+	bool		needwal;
 
 	/*
 	 * For now, parallel operations are required to be strictly read-only.
@@ -5733,6 +5753,10 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	if (oldlen != newlen || htup->t_hoff != tuple->t_data->t_hoff)
 		elog(ERROR, "wrong tuple length");
 
+	needwal = RelationNeedsWAL(relation);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -5743,7 +5767,7 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	MarkBufferDirty(buffer);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(relation))
+	if (needwal)
 	{
 		xl_heap_inplace xlrec;
 		XLogRecPtr	recptr;
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index 256df4de105..0899078a1e2 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -18,6 +18,7 @@
 #include "access/heapam_xlog.h"
 #include "access/htup_details.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "catalog/catalog.h"
 #include "miscadmin.h"
@@ -77,11 +78,11 @@ heap_page_prune_opt(Relation relation, Buffer buffer)
 	TransactionId OldestXmin;
 
 	/*
-	 * We can't write WAL in recovery mode, so there's no point trying to
+	 * We can't write WAL during read-only mode, so there's no point trying to
 	 * clean the page. The primary will likely issue a cleaning WAL record soon
 	 * anyway, so this is no particular loss.
 	 */
-	if (RecoveryInProgress())
+	if (!XLogInsertAllowed())
 		return;
 
 	/*
@@ -185,6 +186,7 @@ heap_page_prune(Relation relation, Buffer buffer, TransactionId OldestXmin,
 	OffsetNumber offnum,
 				maxoff;
 	PruneState	prstate;
+	bool		needwal;
 
 	/*
 	 * Our strategy is to scan the page and make lists of items to change,
@@ -225,6 +227,10 @@ heap_page_prune(Relation relation, Buffer buffer, TransactionId OldestXmin,
 									 &prstate);
 	}
 
+	needwal = RelationNeedsWAL(relation);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Any error while applying the changes is critical */
 	START_CRIT_SECTION();
 
@@ -258,7 +264,7 @@ heap_page_prune(Relation relation, Buffer buffer, TransactionId OldestXmin,
 		/*
 		 * Emit a WAL XLOG_HEAP2_CLEAN record showing what we did
 		 */
-		if (RelationNeedsWAL(relation))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 1bbc4598f75..89ab313c46b 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -59,6 +59,7 @@
 #include "access/parallel.h"
 #include "access/transam.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "catalog/storage.h"
@@ -764,6 +765,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 	BlockNumber nblocks,
 				blkno;
 	HeapTupleData tuple;
+	bool		needwal = RelationNeedsWAL(onerel);
 	TransactionId relfrozenxid = onerel->rd_rel->relfrozenxid;
 	TransactionId relminmxid = onerel->rd_rel->relminmxid;
 	BlockNumber empty_pages,
@@ -1203,6 +1205,9 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 			 */
 			if (!PageIsAllVisible(page))
 			{
+				if (needwal)
+					CheckWALPermitted();
+
 				START_CRIT_SECTION();
 
 				/* mark buffer dirty before writing a WAL record */
@@ -1218,7 +1223,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 				 * page has been previously WAL-logged, and if not, do that
 				 * now.
 				 */
-				if (RelationNeedsWAL(onerel) &&
+				if (needwal &&
 					PageGetLSN(page) == InvalidXLogRecPtr)
 					log_newpage_buffer(buf, true);
 
@@ -1471,6 +1476,9 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 		 */
 		if (nfrozen > 0)
 		{
+			if (needwal)
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 
 			MarkBufferDirty(buf);
@@ -1488,7 +1496,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 			}
 
 			/* Now WAL-log freezing if necessary */
-			if (RelationNeedsWAL(onerel))
+			if (needwal)
 			{
 				XLogRecPtr	recptr;
 
@@ -1910,6 +1918,7 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	TransactionId visibility_cutoff_xid;
 	bool		all_frozen;
 	LVSavedErrInfo saved_err_info;
+	bool		needwal = RelationNeedsWAL(onerel);
 
 	pgstat_progress_update_param(PROGRESS_VACUUM_HEAP_BLKS_VACUUMED, blkno);
 
@@ -1917,6 +1926,9 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	update_vacuum_error_info(vacrelstats, &saved_err_info, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
 							 blkno);
 
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	for (; tupindex < dead_tuples->num_tuples; tupindex++)
@@ -1942,7 +1954,7 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	MarkBufferDirty(buffer);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(onerel))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 0a51678c40d..9606e1752d9 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -88,6 +88,7 @@
 
 #include "access/heapam_xlog.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "miscadmin.h"
 #include "port/pg_bitutils.h"
@@ -249,6 +250,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 	uint8		mapOffset = HEAPBLK_TO_OFFSET(heapBlk);
 	Page		page;
 	uint8	   *map;
+	bool		needwal = RelationNeedsWAL(rel);
 
 #ifdef TRACE_VISIBILITYMAP
 	elog(DEBUG1, "vm_set %s %d", RelationGetRelationName(rel), heapBlk);
@@ -270,6 +272,16 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 	map = (uint8 *) PageGetContents(page);
 	LockBuffer(vmBuf, BUFFER_LOCK_EXCLUSIVE);
 
+	/*
+	 * Can reach here from VACUUM or from startup process, so need not have an
+	 * XID.
+	 *
+	 * Recovery in the startup process never have wal prohibit state, skip
+	 * permission check if reach here in the startup process.
+	 */
+	if (needwal)
+		InRecovery ? AssertWALPermitted() : CheckWALPermitted();
+
 	if (flags != (map[mapByte] >> mapOffset & VISIBILITYMAP_VALID_BITS))
 	{
 		START_CRIT_SECTION();
@@ -277,7 +289,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 		map[mapByte] |= (flags << mapOffset);
 		MarkBufferDirty(vmBuf);
 
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			if (XLogRecPtrIsInvalid(recptr))
 			{
@@ -476,6 +488,7 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		Buffer		mapBuffer;
 		Page		page;
 		char	   *map;
+		bool		needwal;
 
 		newnblocks = truncBlock + 1;
 
@@ -489,8 +502,13 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		page = BufferGetPage(mapBuffer);
 		map = PageGetContents(page);
 
+		needwal = (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded());
+
 		LockBuffer(mapBuffer, BUFFER_LOCK_EXCLUSIVE);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -518,7 +536,7 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		 * during recovery.
 		 */
 		MarkBufferDirty(mapBuffer);
-		if (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded())
+		if (needwal)
 			log_newpage_buffer(mapBuffer, false);
 
 		END_CRIT_SECTION();
diff --git a/src/backend/access/nbtree/nbtdedup.c b/src/backend/access/nbtree/nbtdedup.c
index f6be865b17e..b519a1268e8 100644
--- a/src/backend/access/nbtree/nbtdedup.c
+++ b/src/backend/access/nbtree/nbtdedup.c
@@ -16,6 +16,7 @@
 
 #include "access/nbtree.h"
 #include "access/nbtxlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "utils/rel.h"
 
@@ -271,6 +272,8 @@ _bt_dedup_one_page(Relation rel, Buffer buf, Relation heapRel,
 		nopaque->btpo_flags &= ~BTP_HAS_GARBAGE;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	PageRestoreTempPage(newpage, page);
diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c
index e3a44bc09e0..33cc89ac392 100644
--- a/src/backend/access/nbtree/nbtinsert.c
+++ b/src/backend/access/nbtree/nbtinsert.c
@@ -19,6 +19,7 @@
 #include "access/nbtxlog.h"
 #include "access/tableam.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/lmgr.h"
@@ -1246,6 +1247,8 @@ _bt_insertonpg(Relation rel,
 			}
 		}
 
+		AssertWALPermittedHaveXID();
+
 		/* Do the update.  No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -1900,13 +1903,16 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf,
 			ropaque->btpo_flags |= BTP_SPLIT_END;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Right sibling is locked, new siblings are prepared, but original page
 	 * is not updated yet.
 	 *
 	 * NO EREPORT(ERROR) till right sibling is updated.  We can get away with
 	 * not starting the critical section till here because we haven't been
-	 * scribbling on the original page yet; see comments above.
+	 * scribbling on the original page yet; see the comments above for grabbing
+	 * the right sibling.
 	 */
 	START_CRIT_SECTION();
 
@@ -2469,6 +2475,8 @@ _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf)
 	right_item = CopyIndexTuple(item);
 	BTreeTupleSetDownLink(right_item, rbkno);
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till newroot op is logged */
 	START_CRIT_SECTION();
 
diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c
index 70bac0052fc..295aa1577f1 100644
--- a/src/backend/access/nbtree/nbtpage.c
+++ b/src/backend/access/nbtree/nbtpage.c
@@ -26,6 +26,7 @@
 #include "access/nbtxlog.h"
 #include "access/tableam.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
@@ -179,6 +180,7 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	BTMetaPageData *metad;
 	bool		needsRewrite = false;
 	XLogRecPtr	recptr;
+	bool		needwal;
 
 	/* read the metapage and check if it needs rewrite */
 	metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ);
@@ -202,6 +204,10 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	_bt_unlockbuf(rel, metabuf);
 	_bt_lockbuf(rel, metabuf, BT_WRITE);
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* upgrade meta-page if needed */
@@ -214,7 +220,7 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	MarkBufferDirty(metabuf);
 
 	/* write wal record if needed */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_btree_metadata md;
 
@@ -332,6 +338,7 @@ _bt_getroot(Relation rel, int access)
 	if (metad->btm_root == P_NONE)
 	{
 		Page		metapg;
+		bool		needwal;
 
 		/* If access = BT_READ, caller doesn't want us to create root yet */
 		if (access == BT_READ)
@@ -377,6 +384,10 @@ _bt_getroot(Relation rel, int access)
 		/* Get raw page pointer for metapage */
 		metapg = BufferGetPage(metabuf);
 
+		needwal = RelationNeedsWAL(rel);
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO ELOG(ERROR) till meta is updated */
 		START_CRIT_SECTION();
 
@@ -395,7 +406,7 @@ _bt_getroot(Relation rel, int access)
 		MarkBufferDirty(metabuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			xl_btree_newroot xlrec;
 			XLogRecPtr	recptr;
@@ -1131,6 +1142,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 	char	   *updatedbuf = NULL;
 	Size		updatedbuflen = 0;
 	OffsetNumber updatedoffsets[MaxIndexTuplesPerPage];
+	bool		needwal = RelationNeedsWAL(rel);
 
 	/* Shouldn't be called unless there's something to do */
 	Assert(ndeletable > 0 || nupdatable > 0);
@@ -1145,7 +1157,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 	}
 
 	/* XLOG stuff -- allocate and fill buffer before critical section */
-	if (nupdatable > 0 && RelationNeedsWAL(rel))
+	if (nupdatable > 0 && needwal)
 	{
 		Size		offset = 0;
 
@@ -1175,6 +1187,9 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 		}
 	}
 
+	if (needwal)
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -1235,7 +1250,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 	MarkBufferDirty(buf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		xl_btree_vacuum xlrec_vacuum;
@@ -1302,6 +1317,8 @@ _bt_delitems_delete(Relation rel, Buffer buf,
 		latestRemovedXid =
 			_bt_xid_horizon(rel, heapRel, page, deletable, ndeletable);
 
+	AssertWALPermittedHaveXID();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -1831,6 +1848,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	OffsetNumber nextoffset;
 	IndexTuple	itup;
 	IndexTupleData trunctuple;
+	bool		needwal;
 
 	page = BufferGetPage(leafbuf);
 	opaque = (BTPageOpaque) PageGetSpecialPointer(page);
@@ -1919,6 +1937,10 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	 */
 	PredicateLockPageCombine(rel, leafblkno, leafrightsib);
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -1970,7 +1992,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	MarkBufferDirty(leafbuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_btree_mark_page_halfdead xlrec;
 		XLogRecPtr	recptr;
@@ -2062,6 +2084,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 	int			targetlevel;
 	IndexTuple	leafhikey;
 	BlockNumber nextchild;
+	bool		needwal;
 
 	page = BufferGetPage(leafbuf);
 	opaque = (BTPageOpaque) PageGetSpecialPointer(page);
@@ -2275,6 +2298,10 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 	 * Here we begin doing the deletion.
 	 */
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -2346,7 +2373,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 		MarkBufferDirty(leafbuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_btree_unlink_page xlrec;
 		xl_btree_metadata xlmeta;
diff --git a/src/backend/access/spgist/spgdoinsert.c b/src/backend/access/spgist/spgdoinsert.c
index 934d65b89f2..3c5a15c5d32 100644
--- a/src/backend/access/spgist/spgdoinsert.c
+++ b/src/backend/access/spgist/spgdoinsert.c
@@ -18,6 +18,7 @@
 #include "access/genam.h"
 #include "access/spgist_private.h"
 #include "access/spgxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -214,6 +215,8 @@ addLeafTuple(Relation index, SpGistState *state, SpGistLeafTuple leafTuple,
 	xlrec.offnumParent = InvalidOffsetNumber;
 	xlrec.nodeI = 0;
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	if (current->offnum == InvalidOffsetNumber ||
@@ -458,6 +461,8 @@ moveLeafs(Relation index, SpGistState *state,
 
 	leafdata = leafptr = palloc(size);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/* copy all the old tuples to new page, unless they're dead */
@@ -1109,6 +1114,8 @@ doPickSplit(Relation index, SpGistState *state,
 
 	leafdata = leafptr = (char *) palloc(totalLeafSizes);
 
+	AssertWALPermittedHaveXID();
+
 	/* Here we begin making the changes to the target pages */
 	START_CRIT_SECTION();
 
@@ -1517,6 +1524,8 @@ spgAddNodeAction(Relation index, SpGistState *state,
 	if (PageGetExactFreeSpace(current->page) >=
 		newInnerTuple->size - innerTuple->size)
 	{
+		AssertWALPermittedHaveXID();
+
 		/*
 		 * We can replace the inner tuple by new version in-place
 		 */
@@ -1603,6 +1612,8 @@ spgAddNodeAction(Relation index, SpGistState *state,
 		else
 			xlrec.parentBlk = 2;
 
+		AssertWALPermittedHaveXID();
+
 		START_CRIT_SECTION();
 
 		/* insert new ... */
@@ -1788,6 +1799,8 @@ spgSplitNodeAction(Relation index, SpGistState *state,
 									&xlrec.newPage);
 	}
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c
index bd98707f3c0..5ed12301763 100644
--- a/src/backend/access/spgist/spgvacuum.c
+++ b/src/backend/access/spgist/spgvacuum.c
@@ -19,6 +19,7 @@
 #include "access/spgist_private.h"
 #include "access/spgxlog.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/storage_xlog.h"
 #include "commands/vacuum.h"
@@ -139,6 +140,7 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 	int			nDeletable;
 	OffsetNumber i,
 				max = PageGetMaxOffsetNumber(page);
+	bool		needwal;
 
 	memset(predecessor, 0, sizeof(predecessor));
 	memset(deletable, 0, sizeof(deletable));
@@ -323,6 +325,10 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 	if (nDeletable != xlrec.nDead + xlrec.nPlaceholder + xlrec.nMove)
 		elog(ERROR, "inconsistent counts of deletable tuples");
 
+	needwal = RelationNeedsWAL(index);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Do the updates */
 	START_CRIT_SECTION();
 
@@ -371,7 +377,7 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 
@@ -411,6 +417,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 	OffsetNumber toDelete[MaxIndexTuplesPerPage];
 	OffsetNumber i,
 				max = PageGetMaxOffsetNumber(page);
+	bool		needwal;
 
 	xlrec.nDelete = 0;
 
@@ -447,6 +454,10 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 	if (xlrec.nDelete == 0)
 		return;					/* nothing more to do */
 
+	needwal = RelationNeedsWAL(index);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Do the update */
 	START_CRIT_SECTION();
 
@@ -455,7 +466,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 
@@ -501,10 +512,14 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer)
 	OffsetNumber itemToPlaceholder[MaxIndexTuplesPerPage];
 	OffsetNumber itemnos[MaxIndexTuplesPerPage];
 	spgxlogVacuumRedirect xlrec;
+	bool		needwal = RelationNeedsWAL(index);
 
 	xlrec.nToPlaceholder = 0;
 	xlrec.newestRedirectXid = InvalidTransactionId;
 
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -580,7 +595,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer)
 	if (hasUpdate)
 		MarkBufferDirty(buffer);
 
-	if (hasUpdate && RelationNeedsWAL(index))
+	if (hasUpdate && needwal)
 	{
 		XLogRecPtr	recptr;
 
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index ce84dac0c40..20005798e9a 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -73,6 +73,7 @@
 #include "access/transam.h"
 #include "access/twophase.h"
 #include "access/twophase_rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1143,6 +1144,8 @@ GetNewMultiXactId(int nmembers, MultiXactOffset *offset)
 
 	ExtendMultiXactMember(nextOffset, nmembers);
 
+	CheckWALPermitted();
+
 	/*
 	 * Critical section from here until caller has written the data into the
 	 * just-reserved SLRU space; we don't want to error out with a partly
@@ -2942,7 +2945,7 @@ TruncateMultiXact(MultiXactId newOldestMulti, Oid newOldestMultiDB)
 	mxtruncinfo trunc;
 	MultiXactId earliest;
 
-	Assert(!RecoveryInProgress());
+	Assert(XLogInsertAllowed());
 	Assert(MultiXactState->finishedStartup);
 
 	/*
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 9b2e59bf0ec..141089d977a 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -82,6 +82,7 @@
 #include "access/transam.h"
 #include "access/twophase.h"
 #include "access/twophase_rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1112,6 +1113,8 @@ EndPrepare(GlobalTransaction gxact)
 	 */
 	XLogEnsureRecordSpace(0, records.num_chunks);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	MyProc->delayChkpt = true;
@@ -2204,6 +2207,9 @@ RecordTransactionCommitPrepared(TransactionId xid,
 	replorigin = (replorigin_session_origin != InvalidRepOriginId &&
 				  replorigin_session_origin != DoNotReplicateId);
 
+	/* COMMIT PREPARED need not have an XID */
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* See notes in RecordTransactionCommit */
@@ -2294,6 +2300,9 @@ RecordTransactionAbortPrepared(TransactionId xid,
 		elog(PANIC, "cannot abort transaction %u, it was already committed",
 			 xid);
 
+	/* ROLLBACK PREPARED need not have an XID */
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
diff --git a/src/backend/access/transam/varsup.c b/src/backend/access/transam/varsup.c
index e14b53bf9e3..365de44321d 100644
--- a/src/backend/access/transam/varsup.c
+++ b/src/backend/access/transam/varsup.c
@@ -17,6 +17,7 @@
 #include "access/commit_ts.h"
 #include "access/subtrans.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "commands/dbcommands.h"
@@ -73,6 +74,9 @@ GetNewTransactionId(bool isSubXact)
 	if (RecoveryInProgress())
 		elog(ERROR, "cannot assign TransactionIds during recovery");
 
+	/* Cannot assign transaction id in read-only mode */
+	CheckWALPermitted();
+
 	LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
 
 	full_xid = ShmemVariableCache->nextFullXid;
diff --git a/src/backend/access/transam/walprohibit.c b/src/backend/access/transam/walprohibit.c
index a3f1a750744..6f66804735f 100644
--- a/src/backend/access/transam/walprohibit.c
+++ b/src/backend/access/transam/walprohibit.c
@@ -16,6 +16,16 @@
 #include "postmaster/bgwriter.h"
 #include "storage/procsignal.h"
 
+/*
+ * Assert flag to enforce WAL insert permission check rule before starting a
+ * critical section for the WAL writes.  For this, either of CheckWALPermitted,
+ * AssertWALPermittedHaveXID, or AssertWALPermitted must be called before
+ * starting the critical section.
+ */
+#ifdef USE_ASSERT_CHECKING
+WALPermitCheckState walpermit_checked_state = WALPERMIT_UNCHECKED;
+#endif
+
 /*
  * ProcessBarrierWALProhibit()
  *
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 8ebc4242c25..8f12e9f672b 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -26,6 +26,7 @@
 #include "access/subtrans.h"
 #include "access/transam.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1304,6 +1305,8 @@ RecordTransactionCommit(void)
 		/* Tell bufmgr and smgr to prepare for commit */
 		BufmgrCommit();
 
+		AssertWALPermittedHaveXID();
+
 		/*
 		 * Mark ourselves as within our "commit critical section".  This
 		 * forces any concurrent checkpoint to wait until we've updated
@@ -1664,6 +1667,9 @@ RecordTransactionAbort(bool isSubXact)
 		elog(PANIC, "cannot abort transaction %u, it was already committed",
 			 xid);
 
+	/* We'll be reaching here with valid XID only. */
+	AssertWALPermittedHaveXID();
+
 	/* Fetch the data we need for the abort record */
 	nrels = smgrGetPendingDeletes(false, &rels);
 	nchildren = xactGetCommittedChildren(&children);
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 6d33ea226b1..e3923ea1fe9 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -1028,7 +1028,7 @@ XLogInsertRecord(XLogRecData *rdata,
 
 	/* cross-check on whether we should be here or not */
 	if (!XLogInsertAllowed())
-		elog(ERROR, "cannot make new WAL entries during recovery");
+		elog(ERROR, "cannot make new WAL entries now");
 
 	/*----------
 	 *
@@ -2863,9 +2863,11 @@ XLogFlush(XLogRecPtr record)
 	 * trying to flush the WAL, we should update minRecoveryPoint instead. We
 	 * test XLogInsertAllowed(), not InRecovery, because we need checkpointer
 	 * to act this way too, and because when it tries to write the
-	 * end-of-recovery checkpoint, it should indeed flush.
+	 * end-of-recovery checkpoint, it should indeed flush. Also, WAL prohibit
+	 * state should not restrict WAL flushing. Otherwise, the dirty buffer
+	 * cannot be evicted until WAL has been flushed up to the buffer's LSN.
 	 */
-	if (!XLogInsertAllowed())
+	if (!XLogInsertAllowed() && !IsWALProhibited())
 	{
 		UpdateMinRecoveryPoint(record, false);
 		return;
@@ -8855,6 +8857,8 @@ CreateCheckPoint(int flags)
 	/* sanity check */
 	if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
 		elog(ERROR, "can't create a checkpoint during recovery");
+	if (!XLogInsertAllowed() && !RecoveryInProgress())
+		elog(ERROR, "can't create a checkpoint while system is read only ");
 
 	/*
 	 * Initialize InitXLogInsert working areas before entering the critical
@@ -8884,6 +8888,8 @@ CreateCheckPoint(int flags)
 	MemSet(&CheckpointStats, 0, sizeof(CheckpointStats));
 	CheckpointStats.ckpt_start_t = GetCurrentTimestamp();
 
+	AssertWALPermitted();
+
 	/*
 	 * Use a critical section to force system panic if we have trouble.
 	 */
@@ -9112,6 +9118,8 @@ CreateCheckPoint(int flags)
 	if (!shutdown && XLogStandbyInfoActive())
 		LogStandbySnapshot();
 
+	AssertWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -9269,6 +9277,8 @@ CreateEndOfRecoveryRecord(void)
 
 	LocalSetXLogInsertAllowed();
 
+	AssertWALPermitted();
+
 	START_CRIT_SECTION();
 
 	XLogBeginInsert();
@@ -9915,7 +9925,7 @@ void
 UpdateFullPageWrites(void)
 {
 	XLogCtlInsert *Insert = &XLogCtl->Insert;
-	bool		recoveryInProgress;
+	bool		WALInsertAllowed;
 
 	/*
 	 * Do nothing if full_page_writes has not been changed.
@@ -9929,10 +9939,10 @@ UpdateFullPageWrites(void)
 
 	/*
 	 * Perform this outside critical section so that the WAL insert
-	 * initialization done by RecoveryInProgress() doesn't trigger an
+	 * initialization done by XLogInsertAllowed() doesn't trigger an
 	 * assertion failure.
 	 */
-	recoveryInProgress = RecoveryInProgress();
+	WALInsertAllowed = XLogInsertAllowed();
 
 	START_CRIT_SECTION();
 
@@ -9954,8 +9964,11 @@ UpdateFullPageWrites(void)
 	 * Write an XLOG_FPW_CHANGE record. This allows us to keep track of
 	 * full_page_writes during archive recovery, if required.
 	 */
-	if (XLogStandbyInfoActive() && !recoveryInProgress)
+	if (XLogStandbyInfoActive() && WALInsertAllowed)
 	{
+		/* Assured that WAL permission has been checked */
+		AssertWALPermitted();
+
 		XLogBeginInsert();
 		XLogRegisterData((char *) (&fullPageWrites), sizeof(bool));
 
diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c
index c526bb19281..506d7e97f38 100644
--- a/src/backend/access/transam/xloginsert.c
+++ b/src/backend/access/transam/xloginsert.c
@@ -19,6 +19,7 @@
 
 #include "postgres.h"
 
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xlog_internal.h"
@@ -126,9 +127,14 @@ XLogBeginInsert(void)
 	Assert(mainrdata_last == (XLogRecData *) &mainrdata_head);
 	Assert(mainrdata_len == 0);
 
+	/*
+	 * WAL permission must have checked before entering the critical section.
+	 * Otherwise, WAL prohibited error will force system panic.
+	 */
+	Assert(walpermit_checked_state != WALPERMIT_UNCHECKED || !CritSectionCount);
+
 	/* cross-check on whether we should be here or not */
-	if (!XLogInsertAllowed())
-		elog(ERROR, "cannot make new WAL entries during recovery");
+	CheckWALPermitted();
 
 	if (begininsert_called)
 		elog(ERROR, "XLogBeginInsert was already called");
@@ -210,6 +216,9 @@ XLogResetInsertion(void)
 	mainrdata_last = (XLogRecData *) &mainrdata_head;
 	curinsert_flags = 0;
 	begininsert_called = false;
+
+	/* Reset walpermit_checked flag */
+	RESET_WALPERMIT_CHECKED_STATE();
 }
 
 /*
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 6aab73bfd44..8dacf48db24 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -20,6 +20,7 @@
 #include "access/relation.h"
 #include "access/table.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -378,8 +379,13 @@ fill_seq_with_data(Relation rel, HeapTuple tuple)
 
 	/* check the comment above nextval_internal()'s equivalent call. */
 	if (RelationNeedsWAL(rel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermittedHaveXID();
+	}
+
 	START_CRIT_SECTION();
 
 	MarkBufferDirty(buf);
@@ -766,8 +772,13 @@ nextval_internal(Oid relid, bool check_permissions)
 	 * (Have to do that here, so we're outside the critical section)
 	 */
 	if (logit && RelationNeedsWAL(seqrel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermittedHaveXID();
+	}
+
 	/* ready to change the on-disk (or really, in-buffer) tuple */
 	START_CRIT_SECTION();
 
@@ -977,8 +988,13 @@ do_setval(Oid relid, int64 next, bool iscalled)
 
 	/* check the comment above nextval_internal()'s equivalent call. */
 	if (RelationNeedsWAL(seqrel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermittedHaveXID();
+	}
+
 	/* ready to change the on-disk (or really, in-buffer) tuple */
 	START_CRIT_SECTION();
 
diff --git a/src/backend/commands/variable.c b/src/backend/commands/variable.c
index 484f7ea2c0e..acc34d2ad7c 100644
--- a/src/backend/commands/variable.c
+++ b/src/backend/commands/variable.c
@@ -501,11 +501,14 @@ check_transaction_read_only(bool *newval, void **extra, GucSource source)
 			GUC_check_errmsg("transaction read-write mode must be set before any query");
 			return false;
 		}
-		/* Can't go to r/w mode while recovery is still active */
-		if (RecoveryInProgress())
+		/*
+		 * Can't go to r/w mode while recovery is still active or while in WAL
+		 * prohibit state
+		 */
+		if (!XLogInsertAllowed())
 		{
 			GUC_check_errcode(ERRCODE_FEATURE_NOT_SUPPORTED);
-			GUC_check_errmsg("cannot set transaction read-write mode during recovery");
+			GUC_check_errmsg("cannot set transaction read-write mode while system is read only");
 			return false;
 		}
 	}
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 3e8aa9a0ec3..5404b7bc126 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -935,6 +935,10 @@ RequestCheckpoint(int flags)
 	int			old_failed,
 				old_started;
 
+	/* The checkpoint is allowed in recovery but not in WAL prohibit state */
+	if (!RecoveryInProgress())
+		CheckWALPermitted();
+
 	/*
 	 * If in a standalone backend, just do it ourselves.
 	 */
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index f1ae6f9f844..322b7a385cf 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -3638,13 +3638,15 @@ MarkBufferDirtyHint(Buffer buffer, bool buffer_std)
 		{
 			/*
 			 * If we must not write WAL, due to a relfilenode-specific
-			 * condition or being in recovery, don't dirty the page.  We can
-			 * set the hint, just not dirty the page as a result so the hint
-			 * is lost when we evict the page or shutdown.
+			 * condition or in general, don't dirty the page.  We can
+			 * set the hint, but must not dirty the page as a result, lest
+			 * we trigger WAL generation. Unless the page is dirtied again
+			 * later, the hint will be lost when the page is evicted, or at
+			 * shutdown.
 			 *
 			 * See src/backend/storage/page/README for longer discussion.
 			 */
-			if (RecoveryInProgress() ||
+			if (!XLogInsertAllowed() ||
 				RelFileNodeSkippingWAL(bufHdr->tag.rnode))
 				return;
 
diff --git a/src/backend/storage/freespace/freespace.c b/src/backend/storage/freespace/freespace.c
index 95a21f6cc38..45d7fdf9485 100644
--- a/src/backend/storage/freespace/freespace.c
+++ b/src/backend/storage/freespace/freespace.c
@@ -24,6 +24,7 @@
 #include "postgres.h"
 
 #include "access/htup_details.h"
+#include "access/walprohibit.h"
 #include "access/xlogutils.h"
 #include "miscadmin.h"
 #include "storage/freespace.h"
@@ -285,12 +286,19 @@ FreeSpaceMapPrepareTruncateRel(Relation rel, BlockNumber nblocks)
 	 */
 	if (first_removed_slot > 0)
 	{
+		bool needwal;
+
 		buf = fsm_readbuf(rel, first_removed_address, false);
 		if (!BufferIsValid(buf))
 			return InvalidBlockNumber;	/* nothing to do; the FSM was already
 										 * smaller */
 		LockBuffer(buf, BUFFER_LOCK_EXCLUSIVE);
 
+		needwal = (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded());
+
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -305,7 +313,7 @@ FreeSpaceMapPrepareTruncateRel(Relation rel, BlockNumber nblocks)
 		 * during recovery.
 		 */
 		MarkBufferDirty(buf);
-		if (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded())
+		if (needwal)
 			log_newpage_buffer(buf, false);
 
 		END_CRIT_SECTION();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 95989ce79bd..212312d5ae5 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -794,15 +794,15 @@ LockAcquireExtended(const LOCKTAG *locktag,
 	if (lockmode <= 0 || lockmode > lockMethodTable->numLockModes)
 		elog(ERROR, "unrecognized lock mode: %d", lockmode);
 
-	if (RecoveryInProgress() && !InRecovery &&
+	if (!XLogInsertAllowed() && !InRecovery &&
 		(locktag->locktag_type == LOCKTAG_OBJECT ||
 		 locktag->locktag_type == LOCKTAG_RELATION) &&
 		lockmode > RowExclusiveLock)
 		ereport(ERROR,
 				(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
-				 errmsg("cannot acquire lock mode %s on database objects while recovery is in progress",
+				 errmsg("cannot acquire lock mode %s on database objects while system is read only",
 						lockMethodTable->lockModeNames[lockmode]),
-				 errhint("Only RowExclusiveLock or less can be acquired on database objects during recovery.")));
+				 errhint("Only RowExclusiveLock or less can be acquired on database objects while system is read only")));
 
 #ifdef LOCK_DEBUG
 	if (LOCK_DEBUG_ENABLED(locktag))
diff --git a/src/backend/utils/cache/relmapper.c b/src/backend/utils/cache/relmapper.c
index 671fbb0ed5c..90d7599a57c 100644
--- a/src/backend/utils/cache/relmapper.c
+++ b/src/backend/utils/cache/relmapper.c
@@ -43,6 +43,7 @@
 #include <sys/stat.h>
 #include <unistd.h>
 
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -841,6 +842,8 @@ write_relmap_file(bool shared, RelMapFile *newmap,
 		xl_relmap_update xlrec;
 		XLogRecPtr	lsn;
 
+		AssertWALPermittedHaveXID();
+
 		/* now errors are fatal ... */
 		START_CRIT_SECTION();
 
diff --git a/src/include/access/walprohibit.h b/src/include/access/walprohibit.h
index 163fe0d2fce..3442df5be2f 100644
--- a/src/include/access/walprohibit.h
+++ b/src/include/access/walprohibit.h
@@ -19,8 +19,8 @@ extern bool ProcessBarrierWALProhibit(void);
 extern void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
 
 /* WAL Prohibit States */
-#define	WALPROHIBIT_STATE_READ_WRITE		0x0000
-#define	WALPROHIBIT_STATE_READ_ONLY			0x0001
+#define	WALPROHIBIT_STATE_READ_WRITE		0x0000	/* WAL permitted */
+#define	WALPROHIBIT_STATE_READ_ONLY			0x0001	/* WAL prohibited */
 
 /*
  * The bit is used in state transition from one state to another.  When this
@@ -29,4 +29,50 @@ extern void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
  */
 #define WALPROHIBIT_TRANSITION_IN_PROGRESS	0x0002
 
+/* Never reaches when WAL is prohibited. */
+static inline void
+AssertWALPermitted(void)
+{
+	/*
+	 * Recovery in the startup process never is in wal prohibited state.
+	 */
+	Assert(InRecovery || XLogInsertAllowed());
+
+#ifdef USE_ASSERT_CHECKING
+	walpermit_checked_state = WALPERMIT_CHECKED;
+#endif
+}
+
+/*
+ * XID-bearing transactions are killed off by "ALTER SYSTEM READ ONLY", so any
+ * part of the code that can only be reached with an XID assigned is never
+ * reached when WAL is prohibited.
+ */
+static inline void
+AssertWALPermittedHaveXID(void)
+{
+	/* Must be performing an INSERT, UPDATE or DELETE, so we'll have an XID */
+	Assert(FullTransactionIdIsValid(GetTopFullTransactionIdIfAny()));
+	AssertWALPermitted();
+}
+
+/*
+ * In opposite to the above assertion if a transaction doesn't have valid XID
+ * (e.g. VACUUM) then it won't be killed while changing the system state to WAL
+ * prohibited.  Therefore, we need to explicitly error out before entering into
+ * the critical section.
+ */
+static inline void
+CheckWALPermitted(void)
+{
+	if (!XLogInsertAllowed())
+		ereport(ERROR,
+				(errcode(ERRCODE_READ_ONLY_SQL_TRANSACTION),
+				 errmsg("system is now read only")));
+
+#ifdef USE_ASSERT_CHECKING
+	walpermit_checked_state = WALPERMIT_CHECKED;
+#endif
+}
+
 #endif		/* WALPROHIBIT_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 18bc8a7b904..63459305383 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -94,12 +94,37 @@ extern PGDLLIMPORT volatile uint32 CritSectionCount;
 /* in tcop/postgres.c */
 extern void ProcessInterrupts(void);
 
+#ifdef USE_ASSERT_CHECKING
+typedef enum
+{
+	WALPERMIT_UNCHECKED,
+	WALPERMIT_CHECKED,
+	WALPERMIT_CHECKED_AND_USED
+} WALPermitCheckState;
+
+/* in access/walprohibit.c */
+extern WALPermitCheckState walpermit_checked_state;
+
+/*
+ * Reset walpermit_checked flag when no longer in the critical section.
+ * Otherwise, marked checked and used.
+ */
+#define RESET_WALPERMIT_CHECKED_STATE() \
+do { \
+	walpermit_checked_state = CritSectionCount ? \
+	WALPERMIT_CHECKED_AND_USED : WALPERMIT_UNCHECKED; \
+} while(0)
+#else
+#define RESET_WALPERMIT_CHECKED_STATE() ((void) 0)
+#endif
+
 #ifndef WIN32
 
 #define CHECK_FOR_INTERRUPTS() \
 do { \
 	if (unlikely(InterruptPending)) \
 		ProcessInterrupts(); \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 #else							/* WIN32 */
 
@@ -109,6 +134,7 @@ do { \
 		pgwin32_dispatch_queued_signals(); \
 	if (unlikely(InterruptPending)) \
 		ProcessInterrupts(); \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 #endif							/* WIN32 */
 
@@ -135,6 +161,7 @@ do { \
 do { \
 	Assert(CritSectionCount > 0); \
 	CritSectionCount--; \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 
 
-- 
2.22.0

From a8ea284ce517316a8fbae5c88cf8b2100b61e7be Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Tue, 14 Jul 2020 02:30:44 -0400
Subject: [PATCH v4 5/5] Documentation - WIP

TODOs:

1] TBC, the section for pg_is_in_readonly() function, right now it is under "Recovery Information Functions"
2] Documentation regarding ALTER SYSTEM READ/WRITE
---
 src/backend/access/transam/README | 59 ++++++++++++++++++++++++++++---
 src/backend/storage/page/README   | 12 +++----
 2 files changed, 60 insertions(+), 11 deletions(-)

diff --git a/src/backend/access/transam/README b/src/backend/access/transam/README
index eb9aac5fd39..f62929f1660 100644
--- a/src/backend/access/transam/README
+++ b/src/backend/access/transam/README
@@ -433,8 +433,8 @@ to be modified.
 2. START_CRIT_SECTION()  (Any error during the next three steps must cause a
 PANIC because the shared buffers will contain unlogged changes, which we
 have to ensure don't get to disk.  Obviously, you should check conditions
-such as whether there's enough free space on the page before you start the
-critical section.)
+such as whether there's WAL write permission and enough free space on the page
+before you start the critical section.)
 
 3. Apply the required changes to the shared buffer(s).
 
@@ -477,6 +477,54 @@ with the incomplete-split flag set, it will finish the interrupted split by
 inserting the key to the parent, before proceeding.
 
 
+Read only system state
+----------------------
+
+The system state when it is not currently possible to insert write-ahead log
+records, either because the system is still in recovery or because the system
+forced to WAL prohibited by executing ALTER SYSTEM READ ONLY.  We have a
+lower-level defense in XLogBeginInsert() and elsewhere to stop us from modifying
+data during recovery when !XLogInsertAllowed(), but if XLogBeginInsert() is
+inside the critical section we must not depend on it to report an error.
+Otherwise, it will cause PANIC as mentioned previously.
+
+We do not reach the point where we try to write WAL during recovery but ALTER
+SYSTEM READ ONLY can be executed anytime by the user to stop WAL writing.  Any
+backends which receive read-only system state transition barrier interrupt need
+to stop WAL writing immediately.  For barrier absorption the backed(s) will kill
+the running transaction which has valid XID indicates that the transaction has
+performed and/or planning WAL write.  The transaction which doesn't acquire
+valid XID yet or operation such VACUUM or CONCURRENT CREATE INDEX which not
+necessary have valid XID for WAL will not be prevented while barrier processing,
+and those might hit the error from XLogBeginInsert() while trying to write WAL
+in read only system state.  To prevent such error from XLogBeginInsert() inside
+the critical section the WAL write permission has to check before
+START_CRIT_SECTION().
+
+To enforce the practice to check WAL permission before entering into critical
+section for the WAL write, we have added an assert check flag that indicates
+permission has been checked before calling XLogBeginInsert().  If not,
+XLogBeginInsert() will have assertion failure.  WAL permission check is not
+mandatory if the XLogBeginInsert() is not inside the critical section where
+throwing the error is acceptable.  To get permission check flag set either
+CheckWALPermitted(), AssertWALPermitted_HaveXID(), or AssertWALPermitted()
+should be called before START_CRIT_SECTION().  This flag automatically resets
+while exiting from the critical section.  The rule to place either of permission
+check routines will be:
+
+	The places where WAL write operation in critical can be expected without
+	having valid XID (e.g vacuum) need to protect by CheckWALPermitted(), so
+	that error can be reported outside before critical section.
+
+	The places where INSERT and UPDATE are expected which are never happened
+	without valid XID can be checked using AssertWALPermitted_HaveXID.  So that
+	non-assert build will not have the checking overhead.
+
+	The places we know that we cannot be reached in the read-only state and may
+	or may not have XID, but need to ensure the permission has been checked on
+	assert enabled build should use AssertWALPermitted().
+
+
 Constructing a WAL record
 -------------------------
 
@@ -522,7 +570,8 @@ Details of the API functions:
 
 void XLogBeginInsert(void)
 
-    Must be called before XLogRegisterBuffer and XLogRegisterData.
+	Must be called before XLogRegisterBuffer and XLogRegisterData.  WAL
+	permission must be check before calling it in a critical section.
 
 void XLogResetInsertion(void)
 
@@ -630,8 +679,8 @@ If the buffer is clean and checksums are in use then
 MarkBufferDirtyHint() inserts an XLOG_FPI record to ensure that we
 take a full page image that includes the hint. We do this to avoid
 a partial page write, when we write the dirtied page. WAL is not
-written during recovery, so we simply skip dirtying blocks because
-of hints when in recovery.
+written while in read only (i.e. during recovery or in WAL prohibit state), so
+we simply skip dirtying blocks because of hints when in read only.
 
 If you do decide to optimise away a WAL record, then any calls to
 MarkBufferDirty() must be replaced by MarkBufferDirtyHint(),
diff --git a/src/backend/storage/page/README b/src/backend/storage/page/README
index e30d7ac59ad..15f0bb4b7b5 100644
--- a/src/backend/storage/page/README
+++ b/src/backend/storage/page/README
@@ -56,9 +56,9 @@ WAL is a fatal error and prevents further recovery, whereas a checksum failure
 on a normal data block is a hard error but not a critical one for the server,
 even if it is a very bad thing for the user.
 
-New WAL records cannot be written during recovery, so hint bits set during
-recovery must not dirty the page if the buffer is not already dirty, when
-checksums are enabled.  Systems in Hot-Standby mode may benefit from hint bits
-being set, but with checksums enabled, a page cannot be dirtied after setting a
-hint bit (due to the torn page risk). So, it must wait for full-page images
-containing the hint bit updates to arrive from the primary.
+New WAL records cannot be written during recovery or or while in WAL prohibit
+state, so hint bits set during recovery must not dirty the page if the buffer is
+not already dirty, when checksums are enabled.  Systems in Hot-Standby mode may
+benefit from hint bits being set, but with checksums enabled, a page cannot be
+dirtied after setting a hint bit (due to the torn page risk). So, it must wait
+for full-page images containing the hint bit updates to arrive from the primary.
-- 
2.22.0

From 0cc6c9dc8b942222d0bf09d7b5cbd8f85c94cd30 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 19 Jun 2020 06:29:36 -0400
Subject: [PATCH v4 3/5] Implement ALTER SYSTEM READ ONLY using global barrier.

Implementation:

 1. When a user tried to change server state to WAL-Prohibited using
	ALTER SYSTEM READ ONLY command; AlterSystemSetWALProhibitState()
	raises request to checkpointer by marking current state to inprogress in
	shared memory.  Checkpointer, noticing that the current state is has
	WALPROHIBIT_TRANSITION_IN_PROGRESS flag set, does the barrier request, and
	then acknowledges back to the backend who requested the state change once
	the transition has been completed.  Final state will be updated in control
	file to make it persistent across the system restarts.

 2. When a backend receives the WAL-Prohibited barrier, at that moment if
    it is already in a transaction and the transaction already assigned XID,
    then the backend will be killed by throwing FATAL(XXX: need more discussion
    on this)

 3. Otherwise, if that backend running transaction which yet to get XID
    assigned we don't need to do anything special, simply call
    ResetLocalXLogInsertAllowed() so that any future WAL insert in will check
    XLogInsertAllowed() first which set ready only state appropriately.

 4. A new transaction (from existing or new backend) starts as a read-only
    transaction.

 5. Autovacuum launcher as well as checkpointer will don't do anything in
 	WAL-Prohibited server state until someone wakes us up.  E.g. a backend
	might later on request us to put the system back to read-write.

 6. At shutdown in WAL-Prohibited mode, we'll skip shutdown checkpoint
    and xlog rotation. Starting up again will perform crash recovery(XXX:
    need some discussion on this as well)

 7. ALTER SYSTEM READ ONLY/WRITE is restricted on standby server.

 8. Only super user can toggle WAL-Prohibit state.

 9. Add system_is_read_only GUC show the system state -- will true when system
    is wal prohibited or in recovery.
---
 src/backend/access/transam/Makefile      |   1 +
 src/backend/access/transam/walprohibit.c |  97 +++++++++++++++
 src/backend/access/transam/xact.c        |  49 +++++---
 src/backend/access/transam/xlog.c        | 150 +++++++++++++++++++++--
 src/backend/postmaster/autovacuum.c      |   4 +
 src/backend/postmaster/bgwriter.c        |   2 +-
 src/backend/postmaster/checkpointer.c    | 117 ++++++++++++++++++
 src/backend/postmaster/pgstat.c          |   3 +
 src/backend/storage/ipc/procsignal.c     |  26 +---
 src/backend/tcop/utility.c               |  14 +--
 src/backend/utils/misc/guc.c             |  26 ++++
 src/bin/pg_controldata/pg_controldata.c  |   2 +
 src/include/access/walprohibit.h         |  32 +++++
 src/include/access/xlog.h                |   3 +
 src/include/catalog/pg_control.h         |   3 +
 src/include/pgstat.h                     |   1 +
 src/include/postmaster/bgwriter.h        |   2 +
 src/include/storage/procsignal.h         |   7 +-
 18 files changed, 466 insertions(+), 73 deletions(-)
 create mode 100644 src/backend/access/transam/walprohibit.c
 create mode 100644 src/include/access/walprohibit.h

diff --git a/src/backend/access/transam/Makefile b/src/backend/access/transam/Makefile
index 595e02de722..b5322a69954 100644
--- a/src/backend/access/transam/Makefile
+++ b/src/backend/access/transam/Makefile
@@ -26,6 +26,7 @@ OBJS = \
 	twophase.o \
 	twophase_rmgr.o \
 	varsup.o \
+	walprohibit.o \
 	xact.o \
 	xlog.o \
 	xlogarchive.o \
diff --git a/src/backend/access/transam/walprohibit.c b/src/backend/access/transam/walprohibit.c
new file mode 100644
index 00000000000..a3f1a750744
--- /dev/null
+++ b/src/backend/access/transam/walprohibit.c
@@ -0,0 +1,97 @@
+/*-------------------------------------------------------------------------
+ *
+ * walprohibit.c
+ * 		PostgreSQL write-ahead log prohibit states
+ *
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/backend/access/transam/walprohibit.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/walprohibit.h"
+#include "postmaster/bgwriter.h"
+#include "storage/procsignal.h"
+
+/*
+ * ProcessBarrierWALProhibit()
+ *
+ * Handle WAL prohibit state change request.
+ */
+bool
+ProcessBarrierWALProhibit(void)
+{
+	/*
+	 * Kill off any transactions that have an XID *before* allowing the system
+	 * to go WAL prohibit state.
+	 */
+	if (FullTransactionIdIsValid(GetTopFullTransactionIdIfAny()))
+	{
+		Assert(GetWALProhibitState() & WALPROHIBIT_STATE_READ_ONLY);
+
+		/*
+		 * XXX: Kill off the whole session by throwing FATAL instead of killing
+		 * transaction by throwing ERROR due to following reasons that need be
+		 * thought:
+		 *
+		 * 1. Due to some presents challenges with the wire protocol, we could
+		 * not simply kill of idle transaction.
+		 *
+		 * 2. If we are here in subtransaction then the ERROR will kill the
+		 * current subtransaction only. In the case of invalidations, that
+		 * might be good enough, but for XID assignment it's not, because
+		 * assigning an XID to a subtransaction also causes higher
+		 * sub-transaction levels and the parent transaction to get XIDs.
+		 */
+		ereport(FATAL,
+				(errcode(ERRCODE_ACTIVE_SQL_TRANSACTION),
+				 errmsg("system is now read only"),
+				 errhint("Cannot continue a transaction if it has performed writes while system is read only.")));
+	}
+
+	/* Return to "check" state */
+	ResetLocalXLogInsertAllowed();
+
+	return true;
+}
+
+/*
+ * AlterSystemSetWALProhibitState
+ *
+ * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
+ */
+void
+AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
+{
+	uint32		state;
+
+	if (!superuser())
+		ereport(ERROR,
+				(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+				 errmsg("must be superuser to execute ALTER SYSTEM command")));
+
+	/* Alter WAL prohibit state not allowed during recovery */
+	PreventCommandDuringRecovery("ALTER SYSTEM");
+
+	/* Requested state */
+	state = stmt->WALProhibited ?
+		WALPROHIBIT_STATE_READ_ONLY : WALPROHIBIT_STATE_READ_WRITE;
+
+	/*
+	 * Since we yet to convey this WAL prohibit state to all backend mark it
+	 * in-progress.
+	 */
+	state |= WALPROHIBIT_TRANSITION_IN_PROGRESS;
+
+	if (!SetWALProhibitState(state))
+		return; /* server is already in the desired state */
+
+	/*
+	 * Signal the checkpointer to do the actual state transition, and wait for
+	 * the state change to occur.
+	 */
+	WALProhibitRequest();
+}
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index d4f7c29847f..8ebc4242c25 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -1949,23 +1949,28 @@ StartTransaction(void)
 	Assert(s->prevSecContext == 0);
 
 	/*
-	 * Make sure we've reset xact state variables
+	 * Reset xact state variables.
 	 *
-	 * If recovery is still in progress, mark this transaction as read-only.
-	 * We have lower level defences in XLogInsert and elsewhere to stop us
-	 * from modifying data during recovery, but this gives the normal
-	 * indication to the user that the transaction is read-only.
-	 */
-	if (RecoveryInProgress())
-	{
-		s->startedInRecovery = true;
-		XactReadOnly = true;
-	}
-	else
-	{
-		s->startedInRecovery = false;
-		XactReadOnly = DefaultXactReadOnly;
-	}
+	 * If it is not currently possible to insert write-ahead log records,
+	 * either because we are still in recovery or because ALTER SYSTEM READ
+	 * ONLY has been executed, force this to be a read-only transaction.
+	 * We have lower level defences in XLogBeginInsert() and elsewhere to stop
+	 * us from modifying data during recovery when !XLogInsertAllowed(), but
+	 * this gives the normal indication to the user that the transaction is
+	 * read-only.
+	 *
+	 * On the other hand, we only need to set the startedInRecovery flag when
+	 * the transaction started during recovery, and not when WAL is otherwise
+	 * prohibited. This information is used by RelationGetIndexScan() to
+	 * decide whether to permit (1) relying on existing killed-tuple markings
+	 * and (2) further killing of index tuples. Even when WAL is prohibited
+	 * on the master, it's still the master, so the former is OK; and since
+	 * killing index tuples doesn't generate WAL, the latter is also OK.
+	 * See comments in RelationGetIndexScan() and MarkBufferDirtyHint().
+	 */
+	XactReadOnly = DefaultXactReadOnly || !XLogInsertAllowed();
+	s->startedInRecovery = RecoveryInProgress();
+
 	XactDeferrable = DefaultXactDeferrable;
 	XactIsoLevel = DefaultXactIsoLevel;
 	forceSyncCommit = false;
@@ -4887,9 +4892,11 @@ CommitSubTransaction(void)
 	/*
 	 * We need to restore the upper transaction's read-only state, in case the
 	 * upper is read-write while the child is read-only; GUC will incorrectly
-	 * think it should leave the child state in place.
+	 * think it should leave the child state in place.  Note that the upper
+	 * transaction will be a force to ready-only irrespective of its previous
+	 * status if the server state is WAL prohibited.
 	 */
-	XactReadOnly = s->prevXactReadOnly;
+	XactReadOnly = s->prevXactReadOnly || !XLogInsertAllowed();
 
 	CurrentResourceOwner = s->parent->curTransactionOwner;
 	CurTransactionResourceOwner = s->parent->curTransactionOwner;
@@ -5045,9 +5052,11 @@ AbortSubTransaction(void)
 	/*
 	 * Restore the upper transaction's read-only state, too.  This should be
 	 * redundant with GUC's cleanup but we may as well do it for consistency
-	 * with the commit case.
+	 * with the commit case.  Note that the upper transaction will be a force
+	 * to ready-only irrespective of its previous status if the server state is
+	 * WAL prohibited.
 	 */
-	XactReadOnly = s->prevXactReadOnly;
+	XactReadOnly = s->prevXactReadOnly || !XLogInsertAllowed();
 
 	RESUME_INTERRUPTS();
 }
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 184c6672f3b..6d33ea226b1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -31,6 +31,7 @@
 #include "access/timeline.h"
 #include "access/transam.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog_internal.h"
 #include "access/xlogarchive.h"
@@ -245,9 +246,10 @@ static bool LocalPromoteIsTriggered = false;
  *		0: unconditionally not allowed to insert XLOG
  *		-1: must check RecoveryInProgress(); disallow until it is false
  * Most processes start with -1 and transition to 1 after seeing that recovery
- * is not in progress.  But we can also force the value for special cases.
- * The coding in XLogInsertAllowed() depends on the first two of these states
- * being numerically the same as bool true and false.
+ * is not in progress or the server state is not a WAL prohibited state.  But
+ * we can also force the value for special cases.  The coding in
+ * XLogInsertAllowed() depends on the first two of these states being
+ * numerically the same as bool true and false.
  */
 static int	LocalXLogInsertAllowed = -1;
 
@@ -660,6 +662,12 @@ typedef struct XLogCtlData
 	 */
 	RecoveryState SharedRecoveryState;
 
+	/*
+	 * SharedWALProhibitState indicates current WAL prohibit state.
+	 * Protected by info_lck.
+	 */
+	uint32		SharedWALProhibitState;
+
 	/*
 	 * SharedHotStandbyActive indicates if we allow hot standby queries to be
 	 * run.  Protected by info_lck.
@@ -970,6 +978,7 @@ static void WALInsertLockAcquire(void);
 static void WALInsertLockAcquireExclusive(void);
 static void WALInsertLockRelease(void);
 static void WALInsertLockUpdateInsertingAt(XLogRecPtr insertingAt);
+static inline bool IsWALProhibited(void);
 
 /*
  * Insert an XLOG record represented by an already-constructed chain of data
@@ -7706,6 +7715,15 @@ StartupXLOG(void)
 	XLogCtl->LogwrtRqst.Write = EndOfLog;
 	XLogCtl->LogwrtRqst.Flush = EndOfLog;
 
+	/*
+	 * Before enabling WAL insertion update WAL prohibit state in shared memory
+	 * that will decide the further WAL insert should be allowed or not.
+	 */
+	SpinLockAcquire(&XLogCtl->info_lck);
+	XLogCtl->SharedWALProhibitState = ControlFile->wal_prohibited ?
+		WALPROHIBIT_STATE_READ_ONLY : WALPROHIBIT_STATE_READ_WRITE;
+	SpinLockRelease(&XLogCtl->info_lck);
+
 	/*
 	 * Update full_page_writes in shared memory and write an XLOG_FPW_CHANGE
 	 * record before resource manager writes cleanup WAL records or checkpoint
@@ -7716,7 +7734,15 @@ StartupXLOG(void)
 	UpdateFullPageWrites();
 	LocalXLogInsertAllowed = -1;
 
-	if (InRecovery)
+	/*
+	 * Skip end-of-recovery checkpoint if the system is in WAL prohibited state.
+	 */
+	if (ControlFile->wal_prohibited && InRecovery)
+	{
+		ereport(LOG,
+				(errmsg("skipping startup checkpoint because the system is read only")));
+	}
+	else if (InRecovery)
 	{
 		/*
 		 * Perform a checkpoint to update all our recovery activity to disk.
@@ -7962,6 +7988,83 @@ StartupXLOG(void)
 		RequestCheckpoint(CHECKPOINT_FORCE);
 }
 
+/* Atomically return the current server WAL prohibited state */
+uint32
+GetWALProhibitState(void)
+{
+	uint32		state;
+
+	SpinLockAcquire(&XLogCtl->info_lck);
+	state = XLogCtl->SharedWALProhibitState;
+	SpinLockRelease(&XLogCtl->info_lck);
+
+	return state;
+}
+
+/*
+ * SetWALProhibitState: Change current wal prohibit state to the input state.
+ *
+ * If the server is already completely moved to the requested WAL prohibit
+ * state, or if the desired state is same as the current state, return false,
+ * indicating that the server state did not change. Else return true.
+ */
+bool
+SetWALProhibitState(uint32 new_state)
+{
+	uint32		cur_state;
+
+	cur_state = GetWALProhibitState();
+
+	/* Server is already in requested state */
+	if (new_state == cur_state ||
+		new_state == (cur_state | WALPROHIBIT_TRANSITION_IN_PROGRESS))
+		return false;
+
+	/* Prevent concurrent contrary in progress transition state setting */
+	if ((new_state & WALPROHIBIT_TRANSITION_IN_PROGRESS) &&
+		(cur_state & WALPROHIBIT_TRANSITION_IN_PROGRESS))
+	{
+		if (cur_state & WALPROHIBIT_STATE_READ_ONLY)
+			ereport(ERROR,
+					(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+					 errmsg("system state transition to read only is already in progress"),
+					 errhint("Try after sometime again.")));
+		else
+			ereport(ERROR,
+					(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+					 errmsg("system state transition to read write is already in progress"),
+					 errhint("Try after sometime again.")));
+
+	}
+
+	/* Update new state in share memory */
+	SpinLockAcquire(&XLogCtl->info_lck);
+	XLogCtl->SharedWALProhibitState = new_state;
+	SpinLockRelease(&XLogCtl->info_lck);
+
+	/* Update control file if it is the final state */
+	if (!(new_state & WALPROHIBIT_TRANSITION_IN_PROGRESS))
+	{
+		bool	wal_prohibited = (new_state & WALPROHIBIT_STATE_READ_ONLY) != 0;
+
+		LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+		ControlFile->wal_prohibited = wal_prohibited;
+		UpdateControlFile();
+		LWLockRelease(ControlFileLock);
+	}
+
+	return true;
+}
+
+/*
+ * Is the system still in WAL prohibited state?
+ */
+static inline bool
+IsWALProhibited(void)
+{
+	return (GetWALProhibitState() & WALPROHIBIT_STATE_READ_ONLY) != 0;
+}
+
 /*
  * Checks if recovery has reached a consistent state. When consistency is
  * reached and we have a valid starting standby snapshot, tell postmaster
@@ -8177,9 +8280,9 @@ HotStandbyActiveInReplay(void)
 /*
  * Is this process allowed to insert new WAL records?
  *
- * Ordinarily this is essentially equivalent to !RecoveryInProgress().
- * But we also have provisions for forcing the result "true" or "false"
- * within specific processes regardless of the global state.
+ * Ordinarily this is essentially equivalent to !RecoveryInProgress() and
+ * !IsWALProhibited().  But we also have provisions for forcing the result
+ * "true" or "false" within specific processes regardless of the global state.
  */
 bool
 XLogInsertAllowed(void)
@@ -8193,14 +8296,25 @@ XLogInsertAllowed(void)
 		return (bool) LocalXLogInsertAllowed;
 
 	/*
-	 * Else, must check to see if we're still in recovery.
+	 * Else, must check to see if we're still in recovery
 	 */
 	if (RecoveryInProgress())
 		return false;
 
+	/* Or, in WAL prohibited state */
+	if (IsWALProhibited())
+	{
+		/*
+		 * Set it to "unconditionally false" to avoid checking until it gets
+		 * reset.
+		 */
+		LocalXLogInsertAllowed = 0;
+		return false;
+	}
+
 	/*
-	 * On exit from recovery, reset to "unconditionally true", since there is
-	 * no need to keep checking.
+	 * On exit from recovery or WAL prohibited state, reset to "unconditionally
+	 * true", since there is no need to keep checking.
 	 */
 	LocalXLogInsertAllowed = 1;
 	return true;
@@ -8216,12 +8330,19 @@ static void
 LocalSetXLogInsertAllowed(void)
 {
 	Assert(LocalXLogInsertAllowed == -1);
+
 	LocalXLogInsertAllowed = 1;
 
 	/* Initialize as RecoveryInProgress() would do when switching state */
 	InitXLOGAccess();
 }
 
+void
+ResetLocalXLogInsertAllowed(void)
+{
+	LocalXLogInsertAllowed = -1;
+}
+
 /*
  * Subroutine to try to fetch and validate a prior checkpoint record.
  *
@@ -8513,7 +8634,10 @@ ShutdownXLOG(int code, Datum arg)
 
 	if (RecoveryInProgress())
 		CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
-	else
+	/*
+	 * Can't perform checkpoint or xlog rotation without writing WAL.
+	 */
+	else if (XLogInsertAllowed())
 	{
 		/*
 		 * If archiving is enabled, rotate the last XLOG file so that all the
@@ -8526,6 +8650,10 @@ ShutdownXLOG(int code, Datum arg)
 
 		CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
 	}
+	else
+		ereport(LOG,
+			   (errmsg("skipping shutdown checkpoint because the system is read only")));
+
 	ShutdownCLOG();
 	ShutdownCommitTs();
 	ShutdownSUBTRANS();
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 9c7d4b0c60e..f83f86994db 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -652,6 +652,10 @@ AutoVacLauncherMain(int argc, char *argv[])
 
 		HandleAutoVacLauncherInterrupts();
 
+		/* If the server is read only just go back to sleep. */
+		if (!XLogInsertAllowed())
+			continue;
+
 		/*
 		 * a worker finished, or postmaster signaled failure to start a worker
 		 */
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index 069e27e427f..6c6ff7dc3af 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -268,7 +268,7 @@ BackgroundWriterMain(void)
 		 * Checkpointer, when active, is barely ever in its mainloop and thus
 		 * makes it hard to log regularly.
 		 */
-		if (XLogStandbyInfoActive() && !RecoveryInProgress())
+		if (XLogStandbyInfoActive() && XLogInsertAllowed())
 		{
 			TimestampTz timeout = 0;
 			TimestampTz now = GetCurrentTimestamp();
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b80..3e8aa9a0ec3 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -39,6 +39,7 @@
 #include <sys/time.h>
 #include <time.h>
 
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xlog_internal.h"
 #include "libpq/pqsignal.h"
@@ -127,6 +128,9 @@ typedef struct
 	ConditionVariable start_cv; /* signaled when ckpt_started advances */
 	ConditionVariable done_cv;	/* signaled when ckpt_done advances */
 
+	ConditionVariable walprohibit_cv; /* signaled when requested wal
+										 prohibit state changes */
+
 	uint32		num_backend_writes; /* counts user backend buffer writes */
 	uint32		num_backend_fsync;	/* counts user backend fsync calls */
 
@@ -168,6 +172,7 @@ static bool IsCheckpointOnSchedule(double progress);
 static bool ImmediateCheckpointRequested(void);
 static bool CompactCheckpointerRequestQueue(void);
 static void UpdateSharedMemoryConfig(void);
+static void performWALProhibitStateChange(uint32 wal_state);
 
 /* Signal handlers */
 static void ReqCheckpointHandler(SIGNAL_ARGS);
@@ -332,6 +337,7 @@ CheckpointerMain(void)
 		pg_time_t	now;
 		int			elapsed_secs;
 		int			cur_timeout;
+		uint32		wal_state;
 
 		/* Clear any already-pending wakeups */
 		ResetLatch(MyLatch);
@@ -342,6 +348,28 @@ CheckpointerMain(void)
 		AbsorbSyncRequests();
 		HandleCheckpointerInterrupts();
 
+		wal_state = GetWALProhibitState();
+
+		if (wal_state & WALPROHIBIT_TRANSITION_IN_PROGRESS)
+		{
+			/* Complete WAL prohibit state change request */
+			performWALProhibitStateChange(wal_state);
+			continue;
+		}
+		else if (wal_state & WALPROHIBIT_STATE_READ_ONLY)
+		{
+			/*
+			 * Don't do anything until someone wakes us up.  For example a
+			 * backend might later on request us to put the system back to
+			 * read-write wal prohibit sate.
+			 */
+			(void) WaitLatch(MyLatch, WL_LATCH_SET | WL_EXIT_ON_PM_DEATH, -1,
+							 WAIT_EVENT_CHECKPOINTER_MAIN);
+			continue;
+		}
+
+		Assert(wal_state == WALPROHIBIT_STATE_READ_WRITE);
+
 		/*
 		 * Detect a pending checkpoint request by checking whether the flags
 		 * word in shared memory is nonzero.  We shouldn't need to acquire the
@@ -879,6 +907,7 @@ CheckpointerShmemInit(void)
 		CheckpointerShmem->max_requests = NBuffers;
 		ConditionVariableInit(&CheckpointerShmem->start_cv);
 		ConditionVariableInit(&CheckpointerShmem->done_cv);
+		ConditionVariableInit(&CheckpointerShmem->walprohibit_cv);
 	}
 }
 
@@ -1109,6 +1138,94 @@ ForwardSyncRequest(const FileTag *ftag, SyncRequestType type)
 	return true;
 }
 
+/*
+ * WALProhibitedRequest: Request checkpointer to make the WALProhibitState to
+ * read-only.
+ */
+void
+WALProhibitRequest(void)
+{
+	/* Must not be called from checkpointer */
+	Assert(!AmCheckpointerProcess());
+	Assert(GetWALProhibitState() & WALPROHIBIT_TRANSITION_IN_PROGRESS);
+
+	/*
+	 * If in a standalone backend, just do it ourselves.
+	 */
+	if (!IsPostmasterEnvironment)
+	{
+		performWALProhibitStateChange(GetWALProhibitState());
+		return;
+	}
+
+	if (CheckpointerShmem->checkpointer_pid == 0)
+		elog(ERROR, "checkpointer is not running");
+
+	if (kill(CheckpointerShmem->checkpointer_pid, SIGINT) != 0)
+		elog(ERROR, "could not signal checkpointer: %m");
+
+	/* Wait for the state to change to read-only */
+	ConditionVariablePrepareToSleep(&CheckpointerShmem->walprohibit_cv);
+	for (;;)
+	{
+		/*  We'll be done once in-progress flag bit is cleared */
+		if (!(GetWALProhibitState() & WALPROHIBIT_TRANSITION_IN_PROGRESS))
+			break;
+
+		elog(DEBUG1, "WALProhibitRequest: Waiting for checkpointer");
+		ConditionVariableSleep(&CheckpointerShmem->walprohibit_cv,
+							   WAIT_EVENT_SYSTEM_WALPROHIBIT_STATE_CHANGE);
+	}
+	ConditionVariableCancelSleep();
+	elog(DEBUG1, "Done WALProhibitRequest");
+}
+
+/*
+ * performWALProhibitStateChange: checkpointer will call this to complete
+ * the requested WAL prohibit state transition.
+ */
+static void
+performWALProhibitStateChange(uint32 wal_state)
+{
+	uint64		barrierGeneration;
+
+	/*
+	 * Must be called from checkpointer. Otherwise, it must be single-user
+	 * backend.
+	 */
+	Assert(AmCheckpointerProcess() || !IsPostmasterEnvironment);
+	Assert(wal_state & WALPROHIBIT_TRANSITION_IN_PROGRESS);
+
+	/*
+	 * WAL prohibit state change is initiated. We need to complete the state
+	 * transition by setting requested WAL prohibit state in all backends.
+	 */
+	elog(DEBUG1, "waiting for backends to adopt requested WAL prohibit state");
+
+	/* Emit global barrier */
+	barrierGeneration = EmitProcSignalBarrier(PROCSIGNAL_BARRIER_WALPROHIBIT);
+	WaitForProcSignalBarrier(barrierGeneration);
+
+	/* And flush all writes. */
+	XLogFlush(GetXLogWriteRecPtr());
+
+	/* Set final state by clearing in-progress flag bit */
+	if (SetWALProhibitState(wal_state & ~(WALPROHIBIT_TRANSITION_IN_PROGRESS)))
+	{
+		if ((wal_state & WALPROHIBIT_STATE_READ_ONLY) != 0)
+			ereport(LOG, (errmsg("system is now read only")));
+		else
+		{
+			/* Request checkpoint */
+			RequestCheckpoint(CHECKPOINT_IMMEDIATE);
+			ereport(LOG, (errmsg("system is now read write")));
+		}
+	}
+
+	/* Wake up the backend who requested the state change */
+	ConditionVariableBroadcast(&CheckpointerShmem->walprohibit_cv);
+}
+
 /*
  * CompactCheckpointerRequestQueue
  *		Remove duplicates from the request queue to avoid backend fsyncs.
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 88992c2da2c..0eb40a86b52 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -4057,6 +4057,9 @@ pgstat_get_wait_io(WaitEventIO w)
 		case WAIT_EVENT_REPLICATION_SLOT_WRITE:
 			event_name = "ReplicationSlotWrite";
 			break;
+		case WAIT_EVENT_SYSTEM_WALPROHIBIT_STATE_CHANGE:
+			event_name = "SystemWALProhibitStateChange";
+			break;
 		case WAIT_EVENT_SLRU_FLUSH_SYNC:
 			event_name = "SLRUFlushSync";
 			break;
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 13648887187..b973727a580 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -18,6 +18,7 @@
 #include <unistd.h>
 
 #include "access/parallel.h"
+#include "access/walprohibit.h"
 #include "commands/async.h"
 #include "miscadmin.h"
 #include "pgstat.h"
@@ -96,7 +97,6 @@ static volatile ProcSignalSlot *MyProcSignalSlot = NULL;
 
 static bool CheckProcSignal(ProcSignalReason reason);
 static void CleanupProcSignalState(int status, Datum arg);
-static bool ProcessBarrierPlaceholder(void);
 
 /*
  * ProcSignalShmemSize
@@ -510,9 +510,9 @@ ProcessProcSignalBarrier(void)
 			 * unconditionally, but it's more efficient to call only the ones
 			 * that might need us to do something based on the flags.
 			 */
-			if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_PLACEHOLDER)
-				&& ProcessBarrierPlaceholder())
-				BARRIER_CLEAR_BIT(flags, PROCSIGNAL_BARRIER_PLACEHOLDER);
+			if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_WALPROHIBIT)
+				&& ProcessBarrierWALProhibit())
+				BARRIER_CLEAR_BIT(flags, PROCSIGNAL_BARRIER_WALPROHIBIT);
 		}
 		PG_CATCH();
 		{
@@ -554,24 +554,6 @@ ProcessProcSignalBarrier(void)
 	pg_atomic_write_u64(&MyProcSignalSlot->pss_barrierGeneration, shared_gen);
 }
 
-static bool
-ProcessBarrierPlaceholder(void)
-{
-	/*
-	 * XXX. This is just a placeholder until the first real user of this
-	 * machinery gets committed. Rename PROCSIGNAL_BARRIER_PLACEHOLDER to
-	 * PROCSIGNAL_BARRIER_SOMETHING_ELSE where SOMETHING_ELSE is something
-	 * appropriately descriptive. Get rid of this function and instead have
-	 * ProcessBarrierSomethingElse. Most likely, that function should live in
-	 * the file pertaining to that subsystem, rather than here.
-	 *
-	 * The return value should be 'true' if the barrier was successfully
-	 * absorbed and 'false' if not. Note that returning 'false' can lead to
-	 * very frequent retries, so try hard to make that an uncommon case.
-	 */
-	return true;
-}
-
 /*
  * CheckProcSignal - check to see if a particular reason has been
  * signaled, and clear the signal flag.  Should be called after receiving
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 7af96c77082..d6411e4f3e9 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -19,6 +19,7 @@
 #include "access/htup_details.h"
 #include "access/reloptions.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "catalog/catalog.h"
@@ -85,7 +86,6 @@ static void ProcessUtilitySlow(ParseState *pstate,
 							   DestReceiver *dest,
 							   QueryCompletion *qc);
 static void ExecDropStmt(DropStmt *stmt, bool isTopLevel);
-static void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
 
 /*
  * CommandIsReadOnly: is an executable query read-only?
@@ -3644,15 +3644,3 @@ GetCommandLogLevel(Node *parsetree)
 
 	return lev;
 }
-
-/*
- * AlterSystemSetWALProhibitState
- *
- * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
- */
-static void
-AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
-{
-	/* some code */
-	elog(INFO, "AlterSystemSetWALProhibitState() called");
-}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index abfa95a2314..a2f796164a7 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -225,6 +225,7 @@ static bool check_recovery_target_lsn(char **newval, void **extra, GucSource sou
 static void assign_recovery_target_lsn(const char *newval, void *extra);
 static bool check_primary_slot_name(char **newval, void **extra, GucSource source);
 static bool check_default_with_oids(bool *newval, void **extra, GucSource source);
+static const char *show_system_is_read_only(void);
 
 /* Private functions in guc-file.l that need to be called from guc.c */
 static ConfigVariable *ProcessConfigFileInternal(GucContext context,
@@ -615,6 +616,7 @@ static char *recovery_target_string;
 static char *recovery_target_xid_string;
 static char *recovery_target_name_string;
 static char *recovery_target_lsn_string;
+static bool system_is_read_only;
 
 
 /* should be static, but commands/variable.c needs to get at this */
@@ -2036,6 +2038,18 @@ static struct config_bool ConfigureNamesBool[] =
 		NULL, NULL, NULL
 	},
 
+	{
+		/* Not for general use */
+		{"system_is_read_only", PGC_INTERNAL, WAL,
+			gettext_noop("Shows whether the system is read only."),
+			NULL,
+			GUC_NO_RESET_ALL | GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE
+		},
+		&system_is_read_only,
+		false,
+		NULL, NULL, show_system_is_read_only
+	},
+
 	/* End-of-list marker */
 	{
 		{NULL, 0, 0, NULL, NULL}, NULL, false, NULL, NULL, NULL
@@ -12019,4 +12033,16 @@ check_default_with_oids(bool *newval, void **extra, GucSource source)
 	return true;
 }
 
+/*
+ * NB: The return string should be the same as the _ShowOption() for boolean
+ * type.
+ */
+ static const char *
+ show_system_is_read_only(void)
+{
+	if (!XLogInsertAllowed())
+		return "on";
+	return "off";
+}
+
 #include "guc-file.c"
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index e73639df744..9594df76946 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -290,6 +290,8 @@ main(int argc, char *argv[])
 		   (uint32) ControlFile->backupEndPoint);
 	printf(_("End-of-backup record required:        %s\n"),
 		   ControlFile->backupEndRequired ? _("yes") : _("no"));
+	printf(_("WAL write prohibited:                 %s\n"),
+		   ControlFile->wal_prohibited ? _("yes") : _("no"));
 	printf(_("wal_level setting:                    %s\n"),
 		   wal_level_str(ControlFile->wal_level));
 	printf(_("wal_log_hints setting:                %s\n"),
diff --git a/src/include/access/walprohibit.h b/src/include/access/walprohibit.h
new file mode 100644
index 00000000000..163fe0d2fce
--- /dev/null
+++ b/src/include/access/walprohibit.h
@@ -0,0 +1,32 @@
+/*
+ * walprohibit.h
+ *
+ * PostgreSQL write-ahead log prohibit states
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/include/access/walprohibit.h
+ */
+#ifndef WALPROHIBIT_H
+#define WALPROHIBIT_H
+
+#include "access/xact.h"
+#include "access/xlog.h"
+#include "miscadmin.h"
+#include "nodes/parsenodes.h"
+
+extern bool ProcessBarrierWALProhibit(void);
+extern void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
+
+/* WAL Prohibit States */
+#define	WALPROHIBIT_STATE_READ_WRITE		0x0000
+#define	WALPROHIBIT_STATE_READ_ONLY			0x0001
+
+/*
+ * The bit is used in state transition from one state to another.  When this
+ * bit is set then the state indicated by the 0th position bit is yet to
+ * confirmed.
+ */
+#define WALPROHIBIT_TRANSITION_IN_PROGRESS	0x0002
+
+#endif		/* WALPROHIBIT_H */
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 219a7299e1f..5f5c2146e7e 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -306,6 +306,7 @@ extern RecoveryState GetRecoveryState(void);
 extern bool HotStandbyActive(void);
 extern bool HotStandbyActiveInReplay(void);
 extern bool XLogInsertAllowed(void);
+extern void ResetLocalXLogInsertAllowed(void);
 extern void GetXLogReceiptTime(TimestampTz *rtime, bool *fromStream);
 extern XLogRecPtr GetXLogReplayRecPtr(TimeLineID *replayTLI);
 extern XLogRecPtr GetXLogInsertRecPtr(void);
@@ -325,6 +326,8 @@ extern void XLOGShmemInit(void);
 extern void BootStrapXLOG(void);
 extern void LocalProcessControlFile(bool reset);
 extern void StartupXLOG(void);
+extern uint32 GetWALProhibitState(void);
+extern bool SetWALProhibitState(uint32 new_state);
 extern void ShutdownXLOG(int code, Datum arg);
 extern void InitXLOGAccess(void);
 extern void CreateCheckPoint(int flags);
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index de5670e5382..b32c7723275 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -182,6 +182,9 @@ typedef struct ControlFileData
 	int			max_locks_per_xact;
 	bool		track_commit_timestamp;
 
+	/* WAL prohibited determines if the WAL insert is allowed or not. */
+	bool		wal_prohibited;
+
 	/*
 	 * This data is used to check for hardware-architecture compatibility of
 	 * the database and the backend executable.  We need not check endianness
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 13872013823..780c59f3e48 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -955,6 +955,7 @@ typedef enum
 	WAIT_EVENT_REPLICATION_SLOT_RESTORE_SYNC,
 	WAIT_EVENT_REPLICATION_SLOT_SYNC,
 	WAIT_EVENT_REPLICATION_SLOT_WRITE,
+	WAIT_EVENT_SYSTEM_WALPROHIBIT_STATE_CHANGE,
 	WAIT_EVENT_SLRU_FLUSH_SYNC,
 	WAIT_EVENT_SLRU_READ,
 	WAIT_EVENT_SLRU_SYNC,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e6..e8271b49f6d 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -35,6 +35,8 @@ extern void CheckpointWriteDelay(int flags, double progress);
 
 extern bool ForwardSyncRequest(const FileTag *ftag, SyncRequestType type);
 
+extern void WALProhibitRequest(void);
+
 extern void AbsorbSyncRequests(void);
 
 extern Size CheckpointerShmemSize(void);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f38..bae06202b4a 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -48,12 +48,7 @@ typedef enum
 
 typedef enum
 {
-	/*
-	 * XXX. PROCSIGNAL_BARRIER_PLACEHOLDER should be replaced when the first
-	 * real user of the ProcSignalBarrier mechanism is added. It's just here
-	 * for now because we can't have an empty enum.
-	 */
-	PROCSIGNAL_BARRIER_PLACEHOLDER = 0
+	PROCSIGNAL_BARRIER_WALPROHIBIT = 0
 } ProcSignalBarrierType;
 
 /*
-- 
2.22.0

From 8629758383237c7ed1249fa6038c15fcd34f9ddd Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 19 Jun 2020 06:29:36 -0400
Subject: [PATCH v5 3/5] Implement ALTER SYSTEM READ ONLY using global barrier.

Implementation:

 1. When a user tried to change server state to WAL-Prohibited using
	ALTER SYSTEM READ ONLY command; AlterSystemSetWALProhibitState()
	raises request to checkpointer by marking current state to inprogress in
	shared memory.  Checkpointer, noticing that the current state is has
	WALPROHIBIT_TRANSITION_IN_PROGRESS flag set, does the barrier request, and
	then acknowledges back to the backend who requested the state change once
	the transition has been completed.  Final state will be updated in control
	file to make it persistent across the system restarts.

 2. When a backend receives the WAL-Prohibited barrier, at that moment if
    it is already in a transaction and the transaction already assigned XID,
    then the backend will be killed by throwing FATAL(XXX: need more discussion
    on this)

 3. Otherwise, if that backend running transaction which yet to get XID
    assigned we don't need to do anything special, simply call
    ResetLocalXLogInsertAllowed() so that any future WAL insert in will check
    XLogInsertAllowed() first which set ready only state appropriately.

 4. A new transaction (from existing or new backend) starts as a read-only
    transaction.

 5. Autovacuum launcher as well as checkpointer will don't do anything in
 	WAL-Prohibited server state until someone wakes us up.  E.g. a backend
	might later on request us to put the system back to read-write.

 6. At shutdown in WAL-Prohibited mode, we'll skip shutdown checkpoint
    and xlog rotation. Starting up again will perform crash recovery(XXX:
    need some discussion on this as well)

 7. ALTER SYSTEM READ ONLY/WRITE is restricted on standby server.

 8. Only super user can toggle WAL-Prohibit state.

 9. Add system_is_read_only GUC show the system state -- will true when system
    is wal prohibited or in recovery.
---
 src/backend/access/transam/Makefile      |   1 +
 src/backend/access/transam/walprohibit.c |  97 +++++++++++++++
 src/backend/access/transam/xact.c        |  49 +++++---
 src/backend/access/transam/xlog.c        | 150 +++++++++++++++++++++--
 src/backend/postmaster/autovacuum.c      |   4 +
 src/backend/postmaster/bgwriter.c        |   2 +-
 src/backend/postmaster/checkpointer.c    | 117 ++++++++++++++++++
 src/backend/postmaster/pgstat.c          |   3 +
 src/backend/storage/ipc/procsignal.c     |  26 +---
 src/backend/tcop/utility.c               |  14 +--
 src/backend/utils/misc/guc.c             |  26 ++++
 src/bin/pg_controldata/pg_controldata.c  |   2 +
 src/include/access/walprohibit.h         |  32 +++++
 src/include/access/xlog.h                |   3 +
 src/include/catalog/pg_control.h         |   3 +
 src/include/pgstat.h                     |   1 +
 src/include/postmaster/bgwriter.h        |   2 +
 src/include/storage/procsignal.h         |   7 +-
 18 files changed, 466 insertions(+), 73 deletions(-)
 create mode 100644 src/backend/access/transam/walprohibit.c
 create mode 100644 src/include/access/walprohibit.h

diff --git a/src/backend/access/transam/Makefile b/src/backend/access/transam/Makefile
index 595e02de722..b5322a69954 100644
--- a/src/backend/access/transam/Makefile
+++ b/src/backend/access/transam/Makefile
@@ -26,6 +26,7 @@ OBJS = \
 	twophase.o \
 	twophase_rmgr.o \
 	varsup.o \
+	walprohibit.o \
 	xact.o \
 	xlog.o \
 	xlogarchive.o \
diff --git a/src/backend/access/transam/walprohibit.c b/src/backend/access/transam/walprohibit.c
new file mode 100644
index 00000000000..a3f1a750744
--- /dev/null
+++ b/src/backend/access/transam/walprohibit.c
@@ -0,0 +1,97 @@
+/*-------------------------------------------------------------------------
+ *
+ * walprohibit.c
+ * 		PostgreSQL write-ahead log prohibit states
+ *
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/backend/access/transam/walprohibit.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/walprohibit.h"
+#include "postmaster/bgwriter.h"
+#include "storage/procsignal.h"
+
+/*
+ * ProcessBarrierWALProhibit()
+ *
+ * Handle WAL prohibit state change request.
+ */
+bool
+ProcessBarrierWALProhibit(void)
+{
+	/*
+	 * Kill off any transactions that have an XID *before* allowing the system
+	 * to go WAL prohibit state.
+	 */
+	if (FullTransactionIdIsValid(GetTopFullTransactionIdIfAny()))
+	{
+		Assert(GetWALProhibitState() & WALPROHIBIT_STATE_READ_ONLY);
+
+		/*
+		 * XXX: Kill off the whole session by throwing FATAL instead of killing
+		 * transaction by throwing ERROR due to following reasons that need be
+		 * thought:
+		 *
+		 * 1. Due to some presents challenges with the wire protocol, we could
+		 * not simply kill of idle transaction.
+		 *
+		 * 2. If we are here in subtransaction then the ERROR will kill the
+		 * current subtransaction only. In the case of invalidations, that
+		 * might be good enough, but for XID assignment it's not, because
+		 * assigning an XID to a subtransaction also causes higher
+		 * sub-transaction levels and the parent transaction to get XIDs.
+		 */
+		ereport(FATAL,
+				(errcode(ERRCODE_ACTIVE_SQL_TRANSACTION),
+				 errmsg("system is now read only"),
+				 errhint("Cannot continue a transaction if it has performed writes while system is read only.")));
+	}
+
+	/* Return to "check" state */
+	ResetLocalXLogInsertAllowed();
+
+	return true;
+}
+
+/*
+ * AlterSystemSetWALProhibitState
+ *
+ * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
+ */
+void
+AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
+{
+	uint32		state;
+
+	if (!superuser())
+		ereport(ERROR,
+				(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+				 errmsg("must be superuser to execute ALTER SYSTEM command")));
+
+	/* Alter WAL prohibit state not allowed during recovery */
+	PreventCommandDuringRecovery("ALTER SYSTEM");
+
+	/* Requested state */
+	state = stmt->WALProhibited ?
+		WALPROHIBIT_STATE_READ_ONLY : WALPROHIBIT_STATE_READ_WRITE;
+
+	/*
+	 * Since we yet to convey this WAL prohibit state to all backend mark it
+	 * in-progress.
+	 */
+	state |= WALPROHIBIT_TRANSITION_IN_PROGRESS;
+
+	if (!SetWALProhibitState(state))
+		return; /* server is already in the desired state */
+
+	/*
+	 * Signal the checkpointer to do the actual state transition, and wait for
+	 * the state change to occur.
+	 */
+	WALProhibitRequest();
+}
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index af6afcebb13..98a1943f717 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -1962,23 +1962,28 @@ StartTransaction(void)
 	Assert(s->prevSecContext == 0);
 
 	/*
-	 * Make sure we've reset xact state variables
+	 * Reset xact state variables.
 	 *
-	 * If recovery is still in progress, mark this transaction as read-only.
-	 * We have lower level defences in XLogInsert and elsewhere to stop us
-	 * from modifying data during recovery, but this gives the normal
-	 * indication to the user that the transaction is read-only.
-	 */
-	if (RecoveryInProgress())
-	{
-		s->startedInRecovery = true;
-		XactReadOnly = true;
-	}
-	else
-	{
-		s->startedInRecovery = false;
-		XactReadOnly = DefaultXactReadOnly;
-	}
+	 * If it is not currently possible to insert write-ahead log records,
+	 * either because we are still in recovery or because ALTER SYSTEM READ
+	 * ONLY has been executed, force this to be a read-only transaction.
+	 * We have lower level defences in XLogBeginInsert() and elsewhere to stop
+	 * us from modifying data during recovery when !XLogInsertAllowed(), but
+	 * this gives the normal indication to the user that the transaction is
+	 * read-only.
+	 *
+	 * On the other hand, we only need to set the startedInRecovery flag when
+	 * the transaction started during recovery, and not when WAL is otherwise
+	 * prohibited. This information is used by RelationGetIndexScan() to
+	 * decide whether to permit (1) relying on existing killed-tuple markings
+	 * and (2) further killing of index tuples. Even when WAL is prohibited
+	 * on the master, it's still the master, so the former is OK; and since
+	 * killing index tuples doesn't generate WAL, the latter is also OK.
+	 * See comments in RelationGetIndexScan() and MarkBufferDirtyHint().
+	 */
+	XactReadOnly = DefaultXactReadOnly || !XLogInsertAllowed();
+	s->startedInRecovery = RecoveryInProgress();
+
 	XactDeferrable = DefaultXactDeferrable;
 	XactIsoLevel = DefaultXactIsoLevel;
 	forceSyncCommit = false;
@@ -4903,9 +4908,11 @@ CommitSubTransaction(void)
 	/*
 	 * We need to restore the upper transaction's read-only state, in case the
 	 * upper is read-write while the child is read-only; GUC will incorrectly
-	 * think it should leave the child state in place.
+	 * think it should leave the child state in place.  Note that the upper
+	 * transaction will be a force to ready-only irrespective of its previous
+	 * status if the server state is WAL prohibited.
 	 */
-	XactReadOnly = s->prevXactReadOnly;
+	XactReadOnly = s->prevXactReadOnly || !XLogInsertAllowed();
 
 	CurrentResourceOwner = s->parent->curTransactionOwner;
 	CurTransactionResourceOwner = s->parent->curTransactionOwner;
@@ -5064,9 +5071,11 @@ AbortSubTransaction(void)
 	/*
 	 * Restore the upper transaction's read-only state, too.  This should be
 	 * redundant with GUC's cleanup but we may as well do it for consistency
-	 * with the commit case.
+	 * with the commit case.  Note that the upper transaction will be a force
+	 * to ready-only irrespective of its previous status if the server state is
+	 * WAL prohibited.
 	 */
-	XactReadOnly = s->prevXactReadOnly;
+	XactReadOnly = s->prevXactReadOnly || !XLogInsertAllowed();
 
 	RESUME_INTERRUPTS();
 }
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 09c01ed4ae4..2091eff0d53 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -31,6 +31,7 @@
 #include "access/timeline.h"
 #include "access/transam.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog_internal.h"
 #include "access/xlogarchive.h"
@@ -245,9 +246,10 @@ static bool LocalPromoteIsTriggered = false;
  *		0: unconditionally not allowed to insert XLOG
  *		-1: must check RecoveryInProgress(); disallow until it is false
  * Most processes start with -1 and transition to 1 after seeing that recovery
- * is not in progress.  But we can also force the value for special cases.
- * The coding in XLogInsertAllowed() depends on the first two of these states
- * being numerically the same as bool true and false.
+ * is not in progress or the server state is not a WAL prohibited state.  But
+ * we can also force the value for special cases.  The coding in
+ * XLogInsertAllowed() depends on the first two of these states being
+ * numerically the same as bool true and false.
  */
 static int	LocalXLogInsertAllowed = -1;
 
@@ -657,6 +659,12 @@ typedef struct XLogCtlData
 	 */
 	RecoveryState SharedRecoveryState;
 
+	/*
+	 * SharedWALProhibitState indicates current WAL prohibit state.
+	 * Protected by info_lck.
+	 */
+	uint32		SharedWALProhibitState;
+
 	/*
 	 * SharedHotStandbyActive indicates if we allow hot standby queries to be
 	 * run.  Protected by info_lck.
@@ -967,6 +975,7 @@ static void WALInsertLockAcquire(void);
 static void WALInsertLockAcquireExclusive(void);
 static void WALInsertLockRelease(void);
 static void WALInsertLockUpdateInsertingAt(XLogRecPtr insertingAt);
+static inline bool IsWALProhibited(void);
 
 /*
  * Insert an XLOG record represented by an already-constructed chain of data
@@ -7703,6 +7712,15 @@ StartupXLOG(void)
 	XLogCtl->LogwrtRqst.Write = EndOfLog;
 	XLogCtl->LogwrtRqst.Flush = EndOfLog;
 
+	/*
+	 * Before enabling WAL insertion update WAL prohibit state in shared memory
+	 * that will decide the further WAL insert should be allowed or not.
+	 */
+	SpinLockAcquire(&XLogCtl->info_lck);
+	XLogCtl->SharedWALProhibitState = ControlFile->wal_prohibited ?
+		WALPROHIBIT_STATE_READ_ONLY : WALPROHIBIT_STATE_READ_WRITE;
+	SpinLockRelease(&XLogCtl->info_lck);
+
 	/*
 	 * Update full_page_writes in shared memory and write an XLOG_FPW_CHANGE
 	 * record before resource manager writes cleanup WAL records or checkpoint
@@ -7713,7 +7731,15 @@ StartupXLOG(void)
 	UpdateFullPageWrites();
 	LocalXLogInsertAllowed = -1;
 
-	if (InRecovery)
+	/*
+	 * Skip end-of-recovery checkpoint if the system is in WAL prohibited state.
+	 */
+	if (ControlFile->wal_prohibited && InRecovery)
+	{
+		ereport(LOG,
+				(errmsg("skipping startup checkpoint because the system is read only")));
+	}
+	else if (InRecovery)
 	{
 		/*
 		 * Perform a checkpoint to update all our recovery activity to disk.
@@ -7959,6 +7985,83 @@ StartupXLOG(void)
 		RequestCheckpoint(CHECKPOINT_FORCE);
 }
 
+/* Atomically return the current server WAL prohibited state */
+uint32
+GetWALProhibitState(void)
+{
+	uint32		state;
+
+	SpinLockAcquire(&XLogCtl->info_lck);
+	state = XLogCtl->SharedWALProhibitState;
+	SpinLockRelease(&XLogCtl->info_lck);
+
+	return state;
+}
+
+/*
+ * SetWALProhibitState: Change current wal prohibit state to the input state.
+ *
+ * If the server is already completely moved to the requested WAL prohibit
+ * state, or if the desired state is same as the current state, return false,
+ * indicating that the server state did not change. Else return true.
+ */
+bool
+SetWALProhibitState(uint32 new_state)
+{
+	uint32		cur_state;
+
+	cur_state = GetWALProhibitState();
+
+	/* Server is already in requested state */
+	if (new_state == cur_state ||
+		new_state == (cur_state | WALPROHIBIT_TRANSITION_IN_PROGRESS))
+		return false;
+
+	/* Prevent concurrent contrary in progress transition state setting */
+	if ((new_state & WALPROHIBIT_TRANSITION_IN_PROGRESS) &&
+		(cur_state & WALPROHIBIT_TRANSITION_IN_PROGRESS))
+	{
+		if (cur_state & WALPROHIBIT_STATE_READ_ONLY)
+			ereport(ERROR,
+					(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+					 errmsg("system state transition to read only is already in progress"),
+					 errhint("Try after sometime again.")));
+		else
+			ereport(ERROR,
+					(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+					 errmsg("system state transition to read write is already in progress"),
+					 errhint("Try after sometime again.")));
+
+	}
+
+	/* Update new state in share memory */
+	SpinLockAcquire(&XLogCtl->info_lck);
+	XLogCtl->SharedWALProhibitState = new_state;
+	SpinLockRelease(&XLogCtl->info_lck);
+
+	/* Update control file if it is the final state */
+	if (!(new_state & WALPROHIBIT_TRANSITION_IN_PROGRESS))
+	{
+		bool	wal_prohibited = (new_state & WALPROHIBIT_STATE_READ_ONLY) != 0;
+
+		LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+		ControlFile->wal_prohibited = wal_prohibited;
+		UpdateControlFile();
+		LWLockRelease(ControlFileLock);
+	}
+
+	return true;
+}
+
+/*
+ * Is the system still in WAL prohibited state?
+ */
+static inline bool
+IsWALProhibited(void)
+{
+	return (GetWALProhibitState() & WALPROHIBIT_STATE_READ_ONLY) != 0;
+}
+
 /*
  * Checks if recovery has reached a consistent state. When consistency is
  * reached and we have a valid starting standby snapshot, tell postmaster
@@ -8174,9 +8277,9 @@ HotStandbyActiveInReplay(void)
 /*
  * Is this process allowed to insert new WAL records?
  *
- * Ordinarily this is essentially equivalent to !RecoveryInProgress().
- * But we also have provisions for forcing the result "true" or "false"
- * within specific processes regardless of the global state.
+ * Ordinarily this is essentially equivalent to !RecoveryInProgress() and
+ * !IsWALProhibited().  But we also have provisions for forcing the result
+ * "true" or "false" within specific processes regardless of the global state.
  */
 bool
 XLogInsertAllowed(void)
@@ -8190,14 +8293,25 @@ XLogInsertAllowed(void)
 		return (bool) LocalXLogInsertAllowed;
 
 	/*
-	 * Else, must check to see if we're still in recovery.
+	 * Else, must check to see if we're still in recovery
 	 */
 	if (RecoveryInProgress())
 		return false;
 
+	/* Or, in WAL prohibited state */
+	if (IsWALProhibited())
+	{
+		/*
+		 * Set it to "unconditionally false" to avoid checking until it gets
+		 * reset.
+		 */
+		LocalXLogInsertAllowed = 0;
+		return false;
+	}
+
 	/*
-	 * On exit from recovery, reset to "unconditionally true", since there is
-	 * no need to keep checking.
+	 * On exit from recovery or WAL prohibited state, reset to "unconditionally
+	 * true", since there is no need to keep checking.
 	 */
 	LocalXLogInsertAllowed = 1;
 	return true;
@@ -8213,12 +8327,19 @@ static void
 LocalSetXLogInsertAllowed(void)
 {
 	Assert(LocalXLogInsertAllowed == -1);
+
 	LocalXLogInsertAllowed = 1;
 
 	/* Initialize as RecoveryInProgress() would do when switching state */
 	InitXLOGAccess();
 }
 
+void
+ResetLocalXLogInsertAllowed(void)
+{
+	LocalXLogInsertAllowed = -1;
+}
+
 /*
  * Subroutine to try to fetch and validate a prior checkpoint record.
  *
@@ -8510,7 +8631,10 @@ ShutdownXLOG(int code, Datum arg)
 
 	if (RecoveryInProgress())
 		CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
-	else
+	/*
+	 * Can't perform checkpoint or xlog rotation without writing WAL.
+	 */
+	else if (XLogInsertAllowed())
 	{
 		/*
 		 * If archiving is enabled, rotate the last XLOG file so that all the
@@ -8523,6 +8647,10 @@ ShutdownXLOG(int code, Datum arg)
 
 		CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
 	}
+	else
+		ereport(LOG,
+			   (errmsg("skipping shutdown checkpoint because the system is read only")));
+
 	ShutdownCLOG();
 	ShutdownCommitTs();
 	ShutdownSUBTRANS();
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index c6ec657a936..93273d1e6b4 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -652,6 +652,10 @@ AutoVacLauncherMain(int argc, char *argv[])
 
 		HandleAutoVacLauncherInterrupts();
 
+		/* If the server is read only just go back to sleep. */
+		if (!XLogInsertAllowed())
+			continue;
+
 		/*
 		 * a worker finished, or postmaster signaled failure to start a worker
 		 */
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index 069e27e427f..6c6ff7dc3af 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -268,7 +268,7 @@ BackgroundWriterMain(void)
 		 * Checkpointer, when active, is barely ever in its mainloop and thus
 		 * makes it hard to log regularly.
 		 */
-		if (XLogStandbyInfoActive() && !RecoveryInProgress())
+		if (XLogStandbyInfoActive() && XLogInsertAllowed())
 		{
 			TimestampTz timeout = 0;
 			TimestampTz now = GetCurrentTimestamp();
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b80..3e8aa9a0ec3 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -39,6 +39,7 @@
 #include <sys/time.h>
 #include <time.h>
 
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xlog_internal.h"
 #include "libpq/pqsignal.h"
@@ -127,6 +128,9 @@ typedef struct
 	ConditionVariable start_cv; /* signaled when ckpt_started advances */
 	ConditionVariable done_cv;	/* signaled when ckpt_done advances */
 
+	ConditionVariable walprohibit_cv; /* signaled when requested wal
+										 prohibit state changes */
+
 	uint32		num_backend_writes; /* counts user backend buffer writes */
 	uint32		num_backend_fsync;	/* counts user backend fsync calls */
 
@@ -168,6 +172,7 @@ static bool IsCheckpointOnSchedule(double progress);
 static bool ImmediateCheckpointRequested(void);
 static bool CompactCheckpointerRequestQueue(void);
 static void UpdateSharedMemoryConfig(void);
+static void performWALProhibitStateChange(uint32 wal_state);
 
 /* Signal handlers */
 static void ReqCheckpointHandler(SIGNAL_ARGS);
@@ -332,6 +337,7 @@ CheckpointerMain(void)
 		pg_time_t	now;
 		int			elapsed_secs;
 		int			cur_timeout;
+		uint32		wal_state;
 
 		/* Clear any already-pending wakeups */
 		ResetLatch(MyLatch);
@@ -342,6 +348,28 @@ CheckpointerMain(void)
 		AbsorbSyncRequests();
 		HandleCheckpointerInterrupts();
 
+		wal_state = GetWALProhibitState();
+
+		if (wal_state & WALPROHIBIT_TRANSITION_IN_PROGRESS)
+		{
+			/* Complete WAL prohibit state change request */
+			performWALProhibitStateChange(wal_state);
+			continue;
+		}
+		else if (wal_state & WALPROHIBIT_STATE_READ_ONLY)
+		{
+			/*
+			 * Don't do anything until someone wakes us up.  For example a
+			 * backend might later on request us to put the system back to
+			 * read-write wal prohibit sate.
+			 */
+			(void) WaitLatch(MyLatch, WL_LATCH_SET | WL_EXIT_ON_PM_DEATH, -1,
+							 WAIT_EVENT_CHECKPOINTER_MAIN);
+			continue;
+		}
+
+		Assert(wal_state == WALPROHIBIT_STATE_READ_WRITE);
+
 		/*
 		 * Detect a pending checkpoint request by checking whether the flags
 		 * word in shared memory is nonzero.  We shouldn't need to acquire the
@@ -879,6 +907,7 @@ CheckpointerShmemInit(void)
 		CheckpointerShmem->max_requests = NBuffers;
 		ConditionVariableInit(&CheckpointerShmem->start_cv);
 		ConditionVariableInit(&CheckpointerShmem->done_cv);
+		ConditionVariableInit(&CheckpointerShmem->walprohibit_cv);
 	}
 }
 
@@ -1109,6 +1138,94 @@ ForwardSyncRequest(const FileTag *ftag, SyncRequestType type)
 	return true;
 }
 
+/*
+ * WALProhibitedRequest: Request checkpointer to make the WALProhibitState to
+ * read-only.
+ */
+void
+WALProhibitRequest(void)
+{
+	/* Must not be called from checkpointer */
+	Assert(!AmCheckpointerProcess());
+	Assert(GetWALProhibitState() & WALPROHIBIT_TRANSITION_IN_PROGRESS);
+
+	/*
+	 * If in a standalone backend, just do it ourselves.
+	 */
+	if (!IsPostmasterEnvironment)
+	{
+		performWALProhibitStateChange(GetWALProhibitState());
+		return;
+	}
+
+	if (CheckpointerShmem->checkpointer_pid == 0)
+		elog(ERROR, "checkpointer is not running");
+
+	if (kill(CheckpointerShmem->checkpointer_pid, SIGINT) != 0)
+		elog(ERROR, "could not signal checkpointer: %m");
+
+	/* Wait for the state to change to read-only */
+	ConditionVariablePrepareToSleep(&CheckpointerShmem->walprohibit_cv);
+	for (;;)
+	{
+		/*  We'll be done once in-progress flag bit is cleared */
+		if (!(GetWALProhibitState() & WALPROHIBIT_TRANSITION_IN_PROGRESS))
+			break;
+
+		elog(DEBUG1, "WALProhibitRequest: Waiting for checkpointer");
+		ConditionVariableSleep(&CheckpointerShmem->walprohibit_cv,
+							   WAIT_EVENT_SYSTEM_WALPROHIBIT_STATE_CHANGE);
+	}
+	ConditionVariableCancelSleep();
+	elog(DEBUG1, "Done WALProhibitRequest");
+}
+
+/*
+ * performWALProhibitStateChange: checkpointer will call this to complete
+ * the requested WAL prohibit state transition.
+ */
+static void
+performWALProhibitStateChange(uint32 wal_state)
+{
+	uint64		barrierGeneration;
+
+	/*
+	 * Must be called from checkpointer. Otherwise, it must be single-user
+	 * backend.
+	 */
+	Assert(AmCheckpointerProcess() || !IsPostmasterEnvironment);
+	Assert(wal_state & WALPROHIBIT_TRANSITION_IN_PROGRESS);
+
+	/*
+	 * WAL prohibit state change is initiated. We need to complete the state
+	 * transition by setting requested WAL prohibit state in all backends.
+	 */
+	elog(DEBUG1, "waiting for backends to adopt requested WAL prohibit state");
+
+	/* Emit global barrier */
+	barrierGeneration = EmitProcSignalBarrier(PROCSIGNAL_BARRIER_WALPROHIBIT);
+	WaitForProcSignalBarrier(barrierGeneration);
+
+	/* And flush all writes. */
+	XLogFlush(GetXLogWriteRecPtr());
+
+	/* Set final state by clearing in-progress flag bit */
+	if (SetWALProhibitState(wal_state & ~(WALPROHIBIT_TRANSITION_IN_PROGRESS)))
+	{
+		if ((wal_state & WALPROHIBIT_STATE_READ_ONLY) != 0)
+			ereport(LOG, (errmsg("system is now read only")));
+		else
+		{
+			/* Request checkpoint */
+			RequestCheckpoint(CHECKPOINT_IMMEDIATE);
+			ereport(LOG, (errmsg("system is now read write")));
+		}
+	}
+
+	/* Wake up the backend who requested the state change */
+	ConditionVariableBroadcast(&CheckpointerShmem->walprohibit_cv);
+}
+
 /*
  * CompactCheckpointerRequestQueue
  *		Remove duplicates from the request queue to avoid backend fsyncs.
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 73ce944fb1c..437da6ac473 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -4057,6 +4057,9 @@ pgstat_get_wait_io(WaitEventIO w)
 		case WAIT_EVENT_REPLICATION_SLOT_WRITE:
 			event_name = "ReplicationSlotWrite";
 			break;
+		case WAIT_EVENT_SYSTEM_WALPROHIBIT_STATE_CHANGE:
+			event_name = "SystemWALProhibitStateChange";
+			break;
 		case WAIT_EVENT_SLRU_FLUSH_SYNC:
 			event_name = "SLRUFlushSync";
 			break;
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 13648887187..b973727a580 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -18,6 +18,7 @@
 #include <unistd.h>
 
 #include "access/parallel.h"
+#include "access/walprohibit.h"
 #include "commands/async.h"
 #include "miscadmin.h"
 #include "pgstat.h"
@@ -96,7 +97,6 @@ static volatile ProcSignalSlot *MyProcSignalSlot = NULL;
 
 static bool CheckProcSignal(ProcSignalReason reason);
 static void CleanupProcSignalState(int status, Datum arg);
-static bool ProcessBarrierPlaceholder(void);
 
 /*
  * ProcSignalShmemSize
@@ -510,9 +510,9 @@ ProcessProcSignalBarrier(void)
 			 * unconditionally, but it's more efficient to call only the ones
 			 * that might need us to do something based on the flags.
 			 */
-			if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_PLACEHOLDER)
-				&& ProcessBarrierPlaceholder())
-				BARRIER_CLEAR_BIT(flags, PROCSIGNAL_BARRIER_PLACEHOLDER);
+			if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_WALPROHIBIT)
+				&& ProcessBarrierWALProhibit())
+				BARRIER_CLEAR_BIT(flags, PROCSIGNAL_BARRIER_WALPROHIBIT);
 		}
 		PG_CATCH();
 		{
@@ -554,24 +554,6 @@ ProcessProcSignalBarrier(void)
 	pg_atomic_write_u64(&MyProcSignalSlot->pss_barrierGeneration, shared_gen);
 }
 
-static bool
-ProcessBarrierPlaceholder(void)
-{
-	/*
-	 * XXX. This is just a placeholder until the first real user of this
-	 * machinery gets committed. Rename PROCSIGNAL_BARRIER_PLACEHOLDER to
-	 * PROCSIGNAL_BARRIER_SOMETHING_ELSE where SOMETHING_ELSE is something
-	 * appropriately descriptive. Get rid of this function and instead have
-	 * ProcessBarrierSomethingElse. Most likely, that function should live in
-	 * the file pertaining to that subsystem, rather than here.
-	 *
-	 * The return value should be 'true' if the barrier was successfully
-	 * absorbed and 'false' if not. Note that returning 'false' can lead to
-	 * very frequent retries, so try hard to make that an uncommon case.
-	 */
-	return true;
-}
-
 /*
  * CheckProcSignal - check to see if a particular reason has been
  * signaled, and clear the signal flag.  Should be called after receiving
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 7af96c77082..d6411e4f3e9 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -19,6 +19,7 @@
 #include "access/htup_details.h"
 #include "access/reloptions.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "catalog/catalog.h"
@@ -85,7 +86,6 @@ static void ProcessUtilitySlow(ParseState *pstate,
 							   DestReceiver *dest,
 							   QueryCompletion *qc);
 static void ExecDropStmt(DropStmt *stmt, bool isTopLevel);
-static void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
 
 /*
  * CommandIsReadOnly: is an executable query read-only?
@@ -3644,15 +3644,3 @@ GetCommandLogLevel(Node *parsetree)
 
 	return lev;
 }
-
-/*
- * AlterSystemSetWALProhibitState
- *
- * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
- */
-static void
-AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
-{
-	/* some code */
-	elog(INFO, "AlterSystemSetWALProhibitState() called");
-}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index de87ad6ef70..dfc44136867 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -225,6 +225,7 @@ static bool check_recovery_target_lsn(char **newval, void **extra, GucSource sou
 static void assign_recovery_target_lsn(const char *newval, void *extra);
 static bool check_primary_slot_name(char **newval, void **extra, GucSource source);
 static bool check_default_with_oids(bool *newval, void **extra, GucSource source);
+static const char *show_system_is_read_only(void);
 
 /* Private functions in guc-file.l that need to be called from guc.c */
 static ConfigVariable *ProcessConfigFileInternal(GucContext context,
@@ -615,6 +616,7 @@ static char *recovery_target_string;
 static char *recovery_target_xid_string;
 static char *recovery_target_name_string;
 static char *recovery_target_lsn_string;
+static bool system_is_read_only;
 
 
 /* should be static, but commands/variable.c needs to get at this */
@@ -2036,6 +2038,18 @@ static struct config_bool ConfigureNamesBool[] =
 		NULL, NULL, NULL
 	},
 
+	{
+		/* Not for general use */
+		{"system_is_read_only", PGC_INTERNAL, WAL,
+			gettext_noop("Shows whether the system is read only."),
+			NULL,
+			GUC_NO_RESET_ALL | GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE
+		},
+		&system_is_read_only,
+		false,
+		NULL, NULL, show_system_is_read_only
+	},
+
 	/* End-of-list marker */
 	{
 		{NULL, 0, 0, NULL, NULL}, NULL, false, NULL, NULL, NULL
@@ -12041,4 +12055,16 @@ check_default_with_oids(bool *newval, void **extra, GucSource source)
 	return true;
 }
 
+/*
+ * NB: The return string should be the same as the _ShowOption() for boolean
+ * type.
+ */
+ static const char *
+ show_system_is_read_only(void)
+{
+	if (!XLogInsertAllowed())
+		return "on";
+	return "off";
+}
+
 #include "guc-file.c"
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index 3e00ac0f701..922cd9641d8 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -290,6 +290,8 @@ main(int argc, char *argv[])
 		   (uint32) ControlFile->backupEndPoint);
 	printf(_("End-of-backup record required:        %s\n"),
 		   ControlFile->backupEndRequired ? _("yes") : _("no"));
+	printf(_("WAL write prohibited:                 %s\n"),
+		   ControlFile->wal_prohibited ? _("yes") : _("no"));
 	printf(_("wal_level setting:                    %s\n"),
 		   wal_level_str(ControlFile->wal_level));
 	printf(_("wal_log_hints setting:                %s\n"),
diff --git a/src/include/access/walprohibit.h b/src/include/access/walprohibit.h
new file mode 100644
index 00000000000..163fe0d2fce
--- /dev/null
+++ b/src/include/access/walprohibit.h
@@ -0,0 +1,32 @@
+/*
+ * walprohibit.h
+ *
+ * PostgreSQL write-ahead log prohibit states
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/include/access/walprohibit.h
+ */
+#ifndef WALPROHIBIT_H
+#define WALPROHIBIT_H
+
+#include "access/xact.h"
+#include "access/xlog.h"
+#include "miscadmin.h"
+#include "nodes/parsenodes.h"
+
+extern bool ProcessBarrierWALProhibit(void);
+extern void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
+
+/* WAL Prohibit States */
+#define	WALPROHIBIT_STATE_READ_WRITE		0x0000
+#define	WALPROHIBIT_STATE_READ_ONLY			0x0001
+
+/*
+ * The bit is used in state transition from one state to another.  When this
+ * bit is set then the state indicated by the 0th position bit is yet to
+ * confirmed.
+ */
+#define WALPROHIBIT_TRANSITION_IN_PROGRESS	0x0002
+
+#endif		/* WALPROHIBIT_H */
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 221af87e715..183b2fa5a14 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -306,6 +306,7 @@ extern RecoveryState GetRecoveryState(void);
 extern bool HotStandbyActive(void);
 extern bool HotStandbyActiveInReplay(void);
 extern bool XLogInsertAllowed(void);
+extern void ResetLocalXLogInsertAllowed(void);
 extern void GetXLogReceiptTime(TimestampTz *rtime, bool *fromStream);
 extern XLogRecPtr GetXLogReplayRecPtr(TimeLineID *replayTLI);
 extern XLogRecPtr GetXLogInsertRecPtr(void);
@@ -325,6 +326,8 @@ extern void XLOGShmemInit(void);
 extern void BootStrapXLOG(void);
 extern void LocalProcessControlFile(bool reset);
 extern void StartupXLOG(void);
+extern uint32 GetWALProhibitState(void);
+extern bool SetWALProhibitState(uint32 new_state);
 extern void ShutdownXLOG(int code, Datum arg);
 extern void InitXLOGAccess(void);
 extern void CreateCheckPoint(int flags);
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index 06bed90c5e9..f4dc5412ee6 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -182,6 +182,9 @@ typedef struct ControlFileData
 	int			max_locks_per_xact;
 	bool		track_commit_timestamp;
 
+	/* WAL prohibited determines if the WAL insert is allowed or not. */
+	bool		wal_prohibited;
+
 	/*
 	 * This data is used to check for hardware-architecture compatibility of
 	 * the database and the backend executable.  We need not check endianness
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 13872013823..780c59f3e48 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -955,6 +955,7 @@ typedef enum
 	WAIT_EVENT_REPLICATION_SLOT_RESTORE_SYNC,
 	WAIT_EVENT_REPLICATION_SLOT_SYNC,
 	WAIT_EVENT_REPLICATION_SLOT_WRITE,
+	WAIT_EVENT_SYSTEM_WALPROHIBIT_STATE_CHANGE,
 	WAIT_EVENT_SLRU_FLUSH_SYNC,
 	WAIT_EVENT_SLRU_READ,
 	WAIT_EVENT_SLRU_SYNC,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e6..e8271b49f6d 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -35,6 +35,8 @@ extern void CheckpointWriteDelay(int flags, double progress);
 
 extern bool ForwardSyncRequest(const FileTag *ftag, SyncRequestType type);
 
+extern void WALProhibitRequest(void);
+
 extern void AbsorbSyncRequests(void);
 
 extern Size CheckpointerShmemSize(void);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f38..bae06202b4a 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -48,12 +48,7 @@ typedef enum
 
 typedef enum
 {
-	/*
-	 * XXX. PROCSIGNAL_BARRIER_PLACEHOLDER should be replaced when the first
-	 * real user of the ProcSignalBarrier mechanism is added. It's just here
-	 * for now because we can't have an empty enum.
-	 */
-	PROCSIGNAL_BARRIER_PLACEHOLDER = 0
+	PROCSIGNAL_BARRIER_WALPROHIBIT = 0
 } ProcSignalBarrierType;
 
 /*
-- 
2.22.0

From df3bdd8d8f5c410e02dcf1f6d60317501e71cb8a Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 19 Jun 2020 06:27:53 -0400
Subject: [PATCH v5 1/5] Allow error or refusal while absorbing barriers.

Patch by Robert Haas
---
 src/backend/storage/ipc/procsignal.c | 75 +++++++++++++++++++++++-----
 1 file changed, 63 insertions(+), 12 deletions(-)

diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ece..13648887187 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -87,12 +87,16 @@ typedef struct
 #define BARRIER_SHOULD_CHECK(flags, type) \
 	(((flags) & (((uint32) 1) << (uint32) (type))) != 0)
 
+/* Clear the relevant type bit from the flags. */
+#define BARRIER_CLEAR_BIT(flags, type) \
+	((flags) &= ~(((uint32) 1) << (uint32) (type)))
+
 static ProcSignalHeader *ProcSignal = NULL;
 static volatile ProcSignalSlot *MyProcSignalSlot = NULL;
 
 static bool CheckProcSignal(ProcSignalReason reason);
 static void CleanupProcSignalState(int status, Datum arg);
-static void ProcessBarrierPlaceholder(void);
+static bool ProcessBarrierPlaceholder(void);
 
 /*
  * ProcSignalShmemSize
@@ -486,17 +490,59 @@ ProcessProcSignalBarrier(void)
 	flags = pg_atomic_exchange_u32(&MyProcSignalSlot->pss_barrierCheckMask, 0);
 
 	/*
-	 * Process each type of barrier. It's important that nothing we call from
-	 * here throws an error, because pss_barrierCheckMask has already been
-	 * cleared. If we jumped out of here before processing all barrier types,
-	 * then we'd forget about the need to do so later.
-	 *
-	 * NB: It ought to be OK to call the barrier-processing functions
-	 * unconditionally, but it's more efficient to call only the ones that
-	 * might need us to do something based on the flags.
+	 * If there are no flags set, then we can skip doing any real work.
+	 * Otherwise, establish a PG_TRY block, so that we don't lose track of
+	 * which types of barrier processing are needed if an ERROR occurs.
 	 */
-	if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_PLACEHOLDER))
-		ProcessBarrierPlaceholder();
+	if (flags != 0)
+	{
+		PG_TRY();
+		{
+			/*
+			 * Process each type of barrier. The barrier-processing functions
+			 * should normally return true, but may return false if the barrier
+			 * can't be absorbed at the current time. This should be rare,
+			 * because it's pretty expensive.  Every single
+			 * CHECK_FOR_INTERRUPTS() will return here until we manage to
+			 * absorb the barrier, and that cost will add up in a hurry.
+			 *
+			 * NB: It ought to be OK to call the barrier-processing functions
+			 * unconditionally, but it's more efficient to call only the ones
+			 * that might need us to do something based on the flags.
+			 */
+			if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_PLACEHOLDER)
+				&& ProcessBarrierPlaceholder())
+				BARRIER_CLEAR_BIT(flags, PROCSIGNAL_BARRIER_PLACEHOLDER);
+		}
+		PG_CATCH();
+		{
+			/*
+			 * If an ERROR occurred, add any flags that weren't yet handled
+			 * back into pss_barrierCheckMask, and reset the global variables
+			 * so that we try again the next time we check for interrupts.
+			 */
+			pg_atomic_fetch_or_u32(&MyProcSignalSlot->pss_barrierCheckMask,
+								   flags);
+			ProcSignalBarrierPending = true;
+			InterruptPending = true;
+
+			PG_RE_THROW();
+		}
+		PG_END_TRY();
+
+		/*
+		 * If some barrier was not successfully absorbed, we will have to try
+		 * again later.
+		 */
+		if (flags != 0)
+		{
+			pg_atomic_fetch_or_u32(&MyProcSignalSlot->pss_barrierCheckMask,
+								   flags);
+			ProcSignalBarrierPending = true;
+			InterruptPending = true;
+			return;
+		}
+	}
 
 	/*
 	 * State changes related to all types of barriers that might have been
@@ -508,7 +554,7 @@ ProcessProcSignalBarrier(void)
 	pg_atomic_write_u64(&MyProcSignalSlot->pss_barrierGeneration, shared_gen);
 }
 
-static void
+static bool
 ProcessBarrierPlaceholder(void)
 {
 	/*
@@ -518,7 +564,12 @@ ProcessBarrierPlaceholder(void)
 	 * appropriately descriptive. Get rid of this function and instead have
 	 * ProcessBarrierSomethingElse. Most likely, that function should live in
 	 * the file pertaining to that subsystem, rather than here.
+	 *
+	 * The return value should be 'true' if the barrier was successfully
+	 * absorbed and 'false' if not. Note that returning 'false' can lead to
+	 * very frequent retries, so try hard to make that an uncommon case.
 	 */
+	return true;
 }
 
 /*
-- 
2.22.0

From 6d6fc8f7470ce4b91d56536a6fa14fb761556aed Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Mon, 27 Jul 2020 02:13:36 -0400
Subject: [PATCH v5 4/5] Error or Assert before START_CRIT_SECTION for WAL
 write

Based on the following criteria added the Assert or the Error when system is
prohibited:

 - Added ERROR for the function which can be reachable without valid XID in case
   of VACUUM or CONCURRENT CREATE INDEX.  For that added common static inline
   function CheckWALPermitted().
 - Added ASSERT for the function which cannot be reached without valid XID;
   Assert to ensure XID validation. For that added AssertWALPermitted_HaveXID().

To enforce the rule to have aforesaid assert or error check before entering a
critical section for WAL write, a new walpermit_checked_state assert only flag
is added. If these check is missing then XLogBeginInsert() will have an
assertion if it is in critical section.

If we are not doing WAL insert inside the critical section then the above
checking is not necessary, we can rely on XLogBeginInsert() for that check &
report an error.
---
 src/backend/access/brin/brin.c            |  4 ++
 src/backend/access/brin/brin_pageops.c    | 21 ++++++++--
 src/backend/access/brin/brin_revmap.c     | 10 ++++-
 src/backend/access/gin/ginbtree.c         | 15 +++++--
 src/backend/access/gin/gindatapage.c      | 18 ++++++--
 src/backend/access/gin/ginfast.c          | 11 ++++-
 src/backend/access/gin/gininsert.c        |  4 ++
 src/backend/access/gin/ginutil.c          |  9 +++-
 src/backend/access/gin/ginvacuum.c        | 11 ++++-
 src/backend/access/gist/gist.c            | 25 +++++++++---
 src/backend/access/gist/gistvacuum.c      | 13 +++++-
 src/backend/access/hash/hash.c            | 19 +++++++--
 src/backend/access/hash/hashinsert.c      |  9 +++-
 src/backend/access/hash/hashovfl.c        | 22 ++++++++--
 src/backend/access/hash/hashpage.c        |  9 ++++
 src/backend/access/heap/heapam.c          | 26 +++++++++++-
 src/backend/access/heap/pruneheap.c       | 12 ++++--
 src/backend/access/heap/vacuumlazy.c      | 18 ++++++--
 src/backend/access/heap/visibilitymap.c   | 22 +++++++++-
 src/backend/access/nbtree/nbtdedup.c      |  3 ++
 src/backend/access/nbtree/nbtinsert.c     | 10 ++++-
 src/backend/access/nbtree/nbtpage.c       | 39 +++++++++++++++---
 src/backend/access/spgist/spgdoinsert.c   | 13 ++++++
 src/backend/access/spgist/spgvacuum.c     | 22 ++++++++--
 src/backend/access/transam/multixact.c    |  5 ++-
 src/backend/access/transam/twophase.c     |  9 ++++
 src/backend/access/transam/varsup.c       |  4 ++
 src/backend/access/transam/walprohibit.c  | 10 +++++
 src/backend/access/transam/xact.c         |  6 +++
 src/backend/access/transam/xlog.c         | 27 ++++++++----
 src/backend/access/transam/xloginsert.c   | 13 +++++-
 src/backend/commands/sequence.c           | 16 ++++++++
 src/backend/commands/variable.c           |  9 ++--
 src/backend/postmaster/checkpointer.c     |  4 ++
 src/backend/storage/buffer/bufmgr.c       | 10 +++--
 src/backend/storage/freespace/freespace.c | 10 ++++-
 src/backend/storage/lmgr/lock.c           |  6 +--
 src/backend/utils/cache/relmapper.c       |  3 ++
 src/include/access/walprohibit.h          | 50 ++++++++++++++++++++++-
 src/include/miscadmin.h                   | 27 ++++++++++++
 40 files changed, 503 insertions(+), 71 deletions(-)

diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index 1f72562c603..47142193706 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -24,6 +24,7 @@
 #include "access/relscan.h"
 #include "access/table.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/index.h"
 #include "catalog/pg_am.h"
@@ -759,6 +760,9 @@ brinbuildempty(Relation index)
 		ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermittedHaveXID();
+
 	/* Initialize and xlog metabuffer. */
 	START_CRIT_SECTION();
 	brin_metapage_init(BufferGetPage(metabuf), BrinGetPagesPerRange(index),
diff --git a/src/backend/access/brin/brin_pageops.c b/src/backend/access/brin/brin_pageops.c
index 87de0b855b5..8b377a679ab 100644
--- a/src/backend/access/brin/brin_pageops.c
+++ b/src/backend/access/brin/brin_pageops.c
@@ -14,6 +14,7 @@
 #include "access/brin_pageops.h"
 #include "access/brin_revmap.h"
 #include "access/brin_xlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -65,6 +66,7 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 	Buffer		newbuf;
 	BlockNumber newblk = InvalidBlockNumber;
 	bool		extended;
+	bool		needwal = RelationNeedsWAL(idxrel);
 
 	Assert(newsz == MAXALIGN(newsz));
 
@@ -176,13 +178,16 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 	if (((BrinPageFlags(oldpage) & BRIN_EVACUATE_PAGE) == 0) &&
 		brin_can_do_samepage_update(oldbuf, origsz, newsz))
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 		if (!PageIndexTupleOverwrite(oldpage, oldoff, (Item) unconstify(BrinTuple *, newtup), newsz))
 			elog(ERROR, "failed to replace BRIN tuple");
 		MarkBufferDirty(oldbuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(idxrel))
+		if (needwal)
 		{
 			xl_brin_samepage_update xlrec;
 			XLogRecPtr	recptr;
@@ -240,6 +245,9 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 
 		revmapbuf = brinLockRevmapPageForUpdate(revmap, heapBlk);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		/*
@@ -267,7 +275,7 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 		MarkBufferDirty(revmapbuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(idxrel))
+		if (needwal)
 		{
 			xl_brin_update xlrec;
 			XLogRecPtr	recptr;
@@ -351,6 +359,7 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	Buffer		revmapbuf;
 	ItemPointerData tid;
 	bool		extended;
+	bool		needwal;
 
 	Assert(itemsz == MAXALIGN(itemsz));
 
@@ -405,6 +414,10 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	page = BufferGetPage(*buffer);
 	blk = BufferGetBlockNumber(*buffer);
 
+	needwal = RelationNeedsWAL(idxrel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Execute the actual insertion */
 	START_CRIT_SECTION();
 	if (extended)
@@ -424,7 +437,7 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	MarkBufferDirty(revmapbuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(idxrel))
+	if (needwal)
 	{
 		xl_brin_insert xlrec;
 		XLogRecPtr	recptr;
@@ -881,6 +894,8 @@ brin_initialize_empty_new_buffer(Relation idxrel, Buffer buffer)
 			   "brin_initialize_empty_new_buffer: initializing blank page %u",
 			   BufferGetBlockNumber(buffer)));
 
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 	page = BufferGetPage(buffer);
 	brin_page_init(page, BRIN_PAGETYPE_REGULAR);
diff --git a/src/backend/access/brin/brin_revmap.c b/src/backend/access/brin/brin_revmap.c
index 35746714a7c..fd766da445d 100644
--- a/src/backend/access/brin/brin_revmap.c
+++ b/src/backend/access/brin/brin_revmap.c
@@ -26,6 +26,7 @@
 #include "access/brin_tuple.h"
 #include "access/brin_xlog.h"
 #include "access/rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -340,6 +341,7 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 	OffsetNumber revmapOffset;
 	OffsetNumber regOffset;
 	ItemId		lp;
+	bool		needwal;
 
 	revmap = brinRevmapInitialize(idxrel, &pagesPerRange, NULL);
 
@@ -404,6 +406,10 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 	 * crashed or aborted summarization; remove them silently.
 	 */
 
+	needwal = RelationNeedsWAL(idxrel);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	ItemPointerSetInvalid(&invalidIptr);
@@ -415,7 +421,7 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 	MarkBufferDirty(regBuf);
 	MarkBufferDirty(revmapBuf);
 
-	if (RelationNeedsWAL(idxrel))
+	if (needwal)
 	{
 		xl_brin_desummarize xlrec;
 		XLogRecPtr	recptr;
@@ -613,6 +619,8 @@ revmap_physical_extend(BrinRevmap *revmap)
 		return;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Ok, we have now locked the metapage and the target block. Re-initialize
 	 * the target block as a revmap page, and update the metapage.
diff --git a/src/backend/access/gin/ginbtree.c b/src/backend/access/gin/ginbtree.c
index 8d08b05f515..0d9997463b4 100644
--- a/src/backend/access/gin/ginbtree.c
+++ b/src/backend/access/gin/ginbtree.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/predicate.h"
@@ -333,6 +334,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 {
 	Page		page = BufferGetPage(stack->buffer);
 	bool		result;
+	bool		needwal;
 	GinPlaceToPageRC rc;
 	uint16		xlflags = 0;
 	Page		childpage = NULL;
@@ -378,6 +380,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 								 insertdata, updateblkno,
 								 &ptp_workspace,
 								 &newlpage, &newrpage);
+	needwal = RelationNeedsWAL(btree->index) && !btree->isBuild;
 
 	if (rc == GPTP_NO_WORK)
 	{
@@ -386,10 +389,13 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 	}
 	else if (rc == GPTP_INSERT)
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		/* It will fit, perform the insertion */
 		START_CRIT_SECTION();
 
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogBeginInsert();
 			XLogRegisterBuffer(0, stack->buffer, REGBUF_STANDARD);
@@ -410,7 +416,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 			MarkBufferDirty(childbuf);
 		}
 
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 			ginxlogInsert xlrec;
@@ -548,6 +554,9 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 			}
 		}
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/*
 		 * OK, we have the new contents of the left page in a temporary copy
 		 * now (newlpage), and likewise for the new contents of the
@@ -588,7 +597,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 		}
 
 		/* write WAL record */
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gin/gindatapage.c b/src/backend/access/gin/gindatapage.c
index 7a2690e97f2..0abc5990100 100644
--- a/src/backend/access/gin/gindatapage.c
+++ b/src/backend/access/gin/gindatapage.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "lib/ilist.h"
 #include "miscadmin.h"
@@ -811,6 +812,7 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 	if (removedsomething)
 	{
 		bool		modified;
+		bool		needwal;
 
 		/*
 		 * Make sure we have a palloc'd copy of all segments, after the first
@@ -835,8 +837,12 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 			}
 		}
 
-		if (RelationNeedsWAL(indexrel))
+		needwal = RelationNeedsWAL(indexrel);
+		if (needwal)
+		{
+			CheckWALPermitted();
 			computeLeafRecompressWALData(leaf);
+		}
 
 		/* Apply changes to page */
 		START_CRIT_SECTION();
@@ -845,7 +851,7 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 
 		MarkBufferDirty(buffer);
 
-		if (RelationNeedsWAL(indexrel))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
@@ -1777,6 +1783,7 @@ createPostingTree(Relation index, ItemPointerData *items, uint32 nitems,
 	int			nrootitems;
 	int			rootsize;
 	bool		is_build = (buildStats != NULL);
+	bool		needwal;
 
 	/* Construct the new root page in memory first. */
 	tmppage = (Page) palloc(BLCKSZ);
@@ -1825,12 +1832,17 @@ createPostingTree(Relation index, ItemPointerData *items, uint32 nitems,
 	 */
 	PredicateLockPageSplit(index, BufferGetBlockNumber(entrybuffer), blkno);
 
+	needwal = RelationNeedsWAL(index) && !is_build;
+
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	PageRestoreTempPage(tmppage, page);
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index) && !is_build)
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogCreatePostingTree data;
diff --git a/src/backend/access/gin/ginfast.c b/src/backend/access/gin/ginfast.c
index 2e41b34d8d5..b8c2a993408 100644
--- a/src/backend/access/gin/ginfast.c
+++ b/src/backend/access/gin/ginfast.c
@@ -20,6 +20,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
 #include "catalog/pg_am.h"
@@ -68,6 +69,8 @@ writeListPage(Relation index, Buffer buffer,
 	PGAlignedBlock workspace;
 	char	   *ptr;
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	GinInitBuffer(buffer, GIN_LIST);
@@ -548,6 +551,7 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 	Page		metapage;
 	GinMetaPageData *metadata;
 	BlockNumber blknoToDelete;
+	bool		needwal = RelationNeedsWAL(index);
 
 	metapage = BufferGetPage(metabuffer);
 	metadata = GinPageGetMeta(metapage);
@@ -586,8 +590,11 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 		 * prepare the XLogInsert machinery for that before entering the
 		 * critical section.
 		 */
-		if (RelationNeedsWAL(index))
+		if (needwal)
+		{
+			CheckWALPermitted();
 			XLogEnsureRecordSpace(data.ndeleted, 0);
+		}
 
 		START_CRIT_SECTION();
 
@@ -625,7 +632,7 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 			MarkBufferDirty(buffers[i]);
 		}
 
-		if (RelationNeedsWAL(index))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gin/gininsert.c b/src/backend/access/gin/gininsert.c
index 77433dc8a41..989d82ffcaf 100644
--- a/src/backend/access/gin/gininsert.c
+++ b/src/backend/access/gin/gininsert.c
@@ -17,6 +17,7 @@
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/index.h"
 #include "miscadmin.h"
@@ -447,6 +448,9 @@ ginbuildempty(Relation index)
 		ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(RootBuffer, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermittedHaveXID();
+
 	/* Initialize and xlog metabuffer and root buffer. */
 	START_CRIT_SECTION();
 	GinInitMetabuffer(MetaBuffer);
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index ef9b56fd363..b48ea1a746a 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -17,6 +17,7 @@
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
 #include "access/reloptions.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/pg_collation.h"
 #include "catalog/pg_type.h"
@@ -659,12 +660,18 @@ ginUpdateStats(Relation index, const GinStatsData *stats, bool is_build)
 	Buffer		metabuffer;
 	Page		metapage;
 	GinMetaPageData *metadata;
+	bool		needwal;
 
 	metabuffer = ReadBuffer(index, GIN_METAPAGE_BLKNO);
 	LockBuffer(metabuffer, GIN_EXCLUSIVE);
 	metapage = BufferGetPage(metabuffer);
 	metadata = GinPageGetMeta(metapage);
 
+	needwal = RelationNeedsWAL(index) && !is_build;
+
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	metadata->nTotalPages = stats->nTotalPages;
@@ -684,7 +691,7 @@ ginUpdateStats(Relation index, const GinStatsData *stats, bool is_build)
 
 	MarkBufferDirty(metabuffer);
 
-	if (RelationNeedsWAL(index) && !is_build)
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogUpdateMeta data;
diff --git a/src/backend/access/gin/ginvacuum.c b/src/backend/access/gin/ginvacuum.c
index 9cd6638df62..7c4cbee627a 100644
--- a/src/backend/access/gin/ginvacuum.c
+++ b/src/backend/access/gin/ginvacuum.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "commands/vacuum.h"
 #include "miscadmin.h"
@@ -136,6 +137,7 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	Page		page,
 				parentPage;
 	BlockNumber rightlink;
+	bool		needwal;
 
 	/*
 	 * This function MUST be called only if someone of parent pages hold
@@ -159,6 +161,10 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	 */
 	PredicateLockPageCombine(gvs->index, deleteBlkno, rightlink);
 
+	needwal = RelationNeedsWAL(gvs->index);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* Unlink the page by changing left sibling's rightlink */
@@ -195,7 +201,7 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	MarkBufferDirty(lBuffer);
 	MarkBufferDirty(dBuffer);
 
-	if (RelationNeedsWAL(gvs->index))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogDeletePage data;
@@ -650,6 +656,9 @@ ginbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 
 		if (resPage)
 		{
+			if (RelationNeedsWAL(gvs.index))
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 			PageRestoreTempPage(resPage, page);
 			MarkBufferDirty(buffer);
diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c
index 25b42e38f22..4a870a062ba 100644
--- a/src/backend/access/gist/gist.c
+++ b/src/backend/access/gist/gist.c
@@ -16,6 +16,7 @@
 
 #include "access/gist_private.h"
 #include "access/gistscan.h"
+#include "access/walprohibit.h"
 #include "catalog/pg_collation.h"
 #include "commands/vacuum.h"
 #include "miscadmin.h"
@@ -135,6 +136,9 @@ gistbuildempty(Relation index)
 	buffer = ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermittedHaveXID();
+
 	/* Initialize and xlog buffer */
 	START_CRIT_SECTION();
 	GISTInitBuffer(buffer, F_LEAF);
@@ -234,6 +238,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 	XLogRecPtr	recptr;
 	int			i;
 	bool		is_split;
+	bool		needwal = RelationNeedsWAL(rel);
 
 	/*
 	 * Refuse to modify a page that's incompletely split. This should not
@@ -465,9 +470,12 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 		 * insertion for that. NB: The number of pages and data segments
 		 * specified here must match the calculations in gistXLogSplit()!
 		 */
-		if (!is_build && RelationNeedsWAL(rel))
+		if (!is_build && needwal)
 			XLogEnsureRecordSpace(npage, 1 + npage * 2);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		/*
@@ -500,7 +508,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 			recptr = GistBuildLSN;
 		else
 		{
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 				recptr = gistXLogSplit(is_leaf,
 									   dist, oldrlink, oldnsn, leftchildbuf,
 									   markfollowright);
@@ -526,6 +534,9 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 	}
 	else
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		/*
 		 * Enough space.  We always get here if ntup==0.
 		 */
@@ -567,7 +578,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 			recptr = GistBuildLSN;
 		else
 		{
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 			{
 				OffsetNumber ndeloffs = 0,
 							deloffs[1];
@@ -1641,6 +1652,7 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 	OffsetNumber offnum,
 				maxoff;
 	TransactionId latestRemovedXid = InvalidTransactionId;
+	bool		needwal = RelationNeedsWAL(rel);
 
 	Assert(GistPageIsLeaf(page));
 
@@ -1659,13 +1671,16 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 			deletable[ndeletable++] = offnum;
 	}
 
-	if (XLogStandbyInfoActive() && RelationNeedsWAL(rel))
+	if (XLogStandbyInfoActive() && needwal)
 		latestRemovedXid =
 			index_compute_xid_horizon_for_tuples(rel, heapRel, buffer,
 												 deletable, ndeletable);
 
 	if (ndeletable > 0)
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		PageIndexMultiDelete(page, deletable, ndeletable);
@@ -1682,7 +1697,7 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 		MarkBufferDirty(buffer);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gist/gistvacuum.c b/src/backend/access/gist/gistvacuum.c
index a9c616c7724..bbb3ebb19ad 100644
--- a/src/backend/access/gist/gistvacuum.c
+++ b/src/backend/access/gist/gistvacuum.c
@@ -17,6 +17,7 @@
 #include "access/genam.h"
 #include "access/gist_private.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "commands/vacuum.h"
 #include "lib/integerset.h"
 #include "miscadmin.h"
@@ -260,6 +261,7 @@ gistvacuumpage(GistVacState *vstate, BlockNumber blkno, BlockNumber orig_blkno)
 	Buffer		buffer;
 	Page		page;
 	BlockNumber recurse_to;
+	bool		needwal = RelationNeedsWAL(rel);
 
 restart:
 	recurse_to = InvalidBlockNumber;
@@ -341,6 +343,9 @@ restart:
 		 */
 		if (ntodelete > 0)
 		{
+			if (needwal)
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 
 			MarkBufferDirty(buffer);
@@ -348,7 +353,7 @@ restart:
 			PageIndexMultiDelete(page, todelete, ntodelete);
 			GistMarkTuplesDeleted(page);
 
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 			{
 				XLogRecPtr	recptr;
 
@@ -580,6 +585,7 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	IndexTuple	idxtuple;
 	XLogRecPtr	recptr;
 	FullTransactionId txid;
+	bool		needwal = RelationNeedsWAL(info->index);
 
 	/*
 	 * Check that the leaf is still empty and deletable.
@@ -634,6 +640,9 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	 */
 	txid = ReadNextFullTransactionId();
 
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* mark the page as deleted */
@@ -645,7 +654,7 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	MarkBufferDirty(parentBuffer);
 	PageIndexTupleDelete(parentPage, downlink);
 
-	if (RelationNeedsWAL(info->index))
+	if (needwal)
 		recptr = gistXLogPageDelete(leafBuffer, txid, parentBuffer, downlink);
 	else
 		recptr = gistGetFakeLSN(info->index);
diff --git a/src/backend/access/hash/hash.c b/src/backend/access/hash/hash.c
index 7c9ccf446c8..f4903a43bb5 100644
--- a/src/backend/access/hash/hash.c
+++ b/src/backend/access/hash/hash.c
@@ -22,6 +22,7 @@
 #include "access/hash_xlog.h"
 #include "access/relscan.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "catalog/index.h"
 #include "commands/progress.h"
 #include "commands/vacuum.h"
@@ -467,6 +468,7 @@ hashbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	Buffer		metabuf = InvalidBuffer;
 	HashMetaPage metap;
 	HashMetaPage cachedmetap;
+	bool		needwal;
 
 	tuples_removed = 0;
 	num_index_tuples = 0;
@@ -573,6 +575,10 @@ loop_top:
 		goto loop_top;
 	}
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Okay, we're really done.  Update tuple count in metapage. */
 	START_CRIT_SECTION();
 
@@ -603,7 +609,7 @@ loop_top:
 	MarkBufferDirty(metabuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_hash_update_meta_page xlrec;
 		XLogRecPtr	recptr;
@@ -690,6 +696,7 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 	Buffer		buf;
 	Bucket		new_bucket PG_USED_FOR_ASSERTS_ONLY = InvalidBucket;
 	bool		bucket_dirty = false;
+	bool		needwal = RelationNeedsWAL(rel);
 
 	blkno = bucket_blkno;
 	buf = bucket_buf;
@@ -788,6 +795,9 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		 */
 		if (ndeletable > 0)
 		{
+			if (needwal)
+				CheckWALPermitted();
+
 			/* No ereport(ERROR) until changes are logged */
 			START_CRIT_SECTION();
 
@@ -809,7 +819,7 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 			MarkBufferDirty(buf);
 
 			/* XLOG stuff */
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 			{
 				xl_hash_delete xlrec;
 				XLogRecPtr	recptr;
@@ -883,6 +893,9 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		page = BufferGetPage(bucket_buf);
 		bucket_opaque = (HashPageOpaque) PageGetSpecialPointer(page);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -890,7 +903,7 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		MarkBufferDirty(bucket_buf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c
index 2ebe671967b..2eab69efa91 100644
--- a/src/backend/access/hash/hashinsert.c
+++ b/src/backend/access/hash/hashinsert.c
@@ -17,6 +17,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "storage/buf_internals.h"
 #include "storage/lwlock.h"
@@ -193,6 +194,8 @@ restart_insert:
 	 */
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	AssertWALPermittedHaveXID();
+
 	/* Do the update.  No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -360,6 +363,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 	if (ndeletable > 0)
 	{
 		TransactionId latestRemovedXid;
+		bool		needwal = RelationNeedsWAL(rel);
 
 		latestRemovedXid =
 			index_compute_xid_horizon_for_tuples(rel, hrel, buf,
@@ -370,6 +374,9 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 		 */
 		LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -393,7 +400,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 		MarkBufferDirty(metabuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			xl_hash_vacuum_one_page xlrec;
 			XLogRecPtr	recptr;
diff --git a/src/backend/access/hash/hashovfl.c b/src/backend/access/hash/hashovfl.c
index 00f0a940116..e7c5dd3e3ce 100644
--- a/src/backend/access/hash/hashovfl.c
+++ b/src/backend/access/hash/hashovfl.c
@@ -19,6 +19,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "utils/rel.h"
 
@@ -312,6 +313,8 @@ _hash_addovflpage(Relation rel, Buffer metabuf, Buffer buf, bool retain_pin)
 
 found:
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Do the update.  No ereport(ERROR) until changes are logged. We want to
 	 * log the changes for bitmap page and overflow page together to avoid
@@ -510,6 +513,7 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	Buffer		prevbuf = InvalidBuffer;
 	Buffer		nextbuf = InvalidBuffer;
 	bool		update_metap = false;
+	bool		needwal;
 
 	/* Get information from the doomed page */
 	_hash_checkpage(rel, ovflbuf, LH_OVERFLOW_PAGE);
@@ -573,10 +577,15 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	/* Get write-lock on metapage to update firstfree */
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	needwal = RelationNeedsWAL(rel);
+
 	/* This operation needs to log multiple tuples, prepare WAL for that */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 		XLogEnsureRecordSpace(HASH_XLOG_FREE_OVFL_BUFS, 4 + nitups);
 
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -641,7 +650,7 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	}
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_hash_squeeze_page xlrec;
 		XLogRecPtr	recptr;
@@ -922,14 +931,19 @@ readpage:
 
 				if (nitups > 0)
 				{
+					bool		needwal = RelationNeedsWAL(rel);
+
 					Assert(nitups == ndeletable);
 
 					/*
 					 * This operation needs to log multiple tuples, prepare
 					 * WAL for that.
 					 */
-					if (RelationNeedsWAL(rel))
+					if (needwal)
+					{
+						CheckWALPermitted();
 						XLogEnsureRecordSpace(0, 3 + nitups);
+					}
 
 					START_CRIT_SECTION();
 
@@ -947,7 +961,7 @@ readpage:
 					MarkBufferDirty(rbuf);
 
 					/* XLOG stuff */
-					if (RelationNeedsWAL(rel))
+					if (needwal)
 					{
 						XLogRecPtr	recptr;
 						xl_hash_move_page_contents xlrec;
diff --git a/src/backend/access/hash/hashpage.c b/src/backend/access/hash/hashpage.c
index a664ecf494a..55a867dd375 100644
--- a/src/backend/access/hash/hashpage.c
+++ b/src/backend/access/hash/hashpage.c
@@ -30,6 +30,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "port/pg_bitutils.h"
 #include "storage/lmgr.h"
@@ -816,6 +817,8 @@ restart_expand:
 		goto fail;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Since we are scribbling on the pages in the shared buffers, establish a
 	 * critical section.  Any failure in this next code leaves us with a big
@@ -1172,6 +1175,8 @@ _hash_splitbucket(Relation rel,
 
 				if (PageGetFreeSpaceForMultipleTuples(npage, nitups + 1) < (all_tups_size + itemsz))
 				{
+					AssertWALPermittedHaveXID();
+
 					/*
 					 * Change the shared buffer state in critical section,
 					 * otherwise any error could make it unrecoverable.
@@ -1223,6 +1228,8 @@ _hash_splitbucket(Relation rel,
 		/* Exit loop if no more overflow pages in old bucket */
 		if (!BlockNumberIsValid(oblkno))
 		{
+			AssertWALPermittedHaveXID();
+
 			/*
 			 * Change the shared buffer state in critical section, otherwise
 			 * any error could make it unrecoverable.
@@ -1269,6 +1276,8 @@ _hash_splitbucket(Relation rel,
 	npage = BufferGetPage(bucket_nbuf);
 	nopaque = (HashPageOpaque) PageGetSpecialPointer(npage);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	oopaque->hasho_flag &= ~LH_BUCKET_BEING_SPLIT;
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 8eb276e4644..eb784bbf766 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -47,6 +47,7 @@
 #include "access/transam.h"
 #include "access/valid.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1898,6 +1899,8 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
 	 */
 	CheckForSerializableConflictIn(relation, NULL, InvalidBlockNumber);
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -2174,6 +2177,8 @@ heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples,
 										   &vmbuffer, NULL);
 		page = BufferGetPage(buffer);
 
+		AssertWALPermittedHaveXID();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -2692,6 +2697,8 @@ l1:
 							  xid, LockTupleExclusive, true,
 							  &new_xmax, &new_infomask, &new_infomask2);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -3444,6 +3451,8 @@ l2:
 
 		Assert(HEAP_XMAX_IS_LOCKED_ONLY(infomask_lock_old_tuple));
 
+		AssertWALPermittedHaveXID();
+
 		START_CRIT_SECTION();
 
 		/* Clear obsolete visibility flags ... */
@@ -3617,6 +3626,8 @@ l2:
 										   bms_overlap(modified_attrs, id_attrs),
 										   &old_key_copied);
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -4550,6 +4561,8 @@ failed:
 							  GetCurrentTransactionId(), mode, false,
 							  &xid, &new_infomask, &new_infomask2);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -5341,6 +5354,8 @@ l4:
 								VISIBILITYMAP_ALL_FROZEN))
 			cleared_all_frozen = true;
 
+		AssertWALPermittedHaveXID();
+
 		START_CRIT_SECTION();
 
 		/* ... and set them */
@@ -5499,6 +5514,8 @@ heap_finish_speculative(Relation relation, ItemPointer tid)
 	StaticAssertStmt(MaxOffsetNumber < SpecTokenOffsetNumber,
 					 "invalid speculative token constant");
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -5607,6 +5624,8 @@ heap_abort_speculative(Relation relation, ItemPointer tid)
 		elog(ERROR, "attempted to kill a non-speculative tuple");
 	Assert(!HeapTupleHeaderIsHeapOnly(tp.t_data));
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * No need to check for serializable conflicts here.  There is never a
 	 * need for a combocid, either.  No need to extract replica identity, or
@@ -5723,6 +5742,7 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	HeapTupleHeader htup;
 	uint32		oldlen;
 	uint32		newlen;
+	bool		needwal;
 
 	/*
 	 * For now, parallel operations are required to be strictly read-only.
@@ -5753,6 +5773,10 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	if (oldlen != newlen || htup->t_hoff != tuple->t_data->t_hoff)
 		elog(ERROR, "wrong tuple length");
 
+	needwal = RelationNeedsWAL(relation);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -5763,7 +5787,7 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	MarkBufferDirty(buffer);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(relation))
+	if (needwal)
 	{
 		xl_heap_inplace xlrec;
 		XLogRecPtr	recptr;
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index 3ad4222cb8a..a0afbe76c2b 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -18,6 +18,7 @@
 #include "access/heapam_xlog.h"
 #include "access/htup_details.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "catalog/catalog.h"
 #include "miscadmin.h"
@@ -94,11 +95,11 @@ heap_page_prune_opt(Relation relation, Buffer buffer)
 	Size		minfree;
 
 	/*
-	 * We can't write WAL in recovery mode, so there's no point trying to
+	 * We can't write WAL during read-only mode, so there's no point trying to
 	 * clean the page. The primary will likely issue a cleaning WAL record soon
 	 * anyway, so this is no particular loss.
 	 */
-	if (RecoveryInProgress())
+	if (!XLogInsertAllowed())
 		return;
 
 	/*
@@ -228,6 +229,7 @@ heap_page_prune(Relation relation, Buffer buffer,
 	OffsetNumber offnum,
 				maxoff;
 	PruneState	prstate;
+	bool		needwal;
 
 	/*
 	 * Our strategy is to scan the page and make lists of items to change,
@@ -271,6 +273,10 @@ heap_page_prune(Relation relation, Buffer buffer,
 		ndeleted += heap_prune_chain(buffer, offnum, &prstate);
 	}
 
+	needwal = RelationNeedsWAL(relation);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Any error while applying the changes is critical */
 	START_CRIT_SECTION();
 
@@ -304,7 +310,7 @@ heap_page_prune(Relation relation, Buffer buffer,
 		/*
 		 * Emit a WAL XLOG_HEAP2_CLEAN record showing what we did
 		 */
-		if (RelationNeedsWAL(relation))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 03c8e1ff7ea..d1117608160 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -59,6 +59,7 @@
 #include "access/parallel.h"
 #include "access/transam.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "catalog/storage.h"
@@ -764,6 +765,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 	BlockNumber nblocks,
 				blkno;
 	HeapTupleData tuple;
+	bool		needwal = RelationNeedsWAL(onerel);
 	TransactionId relfrozenxid = onerel->rd_rel->relfrozenxid;
 	TransactionId relminmxid = onerel->rd_rel->relminmxid;
 	BlockNumber empty_pages,
@@ -1206,6 +1208,9 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 			 */
 			if (!PageIsAllVisible(page))
 			{
+				if (needwal)
+					CheckWALPermitted();
+
 				START_CRIT_SECTION();
 
 				/* mark buffer dirty before writing a WAL record */
@@ -1221,7 +1226,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 				 * page has been previously WAL-logged, and if not, do that
 				 * now.
 				 */
-				if (RelationNeedsWAL(onerel) &&
+				if (needwal &&
 					PageGetLSN(page) == InvalidXLogRecPtr)
 					log_newpage_buffer(buf, true);
 
@@ -1482,6 +1487,9 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 		 */
 		if (nfrozen > 0)
 		{
+			if (needwal)
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 
 			MarkBufferDirty(buf);
@@ -1499,7 +1507,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 			}
 
 			/* Now WAL-log freezing if necessary */
-			if (RelationNeedsWAL(onerel))
+			if (needwal)
 			{
 				XLogRecPtr	recptr;
 
@@ -1923,6 +1931,7 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	TransactionId visibility_cutoff_xid;
 	bool		all_frozen;
 	LVSavedErrInfo saved_err_info;
+	bool		needwal = RelationNeedsWAL(onerel);
 
 	pgstat_progress_update_param(PROGRESS_VACUUM_HEAP_BLKS_VACUUMED, blkno);
 
@@ -1930,6 +1939,9 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	update_vacuum_error_info(vacrelstats, &saved_err_info, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
 							 blkno);
 
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	for (; tupindex < dead_tuples->num_tuples; tupindex++)
@@ -1955,7 +1967,7 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	MarkBufferDirty(buffer);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(onerel))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index b1072183bcd..44244363968 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -88,6 +88,7 @@
 
 #include "access/heapam_xlog.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "miscadmin.h"
 #include "port/pg_bitutils.h"
@@ -249,6 +250,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 	uint8		mapOffset = HEAPBLK_TO_OFFSET(heapBlk);
 	Page		page;
 	uint8	   *map;
+	bool		needwal = RelationNeedsWAL(rel);
 
 #ifdef TRACE_VISIBILITYMAP
 	elog(DEBUG1, "vm_set %s %d", RelationGetRelationName(rel), heapBlk);
@@ -270,6 +272,16 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 	map = (uint8 *) PageGetContents(page);
 	LockBuffer(vmBuf, BUFFER_LOCK_EXCLUSIVE);
 
+	/*
+	 * Can reach here from VACUUM or from startup process, so need not have an
+	 * XID.
+	 *
+	 * Recovery in the startup process never have wal prohibit state, skip
+	 * permission check if reach here in the startup process.
+	 */
+	if (needwal)
+		InRecovery ? AssertWALPermitted() : CheckWALPermitted();
+
 	if (flags != (map[mapByte] >> mapOffset & VISIBILITYMAP_VALID_BITS))
 	{
 		START_CRIT_SECTION();
@@ -277,7 +289,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 		map[mapByte] |= (flags << mapOffset);
 		MarkBufferDirty(vmBuf);
 
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			if (XLogRecPtrIsInvalid(recptr))
 			{
@@ -476,6 +488,7 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		Buffer		mapBuffer;
 		Page		page;
 		char	   *map;
+		bool		needwal;
 
 		newnblocks = truncBlock + 1;
 
@@ -489,8 +502,13 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		page = BufferGetPage(mapBuffer);
 		map = PageGetContents(page);
 
+		needwal = (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded());
+
 		LockBuffer(mapBuffer, BUFFER_LOCK_EXCLUSIVE);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -518,7 +536,7 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		 * during recovery.
 		 */
 		MarkBufferDirty(mapBuffer);
-		if (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded())
+		if (needwal)
 			log_newpage_buffer(mapBuffer, false);
 
 		END_CRIT_SECTION();
diff --git a/src/backend/access/nbtree/nbtdedup.c b/src/backend/access/nbtree/nbtdedup.c
index f6be865b17e..b519a1268e8 100644
--- a/src/backend/access/nbtree/nbtdedup.c
+++ b/src/backend/access/nbtree/nbtdedup.c
@@ -16,6 +16,7 @@
 
 #include "access/nbtree.h"
 #include "access/nbtxlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "utils/rel.h"
 
@@ -271,6 +272,8 @@ _bt_dedup_one_page(Relation rel, Buffer buf, Relation heapRel,
 		nopaque->btpo_flags &= ~BTP_HAS_GARBAGE;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	PageRestoreTempPage(newpage, page);
diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c
index d36f7557c87..2c3d8aaecbd 100644
--- a/src/backend/access/nbtree/nbtinsert.c
+++ b/src/backend/access/nbtree/nbtinsert.c
@@ -19,6 +19,7 @@
 #include "access/nbtxlog.h"
 #include "access/tableam.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/lmgr.h"
@@ -1246,6 +1247,8 @@ _bt_insertonpg(Relation rel,
 			}
 		}
 
+		AssertWALPermittedHaveXID();
+
 		/* Do the update.  No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -1898,13 +1901,16 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf,
 			ropaque->btpo_flags |= BTP_SPLIT_END;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Right sibling is locked, new siblings are prepared, but original page
 	 * is not updated yet.
 	 *
 	 * NO EREPORT(ERROR) till right sibling is updated.  We can get away with
 	 * not starting the critical section till here because we haven't been
-	 * scribbling on the original page yet; see comments above.
+	 * scribbling on the original page yet; see the comments above for grabbing
+	 * the right sibling.
 	 */
 	START_CRIT_SECTION();
 
@@ -2467,6 +2473,8 @@ _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf)
 	right_item = CopyIndexTuple(item);
 	BTreeTupleSetDownLink(right_item, rbkno);
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till newroot op is logged */
 	START_CRIT_SECTION();
 
diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c
index 7f392480ac0..8c3fc251a29 100644
--- a/src/backend/access/nbtree/nbtpage.c
+++ b/src/backend/access/nbtree/nbtpage.c
@@ -26,6 +26,7 @@
 #include "access/nbtxlog.h"
 #include "access/tableam.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
@@ -179,6 +180,7 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	BTMetaPageData *metad;
 	bool		needsRewrite = false;
 	XLogRecPtr	recptr;
+	bool		needwal;
 
 	/* read the metapage and check if it needs rewrite */
 	metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ);
@@ -202,6 +204,10 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	_bt_unlockbuf(rel, metabuf);
 	_bt_lockbuf(rel, metabuf, BT_WRITE);
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* upgrade meta-page if needed */
@@ -214,7 +220,7 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	MarkBufferDirty(metabuf);
 
 	/* write wal record if needed */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_btree_metadata md;
 
@@ -332,6 +338,7 @@ _bt_getroot(Relation rel, int access)
 	if (metad->btm_root == P_NONE)
 	{
 		Page		metapg;
+		bool		needwal;
 
 		/* If access = BT_READ, caller doesn't want us to create root yet */
 		if (access == BT_READ)
@@ -377,6 +384,10 @@ _bt_getroot(Relation rel, int access)
 		/* Get raw page pointer for metapage */
 		metapg = BufferGetPage(metabuf);
 
+		needwal = RelationNeedsWAL(rel);
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO ELOG(ERROR) till meta is updated */
 		START_CRIT_SECTION();
 
@@ -395,7 +406,7 @@ _bt_getroot(Relation rel, int access)
 		MarkBufferDirty(metabuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			xl_btree_newroot xlrec;
 			XLogRecPtr	recptr;
@@ -1131,6 +1142,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 	char	   *updatedbuf = NULL;
 	Size		updatedbuflen = 0;
 	OffsetNumber updatedoffsets[MaxIndexTuplesPerPage];
+	bool		needwal = RelationNeedsWAL(rel);
 
 	/* Shouldn't be called unless there's something to do */
 	Assert(ndeletable > 0 || nupdatable > 0);
@@ -1145,7 +1157,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 	}
 
 	/* XLOG stuff -- allocate and fill buffer before critical section */
-	if (nupdatable > 0 && RelationNeedsWAL(rel))
+	if (nupdatable > 0 && needwal)
 	{
 		Size		offset = 0;
 
@@ -1175,6 +1187,9 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 		}
 	}
 
+	if (needwal)
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -1235,7 +1250,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 	MarkBufferDirty(buf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		xl_btree_vacuum xlrec_vacuum;
@@ -1302,6 +1317,8 @@ _bt_delitems_delete(Relation rel, Buffer buf,
 		latestRemovedXid =
 			_bt_xid_horizon(rel, heapRel, page, deletable, ndeletable);
 
+	AssertWALPermittedHaveXID();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -1832,6 +1849,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	OffsetNumber nextoffset;
 	IndexTuple	itup;
 	IndexTupleData trunctuple;
+	bool		needwal;
 
 	page = BufferGetPage(leafbuf);
 	opaque = (BTPageOpaque) PageGetSpecialPointer(page);
@@ -1920,6 +1938,10 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	 */
 	PredicateLockPageCombine(rel, leafblkno, leafrightsib);
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -1971,7 +1993,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	MarkBufferDirty(leafbuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_btree_mark_page_halfdead xlrec;
 		XLogRecPtr	recptr;
@@ -2064,6 +2086,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 	int			targetlevel;
 	IndexTuple	leafhikey;
 	BlockNumber nextchild;
+	bool		needwal;
 
 	page = BufferGetPage(leafbuf);
 	opaque = (BTPageOpaque) PageGetSpecialPointer(page);
@@ -2277,6 +2300,10 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 	 * Here we begin doing the deletion.
 	 */
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -2356,7 +2383,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 		MarkBufferDirty(leafbuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_btree_unlink_page xlrec;
 		xl_btree_metadata xlmeta;
diff --git a/src/backend/access/spgist/spgdoinsert.c b/src/backend/access/spgist/spgdoinsert.c
index 934d65b89f2..3c5a15c5d32 100644
--- a/src/backend/access/spgist/spgdoinsert.c
+++ b/src/backend/access/spgist/spgdoinsert.c
@@ -18,6 +18,7 @@
 #include "access/genam.h"
 #include "access/spgist_private.h"
 #include "access/spgxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -214,6 +215,8 @@ addLeafTuple(Relation index, SpGistState *state, SpGistLeafTuple leafTuple,
 	xlrec.offnumParent = InvalidOffsetNumber;
 	xlrec.nodeI = 0;
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	if (current->offnum == InvalidOffsetNumber ||
@@ -458,6 +461,8 @@ moveLeafs(Relation index, SpGistState *state,
 
 	leafdata = leafptr = palloc(size);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/* copy all the old tuples to new page, unless they're dead */
@@ -1109,6 +1114,8 @@ doPickSplit(Relation index, SpGistState *state,
 
 	leafdata = leafptr = (char *) palloc(totalLeafSizes);
 
+	AssertWALPermittedHaveXID();
+
 	/* Here we begin making the changes to the target pages */
 	START_CRIT_SECTION();
 
@@ -1517,6 +1524,8 @@ spgAddNodeAction(Relation index, SpGistState *state,
 	if (PageGetExactFreeSpace(current->page) >=
 		newInnerTuple->size - innerTuple->size)
 	{
+		AssertWALPermittedHaveXID();
+
 		/*
 		 * We can replace the inner tuple by new version in-place
 		 */
@@ -1603,6 +1612,8 @@ spgAddNodeAction(Relation index, SpGistState *state,
 		else
 			xlrec.parentBlk = 2;
 
+		AssertWALPermittedHaveXID();
+
 		START_CRIT_SECTION();
 
 		/* insert new ... */
@@ -1788,6 +1799,8 @@ spgSplitNodeAction(Relation index, SpGistState *state,
 									&xlrec.newPage);
 	}
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c
index e1c58933f97..3308832b85b 100644
--- a/src/backend/access/spgist/spgvacuum.c
+++ b/src/backend/access/spgist/spgvacuum.c
@@ -19,6 +19,7 @@
 #include "access/spgist_private.h"
 #include "access/spgxlog.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/storage_xlog.h"
 #include "commands/vacuum.h"
@@ -139,6 +140,7 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 	int			nDeletable;
 	OffsetNumber i,
 				max = PageGetMaxOffsetNumber(page);
+	bool		needwal;
 
 	memset(predecessor, 0, sizeof(predecessor));
 	memset(deletable, 0, sizeof(deletable));
@@ -323,6 +325,10 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 	if (nDeletable != xlrec.nDead + xlrec.nPlaceholder + xlrec.nMove)
 		elog(ERROR, "inconsistent counts of deletable tuples");
 
+	needwal = RelationNeedsWAL(index);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Do the updates */
 	START_CRIT_SECTION();
 
@@ -371,7 +377,7 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 
@@ -411,6 +417,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 	OffsetNumber toDelete[MaxIndexTuplesPerPage];
 	OffsetNumber i,
 				max = PageGetMaxOffsetNumber(page);
+	bool		needwal;
 
 	xlrec.nDelete = 0;
 
@@ -447,6 +454,10 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 	if (xlrec.nDelete == 0)
 		return;					/* nothing more to do */
 
+	needwal = RelationNeedsWAL(index);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Do the update */
 	START_CRIT_SECTION();
 
@@ -455,7 +466,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 
@@ -502,6 +513,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer)
 	OffsetNumber itemnos[MaxIndexTuplesPerPage];
 	spgxlogVacuumRedirect xlrec;
 	GlobalVisState *vistest;
+	bool		needwal;
 
 	xlrec.nToPlaceholder = 0;
 	xlrec.newestRedirectXid = InvalidTransactionId;
@@ -509,6 +521,10 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer)
 	/* XXX: providing heap relation would allow more pruning */
 	vistest = GlobalVisTestFor(NULL);
 
+	needwal = RelationNeedsWAL(index);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -584,7 +600,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer)
 	if (hasUpdate)
 		MarkBufferDirty(buffer);
 
-	if (hasUpdate && RelationNeedsWAL(index))
+	if (hasUpdate && needwal)
 	{
 		XLogRecPtr	recptr;
 
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index b8bedca04a4..0a88740764f 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -73,6 +73,7 @@
 #include "access/transam.h"
 #include "access/twophase.h"
 #include "access/twophase_rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1143,6 +1144,8 @@ GetNewMultiXactId(int nmembers, MultiXactOffset *offset)
 
 	ExtendMultiXactMember(nextOffset, nmembers);
 
+	CheckWALPermitted();
+
 	/*
 	 * Critical section from here until caller has written the data into the
 	 * just-reserved SLRU space; we don't want to error out with a partly
@@ -2938,7 +2941,7 @@ TruncateMultiXact(MultiXactId newOldestMulti, Oid newOldestMultiDB)
 	mxtruncinfo trunc;
 	MultiXactId earliest;
 
-	Assert(!RecoveryInProgress());
+	Assert(XLogInsertAllowed());
 	Assert(MultiXactState->finishedStartup);
 
 	/*
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index ef4f9981e35..ff2bc8cc74b 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -82,6 +82,7 @@
 #include "access/transam.h"
 #include "access/twophase.h"
 #include "access/twophase_rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1106,6 +1107,8 @@ EndPrepare(GlobalTransaction gxact)
 	 */
 	XLogEnsureRecordSpace(0, records.num_chunks);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	MyProc->delayChkpt = true;
@@ -2196,6 +2199,9 @@ RecordTransactionCommitPrepared(TransactionId xid,
 	replorigin = (replorigin_session_origin != InvalidRepOriginId &&
 				  replorigin_session_origin != DoNotReplicateId);
 
+	/* COMMIT PREPARED need not have an XID */
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* See notes in RecordTransactionCommit */
@@ -2286,6 +2292,9 @@ RecordTransactionAbortPrepared(TransactionId xid,
 		elog(PANIC, "cannot abort transaction %u, it was already committed",
 			 xid);
 
+	/* ROLLBACK PREPARED need not have an XID */
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
diff --git a/src/backend/access/transam/varsup.c b/src/backend/access/transam/varsup.c
index a4944faa32e..0c7a2362f25 100644
--- a/src/backend/access/transam/varsup.c
+++ b/src/backend/access/transam/varsup.c
@@ -17,6 +17,7 @@
 #include "access/commit_ts.h"
 #include "access/subtrans.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "commands/dbcommands.h"
@@ -75,6 +76,9 @@ GetNewTransactionId(bool isSubXact)
 	if (RecoveryInProgress())
 		elog(ERROR, "cannot assign TransactionIds during recovery");
 
+	/* do not assign transaction id in read-only mode */
+	CheckWALPermitted();
+
 	LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
 
 	full_xid = ShmemVariableCache->nextXid;
diff --git a/src/backend/access/transam/walprohibit.c b/src/backend/access/transam/walprohibit.c
index a3f1a750744..6f66804735f 100644
--- a/src/backend/access/transam/walprohibit.c
+++ b/src/backend/access/transam/walprohibit.c
@@ -16,6 +16,16 @@
 #include "postmaster/bgwriter.h"
 #include "storage/procsignal.h"
 
+/*
+ * Assert flag to enforce WAL insert permission check rule before starting a
+ * critical section for the WAL writes.  For this, either of CheckWALPermitted,
+ * AssertWALPermittedHaveXID, or AssertWALPermitted must be called before
+ * starting the critical section.
+ */
+#ifdef USE_ASSERT_CHECKING
+WALPermitCheckState walpermit_checked_state = WALPERMIT_UNCHECKED;
+#endif
+
 /*
  * ProcessBarrierWALProhibit()
  *
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 98a1943f717..4e8c7bb7dcc 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -26,6 +26,7 @@
 #include "access/subtrans.h"
 #include "access/transam.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1317,6 +1318,8 @@ RecordTransactionCommit(void)
 		/* Tell bufmgr and smgr to prepare for commit */
 		BufmgrCommit();
 
+		AssertWALPermittedHaveXID();
+
 		/*
 		 * Mark ourselves as within our "commit critical section".  This
 		 * forces any concurrent checkpoint to wait until we've updated
@@ -1677,6 +1680,9 @@ RecordTransactionAbort(bool isSubXact)
 		elog(PANIC, "cannot abort transaction %u, it was already committed",
 			 xid);
 
+	/* We'll be reaching here with valid XID only. */
+	AssertWALPermittedHaveXID();
+
 	/* Fetch the data we need for the abort record */
 	nrels = smgrGetPendingDeletes(false, &rels);
 	nchildren = xactGetCommittedChildren(&children);
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 2091eff0d53..211abaca22a 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -1025,7 +1025,7 @@ XLogInsertRecord(XLogRecData *rdata,
 
 	/* cross-check on whether we should be here or not */
 	if (!XLogInsertAllowed())
-		elog(ERROR, "cannot make new WAL entries during recovery");
+		elog(ERROR, "cannot make new WAL entries now");
 
 	/*----------
 	 *
@@ -2860,9 +2860,11 @@ XLogFlush(XLogRecPtr record)
 	 * trying to flush the WAL, we should update minRecoveryPoint instead. We
 	 * test XLogInsertAllowed(), not InRecovery, because we need checkpointer
 	 * to act this way too, and because when it tries to write the
-	 * end-of-recovery checkpoint, it should indeed flush.
+	 * end-of-recovery checkpoint, it should indeed flush. Also, WAL prohibit
+	 * state should not restrict WAL flushing. Otherwise, the dirty buffer
+	 * cannot be evicted until WAL has been flushed up to the buffer's LSN.
 	 */
-	if (!XLogInsertAllowed())
+	if (!XLogInsertAllowed() && !IsWALProhibited())
 	{
 		UpdateMinRecoveryPoint(record, false);
 		return;
@@ -8852,6 +8854,8 @@ CreateCheckPoint(int flags)
 	/* sanity check */
 	if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
 		elog(ERROR, "can't create a checkpoint during recovery");
+	if (!XLogInsertAllowed() && !RecoveryInProgress())
+		elog(ERROR, "can't create a checkpoint while system is read only ");
 
 	/*
 	 * Initialize InitXLogInsert working areas before entering the critical
@@ -8881,6 +8885,8 @@ CreateCheckPoint(int flags)
 	MemSet(&CheckpointStats, 0, sizeof(CheckpointStats));
 	CheckpointStats.ckpt_start_t = GetCurrentTimestamp();
 
+	AssertWALPermitted();
+
 	/*
 	 * Use a critical section to force system panic if we have trouble.
 	 */
@@ -9109,6 +9115,8 @@ CreateCheckPoint(int flags)
 	if (!shutdown && XLogStandbyInfoActive())
 		LogStandbySnapshot();
 
+	AssertWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -9266,6 +9274,8 @@ CreateEndOfRecoveryRecord(void)
 
 	LocalSetXLogInsertAllowed();
 
+	AssertWALPermitted();
+
 	START_CRIT_SECTION();
 
 	XLogBeginInsert();
@@ -9912,7 +9922,7 @@ void
 UpdateFullPageWrites(void)
 {
 	XLogCtlInsert *Insert = &XLogCtl->Insert;
-	bool		recoveryInProgress;
+	bool		WALInsertAllowed;
 
 	/*
 	 * Do nothing if full_page_writes has not been changed.
@@ -9926,10 +9936,10 @@ UpdateFullPageWrites(void)
 
 	/*
 	 * Perform this outside critical section so that the WAL insert
-	 * initialization done by RecoveryInProgress() doesn't trigger an
+	 * initialization done by XLogInsertAllowed() doesn't trigger an
 	 * assertion failure.
 	 */
-	recoveryInProgress = RecoveryInProgress();
+	WALInsertAllowed = XLogInsertAllowed();
 
 	START_CRIT_SECTION();
 
@@ -9951,8 +9961,11 @@ UpdateFullPageWrites(void)
 	 * Write an XLOG_FPW_CHANGE record. This allows us to keep track of
 	 * full_page_writes during archive recovery, if required.
 	 */
-	if (XLogStandbyInfoActive() && !recoveryInProgress)
+	if (XLogStandbyInfoActive() && WALInsertAllowed)
 	{
+		/* Assured that WAL permission has been checked */
+		AssertWALPermitted();
+
 		XLogBeginInsert();
 		XLogRegisterData((char *) (&fullPageWrites), sizeof(bool));
 
diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c
index c526bb19281..506d7e97f38 100644
--- a/src/backend/access/transam/xloginsert.c
+++ b/src/backend/access/transam/xloginsert.c
@@ -19,6 +19,7 @@
 
 #include "postgres.h"
 
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xlog_internal.h"
@@ -126,9 +127,14 @@ XLogBeginInsert(void)
 	Assert(mainrdata_last == (XLogRecData *) &mainrdata_head);
 	Assert(mainrdata_len == 0);
 
+	/*
+	 * WAL permission must have checked before entering the critical section.
+	 * Otherwise, WAL prohibited error will force system panic.
+	 */
+	Assert(walpermit_checked_state != WALPERMIT_UNCHECKED || !CritSectionCount);
+
 	/* cross-check on whether we should be here or not */
-	if (!XLogInsertAllowed())
-		elog(ERROR, "cannot make new WAL entries during recovery");
+	CheckWALPermitted();
 
 	if (begininsert_called)
 		elog(ERROR, "XLogBeginInsert was already called");
@@ -210,6 +216,9 @@ XLogResetInsertion(void)
 	mainrdata_last = (XLogRecData *) &mainrdata_head;
 	curinsert_flags = 0;
 	begininsert_called = false;
+
+	/* Reset walpermit_checked flag */
+	RESET_WALPERMIT_CHECKED_STATE();
 }
 
 /*
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 6aab73bfd44..8dacf48db24 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -20,6 +20,7 @@
 #include "access/relation.h"
 #include "access/table.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -378,8 +379,13 @@ fill_seq_with_data(Relation rel, HeapTuple tuple)
 
 	/* check the comment above nextval_internal()'s equivalent call. */
 	if (RelationNeedsWAL(rel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermittedHaveXID();
+	}
+
 	START_CRIT_SECTION();
 
 	MarkBufferDirty(buf);
@@ -766,8 +772,13 @@ nextval_internal(Oid relid, bool check_permissions)
 	 * (Have to do that here, so we're outside the critical section)
 	 */
 	if (logit && RelationNeedsWAL(seqrel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermittedHaveXID();
+	}
+
 	/* ready to change the on-disk (or really, in-buffer) tuple */
 	START_CRIT_SECTION();
 
@@ -977,8 +988,13 @@ do_setval(Oid relid, int64 next, bool iscalled)
 
 	/* check the comment above nextval_internal()'s equivalent call. */
 	if (RelationNeedsWAL(seqrel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermittedHaveXID();
+	}
+
 	/* ready to change the on-disk (or really, in-buffer) tuple */
 	START_CRIT_SECTION();
 
diff --git a/src/backend/commands/variable.c b/src/backend/commands/variable.c
index 484f7ea2c0e..acc34d2ad7c 100644
--- a/src/backend/commands/variable.c
+++ b/src/backend/commands/variable.c
@@ -501,11 +501,14 @@ check_transaction_read_only(bool *newval, void **extra, GucSource source)
 			GUC_check_errmsg("transaction read-write mode must be set before any query");
 			return false;
 		}
-		/* Can't go to r/w mode while recovery is still active */
-		if (RecoveryInProgress())
+		/*
+		 * Can't go to r/w mode while recovery is still active or while in WAL
+		 * prohibit state
+		 */
+		if (!XLogInsertAllowed())
 		{
 			GUC_check_errcode(ERRCODE_FEATURE_NOT_SUPPORTED);
-			GUC_check_errmsg("cannot set transaction read-write mode during recovery");
+			GUC_check_errmsg("cannot set transaction read-write mode while system is read only");
 			return false;
 		}
 	}
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 3e8aa9a0ec3..5404b7bc126 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -935,6 +935,10 @@ RequestCheckpoint(int flags)
 	int			old_failed,
 				old_started;
 
+	/* The checkpoint is allowed in recovery but not in WAL prohibit state */
+	if (!RecoveryInProgress())
+		CheckWALPermitted();
+
 	/*
 	 * If in a standalone backend, just do it ourselves.
 	 */
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index a2a963bd5b4..186cc47be1d 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -3638,13 +3638,15 @@ MarkBufferDirtyHint(Buffer buffer, bool buffer_std)
 		{
 			/*
 			 * If we must not write WAL, due to a relfilenode-specific
-			 * condition or being in recovery, don't dirty the page.  We can
-			 * set the hint, just not dirty the page as a result so the hint
-			 * is lost when we evict the page or shutdown.
+			 * condition or in general, don't dirty the page.  We can
+			 * set the hint, but must not dirty the page as a result, lest
+			 * we trigger WAL generation. Unless the page is dirtied again
+			 * later, the hint will be lost when the page is evicted, or at
+			 * shutdown.
 			 *
 			 * See src/backend/storage/page/README for longer discussion.
 			 */
-			if (RecoveryInProgress() ||
+			if (!XLogInsertAllowed() ||
 				RelFileNodeSkippingWAL(bufHdr->tag.rnode))
 				return;
 
diff --git a/src/backend/storage/freespace/freespace.c b/src/backend/storage/freespace/freespace.c
index 6a96126b0c2..b05b0fe5f41 100644
--- a/src/backend/storage/freespace/freespace.c
+++ b/src/backend/storage/freespace/freespace.c
@@ -24,6 +24,7 @@
 #include "postgres.h"
 
 #include "access/htup_details.h"
+#include "access/walprohibit.h"
 #include "access/xlogutils.h"
 #include "miscadmin.h"
 #include "storage/freespace.h"
@@ -285,12 +286,19 @@ FreeSpaceMapPrepareTruncateRel(Relation rel, BlockNumber nblocks)
 	 */
 	if (first_removed_slot > 0)
 	{
+		bool needwal;
+
 		buf = fsm_readbuf(rel, first_removed_address, false);
 		if (!BufferIsValid(buf))
 			return InvalidBlockNumber;	/* nothing to do; the FSM was already
 										 * smaller */
 		LockBuffer(buf, BUFFER_LOCK_EXCLUSIVE);
 
+		needwal = (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded());
+
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -305,7 +313,7 @@ FreeSpaceMapPrepareTruncateRel(Relation rel, BlockNumber nblocks)
 		 * during recovery.
 		 */
 		MarkBufferDirty(buf);
-		if (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded())
+		if (needwal)
 			log_newpage_buffer(buf, false);
 
 		END_CRIT_SECTION();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index d86566f4554..f949a290745 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -794,15 +794,15 @@ LockAcquireExtended(const LOCKTAG *locktag,
 	if (lockmode <= 0 || lockmode > lockMethodTable->numLockModes)
 		elog(ERROR, "unrecognized lock mode: %d", lockmode);
 
-	if (RecoveryInProgress() && !InRecovery &&
+	if (!XLogInsertAllowed() && !InRecovery &&
 		(locktag->locktag_type == LOCKTAG_OBJECT ||
 		 locktag->locktag_type == LOCKTAG_RELATION) &&
 		lockmode > RowExclusiveLock)
 		ereport(ERROR,
 				(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
-				 errmsg("cannot acquire lock mode %s on database objects while recovery is in progress",
+				 errmsg("cannot acquire lock mode %s on database objects while system is read only",
 						lockMethodTable->lockModeNames[lockmode]),
-				 errhint("Only RowExclusiveLock or less can be acquired on database objects during recovery.")));
+				 errhint("Only RowExclusiveLock or less can be acquired on database objects while system is read only")));
 
 #ifdef LOCK_DEBUG
 	if (LOCK_DEBUG_ENABLED(locktag))
diff --git a/src/backend/utils/cache/relmapper.c b/src/backend/utils/cache/relmapper.c
index 671fbb0ed5c..90d7599a57c 100644
--- a/src/backend/utils/cache/relmapper.c
+++ b/src/backend/utils/cache/relmapper.c
@@ -43,6 +43,7 @@
 #include <sys/stat.h>
 #include <unistd.h>
 
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -841,6 +842,8 @@ write_relmap_file(bool shared, RelMapFile *newmap,
 		xl_relmap_update xlrec;
 		XLogRecPtr	lsn;
 
+		AssertWALPermittedHaveXID();
+
 		/* now errors are fatal ... */
 		START_CRIT_SECTION();
 
diff --git a/src/include/access/walprohibit.h b/src/include/access/walprohibit.h
index 163fe0d2fce..3442df5be2f 100644
--- a/src/include/access/walprohibit.h
+++ b/src/include/access/walprohibit.h
@@ -19,8 +19,8 @@ extern bool ProcessBarrierWALProhibit(void);
 extern void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
 
 /* WAL Prohibit States */
-#define	WALPROHIBIT_STATE_READ_WRITE		0x0000
-#define	WALPROHIBIT_STATE_READ_ONLY			0x0001
+#define	WALPROHIBIT_STATE_READ_WRITE		0x0000	/* WAL permitted */
+#define	WALPROHIBIT_STATE_READ_ONLY			0x0001	/* WAL prohibited */
 
 /*
  * The bit is used in state transition from one state to another.  When this
@@ -29,4 +29,50 @@ extern void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
  */
 #define WALPROHIBIT_TRANSITION_IN_PROGRESS	0x0002
 
+/* Never reaches when WAL is prohibited. */
+static inline void
+AssertWALPermitted(void)
+{
+	/*
+	 * Recovery in the startup process never is in wal prohibited state.
+	 */
+	Assert(InRecovery || XLogInsertAllowed());
+
+#ifdef USE_ASSERT_CHECKING
+	walpermit_checked_state = WALPERMIT_CHECKED;
+#endif
+}
+
+/*
+ * XID-bearing transactions are killed off by "ALTER SYSTEM READ ONLY", so any
+ * part of the code that can only be reached with an XID assigned is never
+ * reached when WAL is prohibited.
+ */
+static inline void
+AssertWALPermittedHaveXID(void)
+{
+	/* Must be performing an INSERT, UPDATE or DELETE, so we'll have an XID */
+	Assert(FullTransactionIdIsValid(GetTopFullTransactionIdIfAny()));
+	AssertWALPermitted();
+}
+
+/*
+ * In opposite to the above assertion if a transaction doesn't have valid XID
+ * (e.g. VACUUM) then it won't be killed while changing the system state to WAL
+ * prohibited.  Therefore, we need to explicitly error out before entering into
+ * the critical section.
+ */
+static inline void
+CheckWALPermitted(void)
+{
+	if (!XLogInsertAllowed())
+		ereport(ERROR,
+				(errcode(ERRCODE_READ_ONLY_SQL_TRANSACTION),
+				 errmsg("system is now read only")));
+
+#ifdef USE_ASSERT_CHECKING
+	walpermit_checked_state = WALPERMIT_CHECKED;
+#endif
+}
+
 #endif		/* WALPROHIBIT_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e33523984..f3ff120601e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -94,12 +94,37 @@ extern PGDLLIMPORT volatile uint32 CritSectionCount;
 /* in tcop/postgres.c */
 extern void ProcessInterrupts(void);
 
+#ifdef USE_ASSERT_CHECKING
+typedef enum
+{
+	WALPERMIT_UNCHECKED,
+	WALPERMIT_CHECKED,
+	WALPERMIT_CHECKED_AND_USED
+} WALPermitCheckState;
+
+/* in access/walprohibit.c */
+extern WALPermitCheckState walpermit_checked_state;
+
+/*
+ * Reset walpermit_checked flag when no longer in the critical section.
+ * Otherwise, marked checked and used.
+ */
+#define RESET_WALPERMIT_CHECKED_STATE() \
+do { \
+	walpermit_checked_state = CritSectionCount ? \
+	WALPERMIT_CHECKED_AND_USED : WALPERMIT_UNCHECKED; \
+} while(0)
+#else
+#define RESET_WALPERMIT_CHECKED_STATE() ((void) 0)
+#endif
+
 #ifndef WIN32
 
 #define CHECK_FOR_INTERRUPTS() \
 do { \
 	if (unlikely(InterruptPending)) \
 		ProcessInterrupts(); \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 #else							/* WIN32 */
 
@@ -109,6 +134,7 @@ do { \
 		pgwin32_dispatch_queued_signals(); \
 	if (unlikely(InterruptPending)) \
 		ProcessInterrupts(); \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 #endif							/* WIN32 */
 
@@ -135,6 +161,7 @@ do { \
 do { \
 	Assert(CritSectionCount > 0); \
 	CritSectionCount--; \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 
 
-- 
2.22.0

From 32ce3f068a96fb8df185c935b11abb254697d7be Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 27 Mar 2020 05:05:38 -0400
Subject: [PATCH v5 2/5] Add alter system read only/write syntax

Note that syntax doesn't have any implementation.
---
 src/backend/nodes/copyfuncs.c    | 12 ++++++++++++
 src/backend/nodes/equalfuncs.c   |  9 +++++++++
 src/backend/nodes/outfuncs.c     | 12 ++++++++++++
 src/backend/nodes/readfuncs.c    | 15 +++++++++++++++
 src/backend/parser/gram.y        | 13 +++++++++++++
 src/backend/tcop/utility.c       | 20 ++++++++++++++++++++
 src/bin/psql/tab-complete.c      |  6 ++++--
 src/include/nodes/nodes.h        |  1 +
 src/include/nodes/parsenodes.h   | 10 ++++++++++
 src/tools/pgindent/typedefs.list |  1 +
 10 files changed, 97 insertions(+), 2 deletions(-)

diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 89c409de664..ba3393b8ccf 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -4020,6 +4020,15 @@ _copyAlterSystemStmt(const AlterSystemStmt *from)
 	return newnode;
 }
 
+static AlterSystemWALProhibitState *
+_copyAlterSystemWALProhibitState(const AlterSystemWALProhibitState *from)
+{
+	AlterSystemWALProhibitState *newnode = makeNode(AlterSystemWALProhibitState);
+
+	COPY_SCALAR_FIELD(WALProhibited);
+	return newnode;
+}
+
 static CreateSeqStmt *
 _copyCreateSeqStmt(const CreateSeqStmt *from)
 {
@@ -5406,6 +5415,9 @@ copyObjectImpl(const void *from)
 		case T_AlterSystemStmt:
 			retval = _copyAlterSystemStmt(from);
 			break;
+		case T_AlterSystemWALProhibitState:
+			retval = _copyAlterSystemWALProhibitState(from);
+			break;
 		case T_CreateSeqStmt:
 			retval = _copyCreateSeqStmt(from);
 			break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e3f33c40be5..b09bff458af 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1769,6 +1769,12 @@ _equalAlterSystemStmt(const AlterSystemStmt *a, const AlterSystemStmt *b)
 	return true;
 }
 
+static bool
+_equalAlterSystemWALProhibitState(const AlterSystemWALProhibitState *a, const AlterSystemWALProhibitState *b)
+{
+	COMPARE_SCALAR_FIELD(WALProhibited);
+	return true;
+}
 
 static bool
 _equalCreateSeqStmt(const CreateSeqStmt *a, const CreateSeqStmt *b)
@@ -3458,6 +3464,9 @@ equal(const void *a, const void *b)
 		case T_AlterSystemStmt:
 			retval = _equalAlterSystemStmt(a, b);
 			break;
+		case T_AlterSystemWALProhibitState:
+			retval = _equalAlterSystemWALProhibitState(a, b);
+			break;
 		case T_CreateSeqStmt:
 			retval = _equalCreateSeqStmt(a, b);
 			break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index e2f177515da..37f297f39a5 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -1358,6 +1358,15 @@ _outAlternativeSubPlan(StringInfo str, const AlternativeSubPlan *node)
 	WRITE_NODE_FIELD(subplans);
 }
 
+static void
+_outAlterSystemWALProhibitState(StringInfo str,
+								const AlterSystemWALProhibitState *node)
+{
+	WRITE_NODE_TYPE("ALTERSYSTEMWALPROHIBITSTATE");
+
+	WRITE_BOOL_FIELD(WALProhibited);
+}
+
 static void
 _outFieldSelect(StringInfo str, const FieldSelect *node)
 {
@@ -3914,6 +3923,9 @@ outNode(StringInfo str, const void *obj)
 			case T_AlternativeSubPlan:
 				_outAlternativeSubPlan(str, obj);
 				break;
+			case T_AlterSystemWALProhibitState:
+				_outAlterSystemWALProhibitState(str, obj);
+				break;
 			case T_FieldSelect:
 				_outFieldSelect(str, obj);
 				break;
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 42050ab7195..0ac826d3c2f 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -2552,6 +2552,19 @@ _readAlternativeSubPlan(void)
 	READ_DONE();
 }
 
+/*
+ * _readAlterSystemWALProhibitState
+ */
+static AlterSystemWALProhibitState *
+_readAlterSystemWALProhibitState(void)
+{
+	READ_LOCALS(AlterSystemWALProhibitState);
+
+	READ_BOOL_FIELD(WALProhibited);
+
+	READ_DONE();
+}
+
 /*
  * _readExtensibleNode
  */
@@ -2874,6 +2887,8 @@ parseNodeString(void)
 		return_value = _readSubPlan();
 	else if (MATCH("ALTERNATIVESUBPLAN", 18))
 		return_value = _readAlternativeSubPlan();
+	else if (MATCH("ALTERSYSTEMWALPROHIBITSTATE", 27))
+		return_value = _readAlterSystemWALProhibitState();
 	else if (MATCH("EXTENSIBLENODE", 14))
 		return_value = _readExtensibleNode();
 	else if (MATCH("PARTITIONBOUNDSPEC", 18))
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index dbb47d49829..6090d18ec61 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -479,6 +479,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
 
 %type <vsetstmt> generic_set set_rest set_rest_more generic_reset reset_rest
 				 SetResetClause FunctionSetResetClause
+%type <boolean> system_readonly_state
 
 %type <node>	TableElement TypedTableElement ConstraintElem TableFuncElement
 %type <node>	columnDef columnOptions
@@ -10172,8 +10173,20 @@ AlterSystemStmt:
 					n->setstmt = $4;
 					$$ = (Node *)n;
 				}
+			| ALTER SYSTEM_P system_readonly_state
+				{
+					AlterSystemWALProhibitState *n = makeNode(AlterSystemWALProhibitState);
+					n->WALProhibited = $3;
+					$$ = (Node *)n;
+				}
 		;
 
+system_readonly_state:
+			 READ ONLY
+					{ $$ = true; }
+			| READ WRITE
+					{ $$ = false; }
+		;
 
 /*****************************************************************************
  *
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 9b0c376c8cb..7af96c77082 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -85,6 +85,7 @@ static void ProcessUtilitySlow(ParseState *pstate,
 							   DestReceiver *dest,
 							   QueryCompletion *qc);
 static void ExecDropStmt(DropStmt *stmt, bool isTopLevel);
+static void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
 
 /*
  * CommandIsReadOnly: is an executable query read-only?
@@ -219,6 +220,7 @@ ClassifyUtilityCommandAsReadOnly(Node *parsetree)
 				return COMMAND_IS_NOT_READ_ONLY;
 			}
 
+		case T_AlterSystemWALProhibitState:
 		case T_AlterSystemStmt:
 			{
 				/*
@@ -834,6 +836,11 @@ standard_ProcessUtility(PlannedStmt *pstmt,
 			AlterSystemSetConfigFile((AlterSystemStmt *) parsetree);
 			break;
 
+		case T_AlterSystemWALProhibitState:
+			PreventInTransactionBlock(isTopLevel, "ALTER SYSTEM");
+			AlterSystemSetWALProhibitState((AlterSystemWALProhibitState *) parsetree);
+			break;
+
 		case T_VariableSetStmt:
 			ExecSetVariableStmt((VariableSetStmt *) parsetree, isTopLevel);
 			break;
@@ -2772,6 +2779,7 @@ CreateCommandTag(Node *parsetree)
 			tag = CMDTAG_REFRESH_MATERIALIZED_VIEW;
 			break;
 
+		case T_AlterSystemWALProhibitState:
 		case T_AlterSystemStmt:
 			tag = CMDTAG_ALTER_SYSTEM;
 			break;
@@ -3636,3 +3644,15 @@ GetCommandLogLevel(Node *parsetree)
 
 	return lev;
 }
+
+/*
+ * AlterSystemSetWALProhibitState
+ *
+ * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
+ */
+static void
+AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
+{
+	/* some code */
+	elog(INFO, "AlterSystemSetWALProhibitState() called");
+}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index f41785f11c1..408f6260b26 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1864,9 +1864,11 @@ psql_completion(const char *text, int start, int end)
 	/* ALTER SERVER <name> VERSION <version> */
 	else if (Matches("ALTER", "SERVER", MatchAny, "VERSION", MatchAny))
 		COMPLETE_WITH("OPTIONS");
-	/* ALTER SYSTEM SET, RESET, RESET ALL */
+	/* ALTER SYSTEM READ, SET, RESET, RESET ALL */
 	else if (Matches("ALTER", "SYSTEM"))
-		COMPLETE_WITH("SET", "RESET");
+		COMPLETE_WITH("SET", "RESET", "READ");
+	else if (Matches("ALTER", "SYSTEM", "READ"))
+		COMPLETE_WITH("ONLY", "WRITE");
 	else if (Matches("ALTER", "SYSTEM", "SET|RESET"))
 		COMPLETE_WITH_QUERY(Query_for_list_of_alter_system_set_vars);
 	else if (Matches("ALTER", "SYSTEM", "SET", MatchAny))
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 381d84b4e4f..17d6942c734 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -412,6 +412,7 @@ typedef enum NodeTag
 	T_RefreshMatViewStmt,
 	T_ReplicaIdentityStmt,
 	T_AlterSystemStmt,
+	T_AlterSystemWALProhibitState,
 	T_CreatePolicyStmt,
 	T_AlterPolicyStmt,
 	T_CreateTransformStmt,
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 151bcdb7ef5..f2c1ae8e3fe 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -3194,6 +3194,16 @@ typedef struct AlterSystemStmt
 	VariableSetStmt *setstmt;	/* SET subcommand */
 } AlterSystemStmt;
 
+/* ----------------------
+ *		Alter System Read Statement
+ * ----------------------
+ */
+typedef struct AlterSystemWALProhibitState
+{
+	NodeTag		type;
+	bool		WALProhibited;
+} AlterSystemWALProhibitState;
+
 /* ----------------------
  *		Cluster Statement (support pbrown's cluster index implementation)
  * ----------------------
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 3d990463ce9..29d6f6c968d 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -87,6 +87,7 @@ AlterStatsStmt
 AlterSubscriptionStmt
 AlterSubscriptionType
 AlterSystemStmt
+AlterSystemWALProhibitState
 AlterTSConfigType
 AlterTSConfigurationStmt
 AlterTSDictionaryStmt
-- 
2.22.0

From 7b25a9283429d63a7cb34aeba3f0836f40eaa6e9 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Tue, 14 Jul 2020 02:30:44 -0400
Subject: [PATCH v5 5/5] Documentation - WIP

TODOs:

1] TBC, the section for pg_is_in_readonly() function, right now it is under "Recovery Information Functions"
2] Documentation regarding ALTER SYSTEM READ/WRITE
---
 src/backend/access/transam/README | 60 ++++++++++++++++++++++++++++---
 src/backend/storage/page/README   | 12 +++----
 2 files changed, 61 insertions(+), 11 deletions(-)

diff --git a/src/backend/access/transam/README b/src/backend/access/transam/README
index 1edc8180c12..fe7a84f93da 100644
--- a/src/backend/access/transam/README
+++ b/src/backend/access/transam/README
@@ -442,8 +442,8 @@ to be modified.
 2. START_CRIT_SECTION()  (Any error during the next three steps must cause a
 PANIC because the shared buffers will contain unlogged changes, which we
 have to ensure don't get to disk.  Obviously, you should check conditions
-such as whether there's enough free space on the page before you start the
-critical section.)
+such as whether there's WAL write permission and enough free space on the page
+before you start the critical section.)
 
 3. Apply the required changes to the shared buffer(s).
 
@@ -486,6 +486,54 @@ with the incomplete-split flag set, it will finish the interrupted split by
 inserting the key to the parent, before proceeding.
 
 
+Read only system state
+----------------------
+
+The system state when it is not currently possible to insert write-ahead log
+records, either because the system is still in recovery or because the system
+forced to WAL prohibited by executing ALTER SYSTEM READ ONLY.  We have a
+lower-level defense in XLogBeginInsert() and elsewhere to stop us from modifying
+data during recovery when !XLogInsertAllowed(), but if XLogBeginInsert() is
+inside the critical section we must not depend on it to report an error.
+Otherwise, it will cause PANIC as mentioned previously.
+
+We do not reach the point where we try to write WAL during recovery but ALTER
+SYSTEM READ ONLY can be executed anytime by the user to stop WAL writing.  Any
+backends which receive read-only system state transition barrier interrupt need
+to stop WAL writing immediately.  For barrier absorption the backed(s) will kill
+the running transaction which has valid XID indicates that the transaction has
+performed and/or planning WAL write.  The transaction which doesn't acquire
+valid XID yet or operation such VACUUM or CONCURRENT CREATE INDEX which not
+necessary have valid XID for WAL will not be prevented while barrier processing,
+and those might hit the error from XLogBeginInsert() while trying to write WAL
+in read only system state.  To prevent such error from XLogBeginInsert() inside
+the critical section the WAL write permission has to check before
+START_CRIT_SECTION().
+
+To enforce the practice to check WAL permission before entering into critical
+section for the WAL write, we have added an assert check flag that indicates
+permission has been checked before calling XLogBeginInsert().  If not,
+XLogBeginInsert() will have assertion failure.  WAL permission check is not
+mandatory if the XLogBeginInsert() is not inside the critical section where
+throwing the error is acceptable.  To get permission check flag set either
+CheckWALPermitted(), AssertWALPermitted_HaveXID(), or AssertWALPermitted()
+should be called before START_CRIT_SECTION().  This flag automatically resets
+while exiting from the critical section.  The rule to place either of permission
+check routines will be:
+
+	The places where WAL write operation in critical can be expected without
+	having valid XID (e.g vacuum) need to protect by CheckWALPermitted(), so
+	that error can be reported outside before critical section.
+
+	The places where INSERT and UPDATE are expected which are never happened
+	without valid XID can be checked using AssertWALPermitted_HaveXID.  So that
+	non-assert build will not have the checking overhead.
+
+	The places we know that we cannot be reached in the read-only state and may
+	or may not have XID, but need to ensure the permission has been checked on
+	assert enabled build should use AssertWALPermitted().
+
+
 Constructing a WAL record
 -------------------------
 
@@ -531,7 +579,8 @@ Details of the API functions:
 
 void XLogBeginInsert(void)
 
-    Must be called before XLogRegisterBuffer and XLogRegisterData.
+	Must be called before XLogRegisterBuffer and XLogRegisterData.  WAL
+	permission must be check before calling it in a critical section.
 
 void XLogResetInsertion(void)
 
@@ -638,8 +687,9 @@ MarkBufferDirtyHint() to mark the block dirty.
 If the buffer is clean and checksums are in use then MarkBufferDirtyHint()
 inserts an XLOG_FPI_FOR_HINT record to ensure that we take a full page image
 that includes the hint. We do this to avoid a partial page write, when we
-write the dirtied page. WAL is not written during recovery, so we simply skip
-dirtying blocks because of hints when in recovery.
+write the dirtied page. WAL is not written while in read only (i.e. during
+recovery or in WAL prohibit state), so we simply skip dirtying blocks because of
+hints when in recovery.
 
 If you do decide to optimise away a WAL record, then any calls to
 MarkBufferDirty() must be replaced by MarkBufferDirtyHint(),
diff --git a/src/backend/storage/page/README b/src/backend/storage/page/README
index e30d7ac59ad..15f0bb4b7b5 100644
--- a/src/backend/storage/page/README
+++ b/src/backend/storage/page/README
@@ -56,9 +56,9 @@ WAL is a fatal error and prevents further recovery, whereas a checksum failure
 on a normal data block is a hard error but not a critical one for the server,
 even if it is a very bad thing for the user.
 
-New WAL records cannot be written during recovery, so hint bits set during
-recovery must not dirty the page if the buffer is not already dirty, when
-checksums are enabled.  Systems in Hot-Standby mode may benefit from hint bits
-being set, but with checksums enabled, a page cannot be dirtied after setting a
-hint bit (due to the torn page risk). So, it must wait for full-page images
-containing the hint bit updates to arrive from the primary.
+New WAL records cannot be written during recovery or or while in WAL prohibit
+state, so hint bits set during recovery must not dirty the page if the buffer is
+not already dirty, when checksums are enabled.  Systems in Hot-Standby mode may
+benefit from hint bits being set, but with checksums enabled, a page cannot be
+dirtied after setting a hint bit (due to the torn page risk). So, it must wait
+for full-page images containing the hint bit updates to arrive from the primary.
-- 
2.22.0

From c711defd1e1ae7121293b933eba686d5b241ca49 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 27 Mar 2020 05:05:38 -0400
Subject: [PATCH v6 2/5] Add alter system read only/write syntax

Note that syntax doesn't have any implementation.
---
 src/backend/nodes/copyfuncs.c    | 12 ++++++++++++
 src/backend/nodes/equalfuncs.c   |  9 +++++++++
 src/backend/nodes/outfuncs.c     | 12 ++++++++++++
 src/backend/nodes/readfuncs.c    | 15 +++++++++++++++
 src/backend/parser/gram.y        | 13 +++++++++++++
 src/backend/tcop/utility.c       | 20 ++++++++++++++++++++
 src/bin/psql/tab-complete.c      |  6 ++++--
 src/include/nodes/nodes.h        |  1 +
 src/include/nodes/parsenodes.h   | 10 ++++++++++
 src/tools/pgindent/typedefs.list |  1 +
 10 files changed, 97 insertions(+), 2 deletions(-)

diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 89c409de664..ba3393b8ccf 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -4020,6 +4020,15 @@ _copyAlterSystemStmt(const AlterSystemStmt *from)
 	return newnode;
 }
 
+static AlterSystemWALProhibitState *
+_copyAlterSystemWALProhibitState(const AlterSystemWALProhibitState *from)
+{
+	AlterSystemWALProhibitState *newnode = makeNode(AlterSystemWALProhibitState);
+
+	COPY_SCALAR_FIELD(WALProhibited);
+	return newnode;
+}
+
 static CreateSeqStmt *
 _copyCreateSeqStmt(const CreateSeqStmt *from)
 {
@@ -5406,6 +5415,9 @@ copyObjectImpl(const void *from)
 		case T_AlterSystemStmt:
 			retval = _copyAlterSystemStmt(from);
 			break;
+		case T_AlterSystemWALProhibitState:
+			retval = _copyAlterSystemWALProhibitState(from);
+			break;
 		case T_CreateSeqStmt:
 			retval = _copyCreateSeqStmt(from);
 			break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e3f33c40be5..b09bff458af 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1769,6 +1769,12 @@ _equalAlterSystemStmt(const AlterSystemStmt *a, const AlterSystemStmt *b)
 	return true;
 }
 
+static bool
+_equalAlterSystemWALProhibitState(const AlterSystemWALProhibitState *a, const AlterSystemWALProhibitState *b)
+{
+	COMPARE_SCALAR_FIELD(WALProhibited);
+	return true;
+}
 
 static bool
 _equalCreateSeqStmt(const CreateSeqStmt *a, const CreateSeqStmt *b)
@@ -3458,6 +3464,9 @@ equal(const void *a, const void *b)
 		case T_AlterSystemStmt:
 			retval = _equalAlterSystemStmt(a, b);
 			break;
+		case T_AlterSystemWALProhibitState:
+			retval = _equalAlterSystemWALProhibitState(a, b);
+			break;
 		case T_CreateSeqStmt:
 			retval = _equalCreateSeqStmt(a, b);
 			break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index e2f177515da..37f297f39a5 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -1358,6 +1358,15 @@ _outAlternativeSubPlan(StringInfo str, const AlternativeSubPlan *node)
 	WRITE_NODE_FIELD(subplans);
 }
 
+static void
+_outAlterSystemWALProhibitState(StringInfo str,
+								const AlterSystemWALProhibitState *node)
+{
+	WRITE_NODE_TYPE("ALTERSYSTEMWALPROHIBITSTATE");
+
+	WRITE_BOOL_FIELD(WALProhibited);
+}
+
 static void
 _outFieldSelect(StringInfo str, const FieldSelect *node)
 {
@@ -3914,6 +3923,9 @@ outNode(StringInfo str, const void *obj)
 			case T_AlternativeSubPlan:
 				_outAlternativeSubPlan(str, obj);
 				break;
+			case T_AlterSystemWALProhibitState:
+				_outAlterSystemWALProhibitState(str, obj);
+				break;
 			case T_FieldSelect:
 				_outFieldSelect(str, obj);
 				break;
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 42050ab7195..0ac826d3c2f 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -2552,6 +2552,19 @@ _readAlternativeSubPlan(void)
 	READ_DONE();
 }
 
+/*
+ * _readAlterSystemWALProhibitState
+ */
+static AlterSystemWALProhibitState *
+_readAlterSystemWALProhibitState(void)
+{
+	READ_LOCALS(AlterSystemWALProhibitState);
+
+	READ_BOOL_FIELD(WALProhibited);
+
+	READ_DONE();
+}
+
 /*
  * _readExtensibleNode
  */
@@ -2874,6 +2887,8 @@ parseNodeString(void)
 		return_value = _readSubPlan();
 	else if (MATCH("ALTERNATIVESUBPLAN", 18))
 		return_value = _readAlternativeSubPlan();
+	else if (MATCH("ALTERSYSTEMWALPROHIBITSTATE", 27))
+		return_value = _readAlterSystemWALProhibitState();
 	else if (MATCH("EXTENSIBLENODE", 14))
 		return_value = _readExtensibleNode();
 	else if (MATCH("PARTITIONBOUNDSPEC", 18))
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index dbb47d49829..6090d18ec61 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -479,6 +479,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
 
 %type <vsetstmt> generic_set set_rest set_rest_more generic_reset reset_rest
 				 SetResetClause FunctionSetResetClause
+%type <boolean> system_readonly_state
 
 %type <node>	TableElement TypedTableElement ConstraintElem TableFuncElement
 %type <node>	columnDef columnOptions
@@ -10172,8 +10173,20 @@ AlterSystemStmt:
 					n->setstmt = $4;
 					$$ = (Node *)n;
 				}
+			| ALTER SYSTEM_P system_readonly_state
+				{
+					AlterSystemWALProhibitState *n = makeNode(AlterSystemWALProhibitState);
+					n->WALProhibited = $3;
+					$$ = (Node *)n;
+				}
 		;
 
+system_readonly_state:
+			 READ ONLY
+					{ $$ = true; }
+			| READ WRITE
+					{ $$ = false; }
+		;
 
 /*****************************************************************************
  *
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 6154d2c8c63..1730a5402f7 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -85,6 +85,7 @@ static void ProcessUtilitySlow(ParseState *pstate,
 							   DestReceiver *dest,
 							   QueryCompletion *qc);
 static void ExecDropStmt(DropStmt *stmt, bool isTopLevel);
+static void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
 
 /*
  * CommandIsReadOnly: is an executable query read-only?
@@ -219,6 +220,7 @@ ClassifyUtilityCommandAsReadOnly(Node *parsetree)
 				return COMMAND_IS_NOT_READ_ONLY;
 			}
 
+		case T_AlterSystemWALProhibitState:
 		case T_AlterSystemStmt:
 			{
 				/*
@@ -834,6 +836,11 @@ standard_ProcessUtility(PlannedStmt *pstmt,
 			AlterSystemSetConfigFile((AlterSystemStmt *) parsetree);
 			break;
 
+		case T_AlterSystemWALProhibitState:
+			PreventInTransactionBlock(isTopLevel, "ALTER SYSTEM");
+			AlterSystemSetWALProhibitState((AlterSystemWALProhibitState *) parsetree);
+			break;
+
 		case T_VariableSetStmt:
 			ExecSetVariableStmt((VariableSetStmt *) parsetree, isTopLevel);
 			break;
@@ -2806,6 +2813,7 @@ CreateCommandTag(Node *parsetree)
 			tag = CMDTAG_REFRESH_MATERIALIZED_VIEW;
 			break;
 
+		case T_AlterSystemWALProhibitState:
 		case T_AlterSystemStmt:
 			tag = CMDTAG_ALTER_SYSTEM;
 			break;
@@ -3670,3 +3678,15 @@ GetCommandLogLevel(Node *parsetree)
 
 	return lev;
 }
+
+/*
+ * AlterSystemSetWALProhibitState
+ *
+ * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
+ */
+static void
+AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
+{
+	/* some code */
+	elog(INFO, "AlterSystemSetWALProhibitState() called");
+}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index f41785f11c1..408f6260b26 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1864,9 +1864,11 @@ psql_completion(const char *text, int start, int end)
 	/* ALTER SERVER <name> VERSION <version> */
 	else if (Matches("ALTER", "SERVER", MatchAny, "VERSION", MatchAny))
 		COMPLETE_WITH("OPTIONS");
-	/* ALTER SYSTEM SET, RESET, RESET ALL */
+	/* ALTER SYSTEM READ, SET, RESET, RESET ALL */
 	else if (Matches("ALTER", "SYSTEM"))
-		COMPLETE_WITH("SET", "RESET");
+		COMPLETE_WITH("SET", "RESET", "READ");
+	else if (Matches("ALTER", "SYSTEM", "READ"))
+		COMPLETE_WITH("ONLY", "WRITE");
 	else if (Matches("ALTER", "SYSTEM", "SET|RESET"))
 		COMPLETE_WITH_QUERY(Query_for_list_of_alter_system_set_vars);
 	else if (Matches("ALTER", "SYSTEM", "SET", MatchAny))
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 381d84b4e4f..17d6942c734 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -412,6 +412,7 @@ typedef enum NodeTag
 	T_RefreshMatViewStmt,
 	T_ReplicaIdentityStmt,
 	T_AlterSystemStmt,
+	T_AlterSystemWALProhibitState,
 	T_CreatePolicyStmt,
 	T_AlterPolicyStmt,
 	T_CreateTransformStmt,
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 47d4c07306d..f82de3c126f 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -3195,6 +3195,16 @@ typedef struct AlterSystemStmt
 	VariableSetStmt *setstmt;	/* SET subcommand */
 } AlterSystemStmt;
 
+/* ----------------------
+ *		Alter System Read Statement
+ * ----------------------
+ */
+typedef struct AlterSystemWALProhibitState
+{
+	NodeTag		type;
+	bool		WALProhibited;
+} AlterSystemWALProhibitState;
+
 /* ----------------------
  *		Cluster Statement (support pbrown's cluster index implementation)
  * ----------------------
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 3d990463ce9..29d6f6c968d 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -87,6 +87,7 @@ AlterStatsStmt
 AlterSubscriptionStmt
 AlterSubscriptionType
 AlterSystemStmt
+AlterSystemWALProhibitState
 AlterTSConfigType
 AlterTSConfigurationStmt
 AlterTSDictionaryStmt
-- 
2.22.0

From 3230e0b1b77446eba50313466fc50ffddfe61ac7 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 19 Jun 2020 06:27:53 -0400
Subject: [PATCH v6 1/5] Allow error or refusal while absorbing barriers.

Patch by Robert Haas
---
 src/backend/storage/ipc/procsignal.c | 75 +++++++++++++++++++++++-----
 1 file changed, 63 insertions(+), 12 deletions(-)

diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 4fa385b0ece..13648887187 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -87,12 +87,16 @@ typedef struct
 #define BARRIER_SHOULD_CHECK(flags, type) \
 	(((flags) & (((uint32) 1) << (uint32) (type))) != 0)
 
+/* Clear the relevant type bit from the flags. */
+#define BARRIER_CLEAR_BIT(flags, type) \
+	((flags) &= ~(((uint32) 1) << (uint32) (type)))
+
 static ProcSignalHeader *ProcSignal = NULL;
 static volatile ProcSignalSlot *MyProcSignalSlot = NULL;
 
 static bool CheckProcSignal(ProcSignalReason reason);
 static void CleanupProcSignalState(int status, Datum arg);
-static void ProcessBarrierPlaceholder(void);
+static bool ProcessBarrierPlaceholder(void);
 
 /*
  * ProcSignalShmemSize
@@ -486,17 +490,59 @@ ProcessProcSignalBarrier(void)
 	flags = pg_atomic_exchange_u32(&MyProcSignalSlot->pss_barrierCheckMask, 0);
 
 	/*
-	 * Process each type of barrier. It's important that nothing we call from
-	 * here throws an error, because pss_barrierCheckMask has already been
-	 * cleared. If we jumped out of here before processing all barrier types,
-	 * then we'd forget about the need to do so later.
-	 *
-	 * NB: It ought to be OK to call the barrier-processing functions
-	 * unconditionally, but it's more efficient to call only the ones that
-	 * might need us to do something based on the flags.
+	 * If there are no flags set, then we can skip doing any real work.
+	 * Otherwise, establish a PG_TRY block, so that we don't lose track of
+	 * which types of barrier processing are needed if an ERROR occurs.
 	 */
-	if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_PLACEHOLDER))
-		ProcessBarrierPlaceholder();
+	if (flags != 0)
+	{
+		PG_TRY();
+		{
+			/*
+			 * Process each type of barrier. The barrier-processing functions
+			 * should normally return true, but may return false if the barrier
+			 * can't be absorbed at the current time. This should be rare,
+			 * because it's pretty expensive.  Every single
+			 * CHECK_FOR_INTERRUPTS() will return here until we manage to
+			 * absorb the barrier, and that cost will add up in a hurry.
+			 *
+			 * NB: It ought to be OK to call the barrier-processing functions
+			 * unconditionally, but it's more efficient to call only the ones
+			 * that might need us to do something based on the flags.
+			 */
+			if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_PLACEHOLDER)
+				&& ProcessBarrierPlaceholder())
+				BARRIER_CLEAR_BIT(flags, PROCSIGNAL_BARRIER_PLACEHOLDER);
+		}
+		PG_CATCH();
+		{
+			/*
+			 * If an ERROR occurred, add any flags that weren't yet handled
+			 * back into pss_barrierCheckMask, and reset the global variables
+			 * so that we try again the next time we check for interrupts.
+			 */
+			pg_atomic_fetch_or_u32(&MyProcSignalSlot->pss_barrierCheckMask,
+								   flags);
+			ProcSignalBarrierPending = true;
+			InterruptPending = true;
+
+			PG_RE_THROW();
+		}
+		PG_END_TRY();
+
+		/*
+		 * If some barrier was not successfully absorbed, we will have to try
+		 * again later.
+		 */
+		if (flags != 0)
+		{
+			pg_atomic_fetch_or_u32(&MyProcSignalSlot->pss_barrierCheckMask,
+								   flags);
+			ProcSignalBarrierPending = true;
+			InterruptPending = true;
+			return;
+		}
+	}
 
 	/*
 	 * State changes related to all types of barriers that might have been
@@ -508,7 +554,7 @@ ProcessProcSignalBarrier(void)
 	pg_atomic_write_u64(&MyProcSignalSlot->pss_barrierGeneration, shared_gen);
 }
 
-static void
+static bool
 ProcessBarrierPlaceholder(void)
 {
 	/*
@@ -518,7 +564,12 @@ ProcessBarrierPlaceholder(void)
 	 * appropriately descriptive. Get rid of this function and instead have
 	 * ProcessBarrierSomethingElse. Most likely, that function should live in
 	 * the file pertaining to that subsystem, rather than here.
+	 *
+	 * The return value should be 'true' if the barrier was successfully
+	 * absorbed and 'false' if not. Note that returning 'false' can lead to
+	 * very frequent retries, so try hard to make that an uncommon case.
 	 */
+	return true;
 }
 
 /*
-- 
2.22.0

From 6c508b597893a09af4d287284df281b3b1a11b65 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Tue, 14 Jul 2020 02:30:44 -0400
Subject: [PATCH v6 5/5] Documentation - WIP

TODOs:

1] TBC, the section for pg_is_in_readonly() function, right now it is under "Recovery Information Functions"
2] Documentation regarding ALTER SYSTEM READ/WRITE
---
 src/backend/access/transam/README | 60 ++++++++++++++++++++++++++++---
 src/backend/storage/page/README   | 12 +++----
 2 files changed, 61 insertions(+), 11 deletions(-)

diff --git a/src/backend/access/transam/README b/src/backend/access/transam/README
index 1edc8180c12..fe7a84f93da 100644
--- a/src/backend/access/transam/README
+++ b/src/backend/access/transam/README
@@ -442,8 +442,8 @@ to be modified.
 2. START_CRIT_SECTION()  (Any error during the next three steps must cause a
 PANIC because the shared buffers will contain unlogged changes, which we
 have to ensure don't get to disk.  Obviously, you should check conditions
-such as whether there's enough free space on the page before you start the
-critical section.)
+such as whether there's WAL write permission and enough free space on the page
+before you start the critical section.)
 
 3. Apply the required changes to the shared buffer(s).
 
@@ -486,6 +486,54 @@ with the incomplete-split flag set, it will finish the interrupted split by
 inserting the key to the parent, before proceeding.
 
 
+Read only system state
+----------------------
+
+The system state when it is not currently possible to insert write-ahead log
+records, either because the system is still in recovery or because the system
+forced to WAL prohibited by executing ALTER SYSTEM READ ONLY.  We have a
+lower-level defense in XLogBeginInsert() and elsewhere to stop us from modifying
+data during recovery when !XLogInsertAllowed(), but if XLogBeginInsert() is
+inside the critical section we must not depend on it to report an error.
+Otherwise, it will cause PANIC as mentioned previously.
+
+We do not reach the point where we try to write WAL during recovery but ALTER
+SYSTEM READ ONLY can be executed anytime by the user to stop WAL writing.  Any
+backends which receive read-only system state transition barrier interrupt need
+to stop WAL writing immediately.  For barrier absorption the backed(s) will kill
+the running transaction which has valid XID indicates that the transaction has
+performed and/or planning WAL write.  The transaction which doesn't acquire
+valid XID yet or operation such VACUUM or CONCURRENT CREATE INDEX which not
+necessary have valid XID for WAL will not be prevented while barrier processing,
+and those might hit the error from XLogBeginInsert() while trying to write WAL
+in read only system state.  To prevent such error from XLogBeginInsert() inside
+the critical section the WAL write permission has to check before
+START_CRIT_SECTION().
+
+To enforce the practice to check WAL permission before entering into critical
+section for the WAL write, we have added an assert check flag that indicates
+permission has been checked before calling XLogBeginInsert().  If not,
+XLogBeginInsert() will have assertion failure.  WAL permission check is not
+mandatory if the XLogBeginInsert() is not inside the critical section where
+throwing the error is acceptable.  To get permission check flag set either
+CheckWALPermitted(), AssertWALPermitted_HaveXID(), or AssertWALPermitted()
+should be called before START_CRIT_SECTION().  This flag automatically resets
+while exiting from the critical section.  The rule to place either of permission
+check routines will be:
+
+	The places where WAL write operation in critical can be expected without
+	having valid XID (e.g vacuum) need to protect by CheckWALPermitted(), so
+	that error can be reported outside before critical section.
+
+	The places where INSERT and UPDATE are expected which are never happened
+	without valid XID can be checked using AssertWALPermitted_HaveXID.  So that
+	non-assert build will not have the checking overhead.
+
+	The places we know that we cannot be reached in the read-only state and may
+	or may not have XID, but need to ensure the permission has been checked on
+	assert enabled build should use AssertWALPermitted().
+
+
 Constructing a WAL record
 -------------------------
 
@@ -531,7 +579,8 @@ Details of the API functions:
 
 void XLogBeginInsert(void)
 
-    Must be called before XLogRegisterBuffer and XLogRegisterData.
+	Must be called before XLogRegisterBuffer and XLogRegisterData.  WAL
+	permission must be check before calling it in a critical section.
 
 void XLogResetInsertion(void)
 
@@ -638,8 +687,9 @@ MarkBufferDirtyHint() to mark the block dirty.
 If the buffer is clean and checksums are in use then MarkBufferDirtyHint()
 inserts an XLOG_FPI_FOR_HINT record to ensure that we take a full page image
 that includes the hint. We do this to avoid a partial page write, when we
-write the dirtied page. WAL is not written during recovery, so we simply skip
-dirtying blocks because of hints when in recovery.
+write the dirtied page. WAL is not written while in read only (i.e. during
+recovery or in WAL prohibit state), so we simply skip dirtying blocks because of
+hints when in recovery.
 
 If you do decide to optimise away a WAL record, then any calls to
 MarkBufferDirty() must be replaced by MarkBufferDirtyHint(),
diff --git a/src/backend/storage/page/README b/src/backend/storage/page/README
index e30d7ac59ad..15f0bb4b7b5 100644
--- a/src/backend/storage/page/README
+++ b/src/backend/storage/page/README
@@ -56,9 +56,9 @@ WAL is a fatal error and prevents further recovery, whereas a checksum failure
 on a normal data block is a hard error but not a critical one for the server,
 even if it is a very bad thing for the user.
 
-New WAL records cannot be written during recovery, so hint bits set during
-recovery must not dirty the page if the buffer is not already dirty, when
-checksums are enabled.  Systems in Hot-Standby mode may benefit from hint bits
-being set, but with checksums enabled, a page cannot be dirtied after setting a
-hint bit (due to the torn page risk). So, it must wait for full-page images
-containing the hint bit updates to arrive from the primary.
+New WAL records cannot be written during recovery or or while in WAL prohibit
+state, so hint bits set during recovery must not dirty the page if the buffer is
+not already dirty, when checksums are enabled.  Systems in Hot-Standby mode may
+benefit from hint bits being set, but with checksums enabled, a page cannot be
+dirtied after setting a hint bit (due to the torn page risk). So, it must wait
+for full-page images containing the hint bit updates to arrive from the primary.
-- 
2.22.0

From f59329e4a7285c5b132ca74473fe88e5ba537254 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 19 Jun 2020 06:29:36 -0400
Subject: [PATCH v6 3/5] Implement ALTER SYSTEM READ ONLY using global barrier.

Implementation:

 1. When a user tried to change server state to WAL-Prohibited using
    ALTER SYSTEM READ ONLY command; AlterSystemSetWALProhibitState()
    raises request to checkpointer by marking current state to inprogress in
    shared memory.  Checkpointer, noticing that the current state is has
    WALPROHIBIT_TRANSITION_IN_PROGRESS flag set, does the barrier request, and
    then acknowledges back to the backend who requested the state change once
    the transition has been completed.  Final state will be updated in control
    file to make it persistent across the system restarts.

 2. When a backend receives the WAL-Prohibited barrier, at that moment if
    it is already in a transaction and the transaction already assigned XID,
    then the backend will be killed by throwing FATAL(XXX: need more discussion
    on this)

 3. Otherwise, if that backend running transaction which yet to get XID
    assigned we don't need to do anything special, simply call
    ResetLocalXLogInsertAllowed() so that any future WAL insert in will check
    XLogInsertAllowed() first which set ready only state appropriately.

 4. A new transaction (from existing or new backend) starts as a read-only
    transaction.

 5. Autovacuum launcher as well as checkpointer will don't do anything in
    WAL-Prohibited server state until someone wakes us up.  E.g. a backend
    might later on request us to put the system back to read-write.

 6. At shutdown in WAL-Prohibited mode, we'll skip shutdown checkpoint
    and xlog rotation. Starting up again will perform crash recovery(XXX:
    need some discussion on this as well) but the end of recovery checkpoint
    will be skipped and it will be performed when the system changed to
    WAL-Permitted mode.

 7. ALTER SYSTEM READ ONLY/WRITE is restricted on standby server.

 8. Only super user can toggle WAL-Prohibit state.

 9. Add system_is_read_only GUC show the system state -- will true when system
    is wal prohibited or in recovery.
---
 src/backend/access/transam/Makefile      |   1 +
 src/backend/access/transam/walprohibit.c | 321 +++++++++++++++++++++++
 src/backend/access/transam/xact.c        |  37 +--
 src/backend/access/transam/xlog.c        |  84 +++++-
 src/backend/postmaster/autovacuum.c      |   4 +
 src/backend/postmaster/bgwriter.c        |   2 +-
 src/backend/postmaster/checkpointer.c    |  37 +++
 src/backend/postmaster/pgstat.c          |   3 +
 src/backend/storage/ipc/ipci.c           |   6 +
 src/backend/storage/ipc/procsignal.c     |  26 +-
 src/backend/tcop/utility.c               |  14 +-
 src/backend/utils/misc/guc.c             |  26 ++
 src/bin/pg_controldata/pg_controldata.c  |   2 +
 src/include/access/walprohibit.h         |  83 ++++++
 src/include/access/xlog.h                |   2 +
 src/include/catalog/pg_control.h         |   3 +
 src/include/pgstat.h                     |   1 +
 src/include/postmaster/bgwriter.h        |   2 +
 src/include/storage/procsignal.h         |   7 +-
 src/tools/pgindent/typedefs.list         |   1 +
 20 files changed, 594 insertions(+), 68 deletions(-)
 create mode 100644 src/backend/access/transam/walprohibit.c
 create mode 100644 src/include/access/walprohibit.h

diff --git a/src/backend/access/transam/Makefile b/src/backend/access/transam/Makefile
index 595e02de722..b5322a69954 100644
--- a/src/backend/access/transam/Makefile
+++ b/src/backend/access/transam/Makefile
@@ -26,6 +26,7 @@ OBJS = \
 	twophase.o \
 	twophase_rmgr.o \
 	varsup.o \
+	walprohibit.o \
 	xact.o \
 	xlog.o \
 	xlogarchive.o \
diff --git a/src/backend/access/transam/walprohibit.c b/src/backend/access/transam/walprohibit.c
new file mode 100644
index 00000000000..8ab30be1d51
--- /dev/null
+++ b/src/backend/access/transam/walprohibit.c
@@ -0,0 +1,321 @@
+/*-------------------------------------------------------------------------
+ *
+ * walprohibit.c
+ * 		PostgreSQL write-ahead log prohibit states
+ *
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/backend/access/transam/walprohibit.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/walprohibit.h"
+#include "pgstat.h"
+#include "port/atomics.h"
+#include "postmaster/bgwriter.h"
+#include "storage/condition_variable.h"
+#include "storage/procsignal.h"
+#include "storage/shmem.h"
+
+/*
+ * Shared-memory WAL prohibit state
+ */
+typedef struct WALProhibitStateData
+{
+	/* Indicates current WAL prohibit state */
+	pg_atomic_uint32 SharedWALProhibitState;
+
+	/* Startup checkpoint pending */
+	bool		checkpointPending;
+
+	/* Signaled when requested WAL prohibit state changes */
+	ConditionVariable walprohibit_cv;
+} WALProhibitStateData;
+
+static WALProhibitStateData *WALProhibitState = NULL;
+
+static void RequestWALProhibitChange(void);
+
+/*
+ * ProcessBarrierWALProhibit()
+ *
+ * Handle WAL prohibit state change request.
+ */
+bool
+ProcessBarrierWALProhibit(void)
+{
+	/*
+	 * Kill off any transactions that have an XID *before* allowing the system
+	 * to go WAL prohibit state.
+	 */
+	if (FullTransactionIdIsValid(GetTopFullTransactionIdIfAny()))
+	{
+		/* Should be here only for the WAL prohibit state. */
+		Assert(GetWALProhibitState() & WALPROHIBIT_STATE_READ_ONLY);
+
+		/*
+		 * XXX: Kill off the whole session by throwing FATAL instead of
+		 * killing transaction by throwing ERROR due to following reasons that
+		 * need be thought:
+		 *
+		 * 1. Due to some presents challenges with the wire protocol, we could
+		 * not simply kill of idle transaction.
+		 *
+		 * 2. If we are here in subtransaction then the ERROR will kill the
+		 * current subtransaction only. In the case of invalidations, that
+		 * might be good enough, but for XID assignment it's not, because
+		 * assigning an XID to a subtransaction also causes higher
+		 * sub-transaction levels and the parent transaction to get XIDs.
+		 */
+		ereport(FATAL,
+				(errcode(ERRCODE_ACTIVE_SQL_TRANSACTION),
+				 errmsg("system is now read only"),
+				 errhint("Sessions with open write transactions must be terminated.")));
+	}
+
+	/* Return to "check" state */
+	ResetLocalXLogInsertAllowed();
+
+	return true;
+}
+
+/*
+ * AlterSystemSetWALProhibitState()
+ *
+ * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
+ */
+void
+AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
+{
+	uint32		state;
+
+	if (!superuser())
+		ereport(ERROR,
+				(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+				 errmsg("must be superuser to execute ALTER SYSTEM command")));
+
+	/* Alter WAL prohibit state not allowed during recovery */
+	PreventCommandDuringRecovery("ALTER SYSTEM");
+
+	/* Requested state */
+	state = stmt->WALProhibited ?
+		WALPROHIBIT_STATE_READ_ONLY : WALPROHIBIT_STATE_READ_WRITE;
+
+	/*
+	 * Since we yet to convey this WAL prohibit state to all backend mark it
+	 * in-progress.
+	 */
+	state |= WALPROHIBIT_TRANSITION_IN_PROGRESS;
+
+	if (!SetWALProhibitState(state))
+		return;					/* server is already in the desired state */
+
+	/*
+	 * Signal the checkpointer to do the actual state transition, and wait for
+	 * the state change to occur.
+	 */
+	RequestWALProhibitChange();
+}
+
+/*
+ * RequestWALProhibitChange()
+ *
+ * Request checkpointer to make the WALProhibitState to read-only.
+ */
+static void
+RequestWALProhibitChange(void)
+{
+	/* Must not be called from checkpointer */
+	Assert(!AmCheckpointerProcess());
+	Assert(GetWALProhibitState() & WALPROHIBIT_TRANSITION_IN_PROGRESS);
+
+	/*
+	 * If in a standalone backend, just do it ourselves.
+	 */
+	if (!IsPostmasterEnvironment)
+	{
+		CompleteWALProhibitChange(GetWALProhibitState());
+		return;
+	}
+
+	send_signal_to_checkpointer(SIGINT);
+
+	/* Wait for the state to change to read-only */
+	ConditionVariablePrepareToSleep(&WALProhibitState->walprohibit_cv);
+	for (;;)
+	{
+		/* We'll be done once in-progress flag bit is cleared */
+		if (!(GetWALProhibitState() & WALPROHIBIT_TRANSITION_IN_PROGRESS))
+			break;
+
+		ConditionVariableSleep(&WALProhibitState->walprohibit_cv,
+							   WAIT_EVENT_WALPROHIBIT_STATE_CHANGE);
+	}
+	ConditionVariableCancelSleep();
+}
+
+/*
+ * CompleteWALProhibitChange()
+ *
+ * Checkpointer will call this to complete the requested WAL prohibit state
+ * transition.
+ */
+void
+CompleteWALProhibitChange(uint32 wal_state)
+{
+	uint64		barrierGeneration;
+
+	/*
+	 * Must be called from checkpointer. Otherwise, it must be single-user
+	 * backend.
+	 */
+	Assert(AmCheckpointerProcess() || !IsPostmasterEnvironment);
+	Assert(wal_state & WALPROHIBIT_TRANSITION_IN_PROGRESS);
+
+	/*
+	 * WAL prohibit state change is initiated. We need to complete the state
+	 * transition by setting requested WAL prohibit state in all backends.
+	 */
+	elog(DEBUG1, "waiting for backends to adopt requested WAL prohibit state");
+
+	/* Emit global barrier */
+	barrierGeneration = EmitProcSignalBarrier(PROCSIGNAL_BARRIER_WALPROHIBIT);
+	WaitForProcSignalBarrier(barrierGeneration);
+
+	/* And flush all writes. */
+	XLogFlush(GetXLogWriteRecPtr());
+
+	/* Set final state by clearing in-progress flag bit */
+	if (SetWALProhibitState(wal_state & ~(WALPROHIBIT_TRANSITION_IN_PROGRESS)))
+	{
+		bool		wal_prohibited;
+
+		wal_prohibited = (wal_state & WALPROHIBIT_STATE_READ_ONLY) != 0;
+
+		/* Update the control file to make state persistent */
+		SetControlFileWALProhibitFlag(wal_prohibited);
+
+		if (wal_prohibited)
+			ereport(LOG, (errmsg("system is now read only")));
+		else
+		{
+			/*
+			 * Request checkpoint if the end-of-recovery checkpoint has been
+			 * skipped previously.
+			 */
+			if (WALProhibitState->checkpointPending)
+			{
+				RequestCheckpoint(CHECKPOINT_END_OF_RECOVERY |
+								  CHECKPOINT_IMMEDIATE);
+				WALProhibitState->checkpointPending = false;
+			}
+			ereport(LOG, (errmsg("system is now read write")));
+		}
+	}
+
+	/* Wake up the backend who requested the state change */
+	ConditionVariableBroadcast(&WALProhibitState->walprohibit_cv);
+}
+
+/*
+ * GetWALProhibitState()
+ *
+ * Atomically return the current server WAL prohibited state
+ */
+uint32
+GetWALProhibitState(void)
+{
+	return pg_atomic_read_u32(&WALProhibitState->SharedWALProhibitState);
+}
+
+/*
+ * SetWALProhibitState()
+ *
+ * Change current WAL prohibit state to the input state.
+ *
+ * If the server is already completely moved to the requested WAL prohibit
+ * state, or if the desired state is same as the current state, return false,
+ * indicating that the server state did not change. Else return true.
+ */
+bool
+SetWALProhibitState(uint32 new_state)
+{
+	bool		state_updated = false;
+	uint32		cur_state;
+
+	cur_state = GetWALProhibitState();
+
+	/* Server is already in requested state */
+	if (new_state == cur_state ||
+		new_state == (cur_state | WALPROHIBIT_TRANSITION_IN_PROGRESS))
+		return false;
+
+	/* Prevent concurrent contrary in progress transition state setting */
+	if ((new_state & WALPROHIBIT_TRANSITION_IN_PROGRESS) &&
+		(cur_state & WALPROHIBIT_TRANSITION_IN_PROGRESS))
+	{
+		if (cur_state & WALPROHIBIT_STATE_READ_ONLY)
+			ereport(ERROR,
+					(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+					 errmsg("system state transition to read only is already in progress"),
+					 errhint("Try after sometime again.")));
+		else
+			ereport(ERROR,
+					(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+					 errmsg("system state transition to read write is already in progress"),
+					 errhint("Try after sometime again.")));
+	}
+
+	/* Update new state in share memory */
+	state_updated =
+		pg_atomic_compare_exchange_u32(&WALProhibitState->SharedWALProhibitState,
+									   &cur_state, new_state);
+
+	if (!state_updated)
+		ereport(ERROR,
+				(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+				 errmsg("system read write state concurrently changed"),
+				 errhint("Try after sometime again.")));
+
+	return true;
+}
+
+/*
+ * MarkCheckPointSkippedInWalProhibitState()
+ *
+ * Sets checkpoint pending flag so that it can be performed next time while
+ * changing system state to WAL permitted.
+ */
+void
+MarkCheckPointSkippedInWalProhibitState(void)
+{
+	WALProhibitState->checkpointPending = true;
+}
+
+/*
+ * WALProhibitStateShmemInit()
+ *
+ * Initialization of shared memory for WAL prohibit state.
+ */
+void
+WALProhibitStateShmemInit(void)
+{
+	bool		found;
+
+	WALProhibitState = (WALProhibitStateData *)
+		ShmemInitStruct("WAL Prohibit State",
+						sizeof(WALProhibitStateData),
+						&found);
+
+	if (found)
+		return;
+
+	/* First time through ... */
+	memset(WALProhibitState, 0, sizeof(WALProhibitStateData));
+
+	pg_atomic_init_u32(&WALProhibitState->SharedWALProhibitState, 0);
+	ConditionVariableInit(&WALProhibitState->walprohibit_cv);
+}
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index af6afcebb13..188c299bed9 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -1962,23 +1962,28 @@ StartTransaction(void)
 	Assert(s->prevSecContext == 0);
 
 	/*
-	 * Make sure we've reset xact state variables
+	 * Reset xact state variables.
 	 *
-	 * If recovery is still in progress, mark this transaction as read-only.
-	 * We have lower level defences in XLogInsert and elsewhere to stop us
-	 * from modifying data during recovery, but this gives the normal
-	 * indication to the user that the transaction is read-only.
-	 */
-	if (RecoveryInProgress())
-	{
-		s->startedInRecovery = true;
-		XactReadOnly = true;
-	}
-	else
-	{
-		s->startedInRecovery = false;
-		XactReadOnly = DefaultXactReadOnly;
-	}
+	 * If it is not currently possible to insert write-ahead log records,
+	 * either because we are still in recovery or because ALTER SYSTEM READ
+	 * ONLY has been executed, force this to be a read-only transaction.
+	 * We have lower level defences in XLogBeginInsert() and elsewhere to stop
+	 * us from modifying data during recovery when !XLogInsertAllowed(), but
+	 * this gives the normal indication to the user that the transaction is
+	 * read-only.
+	 *
+	 * On the other hand, we only need to set the startedInRecovery flag when
+	 * the transaction started during recovery, and not when WAL is otherwise
+	 * prohibited. This information is used by RelationGetIndexScan() to
+	 * decide whether to permit (1) relying on existing killed-tuple markings
+	 * and (2) further killing of index tuples. Even when WAL is prohibited
+	 * on the master, it's still the master, so the former is OK; and since
+	 * killing index tuples doesn't generate WAL, the latter is also OK.
+	 * See comments in RelationGetIndexScan() and MarkBufferDirtyHint().
+	 */
+	XactReadOnly = DefaultXactReadOnly || !XLogInsertAllowed();
+	s->startedInRecovery = RecoveryInProgress();
+
 	XactDeferrable = DefaultXactDeferrable;
 	XactIsoLevel = DefaultXactIsoLevel;
 	forceSyncCommit = false;
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 09c01ed4ae4..e170df78a87 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -31,6 +31,7 @@
 #include "access/timeline.h"
 #include "access/transam.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog_internal.h"
 #include "access/xlogarchive.h"
@@ -245,9 +246,10 @@ static bool LocalPromoteIsTriggered = false;
  *		0: unconditionally not allowed to insert XLOG
  *		-1: must check RecoveryInProgress(); disallow until it is false
  * Most processes start with -1 and transition to 1 after seeing that recovery
- * is not in progress.  But we can also force the value for special cases.
- * The coding in XLogInsertAllowed() depends on the first two of these states
- * being numerically the same as bool true and false.
+ * is not in progress or the server state is not a WAL prohibited state.  But
+ * we can also force the value for special cases.  The coding in
+ * XLogInsertAllowed() depends on the first two of these states being
+ * numerically the same as bool true and false.
  */
 static int	LocalXLogInsertAllowed = -1;
 
@@ -967,6 +969,7 @@ static void WALInsertLockAcquire(void);
 static void WALInsertLockAcquireExclusive(void);
 static void WALInsertLockRelease(void);
 static void WALInsertLockUpdateInsertingAt(XLogRecPtr insertingAt);
+static inline bool IsWALProhibited(void);
 
 /*
  * Insert an XLOG record represented by an already-constructed chain of data
@@ -7703,6 +7706,14 @@ StartupXLOG(void)
 	XLogCtl->LogwrtRqst.Write = EndOfLog;
 	XLogCtl->LogwrtRqst.Flush = EndOfLog;
 
+	/*
+	 * Before enabling WAL insertion update WAL prohibit state in shared memory
+	 * that will decide the further WAL insert should be allowed or not.
+	 */
+	(void) SetWALProhibitState(ControlFile->wal_prohibited ?
+							   WALPROHIBIT_STATE_READ_ONLY :
+							   WALPROHIBIT_STATE_READ_WRITE);
+
 	/*
 	 * Update full_page_writes in shared memory and write an XLOG_FPW_CHANGE
 	 * record before resource manager writes cleanup WAL records or checkpoint
@@ -7713,7 +7724,16 @@ StartupXLOG(void)
 	UpdateFullPageWrites();
 	LocalXLogInsertAllowed = -1;
 
-	if (InRecovery)
+	/*
+	 * Skip end-of-recovery checkpoint if the system is in WAL prohibited state.
+	 */
+	if (ControlFile->wal_prohibited && InRecovery)
+	{
+		MarkCheckPointSkippedInWalProhibitState();
+		ereport(LOG,
+				(errmsg("skipping startup checkpoint because the system is read only")));
+	}
+	else if (InRecovery)
 	{
 		/*
 		 * Perform a checkpoint to update all our recovery activity to disk.
@@ -7959,6 +7979,25 @@ StartupXLOG(void)
 		RequestCheckpoint(CHECKPOINT_FORCE);
 }
 
+/* Set ControlFile's WAL prohibit flag */
+void
+SetControlFileWALProhibitFlag(bool wal_prohibited)
+{
+	LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+	ControlFile->wal_prohibited = wal_prohibited;
+	UpdateControlFile();
+	LWLockRelease(ControlFileLock);
+}
+
+/*
+ * Is the system still in WAL prohibited state?
+ */
+static inline bool
+IsWALProhibited(void)
+{
+	return (GetWALProhibitState() & WALPROHIBIT_STATE_READ_ONLY) != 0;
+}
+
 /*
  * Checks if recovery has reached a consistent state. When consistency is
  * reached and we have a valid starting standby snapshot, tell postmaster
@@ -8174,9 +8213,9 @@ HotStandbyActiveInReplay(void)
 /*
  * Is this process allowed to insert new WAL records?
  *
- * Ordinarily this is essentially equivalent to !RecoveryInProgress().
- * But we also have provisions for forcing the result "true" or "false"
- * within specific processes regardless of the global state.
+ * Ordinarily this is essentially equivalent to !RecoveryInProgress() and
+ * !IsWALProhibited().  But we also have provisions for forcing the result
+ * "true" or "false" within specific processes regardless of the global state.
  */
 bool
 XLogInsertAllowed(void)
@@ -8195,9 +8234,20 @@ XLogInsertAllowed(void)
 	if (RecoveryInProgress())
 		return false;
 
+	/* Or, in WAL prohibited state */
+	if (IsWALProhibited())
+	{
+		/*
+		 * Set it to "unconditionally false" to avoid checking until it gets
+		 * reset.
+		 */
+		LocalXLogInsertAllowed = 0;
+		return false;
+	}
+
 	/*
-	 * On exit from recovery, reset to "unconditionally true", since there is
-	 * no need to keep checking.
+	 * On exit from recovery or WAL prohibited state, reset to "unconditionally
+	 * true", since there is no need to keep checking.
 	 */
 	LocalXLogInsertAllowed = 1;
 	return true;
@@ -8219,6 +8269,12 @@ LocalSetXLogInsertAllowed(void)
 	InitXLOGAccess();
 }
 
+void
+ResetLocalXLogInsertAllowed(void)
+{
+	LocalXLogInsertAllowed = -1;
+}
+
 /*
  * Subroutine to try to fetch and validate a prior checkpoint record.
  *
@@ -8508,9 +8564,13 @@ ShutdownXLOG(int code, Datum arg)
 	 */
 	WalSndWaitStopping();
 
+	/*
+	 * The restartpoint, checkpoint, or xlog rotation will be performed if the
+	 * WAL writing is permitted.
+	 */
 	if (RecoveryInProgress())
 		CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
-	else
+	else if (XLogInsertAllowed())
 	{
 		/*
 		 * If archiving is enabled, rotate the last XLOG file so that all the
@@ -8523,6 +8583,10 @@ ShutdownXLOG(int code, Datum arg)
 
 		CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
 	}
+	else
+		ereport(LOG,
+			   (errmsg("skipping shutdown checkpoint because the system is read only")));
+
 	ShutdownCLOG();
 	ShutdownCommitTs();
 	ShutdownSUBTRANS();
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 1b8cd7bacd4..aa4cdd57ec1 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -652,6 +652,10 @@ AutoVacLauncherMain(int argc, char *argv[])
 
 		HandleAutoVacLauncherInterrupts();
 
+		/* If the server is read only just go back to sleep. */
+		if (!XLogInsertAllowed())
+			continue;
+
 		/*
 		 * a worker finished, or postmaster signaled failure to start a worker
 		 */
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index 069e27e427f..6c6ff7dc3af 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -268,7 +268,7 @@ BackgroundWriterMain(void)
 		 * Checkpointer, when active, is barely ever in its mainloop and thus
 		 * makes it hard to log regularly.
 		 */
-		if (XLogStandbyInfoActive() && !RecoveryInProgress())
+		if (XLogStandbyInfoActive() && XLogInsertAllowed())
 		{
 			TimestampTz timeout = 0;
 			TimestampTz now = GetCurrentTimestamp();
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b80..b841721c9ec 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -39,6 +39,7 @@
 #include <sys/time.h>
 #include <time.h>
 
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xlog_internal.h"
 #include "libpq/pqsignal.h"
@@ -332,6 +333,7 @@ CheckpointerMain(void)
 		pg_time_t	now;
 		int			elapsed_secs;
 		int			cur_timeout;
+		uint32		wal_state;
 
 		/* Clear any already-pending wakeups */
 		ResetLatch(MyLatch);
@@ -342,6 +344,28 @@ CheckpointerMain(void)
 		AbsorbSyncRequests();
 		HandleCheckpointerInterrupts();
 
+		wal_state = GetWALProhibitState();
+
+		if (wal_state & WALPROHIBIT_TRANSITION_IN_PROGRESS)
+		{
+			/* Complete WAL prohibit state change request */
+			CompleteWALProhibitChange(wal_state);
+			continue;
+		}
+		else if (wal_state & WALPROHIBIT_STATE_READ_ONLY)
+		{
+			/*
+			 * Don't do anything until someone wakes us up.  For example a
+			 * backend might later on request us to put the system back to
+			 * read-write wal prohibit sate.
+			 */
+			(void) WaitLatch(MyLatch, WL_LATCH_SET | WL_EXIT_ON_PM_DEATH, -1,
+							 WAIT_EVENT_CHECKPOINTER_MAIN);
+			continue;
+		}
+
+		Assert(wal_state == WALPROHIBIT_STATE_READ_WRITE);
+
 		/*
 		 * Detect a pending checkpoint request by checking whether the flags
 		 * word in shared memory is nonzero.  We shouldn't need to acquire the
@@ -1323,3 +1347,16 @@ FirstCallSinceLastCheckpoint(void)
 
 	return FirstCall;
 }
+
+/*
+ * send_signal_to_checkpointer allows a process to send a signal to the checkpoint process.
+ */
+void
+send_signal_to_checkpointer(int signum)
+{
+	if (CheckpointerShmem->checkpointer_pid == 0)
+		elog(ERROR, "checkpointer is not running");
+
+	if (kill(CheckpointerShmem->checkpointer_pid, signum) != 0)
+		elog(ERROR, "could not signal checkpointer: %m");
+}
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 8116b236143..03b63a0d05f 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -4060,6 +4060,9 @@ pgstat_get_wait_io(WaitEventIO w)
 		case WAIT_EVENT_REPLICATION_SLOT_WRITE:
 			event_name = "ReplicationSlotWrite";
 			break;
+		case WAIT_EVENT_WALPROHIBIT_STATE_CHANGE:
+			event_name = "SystemWALProhibitStateChange";
+			break;
 		case WAIT_EVENT_SLRU_FLUSH_SYNC:
 			event_name = "SLRUFlushSync";
 			break;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 96c2aaabbd6..2d000ec2ff7 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -22,6 +22,7 @@
 #include "access/subtrans.h"
 #include "access/syncscan.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "commands/async.h"
 #include "miscadmin.h"
 #include "pgstat.h"
@@ -222,6 +223,11 @@ CreateSharedMemoryAndSemaphores(void)
 	MultiXactShmemInit();
 	InitBufferPool();
 
+	/*
+	 * Set up wal probibit shared state
+	 */
+	WALProhibitStateShmemInit();
+
 	/*
 	 * Set up lock manager
 	 */
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 13648887187..b973727a580 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -18,6 +18,7 @@
 #include <unistd.h>
 
 #include "access/parallel.h"
+#include "access/walprohibit.h"
 #include "commands/async.h"
 #include "miscadmin.h"
 #include "pgstat.h"
@@ -96,7 +97,6 @@ static volatile ProcSignalSlot *MyProcSignalSlot = NULL;
 
 static bool CheckProcSignal(ProcSignalReason reason);
 static void CleanupProcSignalState(int status, Datum arg);
-static bool ProcessBarrierPlaceholder(void);
 
 /*
  * ProcSignalShmemSize
@@ -510,9 +510,9 @@ ProcessProcSignalBarrier(void)
 			 * unconditionally, but it's more efficient to call only the ones
 			 * that might need us to do something based on the flags.
 			 */
-			if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_PLACEHOLDER)
-				&& ProcessBarrierPlaceholder())
-				BARRIER_CLEAR_BIT(flags, PROCSIGNAL_BARRIER_PLACEHOLDER);
+			if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_WALPROHIBIT)
+				&& ProcessBarrierWALProhibit())
+				BARRIER_CLEAR_BIT(flags, PROCSIGNAL_BARRIER_WALPROHIBIT);
 		}
 		PG_CATCH();
 		{
@@ -554,24 +554,6 @@ ProcessProcSignalBarrier(void)
 	pg_atomic_write_u64(&MyProcSignalSlot->pss_barrierGeneration, shared_gen);
 }
 
-static bool
-ProcessBarrierPlaceholder(void)
-{
-	/*
-	 * XXX. This is just a placeholder until the first real user of this
-	 * machinery gets committed. Rename PROCSIGNAL_BARRIER_PLACEHOLDER to
-	 * PROCSIGNAL_BARRIER_SOMETHING_ELSE where SOMETHING_ELSE is something
-	 * appropriately descriptive. Get rid of this function and instead have
-	 * ProcessBarrierSomethingElse. Most likely, that function should live in
-	 * the file pertaining to that subsystem, rather than here.
-	 *
-	 * The return value should be 'true' if the barrier was successfully
-	 * absorbed and 'false' if not. Note that returning 'false' can lead to
-	 * very frequent retries, so try hard to make that an uncommon case.
-	 */
-	return true;
-}
-
 /*
  * CheckProcSignal - check to see if a particular reason has been
  * signaled, and clear the signal flag.  Should be called after receiving
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 1730a5402f7..513a7f324fb 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -19,6 +19,7 @@
 #include "access/htup_details.h"
 #include "access/reloptions.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "catalog/catalog.h"
@@ -85,7 +86,6 @@ static void ProcessUtilitySlow(ParseState *pstate,
 							   DestReceiver *dest,
 							   QueryCompletion *qc);
 static void ExecDropStmt(DropStmt *stmt, bool isTopLevel);
-static void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
 
 /*
  * CommandIsReadOnly: is an executable query read-only?
@@ -3678,15 +3678,3 @@ GetCommandLogLevel(Node *parsetree)
 
 	return lev;
 }
-
-/*
- * AlterSystemSetWALProhibitState
- *
- * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
- */
-static void
-AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
-{
-	/* some code */
-	elog(INFO, "AlterSystemSetWALProhibitState() called");
-}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index de87ad6ef70..774628b3cef 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -225,6 +225,7 @@ static bool check_recovery_target_lsn(char **newval, void **extra, GucSource sou
 static void assign_recovery_target_lsn(const char *newval, void *extra);
 static bool check_primary_slot_name(char **newval, void **extra, GucSource source);
 static bool check_default_with_oids(bool *newval, void **extra, GucSource source);
+static const char *show_system_is_read_only(void);
 
 /* Private functions in guc-file.l that need to be called from guc.c */
 static ConfigVariable *ProcessConfigFileInternal(GucContext context,
@@ -615,6 +616,7 @@ static char *recovery_target_string;
 static char *recovery_target_xid_string;
 static char *recovery_target_name_string;
 static char *recovery_target_lsn_string;
+static bool system_is_read_only;
 
 
 /* should be static, but commands/variable.c needs to get at this */
@@ -2036,6 +2038,18 @@ static struct config_bool ConfigureNamesBool[] =
 		NULL, NULL, NULL
 	},
 
+	{
+		/* Not for general use */
+		{"system_is_read_only", PGC_INTERNAL, WAL,
+			gettext_noop("Shows whether the system is read only."),
+			NULL,
+			GUC_NO_RESET_ALL | GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE
+		},
+		&system_is_read_only,
+		false,
+		NULL, NULL, show_system_is_read_only
+	},
+
 	/* End-of-list marker */
 	{
 		{NULL, 0, 0, NULL, NULL}, NULL, false, NULL, NULL, NULL
@@ -12041,4 +12055,16 @@ check_default_with_oids(bool *newval, void **extra, GucSource source)
 	return true;
 }
 
+/*
+ * NB: The return string should be the same as the _ShowOption() for boolean
+ * type.
+ */
+static const char *
+show_system_is_read_only(void)
+{
+	if (!XLogInsertAllowed())
+		return "on";
+	return "off";
+}
+
 #include "guc-file.c"
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index 3e00ac0f701..922cd9641d8 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -290,6 +290,8 @@ main(int argc, char *argv[])
 		   (uint32) ControlFile->backupEndPoint);
 	printf(_("End-of-backup record required:        %s\n"),
 		   ControlFile->backupEndRequired ? _("yes") : _("no"));
+	printf(_("WAL write prohibited:                 %s\n"),
+		   ControlFile->wal_prohibited ? _("yes") : _("no"));
 	printf(_("wal_level setting:                    %s\n"),
 		   wal_level_str(ControlFile->wal_level));
 	printf(_("wal_log_hints setting:                %s\n"),
diff --git a/src/include/access/walprohibit.h b/src/include/access/walprohibit.h
new file mode 100644
index 00000000000..14e1f5b3a2e
--- /dev/null
+++ b/src/include/access/walprohibit.h
@@ -0,0 +1,83 @@
+/*
+ * walprohibit.h
+ *
+ * PostgreSQL write-ahead log prohibit states
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/include/access/walprohibit.h
+ */
+#ifndef WALPROHIBIT_H
+#define WALPROHIBIT_H
+
+#include "access/xact.h"
+#include "access/xlog.h"
+#include "miscadmin.h"
+#include "nodes/parsenodes.h"
+
+extern bool ProcessBarrierWALProhibit(void);
+extern void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
+extern void CompleteWALProhibitChange(uint32 wal_state);
+extern uint32 GetWALProhibitState(void);
+extern bool SetWALProhibitState(uint32 new_state);
+extern void MarkCheckPointSkippedInWalProhibitState(void);
+extern void WALProhibitStateShmemInit(void);
+
+/* WAL Prohibit States */
+#define	WALPROHIBIT_STATE_READ_WRITE		0x0000
+#define	WALPROHIBIT_STATE_READ_ONLY			0x0001
+
+/*
+ * The bit is used in state transition from one state to another.  When this
+ * bit is set then the state indicated by the 0th position bit is yet to
+ * confirmed.
+ */
+#define WALPROHIBIT_TRANSITION_IN_PROGRESS	0x0002
+
+/* Never reaches when WAL is prohibited. */
+static inline void
+AssertWALPermitted(void)
+{
+	/*
+	 * Recovery in the startup process never is in wal prohibited state.
+	 */
+	Assert(InRecovery || XLogInsertAllowed());
+
+#ifdef USE_ASSERT_CHECKING
+	walpermit_checked_state = WALPERMIT_CHECKED;
+#endif
+}
+
+/*
+ * XID-bearing transactions are killed off by "ALTER SYSTEM READ ONLY", so any
+ * part of the code that can only be reached with an XID assigned is never
+ * reached when WAL is prohibited.
+ */
+static inline void
+AssertWALPermittedHaveXID(void)
+{
+	/* Must be performing an INSERT, UPDATE or DELETE, so we'll have an XID */
+	Assert(FullTransactionIdIsValid(GetTopFullTransactionIdIfAny()));
+	AssertWALPermitted();
+}
+
+/*
+ * In opposite to the above assertion if a transaction doesn't have valid XID
+ * (e.g. VACUUM) then it won't be killed while changing the system state to WAL
+ * prohibited.  Therefore, we need to explicitly error out before entering into
+ * the critical section.
+ */
+static inline void
+CheckWALPermitted(void)
+{
+	if (!XLogInsertAllowed())
+		ereport(ERROR,
+				(errcode(ERRCODE_READ_ONLY_SQL_TRANSACTION),
+				 errmsg("system is now read only")));
+
+#ifdef USE_ASSERT_CHECKING
+	walpermit_checked_state = WALPERMIT_CHECKED;
+#endif
+}
+
+#endif							/* WALPROHIBIT_H */
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 221af87e715..2c423d6b609 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -306,6 +306,7 @@ extern RecoveryState GetRecoveryState(void);
 extern bool HotStandbyActive(void);
 extern bool HotStandbyActiveInReplay(void);
 extern bool XLogInsertAllowed(void);
+extern void ResetLocalXLogInsertAllowed(void);
 extern void GetXLogReceiptTime(TimestampTz *rtime, bool *fromStream);
 extern XLogRecPtr GetXLogReplayRecPtr(TimeLineID *replayTLI);
 extern XLogRecPtr GetXLogInsertRecPtr(void);
@@ -325,6 +326,7 @@ extern void XLOGShmemInit(void);
 extern void BootStrapXLOG(void);
 extern void LocalProcessControlFile(bool reset);
 extern void StartupXLOG(void);
+extern void SetControlFileWALProhibitFlag(bool wal_prohibited);
 extern void ShutdownXLOG(int code, Datum arg);
 extern void InitXLOGAccess(void);
 extern void CreateCheckPoint(int flags);
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index 06bed90c5e9..f4dc5412ee6 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -182,6 +182,9 @@ typedef struct ControlFileData
 	int			max_locks_per_xact;
 	bool		track_commit_timestamp;
 
+	/* WAL prohibited determines if the WAL insert is allowed or not. */
+	bool		wal_prohibited;
+
 	/*
 	 * This data is used to check for hardware-architecture compatibility of
 	 * the database and the backend executable.  We need not check endianness
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 807a9c1edf6..76f504ee277 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -956,6 +956,7 @@ typedef enum
 	WAIT_EVENT_REPLICATION_SLOT_RESTORE_SYNC,
 	WAIT_EVENT_REPLICATION_SLOT_SYNC,
 	WAIT_EVENT_REPLICATION_SLOT_WRITE,
+	WAIT_EVENT_WALPROHIBIT_STATE_CHANGE,
 	WAIT_EVENT_SLRU_FLUSH_SYNC,
 	WAIT_EVENT_SLRU_READ,
 	WAIT_EVENT_SLRU_SYNC,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e6..d72aa4c9fa0 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -42,4 +42,6 @@ extern void CheckpointerShmemInit(void);
 
 extern bool FirstCallSinceLastCheckpoint(void);
 
+extern void send_signal_to_checkpointer(int signum);
+
 #endif							/* _BGWRITER_H */
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f38..bae06202b4a 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -48,12 +48,7 @@ typedef enum
 
 typedef enum
 {
-	/*
-	 * XXX. PROCSIGNAL_BARRIER_PLACEHOLDER should be replaced when the first
-	 * real user of the ProcSignalBarrier mechanism is added. It's just here
-	 * for now because we can't have an empty enum.
-	 */
-	PROCSIGNAL_BARRIER_PLACEHOLDER = 0
+	PROCSIGNAL_BARRIER_WALPROHIBIT = 0
 } ProcSignalBarrierType;
 
 /*
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 29d6f6c968d..4411111a78e 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2665,6 +2665,7 @@ WALAvailability
 WALInsertLock
 WALInsertLockPadded
 WALOpenSegment
+WALProhibitStateData
 WALReadError
 WALSegmentCloseCB
 WALSegmentContext
-- 
2.22.0

From 9242327f51d45f2cdbe1b9ece99ed98366d1cbe0 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Mon, 27 Jul 2020 02:13:36 -0400
Subject: [PATCH v6 4/5] Error or Assert before START_CRIT_SECTION for WAL
 write

Based on the following criteria added the Assert or the Error when system is
prohibited:

 - Added ERROR for the function which can be reachable without valid XID in case
   of VACUUM or CONCURRENT CREATE INDEX.  For that added common static inline
   function CheckWALPermitted().
 - Added ASSERT for the function which cannot be reached without valid XID;
   Assert to ensure XID validation. For that added AssertWALPermitted_HaveXID().

To enforce the rule to have aforesaid assert or error check before entering a
critical section for WAL write, a new walpermit_checked_state assert only flag
is added. If these check is missing then XLogBeginInsert() will have an
assertion if it is in critical section.

If we are not doing WAL insert inside the critical section then the above
checking is not necessary, we can rely on XLogBeginInsert() for that check &
report an error.
---
 src/backend/access/brin/brin.c            |  4 +++
 src/backend/access/brin/brin_pageops.c    | 21 ++++++++++--
 src/backend/access/brin/brin_revmap.c     | 10 +++++-
 src/backend/access/gin/ginbtree.c         | 15 +++++++--
 src/backend/access/gin/gindatapage.c      | 18 +++++++++--
 src/backend/access/gin/ginfast.c          | 11 +++++--
 src/backend/access/gin/gininsert.c        |  4 +++
 src/backend/access/gin/ginutil.c          |  9 +++++-
 src/backend/access/gin/ginvacuum.c        | 11 ++++++-
 src/backend/access/gist/gist.c            | 25 ++++++++++++---
 src/backend/access/gist/gistvacuum.c      | 13 ++++++--
 src/backend/access/hash/hash.c            | 19 +++++++++--
 src/backend/access/hash/hashinsert.c      |  9 +++++-
 src/backend/access/hash/hashovfl.c        | 22 ++++++++++---
 src/backend/access/hash/hashpage.c        |  9 ++++++
 src/backend/access/heap/heapam.c          | 26 ++++++++++++++-
 src/backend/access/heap/pruneheap.c       | 12 +++++--
 src/backend/access/heap/vacuumlazy.c      | 18 +++++++++--
 src/backend/access/heap/visibilitymap.c   | 22 +++++++++++--
 src/backend/access/nbtree/nbtdedup.c      |  3 ++
 src/backend/access/nbtree/nbtinsert.c     | 10 +++++-
 src/backend/access/nbtree/nbtpage.c       | 39 +++++++++++++++++++----
 src/backend/access/spgist/spgdoinsert.c   | 13 ++++++++
 src/backend/access/spgist/spgvacuum.c     | 22 +++++++++++--
 src/backend/access/transam/multixact.c    |  5 ++-
 src/backend/access/transam/twophase.c     |  9 ++++++
 src/backend/access/transam/varsup.c       |  4 +++
 src/backend/access/transam/walprohibit.c  | 10 ++++++
 src/backend/access/transam/xact.c         |  6 ++++
 src/backend/access/transam/xlog.c         | 27 ++++++++++++----
 src/backend/access/transam/xloginsert.c   | 13 ++++++--
 src/backend/commands/sequence.c           | 16 ++++++++++
 src/backend/commands/variable.c           |  9 ++++--
 src/backend/postmaster/checkpointer.c     |  4 +++
 src/backend/storage/buffer/bufmgr.c       | 10 +++---
 src/backend/storage/freespace/freespace.c | 10 +++++-
 src/backend/storage/lmgr/lock.c           |  6 ++--
 src/backend/utils/cache/relmapper.c       |  3 ++
 src/include/access/walprohibit.h          |  4 +--
 src/include/miscadmin.h                   | 27 ++++++++++++++++
 40 files changed, 457 insertions(+), 71 deletions(-)

diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index 1f72562c603..47142193706 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -24,6 +24,7 @@
 #include "access/relscan.h"
 #include "access/table.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/index.h"
 #include "catalog/pg_am.h"
@@ -759,6 +760,9 @@ brinbuildempty(Relation index)
 		ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermittedHaveXID();
+
 	/* Initialize and xlog metabuffer. */
 	START_CRIT_SECTION();
 	brin_metapage_init(BufferGetPage(metabuf), BrinGetPagesPerRange(index),
diff --git a/src/backend/access/brin/brin_pageops.c b/src/backend/access/brin/brin_pageops.c
index 87de0b855b5..8b377a679ab 100644
--- a/src/backend/access/brin/brin_pageops.c
+++ b/src/backend/access/brin/brin_pageops.c
@@ -14,6 +14,7 @@
 #include "access/brin_pageops.h"
 #include "access/brin_revmap.h"
 #include "access/brin_xlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -65,6 +66,7 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 	Buffer		newbuf;
 	BlockNumber newblk = InvalidBlockNumber;
 	bool		extended;
+	bool		needwal = RelationNeedsWAL(idxrel);
 
 	Assert(newsz == MAXALIGN(newsz));
 
@@ -176,13 +178,16 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 	if (((BrinPageFlags(oldpage) & BRIN_EVACUATE_PAGE) == 0) &&
 		brin_can_do_samepage_update(oldbuf, origsz, newsz))
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 		if (!PageIndexTupleOverwrite(oldpage, oldoff, (Item) unconstify(BrinTuple *, newtup), newsz))
 			elog(ERROR, "failed to replace BRIN tuple");
 		MarkBufferDirty(oldbuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(idxrel))
+		if (needwal)
 		{
 			xl_brin_samepage_update xlrec;
 			XLogRecPtr	recptr;
@@ -240,6 +245,9 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 
 		revmapbuf = brinLockRevmapPageForUpdate(revmap, heapBlk);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		/*
@@ -267,7 +275,7 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 		MarkBufferDirty(revmapbuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(idxrel))
+		if (needwal)
 		{
 			xl_brin_update xlrec;
 			XLogRecPtr	recptr;
@@ -351,6 +359,7 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	Buffer		revmapbuf;
 	ItemPointerData tid;
 	bool		extended;
+	bool		needwal;
 
 	Assert(itemsz == MAXALIGN(itemsz));
 
@@ -405,6 +414,10 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	page = BufferGetPage(*buffer);
 	blk = BufferGetBlockNumber(*buffer);
 
+	needwal = RelationNeedsWAL(idxrel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Execute the actual insertion */
 	START_CRIT_SECTION();
 	if (extended)
@@ -424,7 +437,7 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	MarkBufferDirty(revmapbuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(idxrel))
+	if (needwal)
 	{
 		xl_brin_insert xlrec;
 		XLogRecPtr	recptr;
@@ -881,6 +894,8 @@ brin_initialize_empty_new_buffer(Relation idxrel, Buffer buffer)
 			   "brin_initialize_empty_new_buffer: initializing blank page %u",
 			   BufferGetBlockNumber(buffer)));
 
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 	page = BufferGetPage(buffer);
 	brin_page_init(page, BRIN_PAGETYPE_REGULAR);
diff --git a/src/backend/access/brin/brin_revmap.c b/src/backend/access/brin/brin_revmap.c
index 35746714a7c..fd766da445d 100644
--- a/src/backend/access/brin/brin_revmap.c
+++ b/src/backend/access/brin/brin_revmap.c
@@ -26,6 +26,7 @@
 #include "access/brin_tuple.h"
 #include "access/brin_xlog.h"
 #include "access/rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -340,6 +341,7 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 	OffsetNumber revmapOffset;
 	OffsetNumber regOffset;
 	ItemId		lp;
+	bool		needwal;
 
 	revmap = brinRevmapInitialize(idxrel, &pagesPerRange, NULL);
 
@@ -404,6 +406,10 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 	 * crashed or aborted summarization; remove them silently.
 	 */
 
+	needwal = RelationNeedsWAL(idxrel);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	ItemPointerSetInvalid(&invalidIptr);
@@ -415,7 +421,7 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 	MarkBufferDirty(regBuf);
 	MarkBufferDirty(revmapBuf);
 
-	if (RelationNeedsWAL(idxrel))
+	if (needwal)
 	{
 		xl_brin_desummarize xlrec;
 		XLogRecPtr	recptr;
@@ -613,6 +619,8 @@ revmap_physical_extend(BrinRevmap *revmap)
 		return;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Ok, we have now locked the metapage and the target block. Re-initialize
 	 * the target block as a revmap page, and update the metapage.
diff --git a/src/backend/access/gin/ginbtree.c b/src/backend/access/gin/ginbtree.c
index 8d08b05f515..0d9997463b4 100644
--- a/src/backend/access/gin/ginbtree.c
+++ b/src/backend/access/gin/ginbtree.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/predicate.h"
@@ -333,6 +334,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 {
 	Page		page = BufferGetPage(stack->buffer);
 	bool		result;
+	bool		needwal;
 	GinPlaceToPageRC rc;
 	uint16		xlflags = 0;
 	Page		childpage = NULL;
@@ -378,6 +380,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 								 insertdata, updateblkno,
 								 &ptp_workspace,
 								 &newlpage, &newrpage);
+	needwal = RelationNeedsWAL(btree->index) && !btree->isBuild;
 
 	if (rc == GPTP_NO_WORK)
 	{
@@ -386,10 +389,13 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 	}
 	else if (rc == GPTP_INSERT)
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		/* It will fit, perform the insertion */
 		START_CRIT_SECTION();
 
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogBeginInsert();
 			XLogRegisterBuffer(0, stack->buffer, REGBUF_STANDARD);
@@ -410,7 +416,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 			MarkBufferDirty(childbuf);
 		}
 
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 			ginxlogInsert xlrec;
@@ -548,6 +554,9 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 			}
 		}
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/*
 		 * OK, we have the new contents of the left page in a temporary copy
 		 * now (newlpage), and likewise for the new contents of the
@@ -588,7 +597,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 		}
 
 		/* write WAL record */
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gin/gindatapage.c b/src/backend/access/gin/gindatapage.c
index 7a2690e97f2..0abc5990100 100644
--- a/src/backend/access/gin/gindatapage.c
+++ b/src/backend/access/gin/gindatapage.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "lib/ilist.h"
 #include "miscadmin.h"
@@ -811,6 +812,7 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 	if (removedsomething)
 	{
 		bool		modified;
+		bool		needwal;
 
 		/*
 		 * Make sure we have a palloc'd copy of all segments, after the first
@@ -835,8 +837,12 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 			}
 		}
 
-		if (RelationNeedsWAL(indexrel))
+		needwal = RelationNeedsWAL(indexrel);
+		if (needwal)
+		{
+			CheckWALPermitted();
 			computeLeafRecompressWALData(leaf);
+		}
 
 		/* Apply changes to page */
 		START_CRIT_SECTION();
@@ -845,7 +851,7 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 
 		MarkBufferDirty(buffer);
 
-		if (RelationNeedsWAL(indexrel))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
@@ -1777,6 +1783,7 @@ createPostingTree(Relation index, ItemPointerData *items, uint32 nitems,
 	int			nrootitems;
 	int			rootsize;
 	bool		is_build = (buildStats != NULL);
+	bool		needwal;
 
 	/* Construct the new root page in memory first. */
 	tmppage = (Page) palloc(BLCKSZ);
@@ -1825,12 +1832,17 @@ createPostingTree(Relation index, ItemPointerData *items, uint32 nitems,
 	 */
 	PredicateLockPageSplit(index, BufferGetBlockNumber(entrybuffer), blkno);
 
+	needwal = RelationNeedsWAL(index) && !is_build;
+
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	PageRestoreTempPage(tmppage, page);
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index) && !is_build)
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogCreatePostingTree data;
diff --git a/src/backend/access/gin/ginfast.c b/src/backend/access/gin/ginfast.c
index 2e41b34d8d5..b8c2a993408 100644
--- a/src/backend/access/gin/ginfast.c
+++ b/src/backend/access/gin/ginfast.c
@@ -20,6 +20,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
 #include "catalog/pg_am.h"
@@ -68,6 +69,8 @@ writeListPage(Relation index, Buffer buffer,
 	PGAlignedBlock workspace;
 	char	   *ptr;
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	GinInitBuffer(buffer, GIN_LIST);
@@ -548,6 +551,7 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 	Page		metapage;
 	GinMetaPageData *metadata;
 	BlockNumber blknoToDelete;
+	bool		needwal = RelationNeedsWAL(index);
 
 	metapage = BufferGetPage(metabuffer);
 	metadata = GinPageGetMeta(metapage);
@@ -586,8 +590,11 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 		 * prepare the XLogInsert machinery for that before entering the
 		 * critical section.
 		 */
-		if (RelationNeedsWAL(index))
+		if (needwal)
+		{
+			CheckWALPermitted();
 			XLogEnsureRecordSpace(data.ndeleted, 0);
+		}
 
 		START_CRIT_SECTION();
 
@@ -625,7 +632,7 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 			MarkBufferDirty(buffers[i]);
 		}
 
-		if (RelationNeedsWAL(index))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gin/gininsert.c b/src/backend/access/gin/gininsert.c
index 77433dc8a41..989d82ffcaf 100644
--- a/src/backend/access/gin/gininsert.c
+++ b/src/backend/access/gin/gininsert.c
@@ -17,6 +17,7 @@
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/index.h"
 #include "miscadmin.h"
@@ -447,6 +448,9 @@ ginbuildempty(Relation index)
 		ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(RootBuffer, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermittedHaveXID();
+
 	/* Initialize and xlog metabuffer and root buffer. */
 	START_CRIT_SECTION();
 	GinInitMetabuffer(MetaBuffer);
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index ef9b56fd363..b48ea1a746a 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -17,6 +17,7 @@
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
 #include "access/reloptions.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/pg_collation.h"
 #include "catalog/pg_type.h"
@@ -659,12 +660,18 @@ ginUpdateStats(Relation index, const GinStatsData *stats, bool is_build)
 	Buffer		metabuffer;
 	Page		metapage;
 	GinMetaPageData *metadata;
+	bool		needwal;
 
 	metabuffer = ReadBuffer(index, GIN_METAPAGE_BLKNO);
 	LockBuffer(metabuffer, GIN_EXCLUSIVE);
 	metapage = BufferGetPage(metabuffer);
 	metadata = GinPageGetMeta(metapage);
 
+	needwal = RelationNeedsWAL(index) && !is_build;
+
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	metadata->nTotalPages = stats->nTotalPages;
@@ -684,7 +691,7 @@ ginUpdateStats(Relation index, const GinStatsData *stats, bool is_build)
 
 	MarkBufferDirty(metabuffer);
 
-	if (RelationNeedsWAL(index) && !is_build)
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogUpdateMeta data;
diff --git a/src/backend/access/gin/ginvacuum.c b/src/backend/access/gin/ginvacuum.c
index 0935a6d9e53..d91ca2b391c 100644
--- a/src/backend/access/gin/ginvacuum.c
+++ b/src/backend/access/gin/ginvacuum.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "commands/vacuum.h"
 #include "miscadmin.h"
@@ -136,6 +137,7 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	Page		page,
 				parentPage;
 	BlockNumber rightlink;
+	bool		needwal;
 
 	/*
 	 * This function MUST be called only if someone of parent pages hold
@@ -159,6 +161,10 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	 */
 	PredicateLockPageCombine(gvs->index, deleteBlkno, rightlink);
 
+	needwal = RelationNeedsWAL(gvs->index);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* Unlink the page by changing left sibling's rightlink */
@@ -195,7 +201,7 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	MarkBufferDirty(lBuffer);
 	MarkBufferDirty(dBuffer);
 
-	if (RelationNeedsWAL(gvs->index))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogDeletePage data;
@@ -650,6 +656,9 @@ ginbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 
 		if (resPage)
 		{
+			if (RelationNeedsWAL(gvs.index))
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 			PageRestoreTempPage(resPage, page);
 			MarkBufferDirty(buffer);
diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c
index 25b42e38f22..4a870a062ba 100644
--- a/src/backend/access/gist/gist.c
+++ b/src/backend/access/gist/gist.c
@@ -16,6 +16,7 @@
 
 #include "access/gist_private.h"
 #include "access/gistscan.h"
+#include "access/walprohibit.h"
 #include "catalog/pg_collation.h"
 #include "commands/vacuum.h"
 #include "miscadmin.h"
@@ -135,6 +136,9 @@ gistbuildempty(Relation index)
 	buffer = ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermittedHaveXID();
+
 	/* Initialize and xlog buffer */
 	START_CRIT_SECTION();
 	GISTInitBuffer(buffer, F_LEAF);
@@ -234,6 +238,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 	XLogRecPtr	recptr;
 	int			i;
 	bool		is_split;
+	bool		needwal = RelationNeedsWAL(rel);
 
 	/*
 	 * Refuse to modify a page that's incompletely split. This should not
@@ -465,9 +470,12 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 		 * insertion for that. NB: The number of pages and data segments
 		 * specified here must match the calculations in gistXLogSplit()!
 		 */
-		if (!is_build && RelationNeedsWAL(rel))
+		if (!is_build && needwal)
 			XLogEnsureRecordSpace(npage, 1 + npage * 2);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		/*
@@ -500,7 +508,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 			recptr = GistBuildLSN;
 		else
 		{
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 				recptr = gistXLogSplit(is_leaf,
 									   dist, oldrlink, oldnsn, leftchildbuf,
 									   markfollowright);
@@ -526,6 +534,9 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 	}
 	else
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		/*
 		 * Enough space.  We always get here if ntup==0.
 		 */
@@ -567,7 +578,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 			recptr = GistBuildLSN;
 		else
 		{
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 			{
 				OffsetNumber ndeloffs = 0,
 							deloffs[1];
@@ -1641,6 +1652,7 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 	OffsetNumber offnum,
 				maxoff;
 	TransactionId latestRemovedXid = InvalidTransactionId;
+	bool		needwal = RelationNeedsWAL(rel);
 
 	Assert(GistPageIsLeaf(page));
 
@@ -1659,13 +1671,16 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 			deletable[ndeletable++] = offnum;
 	}
 
-	if (XLogStandbyInfoActive() && RelationNeedsWAL(rel))
+	if (XLogStandbyInfoActive() && needwal)
 		latestRemovedXid =
 			index_compute_xid_horizon_for_tuples(rel, heapRel, buffer,
 												 deletable, ndeletable);
 
 	if (ndeletable > 0)
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		PageIndexMultiDelete(page, deletable, ndeletable);
@@ -1682,7 +1697,7 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 		MarkBufferDirty(buffer);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gist/gistvacuum.c b/src/backend/access/gist/gistvacuum.c
index a9c616c7724..bbb3ebb19ad 100644
--- a/src/backend/access/gist/gistvacuum.c
+++ b/src/backend/access/gist/gistvacuum.c
@@ -17,6 +17,7 @@
 #include "access/genam.h"
 #include "access/gist_private.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "commands/vacuum.h"
 #include "lib/integerset.h"
 #include "miscadmin.h"
@@ -260,6 +261,7 @@ gistvacuumpage(GistVacState *vstate, BlockNumber blkno, BlockNumber orig_blkno)
 	Buffer		buffer;
 	Page		page;
 	BlockNumber recurse_to;
+	bool		needwal = RelationNeedsWAL(rel);
 
 restart:
 	recurse_to = InvalidBlockNumber;
@@ -341,6 +343,9 @@ restart:
 		 */
 		if (ntodelete > 0)
 		{
+			if (needwal)
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 
 			MarkBufferDirty(buffer);
@@ -348,7 +353,7 @@ restart:
 			PageIndexMultiDelete(page, todelete, ntodelete);
 			GistMarkTuplesDeleted(page);
 
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 			{
 				XLogRecPtr	recptr;
 
@@ -580,6 +585,7 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	IndexTuple	idxtuple;
 	XLogRecPtr	recptr;
 	FullTransactionId txid;
+	bool		needwal = RelationNeedsWAL(info->index);
 
 	/*
 	 * Check that the leaf is still empty and deletable.
@@ -634,6 +640,9 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	 */
 	txid = ReadNextFullTransactionId();
 
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* mark the page as deleted */
@@ -645,7 +654,7 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	MarkBufferDirty(parentBuffer);
 	PageIndexTupleDelete(parentPage, downlink);
 
-	if (RelationNeedsWAL(info->index))
+	if (needwal)
 		recptr = gistXLogPageDelete(leafBuffer, txid, parentBuffer, downlink);
 	else
 		recptr = gistGetFakeLSN(info->index);
diff --git a/src/backend/access/hash/hash.c b/src/backend/access/hash/hash.c
index 7c9ccf446c8..f4903a43bb5 100644
--- a/src/backend/access/hash/hash.c
+++ b/src/backend/access/hash/hash.c
@@ -22,6 +22,7 @@
 #include "access/hash_xlog.h"
 #include "access/relscan.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "catalog/index.h"
 #include "commands/progress.h"
 #include "commands/vacuum.h"
@@ -467,6 +468,7 @@ hashbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	Buffer		metabuf = InvalidBuffer;
 	HashMetaPage metap;
 	HashMetaPage cachedmetap;
+	bool		needwal;
 
 	tuples_removed = 0;
 	num_index_tuples = 0;
@@ -573,6 +575,10 @@ loop_top:
 		goto loop_top;
 	}
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Okay, we're really done.  Update tuple count in metapage. */
 	START_CRIT_SECTION();
 
@@ -603,7 +609,7 @@ loop_top:
 	MarkBufferDirty(metabuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_hash_update_meta_page xlrec;
 		XLogRecPtr	recptr;
@@ -690,6 +696,7 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 	Buffer		buf;
 	Bucket		new_bucket PG_USED_FOR_ASSERTS_ONLY = InvalidBucket;
 	bool		bucket_dirty = false;
+	bool		needwal = RelationNeedsWAL(rel);
 
 	blkno = bucket_blkno;
 	buf = bucket_buf;
@@ -788,6 +795,9 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		 */
 		if (ndeletable > 0)
 		{
+			if (needwal)
+				CheckWALPermitted();
+
 			/* No ereport(ERROR) until changes are logged */
 			START_CRIT_SECTION();
 
@@ -809,7 +819,7 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 			MarkBufferDirty(buf);
 
 			/* XLOG stuff */
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 			{
 				xl_hash_delete xlrec;
 				XLogRecPtr	recptr;
@@ -883,6 +893,9 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		page = BufferGetPage(bucket_buf);
 		bucket_opaque = (HashPageOpaque) PageGetSpecialPointer(page);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -890,7 +903,7 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		MarkBufferDirty(bucket_buf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c
index 2ebe671967b..2eab69efa91 100644
--- a/src/backend/access/hash/hashinsert.c
+++ b/src/backend/access/hash/hashinsert.c
@@ -17,6 +17,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "storage/buf_internals.h"
 #include "storage/lwlock.h"
@@ -193,6 +194,8 @@ restart_insert:
 	 */
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	AssertWALPermittedHaveXID();
+
 	/* Do the update.  No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -360,6 +363,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 	if (ndeletable > 0)
 	{
 		TransactionId latestRemovedXid;
+		bool		needwal = RelationNeedsWAL(rel);
 
 		latestRemovedXid =
 			index_compute_xid_horizon_for_tuples(rel, hrel, buf,
@@ -370,6 +374,9 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 		 */
 		LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -393,7 +400,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 		MarkBufferDirty(metabuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			xl_hash_vacuum_one_page xlrec;
 			XLogRecPtr	recptr;
diff --git a/src/backend/access/hash/hashovfl.c b/src/backend/access/hash/hashovfl.c
index 00f0a940116..e7c5dd3e3ce 100644
--- a/src/backend/access/hash/hashovfl.c
+++ b/src/backend/access/hash/hashovfl.c
@@ -19,6 +19,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "utils/rel.h"
 
@@ -312,6 +313,8 @@ _hash_addovflpage(Relation rel, Buffer metabuf, Buffer buf, bool retain_pin)
 
 found:
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Do the update.  No ereport(ERROR) until changes are logged. We want to
 	 * log the changes for bitmap page and overflow page together to avoid
@@ -510,6 +513,7 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	Buffer		prevbuf = InvalidBuffer;
 	Buffer		nextbuf = InvalidBuffer;
 	bool		update_metap = false;
+	bool		needwal;
 
 	/* Get information from the doomed page */
 	_hash_checkpage(rel, ovflbuf, LH_OVERFLOW_PAGE);
@@ -573,10 +577,15 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	/* Get write-lock on metapage to update firstfree */
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	needwal = RelationNeedsWAL(rel);
+
 	/* This operation needs to log multiple tuples, prepare WAL for that */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 		XLogEnsureRecordSpace(HASH_XLOG_FREE_OVFL_BUFS, 4 + nitups);
 
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -641,7 +650,7 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	}
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_hash_squeeze_page xlrec;
 		XLogRecPtr	recptr;
@@ -922,14 +931,19 @@ readpage:
 
 				if (nitups > 0)
 				{
+					bool		needwal = RelationNeedsWAL(rel);
+
 					Assert(nitups == ndeletable);
 
 					/*
 					 * This operation needs to log multiple tuples, prepare
 					 * WAL for that.
 					 */
-					if (RelationNeedsWAL(rel))
+					if (needwal)
+					{
+						CheckWALPermitted();
 						XLogEnsureRecordSpace(0, 3 + nitups);
+					}
 
 					START_CRIT_SECTION();
 
@@ -947,7 +961,7 @@ readpage:
 					MarkBufferDirty(rbuf);
 
 					/* XLOG stuff */
-					if (RelationNeedsWAL(rel))
+					if (needwal)
 					{
 						XLogRecPtr	recptr;
 						xl_hash_move_page_contents xlrec;
diff --git a/src/backend/access/hash/hashpage.c b/src/backend/access/hash/hashpage.c
index a664ecf494a..55a867dd375 100644
--- a/src/backend/access/hash/hashpage.c
+++ b/src/backend/access/hash/hashpage.c
@@ -30,6 +30,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "port/pg_bitutils.h"
 #include "storage/lmgr.h"
@@ -816,6 +817,8 @@ restart_expand:
 		goto fail;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Since we are scribbling on the pages in the shared buffers, establish a
 	 * critical section.  Any failure in this next code leaves us with a big
@@ -1172,6 +1175,8 @@ _hash_splitbucket(Relation rel,
 
 				if (PageGetFreeSpaceForMultipleTuples(npage, nitups + 1) < (all_tups_size + itemsz))
 				{
+					AssertWALPermittedHaveXID();
+
 					/*
 					 * Change the shared buffer state in critical section,
 					 * otherwise any error could make it unrecoverable.
@@ -1223,6 +1228,8 @@ _hash_splitbucket(Relation rel,
 		/* Exit loop if no more overflow pages in old bucket */
 		if (!BlockNumberIsValid(oblkno))
 		{
+			AssertWALPermittedHaveXID();
+
 			/*
 			 * Change the shared buffer state in critical section, otherwise
 			 * any error could make it unrecoverable.
@@ -1269,6 +1276,8 @@ _hash_splitbucket(Relation rel,
 	npage = BufferGetPage(bucket_nbuf);
 	nopaque = (HashPageOpaque) PageGetSpecialPointer(npage);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	oopaque->hasho_flag &= ~LH_BUCKET_BEING_SPLIT;
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 9b5f417eac4..a411dbb128c 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -47,6 +47,7 @@
 #include "access/transam.h"
 #include "access/valid.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1898,6 +1899,8 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
 	 */
 	CheckForSerializableConflictIn(relation, NULL, InvalidBlockNumber);
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -2174,6 +2177,8 @@ heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples,
 										   &vmbuffer, NULL);
 		page = BufferGetPage(buffer);
 
+		AssertWALPermittedHaveXID();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -2692,6 +2697,8 @@ l1:
 							  xid, LockTupleExclusive, true,
 							  &new_xmax, &new_infomask, &new_infomask2);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -3444,6 +3451,8 @@ l2:
 
 		Assert(HEAP_XMAX_IS_LOCKED_ONLY(infomask_lock_old_tuple));
 
+		AssertWALPermittedHaveXID();
+
 		START_CRIT_SECTION();
 
 		/* Clear obsolete visibility flags ... */
@@ -3617,6 +3626,8 @@ l2:
 										   bms_overlap(modified_attrs, id_attrs),
 										   &old_key_copied);
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -4550,6 +4561,8 @@ failed:
 							  GetCurrentTransactionId(), mode, false,
 							  &xid, &new_infomask, &new_infomask2);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -5341,6 +5354,8 @@ l4:
 								VISIBILITYMAP_ALL_FROZEN))
 			cleared_all_frozen = true;
 
+		AssertWALPermittedHaveXID();
+
 		START_CRIT_SECTION();
 
 		/* ... and set them */
@@ -5499,6 +5514,8 @@ heap_finish_speculative(Relation relation, ItemPointer tid)
 	StaticAssertStmt(MaxOffsetNumber < SpecTokenOffsetNumber,
 					 "invalid speculative token constant");
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -5607,6 +5624,8 @@ heap_abort_speculative(Relation relation, ItemPointer tid)
 		elog(ERROR, "attempted to kill a non-speculative tuple");
 	Assert(!HeapTupleHeaderIsHeapOnly(tp.t_data));
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * No need to check for serializable conflicts here.  There is never a
 	 * need for a combocid, either.  No need to extract replica identity, or
@@ -5723,6 +5742,7 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	HeapTupleHeader htup;
 	uint32		oldlen;
 	uint32		newlen;
+	bool		needwal;
 
 	/*
 	 * For now, parallel operations are required to be strictly read-only.
@@ -5753,6 +5773,10 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	if (oldlen != newlen || htup->t_hoff != tuple->t_data->t_hoff)
 		elog(ERROR, "wrong tuple length");
 
+	needwal = RelationNeedsWAL(relation);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -5763,7 +5787,7 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	MarkBufferDirty(buffer);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(relation))
+	if (needwal)
 	{
 		xl_heap_inplace xlrec;
 		XLogRecPtr	recptr;
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index bc510e2e9b3..9dcae7d2153 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -18,6 +18,7 @@
 #include "access/heapam_xlog.h"
 #include "access/htup_details.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "catalog/catalog.h"
 #include "miscadmin.h"
@@ -94,11 +95,11 @@ heap_page_prune_opt(Relation relation, Buffer buffer)
 	Size		minfree;
 
 	/*
-	 * We can't write WAL in recovery mode, so there's no point trying to
+	 * We can't write WAL during read-only mode, so there's no point trying to
 	 * clean the page. The primary will likely issue a cleaning WAL record soon
 	 * anyway, so this is no particular loss.
 	 */
-	if (RecoveryInProgress())
+	if (!XLogInsertAllowed())
 		return;
 
 	/*
@@ -232,6 +233,7 @@ heap_page_prune(Relation relation, Buffer buffer,
 	OffsetNumber offnum,
 				maxoff;
 	PruneState	prstate;
+	bool		needwal;
 
 	/*
 	 * Our strategy is to scan the page and make lists of items to change,
@@ -286,6 +288,10 @@ heap_page_prune(Relation relation, Buffer buffer,
 	if (off_loc)
 		*off_loc = InvalidOffsetNumber;
 
+	needwal = RelationNeedsWAL(relation);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Any error while applying the changes is critical */
 	START_CRIT_SECTION();
 
@@ -319,7 +325,7 @@ heap_page_prune(Relation relation, Buffer buffer,
 		/*
 		 * Emit a WAL XLOG_HEAP2_CLEAN record showing what we did
 		 */
-		if (RelationNeedsWAL(relation))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 53b1a952543..4eff8865ca7 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -59,6 +59,7 @@
 #include "access/parallel.h"
 #include "access/transam.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "catalog/storage.h"
@@ -759,6 +760,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 	BlockNumber nblocks,
 				blkno;
 	HeapTupleData tuple;
+	bool		needwal = RelationNeedsWAL(onerel);
 	TransactionId relfrozenxid = onerel->rd_rel->relfrozenxid;
 	TransactionId relminmxid = onerel->rd_rel->relminmxid;
 	BlockNumber empty_pages,
@@ -1201,6 +1203,9 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 			 */
 			if (!PageIsAllVisible(page))
 			{
+				if (needwal)
+					CheckWALPermitted();
+
 				START_CRIT_SECTION();
 
 				/* mark buffer dirty before writing a WAL record */
@@ -1216,7 +1221,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 				 * page has been previously WAL-logged, and if not, do that
 				 * now.
 				 */
-				if (RelationNeedsWAL(onerel) &&
+				if (needwal &&
 					PageGetLSN(page) == InvalidXLogRecPtr)
 					log_newpage_buffer(buf, true);
 
@@ -1482,6 +1487,9 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 		 */
 		if (nfrozen > 0)
 		{
+			if (needwal)
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 
 			MarkBufferDirty(buf);
@@ -1499,7 +1507,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 			}
 
 			/* Now WAL-log freezing if necessary */
-			if (RelationNeedsWAL(onerel))
+			if (needwal)
 			{
 				XLogRecPtr	recptr;
 
@@ -1932,6 +1940,7 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	TransactionId visibility_cutoff_xid;
 	bool		all_frozen;
 	LVSavedErrInfo saved_err_info;
+	bool		needwal = RelationNeedsWAL(onerel);
 
 	pgstat_progress_update_param(PROGRESS_VACUUM_HEAP_BLKS_VACUUMED, blkno);
 
@@ -1939,6 +1948,9 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	update_vacuum_error_info(vacrelstats, &saved_err_info, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
 							 blkno, InvalidOffsetNumber);
 
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	for (; tupindex < dead_tuples->num_tuples; tupindex++)
@@ -1964,7 +1976,7 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	MarkBufferDirty(buffer);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(onerel))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index b1072183bcd..44244363968 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -88,6 +88,7 @@
 
 #include "access/heapam_xlog.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "miscadmin.h"
 #include "port/pg_bitutils.h"
@@ -249,6 +250,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 	uint8		mapOffset = HEAPBLK_TO_OFFSET(heapBlk);
 	Page		page;
 	uint8	   *map;
+	bool		needwal = RelationNeedsWAL(rel);
 
 #ifdef TRACE_VISIBILITYMAP
 	elog(DEBUG1, "vm_set %s %d", RelationGetRelationName(rel), heapBlk);
@@ -270,6 +272,16 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 	map = (uint8 *) PageGetContents(page);
 	LockBuffer(vmBuf, BUFFER_LOCK_EXCLUSIVE);
 
+	/*
+	 * Can reach here from VACUUM or from startup process, so need not have an
+	 * XID.
+	 *
+	 * Recovery in the startup process never have wal prohibit state, skip
+	 * permission check if reach here in the startup process.
+	 */
+	if (needwal)
+		InRecovery ? AssertWALPermitted() : CheckWALPermitted();
+
 	if (flags != (map[mapByte] >> mapOffset & VISIBILITYMAP_VALID_BITS))
 	{
 		START_CRIT_SECTION();
@@ -277,7 +289,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 		map[mapByte] |= (flags << mapOffset);
 		MarkBufferDirty(vmBuf);
 
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			if (XLogRecPtrIsInvalid(recptr))
 			{
@@ -476,6 +488,7 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		Buffer		mapBuffer;
 		Page		page;
 		char	   *map;
+		bool		needwal;
 
 		newnblocks = truncBlock + 1;
 
@@ -489,8 +502,13 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		page = BufferGetPage(mapBuffer);
 		map = PageGetContents(page);
 
+		needwal = (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded());
+
 		LockBuffer(mapBuffer, BUFFER_LOCK_EXCLUSIVE);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -518,7 +536,7 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		 * during recovery.
 		 */
 		MarkBufferDirty(mapBuffer);
-		if (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded())
+		if (needwal)
 			log_newpage_buffer(mapBuffer, false);
 
 		END_CRIT_SECTION();
diff --git a/src/backend/access/nbtree/nbtdedup.c b/src/backend/access/nbtree/nbtdedup.c
index f6be865b17e..b519a1268e8 100644
--- a/src/backend/access/nbtree/nbtdedup.c
+++ b/src/backend/access/nbtree/nbtdedup.c
@@ -16,6 +16,7 @@
 
 #include "access/nbtree.h"
 #include "access/nbtxlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "utils/rel.h"
 
@@ -271,6 +272,8 @@ _bt_dedup_one_page(Relation rel, Buffer buf, Relation heapRel,
 		nopaque->btpo_flags &= ~BTP_HAS_GARBAGE;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	PageRestoreTempPage(newpage, page);
diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c
index d36f7557c87..2c3d8aaecbd 100644
--- a/src/backend/access/nbtree/nbtinsert.c
+++ b/src/backend/access/nbtree/nbtinsert.c
@@ -19,6 +19,7 @@
 #include "access/nbtxlog.h"
 #include "access/tableam.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/lmgr.h"
@@ -1246,6 +1247,8 @@ _bt_insertonpg(Relation rel,
 			}
 		}
 
+		AssertWALPermittedHaveXID();
+
 		/* Do the update.  No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -1898,13 +1901,16 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf,
 			ropaque->btpo_flags |= BTP_SPLIT_END;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Right sibling is locked, new siblings are prepared, but original page
 	 * is not updated yet.
 	 *
 	 * NO EREPORT(ERROR) till right sibling is updated.  We can get away with
 	 * not starting the critical section till here because we haven't been
-	 * scribbling on the original page yet; see comments above.
+	 * scribbling on the original page yet; see the comments above for grabbing
+	 * the right sibling.
 	 */
 	START_CRIT_SECTION();
 
@@ -2467,6 +2473,8 @@ _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf)
 	right_item = CopyIndexTuple(item);
 	BTreeTupleSetDownLink(right_item, rbkno);
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till newroot op is logged */
 	START_CRIT_SECTION();
 
diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c
index 7f392480ac0..8c3fc251a29 100644
--- a/src/backend/access/nbtree/nbtpage.c
+++ b/src/backend/access/nbtree/nbtpage.c
@@ -26,6 +26,7 @@
 #include "access/nbtxlog.h"
 #include "access/tableam.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
@@ -179,6 +180,7 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	BTMetaPageData *metad;
 	bool		needsRewrite = false;
 	XLogRecPtr	recptr;
+	bool		needwal;
 
 	/* read the metapage and check if it needs rewrite */
 	metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ);
@@ -202,6 +204,10 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	_bt_unlockbuf(rel, metabuf);
 	_bt_lockbuf(rel, metabuf, BT_WRITE);
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* upgrade meta-page if needed */
@@ -214,7 +220,7 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	MarkBufferDirty(metabuf);
 
 	/* write wal record if needed */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_btree_metadata md;
 
@@ -332,6 +338,7 @@ _bt_getroot(Relation rel, int access)
 	if (metad->btm_root == P_NONE)
 	{
 		Page		metapg;
+		bool		needwal;
 
 		/* If access = BT_READ, caller doesn't want us to create root yet */
 		if (access == BT_READ)
@@ -377,6 +384,10 @@ _bt_getroot(Relation rel, int access)
 		/* Get raw page pointer for metapage */
 		metapg = BufferGetPage(metabuf);
 
+		needwal = RelationNeedsWAL(rel);
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO ELOG(ERROR) till meta is updated */
 		START_CRIT_SECTION();
 
@@ -395,7 +406,7 @@ _bt_getroot(Relation rel, int access)
 		MarkBufferDirty(metabuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			xl_btree_newroot xlrec;
 			XLogRecPtr	recptr;
@@ -1131,6 +1142,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 	char	   *updatedbuf = NULL;
 	Size		updatedbuflen = 0;
 	OffsetNumber updatedoffsets[MaxIndexTuplesPerPage];
+	bool		needwal = RelationNeedsWAL(rel);
 
 	/* Shouldn't be called unless there's something to do */
 	Assert(ndeletable > 0 || nupdatable > 0);
@@ -1145,7 +1157,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 	}
 
 	/* XLOG stuff -- allocate and fill buffer before critical section */
-	if (nupdatable > 0 && RelationNeedsWAL(rel))
+	if (nupdatable > 0 && needwal)
 	{
 		Size		offset = 0;
 
@@ -1175,6 +1187,9 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 		}
 	}
 
+	if (needwal)
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -1235,7 +1250,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 	MarkBufferDirty(buf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		xl_btree_vacuum xlrec_vacuum;
@@ -1302,6 +1317,8 @@ _bt_delitems_delete(Relation rel, Buffer buf,
 		latestRemovedXid =
 			_bt_xid_horizon(rel, heapRel, page, deletable, ndeletable);
 
+	AssertWALPermittedHaveXID();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -1832,6 +1849,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	OffsetNumber nextoffset;
 	IndexTuple	itup;
 	IndexTupleData trunctuple;
+	bool		needwal;
 
 	page = BufferGetPage(leafbuf);
 	opaque = (BTPageOpaque) PageGetSpecialPointer(page);
@@ -1920,6 +1938,10 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	 */
 	PredicateLockPageCombine(rel, leafblkno, leafrightsib);
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -1971,7 +1993,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	MarkBufferDirty(leafbuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_btree_mark_page_halfdead xlrec;
 		XLogRecPtr	recptr;
@@ -2064,6 +2086,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 	int			targetlevel;
 	IndexTuple	leafhikey;
 	BlockNumber nextchild;
+	bool		needwal;
 
 	page = BufferGetPage(leafbuf);
 	opaque = (BTPageOpaque) PageGetSpecialPointer(page);
@@ -2277,6 +2300,10 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 	 * Here we begin doing the deletion.
 	 */
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -2356,7 +2383,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 		MarkBufferDirty(leafbuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_btree_unlink_page xlrec;
 		xl_btree_metadata xlmeta;
diff --git a/src/backend/access/spgist/spgdoinsert.c b/src/backend/access/spgist/spgdoinsert.c
index 934d65b89f2..3c5a15c5d32 100644
--- a/src/backend/access/spgist/spgdoinsert.c
+++ b/src/backend/access/spgist/spgdoinsert.c
@@ -18,6 +18,7 @@
 #include "access/genam.h"
 #include "access/spgist_private.h"
 #include "access/spgxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -214,6 +215,8 @@ addLeafTuple(Relation index, SpGistState *state, SpGistLeafTuple leafTuple,
 	xlrec.offnumParent = InvalidOffsetNumber;
 	xlrec.nodeI = 0;
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	if (current->offnum == InvalidOffsetNumber ||
@@ -458,6 +461,8 @@ moveLeafs(Relation index, SpGistState *state,
 
 	leafdata = leafptr = palloc(size);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/* copy all the old tuples to new page, unless they're dead */
@@ -1109,6 +1114,8 @@ doPickSplit(Relation index, SpGistState *state,
 
 	leafdata = leafptr = (char *) palloc(totalLeafSizes);
 
+	AssertWALPermittedHaveXID();
+
 	/* Here we begin making the changes to the target pages */
 	START_CRIT_SECTION();
 
@@ -1517,6 +1524,8 @@ spgAddNodeAction(Relation index, SpGistState *state,
 	if (PageGetExactFreeSpace(current->page) >=
 		newInnerTuple->size - innerTuple->size)
 	{
+		AssertWALPermittedHaveXID();
+
 		/*
 		 * We can replace the inner tuple by new version in-place
 		 */
@@ -1603,6 +1612,8 @@ spgAddNodeAction(Relation index, SpGistState *state,
 		else
 			xlrec.parentBlk = 2;
 
+		AssertWALPermittedHaveXID();
+
 		START_CRIT_SECTION();
 
 		/* insert new ... */
@@ -1788,6 +1799,8 @@ spgSplitNodeAction(Relation index, SpGistState *state,
 									&xlrec.newPage);
 	}
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c
index e1c58933f97..3308832b85b 100644
--- a/src/backend/access/spgist/spgvacuum.c
+++ b/src/backend/access/spgist/spgvacuum.c
@@ -19,6 +19,7 @@
 #include "access/spgist_private.h"
 #include "access/spgxlog.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/storage_xlog.h"
 #include "commands/vacuum.h"
@@ -139,6 +140,7 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 	int			nDeletable;
 	OffsetNumber i,
 				max = PageGetMaxOffsetNumber(page);
+	bool		needwal;
 
 	memset(predecessor, 0, sizeof(predecessor));
 	memset(deletable, 0, sizeof(deletable));
@@ -323,6 +325,10 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 	if (nDeletable != xlrec.nDead + xlrec.nPlaceholder + xlrec.nMove)
 		elog(ERROR, "inconsistent counts of deletable tuples");
 
+	needwal = RelationNeedsWAL(index);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Do the updates */
 	START_CRIT_SECTION();
 
@@ -371,7 +377,7 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 
@@ -411,6 +417,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 	OffsetNumber toDelete[MaxIndexTuplesPerPage];
 	OffsetNumber i,
 				max = PageGetMaxOffsetNumber(page);
+	bool		needwal;
 
 	xlrec.nDelete = 0;
 
@@ -447,6 +454,10 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 	if (xlrec.nDelete == 0)
 		return;					/* nothing more to do */
 
+	needwal = RelationNeedsWAL(index);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Do the update */
 	START_CRIT_SECTION();
 
@@ -455,7 +466,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 
@@ -502,6 +513,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer)
 	OffsetNumber itemnos[MaxIndexTuplesPerPage];
 	spgxlogVacuumRedirect xlrec;
 	GlobalVisState *vistest;
+	bool		needwal;
 
 	xlrec.nToPlaceholder = 0;
 	xlrec.newestRedirectXid = InvalidTransactionId;
@@ -509,6 +521,10 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer)
 	/* XXX: providing heap relation would allow more pruning */
 	vistest = GlobalVisTestFor(NULL);
 
+	needwal = RelationNeedsWAL(index);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -584,7 +600,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer)
 	if (hasUpdate)
 		MarkBufferDirty(buffer);
 
-	if (hasUpdate && RelationNeedsWAL(index))
+	if (hasUpdate && needwal)
 	{
 		XLogRecPtr	recptr;
 
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index b8bedca04a4..0a88740764f 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -73,6 +73,7 @@
 #include "access/transam.h"
 #include "access/twophase.h"
 #include "access/twophase_rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1143,6 +1144,8 @@ GetNewMultiXactId(int nmembers, MultiXactOffset *offset)
 
 	ExtendMultiXactMember(nextOffset, nmembers);
 
+	CheckWALPermitted();
+
 	/*
 	 * Critical section from here until caller has written the data into the
 	 * just-reserved SLRU space; we don't want to error out with a partly
@@ -2938,7 +2941,7 @@ TruncateMultiXact(MultiXactId newOldestMulti, Oid newOldestMultiDB)
 	mxtruncinfo trunc;
 	MultiXactId earliest;
 
-	Assert(!RecoveryInProgress());
+	Assert(XLogInsertAllowed());
 	Assert(MultiXactState->finishedStartup);
 
 	/*
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index ef4f9981e35..ff2bc8cc74b 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -82,6 +82,7 @@
 #include "access/transam.h"
 #include "access/twophase.h"
 #include "access/twophase_rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1106,6 +1107,8 @@ EndPrepare(GlobalTransaction gxact)
 	 */
 	XLogEnsureRecordSpace(0, records.num_chunks);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	MyProc->delayChkpt = true;
@@ -2196,6 +2199,9 @@ RecordTransactionCommitPrepared(TransactionId xid,
 	replorigin = (replorigin_session_origin != InvalidRepOriginId &&
 				  replorigin_session_origin != DoNotReplicateId);
 
+	/* COMMIT PREPARED need not have an XID */
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* See notes in RecordTransactionCommit */
@@ -2286,6 +2292,9 @@ RecordTransactionAbortPrepared(TransactionId xid,
 		elog(PANIC, "cannot abort transaction %u, it was already committed",
 			 xid);
 
+	/* ROLLBACK PREPARED need not have an XID */
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
diff --git a/src/backend/access/transam/varsup.c b/src/backend/access/transam/varsup.c
index a4944faa32e..0c7a2362f25 100644
--- a/src/backend/access/transam/varsup.c
+++ b/src/backend/access/transam/varsup.c
@@ -17,6 +17,7 @@
 #include "access/commit_ts.h"
 #include "access/subtrans.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "commands/dbcommands.h"
@@ -75,6 +76,9 @@ GetNewTransactionId(bool isSubXact)
 	if (RecoveryInProgress())
 		elog(ERROR, "cannot assign TransactionIds during recovery");
 
+	/* do not assign transaction id in read-only mode */
+	CheckWALPermitted();
+
 	LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
 
 	full_xid = ShmemVariableCache->nextXid;
diff --git a/src/backend/access/transam/walprohibit.c b/src/backend/access/transam/walprohibit.c
index 8ab30be1d51..6b86e5cffcc 100644
--- a/src/backend/access/transam/walprohibit.c
+++ b/src/backend/access/transam/walprohibit.c
@@ -20,6 +20,16 @@
 #include "storage/procsignal.h"
 #include "storage/shmem.h"
 
+/*
+ * Assert flag to enforce WAL insert permission check rule before starting a
+ * critical section for the WAL writes.  For this, either of CheckWALPermitted,
+ * AssertWALPermittedHaveXID, or AssertWALPermitted must be called before
+ * starting the critical section.
+ */
+#ifdef USE_ASSERT_CHECKING
+WALPermitCheckState walpermit_checked_state = WALPERMIT_UNCHECKED;
+#endif
+
 /*
  * Shared-memory WAL prohibit state
  */
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 188c299bed9..abda095e735 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -26,6 +26,7 @@
 #include "access/subtrans.h"
 #include "access/transam.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1317,6 +1318,8 @@ RecordTransactionCommit(void)
 		/* Tell bufmgr and smgr to prepare for commit */
 		BufmgrCommit();
 
+		AssertWALPermittedHaveXID();
+
 		/*
 		 * Mark ourselves as within our "commit critical section".  This
 		 * forces any concurrent checkpoint to wait until we've updated
@@ -1677,6 +1680,9 @@ RecordTransactionAbort(bool isSubXact)
 		elog(PANIC, "cannot abort transaction %u, it was already committed",
 			 xid);
 
+	/* We'll be reaching here with valid XID only. */
+	AssertWALPermittedHaveXID();
+
 	/* Fetch the data we need for the abort record */
 	nrels = smgrGetPendingDeletes(false, &rels);
 	nchildren = xactGetCommittedChildren(&children);
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index e170df78a87..1273cb1ed52 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -1019,7 +1019,7 @@ XLogInsertRecord(XLogRecData *rdata,
 
 	/* cross-check on whether we should be here or not */
 	if (!XLogInsertAllowed())
-		elog(ERROR, "cannot make new WAL entries during recovery");
+		elog(ERROR, "cannot make new WAL entries now");
 
 	/*----------
 	 *
@@ -2854,9 +2854,11 @@ XLogFlush(XLogRecPtr record)
 	 * trying to flush the WAL, we should update minRecoveryPoint instead. We
 	 * test XLogInsertAllowed(), not InRecovery, because we need checkpointer
 	 * to act this way too, and because when it tries to write the
-	 * end-of-recovery checkpoint, it should indeed flush.
+	 * end-of-recovery checkpoint, it should indeed flush. Also, WAL prohibit
+	 * state should not restrict WAL flushing. Otherwise, the dirty buffer
+	 * cannot be evicted until WAL has been flushed up to the buffer's LSN.
 	 */
-	if (!XLogInsertAllowed())
+	if (!XLogInsertAllowed() && !IsWALProhibited())
 	{
 		UpdateMinRecoveryPoint(record, false);
 		return;
@@ -8788,6 +8790,8 @@ CreateCheckPoint(int flags)
 	/* sanity check */
 	if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
 		elog(ERROR, "can't create a checkpoint during recovery");
+	if (!XLogInsertAllowed() && !RecoveryInProgress())
+		elog(ERROR, "can't create a checkpoint while system is read only ");
 
 	/*
 	 * Initialize InitXLogInsert working areas before entering the critical
@@ -8817,6 +8821,8 @@ CreateCheckPoint(int flags)
 	MemSet(&CheckpointStats, 0, sizeof(CheckpointStats));
 	CheckpointStats.ckpt_start_t = GetCurrentTimestamp();
 
+	AssertWALPermitted();
+
 	/*
 	 * Use a critical section to force system panic if we have trouble.
 	 */
@@ -9045,6 +9051,8 @@ CreateCheckPoint(int flags)
 	if (!shutdown && XLogStandbyInfoActive())
 		LogStandbySnapshot();
 
+	AssertWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -9202,6 +9210,8 @@ CreateEndOfRecoveryRecord(void)
 
 	LocalSetXLogInsertAllowed();
 
+	AssertWALPermitted();
+
 	START_CRIT_SECTION();
 
 	XLogBeginInsert();
@@ -9848,7 +9858,7 @@ void
 UpdateFullPageWrites(void)
 {
 	XLogCtlInsert *Insert = &XLogCtl->Insert;
-	bool		recoveryInProgress;
+	bool		WALInsertAllowed;
 
 	/*
 	 * Do nothing if full_page_writes has not been changed.
@@ -9862,10 +9872,10 @@ UpdateFullPageWrites(void)
 
 	/*
 	 * Perform this outside critical section so that the WAL insert
-	 * initialization done by RecoveryInProgress() doesn't trigger an
+	 * initialization done by XLogInsertAllowed() doesn't trigger an
 	 * assertion failure.
 	 */
-	recoveryInProgress = RecoveryInProgress();
+	WALInsertAllowed = XLogInsertAllowed();
 
 	START_CRIT_SECTION();
 
@@ -9887,8 +9897,11 @@ UpdateFullPageWrites(void)
 	 * Write an XLOG_FPW_CHANGE record. This allows us to keep track of
 	 * full_page_writes during archive recovery, if required.
 	 */
-	if (XLogStandbyInfoActive() && !recoveryInProgress)
+	if (XLogStandbyInfoActive() && WALInsertAllowed)
 	{
+		/* Assured that WAL permission has been checked */
+		AssertWALPermitted();
+
 		XLogBeginInsert();
 		XLogRegisterData((char *) (&fullPageWrites), sizeof(bool));
 
diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c
index c526bb19281..506d7e97f38 100644
--- a/src/backend/access/transam/xloginsert.c
+++ b/src/backend/access/transam/xloginsert.c
@@ -19,6 +19,7 @@
 
 #include "postgres.h"
 
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xlog_internal.h"
@@ -126,9 +127,14 @@ XLogBeginInsert(void)
 	Assert(mainrdata_last == (XLogRecData *) &mainrdata_head);
 	Assert(mainrdata_len == 0);
 
+	/*
+	 * WAL permission must have checked before entering the critical section.
+	 * Otherwise, WAL prohibited error will force system panic.
+	 */
+	Assert(walpermit_checked_state != WALPERMIT_UNCHECKED || !CritSectionCount);
+
 	/* cross-check on whether we should be here or not */
-	if (!XLogInsertAllowed())
-		elog(ERROR, "cannot make new WAL entries during recovery");
+	CheckWALPermitted();
 
 	if (begininsert_called)
 		elog(ERROR, "XLogBeginInsert was already called");
@@ -210,6 +216,9 @@ XLogResetInsertion(void)
 	mainrdata_last = (XLogRecData *) &mainrdata_head;
 	curinsert_flags = 0;
 	begininsert_called = false;
+
+	/* Reset walpermit_checked flag */
+	RESET_WALPERMIT_CHECKED_STATE();
 }
 
 /*
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 6aab73bfd44..8dacf48db24 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -20,6 +20,7 @@
 #include "access/relation.h"
 #include "access/table.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -378,8 +379,13 @@ fill_seq_with_data(Relation rel, HeapTuple tuple)
 
 	/* check the comment above nextval_internal()'s equivalent call. */
 	if (RelationNeedsWAL(rel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermittedHaveXID();
+	}
+
 	START_CRIT_SECTION();
 
 	MarkBufferDirty(buf);
@@ -766,8 +772,13 @@ nextval_internal(Oid relid, bool check_permissions)
 	 * (Have to do that here, so we're outside the critical section)
 	 */
 	if (logit && RelationNeedsWAL(seqrel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermittedHaveXID();
+	}
+
 	/* ready to change the on-disk (or really, in-buffer) tuple */
 	START_CRIT_SECTION();
 
@@ -977,8 +988,13 @@ do_setval(Oid relid, int64 next, bool iscalled)
 
 	/* check the comment above nextval_internal()'s equivalent call. */
 	if (RelationNeedsWAL(seqrel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermittedHaveXID();
+	}
+
 	/* ready to change the on-disk (or really, in-buffer) tuple */
 	START_CRIT_SECTION();
 
diff --git a/src/backend/commands/variable.c b/src/backend/commands/variable.c
index 484f7ea2c0e..acc34d2ad7c 100644
--- a/src/backend/commands/variable.c
+++ b/src/backend/commands/variable.c
@@ -501,11 +501,14 @@ check_transaction_read_only(bool *newval, void **extra, GucSource source)
 			GUC_check_errmsg("transaction read-write mode must be set before any query");
 			return false;
 		}
-		/* Can't go to r/w mode while recovery is still active */
-		if (RecoveryInProgress())
+		/*
+		 * Can't go to r/w mode while recovery is still active or while in WAL
+		 * prohibit state
+		 */
+		if (!XLogInsertAllowed())
 		{
 			GUC_check_errcode(ERRCODE_FEATURE_NOT_SUPPORTED);
-			GUC_check_errmsg("cannot set transaction read-write mode during recovery");
+			GUC_check_errmsg("cannot set transaction read-write mode while system is read only");
 			return false;
 		}
 	}
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index b841721c9ec..81337d48d82 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -930,6 +930,10 @@ RequestCheckpoint(int flags)
 	int			old_failed,
 				old_started;
 
+	/* The checkpoint is allowed in recovery but not in WAL prohibit state */
+	if (!RecoveryInProgress())
+		CheckWALPermitted();
+
 	/*
 	 * If in a standalone backend, just do it ourselves.
 	 */
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index a2a963bd5b4..186cc47be1d 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -3638,13 +3638,15 @@ MarkBufferDirtyHint(Buffer buffer, bool buffer_std)
 		{
 			/*
 			 * If we must not write WAL, due to a relfilenode-specific
-			 * condition or being in recovery, don't dirty the page.  We can
-			 * set the hint, just not dirty the page as a result so the hint
-			 * is lost when we evict the page or shutdown.
+			 * condition or in general, don't dirty the page.  We can
+			 * set the hint, but must not dirty the page as a result, lest
+			 * we trigger WAL generation. Unless the page is dirtied again
+			 * later, the hint will be lost when the page is evicted, or at
+			 * shutdown.
 			 *
 			 * See src/backend/storage/page/README for longer discussion.
 			 */
-			if (RecoveryInProgress() ||
+			if (!XLogInsertAllowed() ||
 				RelFileNodeSkippingWAL(bufHdr->tag.rnode))
 				return;
 
diff --git a/src/backend/storage/freespace/freespace.c b/src/backend/storage/freespace/freespace.c
index 6a96126b0c2..b05b0fe5f41 100644
--- a/src/backend/storage/freespace/freespace.c
+++ b/src/backend/storage/freespace/freespace.c
@@ -24,6 +24,7 @@
 #include "postgres.h"
 
 #include "access/htup_details.h"
+#include "access/walprohibit.h"
 #include "access/xlogutils.h"
 #include "miscadmin.h"
 #include "storage/freespace.h"
@@ -285,12 +286,19 @@ FreeSpaceMapPrepareTruncateRel(Relation rel, BlockNumber nblocks)
 	 */
 	if (first_removed_slot > 0)
 	{
+		bool needwal;
+
 		buf = fsm_readbuf(rel, first_removed_address, false);
 		if (!BufferIsValid(buf))
 			return InvalidBlockNumber;	/* nothing to do; the FSM was already
 										 * smaller */
 		LockBuffer(buf, BUFFER_LOCK_EXCLUSIVE);
 
+		needwal = (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded());
+
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -305,7 +313,7 @@ FreeSpaceMapPrepareTruncateRel(Relation rel, BlockNumber nblocks)
 		 * during recovery.
 		 */
 		MarkBufferDirty(buf);
-		if (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded())
+		if (needwal)
 			log_newpage_buffer(buf, false);
 
 		END_CRIT_SECTION();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index d86566f4554..f949a290745 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -794,15 +794,15 @@ LockAcquireExtended(const LOCKTAG *locktag,
 	if (lockmode <= 0 || lockmode > lockMethodTable->numLockModes)
 		elog(ERROR, "unrecognized lock mode: %d", lockmode);
 
-	if (RecoveryInProgress() && !InRecovery &&
+	if (!XLogInsertAllowed() && !InRecovery &&
 		(locktag->locktag_type == LOCKTAG_OBJECT ||
 		 locktag->locktag_type == LOCKTAG_RELATION) &&
 		lockmode > RowExclusiveLock)
 		ereport(ERROR,
 				(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
-				 errmsg("cannot acquire lock mode %s on database objects while recovery is in progress",
+				 errmsg("cannot acquire lock mode %s on database objects while system is read only",
 						lockMethodTable->lockModeNames[lockmode]),
-				 errhint("Only RowExclusiveLock or less can be acquired on database objects during recovery.")));
+				 errhint("Only RowExclusiveLock or less can be acquired on database objects while system is read only")));
 
 #ifdef LOCK_DEBUG
 	if (LOCK_DEBUG_ENABLED(locktag))
diff --git a/src/backend/utils/cache/relmapper.c b/src/backend/utils/cache/relmapper.c
index 671fbb0ed5c..90d7599a57c 100644
--- a/src/backend/utils/cache/relmapper.c
+++ b/src/backend/utils/cache/relmapper.c
@@ -43,6 +43,7 @@
 #include <sys/stat.h>
 #include <unistd.h>
 
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -841,6 +842,8 @@ write_relmap_file(bool shared, RelMapFile *newmap,
 		xl_relmap_update xlrec;
 		XLogRecPtr	lsn;
 
+		AssertWALPermittedHaveXID();
+
 		/* now errors are fatal ... */
 		START_CRIT_SECTION();
 
diff --git a/src/include/access/walprohibit.h b/src/include/access/walprohibit.h
index 14e1f5b3a2e..a876ec8a064 100644
--- a/src/include/access/walprohibit.h
+++ b/src/include/access/walprohibit.h
@@ -24,8 +24,8 @@ extern void MarkCheckPointSkippedInWalProhibitState(void);
 extern void WALProhibitStateShmemInit(void);
 
 /* WAL Prohibit States */
-#define	WALPROHIBIT_STATE_READ_WRITE		0x0000
-#define	WALPROHIBIT_STATE_READ_ONLY			0x0001
+#define	WALPROHIBIT_STATE_READ_WRITE		0x0000	/* WAL permitted */
+#define	WALPROHIBIT_STATE_READ_ONLY			0x0001	/* WAL prohibited */
 
 /*
  * The bit is used in state transition from one state to another.  When this
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e33523984..f3ff120601e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -94,12 +94,37 @@ extern PGDLLIMPORT volatile uint32 CritSectionCount;
 /* in tcop/postgres.c */
 extern void ProcessInterrupts(void);
 
+#ifdef USE_ASSERT_CHECKING
+typedef enum
+{
+	WALPERMIT_UNCHECKED,
+	WALPERMIT_CHECKED,
+	WALPERMIT_CHECKED_AND_USED
+} WALPermitCheckState;
+
+/* in access/walprohibit.c */
+extern WALPermitCheckState walpermit_checked_state;
+
+/*
+ * Reset walpermit_checked flag when no longer in the critical section.
+ * Otherwise, marked checked and used.
+ */
+#define RESET_WALPERMIT_CHECKED_STATE() \
+do { \
+	walpermit_checked_state = CritSectionCount ? \
+	WALPERMIT_CHECKED_AND_USED : WALPERMIT_UNCHECKED; \
+} while(0)
+#else
+#define RESET_WALPERMIT_CHECKED_STATE() ((void) 0)
+#endif
+
 #ifndef WIN32
 
 #define CHECK_FOR_INTERRUPTS() \
 do { \
 	if (unlikely(InterruptPending)) \
 		ProcessInterrupts(); \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 #else							/* WIN32 */
 
@@ -109,6 +134,7 @@ do { \
 		pgwin32_dispatch_queued_signals(); \
 	if (unlikely(InterruptPending)) \
 		ProcessInterrupts(); \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 #endif							/* WIN32 */
 
@@ -135,6 +161,7 @@ do { \
 do { \
 	Assert(CritSectionCount > 0); \
 	CritSectionCount--; \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 
 
-- 
2.22.0

From ac524d266aae75614da81e5868dd78f8bbb4c9db Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Mon, 27 Jul 2020 02:13:36 -0400
Subject: [PATCH v7 4/5] Error or Assert before START_CRIT_SECTION for WAL
 write

Based on the following criteria added the Assert or the Error when system is
prohibited:

 - Added ERROR for the function which can be reachable without valid XID in case
   of VACUUM or CONCURRENT CREATE INDEX.  For that added common static inline
   function CheckWALPermitted().
 - Added ASSERT for the function which cannot be reached without valid XID;
   Assert to ensure XID validation. For that added AssertWALPermitted_HaveXID().

To enforce the rule to have aforesaid assert or error check before entering a
critical section for WAL write, a new walpermit_checked_state assert only flag
is added. If these check is missing then XLogBeginInsert() will have an
assertion if it is in critical section.

If we are not doing WAL insert inside the critical section then the above
checking is not necessary, we can rely on XLogBeginInsert() for that check &
report an error.
---
 contrib/pg_surgery/heap_surgery.c         | 10 ++++--
 src/backend/access/brin/brin.c            |  4 +++
 src/backend/access/brin/brin_pageops.c    | 21 ++++++++++--
 src/backend/access/brin/brin_revmap.c     | 10 +++++-
 src/backend/access/gin/ginbtree.c         | 15 +++++++--
 src/backend/access/gin/gindatapage.c      | 18 +++++++++--
 src/backend/access/gin/ginfast.c          | 11 +++++--
 src/backend/access/gin/gininsert.c        |  4 +++
 src/backend/access/gin/ginutil.c          |  9 +++++-
 src/backend/access/gin/ginvacuum.c        | 11 ++++++-
 src/backend/access/gist/gist.c            | 25 ++++++++++++---
 src/backend/access/gist/gistvacuum.c      | 13 ++++++--
 src/backend/access/hash/hash.c            | 19 +++++++++--
 src/backend/access/hash/hashinsert.c      |  9 +++++-
 src/backend/access/hash/hashovfl.c        | 22 ++++++++++---
 src/backend/access/hash/hashpage.c        |  9 ++++++
 src/backend/access/heap/heapam.c          | 26 ++++++++++++++-
 src/backend/access/heap/pruneheap.c       | 12 +++++--
 src/backend/access/heap/vacuumlazy.c      | 18 +++++++++--
 src/backend/access/heap/visibilitymap.c   | 22 +++++++++++--
 src/backend/access/nbtree/nbtdedup.c      |  3 ++
 src/backend/access/nbtree/nbtinsert.c     | 10 +++++-
 src/backend/access/nbtree/nbtpage.c       | 39 +++++++++++++++++++----
 src/backend/access/spgist/spgdoinsert.c   | 13 ++++++++
 src/backend/access/spgist/spgvacuum.c     | 22 +++++++++++--
 src/backend/access/transam/multixact.c    |  5 ++-
 src/backend/access/transam/twophase.c     |  9 ++++++
 src/backend/access/transam/varsup.c       |  4 +++
 src/backend/access/transam/walprohibit.c  | 10 ++++++
 src/backend/access/transam/xact.c         |  6 ++++
 src/backend/access/transam/xlog.c         | 27 ++++++++++++----
 src/backend/access/transam/xloginsert.c   | 13 ++++++--
 src/backend/commands/sequence.c           | 16 ++++++++++
 src/backend/commands/variable.c           |  9 ++++--
 src/backend/postmaster/checkpointer.c     |  4 +++
 src/backend/storage/buffer/bufmgr.c       | 10 +++---
 src/backend/storage/freespace/freespace.c | 10 +++++-
 src/backend/storage/lmgr/lock.c           |  6 ++--
 src/backend/utils/cache/relmapper.c       |  3 ++
 src/include/access/walprohibit.h          |  4 +--
 src/include/miscadmin.h                   | 27 ++++++++++++++++
 41 files changed, 465 insertions(+), 73 deletions(-)

diff --git a/contrib/pg_surgery/heap_surgery.c b/contrib/pg_surgery/heap_surgery.c
index eb96b4bb36d..53d8c9cea28 100644
--- a/contrib/pg_surgery/heap_surgery.c
+++ b/contrib/pg_surgery/heap_surgery.c
@@ -14,6 +14,7 @@
 
 #include "access/heapam.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "catalog/pg_am_d.h"
 #include "catalog/pg_proc_d.h"
 #include "miscadmin.h"
@@ -89,6 +90,7 @@ heap_force_common(FunctionCallInfo fcinfo, HeapTupleForceOption heap_force_opt)
 	OffsetNumber curr_start_ptr,
 				next_start_ptr;
 	bool		include_this_tid[MaxHeapTuplesPerPage];
+	bool		needwal;
 
 	if (RecoveryInProgress())
 		ereport(ERROR,
@@ -100,6 +102,7 @@ heap_force_common(FunctionCallInfo fcinfo, HeapTupleForceOption heap_force_opt)
 	sanity_check_tid_array(ta, &ntids);
 
 	rel = relation_open(relid, RowExclusiveLock);
+	needwal = RelationNeedsWAL(rel);
 
 	/* Check target relation. */
 	sanity_check_relation(rel);
@@ -217,6 +220,9 @@ heap_force_common(FunctionCallInfo fcinfo, HeapTupleForceOption heap_force_opt)
 		if (heap_force_opt == HEAP_FORCE_KILL && PageIsAllVisible(page))
 			visibilitymap_pin(rel, blkno, &vmbuf);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* No ereport(ERROR) from here until all the changes are logged. */
 		START_CRIT_SECTION();
 
@@ -297,12 +303,12 @@ heap_force_common(FunctionCallInfo fcinfo, HeapTupleForceOption heap_force_opt)
 			MarkBufferDirty(buf);
 
 			/* XLOG stuff */
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 				log_newpage_buffer(buf, true);
 		}
 
 		/* WAL log the VM page if it was modified. */
-		if (did_modify_vm && RelationNeedsWAL(rel))
+		if (did_modify_vm && needwal)
 			log_newpage_buffer(vmbuf, false);
 
 		END_CRIT_SECTION();
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index 1f72562c603..47142193706 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -24,6 +24,7 @@
 #include "access/relscan.h"
 #include "access/table.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/index.h"
 #include "catalog/pg_am.h"
@@ -759,6 +760,9 @@ brinbuildempty(Relation index)
 		ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermittedHaveXID();
+
 	/* Initialize and xlog metabuffer. */
 	START_CRIT_SECTION();
 	brin_metapage_init(BufferGetPage(metabuf), BrinGetPagesPerRange(index),
diff --git a/src/backend/access/brin/brin_pageops.c b/src/backend/access/brin/brin_pageops.c
index 87de0b855b5..8b377a679ab 100644
--- a/src/backend/access/brin/brin_pageops.c
+++ b/src/backend/access/brin/brin_pageops.c
@@ -14,6 +14,7 @@
 #include "access/brin_pageops.h"
 #include "access/brin_revmap.h"
 #include "access/brin_xlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -65,6 +66,7 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 	Buffer		newbuf;
 	BlockNumber newblk = InvalidBlockNumber;
 	bool		extended;
+	bool		needwal = RelationNeedsWAL(idxrel);
 
 	Assert(newsz == MAXALIGN(newsz));
 
@@ -176,13 +178,16 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 	if (((BrinPageFlags(oldpage) & BRIN_EVACUATE_PAGE) == 0) &&
 		brin_can_do_samepage_update(oldbuf, origsz, newsz))
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 		if (!PageIndexTupleOverwrite(oldpage, oldoff, (Item) unconstify(BrinTuple *, newtup), newsz))
 			elog(ERROR, "failed to replace BRIN tuple");
 		MarkBufferDirty(oldbuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(idxrel))
+		if (needwal)
 		{
 			xl_brin_samepage_update xlrec;
 			XLogRecPtr	recptr;
@@ -240,6 +245,9 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 
 		revmapbuf = brinLockRevmapPageForUpdate(revmap, heapBlk);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		/*
@@ -267,7 +275,7 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 		MarkBufferDirty(revmapbuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(idxrel))
+		if (needwal)
 		{
 			xl_brin_update xlrec;
 			XLogRecPtr	recptr;
@@ -351,6 +359,7 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	Buffer		revmapbuf;
 	ItemPointerData tid;
 	bool		extended;
+	bool		needwal;
 
 	Assert(itemsz == MAXALIGN(itemsz));
 
@@ -405,6 +414,10 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	page = BufferGetPage(*buffer);
 	blk = BufferGetBlockNumber(*buffer);
 
+	needwal = RelationNeedsWAL(idxrel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Execute the actual insertion */
 	START_CRIT_SECTION();
 	if (extended)
@@ -424,7 +437,7 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	MarkBufferDirty(revmapbuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(idxrel))
+	if (needwal)
 	{
 		xl_brin_insert xlrec;
 		XLogRecPtr	recptr;
@@ -881,6 +894,8 @@ brin_initialize_empty_new_buffer(Relation idxrel, Buffer buffer)
 			   "brin_initialize_empty_new_buffer: initializing blank page %u",
 			   BufferGetBlockNumber(buffer)));
 
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 	page = BufferGetPage(buffer);
 	brin_page_init(page, BRIN_PAGETYPE_REGULAR);
diff --git a/src/backend/access/brin/brin_revmap.c b/src/backend/access/brin/brin_revmap.c
index 35746714a7c..fd766da445d 100644
--- a/src/backend/access/brin/brin_revmap.c
+++ b/src/backend/access/brin/brin_revmap.c
@@ -26,6 +26,7 @@
 #include "access/brin_tuple.h"
 #include "access/brin_xlog.h"
 #include "access/rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -340,6 +341,7 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 	OffsetNumber revmapOffset;
 	OffsetNumber regOffset;
 	ItemId		lp;
+	bool		needwal;
 
 	revmap = brinRevmapInitialize(idxrel, &pagesPerRange, NULL);
 
@@ -404,6 +406,10 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 	 * crashed or aborted summarization; remove them silently.
 	 */
 
+	needwal = RelationNeedsWAL(idxrel);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	ItemPointerSetInvalid(&invalidIptr);
@@ -415,7 +421,7 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 	MarkBufferDirty(regBuf);
 	MarkBufferDirty(revmapBuf);
 
-	if (RelationNeedsWAL(idxrel))
+	if (needwal)
 	{
 		xl_brin_desummarize xlrec;
 		XLogRecPtr	recptr;
@@ -613,6 +619,8 @@ revmap_physical_extend(BrinRevmap *revmap)
 		return;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Ok, we have now locked the metapage and the target block. Re-initialize
 	 * the target block as a revmap page, and update the metapage.
diff --git a/src/backend/access/gin/ginbtree.c b/src/backend/access/gin/ginbtree.c
index 82788a5c367..f31590dcd75 100644
--- a/src/backend/access/gin/ginbtree.c
+++ b/src/backend/access/gin/ginbtree.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/predicate.h"
@@ -332,6 +333,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 {
 	Page		page = BufferGetPage(stack->buffer);
 	bool		result;
+	bool		needwal;
 	GinPlaceToPageRC rc;
 	uint16		xlflags = 0;
 	Page		childpage = NULL;
@@ -377,6 +379,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 								 insertdata, updateblkno,
 								 &ptp_workspace,
 								 &newlpage, &newrpage);
+	needwal = RelationNeedsWAL(btree->index) && !btree->isBuild;
 
 	if (rc == GPTP_NO_WORK)
 	{
@@ -385,10 +388,13 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 	}
 	else if (rc == GPTP_INSERT)
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		/* It will fit, perform the insertion */
 		START_CRIT_SECTION();
 
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogBeginInsert();
 			XLogRegisterBuffer(0, stack->buffer, REGBUF_STANDARD);
@@ -409,7 +415,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 			MarkBufferDirty(childbuf);
 		}
 
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 			ginxlogInsert xlrec;
@@ -547,6 +553,9 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 			}
 		}
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/*
 		 * OK, we have the new contents of the left page in a temporary copy
 		 * now (newlpage), and likewise for the new contents of the
@@ -587,7 +596,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 		}
 
 		/* write WAL record */
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gin/gindatapage.c b/src/backend/access/gin/gindatapage.c
index 7a2690e97f2..0abc5990100 100644
--- a/src/backend/access/gin/gindatapage.c
+++ b/src/backend/access/gin/gindatapage.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "lib/ilist.h"
 #include "miscadmin.h"
@@ -811,6 +812,7 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 	if (removedsomething)
 	{
 		bool		modified;
+		bool		needwal;
 
 		/*
 		 * Make sure we have a palloc'd copy of all segments, after the first
@@ -835,8 +837,12 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 			}
 		}
 
-		if (RelationNeedsWAL(indexrel))
+		needwal = RelationNeedsWAL(indexrel);
+		if (needwal)
+		{
+			CheckWALPermitted();
 			computeLeafRecompressWALData(leaf);
+		}
 
 		/* Apply changes to page */
 		START_CRIT_SECTION();
@@ -845,7 +851,7 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 
 		MarkBufferDirty(buffer);
 
-		if (RelationNeedsWAL(indexrel))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
@@ -1777,6 +1783,7 @@ createPostingTree(Relation index, ItemPointerData *items, uint32 nitems,
 	int			nrootitems;
 	int			rootsize;
 	bool		is_build = (buildStats != NULL);
+	bool		needwal;
 
 	/* Construct the new root page in memory first. */
 	tmppage = (Page) palloc(BLCKSZ);
@@ -1825,12 +1832,17 @@ createPostingTree(Relation index, ItemPointerData *items, uint32 nitems,
 	 */
 	PredicateLockPageSplit(index, BufferGetBlockNumber(entrybuffer), blkno);
 
+	needwal = RelationNeedsWAL(index) && !is_build;
+
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	PageRestoreTempPage(tmppage, page);
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index) && !is_build)
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogCreatePostingTree data;
diff --git a/src/backend/access/gin/ginfast.c b/src/backend/access/gin/ginfast.c
index 2e41b34d8d5..b8c2a993408 100644
--- a/src/backend/access/gin/ginfast.c
+++ b/src/backend/access/gin/ginfast.c
@@ -20,6 +20,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
 #include "catalog/pg_am.h"
@@ -68,6 +69,8 @@ writeListPage(Relation index, Buffer buffer,
 	PGAlignedBlock workspace;
 	char	   *ptr;
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	GinInitBuffer(buffer, GIN_LIST);
@@ -548,6 +551,7 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 	Page		metapage;
 	GinMetaPageData *metadata;
 	BlockNumber blknoToDelete;
+	bool		needwal = RelationNeedsWAL(index);
 
 	metapage = BufferGetPage(metabuffer);
 	metadata = GinPageGetMeta(metapage);
@@ -586,8 +590,11 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 		 * prepare the XLogInsert machinery for that before entering the
 		 * critical section.
 		 */
-		if (RelationNeedsWAL(index))
+		if (needwal)
+		{
+			CheckWALPermitted();
 			XLogEnsureRecordSpace(data.ndeleted, 0);
+		}
 
 		START_CRIT_SECTION();
 
@@ -625,7 +632,7 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 			MarkBufferDirty(buffers[i]);
 		}
 
-		if (RelationNeedsWAL(index))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gin/gininsert.c b/src/backend/access/gin/gininsert.c
index 77433dc8a41..989d82ffcaf 100644
--- a/src/backend/access/gin/gininsert.c
+++ b/src/backend/access/gin/gininsert.c
@@ -17,6 +17,7 @@
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/index.h"
 #include "miscadmin.h"
@@ -447,6 +448,9 @@ ginbuildempty(Relation index)
 		ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(RootBuffer, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermittedHaveXID();
+
 	/* Initialize and xlog metabuffer and root buffer. */
 	START_CRIT_SECTION();
 	GinInitMetabuffer(MetaBuffer);
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index ef9b56fd363..b48ea1a746a 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -17,6 +17,7 @@
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
 #include "access/reloptions.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/pg_collation.h"
 #include "catalog/pg_type.h"
@@ -659,12 +660,18 @@ ginUpdateStats(Relation index, const GinStatsData *stats, bool is_build)
 	Buffer		metabuffer;
 	Page		metapage;
 	GinMetaPageData *metadata;
+	bool		needwal;
 
 	metabuffer = ReadBuffer(index, GIN_METAPAGE_BLKNO);
 	LockBuffer(metabuffer, GIN_EXCLUSIVE);
 	metapage = BufferGetPage(metabuffer);
 	metadata = GinPageGetMeta(metapage);
 
+	needwal = RelationNeedsWAL(index) && !is_build;
+
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	metadata->nTotalPages = stats->nTotalPages;
@@ -684,7 +691,7 @@ ginUpdateStats(Relation index, const GinStatsData *stats, bool is_build)
 
 	MarkBufferDirty(metabuffer);
 
-	if (RelationNeedsWAL(index) && !is_build)
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogUpdateMeta data;
diff --git a/src/backend/access/gin/ginvacuum.c b/src/backend/access/gin/ginvacuum.c
index 0935a6d9e53..d91ca2b391c 100644
--- a/src/backend/access/gin/ginvacuum.c
+++ b/src/backend/access/gin/ginvacuum.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "commands/vacuum.h"
 #include "miscadmin.h"
@@ -136,6 +137,7 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	Page		page,
 				parentPage;
 	BlockNumber rightlink;
+	bool		needwal;
 
 	/*
 	 * This function MUST be called only if someone of parent pages hold
@@ -159,6 +161,10 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	 */
 	PredicateLockPageCombine(gvs->index, deleteBlkno, rightlink);
 
+	needwal = RelationNeedsWAL(gvs->index);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* Unlink the page by changing left sibling's rightlink */
@@ -195,7 +201,7 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	MarkBufferDirty(lBuffer);
 	MarkBufferDirty(dBuffer);
 
-	if (RelationNeedsWAL(gvs->index))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogDeletePage data;
@@ -650,6 +656,9 @@ ginbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 
 		if (resPage)
 		{
+			if (RelationNeedsWAL(gvs.index))
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 			PageRestoreTempPage(resPage, page);
 			MarkBufferDirty(buffer);
diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c
index 25b42e38f22..4a870a062ba 100644
--- a/src/backend/access/gist/gist.c
+++ b/src/backend/access/gist/gist.c
@@ -16,6 +16,7 @@
 
 #include "access/gist_private.h"
 #include "access/gistscan.h"
+#include "access/walprohibit.h"
 #include "catalog/pg_collation.h"
 #include "commands/vacuum.h"
 #include "miscadmin.h"
@@ -135,6 +136,9 @@ gistbuildempty(Relation index)
 	buffer = ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermittedHaveXID();
+
 	/* Initialize and xlog buffer */
 	START_CRIT_SECTION();
 	GISTInitBuffer(buffer, F_LEAF);
@@ -234,6 +238,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 	XLogRecPtr	recptr;
 	int			i;
 	bool		is_split;
+	bool		needwal = RelationNeedsWAL(rel);
 
 	/*
 	 * Refuse to modify a page that's incompletely split. This should not
@@ -465,9 +470,12 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 		 * insertion for that. NB: The number of pages and data segments
 		 * specified here must match the calculations in gistXLogSplit()!
 		 */
-		if (!is_build && RelationNeedsWAL(rel))
+		if (!is_build && needwal)
 			XLogEnsureRecordSpace(npage, 1 + npage * 2);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		/*
@@ -500,7 +508,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 			recptr = GistBuildLSN;
 		else
 		{
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 				recptr = gistXLogSplit(is_leaf,
 									   dist, oldrlink, oldnsn, leftchildbuf,
 									   markfollowright);
@@ -526,6 +534,9 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 	}
 	else
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		/*
 		 * Enough space.  We always get here if ntup==0.
 		 */
@@ -567,7 +578,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 			recptr = GistBuildLSN;
 		else
 		{
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 			{
 				OffsetNumber ndeloffs = 0,
 							deloffs[1];
@@ -1641,6 +1652,7 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 	OffsetNumber offnum,
 				maxoff;
 	TransactionId latestRemovedXid = InvalidTransactionId;
+	bool		needwal = RelationNeedsWAL(rel);
 
 	Assert(GistPageIsLeaf(page));
 
@@ -1659,13 +1671,16 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 			deletable[ndeletable++] = offnum;
 	}
 
-	if (XLogStandbyInfoActive() && RelationNeedsWAL(rel))
+	if (XLogStandbyInfoActive() && needwal)
 		latestRemovedXid =
 			index_compute_xid_horizon_for_tuples(rel, heapRel, buffer,
 												 deletable, ndeletable);
 
 	if (ndeletable > 0)
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		PageIndexMultiDelete(page, deletable, ndeletable);
@@ -1682,7 +1697,7 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 		MarkBufferDirty(buffer);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gist/gistvacuum.c b/src/backend/access/gist/gistvacuum.c
index a9c616c7724..bbb3ebb19ad 100644
--- a/src/backend/access/gist/gistvacuum.c
+++ b/src/backend/access/gist/gistvacuum.c
@@ -17,6 +17,7 @@
 #include "access/genam.h"
 #include "access/gist_private.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "commands/vacuum.h"
 #include "lib/integerset.h"
 #include "miscadmin.h"
@@ -260,6 +261,7 @@ gistvacuumpage(GistVacState *vstate, BlockNumber blkno, BlockNumber orig_blkno)
 	Buffer		buffer;
 	Page		page;
 	BlockNumber recurse_to;
+	bool		needwal = RelationNeedsWAL(rel);
 
 restart:
 	recurse_to = InvalidBlockNumber;
@@ -341,6 +343,9 @@ restart:
 		 */
 		if (ntodelete > 0)
 		{
+			if (needwal)
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 
 			MarkBufferDirty(buffer);
@@ -348,7 +353,7 @@ restart:
 			PageIndexMultiDelete(page, todelete, ntodelete);
 			GistMarkTuplesDeleted(page);
 
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 			{
 				XLogRecPtr	recptr;
 
@@ -580,6 +585,7 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	IndexTuple	idxtuple;
 	XLogRecPtr	recptr;
 	FullTransactionId txid;
+	bool		needwal = RelationNeedsWAL(info->index);
 
 	/*
 	 * Check that the leaf is still empty and deletable.
@@ -634,6 +640,9 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	 */
 	txid = ReadNextFullTransactionId();
 
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* mark the page as deleted */
@@ -645,7 +654,7 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	MarkBufferDirty(parentBuffer);
 	PageIndexTupleDelete(parentPage, downlink);
 
-	if (RelationNeedsWAL(info->index))
+	if (needwal)
 		recptr = gistXLogPageDelete(leafBuffer, txid, parentBuffer, downlink);
 	else
 		recptr = gistGetFakeLSN(info->index);
diff --git a/src/backend/access/hash/hash.c b/src/backend/access/hash/hash.c
index 7c9ccf446c8..f4903a43bb5 100644
--- a/src/backend/access/hash/hash.c
+++ b/src/backend/access/hash/hash.c
@@ -22,6 +22,7 @@
 #include "access/hash_xlog.h"
 #include "access/relscan.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "catalog/index.h"
 #include "commands/progress.h"
 #include "commands/vacuum.h"
@@ -467,6 +468,7 @@ hashbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	Buffer		metabuf = InvalidBuffer;
 	HashMetaPage metap;
 	HashMetaPage cachedmetap;
+	bool		needwal;
 
 	tuples_removed = 0;
 	num_index_tuples = 0;
@@ -573,6 +575,10 @@ loop_top:
 		goto loop_top;
 	}
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Okay, we're really done.  Update tuple count in metapage. */
 	START_CRIT_SECTION();
 
@@ -603,7 +609,7 @@ loop_top:
 	MarkBufferDirty(metabuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_hash_update_meta_page xlrec;
 		XLogRecPtr	recptr;
@@ -690,6 +696,7 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 	Buffer		buf;
 	Bucket		new_bucket PG_USED_FOR_ASSERTS_ONLY = InvalidBucket;
 	bool		bucket_dirty = false;
+	bool		needwal = RelationNeedsWAL(rel);
 
 	blkno = bucket_blkno;
 	buf = bucket_buf;
@@ -788,6 +795,9 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		 */
 		if (ndeletable > 0)
 		{
+			if (needwal)
+				CheckWALPermitted();
+
 			/* No ereport(ERROR) until changes are logged */
 			START_CRIT_SECTION();
 
@@ -809,7 +819,7 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 			MarkBufferDirty(buf);
 
 			/* XLOG stuff */
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 			{
 				xl_hash_delete xlrec;
 				XLogRecPtr	recptr;
@@ -883,6 +893,9 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		page = BufferGetPage(bucket_buf);
 		bucket_opaque = (HashPageOpaque) PageGetSpecialPointer(page);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -890,7 +903,7 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		MarkBufferDirty(bucket_buf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c
index 2ebe671967b..2eab69efa91 100644
--- a/src/backend/access/hash/hashinsert.c
+++ b/src/backend/access/hash/hashinsert.c
@@ -17,6 +17,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "storage/buf_internals.h"
 #include "storage/lwlock.h"
@@ -193,6 +194,8 @@ restart_insert:
 	 */
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	AssertWALPermittedHaveXID();
+
 	/* Do the update.  No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -360,6 +363,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 	if (ndeletable > 0)
 	{
 		TransactionId latestRemovedXid;
+		bool		needwal = RelationNeedsWAL(rel);
 
 		latestRemovedXid =
 			index_compute_xid_horizon_for_tuples(rel, hrel, buf,
@@ -370,6 +374,9 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 		 */
 		LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -393,7 +400,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 		MarkBufferDirty(metabuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			xl_hash_vacuum_one_page xlrec;
 			XLogRecPtr	recptr;
diff --git a/src/backend/access/hash/hashovfl.c b/src/backend/access/hash/hashovfl.c
index 00f0a940116..e7c5dd3e3ce 100644
--- a/src/backend/access/hash/hashovfl.c
+++ b/src/backend/access/hash/hashovfl.c
@@ -19,6 +19,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "utils/rel.h"
 
@@ -312,6 +313,8 @@ _hash_addovflpage(Relation rel, Buffer metabuf, Buffer buf, bool retain_pin)
 
 found:
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Do the update.  No ereport(ERROR) until changes are logged. We want to
 	 * log the changes for bitmap page and overflow page together to avoid
@@ -510,6 +513,7 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	Buffer		prevbuf = InvalidBuffer;
 	Buffer		nextbuf = InvalidBuffer;
 	bool		update_metap = false;
+	bool		needwal;
 
 	/* Get information from the doomed page */
 	_hash_checkpage(rel, ovflbuf, LH_OVERFLOW_PAGE);
@@ -573,10 +577,15 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	/* Get write-lock on metapage to update firstfree */
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	needwal = RelationNeedsWAL(rel);
+
 	/* This operation needs to log multiple tuples, prepare WAL for that */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 		XLogEnsureRecordSpace(HASH_XLOG_FREE_OVFL_BUFS, 4 + nitups);
 
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -641,7 +650,7 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	}
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_hash_squeeze_page xlrec;
 		XLogRecPtr	recptr;
@@ -922,14 +931,19 @@ readpage:
 
 				if (nitups > 0)
 				{
+					bool		needwal = RelationNeedsWAL(rel);
+
 					Assert(nitups == ndeletable);
 
 					/*
 					 * This operation needs to log multiple tuples, prepare
 					 * WAL for that.
 					 */
-					if (RelationNeedsWAL(rel))
+					if (needwal)
+					{
+						CheckWALPermitted();
 						XLogEnsureRecordSpace(0, 3 + nitups);
+					}
 
 					START_CRIT_SECTION();
 
@@ -947,7 +961,7 @@ readpage:
 					MarkBufferDirty(rbuf);
 
 					/* XLOG stuff */
-					if (RelationNeedsWAL(rel))
+					if (needwal)
 					{
 						XLogRecPtr	recptr;
 						xl_hash_move_page_contents xlrec;
diff --git a/src/backend/access/hash/hashpage.c b/src/backend/access/hash/hashpage.c
index a664ecf494a..55a867dd375 100644
--- a/src/backend/access/hash/hashpage.c
+++ b/src/backend/access/hash/hashpage.c
@@ -30,6 +30,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "port/pg_bitutils.h"
 #include "storage/lmgr.h"
@@ -816,6 +817,8 @@ restart_expand:
 		goto fail;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Since we are scribbling on the pages in the shared buffers, establish a
 	 * critical section.  Any failure in this next code leaves us with a big
@@ -1172,6 +1175,8 @@ _hash_splitbucket(Relation rel,
 
 				if (PageGetFreeSpaceForMultipleTuples(npage, nitups + 1) < (all_tups_size + itemsz))
 				{
+					AssertWALPermittedHaveXID();
+
 					/*
 					 * Change the shared buffer state in critical section,
 					 * otherwise any error could make it unrecoverable.
@@ -1223,6 +1228,8 @@ _hash_splitbucket(Relation rel,
 		/* Exit loop if no more overflow pages in old bucket */
 		if (!BlockNumberIsValid(oblkno))
 		{
+			AssertWALPermittedHaveXID();
+
 			/*
 			 * Change the shared buffer state in critical section, otherwise
 			 * any error could make it unrecoverable.
@@ -1269,6 +1276,8 @@ _hash_splitbucket(Relation rel,
 	npage = BufferGetPage(bucket_nbuf);
 	nopaque = (HashPageOpaque) PageGetSpecialPointer(npage);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	oopaque->hasho_flag &= ~LH_BUCKET_BEING_SPLIT;
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 9b5f417eac4..a411dbb128c 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -47,6 +47,7 @@
 #include "access/transam.h"
 #include "access/valid.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1898,6 +1899,8 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
 	 */
 	CheckForSerializableConflictIn(relation, NULL, InvalidBlockNumber);
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -2174,6 +2177,8 @@ heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples,
 										   &vmbuffer, NULL);
 		page = BufferGetPage(buffer);
 
+		AssertWALPermittedHaveXID();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -2692,6 +2697,8 @@ l1:
 							  xid, LockTupleExclusive, true,
 							  &new_xmax, &new_infomask, &new_infomask2);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -3444,6 +3451,8 @@ l2:
 
 		Assert(HEAP_XMAX_IS_LOCKED_ONLY(infomask_lock_old_tuple));
 
+		AssertWALPermittedHaveXID();
+
 		START_CRIT_SECTION();
 
 		/* Clear obsolete visibility flags ... */
@@ -3617,6 +3626,8 @@ l2:
 										   bms_overlap(modified_attrs, id_attrs),
 										   &old_key_copied);
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -4550,6 +4561,8 @@ failed:
 							  GetCurrentTransactionId(), mode, false,
 							  &xid, &new_infomask, &new_infomask2);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -5341,6 +5354,8 @@ l4:
 								VISIBILITYMAP_ALL_FROZEN))
 			cleared_all_frozen = true;
 
+		AssertWALPermittedHaveXID();
+
 		START_CRIT_SECTION();
 
 		/* ... and set them */
@@ -5499,6 +5514,8 @@ heap_finish_speculative(Relation relation, ItemPointer tid)
 	StaticAssertStmt(MaxOffsetNumber < SpecTokenOffsetNumber,
 					 "invalid speculative token constant");
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -5607,6 +5624,8 @@ heap_abort_speculative(Relation relation, ItemPointer tid)
 		elog(ERROR, "attempted to kill a non-speculative tuple");
 	Assert(!HeapTupleHeaderIsHeapOnly(tp.t_data));
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * No need to check for serializable conflicts here.  There is never a
 	 * need for a combocid, either.  No need to extract replica identity, or
@@ -5723,6 +5742,7 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	HeapTupleHeader htup;
 	uint32		oldlen;
 	uint32		newlen;
+	bool		needwal;
 
 	/*
 	 * For now, parallel operations are required to be strictly read-only.
@@ -5753,6 +5773,10 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	if (oldlen != newlen || htup->t_hoff != tuple->t_data->t_hoff)
 		elog(ERROR, "wrong tuple length");
 
+	needwal = RelationNeedsWAL(relation);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -5763,7 +5787,7 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	MarkBufferDirty(buffer);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(relation))
+	if (needwal)
 	{
 		xl_heap_inplace xlrec;
 		XLogRecPtr	recptr;
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index bc510e2e9b3..9dcae7d2153 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -18,6 +18,7 @@
 #include "access/heapam_xlog.h"
 #include "access/htup_details.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "catalog/catalog.h"
 #include "miscadmin.h"
@@ -94,11 +95,11 @@ heap_page_prune_opt(Relation relation, Buffer buffer)
 	Size		minfree;
 
 	/*
-	 * We can't write WAL in recovery mode, so there's no point trying to
+	 * We can't write WAL during read-only mode, so there's no point trying to
 	 * clean the page. The primary will likely issue a cleaning WAL record soon
 	 * anyway, so this is no particular loss.
 	 */
-	if (RecoveryInProgress())
+	if (!XLogInsertAllowed())
 		return;
 
 	/*
@@ -232,6 +233,7 @@ heap_page_prune(Relation relation, Buffer buffer,
 	OffsetNumber offnum,
 				maxoff;
 	PruneState	prstate;
+	bool		needwal;
 
 	/*
 	 * Our strategy is to scan the page and make lists of items to change,
@@ -286,6 +288,10 @@ heap_page_prune(Relation relation, Buffer buffer,
 	if (off_loc)
 		*off_loc = InvalidOffsetNumber;
 
+	needwal = RelationNeedsWAL(relation);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Any error while applying the changes is critical */
 	START_CRIT_SECTION();
 
@@ -319,7 +325,7 @@ heap_page_prune(Relation relation, Buffer buffer,
 		/*
 		 * Emit a WAL XLOG_HEAP2_CLEAN record showing what we did
 		 */
-		if (RelationNeedsWAL(relation))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 4f2f38168dc..1869df5f03f 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -59,6 +59,7 @@
 #include "access/parallel.h"
 #include "access/transam.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "catalog/storage.h"
@@ -759,6 +760,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 	BlockNumber nblocks,
 				blkno;
 	HeapTupleData tuple;
+	bool		needwal = RelationNeedsWAL(onerel);
 	TransactionId relfrozenxid = onerel->rd_rel->relfrozenxid;
 	TransactionId relminmxid = onerel->rd_rel->relminmxid;
 	BlockNumber empty_pages,
@@ -1201,6 +1203,9 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 			 */
 			if (!PageIsAllVisible(page))
 			{
+				if (needwal)
+					CheckWALPermitted();
+
 				START_CRIT_SECTION();
 
 				/* mark buffer dirty before writing a WAL record */
@@ -1216,7 +1221,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 				 * page has been previously WAL-logged, and if not, do that
 				 * now.
 				 */
-				if (RelationNeedsWAL(onerel) &&
+				if (needwal &&
 					PageGetLSN(page) == InvalidXLogRecPtr)
 					log_newpage_buffer(buf, true);
 
@@ -1482,6 +1487,9 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 		 */
 		if (nfrozen > 0)
 		{
+			if (needwal)
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 
 			MarkBufferDirty(buf);
@@ -1499,7 +1507,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 			}
 
 			/* Now WAL-log freezing if necessary */
-			if (RelationNeedsWAL(onerel))
+			if (needwal)
 			{
 				XLogRecPtr	recptr;
 
@@ -1932,6 +1940,7 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	TransactionId visibility_cutoff_xid;
 	bool		all_frozen;
 	LVSavedErrInfo saved_err_info;
+	bool		needwal = RelationNeedsWAL(onerel);
 
 	pgstat_progress_update_param(PROGRESS_VACUUM_HEAP_BLKS_VACUUMED, blkno);
 
@@ -1939,6 +1948,9 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	update_vacuum_error_info(vacrelstats, &saved_err_info, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
 							 blkno, InvalidOffsetNumber);
 
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	for (; tupindex < dead_tuples->num_tuples; tupindex++)
@@ -1964,7 +1976,7 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	MarkBufferDirty(buffer);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(onerel))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index b1072183bcd..44244363968 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -88,6 +88,7 @@
 
 #include "access/heapam_xlog.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "miscadmin.h"
 #include "port/pg_bitutils.h"
@@ -249,6 +250,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 	uint8		mapOffset = HEAPBLK_TO_OFFSET(heapBlk);
 	Page		page;
 	uint8	   *map;
+	bool		needwal = RelationNeedsWAL(rel);
 
 #ifdef TRACE_VISIBILITYMAP
 	elog(DEBUG1, "vm_set %s %d", RelationGetRelationName(rel), heapBlk);
@@ -270,6 +272,16 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 	map = (uint8 *) PageGetContents(page);
 	LockBuffer(vmBuf, BUFFER_LOCK_EXCLUSIVE);
 
+	/*
+	 * Can reach here from VACUUM or from startup process, so need not have an
+	 * XID.
+	 *
+	 * Recovery in the startup process never have wal prohibit state, skip
+	 * permission check if reach here in the startup process.
+	 */
+	if (needwal)
+		InRecovery ? AssertWALPermitted() : CheckWALPermitted();
+
 	if (flags != (map[mapByte] >> mapOffset & VISIBILITYMAP_VALID_BITS))
 	{
 		START_CRIT_SECTION();
@@ -277,7 +289,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 		map[mapByte] |= (flags << mapOffset);
 		MarkBufferDirty(vmBuf);
 
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			if (XLogRecPtrIsInvalid(recptr))
 			{
@@ -476,6 +488,7 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		Buffer		mapBuffer;
 		Page		page;
 		char	   *map;
+		bool		needwal;
 
 		newnblocks = truncBlock + 1;
 
@@ -489,8 +502,13 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		page = BufferGetPage(mapBuffer);
 		map = PageGetContents(page);
 
+		needwal = (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded());
+
 		LockBuffer(mapBuffer, BUFFER_LOCK_EXCLUSIVE);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -518,7 +536,7 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		 * during recovery.
 		 */
 		MarkBufferDirty(mapBuffer);
-		if (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded())
+		if (needwal)
 			log_newpage_buffer(mapBuffer, false);
 
 		END_CRIT_SECTION();
diff --git a/src/backend/access/nbtree/nbtdedup.c b/src/backend/access/nbtree/nbtdedup.c
index f6be865b17e..b519a1268e8 100644
--- a/src/backend/access/nbtree/nbtdedup.c
+++ b/src/backend/access/nbtree/nbtdedup.c
@@ -16,6 +16,7 @@
 
 #include "access/nbtree.h"
 #include "access/nbtxlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "utils/rel.h"
 
@@ -271,6 +272,8 @@ _bt_dedup_one_page(Relation rel, Buffer buf, Relation heapRel,
 		nopaque->btpo_flags &= ~BTP_HAS_GARBAGE;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	PageRestoreTempPage(newpage, page);
diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c
index d36f7557c87..2c3d8aaecbd 100644
--- a/src/backend/access/nbtree/nbtinsert.c
+++ b/src/backend/access/nbtree/nbtinsert.c
@@ -19,6 +19,7 @@
 #include "access/nbtxlog.h"
 #include "access/tableam.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/lmgr.h"
@@ -1246,6 +1247,8 @@ _bt_insertonpg(Relation rel,
 			}
 		}
 
+		AssertWALPermittedHaveXID();
+
 		/* Do the update.  No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -1898,13 +1901,16 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf,
 			ropaque->btpo_flags |= BTP_SPLIT_END;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Right sibling is locked, new siblings are prepared, but original page
 	 * is not updated yet.
 	 *
 	 * NO EREPORT(ERROR) till right sibling is updated.  We can get away with
 	 * not starting the critical section till here because we haven't been
-	 * scribbling on the original page yet; see comments above.
+	 * scribbling on the original page yet; see the comments above for grabbing
+	 * the right sibling.
 	 */
 	START_CRIT_SECTION();
 
@@ -2467,6 +2473,8 @@ _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf)
 	right_item = CopyIndexTuple(item);
 	BTreeTupleSetDownLink(right_item, rbkno);
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till newroot op is logged */
 	START_CRIT_SECTION();
 
diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c
index 7f392480ac0..8c3fc251a29 100644
--- a/src/backend/access/nbtree/nbtpage.c
+++ b/src/backend/access/nbtree/nbtpage.c
@@ -26,6 +26,7 @@
 #include "access/nbtxlog.h"
 #include "access/tableam.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
@@ -179,6 +180,7 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	BTMetaPageData *metad;
 	bool		needsRewrite = false;
 	XLogRecPtr	recptr;
+	bool		needwal;
 
 	/* read the metapage and check if it needs rewrite */
 	metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ);
@@ -202,6 +204,10 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	_bt_unlockbuf(rel, metabuf);
 	_bt_lockbuf(rel, metabuf, BT_WRITE);
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* upgrade meta-page if needed */
@@ -214,7 +220,7 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	MarkBufferDirty(metabuf);
 
 	/* write wal record if needed */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_btree_metadata md;
 
@@ -332,6 +338,7 @@ _bt_getroot(Relation rel, int access)
 	if (metad->btm_root == P_NONE)
 	{
 		Page		metapg;
+		bool		needwal;
 
 		/* If access = BT_READ, caller doesn't want us to create root yet */
 		if (access == BT_READ)
@@ -377,6 +384,10 @@ _bt_getroot(Relation rel, int access)
 		/* Get raw page pointer for metapage */
 		metapg = BufferGetPage(metabuf);
 
+		needwal = RelationNeedsWAL(rel);
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO ELOG(ERROR) till meta is updated */
 		START_CRIT_SECTION();
 
@@ -395,7 +406,7 @@ _bt_getroot(Relation rel, int access)
 		MarkBufferDirty(metabuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			xl_btree_newroot xlrec;
 			XLogRecPtr	recptr;
@@ -1131,6 +1142,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 	char	   *updatedbuf = NULL;
 	Size		updatedbuflen = 0;
 	OffsetNumber updatedoffsets[MaxIndexTuplesPerPage];
+	bool		needwal = RelationNeedsWAL(rel);
 
 	/* Shouldn't be called unless there's something to do */
 	Assert(ndeletable > 0 || nupdatable > 0);
@@ -1145,7 +1157,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 	}
 
 	/* XLOG stuff -- allocate and fill buffer before critical section */
-	if (nupdatable > 0 && RelationNeedsWAL(rel))
+	if (nupdatable > 0 && needwal)
 	{
 		Size		offset = 0;
 
@@ -1175,6 +1187,9 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 		}
 	}
 
+	if (needwal)
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -1235,7 +1250,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 	MarkBufferDirty(buf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		xl_btree_vacuum xlrec_vacuum;
@@ -1302,6 +1317,8 @@ _bt_delitems_delete(Relation rel, Buffer buf,
 		latestRemovedXid =
 			_bt_xid_horizon(rel, heapRel, page, deletable, ndeletable);
 
+	AssertWALPermittedHaveXID();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -1832,6 +1849,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	OffsetNumber nextoffset;
 	IndexTuple	itup;
 	IndexTupleData trunctuple;
+	bool		needwal;
 
 	page = BufferGetPage(leafbuf);
 	opaque = (BTPageOpaque) PageGetSpecialPointer(page);
@@ -1920,6 +1938,10 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	 */
 	PredicateLockPageCombine(rel, leafblkno, leafrightsib);
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -1971,7 +1993,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	MarkBufferDirty(leafbuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_btree_mark_page_halfdead xlrec;
 		XLogRecPtr	recptr;
@@ -2064,6 +2086,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 	int			targetlevel;
 	IndexTuple	leafhikey;
 	BlockNumber nextchild;
+	bool		needwal;
 
 	page = BufferGetPage(leafbuf);
 	opaque = (BTPageOpaque) PageGetSpecialPointer(page);
@@ -2277,6 +2300,10 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 	 * Here we begin doing the deletion.
 	 */
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -2356,7 +2383,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 		MarkBufferDirty(leafbuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_btree_unlink_page xlrec;
 		xl_btree_metadata xlmeta;
diff --git a/src/backend/access/spgist/spgdoinsert.c b/src/backend/access/spgist/spgdoinsert.c
index 934d65b89f2..3c5a15c5d32 100644
--- a/src/backend/access/spgist/spgdoinsert.c
+++ b/src/backend/access/spgist/spgdoinsert.c
@@ -18,6 +18,7 @@
 #include "access/genam.h"
 #include "access/spgist_private.h"
 #include "access/spgxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -214,6 +215,8 @@ addLeafTuple(Relation index, SpGistState *state, SpGistLeafTuple leafTuple,
 	xlrec.offnumParent = InvalidOffsetNumber;
 	xlrec.nodeI = 0;
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	if (current->offnum == InvalidOffsetNumber ||
@@ -458,6 +461,8 @@ moveLeafs(Relation index, SpGistState *state,
 
 	leafdata = leafptr = palloc(size);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/* copy all the old tuples to new page, unless they're dead */
@@ -1109,6 +1114,8 @@ doPickSplit(Relation index, SpGistState *state,
 
 	leafdata = leafptr = (char *) palloc(totalLeafSizes);
 
+	AssertWALPermittedHaveXID();
+
 	/* Here we begin making the changes to the target pages */
 	START_CRIT_SECTION();
 
@@ -1517,6 +1524,8 @@ spgAddNodeAction(Relation index, SpGistState *state,
 	if (PageGetExactFreeSpace(current->page) >=
 		newInnerTuple->size - innerTuple->size)
 	{
+		AssertWALPermittedHaveXID();
+
 		/*
 		 * We can replace the inner tuple by new version in-place
 		 */
@@ -1603,6 +1612,8 @@ spgAddNodeAction(Relation index, SpGistState *state,
 		else
 			xlrec.parentBlk = 2;
 
+		AssertWALPermittedHaveXID();
+
 		START_CRIT_SECTION();
 
 		/* insert new ... */
@@ -1788,6 +1799,8 @@ spgSplitNodeAction(Relation index, SpGistState *state,
 									&xlrec.newPage);
 	}
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c
index e1c58933f97..3308832b85b 100644
--- a/src/backend/access/spgist/spgvacuum.c
+++ b/src/backend/access/spgist/spgvacuum.c
@@ -19,6 +19,7 @@
 #include "access/spgist_private.h"
 #include "access/spgxlog.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/storage_xlog.h"
 #include "commands/vacuum.h"
@@ -139,6 +140,7 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 	int			nDeletable;
 	OffsetNumber i,
 				max = PageGetMaxOffsetNumber(page);
+	bool		needwal;
 
 	memset(predecessor, 0, sizeof(predecessor));
 	memset(deletable, 0, sizeof(deletable));
@@ -323,6 +325,10 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 	if (nDeletable != xlrec.nDead + xlrec.nPlaceholder + xlrec.nMove)
 		elog(ERROR, "inconsistent counts of deletable tuples");
 
+	needwal = RelationNeedsWAL(index);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Do the updates */
 	START_CRIT_SECTION();
 
@@ -371,7 +377,7 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 
@@ -411,6 +417,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 	OffsetNumber toDelete[MaxIndexTuplesPerPage];
 	OffsetNumber i,
 				max = PageGetMaxOffsetNumber(page);
+	bool		needwal;
 
 	xlrec.nDelete = 0;
 
@@ -447,6 +454,10 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 	if (xlrec.nDelete == 0)
 		return;					/* nothing more to do */
 
+	needwal = RelationNeedsWAL(index);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Do the update */
 	START_CRIT_SECTION();
 
@@ -455,7 +466,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 
@@ -502,6 +513,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer)
 	OffsetNumber itemnos[MaxIndexTuplesPerPage];
 	spgxlogVacuumRedirect xlrec;
 	GlobalVisState *vistest;
+	bool		needwal;
 
 	xlrec.nToPlaceholder = 0;
 	xlrec.newestRedirectXid = InvalidTransactionId;
@@ -509,6 +521,10 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer)
 	/* XXX: providing heap relation would allow more pruning */
 	vistest = GlobalVisTestFor(NULL);
 
+	needwal = RelationNeedsWAL(index);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -584,7 +600,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer)
 	if (hasUpdate)
 		MarkBufferDirty(buffer);
 
-	if (hasUpdate && RelationNeedsWAL(index))
+	if (hasUpdate && needwal)
 	{
 		XLogRecPtr	recptr;
 
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index b8bedca04a4..0a88740764f 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -73,6 +73,7 @@
 #include "access/transam.h"
 #include "access/twophase.h"
 #include "access/twophase_rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1143,6 +1144,8 @@ GetNewMultiXactId(int nmembers, MultiXactOffset *offset)
 
 	ExtendMultiXactMember(nextOffset, nmembers);
 
+	CheckWALPermitted();
+
 	/*
 	 * Critical section from here until caller has written the data into the
 	 * just-reserved SLRU space; we don't want to error out with a partly
@@ -2938,7 +2941,7 @@ TruncateMultiXact(MultiXactId newOldestMulti, Oid newOldestMultiDB)
 	mxtruncinfo trunc;
 	MultiXactId earliest;
 
-	Assert(!RecoveryInProgress());
+	Assert(XLogInsertAllowed());
 	Assert(MultiXactState->finishedStartup);
 
 	/*
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index ef4f9981e35..ff2bc8cc74b 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -82,6 +82,7 @@
 #include "access/transam.h"
 #include "access/twophase.h"
 #include "access/twophase_rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1106,6 +1107,8 @@ EndPrepare(GlobalTransaction gxact)
 	 */
 	XLogEnsureRecordSpace(0, records.num_chunks);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	MyProc->delayChkpt = true;
@@ -2196,6 +2199,9 @@ RecordTransactionCommitPrepared(TransactionId xid,
 	replorigin = (replorigin_session_origin != InvalidRepOriginId &&
 				  replorigin_session_origin != DoNotReplicateId);
 
+	/* COMMIT PREPARED need not have an XID */
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* See notes in RecordTransactionCommit */
@@ -2286,6 +2292,9 @@ RecordTransactionAbortPrepared(TransactionId xid,
 		elog(PANIC, "cannot abort transaction %u, it was already committed",
 			 xid);
 
+	/* ROLLBACK PREPARED need not have an XID */
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
diff --git a/src/backend/access/transam/varsup.c b/src/backend/access/transam/varsup.c
index a4944faa32e..0c7a2362f25 100644
--- a/src/backend/access/transam/varsup.c
+++ b/src/backend/access/transam/varsup.c
@@ -17,6 +17,7 @@
 #include "access/commit_ts.h"
 #include "access/subtrans.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "commands/dbcommands.h"
@@ -75,6 +76,9 @@ GetNewTransactionId(bool isSubXact)
 	if (RecoveryInProgress())
 		elog(ERROR, "cannot assign TransactionIds during recovery");
 
+	/* do not assign transaction id in read-only mode */
+	CheckWALPermitted();
+
 	LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
 
 	full_xid = ShmemVariableCache->nextXid;
diff --git a/src/backend/access/transam/walprohibit.c b/src/backend/access/transam/walprohibit.c
index fc6d6c975f4..d1257d201a0 100644
--- a/src/backend/access/transam/walprohibit.c
+++ b/src/backend/access/transam/walprohibit.c
@@ -24,6 +24,16 @@
 #include "utils/fmgroids.h"
 #include "utils/fmgrprotos.h"
 
+/*
+ * Assert flag to enforce WAL insert permission check rule before starting a
+ * critical section for the WAL writes.  For this, either of CheckWALPermitted,
+ * AssertWALPermittedHaveXID, or AssertWALPermitted must be called before
+ * starting the critical section.
+ */
+#ifdef USE_ASSERT_CHECKING
+WALPermitCheckState walpermit_checked_state = WALPERMIT_UNCHECKED;
+#endif
+
 /*
  * Shared-memory WAL prohibit state
  */
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 188c299bed9..abda095e735 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -26,6 +26,7 @@
 #include "access/subtrans.h"
 #include "access/transam.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1317,6 +1318,8 @@ RecordTransactionCommit(void)
 		/* Tell bufmgr and smgr to prepare for commit */
 		BufmgrCommit();
 
+		AssertWALPermittedHaveXID();
+
 		/*
 		 * Mark ourselves as within our "commit critical section".  This
 		 * forces any concurrent checkpoint to wait until we've updated
@@ -1677,6 +1680,9 @@ RecordTransactionAbort(bool isSubXact)
 		elog(PANIC, "cannot abort transaction %u, it was already committed",
 			 xid);
 
+	/* We'll be reaching here with valid XID only. */
+	AssertWALPermittedHaveXID();
+
 	/* Fetch the data we need for the abort record */
 	nrels = smgrGetPendingDeletes(false, &rels);
 	nchildren = xactGetCommittedChildren(&children);
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 6a192e01211..974bb36c51b 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -1024,7 +1024,7 @@ XLogInsertRecord(XLogRecData *rdata,
 
 	/* cross-check on whether we should be here or not */
 	if (!XLogInsertAllowed())
-		elog(ERROR, "cannot make new WAL entries during recovery");
+		elog(ERROR, "cannot make new WAL entries now");
 
 	/*----------
 	 *
@@ -2859,9 +2859,11 @@ XLogFlush(XLogRecPtr record)
 	 * trying to flush the WAL, we should update minRecoveryPoint instead. We
 	 * test XLogInsertAllowed(), not InRecovery, because we need checkpointer
 	 * to act this way too, and because when it tries to write the
-	 * end-of-recovery checkpoint, it should indeed flush.
+	 * end-of-recovery checkpoint, it should indeed flush. Also, WAL prohibit
+	 * state should not restrict WAL flushing. Otherwise, the dirty buffer
+	 * cannot be evicted until WAL has been flushed up to the buffer's LSN.
 	 */
-	if (!XLogInsertAllowed())
+	if (!XLogInsertAllowed() && !IsWALProhibited())
 	{
 		UpdateMinRecoveryPoint(record, false);
 		return;
@@ -8818,6 +8820,8 @@ CreateCheckPoint(int flags)
 	/* sanity check */
 	if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
 		elog(ERROR, "can't create a checkpoint during recovery");
+	if (!XLogInsertAllowed() && !RecoveryInProgress())
+		elog(ERROR, "can't create a checkpoint while system is read only ");
 
 	/*
 	 * Initialize InitXLogInsert working areas before entering the critical
@@ -8847,6 +8851,8 @@ CreateCheckPoint(int flags)
 	MemSet(&CheckpointStats, 0, sizeof(CheckpointStats));
 	CheckpointStats.ckpt_start_t = GetCurrentTimestamp();
 
+	AssertWALPermitted();
+
 	/*
 	 * Use a critical section to force system panic if we have trouble.
 	 */
@@ -9075,6 +9081,8 @@ CreateCheckPoint(int flags)
 	if (!shutdown && XLogStandbyInfoActive())
 		LogStandbySnapshot();
 
+	AssertWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -9232,6 +9240,8 @@ CreateEndOfRecoveryRecord(void)
 
 	LocalSetXLogInsertAllowed();
 
+	AssertWALPermitted();
+
 	START_CRIT_SECTION();
 
 	XLogBeginInsert();
@@ -9878,7 +9888,7 @@ void
 UpdateFullPageWrites(void)
 {
 	XLogCtlInsert *Insert = &XLogCtl->Insert;
-	bool		recoveryInProgress;
+	bool		WALInsertAllowed;
 
 	/*
 	 * Do nothing if full_page_writes has not been changed.
@@ -9892,10 +9902,10 @@ UpdateFullPageWrites(void)
 
 	/*
 	 * Perform this outside critical section so that the WAL insert
-	 * initialization done by RecoveryInProgress() doesn't trigger an
+	 * initialization done by XLogInsertAllowed() doesn't trigger an
 	 * assertion failure.
 	 */
-	recoveryInProgress = RecoveryInProgress();
+	WALInsertAllowed = XLogInsertAllowed();
 
 	START_CRIT_SECTION();
 
@@ -9917,8 +9927,11 @@ UpdateFullPageWrites(void)
 	 * Write an XLOG_FPW_CHANGE record. This allows us to keep track of
 	 * full_page_writes during archive recovery, if required.
 	 */
-	if (XLogStandbyInfoActive() && !recoveryInProgress)
+	if (XLogStandbyInfoActive() && WALInsertAllowed)
 	{
+		/* Assured that WAL permission has been checked */
+		AssertWALPermitted();
+
 		XLogBeginInsert();
 		XLogRegisterData((char *) (&fullPageWrites), sizeof(bool));
 
diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c
index c526bb19281..506d7e97f38 100644
--- a/src/backend/access/transam/xloginsert.c
+++ b/src/backend/access/transam/xloginsert.c
@@ -19,6 +19,7 @@
 
 #include "postgres.h"
 
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xlog_internal.h"
@@ -126,9 +127,14 @@ XLogBeginInsert(void)
 	Assert(mainrdata_last == (XLogRecData *) &mainrdata_head);
 	Assert(mainrdata_len == 0);
 
+	/*
+	 * WAL permission must have checked before entering the critical section.
+	 * Otherwise, WAL prohibited error will force system panic.
+	 */
+	Assert(walpermit_checked_state != WALPERMIT_UNCHECKED || !CritSectionCount);
+
 	/* cross-check on whether we should be here or not */
-	if (!XLogInsertAllowed())
-		elog(ERROR, "cannot make new WAL entries during recovery");
+	CheckWALPermitted();
 
 	if (begininsert_called)
 		elog(ERROR, "XLogBeginInsert was already called");
@@ -210,6 +216,9 @@ XLogResetInsertion(void)
 	mainrdata_last = (XLogRecData *) &mainrdata_head;
 	curinsert_flags = 0;
 	begininsert_called = false;
+
+	/* Reset walpermit_checked flag */
+	RESET_WALPERMIT_CHECKED_STATE();
 }
 
 /*
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 6aab73bfd44..8dacf48db24 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -20,6 +20,7 @@
 #include "access/relation.h"
 #include "access/table.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -378,8 +379,13 @@ fill_seq_with_data(Relation rel, HeapTuple tuple)
 
 	/* check the comment above nextval_internal()'s equivalent call. */
 	if (RelationNeedsWAL(rel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermittedHaveXID();
+	}
+
 	START_CRIT_SECTION();
 
 	MarkBufferDirty(buf);
@@ -766,8 +772,13 @@ nextval_internal(Oid relid, bool check_permissions)
 	 * (Have to do that here, so we're outside the critical section)
 	 */
 	if (logit && RelationNeedsWAL(seqrel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermittedHaveXID();
+	}
+
 	/* ready to change the on-disk (or really, in-buffer) tuple */
 	START_CRIT_SECTION();
 
@@ -977,8 +988,13 @@ do_setval(Oid relid, int64 next, bool iscalled)
 
 	/* check the comment above nextval_internal()'s equivalent call. */
 	if (RelationNeedsWAL(seqrel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermittedHaveXID();
+	}
+
 	/* ready to change the on-disk (or really, in-buffer) tuple */
 	START_CRIT_SECTION();
 
diff --git a/src/backend/commands/variable.c b/src/backend/commands/variable.c
index 484f7ea2c0e..acc34d2ad7c 100644
--- a/src/backend/commands/variable.c
+++ b/src/backend/commands/variable.c
@@ -501,11 +501,14 @@ check_transaction_read_only(bool *newval, void **extra, GucSource source)
 			GUC_check_errmsg("transaction read-write mode must be set before any query");
 			return false;
 		}
-		/* Can't go to r/w mode while recovery is still active */
-		if (RecoveryInProgress())
+		/*
+		 * Can't go to r/w mode while recovery is still active or while in WAL
+		 * prohibit state
+		 */
+		if (!XLogInsertAllowed())
 		{
 			GUC_check_errcode(ERRCODE_FEATURE_NOT_SUPPORTED);
-			GUC_check_errmsg("cannot set transaction read-write mode during recovery");
+			GUC_check_errmsg("cannot set transaction read-write mode while system is read only");
 			return false;
 		}
 	}
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 97d8a62fd06..3ff6b954f1b 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -944,6 +944,10 @@ RequestCheckpoint(int flags)
 	int			old_failed,
 				old_started;
 
+	/* The checkpoint is allowed in recovery but not in WAL prohibit state */
+	if (!RecoveryInProgress())
+		CheckWALPermitted();
+
 	/*
 	 * If in a standalone backend, just do it ourselves.
 	 */
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index a2a963bd5b4..186cc47be1d 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -3638,13 +3638,15 @@ MarkBufferDirtyHint(Buffer buffer, bool buffer_std)
 		{
 			/*
 			 * If we must not write WAL, due to a relfilenode-specific
-			 * condition or being in recovery, don't dirty the page.  We can
-			 * set the hint, just not dirty the page as a result so the hint
-			 * is lost when we evict the page or shutdown.
+			 * condition or in general, don't dirty the page.  We can
+			 * set the hint, but must not dirty the page as a result, lest
+			 * we trigger WAL generation. Unless the page is dirtied again
+			 * later, the hint will be lost when the page is evicted, or at
+			 * shutdown.
 			 *
 			 * See src/backend/storage/page/README for longer discussion.
 			 */
-			if (RecoveryInProgress() ||
+			if (!XLogInsertAllowed() ||
 				RelFileNodeSkippingWAL(bufHdr->tag.rnode))
 				return;
 
diff --git a/src/backend/storage/freespace/freespace.c b/src/backend/storage/freespace/freespace.c
index 6a96126b0c2..b05b0fe5f41 100644
--- a/src/backend/storage/freespace/freespace.c
+++ b/src/backend/storage/freespace/freespace.c
@@ -24,6 +24,7 @@
 #include "postgres.h"
 
 #include "access/htup_details.h"
+#include "access/walprohibit.h"
 #include "access/xlogutils.h"
 #include "miscadmin.h"
 #include "storage/freespace.h"
@@ -285,12 +286,19 @@ FreeSpaceMapPrepareTruncateRel(Relation rel, BlockNumber nblocks)
 	 */
 	if (first_removed_slot > 0)
 	{
+		bool needwal;
+
 		buf = fsm_readbuf(rel, first_removed_address, false);
 		if (!BufferIsValid(buf))
 			return InvalidBlockNumber;	/* nothing to do; the FSM was already
 										 * smaller */
 		LockBuffer(buf, BUFFER_LOCK_EXCLUSIVE);
 
+		needwal = (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded());
+
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -305,7 +313,7 @@ FreeSpaceMapPrepareTruncateRel(Relation rel, BlockNumber nblocks)
 		 * during recovery.
 		 */
 		MarkBufferDirty(buf);
-		if (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded())
+		if (needwal)
 			log_newpage_buffer(buf, false);
 
 		END_CRIT_SECTION();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index d86566f4554..f949a290745 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -794,15 +794,15 @@ LockAcquireExtended(const LOCKTAG *locktag,
 	if (lockmode <= 0 || lockmode > lockMethodTable->numLockModes)
 		elog(ERROR, "unrecognized lock mode: %d", lockmode);
 
-	if (RecoveryInProgress() && !InRecovery &&
+	if (!XLogInsertAllowed() && !InRecovery &&
 		(locktag->locktag_type == LOCKTAG_OBJECT ||
 		 locktag->locktag_type == LOCKTAG_RELATION) &&
 		lockmode > RowExclusiveLock)
 		ereport(ERROR,
 				(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
-				 errmsg("cannot acquire lock mode %s on database objects while recovery is in progress",
+				 errmsg("cannot acquire lock mode %s on database objects while system is read only",
 						lockMethodTable->lockModeNames[lockmode]),
-				 errhint("Only RowExclusiveLock or less can be acquired on database objects during recovery.")));
+				 errhint("Only RowExclusiveLock or less can be acquired on database objects while system is read only")));
 
 #ifdef LOCK_DEBUG
 	if (LOCK_DEBUG_ENABLED(locktag))
diff --git a/src/backend/utils/cache/relmapper.c b/src/backend/utils/cache/relmapper.c
index 671fbb0ed5c..90d7599a57c 100644
--- a/src/backend/utils/cache/relmapper.c
+++ b/src/backend/utils/cache/relmapper.c
@@ -43,6 +43,7 @@
 #include <sys/stat.h>
 #include <unistd.h>
 
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -841,6 +842,8 @@ write_relmap_file(bool shared, RelMapFile *newmap,
 		xl_relmap_update xlrec;
 		XLogRecPtr	lsn;
 
+		AssertWALPermittedHaveXID();
+
 		/* now errors are fatal ... */
 		START_CRIT_SECTION();
 
diff --git a/src/include/access/walprohibit.h b/src/include/access/walprohibit.h
index 9d308a11cd0..574a4eba1a4 100644
--- a/src/include/access/walprohibit.h
+++ b/src/include/access/walprohibit.h
@@ -24,8 +24,8 @@ extern void MarkCheckPointSkippedInWalProhibitState(void);
 extern void WALProhibitStateShmemInit(void);
 
 /* WAL Prohibit States */
-#define	WALPROHIBIT_STATE_READ_WRITE		0x0000
-#define	WALPROHIBIT_STATE_READ_ONLY			0x0001
+#define	WALPROHIBIT_STATE_READ_WRITE		0x0000	/* WAL permitted */
+#define	WALPROHIBIT_STATE_READ_ONLY			0x0001	/* WAL prohibited */
 
 /*
  * The bit is used in state transition from one state to another.  When this
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e33523984..f3ff120601e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -94,12 +94,37 @@ extern PGDLLIMPORT volatile uint32 CritSectionCount;
 /* in tcop/postgres.c */
 extern void ProcessInterrupts(void);
 
+#ifdef USE_ASSERT_CHECKING
+typedef enum
+{
+	WALPERMIT_UNCHECKED,
+	WALPERMIT_CHECKED,
+	WALPERMIT_CHECKED_AND_USED
+} WALPermitCheckState;
+
+/* in access/walprohibit.c */
+extern WALPermitCheckState walpermit_checked_state;
+
+/*
+ * Reset walpermit_checked flag when no longer in the critical section.
+ * Otherwise, marked checked and used.
+ */
+#define RESET_WALPERMIT_CHECKED_STATE() \
+do { \
+	walpermit_checked_state = CritSectionCount ? \
+	WALPERMIT_CHECKED_AND_USED : WALPERMIT_UNCHECKED; \
+} while(0)
+#else
+#define RESET_WALPERMIT_CHECKED_STATE() ((void) 0)
+#endif
+
 #ifndef WIN32
 
 #define CHECK_FOR_INTERRUPTS() \
 do { \
 	if (unlikely(InterruptPending)) \
 		ProcessInterrupts(); \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 #else							/* WIN32 */
 
@@ -109,6 +134,7 @@ do { \
 		pgwin32_dispatch_queued_signals(); \
 	if (unlikely(InterruptPending)) \
 		ProcessInterrupts(); \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 #endif							/* WIN32 */
 
@@ -135,6 +161,7 @@ do { \
 do { \
 	Assert(CritSectionCount > 0); \
 	CritSectionCount--; \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 
 
-- 
2.22.0

From fcd2b4368f7b37aba497c8a314a2929b6c31e7d0 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 19 Jun 2020 06:29:36 -0400
Subject: [PATCH v7 3/5] WIP - Implement ALTER SYSTEM READ ONLY using global
 barrier.

Implementation:

 1. When a user tried to change server state to WAL-Prohibited using
    ALTER SYSTEM READ ONLY command; AlterSystemSetWALProhibitState()
    raises request to checkpointer by marking current state to inprogress in
    shared memory.  Checkpointer, noticing that the current state has
    WALPROHIBIT_TRANSITION_IN_PROGRESS flag set, does the barrier request, and
    then acknowledges back to the backend who requested the state change once
    the transition has been completed.  Final state will be updated in control
    file to make it persistent across the system restarts.

 2. When a backend receives the WAL-Prohibited barrier, at that moment if
    it is already in a transaction and the transaction already assigned XID,
    then the backend will be killed by throwing FATAL(XXX: need more discussion
    on this)

 3. Otherwise, if that backend running transaction without valid XID then, we
    don't need to do anything special right now, simply call
    ResetLocalXLogInsertAllowed() so that any future WAL insert in will check
    XLogInsertAllowed() first which set ready only state appropriately.

 4. A new transaction (in an existing or in a new backend) starts as a
    read-only transaction.

 5. Autovacuum launcher as well as checkpointer will not do anything in
    WAL-Prohibited server state until someone wakes us up.  E.g. a backend
    might later on request us to put the system back to read-write.
 5. The Autovacuum launcher, as well as the checkpointer, will not do
    anything while in the WAL-Prohibited server state until someone wakes
    up.  E.g. user might, later on, request us to put the system back to
    read-write by executing ALTER SYSTEM READ WRITE.

 6. At shutdown in WAL-Prohibited mode, we'll skip shutdown checkpoint
    and xlog rotation. Starting up again will perform crash recovery(XXX:
    need some discussion on this as well) but the end of recovery checkpoint
    will be skipped and it will be performed when the system changed to
    WAL-Permitted mode.

 7. ALTER SYSTEM READ ONLY/WRITE is restricted on standby server.

 8. To execute ALTER SYSTEM READ ONLY/WRITE, the user should have execute
    permssion on pg_alter_wal_prohibit_state() function.

 9. Add system_is_read_only GUC show the system state -- will true when system
    is wal prohibited or in recovery.

======
TODOs
======
 1. TBC, WALProhibitLock is added to protect shared walprohibit state, is
    that correct or overkilled?

 2. TBC, not using any lock in GetWALProhibitState() to read the state.
    Since shared wal prohibit state can be updated only with exclusive
    WALProhibitLock, so reading via volatile pointer should be enough?

 3. TBC, name of SQL callable function picked is
    pg_alter_wal_prohibit_state(), could be better.

 4. TBC, Error message and hint added in AlterSystemSetWALProhibitState().

 5. TBC, SetWALProhibitState() parameter is changed as per Andres' suggestion.
---
 src/backend/access/transam/Makefile      |   1 +
 src/backend/access/transam/walprohibit.c | 350 +++++++++++++++++++++++
 src/backend/access/transam/xact.c        |  37 +--
 src/backend/access/transam/xlog.c        | 114 +++++++-
 src/backend/catalog/system_views.sql     |   2 +
 src/backend/postmaster/autovacuum.c      |   4 +
 src/backend/postmaster/bgwriter.c        |   2 +-
 src/backend/postmaster/checkpointer.c    |  37 +++
 src/backend/postmaster/pgstat.c          |   3 +
 src/backend/storage/ipc/ipci.c           |   6 +
 src/backend/storage/ipc/procsignal.c     |  26 +-
 src/backend/storage/lmgr/lwlocknames.txt |   1 +
 src/backend/tcop/utility.c               |  15 +-
 src/backend/utils/misc/guc.c             |  26 ++
 src/bin/pg_controldata/pg_controldata.c  |   2 +
 src/include/access/walprohibit.h         |  83 ++++++
 src/include/access/xlog.h                |   4 +
 src/include/catalog/pg_control.h         |   3 +
 src/include/catalog/pg_proc.dat          |   4 +
 src/include/pgstat.h                     |   1 +
 src/include/postmaster/bgwriter.h        |   2 +
 src/include/storage/procsignal.h         |   7 +-
 src/tools/pgindent/typedefs.list         |   1 +
 23 files changed, 662 insertions(+), 69 deletions(-)
 create mode 100644 src/backend/access/transam/walprohibit.c
 create mode 100644 src/include/access/walprohibit.h

diff --git a/src/backend/access/transam/Makefile b/src/backend/access/transam/Makefile
index 595e02de722..b5322a69954 100644
--- a/src/backend/access/transam/Makefile
+++ b/src/backend/access/transam/Makefile
@@ -26,6 +26,7 @@ OBJS = \
 	twophase.o \
 	twophase_rmgr.o \
 	varsup.o \
+	walprohibit.o \
 	xact.o \
 	xlog.o \
 	xlogarchive.o \
diff --git a/src/backend/access/transam/walprohibit.c b/src/backend/access/transam/walprohibit.c
new file mode 100644
index 00000000000..fc6d6c975f4
--- /dev/null
+++ b/src/backend/access/transam/walprohibit.c
@@ -0,0 +1,350 @@
+/*-------------------------------------------------------------------------
+ *
+ * walprohibit.c
+ * 		PostgreSQL write-ahead log prohibit states
+ *
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/backend/access/transam/walprohibit.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/walprohibit.h"
+#include "fmgr.h"
+#include "pgstat.h"
+#include "port/atomics.h"
+#include "postmaster/bgwriter.h"
+#include "storage/condition_variable.h"
+#include "storage/procsignal.h"
+#include "storage/shmem.h"
+#include "utils/acl.h"
+#include "utils/fmgroids.h"
+#include "utils/fmgrprotos.h"
+
+/*
+ * Shared-memory WAL prohibit state
+ */
+typedef struct WALProhibitStateData
+{
+	/*
+	 * Indicates current WAL prohibit state.
+	 * Update protected by WALProhibitLock.
+	 */
+	uint32		shared_walprohibit_state;
+
+	/* Signaled when requested WAL prohibit state changes */
+	ConditionVariable walprohibit_cv;
+} WALProhibitStateData;
+
+static WALProhibitStateData *WALProhibitState = NULL;
+
+static void RequestWALProhibitChange(void);
+
+/*
+ * ProcessBarrierWALProhibit()
+ *
+ * Handle WAL prohibit state change request.
+ */
+bool
+ProcessBarrierWALProhibit(void)
+{
+	/*
+	 * Kill off any transactions that have an XID *before* allowing the system
+	 * to go WAL prohibit state.
+	 */
+	if (FullTransactionIdIsValid(GetTopFullTransactionIdIfAny()))
+	{
+		/* Should be here only for the WAL prohibit state. */
+		Assert(GetWALProhibitState() & WALPROHIBIT_STATE_READ_ONLY);
+
+		/*
+		 * XXX: Kill off the whole session by throwing FATAL instead of
+		 * killing transaction by throwing ERROR due to following reasons that
+		 * need be thought:
+		 *
+		 * 1. Due to some presents challenges with the wire protocol, we could
+		 * not simply kill of idle transaction.
+		 *
+		 * 2. If we are here in subtransaction then the ERROR will kill the
+		 * current subtransaction only. In the case of invalidations, that
+		 * might be good enough, but for XID assignment it's not, because
+		 * assigning an XID to a subtransaction also causes higher
+		 * sub-transaction levels and the parent transaction to get XIDs.
+		 */
+		ereport(FATAL,
+				(errcode(ERRCODE_ACTIVE_SQL_TRANSACTION),
+				 errmsg("system is now read only"),
+				 errhint("Sessions with open write transactions must be terminated.")));
+	}
+
+	/* Return to "check" state */
+	ResetLocalXLogInsertAllowed();
+
+	return true;
+}
+
+/*
+ * AlterSystemSetWALProhibitState()
+ *
+ * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
+ */
+void
+AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
+{
+	/* Check permission for pg_alter_wal_prohibit_state() */
+	if (pg_proc_aclcheck(F_PG_ALTER_WAL_PROHIBIT_STATE,
+						 GetUserId(), ACL_EXECUTE) != ACLCHECK_OK)
+		ereport(ERROR,
+				(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+				 errmsg("permission denied for command ALTER SYSTEM"),
+				 errhint("Get execute permission for pg_alter_wal_prohibit_state() to this user.")));
+
+	/* Alter WAL prohibit state not allowed during recovery */
+	PreventCommandDuringRecovery("ALTER SYSTEM");
+
+	/* Execute function to alter wal prohibit state */
+	(void) OidFunctionCall1(F_PG_ALTER_WAL_PROHIBIT_STATE,
+							BoolGetDatum(stmt->walprohibited));
+}
+
+/*
+ * pg_alter_wal_prohibit_state()
+ *
+ * SQL callable function to alter system read write state.
+ */
+Datum
+pg_alter_wal_prohibit_state(PG_FUNCTION_ARGS)
+{
+	bool		walprohibited = PG_GETARG_BOOL(0);
+
+	/* Alter WAL prohibit state not allowed during recovery */
+	PreventCommandDuringRecovery("pg_alter_wal_prohibit_state()");
+
+	/*
+	 * It is not a final state since we yet to convey this WAL prohibit state to
+	 * all backend.
+	 */
+	if (!SetWALProhibitState(walprohibited, false))
+	{
+		/* Server is already in the desired state */
+		PG_RETURN_BOOL(false);
+	}
+
+	/*
+	 * Signal the checkpointer to do the actual state transition, and wait for
+	 * the state change to occur.
+	 */
+	RequestWALProhibitChange();
+
+	/* Server state changed to the desired state */
+	PG_RETURN_BOOL(true);
+}
+
+/*
+ * RequestWALProhibitChange()
+ *
+ * Request checkpointer to make the WALProhibitState to read-only.
+ */
+static void
+RequestWALProhibitChange(void)
+{
+	/* Must not be called from checkpointer */
+	Assert(!AmCheckpointerProcess());
+	Assert(GetWALProhibitState() & WALPROHIBIT_TRANSITION_IN_PROGRESS);
+
+	/*
+	 * If in a standalone backend, just do it ourselves.
+	 */
+	if (!IsPostmasterEnvironment)
+	{
+		CompleteWALProhibitChange(GetWALProhibitState());
+		return;
+	}
+
+	SendsSignalToCheckpointer(SIGINT);
+
+	/* Wait for the state to change to read-only */
+	ConditionVariablePrepareToSleep(&WALProhibitState->walprohibit_cv);
+	for (;;)
+	{
+		/* We'll be done once in-progress flag bit is cleared */
+		if (!(GetWALProhibitState() & WALPROHIBIT_TRANSITION_IN_PROGRESS))
+			break;
+
+		ConditionVariableSleep(&WALProhibitState->walprohibit_cv,
+							   WAIT_EVENT_WALPROHIBIT_STATE_CHANGE);
+	}
+	ConditionVariableCancelSleep();
+}
+
+/*
+ * CompleteWALProhibitChange()
+ *
+ * Checkpointer will call this to complete the requested WAL prohibit state
+ * transition.
+ */
+void
+CompleteWALProhibitChange(uint32 wal_state)
+{
+	uint64		barrierGeneration;
+	bool		wal_prohibited;
+
+	/*
+	 * Must be called from checkpointer. Otherwise, it must be single-user
+	 * backend.
+	 */
+	Assert(AmCheckpointerProcess() || !IsPostmasterEnvironment);
+	Assert(wal_state & WALPROHIBIT_TRANSITION_IN_PROGRESS);
+
+	/*
+	 * WAL prohibit state change is initiated. We need to complete the state
+	 * transition by setting requested WAL prohibit state in all backends.
+	 */
+	elog(DEBUG1, "waiting for backends to adopt requested WAL prohibit state");
+
+	/* Emit global barrier */
+	barrierGeneration = EmitProcSignalBarrier(PROCSIGNAL_BARRIER_WALPROHIBIT);
+	WaitForProcSignalBarrier(barrierGeneration);
+
+	/* And flush all inserts. */
+	XLogFlush(GetXLogInsertRecPtr());
+
+	wal_prohibited = (wal_state & WALPROHIBIT_STATE_READ_ONLY);
+
+	/* Set the final state */
+	if (SetWALProhibitState(wal_prohibited, true))
+	{
+		/* Update the control file to make state persistent */
+		SetControlFileWALProhibitFlag(wal_prohibited);
+
+		if (wal_prohibited)
+			ereport(LOG, (errmsg("system is now read only")));
+		else
+		{
+			/*
+			 * Request checkpoint if the end-of-recovery checkpoint has been
+			 * skipped previously.
+			 */
+			if (LastCheckPointIsSkipped())
+			{
+				RequestCheckpoint(CHECKPOINT_IMMEDIATE);
+				SetLastCheckPointSkipped(false);
+			}
+			ereport(LOG, (errmsg("system is now read write")));
+		}
+	}
+
+	/* Wake up all backends waiting on this. */
+	ConditionVariableBroadcast(&WALProhibitState->walprohibit_cv);
+}
+
+/*
+ * GetWALProhibitState()
+ *
+ * Return the current server WAL prohibited state
+ */
+uint32
+GetWALProhibitState(void)
+{
+	/*
+	 * Use volatile pointer to make sure we make a fresh read of the
+	 * shared variable.
+	 */
+	volatile WALProhibitStateData *cur_state = WALProhibitState;
+
+	return cur_state->shared_walprohibit_state;
+}
+
+/*
+ * SetWALProhibitState()
+ *
+ * Change current WAL prohibit state to the input state.
+ *
+ * If the server is already completely moved to the requested WAL prohibit
+ * state, or if the desired state is same as the current state, return false,
+ * indicating that the server state did not change. Else return true.
+ *
+ */
+bool
+SetWALProhibitState(bool wal_prohibited, bool is_final_state)
+{
+	uint32		new_state;
+	uint32		cur_state;
+
+	/*
+	 * Only checkpointer or startup process or single-user can set the final wal
+	 * prohibit state.
+	 */
+	Assert(!is_final_state ||  AmCheckpointerProcess() || AmStartupProcess() ||
+		   !IsPostmasterEnvironment);
+
+	/* Compute new state */
+	new_state = wal_prohibited ?
+		WALPROHIBIT_STATE_READ_ONLY : WALPROHIBIT_STATE_READ_WRITE;
+
+	if (!is_final_state)
+		new_state |= WALPROHIBIT_TRANSITION_IN_PROGRESS;
+
+	LWLockAcquire(WALProhibitLock, LW_EXCLUSIVE);
+
+	/* Get the current state */
+	cur_state = GetWALProhibitState();
+
+	/* Server is already in requested state */
+	if (new_state == cur_state ||
+		new_state == (cur_state | WALPROHIBIT_TRANSITION_IN_PROGRESS))
+	{
+		LWLockRelease(WALProhibitLock);
+		return false;
+	}
+
+	/* Prevent concurrent contrary in progress transition state setting */
+	if ((new_state & WALPROHIBIT_TRANSITION_IN_PROGRESS) &&
+		(cur_state & WALPROHIBIT_TRANSITION_IN_PROGRESS))
+	{
+		if (cur_state & WALPROHIBIT_STATE_READ_ONLY)
+			ereport(ERROR,
+					(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+					 errmsg("system state transition to read only is already in progress"),
+					 errhint("Try after sometime again.")));
+		else
+			ereport(ERROR,
+					(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+					 errmsg("system state transition to read write is already in progress"),
+					 errhint("Try after sometime again.")));
+	}
+
+	/* Update new state in share memory */
+	WALProhibitState->shared_walprohibit_state = new_state;
+
+	LWLockRelease(WALProhibitLock);
+
+	return true;
+}
+
+/*
+ * WALProhibitStateShmemInit()
+ *
+ * Initialization of shared memory for WAL prohibit state.
+ */
+void
+WALProhibitStateShmemInit(void)
+{
+	bool		found;
+
+	WALProhibitState = (WALProhibitStateData *)
+		ShmemInitStruct("WAL Prohibit State",
+						sizeof(WALProhibitStateData),
+						&found);
+
+	if (found)
+		return;
+
+	/* First time through ... */
+	memset(WALProhibitState, 0, sizeof(WALProhibitStateData));
+
+	ConditionVariableInit(&WALProhibitState->walprohibit_cv);
+}
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index af6afcebb13..188c299bed9 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -1962,23 +1962,28 @@ StartTransaction(void)
 	Assert(s->prevSecContext == 0);
 
 	/*
-	 * Make sure we've reset xact state variables
+	 * Reset xact state variables.
 	 *
-	 * If recovery is still in progress, mark this transaction as read-only.
-	 * We have lower level defences in XLogInsert and elsewhere to stop us
-	 * from modifying data during recovery, but this gives the normal
-	 * indication to the user that the transaction is read-only.
-	 */
-	if (RecoveryInProgress())
-	{
-		s->startedInRecovery = true;
-		XactReadOnly = true;
-	}
-	else
-	{
-		s->startedInRecovery = false;
-		XactReadOnly = DefaultXactReadOnly;
-	}
+	 * If it is not currently possible to insert write-ahead log records,
+	 * either because we are still in recovery or because ALTER SYSTEM READ
+	 * ONLY has been executed, force this to be a read-only transaction.
+	 * We have lower level defences in XLogBeginInsert() and elsewhere to stop
+	 * us from modifying data during recovery when !XLogInsertAllowed(), but
+	 * this gives the normal indication to the user that the transaction is
+	 * read-only.
+	 *
+	 * On the other hand, we only need to set the startedInRecovery flag when
+	 * the transaction started during recovery, and not when WAL is otherwise
+	 * prohibited. This information is used by RelationGetIndexScan() to
+	 * decide whether to permit (1) relying on existing killed-tuple markings
+	 * and (2) further killing of index tuples. Even when WAL is prohibited
+	 * on the master, it's still the master, so the former is OK; and since
+	 * killing index tuples doesn't generate WAL, the latter is also OK.
+	 * See comments in RelationGetIndexScan() and MarkBufferDirtyHint().
+	 */
+	XactReadOnly = DefaultXactReadOnly || !XLogInsertAllowed();
+	s->startedInRecovery = RecoveryInProgress();
+
 	XactDeferrable = DefaultXactDeferrable;
 	XactIsoLevel = DefaultXactIsoLevel;
 	forceSyncCommit = false;
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index a38371a64f9..6a192e01211 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -31,6 +31,7 @@
 #include "access/timeline.h"
 #include "access/transam.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog_internal.h"
 #include "access/xlogarchive.h"
@@ -245,9 +246,10 @@ static bool LocalPromoteIsTriggered = false;
  *		0: unconditionally not allowed to insert XLOG
  *		-1: must check RecoveryInProgress(); disallow until it is false
  * Most processes start with -1 and transition to 1 after seeing that recovery
- * is not in progress.  But we can also force the value for special cases.
- * The coding in XLogInsertAllowed() depends on the first two of these states
- * being numerically the same as bool true and false.
+ * is not in progress or the server state is not a WAL prohibited state.  But
+ * we can also force the value for special cases.  The coding in
+ * XLogInsertAllowed() depends on the first two of these states being
+ * numerically the same as bool true and false.
  */
 static int	LocalXLogInsertAllowed = -1;
 
@@ -722,6 +724,11 @@ typedef struct XLogCtlData
 	 */
 	XLogRecPtr	lastFpwDisableRecPtr;
 
+	/*
+	 * lastCheckPointSkipped indicates if the last checkpoint is skipped.
+	 */
+	bool		lastCheckPointSkipped;
+
 	slock_t		info_lck;		/* locks shared variables shown above */
 } XLogCtlData;
 
@@ -967,6 +974,7 @@ static void WALInsertLockAcquire(void);
 static void WALInsertLockAcquireExclusive(void);
 static void WALInsertLockRelease(void);
 static void WALInsertLockUpdateInsertingAt(XLogRecPtr insertingAt);
+static inline bool IsWALProhibited(void);
 
 /*
  * Insert an XLOG record represented by an already-constructed chain of data
@@ -6195,6 +6203,32 @@ SetCurrentChunkStartTime(TimestampTz xtime)
 	SpinLockRelease(&XLogCtl->info_lck);
 }
 
+/*
+ * Set or unset flag to indicating that the last checkpoint has been skipped.
+ */
+void
+SetLastCheckPointSkipped(bool ChkptSkip)
+{
+	SpinLockAcquire(&XLogCtl->info_lck);
+	XLogCtl->lastCheckPointSkipped = ChkptSkip;
+	SpinLockRelease(&XLogCtl->info_lck);
+}
+
+/*
+ * Return value of lastCheckPointSkipped flag.
+ */
+bool
+LastCheckPointIsSkipped(void)
+{
+	bool	ChkptSkipped;
+
+	SpinLockAcquire(&XLogCtl->info_lck);
+	ChkptSkipped = XLogCtl->lastCheckPointSkipped;
+	SpinLockRelease(&XLogCtl->info_lck);
+
+	return ChkptSkipped;
+}
+
 /*
  * Fetch timestamp of latest processed commit/abort record.
  * Startup process maintains an accurate local copy in XLogReceiptTime
@@ -7703,6 +7737,12 @@ StartupXLOG(void)
 	XLogCtl->LogwrtRqst.Write = EndOfLog;
 	XLogCtl->LogwrtRqst.Flush = EndOfLog;
 
+	/*
+	 * Before enabling WAL insertion update WAL prohibit state in shared memory
+	 * that will decide the further WAL insert should be allowed or not.
+	 */
+	(void) SetWALProhibitState(ControlFile->wal_prohibited, true);
+
 	/*
 	 * Update full_page_writes in shared memory and write an XLOG_FPW_CHANGE
 	 * record before resource manager writes cleanup WAL records or checkpoint
@@ -7713,7 +7753,17 @@ StartupXLOG(void)
 	UpdateFullPageWrites();
 	LocalXLogInsertAllowed = -1;
 
-	if (InRecovery)
+	/*
+	 * Skip end-of-recovery checkpoint if the system is in WAL prohibited state.
+	 */
+	if (ControlFile->wal_prohibited && InRecovery)
+	{
+		SetLastCheckPointSkipped(true);
+
+		ereport(LOG,
+				(errmsg("skipping startup checkpoint because the system is read only")));
+	}
+	else if (InRecovery)
 	{
 		/*
 		 * Perform a checkpoint to update all our recovery activity to disk.
@@ -7959,6 +8009,25 @@ StartupXLOG(void)
 		RequestCheckpoint(CHECKPOINT_FORCE);
 }
 
+/* Set ControlFile's WAL prohibit flag */
+void
+SetControlFileWALProhibitFlag(bool wal_prohibited)
+{
+	LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+	ControlFile->wal_prohibited = wal_prohibited;
+	UpdateControlFile();
+	LWLockRelease(ControlFileLock);
+}
+
+/*
+ * Is the system still in WAL prohibited state?
+ */
+static inline bool
+IsWALProhibited(void)
+{
+	return (GetWALProhibitState() & WALPROHIBIT_STATE_READ_ONLY) != 0;
+}
+
 /*
  * Checks if recovery has reached a consistent state. When consistency is
  * reached and we have a valid starting standby snapshot, tell postmaster
@@ -8174,9 +8243,9 @@ HotStandbyActiveInReplay(void)
 /*
  * Is this process allowed to insert new WAL records?
  *
- * Ordinarily this is essentially equivalent to !RecoveryInProgress().
- * But we also have provisions for forcing the result "true" or "false"
- * within specific processes regardless of the global state.
+ * Ordinarily this is essentially equivalent to !RecoveryInProgress() and
+ * !IsWALProhibited().  But we also have provisions for forcing the result
+ * "true" or "false" within specific processes regardless of the global state.
  */
 bool
 XLogInsertAllowed(void)
@@ -8195,9 +8264,20 @@ XLogInsertAllowed(void)
 	if (RecoveryInProgress())
 		return false;
 
+	/* Or, in WAL prohibited state */
+	if (IsWALProhibited())
+	{
+		/*
+		 * Set it to "unconditionally false" to avoid checking until it gets
+		 * reset.
+		 */
+		LocalXLogInsertAllowed = 0;
+		return false;
+	}
+
 	/*
-	 * On exit from recovery, reset to "unconditionally true", since there is
-	 * no need to keep checking.
+	 * On exit from recovery or WAL prohibited state, reset to "unconditionally
+	 * true", since there is no need to keep checking.
 	 */
 	LocalXLogInsertAllowed = 1;
 	return true;
@@ -8219,6 +8299,12 @@ LocalSetXLogInsertAllowed(void)
 	InitXLOGAccess();
 }
 
+void
+ResetLocalXLogInsertAllowed(void)
+{
+	LocalXLogInsertAllowed = -1;
+}
+
 /*
  * Subroutine to try to fetch and validate a prior checkpoint record.
  *
@@ -8508,9 +8594,13 @@ ShutdownXLOG(int code, Datum arg)
 	 */
 	WalSndWaitStopping();
 
+	/*
+	 * The restartpoint, checkpoint, or xlog rotation will be performed if the
+	 * WAL writing is permitted.
+	 */
 	if (RecoveryInProgress())
 		CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
-	else
+	else if (XLogInsertAllowed())
 	{
 		/*
 		 * If archiving is enabled, rotate the last XLOG file so that all the
@@ -8523,6 +8613,10 @@ ShutdownXLOG(int code, Datum arg)
 
 		CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
 	}
+	else
+		ereport(LOG,
+			   (errmsg("skipping shutdown checkpoint because the system is read only")));
+
 	ShutdownCLOG();
 	ShutdownCommitTs();
 	ShutdownSUBTRANS();
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index ed4f3f142d8..79da249dd5c 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1485,6 +1485,8 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
 REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
 REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
 
+REVOKE EXECUTE ON FUNCTION pg_alter_wal_prohibit_state(bool) FROM public;
+
 --
 -- We also set up some things as accessible to standard roles.
 --
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 19ba26b914e..d40d2bce7f8 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -658,6 +658,10 @@ AutoVacLauncherMain(int argc, char *argv[])
 
 		HandleAutoVacLauncherInterrupts();
 
+		/* If the server is read only just go back to sleep. */
+		if (!XLogInsertAllowed())
+			continue;
+
 		/*
 		 * a worker finished, or postmaster signaled failure to start a worker
 		 */
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index c96568149fe..3a8a40f7f3b 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -282,7 +282,7 @@ BackgroundWriterMain(void)
 		 * Checkpointer, when active, is barely ever in its mainloop and thus
 		 * makes it hard to log regularly.
 		 */
-		if (XLogStandbyInfoActive() && !RecoveryInProgress())
+		if (XLogStandbyInfoActive() && XLogInsertAllowed())
 		{
 			TimestampTz timeout = 0;
 			TimestampTz now = GetCurrentTimestamp();
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 45f5deca72e..97d8a62fd06 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -39,6 +39,7 @@
 #include <sys/time.h>
 #include <time.h>
 
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xlog_internal.h"
 #include "libpq/pqsignal.h"
@@ -346,6 +347,7 @@ CheckpointerMain(void)
 		pg_time_t	now;
 		int			elapsed_secs;
 		int			cur_timeout;
+		uint32		wal_state;
 
 		/* Clear any already-pending wakeups */
 		ResetLatch(MyLatch);
@@ -356,6 +358,28 @@ CheckpointerMain(void)
 		AbsorbSyncRequests();
 		HandleCheckpointerInterrupts();
 
+		wal_state = GetWALProhibitState();
+
+		if (wal_state & WALPROHIBIT_TRANSITION_IN_PROGRESS)
+		{
+			/* Complete WAL prohibit state change request */
+			CompleteWALProhibitChange(wal_state);
+			continue;
+		}
+		else if (wal_state & WALPROHIBIT_STATE_READ_ONLY)
+		{
+			/*
+			 * Don't do anything until someone wakes us up.  For example a
+			 * backend might later on request us to put the system back to
+			 * read-write wal prohibit sate.
+			 */
+			(void) WaitLatch(MyLatch, WL_LATCH_SET | WL_EXIT_ON_PM_DEATH, -1,
+							 WAIT_EVENT_CHECKPOINTER_MAIN);
+			continue;
+		}
+
+		Assert(wal_state == WALPROHIBIT_STATE_READ_WRITE);
+
 		/*
 		 * Detect a pending checkpoint request by checking whether the flags
 		 * word in shared memory is nonzero.  We shouldn't need to acquire the
@@ -1337,3 +1361,16 @@ FirstCallSinceLastCheckpoint(void)
 
 	return FirstCall;
 }
+
+/*
+ * SendsSignalToCheckpointer allows a process to send a signal to the checkpoint process.
+ */
+void
+SendsSignalToCheckpointer(int signum)
+{
+	if (CheckpointerShmem->checkpointer_pid == 0)
+		elog(ERROR, "checkpointer is not running");
+
+	if (kill(CheckpointerShmem->checkpointer_pid, signum) != 0)
+		elog(ERROR, "could not signal checkpointer: %m");
+}
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index e6be2b7836a..95a738d7f25 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -4060,6 +4060,9 @@ pgstat_get_wait_io(WaitEventIO w)
 		case WAIT_EVENT_REPLICATION_SLOT_WRITE:
 			event_name = "ReplicationSlotWrite";
 			break;
+		case WAIT_EVENT_WALPROHIBIT_STATE_CHANGE:
+			event_name = "SystemWALProhibitStateChange";
+			break;
 		case WAIT_EVENT_SLRU_FLUSH_SYNC:
 			event_name = "SLRUFlushSync";
 			break;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 96c2aaabbd6..2d000ec2ff7 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -22,6 +22,7 @@
 #include "access/subtrans.h"
 #include "access/syncscan.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "commands/async.h"
 #include "miscadmin.h"
 #include "pgstat.h"
@@ -222,6 +223,11 @@ CreateSharedMemoryAndSemaphores(void)
 	MultiXactShmemInit();
 	InitBufferPool();
 
+	/*
+	 * Set up wal probibit shared state
+	 */
+	WALProhibitStateShmemInit();
+
 	/*
 	 * Set up lock manager
 	 */
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 63cb70bcaa4..4f1b67f9d04 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -18,6 +18,7 @@
 #include <unistd.h>
 
 #include "access/parallel.h"
+#include "access/walprohibit.h"
 #include "commands/async.h"
 #include "miscadmin.h"
 #include "pgstat.h"
@@ -96,7 +97,6 @@ static volatile ProcSignalSlot *MyProcSignalSlot = NULL;
 
 static bool CheckProcSignal(ProcSignalReason reason);
 static void CleanupProcSignalState(int status, Datum arg);
-static bool ProcessBarrierPlaceholder(void);
 
 /*
  * ProcSignalShmemSize
@@ -510,9 +510,9 @@ ProcessProcSignalBarrier(void)
 			 * unconditionally, but it's more efficient to call only the ones
 			 * that might need us to do something based on the flags.
 			 */
-			if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_PLACEHOLDER)
-				&& ProcessBarrierPlaceholder())
-				BARRIER_CLEAR_BIT(flags, PROCSIGNAL_BARRIER_PLACEHOLDER);
+			if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_WALPROHIBIT)
+				&& ProcessBarrierWALProhibit())
+				BARRIER_CLEAR_BIT(flags, PROCSIGNAL_BARRIER_WALPROHIBIT);
 		}
 		PG_CATCH();
 		{
@@ -554,24 +554,6 @@ ProcessProcSignalBarrier(void)
 	pg_atomic_write_u64(&MyProcSignalSlot->pss_barrierGeneration, shared_gen);
 }
 
-static bool
-ProcessBarrierPlaceholder(void)
-{
-	/*
-	 * XXX. This is just a placeholder until the first real user of this
-	 * machinery gets committed. Rename PROCSIGNAL_BARRIER_PLACEHOLDER to
-	 * PROCSIGNAL_BARRIER_SOMETHING_ELSE where SOMETHING_ELSE is something
-	 * appropriately descriptive. Get rid of this function and instead have
-	 * ProcessBarrierSomethingElse. Most likely, that function should live in
-	 * the file pertaining to that subsystem, rather than here.
-	 *
-	 * The return value should be 'true' if the barrier was successfully
-	 * absorbed and 'false' if not. Note that returning 'false' can lead to
-	 * very frequent retries, so try hard to make that an uncommon case.
-	 */
-	return true;
-}
-
 /*
  * CheckProcSignal - check to see if a particular reason has been
  * signaled, and clear the signal flag.  Should be called after receiving
diff --git a/src/backend/storage/lmgr/lwlocknames.txt b/src/backend/storage/lmgr/lwlocknames.txt
index 774292fd942..6824ce0aa3f 100644
--- a/src/backend/storage/lmgr/lwlocknames.txt
+++ b/src/backend/storage/lmgr/lwlocknames.txt
@@ -53,3 +53,4 @@ XactTruncationLock					44
 # 45 was XactTruncationLock until removal of BackendRandomLock
 WrapLimitsVacuumLock				46
 NotifyQueueTailLock					47
+WALProhibitLock						48
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 74c2162cd59..05eac206182 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -19,6 +19,7 @@
 #include "access/htup_details.h"
 #include "access/reloptions.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "catalog/catalog.h"
@@ -85,7 +86,6 @@ static void ProcessUtilitySlow(ParseState *pstate,
 							   DestReceiver *dest,
 							   QueryCompletion *qc);
 static void ExecDropStmt(DropStmt *stmt, bool isTopLevel);
-static void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
 
 /*
  * CommandIsReadOnly: is an executable query read-only?
@@ -3691,16 +3691,3 @@ GetCommandLogLevel(Node *parsetree)
 
 	return lev;
 }
-
-/*
- * AlterSystemSetWALProhibitState
- *
- * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
- */
-static void
-AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
-{
-	ereport(ERROR,
-			(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
-			 errmsg("ALTER SYSTEM READ { ONLY | WRITE } not implemented")));
-}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 596bcb7b842..24113249f67 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -225,6 +225,7 @@ static bool check_recovery_target_lsn(char **newval, void **extra, GucSource sou
 static void assign_recovery_target_lsn(const char *newval, void *extra);
 static bool check_primary_slot_name(char **newval, void **extra, GucSource source);
 static bool check_default_with_oids(bool *newval, void **extra, GucSource source);
+static const char *show_system_is_read_only(void);
 
 /* Private functions in guc-file.l that need to be called from guc.c */
 static ConfigVariable *ProcessConfigFileInternal(GucContext context,
@@ -615,6 +616,7 @@ static char *recovery_target_string;
 static char *recovery_target_xid_string;
 static char *recovery_target_name_string;
 static char *recovery_target_lsn_string;
+static bool system_is_read_only;
 
 
 /* should be static, but commands/variable.c needs to get at this */
@@ -2036,6 +2038,18 @@ static struct config_bool ConfigureNamesBool[] =
 		NULL, NULL, NULL
 	},
 
+	{
+		/* Not for general use */
+		{"system_is_read_only", PGC_INTERNAL, WAL,
+			gettext_noop("Shows whether the system is read only."),
+			NULL,
+			GUC_NO_RESET_ALL | GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE
+		},
+		&system_is_read_only,
+		false,
+		NULL, NULL, show_system_is_read_only
+	},
+
 	/* End-of-list marker */
 	{
 		{NULL, 0, 0, NULL, NULL}, NULL, false, NULL, NULL, NULL
@@ -12041,4 +12055,16 @@ check_default_with_oids(bool *newval, void **extra, GucSource source)
 	return true;
 }
 
+/*
+ * NB: The return string should be the same as the _ShowOption() for boolean
+ * type.
+ */
+static const char *
+show_system_is_read_only(void)
+{
+	if (!XLogInsertAllowed())
+		return "on";
+	return "off";
+}
+
 #include "guc-file.c"
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index 3e00ac0f701..922cd9641d8 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -290,6 +290,8 @@ main(int argc, char *argv[])
 		   (uint32) ControlFile->backupEndPoint);
 	printf(_("End-of-backup record required:        %s\n"),
 		   ControlFile->backupEndRequired ? _("yes") : _("no"));
+	printf(_("WAL write prohibited:                 %s\n"),
+		   ControlFile->wal_prohibited ? _("yes") : _("no"));
 	printf(_("wal_level setting:                    %s\n"),
 		   wal_level_str(ControlFile->wal_level));
 	printf(_("wal_log_hints setting:                %s\n"),
diff --git a/src/include/access/walprohibit.h b/src/include/access/walprohibit.h
new file mode 100644
index 00000000000..9d308a11cd0
--- /dev/null
+++ b/src/include/access/walprohibit.h
@@ -0,0 +1,83 @@
+/*
+ * walprohibit.h
+ *
+ * PostgreSQL write-ahead log prohibit states
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/include/access/walprohibit.h
+ */
+#ifndef WALPROHIBIT_H
+#define WALPROHIBIT_H
+
+#include "access/xact.h"
+#include "access/xlog.h"
+#include "miscadmin.h"
+#include "nodes/parsenodes.h"
+
+extern bool ProcessBarrierWALProhibit(void);
+extern void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
+extern void CompleteWALProhibitChange(uint32 wal_state);
+extern uint32 GetWALProhibitState(void);
+extern bool SetWALProhibitState(bool wal_prohibited, bool is_final_state);
+extern void MarkCheckPointSkippedInWalProhibitState(void);
+extern void WALProhibitStateShmemInit(void);
+
+/* WAL Prohibit States */
+#define	WALPROHIBIT_STATE_READ_WRITE		0x0000
+#define	WALPROHIBIT_STATE_READ_ONLY			0x0001
+
+/*
+ * The bit is used in state transition from one state to another.  When this
+ * bit is set then the state indicated by the 0th position bit is yet to
+ * confirmed.
+ */
+#define WALPROHIBIT_TRANSITION_IN_PROGRESS	0x0002
+
+/* Never reaches when WAL is prohibited. */
+static inline void
+AssertWALPermitted(void)
+{
+	/*
+	 * Recovery in the startup process never is in wal prohibited state.
+	 */
+	Assert(InRecovery || XLogInsertAllowed());
+
+#ifdef USE_ASSERT_CHECKING
+	walpermit_checked_state = WALPERMIT_CHECKED;
+#endif
+}
+
+/*
+ * XID-bearing transactions are killed off by "ALTER SYSTEM READ ONLY", so any
+ * part of the code that can only be reached with an XID assigned is never
+ * reached when WAL is prohibited.
+ */
+static inline void
+AssertWALPermittedHaveXID(void)
+{
+	/* Must be performing an INSERT, UPDATE or DELETE, so we'll have an XID */
+	Assert(FullTransactionIdIsValid(GetTopFullTransactionIdIfAny()));
+	AssertWALPermitted();
+}
+
+/*
+ * In opposite to the above assertion if a transaction doesn't have valid XID
+ * (e.g. VACUUM) then it won't be killed while changing the system state to WAL
+ * prohibited.  Therefore, we need to explicitly error out before entering into
+ * the critical section.
+ */
+static inline void
+CheckWALPermitted(void)
+{
+	if (!XLogInsertAllowed())
+		ereport(ERROR,
+				(errcode(ERRCODE_READ_ONLY_SQL_TRANSACTION),
+				 errmsg("system is now read only")));
+
+#ifdef USE_ASSERT_CHECKING
+	walpermit_checked_state = WALPERMIT_CHECKED;
+#endif
+}
+
+#endif							/* WALPROHIBIT_H */
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 221af87e715..2bcd37894f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -306,6 +306,7 @@ extern RecoveryState GetRecoveryState(void);
 extern bool HotStandbyActive(void);
 extern bool HotStandbyActiveInReplay(void);
 extern bool XLogInsertAllowed(void);
+extern void ResetLocalXLogInsertAllowed(void);
 extern void GetXLogReceiptTime(TimestampTz *rtime, bool *fromStream);
 extern XLogRecPtr GetXLogReplayRecPtr(TimeLineID *replayTLI);
 extern XLogRecPtr GetXLogInsertRecPtr(void);
@@ -314,6 +315,8 @@ extern bool RecoveryIsPaused(void);
 extern void SetRecoveryPause(bool recoveryPause);
 extern TimestampTz GetLatestXTime(void);
 extern TimestampTz GetCurrentChunkReplayStartTime(void);
+extern void SetLastCheckPointSkipped(bool ChkptSkip);
+extern bool LastCheckPointIsSkipped(void);
 
 extern void UpdateControlFile(void);
 extern uint64 GetSystemIdentifier(void);
@@ -325,6 +328,7 @@ extern void XLOGShmemInit(void);
 extern void BootStrapXLOG(void);
 extern void LocalProcessControlFile(bool reset);
 extern void StartupXLOG(void);
+extern void SetControlFileWALProhibitFlag(bool wal_prohibited);
 extern void ShutdownXLOG(int code, Datum arg);
 extern void InitXLOGAccess(void);
 extern void CreateCheckPoint(int flags);
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index 06bed90c5e9..f4dc5412ee6 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -182,6 +182,9 @@ typedef struct ControlFileData
 	int			max_locks_per_xact;
 	bool		track_commit_timestamp;
 
+	/* WAL prohibited determines if the WAL insert is allowed or not. */
+	bool		wal_prohibited;
+
 	/*
 	 * This data is used to check for hardware-architecture compatibility of
 	 * the database and the backend executable.  We need not check endianness
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 687509ba926..246a9de91e3 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10964,6 +10964,10 @@
   proname => 'pg_partition_root', prorettype => 'regclass',
   proargtypes => 'regclass', prosrc => 'pg_partition_root' },
 
+{ oid => '4142', descr => 'alter system read only state',
+  proname => 'pg_alter_wal_prohibit_state', prorettype => 'bool',
+  proargtypes => 'bool', prosrc => 'pg_alter_wal_prohibit_state' },
+
 { oid => '4350', descr => 'Unicode normalization',
   proname => 'normalize', prorettype => 'text', proargtypes => 'text text',
   prosrc => 'unicode_normalize_func' },
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 0dfbac46b4b..f9ff2360b35 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -956,6 +956,7 @@ typedef enum
 	WAIT_EVENT_REPLICATION_SLOT_RESTORE_SYNC,
 	WAIT_EVENT_REPLICATION_SLOT_SYNC,
 	WAIT_EVENT_REPLICATION_SLOT_WRITE,
+	WAIT_EVENT_WALPROHIBIT_STATE_CHANGE,
 	WAIT_EVENT_SLRU_FLUSH_SYNC,
 	WAIT_EVENT_SLRU_READ,
 	WAIT_EVENT_SLRU_SYNC,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e6..ad5e3ba5724 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -42,4 +42,6 @@ extern void CheckpointerShmemInit(void);
 
 extern bool FirstCallSinceLastCheckpoint(void);
 
+extern void SendsSignalToCheckpointer(int signum);
+
 #endif							/* _BGWRITER_H */
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f38..bae06202b4a 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -48,12 +48,7 @@ typedef enum
 
 typedef enum
 {
-	/*
-	 * XXX. PROCSIGNAL_BARRIER_PLACEHOLDER should be replaced when the first
-	 * real user of the ProcSignalBarrier mechanism is added. It's just here
-	 * for now because we can't have an empty enum.
-	 */
-	PROCSIGNAL_BARRIER_PLACEHOLDER = 0
+	PROCSIGNAL_BARRIER_WALPROHIBIT = 0
 } ProcSignalBarrierType;
 
 /*
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 74af9665d00..f16efeb5d6a 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2670,6 +2670,7 @@ WALAvailability
 WALInsertLock
 WALInsertLockPadded
 WALOpenSegment
+WALProhibitStateData
 WALReadError
 WALSegmentCloseCB
 WALSegmentContext
-- 
2.22.0

From 0797fd2e9adf52066b605dafc14a707962611165 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 27 Mar 2020 05:05:38 -0400
Subject: [PATCH v7 2/5] Add alter system read only/write syntax

Note that syntax doesn't have any implementation.
---
 src/backend/nodes/copyfuncs.c    | 12 ++++++++++++
 src/backend/nodes/equalfuncs.c   |  9 +++++++++
 src/backend/nodes/outfuncs.c     | 12 ++++++++++++
 src/backend/nodes/readfuncs.c    | 15 +++++++++++++++
 src/backend/parser/gram.y        | 13 +++++++++++++
 src/backend/tcop/utility.c       | 21 +++++++++++++++++++++
 src/bin/psql/tab-complete.c      |  6 ++++--
 src/include/nodes/nodes.h        |  1 +
 src/include/nodes/parsenodes.h   | 10 ++++++++++
 src/tools/pgindent/typedefs.list |  1 +
 10 files changed, 98 insertions(+), 2 deletions(-)

diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 0409a40b82a..b3c055a4b7e 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -4020,6 +4020,15 @@ _copyAlterSystemStmt(const AlterSystemStmt *from)
 	return newnode;
 }
 
+static AlterSystemWALProhibitState *
+_copyAlterSystemWALProhibitState(const AlterSystemWALProhibitState *from)
+{
+	AlterSystemWALProhibitState *newnode = makeNode(AlterSystemWALProhibitState);
+
+	COPY_SCALAR_FIELD(walprohibited);
+	return newnode;
+}
+
 static CreateSeqStmt *
 _copyCreateSeqStmt(const CreateSeqStmt *from)
 {
@@ -5405,6 +5414,9 @@ copyObjectImpl(const void *from)
 		case T_AlterSystemStmt:
 			retval = _copyAlterSystemStmt(from);
 			break;
+		case T_AlterSystemWALProhibitState:
+			retval = _copyAlterSystemWALProhibitState(from);
+			break;
 		case T_CreateSeqStmt:
 			retval = _copyCreateSeqStmt(from);
 			break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e2d1b987bf4..5f5f289b8af 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1769,6 +1769,12 @@ _equalAlterSystemStmt(const AlterSystemStmt *a, const AlterSystemStmt *b)
 	return true;
 }
 
+static bool
+_equalAlterSystemWALProhibitState(const AlterSystemWALProhibitState *a, const AlterSystemWALProhibitState *b)
+{
+	COMPARE_SCALAR_FIELD(walprohibited);
+	return true;
+}
 
 static bool
 _equalCreateSeqStmt(const CreateSeqStmt *a, const CreateSeqStmt *b)
@@ -3457,6 +3463,9 @@ equal(const void *a, const void *b)
 		case T_AlterSystemStmt:
 			retval = _equalAlterSystemStmt(a, b);
 			break;
+		case T_AlterSystemWALProhibitState:
+			retval = _equalAlterSystemWALProhibitState(a, b);
+			break;
 		case T_CreateSeqStmt:
 			retval = _equalCreateSeqStmt(a, b);
 			break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index e2f177515da..4b98ed7f122 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -1358,6 +1358,15 @@ _outAlternativeSubPlan(StringInfo str, const AlternativeSubPlan *node)
 	WRITE_NODE_FIELD(subplans);
 }
 
+static void
+_outAlterSystemWALProhibitState(StringInfo str,
+								const AlterSystemWALProhibitState *node)
+{
+	WRITE_NODE_TYPE("ALTERSYSTEMWALPROHIBITSTATE");
+
+	WRITE_BOOL_FIELD(walprohibited);
+}
+
 static void
 _outFieldSelect(StringInfo str, const FieldSelect *node)
 {
@@ -3914,6 +3923,9 @@ outNode(StringInfo str, const void *obj)
 			case T_AlternativeSubPlan:
 				_outAlternativeSubPlan(str, obj);
 				break;
+			case T_AlterSystemWALProhibitState:
+				_outAlterSystemWALProhibitState(str, obj);
+				break;
 			case T_FieldSelect:
 				_outFieldSelect(str, obj);
 				break;
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 42050ab7195..beb6540ecb9 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -2552,6 +2552,19 @@ _readAlternativeSubPlan(void)
 	READ_DONE();
 }
 
+/*
+ * _readAlterSystemWALProhibitState
+ */
+static AlterSystemWALProhibitState *
+_readAlterSystemWALProhibitState(void)
+{
+	READ_LOCALS(AlterSystemWALProhibitState);
+
+	READ_BOOL_FIELD(walprohibited);
+
+	READ_DONE();
+}
+
 /*
  * _readExtensibleNode
  */
@@ -2874,6 +2887,8 @@ parseNodeString(void)
 		return_value = _readSubPlan();
 	else if (MATCH("ALTERNATIVESUBPLAN", 18))
 		return_value = _readAlternativeSubPlan();
+	else if (MATCH("ALTERSYSTEMWALPROHIBITSTATE", 27))
+		return_value = _readAlterSystemWALProhibitState();
 	else if (MATCH("EXTENSIBLENODE", 14))
 		return_value = _readExtensibleNode();
 	else if (MATCH("PARTITIONBOUNDSPEC", 18))
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9f47745ee24..c900e5d8319 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -479,6 +479,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
 
 %type <vsetstmt> generic_set set_rest set_rest_more generic_reset reset_rest
 				 SetResetClause FunctionSetResetClause
+%type <boolean> system_readonly_state
 
 %type <node>	TableElement TypedTableElement ConstraintElem TableFuncElement
 %type <node>	columnDef columnOptions
@@ -10176,8 +10177,20 @@ AlterSystemStmt:
 					n->setstmt = $4;
 					$$ = (Node *)n;
 				}
+			| ALTER SYSTEM_P system_readonly_state
+				{
+					AlterSystemWALProhibitState *n = makeNode(AlterSystemWALProhibitState);
+					n->walprohibited = $3;
+					$$ = (Node *)n;
+				}
 		;
 
+system_readonly_state:
+			 READ ONLY
+					{ $$ = true; }
+			| READ WRITE
+					{ $$ = false; }
+		;
 
 /*****************************************************************************
  *
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 9a35147b26a..74c2162cd59 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -85,6 +85,7 @@ static void ProcessUtilitySlow(ParseState *pstate,
 							   DestReceiver *dest,
 							   QueryCompletion *qc);
 static void ExecDropStmt(DropStmt *stmt, bool isTopLevel);
+static void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
 
 /*
  * CommandIsReadOnly: is an executable query read-only?
@@ -219,6 +220,7 @@ ClassifyUtilityCommandAsReadOnly(Node *parsetree)
 				return COMMAND_IS_NOT_READ_ONLY;
 			}
 
+		case T_AlterSystemWALProhibitState:
 		case T_AlterSystemStmt:
 			{
 				/*
@@ -834,6 +836,11 @@ standard_ProcessUtility(PlannedStmt *pstmt,
 			AlterSystemSetConfigFile((AlterSystemStmt *) parsetree);
 			break;
 
+		case T_AlterSystemWALProhibitState:
+			PreventInTransactionBlock(isTopLevel, "ALTER SYSTEM");
+			AlterSystemSetWALProhibitState((AlterSystemWALProhibitState *) parsetree);
+			break;
+
 		case T_VariableSetStmt:
 			ExecSetVariableStmt((VariableSetStmt *) parsetree, isTopLevel);
 			break;
@@ -2819,6 +2826,7 @@ CreateCommandTag(Node *parsetree)
 			tag = CMDTAG_REFRESH_MATERIALIZED_VIEW;
 			break;
 
+		case T_AlterSystemWALProhibitState:
 		case T_AlterSystemStmt:
 			tag = CMDTAG_ALTER_SYSTEM;
 			break;
@@ -3683,3 +3691,16 @@ GetCommandLogLevel(Node *parsetree)
 
 	return lev;
 }
+
+/*
+ * AlterSystemSetWALProhibitState
+ *
+ * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
+ */
+static void
+AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
+{
+	ereport(ERROR,
+			(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+			 errmsg("ALTER SYSTEM READ { ONLY | WRITE } not implemented")));
+}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index f41785f11c1..408f6260b26 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1864,9 +1864,11 @@ psql_completion(const char *text, int start, int end)
 	/* ALTER SERVER <name> VERSION <version> */
 	else if (Matches("ALTER", "SERVER", MatchAny, "VERSION", MatchAny))
 		COMPLETE_WITH("OPTIONS");
-	/* ALTER SYSTEM SET, RESET, RESET ALL */
+	/* ALTER SYSTEM READ, SET, RESET, RESET ALL */
 	else if (Matches("ALTER", "SYSTEM"))
-		COMPLETE_WITH("SET", "RESET");
+		COMPLETE_WITH("SET", "RESET", "READ");
+	else if (Matches("ALTER", "SYSTEM", "READ"))
+		COMPLETE_WITH("ONLY", "WRITE");
 	else if (Matches("ALTER", "SYSTEM", "SET|RESET"))
 		COMPLETE_WITH_QUERY(Query_for_list_of_alter_system_set_vars);
 	else if (Matches("ALTER", "SYSTEM", "SET", MatchAny))
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 381d84b4e4f..17d6942c734 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -412,6 +412,7 @@ typedef enum NodeTag
 	T_RefreshMatViewStmt,
 	T_ReplicaIdentityStmt,
 	T_AlterSystemStmt,
+	T_AlterSystemWALProhibitState,
 	T_CreatePolicyStmt,
 	T_AlterPolicyStmt,
 	T_CreateTransformStmt,
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index e83329fd6d1..2ecaeff031b 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -3195,6 +3195,16 @@ typedef struct AlterSystemStmt
 	VariableSetStmt *setstmt;	/* SET subcommand */
 } AlterSystemStmt;
 
+/* ----------------------
+ *		Alter System Read Statement
+ * ----------------------
+ */
+typedef struct AlterSystemWALProhibitState
+{
+	NodeTag		type;
+	bool		walprohibited;
+} AlterSystemWALProhibitState;
+
 /* ----------------------
  *		Cluster Statement (support pbrown's cluster index implementation)
  * ----------------------
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index b1afb345c36..74af9665d00 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -87,6 +87,7 @@ AlterStatsStmt
 AlterSubscriptionStmt
 AlterSubscriptionType
 AlterSystemStmt
+AlterSystemWALProhibitState
 AlterTSConfigType
 AlterTSConfigurationStmt
 AlterTSDictionaryStmt
-- 
2.22.0

From 9b0537dcece5e12cffa8709494e924790b24822c Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 19 Jun 2020 06:27:53 -0400
Subject: [PATCH v7 1/5] Allow error or refusal while absorbing barriers.

Patch by Robert Haas
---
 src/backend/storage/ipc/procsignal.c | 75 +++++++++++++++++++++++-----
 1 file changed, 63 insertions(+), 12 deletions(-)

diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index ffe67acea1c..63cb70bcaa4 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -87,12 +87,16 @@ typedef struct
 #define BARRIER_SHOULD_CHECK(flags, type) \
 	(((flags) & (((uint32) 1) << (uint32) (type))) != 0)
 
+/* Clear the relevant type bit from the flags. */
+#define BARRIER_CLEAR_BIT(flags, type) \
+	((flags) &= ~(((uint32) 1) << (uint32) (type)))
+
 static ProcSignalHeader *ProcSignal = NULL;
 static volatile ProcSignalSlot *MyProcSignalSlot = NULL;
 
 static bool CheckProcSignal(ProcSignalReason reason);
 static void CleanupProcSignalState(int status, Datum arg);
-static void ProcessBarrierPlaceholder(void);
+static bool ProcessBarrierPlaceholder(void);
 
 /*
  * ProcSignalShmemSize
@@ -486,17 +490,59 @@ ProcessProcSignalBarrier(void)
 	flags = pg_atomic_exchange_u32(&MyProcSignalSlot->pss_barrierCheckMask, 0);
 
 	/*
-	 * Process each type of barrier. It's important that nothing we call from
-	 * here throws an error, because pss_barrierCheckMask has already been
-	 * cleared. If we jumped out of here before processing all barrier types,
-	 * then we'd forget about the need to do so later.
-	 *
-	 * NB: It ought to be OK to call the barrier-processing functions
-	 * unconditionally, but it's more efficient to call only the ones that
-	 * might need us to do something based on the flags.
+	 * If there are no flags set, then we can skip doing any real work.
+	 * Otherwise, establish a PG_TRY block, so that we don't lose track of
+	 * which types of barrier processing are needed if an ERROR occurs.
 	 */
-	if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_PLACEHOLDER))
-		ProcessBarrierPlaceholder();
+	if (flags != 0)
+	{
+		PG_TRY();
+		{
+			/*
+			 * Process each type of barrier. The barrier-processing functions
+			 * should normally return true, but may return false if the barrier
+			 * can't be absorbed at the current time. This should be rare,
+			 * because it's pretty expensive.  Every single
+			 * CHECK_FOR_INTERRUPTS() will return here until we manage to
+			 * absorb the barrier, and that cost will add up in a hurry.
+			 *
+			 * NB: It ought to be OK to call the barrier-processing functions
+			 * unconditionally, but it's more efficient to call only the ones
+			 * that might need us to do something based on the flags.
+			 */
+			if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_PLACEHOLDER)
+				&& ProcessBarrierPlaceholder())
+				BARRIER_CLEAR_BIT(flags, PROCSIGNAL_BARRIER_PLACEHOLDER);
+		}
+		PG_CATCH();
+		{
+			/*
+			 * If an ERROR occurred, add any flags that weren't yet handled
+			 * back into pss_barrierCheckMask, and reset the global variables
+			 * so that we try again the next time we check for interrupts.
+			 */
+			pg_atomic_fetch_or_u32(&MyProcSignalSlot->pss_barrierCheckMask,
+								   flags);
+			ProcSignalBarrierPending = true;
+			InterruptPending = true;
+
+			PG_RE_THROW();
+		}
+		PG_END_TRY();
+
+		/*
+		 * If some barrier was not successfully absorbed, we will have to try
+		 * again later.
+		 */
+		if (flags != 0)
+		{
+			pg_atomic_fetch_or_u32(&MyProcSignalSlot->pss_barrierCheckMask,
+								   flags);
+			ProcSignalBarrierPending = true;
+			InterruptPending = true;
+			return;
+		}
+	}
 
 	/*
 	 * State changes related to all types of barriers that might have been
@@ -508,7 +554,7 @@ ProcessProcSignalBarrier(void)
 	pg_atomic_write_u64(&MyProcSignalSlot->pss_barrierGeneration, shared_gen);
 }
 
-static void
+static bool
 ProcessBarrierPlaceholder(void)
 {
 	/*
@@ -518,7 +564,12 @@ ProcessBarrierPlaceholder(void)
 	 * appropriately descriptive. Get rid of this function and instead have
 	 * ProcessBarrierSomethingElse. Most likely, that function should live in
 	 * the file pertaining to that subsystem, rather than here.
+	 *
+	 * The return value should be 'true' if the barrier was successfully
+	 * absorbed and 'false' if not. Note that returning 'false' can lead to
+	 * very frequent retries, so try hard to make that an uncommon case.
 	 */
+	return true;
 }
 
 /*
-- 
2.22.0

From 4b18e2a3c598b9abd040c2ba1ba844b012f0c3a0 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Tue, 14 Jul 2020 02:30:44 -0400
Subject: [PATCH v7 5/5] WIP - Documentation.

TODOs:

1] TBC, the section for pg_is_in_readonly() function, right now it is under "Recovery Information Functions"
2] Documentation regarding ALTER SYSTEM READ/WRITE
---
 src/backend/access/transam/README | 60 ++++++++++++++++++++++++++++---
 src/backend/storage/page/README   | 12 +++----
 2 files changed, 61 insertions(+), 11 deletions(-)

diff --git a/src/backend/access/transam/README b/src/backend/access/transam/README
index 1edc8180c12..fe7a84f93da 100644
--- a/src/backend/access/transam/README
+++ b/src/backend/access/transam/README
@@ -442,8 +442,8 @@ to be modified.
 2. START_CRIT_SECTION()  (Any error during the next three steps must cause a
 PANIC because the shared buffers will contain unlogged changes, which we
 have to ensure don't get to disk.  Obviously, you should check conditions
-such as whether there's enough free space on the page before you start the
-critical section.)
+such as whether there's WAL write permission and enough free space on the page
+before you start the critical section.)
 
 3. Apply the required changes to the shared buffer(s).
 
@@ -486,6 +486,54 @@ with the incomplete-split flag set, it will finish the interrupted split by
 inserting the key to the parent, before proceeding.
 
 
+Read only system state
+----------------------
+
+The system state when it is not currently possible to insert write-ahead log
+records, either because the system is still in recovery or because the system
+forced to WAL prohibited by executing ALTER SYSTEM READ ONLY.  We have a
+lower-level defense in XLogBeginInsert() and elsewhere to stop us from modifying
+data during recovery when !XLogInsertAllowed(), but if XLogBeginInsert() is
+inside the critical section we must not depend on it to report an error.
+Otherwise, it will cause PANIC as mentioned previously.
+
+We do not reach the point where we try to write WAL during recovery but ALTER
+SYSTEM READ ONLY can be executed anytime by the user to stop WAL writing.  Any
+backends which receive read-only system state transition barrier interrupt need
+to stop WAL writing immediately.  For barrier absorption the backed(s) will kill
+the running transaction which has valid XID indicates that the transaction has
+performed and/or planning WAL write.  The transaction which doesn't acquire
+valid XID yet or operation such VACUUM or CONCURRENT CREATE INDEX which not
+necessary have valid XID for WAL will not be prevented while barrier processing,
+and those might hit the error from XLogBeginInsert() while trying to write WAL
+in read only system state.  To prevent such error from XLogBeginInsert() inside
+the critical section the WAL write permission has to check before
+START_CRIT_SECTION().
+
+To enforce the practice to check WAL permission before entering into critical
+section for the WAL write, we have added an assert check flag that indicates
+permission has been checked before calling XLogBeginInsert().  If not,
+XLogBeginInsert() will have assertion failure.  WAL permission check is not
+mandatory if the XLogBeginInsert() is not inside the critical section where
+throwing the error is acceptable.  To get permission check flag set either
+CheckWALPermitted(), AssertWALPermitted_HaveXID(), or AssertWALPermitted()
+should be called before START_CRIT_SECTION().  This flag automatically resets
+while exiting from the critical section.  The rule to place either of permission
+check routines will be:
+
+	The places where WAL write operation in critical can be expected without
+	having valid XID (e.g vacuum) need to protect by CheckWALPermitted(), so
+	that error can be reported outside before critical section.
+
+	The places where INSERT and UPDATE are expected which are never happened
+	without valid XID can be checked using AssertWALPermitted_HaveXID.  So that
+	non-assert build will not have the checking overhead.
+
+	The places we know that we cannot be reached in the read-only state and may
+	or may not have XID, but need to ensure the permission has been checked on
+	assert enabled build should use AssertWALPermitted().
+
+
 Constructing a WAL record
 -------------------------
 
@@ -531,7 +579,8 @@ Details of the API functions:
 
 void XLogBeginInsert(void)
 
-    Must be called before XLogRegisterBuffer and XLogRegisterData.
+	Must be called before XLogRegisterBuffer and XLogRegisterData.  WAL
+	permission must be check before calling it in a critical section.
 
 void XLogResetInsertion(void)
 
@@ -638,8 +687,9 @@ MarkBufferDirtyHint() to mark the block dirty.
 If the buffer is clean and checksums are in use then MarkBufferDirtyHint()
 inserts an XLOG_FPI_FOR_HINT record to ensure that we take a full page image
 that includes the hint. We do this to avoid a partial page write, when we
-write the dirtied page. WAL is not written during recovery, so we simply skip
-dirtying blocks because of hints when in recovery.
+write the dirtied page. WAL is not written while in read only (i.e. during
+recovery or in WAL prohibit state), so we simply skip dirtying blocks because of
+hints when in recovery.
 
 If you do decide to optimise away a WAL record, then any calls to
 MarkBufferDirty() must be replaced by MarkBufferDirtyHint(),
diff --git a/src/backend/storage/page/README b/src/backend/storage/page/README
index e30d7ac59ad..15f0bb4b7b5 100644
--- a/src/backend/storage/page/README
+++ b/src/backend/storage/page/README
@@ -56,9 +56,9 @@ WAL is a fatal error and prevents further recovery, whereas a checksum failure
 on a normal data block is a hard error but not a critical one for the server,
 even if it is a very bad thing for the user.
 
-New WAL records cannot be written during recovery, so hint bits set during
-recovery must not dirty the page if the buffer is not already dirty, when
-checksums are enabled.  Systems in Hot-Standby mode may benefit from hint bits
-being set, but with checksums enabled, a page cannot be dirtied after setting a
-hint bit (due to the torn page risk). So, it must wait for full-page images
-containing the hint bit updates to arrive from the primary.
+New WAL records cannot be written during recovery or or while in WAL prohibit
+state, so hint bits set during recovery must not dirty the page if the buffer is
+not already dirty, when checksums are enabled.  Systems in Hot-Standby mode may
+benefit from hint bits being set, but with checksums enabled, a page cannot be
+dirtied after setting a hint bit (due to the torn page risk). So, it must wait
+for full-page images containing the hint bit updates to arrive from the primary.
-- 
2.22.0

From f1b4606aac0b28d22366e79a5e1168355d0a9589 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 27 Mar 2020 05:05:38 -0400
Subject: [PATCH v8 2/5] Add alter system read only/write syntax

Note that syntax doesn't have any implementation.
---
 src/backend/nodes/copyfuncs.c    | 12 ++++++++++++
 src/backend/nodes/equalfuncs.c   |  9 +++++++++
 src/backend/nodes/outfuncs.c     | 12 ++++++++++++
 src/backend/nodes/readfuncs.c    | 15 +++++++++++++++
 src/backend/parser/gram.y        | 13 +++++++++++++
 src/backend/tcop/utility.c       | 21 +++++++++++++++++++++
 src/bin/psql/tab-complete.c      |  6 ++++--
 src/include/nodes/nodes.h        |  1 +
 src/include/nodes/parsenodes.h   | 10 ++++++++++
 src/tools/pgindent/typedefs.list |  1 +
 10 files changed, 98 insertions(+), 2 deletions(-)

diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 0409a40b82a..b3c055a4b7e 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -4020,6 +4020,15 @@ _copyAlterSystemStmt(const AlterSystemStmt *from)
 	return newnode;
 }
 
+static AlterSystemWALProhibitState *
+_copyAlterSystemWALProhibitState(const AlterSystemWALProhibitState *from)
+{
+	AlterSystemWALProhibitState *newnode = makeNode(AlterSystemWALProhibitState);
+
+	COPY_SCALAR_FIELD(walprohibited);
+	return newnode;
+}
+
 static CreateSeqStmt *
 _copyCreateSeqStmt(const CreateSeqStmt *from)
 {
@@ -5405,6 +5414,9 @@ copyObjectImpl(const void *from)
 		case T_AlterSystemStmt:
 			retval = _copyAlterSystemStmt(from);
 			break;
+		case T_AlterSystemWALProhibitState:
+			retval = _copyAlterSystemWALProhibitState(from);
+			break;
 		case T_CreateSeqStmt:
 			retval = _copyCreateSeqStmt(from);
 			break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e2d1b987bf4..5f5f289b8af 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1769,6 +1769,12 @@ _equalAlterSystemStmt(const AlterSystemStmt *a, const AlterSystemStmt *b)
 	return true;
 }
 
+static bool
+_equalAlterSystemWALProhibitState(const AlterSystemWALProhibitState *a, const AlterSystemWALProhibitState *b)
+{
+	COMPARE_SCALAR_FIELD(walprohibited);
+	return true;
+}
 
 static bool
 _equalCreateSeqStmt(const CreateSeqStmt *a, const CreateSeqStmt *b)
@@ -3457,6 +3463,9 @@ equal(const void *a, const void *b)
 		case T_AlterSystemStmt:
 			retval = _equalAlterSystemStmt(a, b);
 			break;
+		case T_AlterSystemWALProhibitState:
+			retval = _equalAlterSystemWALProhibitState(a, b);
+			break;
 		case T_CreateSeqStmt:
 			retval = _equalCreateSeqStmt(a, b);
 			break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index e2f177515da..4b98ed7f122 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -1358,6 +1358,15 @@ _outAlternativeSubPlan(StringInfo str, const AlternativeSubPlan *node)
 	WRITE_NODE_FIELD(subplans);
 }
 
+static void
+_outAlterSystemWALProhibitState(StringInfo str,
+								const AlterSystemWALProhibitState *node)
+{
+	WRITE_NODE_TYPE("ALTERSYSTEMWALPROHIBITSTATE");
+
+	WRITE_BOOL_FIELD(walprohibited);
+}
+
 static void
 _outFieldSelect(StringInfo str, const FieldSelect *node)
 {
@@ -3914,6 +3923,9 @@ outNode(StringInfo str, const void *obj)
 			case T_AlternativeSubPlan:
 				_outAlternativeSubPlan(str, obj);
 				break;
+			case T_AlterSystemWALProhibitState:
+				_outAlterSystemWALProhibitState(str, obj);
+				break;
 			case T_FieldSelect:
 				_outFieldSelect(str, obj);
 				break;
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 42050ab7195..beb6540ecb9 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -2552,6 +2552,19 @@ _readAlternativeSubPlan(void)
 	READ_DONE();
 }
 
+/*
+ * _readAlterSystemWALProhibitState
+ */
+static AlterSystemWALProhibitState *
+_readAlterSystemWALProhibitState(void)
+{
+	READ_LOCALS(AlterSystemWALProhibitState);
+
+	READ_BOOL_FIELD(walprohibited);
+
+	READ_DONE();
+}
+
 /*
  * _readExtensibleNode
  */
@@ -2874,6 +2887,8 @@ parseNodeString(void)
 		return_value = _readSubPlan();
 	else if (MATCH("ALTERNATIVESUBPLAN", 18))
 		return_value = _readAlternativeSubPlan();
+	else if (MATCH("ALTERSYSTEMWALPROHIBITSTATE", 27))
+		return_value = _readAlterSystemWALProhibitState();
 	else if (MATCH("EXTENSIBLENODE", 14))
 		return_value = _readExtensibleNode();
 	else if (MATCH("PARTITIONBOUNDSPEC", 18))
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 17653ef3a79..a0514e66c4a 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -479,6 +479,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
 
 %type <vsetstmt> generic_set set_rest set_rest_more generic_reset reset_rest
 				 SetResetClause FunctionSetResetClause
+%type <boolean> system_readonly_state
 
 %type <node>	TableElement TypedTableElement ConstraintElem TableFuncElement
 %type <node>	columnDef columnOptions
@@ -10173,8 +10174,20 @@ AlterSystemStmt:
 					n->setstmt = $4;
 					$$ = (Node *)n;
 				}
+			| ALTER SYSTEM_P system_readonly_state
+				{
+					AlterSystemWALProhibitState *n = makeNode(AlterSystemWALProhibitState);
+					n->walprohibited = $3;
+					$$ = (Node *)n;
+				}
 		;
 
+system_readonly_state:
+			 READ ONLY
+					{ $$ = true; }
+			| READ WRITE
+					{ $$ = false; }
+		;
 
 /*****************************************************************************
  *
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 9a35147b26a..74c2162cd59 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -85,6 +85,7 @@ static void ProcessUtilitySlow(ParseState *pstate,
 							   DestReceiver *dest,
 							   QueryCompletion *qc);
 static void ExecDropStmt(DropStmt *stmt, bool isTopLevel);
+static void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
 
 /*
  * CommandIsReadOnly: is an executable query read-only?
@@ -219,6 +220,7 @@ ClassifyUtilityCommandAsReadOnly(Node *parsetree)
 				return COMMAND_IS_NOT_READ_ONLY;
 			}
 
+		case T_AlterSystemWALProhibitState:
 		case T_AlterSystemStmt:
 			{
 				/*
@@ -834,6 +836,11 @@ standard_ProcessUtility(PlannedStmt *pstmt,
 			AlterSystemSetConfigFile((AlterSystemStmt *) parsetree);
 			break;
 
+		case T_AlterSystemWALProhibitState:
+			PreventInTransactionBlock(isTopLevel, "ALTER SYSTEM");
+			AlterSystemSetWALProhibitState((AlterSystemWALProhibitState *) parsetree);
+			break;
+
 		case T_VariableSetStmt:
 			ExecSetVariableStmt((VariableSetStmt *) parsetree, isTopLevel);
 			break;
@@ -2819,6 +2826,7 @@ CreateCommandTag(Node *parsetree)
 			tag = CMDTAG_REFRESH_MATERIALIZED_VIEW;
 			break;
 
+		case T_AlterSystemWALProhibitState:
 		case T_AlterSystemStmt:
 			tag = CMDTAG_ALTER_SYSTEM;
 			break;
@@ -3683,3 +3691,16 @@ GetCommandLogLevel(Node *parsetree)
 
 	return lev;
 }
+
+/*
+ * AlterSystemSetWALProhibitState
+ *
+ * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
+ */
+static void
+AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
+{
+	ereport(ERROR,
+			(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+			 errmsg("ALTER SYSTEM READ { ONLY | WRITE } not implemented")));
+}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9c6f5ecb6a8..d28d8cea773 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1864,9 +1864,11 @@ psql_completion(const char *text, int start, int end)
 	/* ALTER SERVER <name> VERSION <version> */
 	else if (Matches("ALTER", "SERVER", MatchAny, "VERSION", MatchAny))
 		COMPLETE_WITH("OPTIONS");
-	/* ALTER SYSTEM SET, RESET, RESET ALL */
+	/* ALTER SYSTEM READ, SET, RESET, RESET ALL */
 	else if (Matches("ALTER", "SYSTEM"))
-		COMPLETE_WITH("SET", "RESET");
+		COMPLETE_WITH("SET", "RESET", "READ");
+	else if (Matches("ALTER", "SYSTEM", "READ"))
+		COMPLETE_WITH("ONLY", "WRITE");
 	else if (Matches("ALTER", "SYSTEM", "SET|RESET"))
 		COMPLETE_WITH_QUERY(Query_for_list_of_alter_system_set_vars);
 	else if (Matches("ALTER", "SYSTEM", "SET", MatchAny))
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 381d84b4e4f..17d6942c734 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -412,6 +412,7 @@ typedef enum NodeTag
 	T_RefreshMatViewStmt,
 	T_ReplicaIdentityStmt,
 	T_AlterSystemStmt,
+	T_AlterSystemWALProhibitState,
 	T_CreatePolicyStmt,
 	T_AlterPolicyStmt,
 	T_CreateTransformStmt,
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 60c2f454660..340ee87f1bc 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -3196,6 +3196,16 @@ typedef struct AlterSystemStmt
 	VariableSetStmt *setstmt;	/* SET subcommand */
 } AlterSystemStmt;
 
+/* ----------------------
+ *		Alter System Read Statement
+ * ----------------------
+ */
+typedef struct AlterSystemWALProhibitState
+{
+	NodeTag		type;
+	bool		walprohibited;
+} AlterSystemWALProhibitState;
+
 /* ----------------------
  *		Cluster Statement (support pbrown's cluster index implementation)
  * ----------------------
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index b1afb345c36..74af9665d00 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -87,6 +87,7 @@ AlterStatsStmt
 AlterSubscriptionStmt
 AlterSubscriptionType
 AlterSystemStmt
+AlterSystemWALProhibitState
 AlterTSConfigType
 AlterTSConfigurationStmt
 AlterTSDictionaryStmt
-- 
2.22.0

From 57458536d9dbfe21fb68a30aa43f7d2c12c13a80 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 19 Jun 2020 06:27:53 -0400
Subject: [PATCH v8 1/5] Allow error or refusal while absorbing barriers.

Patch by Robert Haas
---
 src/backend/storage/ipc/procsignal.c | 75 +++++++++++++++++++++++-----
 1 file changed, 63 insertions(+), 12 deletions(-)

diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index ffe67acea1c..63cb70bcaa4 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -87,12 +87,16 @@ typedef struct
 #define BARRIER_SHOULD_CHECK(flags, type) \
 	(((flags) & (((uint32) 1) << (uint32) (type))) != 0)
 
+/* Clear the relevant type bit from the flags. */
+#define BARRIER_CLEAR_BIT(flags, type) \
+	((flags) &= ~(((uint32) 1) << (uint32) (type)))
+
 static ProcSignalHeader *ProcSignal = NULL;
 static volatile ProcSignalSlot *MyProcSignalSlot = NULL;
 
 static bool CheckProcSignal(ProcSignalReason reason);
 static void CleanupProcSignalState(int status, Datum arg);
-static void ProcessBarrierPlaceholder(void);
+static bool ProcessBarrierPlaceholder(void);
 
 /*
  * ProcSignalShmemSize
@@ -486,17 +490,59 @@ ProcessProcSignalBarrier(void)
 	flags = pg_atomic_exchange_u32(&MyProcSignalSlot->pss_barrierCheckMask, 0);
 
 	/*
-	 * Process each type of barrier. It's important that nothing we call from
-	 * here throws an error, because pss_barrierCheckMask has already been
-	 * cleared. If we jumped out of here before processing all barrier types,
-	 * then we'd forget about the need to do so later.
-	 *
-	 * NB: It ought to be OK to call the barrier-processing functions
-	 * unconditionally, but it's more efficient to call only the ones that
-	 * might need us to do something based on the flags.
+	 * If there are no flags set, then we can skip doing any real work.
+	 * Otherwise, establish a PG_TRY block, so that we don't lose track of
+	 * which types of barrier processing are needed if an ERROR occurs.
 	 */
-	if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_PLACEHOLDER))
-		ProcessBarrierPlaceholder();
+	if (flags != 0)
+	{
+		PG_TRY();
+		{
+			/*
+			 * Process each type of barrier. The barrier-processing functions
+			 * should normally return true, but may return false if the barrier
+			 * can't be absorbed at the current time. This should be rare,
+			 * because it's pretty expensive.  Every single
+			 * CHECK_FOR_INTERRUPTS() will return here until we manage to
+			 * absorb the barrier, and that cost will add up in a hurry.
+			 *
+			 * NB: It ought to be OK to call the barrier-processing functions
+			 * unconditionally, but it's more efficient to call only the ones
+			 * that might need us to do something based on the flags.
+			 */
+			if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_PLACEHOLDER)
+				&& ProcessBarrierPlaceholder())
+				BARRIER_CLEAR_BIT(flags, PROCSIGNAL_BARRIER_PLACEHOLDER);
+		}
+		PG_CATCH();
+		{
+			/*
+			 * If an ERROR occurred, add any flags that weren't yet handled
+			 * back into pss_barrierCheckMask, and reset the global variables
+			 * so that we try again the next time we check for interrupts.
+			 */
+			pg_atomic_fetch_or_u32(&MyProcSignalSlot->pss_barrierCheckMask,
+								   flags);
+			ProcSignalBarrierPending = true;
+			InterruptPending = true;
+
+			PG_RE_THROW();
+		}
+		PG_END_TRY();
+
+		/*
+		 * If some barrier was not successfully absorbed, we will have to try
+		 * again later.
+		 */
+		if (flags != 0)
+		{
+			pg_atomic_fetch_or_u32(&MyProcSignalSlot->pss_barrierCheckMask,
+								   flags);
+			ProcSignalBarrierPending = true;
+			InterruptPending = true;
+			return;
+		}
+	}
 
 	/*
 	 * State changes related to all types of barriers that might have been
@@ -508,7 +554,7 @@ ProcessProcSignalBarrier(void)
 	pg_atomic_write_u64(&MyProcSignalSlot->pss_barrierGeneration, shared_gen);
 }
 
-static void
+static bool
 ProcessBarrierPlaceholder(void)
 {
 	/*
@@ -518,7 +564,12 @@ ProcessBarrierPlaceholder(void)
 	 * appropriately descriptive. Get rid of this function and instead have
 	 * ProcessBarrierSomethingElse. Most likely, that function should live in
 	 * the file pertaining to that subsystem, rather than here.
+	 *
+	 * The return value should be 'true' if the barrier was successfully
+	 * absorbed and 'false' if not. Note that returning 'false' can lead to
+	 * very frequent retries, so try hard to make that an uncommon case.
 	 */
+	return true;
 }
 
 /*
-- 
2.22.0

From 3a3fe80a0fe513411248ed4eaa3322ada3c2c41b Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Tue, 14 Jul 2020 02:30:44 -0400
Subject: [PATCH v8 5/5] WIP - Documentation.

TODOs:

1] TBC, the section for pg_is_in_readonly() function, right now it is under "Recovery Information Functions"
2] Documentation regarding ALTER SYSTEM READ/WRITE
---
 src/backend/access/transam/README | 60 ++++++++++++++++++++++++++++---
 src/backend/storage/page/README   | 12 +++----
 2 files changed, 61 insertions(+), 11 deletions(-)

diff --git a/src/backend/access/transam/README b/src/backend/access/transam/README
index 1edc8180c12..fe7a84f93da 100644
--- a/src/backend/access/transam/README
+++ b/src/backend/access/transam/README
@@ -442,8 +442,8 @@ to be modified.
 2. START_CRIT_SECTION()  (Any error during the next three steps must cause a
 PANIC because the shared buffers will contain unlogged changes, which we
 have to ensure don't get to disk.  Obviously, you should check conditions
-such as whether there's enough free space on the page before you start the
-critical section.)
+such as whether there's WAL write permission and enough free space on the page
+before you start the critical section.)
 
 3. Apply the required changes to the shared buffer(s).
 
@@ -486,6 +486,54 @@ with the incomplete-split flag set, it will finish the interrupted split by
 inserting the key to the parent, before proceeding.
 
 
+Read only system state
+----------------------
+
+The system state when it is not currently possible to insert write-ahead log
+records, either because the system is still in recovery or because the system
+forced to WAL prohibited by executing ALTER SYSTEM READ ONLY.  We have a
+lower-level defense in XLogBeginInsert() and elsewhere to stop us from modifying
+data during recovery when !XLogInsertAllowed(), but if XLogBeginInsert() is
+inside the critical section we must not depend on it to report an error.
+Otherwise, it will cause PANIC as mentioned previously.
+
+We do not reach the point where we try to write WAL during recovery but ALTER
+SYSTEM READ ONLY can be executed anytime by the user to stop WAL writing.  Any
+backends which receive read-only system state transition barrier interrupt need
+to stop WAL writing immediately.  For barrier absorption the backed(s) will kill
+the running transaction which has valid XID indicates that the transaction has
+performed and/or planning WAL write.  The transaction which doesn't acquire
+valid XID yet or operation such VACUUM or CONCURRENT CREATE INDEX which not
+necessary have valid XID for WAL will not be prevented while barrier processing,
+and those might hit the error from XLogBeginInsert() while trying to write WAL
+in read only system state.  To prevent such error from XLogBeginInsert() inside
+the critical section the WAL write permission has to check before
+START_CRIT_SECTION().
+
+To enforce the practice to check WAL permission before entering into critical
+section for the WAL write, we have added an assert check flag that indicates
+permission has been checked before calling XLogBeginInsert().  If not,
+XLogBeginInsert() will have assertion failure.  WAL permission check is not
+mandatory if the XLogBeginInsert() is not inside the critical section where
+throwing the error is acceptable.  To get permission check flag set either
+CheckWALPermitted(), AssertWALPermitted_HaveXID(), or AssertWALPermitted()
+should be called before START_CRIT_SECTION().  This flag automatically resets
+while exiting from the critical section.  The rule to place either of permission
+check routines will be:
+
+	The places where WAL write operation in critical can be expected without
+	having valid XID (e.g vacuum) need to protect by CheckWALPermitted(), so
+	that error can be reported outside before critical section.
+
+	The places where INSERT and UPDATE are expected which are never happened
+	without valid XID can be checked using AssertWALPermitted_HaveXID.  So that
+	non-assert build will not have the checking overhead.
+
+	The places we know that we cannot be reached in the read-only state and may
+	or may not have XID, but need to ensure the permission has been checked on
+	assert enabled build should use AssertWALPermitted().
+
+
 Constructing a WAL record
 -------------------------
 
@@ -531,7 +579,8 @@ Details of the API functions:
 
 void XLogBeginInsert(void)
 
-    Must be called before XLogRegisterBuffer and XLogRegisterData.
+	Must be called before XLogRegisterBuffer and XLogRegisterData.  WAL
+	permission must be check before calling it in a critical section.
 
 void XLogResetInsertion(void)
 
@@ -638,8 +687,9 @@ MarkBufferDirtyHint() to mark the block dirty.
 If the buffer is clean and checksums are in use then MarkBufferDirtyHint()
 inserts an XLOG_FPI_FOR_HINT record to ensure that we take a full page image
 that includes the hint. We do this to avoid a partial page write, when we
-write the dirtied page. WAL is not written during recovery, so we simply skip
-dirtying blocks because of hints when in recovery.
+write the dirtied page. WAL is not written while in read only (i.e. during
+recovery or in WAL prohibit state), so we simply skip dirtying blocks because of
+hints when in recovery.
 
 If you do decide to optimise away a WAL record, then any calls to
 MarkBufferDirty() must be replaced by MarkBufferDirtyHint(),
diff --git a/src/backend/storage/page/README b/src/backend/storage/page/README
index e30d7ac59ad..15f0bb4b7b5 100644
--- a/src/backend/storage/page/README
+++ b/src/backend/storage/page/README
@@ -56,9 +56,9 @@ WAL is a fatal error and prevents further recovery, whereas a checksum failure
 on a normal data block is a hard error but not a critical one for the server,
 even if it is a very bad thing for the user.
 
-New WAL records cannot be written during recovery, so hint bits set during
-recovery must not dirty the page if the buffer is not already dirty, when
-checksums are enabled.  Systems in Hot-Standby mode may benefit from hint bits
-being set, but with checksums enabled, a page cannot be dirtied after setting a
-hint bit (due to the torn page risk). So, it must wait for full-page images
-containing the hint bit updates to arrive from the primary.
+New WAL records cannot be written during recovery or or while in WAL prohibit
+state, so hint bits set during recovery must not dirty the page if the buffer is
+not already dirty, when checksums are enabled.  Systems in Hot-Standby mode may
+benefit from hint bits being set, but with checksums enabled, a page cannot be
+dirtied after setting a hint bit (due to the torn page risk). So, it must wait
+for full-page images containing the hint bit updates to arrive from the primary.
-- 
2.22.0

From 0a3c4071145bb050b7e5e8101471a592e5639b0d Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Mon, 27 Jul 2020 02:13:36 -0400
Subject: [PATCH v8 4/5] Error or Assert before START_CRIT_SECTION for WAL
 write

Based on the following criteria added the Assert or the Error when system is
prohibited:

 - Added ERROR for the function which can be reachable without valid XID in case
   of VACUUM or CONCURRENT CREATE INDEX.  For that added common static inline
   function CheckWALPermitted().
 - Added ASSERT for the function which cannot be reached without valid XID;
   Assert to ensure XID validation. For that added AssertWALPermitted_HaveXID().

To enforce the rule to have aforesaid assert or error check before entering a
critical section for WAL write, a new walpermit_checked_state assert only flag
is added. If these check is missing then XLogBeginInsert() will have an
assertion if it is in critical section.

If we are not doing WAL insert inside the critical section then the above
checking is not necessary, we can rely on XLogBeginInsert() for that check &
report an error.
---
 contrib/pg_surgery/heap_surgery.c         | 10 ++++--
 src/backend/access/brin/brin.c            |  4 +++
 src/backend/access/brin/brin_pageops.c    | 21 ++++++++++--
 src/backend/access/brin/brin_revmap.c     | 10 +++++-
 src/backend/access/gin/ginbtree.c         | 15 +++++++--
 src/backend/access/gin/gindatapage.c      | 18 +++++++++--
 src/backend/access/gin/ginfast.c          | 11 +++++--
 src/backend/access/gin/gininsert.c        |  4 +++
 src/backend/access/gin/ginutil.c          |  9 +++++-
 src/backend/access/gin/ginvacuum.c        | 11 ++++++-
 src/backend/access/gist/gist.c            | 25 ++++++++++++---
 src/backend/access/gist/gistvacuum.c      | 13 ++++++--
 src/backend/access/hash/hash.c            | 19 +++++++++--
 src/backend/access/hash/hashinsert.c      |  9 +++++-
 src/backend/access/hash/hashovfl.c        | 22 ++++++++++---
 src/backend/access/hash/hashpage.c        |  9 ++++++
 src/backend/access/heap/heapam.c          | 26 ++++++++++++++-
 src/backend/access/heap/pruneheap.c       | 12 +++++--
 src/backend/access/heap/vacuumlazy.c      | 18 +++++++++--
 src/backend/access/heap/visibilitymap.c   | 22 +++++++++++--
 src/backend/access/nbtree/nbtdedup.c      |  3 ++
 src/backend/access/nbtree/nbtinsert.c     | 10 +++++-
 src/backend/access/nbtree/nbtpage.c       | 39 +++++++++++++++++++----
 src/backend/access/spgist/spgdoinsert.c   | 13 ++++++++
 src/backend/access/spgist/spgvacuum.c     | 22 +++++++++++--
 src/backend/access/transam/multixact.c    |  5 ++-
 src/backend/access/transam/twophase.c     |  9 ++++++
 src/backend/access/transam/varsup.c       |  4 +++
 src/backend/access/transam/walprohibit.c  | 10 ++++++
 src/backend/access/transam/xact.c         |  6 ++++
 src/backend/access/transam/xlog.c         | 27 ++++++++++++----
 src/backend/access/transam/xloginsert.c   | 13 ++++++--
 src/backend/commands/sequence.c           | 16 ++++++++++
 src/backend/commands/variable.c           |  9 ++++--
 src/backend/postmaster/checkpointer.c     |  4 +++
 src/backend/storage/buffer/bufmgr.c       | 10 +++---
 src/backend/storage/freespace/freespace.c | 10 +++++-
 src/backend/storage/lmgr/lock.c           |  6 ++--
 src/backend/utils/cache/relmapper.c       |  3 ++
 src/include/miscadmin.h                   | 27 ++++++++++++++++
 40 files changed, 463 insertions(+), 71 deletions(-)

diff --git a/contrib/pg_surgery/heap_surgery.c b/contrib/pg_surgery/heap_surgery.c
index eb96b4bb36d..53d8c9cea28 100644
--- a/contrib/pg_surgery/heap_surgery.c
+++ b/contrib/pg_surgery/heap_surgery.c
@@ -14,6 +14,7 @@
 
 #include "access/heapam.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "catalog/pg_am_d.h"
 #include "catalog/pg_proc_d.h"
 #include "miscadmin.h"
@@ -89,6 +90,7 @@ heap_force_common(FunctionCallInfo fcinfo, HeapTupleForceOption heap_force_opt)
 	OffsetNumber curr_start_ptr,
 				next_start_ptr;
 	bool		include_this_tid[MaxHeapTuplesPerPage];
+	bool		needwal;
 
 	if (RecoveryInProgress())
 		ereport(ERROR,
@@ -100,6 +102,7 @@ heap_force_common(FunctionCallInfo fcinfo, HeapTupleForceOption heap_force_opt)
 	sanity_check_tid_array(ta, &ntids);
 
 	rel = relation_open(relid, RowExclusiveLock);
+	needwal = RelationNeedsWAL(rel);
 
 	/* Check target relation. */
 	sanity_check_relation(rel);
@@ -217,6 +220,9 @@ heap_force_common(FunctionCallInfo fcinfo, HeapTupleForceOption heap_force_opt)
 		if (heap_force_opt == HEAP_FORCE_KILL && PageIsAllVisible(page))
 			visibilitymap_pin(rel, blkno, &vmbuf);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* No ereport(ERROR) from here until all the changes are logged. */
 		START_CRIT_SECTION();
 
@@ -297,12 +303,12 @@ heap_force_common(FunctionCallInfo fcinfo, HeapTupleForceOption heap_force_opt)
 			MarkBufferDirty(buf);
 
 			/* XLOG stuff */
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 				log_newpage_buffer(buf, true);
 		}
 
 		/* WAL log the VM page if it was modified. */
-		if (did_modify_vm && RelationNeedsWAL(rel))
+		if (did_modify_vm && needwal)
 			log_newpage_buffer(vmbuf, false);
 
 		END_CRIT_SECTION();
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index 1f72562c603..47142193706 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -24,6 +24,7 @@
 #include "access/relscan.h"
 #include "access/table.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/index.h"
 #include "catalog/pg_am.h"
@@ -759,6 +760,9 @@ brinbuildempty(Relation index)
 		ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermittedHaveXID();
+
 	/* Initialize and xlog metabuffer. */
 	START_CRIT_SECTION();
 	brin_metapage_init(BufferGetPage(metabuf), BrinGetPagesPerRange(index),
diff --git a/src/backend/access/brin/brin_pageops.c b/src/backend/access/brin/brin_pageops.c
index 87de0b855b5..8b377a679ab 100644
--- a/src/backend/access/brin/brin_pageops.c
+++ b/src/backend/access/brin/brin_pageops.c
@@ -14,6 +14,7 @@
 #include "access/brin_pageops.h"
 #include "access/brin_revmap.h"
 #include "access/brin_xlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -65,6 +66,7 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 	Buffer		newbuf;
 	BlockNumber newblk = InvalidBlockNumber;
 	bool		extended;
+	bool		needwal = RelationNeedsWAL(idxrel);
 
 	Assert(newsz == MAXALIGN(newsz));
 
@@ -176,13 +178,16 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 	if (((BrinPageFlags(oldpage) & BRIN_EVACUATE_PAGE) == 0) &&
 		brin_can_do_samepage_update(oldbuf, origsz, newsz))
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 		if (!PageIndexTupleOverwrite(oldpage, oldoff, (Item) unconstify(BrinTuple *, newtup), newsz))
 			elog(ERROR, "failed to replace BRIN tuple");
 		MarkBufferDirty(oldbuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(idxrel))
+		if (needwal)
 		{
 			xl_brin_samepage_update xlrec;
 			XLogRecPtr	recptr;
@@ -240,6 +245,9 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 
 		revmapbuf = brinLockRevmapPageForUpdate(revmap, heapBlk);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		/*
@@ -267,7 +275,7 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 		MarkBufferDirty(revmapbuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(idxrel))
+		if (needwal)
 		{
 			xl_brin_update xlrec;
 			XLogRecPtr	recptr;
@@ -351,6 +359,7 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	Buffer		revmapbuf;
 	ItemPointerData tid;
 	bool		extended;
+	bool		needwal;
 
 	Assert(itemsz == MAXALIGN(itemsz));
 
@@ -405,6 +414,10 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	page = BufferGetPage(*buffer);
 	blk = BufferGetBlockNumber(*buffer);
 
+	needwal = RelationNeedsWAL(idxrel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Execute the actual insertion */
 	START_CRIT_SECTION();
 	if (extended)
@@ -424,7 +437,7 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	MarkBufferDirty(revmapbuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(idxrel))
+	if (needwal)
 	{
 		xl_brin_insert xlrec;
 		XLogRecPtr	recptr;
@@ -881,6 +894,8 @@ brin_initialize_empty_new_buffer(Relation idxrel, Buffer buffer)
 			   "brin_initialize_empty_new_buffer: initializing blank page %u",
 			   BufferGetBlockNumber(buffer)));
 
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 	page = BufferGetPage(buffer);
 	brin_page_init(page, BRIN_PAGETYPE_REGULAR);
diff --git a/src/backend/access/brin/brin_revmap.c b/src/backend/access/brin/brin_revmap.c
index 35746714a7c..fd766da445d 100644
--- a/src/backend/access/brin/brin_revmap.c
+++ b/src/backend/access/brin/brin_revmap.c
@@ -26,6 +26,7 @@
 #include "access/brin_tuple.h"
 #include "access/brin_xlog.h"
 #include "access/rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -340,6 +341,7 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 	OffsetNumber revmapOffset;
 	OffsetNumber regOffset;
 	ItemId		lp;
+	bool		needwal;
 
 	revmap = brinRevmapInitialize(idxrel, &pagesPerRange, NULL);
 
@@ -404,6 +406,10 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 	 * crashed or aborted summarization; remove them silently.
 	 */
 
+	needwal = RelationNeedsWAL(idxrel);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	ItemPointerSetInvalid(&invalidIptr);
@@ -415,7 +421,7 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 	MarkBufferDirty(regBuf);
 	MarkBufferDirty(revmapBuf);
 
-	if (RelationNeedsWAL(idxrel))
+	if (needwal)
 	{
 		xl_brin_desummarize xlrec;
 		XLogRecPtr	recptr;
@@ -613,6 +619,8 @@ revmap_physical_extend(BrinRevmap *revmap)
 		return;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Ok, we have now locked the metapage and the target block. Re-initialize
 	 * the target block as a revmap page, and update the metapage.
diff --git a/src/backend/access/gin/ginbtree.c b/src/backend/access/gin/ginbtree.c
index 82788a5c367..f31590dcd75 100644
--- a/src/backend/access/gin/ginbtree.c
+++ b/src/backend/access/gin/ginbtree.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/predicate.h"
@@ -332,6 +333,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 {
 	Page		page = BufferGetPage(stack->buffer);
 	bool		result;
+	bool		needwal;
 	GinPlaceToPageRC rc;
 	uint16		xlflags = 0;
 	Page		childpage = NULL;
@@ -377,6 +379,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 								 insertdata, updateblkno,
 								 &ptp_workspace,
 								 &newlpage, &newrpage);
+	needwal = RelationNeedsWAL(btree->index) && !btree->isBuild;
 
 	if (rc == GPTP_NO_WORK)
 	{
@@ -385,10 +388,13 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 	}
 	else if (rc == GPTP_INSERT)
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		/* It will fit, perform the insertion */
 		START_CRIT_SECTION();
 
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogBeginInsert();
 			XLogRegisterBuffer(0, stack->buffer, REGBUF_STANDARD);
@@ -409,7 +415,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 			MarkBufferDirty(childbuf);
 		}
 
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 			ginxlogInsert xlrec;
@@ -547,6 +553,9 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 			}
 		}
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/*
 		 * OK, we have the new contents of the left page in a temporary copy
 		 * now (newlpage), and likewise for the new contents of the
@@ -587,7 +596,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 		}
 
 		/* write WAL record */
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gin/gindatapage.c b/src/backend/access/gin/gindatapage.c
index 7a2690e97f2..0abc5990100 100644
--- a/src/backend/access/gin/gindatapage.c
+++ b/src/backend/access/gin/gindatapage.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "lib/ilist.h"
 #include "miscadmin.h"
@@ -811,6 +812,7 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 	if (removedsomething)
 	{
 		bool		modified;
+		bool		needwal;
 
 		/*
 		 * Make sure we have a palloc'd copy of all segments, after the first
@@ -835,8 +837,12 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 			}
 		}
 
-		if (RelationNeedsWAL(indexrel))
+		needwal = RelationNeedsWAL(indexrel);
+		if (needwal)
+		{
+			CheckWALPermitted();
 			computeLeafRecompressWALData(leaf);
+		}
 
 		/* Apply changes to page */
 		START_CRIT_SECTION();
@@ -845,7 +851,7 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 
 		MarkBufferDirty(buffer);
 
-		if (RelationNeedsWAL(indexrel))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
@@ -1777,6 +1783,7 @@ createPostingTree(Relation index, ItemPointerData *items, uint32 nitems,
 	int			nrootitems;
 	int			rootsize;
 	bool		is_build = (buildStats != NULL);
+	bool		needwal;
 
 	/* Construct the new root page in memory first. */
 	tmppage = (Page) palloc(BLCKSZ);
@@ -1825,12 +1832,17 @@ createPostingTree(Relation index, ItemPointerData *items, uint32 nitems,
 	 */
 	PredicateLockPageSplit(index, BufferGetBlockNumber(entrybuffer), blkno);
 
+	needwal = RelationNeedsWAL(index) && !is_build;
+
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	PageRestoreTempPage(tmppage, page);
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index) && !is_build)
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogCreatePostingTree data;
diff --git a/src/backend/access/gin/ginfast.c b/src/backend/access/gin/ginfast.c
index 2e41b34d8d5..b8c2a993408 100644
--- a/src/backend/access/gin/ginfast.c
+++ b/src/backend/access/gin/ginfast.c
@@ -20,6 +20,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
 #include "catalog/pg_am.h"
@@ -68,6 +69,8 @@ writeListPage(Relation index, Buffer buffer,
 	PGAlignedBlock workspace;
 	char	   *ptr;
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	GinInitBuffer(buffer, GIN_LIST);
@@ -548,6 +551,7 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 	Page		metapage;
 	GinMetaPageData *metadata;
 	BlockNumber blknoToDelete;
+	bool		needwal = RelationNeedsWAL(index);
 
 	metapage = BufferGetPage(metabuffer);
 	metadata = GinPageGetMeta(metapage);
@@ -586,8 +590,11 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 		 * prepare the XLogInsert machinery for that before entering the
 		 * critical section.
 		 */
-		if (RelationNeedsWAL(index))
+		if (needwal)
+		{
+			CheckWALPermitted();
 			XLogEnsureRecordSpace(data.ndeleted, 0);
+		}
 
 		START_CRIT_SECTION();
 
@@ -625,7 +632,7 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 			MarkBufferDirty(buffers[i]);
 		}
 
-		if (RelationNeedsWAL(index))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gin/gininsert.c b/src/backend/access/gin/gininsert.c
index 77433dc8a41..989d82ffcaf 100644
--- a/src/backend/access/gin/gininsert.c
+++ b/src/backend/access/gin/gininsert.c
@@ -17,6 +17,7 @@
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/index.h"
 #include "miscadmin.h"
@@ -447,6 +448,9 @@ ginbuildempty(Relation index)
 		ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(RootBuffer, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermittedHaveXID();
+
 	/* Initialize and xlog metabuffer and root buffer. */
 	START_CRIT_SECTION();
 	GinInitMetabuffer(MetaBuffer);
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index ef9b56fd363..b48ea1a746a 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -17,6 +17,7 @@
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
 #include "access/reloptions.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/pg_collation.h"
 #include "catalog/pg_type.h"
@@ -659,12 +660,18 @@ ginUpdateStats(Relation index, const GinStatsData *stats, bool is_build)
 	Buffer		metabuffer;
 	Page		metapage;
 	GinMetaPageData *metadata;
+	bool		needwal;
 
 	metabuffer = ReadBuffer(index, GIN_METAPAGE_BLKNO);
 	LockBuffer(metabuffer, GIN_EXCLUSIVE);
 	metapage = BufferGetPage(metabuffer);
 	metadata = GinPageGetMeta(metapage);
 
+	needwal = RelationNeedsWAL(index) && !is_build;
+
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	metadata->nTotalPages = stats->nTotalPages;
@@ -684,7 +691,7 @@ ginUpdateStats(Relation index, const GinStatsData *stats, bool is_build)
 
 	MarkBufferDirty(metabuffer);
 
-	if (RelationNeedsWAL(index) && !is_build)
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogUpdateMeta data;
diff --git a/src/backend/access/gin/ginvacuum.c b/src/backend/access/gin/ginvacuum.c
index 0935a6d9e53..d91ca2b391c 100644
--- a/src/backend/access/gin/ginvacuum.c
+++ b/src/backend/access/gin/ginvacuum.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "commands/vacuum.h"
 #include "miscadmin.h"
@@ -136,6 +137,7 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	Page		page,
 				parentPage;
 	BlockNumber rightlink;
+	bool		needwal;
 
 	/*
 	 * This function MUST be called only if someone of parent pages hold
@@ -159,6 +161,10 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	 */
 	PredicateLockPageCombine(gvs->index, deleteBlkno, rightlink);
 
+	needwal = RelationNeedsWAL(gvs->index);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* Unlink the page by changing left sibling's rightlink */
@@ -195,7 +201,7 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	MarkBufferDirty(lBuffer);
 	MarkBufferDirty(dBuffer);
 
-	if (RelationNeedsWAL(gvs->index))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogDeletePage data;
@@ -650,6 +656,9 @@ ginbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 
 		if (resPage)
 		{
+			if (RelationNeedsWAL(gvs.index))
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 			PageRestoreTempPage(resPage, page);
 			MarkBufferDirty(buffer);
diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c
index 25b42e38f22..4a870a062ba 100644
--- a/src/backend/access/gist/gist.c
+++ b/src/backend/access/gist/gist.c
@@ -16,6 +16,7 @@
 
 #include "access/gist_private.h"
 #include "access/gistscan.h"
+#include "access/walprohibit.h"
 #include "catalog/pg_collation.h"
 #include "commands/vacuum.h"
 #include "miscadmin.h"
@@ -135,6 +136,9 @@ gistbuildempty(Relation index)
 	buffer = ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermittedHaveXID();
+
 	/* Initialize and xlog buffer */
 	START_CRIT_SECTION();
 	GISTInitBuffer(buffer, F_LEAF);
@@ -234,6 +238,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 	XLogRecPtr	recptr;
 	int			i;
 	bool		is_split;
+	bool		needwal = RelationNeedsWAL(rel);
 
 	/*
 	 * Refuse to modify a page that's incompletely split. This should not
@@ -465,9 +470,12 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 		 * insertion for that. NB: The number of pages and data segments
 		 * specified here must match the calculations in gistXLogSplit()!
 		 */
-		if (!is_build && RelationNeedsWAL(rel))
+		if (!is_build && needwal)
 			XLogEnsureRecordSpace(npage, 1 + npage * 2);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		/*
@@ -500,7 +508,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 			recptr = GistBuildLSN;
 		else
 		{
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 				recptr = gistXLogSplit(is_leaf,
 									   dist, oldrlink, oldnsn, leftchildbuf,
 									   markfollowright);
@@ -526,6 +534,9 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 	}
 	else
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		/*
 		 * Enough space.  We always get here if ntup==0.
 		 */
@@ -567,7 +578,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 			recptr = GistBuildLSN;
 		else
 		{
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 			{
 				OffsetNumber ndeloffs = 0,
 							deloffs[1];
@@ -1641,6 +1652,7 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 	OffsetNumber offnum,
 				maxoff;
 	TransactionId latestRemovedXid = InvalidTransactionId;
+	bool		needwal = RelationNeedsWAL(rel);
 
 	Assert(GistPageIsLeaf(page));
 
@@ -1659,13 +1671,16 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 			deletable[ndeletable++] = offnum;
 	}
 
-	if (XLogStandbyInfoActive() && RelationNeedsWAL(rel))
+	if (XLogStandbyInfoActive() && needwal)
 		latestRemovedXid =
 			index_compute_xid_horizon_for_tuples(rel, heapRel, buffer,
 												 deletable, ndeletable);
 
 	if (ndeletable > 0)
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		PageIndexMultiDelete(page, deletable, ndeletable);
@@ -1682,7 +1697,7 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 		MarkBufferDirty(buffer);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gist/gistvacuum.c b/src/backend/access/gist/gistvacuum.c
index a9c616c7724..bbb3ebb19ad 100644
--- a/src/backend/access/gist/gistvacuum.c
+++ b/src/backend/access/gist/gistvacuum.c
@@ -17,6 +17,7 @@
 #include "access/genam.h"
 #include "access/gist_private.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "commands/vacuum.h"
 #include "lib/integerset.h"
 #include "miscadmin.h"
@@ -260,6 +261,7 @@ gistvacuumpage(GistVacState *vstate, BlockNumber blkno, BlockNumber orig_blkno)
 	Buffer		buffer;
 	Page		page;
 	BlockNumber recurse_to;
+	bool		needwal = RelationNeedsWAL(rel);
 
 restart:
 	recurse_to = InvalidBlockNumber;
@@ -341,6 +343,9 @@ restart:
 		 */
 		if (ntodelete > 0)
 		{
+			if (needwal)
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 
 			MarkBufferDirty(buffer);
@@ -348,7 +353,7 @@ restart:
 			PageIndexMultiDelete(page, todelete, ntodelete);
 			GistMarkTuplesDeleted(page);
 
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 			{
 				XLogRecPtr	recptr;
 
@@ -580,6 +585,7 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	IndexTuple	idxtuple;
 	XLogRecPtr	recptr;
 	FullTransactionId txid;
+	bool		needwal = RelationNeedsWAL(info->index);
 
 	/*
 	 * Check that the leaf is still empty and deletable.
@@ -634,6 +640,9 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	 */
 	txid = ReadNextFullTransactionId();
 
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* mark the page as deleted */
@@ -645,7 +654,7 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	MarkBufferDirty(parentBuffer);
 	PageIndexTupleDelete(parentPage, downlink);
 
-	if (RelationNeedsWAL(info->index))
+	if (needwal)
 		recptr = gistXLogPageDelete(leafBuffer, txid, parentBuffer, downlink);
 	else
 		recptr = gistGetFakeLSN(info->index);
diff --git a/src/backend/access/hash/hash.c b/src/backend/access/hash/hash.c
index 7c9ccf446c8..f4903a43bb5 100644
--- a/src/backend/access/hash/hash.c
+++ b/src/backend/access/hash/hash.c
@@ -22,6 +22,7 @@
 #include "access/hash_xlog.h"
 #include "access/relscan.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "catalog/index.h"
 #include "commands/progress.h"
 #include "commands/vacuum.h"
@@ -467,6 +468,7 @@ hashbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	Buffer		metabuf = InvalidBuffer;
 	HashMetaPage metap;
 	HashMetaPage cachedmetap;
+	bool		needwal;
 
 	tuples_removed = 0;
 	num_index_tuples = 0;
@@ -573,6 +575,10 @@ loop_top:
 		goto loop_top;
 	}
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Okay, we're really done.  Update tuple count in metapage. */
 	START_CRIT_SECTION();
 
@@ -603,7 +609,7 @@ loop_top:
 	MarkBufferDirty(metabuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_hash_update_meta_page xlrec;
 		XLogRecPtr	recptr;
@@ -690,6 +696,7 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 	Buffer		buf;
 	Bucket		new_bucket PG_USED_FOR_ASSERTS_ONLY = InvalidBucket;
 	bool		bucket_dirty = false;
+	bool		needwal = RelationNeedsWAL(rel);
 
 	blkno = bucket_blkno;
 	buf = bucket_buf;
@@ -788,6 +795,9 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		 */
 		if (ndeletable > 0)
 		{
+			if (needwal)
+				CheckWALPermitted();
+
 			/* No ereport(ERROR) until changes are logged */
 			START_CRIT_SECTION();
 
@@ -809,7 +819,7 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 			MarkBufferDirty(buf);
 
 			/* XLOG stuff */
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 			{
 				xl_hash_delete xlrec;
 				XLogRecPtr	recptr;
@@ -883,6 +893,9 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		page = BufferGetPage(bucket_buf);
 		bucket_opaque = (HashPageOpaque) PageGetSpecialPointer(page);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -890,7 +903,7 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		MarkBufferDirty(bucket_buf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c
index 2ebe671967b..2eab69efa91 100644
--- a/src/backend/access/hash/hashinsert.c
+++ b/src/backend/access/hash/hashinsert.c
@@ -17,6 +17,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "storage/buf_internals.h"
 #include "storage/lwlock.h"
@@ -193,6 +194,8 @@ restart_insert:
 	 */
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	AssertWALPermittedHaveXID();
+
 	/* Do the update.  No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -360,6 +363,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 	if (ndeletable > 0)
 	{
 		TransactionId latestRemovedXid;
+		bool		needwal = RelationNeedsWAL(rel);
 
 		latestRemovedXid =
 			index_compute_xid_horizon_for_tuples(rel, hrel, buf,
@@ -370,6 +374,9 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 		 */
 		LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -393,7 +400,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 		MarkBufferDirty(metabuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			xl_hash_vacuum_one_page xlrec;
 			XLogRecPtr	recptr;
diff --git a/src/backend/access/hash/hashovfl.c b/src/backend/access/hash/hashovfl.c
index 00f0a940116..e7c5dd3e3ce 100644
--- a/src/backend/access/hash/hashovfl.c
+++ b/src/backend/access/hash/hashovfl.c
@@ -19,6 +19,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "utils/rel.h"
 
@@ -312,6 +313,8 @@ _hash_addovflpage(Relation rel, Buffer metabuf, Buffer buf, bool retain_pin)
 
 found:
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Do the update.  No ereport(ERROR) until changes are logged. We want to
 	 * log the changes for bitmap page and overflow page together to avoid
@@ -510,6 +513,7 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	Buffer		prevbuf = InvalidBuffer;
 	Buffer		nextbuf = InvalidBuffer;
 	bool		update_metap = false;
+	bool		needwal;
 
 	/* Get information from the doomed page */
 	_hash_checkpage(rel, ovflbuf, LH_OVERFLOW_PAGE);
@@ -573,10 +577,15 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	/* Get write-lock on metapage to update firstfree */
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	needwal = RelationNeedsWAL(rel);
+
 	/* This operation needs to log multiple tuples, prepare WAL for that */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 		XLogEnsureRecordSpace(HASH_XLOG_FREE_OVFL_BUFS, 4 + nitups);
 
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -641,7 +650,7 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	}
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_hash_squeeze_page xlrec;
 		XLogRecPtr	recptr;
@@ -922,14 +931,19 @@ readpage:
 
 				if (nitups > 0)
 				{
+					bool		needwal = RelationNeedsWAL(rel);
+
 					Assert(nitups == ndeletable);
 
 					/*
 					 * This operation needs to log multiple tuples, prepare
 					 * WAL for that.
 					 */
-					if (RelationNeedsWAL(rel))
+					if (needwal)
+					{
+						CheckWALPermitted();
 						XLogEnsureRecordSpace(0, 3 + nitups);
+					}
 
 					START_CRIT_SECTION();
 
@@ -947,7 +961,7 @@ readpage:
 					MarkBufferDirty(rbuf);
 
 					/* XLOG stuff */
-					if (RelationNeedsWAL(rel))
+					if (needwal)
 					{
 						XLogRecPtr	recptr;
 						xl_hash_move_page_contents xlrec;
diff --git a/src/backend/access/hash/hashpage.c b/src/backend/access/hash/hashpage.c
index a664ecf494a..55a867dd375 100644
--- a/src/backend/access/hash/hashpage.c
+++ b/src/backend/access/hash/hashpage.c
@@ -30,6 +30,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "port/pg_bitutils.h"
 #include "storage/lmgr.h"
@@ -816,6 +817,8 @@ restart_expand:
 		goto fail;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Since we are scribbling on the pages in the shared buffers, establish a
 	 * critical section.  Any failure in this next code leaves us with a big
@@ -1172,6 +1175,8 @@ _hash_splitbucket(Relation rel,
 
 				if (PageGetFreeSpaceForMultipleTuples(npage, nitups + 1) < (all_tups_size + itemsz))
 				{
+					AssertWALPermittedHaveXID();
+
 					/*
 					 * Change the shared buffer state in critical section,
 					 * otherwise any error could make it unrecoverable.
@@ -1223,6 +1228,8 @@ _hash_splitbucket(Relation rel,
 		/* Exit loop if no more overflow pages in old bucket */
 		if (!BlockNumberIsValid(oblkno))
 		{
+			AssertWALPermittedHaveXID();
+
 			/*
 			 * Change the shared buffer state in critical section, otherwise
 			 * any error could make it unrecoverable.
@@ -1269,6 +1276,8 @@ _hash_splitbucket(Relation rel,
 	npage = BufferGetPage(bucket_nbuf);
 	nopaque = (HashPageOpaque) PageGetSpecialPointer(npage);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	oopaque->hasho_flag &= ~LH_BUCKET_BEING_SPLIT;
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 1585861a021..4d6052224fa 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -47,6 +47,7 @@
 #include "access/transam.h"
 #include "access/valid.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1898,6 +1899,8 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
 	 */
 	CheckForSerializableConflictIn(relation, NULL, InvalidBlockNumber);
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -2172,6 +2175,8 @@ heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples,
 										   &vmbuffer, NULL);
 		page = BufferGetPage(buffer);
 
+		AssertWALPermittedHaveXID();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -2690,6 +2695,8 @@ l1:
 							  xid, LockTupleExclusive, true,
 							  &new_xmax, &new_infomask, &new_infomask2);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -3442,6 +3449,8 @@ l2:
 
 		Assert(HEAP_XMAX_IS_LOCKED_ONLY(infomask_lock_old_tuple));
 
+		AssertWALPermittedHaveXID();
+
 		START_CRIT_SECTION();
 
 		/* Clear obsolete visibility flags ... */
@@ -3615,6 +3624,8 @@ l2:
 										   bms_overlap(modified_attrs, id_attrs),
 										   &old_key_copied);
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -4548,6 +4559,8 @@ failed:
 							  GetCurrentTransactionId(), mode, false,
 							  &xid, &new_infomask, &new_infomask2);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -5339,6 +5352,8 @@ l4:
 								VISIBILITYMAP_ALL_FROZEN))
 			cleared_all_frozen = true;
 
+		AssertWALPermittedHaveXID();
+
 		START_CRIT_SECTION();
 
 		/* ... and set them */
@@ -5497,6 +5512,8 @@ heap_finish_speculative(Relation relation, ItemPointer tid)
 	StaticAssertStmt(MaxOffsetNumber < SpecTokenOffsetNumber,
 					 "invalid speculative token constant");
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -5605,6 +5622,8 @@ heap_abort_speculative(Relation relation, ItemPointer tid)
 		elog(ERROR, "attempted to kill a non-speculative tuple");
 	Assert(!HeapTupleHeaderIsHeapOnly(tp.t_data));
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * No need to check for serializable conflicts here.  There is never a
 	 * need for a combocid, either.  No need to extract replica identity, or
@@ -5721,6 +5740,7 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	HeapTupleHeader htup;
 	uint32		oldlen;
 	uint32		newlen;
+	bool		needwal;
 
 	/*
 	 * For now, we don't allow parallel updates.  Unlike a regular update,
@@ -5751,6 +5771,10 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	if (oldlen != newlen || htup->t_hoff != tuple->t_data->t_hoff)
 		elog(ERROR, "wrong tuple length");
 
+	needwal = RelationNeedsWAL(relation);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -5761,7 +5785,7 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	MarkBufferDirty(buffer);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(relation))
+	if (needwal)
 	{
 		xl_heap_inplace xlrec;
 		XLogRecPtr	recptr;
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index bc510e2e9b3..9dcae7d2153 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -18,6 +18,7 @@
 #include "access/heapam_xlog.h"
 #include "access/htup_details.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "catalog/catalog.h"
 #include "miscadmin.h"
@@ -94,11 +95,11 @@ heap_page_prune_opt(Relation relation, Buffer buffer)
 	Size		minfree;
 
 	/*
-	 * We can't write WAL in recovery mode, so there's no point trying to
+	 * We can't write WAL during read-only mode, so there's no point trying to
 	 * clean the page. The primary will likely issue a cleaning WAL record soon
 	 * anyway, so this is no particular loss.
 	 */
-	if (RecoveryInProgress())
+	if (!XLogInsertAllowed())
 		return;
 
 	/*
@@ -232,6 +233,7 @@ heap_page_prune(Relation relation, Buffer buffer,
 	OffsetNumber offnum,
 				maxoff;
 	PruneState	prstate;
+	bool		needwal;
 
 	/*
 	 * Our strategy is to scan the page and make lists of items to change,
@@ -286,6 +288,10 @@ heap_page_prune(Relation relation, Buffer buffer,
 	if (off_loc)
 		*off_loc = InvalidOffsetNumber;
 
+	needwal = RelationNeedsWAL(relation);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Any error while applying the changes is critical */
 	START_CRIT_SECTION();
 
@@ -319,7 +325,7 @@ heap_page_prune(Relation relation, Buffer buffer,
 		/*
 		 * Emit a WAL XLOG_HEAP2_CLEAN record showing what we did
 		 */
-		if (RelationNeedsWAL(relation))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 4f2f38168dc..1869df5f03f 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -59,6 +59,7 @@
 #include "access/parallel.h"
 #include "access/transam.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "catalog/storage.h"
@@ -759,6 +760,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 	BlockNumber nblocks,
 				blkno;
 	HeapTupleData tuple;
+	bool		needwal = RelationNeedsWAL(onerel);
 	TransactionId relfrozenxid = onerel->rd_rel->relfrozenxid;
 	TransactionId relminmxid = onerel->rd_rel->relminmxid;
 	BlockNumber empty_pages,
@@ -1201,6 +1203,9 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 			 */
 			if (!PageIsAllVisible(page))
 			{
+				if (needwal)
+					CheckWALPermitted();
+
 				START_CRIT_SECTION();
 
 				/* mark buffer dirty before writing a WAL record */
@@ -1216,7 +1221,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 				 * page has been previously WAL-logged, and if not, do that
 				 * now.
 				 */
-				if (RelationNeedsWAL(onerel) &&
+				if (needwal &&
 					PageGetLSN(page) == InvalidXLogRecPtr)
 					log_newpage_buffer(buf, true);
 
@@ -1482,6 +1487,9 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 		 */
 		if (nfrozen > 0)
 		{
+			if (needwal)
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 
 			MarkBufferDirty(buf);
@@ -1499,7 +1507,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 			}
 
 			/* Now WAL-log freezing if necessary */
-			if (RelationNeedsWAL(onerel))
+			if (needwal)
 			{
 				XLogRecPtr	recptr;
 
@@ -1932,6 +1940,7 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	TransactionId visibility_cutoff_xid;
 	bool		all_frozen;
 	LVSavedErrInfo saved_err_info;
+	bool		needwal = RelationNeedsWAL(onerel);
 
 	pgstat_progress_update_param(PROGRESS_VACUUM_HEAP_BLKS_VACUUMED, blkno);
 
@@ -1939,6 +1948,9 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	update_vacuum_error_info(vacrelstats, &saved_err_info, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
 							 blkno, InvalidOffsetNumber);
 
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	for (; tupindex < dead_tuples->num_tuples; tupindex++)
@@ -1964,7 +1976,7 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	MarkBufferDirty(buffer);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(onerel))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index b1072183bcd..44244363968 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -88,6 +88,7 @@
 
 #include "access/heapam_xlog.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "miscadmin.h"
 #include "port/pg_bitutils.h"
@@ -249,6 +250,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 	uint8		mapOffset = HEAPBLK_TO_OFFSET(heapBlk);
 	Page		page;
 	uint8	   *map;
+	bool		needwal = RelationNeedsWAL(rel);
 
 #ifdef TRACE_VISIBILITYMAP
 	elog(DEBUG1, "vm_set %s %d", RelationGetRelationName(rel), heapBlk);
@@ -270,6 +272,16 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 	map = (uint8 *) PageGetContents(page);
 	LockBuffer(vmBuf, BUFFER_LOCK_EXCLUSIVE);
 
+	/*
+	 * Can reach here from VACUUM or from startup process, so need not have an
+	 * XID.
+	 *
+	 * Recovery in the startup process never have wal prohibit state, skip
+	 * permission check if reach here in the startup process.
+	 */
+	if (needwal)
+		InRecovery ? AssertWALPermitted() : CheckWALPermitted();
+
 	if (flags != (map[mapByte] >> mapOffset & VISIBILITYMAP_VALID_BITS))
 	{
 		START_CRIT_SECTION();
@@ -277,7 +289,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 		map[mapByte] |= (flags << mapOffset);
 		MarkBufferDirty(vmBuf);
 
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			if (XLogRecPtrIsInvalid(recptr))
 			{
@@ -476,6 +488,7 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		Buffer		mapBuffer;
 		Page		page;
 		char	   *map;
+		bool		needwal;
 
 		newnblocks = truncBlock + 1;
 
@@ -489,8 +502,13 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		page = BufferGetPage(mapBuffer);
 		map = PageGetContents(page);
 
+		needwal = (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded());
+
 		LockBuffer(mapBuffer, BUFFER_LOCK_EXCLUSIVE);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -518,7 +536,7 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		 * during recovery.
 		 */
 		MarkBufferDirty(mapBuffer);
-		if (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded())
+		if (needwal)
 			log_newpage_buffer(mapBuffer, false);
 
 		END_CRIT_SECTION();
diff --git a/src/backend/access/nbtree/nbtdedup.c b/src/backend/access/nbtree/nbtdedup.c
index f6be865b17e..b519a1268e8 100644
--- a/src/backend/access/nbtree/nbtdedup.c
+++ b/src/backend/access/nbtree/nbtdedup.c
@@ -16,6 +16,7 @@
 
 #include "access/nbtree.h"
 #include "access/nbtxlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "utils/rel.h"
 
@@ -271,6 +272,8 @@ _bt_dedup_one_page(Relation rel, Buffer buf, Relation heapRel,
 		nopaque->btpo_flags &= ~BTP_HAS_GARBAGE;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	PageRestoreTempPage(newpage, page);
diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c
index d36f7557c87..2c3d8aaecbd 100644
--- a/src/backend/access/nbtree/nbtinsert.c
+++ b/src/backend/access/nbtree/nbtinsert.c
@@ -19,6 +19,7 @@
 #include "access/nbtxlog.h"
 #include "access/tableam.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/lmgr.h"
@@ -1246,6 +1247,8 @@ _bt_insertonpg(Relation rel,
 			}
 		}
 
+		AssertWALPermittedHaveXID();
+
 		/* Do the update.  No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -1898,13 +1901,16 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf,
 			ropaque->btpo_flags |= BTP_SPLIT_END;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Right sibling is locked, new siblings are prepared, but original page
 	 * is not updated yet.
 	 *
 	 * NO EREPORT(ERROR) till right sibling is updated.  We can get away with
 	 * not starting the critical section till here because we haven't been
-	 * scribbling on the original page yet; see comments above.
+	 * scribbling on the original page yet; see the comments above for grabbing
+	 * the right sibling.
 	 */
 	START_CRIT_SECTION();
 
@@ -2467,6 +2473,8 @@ _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf)
 	right_item = CopyIndexTuple(item);
 	BTreeTupleSetDownLink(right_item, rbkno);
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till newroot op is logged */
 	START_CRIT_SECTION();
 
diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c
index 7f392480ac0..8c3fc251a29 100644
--- a/src/backend/access/nbtree/nbtpage.c
+++ b/src/backend/access/nbtree/nbtpage.c
@@ -26,6 +26,7 @@
 #include "access/nbtxlog.h"
 #include "access/tableam.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
@@ -179,6 +180,7 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	BTMetaPageData *metad;
 	bool		needsRewrite = false;
 	XLogRecPtr	recptr;
+	bool		needwal;
 
 	/* read the metapage and check if it needs rewrite */
 	metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ);
@@ -202,6 +204,10 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	_bt_unlockbuf(rel, metabuf);
 	_bt_lockbuf(rel, metabuf, BT_WRITE);
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* upgrade meta-page if needed */
@@ -214,7 +220,7 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	MarkBufferDirty(metabuf);
 
 	/* write wal record if needed */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_btree_metadata md;
 
@@ -332,6 +338,7 @@ _bt_getroot(Relation rel, int access)
 	if (metad->btm_root == P_NONE)
 	{
 		Page		metapg;
+		bool		needwal;
 
 		/* If access = BT_READ, caller doesn't want us to create root yet */
 		if (access == BT_READ)
@@ -377,6 +384,10 @@ _bt_getroot(Relation rel, int access)
 		/* Get raw page pointer for metapage */
 		metapg = BufferGetPage(metabuf);
 
+		needwal = RelationNeedsWAL(rel);
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO ELOG(ERROR) till meta is updated */
 		START_CRIT_SECTION();
 
@@ -395,7 +406,7 @@ _bt_getroot(Relation rel, int access)
 		MarkBufferDirty(metabuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			xl_btree_newroot xlrec;
 			XLogRecPtr	recptr;
@@ -1131,6 +1142,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 	char	   *updatedbuf = NULL;
 	Size		updatedbuflen = 0;
 	OffsetNumber updatedoffsets[MaxIndexTuplesPerPage];
+	bool		needwal = RelationNeedsWAL(rel);
 
 	/* Shouldn't be called unless there's something to do */
 	Assert(ndeletable > 0 || nupdatable > 0);
@@ -1145,7 +1157,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 	}
 
 	/* XLOG stuff -- allocate and fill buffer before critical section */
-	if (nupdatable > 0 && RelationNeedsWAL(rel))
+	if (nupdatable > 0 && needwal)
 	{
 		Size		offset = 0;
 
@@ -1175,6 +1187,9 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 		}
 	}
 
+	if (needwal)
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -1235,7 +1250,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 	MarkBufferDirty(buf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		xl_btree_vacuum xlrec_vacuum;
@@ -1302,6 +1317,8 @@ _bt_delitems_delete(Relation rel, Buffer buf,
 		latestRemovedXid =
 			_bt_xid_horizon(rel, heapRel, page, deletable, ndeletable);
 
+	AssertWALPermittedHaveXID();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -1832,6 +1849,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	OffsetNumber nextoffset;
 	IndexTuple	itup;
 	IndexTupleData trunctuple;
+	bool		needwal;
 
 	page = BufferGetPage(leafbuf);
 	opaque = (BTPageOpaque) PageGetSpecialPointer(page);
@@ -1920,6 +1938,10 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	 */
 	PredicateLockPageCombine(rel, leafblkno, leafrightsib);
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -1971,7 +1993,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	MarkBufferDirty(leafbuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_btree_mark_page_halfdead xlrec;
 		XLogRecPtr	recptr;
@@ -2064,6 +2086,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 	int			targetlevel;
 	IndexTuple	leafhikey;
 	BlockNumber nextchild;
+	bool		needwal;
 
 	page = BufferGetPage(leafbuf);
 	opaque = (BTPageOpaque) PageGetSpecialPointer(page);
@@ -2277,6 +2300,10 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 	 * Here we begin doing the deletion.
 	 */
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -2356,7 +2383,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 		MarkBufferDirty(leafbuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_btree_unlink_page xlrec;
 		xl_btree_metadata xlmeta;
diff --git a/src/backend/access/spgist/spgdoinsert.c b/src/backend/access/spgist/spgdoinsert.c
index 934d65b89f2..3c5a15c5d32 100644
--- a/src/backend/access/spgist/spgdoinsert.c
+++ b/src/backend/access/spgist/spgdoinsert.c
@@ -18,6 +18,7 @@
 #include "access/genam.h"
 #include "access/spgist_private.h"
 #include "access/spgxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -214,6 +215,8 @@ addLeafTuple(Relation index, SpGistState *state, SpGistLeafTuple leafTuple,
 	xlrec.offnumParent = InvalidOffsetNumber;
 	xlrec.nodeI = 0;
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	if (current->offnum == InvalidOffsetNumber ||
@@ -458,6 +461,8 @@ moveLeafs(Relation index, SpGistState *state,
 
 	leafdata = leafptr = palloc(size);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/* copy all the old tuples to new page, unless they're dead */
@@ -1109,6 +1114,8 @@ doPickSplit(Relation index, SpGistState *state,
 
 	leafdata = leafptr = (char *) palloc(totalLeafSizes);
 
+	AssertWALPermittedHaveXID();
+
 	/* Here we begin making the changes to the target pages */
 	START_CRIT_SECTION();
 
@@ -1517,6 +1524,8 @@ spgAddNodeAction(Relation index, SpGistState *state,
 	if (PageGetExactFreeSpace(current->page) >=
 		newInnerTuple->size - innerTuple->size)
 	{
+		AssertWALPermittedHaveXID();
+
 		/*
 		 * We can replace the inner tuple by new version in-place
 		 */
@@ -1603,6 +1612,8 @@ spgAddNodeAction(Relation index, SpGistState *state,
 		else
 			xlrec.parentBlk = 2;
 
+		AssertWALPermittedHaveXID();
+
 		START_CRIT_SECTION();
 
 		/* insert new ... */
@@ -1788,6 +1799,8 @@ spgSplitNodeAction(Relation index, SpGistState *state,
 									&xlrec.newPage);
 	}
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c
index e1c58933f97..3308832b85b 100644
--- a/src/backend/access/spgist/spgvacuum.c
+++ b/src/backend/access/spgist/spgvacuum.c
@@ -19,6 +19,7 @@
 #include "access/spgist_private.h"
 #include "access/spgxlog.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/storage_xlog.h"
 #include "commands/vacuum.h"
@@ -139,6 +140,7 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 	int			nDeletable;
 	OffsetNumber i,
 				max = PageGetMaxOffsetNumber(page);
+	bool		needwal;
 
 	memset(predecessor, 0, sizeof(predecessor));
 	memset(deletable, 0, sizeof(deletable));
@@ -323,6 +325,10 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 	if (nDeletable != xlrec.nDead + xlrec.nPlaceholder + xlrec.nMove)
 		elog(ERROR, "inconsistent counts of deletable tuples");
 
+	needwal = RelationNeedsWAL(index);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Do the updates */
 	START_CRIT_SECTION();
 
@@ -371,7 +377,7 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 
@@ -411,6 +417,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 	OffsetNumber toDelete[MaxIndexTuplesPerPage];
 	OffsetNumber i,
 				max = PageGetMaxOffsetNumber(page);
+	bool		needwal;
 
 	xlrec.nDelete = 0;
 
@@ -447,6 +454,10 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 	if (xlrec.nDelete == 0)
 		return;					/* nothing more to do */
 
+	needwal = RelationNeedsWAL(index);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Do the update */
 	START_CRIT_SECTION();
 
@@ -455,7 +466,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 
@@ -502,6 +513,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer)
 	OffsetNumber itemnos[MaxIndexTuplesPerPage];
 	spgxlogVacuumRedirect xlrec;
 	GlobalVisState *vistest;
+	bool		needwal;
 
 	xlrec.nToPlaceholder = 0;
 	xlrec.newestRedirectXid = InvalidTransactionId;
@@ -509,6 +521,10 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer)
 	/* XXX: providing heap relation would allow more pruning */
 	vistest = GlobalVisTestFor(NULL);
 
+	needwal = RelationNeedsWAL(index);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -584,7 +600,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer)
 	if (hasUpdate)
 		MarkBufferDirty(buffer);
 
-	if (hasUpdate && RelationNeedsWAL(index))
+	if (hasUpdate && needwal)
 	{
 		XLogRecPtr	recptr;
 
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index b8bedca04a4..0a88740764f 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -73,6 +73,7 @@
 #include "access/transam.h"
 #include "access/twophase.h"
 #include "access/twophase_rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1143,6 +1144,8 @@ GetNewMultiXactId(int nmembers, MultiXactOffset *offset)
 
 	ExtendMultiXactMember(nextOffset, nmembers);
 
+	CheckWALPermitted();
+
 	/*
 	 * Critical section from here until caller has written the data into the
 	 * just-reserved SLRU space; we don't want to error out with a partly
@@ -2938,7 +2941,7 @@ TruncateMultiXact(MultiXactId newOldestMulti, Oid newOldestMultiDB)
 	mxtruncinfo trunc;
 	MultiXactId earliest;
 
-	Assert(!RecoveryInProgress());
+	Assert(XLogInsertAllowed());
 	Assert(MultiXactState->finishedStartup);
 
 	/*
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index ef4f9981e35..ff2bc8cc74b 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -82,6 +82,7 @@
 #include "access/transam.h"
 #include "access/twophase.h"
 #include "access/twophase_rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1106,6 +1107,8 @@ EndPrepare(GlobalTransaction gxact)
 	 */
 	XLogEnsureRecordSpace(0, records.num_chunks);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	MyProc->delayChkpt = true;
@@ -2196,6 +2199,9 @@ RecordTransactionCommitPrepared(TransactionId xid,
 	replorigin = (replorigin_session_origin != InvalidRepOriginId &&
 				  replorigin_session_origin != DoNotReplicateId);
 
+	/* COMMIT PREPARED need not have an XID */
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* See notes in RecordTransactionCommit */
@@ -2286,6 +2292,9 @@ RecordTransactionAbortPrepared(TransactionId xid,
 		elog(PANIC, "cannot abort transaction %u, it was already committed",
 			 xid);
 
+	/* ROLLBACK PREPARED need not have an XID */
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
diff --git a/src/backend/access/transam/varsup.c b/src/backend/access/transam/varsup.c
index a4944faa32e..0c7a2362f25 100644
--- a/src/backend/access/transam/varsup.c
+++ b/src/backend/access/transam/varsup.c
@@ -17,6 +17,7 @@
 #include "access/commit_ts.h"
 #include "access/subtrans.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "commands/dbcommands.h"
@@ -75,6 +76,9 @@ GetNewTransactionId(bool isSubXact)
 	if (RecoveryInProgress())
 		elog(ERROR, "cannot assign TransactionIds during recovery");
 
+	/* do not assign transaction id in read-only mode */
+	CheckWALPermitted();
+
 	LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
 
 	full_xid = ShmemVariableCache->nextXid;
diff --git a/src/backend/access/transam/walprohibit.c b/src/backend/access/transam/walprohibit.c
index 75f3924cc97..f273f75b41f 100644
--- a/src/backend/access/transam/walprohibit.c
+++ b/src/backend/access/transam/walprohibit.c
@@ -24,6 +24,16 @@
 #include "utils/fmgroids.h"
 #include "utils/fmgrprotos.h"
 
+/*
+ * Assert flag to enforce WAL insert permission check rule before starting a
+ * critical section for the WAL writes.  For this, either of CheckWALPermitted,
+ * AssertWALPermittedHaveXID, or AssertWALPermitted must be called before
+ * starting the critical section.
+ */
+#ifdef USE_ASSERT_CHECKING
+WALPermitCheckState walpermit_checked_state = WALPERMIT_UNCHECKED;
+#endif
+
 /*
  * Shared-memory WAL prohibit state
  */
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 188c299bed9..abda095e735 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -26,6 +26,7 @@
 #include "access/subtrans.h"
 #include "access/transam.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1317,6 +1318,8 @@ RecordTransactionCommit(void)
 		/* Tell bufmgr and smgr to prepare for commit */
 		BufmgrCommit();
 
+		AssertWALPermittedHaveXID();
+
 		/*
 		 * Mark ourselves as within our "commit critical section".  This
 		 * forces any concurrent checkpoint to wait until we've updated
@@ -1677,6 +1680,9 @@ RecordTransactionAbort(bool isSubXact)
 		elog(PANIC, "cannot abort transaction %u, it was already committed",
 			 xid);
 
+	/* We'll be reaching here with valid XID only. */
+	AssertWALPermittedHaveXID();
+
 	/* Fetch the data we need for the abort record */
 	nrels = smgrGetPendingDeletes(false, &rels);
 	nchildren = xactGetCommittedChildren(&children);
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index bfba75bbe80..253409ca065 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -1025,7 +1025,7 @@ XLogInsertRecord(XLogRecData *rdata,
 
 	/* cross-check on whether we should be here or not */
 	if (!XLogInsertAllowed())
-		elog(ERROR, "cannot make new WAL entries during recovery");
+		elog(ERROR, "cannot make new WAL entries now");
 
 	/*----------
 	 *
@@ -2860,9 +2860,11 @@ XLogFlush(XLogRecPtr record)
 	 * trying to flush the WAL, we should update minRecoveryPoint instead. We
 	 * test XLogInsertAllowed(), not InRecovery, because we need checkpointer
 	 * to act this way too, and because when it tries to write the
-	 * end-of-recovery checkpoint, it should indeed flush.
+	 * end-of-recovery checkpoint, it should indeed flush. Also, WAL prohibit
+	 * state should not restrict WAL flushing. Otherwise, the dirty buffer
+	 * cannot be evicted until WAL has been flushed up to the buffer's LSN.
 	 */
-	if (!XLogInsertAllowed())
+	if (!XLogInsertAllowed() && !IsWALProhibited())
 	{
 		UpdateMinRecoveryPoint(record, false);
 		return;
@@ -8826,6 +8828,8 @@ CreateCheckPoint(int flags)
 	/* sanity check */
 	if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
 		elog(ERROR, "can't create a checkpoint during recovery");
+	if (!XLogInsertAllowed() && !RecoveryInProgress())
+		elog(ERROR, "can't create a checkpoint while system is read only ");
 
 	/*
 	 * Initialize InitXLogInsert working areas before entering the critical
@@ -8855,6 +8859,8 @@ CreateCheckPoint(int flags)
 	MemSet(&CheckpointStats, 0, sizeof(CheckpointStats));
 	CheckpointStats.ckpt_start_t = GetCurrentTimestamp();
 
+	AssertWALPermitted();
+
 	/*
 	 * Use a critical section to force system panic if we have trouble.
 	 */
@@ -9083,6 +9089,8 @@ CreateCheckPoint(int flags)
 	if (!shutdown && XLogStandbyInfoActive())
 		LogStandbySnapshot();
 
+	AssertWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -9240,6 +9248,8 @@ CreateEndOfRecoveryRecord(void)
 
 	LocalSetXLogInsertAllowed();
 
+	AssertWALPermitted();
+
 	START_CRIT_SECTION();
 
 	XLogBeginInsert();
@@ -9886,7 +9896,7 @@ void
 UpdateFullPageWrites(void)
 {
 	XLogCtlInsert *Insert = &XLogCtl->Insert;
-	bool		recoveryInProgress;
+	bool		WALInsertAllowed;
 
 	/*
 	 * Do nothing if full_page_writes has not been changed.
@@ -9900,10 +9910,10 @@ UpdateFullPageWrites(void)
 
 	/*
 	 * Perform this outside critical section so that the WAL insert
-	 * initialization done by RecoveryInProgress() doesn't trigger an
+	 * initialization done by XLogInsertAllowed() doesn't trigger an
 	 * assertion failure.
 	 */
-	recoveryInProgress = RecoveryInProgress();
+	WALInsertAllowed = XLogInsertAllowed();
 
 	START_CRIT_SECTION();
 
@@ -9925,8 +9935,11 @@ UpdateFullPageWrites(void)
 	 * Write an XLOG_FPW_CHANGE record. This allows us to keep track of
 	 * full_page_writes during archive recovery, if required.
 	 */
-	if (XLogStandbyInfoActive() && !recoveryInProgress)
+	if (XLogStandbyInfoActive() && WALInsertAllowed)
 	{
+		/* Assured that WAL permission has been checked */
+		AssertWALPermitted();
+
 		XLogBeginInsert();
 		XLogRegisterData((char *) (&fullPageWrites), sizeof(bool));
 
diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c
index 1f0e4e01e69..710806143d4 100644
--- a/src/backend/access/transam/xloginsert.c
+++ b/src/backend/access/transam/xloginsert.c
@@ -19,6 +19,7 @@
 
 #include "postgres.h"
 
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xlog_internal.h"
@@ -126,9 +127,14 @@ XLogBeginInsert(void)
 	Assert(mainrdata_last == (XLogRecData *) &mainrdata_head);
 	Assert(mainrdata_len == 0);
 
+	/*
+	 * WAL permission must have checked before entering the critical section.
+	 * Otherwise, WAL prohibited error will force system panic.
+	 */
+	Assert(walpermit_checked_state != WALPERMIT_UNCHECKED || !CritSectionCount);
+
 	/* cross-check on whether we should be here or not */
-	if (!XLogInsertAllowed())
-		elog(ERROR, "cannot make new WAL entries during recovery");
+	CheckWALPermitted();
 
 	if (begininsert_called)
 		elog(ERROR, "XLogBeginInsert was already called");
@@ -210,6 +216,9 @@ XLogResetInsertion(void)
 	mainrdata_last = (XLogRecData *) &mainrdata_head;
 	curinsert_flags = 0;
 	begininsert_called = false;
+
+	/* Reset walpermit_checked flag */
+	RESET_WALPERMIT_CHECKED_STATE();
 }
 
 /*
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 6aab73bfd44..8dacf48db24 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -20,6 +20,7 @@
 #include "access/relation.h"
 #include "access/table.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -378,8 +379,13 @@ fill_seq_with_data(Relation rel, HeapTuple tuple)
 
 	/* check the comment above nextval_internal()'s equivalent call. */
 	if (RelationNeedsWAL(rel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermittedHaveXID();
+	}
+
 	START_CRIT_SECTION();
 
 	MarkBufferDirty(buf);
@@ -766,8 +772,13 @@ nextval_internal(Oid relid, bool check_permissions)
 	 * (Have to do that here, so we're outside the critical section)
 	 */
 	if (logit && RelationNeedsWAL(seqrel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermittedHaveXID();
+	}
+
 	/* ready to change the on-disk (or really, in-buffer) tuple */
 	START_CRIT_SECTION();
 
@@ -977,8 +988,13 @@ do_setval(Oid relid, int64 next, bool iscalled)
 
 	/* check the comment above nextval_internal()'s equivalent call. */
 	if (RelationNeedsWAL(seqrel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermittedHaveXID();
+	}
+
 	/* ready to change the on-disk (or really, in-buffer) tuple */
 	START_CRIT_SECTION();
 
diff --git a/src/backend/commands/variable.c b/src/backend/commands/variable.c
index 484f7ea2c0e..acc34d2ad7c 100644
--- a/src/backend/commands/variable.c
+++ b/src/backend/commands/variable.c
@@ -501,11 +501,14 @@ check_transaction_read_only(bool *newval, void **extra, GucSource source)
 			GUC_check_errmsg("transaction read-write mode must be set before any query");
 			return false;
 		}
-		/* Can't go to r/w mode while recovery is still active */
-		if (RecoveryInProgress())
+		/*
+		 * Can't go to r/w mode while recovery is still active or while in WAL
+		 * prohibit state
+		 */
+		if (!XLogInsertAllowed())
 		{
 			GUC_check_errcode(ERRCODE_FEATURE_NOT_SUPPORTED);
-			GUC_check_errmsg("cannot set transaction read-write mode during recovery");
+			GUC_check_errmsg("cannot set transaction read-write mode while system is read only");
 			return false;
 		}
 	}
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index e2ff484d367..a12e026ae02 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -942,6 +942,10 @@ RequestCheckpoint(int flags)
 	int			old_failed,
 				old_started;
 
+	/* The checkpoint is allowed in recovery but not in WAL prohibit state */
+	if (!RecoveryInProgress())
+		CheckWALPermitted();
+
 	/*
 	 * If in a standalone backend, just do it ourselves.
 	 */
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index a2a963bd5b4..186cc47be1d 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -3638,13 +3638,15 @@ MarkBufferDirtyHint(Buffer buffer, bool buffer_std)
 		{
 			/*
 			 * If we must not write WAL, due to a relfilenode-specific
-			 * condition or being in recovery, don't dirty the page.  We can
-			 * set the hint, just not dirty the page as a result so the hint
-			 * is lost when we evict the page or shutdown.
+			 * condition or in general, don't dirty the page.  We can
+			 * set the hint, but must not dirty the page as a result, lest
+			 * we trigger WAL generation. Unless the page is dirtied again
+			 * later, the hint will be lost when the page is evicted, or at
+			 * shutdown.
 			 *
 			 * See src/backend/storage/page/README for longer discussion.
 			 */
-			if (RecoveryInProgress() ||
+			if (!XLogInsertAllowed() ||
 				RelFileNodeSkippingWAL(bufHdr->tag.rnode))
 				return;
 
diff --git a/src/backend/storage/freespace/freespace.c b/src/backend/storage/freespace/freespace.c
index 6a96126b0c2..b05b0fe5f41 100644
--- a/src/backend/storage/freespace/freespace.c
+++ b/src/backend/storage/freespace/freespace.c
@@ -24,6 +24,7 @@
 #include "postgres.h"
 
 #include "access/htup_details.h"
+#include "access/walprohibit.h"
 #include "access/xlogutils.h"
 #include "miscadmin.h"
 #include "storage/freespace.h"
@@ -285,12 +286,19 @@ FreeSpaceMapPrepareTruncateRel(Relation rel, BlockNumber nblocks)
 	 */
 	if (first_removed_slot > 0)
 	{
+		bool needwal;
+
 		buf = fsm_readbuf(rel, first_removed_address, false);
 		if (!BufferIsValid(buf))
 			return InvalidBlockNumber;	/* nothing to do; the FSM was already
 										 * smaller */
 		LockBuffer(buf, BUFFER_LOCK_EXCLUSIVE);
 
+		needwal = (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded());
+
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -305,7 +313,7 @@ FreeSpaceMapPrepareTruncateRel(Relation rel, BlockNumber nblocks)
 		 * during recovery.
 		 */
 		MarkBufferDirty(buf);
-		if (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded())
+		if (needwal)
 			log_newpage_buffer(buf, false);
 
 		END_CRIT_SECTION();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index d86566f4554..f949a290745 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -794,15 +794,15 @@ LockAcquireExtended(const LOCKTAG *locktag,
 	if (lockmode <= 0 || lockmode > lockMethodTable->numLockModes)
 		elog(ERROR, "unrecognized lock mode: %d", lockmode);
 
-	if (RecoveryInProgress() && !InRecovery &&
+	if (!XLogInsertAllowed() && !InRecovery &&
 		(locktag->locktag_type == LOCKTAG_OBJECT ||
 		 locktag->locktag_type == LOCKTAG_RELATION) &&
 		lockmode > RowExclusiveLock)
 		ereport(ERROR,
 				(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
-				 errmsg("cannot acquire lock mode %s on database objects while recovery is in progress",
+				 errmsg("cannot acquire lock mode %s on database objects while system is read only",
 						lockMethodTable->lockModeNames[lockmode]),
-				 errhint("Only RowExclusiveLock or less can be acquired on database objects during recovery.")));
+				 errhint("Only RowExclusiveLock or less can be acquired on database objects while system is read only")));
 
 #ifdef LOCK_DEBUG
 	if (LOCK_DEBUG_ENABLED(locktag))
diff --git a/src/backend/utils/cache/relmapper.c b/src/backend/utils/cache/relmapper.c
index 671fbb0ed5c..90d7599a57c 100644
--- a/src/backend/utils/cache/relmapper.c
+++ b/src/backend/utils/cache/relmapper.c
@@ -43,6 +43,7 @@
 #include <sys/stat.h>
 #include <unistd.h>
 
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -841,6 +842,8 @@ write_relmap_file(bool shared, RelMapFile *newmap,
 		xl_relmap_update xlrec;
 		XLogRecPtr	lsn;
 
+		AssertWALPermittedHaveXID();
+
 		/* now errors are fatal ... */
 		START_CRIT_SECTION();
 
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e33523984..f3ff120601e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -94,12 +94,37 @@ extern PGDLLIMPORT volatile uint32 CritSectionCount;
 /* in tcop/postgres.c */
 extern void ProcessInterrupts(void);
 
+#ifdef USE_ASSERT_CHECKING
+typedef enum
+{
+	WALPERMIT_UNCHECKED,
+	WALPERMIT_CHECKED,
+	WALPERMIT_CHECKED_AND_USED
+} WALPermitCheckState;
+
+/* in access/walprohibit.c */
+extern WALPermitCheckState walpermit_checked_state;
+
+/*
+ * Reset walpermit_checked flag when no longer in the critical section.
+ * Otherwise, marked checked and used.
+ */
+#define RESET_WALPERMIT_CHECKED_STATE() \
+do { \
+	walpermit_checked_state = CritSectionCount ? \
+	WALPERMIT_CHECKED_AND_USED : WALPERMIT_UNCHECKED; \
+} while(0)
+#else
+#define RESET_WALPERMIT_CHECKED_STATE() ((void) 0)
+#endif
+
 #ifndef WIN32
 
 #define CHECK_FOR_INTERRUPTS() \
 do { \
 	if (unlikely(InterruptPending)) \
 		ProcessInterrupts(); \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 #else							/* WIN32 */
 
@@ -109,6 +134,7 @@ do { \
 		pgwin32_dispatch_queued_signals(); \
 	if (unlikely(InterruptPending)) \
 		ProcessInterrupts(); \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 #endif							/* WIN32 */
 
@@ -135,6 +161,7 @@ do { \
 do { \
 	Assert(CritSectionCount > 0); \
 	CritSectionCount--; \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 
 
-- 
2.22.0

From c0db80b3d85630a026dc722ae0abe7dd51e6ffc8 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 19 Jun 2020 06:29:36 -0400
Subject: [PATCH v8 3/5] Implement ALTER SYSTEM READ ONLY using global barrier.

Implementation:

 1. When a user tried to change server state to WAL-Prohibited using
    ALTER SYSTEM READ ONLY command or by calling
    pg_alter_wal_prohibit_state(true) sql function, the current state
    generation to inprogress in shared memory marked and signaled
    checkpointer process.  Checkpointer, noticing that the current state
    generation has WALPROHIBIT_TRANSITION_IN_PROGRESS flag set, does the
    barrier request, and then acknowledges back to the backend who
    requested the state change once the transition has been completed.
    Final state will be updated in control file to make it persistent
    across the system restarts.

 2. When a backend receives the WAL-Prohibited barrier, at that moment if
    it is already in a transaction and the transaction already assigned XID,
    then the backend will be killed by throwing FATAL(XXX: need more discussion
    on this)

 3. Otherwise, if that backend running transaction without valid XID then, we
    don't need to do anything special right now, simply call
    ResetLocalXLogInsertAllowed() so that any future WAL insert in will check
    XLogInsertAllowed() first which set ready only state appropriately.

 4. A new transaction (in an existing or in a new backend) starts as a
    read-only transaction.

 5. Autovacuum launcher as well as checkpointer will not do anything in
    WAL-Prohibited server state until someone wakes us up.  E.g. a backend
    might later on request us to put the system back to read-write.
 5. The Autovacuum launcher, as well as the checkpointer, will not do
    anything while in the WAL-Prohibited server state until someone wakes
    up.  E.g. user might, later on, request us to put the system back to
    read-write by executing ALTER SYSTEM READ WRITE.

 6. At shutdown in WAL-Prohibited mode, we'll skip shutdown checkpoint
    and xlog rotation. Starting up again will perform crash recovery(XXX:
    need some discussion on this as well) but the end of recovery checkpoint
    will be skipped and it will be performed when the system changed to
    WAL-Permitted mode.

 7. ALTER SYSTEM READ ONLY/WRITE is restricted on standby server.

 8. To execute ALTER SYSTEM READ ONLY/WRITE, the user should have execute
    permssion on pg_alter_wal_prohibit_state() function.

 9. Add system_is_read_only GUC show the system state -- will true when system
    is wal prohibited or in recovery.
---
 src/backend/access/transam/Makefile      |   1 +
 src/backend/access/transam/walprohibit.c | 390 +++++++++++++++++++++++
 src/backend/access/transam/xact.c        |  37 ++-
 src/backend/access/transam/xlog.c        | 117 ++++++-
 src/backend/catalog/system_views.sql     |   2 +
 src/backend/postmaster/autovacuum.c      |   4 +
 src/backend/postmaster/bgwriter.c        |   2 +-
 src/backend/postmaster/checkpointer.c    |  39 +++
 src/backend/postmaster/pgstat.c          |   3 +
 src/backend/storage/ipc/ipci.c           |   6 +
 src/backend/storage/ipc/procsignal.c     |  26 +-
 src/backend/tcop/utility.c               |  15 +-
 src/backend/utils/misc/guc.c             |  26 ++
 src/bin/pg_controldata/pg_controldata.c  |   2 +
 src/include/access/walprohibit.h         |  94 ++++++
 src/include/access/xlog.h                |   4 +
 src/include/catalog/pg_control.h         |   3 +
 src/include/catalog/pg_proc.dat          |   4 +
 src/include/pgstat.h                     |   1 +
 src/include/postmaster/bgwriter.h        |   2 +
 src/include/storage/procsignal.h         |   7 +-
 src/tools/pgindent/typedefs.list         |   1 +
 22 files changed, 717 insertions(+), 69 deletions(-)
 create mode 100644 src/backend/access/transam/walprohibit.c
 create mode 100644 src/include/access/walprohibit.h

diff --git a/src/backend/access/transam/Makefile b/src/backend/access/transam/Makefile
index 595e02de722..b5322a69954 100644
--- a/src/backend/access/transam/Makefile
+++ b/src/backend/access/transam/Makefile
@@ -26,6 +26,7 @@ OBJS = \
 	twophase.o \
 	twophase_rmgr.o \
 	varsup.o \
+	walprohibit.o \
 	xact.o \
 	xlog.o \
 	xlogarchive.o \
diff --git a/src/backend/access/transam/walprohibit.c b/src/backend/access/transam/walprohibit.c
new file mode 100644
index 00000000000..75f3924cc97
--- /dev/null
+++ b/src/backend/access/transam/walprohibit.c
@@ -0,0 +1,390 @@
+/*-------------------------------------------------------------------------
+ *
+ * walprohibit.c
+ * 		PostgreSQL write-ahead log prohibit states
+ *
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/backend/access/transam/walprohibit.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/walprohibit.h"
+#include "fmgr.h"
+#include "pgstat.h"
+#include "port/atomics.h"
+#include "postmaster/bgwriter.h"
+#include "storage/condition_variable.h"
+#include "storage/procsignal.h"
+#include "storage/shmem.h"
+#include "utils/acl.h"
+#include "utils/fmgroids.h"
+#include "utils/fmgrprotos.h"
+
+/*
+ * Shared-memory WAL prohibit state
+ */
+typedef struct WALProhibitStateData
+{
+	/*
+	 * Indicates current WAL prohibit state generation and the last two bits of
+	 * this generation indicates current wal prohibit state.
+	 */
+	pg_atomic_uint32 shared_state_generation;
+
+	/* Signaled when requested WAL prohibit state changes */
+	ConditionVariable walprohibit_cv;
+} WALProhibitStateData;
+
+static WALProhibitStateData *WALProhibitState = NULL;
+
+static void RequestWALProhibitChange(uint32 cur_state_gen);
+
+/*
+ * ProcessBarrierWALProhibit()
+ *
+ * Handle WAL prohibit state change request.
+ */
+bool
+ProcessBarrierWALProhibit(void)
+{
+	/*
+	 * Kill off any transactions that have an XID *before* allowing the system
+	 * to go WAL prohibit state.
+	 */
+	if (FullTransactionIdIsValid(GetTopFullTransactionIdIfAny()))
+	{
+		/*
+		 * Should be here only while transiting towards the WAL prohibit state.
+		 */
+		Assert(WALPROHIBIT_CURRENT_STATE(GetWALProhibitStateGen()) ==
+			   WALPROHIBIT_STATE_GOING_READ_ONLY);
+
+		/*
+		 * XXX: Kill off the whole session by throwing FATAL instead of
+		 * killing transaction by throwing ERROR due to following reasons that
+		 * need be thought:
+		 *
+		 * 1. Due to some presents challenges with the wire protocol, we could
+		 * not simply kill of idle transaction.
+		 *
+		 * 2. If we are here in subtransaction then the ERROR will kill the
+		 * current subtransaction only. In the case of invalidations, that
+		 * might be good enough, but for XID assignment it's not, because
+		 * assigning an XID to a subtransaction also causes higher
+		 * sub-transaction levels and the parent transaction to get XIDs.
+		 */
+		ereport(FATAL,
+				(errcode(ERRCODE_ACTIVE_SQL_TRANSACTION),
+				 errmsg("system is now read only"),
+				 errhint("Sessions with open write transactions must be terminated.")));
+	}
+
+	/* Return to "check" state */
+	ResetLocalXLogInsertAllowed();
+
+	return true;
+}
+
+/*
+ * AlterSystemSetWALProhibitState()
+ *
+ * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
+ */
+void
+AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
+{
+	/* Check permission for pg_alter_wal_prohibit_state() */
+	if (pg_proc_aclcheck(F_PG_ALTER_WAL_PROHIBIT_STATE,
+						 GetUserId(), ACL_EXECUTE) != ACLCHECK_OK)
+		ereport(ERROR,
+				(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+				 errmsg("permission denied for command ALTER SYSTEM"),
+				 errhint("Get execute permission for pg_alter_wal_prohibit_state() to this user.")));
+
+	/* Alter WAL prohibit state not allowed during recovery */
+	PreventCommandDuringRecovery("ALTER SYSTEM");
+
+	/* Execute function to alter wal prohibit state */
+	(void) OidFunctionCall1(F_PG_ALTER_WAL_PROHIBIT_STATE,
+							BoolGetDatum(stmt->walprohibited));
+}
+
+/*
+ * pg_alter_wal_prohibit_state()
+ *
+ * SQL callable function to alter system read write state.
+ */
+Datum
+pg_alter_wal_prohibit_state(PG_FUNCTION_ARGS)
+{
+	bool		walprohibited = PG_GETARG_BOOL(0);
+	uint32		cur_state_gen;
+
+	/* Alter WAL prohibit state not allowed during recovery */
+	PreventCommandDuringRecovery("pg_alter_wal_prohibit_state()");
+
+	/*
+	 * It is not a final state since we yet to convey this WAL prohibit state to
+	 * all backend.
+	 */
+	cur_state_gen = SetWALProhibitState(walprohibited, false);
+
+	/* Server is already in requested state */
+	if (!cur_state_gen)
+		PG_RETURN_VOID();
+
+	/*
+	 * Signal the checkpointer to do the actual state transition, and wait for
+	 * the state change to occur.
+	 */
+	RequestWALProhibitChange(cur_state_gen);
+
+	PG_RETURN_VOID();
+}
+
+/*
+ * RequestWALProhibitChange()
+ *
+ * Request checkpointer to make the WALProhibitState to read-only.
+ */
+static void
+RequestWALProhibitChange(uint32 cur_state_gen)
+{
+	/* Must not be called from checkpointer */
+	Assert(!AmCheckpointerProcess());
+	Assert(GetWALProhibitStateGen() & WALPROHIBIT_TRANSITION_IN_PROGRESS);
+
+	/*
+	 * If in a standalone backend, just do it ourselves.
+	 */
+	if (!IsPostmasterEnvironment)
+	{
+		CompleteWALProhibitChange(cur_state_gen);
+		return;
+	}
+
+	/* Signal checkpointer process */
+	SendsSignalToCheckpointer(SIGINT);
+
+	/* Wait for the state to change to read-only */
+	ConditionVariablePrepareToSleep(&WALProhibitState->walprohibit_cv);
+	for (;;)
+	{
+		/* We'll be done once wal prohibit state generation changes */
+		if (GetWALProhibitStateGen() != cur_state_gen)
+			break;
+
+		ConditionVariableSleep(&WALProhibitState->walprohibit_cv,
+							   WAIT_EVENT_WALPROHIBIT_STATE_CHANGE);
+	}
+	ConditionVariableCancelSleep();
+}
+
+/*
+ * CompleteWALProhibitChange()
+ *
+ * Checkpointer will call this to complete the requested WAL prohibit state
+ * transition.
+ */
+void
+CompleteWALProhibitChange(uint32 cur_state_gen)
+{
+	uint64		barrier_gen;
+	bool		wal_prohibited;
+
+	/*
+	 * Must be called from checkpointer. Otherwise, it must be single-user
+	 * backend.
+	 */
+	Assert(AmCheckpointerProcess() || !IsPostmasterEnvironment);
+	Assert(cur_state_gen & WALPROHIBIT_TRANSITION_IN_PROGRESS);
+
+	/*
+	 * WAL prohibit state change is initiated. We need to complete the state
+	 * transition by setting requested WAL prohibit state in all backends.
+	 */
+	elog(DEBUG1, "waiting for backends to adopt requested WAL prohibit state change");
+
+	/* Emit global barrier */
+	barrier_gen = EmitProcSignalBarrier(PROCSIGNAL_BARRIER_WALPROHIBIT);
+	WaitForProcSignalBarrier(barrier_gen);
+
+	/* And flush all inserts. */
+	XLogFlush(GetXLogInsertRecPtr());
+
+	wal_prohibited =
+		(WALPROHIBIT_NEXT_STATE(cur_state_gen) == WALPROHIBIT_STATE_READ_ONLY);
+
+	/* Set the final state */
+	(void) SetWALProhibitState(wal_prohibited, true);
+
+	/* Update the control file to make state persistent */
+	SetControlFileWALProhibitFlag(wal_prohibited);
+
+	if (wal_prohibited)
+		ereport(LOG, (errmsg("system is now read only")));
+	else
+	{
+		/*
+		 * Request checkpoint if the end-of-recovery checkpoint has been skipped
+		 * previously.
+		 */
+		if (LastCheckPointIsSkipped())
+		{
+			RequestCheckpoint(CHECKPOINT_IMMEDIATE);
+			SetLastCheckPointSkipped(false);
+		}
+		ereport(LOG, (errmsg("system is now read write")));
+	}
+
+	/* Wake up all backends waiting on this. */
+	ConditionVariableBroadcast(&WALProhibitState->walprohibit_cv);
+}
+
+/*
+ * GetWALProhibitStateGen()
+ *
+ * Atomically return the current server WAL prohibited state generation.
+ */
+uint32
+GetWALProhibitStateGen(void)
+{
+	return pg_atomic_read_u32(&WALProhibitState->shared_state_generation);
+}
+
+/*
+ * SetWALProhibitState()
+ *
+ * Increments current shared WAL prohibit state generation concerning to
+ * requested state and returns the same.
+ *
+ * For the transition state request where is_final_state is false if the server
+ * desired transition state is the same as the current state which might have
+ * been requested by some other backend and has been proceeded then the current
+ * wal prohibit generation will be returned so that this backend can wait until
+ * the shared wal prohibited generation change for the final state.  And, if the
+ * server is already completely moved to the requested state then the requester
+ * backend doesn't need to wait, in that case, 0 will be returned.
+ *
+ * For the final state request which can be only requested by the checkpointer
+ * or by the single-user so that there is no chance that the server already is
+ * in the desired final state.
+ */
+uint32
+SetWALProhibitState(bool wal_prohibited, bool is_final_state)
+{
+	uint32		new_state;
+	uint32		cur_state;
+	uint32		cur_state_gen;
+	uint32		next_state_gen;
+
+	/* Get the current state */
+	cur_state_gen = GetWALProhibitStateGen();
+	cur_state = WALPROHIBIT_CURRENT_STATE(cur_state_gen);
+
+	/* Compute new state */
+	if (is_final_state)
+	{
+		/*
+		 * Only checkpointer or single-user can set the final wal prohibit
+		 * state.
+		 */
+		Assert(AmCheckpointerProcess() || !IsPostmasterEnvironment);
+
+		new_state = wal_prohibited ?
+			WALPROHIBIT_STATE_READ_ONLY : WALPROHIBIT_STATE_READ_WRITE;
+
+		/*
+		 * There won't be any other process for the final state setting so that
+		 * the next final state will be the desired state.
+		 */
+		Assert(WALPROHIBIT_NEXT_STATE(cur_state) == new_state);
+	}
+	else
+	{
+		new_state = wal_prohibited ?
+			WALPROHIBIT_STATE_GOING_READ_ONLY :
+			WALPROHIBIT_STATE_GOING_READ_WRITE;
+
+		/* Server is already in the requested transition state */
+		if (cur_state == new_state)
+			return cur_state;		/* Wait for state transition completion */
+
+		/* Server is already in requested state */
+		if (WALPROHIBIT_NEXT_STATE(new_state) == cur_state)
+			return 0;		/* No wait is needed */
+
+		/* Prevent concurrent contrary in progress transition state setting */
+		if (cur_state & WALPROHIBIT_TRANSITION_IN_PROGRESS)
+		{
+			if (cur_state & WALPROHIBIT_STATE_READ_ONLY)
+				ereport(ERROR,
+						(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+						 errmsg("system state transition to read only is already in progress"),
+						 errhint("Try after sometime again.")));
+			else
+				ereport(ERROR,
+						(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+						 errmsg("system state transition to read write is already in progress"),
+						 errhint("Try after sometime again.")));
+		}
+	}
+
+	/*
+	 * Update new state generation in share memory only if the state generation
+	 * hasn't changed until now we have checked.
+	 */
+	next_state_gen = cur_state_gen + 1;
+	(void) pg_atomic_compare_exchange_u32(&WALProhibitState->shared_state_generation,
+										  &cur_state_gen, next_state_gen);
+
+	/* To be sure that any later reads of memory happen strictly after this. */
+	pg_memory_barrier();
+
+	return next_state_gen;
+}
+
+/*
+ * WALProhibitStateGenerationInit()
+ *
+ * Initialization of shared wal prohibit state generation.
+ */
+void
+WALProhibitStateGenerationInit(bool wal_prohibited)
+{
+	uint32	new_state;
+
+	Assert(AmStartupProcess() || !IsPostmasterEnvironment);
+
+	new_state = wal_prohibited ?
+		WALPROHIBIT_STATE_READ_ONLY : WALPROHIBIT_STATE_READ_WRITE;
+
+	pg_atomic_init_u32(&WALProhibitState->shared_state_generation, new_state);
+}
+
+/*
+ * WALProhibitStateShmemInit()
+ *
+ * Initialization of shared memory for WAL prohibit state.
+ */
+void
+WALProhibitStateShmemInit(void)
+{
+	bool		found;
+
+	WALProhibitState = (WALProhibitStateData *)
+		ShmemInitStruct("WAL Prohibit State",
+						sizeof(WALProhibitStateData),
+						&found);
+
+	if (found)
+		return;
+
+	/* First time through ... */
+	memset(WALProhibitState, 0, sizeof(WALProhibitStateData));
+	ConditionVariableInit(&WALProhibitState->walprohibit_cv);
+}
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index af6afcebb13..188c299bed9 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -1962,23 +1962,28 @@ StartTransaction(void)
 	Assert(s->prevSecContext == 0);
 
 	/*
-	 * Make sure we've reset xact state variables
+	 * Reset xact state variables.
 	 *
-	 * If recovery is still in progress, mark this transaction as read-only.
-	 * We have lower level defences in XLogInsert and elsewhere to stop us
-	 * from modifying data during recovery, but this gives the normal
-	 * indication to the user that the transaction is read-only.
-	 */
-	if (RecoveryInProgress())
-	{
-		s->startedInRecovery = true;
-		XactReadOnly = true;
-	}
-	else
-	{
-		s->startedInRecovery = false;
-		XactReadOnly = DefaultXactReadOnly;
-	}
+	 * If it is not currently possible to insert write-ahead log records,
+	 * either because we are still in recovery or because ALTER SYSTEM READ
+	 * ONLY has been executed, force this to be a read-only transaction.
+	 * We have lower level defences in XLogBeginInsert() and elsewhere to stop
+	 * us from modifying data during recovery when !XLogInsertAllowed(), but
+	 * this gives the normal indication to the user that the transaction is
+	 * read-only.
+	 *
+	 * On the other hand, we only need to set the startedInRecovery flag when
+	 * the transaction started during recovery, and not when WAL is otherwise
+	 * prohibited. This information is used by RelationGetIndexScan() to
+	 * decide whether to permit (1) relying on existing killed-tuple markings
+	 * and (2) further killing of index tuples. Even when WAL is prohibited
+	 * on the master, it's still the master, so the former is OK; and since
+	 * killing index tuples doesn't generate WAL, the latter is also OK.
+	 * See comments in RelationGetIndexScan() and MarkBufferDirtyHint().
+	 */
+	XactReadOnly = DefaultXactReadOnly || !XLogInsertAllowed();
+	s->startedInRecovery = RecoveryInProgress();
+
 	XactDeferrable = DefaultXactDeferrable;
 	XactIsoLevel = DefaultXactIsoLevel;
 	forceSyncCommit = false;
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 61754312e26..bfba75bbe80 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -31,6 +31,7 @@
 #include "access/timeline.h"
 #include "access/transam.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog_internal.h"
 #include "access/xlogarchive.h"
@@ -246,9 +247,10 @@ static bool LocalPromoteIsTriggered = false;
  *		0: unconditionally not allowed to insert XLOG
  *		-1: must check RecoveryInProgress(); disallow until it is false
  * Most processes start with -1 and transition to 1 after seeing that recovery
- * is not in progress.  But we can also force the value for special cases.
- * The coding in XLogInsertAllowed() depends on the first two of these states
- * being numerically the same as bool true and false.
+ * is not in progress or the server state is not a WAL prohibited state.  But
+ * we can also force the value for special cases.  The coding in
+ * XLogInsertAllowed() depends on the first two of these states being
+ * numerically the same as bool true and false.
  */
 static int	LocalXLogInsertAllowed = -1;
 
@@ -723,6 +725,11 @@ typedef struct XLogCtlData
 	 */
 	XLogRecPtr	lastFpwDisableRecPtr;
 
+	/*
+	 * lastCheckPointSkipped indicates if the last checkpoint is skipped.
+	 */
+	bool		lastCheckPointSkipped;
+
 	slock_t		info_lck;		/* locks shared variables shown above */
 } XLogCtlData;
 
@@ -968,6 +975,7 @@ static void WALInsertLockAcquire(void);
 static void WALInsertLockAcquireExclusive(void);
 static void WALInsertLockRelease(void);
 static void WALInsertLockUpdateInsertingAt(XLogRecPtr insertingAt);
+static inline bool IsWALProhibited(void);
 
 /*
  * Insert an XLOG record represented by an already-constructed chain of data
@@ -6196,6 +6204,32 @@ SetCurrentChunkStartTime(TimestampTz xtime)
 	SpinLockRelease(&XLogCtl->info_lck);
 }
 
+/*
+ * Set or unset flag to indicating that the last checkpoint has been skipped.
+ */
+void
+SetLastCheckPointSkipped(bool ChkptSkip)
+{
+	SpinLockAcquire(&XLogCtl->info_lck);
+	XLogCtl->lastCheckPointSkipped = ChkptSkip;
+	SpinLockRelease(&XLogCtl->info_lck);
+}
+
+/*
+ * Return value of lastCheckPointSkipped flag.
+ */
+bool
+LastCheckPointIsSkipped(void)
+{
+	bool	ChkptSkipped;
+
+	SpinLockAcquire(&XLogCtl->info_lck);
+	ChkptSkipped = XLogCtl->lastCheckPointSkipped;
+	SpinLockRelease(&XLogCtl->info_lck);
+
+	return ChkptSkipped;
+}
+
 /*
  * Fetch timestamp of latest processed commit/abort record.
  * Startup process maintains an accurate local copy in XLogReceiptTime
@@ -7708,6 +7742,12 @@ StartupXLOG(void)
 	XLogCtl->LogwrtRqst.Write = EndOfLog;
 	XLogCtl->LogwrtRqst.Flush = EndOfLog;
 
+	/*
+	 * Before enabling WAL insertion, initialize WAL prohibit state in shared
+	 * memory that will decide the further WAL insert should be allowed or not.
+	 */
+	WALProhibitStateGenerationInit(ControlFile->wal_prohibited);
+
 	/*
 	 * Update full_page_writes in shared memory and write an XLOG_FPW_CHANGE
 	 * record before resource manager writes cleanup WAL records or checkpoint
@@ -7718,7 +7758,17 @@ StartupXLOG(void)
 	UpdateFullPageWrites();
 	LocalXLogInsertAllowed = -1;
 
-	if (InRecovery)
+	/*
+	 * Skip end-of-recovery checkpoint if the system is in WAL prohibited state.
+	 */
+	if (ControlFile->wal_prohibited && InRecovery)
+	{
+		SetLastCheckPointSkipped(true);
+
+		ereport(LOG,
+				(errmsg("skipping startup checkpoint because the system is read only")));
+	}
+	else if (InRecovery)
 	{
 		/*
 		 * Perform a checkpoint to update all our recovery activity to disk.
@@ -7964,6 +8014,28 @@ StartupXLOG(void)
 		RequestCheckpoint(CHECKPOINT_FORCE);
 }
 
+/* Set ControlFile's WAL prohibit flag */
+void
+SetControlFileWALProhibitFlag(bool wal_prohibited)
+{
+	LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+	ControlFile->wal_prohibited = wal_prohibited;
+	UpdateControlFile();
+	LWLockRelease(ControlFileLock);
+}
+
+/*
+ * Is the system still in WAL prohibited state?
+ */
+static inline bool
+IsWALProhibited(void)
+{
+	uint32 		cur_state = WALPROHIBIT_CURRENT_STATE(GetWALProhibitStateGen());
+
+	return (cur_state != WALPROHIBIT_STATE_READ_WRITE &&
+			cur_state != WALPROHIBIT_STATE_GOING_READ_WRITE);
+}
+
 /*
  * Checks if recovery has reached a consistent state. When consistency is
  * reached and we have a valid starting standby snapshot, tell postmaster
@@ -8179,9 +8251,9 @@ HotStandbyActiveInReplay(void)
 /*
  * Is this process allowed to insert new WAL records?
  *
- * Ordinarily this is essentially equivalent to !RecoveryInProgress().
- * But we also have provisions for forcing the result "true" or "false"
- * within specific processes regardless of the global state.
+ * Ordinarily this is essentially equivalent to !RecoveryInProgress() and
+ * !IsWALProhibited().  But we also have provisions for forcing the result
+ * "true" or "false" within specific processes regardless of the global state.
  */
 bool
 XLogInsertAllowed(void)
@@ -8200,9 +8272,20 @@ XLogInsertAllowed(void)
 	if (RecoveryInProgress())
 		return false;
 
+	/* Or, in WAL prohibited state */
+	if (IsWALProhibited())
+	{
+		/*
+		 * Set it to "unconditionally false" to avoid checking until it gets
+		 * reset.
+		 */
+		LocalXLogInsertAllowed = 0;
+		return false;
+	}
+
 	/*
-	 * On exit from recovery, reset to "unconditionally true", since there is
-	 * no need to keep checking.
+	 * On exit from recovery or WAL prohibited state, reset to "unconditionally
+	 * true", since there is no need to keep checking.
 	 */
 	LocalXLogInsertAllowed = 1;
 	return true;
@@ -8224,6 +8307,12 @@ LocalSetXLogInsertAllowed(void)
 	InitXLOGAccess();
 }
 
+void
+ResetLocalXLogInsertAllowed(void)
+{
+	LocalXLogInsertAllowed = -1;
+}
+
 /*
  * Subroutine to try to fetch and validate a prior checkpoint record.
  *
@@ -8513,9 +8602,13 @@ ShutdownXLOG(int code, Datum arg)
 	 */
 	WalSndWaitStopping();
 
+	/*
+	 * The restartpoint, checkpoint, or xlog rotation will be performed if the
+	 * WAL writing is permitted.
+	 */
 	if (RecoveryInProgress())
 		CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
-	else
+	else if (XLogInsertAllowed())
 	{
 		/*
 		 * If archiving is enabled, rotate the last XLOG file so that all the
@@ -8528,6 +8621,10 @@ ShutdownXLOG(int code, Datum arg)
 
 		CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
 	}
+	else
+		ereport(LOG,
+			   (errmsg("skipping shutdown checkpoint because the system is read only")));
+
 	ShutdownCLOG();
 	ShutdownCommitTs();
 	ShutdownSUBTRANS();
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index ed4f3f142d8..79da249dd5c 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1485,6 +1485,8 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
 REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
 REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
 
+REVOKE EXECUTE ON FUNCTION pg_alter_wal_prohibit_state(bool) FROM public;
+
 --
 -- We also set up some things as accessible to standard roles.
 --
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 2cef56f115f..efee35cbc94 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -659,6 +659,10 @@ AutoVacLauncherMain(int argc, char *argv[])
 
 		HandleAutoVacLauncherInterrupts();
 
+		/* If the server is read only just go back to sleep. */
+		if (!XLogInsertAllowed())
+			continue;
+
 		/*
 		 * a worker finished, or postmaster signaled failure to start a worker
 		 */
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index a7afa758b61..1d9c46de20a 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -278,7 +278,7 @@ BackgroundWriterMain(void)
 		 * Checkpointer, when active, is barely ever in its mainloop and thus
 		 * makes it hard to log regularly.
 		 */
-		if (XLogStandbyInfoActive() && !RecoveryInProgress())
+		if (XLogStandbyInfoActive() && XLogInsertAllowed())
 		{
 			TimestampTz timeout = 0;
 			TimestampTz now = GetCurrentTimestamp();
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 3e7dcd4f764..e2ff484d367 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -39,6 +39,7 @@
 #include <sys/time.h>
 #include <time.h>
 
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xlog_internal.h"
 #include "libpq/pqsignal.h"
@@ -342,6 +343,7 @@ CheckpointerMain(void)
 		pg_time_t	now;
 		int			elapsed_secs;
 		int			cur_timeout;
+		uint32		wal_state_gen;
 
 		/* Clear any already-pending wakeups */
 		ResetLatch(MyLatch);
@@ -352,6 +354,30 @@ CheckpointerMain(void)
 		AbsorbSyncRequests();
 		HandleCheckpointerInterrupts();
 
+		wal_state_gen = GetWALProhibitStateGen();
+
+		if (wal_state_gen & WALPROHIBIT_TRANSITION_IN_PROGRESS)
+		{
+			/* Complete WAL prohibit state change request */
+			CompleteWALProhibitChange(wal_state_gen);
+			continue;
+		}
+		else if (WALPROHIBIT_CURRENT_STATE(wal_state_gen) ==
+				 WALPROHIBIT_STATE_READ_ONLY)
+		{
+			/*
+			 * Don't do anything until someone wakes us up.  For example a
+			 * backend might later on request us to put the system back to
+			 * read-write wal prohibit sate.
+			 */
+			(void) WaitLatch(MyLatch, WL_LATCH_SET | WL_EXIT_ON_PM_DEATH, -1,
+							 WAIT_EVENT_CHECKPOINTER_MAIN);
+			continue;
+		}
+
+		Assert(WALPROHIBIT_CURRENT_STATE(wal_state_gen) ==
+			   WALPROHIBIT_STATE_READ_WRITE);
+
 		/*
 		 * Detect a pending checkpoint request by checking whether the flags
 		 * word in shared memory is nonzero.  We shouldn't need to acquire the
@@ -1333,3 +1359,16 @@ FirstCallSinceLastCheckpoint(void)
 
 	return FirstCall;
 }
+
+/*
+ * SendsSignalToCheckpointer allows a process to send a signal to the checkpoint process.
+ */
+void
+SendsSignalToCheckpointer(int signum)
+{
+	if (CheckpointerShmem->checkpointer_pid == 0)
+		elog(ERROR, "checkpointer is not running");
+
+	if (kill(CheckpointerShmem->checkpointer_pid, signum) != 0)
+		elog(ERROR, "could not signal checkpointer: %m");
+}
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index e6be2b7836a..95a738d7f25 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -4060,6 +4060,9 @@ pgstat_get_wait_io(WaitEventIO w)
 		case WAIT_EVENT_REPLICATION_SLOT_WRITE:
 			event_name = "ReplicationSlotWrite";
 			break;
+		case WAIT_EVENT_WALPROHIBIT_STATE_CHANGE:
+			event_name = "SystemWALProhibitStateChange";
+			break;
 		case WAIT_EVENT_SLRU_FLUSH_SYNC:
 			event_name = "SLRUFlushSync";
 			break;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 96c2aaabbd6..2d000ec2ff7 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -22,6 +22,7 @@
 #include "access/subtrans.h"
 #include "access/syncscan.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "commands/async.h"
 #include "miscadmin.h"
 #include "pgstat.h"
@@ -222,6 +223,11 @@ CreateSharedMemoryAndSemaphores(void)
 	MultiXactShmemInit();
 	InitBufferPool();
 
+	/*
+	 * Set up wal probibit shared state
+	 */
+	WALProhibitStateShmemInit();
+
 	/*
 	 * Set up lock manager
 	 */
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 63cb70bcaa4..4f1b67f9d04 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -18,6 +18,7 @@
 #include <unistd.h>
 
 #include "access/parallel.h"
+#include "access/walprohibit.h"
 #include "commands/async.h"
 #include "miscadmin.h"
 #include "pgstat.h"
@@ -96,7 +97,6 @@ static volatile ProcSignalSlot *MyProcSignalSlot = NULL;
 
 static bool CheckProcSignal(ProcSignalReason reason);
 static void CleanupProcSignalState(int status, Datum arg);
-static bool ProcessBarrierPlaceholder(void);
 
 /*
  * ProcSignalShmemSize
@@ -510,9 +510,9 @@ ProcessProcSignalBarrier(void)
 			 * unconditionally, but it's more efficient to call only the ones
 			 * that might need us to do something based on the flags.
 			 */
-			if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_PLACEHOLDER)
-				&& ProcessBarrierPlaceholder())
-				BARRIER_CLEAR_BIT(flags, PROCSIGNAL_BARRIER_PLACEHOLDER);
+			if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_WALPROHIBIT)
+				&& ProcessBarrierWALProhibit())
+				BARRIER_CLEAR_BIT(flags, PROCSIGNAL_BARRIER_WALPROHIBIT);
 		}
 		PG_CATCH();
 		{
@@ -554,24 +554,6 @@ ProcessProcSignalBarrier(void)
 	pg_atomic_write_u64(&MyProcSignalSlot->pss_barrierGeneration, shared_gen);
 }
 
-static bool
-ProcessBarrierPlaceholder(void)
-{
-	/*
-	 * XXX. This is just a placeholder until the first real user of this
-	 * machinery gets committed. Rename PROCSIGNAL_BARRIER_PLACEHOLDER to
-	 * PROCSIGNAL_BARRIER_SOMETHING_ELSE where SOMETHING_ELSE is something
-	 * appropriately descriptive. Get rid of this function and instead have
-	 * ProcessBarrierSomethingElse. Most likely, that function should live in
-	 * the file pertaining to that subsystem, rather than here.
-	 *
-	 * The return value should be 'true' if the barrier was successfully
-	 * absorbed and 'false' if not. Note that returning 'false' can lead to
-	 * very frequent retries, so try hard to make that an uncommon case.
-	 */
-	return true;
-}
-
 /*
  * CheckProcSignal - check to see if a particular reason has been
  * signaled, and clear the signal flag.  Should be called after receiving
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 74c2162cd59..05eac206182 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -19,6 +19,7 @@
 #include "access/htup_details.h"
 #include "access/reloptions.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "catalog/catalog.h"
@@ -85,7 +86,6 @@ static void ProcessUtilitySlow(ParseState *pstate,
 							   DestReceiver *dest,
 							   QueryCompletion *qc);
 static void ExecDropStmt(DropStmt *stmt, bool isTopLevel);
-static void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
 
 /*
  * CommandIsReadOnly: is an executable query read-only?
@@ -3691,16 +3691,3 @@ GetCommandLogLevel(Node *parsetree)
 
 	return lev;
 }
-
-/*
- * AlterSystemSetWALProhibitState
- *
- * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
- */
-static void
-AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
-{
-	ereport(ERROR,
-			(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
-			 errmsg("ALTER SYSTEM READ { ONLY | WRITE } not implemented")));
-}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 596bcb7b842..24113249f67 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -225,6 +225,7 @@ static bool check_recovery_target_lsn(char **newval, void **extra, GucSource sou
 static void assign_recovery_target_lsn(const char *newval, void *extra);
 static bool check_primary_slot_name(char **newval, void **extra, GucSource source);
 static bool check_default_with_oids(bool *newval, void **extra, GucSource source);
+static const char *show_system_is_read_only(void);
 
 /* Private functions in guc-file.l that need to be called from guc.c */
 static ConfigVariable *ProcessConfigFileInternal(GucContext context,
@@ -615,6 +616,7 @@ static char *recovery_target_string;
 static char *recovery_target_xid_string;
 static char *recovery_target_name_string;
 static char *recovery_target_lsn_string;
+static bool system_is_read_only;
 
 
 /* should be static, but commands/variable.c needs to get at this */
@@ -2036,6 +2038,18 @@ static struct config_bool ConfigureNamesBool[] =
 		NULL, NULL, NULL
 	},
 
+	{
+		/* Not for general use */
+		{"system_is_read_only", PGC_INTERNAL, WAL,
+			gettext_noop("Shows whether the system is read only."),
+			NULL,
+			GUC_NO_RESET_ALL | GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE
+		},
+		&system_is_read_only,
+		false,
+		NULL, NULL, show_system_is_read_only
+	},
+
 	/* End-of-list marker */
 	{
 		{NULL, 0, 0, NULL, NULL}, NULL, false, NULL, NULL, NULL
@@ -12041,4 +12055,16 @@ check_default_with_oids(bool *newval, void **extra, GucSource source)
 	return true;
 }
 
+/*
+ * NB: The return string should be the same as the _ShowOption() for boolean
+ * type.
+ */
+static const char *
+show_system_is_read_only(void)
+{
+	if (!XLogInsertAllowed())
+		return "on";
+	return "off";
+}
+
 #include "guc-file.c"
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index 3e00ac0f701..922cd9641d8 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -290,6 +290,8 @@ main(int argc, char *argv[])
 		   (uint32) ControlFile->backupEndPoint);
 	printf(_("End-of-backup record required:        %s\n"),
 		   ControlFile->backupEndRequired ? _("yes") : _("no"));
+	printf(_("WAL write prohibited:                 %s\n"),
+		   ControlFile->wal_prohibited ? _("yes") : _("no"));
 	printf(_("wal_level setting:                    %s\n"),
 		   wal_level_str(ControlFile->wal_level));
 	printf(_("wal_log_hints setting:                %s\n"),
diff --git a/src/include/access/walprohibit.h b/src/include/access/walprohibit.h
new file mode 100644
index 00000000000..61836d61844
--- /dev/null
+++ b/src/include/access/walprohibit.h
@@ -0,0 +1,94 @@
+/*
+ * walprohibit.h
+ *
+ * PostgreSQL write-ahead log prohibit states
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/include/access/walprohibit.h
+ */
+#ifndef WALPROHIBIT_H
+#define WALPROHIBIT_H
+
+#include "access/xact.h"
+#include "access/xlog.h"
+#include "miscadmin.h"
+#include "nodes/parsenodes.h"
+
+extern bool ProcessBarrierWALProhibit(void);
+extern void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
+extern void CompleteWALProhibitChange(uint32 wal_state);
+extern uint32 GetWALProhibitStateGen(void);
+extern uint32 SetWALProhibitState(bool wal_prohibited, bool is_final_state);
+extern void MarkCheckPointSkippedInWalProhibitState(void);
+extern void WALProhibitStateGenerationInit(bool wal_prohibited);
+extern void WALProhibitStateShmemInit(void);
+
+/*
+ * The WAL Prohibit States.
+ *
+ * 	The odd number represents the transition state and whereas the even number
+ * 	represents the final state.  These states can be distinguished by checking
+ * 	the 0th bits aka transition bit.
+ */
+#define	WALPROHIBIT_STATE_READ_WRITE		(uint32) 0	/* WAL permitted */
+#define	WALPROHIBIT_STATE_GOING_READ_ONLY	(uint32) 1
+#define	WALPROHIBIT_STATE_READ_ONLY			(uint32) 2	/* WAL prohibited */
+#define	WALPROHIBIT_STATE_GOING_READ_WRITE	(uint32) 3
+
+/* The transition bit to distinguish states.  */
+#define	WALPROHIBIT_TRANSITION_IN_PROGRESS	((uint32) 1 << 0)
+
+/* Extract last two bits */
+#define	WALPROHIBIT_CURRENT_STATE(stateGeneration)	\
+	((uint32)(stateGeneration) & ((uint32) ((1 << 2) - 1)))
+#define	WALPROHIBIT_NEXT_STATE(stateGeneration)	\
+	WALPROHIBIT_CURRENT_STATE((stateGeneration + 1))
+
+/* Never reaches when WAL is prohibited. */
+static inline void
+AssertWALPermitted(void)
+{
+	/*
+	 * Recovery in the startup process never is in wal prohibited state.
+	 */
+	Assert(InRecovery || XLogInsertAllowed());
+
+#ifdef USE_ASSERT_CHECKING
+	walpermit_checked_state = WALPERMIT_CHECKED;
+#endif
+}
+
+/*
+ * XID-bearing transactions are killed off by "ALTER SYSTEM READ ONLY", so any
+ * part of the code that can only be reached with an XID assigned is never
+ * reached when WAL is prohibited.
+ */
+static inline void
+AssertWALPermittedHaveXID(void)
+{
+	/* Must be performing an INSERT, UPDATE or DELETE, so we'll have an XID */
+	Assert(FullTransactionIdIsValid(GetTopFullTransactionIdIfAny()));
+	AssertWALPermitted();
+}
+
+/*
+ * In opposite to the above assertion if a transaction doesn't have valid XID
+ * (e.g. VACUUM) then it won't be killed while changing the system state to WAL
+ * prohibited.  Therefore, we need to explicitly error out before entering into
+ * the critical section.
+ */
+static inline void
+CheckWALPermitted(void)
+{
+	if (!XLogInsertAllowed())
+		ereport(ERROR,
+				(errcode(ERRCODE_READ_ONLY_SQL_TRANSACTION),
+				 errmsg("system is now read only")));
+
+#ifdef USE_ASSERT_CHECKING
+	walpermit_checked_state = WALPERMIT_CHECKED;
+#endif
+}
+
+#endif							/* WALPROHIBIT_H */
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 221af87e715..2bcd37894f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -306,6 +306,7 @@ extern RecoveryState GetRecoveryState(void);
 extern bool HotStandbyActive(void);
 extern bool HotStandbyActiveInReplay(void);
 extern bool XLogInsertAllowed(void);
+extern void ResetLocalXLogInsertAllowed(void);
 extern void GetXLogReceiptTime(TimestampTz *rtime, bool *fromStream);
 extern XLogRecPtr GetXLogReplayRecPtr(TimeLineID *replayTLI);
 extern XLogRecPtr GetXLogInsertRecPtr(void);
@@ -314,6 +315,8 @@ extern bool RecoveryIsPaused(void);
 extern void SetRecoveryPause(bool recoveryPause);
 extern TimestampTz GetLatestXTime(void);
 extern TimestampTz GetCurrentChunkReplayStartTime(void);
+extern void SetLastCheckPointSkipped(bool ChkptSkip);
+extern bool LastCheckPointIsSkipped(void);
 
 extern void UpdateControlFile(void);
 extern uint64 GetSystemIdentifier(void);
@@ -325,6 +328,7 @@ extern void XLOGShmemInit(void);
 extern void BootStrapXLOG(void);
 extern void LocalProcessControlFile(bool reset);
 extern void StartupXLOG(void);
+extern void SetControlFileWALProhibitFlag(bool wal_prohibited);
 extern void ShutdownXLOG(int code, Datum arg);
 extern void InitXLOGAccess(void);
 extern void CreateCheckPoint(int flags);
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index 06bed90c5e9..f4dc5412ee6 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -182,6 +182,9 @@ typedef struct ControlFileData
 	int			max_locks_per_xact;
 	bool		track_commit_timestamp;
 
+	/* WAL prohibited determines if the WAL insert is allowed or not. */
+	bool		wal_prohibited;
+
 	/*
 	 * This data is used to check for hardware-architecture compatibility of
 	 * the database and the backend executable.  We need not check endianness
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f48f5fb4d99..41b8fe02b3e 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10964,6 +10964,10 @@
   proname => 'pg_partition_root', prorettype => 'regclass',
   proargtypes => 'regclass', prosrc => 'pg_partition_root' },
 
+{ oid => '4142', descr => 'alter system read only state',
+  proname => 'pg_alter_wal_prohibit_state', prorettype => 'void',
+  proargtypes => 'bool', prosrc => 'pg_alter_wal_prohibit_state' },
+
 { oid => '4350', descr => 'Unicode normalization',
   proname => 'normalize', prorettype => 'text', proargtypes => 'text text',
   prosrc => 'unicode_normalize_func' },
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 0dfbac46b4b..f9ff2360b35 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -956,6 +956,7 @@ typedef enum
 	WAIT_EVENT_REPLICATION_SLOT_RESTORE_SYNC,
 	WAIT_EVENT_REPLICATION_SLOT_SYNC,
 	WAIT_EVENT_REPLICATION_SLOT_WRITE,
+	WAIT_EVENT_WALPROHIBIT_STATE_CHANGE,
 	WAIT_EVENT_SLRU_FLUSH_SYNC,
 	WAIT_EVENT_SLRU_READ,
 	WAIT_EVENT_SLRU_SYNC,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e6..ad5e3ba5724 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -42,4 +42,6 @@ extern void CheckpointerShmemInit(void);
 
 extern bool FirstCallSinceLastCheckpoint(void);
 
+extern void SendsSignalToCheckpointer(int signum);
+
 #endif							/* _BGWRITER_H */
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f38..bae06202b4a 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -48,12 +48,7 @@ typedef enum
 
 typedef enum
 {
-	/*
-	 * XXX. PROCSIGNAL_BARRIER_PLACEHOLDER should be replaced when the first
-	 * real user of the ProcSignalBarrier mechanism is added. It's just here
-	 * for now because we can't have an empty enum.
-	 */
-	PROCSIGNAL_BARRIER_PLACEHOLDER = 0
+	PROCSIGNAL_BARRIER_WALPROHIBIT = 0
 } ProcSignalBarrierType;
 
 /*
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 74af9665d00..f16efeb5d6a 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2670,6 +2670,7 @@ WALAvailability
 WALInsertLock
 WALInsertLockPadded
 WALOpenSegment
+WALProhibitStateData
 WALReadError
 WALSegmentCloseCB
 WALSegmentContext
-- 
2.22.0

From 5f1ec1b1aa8d26c3d81bec9e65207861fd0dcb86 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 19 Jun 2020 06:27:53 -0400
Subject: [PATCH v9 1/5] Allow error or refusal while absorbing barriers.

Patch by Robert Haas
---
 src/backend/storage/ipc/procsignal.c | 75 +++++++++++++++++++++++-----
 1 file changed, 63 insertions(+), 12 deletions(-)

diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index ffe67acea1c..63cb70bcaa4 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -87,12 +87,16 @@ typedef struct
 #define BARRIER_SHOULD_CHECK(flags, type) \
 	(((flags) & (((uint32) 1) << (uint32) (type))) != 0)
 
+/* Clear the relevant type bit from the flags. */
+#define BARRIER_CLEAR_BIT(flags, type) \
+	((flags) &= ~(((uint32) 1) << (uint32) (type)))
+
 static ProcSignalHeader *ProcSignal = NULL;
 static volatile ProcSignalSlot *MyProcSignalSlot = NULL;
 
 static bool CheckProcSignal(ProcSignalReason reason);
 static void CleanupProcSignalState(int status, Datum arg);
-static void ProcessBarrierPlaceholder(void);
+static bool ProcessBarrierPlaceholder(void);
 
 /*
  * ProcSignalShmemSize
@@ -486,17 +490,59 @@ ProcessProcSignalBarrier(void)
 	flags = pg_atomic_exchange_u32(&MyProcSignalSlot->pss_barrierCheckMask, 0);
 
 	/*
-	 * Process each type of barrier. It's important that nothing we call from
-	 * here throws an error, because pss_barrierCheckMask has already been
-	 * cleared. If we jumped out of here before processing all barrier types,
-	 * then we'd forget about the need to do so later.
-	 *
-	 * NB: It ought to be OK to call the barrier-processing functions
-	 * unconditionally, but it's more efficient to call only the ones that
-	 * might need us to do something based on the flags.
+	 * If there are no flags set, then we can skip doing any real work.
+	 * Otherwise, establish a PG_TRY block, so that we don't lose track of
+	 * which types of barrier processing are needed if an ERROR occurs.
 	 */
-	if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_PLACEHOLDER))
-		ProcessBarrierPlaceholder();
+	if (flags != 0)
+	{
+		PG_TRY();
+		{
+			/*
+			 * Process each type of barrier. The barrier-processing functions
+			 * should normally return true, but may return false if the barrier
+			 * can't be absorbed at the current time. This should be rare,
+			 * because it's pretty expensive.  Every single
+			 * CHECK_FOR_INTERRUPTS() will return here until we manage to
+			 * absorb the barrier, and that cost will add up in a hurry.
+			 *
+			 * NB: It ought to be OK to call the barrier-processing functions
+			 * unconditionally, but it's more efficient to call only the ones
+			 * that might need us to do something based on the flags.
+			 */
+			if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_PLACEHOLDER)
+				&& ProcessBarrierPlaceholder())
+				BARRIER_CLEAR_BIT(flags, PROCSIGNAL_BARRIER_PLACEHOLDER);
+		}
+		PG_CATCH();
+		{
+			/*
+			 * If an ERROR occurred, add any flags that weren't yet handled
+			 * back into pss_barrierCheckMask, and reset the global variables
+			 * so that we try again the next time we check for interrupts.
+			 */
+			pg_atomic_fetch_or_u32(&MyProcSignalSlot->pss_barrierCheckMask,
+								   flags);
+			ProcSignalBarrierPending = true;
+			InterruptPending = true;
+
+			PG_RE_THROW();
+		}
+		PG_END_TRY();
+
+		/*
+		 * If some barrier was not successfully absorbed, we will have to try
+		 * again later.
+		 */
+		if (flags != 0)
+		{
+			pg_atomic_fetch_or_u32(&MyProcSignalSlot->pss_barrierCheckMask,
+								   flags);
+			ProcSignalBarrierPending = true;
+			InterruptPending = true;
+			return;
+		}
+	}
 
 	/*
 	 * State changes related to all types of barriers that might have been
@@ -508,7 +554,7 @@ ProcessProcSignalBarrier(void)
 	pg_atomic_write_u64(&MyProcSignalSlot->pss_barrierGeneration, shared_gen);
 }
 
-static void
+static bool
 ProcessBarrierPlaceholder(void)
 {
 	/*
@@ -518,7 +564,12 @@ ProcessBarrierPlaceholder(void)
 	 * appropriately descriptive. Get rid of this function and instead have
 	 * ProcessBarrierSomethingElse. Most likely, that function should live in
 	 * the file pertaining to that subsystem, rather than here.
+	 *
+	 * The return value should be 'true' if the barrier was successfully
+	 * absorbed and 'false' if not. Note that returning 'false' can lead to
+	 * very frequent retries, so try hard to make that an uncommon case.
 	 */
+	return true;
 }
 
 /*
-- 
2.22.0

From fed340cae51218fa3c4e0ae0cd80e8e08f50a1f0 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 27 Mar 2020 05:05:38 -0400
Subject: [PATCH v9 2/5] Add alter system read only/write syntax

Note that syntax doesn't have any implementation.
---
 src/backend/nodes/copyfuncs.c    | 12 ++++++++++++
 src/backend/nodes/equalfuncs.c   |  9 +++++++++
 src/backend/nodes/outfuncs.c     | 12 ++++++++++++
 src/backend/nodes/readfuncs.c    | 15 +++++++++++++++
 src/backend/parser/gram.y        | 13 +++++++++++++
 src/backend/tcop/utility.c       | 21 +++++++++++++++++++++
 src/bin/psql/tab-complete.c      |  6 ++++--
 src/include/nodes/nodes.h        |  1 +
 src/include/nodes/parsenodes.h   | 10 ++++++++++
 src/tools/pgindent/typedefs.list |  1 +
 10 files changed, 98 insertions(+), 2 deletions(-)

diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 0409a40b82a..b3c055a4b7e 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -4020,6 +4020,15 @@ _copyAlterSystemStmt(const AlterSystemStmt *from)
 	return newnode;
 }
 
+static AlterSystemWALProhibitState *
+_copyAlterSystemWALProhibitState(const AlterSystemWALProhibitState *from)
+{
+	AlterSystemWALProhibitState *newnode = makeNode(AlterSystemWALProhibitState);
+
+	COPY_SCALAR_FIELD(walprohibited);
+	return newnode;
+}
+
 static CreateSeqStmt *
 _copyCreateSeqStmt(const CreateSeqStmt *from)
 {
@@ -5405,6 +5414,9 @@ copyObjectImpl(const void *from)
 		case T_AlterSystemStmt:
 			retval = _copyAlterSystemStmt(from);
 			break;
+		case T_AlterSystemWALProhibitState:
+			retval = _copyAlterSystemWALProhibitState(from);
+			break;
 		case T_CreateSeqStmt:
 			retval = _copyCreateSeqStmt(from);
 			break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e2d1b987bf4..5f5f289b8af 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1769,6 +1769,12 @@ _equalAlterSystemStmt(const AlterSystemStmt *a, const AlterSystemStmt *b)
 	return true;
 }
 
+static bool
+_equalAlterSystemWALProhibitState(const AlterSystemWALProhibitState *a, const AlterSystemWALProhibitState *b)
+{
+	COMPARE_SCALAR_FIELD(walprohibited);
+	return true;
+}
 
 static bool
 _equalCreateSeqStmt(const CreateSeqStmt *a, const CreateSeqStmt *b)
@@ -3457,6 +3463,9 @@ equal(const void *a, const void *b)
 		case T_AlterSystemStmt:
 			retval = _equalAlterSystemStmt(a, b);
 			break;
+		case T_AlterSystemWALProhibitState:
+			retval = _equalAlterSystemWALProhibitState(a, b);
+			break;
 		case T_CreateSeqStmt:
 			retval = _equalCreateSeqStmt(a, b);
 			break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index f0386480ab8..c8f89e6f635 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -1358,6 +1358,15 @@ _outAlternativeSubPlan(StringInfo str, const AlternativeSubPlan *node)
 	WRITE_NODE_FIELD(subplans);
 }
 
+static void
+_outAlterSystemWALProhibitState(StringInfo str,
+								const AlterSystemWALProhibitState *node)
+{
+	WRITE_NODE_TYPE("ALTERSYSTEMWALPROHIBITSTATE");
+
+	WRITE_BOOL_FIELD(walprohibited);
+}
+
 static void
 _outFieldSelect(StringInfo str, const FieldSelect *node)
 {
@@ -3915,6 +3924,9 @@ outNode(StringInfo str, const void *obj)
 			case T_AlternativeSubPlan:
 				_outAlternativeSubPlan(str, obj);
 				break;
+			case T_AlterSystemWALProhibitState:
+				_outAlterSystemWALProhibitState(str, obj);
+				break;
 			case T_FieldSelect:
 				_outFieldSelect(str, obj);
 				break;
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 42050ab7195..beb6540ecb9 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -2552,6 +2552,19 @@ _readAlternativeSubPlan(void)
 	READ_DONE();
 }
 
+/*
+ * _readAlterSystemWALProhibitState
+ */
+static AlterSystemWALProhibitState *
+_readAlterSystemWALProhibitState(void)
+{
+	READ_LOCALS(AlterSystemWALProhibitState);
+
+	READ_BOOL_FIELD(walprohibited);
+
+	READ_DONE();
+}
+
 /*
  * _readExtensibleNode
  */
@@ -2874,6 +2887,8 @@ parseNodeString(void)
 		return_value = _readSubPlan();
 	else if (MATCH("ALTERNATIVESUBPLAN", 18))
 		return_value = _readAlternativeSubPlan();
+	else if (MATCH("ALTERSYSTEMWALPROHIBITSTATE", 27))
+		return_value = _readAlterSystemWALProhibitState();
 	else if (MATCH("EXTENSIBLENODE", 14))
 		return_value = _readExtensibleNode();
 	else if (MATCH("PARTITIONBOUNDSPEC", 18))
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 17653ef3a79..a0514e66c4a 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -479,6 +479,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
 
 %type <vsetstmt> generic_set set_rest set_rest_more generic_reset reset_rest
 				 SetResetClause FunctionSetResetClause
+%type <boolean> system_readonly_state
 
 %type <node>	TableElement TypedTableElement ConstraintElem TableFuncElement
 %type <node>	columnDef columnOptions
@@ -10173,8 +10174,20 @@ AlterSystemStmt:
 					n->setstmt = $4;
 					$$ = (Node *)n;
 				}
+			| ALTER SYSTEM_P system_readonly_state
+				{
+					AlterSystemWALProhibitState *n = makeNode(AlterSystemWALProhibitState);
+					n->walprohibited = $3;
+					$$ = (Node *)n;
+				}
 		;
 
+system_readonly_state:
+			 READ ONLY
+					{ $$ = true; }
+			| READ WRITE
+					{ $$ = false; }
+		;
 
 /*****************************************************************************
  *
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 9a35147b26a..74c2162cd59 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -85,6 +85,7 @@ static void ProcessUtilitySlow(ParseState *pstate,
 							   DestReceiver *dest,
 							   QueryCompletion *qc);
 static void ExecDropStmt(DropStmt *stmt, bool isTopLevel);
+static void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
 
 /*
  * CommandIsReadOnly: is an executable query read-only?
@@ -219,6 +220,7 @@ ClassifyUtilityCommandAsReadOnly(Node *parsetree)
 				return COMMAND_IS_NOT_READ_ONLY;
 			}
 
+		case T_AlterSystemWALProhibitState:
 		case T_AlterSystemStmt:
 			{
 				/*
@@ -834,6 +836,11 @@ standard_ProcessUtility(PlannedStmt *pstmt,
 			AlterSystemSetConfigFile((AlterSystemStmt *) parsetree);
 			break;
 
+		case T_AlterSystemWALProhibitState:
+			PreventInTransactionBlock(isTopLevel, "ALTER SYSTEM");
+			AlterSystemSetWALProhibitState((AlterSystemWALProhibitState *) parsetree);
+			break;
+
 		case T_VariableSetStmt:
 			ExecSetVariableStmt((VariableSetStmt *) parsetree, isTopLevel);
 			break;
@@ -2819,6 +2826,7 @@ CreateCommandTag(Node *parsetree)
 			tag = CMDTAG_REFRESH_MATERIALIZED_VIEW;
 			break;
 
+		case T_AlterSystemWALProhibitState:
 		case T_AlterSystemStmt:
 			tag = CMDTAG_ALTER_SYSTEM;
 			break;
@@ -3683,3 +3691,16 @@ GetCommandLogLevel(Node *parsetree)
 
 	return lev;
 }
+
+/*
+ * AlterSystemSetWALProhibitState
+ *
+ * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
+ */
+static void
+AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
+{
+	ereport(ERROR,
+			(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+			 errmsg("ALTER SYSTEM READ { ONLY | WRITE } not implemented")));
+}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 24c7b414cf3..9356f13e509 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1864,9 +1864,11 @@ psql_completion(const char *text, int start, int end)
 	/* ALTER SERVER <name> VERSION <version> */
 	else if (Matches("ALTER", "SERVER", MatchAny, "VERSION", MatchAny))
 		COMPLETE_WITH("OPTIONS");
-	/* ALTER SYSTEM SET, RESET, RESET ALL */
+	/* ALTER SYSTEM READ, SET, RESET, RESET ALL */
 	else if (Matches("ALTER", "SYSTEM"))
-		COMPLETE_WITH("SET", "RESET");
+		COMPLETE_WITH("SET", "RESET", "READ");
+	else if (Matches("ALTER", "SYSTEM", "READ"))
+		COMPLETE_WITH("ONLY", "WRITE");
 	else if (Matches("ALTER", "SYSTEM", "SET|RESET"))
 		COMPLETE_WITH_QUERY(Query_for_list_of_alter_system_set_vars);
 	else if (Matches("ALTER", "SYSTEM", "SET", MatchAny))
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 7ddd8c011bf..7b233925692 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -411,6 +411,7 @@ typedef enum NodeTag
 	T_RefreshMatViewStmt,
 	T_ReplicaIdentityStmt,
 	T_AlterSystemStmt,
+	T_AlterSystemWALProhibitState,
 	T_CreatePolicyStmt,
 	T_AlterPolicyStmt,
 	T_CreateTransformStmt,
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 60c2f454660..340ee87f1bc 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -3196,6 +3196,16 @@ typedef struct AlterSystemStmt
 	VariableSetStmt *setstmt;	/* SET subcommand */
 } AlterSystemStmt;
 
+/* ----------------------
+ *		Alter System Read Statement
+ * ----------------------
+ */
+typedef struct AlterSystemWALProhibitState
+{
+	NodeTag		type;
+	bool		walprohibited;
+} AlterSystemWALProhibitState;
+
 /* ----------------------
  *		Cluster Statement (support pbrown's cluster index implementation)
  * ----------------------
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 9cd1179af63..db6f86e02ab 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -87,6 +87,7 @@ AlterStatsStmt
 AlterSubscriptionStmt
 AlterSubscriptionType
 AlterSystemStmt
+AlterSystemWALProhibitState
 AlterTSConfigType
 AlterTSConfigurationStmt
 AlterTSDictionaryStmt
-- 
2.22.0

From d384897832a832b268952fb7312be1a110e96f1e Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Tue, 14 Jul 2020 02:30:44 -0400
Subject: [PATCH v9 5/5] WIP - Documentation.

TODOs:

1] Documentation regarding ALTER SYSTEM READ/WRITE
---
 src/backend/access/transam/README | 60 ++++++++++++++++++++++++++++---
 src/backend/storage/page/README   | 12 +++----
 2 files changed, 61 insertions(+), 11 deletions(-)

diff --git a/src/backend/access/transam/README b/src/backend/access/transam/README
index 1edc8180c12..fe7a84f93da 100644
--- a/src/backend/access/transam/README
+++ b/src/backend/access/transam/README
@@ -442,8 +442,8 @@ to be modified.
 2. START_CRIT_SECTION()  (Any error during the next three steps must cause a
 PANIC because the shared buffers will contain unlogged changes, which we
 have to ensure don't get to disk.  Obviously, you should check conditions
-such as whether there's enough free space on the page before you start the
-critical section.)
+such as whether there's WAL write permission and enough free space on the page
+before you start the critical section.)
 
 3. Apply the required changes to the shared buffer(s).
 
@@ -486,6 +486,54 @@ with the incomplete-split flag set, it will finish the interrupted split by
 inserting the key to the parent, before proceeding.
 
 
+Read only system state
+----------------------
+
+The system state when it is not currently possible to insert write-ahead log
+records, either because the system is still in recovery or because the system
+forced to WAL prohibited by executing ALTER SYSTEM READ ONLY.  We have a
+lower-level defense in XLogBeginInsert() and elsewhere to stop us from modifying
+data during recovery when !XLogInsertAllowed(), but if XLogBeginInsert() is
+inside the critical section we must not depend on it to report an error.
+Otherwise, it will cause PANIC as mentioned previously.
+
+We do not reach the point where we try to write WAL during recovery but ALTER
+SYSTEM READ ONLY can be executed anytime by the user to stop WAL writing.  Any
+backends which receive read-only system state transition barrier interrupt need
+to stop WAL writing immediately.  For barrier absorption the backed(s) will kill
+the running transaction which has valid XID indicates that the transaction has
+performed and/or planning WAL write.  The transaction which doesn't acquire
+valid XID yet or operation such VACUUM or CONCURRENT CREATE INDEX which not
+necessary have valid XID for WAL will not be prevented while barrier processing,
+and those might hit the error from XLogBeginInsert() while trying to write WAL
+in read only system state.  To prevent such error from XLogBeginInsert() inside
+the critical section the WAL write permission has to check before
+START_CRIT_SECTION().
+
+To enforce the practice to check WAL permission before entering into critical
+section for the WAL write, we have added an assert check flag that indicates
+permission has been checked before calling XLogBeginInsert().  If not,
+XLogBeginInsert() will have assertion failure.  WAL permission check is not
+mandatory if the XLogBeginInsert() is not inside the critical section where
+throwing the error is acceptable.  To get permission check flag set either
+CheckWALPermitted(), AssertWALPermitted_HaveXID(), or AssertWALPermitted()
+should be called before START_CRIT_SECTION().  This flag automatically resets
+while exiting from the critical section.  The rule to place either of permission
+check routines will be:
+
+	The places where WAL write operation in critical can be expected without
+	having valid XID (e.g vacuum) need to protect by CheckWALPermitted(), so
+	that error can be reported outside before critical section.
+
+	The places where INSERT and UPDATE are expected which are never happened
+	without valid XID can be checked using AssertWALPermitted_HaveXID.  So that
+	non-assert build will not have the checking overhead.
+
+	The places we know that we cannot be reached in the read-only state and may
+	or may not have XID, but need to ensure the permission has been checked on
+	assert enabled build should use AssertWALPermitted().
+
+
 Constructing a WAL record
 -------------------------
 
@@ -531,7 +579,8 @@ Details of the API functions:
 
 void XLogBeginInsert(void)
 
-    Must be called before XLogRegisterBuffer and XLogRegisterData.
+	Must be called before XLogRegisterBuffer and XLogRegisterData.  WAL
+	permission must be check before calling it in a critical section.
 
 void XLogResetInsertion(void)
 
@@ -638,8 +687,9 @@ MarkBufferDirtyHint() to mark the block dirty.
 If the buffer is clean and checksums are in use then MarkBufferDirtyHint()
 inserts an XLOG_FPI_FOR_HINT record to ensure that we take a full page image
 that includes the hint. We do this to avoid a partial page write, when we
-write the dirtied page. WAL is not written during recovery, so we simply skip
-dirtying blocks because of hints when in recovery.
+write the dirtied page. WAL is not written while in read only (i.e. during
+recovery or in WAL prohibit state), so we simply skip dirtying blocks because of
+hints when in recovery.
 
 If you do decide to optimise away a WAL record, then any calls to
 MarkBufferDirty() must be replaced by MarkBufferDirtyHint(),
diff --git a/src/backend/storage/page/README b/src/backend/storage/page/README
index e30d7ac59ad..15f0bb4b7b5 100644
--- a/src/backend/storage/page/README
+++ b/src/backend/storage/page/README
@@ -56,9 +56,9 @@ WAL is a fatal error and prevents further recovery, whereas a checksum failure
 on a normal data block is a hard error but not a critical one for the server,
 even if it is a very bad thing for the user.
 
-New WAL records cannot be written during recovery, so hint bits set during
-recovery must not dirty the page if the buffer is not already dirty, when
-checksums are enabled.  Systems in Hot-Standby mode may benefit from hint bits
-being set, but with checksums enabled, a page cannot be dirtied after setting a
-hint bit (due to the torn page risk). So, it must wait for full-page images
-containing the hint bit updates to arrive from the primary.
+New WAL records cannot be written during recovery or or while in WAL prohibit
+state, so hint bits set during recovery must not dirty the page if the buffer is
+not already dirty, when checksums are enabled.  Systems in Hot-Standby mode may
+benefit from hint bits being set, but with checksums enabled, a page cannot be
+dirtied after setting a hint bit (due to the torn page risk). So, it must wait
+for full-page images containing the hint bit updates to arrive from the primary.
-- 
2.22.0

From 86bd090ce66684c582fe88821b2e4fcafc0cf71a Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 19 Jun 2020 06:29:36 -0400
Subject: [PATCH v9 3/5] Implement ALTER SYSTEM READ ONLY using global barrier.

Implementation:

 1. When a user tried to change server state to WAL-Prohibited using
    ALTER SYSTEM READ ONLY command or by calling
    pg_alter_wal_prohibit_state(true) sql function, the current state
    generation to inprogress in shared memory marked and signaled
    checkpointer process.  Checkpointer, noticing that the current state
    generation has WALPROHIBIT_TRANSITION_IN_PROGRESS flag set, does the
    barrier request, and then acknowledges back to the backend who
    requested the state change once the transition has been completed.
    Final state will be updated in control file to make it persistent
    across the system restarts.

 2. When a backend receives the WAL-Prohibited barrier, at that moment if
    it is already in a transaction and the transaction already assigned XID,
    then the backend will be killed by throwing FATAL(XXX: need more discussion
    on this)

 3. Otherwise, if that backend running transaction without valid XID then, we
    don't need to do anything special right now, simply call
    ResetLocalXLogInsertAllowed() so that any future WAL insert in will check
    XLogInsertAllowed() first which set ready only state appropriately.

 4. A new transaction (in an existing or in a new backend) starts as a
    read-only transaction.

 5. Autovacuum launcher as well as checkpointer will not do anything in
    WAL-Prohibited server state until someone wakes us up.  E.g. a backend
    might later on request us to put the system back to read-write.
 5. The Autovacuum launcher, as well as the checkpointer, will not do
    anything while in the WAL-Prohibited server state until someone wakes
    up.  E.g. user might, later on, request us to put the system back to
    read-write by executing ALTER SYSTEM READ WRITE.

 6. At shutdown in WAL-Prohibited mode, we'll skip shutdown checkpoint
    and xlog rotation. Starting up again will perform crash recovery(XXX:
    need some discussion on this as well) but the end of recovery checkpoint
    will be skipped and it will be performed when the system changed to
    WAL-Permitted mode.

 7. ALTER SYSTEM READ ONLY/WRITE is restricted on standby server.

 8. To execute ALTER SYSTEM READ ONLY/WRITE, the user should have execute
    permssion on pg_alter_wal_prohibit_state() function.

 9. Add system_is_read_only GUC show the system state -- will true when system
    is wal prohibited or in recovery.
---
 src/backend/access/transam/Makefile      |   1 +
 src/backend/access/transam/walprohibit.c | 390 +++++++++++++++++++++++
 src/backend/access/transam/xact.c        |  37 ++-
 src/backend/access/transam/xlog.c        | 116 ++++++-
 src/backend/catalog/system_views.sql     |   2 +
 src/backend/postmaster/autovacuum.c      |   4 +
 src/backend/postmaster/bgwriter.c        |   2 +-
 src/backend/postmaster/checkpointer.c    |  39 +++
 src/backend/postmaster/pgstat.c          |   3 +
 src/backend/storage/ipc/ipci.c           |   6 +
 src/backend/storage/ipc/procsignal.c     |  26 +-
 src/backend/tcop/utility.c               |  15 +-
 src/backend/utils/misc/guc.c             |  26 ++
 src/bin/pg_controldata/pg_controldata.c  |   2 +
 src/include/access/walprohibit.h         |  94 ++++++
 src/include/access/xlog.h                |   4 +
 src/include/catalog/pg_control.h         |   3 +
 src/include/catalog/pg_proc.dat          |   4 +
 src/include/pgstat.h                     |   1 +
 src/include/postmaster/bgwriter.h        |   2 +
 src/include/storage/procsignal.h         |   7 +-
 src/tools/pgindent/typedefs.list         |   1 +
 22 files changed, 716 insertions(+), 69 deletions(-)
 create mode 100644 src/backend/access/transam/walprohibit.c
 create mode 100644 src/include/access/walprohibit.h

diff --git a/src/backend/access/transam/Makefile b/src/backend/access/transam/Makefile
index 595e02de722..b5322a69954 100644
--- a/src/backend/access/transam/Makefile
+++ b/src/backend/access/transam/Makefile
@@ -26,6 +26,7 @@ OBJS = \
 	twophase.o \
 	twophase_rmgr.o \
 	varsup.o \
+	walprohibit.o \
 	xact.o \
 	xlog.o \
 	xlogarchive.o \
diff --git a/src/backend/access/transam/walprohibit.c b/src/backend/access/transam/walprohibit.c
new file mode 100644
index 00000000000..00c8894d806
--- /dev/null
+++ b/src/backend/access/transam/walprohibit.c
@@ -0,0 +1,390 @@
+/*-------------------------------------------------------------------------
+ *
+ * walprohibit.c
+ * 		PostgreSQL write-ahead log prohibit states
+ *
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/backend/access/transam/walprohibit.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/walprohibit.h"
+#include "fmgr.h"
+#include "pgstat.h"
+#include "port/atomics.h"
+#include "postmaster/bgwriter.h"
+#include "storage/condition_variable.h"
+#include "storage/procsignal.h"
+#include "storage/shmem.h"
+#include "utils/acl.h"
+#include "utils/fmgroids.h"
+#include "utils/fmgrprotos.h"
+
+/*
+ * Shared-memory WAL prohibit state
+ */
+typedef struct WALProhibitStateData
+{
+	/*
+	 * Indicates current WAL prohibit state generation and the last two bits of
+	 * this generation indicates current wal prohibit state.
+	 */
+	pg_atomic_uint32 shared_state_generation;
+
+	/* Signaled when requested WAL prohibit state changes */
+	ConditionVariable walprohibit_cv;
+} WALProhibitStateData;
+
+static WALProhibitStateData *WALProhibitState = NULL;
+
+static void RequestWALProhibitChange(uint32 cur_state_gen);
+
+/*
+ * ProcessBarrierWALProhibit()
+ *
+ * Handle WAL prohibit state change request.
+ */
+bool
+ProcessBarrierWALProhibit(void)
+{
+	/*
+	 * Kill off any transactions that have an XID *before* allowing the system
+	 * to go WAL prohibit state.
+	 */
+	if (FullTransactionIdIsValid(GetTopFullTransactionIdIfAny()))
+	{
+		/*
+		 * Should be here only while transiting towards the WAL prohibit state.
+		 */
+		Assert(WALPROHIBIT_CURRENT_STATE(GetWALProhibitStateGen()) ==
+			   WALPROHIBIT_STATE_GOING_READ_ONLY);
+
+		/*
+		 * XXX: Kill off the whole session by throwing FATAL instead of
+		 * killing transaction by throwing ERROR due to following reasons that
+		 * need be thought:
+		 *
+		 * 1. Due to some presents challenges with the wire protocol, we could
+		 * not simply kill of idle transaction.
+		 *
+		 * 2. If we are here in subtransaction then the ERROR will kill the
+		 * current subtransaction only. In the case of invalidations, that
+		 * might be good enough, but for XID assignment it's not, because
+		 * assigning an XID to a subtransaction also causes higher
+		 * sub-transaction levels and the parent transaction to get XIDs.
+		 */
+		ereport(FATAL,
+				(errcode(ERRCODE_ACTIVE_SQL_TRANSACTION),
+				 errmsg("system is now read only"),
+				 errhint("Sessions with open write transactions must be terminated.")));
+	}
+
+	/* Return to "check" state */
+	ResetLocalXLogInsertAllowed();
+
+	return true;
+}
+
+/*
+ * AlterSystemSetWALProhibitState()
+ *
+ * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
+ */
+void
+AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
+{
+	/* Check permission for pg_alter_wal_prohibit_state() */
+	if (pg_proc_aclcheck(F_PG_ALTER_WAL_PROHIBIT_STATE,
+						 GetUserId(), ACL_EXECUTE) != ACLCHECK_OK)
+		ereport(ERROR,
+				(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+				 errmsg("permission denied for command ALTER SYSTEM"),
+				 errhint("Get execute permission for pg_alter_wal_prohibit_state() to this user.")));
+
+	/* Alter WAL prohibit state not allowed during recovery */
+	PreventCommandDuringRecovery("ALTER SYSTEM");
+
+	/* Execute function to alter wal prohibit state */
+	(void) OidFunctionCall1(F_PG_ALTER_WAL_PROHIBIT_STATE,
+							BoolGetDatum(stmt->walprohibited));
+}
+
+/*
+ * pg_alter_wal_prohibit_state()
+ *
+ * SQL callable function to alter system read write state.
+ */
+Datum
+pg_alter_wal_prohibit_state(PG_FUNCTION_ARGS)
+{
+	bool		walprohibited = PG_GETARG_BOOL(0);
+	uint32		cur_state_gen;
+
+	/* Alter WAL prohibit state not allowed during recovery */
+	PreventCommandDuringRecovery("pg_alter_wal_prohibit_state()");
+
+	/*
+	 * It is not a final state since we yet to convey this WAL prohibit state to
+	 * all backend.
+	 */
+	cur_state_gen = SetWALProhibitState(walprohibited, false);
+
+	/* Server is already in requested state */
+	if (!cur_state_gen)
+		PG_RETURN_VOID();
+
+	/*
+	 * Signal the checkpointer to do the actual state transition, and wait for
+	 * the state change to occur.
+	 */
+	RequestWALProhibitChange(cur_state_gen);
+
+	PG_RETURN_VOID();
+}
+
+/*
+ * RequestWALProhibitChange()
+ *
+ * Request checkpointer to make the WALProhibitState to read-only.
+ */
+static void
+RequestWALProhibitChange(uint32 cur_state_gen)
+{
+	/* Must not be called from checkpointer */
+	Assert(!AmCheckpointerProcess());
+	Assert(GetWALProhibitStateGen() & WALPROHIBIT_TRANSITION_IN_PROGRESS);
+
+	/*
+	 * If in a standalone backend, just do it ourselves.
+	 */
+	if (!IsPostmasterEnvironment)
+	{
+		CompleteWALProhibitChange(cur_state_gen);
+		return;
+	}
+
+	/* Signal checkpointer process */
+	SendsSignalToCheckpointer(SIGINT);
+
+	/* Wait for the state to change to read-only */
+	ConditionVariablePrepareToSleep(&WALProhibitState->walprohibit_cv);
+	for (;;)
+	{
+		/* We'll be done once wal prohibit state generation changes */
+		if (GetWALProhibitStateGen() != cur_state_gen)
+			break;
+
+		ConditionVariableSleep(&WALProhibitState->walprohibit_cv,
+							   WAIT_EVENT_WALPROHIBIT_STATE_CHANGE);
+	}
+	ConditionVariableCancelSleep();
+}
+
+/*
+ * CompleteWALProhibitChange()
+ *
+ * Checkpointer will call this to complete the requested WAL prohibit state
+ * transition.
+ */
+void
+CompleteWALProhibitChange(uint32 cur_state_gen)
+{
+	uint64		barrier_gen;
+	bool		wal_prohibited;
+
+	/*
+	 * Must be called from checkpointer. Otherwise, it must be single-user
+	 * backend.
+	 */
+	Assert(AmCheckpointerProcess() || !IsPostmasterEnvironment);
+	Assert(cur_state_gen & WALPROHIBIT_TRANSITION_IN_PROGRESS);
+
+	/*
+	 * WAL prohibit state change is initiated. We need to complete the state
+	 * transition by setting requested WAL prohibit state in all backends.
+	 */
+	elog(DEBUG1, "waiting for backends to adopt requested WAL prohibit state change");
+
+	/* Emit global barrier */
+	barrier_gen = EmitProcSignalBarrier(PROCSIGNAL_BARRIER_WALPROHIBIT);
+	WaitForProcSignalBarrier(barrier_gen);
+
+	/* And flush all inserts. */
+	XLogFlush(GetXLogInsertRecPtr());
+
+	wal_prohibited =
+		(WALPROHIBIT_NEXT_STATE(cur_state_gen) == WALPROHIBIT_STATE_READ_ONLY);
+
+	/* Set the final state */
+	(void) SetWALProhibitState(wal_prohibited, true);
+
+	/* Update the control file to make state persistent */
+	SetControlFileWALProhibitFlag(wal_prohibited);
+
+	if (wal_prohibited)
+		ereport(LOG, (errmsg("system is now read only")));
+	else
+	{
+		/*
+		 * Request checkpoint if the end-of-recovery checkpoint has been skipped
+		 * previously.
+		 */
+		if (LastCheckPointIsSkipped())
+		{
+			RequestCheckpoint(CHECKPOINT_IMMEDIATE);
+			SetLastCheckPointSkipped(false);
+		}
+		ereport(LOG, (errmsg("system is now read write")));
+	}
+
+	/* Wake up all backends waiting on this. */
+	ConditionVariableBroadcast(&WALProhibitState->walprohibit_cv);
+}
+
+/*
+ * GetWALProhibitStateGen()
+ *
+ * Atomically return the current server WAL prohibited state generation.
+ */
+uint32
+GetWALProhibitStateGen(void)
+{
+	return pg_atomic_read_u32(&WALProhibitState->shared_state_generation);
+}
+
+/*
+ * SetWALProhibitState()
+ *
+ * Increments current shared WAL prohibit state generation concerning to
+ * requested state and returns the same.
+ *
+ * For the transition state request where is_final_state is false if the server
+ * desired transition state is the same as the current state which might have
+ * been requested by some other backend and has been proceeded then the current
+ * wal prohibit generation will be returned so that this backend can wait until
+ * the shared wal prohibited generation change for the final state.  And, if the
+ * server is already completely moved to the requested state then the requester
+ * backend doesn't need to wait, in that case, 0 will be returned.
+ *
+ * The final state can only be requested by the checkpointer or by the
+ * single-user so that there will be no chance that the server is already in the
+ * desired final state.
+ */
+uint32
+SetWALProhibitState(bool wal_prohibited, bool is_final_state)
+{
+	uint32		new_state;
+	uint32		cur_state;
+	uint32		cur_state_gen;
+	uint32		next_state_gen;
+
+	/* Get the current state */
+	cur_state_gen = GetWALProhibitStateGen();
+	cur_state = WALPROHIBIT_CURRENT_STATE(cur_state_gen);
+
+	/* Compute new state */
+	if (is_final_state)
+	{
+		/*
+		 * Only checkpointer or single-user can set the final wal prohibit
+		 * state.
+		 */
+		Assert(AmCheckpointerProcess() || !IsPostmasterEnvironment);
+
+		new_state = wal_prohibited ?
+			WALPROHIBIT_STATE_READ_ONLY : WALPROHIBIT_STATE_READ_WRITE;
+
+		/*
+		 * There won't be any other process for the final state setting so that
+		 * the next final state will be the desired state.
+		 */
+		Assert(WALPROHIBIT_NEXT_STATE(cur_state) == new_state);
+	}
+	else
+	{
+		new_state = wal_prohibited ?
+			WALPROHIBIT_STATE_GOING_READ_ONLY :
+			WALPROHIBIT_STATE_GOING_READ_WRITE;
+
+		/* Server is already in the requested transition state */
+		if (cur_state == new_state)
+			return cur_state;		/* Wait for state transition completion */
+
+		/* Server is already in requested state */
+		if (WALPROHIBIT_NEXT_STATE(new_state) == cur_state)
+			return 0;		/* No wait is needed */
+
+		/* Prevent concurrent contrary in progress transition state setting */
+		if (cur_state & WALPROHIBIT_TRANSITION_IN_PROGRESS)
+		{
+			if (cur_state & WALPROHIBIT_STATE_READ_ONLY)
+				ereport(ERROR,
+						(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+						 errmsg("system state transition to read only is already in progress"),
+						 errhint("Try after sometime again.")));
+			else
+				ereport(ERROR,
+						(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+						 errmsg("system state transition to read write is already in progress"),
+						 errhint("Try after sometime again.")));
+		}
+	}
+
+	/*
+	 * Update new state generation in share memory only if the state generation
+	 * hasn't changed until now we have checked.
+	 */
+	next_state_gen = cur_state_gen + 1;
+	(void) pg_atomic_compare_exchange_u32(&WALProhibitState->shared_state_generation,
+										  &cur_state_gen, next_state_gen);
+
+	/* To be sure that any later reads of memory happen strictly after this. */
+	pg_memory_barrier();
+
+	return next_state_gen;
+}
+
+/*
+ * WALProhibitStateGenerationInit()
+ *
+ * Initialization of shared wal prohibit state generation.
+ */
+void
+WALProhibitStateGenerationInit(bool wal_prohibited)
+{
+	uint32	new_state;
+
+	Assert(AmStartupProcess() || !IsPostmasterEnvironment);
+
+	new_state = wal_prohibited ?
+		WALPROHIBIT_STATE_READ_ONLY : WALPROHIBIT_STATE_READ_WRITE;
+
+	pg_atomic_init_u32(&WALProhibitState->shared_state_generation, new_state);
+}
+
+/*
+ * WALProhibitStateShmemInit()
+ *
+ * Initialization of shared memory for WAL prohibit state.
+ */
+void
+WALProhibitStateShmemInit(void)
+{
+	bool		found;
+
+	WALProhibitState = (WALProhibitStateData *)
+		ShmemInitStruct("WAL Prohibit State",
+						sizeof(WALProhibitStateData),
+						&found);
+
+	if (found)
+		return;
+
+	/* First time through ... */
+	memset(WALProhibitState, 0, sizeof(WALProhibitStateData));
+	ConditionVariableInit(&WALProhibitState->walprohibit_cv);
+}
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index af6afcebb13..188c299bed9 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -1962,23 +1962,28 @@ StartTransaction(void)
 	Assert(s->prevSecContext == 0);
 
 	/*
-	 * Make sure we've reset xact state variables
+	 * Reset xact state variables.
 	 *
-	 * If recovery is still in progress, mark this transaction as read-only.
-	 * We have lower level defences in XLogInsert and elsewhere to stop us
-	 * from modifying data during recovery, but this gives the normal
-	 * indication to the user that the transaction is read-only.
-	 */
-	if (RecoveryInProgress())
-	{
-		s->startedInRecovery = true;
-		XactReadOnly = true;
-	}
-	else
-	{
-		s->startedInRecovery = false;
-		XactReadOnly = DefaultXactReadOnly;
-	}
+	 * If it is not currently possible to insert write-ahead log records,
+	 * either because we are still in recovery or because ALTER SYSTEM READ
+	 * ONLY has been executed, force this to be a read-only transaction.
+	 * We have lower level defences in XLogBeginInsert() and elsewhere to stop
+	 * us from modifying data during recovery when !XLogInsertAllowed(), but
+	 * this gives the normal indication to the user that the transaction is
+	 * read-only.
+	 *
+	 * On the other hand, we only need to set the startedInRecovery flag when
+	 * the transaction started during recovery, and not when WAL is otherwise
+	 * prohibited. This information is used by RelationGetIndexScan() to
+	 * decide whether to permit (1) relying on existing killed-tuple markings
+	 * and (2) further killing of index tuples. Even when WAL is prohibited
+	 * on the master, it's still the master, so the former is OK; and since
+	 * killing index tuples doesn't generate WAL, the latter is also OK.
+	 * See comments in RelationGetIndexScan() and MarkBufferDirtyHint().
+	 */
+	XactReadOnly = DefaultXactReadOnly || !XLogInsertAllowed();
+	s->startedInRecovery = RecoveryInProgress();
+
 	XactDeferrable = DefaultXactDeferrable;
 	XactIsoLevel = DefaultXactIsoLevel;
 	forceSyncCommit = false;
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 79a77ebbfe2..ef373dc4d6b 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -31,6 +31,7 @@
 #include "access/timeline.h"
 #include "access/transam.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog_internal.h"
 #include "access/xlogarchive.h"
@@ -246,9 +247,10 @@ static bool LocalPromoteIsTriggered = false;
  *		0: unconditionally not allowed to insert XLOG
  *		-1: must check RecoveryInProgress(); disallow until it is false
  * Most processes start with -1 and transition to 1 after seeing that recovery
- * is not in progress.  But we can also force the value for special cases.
- * The coding in XLogInsertAllowed() depends on the first two of these states
- * being numerically the same as bool true and false.
+ * is not in progress or the server state is not a WAL prohibited state.  But
+ * we can also force the value for special cases.  The coding in
+ * XLogInsertAllowed() depends on the first two of these states being
+ * numerically the same as bool true and false.
  */
 static int	LocalXLogInsertAllowed = -1;
 
@@ -723,6 +725,11 @@ typedef struct XLogCtlData
 	 */
 	XLogRecPtr	lastFpwDisableRecPtr;
 
+	/*
+	 * lastCheckPointSkipped indicates if the last checkpoint is skipped.
+	 */
+	bool		lastCheckPointSkipped;
+
 	slock_t		info_lck;		/* locks shared variables shown above */
 } XLogCtlData;
 
@@ -968,6 +975,7 @@ static void WALInsertLockAcquire(void);
 static void WALInsertLockAcquireExclusive(void);
 static void WALInsertLockRelease(void);
 static void WALInsertLockUpdateInsertingAt(XLogRecPtr insertingAt);
+static inline bool IsWALProhibited(void);
 
 /*
  * Insert an XLOG record represented by an already-constructed chain of data
@@ -6196,6 +6204,32 @@ SetCurrentChunkStartTime(TimestampTz xtime)
 	SpinLockRelease(&XLogCtl->info_lck);
 }
 
+/*
+ * Set or unset flag to indicating that the last checkpoint has been skipped.
+ */
+void
+SetLastCheckPointSkipped(bool ChkptSkip)
+{
+	SpinLockAcquire(&XLogCtl->info_lck);
+	XLogCtl->lastCheckPointSkipped = ChkptSkip;
+	SpinLockRelease(&XLogCtl->info_lck);
+}
+
+/*
+ * Return value of lastCheckPointSkipped flag.
+ */
+bool
+LastCheckPointIsSkipped(void)
+{
+	bool	ChkptSkipped;
+
+	SpinLockAcquire(&XLogCtl->info_lck);
+	ChkptSkipped = XLogCtl->lastCheckPointSkipped;
+	SpinLockRelease(&XLogCtl->info_lck);
+
+	return ChkptSkipped;
+}
+
 /*
  * Fetch timestamp of latest processed commit/abort record.
  * Startup process maintains an accurate local copy in XLogReceiptTime
@@ -7708,6 +7742,12 @@ StartupXLOG(void)
 	XLogCtl->LogwrtRqst.Write = EndOfLog;
 	XLogCtl->LogwrtRqst.Flush = EndOfLog;
 
+	/*
+	 * Before enabling WAL insertion, initialize WAL prohibit state in shared
+	 * memory that will decide the further WAL insert should be allowed or not.
+	 */
+	WALProhibitStateGenerationInit(ControlFile->wal_prohibited);
+
 	/*
 	 * Update full_page_writes in shared memory and write an XLOG_FPW_CHANGE
 	 * record before resource manager writes cleanup WAL records or checkpoint
@@ -7718,7 +7758,17 @@ StartupXLOG(void)
 	UpdateFullPageWrites();
 	LocalXLogInsertAllowed = -1;
 
-	if (InRecovery)
+	/*
+	 * Skip end-of-recovery checkpoint if the system is in WAL prohibited state.
+	 */
+	if (ControlFile->wal_prohibited && InRecovery)
+	{
+		SetLastCheckPointSkipped(true);
+
+		ereport(LOG,
+				(errmsg("skipping startup checkpoint because the system is read only")));
+	}
+	else if (InRecovery)
 	{
 		/*
 		 * Perform a checkpoint to update all our recovery activity to disk.
@@ -7964,6 +8014,28 @@ StartupXLOG(void)
 		RequestCheckpoint(CHECKPOINT_FORCE);
 }
 
+/* Set ControlFile's WAL prohibit flag */
+void
+SetControlFileWALProhibitFlag(bool wal_prohibited)
+{
+	LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+	ControlFile->wal_prohibited = wal_prohibited;
+	UpdateControlFile();
+	LWLockRelease(ControlFileLock);
+}
+
+/*
+ * Is the system still in WAL prohibited state?
+ */
+static inline bool
+IsWALProhibited(void)
+{
+	uint32 		cur_state = WALPROHIBIT_CURRENT_STATE(GetWALProhibitStateGen());
+
+	return (cur_state != WALPROHIBIT_STATE_READ_WRITE &&
+			cur_state != WALPROHIBIT_STATE_GOING_READ_WRITE);
+}
+
 /*
  * Checks if recovery has reached a consistent state. When consistency is
  * reached and we have a valid starting standby snapshot, tell postmaster
@@ -8179,9 +8251,9 @@ HotStandbyActiveInReplay(void)
 /*
  * Is this process allowed to insert new WAL records?
  *
- * Ordinarily this is essentially equivalent to !RecoveryInProgress().
- * But we also have provisions for forcing the result "true" or "false"
- * within specific processes regardless of the global state.
+ * Ordinarily this is essentially equivalent to !RecoveryInProgress() and
+ * !IsWALProhibited().  But we also have provisions for forcing the result
+ * "true" or "false" within specific processes regardless of the global state.
  */
 bool
 XLogInsertAllowed(void)
@@ -8200,9 +8272,20 @@ XLogInsertAllowed(void)
 	if (RecoveryInProgress())
 		return false;
 
+	/* Or, in WAL prohibited state */
+	if (IsWALProhibited())
+	{
+		/*
+		 * Set it to "unconditionally false" to avoid checking until it gets
+		 * reset.
+		 */
+		LocalXLogInsertAllowed = 0;
+		return false;
+	}
+
 	/*
-	 * On exit from recovery, reset to "unconditionally true", since there is
-	 * no need to keep checking.
+	 * On exit from recovery or WAL prohibited state, reset to "unconditionally
+	 * true", since there is no need to keep checking.
 	 */
 	LocalXLogInsertAllowed = 1;
 	return true;
@@ -8224,6 +8307,12 @@ LocalSetXLogInsertAllowed(void)
 	InitXLOGAccess();
 }
 
+void
+ResetLocalXLogInsertAllowed(void)
+{
+	LocalXLogInsertAllowed = -1;
+}
+
 /*
  * Subroutine to try to fetch and validate a prior checkpoint record.
  *
@@ -8513,9 +8602,13 @@ ShutdownXLOG(int code, Datum arg)
 	 */
 	WalSndWaitStopping();
 
+	/*
+	 * The restartpoint, checkpoint, or xlog rotation will be performed if the
+	 * WAL writing is permitted.
+	 */
 	if (RecoveryInProgress())
 		CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
-	else
+	else if (XLogInsertAllowed())
 	{
 		/*
 		 * If archiving is enabled, rotate the last XLOG file so that all the
@@ -8528,6 +8621,9 @@ ShutdownXLOG(int code, Datum arg)
 
 		CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
 	}
+	else
+		ereport(LOG,
+			   (errmsg("skipping shutdown checkpoint because the system is read only")));
 }
 
 /*
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index ed4f3f142d8..79da249dd5c 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1485,6 +1485,8 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
 REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
 REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
 
+REVOKE EXECUTE ON FUNCTION pg_alter_wal_prohibit_state(bool) FROM public;
+
 --
 -- We also set up some things as accessible to standard roles.
 --
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 2cef56f115f..efee35cbc94 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -659,6 +659,10 @@ AutoVacLauncherMain(int argc, char *argv[])
 
 		HandleAutoVacLauncherInterrupts();
 
+		/* If the server is read only just go back to sleep. */
+		if (!XLogInsertAllowed())
+			continue;
+
 		/*
 		 * a worker finished, or postmaster signaled failure to start a worker
 		 */
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index a7afa758b61..1d9c46de20a 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -278,7 +278,7 @@ BackgroundWriterMain(void)
 		 * Checkpointer, when active, is barely ever in its mainloop and thus
 		 * makes it hard to log regularly.
 		 */
-		if (XLogStandbyInfoActive() && !RecoveryInProgress())
+		if (XLogStandbyInfoActive() && XLogInsertAllowed())
 		{
 			TimestampTz timeout = 0;
 			TimestampTz now = GetCurrentTimestamp();
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 3e7dcd4f764..e2ff484d367 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -39,6 +39,7 @@
 #include <sys/time.h>
 #include <time.h>
 
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xlog_internal.h"
 #include "libpq/pqsignal.h"
@@ -342,6 +343,7 @@ CheckpointerMain(void)
 		pg_time_t	now;
 		int			elapsed_secs;
 		int			cur_timeout;
+		uint32		wal_state_gen;
 
 		/* Clear any already-pending wakeups */
 		ResetLatch(MyLatch);
@@ -352,6 +354,30 @@ CheckpointerMain(void)
 		AbsorbSyncRequests();
 		HandleCheckpointerInterrupts();
 
+		wal_state_gen = GetWALProhibitStateGen();
+
+		if (wal_state_gen & WALPROHIBIT_TRANSITION_IN_PROGRESS)
+		{
+			/* Complete WAL prohibit state change request */
+			CompleteWALProhibitChange(wal_state_gen);
+			continue;
+		}
+		else if (WALPROHIBIT_CURRENT_STATE(wal_state_gen) ==
+				 WALPROHIBIT_STATE_READ_ONLY)
+		{
+			/*
+			 * Don't do anything until someone wakes us up.  For example a
+			 * backend might later on request us to put the system back to
+			 * read-write wal prohibit sate.
+			 */
+			(void) WaitLatch(MyLatch, WL_LATCH_SET | WL_EXIT_ON_PM_DEATH, -1,
+							 WAIT_EVENT_CHECKPOINTER_MAIN);
+			continue;
+		}
+
+		Assert(WALPROHIBIT_CURRENT_STATE(wal_state_gen) ==
+			   WALPROHIBIT_STATE_READ_WRITE);
+
 		/*
 		 * Detect a pending checkpoint request by checking whether the flags
 		 * word in shared memory is nonzero.  We shouldn't need to acquire the
@@ -1333,3 +1359,16 @@ FirstCallSinceLastCheckpoint(void)
 
 	return FirstCall;
 }
+
+/*
+ * SendsSignalToCheckpointer allows a process to send a signal to the checkpoint process.
+ */
+void
+SendsSignalToCheckpointer(int signum)
+{
+	if (CheckpointerShmem->checkpointer_pid == 0)
+		elog(ERROR, "checkpointer is not running");
+
+	if (kill(CheckpointerShmem->checkpointer_pid, signum) != 0)
+		elog(ERROR, "could not signal checkpointer: %m");
+}
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index e6be2b7836a..95a738d7f25 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -4060,6 +4060,9 @@ pgstat_get_wait_io(WaitEventIO w)
 		case WAIT_EVENT_REPLICATION_SLOT_WRITE:
 			event_name = "ReplicationSlotWrite";
 			break;
+		case WAIT_EVENT_WALPROHIBIT_STATE_CHANGE:
+			event_name = "SystemWALProhibitStateChange";
+			break;
 		case WAIT_EVENT_SLRU_FLUSH_SYNC:
 			event_name = "SLRUFlushSync";
 			break;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 96c2aaabbd6..2d000ec2ff7 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -22,6 +22,7 @@
 #include "access/subtrans.h"
 #include "access/syncscan.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "commands/async.h"
 #include "miscadmin.h"
 #include "pgstat.h"
@@ -222,6 +223,11 @@ CreateSharedMemoryAndSemaphores(void)
 	MultiXactShmemInit();
 	InitBufferPool();
 
+	/*
+	 * Set up wal probibit shared state
+	 */
+	WALProhibitStateShmemInit();
+
 	/*
 	 * Set up lock manager
 	 */
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 63cb70bcaa4..4f1b67f9d04 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -18,6 +18,7 @@
 #include <unistd.h>
 
 #include "access/parallel.h"
+#include "access/walprohibit.h"
 #include "commands/async.h"
 #include "miscadmin.h"
 #include "pgstat.h"
@@ -96,7 +97,6 @@ static volatile ProcSignalSlot *MyProcSignalSlot = NULL;
 
 static bool CheckProcSignal(ProcSignalReason reason);
 static void CleanupProcSignalState(int status, Datum arg);
-static bool ProcessBarrierPlaceholder(void);
 
 /*
  * ProcSignalShmemSize
@@ -510,9 +510,9 @@ ProcessProcSignalBarrier(void)
 			 * unconditionally, but it's more efficient to call only the ones
 			 * that might need us to do something based on the flags.
 			 */
-			if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_PLACEHOLDER)
-				&& ProcessBarrierPlaceholder())
-				BARRIER_CLEAR_BIT(flags, PROCSIGNAL_BARRIER_PLACEHOLDER);
+			if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_WALPROHIBIT)
+				&& ProcessBarrierWALProhibit())
+				BARRIER_CLEAR_BIT(flags, PROCSIGNAL_BARRIER_WALPROHIBIT);
 		}
 		PG_CATCH();
 		{
@@ -554,24 +554,6 @@ ProcessProcSignalBarrier(void)
 	pg_atomic_write_u64(&MyProcSignalSlot->pss_barrierGeneration, shared_gen);
 }
 
-static bool
-ProcessBarrierPlaceholder(void)
-{
-	/*
-	 * XXX. This is just a placeholder until the first real user of this
-	 * machinery gets committed. Rename PROCSIGNAL_BARRIER_PLACEHOLDER to
-	 * PROCSIGNAL_BARRIER_SOMETHING_ELSE where SOMETHING_ELSE is something
-	 * appropriately descriptive. Get rid of this function and instead have
-	 * ProcessBarrierSomethingElse. Most likely, that function should live in
-	 * the file pertaining to that subsystem, rather than here.
-	 *
-	 * The return value should be 'true' if the barrier was successfully
-	 * absorbed and 'false' if not. Note that returning 'false' can lead to
-	 * very frequent retries, so try hard to make that an uncommon case.
-	 */
-	return true;
-}
-
 /*
  * CheckProcSignal - check to see if a particular reason has been
  * signaled, and clear the signal flag.  Should be called after receiving
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 74c2162cd59..05eac206182 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -19,6 +19,7 @@
 #include "access/htup_details.h"
 #include "access/reloptions.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "catalog/catalog.h"
@@ -85,7 +86,6 @@ static void ProcessUtilitySlow(ParseState *pstate,
 							   DestReceiver *dest,
 							   QueryCompletion *qc);
 static void ExecDropStmt(DropStmt *stmt, bool isTopLevel);
-static void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
 
 /*
  * CommandIsReadOnly: is an executable query read-only?
@@ -3691,16 +3691,3 @@ GetCommandLogLevel(Node *parsetree)
 
 	return lev;
 }
-
-/*
- * AlterSystemSetWALProhibitState
- *
- * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
- */
-static void
-AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
-{
-	ereport(ERROR,
-			(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
-			 errmsg("ALTER SYSTEM READ { ONLY | WRITE } not implemented")));
-}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 596bcb7b842..24113249f67 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -225,6 +225,7 @@ static bool check_recovery_target_lsn(char **newval, void **extra, GucSource sou
 static void assign_recovery_target_lsn(const char *newval, void *extra);
 static bool check_primary_slot_name(char **newval, void **extra, GucSource source);
 static bool check_default_with_oids(bool *newval, void **extra, GucSource source);
+static const char *show_system_is_read_only(void);
 
 /* Private functions in guc-file.l that need to be called from guc.c */
 static ConfigVariable *ProcessConfigFileInternal(GucContext context,
@@ -615,6 +616,7 @@ static char *recovery_target_string;
 static char *recovery_target_xid_string;
 static char *recovery_target_name_string;
 static char *recovery_target_lsn_string;
+static bool system_is_read_only;
 
 
 /* should be static, but commands/variable.c needs to get at this */
@@ -2036,6 +2038,18 @@ static struct config_bool ConfigureNamesBool[] =
 		NULL, NULL, NULL
 	},
 
+	{
+		/* Not for general use */
+		{"system_is_read_only", PGC_INTERNAL, WAL,
+			gettext_noop("Shows whether the system is read only."),
+			NULL,
+			GUC_NO_RESET_ALL | GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE
+		},
+		&system_is_read_only,
+		false,
+		NULL, NULL, show_system_is_read_only
+	},
+
 	/* End-of-list marker */
 	{
 		{NULL, 0, 0, NULL, NULL}, NULL, false, NULL, NULL, NULL
@@ -12041,4 +12055,16 @@ check_default_with_oids(bool *newval, void **extra, GucSource source)
 	return true;
 }
 
+/*
+ * NB: The return string should be the same as the _ShowOption() for boolean
+ * type.
+ */
+static const char *
+show_system_is_read_only(void)
+{
+	if (!XLogInsertAllowed())
+		return "on";
+	return "off";
+}
+
 #include "guc-file.c"
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index 3e00ac0f701..922cd9641d8 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -290,6 +290,8 @@ main(int argc, char *argv[])
 		   (uint32) ControlFile->backupEndPoint);
 	printf(_("End-of-backup record required:        %s\n"),
 		   ControlFile->backupEndRequired ? _("yes") : _("no"));
+	printf(_("WAL write prohibited:                 %s\n"),
+		   ControlFile->wal_prohibited ? _("yes") : _("no"));
 	printf(_("wal_level setting:                    %s\n"),
 		   wal_level_str(ControlFile->wal_level));
 	printf(_("wal_log_hints setting:                %s\n"),
diff --git a/src/include/access/walprohibit.h b/src/include/access/walprohibit.h
new file mode 100644
index 00000000000..61836d61844
--- /dev/null
+++ b/src/include/access/walprohibit.h
@@ -0,0 +1,94 @@
+/*
+ * walprohibit.h
+ *
+ * PostgreSQL write-ahead log prohibit states
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/include/access/walprohibit.h
+ */
+#ifndef WALPROHIBIT_H
+#define WALPROHIBIT_H
+
+#include "access/xact.h"
+#include "access/xlog.h"
+#include "miscadmin.h"
+#include "nodes/parsenodes.h"
+
+extern bool ProcessBarrierWALProhibit(void);
+extern void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
+extern void CompleteWALProhibitChange(uint32 wal_state);
+extern uint32 GetWALProhibitStateGen(void);
+extern uint32 SetWALProhibitState(bool wal_prohibited, bool is_final_state);
+extern void MarkCheckPointSkippedInWalProhibitState(void);
+extern void WALProhibitStateGenerationInit(bool wal_prohibited);
+extern void WALProhibitStateShmemInit(void);
+
+/*
+ * The WAL Prohibit States.
+ *
+ * 	The odd number represents the transition state and whereas the even number
+ * 	represents the final state.  These states can be distinguished by checking
+ * 	the 0th bits aka transition bit.
+ */
+#define	WALPROHIBIT_STATE_READ_WRITE		(uint32) 0	/* WAL permitted */
+#define	WALPROHIBIT_STATE_GOING_READ_ONLY	(uint32) 1
+#define	WALPROHIBIT_STATE_READ_ONLY			(uint32) 2	/* WAL prohibited */
+#define	WALPROHIBIT_STATE_GOING_READ_WRITE	(uint32) 3
+
+/* The transition bit to distinguish states.  */
+#define	WALPROHIBIT_TRANSITION_IN_PROGRESS	((uint32) 1 << 0)
+
+/* Extract last two bits */
+#define	WALPROHIBIT_CURRENT_STATE(stateGeneration)	\
+	((uint32)(stateGeneration) & ((uint32) ((1 << 2) - 1)))
+#define	WALPROHIBIT_NEXT_STATE(stateGeneration)	\
+	WALPROHIBIT_CURRENT_STATE((stateGeneration + 1))
+
+/* Never reaches when WAL is prohibited. */
+static inline void
+AssertWALPermitted(void)
+{
+	/*
+	 * Recovery in the startup process never is in wal prohibited state.
+	 */
+	Assert(InRecovery || XLogInsertAllowed());
+
+#ifdef USE_ASSERT_CHECKING
+	walpermit_checked_state = WALPERMIT_CHECKED;
+#endif
+}
+
+/*
+ * XID-bearing transactions are killed off by "ALTER SYSTEM READ ONLY", so any
+ * part of the code that can only be reached with an XID assigned is never
+ * reached when WAL is prohibited.
+ */
+static inline void
+AssertWALPermittedHaveXID(void)
+{
+	/* Must be performing an INSERT, UPDATE or DELETE, so we'll have an XID */
+	Assert(FullTransactionIdIsValid(GetTopFullTransactionIdIfAny()));
+	AssertWALPermitted();
+}
+
+/*
+ * In opposite to the above assertion if a transaction doesn't have valid XID
+ * (e.g. VACUUM) then it won't be killed while changing the system state to WAL
+ * prohibited.  Therefore, we need to explicitly error out before entering into
+ * the critical section.
+ */
+static inline void
+CheckWALPermitted(void)
+{
+	if (!XLogInsertAllowed())
+		ereport(ERROR,
+				(errcode(ERRCODE_READ_ONLY_SQL_TRANSACTION),
+				 errmsg("system is now read only")));
+
+#ifdef USE_ASSERT_CHECKING
+	walpermit_checked_state = WALPERMIT_CHECKED;
+#endif
+}
+
+#endif							/* WALPROHIBIT_H */
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 221af87e715..2bcd37894f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -306,6 +306,7 @@ extern RecoveryState GetRecoveryState(void);
 extern bool HotStandbyActive(void);
 extern bool HotStandbyActiveInReplay(void);
 extern bool XLogInsertAllowed(void);
+extern void ResetLocalXLogInsertAllowed(void);
 extern void GetXLogReceiptTime(TimestampTz *rtime, bool *fromStream);
 extern XLogRecPtr GetXLogReplayRecPtr(TimeLineID *replayTLI);
 extern XLogRecPtr GetXLogInsertRecPtr(void);
@@ -314,6 +315,8 @@ extern bool RecoveryIsPaused(void);
 extern void SetRecoveryPause(bool recoveryPause);
 extern TimestampTz GetLatestXTime(void);
 extern TimestampTz GetCurrentChunkReplayStartTime(void);
+extern void SetLastCheckPointSkipped(bool ChkptSkip);
+extern bool LastCheckPointIsSkipped(void);
 
 extern void UpdateControlFile(void);
 extern uint64 GetSystemIdentifier(void);
@@ -325,6 +328,7 @@ extern void XLOGShmemInit(void);
 extern void BootStrapXLOG(void);
 extern void LocalProcessControlFile(bool reset);
 extern void StartupXLOG(void);
+extern void SetControlFileWALProhibitFlag(bool wal_prohibited);
 extern void ShutdownXLOG(int code, Datum arg);
 extern void InitXLOGAccess(void);
 extern void CreateCheckPoint(int flags);
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index 06bed90c5e9..f4dc5412ee6 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -182,6 +182,9 @@ typedef struct ControlFileData
 	int			max_locks_per_xact;
 	bool		track_commit_timestamp;
 
+	/* WAL prohibited determines if the WAL insert is allowed or not. */
+	bool		wal_prohibited;
+
 	/*
 	 * This data is used to check for hardware-architecture compatibility of
 	 * the database and the backend executable.  We need not check endianness
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f48f5fb4d99..41b8fe02b3e 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10964,6 +10964,10 @@
   proname => 'pg_partition_root', prorettype => 'regclass',
   proargtypes => 'regclass', prosrc => 'pg_partition_root' },
 
+{ oid => '4142', descr => 'alter system read only state',
+  proname => 'pg_alter_wal_prohibit_state', prorettype => 'void',
+  proargtypes => 'bool', prosrc => 'pg_alter_wal_prohibit_state' },
+
 { oid => '4350', descr => 'Unicode normalization',
   proname => 'normalize', prorettype => 'text', proargtypes => 'text text',
   prosrc => 'unicode_normalize_func' },
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 0dfbac46b4b..f9ff2360b35 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -956,6 +956,7 @@ typedef enum
 	WAIT_EVENT_REPLICATION_SLOT_RESTORE_SYNC,
 	WAIT_EVENT_REPLICATION_SLOT_SYNC,
 	WAIT_EVENT_REPLICATION_SLOT_WRITE,
+	WAIT_EVENT_WALPROHIBIT_STATE_CHANGE,
 	WAIT_EVENT_SLRU_FLUSH_SYNC,
 	WAIT_EVENT_SLRU_READ,
 	WAIT_EVENT_SLRU_SYNC,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e6..ad5e3ba5724 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -42,4 +42,6 @@ extern void CheckpointerShmemInit(void);
 
 extern bool FirstCallSinceLastCheckpoint(void);
 
+extern void SendsSignalToCheckpointer(int signum);
+
 #endif							/* _BGWRITER_H */
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f38..bae06202b4a 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -48,12 +48,7 @@ typedef enum
 
 typedef enum
 {
-	/*
-	 * XXX. PROCSIGNAL_BARRIER_PLACEHOLDER should be replaced when the first
-	 * real user of the ProcSignalBarrier mechanism is added. It's just here
-	 * for now because we can't have an empty enum.
-	 */
-	PROCSIGNAL_BARRIER_PLACEHOLDER = 0
+	PROCSIGNAL_BARRIER_WALPROHIBIT = 0
 } ProcSignalBarrierType;
 
 /*
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index db6f86e02ab..b89f8f027b1 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2671,6 +2671,7 @@ WALAvailability
 WALInsertLock
 WALInsertLockPadded
 WALOpenSegment
+WALProhibitStateData
 WALReadError
 WALSegmentCloseCB
 WALSegmentContext
-- 
2.22.0

From 039ff18750b69097941d41a989819d1725bc2d90 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Mon, 27 Jul 2020 02:13:36 -0400
Subject: [PATCH v9 4/5] Error or Assert before START_CRIT_SECTION for WAL
 write

Based on the following criteria added the Assert or the Error when system is
prohibited:

 - Added ERROR for the function which can be reachable without valid XID in case
   of VACUUM or CONCURRENT CREATE INDEX.  For that added common static inline
   function CheckWALPermitted().
 - Added ASSERT for the function which cannot be reached without valid XID;
   Assert to ensure XID validation. For that added AssertWALPermitted_HaveXID().

To enforce the rule to have aforesaid assert or error check before entering a
critical section for WAL write, a new walpermit_checked_state assert only flag
is added. If these check is missing then XLogBeginInsert() will have an
assertion if it is in critical section.

If we are not doing WAL insert inside the critical section then the above
checking is not necessary, we can rely on XLogBeginInsert() for that check &
report an error.
---
 contrib/pg_surgery/heap_surgery.c         | 10 ++++--
 src/backend/access/brin/brin.c            |  4 +++
 src/backend/access/brin/brin_pageops.c    | 21 ++++++++++--
 src/backend/access/brin/brin_revmap.c     | 10 +++++-
 src/backend/access/gin/ginbtree.c         | 15 +++++++--
 src/backend/access/gin/gindatapage.c      | 18 +++++++++--
 src/backend/access/gin/ginfast.c          | 11 +++++--
 src/backend/access/gin/gininsert.c        |  4 +++
 src/backend/access/gin/ginutil.c          |  9 +++++-
 src/backend/access/gin/ginvacuum.c        | 11 ++++++-
 src/backend/access/gist/gist.c            | 25 ++++++++++++---
 src/backend/access/gist/gistvacuum.c      | 13 ++++++--
 src/backend/access/hash/hash.c            | 19 +++++++++--
 src/backend/access/hash/hashinsert.c      |  9 +++++-
 src/backend/access/hash/hashovfl.c        | 22 ++++++++++---
 src/backend/access/hash/hashpage.c        |  9 ++++++
 src/backend/access/heap/heapam.c          | 26 ++++++++++++++-
 src/backend/access/heap/pruneheap.c       | 12 +++++--
 src/backend/access/heap/vacuumlazy.c      | 18 +++++++++--
 src/backend/access/heap/visibilitymap.c   | 22 +++++++++++--
 src/backend/access/nbtree/nbtdedup.c      |  3 ++
 src/backend/access/nbtree/nbtinsert.c     | 10 +++++-
 src/backend/access/nbtree/nbtpage.c       | 39 +++++++++++++++++++----
 src/backend/access/spgist/spgdoinsert.c   | 13 ++++++++
 src/backend/access/spgist/spgvacuum.c     | 22 +++++++++++--
 src/backend/access/transam/multixact.c    |  5 ++-
 src/backend/access/transam/twophase.c     |  9 ++++++
 src/backend/access/transam/varsup.c       |  4 +++
 src/backend/access/transam/walprohibit.c  | 10 ++++++
 src/backend/access/transam/xact.c         |  6 ++++
 src/backend/access/transam/xlog.c         | 27 ++++++++++++----
 src/backend/access/transam/xloginsert.c   | 13 ++++++--
 src/backend/commands/sequence.c           | 16 ++++++++++
 src/backend/commands/variable.c           |  9 ++++--
 src/backend/postmaster/checkpointer.c     |  4 +++
 src/backend/storage/buffer/bufmgr.c       | 10 +++---
 src/backend/storage/freespace/freespace.c | 10 +++++-
 src/backend/storage/lmgr/lock.c           |  6 ++--
 src/backend/utils/cache/relmapper.c       |  3 ++
 src/include/miscadmin.h                   | 27 ++++++++++++++++
 40 files changed, 463 insertions(+), 71 deletions(-)

diff --git a/contrib/pg_surgery/heap_surgery.c b/contrib/pg_surgery/heap_surgery.c
index eb96b4bb36d..53d8c9cea28 100644
--- a/contrib/pg_surgery/heap_surgery.c
+++ b/contrib/pg_surgery/heap_surgery.c
@@ -14,6 +14,7 @@
 
 #include "access/heapam.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "catalog/pg_am_d.h"
 #include "catalog/pg_proc_d.h"
 #include "miscadmin.h"
@@ -89,6 +90,7 @@ heap_force_common(FunctionCallInfo fcinfo, HeapTupleForceOption heap_force_opt)
 	OffsetNumber curr_start_ptr,
 				next_start_ptr;
 	bool		include_this_tid[MaxHeapTuplesPerPage];
+	bool		needwal;
 
 	if (RecoveryInProgress())
 		ereport(ERROR,
@@ -100,6 +102,7 @@ heap_force_common(FunctionCallInfo fcinfo, HeapTupleForceOption heap_force_opt)
 	sanity_check_tid_array(ta, &ntids);
 
 	rel = relation_open(relid, RowExclusiveLock);
+	needwal = RelationNeedsWAL(rel);
 
 	/* Check target relation. */
 	sanity_check_relation(rel);
@@ -217,6 +220,9 @@ heap_force_common(FunctionCallInfo fcinfo, HeapTupleForceOption heap_force_opt)
 		if (heap_force_opt == HEAP_FORCE_KILL && PageIsAllVisible(page))
 			visibilitymap_pin(rel, blkno, &vmbuf);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* No ereport(ERROR) from here until all the changes are logged. */
 		START_CRIT_SECTION();
 
@@ -297,12 +303,12 @@ heap_force_common(FunctionCallInfo fcinfo, HeapTupleForceOption heap_force_opt)
 			MarkBufferDirty(buf);
 
 			/* XLOG stuff */
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 				log_newpage_buffer(buf, true);
 		}
 
 		/* WAL log the VM page if it was modified. */
-		if (did_modify_vm && RelationNeedsWAL(rel))
+		if (did_modify_vm && needwal)
 			log_newpage_buffer(vmbuf, false);
 
 		END_CRIT_SECTION();
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index 1f72562c603..47142193706 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -24,6 +24,7 @@
 #include "access/relscan.h"
 #include "access/table.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/index.h"
 #include "catalog/pg_am.h"
@@ -759,6 +760,9 @@ brinbuildempty(Relation index)
 		ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermittedHaveXID();
+
 	/* Initialize and xlog metabuffer. */
 	START_CRIT_SECTION();
 	brin_metapage_init(BufferGetPage(metabuf), BrinGetPagesPerRange(index),
diff --git a/src/backend/access/brin/brin_pageops.c b/src/backend/access/brin/brin_pageops.c
index 87de0b855b5..8b377a679ab 100644
--- a/src/backend/access/brin/brin_pageops.c
+++ b/src/backend/access/brin/brin_pageops.c
@@ -14,6 +14,7 @@
 #include "access/brin_pageops.h"
 #include "access/brin_revmap.h"
 #include "access/brin_xlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -65,6 +66,7 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 	Buffer		newbuf;
 	BlockNumber newblk = InvalidBlockNumber;
 	bool		extended;
+	bool		needwal = RelationNeedsWAL(idxrel);
 
 	Assert(newsz == MAXALIGN(newsz));
 
@@ -176,13 +178,16 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 	if (((BrinPageFlags(oldpage) & BRIN_EVACUATE_PAGE) == 0) &&
 		brin_can_do_samepage_update(oldbuf, origsz, newsz))
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 		if (!PageIndexTupleOverwrite(oldpage, oldoff, (Item) unconstify(BrinTuple *, newtup), newsz))
 			elog(ERROR, "failed to replace BRIN tuple");
 		MarkBufferDirty(oldbuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(idxrel))
+		if (needwal)
 		{
 			xl_brin_samepage_update xlrec;
 			XLogRecPtr	recptr;
@@ -240,6 +245,9 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 
 		revmapbuf = brinLockRevmapPageForUpdate(revmap, heapBlk);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		/*
@@ -267,7 +275,7 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 		MarkBufferDirty(revmapbuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(idxrel))
+		if (needwal)
 		{
 			xl_brin_update xlrec;
 			XLogRecPtr	recptr;
@@ -351,6 +359,7 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	Buffer		revmapbuf;
 	ItemPointerData tid;
 	bool		extended;
+	bool		needwal;
 
 	Assert(itemsz == MAXALIGN(itemsz));
 
@@ -405,6 +414,10 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	page = BufferGetPage(*buffer);
 	blk = BufferGetBlockNumber(*buffer);
 
+	needwal = RelationNeedsWAL(idxrel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Execute the actual insertion */
 	START_CRIT_SECTION();
 	if (extended)
@@ -424,7 +437,7 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	MarkBufferDirty(revmapbuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(idxrel))
+	if (needwal)
 	{
 		xl_brin_insert xlrec;
 		XLogRecPtr	recptr;
@@ -881,6 +894,8 @@ brin_initialize_empty_new_buffer(Relation idxrel, Buffer buffer)
 			   "brin_initialize_empty_new_buffer: initializing blank page %u",
 			   BufferGetBlockNumber(buffer)));
 
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 	page = BufferGetPage(buffer);
 	brin_page_init(page, BRIN_PAGETYPE_REGULAR);
diff --git a/src/backend/access/brin/brin_revmap.c b/src/backend/access/brin/brin_revmap.c
index 35746714a7c..fd766da445d 100644
--- a/src/backend/access/brin/brin_revmap.c
+++ b/src/backend/access/brin/brin_revmap.c
@@ -26,6 +26,7 @@
 #include "access/brin_tuple.h"
 #include "access/brin_xlog.h"
 #include "access/rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -340,6 +341,7 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 	OffsetNumber revmapOffset;
 	OffsetNumber regOffset;
 	ItemId		lp;
+	bool		needwal;
 
 	revmap = brinRevmapInitialize(idxrel, &pagesPerRange, NULL);
 
@@ -404,6 +406,10 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 	 * crashed or aborted summarization; remove them silently.
 	 */
 
+	needwal = RelationNeedsWAL(idxrel);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	ItemPointerSetInvalid(&invalidIptr);
@@ -415,7 +421,7 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 	MarkBufferDirty(regBuf);
 	MarkBufferDirty(revmapBuf);
 
-	if (RelationNeedsWAL(idxrel))
+	if (needwal)
 	{
 		xl_brin_desummarize xlrec;
 		XLogRecPtr	recptr;
@@ -613,6 +619,8 @@ revmap_physical_extend(BrinRevmap *revmap)
 		return;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Ok, we have now locked the metapage and the target block. Re-initialize
 	 * the target block as a revmap page, and update the metapage.
diff --git a/src/backend/access/gin/ginbtree.c b/src/backend/access/gin/ginbtree.c
index 82788a5c367..f31590dcd75 100644
--- a/src/backend/access/gin/ginbtree.c
+++ b/src/backend/access/gin/ginbtree.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/predicate.h"
@@ -332,6 +333,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 {
 	Page		page = BufferGetPage(stack->buffer);
 	bool		result;
+	bool		needwal;
 	GinPlaceToPageRC rc;
 	uint16		xlflags = 0;
 	Page		childpage = NULL;
@@ -377,6 +379,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 								 insertdata, updateblkno,
 								 &ptp_workspace,
 								 &newlpage, &newrpage);
+	needwal = RelationNeedsWAL(btree->index) && !btree->isBuild;
 
 	if (rc == GPTP_NO_WORK)
 	{
@@ -385,10 +388,13 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 	}
 	else if (rc == GPTP_INSERT)
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		/* It will fit, perform the insertion */
 		START_CRIT_SECTION();
 
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogBeginInsert();
 			XLogRegisterBuffer(0, stack->buffer, REGBUF_STANDARD);
@@ -409,7 +415,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 			MarkBufferDirty(childbuf);
 		}
 
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 			ginxlogInsert xlrec;
@@ -547,6 +553,9 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 			}
 		}
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/*
 		 * OK, we have the new contents of the left page in a temporary copy
 		 * now (newlpage), and likewise for the new contents of the
@@ -587,7 +596,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 		}
 
 		/* write WAL record */
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gin/gindatapage.c b/src/backend/access/gin/gindatapage.c
index 7a2690e97f2..0abc5990100 100644
--- a/src/backend/access/gin/gindatapage.c
+++ b/src/backend/access/gin/gindatapage.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "lib/ilist.h"
 #include "miscadmin.h"
@@ -811,6 +812,7 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 	if (removedsomething)
 	{
 		bool		modified;
+		bool		needwal;
 
 		/*
 		 * Make sure we have a palloc'd copy of all segments, after the first
@@ -835,8 +837,12 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 			}
 		}
 
-		if (RelationNeedsWAL(indexrel))
+		needwal = RelationNeedsWAL(indexrel);
+		if (needwal)
+		{
+			CheckWALPermitted();
 			computeLeafRecompressWALData(leaf);
+		}
 
 		/* Apply changes to page */
 		START_CRIT_SECTION();
@@ -845,7 +851,7 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 
 		MarkBufferDirty(buffer);
 
-		if (RelationNeedsWAL(indexrel))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
@@ -1777,6 +1783,7 @@ createPostingTree(Relation index, ItemPointerData *items, uint32 nitems,
 	int			nrootitems;
 	int			rootsize;
 	bool		is_build = (buildStats != NULL);
+	bool		needwal;
 
 	/* Construct the new root page in memory first. */
 	tmppage = (Page) palloc(BLCKSZ);
@@ -1825,12 +1832,17 @@ createPostingTree(Relation index, ItemPointerData *items, uint32 nitems,
 	 */
 	PredicateLockPageSplit(index, BufferGetBlockNumber(entrybuffer), blkno);
 
+	needwal = RelationNeedsWAL(index) && !is_build;
+
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	PageRestoreTempPage(tmppage, page);
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index) && !is_build)
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogCreatePostingTree data;
diff --git a/src/backend/access/gin/ginfast.c b/src/backend/access/gin/ginfast.c
index 2e41b34d8d5..b8c2a993408 100644
--- a/src/backend/access/gin/ginfast.c
+++ b/src/backend/access/gin/ginfast.c
@@ -20,6 +20,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
 #include "catalog/pg_am.h"
@@ -68,6 +69,8 @@ writeListPage(Relation index, Buffer buffer,
 	PGAlignedBlock workspace;
 	char	   *ptr;
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	GinInitBuffer(buffer, GIN_LIST);
@@ -548,6 +551,7 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 	Page		metapage;
 	GinMetaPageData *metadata;
 	BlockNumber blknoToDelete;
+	bool		needwal = RelationNeedsWAL(index);
 
 	metapage = BufferGetPage(metabuffer);
 	metadata = GinPageGetMeta(metapage);
@@ -586,8 +590,11 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 		 * prepare the XLogInsert machinery for that before entering the
 		 * critical section.
 		 */
-		if (RelationNeedsWAL(index))
+		if (needwal)
+		{
+			CheckWALPermitted();
 			XLogEnsureRecordSpace(data.ndeleted, 0);
+		}
 
 		START_CRIT_SECTION();
 
@@ -625,7 +632,7 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 			MarkBufferDirty(buffers[i]);
 		}
 
-		if (RelationNeedsWAL(index))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gin/gininsert.c b/src/backend/access/gin/gininsert.c
index 77433dc8a41..989d82ffcaf 100644
--- a/src/backend/access/gin/gininsert.c
+++ b/src/backend/access/gin/gininsert.c
@@ -17,6 +17,7 @@
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/index.h"
 #include "miscadmin.h"
@@ -447,6 +448,9 @@ ginbuildempty(Relation index)
 		ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(RootBuffer, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermittedHaveXID();
+
 	/* Initialize and xlog metabuffer and root buffer. */
 	START_CRIT_SECTION();
 	GinInitMetabuffer(MetaBuffer);
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index ef9b56fd363..b48ea1a746a 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -17,6 +17,7 @@
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
 #include "access/reloptions.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/pg_collation.h"
 #include "catalog/pg_type.h"
@@ -659,12 +660,18 @@ ginUpdateStats(Relation index, const GinStatsData *stats, bool is_build)
 	Buffer		metabuffer;
 	Page		metapage;
 	GinMetaPageData *metadata;
+	bool		needwal;
 
 	metabuffer = ReadBuffer(index, GIN_METAPAGE_BLKNO);
 	LockBuffer(metabuffer, GIN_EXCLUSIVE);
 	metapage = BufferGetPage(metabuffer);
 	metadata = GinPageGetMeta(metapage);
 
+	needwal = RelationNeedsWAL(index) && !is_build;
+
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	metadata->nTotalPages = stats->nTotalPages;
@@ -684,7 +691,7 @@ ginUpdateStats(Relation index, const GinStatsData *stats, bool is_build)
 
 	MarkBufferDirty(metabuffer);
 
-	if (RelationNeedsWAL(index) && !is_build)
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogUpdateMeta data;
diff --git a/src/backend/access/gin/ginvacuum.c b/src/backend/access/gin/ginvacuum.c
index 0935a6d9e53..d91ca2b391c 100644
--- a/src/backend/access/gin/ginvacuum.c
+++ b/src/backend/access/gin/ginvacuum.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "commands/vacuum.h"
 #include "miscadmin.h"
@@ -136,6 +137,7 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	Page		page,
 				parentPage;
 	BlockNumber rightlink;
+	bool		needwal;
 
 	/*
 	 * This function MUST be called only if someone of parent pages hold
@@ -159,6 +161,10 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	 */
 	PredicateLockPageCombine(gvs->index, deleteBlkno, rightlink);
 
+	needwal = RelationNeedsWAL(gvs->index);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* Unlink the page by changing left sibling's rightlink */
@@ -195,7 +201,7 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	MarkBufferDirty(lBuffer);
 	MarkBufferDirty(dBuffer);
 
-	if (RelationNeedsWAL(gvs->index))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogDeletePage data;
@@ -650,6 +656,9 @@ ginbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 
 		if (resPage)
 		{
+			if (RelationNeedsWAL(gvs.index))
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 			PageRestoreTempPage(resPage, page);
 			MarkBufferDirty(buffer);
diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c
index 25b42e38f22..4a870a062ba 100644
--- a/src/backend/access/gist/gist.c
+++ b/src/backend/access/gist/gist.c
@@ -16,6 +16,7 @@
 
 #include "access/gist_private.h"
 #include "access/gistscan.h"
+#include "access/walprohibit.h"
 #include "catalog/pg_collation.h"
 #include "commands/vacuum.h"
 #include "miscadmin.h"
@@ -135,6 +136,9 @@ gistbuildempty(Relation index)
 	buffer = ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermittedHaveXID();
+
 	/* Initialize and xlog buffer */
 	START_CRIT_SECTION();
 	GISTInitBuffer(buffer, F_LEAF);
@@ -234,6 +238,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 	XLogRecPtr	recptr;
 	int			i;
 	bool		is_split;
+	bool		needwal = RelationNeedsWAL(rel);
 
 	/*
 	 * Refuse to modify a page that's incompletely split. This should not
@@ -465,9 +470,12 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 		 * insertion for that. NB: The number of pages and data segments
 		 * specified here must match the calculations in gistXLogSplit()!
 		 */
-		if (!is_build && RelationNeedsWAL(rel))
+		if (!is_build && needwal)
 			XLogEnsureRecordSpace(npage, 1 + npage * 2);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		/*
@@ -500,7 +508,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 			recptr = GistBuildLSN;
 		else
 		{
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 				recptr = gistXLogSplit(is_leaf,
 									   dist, oldrlink, oldnsn, leftchildbuf,
 									   markfollowright);
@@ -526,6 +534,9 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 	}
 	else
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		/*
 		 * Enough space.  We always get here if ntup==0.
 		 */
@@ -567,7 +578,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 			recptr = GistBuildLSN;
 		else
 		{
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 			{
 				OffsetNumber ndeloffs = 0,
 							deloffs[1];
@@ -1641,6 +1652,7 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 	OffsetNumber offnum,
 				maxoff;
 	TransactionId latestRemovedXid = InvalidTransactionId;
+	bool		needwal = RelationNeedsWAL(rel);
 
 	Assert(GistPageIsLeaf(page));
 
@@ -1659,13 +1671,16 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 			deletable[ndeletable++] = offnum;
 	}
 
-	if (XLogStandbyInfoActive() && RelationNeedsWAL(rel))
+	if (XLogStandbyInfoActive() && needwal)
 		latestRemovedXid =
 			index_compute_xid_horizon_for_tuples(rel, heapRel, buffer,
 												 deletable, ndeletable);
 
 	if (ndeletable > 0)
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		PageIndexMultiDelete(page, deletable, ndeletable);
@@ -1682,7 +1697,7 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 		MarkBufferDirty(buffer);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gist/gistvacuum.c b/src/backend/access/gist/gistvacuum.c
index a9c616c7724..bbb3ebb19ad 100644
--- a/src/backend/access/gist/gistvacuum.c
+++ b/src/backend/access/gist/gistvacuum.c
@@ -17,6 +17,7 @@
 #include "access/genam.h"
 #include "access/gist_private.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "commands/vacuum.h"
 #include "lib/integerset.h"
 #include "miscadmin.h"
@@ -260,6 +261,7 @@ gistvacuumpage(GistVacState *vstate, BlockNumber blkno, BlockNumber orig_blkno)
 	Buffer		buffer;
 	Page		page;
 	BlockNumber recurse_to;
+	bool		needwal = RelationNeedsWAL(rel);
 
 restart:
 	recurse_to = InvalidBlockNumber;
@@ -341,6 +343,9 @@ restart:
 		 */
 		if (ntodelete > 0)
 		{
+			if (needwal)
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 
 			MarkBufferDirty(buffer);
@@ -348,7 +353,7 @@ restart:
 			PageIndexMultiDelete(page, todelete, ntodelete);
 			GistMarkTuplesDeleted(page);
 
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 			{
 				XLogRecPtr	recptr;
 
@@ -580,6 +585,7 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	IndexTuple	idxtuple;
 	XLogRecPtr	recptr;
 	FullTransactionId txid;
+	bool		needwal = RelationNeedsWAL(info->index);
 
 	/*
 	 * Check that the leaf is still empty and deletable.
@@ -634,6 +640,9 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	 */
 	txid = ReadNextFullTransactionId();
 
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* mark the page as deleted */
@@ -645,7 +654,7 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	MarkBufferDirty(parentBuffer);
 	PageIndexTupleDelete(parentPage, downlink);
 
-	if (RelationNeedsWAL(info->index))
+	if (needwal)
 		recptr = gistXLogPageDelete(leafBuffer, txid, parentBuffer, downlink);
 	else
 		recptr = gistGetFakeLSN(info->index);
diff --git a/src/backend/access/hash/hash.c b/src/backend/access/hash/hash.c
index 7c9ccf446c8..f4903a43bb5 100644
--- a/src/backend/access/hash/hash.c
+++ b/src/backend/access/hash/hash.c
@@ -22,6 +22,7 @@
 #include "access/hash_xlog.h"
 #include "access/relscan.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "catalog/index.h"
 #include "commands/progress.h"
 #include "commands/vacuum.h"
@@ -467,6 +468,7 @@ hashbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	Buffer		metabuf = InvalidBuffer;
 	HashMetaPage metap;
 	HashMetaPage cachedmetap;
+	bool		needwal;
 
 	tuples_removed = 0;
 	num_index_tuples = 0;
@@ -573,6 +575,10 @@ loop_top:
 		goto loop_top;
 	}
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Okay, we're really done.  Update tuple count in metapage. */
 	START_CRIT_SECTION();
 
@@ -603,7 +609,7 @@ loop_top:
 	MarkBufferDirty(metabuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_hash_update_meta_page xlrec;
 		XLogRecPtr	recptr;
@@ -690,6 +696,7 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 	Buffer		buf;
 	Bucket		new_bucket PG_USED_FOR_ASSERTS_ONLY = InvalidBucket;
 	bool		bucket_dirty = false;
+	bool		needwal = RelationNeedsWAL(rel);
 
 	blkno = bucket_blkno;
 	buf = bucket_buf;
@@ -788,6 +795,9 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		 */
 		if (ndeletable > 0)
 		{
+			if (needwal)
+				CheckWALPermitted();
+
 			/* No ereport(ERROR) until changes are logged */
 			START_CRIT_SECTION();
 
@@ -809,7 +819,7 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 			MarkBufferDirty(buf);
 
 			/* XLOG stuff */
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 			{
 				xl_hash_delete xlrec;
 				XLogRecPtr	recptr;
@@ -883,6 +893,9 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		page = BufferGetPage(bucket_buf);
 		bucket_opaque = (HashPageOpaque) PageGetSpecialPointer(page);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -890,7 +903,7 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		MarkBufferDirty(bucket_buf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c
index 2ebe671967b..2eab69efa91 100644
--- a/src/backend/access/hash/hashinsert.c
+++ b/src/backend/access/hash/hashinsert.c
@@ -17,6 +17,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "storage/buf_internals.h"
 #include "storage/lwlock.h"
@@ -193,6 +194,8 @@ restart_insert:
 	 */
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	AssertWALPermittedHaveXID();
+
 	/* Do the update.  No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -360,6 +363,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 	if (ndeletable > 0)
 	{
 		TransactionId latestRemovedXid;
+		bool		needwal = RelationNeedsWAL(rel);
 
 		latestRemovedXid =
 			index_compute_xid_horizon_for_tuples(rel, hrel, buf,
@@ -370,6 +374,9 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 		 */
 		LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -393,7 +400,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 		MarkBufferDirty(metabuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			xl_hash_vacuum_one_page xlrec;
 			XLogRecPtr	recptr;
diff --git a/src/backend/access/hash/hashovfl.c b/src/backend/access/hash/hashovfl.c
index 00f0a940116..e7c5dd3e3ce 100644
--- a/src/backend/access/hash/hashovfl.c
+++ b/src/backend/access/hash/hashovfl.c
@@ -19,6 +19,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "utils/rel.h"
 
@@ -312,6 +313,8 @@ _hash_addovflpage(Relation rel, Buffer metabuf, Buffer buf, bool retain_pin)
 
 found:
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Do the update.  No ereport(ERROR) until changes are logged. We want to
 	 * log the changes for bitmap page and overflow page together to avoid
@@ -510,6 +513,7 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	Buffer		prevbuf = InvalidBuffer;
 	Buffer		nextbuf = InvalidBuffer;
 	bool		update_metap = false;
+	bool		needwal;
 
 	/* Get information from the doomed page */
 	_hash_checkpage(rel, ovflbuf, LH_OVERFLOW_PAGE);
@@ -573,10 +577,15 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	/* Get write-lock on metapage to update firstfree */
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	needwal = RelationNeedsWAL(rel);
+
 	/* This operation needs to log multiple tuples, prepare WAL for that */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 		XLogEnsureRecordSpace(HASH_XLOG_FREE_OVFL_BUFS, 4 + nitups);
 
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -641,7 +650,7 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	}
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_hash_squeeze_page xlrec;
 		XLogRecPtr	recptr;
@@ -922,14 +931,19 @@ readpage:
 
 				if (nitups > 0)
 				{
+					bool		needwal = RelationNeedsWAL(rel);
+
 					Assert(nitups == ndeletable);
 
 					/*
 					 * This operation needs to log multiple tuples, prepare
 					 * WAL for that.
 					 */
-					if (RelationNeedsWAL(rel))
+					if (needwal)
+					{
+						CheckWALPermitted();
 						XLogEnsureRecordSpace(0, 3 + nitups);
+					}
 
 					START_CRIT_SECTION();
 
@@ -947,7 +961,7 @@ readpage:
 					MarkBufferDirty(rbuf);
 
 					/* XLOG stuff */
-					if (RelationNeedsWAL(rel))
+					if (needwal)
 					{
 						XLogRecPtr	recptr;
 						xl_hash_move_page_contents xlrec;
diff --git a/src/backend/access/hash/hashpage.c b/src/backend/access/hash/hashpage.c
index a664ecf494a..55a867dd375 100644
--- a/src/backend/access/hash/hashpage.c
+++ b/src/backend/access/hash/hashpage.c
@@ -30,6 +30,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "port/pg_bitutils.h"
 #include "storage/lmgr.h"
@@ -816,6 +817,8 @@ restart_expand:
 		goto fail;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Since we are scribbling on the pages in the shared buffers, establish a
 	 * critical section.  Any failure in this next code leaves us with a big
@@ -1172,6 +1175,8 @@ _hash_splitbucket(Relation rel,
 
 				if (PageGetFreeSpaceForMultipleTuples(npage, nitups + 1) < (all_tups_size + itemsz))
 				{
+					AssertWALPermittedHaveXID();
+
 					/*
 					 * Change the shared buffer state in critical section,
 					 * otherwise any error could make it unrecoverable.
@@ -1223,6 +1228,8 @@ _hash_splitbucket(Relation rel,
 		/* Exit loop if no more overflow pages in old bucket */
 		if (!BlockNumberIsValid(oblkno))
 		{
+			AssertWALPermittedHaveXID();
+
 			/*
 			 * Change the shared buffer state in critical section, otherwise
 			 * any error could make it unrecoverable.
@@ -1269,6 +1276,8 @@ _hash_splitbucket(Relation rel,
 	npage = BufferGetPage(bucket_nbuf);
 	nopaque = (HashPageOpaque) PageGetSpecialPointer(npage);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	oopaque->hasho_flag &= ~LH_BUCKET_BEING_SPLIT;
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 1585861a021..4d6052224fa 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -47,6 +47,7 @@
 #include "access/transam.h"
 #include "access/valid.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1898,6 +1899,8 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
 	 */
 	CheckForSerializableConflictIn(relation, NULL, InvalidBlockNumber);
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -2172,6 +2175,8 @@ heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples,
 										   &vmbuffer, NULL);
 		page = BufferGetPage(buffer);
 
+		AssertWALPermittedHaveXID();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -2690,6 +2695,8 @@ l1:
 							  xid, LockTupleExclusive, true,
 							  &new_xmax, &new_infomask, &new_infomask2);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -3442,6 +3449,8 @@ l2:
 
 		Assert(HEAP_XMAX_IS_LOCKED_ONLY(infomask_lock_old_tuple));
 
+		AssertWALPermittedHaveXID();
+
 		START_CRIT_SECTION();
 
 		/* Clear obsolete visibility flags ... */
@@ -3615,6 +3624,8 @@ l2:
 										   bms_overlap(modified_attrs, id_attrs),
 										   &old_key_copied);
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -4548,6 +4559,8 @@ failed:
 							  GetCurrentTransactionId(), mode, false,
 							  &xid, &new_infomask, &new_infomask2);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -5339,6 +5352,8 @@ l4:
 								VISIBILITYMAP_ALL_FROZEN))
 			cleared_all_frozen = true;
 
+		AssertWALPermittedHaveXID();
+
 		START_CRIT_SECTION();
 
 		/* ... and set them */
@@ -5497,6 +5512,8 @@ heap_finish_speculative(Relation relation, ItemPointer tid)
 	StaticAssertStmt(MaxOffsetNumber < SpecTokenOffsetNumber,
 					 "invalid speculative token constant");
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -5605,6 +5622,8 @@ heap_abort_speculative(Relation relation, ItemPointer tid)
 		elog(ERROR, "attempted to kill a non-speculative tuple");
 	Assert(!HeapTupleHeaderIsHeapOnly(tp.t_data));
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * No need to check for serializable conflicts here.  There is never a
 	 * need for a combocid, either.  No need to extract replica identity, or
@@ -5721,6 +5740,7 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	HeapTupleHeader htup;
 	uint32		oldlen;
 	uint32		newlen;
+	bool		needwal;
 
 	/*
 	 * For now, we don't allow parallel updates.  Unlike a regular update,
@@ -5751,6 +5771,10 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	if (oldlen != newlen || htup->t_hoff != tuple->t_data->t_hoff)
 		elog(ERROR, "wrong tuple length");
 
+	needwal = RelationNeedsWAL(relation);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -5761,7 +5785,7 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	MarkBufferDirty(buffer);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(relation))
+	if (needwal)
 	{
 		xl_heap_inplace xlrec;
 		XLogRecPtr	recptr;
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index bc510e2e9b3..9dcae7d2153 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -18,6 +18,7 @@
 #include "access/heapam_xlog.h"
 #include "access/htup_details.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "catalog/catalog.h"
 #include "miscadmin.h"
@@ -94,11 +95,11 @@ heap_page_prune_opt(Relation relation, Buffer buffer)
 	Size		minfree;
 
 	/*
-	 * We can't write WAL in recovery mode, so there's no point trying to
+	 * We can't write WAL during read-only mode, so there's no point trying to
 	 * clean the page. The primary will likely issue a cleaning WAL record soon
 	 * anyway, so this is no particular loss.
 	 */
-	if (RecoveryInProgress())
+	if (!XLogInsertAllowed())
 		return;
 
 	/*
@@ -232,6 +233,7 @@ heap_page_prune(Relation relation, Buffer buffer,
 	OffsetNumber offnum,
 				maxoff;
 	PruneState	prstate;
+	bool		needwal;
 
 	/*
 	 * Our strategy is to scan the page and make lists of items to change,
@@ -286,6 +288,10 @@ heap_page_prune(Relation relation, Buffer buffer,
 	if (off_loc)
 		*off_loc = InvalidOffsetNumber;
 
+	needwal = RelationNeedsWAL(relation);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Any error while applying the changes is critical */
 	START_CRIT_SECTION();
 
@@ -319,7 +325,7 @@ heap_page_prune(Relation relation, Buffer buffer,
 		/*
 		 * Emit a WAL XLOG_HEAP2_CLEAN record showing what we did
 		 */
-		if (RelationNeedsWAL(relation))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 4f2f38168dc..1869df5f03f 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -59,6 +59,7 @@
 #include "access/parallel.h"
 #include "access/transam.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "catalog/storage.h"
@@ -759,6 +760,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 	BlockNumber nblocks,
 				blkno;
 	HeapTupleData tuple;
+	bool		needwal = RelationNeedsWAL(onerel);
 	TransactionId relfrozenxid = onerel->rd_rel->relfrozenxid;
 	TransactionId relminmxid = onerel->rd_rel->relminmxid;
 	BlockNumber empty_pages,
@@ -1201,6 +1203,9 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 			 */
 			if (!PageIsAllVisible(page))
 			{
+				if (needwal)
+					CheckWALPermitted();
+
 				START_CRIT_SECTION();
 
 				/* mark buffer dirty before writing a WAL record */
@@ -1216,7 +1221,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 				 * page has been previously WAL-logged, and if not, do that
 				 * now.
 				 */
-				if (RelationNeedsWAL(onerel) &&
+				if (needwal &&
 					PageGetLSN(page) == InvalidXLogRecPtr)
 					log_newpage_buffer(buf, true);
 
@@ -1482,6 +1487,9 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 		 */
 		if (nfrozen > 0)
 		{
+			if (needwal)
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 
 			MarkBufferDirty(buf);
@@ -1499,7 +1507,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 			}
 
 			/* Now WAL-log freezing if necessary */
-			if (RelationNeedsWAL(onerel))
+			if (needwal)
 			{
 				XLogRecPtr	recptr;
 
@@ -1932,6 +1940,7 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	TransactionId visibility_cutoff_xid;
 	bool		all_frozen;
 	LVSavedErrInfo saved_err_info;
+	bool		needwal = RelationNeedsWAL(onerel);
 
 	pgstat_progress_update_param(PROGRESS_VACUUM_HEAP_BLKS_VACUUMED, blkno);
 
@@ -1939,6 +1948,9 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	update_vacuum_error_info(vacrelstats, &saved_err_info, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
 							 blkno, InvalidOffsetNumber);
 
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	for (; tupindex < dead_tuples->num_tuples; tupindex++)
@@ -1964,7 +1976,7 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	MarkBufferDirty(buffer);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(onerel))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index b1072183bcd..44244363968 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -88,6 +88,7 @@
 
 #include "access/heapam_xlog.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "miscadmin.h"
 #include "port/pg_bitutils.h"
@@ -249,6 +250,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 	uint8		mapOffset = HEAPBLK_TO_OFFSET(heapBlk);
 	Page		page;
 	uint8	   *map;
+	bool		needwal = RelationNeedsWAL(rel);
 
 #ifdef TRACE_VISIBILITYMAP
 	elog(DEBUG1, "vm_set %s %d", RelationGetRelationName(rel), heapBlk);
@@ -270,6 +272,16 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 	map = (uint8 *) PageGetContents(page);
 	LockBuffer(vmBuf, BUFFER_LOCK_EXCLUSIVE);
 
+	/*
+	 * Can reach here from VACUUM or from startup process, so need not have an
+	 * XID.
+	 *
+	 * Recovery in the startup process never have wal prohibit state, skip
+	 * permission check if reach here in the startup process.
+	 */
+	if (needwal)
+		InRecovery ? AssertWALPermitted() : CheckWALPermitted();
+
 	if (flags != (map[mapByte] >> mapOffset & VISIBILITYMAP_VALID_BITS))
 	{
 		START_CRIT_SECTION();
@@ -277,7 +289,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 		map[mapByte] |= (flags << mapOffset);
 		MarkBufferDirty(vmBuf);
 
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			if (XLogRecPtrIsInvalid(recptr))
 			{
@@ -476,6 +488,7 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		Buffer		mapBuffer;
 		Page		page;
 		char	   *map;
+		bool		needwal;
 
 		newnblocks = truncBlock + 1;
 
@@ -489,8 +502,13 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		page = BufferGetPage(mapBuffer);
 		map = PageGetContents(page);
 
+		needwal = (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded());
+
 		LockBuffer(mapBuffer, BUFFER_LOCK_EXCLUSIVE);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -518,7 +536,7 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		 * during recovery.
 		 */
 		MarkBufferDirty(mapBuffer);
-		if (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded())
+		if (needwal)
 			log_newpage_buffer(mapBuffer, false);
 
 		END_CRIT_SECTION();
diff --git a/src/backend/access/nbtree/nbtdedup.c b/src/backend/access/nbtree/nbtdedup.c
index f6be865b17e..b519a1268e8 100644
--- a/src/backend/access/nbtree/nbtdedup.c
+++ b/src/backend/access/nbtree/nbtdedup.c
@@ -16,6 +16,7 @@
 
 #include "access/nbtree.h"
 #include "access/nbtxlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "utils/rel.h"
 
@@ -271,6 +272,8 @@ _bt_dedup_one_page(Relation rel, Buffer buf, Relation heapRel,
 		nopaque->btpo_flags &= ~BTP_HAS_GARBAGE;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	PageRestoreTempPage(newpage, page);
diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c
index d36f7557c87..2c3d8aaecbd 100644
--- a/src/backend/access/nbtree/nbtinsert.c
+++ b/src/backend/access/nbtree/nbtinsert.c
@@ -19,6 +19,7 @@
 #include "access/nbtxlog.h"
 #include "access/tableam.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/lmgr.h"
@@ -1246,6 +1247,8 @@ _bt_insertonpg(Relation rel,
 			}
 		}
 
+		AssertWALPermittedHaveXID();
+
 		/* Do the update.  No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -1898,13 +1901,16 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf,
 			ropaque->btpo_flags |= BTP_SPLIT_END;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Right sibling is locked, new siblings are prepared, but original page
 	 * is not updated yet.
 	 *
 	 * NO EREPORT(ERROR) till right sibling is updated.  We can get away with
 	 * not starting the critical section till here because we haven't been
-	 * scribbling on the original page yet; see comments above.
+	 * scribbling on the original page yet; see the comments above for grabbing
+	 * the right sibling.
 	 */
 	START_CRIT_SECTION();
 
@@ -2467,6 +2473,8 @@ _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf)
 	right_item = CopyIndexTuple(item);
 	BTreeTupleSetDownLink(right_item, rbkno);
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till newroot op is logged */
 	START_CRIT_SECTION();
 
diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c
index 7f392480ac0..8c3fc251a29 100644
--- a/src/backend/access/nbtree/nbtpage.c
+++ b/src/backend/access/nbtree/nbtpage.c
@@ -26,6 +26,7 @@
 #include "access/nbtxlog.h"
 #include "access/tableam.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
@@ -179,6 +180,7 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	BTMetaPageData *metad;
 	bool		needsRewrite = false;
 	XLogRecPtr	recptr;
+	bool		needwal;
 
 	/* read the metapage and check if it needs rewrite */
 	metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ);
@@ -202,6 +204,10 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	_bt_unlockbuf(rel, metabuf);
 	_bt_lockbuf(rel, metabuf, BT_WRITE);
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* upgrade meta-page if needed */
@@ -214,7 +220,7 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	MarkBufferDirty(metabuf);
 
 	/* write wal record if needed */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_btree_metadata md;
 
@@ -332,6 +338,7 @@ _bt_getroot(Relation rel, int access)
 	if (metad->btm_root == P_NONE)
 	{
 		Page		metapg;
+		bool		needwal;
 
 		/* If access = BT_READ, caller doesn't want us to create root yet */
 		if (access == BT_READ)
@@ -377,6 +384,10 @@ _bt_getroot(Relation rel, int access)
 		/* Get raw page pointer for metapage */
 		metapg = BufferGetPage(metabuf);
 
+		needwal = RelationNeedsWAL(rel);
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO ELOG(ERROR) till meta is updated */
 		START_CRIT_SECTION();
 
@@ -395,7 +406,7 @@ _bt_getroot(Relation rel, int access)
 		MarkBufferDirty(metabuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			xl_btree_newroot xlrec;
 			XLogRecPtr	recptr;
@@ -1131,6 +1142,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 	char	   *updatedbuf = NULL;
 	Size		updatedbuflen = 0;
 	OffsetNumber updatedoffsets[MaxIndexTuplesPerPage];
+	bool		needwal = RelationNeedsWAL(rel);
 
 	/* Shouldn't be called unless there's something to do */
 	Assert(ndeletable > 0 || nupdatable > 0);
@@ -1145,7 +1157,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 	}
 
 	/* XLOG stuff -- allocate and fill buffer before critical section */
-	if (nupdatable > 0 && RelationNeedsWAL(rel))
+	if (nupdatable > 0 && needwal)
 	{
 		Size		offset = 0;
 
@@ -1175,6 +1187,9 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 		}
 	}
 
+	if (needwal)
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -1235,7 +1250,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 	MarkBufferDirty(buf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		xl_btree_vacuum xlrec_vacuum;
@@ -1302,6 +1317,8 @@ _bt_delitems_delete(Relation rel, Buffer buf,
 		latestRemovedXid =
 			_bt_xid_horizon(rel, heapRel, page, deletable, ndeletable);
 
+	AssertWALPermittedHaveXID();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -1832,6 +1849,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	OffsetNumber nextoffset;
 	IndexTuple	itup;
 	IndexTupleData trunctuple;
+	bool		needwal;
 
 	page = BufferGetPage(leafbuf);
 	opaque = (BTPageOpaque) PageGetSpecialPointer(page);
@@ -1920,6 +1938,10 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	 */
 	PredicateLockPageCombine(rel, leafblkno, leafrightsib);
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -1971,7 +1993,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	MarkBufferDirty(leafbuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_btree_mark_page_halfdead xlrec;
 		XLogRecPtr	recptr;
@@ -2064,6 +2086,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 	int			targetlevel;
 	IndexTuple	leafhikey;
 	BlockNumber nextchild;
+	bool		needwal;
 
 	page = BufferGetPage(leafbuf);
 	opaque = (BTPageOpaque) PageGetSpecialPointer(page);
@@ -2277,6 +2300,10 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 	 * Here we begin doing the deletion.
 	 */
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -2356,7 +2383,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 		MarkBufferDirty(leafbuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_btree_unlink_page xlrec;
 		xl_btree_metadata xlmeta;
diff --git a/src/backend/access/spgist/spgdoinsert.c b/src/backend/access/spgist/spgdoinsert.c
index 934d65b89f2..3c5a15c5d32 100644
--- a/src/backend/access/spgist/spgdoinsert.c
+++ b/src/backend/access/spgist/spgdoinsert.c
@@ -18,6 +18,7 @@
 #include "access/genam.h"
 #include "access/spgist_private.h"
 #include "access/spgxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -214,6 +215,8 @@ addLeafTuple(Relation index, SpGistState *state, SpGistLeafTuple leafTuple,
 	xlrec.offnumParent = InvalidOffsetNumber;
 	xlrec.nodeI = 0;
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	if (current->offnum == InvalidOffsetNumber ||
@@ -458,6 +461,8 @@ moveLeafs(Relation index, SpGistState *state,
 
 	leafdata = leafptr = palloc(size);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/* copy all the old tuples to new page, unless they're dead */
@@ -1109,6 +1114,8 @@ doPickSplit(Relation index, SpGistState *state,
 
 	leafdata = leafptr = (char *) palloc(totalLeafSizes);
 
+	AssertWALPermittedHaveXID();
+
 	/* Here we begin making the changes to the target pages */
 	START_CRIT_SECTION();
 
@@ -1517,6 +1524,8 @@ spgAddNodeAction(Relation index, SpGistState *state,
 	if (PageGetExactFreeSpace(current->page) >=
 		newInnerTuple->size - innerTuple->size)
 	{
+		AssertWALPermittedHaveXID();
+
 		/*
 		 * We can replace the inner tuple by new version in-place
 		 */
@@ -1603,6 +1612,8 @@ spgAddNodeAction(Relation index, SpGistState *state,
 		else
 			xlrec.parentBlk = 2;
 
+		AssertWALPermittedHaveXID();
+
 		START_CRIT_SECTION();
 
 		/* insert new ... */
@@ -1788,6 +1799,8 @@ spgSplitNodeAction(Relation index, SpGistState *state,
 									&xlrec.newPage);
 	}
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c
index e1c58933f97..3308832b85b 100644
--- a/src/backend/access/spgist/spgvacuum.c
+++ b/src/backend/access/spgist/spgvacuum.c
@@ -19,6 +19,7 @@
 #include "access/spgist_private.h"
 #include "access/spgxlog.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/storage_xlog.h"
 #include "commands/vacuum.h"
@@ -139,6 +140,7 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 	int			nDeletable;
 	OffsetNumber i,
 				max = PageGetMaxOffsetNumber(page);
+	bool		needwal;
 
 	memset(predecessor, 0, sizeof(predecessor));
 	memset(deletable, 0, sizeof(deletable));
@@ -323,6 +325,10 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 	if (nDeletable != xlrec.nDead + xlrec.nPlaceholder + xlrec.nMove)
 		elog(ERROR, "inconsistent counts of deletable tuples");
 
+	needwal = RelationNeedsWAL(index);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Do the updates */
 	START_CRIT_SECTION();
 
@@ -371,7 +377,7 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 
@@ -411,6 +417,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 	OffsetNumber toDelete[MaxIndexTuplesPerPage];
 	OffsetNumber i,
 				max = PageGetMaxOffsetNumber(page);
+	bool		needwal;
 
 	xlrec.nDelete = 0;
 
@@ -447,6 +454,10 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 	if (xlrec.nDelete == 0)
 		return;					/* nothing more to do */
 
+	needwal = RelationNeedsWAL(index);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Do the update */
 	START_CRIT_SECTION();
 
@@ -455,7 +466,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 
@@ -502,6 +513,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer)
 	OffsetNumber itemnos[MaxIndexTuplesPerPage];
 	spgxlogVacuumRedirect xlrec;
 	GlobalVisState *vistest;
+	bool		needwal;
 
 	xlrec.nToPlaceholder = 0;
 	xlrec.newestRedirectXid = InvalidTransactionId;
@@ -509,6 +521,10 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer)
 	/* XXX: providing heap relation would allow more pruning */
 	vistest = GlobalVisTestFor(NULL);
 
+	needwal = RelationNeedsWAL(index);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -584,7 +600,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer)
 	if (hasUpdate)
 		MarkBufferDirty(buffer);
 
-	if (hasUpdate && RelationNeedsWAL(index))
+	if (hasUpdate && needwal)
 	{
 		XLogRecPtr	recptr;
 
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index a2ce617c8ce..2a42d784f7a 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -73,6 +73,7 @@
 #include "access/transam.h"
 #include "access/twophase.h"
 #include "access/twophase_rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1143,6 +1144,8 @@ GetNewMultiXactId(int nmembers, MultiXactOffset *offset)
 
 	ExtendMultiXactMember(nextOffset, nmembers);
 
+	CheckWALPermitted();
+
 	/*
 	 * Critical section from here until caller has written the data into the
 	 * just-reserved SLRU space; we don't want to error out with a partly
@@ -2927,7 +2930,7 @@ TruncateMultiXact(MultiXactId newOldestMulti, Oid newOldestMultiDB)
 	mxtruncinfo trunc;
 	MultiXactId earliest;
 
-	Assert(!RecoveryInProgress());
+	Assert(XLogInsertAllowed());
 	Assert(MultiXactState->finishedStartup);
 
 	/*
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 79400604431..68ffea44e0d 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -82,6 +82,7 @@
 #include "access/transam.h"
 #include "access/twophase.h"
 #include "access/twophase_rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1106,6 +1107,8 @@ EndPrepare(GlobalTransaction gxact)
 	 */
 	XLogEnsureRecordSpace(0, records.num_chunks);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	MyProc->delayChkpt = true;
@@ -2196,6 +2199,9 @@ RecordTransactionCommitPrepared(TransactionId xid,
 	replorigin = (replorigin_session_origin != InvalidRepOriginId &&
 				  replorigin_session_origin != DoNotReplicateId);
 
+	/* COMMIT PREPARED need not have an XID */
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* See notes in RecordTransactionCommit */
@@ -2286,6 +2292,9 @@ RecordTransactionAbortPrepared(TransactionId xid,
 		elog(PANIC, "cannot abort transaction %u, it was already committed",
 			 xid);
 
+	/* ROLLBACK PREPARED need not have an XID */
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
diff --git a/src/backend/access/transam/varsup.c b/src/backend/access/transam/varsup.c
index a4944faa32e..0c7a2362f25 100644
--- a/src/backend/access/transam/varsup.c
+++ b/src/backend/access/transam/varsup.c
@@ -17,6 +17,7 @@
 #include "access/commit_ts.h"
 #include "access/subtrans.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "commands/dbcommands.h"
@@ -75,6 +76,9 @@ GetNewTransactionId(bool isSubXact)
 	if (RecoveryInProgress())
 		elog(ERROR, "cannot assign TransactionIds during recovery");
 
+	/* do not assign transaction id in read-only mode */
+	CheckWALPermitted();
+
 	LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
 
 	full_xid = ShmemVariableCache->nextXid;
diff --git a/src/backend/access/transam/walprohibit.c b/src/backend/access/transam/walprohibit.c
index 00c8894d806..f2b5bf2871a 100644
--- a/src/backend/access/transam/walprohibit.c
+++ b/src/backend/access/transam/walprohibit.c
@@ -24,6 +24,16 @@
 #include "utils/fmgroids.h"
 #include "utils/fmgrprotos.h"
 
+/*
+ * Assert flag to enforce WAL insert permission check rule before starting a
+ * critical section for the WAL writes.  For this, either of
+ * CheckWALPermitted(), AssertWALPermittedHaveXID(), or AssertWALPermitted()
+ * must be called before starting the critical section.
+ */
+#ifdef USE_ASSERT_CHECKING
+WALPermitCheckState walpermit_checked_state = WALPERMIT_UNCHECKED;
+#endif
+
 /*
  * Shared-memory WAL prohibit state
  */
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 188c299bed9..abda095e735 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -26,6 +26,7 @@
 #include "access/subtrans.h"
 #include "access/transam.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1317,6 +1318,8 @@ RecordTransactionCommit(void)
 		/* Tell bufmgr and smgr to prepare for commit */
 		BufmgrCommit();
 
+		AssertWALPermittedHaveXID();
+
 		/*
 		 * Mark ourselves as within our "commit critical section".  This
 		 * forces any concurrent checkpoint to wait until we've updated
@@ -1677,6 +1680,9 @@ RecordTransactionAbort(bool isSubXact)
 		elog(PANIC, "cannot abort transaction %u, it was already committed",
 			 xid);
 
+	/* We'll be reaching here with valid XID only. */
+	AssertWALPermittedHaveXID();
+
 	/* Fetch the data we need for the abort record */
 	nrels = smgrGetPendingDeletes(false, &rels);
 	nchildren = xactGetCommittedChildren(&children);
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index ef373dc4d6b..f83baef244e 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -1025,7 +1025,7 @@ XLogInsertRecord(XLogRecData *rdata,
 
 	/* cross-check on whether we should be here or not */
 	if (!XLogInsertAllowed())
-		elog(ERROR, "cannot make new WAL entries during recovery");
+		elog(ERROR, "cannot make new WAL entries now");
 
 	/*----------
 	 *
@@ -2860,9 +2860,11 @@ XLogFlush(XLogRecPtr record)
 	 * trying to flush the WAL, we should update minRecoveryPoint instead. We
 	 * test XLogInsertAllowed(), not InRecovery, because we need checkpointer
 	 * to act this way too, and because when it tries to write the
-	 * end-of-recovery checkpoint, it should indeed flush.
+	 * end-of-recovery checkpoint, it should indeed flush. Also, WAL prohibit
+	 * state should not restrict WAL flushing. Otherwise, the dirty buffer
+	 * cannot be evicted until WAL has been flushed up to the buffer's LSN.
 	 */
-	if (!XLogInsertAllowed())
+	if (!XLogInsertAllowed() && !IsWALProhibited())
 	{
 		UpdateMinRecoveryPoint(record, false);
 		return;
@@ -8821,6 +8823,8 @@ CreateCheckPoint(int flags)
 	/* sanity check */
 	if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
 		elog(ERROR, "can't create a checkpoint during recovery");
+	if (!XLogInsertAllowed() && !RecoveryInProgress())
+		elog(ERROR, "can't create a checkpoint while system is read only ");
 
 	/*
 	 * Initialize InitXLogInsert working areas before entering the critical
@@ -8850,6 +8854,8 @@ CreateCheckPoint(int flags)
 	MemSet(&CheckpointStats, 0, sizeof(CheckpointStats));
 	CheckpointStats.ckpt_start_t = GetCurrentTimestamp();
 
+	AssertWALPermitted();
+
 	/*
 	 * Use a critical section to force system panic if we have trouble.
 	 */
@@ -9078,6 +9084,8 @@ CreateCheckPoint(int flags)
 	if (!shutdown && XLogStandbyInfoActive())
 		LogStandbySnapshot();
 
+	AssertWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -9235,6 +9243,8 @@ CreateEndOfRecoveryRecord(void)
 
 	LocalSetXLogInsertAllowed();
 
+	AssertWALPermitted();
+
 	START_CRIT_SECTION();
 
 	XLogBeginInsert();
@@ -9893,7 +9903,7 @@ void
 UpdateFullPageWrites(void)
 {
 	XLogCtlInsert *Insert = &XLogCtl->Insert;
-	bool		recoveryInProgress;
+	bool		WALInsertAllowed;
 
 	/*
 	 * Do nothing if full_page_writes has not been changed.
@@ -9907,10 +9917,10 @@ UpdateFullPageWrites(void)
 
 	/*
 	 * Perform this outside critical section so that the WAL insert
-	 * initialization done by RecoveryInProgress() doesn't trigger an
+	 * initialization done by XLogInsertAllowed() doesn't trigger an
 	 * assertion failure.
 	 */
-	recoveryInProgress = RecoveryInProgress();
+	WALInsertAllowed = XLogInsertAllowed();
 
 	START_CRIT_SECTION();
 
@@ -9932,8 +9942,11 @@ UpdateFullPageWrites(void)
 	 * Write an XLOG_FPW_CHANGE record. This allows us to keep track of
 	 * full_page_writes during archive recovery, if required.
 	 */
-	if (XLogStandbyInfoActive() && !recoveryInProgress)
+	if (XLogStandbyInfoActive() && WALInsertAllowed)
 	{
+		/* Assured that WAL permission has been checked */
+		AssertWALPermitted();
+
 		XLogBeginInsert();
 		XLogRegisterData((char *) (&fullPageWrites), sizeof(bool));
 
diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c
index 1f0e4e01e69..710806143d4 100644
--- a/src/backend/access/transam/xloginsert.c
+++ b/src/backend/access/transam/xloginsert.c
@@ -19,6 +19,7 @@
 
 #include "postgres.h"
 
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xlog_internal.h"
@@ -126,9 +127,14 @@ XLogBeginInsert(void)
 	Assert(mainrdata_last == (XLogRecData *) &mainrdata_head);
 	Assert(mainrdata_len == 0);
 
+	/*
+	 * WAL permission must have checked before entering the critical section.
+	 * Otherwise, WAL prohibited error will force system panic.
+	 */
+	Assert(walpermit_checked_state != WALPERMIT_UNCHECKED || !CritSectionCount);
+
 	/* cross-check on whether we should be here or not */
-	if (!XLogInsertAllowed())
-		elog(ERROR, "cannot make new WAL entries during recovery");
+	CheckWALPermitted();
 
 	if (begininsert_called)
 		elog(ERROR, "XLogBeginInsert was already called");
@@ -210,6 +216,9 @@ XLogResetInsertion(void)
 	mainrdata_last = (XLogRecData *) &mainrdata_head;
 	curinsert_flags = 0;
 	begininsert_called = false;
+
+	/* Reset walpermit_checked flag */
+	RESET_WALPERMIT_CHECKED_STATE();
 }
 
 /*
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 632b34af610..b01ad5a966a 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -20,6 +20,7 @@
 #include "access/relation.h"
 #include "access/table.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -378,8 +379,13 @@ fill_seq_with_data(Relation rel, HeapTuple tuple)
 
 	/* check the comment above nextval_internal()'s equivalent call. */
 	if (RelationNeedsWAL(rel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermittedHaveXID();
+	}
+
 	START_CRIT_SECTION();
 
 	MarkBufferDirty(buf);
@@ -766,8 +772,13 @@ nextval_internal(Oid relid, bool check_permissions)
 	 * (Have to do that here, so we're outside the critical section)
 	 */
 	if (logit && RelationNeedsWAL(seqrel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermittedHaveXID();
+	}
+
 	/* ready to change the on-disk (or really, in-buffer) tuple */
 	START_CRIT_SECTION();
 
@@ -977,8 +988,13 @@ do_setval(Oid relid, int64 next, bool iscalled)
 
 	/* check the comment above nextval_internal()'s equivalent call. */
 	if (RelationNeedsWAL(seqrel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermittedHaveXID();
+	}
+
 	/* ready to change the on-disk (or really, in-buffer) tuple */
 	START_CRIT_SECTION();
 
diff --git a/src/backend/commands/variable.c b/src/backend/commands/variable.c
index 484f7ea2c0e..acc34d2ad7c 100644
--- a/src/backend/commands/variable.c
+++ b/src/backend/commands/variable.c
@@ -501,11 +501,14 @@ check_transaction_read_only(bool *newval, void **extra, GucSource source)
 			GUC_check_errmsg("transaction read-write mode must be set before any query");
 			return false;
 		}
-		/* Can't go to r/w mode while recovery is still active */
-		if (RecoveryInProgress())
+		/*
+		 * Can't go to r/w mode while recovery is still active or while in WAL
+		 * prohibit state
+		 */
+		if (!XLogInsertAllowed())
 		{
 			GUC_check_errcode(ERRCODE_FEATURE_NOT_SUPPORTED);
-			GUC_check_errmsg("cannot set transaction read-write mode during recovery");
+			GUC_check_errmsg("cannot set transaction read-write mode while system is read only");
 			return false;
 		}
 	}
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index e2ff484d367..a12e026ae02 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -942,6 +942,10 @@ RequestCheckpoint(int flags)
 	int			old_failed,
 				old_started;
 
+	/* The checkpoint is allowed in recovery but not in WAL prohibit state */
+	if (!RecoveryInProgress())
+		CheckWALPermitted();
+
 	/*
 	 * If in a standalone backend, just do it ourselves.
 	 */
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index e549fa1d309..5bd898833f5 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -3631,13 +3631,15 @@ MarkBufferDirtyHint(Buffer buffer, bool buffer_std)
 		{
 			/*
 			 * If we must not write WAL, due to a relfilenode-specific
-			 * condition or being in recovery, don't dirty the page.  We can
-			 * set the hint, just not dirty the page as a result so the hint
-			 * is lost when we evict the page or shutdown.
+			 * condition or in general, don't dirty the page.  We can
+			 * set the hint, but must not dirty the page as a result, lest
+			 * we trigger WAL generation. Unless the page is dirtied again
+			 * later, the hint will be lost when the page is evicted, or at
+			 * shutdown.
 			 *
 			 * See src/backend/storage/page/README for longer discussion.
 			 */
-			if (RecoveryInProgress() ||
+			if (!XLogInsertAllowed() ||
 				RelFileNodeSkippingWAL(bufHdr->tag.rnode))
 				return;
 
diff --git a/src/backend/storage/freespace/freespace.c b/src/backend/storage/freespace/freespace.c
index 6a96126b0c2..b05b0fe5f41 100644
--- a/src/backend/storage/freespace/freespace.c
+++ b/src/backend/storage/freespace/freespace.c
@@ -24,6 +24,7 @@
 #include "postgres.h"
 
 #include "access/htup_details.h"
+#include "access/walprohibit.h"
 #include "access/xlogutils.h"
 #include "miscadmin.h"
 #include "storage/freespace.h"
@@ -285,12 +286,19 @@ FreeSpaceMapPrepareTruncateRel(Relation rel, BlockNumber nblocks)
 	 */
 	if (first_removed_slot > 0)
 	{
+		bool needwal;
+
 		buf = fsm_readbuf(rel, first_removed_address, false);
 		if (!BufferIsValid(buf))
 			return InvalidBlockNumber;	/* nothing to do; the FSM was already
 										 * smaller */
 		LockBuffer(buf, BUFFER_LOCK_EXCLUSIVE);
 
+		needwal = (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded());
+
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -305,7 +313,7 @@ FreeSpaceMapPrepareTruncateRel(Relation rel, BlockNumber nblocks)
 		 * during recovery.
 		 */
 		MarkBufferDirty(buf);
-		if (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded())
+		if (needwal)
 			log_newpage_buffer(buf, false);
 
 		END_CRIT_SECTION();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index d86566f4554..f949a290745 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -794,15 +794,15 @@ LockAcquireExtended(const LOCKTAG *locktag,
 	if (lockmode <= 0 || lockmode > lockMethodTable->numLockModes)
 		elog(ERROR, "unrecognized lock mode: %d", lockmode);
 
-	if (RecoveryInProgress() && !InRecovery &&
+	if (!XLogInsertAllowed() && !InRecovery &&
 		(locktag->locktag_type == LOCKTAG_OBJECT ||
 		 locktag->locktag_type == LOCKTAG_RELATION) &&
 		lockmode > RowExclusiveLock)
 		ereport(ERROR,
 				(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
-				 errmsg("cannot acquire lock mode %s on database objects while recovery is in progress",
+				 errmsg("cannot acquire lock mode %s on database objects while system is read only",
 						lockMethodTable->lockModeNames[lockmode]),
-				 errhint("Only RowExclusiveLock or less can be acquired on database objects during recovery.")));
+				 errhint("Only RowExclusiveLock or less can be acquired on database objects while system is read only")));
 
 #ifdef LOCK_DEBUG
 	if (LOCK_DEBUG_ENABLED(locktag))
diff --git a/src/backend/utils/cache/relmapper.c b/src/backend/utils/cache/relmapper.c
index 671fbb0ed5c..90d7599a57c 100644
--- a/src/backend/utils/cache/relmapper.c
+++ b/src/backend/utils/cache/relmapper.c
@@ -43,6 +43,7 @@
 #include <sys/stat.h>
 #include <unistd.h>
 
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -841,6 +842,8 @@ write_relmap_file(bool shared, RelMapFile *newmap,
 		xl_relmap_update xlrec;
 		XLogRecPtr	lsn;
 
+		AssertWALPermittedHaveXID();
+
 		/* now errors are fatal ... */
 		START_CRIT_SECTION();
 
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e33523984..f3ff120601e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -94,12 +94,37 @@ extern PGDLLIMPORT volatile uint32 CritSectionCount;
 /* in tcop/postgres.c */
 extern void ProcessInterrupts(void);
 
+#ifdef USE_ASSERT_CHECKING
+typedef enum
+{
+	WALPERMIT_UNCHECKED,
+	WALPERMIT_CHECKED,
+	WALPERMIT_CHECKED_AND_USED
+} WALPermitCheckState;
+
+/* in access/walprohibit.c */
+extern WALPermitCheckState walpermit_checked_state;
+
+/*
+ * Reset walpermit_checked flag when no longer in the critical section.
+ * Otherwise, marked checked and used.
+ */
+#define RESET_WALPERMIT_CHECKED_STATE() \
+do { \
+	walpermit_checked_state = CritSectionCount ? \
+	WALPERMIT_CHECKED_AND_USED : WALPERMIT_UNCHECKED; \
+} while(0)
+#else
+#define RESET_WALPERMIT_CHECKED_STATE() ((void) 0)
+#endif
+
 #ifndef WIN32
 
 #define CHECK_FOR_INTERRUPTS() \
 do { \
 	if (unlikely(InterruptPending)) \
 		ProcessInterrupts(); \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 #else							/* WIN32 */
 
@@ -109,6 +134,7 @@ do { \
 		pgwin32_dispatch_queued_signals(); \
 	if (unlikely(InterruptPending)) \
 		ProcessInterrupts(); \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 #endif							/* WIN32 */
 
@@ -135,6 +161,7 @@ do { \
 do { \
 	Assert(CritSectionCount > 0); \
 	CritSectionCount--; \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 
 
-- 
2.22.0

From bc586c860138e92216ac22cf6ce8ff521ec7144a Mon Sep 17 00:00:00 2001
From: Robert Haas <rhaas@postgresql.org>
Date: Tue, 6 Oct 2020 15:26:43 -0400
Subject: [PATCH 1/2] Allow for error or refusal while absorbing barriers.

Previously, the per-barrier-type functions tasked with absorbing
them were expected to always succeed and never throw an error.
However, that's a bit inconvenient. Further study has revealed that
there are realistic cases where it might not be possible to absorb
a ProcSignalBarrier without terminating the transaction, or even
the whole backend. Similarly, for some barrier types, there might
be other reasons where it's not reasonably possible to absorb the
barrier at certain points in the code, so provide a way for a
per-barrier-type function to reject absorbing the barrier.

Patch by me, reviewed by Andres Freund.

Discussion: http://postgr.es/m/20200908182005.xya7wetdh3pndzim@alap3.anarazel.de
---
 src/backend/storage/ipc/procsignal.c | 128 ++++++++++++++++++++++++---
 1 file changed, 116 insertions(+), 12 deletions(-)

diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index ffe67acea1..abdae58c47 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -18,6 +18,7 @@
 #include <unistd.h>
 
 #include "access/parallel.h"
+#include "port/pg_bitutils.h"
 #include "commands/async.h"
 #include "miscadmin.h"
 #include "pgstat.h"
@@ -87,12 +88,17 @@ typedef struct
 #define BARRIER_SHOULD_CHECK(flags, type) \
 	(((flags) & (((uint32) 1) << (uint32) (type))) != 0)
 
+/* Clear the relevant type bit from the flags. */
+#define BARRIER_CLEAR_BIT(flags, type) \
+	((flags) &= ~(((uint32) 1) << (uint32) (type)))
+
 static ProcSignalHeader *ProcSignal = NULL;
 static volatile ProcSignalSlot *MyProcSignalSlot = NULL;
 
 static bool CheckProcSignal(ProcSignalReason reason);
 static void CleanupProcSignalState(int status, Datum arg);
-static void ProcessBarrierPlaceholder(void);
+static void ResetProcSignalBarrierBits(uint32 flags);
+static bool ProcessBarrierPlaceholder(void);
 
 /*
  * ProcSignalShmemSize
@@ -394,6 +400,12 @@ WaitForProcSignalBarrier(uint64 generation)
 		volatile ProcSignalSlot *slot = &ProcSignal->psh_slot[i];
 		uint64		oldval;
 
+		/*
+		 * It's important that we check only pss_barrierGeneration here and
+		 * not pss_barrierCheckMask. Bits in pss_barrierCheckMask get cleared
+		 * before the barrier is actually absorbed, but pss_barrierGeneration
+		 * is updated only afterward.
+		 */
 		oldval = pg_atomic_read_u64(&slot->pss_barrierGeneration);
 		while (oldval < generation)
 		{
@@ -453,7 +465,7 @@ ProcessProcSignalBarrier(void)
 {
 	uint64		local_gen;
 	uint64		shared_gen;
-	uint32		flags;
+	volatile uint32		flags;
 
 	Assert(MyProcSignalSlot);
 
@@ -482,21 +494,95 @@ ProcessProcSignalBarrier(void)
 	 * read of the barrier generation above happens before we atomically
 	 * extract the flags, and that any subsequent state changes happen
 	 * afterward.
+	 *
+	 * NB: In order to avoid race conditions, we must zero pss_barrierCheckMask
+	 * first and only afterwards try to do barrier processing. If we did it
+	 * in the other order, someone could send us another barrier of some
+	 * type right after we called the barrier-processing function but before
+	 * we cleared the bit. We would have no way of knowing that the bit needs
+	 * to stay set in that case, so the need to call the barrier-processing
+	 * function again would just get forgotten. So instead, we tentatively
+	 * clear all the bits and then put back any for which we don't manage
+	 * to successfully absorb the barrier.
 	 */
 	flags = pg_atomic_exchange_u32(&MyProcSignalSlot->pss_barrierCheckMask, 0);
 
 	/*
-	 * Process each type of barrier. It's important that nothing we call from
-	 * here throws an error, because pss_barrierCheckMask has already been
-	 * cleared. If we jumped out of here before processing all barrier types,
-	 * then we'd forget about the need to do so later.
-	 *
-	 * NB: It ought to be OK to call the barrier-processing functions
-	 * unconditionally, but it's more efficient to call only the ones that
-	 * might need us to do something based on the flags.
+	 * If there are no flags set, then we can skip doing any real work.
+	 * Otherwise, establish a PG_TRY block, so that we don't lose track of
+	 * which types of barrier processing are needed if an ERROR occurs.
 	 */
-	if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_PLACEHOLDER))
-		ProcessBarrierPlaceholder();
+	if (flags != 0)
+	{
+		bool	success = true;
+
+		PG_TRY();
+		{
+			/*
+			 * Process each type of barrier. The barrier-processing functions
+			 * should normally return true, but may return false if the barrier
+			 * can't be absorbed at the current time. This should be rare,
+			 * because it's pretty expensive.  Every single
+			 * CHECK_FOR_INTERRUPTS() will return here until we manage to
+			 * absorb the barrier, and that cost will add up in a hurry.
+			 *
+			 * NB: It ought to be OK to call the barrier-processing functions
+			 * unconditionally, but it's more efficient to call only the ones
+			 * that might need us to do something based on the flags.
+			 */
+			while (flags != 0)
+			{
+				ProcSignalBarrierType	type;
+				bool processed = true;
+
+				type = (ProcSignalBarrierType) pg_rightmost_one_pos32(flags);
+				switch (type)
+				{
+					case PROCSIGNAL_BARRIER_PLACEHOLDER:
+						processed = ProcessBarrierPlaceholder();
+						break;
+				}
+
+				/*
+				 * To avoid an infinite loop, we must always unset the bit
+				 * in flags.
+				 */
+				BARRIER_CLEAR_BIT(flags, type);
+
+				/*
+				 * If we failed to process the barrier, reset the shared bit
+				 * so we try again later, and set a flag so that we don't bump
+				 * our generation.
+				 */
+				if (!processed)
+				{
+					ResetProcSignalBarrierBits(((uint32) 1) << type);
+					success = false;
+				}
+			}
+		}
+		PG_CATCH();
+		{
+			/*
+			 * If an ERROR occurred, we'll need to try again later to handle
+			 * that barrier type and any others that haven't been handled yet
+			 * or weren't successfully absorbed.
+			 */
+			ResetProcSignalBarrierBits(flags);
+			PG_RE_THROW();
+		}
+		PG_END_TRY();
+
+		/*
+		 * If some barrier types were not successfully absorbed, we will have
+		 * to try again later.
+		 */
+		if (!success)
+		{
+			ResetProcSignalBarrierBits(flags);
+			return;
+		}
+	}
 
 	/*
 	 * State changes related to all types of barriers that might have been
@@ -508,7 +594,20 @@ ProcessProcSignalBarrier(void)
 	pg_atomic_write_u64(&MyProcSignalSlot->pss_barrierGeneration, shared_gen);
 }
 
+/*
+ * If it turns out that we couldn't absorb one or more barrier types, either
+ * because the barrier-processing functions returned false or due to an error,
+ * arrange for processing to be retried later.
+ */
 static void
+ResetProcSignalBarrierBits(uint32 flags)
+{
+	pg_atomic_fetch_or_u32(&MyProcSignalSlot->pss_barrierCheckMask, flags);
+	ProcSignalBarrierPending = true;
+	InterruptPending = true;
+}
+
+static bool
 ProcessBarrierPlaceholder(void)
 {
 	/*
@@ -518,7 +617,12 @@ ProcessBarrierPlaceholder(void)
 	 * appropriately descriptive. Get rid of this function and instead have
 	 * ProcessBarrierSomethingElse. Most likely, that function should live in
 	 * the file pertaining to that subsystem, rather than here.
+	 *
+	 * The return value should be 'true' if the barrier was successfully
+	 * absorbed and 'false' if not. Note that returning 'false' can lead to
+	 * very frequent retries, so try hard to make that an uncommon case.
 	 */
+	return true;
 }
 
 /*
-- 
2.24.3 (Apple Git-128)

From abf7f6042585c6fc45938fd90f964789eade0d6b Mon Sep 17 00:00:00 2001
From: Robert Haas <rhaas@postgresql.org>
Date: Wed, 7 Oct 2020 13:04:16 -0400
Subject: [PATCH 2/2] Test module for barriers. NOT FOR COMMIT.

---
 contrib/barrier/Makefile         | 23 ++++++++++++
 contrib/barrier/barrier--1.0.sql | 14 +++++++
 contrib/barrier/barrier.c        | 63 ++++++++++++++++++++++++++++++++
 contrib/barrier/barrier.control  |  5 +++
 4 files changed, 105 insertions(+)
 create mode 100644 contrib/barrier/Makefile
 create mode 100644 contrib/barrier/barrier--1.0.sql
 create mode 100644 contrib/barrier/barrier.c
 create mode 100644 contrib/barrier/barrier.control

diff --git a/contrib/barrier/Makefile b/contrib/barrier/Makefile
new file mode 100644
index 0000000000..71f59f6629
--- /dev/null
+++ b/contrib/barrier/Makefile
@@ -0,0 +1,23 @@
+# contrib/barrier/Makefile
+
+MODULE_big = barrier
+OBJS = \
+	$(WIN32RES) \
+	barrier.o
+
+EXTENSION = barrier
+DATA = barrier--1.0.sql
+PGFILEDESC = "barrier - barrier test code NOT FOR COMMIT"
+
+REGRESS = barrier
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/barrier
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/barrier/barrier--1.0.sql b/contrib/barrier/barrier--1.0.sql
new file mode 100644
index 0000000000..66cae976a9
--- /dev/null
+++ b/contrib/barrier/barrier--1.0.sql
@@ -0,0 +1,14 @@
+/* contrib/barrier/barrier--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION barrier" to load this file. \quit
+
+CREATE FUNCTION emit_barrier(barrier_type text, count integer default 1)
+RETURNS void
+AS 'MODULE_PATHNAME', 'emit_barrier'
+LANGUAGE C STRICT;
+
+CREATE FUNCTION wait_barrier(barrier_type text)
+RETURNS void
+AS 'MODULE_PATHNAME', 'wait_barrier'
+LANGUAGE C STRICT;
diff --git a/contrib/barrier/barrier.c b/contrib/barrier/barrier.c
new file mode 100644
index 0000000000..a0b9843992
--- /dev/null
+++ b/contrib/barrier/barrier.c
@@ -0,0 +1,63 @@
+/*-------------------------------------------------------------------------
+ *
+ * barrier.c
+ *	  emit ProcSignalBarriers for testing purposes
+ *
+ * Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ *
+ *	  contrib/barrier/barrier.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "funcapi.h"
+#include "miscadmin.h"
+#include "storage/procsignal.h"
+#include "utils/builtins.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(emit_barrier);
+PG_FUNCTION_INFO_V1(wait_barrier);
+
+static ProcSignalBarrierType
+get_barrier_type(text *barrier_type)
+{
+	char	   *btype = text_to_cstring(barrier_type);
+
+	if (strcmp(btype, "placeholder") == 0)
+		return PROCSIGNAL_BARRIER_PLACEHOLDER;
+
+	elog(ERROR, "unknown barrier type: \"%s\"", btype);
+}
+
+Datum
+emit_barrier(PG_FUNCTION_ARGS)
+{
+	text	   *barrier_type = PG_GETARG_TEXT_PP(0);
+	int32		count = PG_GETARG_INT32(1);
+	int32		i;
+	ProcSignalBarrierType t = get_barrier_type(barrier_type);
+
+	for (i = 0; i < count; ++i)
+	{
+		CHECK_FOR_INTERRUPTS();
+		EmitProcSignalBarrier(t);
+	}
+
+	PG_RETURN_VOID();
+}
+
+Datum
+wait_barrier(PG_FUNCTION_ARGS)
+{
+	text	   *barrier_type = PG_GETARG_TEXT_PP(0);
+	ProcSignalBarrierType t = get_barrier_type(barrier_type);
+	uint64		generation;
+
+	generation = EmitProcSignalBarrier(t);
+	elog(NOTICE, "waiting for barrier");
+	WaitForProcSignalBarrier(generation);
+
+	PG_RETURN_VOID();
+}
diff --git a/contrib/barrier/barrier.control b/contrib/barrier/barrier.control
new file mode 100644
index 0000000000..425ffc1543
--- /dev/null
+++ b/contrib/barrier/barrier.control
@@ -0,0 +1,5 @@
+# barrier extension
+comment = 'emit ProcSignalBarrier for test purposes'
+default_version = '1.0'
+module_pathname = '$libdir/barrier'
+relocatable = true
-- 
2.24.3 (Apple Git-128)

From f71b0841e8403fdc7f3ccb203eccebad15eabc0f Mon Sep 17 00:00:00 2001
From: Robert Haas <rhaas@postgresql.org>
Date: Tue, 6 Oct 2020 15:26:43 -0400
Subject: [PATCH v10 1/6] Allow for error or refusal while absorbing barriers.

Previously, the per-barrier-type functions tasked with absorbing
them were expected to always succeed and never throw an error.
However, that's a bit inconvenient. Further study has revealed that
there are realistic cases where it might not be possible to absorb
a ProcSignalBarrier without terminating the transaction, or even
the whole backend. Similarly, for some barrier types, there might
be other reasons where it's not reasonably possible to absorb the
barrier at certain points in the code, so provide a way for a
per-barrier-type function to reject absorbing the barrier.

Patch by me, reviewed by Andres Freund.

Discussion: http://postgr.es/m/20200908182005.xya7wetdh3pndzim@alap3.anarazel.de
---
 src/backend/storage/ipc/procsignal.c | 128 ++++++++++++++++++++++++---
 1 file changed, 116 insertions(+), 12 deletions(-)

diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index ffe67acea1c..abdae58c476 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -18,6 +18,7 @@
 #include <unistd.h>
 
 #include "access/parallel.h"
+#include "port/pg_bitutils.h"
 #include "commands/async.h"
 #include "miscadmin.h"
 #include "pgstat.h"
@@ -87,12 +88,17 @@ typedef struct
 #define BARRIER_SHOULD_CHECK(flags, type) \
 	(((flags) & (((uint32) 1) << (uint32) (type))) != 0)
 
+/* Clear the relevant type bit from the flags. */
+#define BARRIER_CLEAR_BIT(flags, type) \
+	((flags) &= ~(((uint32) 1) << (uint32) (type)))
+
 static ProcSignalHeader *ProcSignal = NULL;
 static volatile ProcSignalSlot *MyProcSignalSlot = NULL;
 
 static bool CheckProcSignal(ProcSignalReason reason);
 static void CleanupProcSignalState(int status, Datum arg);
-static void ProcessBarrierPlaceholder(void);
+static void ResetProcSignalBarrierBits(uint32 flags);
+static bool ProcessBarrierPlaceholder(void);
 
 /*
  * ProcSignalShmemSize
@@ -394,6 +400,12 @@ WaitForProcSignalBarrier(uint64 generation)
 		volatile ProcSignalSlot *slot = &ProcSignal->psh_slot[i];
 		uint64		oldval;
 
+		/*
+		 * It's important that we check only pss_barrierGeneration here and
+		 * not pss_barrierCheckMask. Bits in pss_barrierCheckMask get cleared
+		 * before the barrier is actually absorbed, but pss_barrierGeneration
+		 * is updated only afterward.
+		 */
 		oldval = pg_atomic_read_u64(&slot->pss_barrierGeneration);
 		while (oldval < generation)
 		{
@@ -453,7 +465,7 @@ ProcessProcSignalBarrier(void)
 {
 	uint64		local_gen;
 	uint64		shared_gen;
-	uint32		flags;
+	volatile uint32		flags;
 
 	Assert(MyProcSignalSlot);
 
@@ -482,21 +494,95 @@ ProcessProcSignalBarrier(void)
 	 * read of the barrier generation above happens before we atomically
 	 * extract the flags, and that any subsequent state changes happen
 	 * afterward.
+	 *
+	 * NB: In order to avoid race conditions, we must zero pss_barrierCheckMask
+	 * first and only afterwards try to do barrier processing. If we did it
+	 * in the other order, someone could send us another barrier of some
+	 * type right after we called the barrier-processing function but before
+	 * we cleared the bit. We would have no way of knowing that the bit needs
+	 * to stay set in that case, so the need to call the barrier-processing
+	 * function again would just get forgotten. So instead, we tentatively
+	 * clear all the bits and then put back any for which we don't manage
+	 * to successfully absorb the barrier.
 	 */
 	flags = pg_atomic_exchange_u32(&MyProcSignalSlot->pss_barrierCheckMask, 0);
 
 	/*
-	 * Process each type of barrier. It's important that nothing we call from
-	 * here throws an error, because pss_barrierCheckMask has already been
-	 * cleared. If we jumped out of here before processing all barrier types,
-	 * then we'd forget about the need to do so later.
-	 *
-	 * NB: It ought to be OK to call the barrier-processing functions
-	 * unconditionally, but it's more efficient to call only the ones that
-	 * might need us to do something based on the flags.
+	 * If there are no flags set, then we can skip doing any real work.
+	 * Otherwise, establish a PG_TRY block, so that we don't lose track of
+	 * which types of barrier processing are needed if an ERROR occurs.
 	 */
-	if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_PLACEHOLDER))
-		ProcessBarrierPlaceholder();
+	if (flags != 0)
+	{
+		bool	success = true;
+
+		PG_TRY();
+		{
+			/*
+			 * Process each type of barrier. The barrier-processing functions
+			 * should normally return true, but may return false if the barrier
+			 * can't be absorbed at the current time. This should be rare,
+			 * because it's pretty expensive.  Every single
+			 * CHECK_FOR_INTERRUPTS() will return here until we manage to
+			 * absorb the barrier, and that cost will add up in a hurry.
+			 *
+			 * NB: It ought to be OK to call the barrier-processing functions
+			 * unconditionally, but it's more efficient to call only the ones
+			 * that might need us to do something based on the flags.
+			 */
+			while (flags != 0)
+			{
+				ProcSignalBarrierType	type;
+				bool processed = true;
+
+				type = (ProcSignalBarrierType) pg_rightmost_one_pos32(flags);
+				switch (type)
+				{
+					case PROCSIGNAL_BARRIER_PLACEHOLDER:
+						processed = ProcessBarrierPlaceholder();
+						break;
+				}
+
+				/*
+				 * To avoid an infinite loop, we must always unset the bit
+				 * in flags.
+				 */
+				BARRIER_CLEAR_BIT(flags, type);
+
+				/*
+				 * If we failed to process the barrier, reset the shared bit
+				 * so we try again later, and set a flag so that we don't bump
+				 * our generation.
+				 */
+				if (!processed)
+				{
+					ResetProcSignalBarrierBits(((uint32) 1) << type);
+					success = false;
+				}
+			}
+		}
+		PG_CATCH();
+		{
+			/*
+			 * If an ERROR occurred, we'll need to try again later to handle
+			 * that barrier type and any others that haven't been handled yet
+			 * or weren't successfully absorbed.
+			 */
+			ResetProcSignalBarrierBits(flags);
+			PG_RE_THROW();
+		}
+		PG_END_TRY();
+
+		/*
+		 * If some barrier types were not successfully absorbed, we will have
+		 * to try again later.
+		 */
+		if (!success)
+		{
+			ResetProcSignalBarrierBits(flags);
+			return;
+		}
+	}
 
 	/*
 	 * State changes related to all types of barriers that might have been
@@ -508,7 +594,20 @@ ProcessProcSignalBarrier(void)
 	pg_atomic_write_u64(&MyProcSignalSlot->pss_barrierGeneration, shared_gen);
 }
 
+/*
+ * If it turns out that we couldn't absorb one or more barrier types, either
+ * because the barrier-processing functions returned false or due to an error,
+ * arrange for processing to be retried later.
+ */
 static void
+ResetProcSignalBarrierBits(uint32 flags)
+{
+	pg_atomic_fetch_or_u32(&MyProcSignalSlot->pss_barrierCheckMask, flags);
+	ProcSignalBarrierPending = true;
+	InterruptPending = true;
+}
+
+static bool
 ProcessBarrierPlaceholder(void)
 {
 	/*
@@ -518,7 +617,12 @@ ProcessBarrierPlaceholder(void)
 	 * appropriately descriptive. Get rid of this function and instead have
 	 * ProcessBarrierSomethingElse. Most likely, that function should live in
 	 * the file pertaining to that subsystem, rather than here.
+	 *
+	 * The return value should be 'true' if the barrier was successfully
+	 * absorbed and 'false' if not. Note that returning 'false' can lead to
+	 * very frequent retries, so try hard to make that an uncommon case.
 	 */
+	return true;
 }
 
 /*
-- 
2.18.0

From 9f19c93f11de7111b9c85b3290414062f8f6e8ca Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Mon, 27 Jul 2020 02:13:36 -0400
Subject: [PATCH v10 5/6] Error or Assert before START_CRIT_SECTION for WAL
 write

Based on the following criteria added the Assert or the Error when system is
prohibited:

 - Added ERROR for the function which can be reachable without valid XID in case
   of VACUUM or CONCURRENT CREATE INDEX.  For that added common static inline
   function CheckWALPermitted().
 - Added ASSERT for the function which cannot be reached without valid XID;
   Assert to ensure XID validation. For that added AssertWALPermitted_HaveXID().

To enforce the rule to have aforesaid assert or error check before entering a
critical section for WAL write, a new walpermit_checked_state assert only flag
is added. If these check is missing then XLogBeginInsert() will have an
assertion if it is in critical section.

If we are not doing WAL insert inside the critical section then the above
checking is not necessary, we can rely on XLogBeginInsert() for that check &
report an error.
---
 contrib/pg_surgery/heap_surgery.c         | 10 ++++--
 src/backend/access/brin/brin.c            |  4 +++
 src/backend/access/brin/brin_pageops.c    | 21 ++++++++++--
 src/backend/access/brin/brin_revmap.c     | 10 +++++-
 src/backend/access/gin/ginbtree.c         | 15 +++++++--
 src/backend/access/gin/gindatapage.c      | 18 +++++++++--
 src/backend/access/gin/ginfast.c          | 11 +++++--
 src/backend/access/gin/gininsert.c        |  4 +++
 src/backend/access/gin/ginutil.c          |  9 +++++-
 src/backend/access/gin/ginvacuum.c        | 11 ++++++-
 src/backend/access/gist/gist.c            | 25 ++++++++++++---
 src/backend/access/gist/gistvacuum.c      | 13 ++++++--
 src/backend/access/hash/hash.c            | 19 +++++++++--
 src/backend/access/hash/hashinsert.c      |  9 +++++-
 src/backend/access/hash/hashovfl.c        | 22 ++++++++++---
 src/backend/access/hash/hashpage.c        |  9 ++++++
 src/backend/access/heap/heapam.c          | 26 ++++++++++++++-
 src/backend/access/heap/pruneheap.c       | 12 +++++--
 src/backend/access/heap/vacuumlazy.c      | 18 +++++++++--
 src/backend/access/heap/visibilitymap.c   | 22 +++++++++++--
 src/backend/access/nbtree/nbtdedup.c      |  3 ++
 src/backend/access/nbtree/nbtinsert.c     | 10 +++++-
 src/backend/access/nbtree/nbtpage.c       | 39 +++++++++++++++++++----
 src/backend/access/spgist/spgdoinsert.c   | 13 ++++++++
 src/backend/access/spgist/spgvacuum.c     | 22 +++++++++++--
 src/backend/access/transam/multixact.c    |  5 ++-
 src/backend/access/transam/twophase.c     |  9 ++++++
 src/backend/access/transam/varsup.c       |  4 +++
 src/backend/access/transam/walprohibit.c  | 10 ++++++
 src/backend/access/transam/xact.c         |  6 ++++
 src/backend/access/transam/xlog.c         | 27 ++++++++++++----
 src/backend/access/transam/xloginsert.c   | 13 ++++++--
 src/backend/commands/sequence.c           | 16 ++++++++++
 src/backend/commands/variable.c           |  9 ++++--
 src/backend/postmaster/checkpointer.c     |  4 +++
 src/backend/storage/buffer/bufmgr.c       | 10 +++---
 src/backend/storage/freespace/freespace.c | 10 +++++-
 src/backend/storage/lmgr/lock.c           |  6 ++--
 src/backend/utils/cache/relmapper.c       |  3 ++
 src/include/miscadmin.h                   | 27 ++++++++++++++++
 40 files changed, 463 insertions(+), 71 deletions(-)

diff --git a/contrib/pg_surgery/heap_surgery.c b/contrib/pg_surgery/heap_surgery.c
index eb96b4bb36d..53d8c9cea28 100644
--- a/contrib/pg_surgery/heap_surgery.c
+++ b/contrib/pg_surgery/heap_surgery.c
@@ -14,6 +14,7 @@
 
 #include "access/heapam.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "catalog/pg_am_d.h"
 #include "catalog/pg_proc_d.h"
 #include "miscadmin.h"
@@ -89,6 +90,7 @@ heap_force_common(FunctionCallInfo fcinfo, HeapTupleForceOption heap_force_opt)
 	OffsetNumber curr_start_ptr,
 				next_start_ptr;
 	bool		include_this_tid[MaxHeapTuplesPerPage];
+	bool		needwal;
 
 	if (RecoveryInProgress())
 		ereport(ERROR,
@@ -100,6 +102,7 @@ heap_force_common(FunctionCallInfo fcinfo, HeapTupleForceOption heap_force_opt)
 	sanity_check_tid_array(ta, &ntids);
 
 	rel = relation_open(relid, RowExclusiveLock);
+	needwal = RelationNeedsWAL(rel);
 
 	/* Check target relation. */
 	sanity_check_relation(rel);
@@ -217,6 +220,9 @@ heap_force_common(FunctionCallInfo fcinfo, HeapTupleForceOption heap_force_opt)
 		if (heap_force_opt == HEAP_FORCE_KILL && PageIsAllVisible(page))
 			visibilitymap_pin(rel, blkno, &vmbuf);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* No ereport(ERROR) from here until all the changes are logged. */
 		START_CRIT_SECTION();
 
@@ -297,12 +303,12 @@ heap_force_common(FunctionCallInfo fcinfo, HeapTupleForceOption heap_force_opt)
 			MarkBufferDirty(buf);
 
 			/* XLOG stuff */
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 				log_newpage_buffer(buf, true);
 		}
 
 		/* WAL log the VM page if it was modified. */
-		if (did_modify_vm && RelationNeedsWAL(rel))
+		if (did_modify_vm && needwal)
 			log_newpage_buffer(vmbuf, false);
 
 		END_CRIT_SECTION();
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index 1f72562c603..47142193706 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -24,6 +24,7 @@
 #include "access/relscan.h"
 #include "access/table.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/index.h"
 #include "catalog/pg_am.h"
@@ -759,6 +760,9 @@ brinbuildempty(Relation index)
 		ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermittedHaveXID();
+
 	/* Initialize and xlog metabuffer. */
 	START_CRIT_SECTION();
 	brin_metapage_init(BufferGetPage(metabuf), BrinGetPagesPerRange(index),
diff --git a/src/backend/access/brin/brin_pageops.c b/src/backend/access/brin/brin_pageops.c
index 87de0b855b5..8b377a679ab 100644
--- a/src/backend/access/brin/brin_pageops.c
+++ b/src/backend/access/brin/brin_pageops.c
@@ -14,6 +14,7 @@
 #include "access/brin_pageops.h"
 #include "access/brin_revmap.h"
 #include "access/brin_xlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -65,6 +66,7 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 	Buffer		newbuf;
 	BlockNumber newblk = InvalidBlockNumber;
 	bool		extended;
+	bool		needwal = RelationNeedsWAL(idxrel);
 
 	Assert(newsz == MAXALIGN(newsz));
 
@@ -176,13 +178,16 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 	if (((BrinPageFlags(oldpage) & BRIN_EVACUATE_PAGE) == 0) &&
 		brin_can_do_samepage_update(oldbuf, origsz, newsz))
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 		if (!PageIndexTupleOverwrite(oldpage, oldoff, (Item) unconstify(BrinTuple *, newtup), newsz))
 			elog(ERROR, "failed to replace BRIN tuple");
 		MarkBufferDirty(oldbuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(idxrel))
+		if (needwal)
 		{
 			xl_brin_samepage_update xlrec;
 			XLogRecPtr	recptr;
@@ -240,6 +245,9 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 
 		revmapbuf = brinLockRevmapPageForUpdate(revmap, heapBlk);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		/*
@@ -267,7 +275,7 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 		MarkBufferDirty(revmapbuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(idxrel))
+		if (needwal)
 		{
 			xl_brin_update xlrec;
 			XLogRecPtr	recptr;
@@ -351,6 +359,7 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	Buffer		revmapbuf;
 	ItemPointerData tid;
 	bool		extended;
+	bool		needwal;
 
 	Assert(itemsz == MAXALIGN(itemsz));
 
@@ -405,6 +414,10 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	page = BufferGetPage(*buffer);
 	blk = BufferGetBlockNumber(*buffer);
 
+	needwal = RelationNeedsWAL(idxrel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Execute the actual insertion */
 	START_CRIT_SECTION();
 	if (extended)
@@ -424,7 +437,7 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	MarkBufferDirty(revmapbuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(idxrel))
+	if (needwal)
 	{
 		xl_brin_insert xlrec;
 		XLogRecPtr	recptr;
@@ -881,6 +894,8 @@ brin_initialize_empty_new_buffer(Relation idxrel, Buffer buffer)
 			   "brin_initialize_empty_new_buffer: initializing blank page %u",
 			   BufferGetBlockNumber(buffer)));
 
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 	page = BufferGetPage(buffer);
 	brin_page_init(page, BRIN_PAGETYPE_REGULAR);
diff --git a/src/backend/access/brin/brin_revmap.c b/src/backend/access/brin/brin_revmap.c
index 35746714a7c..fd766da445d 100644
--- a/src/backend/access/brin/brin_revmap.c
+++ b/src/backend/access/brin/brin_revmap.c
@@ -26,6 +26,7 @@
 #include "access/brin_tuple.h"
 #include "access/brin_xlog.h"
 #include "access/rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -340,6 +341,7 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 	OffsetNumber revmapOffset;
 	OffsetNumber regOffset;
 	ItemId		lp;
+	bool		needwal;
 
 	revmap = brinRevmapInitialize(idxrel, &pagesPerRange, NULL);
 
@@ -404,6 +406,10 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 	 * crashed or aborted summarization; remove them silently.
 	 */
 
+	needwal = RelationNeedsWAL(idxrel);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	ItemPointerSetInvalid(&invalidIptr);
@@ -415,7 +421,7 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 	MarkBufferDirty(regBuf);
 	MarkBufferDirty(revmapBuf);
 
-	if (RelationNeedsWAL(idxrel))
+	if (needwal)
 	{
 		xl_brin_desummarize xlrec;
 		XLogRecPtr	recptr;
@@ -613,6 +619,8 @@ revmap_physical_extend(BrinRevmap *revmap)
 		return;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Ok, we have now locked the metapage and the target block. Re-initialize
 	 * the target block as a revmap page, and update the metapage.
diff --git a/src/backend/access/gin/ginbtree.c b/src/backend/access/gin/ginbtree.c
index 82788a5c367..f31590dcd75 100644
--- a/src/backend/access/gin/ginbtree.c
+++ b/src/backend/access/gin/ginbtree.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/predicate.h"
@@ -332,6 +333,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 {
 	Page		page = BufferGetPage(stack->buffer);
 	bool		result;
+	bool		needwal;
 	GinPlaceToPageRC rc;
 	uint16		xlflags = 0;
 	Page		childpage = NULL;
@@ -377,6 +379,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 								 insertdata, updateblkno,
 								 &ptp_workspace,
 								 &newlpage, &newrpage);
+	needwal = RelationNeedsWAL(btree->index) && !btree->isBuild;
 
 	if (rc == GPTP_NO_WORK)
 	{
@@ -385,10 +388,13 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 	}
 	else if (rc == GPTP_INSERT)
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		/* It will fit, perform the insertion */
 		START_CRIT_SECTION();
 
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogBeginInsert();
 			XLogRegisterBuffer(0, stack->buffer, REGBUF_STANDARD);
@@ -409,7 +415,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 			MarkBufferDirty(childbuf);
 		}
 
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 			ginxlogInsert xlrec;
@@ -547,6 +553,9 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 			}
 		}
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/*
 		 * OK, we have the new contents of the left page in a temporary copy
 		 * now (newlpage), and likewise for the new contents of the
@@ -587,7 +596,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 		}
 
 		/* write WAL record */
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gin/gindatapage.c b/src/backend/access/gin/gindatapage.c
index 7a2690e97f2..0abc5990100 100644
--- a/src/backend/access/gin/gindatapage.c
+++ b/src/backend/access/gin/gindatapage.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "lib/ilist.h"
 #include "miscadmin.h"
@@ -811,6 +812,7 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 	if (removedsomething)
 	{
 		bool		modified;
+		bool		needwal;
 
 		/*
 		 * Make sure we have a palloc'd copy of all segments, after the first
@@ -835,8 +837,12 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 			}
 		}
 
-		if (RelationNeedsWAL(indexrel))
+		needwal = RelationNeedsWAL(indexrel);
+		if (needwal)
+		{
+			CheckWALPermitted();
 			computeLeafRecompressWALData(leaf);
+		}
 
 		/* Apply changes to page */
 		START_CRIT_SECTION();
@@ -845,7 +851,7 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 
 		MarkBufferDirty(buffer);
 
-		if (RelationNeedsWAL(indexrel))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
@@ -1777,6 +1783,7 @@ createPostingTree(Relation index, ItemPointerData *items, uint32 nitems,
 	int			nrootitems;
 	int			rootsize;
 	bool		is_build = (buildStats != NULL);
+	bool		needwal;
 
 	/* Construct the new root page in memory first. */
 	tmppage = (Page) palloc(BLCKSZ);
@@ -1825,12 +1832,17 @@ createPostingTree(Relation index, ItemPointerData *items, uint32 nitems,
 	 */
 	PredicateLockPageSplit(index, BufferGetBlockNumber(entrybuffer), blkno);
 
+	needwal = RelationNeedsWAL(index) && !is_build;
+
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	PageRestoreTempPage(tmppage, page);
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index) && !is_build)
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogCreatePostingTree data;
diff --git a/src/backend/access/gin/ginfast.c b/src/backend/access/gin/ginfast.c
index 2e41b34d8d5..b8c2a993408 100644
--- a/src/backend/access/gin/ginfast.c
+++ b/src/backend/access/gin/ginfast.c
@@ -20,6 +20,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
 #include "catalog/pg_am.h"
@@ -68,6 +69,8 @@ writeListPage(Relation index, Buffer buffer,
 	PGAlignedBlock workspace;
 	char	   *ptr;
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	GinInitBuffer(buffer, GIN_LIST);
@@ -548,6 +551,7 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 	Page		metapage;
 	GinMetaPageData *metadata;
 	BlockNumber blknoToDelete;
+	bool		needwal = RelationNeedsWAL(index);
 
 	metapage = BufferGetPage(metabuffer);
 	metadata = GinPageGetMeta(metapage);
@@ -586,8 +590,11 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 		 * prepare the XLogInsert machinery for that before entering the
 		 * critical section.
 		 */
-		if (RelationNeedsWAL(index))
+		if (needwal)
+		{
+			CheckWALPermitted();
 			XLogEnsureRecordSpace(data.ndeleted, 0);
+		}
 
 		START_CRIT_SECTION();
 
@@ -625,7 +632,7 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 			MarkBufferDirty(buffers[i]);
 		}
 
-		if (RelationNeedsWAL(index))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gin/gininsert.c b/src/backend/access/gin/gininsert.c
index 77433dc8a41..989d82ffcaf 100644
--- a/src/backend/access/gin/gininsert.c
+++ b/src/backend/access/gin/gininsert.c
@@ -17,6 +17,7 @@
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/index.h"
 #include "miscadmin.h"
@@ -447,6 +448,9 @@ ginbuildempty(Relation index)
 		ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(RootBuffer, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermittedHaveXID();
+
 	/* Initialize and xlog metabuffer and root buffer. */
 	START_CRIT_SECTION();
 	GinInitMetabuffer(MetaBuffer);
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index ef9b56fd363..b48ea1a746a 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -17,6 +17,7 @@
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
 #include "access/reloptions.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/pg_collation.h"
 #include "catalog/pg_type.h"
@@ -659,12 +660,18 @@ ginUpdateStats(Relation index, const GinStatsData *stats, bool is_build)
 	Buffer		metabuffer;
 	Page		metapage;
 	GinMetaPageData *metadata;
+	bool		needwal;
 
 	metabuffer = ReadBuffer(index, GIN_METAPAGE_BLKNO);
 	LockBuffer(metabuffer, GIN_EXCLUSIVE);
 	metapage = BufferGetPage(metabuffer);
 	metadata = GinPageGetMeta(metapage);
 
+	needwal = RelationNeedsWAL(index) && !is_build;
+
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	metadata->nTotalPages = stats->nTotalPages;
@@ -684,7 +691,7 @@ ginUpdateStats(Relation index, const GinStatsData *stats, bool is_build)
 
 	MarkBufferDirty(metabuffer);
 
-	if (RelationNeedsWAL(index) && !is_build)
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogUpdateMeta data;
diff --git a/src/backend/access/gin/ginvacuum.c b/src/backend/access/gin/ginvacuum.c
index 0935a6d9e53..d91ca2b391c 100644
--- a/src/backend/access/gin/ginvacuum.c
+++ b/src/backend/access/gin/ginvacuum.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "commands/vacuum.h"
 #include "miscadmin.h"
@@ -136,6 +137,7 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	Page		page,
 				parentPage;
 	BlockNumber rightlink;
+	bool		needwal;
 
 	/*
 	 * This function MUST be called only if someone of parent pages hold
@@ -159,6 +161,10 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	 */
 	PredicateLockPageCombine(gvs->index, deleteBlkno, rightlink);
 
+	needwal = RelationNeedsWAL(gvs->index);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* Unlink the page by changing left sibling's rightlink */
@@ -195,7 +201,7 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	MarkBufferDirty(lBuffer);
 	MarkBufferDirty(dBuffer);
 
-	if (RelationNeedsWAL(gvs->index))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogDeletePage data;
@@ -650,6 +656,9 @@ ginbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 
 		if (resPage)
 		{
+			if (RelationNeedsWAL(gvs.index))
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 			PageRestoreTempPage(resPage, page);
 			MarkBufferDirty(buffer);
diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c
index 25b42e38f22..4a870a062ba 100644
--- a/src/backend/access/gist/gist.c
+++ b/src/backend/access/gist/gist.c
@@ -16,6 +16,7 @@
 
 #include "access/gist_private.h"
 #include "access/gistscan.h"
+#include "access/walprohibit.h"
 #include "catalog/pg_collation.h"
 #include "commands/vacuum.h"
 #include "miscadmin.h"
@@ -135,6 +136,9 @@ gistbuildempty(Relation index)
 	buffer = ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermittedHaveXID();
+
 	/* Initialize and xlog buffer */
 	START_CRIT_SECTION();
 	GISTInitBuffer(buffer, F_LEAF);
@@ -234,6 +238,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 	XLogRecPtr	recptr;
 	int			i;
 	bool		is_split;
+	bool		needwal = RelationNeedsWAL(rel);
 
 	/*
 	 * Refuse to modify a page that's incompletely split. This should not
@@ -465,9 +470,12 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 		 * insertion for that. NB: The number of pages and data segments
 		 * specified here must match the calculations in gistXLogSplit()!
 		 */
-		if (!is_build && RelationNeedsWAL(rel))
+		if (!is_build && needwal)
 			XLogEnsureRecordSpace(npage, 1 + npage * 2);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		/*
@@ -500,7 +508,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 			recptr = GistBuildLSN;
 		else
 		{
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 				recptr = gistXLogSplit(is_leaf,
 									   dist, oldrlink, oldnsn, leftchildbuf,
 									   markfollowright);
@@ -526,6 +534,9 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 	}
 	else
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		/*
 		 * Enough space.  We always get here if ntup==0.
 		 */
@@ -567,7 +578,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 			recptr = GistBuildLSN;
 		else
 		{
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 			{
 				OffsetNumber ndeloffs = 0,
 							deloffs[1];
@@ -1641,6 +1652,7 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 	OffsetNumber offnum,
 				maxoff;
 	TransactionId latestRemovedXid = InvalidTransactionId;
+	bool		needwal = RelationNeedsWAL(rel);
 
 	Assert(GistPageIsLeaf(page));
 
@@ -1659,13 +1671,16 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 			deletable[ndeletable++] = offnum;
 	}
 
-	if (XLogStandbyInfoActive() && RelationNeedsWAL(rel))
+	if (XLogStandbyInfoActive() && needwal)
 		latestRemovedXid =
 			index_compute_xid_horizon_for_tuples(rel, heapRel, buffer,
 												 deletable, ndeletable);
 
 	if (ndeletable > 0)
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		PageIndexMultiDelete(page, deletable, ndeletable);
@@ -1682,7 +1697,7 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 		MarkBufferDirty(buffer);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gist/gistvacuum.c b/src/backend/access/gist/gistvacuum.c
index a9c616c7724..bbb3ebb19ad 100644
--- a/src/backend/access/gist/gistvacuum.c
+++ b/src/backend/access/gist/gistvacuum.c
@@ -17,6 +17,7 @@
 #include "access/genam.h"
 #include "access/gist_private.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "commands/vacuum.h"
 #include "lib/integerset.h"
 #include "miscadmin.h"
@@ -260,6 +261,7 @@ gistvacuumpage(GistVacState *vstate, BlockNumber blkno, BlockNumber orig_blkno)
 	Buffer		buffer;
 	Page		page;
 	BlockNumber recurse_to;
+	bool		needwal = RelationNeedsWAL(rel);
 
 restart:
 	recurse_to = InvalidBlockNumber;
@@ -341,6 +343,9 @@ restart:
 		 */
 		if (ntodelete > 0)
 		{
+			if (needwal)
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 
 			MarkBufferDirty(buffer);
@@ -348,7 +353,7 @@ restart:
 			PageIndexMultiDelete(page, todelete, ntodelete);
 			GistMarkTuplesDeleted(page);
 
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 			{
 				XLogRecPtr	recptr;
 
@@ -580,6 +585,7 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	IndexTuple	idxtuple;
 	XLogRecPtr	recptr;
 	FullTransactionId txid;
+	bool		needwal = RelationNeedsWAL(info->index);
 
 	/*
 	 * Check that the leaf is still empty and deletable.
@@ -634,6 +640,9 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	 */
 	txid = ReadNextFullTransactionId();
 
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* mark the page as deleted */
@@ -645,7 +654,7 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	MarkBufferDirty(parentBuffer);
 	PageIndexTupleDelete(parentPage, downlink);
 
-	if (RelationNeedsWAL(info->index))
+	if (needwal)
 		recptr = gistXLogPageDelete(leafBuffer, txid, parentBuffer, downlink);
 	else
 		recptr = gistGetFakeLSN(info->index);
diff --git a/src/backend/access/hash/hash.c b/src/backend/access/hash/hash.c
index 7c9ccf446c8..f4903a43bb5 100644
--- a/src/backend/access/hash/hash.c
+++ b/src/backend/access/hash/hash.c
@@ -22,6 +22,7 @@
 #include "access/hash_xlog.h"
 #include "access/relscan.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "catalog/index.h"
 #include "commands/progress.h"
 #include "commands/vacuum.h"
@@ -467,6 +468,7 @@ hashbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	Buffer		metabuf = InvalidBuffer;
 	HashMetaPage metap;
 	HashMetaPage cachedmetap;
+	bool		needwal;
 
 	tuples_removed = 0;
 	num_index_tuples = 0;
@@ -573,6 +575,10 @@ loop_top:
 		goto loop_top;
 	}
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Okay, we're really done.  Update tuple count in metapage. */
 	START_CRIT_SECTION();
 
@@ -603,7 +609,7 @@ loop_top:
 	MarkBufferDirty(metabuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_hash_update_meta_page xlrec;
 		XLogRecPtr	recptr;
@@ -690,6 +696,7 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 	Buffer		buf;
 	Bucket		new_bucket PG_USED_FOR_ASSERTS_ONLY = InvalidBucket;
 	bool		bucket_dirty = false;
+	bool		needwal = RelationNeedsWAL(rel);
 
 	blkno = bucket_blkno;
 	buf = bucket_buf;
@@ -788,6 +795,9 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		 */
 		if (ndeletable > 0)
 		{
+			if (needwal)
+				CheckWALPermitted();
+
 			/* No ereport(ERROR) until changes are logged */
 			START_CRIT_SECTION();
 
@@ -809,7 +819,7 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 			MarkBufferDirty(buf);
 
 			/* XLOG stuff */
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 			{
 				xl_hash_delete xlrec;
 				XLogRecPtr	recptr;
@@ -883,6 +893,9 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		page = BufferGetPage(bucket_buf);
 		bucket_opaque = (HashPageOpaque) PageGetSpecialPointer(page);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -890,7 +903,7 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		MarkBufferDirty(bucket_buf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c
index 2ebe671967b..2eab69efa91 100644
--- a/src/backend/access/hash/hashinsert.c
+++ b/src/backend/access/hash/hashinsert.c
@@ -17,6 +17,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "storage/buf_internals.h"
 #include "storage/lwlock.h"
@@ -193,6 +194,8 @@ restart_insert:
 	 */
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	AssertWALPermittedHaveXID();
+
 	/* Do the update.  No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -360,6 +363,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 	if (ndeletable > 0)
 	{
 		TransactionId latestRemovedXid;
+		bool		needwal = RelationNeedsWAL(rel);
 
 		latestRemovedXid =
 			index_compute_xid_horizon_for_tuples(rel, hrel, buf,
@@ -370,6 +374,9 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 		 */
 		LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -393,7 +400,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 		MarkBufferDirty(metabuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			xl_hash_vacuum_one_page xlrec;
 			XLogRecPtr	recptr;
diff --git a/src/backend/access/hash/hashovfl.c b/src/backend/access/hash/hashovfl.c
index 00f0a940116..e7c5dd3e3ce 100644
--- a/src/backend/access/hash/hashovfl.c
+++ b/src/backend/access/hash/hashovfl.c
@@ -19,6 +19,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "utils/rel.h"
 
@@ -312,6 +313,8 @@ _hash_addovflpage(Relation rel, Buffer metabuf, Buffer buf, bool retain_pin)
 
 found:
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Do the update.  No ereport(ERROR) until changes are logged. We want to
 	 * log the changes for bitmap page and overflow page together to avoid
@@ -510,6 +513,7 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	Buffer		prevbuf = InvalidBuffer;
 	Buffer		nextbuf = InvalidBuffer;
 	bool		update_metap = false;
+	bool		needwal;
 
 	/* Get information from the doomed page */
 	_hash_checkpage(rel, ovflbuf, LH_OVERFLOW_PAGE);
@@ -573,10 +577,15 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	/* Get write-lock on metapage to update firstfree */
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	needwal = RelationNeedsWAL(rel);
+
 	/* This operation needs to log multiple tuples, prepare WAL for that */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 		XLogEnsureRecordSpace(HASH_XLOG_FREE_OVFL_BUFS, 4 + nitups);
 
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -641,7 +650,7 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	}
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_hash_squeeze_page xlrec;
 		XLogRecPtr	recptr;
@@ -922,14 +931,19 @@ readpage:
 
 				if (nitups > 0)
 				{
+					bool		needwal = RelationNeedsWAL(rel);
+
 					Assert(nitups == ndeletable);
 
 					/*
 					 * This operation needs to log multiple tuples, prepare
 					 * WAL for that.
 					 */
-					if (RelationNeedsWAL(rel))
+					if (needwal)
+					{
+						CheckWALPermitted();
 						XLogEnsureRecordSpace(0, 3 + nitups);
+					}
 
 					START_CRIT_SECTION();
 
@@ -947,7 +961,7 @@ readpage:
 					MarkBufferDirty(rbuf);
 
 					/* XLOG stuff */
-					if (RelationNeedsWAL(rel))
+					if (needwal)
 					{
 						XLogRecPtr	recptr;
 						xl_hash_move_page_contents xlrec;
diff --git a/src/backend/access/hash/hashpage.c b/src/backend/access/hash/hashpage.c
index a664ecf494a..55a867dd375 100644
--- a/src/backend/access/hash/hashpage.c
+++ b/src/backend/access/hash/hashpage.c
@@ -30,6 +30,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "port/pg_bitutils.h"
 #include "storage/lmgr.h"
@@ -816,6 +817,8 @@ restart_expand:
 		goto fail;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Since we are scribbling on the pages in the shared buffers, establish a
 	 * critical section.  Any failure in this next code leaves us with a big
@@ -1172,6 +1175,8 @@ _hash_splitbucket(Relation rel,
 
 				if (PageGetFreeSpaceForMultipleTuples(npage, nitups + 1) < (all_tups_size + itemsz))
 				{
+					AssertWALPermittedHaveXID();
+
 					/*
 					 * Change the shared buffer state in critical section,
 					 * otherwise any error could make it unrecoverable.
@@ -1223,6 +1228,8 @@ _hash_splitbucket(Relation rel,
 		/* Exit loop if no more overflow pages in old bucket */
 		if (!BlockNumberIsValid(oblkno))
 		{
+			AssertWALPermittedHaveXID();
+
 			/*
 			 * Change the shared buffer state in critical section, otherwise
 			 * any error could make it unrecoverable.
@@ -1269,6 +1276,8 @@ _hash_splitbucket(Relation rel,
 	npage = BufferGetPage(bucket_nbuf);
 	nopaque = (HashPageOpaque) PageGetSpecialPointer(npage);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	oopaque->hasho_flag &= ~LH_BUCKET_BEING_SPLIT;
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 1585861a021..4d6052224fa 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -47,6 +47,7 @@
 #include "access/transam.h"
 #include "access/valid.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1898,6 +1899,8 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
 	 */
 	CheckForSerializableConflictIn(relation, NULL, InvalidBlockNumber);
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -2172,6 +2175,8 @@ heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples,
 										   &vmbuffer, NULL);
 		page = BufferGetPage(buffer);
 
+		AssertWALPermittedHaveXID();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -2690,6 +2695,8 @@ l1:
 							  xid, LockTupleExclusive, true,
 							  &new_xmax, &new_infomask, &new_infomask2);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -3442,6 +3449,8 @@ l2:
 
 		Assert(HEAP_XMAX_IS_LOCKED_ONLY(infomask_lock_old_tuple));
 
+		AssertWALPermittedHaveXID();
+
 		START_CRIT_SECTION();
 
 		/* Clear obsolete visibility flags ... */
@@ -3615,6 +3624,8 @@ l2:
 										   bms_overlap(modified_attrs, id_attrs),
 										   &old_key_copied);
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -4548,6 +4559,8 @@ failed:
 							  GetCurrentTransactionId(), mode, false,
 							  &xid, &new_infomask, &new_infomask2);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -5339,6 +5352,8 @@ l4:
 								VISIBILITYMAP_ALL_FROZEN))
 			cleared_all_frozen = true;
 
+		AssertWALPermittedHaveXID();
+
 		START_CRIT_SECTION();
 
 		/* ... and set them */
@@ -5497,6 +5512,8 @@ heap_finish_speculative(Relation relation, ItemPointer tid)
 	StaticAssertStmt(MaxOffsetNumber < SpecTokenOffsetNumber,
 					 "invalid speculative token constant");
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -5605,6 +5622,8 @@ heap_abort_speculative(Relation relation, ItemPointer tid)
 		elog(ERROR, "attempted to kill a non-speculative tuple");
 	Assert(!HeapTupleHeaderIsHeapOnly(tp.t_data));
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * No need to check for serializable conflicts here.  There is never a
 	 * need for a combocid, either.  No need to extract replica identity, or
@@ -5721,6 +5740,7 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	HeapTupleHeader htup;
 	uint32		oldlen;
 	uint32		newlen;
+	bool		needwal;
 
 	/*
 	 * For now, we don't allow parallel updates.  Unlike a regular update,
@@ -5751,6 +5771,10 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	if (oldlen != newlen || htup->t_hoff != tuple->t_data->t_hoff)
 		elog(ERROR, "wrong tuple length");
 
+	needwal = RelationNeedsWAL(relation);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -5761,7 +5785,7 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	MarkBufferDirty(buffer);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(relation))
+	if (needwal)
 	{
 		xl_heap_inplace xlrec;
 		XLogRecPtr	recptr;
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index bc510e2e9b3..9dcae7d2153 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -18,6 +18,7 @@
 #include "access/heapam_xlog.h"
 #include "access/htup_details.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "catalog/catalog.h"
 #include "miscadmin.h"
@@ -94,11 +95,11 @@ heap_page_prune_opt(Relation relation, Buffer buffer)
 	Size		minfree;
 
 	/*
-	 * We can't write WAL in recovery mode, so there's no point trying to
+	 * We can't write WAL during read-only mode, so there's no point trying to
 	 * clean the page. The primary will likely issue a cleaning WAL record soon
 	 * anyway, so this is no particular loss.
 	 */
-	if (RecoveryInProgress())
+	if (!XLogInsertAllowed())
 		return;
 
 	/*
@@ -232,6 +233,7 @@ heap_page_prune(Relation relation, Buffer buffer,
 	OffsetNumber offnum,
 				maxoff;
 	PruneState	prstate;
+	bool		needwal;
 
 	/*
 	 * Our strategy is to scan the page and make lists of items to change,
@@ -286,6 +288,10 @@ heap_page_prune(Relation relation, Buffer buffer,
 	if (off_loc)
 		*off_loc = InvalidOffsetNumber;
 
+	needwal = RelationNeedsWAL(relation);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Any error while applying the changes is critical */
 	START_CRIT_SECTION();
 
@@ -319,7 +325,7 @@ heap_page_prune(Relation relation, Buffer buffer,
 		/*
 		 * Emit a WAL XLOG_HEAP2_CLEAN record showing what we did
 		 */
-		if (RelationNeedsWAL(relation))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 4f2f38168dc..1869df5f03f 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -59,6 +59,7 @@
 #include "access/parallel.h"
 #include "access/transam.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "catalog/storage.h"
@@ -759,6 +760,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 	BlockNumber nblocks,
 				blkno;
 	HeapTupleData tuple;
+	bool		needwal = RelationNeedsWAL(onerel);
 	TransactionId relfrozenxid = onerel->rd_rel->relfrozenxid;
 	TransactionId relminmxid = onerel->rd_rel->relminmxid;
 	BlockNumber empty_pages,
@@ -1201,6 +1203,9 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 			 */
 			if (!PageIsAllVisible(page))
 			{
+				if (needwal)
+					CheckWALPermitted();
+
 				START_CRIT_SECTION();
 
 				/* mark buffer dirty before writing a WAL record */
@@ -1216,7 +1221,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 				 * page has been previously WAL-logged, and if not, do that
 				 * now.
 				 */
-				if (RelationNeedsWAL(onerel) &&
+				if (needwal &&
 					PageGetLSN(page) == InvalidXLogRecPtr)
 					log_newpage_buffer(buf, true);
 
@@ -1482,6 +1487,9 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 		 */
 		if (nfrozen > 0)
 		{
+			if (needwal)
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 
 			MarkBufferDirty(buf);
@@ -1499,7 +1507,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 			}
 
 			/* Now WAL-log freezing if necessary */
-			if (RelationNeedsWAL(onerel))
+			if (needwal)
 			{
 				XLogRecPtr	recptr;
 
@@ -1932,6 +1940,7 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	TransactionId visibility_cutoff_xid;
 	bool		all_frozen;
 	LVSavedErrInfo saved_err_info;
+	bool		needwal = RelationNeedsWAL(onerel);
 
 	pgstat_progress_update_param(PROGRESS_VACUUM_HEAP_BLKS_VACUUMED, blkno);
 
@@ -1939,6 +1948,9 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	update_vacuum_error_info(vacrelstats, &saved_err_info, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
 							 blkno, InvalidOffsetNumber);
 
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	for (; tupindex < dead_tuples->num_tuples; tupindex++)
@@ -1964,7 +1976,7 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	MarkBufferDirty(buffer);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(onerel))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index b1072183bcd..44244363968 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -88,6 +88,7 @@
 
 #include "access/heapam_xlog.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "miscadmin.h"
 #include "port/pg_bitutils.h"
@@ -249,6 +250,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 	uint8		mapOffset = HEAPBLK_TO_OFFSET(heapBlk);
 	Page		page;
 	uint8	   *map;
+	bool		needwal = RelationNeedsWAL(rel);
 
 #ifdef TRACE_VISIBILITYMAP
 	elog(DEBUG1, "vm_set %s %d", RelationGetRelationName(rel), heapBlk);
@@ -270,6 +272,16 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 	map = (uint8 *) PageGetContents(page);
 	LockBuffer(vmBuf, BUFFER_LOCK_EXCLUSIVE);
 
+	/*
+	 * Can reach here from VACUUM or from startup process, so need not have an
+	 * XID.
+	 *
+	 * Recovery in the startup process never have wal prohibit state, skip
+	 * permission check if reach here in the startup process.
+	 */
+	if (needwal)
+		InRecovery ? AssertWALPermitted() : CheckWALPermitted();
+
 	if (flags != (map[mapByte] >> mapOffset & VISIBILITYMAP_VALID_BITS))
 	{
 		START_CRIT_SECTION();
@@ -277,7 +289,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 		map[mapByte] |= (flags << mapOffset);
 		MarkBufferDirty(vmBuf);
 
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			if (XLogRecPtrIsInvalid(recptr))
 			{
@@ -476,6 +488,7 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		Buffer		mapBuffer;
 		Page		page;
 		char	   *map;
+		bool		needwal;
 
 		newnblocks = truncBlock + 1;
 
@@ -489,8 +502,13 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		page = BufferGetPage(mapBuffer);
 		map = PageGetContents(page);
 
+		needwal = (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded());
+
 		LockBuffer(mapBuffer, BUFFER_LOCK_EXCLUSIVE);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -518,7 +536,7 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		 * during recovery.
 		 */
 		MarkBufferDirty(mapBuffer);
-		if (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded())
+		if (needwal)
 			log_newpage_buffer(mapBuffer, false);
 
 		END_CRIT_SECTION();
diff --git a/src/backend/access/nbtree/nbtdedup.c b/src/backend/access/nbtree/nbtdedup.c
index f6be865b17e..b519a1268e8 100644
--- a/src/backend/access/nbtree/nbtdedup.c
+++ b/src/backend/access/nbtree/nbtdedup.c
@@ -16,6 +16,7 @@
 
 #include "access/nbtree.h"
 #include "access/nbtxlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "utils/rel.h"
 
@@ -271,6 +272,8 @@ _bt_dedup_one_page(Relation rel, Buffer buf, Relation heapRel,
 		nopaque->btpo_flags &= ~BTP_HAS_GARBAGE;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	PageRestoreTempPage(newpage, page);
diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c
index d36f7557c87..2c3d8aaecbd 100644
--- a/src/backend/access/nbtree/nbtinsert.c
+++ b/src/backend/access/nbtree/nbtinsert.c
@@ -19,6 +19,7 @@
 #include "access/nbtxlog.h"
 #include "access/tableam.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/lmgr.h"
@@ -1246,6 +1247,8 @@ _bt_insertonpg(Relation rel,
 			}
 		}
 
+		AssertWALPermittedHaveXID();
+
 		/* Do the update.  No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -1898,13 +1901,16 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf,
 			ropaque->btpo_flags |= BTP_SPLIT_END;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Right sibling is locked, new siblings are prepared, but original page
 	 * is not updated yet.
 	 *
 	 * NO EREPORT(ERROR) till right sibling is updated.  We can get away with
 	 * not starting the critical section till here because we haven't been
-	 * scribbling on the original page yet; see comments above.
+	 * scribbling on the original page yet; see the comments above for grabbing
+	 * the right sibling.
 	 */
 	START_CRIT_SECTION();
 
@@ -2467,6 +2473,8 @@ _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf)
 	right_item = CopyIndexTuple(item);
 	BTreeTupleSetDownLink(right_item, rbkno);
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till newroot op is logged */
 	START_CRIT_SECTION();
 
diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c
index 7f392480ac0..8c3fc251a29 100644
--- a/src/backend/access/nbtree/nbtpage.c
+++ b/src/backend/access/nbtree/nbtpage.c
@@ -26,6 +26,7 @@
 #include "access/nbtxlog.h"
 #include "access/tableam.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
@@ -179,6 +180,7 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	BTMetaPageData *metad;
 	bool		needsRewrite = false;
 	XLogRecPtr	recptr;
+	bool		needwal;
 
 	/* read the metapage and check if it needs rewrite */
 	metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ);
@@ -202,6 +204,10 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	_bt_unlockbuf(rel, metabuf);
 	_bt_lockbuf(rel, metabuf, BT_WRITE);
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* upgrade meta-page if needed */
@@ -214,7 +220,7 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	MarkBufferDirty(metabuf);
 
 	/* write wal record if needed */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_btree_metadata md;
 
@@ -332,6 +338,7 @@ _bt_getroot(Relation rel, int access)
 	if (metad->btm_root == P_NONE)
 	{
 		Page		metapg;
+		bool		needwal;
 
 		/* If access = BT_READ, caller doesn't want us to create root yet */
 		if (access == BT_READ)
@@ -377,6 +384,10 @@ _bt_getroot(Relation rel, int access)
 		/* Get raw page pointer for metapage */
 		metapg = BufferGetPage(metabuf);
 
+		needwal = RelationNeedsWAL(rel);
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO ELOG(ERROR) till meta is updated */
 		START_CRIT_SECTION();
 
@@ -395,7 +406,7 @@ _bt_getroot(Relation rel, int access)
 		MarkBufferDirty(metabuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			xl_btree_newroot xlrec;
 			XLogRecPtr	recptr;
@@ -1131,6 +1142,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 	char	   *updatedbuf = NULL;
 	Size		updatedbuflen = 0;
 	OffsetNumber updatedoffsets[MaxIndexTuplesPerPage];
+	bool		needwal = RelationNeedsWAL(rel);
 
 	/* Shouldn't be called unless there's something to do */
 	Assert(ndeletable > 0 || nupdatable > 0);
@@ -1145,7 +1157,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 	}
 
 	/* XLOG stuff -- allocate and fill buffer before critical section */
-	if (nupdatable > 0 && RelationNeedsWAL(rel))
+	if (nupdatable > 0 && needwal)
 	{
 		Size		offset = 0;
 
@@ -1175,6 +1187,9 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 		}
 	}
 
+	if (needwal)
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -1235,7 +1250,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 	MarkBufferDirty(buf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		xl_btree_vacuum xlrec_vacuum;
@@ -1302,6 +1317,8 @@ _bt_delitems_delete(Relation rel, Buffer buf,
 		latestRemovedXid =
 			_bt_xid_horizon(rel, heapRel, page, deletable, ndeletable);
 
+	AssertWALPermittedHaveXID();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -1832,6 +1849,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	OffsetNumber nextoffset;
 	IndexTuple	itup;
 	IndexTupleData trunctuple;
+	bool		needwal;
 
 	page = BufferGetPage(leafbuf);
 	opaque = (BTPageOpaque) PageGetSpecialPointer(page);
@@ -1920,6 +1938,10 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	 */
 	PredicateLockPageCombine(rel, leafblkno, leafrightsib);
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -1971,7 +1993,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	MarkBufferDirty(leafbuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_btree_mark_page_halfdead xlrec;
 		XLogRecPtr	recptr;
@@ -2064,6 +2086,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 	int			targetlevel;
 	IndexTuple	leafhikey;
 	BlockNumber nextchild;
+	bool		needwal;
 
 	page = BufferGetPage(leafbuf);
 	opaque = (BTPageOpaque) PageGetSpecialPointer(page);
@@ -2277,6 +2300,10 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 	 * Here we begin doing the deletion.
 	 */
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -2356,7 +2383,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 		MarkBufferDirty(leafbuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_btree_unlink_page xlrec;
 		xl_btree_metadata xlmeta;
diff --git a/src/backend/access/spgist/spgdoinsert.c b/src/backend/access/spgist/spgdoinsert.c
index 934d65b89f2..3c5a15c5d32 100644
--- a/src/backend/access/spgist/spgdoinsert.c
+++ b/src/backend/access/spgist/spgdoinsert.c
@@ -18,6 +18,7 @@
 #include "access/genam.h"
 #include "access/spgist_private.h"
 #include "access/spgxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -214,6 +215,8 @@ addLeafTuple(Relation index, SpGistState *state, SpGistLeafTuple leafTuple,
 	xlrec.offnumParent = InvalidOffsetNumber;
 	xlrec.nodeI = 0;
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	if (current->offnum == InvalidOffsetNumber ||
@@ -458,6 +461,8 @@ moveLeafs(Relation index, SpGistState *state,
 
 	leafdata = leafptr = palloc(size);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/* copy all the old tuples to new page, unless they're dead */
@@ -1109,6 +1114,8 @@ doPickSplit(Relation index, SpGistState *state,
 
 	leafdata = leafptr = (char *) palloc(totalLeafSizes);
 
+	AssertWALPermittedHaveXID();
+
 	/* Here we begin making the changes to the target pages */
 	START_CRIT_SECTION();
 
@@ -1517,6 +1524,8 @@ spgAddNodeAction(Relation index, SpGistState *state,
 	if (PageGetExactFreeSpace(current->page) >=
 		newInnerTuple->size - innerTuple->size)
 	{
+		AssertWALPermittedHaveXID();
+
 		/*
 		 * We can replace the inner tuple by new version in-place
 		 */
@@ -1603,6 +1612,8 @@ spgAddNodeAction(Relation index, SpGistState *state,
 		else
 			xlrec.parentBlk = 2;
 
+		AssertWALPermittedHaveXID();
+
 		START_CRIT_SECTION();
 
 		/* insert new ... */
@@ -1788,6 +1799,8 @@ spgSplitNodeAction(Relation index, SpGistState *state,
 									&xlrec.newPage);
 	}
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c
index e1c58933f97..3308832b85b 100644
--- a/src/backend/access/spgist/spgvacuum.c
+++ b/src/backend/access/spgist/spgvacuum.c
@@ -19,6 +19,7 @@
 #include "access/spgist_private.h"
 #include "access/spgxlog.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/storage_xlog.h"
 #include "commands/vacuum.h"
@@ -139,6 +140,7 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 	int			nDeletable;
 	OffsetNumber i,
 				max = PageGetMaxOffsetNumber(page);
+	bool		needwal;
 
 	memset(predecessor, 0, sizeof(predecessor));
 	memset(deletable, 0, sizeof(deletable));
@@ -323,6 +325,10 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 	if (nDeletable != xlrec.nDead + xlrec.nPlaceholder + xlrec.nMove)
 		elog(ERROR, "inconsistent counts of deletable tuples");
 
+	needwal = RelationNeedsWAL(index);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Do the updates */
 	START_CRIT_SECTION();
 
@@ -371,7 +377,7 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 
@@ -411,6 +417,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 	OffsetNumber toDelete[MaxIndexTuplesPerPage];
 	OffsetNumber i,
 				max = PageGetMaxOffsetNumber(page);
+	bool		needwal;
 
 	xlrec.nDelete = 0;
 
@@ -447,6 +454,10 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 	if (xlrec.nDelete == 0)
 		return;					/* nothing more to do */
 
+	needwal = RelationNeedsWAL(index);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Do the update */
 	START_CRIT_SECTION();
 
@@ -455,7 +466,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 
@@ -502,6 +513,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer)
 	OffsetNumber itemnos[MaxIndexTuplesPerPage];
 	spgxlogVacuumRedirect xlrec;
 	GlobalVisState *vistest;
+	bool		needwal;
 
 	xlrec.nToPlaceholder = 0;
 	xlrec.newestRedirectXid = InvalidTransactionId;
@@ -509,6 +521,10 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer)
 	/* XXX: providing heap relation would allow more pruning */
 	vistest = GlobalVisTestFor(NULL);
 
+	needwal = RelationNeedsWAL(index);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -584,7 +600,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer)
 	if (hasUpdate)
 		MarkBufferDirty(buffer);
 
-	if (hasUpdate && RelationNeedsWAL(index))
+	if (hasUpdate && needwal)
 	{
 		XLogRecPtr	recptr;
 
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index 43653fe5721..95cb3a94cf1 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -73,6 +73,7 @@
 #include "access/transam.h"
 #include "access/twophase.h"
 #include "access/twophase_rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1162,6 +1163,8 @@ GetNewMultiXactId(int nmembers, MultiXactOffset *offset)
 
 	ExtendMultiXactMember(nextOffset, nmembers);
 
+	CheckWALPermitted();
+
 	/*
 	 * Critical section from here until caller has written the data into the
 	 * just-reserved SLRU space; we don't want to error out with a partly
@@ -2946,7 +2949,7 @@ TruncateMultiXact(MultiXactId newOldestMulti, Oid newOldestMultiDB)
 	mxtruncinfo trunc;
 	MultiXactId earliest;
 
-	Assert(!RecoveryInProgress());
+	Assert(XLogInsertAllowed());
 	Assert(MultiXactState->finishedStartup);
 
 	/*
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 79400604431..68ffea44e0d 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -82,6 +82,7 @@
 #include "access/transam.h"
 #include "access/twophase.h"
 #include "access/twophase_rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1106,6 +1107,8 @@ EndPrepare(GlobalTransaction gxact)
 	 */
 	XLogEnsureRecordSpace(0, records.num_chunks);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	MyProc->delayChkpt = true;
@@ -2196,6 +2199,9 @@ RecordTransactionCommitPrepared(TransactionId xid,
 	replorigin = (replorigin_session_origin != InvalidRepOriginId &&
 				  replorigin_session_origin != DoNotReplicateId);
 
+	/* COMMIT PREPARED need not have an XID */
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* See notes in RecordTransactionCommit */
@@ -2286,6 +2292,9 @@ RecordTransactionAbortPrepared(TransactionId xid,
 		elog(PANIC, "cannot abort transaction %u, it was already committed",
 			 xid);
 
+	/* ROLLBACK PREPARED need not have an XID */
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
diff --git a/src/backend/access/transam/varsup.c b/src/backend/access/transam/varsup.c
index a4944faa32e..0c7a2362f25 100644
--- a/src/backend/access/transam/varsup.c
+++ b/src/backend/access/transam/varsup.c
@@ -17,6 +17,7 @@
 #include "access/commit_ts.h"
 #include "access/subtrans.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "commands/dbcommands.h"
@@ -75,6 +76,9 @@ GetNewTransactionId(bool isSubXact)
 	if (RecoveryInProgress())
 		elog(ERROR, "cannot assign TransactionIds during recovery");
 
+	/* do not assign transaction id in read-only mode */
+	CheckWALPermitted();
+
 	LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
 
 	full_xid = ShmemVariableCache->nextXid;
diff --git a/src/backend/access/transam/walprohibit.c b/src/backend/access/transam/walprohibit.c
index 00c8894d806..f2b5bf2871a 100644
--- a/src/backend/access/transam/walprohibit.c
+++ b/src/backend/access/transam/walprohibit.c
@@ -24,6 +24,16 @@
 #include "utils/fmgroids.h"
 #include "utils/fmgrprotos.h"
 
+/*
+ * Assert flag to enforce WAL insert permission check rule before starting a
+ * critical section for the WAL writes.  For this, either of
+ * CheckWALPermitted(), AssertWALPermittedHaveXID(), or AssertWALPermitted()
+ * must be called before starting the critical section.
+ */
+#ifdef USE_ASSERT_CHECKING
+WALPermitCheckState walpermit_checked_state = WALPERMIT_UNCHECKED;
+#endif
+
 /*
  * Shared-memory WAL prohibit state
  */
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 188c299bed9..abda095e735 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -26,6 +26,7 @@
 #include "access/subtrans.h"
 #include "access/transam.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1317,6 +1318,8 @@ RecordTransactionCommit(void)
 		/* Tell bufmgr and smgr to prepare for commit */
 		BufmgrCommit();
 
+		AssertWALPermittedHaveXID();
+
 		/*
 		 * Mark ourselves as within our "commit critical section".  This
 		 * forces any concurrent checkpoint to wait until we've updated
@@ -1677,6 +1680,9 @@ RecordTransactionAbort(bool isSubXact)
 		elog(PANIC, "cannot abort transaction %u, it was already committed",
 			 xid);
 
+	/* We'll be reaching here with valid XID only. */
+	AssertWALPermittedHaveXID();
+
 	/* Fetch the data we need for the abort record */
 	nrels = smgrGetPendingDeletes(false, &rels);
 	nchildren = xactGetCommittedChildren(&children);
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 8c51c554dc0..5d0faee6989 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -1026,7 +1026,7 @@ XLogInsertRecord(XLogRecData *rdata,
 
 	/* cross-check on whether we should be here or not */
 	if (!XLogInsertAllowed())
-		elog(ERROR, "cannot make new WAL entries during recovery");
+		elog(ERROR, "cannot make new WAL entries now");
 
 	/*----------
 	 *
@@ -2862,9 +2862,11 @@ XLogFlush(XLogRecPtr record)
 	 * trying to flush the WAL, we should update minRecoveryPoint instead. We
 	 * test XLogInsertAllowed(), not InRecovery, because we need checkpointer
 	 * to act this way too, and because when it tries to write the
-	 * end-of-recovery checkpoint, it should indeed flush.
+	 * end-of-recovery checkpoint, it should indeed flush. Also, WAL prohibit
+	 * state should not restrict WAL flushing. Otherwise, the dirty buffer
+	 * cannot be evicted until WAL has been flushed up to the buffer's LSN.
 	 */
-	if (!XLogInsertAllowed())
+	if (!XLogInsertAllowed() && !IsWALProhibited())
 	{
 		UpdateMinRecoveryPoint(record, false);
 		return;
@@ -8823,6 +8825,8 @@ CreateCheckPoint(int flags)
 	/* sanity check */
 	if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
 		elog(ERROR, "can't create a checkpoint during recovery");
+	if (!XLogInsertAllowed() && !RecoveryInProgress())
+		elog(ERROR, "can't create a checkpoint while system is read only ");
 
 	/*
 	 * Initialize InitXLogInsert working areas before entering the critical
@@ -8852,6 +8856,8 @@ CreateCheckPoint(int flags)
 	MemSet(&CheckpointStats, 0, sizeof(CheckpointStats));
 	CheckpointStats.ckpt_start_t = GetCurrentTimestamp();
 
+	AssertWALPermitted();
+
 	/*
 	 * Use a critical section to force system panic if we have trouble.
 	 */
@@ -9080,6 +9086,8 @@ CreateCheckPoint(int flags)
 	if (!shutdown && XLogStandbyInfoActive())
 		LogStandbySnapshot();
 
+	AssertWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -9237,6 +9245,8 @@ CreateEndOfRecoveryRecord(void)
 
 	LocalSetXLogInsertAllowed();
 
+	AssertWALPermitted();
+
 	START_CRIT_SECTION();
 
 	XLogBeginInsert();
@@ -9895,7 +9905,7 @@ void
 UpdateFullPageWrites(void)
 {
 	XLogCtlInsert *Insert = &XLogCtl->Insert;
-	bool		recoveryInProgress;
+	bool		WALInsertAllowed;
 
 	/*
 	 * Do nothing if full_page_writes has not been changed.
@@ -9909,10 +9919,10 @@ UpdateFullPageWrites(void)
 
 	/*
 	 * Perform this outside critical section so that the WAL insert
-	 * initialization done by RecoveryInProgress() doesn't trigger an
+	 * initialization done by XLogInsertAllowed() doesn't trigger an
 	 * assertion failure.
 	 */
-	recoveryInProgress = RecoveryInProgress();
+	WALInsertAllowed = XLogInsertAllowed();
 
 	START_CRIT_SECTION();
 
@@ -9934,8 +9944,11 @@ UpdateFullPageWrites(void)
 	 * Write an XLOG_FPW_CHANGE record. This allows us to keep track of
 	 * full_page_writes during archive recovery, if required.
 	 */
-	if (XLogStandbyInfoActive() && !recoveryInProgress)
+	if (XLogStandbyInfoActive() && WALInsertAllowed)
 	{
+		/* Assured that WAL permission has been checked */
+		AssertWALPermitted();
+
 		XLogBeginInsert();
 		XLogRegisterData((char *) (&fullPageWrites), sizeof(bool));
 
diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c
index 1f0e4e01e69..710806143d4 100644
--- a/src/backend/access/transam/xloginsert.c
+++ b/src/backend/access/transam/xloginsert.c
@@ -19,6 +19,7 @@
 
 #include "postgres.h"
 
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xlog_internal.h"
@@ -126,9 +127,14 @@ XLogBeginInsert(void)
 	Assert(mainrdata_last == (XLogRecData *) &mainrdata_head);
 	Assert(mainrdata_len == 0);
 
+	/*
+	 * WAL permission must have checked before entering the critical section.
+	 * Otherwise, WAL prohibited error will force system panic.
+	 */
+	Assert(walpermit_checked_state != WALPERMIT_UNCHECKED || !CritSectionCount);
+
 	/* cross-check on whether we should be here or not */
-	if (!XLogInsertAllowed())
-		elog(ERROR, "cannot make new WAL entries during recovery");
+	CheckWALPermitted();
 
 	if (begininsert_called)
 		elog(ERROR, "XLogBeginInsert was already called");
@@ -210,6 +216,9 @@ XLogResetInsertion(void)
 	mainrdata_last = (XLogRecData *) &mainrdata_head;
 	curinsert_flags = 0;
 	begininsert_called = false;
+
+	/* Reset walpermit_checked flag */
+	RESET_WALPERMIT_CHECKED_STATE();
 }
 
 /*
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 632b34af610..b01ad5a966a 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -20,6 +20,7 @@
 #include "access/relation.h"
 #include "access/table.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -378,8 +379,13 @@ fill_seq_with_data(Relation rel, HeapTuple tuple)
 
 	/* check the comment above nextval_internal()'s equivalent call. */
 	if (RelationNeedsWAL(rel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermittedHaveXID();
+	}
+
 	START_CRIT_SECTION();
 
 	MarkBufferDirty(buf);
@@ -766,8 +772,13 @@ nextval_internal(Oid relid, bool check_permissions)
 	 * (Have to do that here, so we're outside the critical section)
 	 */
 	if (logit && RelationNeedsWAL(seqrel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermittedHaveXID();
+	}
+
 	/* ready to change the on-disk (or really, in-buffer) tuple */
 	START_CRIT_SECTION();
 
@@ -977,8 +988,13 @@ do_setval(Oid relid, int64 next, bool iscalled)
 
 	/* check the comment above nextval_internal()'s equivalent call. */
 	if (RelationNeedsWAL(seqrel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermittedHaveXID();
+	}
+
 	/* ready to change the on-disk (or really, in-buffer) tuple */
 	START_CRIT_SECTION();
 
diff --git a/src/backend/commands/variable.c b/src/backend/commands/variable.c
index 484f7ea2c0e..acc34d2ad7c 100644
--- a/src/backend/commands/variable.c
+++ b/src/backend/commands/variable.c
@@ -501,11 +501,14 @@ check_transaction_read_only(bool *newval, void **extra, GucSource source)
 			GUC_check_errmsg("transaction read-write mode must be set before any query");
 			return false;
 		}
-		/* Can't go to r/w mode while recovery is still active */
-		if (RecoveryInProgress())
+		/*
+		 * Can't go to r/w mode while recovery is still active or while in WAL
+		 * prohibit state
+		 */
+		if (!XLogInsertAllowed())
 		{
 			GUC_check_errcode(ERRCODE_FEATURE_NOT_SUPPORTED);
-			GUC_check_errmsg("cannot set transaction read-write mode during recovery");
+			GUC_check_errmsg("cannot set transaction read-write mode while system is read only");
 			return false;
 		}
 	}
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index bd66705a9af..da4b8d502ad 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -945,6 +945,10 @@ RequestCheckpoint(int flags)
 	int			old_failed,
 				old_started;
 
+	/* The checkpoint is allowed in recovery but not in WAL prohibit state */
+	if (!RecoveryInProgress())
+		CheckWALPermitted();
+
 	/*
 	 * If in a standalone backend, just do it ourselves.
 	 */
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 2fa0b065a28..38bce5c5ec7 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -3633,13 +3633,15 @@ MarkBufferDirtyHint(Buffer buffer, bool buffer_std)
 		{
 			/*
 			 * If we must not write WAL, due to a relfilenode-specific
-			 * condition or being in recovery, don't dirty the page.  We can
-			 * set the hint, just not dirty the page as a result so the hint
-			 * is lost when we evict the page or shutdown.
+			 * condition or in general, don't dirty the page.  We can
+			 * set the hint, but must not dirty the page as a result, lest
+			 * we trigger WAL generation. Unless the page is dirtied again
+			 * later, the hint will be lost when the page is evicted, or at
+			 * shutdown.
 			 *
 			 * See src/backend/storage/page/README for longer discussion.
 			 */
-			if (RecoveryInProgress() ||
+			if (!XLogInsertAllowed() ||
 				RelFileNodeSkippingWAL(bufHdr->tag.rnode))
 				return;
 
diff --git a/src/backend/storage/freespace/freespace.c b/src/backend/storage/freespace/freespace.c
index 6a96126b0c2..b05b0fe5f41 100644
--- a/src/backend/storage/freespace/freespace.c
+++ b/src/backend/storage/freespace/freespace.c
@@ -24,6 +24,7 @@
 #include "postgres.h"
 
 #include "access/htup_details.h"
+#include "access/walprohibit.h"
 #include "access/xlogutils.h"
 #include "miscadmin.h"
 #include "storage/freespace.h"
@@ -285,12 +286,19 @@ FreeSpaceMapPrepareTruncateRel(Relation rel, BlockNumber nblocks)
 	 */
 	if (first_removed_slot > 0)
 	{
+		bool needwal;
+
 		buf = fsm_readbuf(rel, first_removed_address, false);
 		if (!BufferIsValid(buf))
 			return InvalidBlockNumber;	/* nothing to do; the FSM was already
 										 * smaller */
 		LockBuffer(buf, BUFFER_LOCK_EXCLUSIVE);
 
+		needwal = (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded());
+
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -305,7 +313,7 @@ FreeSpaceMapPrepareTruncateRel(Relation rel, BlockNumber nblocks)
 		 * during recovery.
 		 */
 		MarkBufferDirty(buf);
-		if (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded())
+		if (needwal)
 			log_newpage_buffer(buf, false);
 
 		END_CRIT_SECTION();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index d86566f4554..f949a290745 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -794,15 +794,15 @@ LockAcquireExtended(const LOCKTAG *locktag,
 	if (lockmode <= 0 || lockmode > lockMethodTable->numLockModes)
 		elog(ERROR, "unrecognized lock mode: %d", lockmode);
 
-	if (RecoveryInProgress() && !InRecovery &&
+	if (!XLogInsertAllowed() && !InRecovery &&
 		(locktag->locktag_type == LOCKTAG_OBJECT ||
 		 locktag->locktag_type == LOCKTAG_RELATION) &&
 		lockmode > RowExclusiveLock)
 		ereport(ERROR,
 				(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
-				 errmsg("cannot acquire lock mode %s on database objects while recovery is in progress",
+				 errmsg("cannot acquire lock mode %s on database objects while system is read only",
 						lockMethodTable->lockModeNames[lockmode]),
-				 errhint("Only RowExclusiveLock or less can be acquired on database objects during recovery.")));
+				 errhint("Only RowExclusiveLock or less can be acquired on database objects while system is read only")));
 
 #ifdef LOCK_DEBUG
 	if (LOCK_DEBUG_ENABLED(locktag))
diff --git a/src/backend/utils/cache/relmapper.c b/src/backend/utils/cache/relmapper.c
index 671fbb0ed5c..90d7599a57c 100644
--- a/src/backend/utils/cache/relmapper.c
+++ b/src/backend/utils/cache/relmapper.c
@@ -43,6 +43,7 @@
 #include <sys/stat.h>
 #include <unistd.h>
 
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -841,6 +842,8 @@ write_relmap_file(bool shared, RelMapFile *newmap,
 		xl_relmap_update xlrec;
 		XLogRecPtr	lsn;
 
+		AssertWALPermittedHaveXID();
+
 		/* now errors are fatal ... */
 		START_CRIT_SECTION();
 
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e33523984..f3ff120601e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -94,12 +94,37 @@ extern PGDLLIMPORT volatile uint32 CritSectionCount;
 /* in tcop/postgres.c */
 extern void ProcessInterrupts(void);
 
+#ifdef USE_ASSERT_CHECKING
+typedef enum
+{
+	WALPERMIT_UNCHECKED,
+	WALPERMIT_CHECKED,
+	WALPERMIT_CHECKED_AND_USED
+} WALPermitCheckState;
+
+/* in access/walprohibit.c */
+extern WALPermitCheckState walpermit_checked_state;
+
+/*
+ * Reset walpermit_checked flag when no longer in the critical section.
+ * Otherwise, marked checked and used.
+ */
+#define RESET_WALPERMIT_CHECKED_STATE() \
+do { \
+	walpermit_checked_state = CritSectionCount ? \
+	WALPERMIT_CHECKED_AND_USED : WALPERMIT_UNCHECKED; \
+} while(0)
+#else
+#define RESET_WALPERMIT_CHECKED_STATE() ((void) 0)
+#endif
+
 #ifndef WIN32
 
 #define CHECK_FOR_INTERRUPTS() \
 do { \
 	if (unlikely(InterruptPending)) \
 		ProcessInterrupts(); \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 #else							/* WIN32 */
 
@@ -109,6 +134,7 @@ do { \
 		pgwin32_dispatch_queued_signals(); \
 	if (unlikely(InterruptPending)) \
 		ProcessInterrupts(); \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 #endif							/* WIN32 */
 
@@ -135,6 +161,7 @@ do { \
 do { \
 	Assert(CritSectionCount > 0); \
 	CritSectionCount--; \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 
 
-- 
2.18.0

From 9e38853c92c49f2f45f05c8730e3de1dc2c0e80a Mon Sep 17 00:00:00 2001
From: Robert Haas <rhaas@postgresql.org>
Date: Wed, 7 Oct 2020 13:04:16 -0400
Subject: [PATCH v10 2/6] Test module for barriers. NOT FOR COMMIT.

---
 contrib/barrier/Makefile         | 23 ++++++++++++
 contrib/barrier/barrier--1.0.sql | 14 +++++++
 contrib/barrier/barrier.c        | 63 ++++++++++++++++++++++++++++++++
 contrib/barrier/barrier.control  |  5 +++
 4 files changed, 105 insertions(+)
 create mode 100644 contrib/barrier/Makefile
 create mode 100644 contrib/barrier/barrier--1.0.sql
 create mode 100644 contrib/barrier/barrier.c
 create mode 100644 contrib/barrier/barrier.control

diff --git a/contrib/barrier/Makefile b/contrib/barrier/Makefile
new file mode 100644
index 00000000000..71f59f6629e
--- /dev/null
+++ b/contrib/barrier/Makefile
@@ -0,0 +1,23 @@
+# contrib/barrier/Makefile
+
+MODULE_big = barrier
+OBJS = \
+	$(WIN32RES) \
+	barrier.o
+
+EXTENSION = barrier
+DATA = barrier--1.0.sql
+PGFILEDESC = "barrier - barrier test code NOT FOR COMMIT"
+
+REGRESS = barrier
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/barrier
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/barrier/barrier--1.0.sql b/contrib/barrier/barrier--1.0.sql
new file mode 100644
index 00000000000..66cae976a96
--- /dev/null
+++ b/contrib/barrier/barrier--1.0.sql
@@ -0,0 +1,14 @@
+/* contrib/barrier/barrier--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION barrier" to load this file. \quit
+
+CREATE FUNCTION emit_barrier(barrier_type text, count integer default 1)
+RETURNS void
+AS 'MODULE_PATHNAME', 'emit_barrier'
+LANGUAGE C STRICT;
+
+CREATE FUNCTION wait_barrier(barrier_type text)
+RETURNS void
+AS 'MODULE_PATHNAME', 'wait_barrier'
+LANGUAGE C STRICT;
diff --git a/contrib/barrier/barrier.c b/contrib/barrier/barrier.c
new file mode 100644
index 00000000000..a0b98439924
--- /dev/null
+++ b/contrib/barrier/barrier.c
@@ -0,0 +1,63 @@
+/*-------------------------------------------------------------------------
+ *
+ * barrier.c
+ *	  emit ProcSignalBarriers for testing purposes
+ *
+ * Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ *
+ *	  contrib/barrier/barrier.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "funcapi.h"
+#include "miscadmin.h"
+#include "storage/procsignal.h"
+#include "utils/builtins.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(emit_barrier);
+PG_FUNCTION_INFO_V1(wait_barrier);
+
+static ProcSignalBarrierType
+get_barrier_type(text *barrier_type)
+{
+	char	   *btype = text_to_cstring(barrier_type);
+
+	if (strcmp(btype, "placeholder") == 0)
+		return PROCSIGNAL_BARRIER_PLACEHOLDER;
+
+	elog(ERROR, "unknown barrier type: \"%s\"", btype);
+}
+
+Datum
+emit_barrier(PG_FUNCTION_ARGS)
+{
+	text	   *barrier_type = PG_GETARG_TEXT_PP(0);
+	int32		count = PG_GETARG_INT32(1);
+	int32		i;
+	ProcSignalBarrierType t = get_barrier_type(barrier_type);
+
+	for (i = 0; i < count; ++i)
+	{
+		CHECK_FOR_INTERRUPTS();
+		EmitProcSignalBarrier(t);
+	}
+
+	PG_RETURN_VOID();
+}
+
+Datum
+wait_barrier(PG_FUNCTION_ARGS)
+{
+	text	   *barrier_type = PG_GETARG_TEXT_PP(0);
+	ProcSignalBarrierType t = get_barrier_type(barrier_type);
+	uint64		generation;
+
+	generation = EmitProcSignalBarrier(t);
+	elog(NOTICE, "waiting for barrier");
+	WaitForProcSignalBarrier(generation);
+
+	PG_RETURN_VOID();
+}
diff --git a/contrib/barrier/barrier.control b/contrib/barrier/barrier.control
new file mode 100644
index 00000000000..425ffc15432
--- /dev/null
+++ b/contrib/barrier/barrier.control
@@ -0,0 +1,5 @@
+# barrier extension
+comment = 'emit ProcSignalBarrier for test purposes'
+default_version = '1.0'
+module_pathname = '$libdir/barrier'
+relocatable = true
-- 
2.18.0

From 49b1140498d17eb49c18cc385230e13664b12f52 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 27 Mar 2020 05:05:38 -0400
Subject: [PATCH v10 3/6] Add alter system read only/write syntax

Note that syntax doesn't have any implementation.
---
 src/backend/nodes/copyfuncs.c    | 12 ++++++++++++
 src/backend/nodes/equalfuncs.c   |  9 +++++++++
 src/backend/nodes/outfuncs.c     | 12 ++++++++++++
 src/backend/nodes/readfuncs.c    | 15 +++++++++++++++
 src/backend/parser/gram.y        | 13 +++++++++++++
 src/backend/tcop/utility.c       | 21 +++++++++++++++++++++
 src/bin/psql/tab-complete.c      |  6 ++++--
 src/include/nodes/nodes.h        |  1 +
 src/include/nodes/parsenodes.h   | 10 ++++++++++
 src/tools/pgindent/typedefs.list |  1 +
 10 files changed, 98 insertions(+), 2 deletions(-)

diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 2b4d7654cc7..e1c6b2364b4 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -4018,6 +4018,15 @@ _copyAlterSystemStmt(const AlterSystemStmt *from)
 	return newnode;
 }
 
+static AlterSystemWALProhibitState *
+_copyAlterSystemWALProhibitState(const AlterSystemWALProhibitState *from)
+{
+	AlterSystemWALProhibitState *newnode = makeNode(AlterSystemWALProhibitState);
+
+	COPY_SCALAR_FIELD(walprohibited);
+	return newnode;
+}
+
 static CreateSeqStmt *
 _copyCreateSeqStmt(const CreateSeqStmt *from)
 {
@@ -5403,6 +5412,9 @@ copyObjectImpl(const void *from)
 		case T_AlterSystemStmt:
 			retval = _copyAlterSystemStmt(from);
 			break;
+		case T_AlterSystemWALProhibitState:
+			retval = _copyAlterSystemWALProhibitState(from);
+			break;
 		case T_CreateSeqStmt:
 			retval = _copyCreateSeqStmt(from);
 			break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e2d1b987bf4..5f5f289b8af 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1769,6 +1769,12 @@ _equalAlterSystemStmt(const AlterSystemStmt *a, const AlterSystemStmt *b)
 	return true;
 }
 
+static bool
+_equalAlterSystemWALProhibitState(const AlterSystemWALProhibitState *a, const AlterSystemWALProhibitState *b)
+{
+	COMPARE_SCALAR_FIELD(walprohibited);
+	return true;
+}
 
 static bool
 _equalCreateSeqStmt(const CreateSeqStmt *a, const CreateSeqStmt *b)
@@ -3457,6 +3463,9 @@ equal(const void *a, const void *b)
 		case T_AlterSystemStmt:
 			retval = _equalAlterSystemStmt(a, b);
 			break;
+		case T_AlterSystemWALProhibitState:
+			retval = _equalAlterSystemWALProhibitState(a, b);
+			break;
 		case T_CreateSeqStmt:
 			retval = _equalCreateSeqStmt(a, b);
 			break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 08a049232e0..af74a781782 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -1356,6 +1356,15 @@ _outAlternativeSubPlan(StringInfo str, const AlternativeSubPlan *node)
 	WRITE_NODE_FIELD(subplans);
 }
 
+static void
+_outAlterSystemWALProhibitState(StringInfo str,
+								const AlterSystemWALProhibitState *node)
+{
+	WRITE_NODE_TYPE("ALTERSYSTEMWALPROHIBITSTATE");
+
+	WRITE_BOOL_FIELD(walprohibited);
+}
+
 static void
 _outFieldSelect(StringInfo str, const FieldSelect *node)
 {
@@ -3912,6 +3921,9 @@ outNode(StringInfo str, const void *obj)
 			case T_AlternativeSubPlan:
 				_outAlternativeSubPlan(str, obj);
 				break;
+			case T_AlterSystemWALProhibitState:
+				_outAlterSystemWALProhibitState(str, obj);
+				break;
 			case T_FieldSelect:
 				_outFieldSelect(str, obj);
 				break;
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index ab7b535caae..d5acb438f6d 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -2550,6 +2550,19 @@ _readAlternativeSubPlan(void)
 	READ_DONE();
 }
 
+/*
+ * _readAlterSystemWALProhibitState
+ */
+static AlterSystemWALProhibitState *
+_readAlterSystemWALProhibitState(void)
+{
+	READ_LOCALS(AlterSystemWALProhibitState);
+
+	READ_BOOL_FIELD(walprohibited);
+
+	READ_DONE();
+}
+
 /*
  * _readExtensibleNode
  */
@@ -2872,6 +2885,8 @@ parseNodeString(void)
 		return_value = _readSubPlan();
 	else if (MATCH("ALTERNATIVESUBPLAN", 18))
 		return_value = _readAlternativeSubPlan();
+	else if (MATCH("ALTERSYSTEMWALPROHIBITSTATE", 27))
+		return_value = _readAlterSystemWALProhibitState();
 	else if (MATCH("EXTENSIBLENODE", 14))
 		return_value = _readExtensibleNode();
 	else if (MATCH("PARTITIONBOUNDSPEC", 18))
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 480d1683468..5f306456231 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -479,6 +479,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
 
 %type <vsetstmt> generic_set set_rest set_rest_more generic_reset reset_rest
 				 SetResetClause FunctionSetResetClause
+%type <boolean> system_readonly_state
 
 %type <node>	TableElement TypedTableElement ConstraintElem TableFuncElement
 %type <node>	columnDef columnOptions
@@ -10204,8 +10205,20 @@ AlterSystemStmt:
 					n->setstmt = $4;
 					$$ = (Node *)n;
 				}
+			| ALTER SYSTEM_P system_readonly_state
+				{
+					AlterSystemWALProhibitState *n = makeNode(AlterSystemWALProhibitState);
+					n->walprohibited = $3;
+					$$ = (Node *)n;
+				}
 		;
 
+system_readonly_state:
+			 READ ONLY
+					{ $$ = true; }
+			| READ WRITE
+					{ $$ = false; }
+		;
 
 /*****************************************************************************
  *
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 9a35147b26a..74c2162cd59 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -85,6 +85,7 @@ static void ProcessUtilitySlow(ParseState *pstate,
 							   DestReceiver *dest,
 							   QueryCompletion *qc);
 static void ExecDropStmt(DropStmt *stmt, bool isTopLevel);
+static void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
 
 /*
  * CommandIsReadOnly: is an executable query read-only?
@@ -219,6 +220,7 @@ ClassifyUtilityCommandAsReadOnly(Node *parsetree)
 				return COMMAND_IS_NOT_READ_ONLY;
 			}
 
+		case T_AlterSystemWALProhibitState:
 		case T_AlterSystemStmt:
 			{
 				/*
@@ -834,6 +836,11 @@ standard_ProcessUtility(PlannedStmt *pstmt,
 			AlterSystemSetConfigFile((AlterSystemStmt *) parsetree);
 			break;
 
+		case T_AlterSystemWALProhibitState:
+			PreventInTransactionBlock(isTopLevel, "ALTER SYSTEM");
+			AlterSystemSetWALProhibitState((AlterSystemWALProhibitState *) parsetree);
+			break;
+
 		case T_VariableSetStmt:
 			ExecSetVariableStmt((VariableSetStmt *) parsetree, isTopLevel);
 			break;
@@ -2819,6 +2826,7 @@ CreateCommandTag(Node *parsetree)
 			tag = CMDTAG_REFRESH_MATERIALIZED_VIEW;
 			break;
 
+		case T_AlterSystemWALProhibitState:
 		case T_AlterSystemStmt:
 			tag = CMDTAG_ALTER_SYSTEM;
 			break;
@@ -3683,3 +3691,16 @@ GetCommandLogLevel(Node *parsetree)
 
 	return lev;
 }
+
+/*
+ * AlterSystemSetWALProhibitState
+ *
+ * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
+ */
+static void
+AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
+{
+	ereport(ERROR,
+			(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+			 errmsg("ALTER SYSTEM READ { ONLY | WRITE } not implemented")));
+}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index b2b4f1fd4d1..46b8ccbdd6f 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1864,9 +1864,11 @@ psql_completion(const char *text, int start, int end)
 	/* ALTER SERVER <name> VERSION <version> */
 	else if (Matches("ALTER", "SERVER", MatchAny, "VERSION", MatchAny))
 		COMPLETE_WITH("OPTIONS");
-	/* ALTER SYSTEM SET, RESET, RESET ALL */
+	/* ALTER SYSTEM READ, SET, RESET, RESET ALL */
 	else if (Matches("ALTER", "SYSTEM"))
-		COMPLETE_WITH("SET", "RESET");
+		COMPLETE_WITH("SET", "RESET", "READ");
+	else if (Matches("ALTER", "SYSTEM", "READ"))
+		COMPLETE_WITH("ONLY", "WRITE");
 	else if (Matches("ALTER", "SYSTEM", "SET|RESET"))
 		COMPLETE_WITH_QUERY(Query_for_list_of_alter_system_set_vars);
 	else if (Matches("ALTER", "SYSTEM", "SET", MatchAny))
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 7ddd8c011bf..7b233925692 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -411,6 +411,7 @@ typedef enum NodeTag
 	T_RefreshMatViewStmt,
 	T_ReplicaIdentityStmt,
 	T_AlterSystemStmt,
+	T_AlterSystemWALProhibitState,
 	T_CreatePolicyStmt,
 	T_AlterPolicyStmt,
 	T_CreateTransformStmt,
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 60c2f454660..340ee87f1bc 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -3196,6 +3196,16 @@ typedef struct AlterSystemStmt
 	VariableSetStmt *setstmt;	/* SET subcommand */
 } AlterSystemStmt;
 
+/* ----------------------
+ *		Alter System Read Statement
+ * ----------------------
+ */
+typedef struct AlterSystemWALProhibitState
+{
+	NodeTag		type;
+	bool		walprohibited;
+} AlterSystemWALProhibitState;
+
 /* ----------------------
  *		Cluster Statement (support pbrown's cluster index implementation)
  * ----------------------
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index b6acade6c67..4f524a36f80 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -87,6 +87,7 @@ AlterStatsStmt
 AlterSubscriptionStmt
 AlterSubscriptionType
 AlterSystemStmt
+AlterSystemWALProhibitState
 AlterTSConfigType
 AlterTSConfigurationStmt
 AlterTSDictionaryStmt
-- 
2.18.0

From 7b05eeb593d6508715500fee2712040210815844 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 19 Jun 2020 06:29:36 -0400
Subject: [PATCH v10 4/6] Implement ALTER SYSTEM READ ONLY using global
 barrier.

Implementation:

 1. When a user tried to change server state to WAL-Prohibited using
    ALTER SYSTEM READ ONLY command or by calling
    pg_alter_wal_prohibit_state(true) sql function, the current state
    generation to inprogress in shared memory marked and signaled
    checkpointer process.  Checkpointer, noticing that the current state
    generation has WALPROHIBIT_TRANSITION_IN_PROGRESS flag set, does the
    barrier request, and then acknowledges back to the backend who
    requested the state change once the transition has been completed.
    Final state will be updated in control file to make it persistent
    across the system restarts.

 2. When a backend receives the WAL-Prohibited barrier, at that moment if
    it is already in a transaction and the transaction already assigned XID,
    then the backend will be killed by throwing FATAL(XXX: need more discussion
    on this)

 3. Otherwise, if that backend running transaction without valid XID then, we
    don't need to do anything special right now, simply call
    ResetLocalXLogInsertAllowed() so that any future WAL insert in will check
    XLogInsertAllowed() first which set ready only state appropriately.

 4. A new transaction (in an existing or in a new backend) starts as a
    read-only transaction.

 5. Autovacuum launcher as well as checkpointer will not do anything in
    WAL-Prohibited server state until someone wakes us up.  E.g. a backend
    might later on request us to put the system back to read-write.
 5. The Autovacuum launcher, as well as the checkpointer, will not do
    anything while in the WAL-Prohibited server state until someone wakes
    up.  E.g. user might, later on, request us to put the system back to
    read-write by executing ALTER SYSTEM READ WRITE.

 6. At shutdown in WAL-Prohibited mode, we'll skip shutdown checkpoint
    and xlog rotation. Starting up again will perform crash recovery(XXX:
    need some discussion on this as well) but the end of recovery checkpoint
    will be skipped and it will be performed when the system changed to
    WAL-Permitted mode.

 7. ALTER SYSTEM READ ONLY/WRITE is restricted on standby server.

 8. To execute ALTER SYSTEM READ ONLY/WRITE, the user should have execute
    permssion on pg_alter_wal_prohibit_state() function.

 9. Add system_is_read_only GUC show the system state -- will true when system
    is wal prohibited or in recovery.
---
 src/backend/access/transam/Makefile      |   1 +
 src/backend/access/transam/walprohibit.c | 390 +++++++++++++++++++++++
 src/backend/access/transam/xact.c        |  37 ++-
 src/backend/access/transam/xlog.c        | 116 ++++++-
 src/backend/catalog/system_views.sql     |   2 +
 src/backend/postmaster/autovacuum.c      |   4 +
 src/backend/postmaster/bgwriter.c        |   2 +-
 src/backend/postmaster/checkpointer.c    |  39 +++
 src/backend/postmaster/pgstat.c          |   3 +
 src/backend/storage/ipc/ipci.c           |   6 +
 src/backend/storage/ipc/procsignal.c     |  24 +-
 src/backend/tcop/utility.c               |  15 +-
 src/backend/utils/misc/guc.c             |  26 ++
 src/bin/pg_controldata/pg_controldata.c  |   2 +
 src/include/access/walprohibit.h         |  94 ++++++
 src/include/access/xlog.h                |   4 +
 src/include/catalog/pg_control.h         |   3 +
 src/include/catalog/pg_proc.dat          |   4 +
 src/include/pgstat.h                     |   1 +
 src/include/postmaster/bgwriter.h        |   2 +
 src/include/storage/procsignal.h         |   7 +-
 src/tools/pgindent/typedefs.list         |   1 +
 22 files changed, 715 insertions(+), 68 deletions(-)
 create mode 100644 src/backend/access/transam/walprohibit.c
 create mode 100644 src/include/access/walprohibit.h

diff --git a/src/backend/access/transam/Makefile b/src/backend/access/transam/Makefile
index 595e02de722..b5322a69954 100644
--- a/src/backend/access/transam/Makefile
+++ b/src/backend/access/transam/Makefile
@@ -26,6 +26,7 @@ OBJS = \
 	twophase.o \
 	twophase_rmgr.o \
 	varsup.o \
+	walprohibit.o \
 	xact.o \
 	xlog.o \
 	xlogarchive.o \
diff --git a/src/backend/access/transam/walprohibit.c b/src/backend/access/transam/walprohibit.c
new file mode 100644
index 00000000000..00c8894d806
--- /dev/null
+++ b/src/backend/access/transam/walprohibit.c
@@ -0,0 +1,390 @@
+/*-------------------------------------------------------------------------
+ *
+ * walprohibit.c
+ * 		PostgreSQL write-ahead log prohibit states
+ *
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/backend/access/transam/walprohibit.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/walprohibit.h"
+#include "fmgr.h"
+#include "pgstat.h"
+#include "port/atomics.h"
+#include "postmaster/bgwriter.h"
+#include "storage/condition_variable.h"
+#include "storage/procsignal.h"
+#include "storage/shmem.h"
+#include "utils/acl.h"
+#include "utils/fmgroids.h"
+#include "utils/fmgrprotos.h"
+
+/*
+ * Shared-memory WAL prohibit state
+ */
+typedef struct WALProhibitStateData
+{
+	/*
+	 * Indicates current WAL prohibit state generation and the last two bits of
+	 * this generation indicates current wal prohibit state.
+	 */
+	pg_atomic_uint32 shared_state_generation;
+
+	/* Signaled when requested WAL prohibit state changes */
+	ConditionVariable walprohibit_cv;
+} WALProhibitStateData;
+
+static WALProhibitStateData *WALProhibitState = NULL;
+
+static void RequestWALProhibitChange(uint32 cur_state_gen);
+
+/*
+ * ProcessBarrierWALProhibit()
+ *
+ * Handle WAL prohibit state change request.
+ */
+bool
+ProcessBarrierWALProhibit(void)
+{
+	/*
+	 * Kill off any transactions that have an XID *before* allowing the system
+	 * to go WAL prohibit state.
+	 */
+	if (FullTransactionIdIsValid(GetTopFullTransactionIdIfAny()))
+	{
+		/*
+		 * Should be here only while transiting towards the WAL prohibit state.
+		 */
+		Assert(WALPROHIBIT_CURRENT_STATE(GetWALProhibitStateGen()) ==
+			   WALPROHIBIT_STATE_GOING_READ_ONLY);
+
+		/*
+		 * XXX: Kill off the whole session by throwing FATAL instead of
+		 * killing transaction by throwing ERROR due to following reasons that
+		 * need be thought:
+		 *
+		 * 1. Due to some presents challenges with the wire protocol, we could
+		 * not simply kill of idle transaction.
+		 *
+		 * 2. If we are here in subtransaction then the ERROR will kill the
+		 * current subtransaction only. In the case of invalidations, that
+		 * might be good enough, but for XID assignment it's not, because
+		 * assigning an XID to a subtransaction also causes higher
+		 * sub-transaction levels and the parent transaction to get XIDs.
+		 */
+		ereport(FATAL,
+				(errcode(ERRCODE_ACTIVE_SQL_TRANSACTION),
+				 errmsg("system is now read only"),
+				 errhint("Sessions with open write transactions must be terminated.")));
+	}
+
+	/* Return to "check" state */
+	ResetLocalXLogInsertAllowed();
+
+	return true;
+}
+
+/*
+ * AlterSystemSetWALProhibitState()
+ *
+ * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
+ */
+void
+AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
+{
+	/* Check permission for pg_alter_wal_prohibit_state() */
+	if (pg_proc_aclcheck(F_PG_ALTER_WAL_PROHIBIT_STATE,
+						 GetUserId(), ACL_EXECUTE) != ACLCHECK_OK)
+		ereport(ERROR,
+				(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+				 errmsg("permission denied for command ALTER SYSTEM"),
+				 errhint("Get execute permission for pg_alter_wal_prohibit_state() to this user.")));
+
+	/* Alter WAL prohibit state not allowed during recovery */
+	PreventCommandDuringRecovery("ALTER SYSTEM");
+
+	/* Execute function to alter wal prohibit state */
+	(void) OidFunctionCall1(F_PG_ALTER_WAL_PROHIBIT_STATE,
+							BoolGetDatum(stmt->walprohibited));
+}
+
+/*
+ * pg_alter_wal_prohibit_state()
+ *
+ * SQL callable function to alter system read write state.
+ */
+Datum
+pg_alter_wal_prohibit_state(PG_FUNCTION_ARGS)
+{
+	bool		walprohibited = PG_GETARG_BOOL(0);
+	uint32		cur_state_gen;
+
+	/* Alter WAL prohibit state not allowed during recovery */
+	PreventCommandDuringRecovery("pg_alter_wal_prohibit_state()");
+
+	/*
+	 * It is not a final state since we yet to convey this WAL prohibit state to
+	 * all backend.
+	 */
+	cur_state_gen = SetWALProhibitState(walprohibited, false);
+
+	/* Server is already in requested state */
+	if (!cur_state_gen)
+		PG_RETURN_VOID();
+
+	/*
+	 * Signal the checkpointer to do the actual state transition, and wait for
+	 * the state change to occur.
+	 */
+	RequestWALProhibitChange(cur_state_gen);
+
+	PG_RETURN_VOID();
+}
+
+/*
+ * RequestWALProhibitChange()
+ *
+ * Request checkpointer to make the WALProhibitState to read-only.
+ */
+static void
+RequestWALProhibitChange(uint32 cur_state_gen)
+{
+	/* Must not be called from checkpointer */
+	Assert(!AmCheckpointerProcess());
+	Assert(GetWALProhibitStateGen() & WALPROHIBIT_TRANSITION_IN_PROGRESS);
+
+	/*
+	 * If in a standalone backend, just do it ourselves.
+	 */
+	if (!IsPostmasterEnvironment)
+	{
+		CompleteWALProhibitChange(cur_state_gen);
+		return;
+	}
+
+	/* Signal checkpointer process */
+	SendsSignalToCheckpointer(SIGINT);
+
+	/* Wait for the state to change to read-only */
+	ConditionVariablePrepareToSleep(&WALProhibitState->walprohibit_cv);
+	for (;;)
+	{
+		/* We'll be done once wal prohibit state generation changes */
+		if (GetWALProhibitStateGen() != cur_state_gen)
+			break;
+
+		ConditionVariableSleep(&WALProhibitState->walprohibit_cv,
+							   WAIT_EVENT_WALPROHIBIT_STATE_CHANGE);
+	}
+	ConditionVariableCancelSleep();
+}
+
+/*
+ * CompleteWALProhibitChange()
+ *
+ * Checkpointer will call this to complete the requested WAL prohibit state
+ * transition.
+ */
+void
+CompleteWALProhibitChange(uint32 cur_state_gen)
+{
+	uint64		barrier_gen;
+	bool		wal_prohibited;
+
+	/*
+	 * Must be called from checkpointer. Otherwise, it must be single-user
+	 * backend.
+	 */
+	Assert(AmCheckpointerProcess() || !IsPostmasterEnvironment);
+	Assert(cur_state_gen & WALPROHIBIT_TRANSITION_IN_PROGRESS);
+
+	/*
+	 * WAL prohibit state change is initiated. We need to complete the state
+	 * transition by setting requested WAL prohibit state in all backends.
+	 */
+	elog(DEBUG1, "waiting for backends to adopt requested WAL prohibit state change");
+
+	/* Emit global barrier */
+	barrier_gen = EmitProcSignalBarrier(PROCSIGNAL_BARRIER_WALPROHIBIT);
+	WaitForProcSignalBarrier(barrier_gen);
+
+	/* And flush all inserts. */
+	XLogFlush(GetXLogInsertRecPtr());
+
+	wal_prohibited =
+		(WALPROHIBIT_NEXT_STATE(cur_state_gen) == WALPROHIBIT_STATE_READ_ONLY);
+
+	/* Set the final state */
+	(void) SetWALProhibitState(wal_prohibited, true);
+
+	/* Update the control file to make state persistent */
+	SetControlFileWALProhibitFlag(wal_prohibited);
+
+	if (wal_prohibited)
+		ereport(LOG, (errmsg("system is now read only")));
+	else
+	{
+		/*
+		 * Request checkpoint if the end-of-recovery checkpoint has been skipped
+		 * previously.
+		 */
+		if (LastCheckPointIsSkipped())
+		{
+			RequestCheckpoint(CHECKPOINT_IMMEDIATE);
+			SetLastCheckPointSkipped(false);
+		}
+		ereport(LOG, (errmsg("system is now read write")));
+	}
+
+	/* Wake up all backends waiting on this. */
+	ConditionVariableBroadcast(&WALProhibitState->walprohibit_cv);
+}
+
+/*
+ * GetWALProhibitStateGen()
+ *
+ * Atomically return the current server WAL prohibited state generation.
+ */
+uint32
+GetWALProhibitStateGen(void)
+{
+	return pg_atomic_read_u32(&WALProhibitState->shared_state_generation);
+}
+
+/*
+ * SetWALProhibitState()
+ *
+ * Increments current shared WAL prohibit state generation concerning to
+ * requested state and returns the same.
+ *
+ * For the transition state request where is_final_state is false if the server
+ * desired transition state is the same as the current state which might have
+ * been requested by some other backend and has been proceeded then the current
+ * wal prohibit generation will be returned so that this backend can wait until
+ * the shared wal prohibited generation change for the final state.  And, if the
+ * server is already completely moved to the requested state then the requester
+ * backend doesn't need to wait, in that case, 0 will be returned.
+ *
+ * The final state can only be requested by the checkpointer or by the
+ * single-user so that there will be no chance that the server is already in the
+ * desired final state.
+ */
+uint32
+SetWALProhibitState(bool wal_prohibited, bool is_final_state)
+{
+	uint32		new_state;
+	uint32		cur_state;
+	uint32		cur_state_gen;
+	uint32		next_state_gen;
+
+	/* Get the current state */
+	cur_state_gen = GetWALProhibitStateGen();
+	cur_state = WALPROHIBIT_CURRENT_STATE(cur_state_gen);
+
+	/* Compute new state */
+	if (is_final_state)
+	{
+		/*
+		 * Only checkpointer or single-user can set the final wal prohibit
+		 * state.
+		 */
+		Assert(AmCheckpointerProcess() || !IsPostmasterEnvironment);
+
+		new_state = wal_prohibited ?
+			WALPROHIBIT_STATE_READ_ONLY : WALPROHIBIT_STATE_READ_WRITE;
+
+		/*
+		 * There won't be any other process for the final state setting so that
+		 * the next final state will be the desired state.
+		 */
+		Assert(WALPROHIBIT_NEXT_STATE(cur_state) == new_state);
+	}
+	else
+	{
+		new_state = wal_prohibited ?
+			WALPROHIBIT_STATE_GOING_READ_ONLY :
+			WALPROHIBIT_STATE_GOING_READ_WRITE;
+
+		/* Server is already in the requested transition state */
+		if (cur_state == new_state)
+			return cur_state;		/* Wait for state transition completion */
+
+		/* Server is already in requested state */
+		if (WALPROHIBIT_NEXT_STATE(new_state) == cur_state)
+			return 0;		/* No wait is needed */
+
+		/* Prevent concurrent contrary in progress transition state setting */
+		if (cur_state & WALPROHIBIT_TRANSITION_IN_PROGRESS)
+		{
+			if (cur_state & WALPROHIBIT_STATE_READ_ONLY)
+				ereport(ERROR,
+						(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+						 errmsg("system state transition to read only is already in progress"),
+						 errhint("Try after sometime again.")));
+			else
+				ereport(ERROR,
+						(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+						 errmsg("system state transition to read write is already in progress"),
+						 errhint("Try after sometime again.")));
+		}
+	}
+
+	/*
+	 * Update new state generation in share memory only if the state generation
+	 * hasn't changed until now we have checked.
+	 */
+	next_state_gen = cur_state_gen + 1;
+	(void) pg_atomic_compare_exchange_u32(&WALProhibitState->shared_state_generation,
+										  &cur_state_gen, next_state_gen);
+
+	/* To be sure that any later reads of memory happen strictly after this. */
+	pg_memory_barrier();
+
+	return next_state_gen;
+}
+
+/*
+ * WALProhibitStateGenerationInit()
+ *
+ * Initialization of shared wal prohibit state generation.
+ */
+void
+WALProhibitStateGenerationInit(bool wal_prohibited)
+{
+	uint32	new_state;
+
+	Assert(AmStartupProcess() || !IsPostmasterEnvironment);
+
+	new_state = wal_prohibited ?
+		WALPROHIBIT_STATE_READ_ONLY : WALPROHIBIT_STATE_READ_WRITE;
+
+	pg_atomic_init_u32(&WALProhibitState->shared_state_generation, new_state);
+}
+
+/*
+ * WALProhibitStateShmemInit()
+ *
+ * Initialization of shared memory for WAL prohibit state.
+ */
+void
+WALProhibitStateShmemInit(void)
+{
+	bool		found;
+
+	WALProhibitState = (WALProhibitStateData *)
+		ShmemInitStruct("WAL Prohibit State",
+						sizeof(WALProhibitStateData),
+						&found);
+
+	if (found)
+		return;
+
+	/* First time through ... */
+	memset(WALProhibitState, 0, sizeof(WALProhibitStateData));
+	ConditionVariableInit(&WALProhibitState->walprohibit_cv);
+}
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index af6afcebb13..188c299bed9 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -1962,23 +1962,28 @@ StartTransaction(void)
 	Assert(s->prevSecContext == 0);
 
 	/*
-	 * Make sure we've reset xact state variables
+	 * Reset xact state variables.
 	 *
-	 * If recovery is still in progress, mark this transaction as read-only.
-	 * We have lower level defences in XLogInsert and elsewhere to stop us
-	 * from modifying data during recovery, but this gives the normal
-	 * indication to the user that the transaction is read-only.
-	 */
-	if (RecoveryInProgress())
-	{
-		s->startedInRecovery = true;
-		XactReadOnly = true;
-	}
-	else
-	{
-		s->startedInRecovery = false;
-		XactReadOnly = DefaultXactReadOnly;
-	}
+	 * If it is not currently possible to insert write-ahead log records,
+	 * either because we are still in recovery or because ALTER SYSTEM READ
+	 * ONLY has been executed, force this to be a read-only transaction.
+	 * We have lower level defences in XLogBeginInsert() and elsewhere to stop
+	 * us from modifying data during recovery when !XLogInsertAllowed(), but
+	 * this gives the normal indication to the user that the transaction is
+	 * read-only.
+	 *
+	 * On the other hand, we only need to set the startedInRecovery flag when
+	 * the transaction started during recovery, and not when WAL is otherwise
+	 * prohibited. This information is used by RelationGetIndexScan() to
+	 * decide whether to permit (1) relying on existing killed-tuple markings
+	 * and (2) further killing of index tuples. Even when WAL is prohibited
+	 * on the master, it's still the master, so the former is OK; and since
+	 * killing index tuples doesn't generate WAL, the latter is also OK.
+	 * See comments in RelationGetIndexScan() and MarkBufferDirtyHint().
+	 */
+	XactReadOnly = DefaultXactReadOnly || !XLogInsertAllowed();
+	s->startedInRecovery = RecoveryInProgress();
+
 	XactDeferrable = DefaultXactDeferrable;
 	XactIsoLevel = DefaultXactIsoLevel;
 	forceSyncCommit = false;
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 52a67b11701..8c51c554dc0 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -31,6 +31,7 @@
 #include "access/timeline.h"
 #include "access/transam.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog_internal.h"
 #include "access/xlogarchive.h"
@@ -246,9 +247,10 @@ static bool LocalPromoteIsTriggered = false;
  *		0: unconditionally not allowed to insert XLOG
  *		-1: must check RecoveryInProgress(); disallow until it is false
  * Most processes start with -1 and transition to 1 after seeing that recovery
- * is not in progress.  But we can also force the value for special cases.
- * The coding in XLogInsertAllowed() depends on the first two of these states
- * being numerically the same as bool true and false.
+ * is not in progress or the server state is not a WAL prohibited state.  But
+ * we can also force the value for special cases.  The coding in
+ * XLogInsertAllowed() depends on the first two of these states being
+ * numerically the same as bool true and false.
  */
 static int	LocalXLogInsertAllowed = -1;
 
@@ -723,6 +725,11 @@ typedef struct XLogCtlData
 	 */
 	XLogRecPtr	lastFpwDisableRecPtr;
 
+	/*
+	 * lastCheckPointSkipped indicates if the last checkpoint is skipped.
+	 */
+	bool		lastCheckPointSkipped;
+
 	slock_t		info_lck;		/* locks shared variables shown above */
 } XLogCtlData;
 
@@ -969,6 +976,7 @@ static void WALInsertLockAcquire(void);
 static void WALInsertLockAcquireExclusive(void);
 static void WALInsertLockRelease(void);
 static void WALInsertLockUpdateInsertingAt(XLogRecPtr insertingAt);
+static inline bool IsWALProhibited(void);
 
 /*
  * Insert an XLOG record represented by an already-constructed chain of data
@@ -6198,6 +6206,32 @@ SetCurrentChunkStartTime(TimestampTz xtime)
 	SpinLockRelease(&XLogCtl->info_lck);
 }
 
+/*
+ * Set or unset flag to indicating that the last checkpoint has been skipped.
+ */
+void
+SetLastCheckPointSkipped(bool ChkptSkip)
+{
+	SpinLockAcquire(&XLogCtl->info_lck);
+	XLogCtl->lastCheckPointSkipped = ChkptSkip;
+	SpinLockRelease(&XLogCtl->info_lck);
+}
+
+/*
+ * Return value of lastCheckPointSkipped flag.
+ */
+bool
+LastCheckPointIsSkipped(void)
+{
+	bool	ChkptSkipped;
+
+	SpinLockAcquire(&XLogCtl->info_lck);
+	ChkptSkipped = XLogCtl->lastCheckPointSkipped;
+	SpinLockRelease(&XLogCtl->info_lck);
+
+	return ChkptSkipped;
+}
+
 /*
  * Fetch timestamp of latest processed commit/abort record.
  * Startup process maintains an accurate local copy in XLogReceiptTime
@@ -7710,6 +7744,12 @@ StartupXLOG(void)
 	XLogCtl->LogwrtRqst.Write = EndOfLog;
 	XLogCtl->LogwrtRqst.Flush = EndOfLog;
 
+	/*
+	 * Before enabling WAL insertion, initialize WAL prohibit state in shared
+	 * memory that will decide the further WAL insert should be allowed or not.
+	 */
+	WALProhibitStateGenerationInit(ControlFile->wal_prohibited);
+
 	/*
 	 * Update full_page_writes in shared memory and write an XLOG_FPW_CHANGE
 	 * record before resource manager writes cleanup WAL records or checkpoint
@@ -7720,7 +7760,17 @@ StartupXLOG(void)
 	UpdateFullPageWrites();
 	LocalXLogInsertAllowed = -1;
 
-	if (InRecovery)
+	/*
+	 * Skip end-of-recovery checkpoint if the system is in WAL prohibited state.
+	 */
+	if (ControlFile->wal_prohibited && InRecovery)
+	{
+		SetLastCheckPointSkipped(true);
+
+		ereport(LOG,
+				(errmsg("skipping startup checkpoint because the system is read only")));
+	}
+	else if (InRecovery)
 	{
 		/*
 		 * Perform a checkpoint to update all our recovery activity to disk.
@@ -7966,6 +8016,28 @@ StartupXLOG(void)
 		RequestCheckpoint(CHECKPOINT_FORCE);
 }
 
+/* Set ControlFile's WAL prohibit flag */
+void
+SetControlFileWALProhibitFlag(bool wal_prohibited)
+{
+	LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+	ControlFile->wal_prohibited = wal_prohibited;
+	UpdateControlFile();
+	LWLockRelease(ControlFileLock);
+}
+
+/*
+ * Is the system still in WAL prohibited state?
+ */
+static inline bool
+IsWALProhibited(void)
+{
+	uint32 		cur_state = WALPROHIBIT_CURRENT_STATE(GetWALProhibitStateGen());
+
+	return (cur_state != WALPROHIBIT_STATE_READ_WRITE &&
+			cur_state != WALPROHIBIT_STATE_GOING_READ_WRITE);
+}
+
 /*
  * Checks if recovery has reached a consistent state. When consistency is
  * reached and we have a valid starting standby snapshot, tell postmaster
@@ -8181,9 +8253,9 @@ HotStandbyActiveInReplay(void)
 /*
  * Is this process allowed to insert new WAL records?
  *
- * Ordinarily this is essentially equivalent to !RecoveryInProgress().
- * But we also have provisions for forcing the result "true" or "false"
- * within specific processes regardless of the global state.
+ * Ordinarily this is essentially equivalent to !RecoveryInProgress() and
+ * !IsWALProhibited().  But we also have provisions for forcing the result
+ * "true" or "false" within specific processes regardless of the global state.
  */
 bool
 XLogInsertAllowed(void)
@@ -8202,9 +8274,20 @@ XLogInsertAllowed(void)
 	if (RecoveryInProgress())
 		return false;
 
+	/* Or, in WAL prohibited state */
+	if (IsWALProhibited())
+	{
+		/*
+		 * Set it to "unconditionally false" to avoid checking until it gets
+		 * reset.
+		 */
+		LocalXLogInsertAllowed = 0;
+		return false;
+	}
+
 	/*
-	 * On exit from recovery, reset to "unconditionally true", since there is
-	 * no need to keep checking.
+	 * On exit from recovery or WAL prohibited state, reset to "unconditionally
+	 * true", since there is no need to keep checking.
 	 */
 	LocalXLogInsertAllowed = 1;
 	return true;
@@ -8226,6 +8309,12 @@ LocalSetXLogInsertAllowed(void)
 	InitXLOGAccess();
 }
 
+void
+ResetLocalXLogInsertAllowed(void)
+{
+	LocalXLogInsertAllowed = -1;
+}
+
 /*
  * Subroutine to try to fetch and validate a prior checkpoint record.
  *
@@ -8515,9 +8604,13 @@ ShutdownXLOG(int code, Datum arg)
 	 */
 	WalSndWaitStopping();
 
+	/*
+	 * The restartpoint, checkpoint, or xlog rotation will be performed if the
+	 * WAL writing is permitted.
+	 */
 	if (RecoveryInProgress())
 		CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
-	else
+	else if (XLogInsertAllowed())
 	{
 		/*
 		 * If archiving is enabled, rotate the last XLOG file so that all the
@@ -8530,6 +8623,9 @@ ShutdownXLOG(int code, Datum arg)
 
 		CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
 	}
+	else
+		ereport(LOG,
+			   (errmsg("skipping shutdown checkpoint because the system is read only")));
 }
 
 /*
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index c6dd084fbcc..c6c2b3b6332 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1510,6 +1510,8 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
 REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
 REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
 
+REVOKE EXECUTE ON FUNCTION pg_alter_wal_prohibit_state(bool) FROM public;
+
 --
 -- We also set up some things as accessible to standard roles.
 --
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 2cef56f115f..efee35cbc94 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -659,6 +659,10 @@ AutoVacLauncherMain(int argc, char *argv[])
 
 		HandleAutoVacLauncherInterrupts();
 
+		/* If the server is read only just go back to sleep. */
+		if (!XLogInsertAllowed())
+			continue;
+
 		/*
 		 * a worker finished, or postmaster signaled failure to start a worker
 		 */
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index a7afa758b61..1d9c46de20a 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -278,7 +278,7 @@ BackgroundWriterMain(void)
 		 * Checkpointer, when active, is barely ever in its mainloop and thus
 		 * makes it hard to log regularly.
 		 */
-		if (XLogStandbyInfoActive() && !RecoveryInProgress())
+		if (XLogStandbyInfoActive() && XLogInsertAllowed())
 		{
 			TimestampTz timeout = 0;
 			TimestampTz now = GetCurrentTimestamp();
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 429c8010ef4..bd66705a9af 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -39,6 +39,7 @@
 #include <sys/time.h>
 #include <time.h>
 
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xlog_internal.h"
 #include "libpq/pqsignal.h"
@@ -342,6 +343,7 @@ CheckpointerMain(void)
 		pg_time_t	now;
 		int			elapsed_secs;
 		int			cur_timeout;
+		uint32		wal_state_gen;
 
 		/* Clear any already-pending wakeups */
 		ResetLatch(MyLatch);
@@ -352,6 +354,30 @@ CheckpointerMain(void)
 		AbsorbSyncRequests();
 		HandleCheckpointerInterrupts();
 
+		wal_state_gen = GetWALProhibitStateGen();
+
+		if (wal_state_gen & WALPROHIBIT_TRANSITION_IN_PROGRESS)
+		{
+			/* Complete WAL prohibit state change request */
+			CompleteWALProhibitChange(wal_state_gen);
+			continue;
+		}
+		else if (WALPROHIBIT_CURRENT_STATE(wal_state_gen) ==
+				 WALPROHIBIT_STATE_READ_ONLY)
+		{
+			/*
+			 * Don't do anything until someone wakes us up.  For example a
+			 * backend might later on request us to put the system back to
+			 * read-write wal prohibit sate.
+			 */
+			(void) WaitLatch(MyLatch, WL_LATCH_SET | WL_EXIT_ON_PM_DEATH, -1,
+							 WAIT_EVENT_CHECKPOINTER_MAIN);
+			continue;
+		}
+
+		Assert(WALPROHIBIT_CURRENT_STATE(wal_state_gen) ==
+			   WALPROHIBIT_STATE_READ_WRITE);
+
 		/*
 		 * Detect a pending checkpoint request by checking whether the flags
 		 * word in shared memory is nonzero.  We shouldn't need to acquire the
@@ -1336,3 +1362,16 @@ FirstCallSinceLastCheckpoint(void)
 
 	return FirstCall;
 }
+
+/*
+ * SendsSignalToCheckpointer allows a process to send a signal to the checkpoint process.
+ */
+void
+SendsSignalToCheckpointer(int signum)
+{
+	if (CheckpointerShmem->checkpointer_pid == 0)
+		elog(ERROR, "checkpointer is not running");
+
+	if (kill(CheckpointerShmem->checkpointer_pid, signum) != 0)
+		elog(ERROR, "could not signal checkpointer: %m");
+}
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 822f0ebc628..f020ff7e5a0 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -4203,6 +4203,9 @@ pgstat_get_wait_io(WaitEventIO w)
 		case WAIT_EVENT_REPLICATION_SLOT_WRITE:
 			event_name = "ReplicationSlotWrite";
 			break;
+		case WAIT_EVENT_WALPROHIBIT_STATE_CHANGE:
+			event_name = "SystemWALProhibitStateChange";
+			break;
 		case WAIT_EVENT_SLRU_FLUSH_SYNC:
 			event_name = "SLRUFlushSync";
 			break;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 96c2aaabbd6..2d000ec2ff7 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -22,6 +22,7 @@
 #include "access/subtrans.h"
 #include "access/syncscan.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "commands/async.h"
 #include "miscadmin.h"
 #include "pgstat.h"
@@ -222,6 +223,11 @@ CreateSharedMemoryAndSemaphores(void)
 	MultiXactShmemInit();
 	InitBufferPool();
 
+	/*
+	 * Set up wal probibit shared state
+	 */
+	WALProhibitStateShmemInit();
+
 	/*
 	 * Set up lock manager
 	 */
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index abdae58c476..ab5b5c888fb 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -18,6 +18,7 @@
 #include <unistd.h>
 
 #include "access/parallel.h"
+#include "access/walprohibit.h"
 #include "port/pg_bitutils.h"
 #include "commands/async.h"
 #include "miscadmin.h"
@@ -98,7 +99,6 @@ static volatile ProcSignalSlot *MyProcSignalSlot = NULL;
 static bool CheckProcSignal(ProcSignalReason reason);
 static void CleanupProcSignalState(int status, Datum arg);
 static void ResetProcSignalBarrierBits(uint32 flags);
-static bool ProcessBarrierPlaceholder(void);
 
 /*
  * ProcSignalShmemSize
@@ -538,8 +538,8 @@ ProcessProcSignalBarrier(void)
 				type = (ProcSignalBarrierType) pg_rightmost_one_pos32(flags);
 				switch (type)
 				{
-					case PROCSIGNAL_BARRIER_PLACEHOLDER:
-						processed = ProcessBarrierPlaceholder();
+					case PROCSIGNAL_BARRIER_WALPROHIBIT:
+						processed = ProcessBarrierWALProhibit();
 						break;
 				}
 
@@ -607,24 +607,6 @@ ResetProcSignalBarrierBits(uint32 flags)
 	InterruptPending = true;
 }
 
-static bool
-ProcessBarrierPlaceholder(void)
-{
-	/*
-	 * XXX. This is just a placeholder until the first real user of this
-	 * machinery gets committed. Rename PROCSIGNAL_BARRIER_PLACEHOLDER to
-	 * PROCSIGNAL_BARRIER_SOMETHING_ELSE where SOMETHING_ELSE is something
-	 * appropriately descriptive. Get rid of this function and instead have
-	 * ProcessBarrierSomethingElse. Most likely, that function should live in
-	 * the file pertaining to that subsystem, rather than here.
-	 *
-	 * The return value should be 'true' if the barrier was successfully
-	 * absorbed and 'false' if not. Note that returning 'false' can lead to
-	 * very frequent retries, so try hard to make that an uncommon case.
-	 */
-	return true;
-}
-
 /*
  * CheckProcSignal - check to see if a particular reason has been
  * signaled, and clear the signal flag.  Should be called after receiving
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 74c2162cd59..05eac206182 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -19,6 +19,7 @@
 #include "access/htup_details.h"
 #include "access/reloptions.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "catalog/catalog.h"
@@ -85,7 +86,6 @@ static void ProcessUtilitySlow(ParseState *pstate,
 							   DestReceiver *dest,
 							   QueryCompletion *qc);
 static void ExecDropStmt(DropStmt *stmt, bool isTopLevel);
-static void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
 
 /*
  * CommandIsReadOnly: is an executable query read-only?
@@ -3691,16 +3691,3 @@ GetCommandLogLevel(Node *parsetree)
 
 	return lev;
 }
-
-/*
- * AlterSystemSetWALProhibitState
- *
- * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
- */
-static void
-AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
-{
-	ereport(ERROR,
-			(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
-			 errmsg("ALTER SYSTEM READ { ONLY | WRITE } not implemented")));
-}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index a62d64eaa47..9c6d89627eb 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -225,6 +225,7 @@ static bool check_recovery_target_lsn(char **newval, void **extra, GucSource sou
 static void assign_recovery_target_lsn(const char *newval, void *extra);
 static bool check_primary_slot_name(char **newval, void **extra, GucSource source);
 static bool check_default_with_oids(bool *newval, void **extra, GucSource source);
+static const char *show_system_is_read_only(void);
 
 /* Private functions in guc-file.l that need to be called from guc.c */
 static ConfigVariable *ProcessConfigFileInternal(GucContext context,
@@ -615,6 +616,7 @@ static char *recovery_target_string;
 static char *recovery_target_xid_string;
 static char *recovery_target_name_string;
 static char *recovery_target_lsn_string;
+static bool system_is_read_only;
 
 
 /* should be static, but commands/variable.c needs to get at this */
@@ -2036,6 +2038,18 @@ static struct config_bool ConfigureNamesBool[] =
 		NULL, NULL, NULL
 	},
 
+	{
+		/* Not for general use */
+		{"system_is_read_only", PGC_INTERNAL, WAL,
+			gettext_noop("Shows whether the system is read only."),
+			NULL,
+			GUC_NO_RESET_ALL | GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE
+		},
+		&system_is_read_only,
+		false,
+		NULL, NULL, show_system_is_read_only
+	},
+
 	/* End-of-list marker */
 	{
 		{NULL, 0, 0, NULL, NULL}, NULL, false, NULL, NULL, NULL
@@ -12069,4 +12083,16 @@ check_default_with_oids(bool *newval, void **extra, GucSource source)
 	return true;
 }
 
+/*
+ * NB: The return string should be the same as the _ShowOption() for boolean
+ * type.
+ */
+static const char *
+show_system_is_read_only(void)
+{
+	if (!XLogInsertAllowed())
+		return "on";
+	return "off";
+}
+
 #include "guc-file.c"
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index 3e00ac0f701..922cd9641d8 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -290,6 +290,8 @@ main(int argc, char *argv[])
 		   (uint32) ControlFile->backupEndPoint);
 	printf(_("End-of-backup record required:        %s\n"),
 		   ControlFile->backupEndRequired ? _("yes") : _("no"));
+	printf(_("WAL write prohibited:                 %s\n"),
+		   ControlFile->wal_prohibited ? _("yes") : _("no"));
 	printf(_("wal_level setting:                    %s\n"),
 		   wal_level_str(ControlFile->wal_level));
 	printf(_("wal_log_hints setting:                %s\n"),
diff --git a/src/include/access/walprohibit.h b/src/include/access/walprohibit.h
new file mode 100644
index 00000000000..61836d61844
--- /dev/null
+++ b/src/include/access/walprohibit.h
@@ -0,0 +1,94 @@
+/*
+ * walprohibit.h
+ *
+ * PostgreSQL write-ahead log prohibit states
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/include/access/walprohibit.h
+ */
+#ifndef WALPROHIBIT_H
+#define WALPROHIBIT_H
+
+#include "access/xact.h"
+#include "access/xlog.h"
+#include "miscadmin.h"
+#include "nodes/parsenodes.h"
+
+extern bool ProcessBarrierWALProhibit(void);
+extern void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
+extern void CompleteWALProhibitChange(uint32 wal_state);
+extern uint32 GetWALProhibitStateGen(void);
+extern uint32 SetWALProhibitState(bool wal_prohibited, bool is_final_state);
+extern void MarkCheckPointSkippedInWalProhibitState(void);
+extern void WALProhibitStateGenerationInit(bool wal_prohibited);
+extern void WALProhibitStateShmemInit(void);
+
+/*
+ * The WAL Prohibit States.
+ *
+ * 	The odd number represents the transition state and whereas the even number
+ * 	represents the final state.  These states can be distinguished by checking
+ * 	the 0th bits aka transition bit.
+ */
+#define	WALPROHIBIT_STATE_READ_WRITE		(uint32) 0	/* WAL permitted */
+#define	WALPROHIBIT_STATE_GOING_READ_ONLY	(uint32) 1
+#define	WALPROHIBIT_STATE_READ_ONLY			(uint32) 2	/* WAL prohibited */
+#define	WALPROHIBIT_STATE_GOING_READ_WRITE	(uint32) 3
+
+/* The transition bit to distinguish states.  */
+#define	WALPROHIBIT_TRANSITION_IN_PROGRESS	((uint32) 1 << 0)
+
+/* Extract last two bits */
+#define	WALPROHIBIT_CURRENT_STATE(stateGeneration)	\
+	((uint32)(stateGeneration) & ((uint32) ((1 << 2) - 1)))
+#define	WALPROHIBIT_NEXT_STATE(stateGeneration)	\
+	WALPROHIBIT_CURRENT_STATE((stateGeneration + 1))
+
+/* Never reaches when WAL is prohibited. */
+static inline void
+AssertWALPermitted(void)
+{
+	/*
+	 * Recovery in the startup process never is in wal prohibited state.
+	 */
+	Assert(InRecovery || XLogInsertAllowed());
+
+#ifdef USE_ASSERT_CHECKING
+	walpermit_checked_state = WALPERMIT_CHECKED;
+#endif
+}
+
+/*
+ * XID-bearing transactions are killed off by "ALTER SYSTEM READ ONLY", so any
+ * part of the code that can only be reached with an XID assigned is never
+ * reached when WAL is prohibited.
+ */
+static inline void
+AssertWALPermittedHaveXID(void)
+{
+	/* Must be performing an INSERT, UPDATE or DELETE, so we'll have an XID */
+	Assert(FullTransactionIdIsValid(GetTopFullTransactionIdIfAny()));
+	AssertWALPermitted();
+}
+
+/*
+ * In opposite to the above assertion if a transaction doesn't have valid XID
+ * (e.g. VACUUM) then it won't be killed while changing the system state to WAL
+ * prohibited.  Therefore, we need to explicitly error out before entering into
+ * the critical section.
+ */
+static inline void
+CheckWALPermitted(void)
+{
+	if (!XLogInsertAllowed())
+		ereport(ERROR,
+				(errcode(ERRCODE_READ_ONLY_SQL_TRANSACTION),
+				 errmsg("system is now read only")));
+
+#ifdef USE_ASSERT_CHECKING
+	walpermit_checked_state = WALPERMIT_CHECKED;
+#endif
+}
+
+#endif							/* WALPROHIBIT_H */
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 221af87e715..2bcd37894f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -306,6 +306,7 @@ extern RecoveryState GetRecoveryState(void);
 extern bool HotStandbyActive(void);
 extern bool HotStandbyActiveInReplay(void);
 extern bool XLogInsertAllowed(void);
+extern void ResetLocalXLogInsertAllowed(void);
 extern void GetXLogReceiptTime(TimestampTz *rtime, bool *fromStream);
 extern XLogRecPtr GetXLogReplayRecPtr(TimeLineID *replayTLI);
 extern XLogRecPtr GetXLogInsertRecPtr(void);
@@ -314,6 +315,8 @@ extern bool RecoveryIsPaused(void);
 extern void SetRecoveryPause(bool recoveryPause);
 extern TimestampTz GetLatestXTime(void);
 extern TimestampTz GetCurrentChunkReplayStartTime(void);
+extern void SetLastCheckPointSkipped(bool ChkptSkip);
+extern bool LastCheckPointIsSkipped(void);
 
 extern void UpdateControlFile(void);
 extern uint64 GetSystemIdentifier(void);
@@ -325,6 +328,7 @@ extern void XLOGShmemInit(void);
 extern void BootStrapXLOG(void);
 extern void LocalProcessControlFile(bool reset);
 extern void StartupXLOG(void);
+extern void SetControlFileWALProhibitFlag(bool wal_prohibited);
 extern void ShutdownXLOG(int code, Datum arg);
 extern void InitXLOGAccess(void);
 extern void CreateCheckPoint(int flags);
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index 06bed90c5e9..f4dc5412ee6 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -182,6 +182,9 @@ typedef struct ControlFileData
 	int			max_locks_per_xact;
 	bool		track_commit_timestamp;
 
+	/* WAL prohibited determines if the WAL insert is allowed or not. */
+	bool		wal_prohibited;
+
 	/*
 	 * This data is used to check for hardware-architecture compatibility of
 	 * the database and the backend executable.  We need not check endianness
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index a66870bcc08..5759594e157 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10991,6 +10991,10 @@
   proname => 'pg_partition_root', prorettype => 'regclass',
   proargtypes => 'regclass', prosrc => 'pg_partition_root' },
 
+{ oid => '4142', descr => 'alter system read only state',
+  proname => 'pg_alter_wal_prohibit_state', prorettype => 'void',
+  proargtypes => 'bool', prosrc => 'pg_alter_wal_prohibit_state' },
+
 { oid => '4350', descr => 'Unicode normalization',
   proname => 'normalize', prorettype => 'text', proargtypes => 'text text',
   prosrc => 'unicode_normalize_func' },
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index a821ff4f158..b226592db90 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -1021,6 +1021,7 @@ typedef enum
 	WAIT_EVENT_REPLICATION_SLOT_RESTORE_SYNC,
 	WAIT_EVENT_REPLICATION_SLOT_SYNC,
 	WAIT_EVENT_REPLICATION_SLOT_WRITE,
+	WAIT_EVENT_WALPROHIBIT_STATE_CHANGE,
 	WAIT_EVENT_SLRU_FLUSH_SYNC,
 	WAIT_EVENT_SLRU_READ,
 	WAIT_EVENT_SLRU_SYNC,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e6..ad5e3ba5724 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -42,4 +42,6 @@ extern void CheckpointerShmemInit(void);
 
 extern bool FirstCallSinceLastCheckpoint(void);
 
+extern void SendsSignalToCheckpointer(int signum);
+
 #endif							/* _BGWRITER_H */
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f38..bae06202b4a 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -48,12 +48,7 @@ typedef enum
 
 typedef enum
 {
-	/*
-	 * XXX. PROCSIGNAL_BARRIER_PLACEHOLDER should be replaced when the first
-	 * real user of the ProcSignalBarrier mechanism is added. It's just here
-	 * for now because we can't have an empty enum.
-	 */
-	PROCSIGNAL_BARRIER_PLACEHOLDER = 0
+	PROCSIGNAL_BARRIER_WALPROHIBIT = 0
 } ProcSignalBarrierType;
 
 /*
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 4f524a36f80..aa8f61b3c98 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2679,6 +2679,7 @@ WALAvailability
 WALInsertLock
 WALInsertLockPadded
 WALOpenSegment
+WALProhibitStateData
 WALReadError
 WALSegmentCloseCB
 WALSegmentContext
-- 
2.18.0

From 2badd4eaa98f517eaa4fed5456cb39fad3c07ae2 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Tue, 14 Jul 2020 02:30:44 -0400
Subject: [PATCH v10 6/6] WIP - Documentation.

TODOs:

1] Documentation regarding ALTER SYSTEM READ/WRITE
---
 src/backend/access/transam/README | 60 ++++++++++++++++++++++++++++---
 src/backend/storage/page/README   | 12 +++----
 2 files changed, 61 insertions(+), 11 deletions(-)

diff --git a/src/backend/access/transam/README b/src/backend/access/transam/README
index 1edc8180c12..fe7a84f93da 100644
--- a/src/backend/access/transam/README
+++ b/src/backend/access/transam/README
@@ -442,8 +442,8 @@ to be modified.
 2. START_CRIT_SECTION()  (Any error during the next three steps must cause a
 PANIC because the shared buffers will contain unlogged changes, which we
 have to ensure don't get to disk.  Obviously, you should check conditions
-such as whether there's enough free space on the page before you start the
-critical section.)
+such as whether there's WAL write permission and enough free space on the page
+before you start the critical section.)
 
 3. Apply the required changes to the shared buffer(s).
 
@@ -486,6 +486,54 @@ with the incomplete-split flag set, it will finish the interrupted split by
 inserting the key to the parent, before proceeding.
 
 
+Read only system state
+----------------------
+
+The system state when it is not currently possible to insert write-ahead log
+records, either because the system is still in recovery or because the system
+forced to WAL prohibited by executing ALTER SYSTEM READ ONLY.  We have a
+lower-level defense in XLogBeginInsert() and elsewhere to stop us from modifying
+data during recovery when !XLogInsertAllowed(), but if XLogBeginInsert() is
+inside the critical section we must not depend on it to report an error.
+Otherwise, it will cause PANIC as mentioned previously.
+
+We do not reach the point where we try to write WAL during recovery but ALTER
+SYSTEM READ ONLY can be executed anytime by the user to stop WAL writing.  Any
+backends which receive read-only system state transition barrier interrupt need
+to stop WAL writing immediately.  For barrier absorption the backed(s) will kill
+the running transaction which has valid XID indicates that the transaction has
+performed and/or planning WAL write.  The transaction which doesn't acquire
+valid XID yet or operation such VACUUM or CONCURRENT CREATE INDEX which not
+necessary have valid XID for WAL will not be prevented while barrier processing,
+and those might hit the error from XLogBeginInsert() while trying to write WAL
+in read only system state.  To prevent such error from XLogBeginInsert() inside
+the critical section the WAL write permission has to check before
+START_CRIT_SECTION().
+
+To enforce the practice to check WAL permission before entering into critical
+section for the WAL write, we have added an assert check flag that indicates
+permission has been checked before calling XLogBeginInsert().  If not,
+XLogBeginInsert() will have assertion failure.  WAL permission check is not
+mandatory if the XLogBeginInsert() is not inside the critical section where
+throwing the error is acceptable.  To get permission check flag set either
+CheckWALPermitted(), AssertWALPermitted_HaveXID(), or AssertWALPermitted()
+should be called before START_CRIT_SECTION().  This flag automatically resets
+while exiting from the critical section.  The rule to place either of permission
+check routines will be:
+
+	The places where WAL write operation in critical can be expected without
+	having valid XID (e.g vacuum) need to protect by CheckWALPermitted(), so
+	that error can be reported outside before critical section.
+
+	The places where INSERT and UPDATE are expected which are never happened
+	without valid XID can be checked using AssertWALPermitted_HaveXID.  So that
+	non-assert build will not have the checking overhead.
+
+	The places we know that we cannot be reached in the read-only state and may
+	or may not have XID, but need to ensure the permission has been checked on
+	assert enabled build should use AssertWALPermitted().
+
+
 Constructing a WAL record
 -------------------------
 
@@ -531,7 +579,8 @@ Details of the API functions:
 
 void XLogBeginInsert(void)
 
-    Must be called before XLogRegisterBuffer and XLogRegisterData.
+	Must be called before XLogRegisterBuffer and XLogRegisterData.  WAL
+	permission must be check before calling it in a critical section.
 
 void XLogResetInsertion(void)
 
@@ -638,8 +687,9 @@ MarkBufferDirtyHint() to mark the block dirty.
 If the buffer is clean and checksums are in use then MarkBufferDirtyHint()
 inserts an XLOG_FPI_FOR_HINT record to ensure that we take a full page image
 that includes the hint. We do this to avoid a partial page write, when we
-write the dirtied page. WAL is not written during recovery, so we simply skip
-dirtying blocks because of hints when in recovery.
+write the dirtied page. WAL is not written while in read only (i.e. during
+recovery or in WAL prohibit state), so we simply skip dirtying blocks because of
+hints when in recovery.
 
 If you do decide to optimise away a WAL record, then any calls to
 MarkBufferDirty() must be replaced by MarkBufferDirtyHint(),
diff --git a/src/backend/storage/page/README b/src/backend/storage/page/README
index e30d7ac59ad..15f0bb4b7b5 100644
--- a/src/backend/storage/page/README
+++ b/src/backend/storage/page/README
@@ -56,9 +56,9 @@ WAL is a fatal error and prevents further recovery, whereas a checksum failure
 on a normal data block is a hard error but not a critical one for the server,
 even if it is a very bad thing for the user.
 
-New WAL records cannot be written during recovery, so hint bits set during
-recovery must not dirty the page if the buffer is not already dirty, when
-checksums are enabled.  Systems in Hot-Standby mode may benefit from hint bits
-being set, but with checksums enabled, a page cannot be dirtied after setting a
-hint bit (due to the torn page risk). So, it must wait for full-page images
-containing the hint bit updates to arrive from the primary.
+New WAL records cannot be written during recovery or or while in WAL prohibit
+state, so hint bits set during recovery must not dirty the page if the buffer is
+not already dirty, when checksums are enabled.  Systems in Hot-Standby mode may
+benefit from hint bits being set, but with checksums enabled, a page cannot be
+dirtied after setting a hint bit (due to the torn page risk). So, it must wait
+for full-page images containing the hint bit updates to arrive from the primary.
-- 
2.18.0

From c46845a9b01087808d13d9013915d6f301a7699b Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 19 Jun 2020 06:29:36 -0400
Subject: [PATCH v11 4/6] Implement ALTER SYSTEM READ ONLY using global
 barrier.

Implementation:

 1. When a user tried to change server state to WAL-Prohibited using
    ALTER SYSTEM READ ONLY command or by calling
    pg_alter_wal_prohibit_state(true) sql function, the current state
    generation to inprogress in shared memory marked and signaled
    checkpointer process.  Checkpointer, noticing that the current state
    generation has WALPROHIBIT_TRANSITION_IN_PROGRESS flag set, does the
    barrier request, and then acknowledges back to the backend who
    requested the state change once the transition has been completed.
    Final state will be updated in control file to make it persistent
    across the system restarts.

 2. When a backend receives the WAL-Prohibited barrier, at that moment if
    it is already in a transaction and the transaction already assigned XID,
    then the backend will be killed by throwing FATAL(XXX: need more discussion
    on this)

 3. Otherwise, if that backend running transaction without valid XID then, we
    don't need to do anything special right now, simply call
    ResetLocalXLogInsertAllowed() so that any future WAL insert in will check
    XLogInsertAllowed() first which set ready only state appropriately.

 4. A new transaction (in an existing or in a new backend) starts as a
    read-only transaction.

 5. Autovacuum launcher as well as checkpointer will not do anything in
    WAL-Prohibited server state until someone wakes us up.  E.g. a backend
    might later on request us to put the system back to read-write.
 5. The Autovacuum launcher, as well as the checkpointer, will not do
    anything while in the WAL-Prohibited server state until someone wakes
    up.  E.g. user might, later on, request us to put the system back to
    read-write by executing ALTER SYSTEM READ WRITE.

 6. At shutdown in WAL-Prohibited mode, we'll skip shutdown checkpoint
    and xlog rotation. Starting up again will perform crash recovery(XXX:
    need some discussion on this as well) but the end of recovery checkpoint
    will be skipped and it will be performed when the system changed to
    WAL-Permitted mode.

 7. ALTER SYSTEM READ ONLY/WRITE is restricted on standby server.

 8. To execute ALTER SYSTEM READ ONLY/WRITE, the user should have execute
    permssion on pg_alter_wal_prohibit_state() function.

 9. Add system_is_read_only GUC show the system state -- will true when system
    is wal prohibited or in recovery.
---
 src/backend/access/transam/Makefile      |   1 +
 src/backend/access/transam/walprohibit.c | 390 +++++++++++++++++++++++
 src/backend/access/transam/xact.c        |  37 ++-
 src/backend/access/transam/xlog.c        | 116 ++++++-
 src/backend/catalog/system_views.sql     |   2 +
 src/backend/postmaster/autovacuum.c      |   4 +
 src/backend/postmaster/bgwriter.c        |   2 +-
 src/backend/postmaster/checkpointer.c    |  39 +++
 src/backend/postmaster/pgstat.c          |   3 +
 src/backend/storage/ipc/ipci.c           |   6 +
 src/backend/storage/ipc/procsignal.c     |  24 +-
 src/backend/tcop/utility.c               |  15 +-
 src/backend/utils/misc/guc.c             |  26 ++
 src/bin/pg_controldata/pg_controldata.c  |   2 +
 src/include/access/walprohibit.h         |  94 ++++++
 src/include/access/xlog.h                |   4 +
 src/include/catalog/pg_control.h         |   3 +
 src/include/catalog/pg_proc.dat          |   4 +
 src/include/pgstat.h                     |   1 +
 src/include/postmaster/bgwriter.h        |   2 +
 src/include/storage/procsignal.h         |   7 +-
 src/tools/pgindent/typedefs.list         |   1 +
 22 files changed, 715 insertions(+), 68 deletions(-)
 create mode 100644 src/backend/access/transam/walprohibit.c
 create mode 100644 src/include/access/walprohibit.h

diff --git a/src/backend/access/transam/Makefile b/src/backend/access/transam/Makefile
index 595e02de722..b5322a69954 100644
--- a/src/backend/access/transam/Makefile
+++ b/src/backend/access/transam/Makefile
@@ -26,6 +26,7 @@ OBJS = \
 	twophase.o \
 	twophase_rmgr.o \
 	varsup.o \
+	walprohibit.o \
 	xact.o \
 	xlog.o \
 	xlogarchive.o \
diff --git a/src/backend/access/transam/walprohibit.c b/src/backend/access/transam/walprohibit.c
new file mode 100644
index 00000000000..00c8894d806
--- /dev/null
+++ b/src/backend/access/transam/walprohibit.c
@@ -0,0 +1,390 @@
+/*-------------------------------------------------------------------------
+ *
+ * walprohibit.c
+ * 		PostgreSQL write-ahead log prohibit states
+ *
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/backend/access/transam/walprohibit.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/walprohibit.h"
+#include "fmgr.h"
+#include "pgstat.h"
+#include "port/atomics.h"
+#include "postmaster/bgwriter.h"
+#include "storage/condition_variable.h"
+#include "storage/procsignal.h"
+#include "storage/shmem.h"
+#include "utils/acl.h"
+#include "utils/fmgroids.h"
+#include "utils/fmgrprotos.h"
+
+/*
+ * Shared-memory WAL prohibit state
+ */
+typedef struct WALProhibitStateData
+{
+	/*
+	 * Indicates current WAL prohibit state generation and the last two bits of
+	 * this generation indicates current wal prohibit state.
+	 */
+	pg_atomic_uint32 shared_state_generation;
+
+	/* Signaled when requested WAL prohibit state changes */
+	ConditionVariable walprohibit_cv;
+} WALProhibitStateData;
+
+static WALProhibitStateData *WALProhibitState = NULL;
+
+static void RequestWALProhibitChange(uint32 cur_state_gen);
+
+/*
+ * ProcessBarrierWALProhibit()
+ *
+ * Handle WAL prohibit state change request.
+ */
+bool
+ProcessBarrierWALProhibit(void)
+{
+	/*
+	 * Kill off any transactions that have an XID *before* allowing the system
+	 * to go WAL prohibit state.
+	 */
+	if (FullTransactionIdIsValid(GetTopFullTransactionIdIfAny()))
+	{
+		/*
+		 * Should be here only while transiting towards the WAL prohibit state.
+		 */
+		Assert(WALPROHIBIT_CURRENT_STATE(GetWALProhibitStateGen()) ==
+			   WALPROHIBIT_STATE_GOING_READ_ONLY);
+
+		/*
+		 * XXX: Kill off the whole session by throwing FATAL instead of
+		 * killing transaction by throwing ERROR due to following reasons that
+		 * need be thought:
+		 *
+		 * 1. Due to some presents challenges with the wire protocol, we could
+		 * not simply kill of idle transaction.
+		 *
+		 * 2. If we are here in subtransaction then the ERROR will kill the
+		 * current subtransaction only. In the case of invalidations, that
+		 * might be good enough, but for XID assignment it's not, because
+		 * assigning an XID to a subtransaction also causes higher
+		 * sub-transaction levels and the parent transaction to get XIDs.
+		 */
+		ereport(FATAL,
+				(errcode(ERRCODE_ACTIVE_SQL_TRANSACTION),
+				 errmsg("system is now read only"),
+				 errhint("Sessions with open write transactions must be terminated.")));
+	}
+
+	/* Return to "check" state */
+	ResetLocalXLogInsertAllowed();
+
+	return true;
+}
+
+/*
+ * AlterSystemSetWALProhibitState()
+ *
+ * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
+ */
+void
+AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
+{
+	/* Check permission for pg_alter_wal_prohibit_state() */
+	if (pg_proc_aclcheck(F_PG_ALTER_WAL_PROHIBIT_STATE,
+						 GetUserId(), ACL_EXECUTE) != ACLCHECK_OK)
+		ereport(ERROR,
+				(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+				 errmsg("permission denied for command ALTER SYSTEM"),
+				 errhint("Get execute permission for pg_alter_wal_prohibit_state() to this user.")));
+
+	/* Alter WAL prohibit state not allowed during recovery */
+	PreventCommandDuringRecovery("ALTER SYSTEM");
+
+	/* Execute function to alter wal prohibit state */
+	(void) OidFunctionCall1(F_PG_ALTER_WAL_PROHIBIT_STATE,
+							BoolGetDatum(stmt->walprohibited));
+}
+
+/*
+ * pg_alter_wal_prohibit_state()
+ *
+ * SQL callable function to alter system read write state.
+ */
+Datum
+pg_alter_wal_prohibit_state(PG_FUNCTION_ARGS)
+{
+	bool		walprohibited = PG_GETARG_BOOL(0);
+	uint32		cur_state_gen;
+
+	/* Alter WAL prohibit state not allowed during recovery */
+	PreventCommandDuringRecovery("pg_alter_wal_prohibit_state()");
+
+	/*
+	 * It is not a final state since we yet to convey this WAL prohibit state to
+	 * all backend.
+	 */
+	cur_state_gen = SetWALProhibitState(walprohibited, false);
+
+	/* Server is already in requested state */
+	if (!cur_state_gen)
+		PG_RETURN_VOID();
+
+	/*
+	 * Signal the checkpointer to do the actual state transition, and wait for
+	 * the state change to occur.
+	 */
+	RequestWALProhibitChange(cur_state_gen);
+
+	PG_RETURN_VOID();
+}
+
+/*
+ * RequestWALProhibitChange()
+ *
+ * Request checkpointer to make the WALProhibitState to read-only.
+ */
+static void
+RequestWALProhibitChange(uint32 cur_state_gen)
+{
+	/* Must not be called from checkpointer */
+	Assert(!AmCheckpointerProcess());
+	Assert(GetWALProhibitStateGen() & WALPROHIBIT_TRANSITION_IN_PROGRESS);
+
+	/*
+	 * If in a standalone backend, just do it ourselves.
+	 */
+	if (!IsPostmasterEnvironment)
+	{
+		CompleteWALProhibitChange(cur_state_gen);
+		return;
+	}
+
+	/* Signal checkpointer process */
+	SendsSignalToCheckpointer(SIGINT);
+
+	/* Wait for the state to change to read-only */
+	ConditionVariablePrepareToSleep(&WALProhibitState->walprohibit_cv);
+	for (;;)
+	{
+		/* We'll be done once wal prohibit state generation changes */
+		if (GetWALProhibitStateGen() != cur_state_gen)
+			break;
+
+		ConditionVariableSleep(&WALProhibitState->walprohibit_cv,
+							   WAIT_EVENT_WALPROHIBIT_STATE_CHANGE);
+	}
+	ConditionVariableCancelSleep();
+}
+
+/*
+ * CompleteWALProhibitChange()
+ *
+ * Checkpointer will call this to complete the requested WAL prohibit state
+ * transition.
+ */
+void
+CompleteWALProhibitChange(uint32 cur_state_gen)
+{
+	uint64		barrier_gen;
+	bool		wal_prohibited;
+
+	/*
+	 * Must be called from checkpointer. Otherwise, it must be single-user
+	 * backend.
+	 */
+	Assert(AmCheckpointerProcess() || !IsPostmasterEnvironment);
+	Assert(cur_state_gen & WALPROHIBIT_TRANSITION_IN_PROGRESS);
+
+	/*
+	 * WAL prohibit state change is initiated. We need to complete the state
+	 * transition by setting requested WAL prohibit state in all backends.
+	 */
+	elog(DEBUG1, "waiting for backends to adopt requested WAL prohibit state change");
+
+	/* Emit global barrier */
+	barrier_gen = EmitProcSignalBarrier(PROCSIGNAL_BARRIER_WALPROHIBIT);
+	WaitForProcSignalBarrier(barrier_gen);
+
+	/* And flush all inserts. */
+	XLogFlush(GetXLogInsertRecPtr());
+
+	wal_prohibited =
+		(WALPROHIBIT_NEXT_STATE(cur_state_gen) == WALPROHIBIT_STATE_READ_ONLY);
+
+	/* Set the final state */
+	(void) SetWALProhibitState(wal_prohibited, true);
+
+	/* Update the control file to make state persistent */
+	SetControlFileWALProhibitFlag(wal_prohibited);
+
+	if (wal_prohibited)
+		ereport(LOG, (errmsg("system is now read only")));
+	else
+	{
+		/*
+		 * Request checkpoint if the end-of-recovery checkpoint has been skipped
+		 * previously.
+		 */
+		if (LastCheckPointIsSkipped())
+		{
+			RequestCheckpoint(CHECKPOINT_IMMEDIATE);
+			SetLastCheckPointSkipped(false);
+		}
+		ereport(LOG, (errmsg("system is now read write")));
+	}
+
+	/* Wake up all backends waiting on this. */
+	ConditionVariableBroadcast(&WALProhibitState->walprohibit_cv);
+}
+
+/*
+ * GetWALProhibitStateGen()
+ *
+ * Atomically return the current server WAL prohibited state generation.
+ */
+uint32
+GetWALProhibitStateGen(void)
+{
+	return pg_atomic_read_u32(&WALProhibitState->shared_state_generation);
+}
+
+/*
+ * SetWALProhibitState()
+ *
+ * Increments current shared WAL prohibit state generation concerning to
+ * requested state and returns the same.
+ *
+ * For the transition state request where is_final_state is false if the server
+ * desired transition state is the same as the current state which might have
+ * been requested by some other backend and has been proceeded then the current
+ * wal prohibit generation will be returned so that this backend can wait until
+ * the shared wal prohibited generation change for the final state.  And, if the
+ * server is already completely moved to the requested state then the requester
+ * backend doesn't need to wait, in that case, 0 will be returned.
+ *
+ * The final state can only be requested by the checkpointer or by the
+ * single-user so that there will be no chance that the server is already in the
+ * desired final state.
+ */
+uint32
+SetWALProhibitState(bool wal_prohibited, bool is_final_state)
+{
+	uint32		new_state;
+	uint32		cur_state;
+	uint32		cur_state_gen;
+	uint32		next_state_gen;
+
+	/* Get the current state */
+	cur_state_gen = GetWALProhibitStateGen();
+	cur_state = WALPROHIBIT_CURRENT_STATE(cur_state_gen);
+
+	/* Compute new state */
+	if (is_final_state)
+	{
+		/*
+		 * Only checkpointer or single-user can set the final wal prohibit
+		 * state.
+		 */
+		Assert(AmCheckpointerProcess() || !IsPostmasterEnvironment);
+
+		new_state = wal_prohibited ?
+			WALPROHIBIT_STATE_READ_ONLY : WALPROHIBIT_STATE_READ_WRITE;
+
+		/*
+		 * There won't be any other process for the final state setting so that
+		 * the next final state will be the desired state.
+		 */
+		Assert(WALPROHIBIT_NEXT_STATE(cur_state) == new_state);
+	}
+	else
+	{
+		new_state = wal_prohibited ?
+			WALPROHIBIT_STATE_GOING_READ_ONLY :
+			WALPROHIBIT_STATE_GOING_READ_WRITE;
+
+		/* Server is already in the requested transition state */
+		if (cur_state == new_state)
+			return cur_state;		/* Wait for state transition completion */
+
+		/* Server is already in requested state */
+		if (WALPROHIBIT_NEXT_STATE(new_state) == cur_state)
+			return 0;		/* No wait is needed */
+
+		/* Prevent concurrent contrary in progress transition state setting */
+		if (cur_state & WALPROHIBIT_TRANSITION_IN_PROGRESS)
+		{
+			if (cur_state & WALPROHIBIT_STATE_READ_ONLY)
+				ereport(ERROR,
+						(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+						 errmsg("system state transition to read only is already in progress"),
+						 errhint("Try after sometime again.")));
+			else
+				ereport(ERROR,
+						(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+						 errmsg("system state transition to read write is already in progress"),
+						 errhint("Try after sometime again.")));
+		}
+	}
+
+	/*
+	 * Update new state generation in share memory only if the state generation
+	 * hasn't changed until now we have checked.
+	 */
+	next_state_gen = cur_state_gen + 1;
+	(void) pg_atomic_compare_exchange_u32(&WALProhibitState->shared_state_generation,
+										  &cur_state_gen, next_state_gen);
+
+	/* To be sure that any later reads of memory happen strictly after this. */
+	pg_memory_barrier();
+
+	return next_state_gen;
+}
+
+/*
+ * WALProhibitStateGenerationInit()
+ *
+ * Initialization of shared wal prohibit state generation.
+ */
+void
+WALProhibitStateGenerationInit(bool wal_prohibited)
+{
+	uint32	new_state;
+
+	Assert(AmStartupProcess() || !IsPostmasterEnvironment);
+
+	new_state = wal_prohibited ?
+		WALPROHIBIT_STATE_READ_ONLY : WALPROHIBIT_STATE_READ_WRITE;
+
+	pg_atomic_init_u32(&WALProhibitState->shared_state_generation, new_state);
+}
+
+/*
+ * WALProhibitStateShmemInit()
+ *
+ * Initialization of shared memory for WAL prohibit state.
+ */
+void
+WALProhibitStateShmemInit(void)
+{
+	bool		found;
+
+	WALProhibitState = (WALProhibitStateData *)
+		ShmemInitStruct("WAL Prohibit State",
+						sizeof(WALProhibitStateData),
+						&found);
+
+	if (found)
+		return;
+
+	/* First time through ... */
+	memset(WALProhibitState, 0, sizeof(WALProhibitStateData));
+	ConditionVariableInit(&WALProhibitState->walprohibit_cv);
+}
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 03c553e7eaa..485e441225b 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -1962,23 +1962,28 @@ StartTransaction(void)
 	Assert(s->prevSecContext == 0);
 
 	/*
-	 * Make sure we've reset xact state variables
+	 * Reset xact state variables.
 	 *
-	 * If recovery is still in progress, mark this transaction as read-only.
-	 * We have lower level defences in XLogInsert and elsewhere to stop us
-	 * from modifying data during recovery, but this gives the normal
-	 * indication to the user that the transaction is read-only.
-	 */
-	if (RecoveryInProgress())
-	{
-		s->startedInRecovery = true;
-		XactReadOnly = true;
-	}
-	else
-	{
-		s->startedInRecovery = false;
-		XactReadOnly = DefaultXactReadOnly;
-	}
+	 * If it is not currently possible to insert write-ahead log records,
+	 * either because we are still in recovery or because ALTER SYSTEM READ
+	 * ONLY has been executed, force this to be a read-only transaction.
+	 * We have lower level defences in XLogBeginInsert() and elsewhere to stop
+	 * us from modifying data during recovery when !XLogInsertAllowed(), but
+	 * this gives the normal indication to the user that the transaction is
+	 * read-only.
+	 *
+	 * On the other hand, we only need to set the startedInRecovery flag when
+	 * the transaction started during recovery, and not when WAL is otherwise
+	 * prohibited. This information is used by RelationGetIndexScan() to
+	 * decide whether to permit (1) relying on existing killed-tuple markings
+	 * and (2) further killing of index tuples. Even when WAL is prohibited
+	 * on the master, it's still the master, so the former is OK; and since
+	 * killing index tuples doesn't generate WAL, the latter is also OK.
+	 * See comments in RelationGetIndexScan() and MarkBufferDirtyHint().
+	 */
+	XactReadOnly = DefaultXactReadOnly || !XLogInsertAllowed();
+	s->startedInRecovery = RecoveryInProgress();
+
 	XactDeferrable = DefaultXactDeferrable;
 	XactIsoLevel = DefaultXactIsoLevel;
 	forceSyncCommit = false;
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13f1d8c3dc7..659376bb327 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -31,6 +31,7 @@
 #include "access/timeline.h"
 #include "access/transam.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog_internal.h"
 #include "access/xlogarchive.h"
@@ -246,9 +247,10 @@ static bool LocalPromoteIsTriggered = false;
  *		0: unconditionally not allowed to insert XLOG
  *		-1: must check RecoveryInProgress(); disallow until it is false
  * Most processes start with -1 and transition to 1 after seeing that recovery
- * is not in progress.  But we can also force the value for special cases.
- * The coding in XLogInsertAllowed() depends on the first two of these states
- * being numerically the same as bool true and false.
+ * is not in progress or the server state is not a WAL prohibited state.  But
+ * we can also force the value for special cases.  The coding in
+ * XLogInsertAllowed() depends on the first two of these states being
+ * numerically the same as bool true and false.
  */
 static int	LocalXLogInsertAllowed = -1;
 
@@ -723,6 +725,11 @@ typedef struct XLogCtlData
 	 */
 	XLogRecPtr	lastFpwDisableRecPtr;
 
+	/*
+	 * lastCheckPointSkipped indicates if the last checkpoint is skipped.
+	 */
+	bool		lastCheckPointSkipped;
+
 	slock_t		info_lck;		/* locks shared variables shown above */
 } XLogCtlData;
 
@@ -969,6 +976,7 @@ static void WALInsertLockAcquire(void);
 static void WALInsertLockAcquireExclusive(void);
 static void WALInsertLockRelease(void);
 static void WALInsertLockUpdateInsertingAt(XLogRecPtr insertingAt);
+static inline bool IsWALProhibited(void);
 
 /*
  * Insert an XLOG record represented by an already-constructed chain of data
@@ -6191,6 +6199,32 @@ SetCurrentChunkStartTime(TimestampTz xtime)
 	SpinLockRelease(&XLogCtl->info_lck);
 }
 
+/*
+ * Set or unset flag to indicating that the last checkpoint has been skipped.
+ */
+void
+SetLastCheckPointSkipped(bool ChkptSkip)
+{
+	SpinLockAcquire(&XLogCtl->info_lck);
+	XLogCtl->lastCheckPointSkipped = ChkptSkip;
+	SpinLockRelease(&XLogCtl->info_lck);
+}
+
+/*
+ * Return value of lastCheckPointSkipped flag.
+ */
+bool
+LastCheckPointIsSkipped(void)
+{
+	bool	ChkptSkipped;
+
+	SpinLockAcquire(&XLogCtl->info_lck);
+	ChkptSkipped = XLogCtl->lastCheckPointSkipped;
+	SpinLockRelease(&XLogCtl->info_lck);
+
+	return ChkptSkipped;
+}
+
 /*
  * Fetch timestamp of latest processed commit/abort record.
  * Startup process maintains an accurate local copy in XLogReceiptTime
@@ -7704,6 +7738,12 @@ StartupXLOG(void)
 	XLogCtl->LogwrtRqst.Write = EndOfLog;
 	XLogCtl->LogwrtRqst.Flush = EndOfLog;
 
+	/*
+	 * Before enabling WAL insertion, initialize WAL prohibit state in shared
+	 * memory that will decide the further WAL insert should be allowed or not.
+	 */
+	WALProhibitStateGenerationInit(ControlFile->wal_prohibited);
+
 	/*
 	 * Update full_page_writes in shared memory and write an XLOG_FPW_CHANGE
 	 * record before resource manager writes cleanup WAL records or checkpoint
@@ -7714,7 +7754,17 @@ StartupXLOG(void)
 	UpdateFullPageWrites();
 	LocalXLogInsertAllowed = -1;
 
-	if (InRecovery)
+	/*
+	 * Skip end-of-recovery checkpoint if the system is in WAL prohibited state.
+	 */
+	if (ControlFile->wal_prohibited && InRecovery)
+	{
+		SetLastCheckPointSkipped(true);
+
+		ereport(LOG,
+				(errmsg("skipping startup checkpoint because the system is read only")));
+	}
+	else if (InRecovery)
 	{
 		/*
 		 * Perform a checkpoint to update all our recovery activity to disk.
@@ -7960,6 +8010,28 @@ StartupXLOG(void)
 		RequestCheckpoint(CHECKPOINT_FORCE);
 }
 
+/* Set ControlFile's WAL prohibit flag */
+void
+SetControlFileWALProhibitFlag(bool wal_prohibited)
+{
+	LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+	ControlFile->wal_prohibited = wal_prohibited;
+	UpdateControlFile();
+	LWLockRelease(ControlFileLock);
+}
+
+/*
+ * Is the system still in WAL prohibited state?
+ */
+static inline bool
+IsWALProhibited(void)
+{
+	uint32 		cur_state = WALPROHIBIT_CURRENT_STATE(GetWALProhibitStateGen());
+
+	return (cur_state != WALPROHIBIT_STATE_READ_WRITE &&
+			cur_state != WALPROHIBIT_STATE_GOING_READ_WRITE);
+}
+
 /*
  * Checks if recovery has reached a consistent state. When consistency is
  * reached and we have a valid starting standby snapshot, tell postmaster
@@ -8175,9 +8247,9 @@ HotStandbyActiveInReplay(void)
 /*
  * Is this process allowed to insert new WAL records?
  *
- * Ordinarily this is essentially equivalent to !RecoveryInProgress().
- * But we also have provisions for forcing the result "true" or "false"
- * within specific processes regardless of the global state.
+ * Ordinarily this is essentially equivalent to !RecoveryInProgress() and
+ * !IsWALProhibited().  But we also have provisions for forcing the result
+ * "true" or "false" within specific processes regardless of the global state.
  */
 bool
 XLogInsertAllowed(void)
@@ -8196,9 +8268,20 @@ XLogInsertAllowed(void)
 	if (RecoveryInProgress())
 		return false;
 
+	/* Or, in WAL prohibited state */
+	if (IsWALProhibited())
+	{
+		/*
+		 * Set it to "unconditionally false" to avoid checking until it gets
+		 * reset.
+		 */
+		LocalXLogInsertAllowed = 0;
+		return false;
+	}
+
 	/*
-	 * On exit from recovery, reset to "unconditionally true", since there is
-	 * no need to keep checking.
+	 * On exit from recovery or WAL prohibited state, reset to "unconditionally
+	 * true", since there is no need to keep checking.
 	 */
 	LocalXLogInsertAllowed = 1;
 	return true;
@@ -8220,6 +8303,12 @@ LocalSetXLogInsertAllowed(void)
 	InitXLOGAccess();
 }
 
+void
+ResetLocalXLogInsertAllowed(void)
+{
+	LocalXLogInsertAllowed = -1;
+}
+
 /*
  * Subroutine to try to fetch and validate a prior checkpoint record.
  *
@@ -8509,9 +8598,13 @@ ShutdownXLOG(int code, Datum arg)
 	 */
 	WalSndWaitStopping();
 
+	/*
+	 * The restartpoint, checkpoint, or xlog rotation will be performed if the
+	 * WAL writing is permitted.
+	 */
 	if (RecoveryInProgress())
 		CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
-	else
+	else if (XLogInsertAllowed())
 	{
 		/*
 		 * If archiving is enabled, rotate the last XLOG file so that all the
@@ -8524,6 +8617,9 @@ ShutdownXLOG(int code, Datum arg)
 
 		CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
 	}
+	else
+		ereport(LOG,
+			   (errmsg("skipping shutdown checkpoint because the system is read only")));
 }
 
 /*
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 2e4aa1c4b66..717104f0216 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1504,6 +1504,8 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
 REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
 REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
 
+REVOKE EXECUTE ON FUNCTION pg_alter_wal_prohibit_state(bool) FROM public;
+
 --
 -- We also set up some things as accessible to standard roles.
 --
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index aa5b97fbacb..9c553804d96 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -659,6 +659,10 @@ AutoVacLauncherMain(int argc, char *argv[])
 
 		HandleAutoVacLauncherInterrupts();
 
+		/* If the server is read only just go back to sleep. */
+		if (!XLogInsertAllowed())
+			continue;
+
 		/*
 		 * a worker finished, or postmaster signaled failure to start a worker
 		 */
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index a7afa758b61..1d9c46de20a 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -278,7 +278,7 @@ BackgroundWriterMain(void)
 		 * Checkpointer, when active, is barely ever in its mainloop and thus
 		 * makes it hard to log regularly.
 		 */
-		if (XLogStandbyInfoActive() && !RecoveryInProgress())
+		if (XLogStandbyInfoActive() && XLogInsertAllowed())
 		{
 			TimestampTz timeout = 0;
 			TimestampTz now = GetCurrentTimestamp();
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 429c8010ef4..bd66705a9af 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -39,6 +39,7 @@
 #include <sys/time.h>
 #include <time.h>
 
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xlog_internal.h"
 #include "libpq/pqsignal.h"
@@ -342,6 +343,7 @@ CheckpointerMain(void)
 		pg_time_t	now;
 		int			elapsed_secs;
 		int			cur_timeout;
+		uint32		wal_state_gen;
 
 		/* Clear any already-pending wakeups */
 		ResetLatch(MyLatch);
@@ -352,6 +354,30 @@ CheckpointerMain(void)
 		AbsorbSyncRequests();
 		HandleCheckpointerInterrupts();
 
+		wal_state_gen = GetWALProhibitStateGen();
+
+		if (wal_state_gen & WALPROHIBIT_TRANSITION_IN_PROGRESS)
+		{
+			/* Complete WAL prohibit state change request */
+			CompleteWALProhibitChange(wal_state_gen);
+			continue;
+		}
+		else if (WALPROHIBIT_CURRENT_STATE(wal_state_gen) ==
+				 WALPROHIBIT_STATE_READ_ONLY)
+		{
+			/*
+			 * Don't do anything until someone wakes us up.  For example a
+			 * backend might later on request us to put the system back to
+			 * read-write wal prohibit sate.
+			 */
+			(void) WaitLatch(MyLatch, WL_LATCH_SET | WL_EXIT_ON_PM_DEATH, -1,
+							 WAIT_EVENT_CHECKPOINTER_MAIN);
+			continue;
+		}
+
+		Assert(WALPROHIBIT_CURRENT_STATE(wal_state_gen) ==
+			   WALPROHIBIT_STATE_READ_WRITE);
+
 		/*
 		 * Detect a pending checkpoint request by checking whether the flags
 		 * word in shared memory is nonzero.  We shouldn't need to acquire the
@@ -1336,3 +1362,16 @@ FirstCallSinceLastCheckpoint(void)
 
 	return FirstCall;
 }
+
+/*
+ * SendsSignalToCheckpointer allows a process to send a signal to the checkpoint process.
+ */
+void
+SendsSignalToCheckpointer(int signum)
+{
+	if (CheckpointerShmem->checkpointer_pid == 0)
+		elog(ERROR, "checkpointer is not running");
+
+	if (kill(CheckpointerShmem->checkpointer_pid, signum) != 0)
+		elog(ERROR, "could not signal checkpointer: %m");
+}
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index e76e627c6b2..3fb5dba318a 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -4206,6 +4206,9 @@ pgstat_get_wait_io(WaitEventIO w)
 		case WAIT_EVENT_REPLICATION_SLOT_WRITE:
 			event_name = "ReplicationSlotWrite";
 			break;
+		case WAIT_EVENT_WALPROHIBIT_STATE_CHANGE:
+			event_name = "SystemWALProhibitStateChange";
+			break;
 		case WAIT_EVENT_SLRU_FLUSH_SYNC:
 			event_name = "SLRUFlushSync";
 			break;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 96c2aaabbd6..2d000ec2ff7 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -22,6 +22,7 @@
 #include "access/subtrans.h"
 #include "access/syncscan.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "commands/async.h"
 #include "miscadmin.h"
 #include "pgstat.h"
@@ -222,6 +223,11 @@ CreateSharedMemoryAndSemaphores(void)
 	MultiXactShmemInit();
 	InitBufferPool();
 
+	/*
+	 * Set up wal probibit shared state
+	 */
+	WALProhibitStateShmemInit();
+
 	/*
 	 * Set up lock manager
 	 */
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 9c3ebccd3e2..b98f88122b6 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -18,6 +18,7 @@
 #include <unistd.h>
 
 #include "access/parallel.h"
+#include "access/walprohibit.h"
 #include "port/pg_bitutils.h"
 #include "commands/async.h"
 #include "miscadmin.h"
@@ -98,7 +99,6 @@ static volatile ProcSignalSlot *MyProcSignalSlot = NULL;
 static bool CheckProcSignal(ProcSignalReason reason);
 static void CleanupProcSignalState(int status, Datum arg);
 static void ResetProcSignalBarrierBits(uint32 flags);
-static bool ProcessBarrierPlaceholder(void);
 
 /*
  * ProcSignalShmemSize
@@ -538,8 +538,8 @@ ProcessProcSignalBarrier(void)
 				type = (ProcSignalBarrierType) pg_rightmost_one_pos32(flags);
 				switch (type)
 				{
-					case PROCSIGNAL_BARRIER_PLACEHOLDER:
-						processed = ProcessBarrierPlaceholder();
+					case PROCSIGNAL_BARRIER_WALPROHIBIT:
+						processed = ProcessBarrierWALProhibit();
 						break;
 				}
 
@@ -604,24 +604,6 @@ ResetProcSignalBarrierBits(uint32 flags)
 	InterruptPending = true;
 }
 
-static bool
-ProcessBarrierPlaceholder(void)
-{
-	/*
-	 * XXX. This is just a placeholder until the first real user of this
-	 * machinery gets committed. Rename PROCSIGNAL_BARRIER_PLACEHOLDER to
-	 * PROCSIGNAL_BARRIER_SOMETHING_ELSE where SOMETHING_ELSE is something
-	 * appropriately descriptive. Get rid of this function and instead have
-	 * ProcessBarrierSomethingElse. Most likely, that function should live in
-	 * the file pertaining to that subsystem, rather than here.
-	 *
-	 * The return value should be 'true' if the barrier was successfully
-	 * absorbed and 'false' if not. Note that returning 'false' can lead to
-	 * very frequent retries, so try hard to make that an uncommon case.
-	 */
-	return true;
-}
-
 /*
  * CheckProcSignal - check to see if a particular reason has been
  * signaled, and clear the signal flag.  Should be called after receiving
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index f68b11d0af1..413913457f4 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -19,6 +19,7 @@
 #include "access/htup_details.h"
 #include "access/reloptions.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "catalog/catalog.h"
@@ -85,7 +86,6 @@ static void ProcessUtilitySlow(ParseState *pstate,
 							   DestReceiver *dest,
 							   QueryCompletion *qc);
 static void ExecDropStmt(DropStmt *stmt, bool isTopLevel);
-static void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
 
 /*
  * CommandIsReadOnly: is an executable query read-only?
@@ -3677,16 +3677,3 @@ GetCommandLogLevel(Node *parsetree)
 
 	return lev;
 }
-
-/*
- * AlterSystemSetWALProhibitState
- *
- * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
- */
-static void
-AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
-{
-	ereport(ERROR,
-			(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
-			 errmsg("ALTER SYSTEM READ { ONLY | WRITE } not implemented")));
-}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index bb34630e8e4..888015c5355 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -225,6 +225,7 @@ static bool check_recovery_target_lsn(char **newval, void **extra, GucSource sou
 static void assign_recovery_target_lsn(const char *newval, void *extra);
 static bool check_primary_slot_name(char **newval, void **extra, GucSource source);
 static bool check_default_with_oids(bool *newval, void **extra, GucSource source);
+static const char *show_system_is_read_only(void);
 
 /* Private functions in guc-file.l that need to be called from guc.c */
 static ConfigVariable *ProcessConfigFileInternal(GucContext context,
@@ -615,6 +616,7 @@ static char *recovery_target_string;
 static char *recovery_target_xid_string;
 static char *recovery_target_name_string;
 static char *recovery_target_lsn_string;
+static bool system_is_read_only;
 
 
 /* should be static, but commands/variable.c needs to get at this */
@@ -2036,6 +2038,18 @@ static struct config_bool ConfigureNamesBool[] =
 		NULL, NULL, NULL
 	},
 
+	{
+		/* Not for general use */
+		{"system_is_read_only", PGC_INTERNAL, WAL,
+			gettext_noop("Shows whether the system is read only."),
+			NULL,
+			GUC_NO_RESET_ALL | GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE
+		},
+		&system_is_read_only,
+		false,
+		NULL, NULL, show_system_is_read_only
+	},
+
 	/* End-of-list marker */
 	{
 		{NULL, 0, 0, NULL, NULL}, NULL, false, NULL, NULL, NULL
@@ -12069,4 +12083,16 @@ check_default_with_oids(bool *newval, void **extra, GucSource source)
 	return true;
 }
 
+/*
+ * NB: The return string should be the same as the _ShowOption() for boolean
+ * type.
+ */
+static const char *
+show_system_is_read_only(void)
+{
+	if (!XLogInsertAllowed())
+		return "on";
+	return "off";
+}
+
 #include "guc-file.c"
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index 3e00ac0f701..922cd9641d8 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -290,6 +290,8 @@ main(int argc, char *argv[])
 		   (uint32) ControlFile->backupEndPoint);
 	printf(_("End-of-backup record required:        %s\n"),
 		   ControlFile->backupEndRequired ? _("yes") : _("no"));
+	printf(_("WAL write prohibited:                 %s\n"),
+		   ControlFile->wal_prohibited ? _("yes") : _("no"));
 	printf(_("wal_level setting:                    %s\n"),
 		   wal_level_str(ControlFile->wal_level));
 	printf(_("wal_log_hints setting:                %s\n"),
diff --git a/src/include/access/walprohibit.h b/src/include/access/walprohibit.h
new file mode 100644
index 00000000000..61836d61844
--- /dev/null
+++ b/src/include/access/walprohibit.h
@@ -0,0 +1,94 @@
+/*
+ * walprohibit.h
+ *
+ * PostgreSQL write-ahead log prohibit states
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/include/access/walprohibit.h
+ */
+#ifndef WALPROHIBIT_H
+#define WALPROHIBIT_H
+
+#include "access/xact.h"
+#include "access/xlog.h"
+#include "miscadmin.h"
+#include "nodes/parsenodes.h"
+
+extern bool ProcessBarrierWALProhibit(void);
+extern void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
+extern void CompleteWALProhibitChange(uint32 wal_state);
+extern uint32 GetWALProhibitStateGen(void);
+extern uint32 SetWALProhibitState(bool wal_prohibited, bool is_final_state);
+extern void MarkCheckPointSkippedInWalProhibitState(void);
+extern void WALProhibitStateGenerationInit(bool wal_prohibited);
+extern void WALProhibitStateShmemInit(void);
+
+/*
+ * The WAL Prohibit States.
+ *
+ * 	The odd number represents the transition state and whereas the even number
+ * 	represents the final state.  These states can be distinguished by checking
+ * 	the 0th bits aka transition bit.
+ */
+#define	WALPROHIBIT_STATE_READ_WRITE		(uint32) 0	/* WAL permitted */
+#define	WALPROHIBIT_STATE_GOING_READ_ONLY	(uint32) 1
+#define	WALPROHIBIT_STATE_READ_ONLY			(uint32) 2	/* WAL prohibited */
+#define	WALPROHIBIT_STATE_GOING_READ_WRITE	(uint32) 3
+
+/* The transition bit to distinguish states.  */
+#define	WALPROHIBIT_TRANSITION_IN_PROGRESS	((uint32) 1 << 0)
+
+/* Extract last two bits */
+#define	WALPROHIBIT_CURRENT_STATE(stateGeneration)	\
+	((uint32)(stateGeneration) & ((uint32) ((1 << 2) - 1)))
+#define	WALPROHIBIT_NEXT_STATE(stateGeneration)	\
+	WALPROHIBIT_CURRENT_STATE((stateGeneration + 1))
+
+/* Never reaches when WAL is prohibited. */
+static inline void
+AssertWALPermitted(void)
+{
+	/*
+	 * Recovery in the startup process never is in wal prohibited state.
+	 */
+	Assert(InRecovery || XLogInsertAllowed());
+
+#ifdef USE_ASSERT_CHECKING
+	walpermit_checked_state = WALPERMIT_CHECKED;
+#endif
+}
+
+/*
+ * XID-bearing transactions are killed off by "ALTER SYSTEM READ ONLY", so any
+ * part of the code that can only be reached with an XID assigned is never
+ * reached when WAL is prohibited.
+ */
+static inline void
+AssertWALPermittedHaveXID(void)
+{
+	/* Must be performing an INSERT, UPDATE or DELETE, so we'll have an XID */
+	Assert(FullTransactionIdIsValid(GetTopFullTransactionIdIfAny()));
+	AssertWALPermitted();
+}
+
+/*
+ * In opposite to the above assertion if a transaction doesn't have valid XID
+ * (e.g. VACUUM) then it won't be killed while changing the system state to WAL
+ * prohibited.  Therefore, we need to explicitly error out before entering into
+ * the critical section.
+ */
+static inline void
+CheckWALPermitted(void)
+{
+	if (!XLogInsertAllowed())
+		ereport(ERROR,
+				(errcode(ERRCODE_READ_ONLY_SQL_TRANSACTION),
+				 errmsg("system is now read only")));
+
+#ifdef USE_ASSERT_CHECKING
+	walpermit_checked_state = WALPERMIT_CHECKED;
+#endif
+}
+
+#endif							/* WALPROHIBIT_H */
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 221af87e715..2bcd37894f9 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -306,6 +306,7 @@ extern RecoveryState GetRecoveryState(void);
 extern bool HotStandbyActive(void);
 extern bool HotStandbyActiveInReplay(void);
 extern bool XLogInsertAllowed(void);
+extern void ResetLocalXLogInsertAllowed(void);
 extern void GetXLogReceiptTime(TimestampTz *rtime, bool *fromStream);
 extern XLogRecPtr GetXLogReplayRecPtr(TimeLineID *replayTLI);
 extern XLogRecPtr GetXLogInsertRecPtr(void);
@@ -314,6 +315,8 @@ extern bool RecoveryIsPaused(void);
 extern void SetRecoveryPause(bool recoveryPause);
 extern TimestampTz GetLatestXTime(void);
 extern TimestampTz GetCurrentChunkReplayStartTime(void);
+extern void SetLastCheckPointSkipped(bool ChkptSkip);
+extern bool LastCheckPointIsSkipped(void);
 
 extern void UpdateControlFile(void);
 extern uint64 GetSystemIdentifier(void);
@@ -325,6 +328,7 @@ extern void XLOGShmemInit(void);
 extern void BootStrapXLOG(void);
 extern void LocalProcessControlFile(bool reset);
 extern void StartupXLOG(void);
+extern void SetControlFileWALProhibitFlag(bool wal_prohibited);
 extern void ShutdownXLOG(int code, Datum arg);
 extern void InitXLOGAccess(void);
 extern void CreateCheckPoint(int flags);
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index 06bed90c5e9..f4dc5412ee6 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -182,6 +182,9 @@ typedef struct ControlFileData
 	int			max_locks_per_xact;
 	bool		track_commit_timestamp;
 
+	/* WAL prohibited determines if the WAL insert is allowed or not. */
+	bool		wal_prohibited;
+
 	/*
 	 * This data is used to check for hardware-architecture compatibility of
 	 * the database and the backend executable.  We need not check endianness
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 33dacfd3403..ca9e853fe46 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11004,6 +11004,10 @@
   proname => 'pg_partition_root', prorettype => 'regclass',
   proargtypes => 'regclass', prosrc => 'pg_partition_root' },
 
+{ oid => '4142', descr => 'alter system read only state',
+  proname => 'pg_alter_wal_prohibit_state', prorettype => 'void',
+  proargtypes => 'bool', prosrc => 'pg_alter_wal_prohibit_state' },
+
 { oid => '4350', descr => 'Unicode normalization',
   proname => 'normalize', prorettype => 'text', proargtypes => 'text text',
   prosrc => 'unicode_normalize_func' },
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 257e515bfe7..3c1ad43b4b0 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -1027,6 +1027,7 @@ typedef enum
 	WAIT_EVENT_REPLICATION_SLOT_RESTORE_SYNC,
 	WAIT_EVENT_REPLICATION_SLOT_SYNC,
 	WAIT_EVENT_REPLICATION_SLOT_WRITE,
+	WAIT_EVENT_WALPROHIBIT_STATE_CHANGE,
 	WAIT_EVENT_SLRU_FLUSH_SYNC,
 	WAIT_EVENT_SLRU_READ,
 	WAIT_EVENT_SLRU_SYNC,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e6..ad5e3ba5724 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -42,4 +42,6 @@ extern void CheckpointerShmemInit(void);
 
 extern bool FirstCallSinceLastCheckpoint(void);
 
+extern void SendsSignalToCheckpointer(int signum);
+
 #endif							/* _BGWRITER_H */
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 5cb39697f38..bae06202b4a 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -48,12 +48,7 @@ typedef enum
 
 typedef enum
 {
-	/*
-	 * XXX. PROCSIGNAL_BARRIER_PLACEHOLDER should be replaced when the first
-	 * real user of the ProcSignalBarrier mechanism is added. It's just here
-	 * for now because we can't have an empty enum.
-	 */
-	PROCSIGNAL_BARRIER_PLACEHOLDER = 0
+	PROCSIGNAL_BARRIER_WALPROHIBIT = 0
 } ProcSignalBarrierType;
 
 /*
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 37bfa69edf0..fe735b548e5 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2681,6 +2681,7 @@ WALAvailability
 WALInsertLock
 WALInsertLockPadded
 WALOpenSegment
+WALProhibitStateData
 WALReadError
 WALSegmentCloseCB
 WALSegmentContext
-- 
2.18.0

From 3e151947dc16587d59d4f2cb3272a8b22d414da1 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Mon, 27 Jul 2020 02:13:36 -0400
Subject: [PATCH v11 5/6] Error or Assert before START_CRIT_SECTION for WAL
 write

Based on the following criteria added the Assert or the Error when system is
prohibited:

 - Added ERROR for the function which can be reachable without valid XID in case
   of VACUUM or CONCURRENT CREATE INDEX.  For that added common static inline
   function CheckWALPermitted().
 - Added ASSERT for the function which cannot be reached without valid XID;
   Assert to ensure XID validation. For that added AssertWALPermitted_HaveXID().

To enforce the rule to have aforesaid assert or error check before entering a
critical section for WAL write, a new walpermit_checked_state assert only flag
is added. If these check is missing then XLogBeginInsert() will have an
assertion if it is in critical section.

If we are not doing WAL insert inside the critical section then the above
checking is not necessary, we can rely on XLogBeginInsert() for that check &
report an error.
---
 contrib/pg_surgery/heap_surgery.c         | 10 ++++--
 src/backend/access/brin/brin.c            |  4 +++
 src/backend/access/brin/brin_pageops.c    | 21 ++++++++++--
 src/backend/access/brin/brin_revmap.c     | 10 +++++-
 src/backend/access/gin/ginbtree.c         | 15 +++++++--
 src/backend/access/gin/gindatapage.c      | 18 +++++++++--
 src/backend/access/gin/ginfast.c          | 11 +++++--
 src/backend/access/gin/gininsert.c        |  4 +++
 src/backend/access/gin/ginutil.c          |  9 +++++-
 src/backend/access/gin/ginvacuum.c        | 11 ++++++-
 src/backend/access/gist/gist.c            | 25 ++++++++++++---
 src/backend/access/gist/gistvacuum.c      | 13 ++++++--
 src/backend/access/hash/hash.c            | 19 +++++++++--
 src/backend/access/hash/hashinsert.c      |  9 +++++-
 src/backend/access/hash/hashovfl.c        | 22 ++++++++++---
 src/backend/access/hash/hashpage.c        |  9 ++++++
 src/backend/access/heap/heapam.c          | 26 ++++++++++++++-
 src/backend/access/heap/pruneheap.c       | 12 +++++--
 src/backend/access/heap/vacuumlazy.c      | 18 +++++++++--
 src/backend/access/heap/visibilitymap.c   | 22 +++++++++++--
 src/backend/access/nbtree/nbtdedup.c      |  3 ++
 src/backend/access/nbtree/nbtinsert.c     | 10 +++++-
 src/backend/access/nbtree/nbtpage.c       | 39 +++++++++++++++++++----
 src/backend/access/spgist/spgdoinsert.c   | 13 ++++++++
 src/backend/access/spgist/spgvacuum.c     | 22 +++++++++++--
 src/backend/access/transam/multixact.c    |  5 ++-
 src/backend/access/transam/twophase.c     |  9 ++++++
 src/backend/access/transam/varsup.c       |  4 +++
 src/backend/access/transam/walprohibit.c  | 10 ++++++
 src/backend/access/transam/xact.c         |  6 ++++
 src/backend/access/transam/xlog.c         | 27 ++++++++++++----
 src/backend/access/transam/xloginsert.c   | 13 ++++++--
 src/backend/commands/sequence.c           | 16 ++++++++++
 src/backend/commands/variable.c           |  9 ++++--
 src/backend/postmaster/checkpointer.c     |  4 +++
 src/backend/storage/buffer/bufmgr.c       | 10 +++---
 src/backend/storage/freespace/freespace.c | 10 +++++-
 src/backend/storage/lmgr/lock.c           |  6 ++--
 src/backend/utils/cache/relmapper.c       |  3 ++
 src/include/miscadmin.h                   | 27 ++++++++++++++++
 40 files changed, 463 insertions(+), 71 deletions(-)

diff --git a/contrib/pg_surgery/heap_surgery.c b/contrib/pg_surgery/heap_surgery.c
index eb96b4bb36d..53d8c9cea28 100644
--- a/contrib/pg_surgery/heap_surgery.c
+++ b/contrib/pg_surgery/heap_surgery.c
@@ -14,6 +14,7 @@
 
 #include "access/heapam.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "catalog/pg_am_d.h"
 #include "catalog/pg_proc_d.h"
 #include "miscadmin.h"
@@ -89,6 +90,7 @@ heap_force_common(FunctionCallInfo fcinfo, HeapTupleForceOption heap_force_opt)
 	OffsetNumber curr_start_ptr,
 				next_start_ptr;
 	bool		include_this_tid[MaxHeapTuplesPerPage];
+	bool		needwal;
 
 	if (RecoveryInProgress())
 		ereport(ERROR,
@@ -100,6 +102,7 @@ heap_force_common(FunctionCallInfo fcinfo, HeapTupleForceOption heap_force_opt)
 	sanity_check_tid_array(ta, &ntids);
 
 	rel = relation_open(relid, RowExclusiveLock);
+	needwal = RelationNeedsWAL(rel);
 
 	/* Check target relation. */
 	sanity_check_relation(rel);
@@ -217,6 +220,9 @@ heap_force_common(FunctionCallInfo fcinfo, HeapTupleForceOption heap_force_opt)
 		if (heap_force_opt == HEAP_FORCE_KILL && PageIsAllVisible(page))
 			visibilitymap_pin(rel, blkno, &vmbuf);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* No ereport(ERROR) from here until all the changes are logged. */
 		START_CRIT_SECTION();
 
@@ -297,12 +303,12 @@ heap_force_common(FunctionCallInfo fcinfo, HeapTupleForceOption heap_force_opt)
 			MarkBufferDirty(buf);
 
 			/* XLOG stuff */
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 				log_newpage_buffer(buf, true);
 		}
 
 		/* WAL log the VM page if it was modified. */
-		if (did_modify_vm && RelationNeedsWAL(rel))
+		if (did_modify_vm && needwal)
 			log_newpage_buffer(vmbuf, false);
 
 		END_CRIT_SECTION();
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index 1f72562c603..47142193706 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -24,6 +24,7 @@
 #include "access/relscan.h"
 #include "access/table.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/index.h"
 #include "catalog/pg_am.h"
@@ -759,6 +760,9 @@ brinbuildempty(Relation index)
 		ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermittedHaveXID();
+
 	/* Initialize and xlog metabuffer. */
 	START_CRIT_SECTION();
 	brin_metapage_init(BufferGetPage(metabuf), BrinGetPagesPerRange(index),
diff --git a/src/backend/access/brin/brin_pageops.c b/src/backend/access/brin/brin_pageops.c
index 87de0b855b5..8b377a679ab 100644
--- a/src/backend/access/brin/brin_pageops.c
+++ b/src/backend/access/brin/brin_pageops.c
@@ -14,6 +14,7 @@
 #include "access/brin_pageops.h"
 #include "access/brin_revmap.h"
 #include "access/brin_xlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -65,6 +66,7 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 	Buffer		newbuf;
 	BlockNumber newblk = InvalidBlockNumber;
 	bool		extended;
+	bool		needwal = RelationNeedsWAL(idxrel);
 
 	Assert(newsz == MAXALIGN(newsz));
 
@@ -176,13 +178,16 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 	if (((BrinPageFlags(oldpage) & BRIN_EVACUATE_PAGE) == 0) &&
 		brin_can_do_samepage_update(oldbuf, origsz, newsz))
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 		if (!PageIndexTupleOverwrite(oldpage, oldoff, (Item) unconstify(BrinTuple *, newtup), newsz))
 			elog(ERROR, "failed to replace BRIN tuple");
 		MarkBufferDirty(oldbuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(idxrel))
+		if (needwal)
 		{
 			xl_brin_samepage_update xlrec;
 			XLogRecPtr	recptr;
@@ -240,6 +245,9 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 
 		revmapbuf = brinLockRevmapPageForUpdate(revmap, heapBlk);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		/*
@@ -267,7 +275,7 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 		MarkBufferDirty(revmapbuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(idxrel))
+		if (needwal)
 		{
 			xl_brin_update xlrec;
 			XLogRecPtr	recptr;
@@ -351,6 +359,7 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	Buffer		revmapbuf;
 	ItemPointerData tid;
 	bool		extended;
+	bool		needwal;
 
 	Assert(itemsz == MAXALIGN(itemsz));
 
@@ -405,6 +414,10 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	page = BufferGetPage(*buffer);
 	blk = BufferGetBlockNumber(*buffer);
 
+	needwal = RelationNeedsWAL(idxrel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Execute the actual insertion */
 	START_CRIT_SECTION();
 	if (extended)
@@ -424,7 +437,7 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	MarkBufferDirty(revmapbuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(idxrel))
+	if (needwal)
 	{
 		xl_brin_insert xlrec;
 		XLogRecPtr	recptr;
@@ -881,6 +894,8 @@ brin_initialize_empty_new_buffer(Relation idxrel, Buffer buffer)
 			   "brin_initialize_empty_new_buffer: initializing blank page %u",
 			   BufferGetBlockNumber(buffer)));
 
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 	page = BufferGetPage(buffer);
 	brin_page_init(page, BRIN_PAGETYPE_REGULAR);
diff --git a/src/backend/access/brin/brin_revmap.c b/src/backend/access/brin/brin_revmap.c
index 35746714a7c..fd766da445d 100644
--- a/src/backend/access/brin/brin_revmap.c
+++ b/src/backend/access/brin/brin_revmap.c
@@ -26,6 +26,7 @@
 #include "access/brin_tuple.h"
 #include "access/brin_xlog.h"
 #include "access/rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -340,6 +341,7 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 	OffsetNumber revmapOffset;
 	OffsetNumber regOffset;
 	ItemId		lp;
+	bool		needwal;
 
 	revmap = brinRevmapInitialize(idxrel, &pagesPerRange, NULL);
 
@@ -404,6 +406,10 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 	 * crashed or aborted summarization; remove them silently.
 	 */
 
+	needwal = RelationNeedsWAL(idxrel);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	ItemPointerSetInvalid(&invalidIptr);
@@ -415,7 +421,7 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 	MarkBufferDirty(regBuf);
 	MarkBufferDirty(revmapBuf);
 
-	if (RelationNeedsWAL(idxrel))
+	if (needwal)
 	{
 		xl_brin_desummarize xlrec;
 		XLogRecPtr	recptr;
@@ -613,6 +619,8 @@ revmap_physical_extend(BrinRevmap *revmap)
 		return;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Ok, we have now locked the metapage and the target block. Re-initialize
 	 * the target block as a revmap page, and update the metapage.
diff --git a/src/backend/access/gin/ginbtree.c b/src/backend/access/gin/ginbtree.c
index 82788a5c367..f31590dcd75 100644
--- a/src/backend/access/gin/ginbtree.c
+++ b/src/backend/access/gin/ginbtree.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/predicate.h"
@@ -332,6 +333,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 {
 	Page		page = BufferGetPage(stack->buffer);
 	bool		result;
+	bool		needwal;
 	GinPlaceToPageRC rc;
 	uint16		xlflags = 0;
 	Page		childpage = NULL;
@@ -377,6 +379,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 								 insertdata, updateblkno,
 								 &ptp_workspace,
 								 &newlpage, &newrpage);
+	needwal = RelationNeedsWAL(btree->index) && !btree->isBuild;
 
 	if (rc == GPTP_NO_WORK)
 	{
@@ -385,10 +388,13 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 	}
 	else if (rc == GPTP_INSERT)
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		/* It will fit, perform the insertion */
 		START_CRIT_SECTION();
 
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogBeginInsert();
 			XLogRegisterBuffer(0, stack->buffer, REGBUF_STANDARD);
@@ -409,7 +415,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 			MarkBufferDirty(childbuf);
 		}
 
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 			ginxlogInsert xlrec;
@@ -547,6 +553,9 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 			}
 		}
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/*
 		 * OK, we have the new contents of the left page in a temporary copy
 		 * now (newlpage), and likewise for the new contents of the
@@ -587,7 +596,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 		}
 
 		/* write WAL record */
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gin/gindatapage.c b/src/backend/access/gin/gindatapage.c
index 7a2690e97f2..0abc5990100 100644
--- a/src/backend/access/gin/gindatapage.c
+++ b/src/backend/access/gin/gindatapage.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "lib/ilist.h"
 #include "miscadmin.h"
@@ -811,6 +812,7 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 	if (removedsomething)
 	{
 		bool		modified;
+		bool		needwal;
 
 		/*
 		 * Make sure we have a palloc'd copy of all segments, after the first
@@ -835,8 +837,12 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 			}
 		}
 
-		if (RelationNeedsWAL(indexrel))
+		needwal = RelationNeedsWAL(indexrel);
+		if (needwal)
+		{
+			CheckWALPermitted();
 			computeLeafRecompressWALData(leaf);
+		}
 
 		/* Apply changes to page */
 		START_CRIT_SECTION();
@@ -845,7 +851,7 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 
 		MarkBufferDirty(buffer);
 
-		if (RelationNeedsWAL(indexrel))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
@@ -1777,6 +1783,7 @@ createPostingTree(Relation index, ItemPointerData *items, uint32 nitems,
 	int			nrootitems;
 	int			rootsize;
 	bool		is_build = (buildStats != NULL);
+	bool		needwal;
 
 	/* Construct the new root page in memory first. */
 	tmppage = (Page) palloc(BLCKSZ);
@@ -1825,12 +1832,17 @@ createPostingTree(Relation index, ItemPointerData *items, uint32 nitems,
 	 */
 	PredicateLockPageSplit(index, BufferGetBlockNumber(entrybuffer), blkno);
 
+	needwal = RelationNeedsWAL(index) && !is_build;
+
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	PageRestoreTempPage(tmppage, page);
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index) && !is_build)
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogCreatePostingTree data;
diff --git a/src/backend/access/gin/ginfast.c b/src/backend/access/gin/ginfast.c
index 2e41b34d8d5..b8c2a993408 100644
--- a/src/backend/access/gin/ginfast.c
+++ b/src/backend/access/gin/ginfast.c
@@ -20,6 +20,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
 #include "catalog/pg_am.h"
@@ -68,6 +69,8 @@ writeListPage(Relation index, Buffer buffer,
 	PGAlignedBlock workspace;
 	char	   *ptr;
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	GinInitBuffer(buffer, GIN_LIST);
@@ -548,6 +551,7 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 	Page		metapage;
 	GinMetaPageData *metadata;
 	BlockNumber blknoToDelete;
+	bool		needwal = RelationNeedsWAL(index);
 
 	metapage = BufferGetPage(metabuffer);
 	metadata = GinPageGetMeta(metapage);
@@ -586,8 +590,11 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 		 * prepare the XLogInsert machinery for that before entering the
 		 * critical section.
 		 */
-		if (RelationNeedsWAL(index))
+		if (needwal)
+		{
+			CheckWALPermitted();
 			XLogEnsureRecordSpace(data.ndeleted, 0);
+		}
 
 		START_CRIT_SECTION();
 
@@ -625,7 +632,7 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 			MarkBufferDirty(buffers[i]);
 		}
 
-		if (RelationNeedsWAL(index))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gin/gininsert.c b/src/backend/access/gin/gininsert.c
index 77433dc8a41..989d82ffcaf 100644
--- a/src/backend/access/gin/gininsert.c
+++ b/src/backend/access/gin/gininsert.c
@@ -17,6 +17,7 @@
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/index.h"
 #include "miscadmin.h"
@@ -447,6 +448,9 @@ ginbuildempty(Relation index)
 		ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(RootBuffer, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermittedHaveXID();
+
 	/* Initialize and xlog metabuffer and root buffer. */
 	START_CRIT_SECTION();
 	GinInitMetabuffer(MetaBuffer);
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index ef9b56fd363..b48ea1a746a 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -17,6 +17,7 @@
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
 #include "access/reloptions.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/pg_collation.h"
 #include "catalog/pg_type.h"
@@ -659,12 +660,18 @@ ginUpdateStats(Relation index, const GinStatsData *stats, bool is_build)
 	Buffer		metabuffer;
 	Page		metapage;
 	GinMetaPageData *metadata;
+	bool		needwal;
 
 	metabuffer = ReadBuffer(index, GIN_METAPAGE_BLKNO);
 	LockBuffer(metabuffer, GIN_EXCLUSIVE);
 	metapage = BufferGetPage(metabuffer);
 	metadata = GinPageGetMeta(metapage);
 
+	needwal = RelationNeedsWAL(index) && !is_build;
+
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	metadata->nTotalPages = stats->nTotalPages;
@@ -684,7 +691,7 @@ ginUpdateStats(Relation index, const GinStatsData *stats, bool is_build)
 
 	MarkBufferDirty(metabuffer);
 
-	if (RelationNeedsWAL(index) && !is_build)
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogUpdateMeta data;
diff --git a/src/backend/access/gin/ginvacuum.c b/src/backend/access/gin/ginvacuum.c
index 0935a6d9e53..d91ca2b391c 100644
--- a/src/backend/access/gin/ginvacuum.c
+++ b/src/backend/access/gin/ginvacuum.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "commands/vacuum.h"
 #include "miscadmin.h"
@@ -136,6 +137,7 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	Page		page,
 				parentPage;
 	BlockNumber rightlink;
+	bool		needwal;
 
 	/*
 	 * This function MUST be called only if someone of parent pages hold
@@ -159,6 +161,10 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	 */
 	PredicateLockPageCombine(gvs->index, deleteBlkno, rightlink);
 
+	needwal = RelationNeedsWAL(gvs->index);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* Unlink the page by changing left sibling's rightlink */
@@ -195,7 +201,7 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	MarkBufferDirty(lBuffer);
 	MarkBufferDirty(dBuffer);
 
-	if (RelationNeedsWAL(gvs->index))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogDeletePage data;
@@ -650,6 +656,9 @@ ginbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 
 		if (resPage)
 		{
+			if (RelationNeedsWAL(gvs.index))
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 			PageRestoreTempPage(resPage, page);
 			MarkBufferDirty(buffer);
diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c
index 25b42e38f22..4a870a062ba 100644
--- a/src/backend/access/gist/gist.c
+++ b/src/backend/access/gist/gist.c
@@ -16,6 +16,7 @@
 
 #include "access/gist_private.h"
 #include "access/gistscan.h"
+#include "access/walprohibit.h"
 #include "catalog/pg_collation.h"
 #include "commands/vacuum.h"
 #include "miscadmin.h"
@@ -135,6 +136,9 @@ gistbuildempty(Relation index)
 	buffer = ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermittedHaveXID();
+
 	/* Initialize and xlog buffer */
 	START_CRIT_SECTION();
 	GISTInitBuffer(buffer, F_LEAF);
@@ -234,6 +238,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 	XLogRecPtr	recptr;
 	int			i;
 	bool		is_split;
+	bool		needwal = RelationNeedsWAL(rel);
 
 	/*
 	 * Refuse to modify a page that's incompletely split. This should not
@@ -465,9 +470,12 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 		 * insertion for that. NB: The number of pages and data segments
 		 * specified here must match the calculations in gistXLogSplit()!
 		 */
-		if (!is_build && RelationNeedsWAL(rel))
+		if (!is_build && needwal)
 			XLogEnsureRecordSpace(npage, 1 + npage * 2);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		/*
@@ -500,7 +508,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 			recptr = GistBuildLSN;
 		else
 		{
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 				recptr = gistXLogSplit(is_leaf,
 									   dist, oldrlink, oldnsn, leftchildbuf,
 									   markfollowright);
@@ -526,6 +534,9 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 	}
 	else
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		/*
 		 * Enough space.  We always get here if ntup==0.
 		 */
@@ -567,7 +578,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 			recptr = GistBuildLSN;
 		else
 		{
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 			{
 				OffsetNumber ndeloffs = 0,
 							deloffs[1];
@@ -1641,6 +1652,7 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 	OffsetNumber offnum,
 				maxoff;
 	TransactionId latestRemovedXid = InvalidTransactionId;
+	bool		needwal = RelationNeedsWAL(rel);
 
 	Assert(GistPageIsLeaf(page));
 
@@ -1659,13 +1671,16 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 			deletable[ndeletable++] = offnum;
 	}
 
-	if (XLogStandbyInfoActive() && RelationNeedsWAL(rel))
+	if (XLogStandbyInfoActive() && needwal)
 		latestRemovedXid =
 			index_compute_xid_horizon_for_tuples(rel, heapRel, buffer,
 												 deletable, ndeletable);
 
 	if (ndeletable > 0)
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		PageIndexMultiDelete(page, deletable, ndeletable);
@@ -1682,7 +1697,7 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 		MarkBufferDirty(buffer);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gist/gistvacuum.c b/src/backend/access/gist/gistvacuum.c
index a9c616c7724..bbb3ebb19ad 100644
--- a/src/backend/access/gist/gistvacuum.c
+++ b/src/backend/access/gist/gistvacuum.c
@@ -17,6 +17,7 @@
 #include "access/genam.h"
 #include "access/gist_private.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "commands/vacuum.h"
 #include "lib/integerset.h"
 #include "miscadmin.h"
@@ -260,6 +261,7 @@ gistvacuumpage(GistVacState *vstate, BlockNumber blkno, BlockNumber orig_blkno)
 	Buffer		buffer;
 	Page		page;
 	BlockNumber recurse_to;
+	bool		needwal = RelationNeedsWAL(rel);
 
 restart:
 	recurse_to = InvalidBlockNumber;
@@ -341,6 +343,9 @@ restart:
 		 */
 		if (ntodelete > 0)
 		{
+			if (needwal)
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 
 			MarkBufferDirty(buffer);
@@ -348,7 +353,7 @@ restart:
 			PageIndexMultiDelete(page, todelete, ntodelete);
 			GistMarkTuplesDeleted(page);
 
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 			{
 				XLogRecPtr	recptr;
 
@@ -580,6 +585,7 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	IndexTuple	idxtuple;
 	XLogRecPtr	recptr;
 	FullTransactionId txid;
+	bool		needwal = RelationNeedsWAL(info->index);
 
 	/*
 	 * Check that the leaf is still empty and deletable.
@@ -634,6 +640,9 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	 */
 	txid = ReadNextFullTransactionId();
 
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* mark the page as deleted */
@@ -645,7 +654,7 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	MarkBufferDirty(parentBuffer);
 	PageIndexTupleDelete(parentPage, downlink);
 
-	if (RelationNeedsWAL(info->index))
+	if (needwal)
 		recptr = gistXLogPageDelete(leafBuffer, txid, parentBuffer, downlink);
 	else
 		recptr = gistGetFakeLSN(info->index);
diff --git a/src/backend/access/hash/hash.c b/src/backend/access/hash/hash.c
index 7c9ccf446c8..f4903a43bb5 100644
--- a/src/backend/access/hash/hash.c
+++ b/src/backend/access/hash/hash.c
@@ -22,6 +22,7 @@
 #include "access/hash_xlog.h"
 #include "access/relscan.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "catalog/index.h"
 #include "commands/progress.h"
 #include "commands/vacuum.h"
@@ -467,6 +468,7 @@ hashbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	Buffer		metabuf = InvalidBuffer;
 	HashMetaPage metap;
 	HashMetaPage cachedmetap;
+	bool		needwal;
 
 	tuples_removed = 0;
 	num_index_tuples = 0;
@@ -573,6 +575,10 @@ loop_top:
 		goto loop_top;
 	}
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Okay, we're really done.  Update tuple count in metapage. */
 	START_CRIT_SECTION();
 
@@ -603,7 +609,7 @@ loop_top:
 	MarkBufferDirty(metabuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_hash_update_meta_page xlrec;
 		XLogRecPtr	recptr;
@@ -690,6 +696,7 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 	Buffer		buf;
 	Bucket		new_bucket PG_USED_FOR_ASSERTS_ONLY = InvalidBucket;
 	bool		bucket_dirty = false;
+	bool		needwal = RelationNeedsWAL(rel);
 
 	blkno = bucket_blkno;
 	buf = bucket_buf;
@@ -788,6 +795,9 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		 */
 		if (ndeletable > 0)
 		{
+			if (needwal)
+				CheckWALPermitted();
+
 			/* No ereport(ERROR) until changes are logged */
 			START_CRIT_SECTION();
 
@@ -809,7 +819,7 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 			MarkBufferDirty(buf);
 
 			/* XLOG stuff */
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 			{
 				xl_hash_delete xlrec;
 				XLogRecPtr	recptr;
@@ -883,6 +893,9 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		page = BufferGetPage(bucket_buf);
 		bucket_opaque = (HashPageOpaque) PageGetSpecialPointer(page);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -890,7 +903,7 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		MarkBufferDirty(bucket_buf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c
index 2ebe671967b..2eab69efa91 100644
--- a/src/backend/access/hash/hashinsert.c
+++ b/src/backend/access/hash/hashinsert.c
@@ -17,6 +17,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "storage/buf_internals.h"
 #include "storage/lwlock.h"
@@ -193,6 +194,8 @@ restart_insert:
 	 */
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	AssertWALPermittedHaveXID();
+
 	/* Do the update.  No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -360,6 +363,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 	if (ndeletable > 0)
 	{
 		TransactionId latestRemovedXid;
+		bool		needwal = RelationNeedsWAL(rel);
 
 		latestRemovedXid =
 			index_compute_xid_horizon_for_tuples(rel, hrel, buf,
@@ -370,6 +374,9 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 		 */
 		LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -393,7 +400,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 		MarkBufferDirty(metabuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			xl_hash_vacuum_one_page xlrec;
 			XLogRecPtr	recptr;
diff --git a/src/backend/access/hash/hashovfl.c b/src/backend/access/hash/hashovfl.c
index 00f0a940116..e7c5dd3e3ce 100644
--- a/src/backend/access/hash/hashovfl.c
+++ b/src/backend/access/hash/hashovfl.c
@@ -19,6 +19,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "utils/rel.h"
 
@@ -312,6 +313,8 @@ _hash_addovflpage(Relation rel, Buffer metabuf, Buffer buf, bool retain_pin)
 
 found:
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Do the update.  No ereport(ERROR) until changes are logged. We want to
 	 * log the changes for bitmap page and overflow page together to avoid
@@ -510,6 +513,7 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	Buffer		prevbuf = InvalidBuffer;
 	Buffer		nextbuf = InvalidBuffer;
 	bool		update_metap = false;
+	bool		needwal;
 
 	/* Get information from the doomed page */
 	_hash_checkpage(rel, ovflbuf, LH_OVERFLOW_PAGE);
@@ -573,10 +577,15 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	/* Get write-lock on metapage to update firstfree */
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	needwal = RelationNeedsWAL(rel);
+
 	/* This operation needs to log multiple tuples, prepare WAL for that */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 		XLogEnsureRecordSpace(HASH_XLOG_FREE_OVFL_BUFS, 4 + nitups);
 
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -641,7 +650,7 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	}
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_hash_squeeze_page xlrec;
 		XLogRecPtr	recptr;
@@ -922,14 +931,19 @@ readpage:
 
 				if (nitups > 0)
 				{
+					bool		needwal = RelationNeedsWAL(rel);
+
 					Assert(nitups == ndeletable);
 
 					/*
 					 * This operation needs to log multiple tuples, prepare
 					 * WAL for that.
 					 */
-					if (RelationNeedsWAL(rel))
+					if (needwal)
+					{
+						CheckWALPermitted();
 						XLogEnsureRecordSpace(0, 3 + nitups);
+					}
 
 					START_CRIT_SECTION();
 
@@ -947,7 +961,7 @@ readpage:
 					MarkBufferDirty(rbuf);
 
 					/* XLOG stuff */
-					if (RelationNeedsWAL(rel))
+					if (needwal)
 					{
 						XLogRecPtr	recptr;
 						xl_hash_move_page_contents xlrec;
diff --git a/src/backend/access/hash/hashpage.c b/src/backend/access/hash/hashpage.c
index a664ecf494a..55a867dd375 100644
--- a/src/backend/access/hash/hashpage.c
+++ b/src/backend/access/hash/hashpage.c
@@ -30,6 +30,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "port/pg_bitutils.h"
 #include "storage/lmgr.h"
@@ -816,6 +817,8 @@ restart_expand:
 		goto fail;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Since we are scribbling on the pages in the shared buffers, establish a
 	 * critical section.  Any failure in this next code leaves us with a big
@@ -1172,6 +1175,8 @@ _hash_splitbucket(Relation rel,
 
 				if (PageGetFreeSpaceForMultipleTuples(npage, nitups + 1) < (all_tups_size + itemsz))
 				{
+					AssertWALPermittedHaveXID();
+
 					/*
 					 * Change the shared buffer state in critical section,
 					 * otherwise any error could make it unrecoverable.
@@ -1223,6 +1228,8 @@ _hash_splitbucket(Relation rel,
 		/* Exit loop if no more overflow pages in old bucket */
 		if (!BlockNumberIsValid(oblkno))
 		{
+			AssertWALPermittedHaveXID();
+
 			/*
 			 * Change the shared buffer state in critical section, otherwise
 			 * any error could make it unrecoverable.
@@ -1269,6 +1276,8 @@ _hash_splitbucket(Relation rel,
 	npage = BufferGetPage(bucket_nbuf);
 	nopaque = (HashPageOpaque) PageGetSpecialPointer(npage);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	oopaque->hasho_flag &= ~LH_BUCKET_BEING_SPLIT;
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 1b2f70499e5..bde45e2f6e6 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -47,6 +47,7 @@
 #include "access/transam.h"
 #include "access/valid.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1898,6 +1899,8 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
 	 */
 	CheckForSerializableConflictIn(relation, NULL, InvalidBlockNumber);
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -2172,6 +2175,8 @@ heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples,
 										   &vmbuffer, NULL);
 		page = BufferGetPage(buffer);
 
+		AssertWALPermittedHaveXID();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -2690,6 +2695,8 @@ l1:
 							  xid, LockTupleExclusive, true,
 							  &new_xmax, &new_infomask, &new_infomask2);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -3442,6 +3449,8 @@ l2:
 
 		Assert(HEAP_XMAX_IS_LOCKED_ONLY(infomask_lock_old_tuple));
 
+		AssertWALPermittedHaveXID();
+
 		START_CRIT_SECTION();
 
 		/* Clear obsolete visibility flags ... */
@@ -3615,6 +3624,8 @@ l2:
 										   bms_overlap(modified_attrs, id_attrs),
 										   &old_key_copied);
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -4548,6 +4559,8 @@ failed:
 							  GetCurrentTransactionId(), mode, false,
 							  &xid, &new_infomask, &new_infomask2);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -5339,6 +5352,8 @@ l4:
 								VISIBILITYMAP_ALL_FROZEN))
 			cleared_all_frozen = true;
 
+		AssertWALPermittedHaveXID();
+
 		START_CRIT_SECTION();
 
 		/* ... and set them */
@@ -5497,6 +5512,8 @@ heap_finish_speculative(Relation relation, ItemPointer tid)
 	StaticAssertStmt(MaxOffsetNumber < SpecTokenOffsetNumber,
 					 "invalid speculative token constant");
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -5605,6 +5622,8 @@ heap_abort_speculative(Relation relation, ItemPointer tid)
 		elog(ERROR, "attempted to kill a non-speculative tuple");
 	Assert(!HeapTupleHeaderIsHeapOnly(tp.t_data));
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * No need to check for serializable conflicts here.  There is never a
 	 * need for a combocid, either.  No need to extract replica identity, or
@@ -5721,6 +5740,7 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	HeapTupleHeader htup;
 	uint32		oldlen;
 	uint32		newlen;
+	bool		needwal;
 
 	/*
 	 * For now, we don't allow parallel updates.  Unlike a regular update,
@@ -5751,6 +5771,10 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	if (oldlen != newlen || htup->t_hoff != tuple->t_data->t_hoff)
 		elog(ERROR, "wrong tuple length");
 
+	needwal = RelationNeedsWAL(relation);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -5761,7 +5785,7 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	MarkBufferDirty(buffer);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(relation))
+	if (needwal)
 	{
 		xl_heap_inplace xlrec;
 		XLogRecPtr	recptr;
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index 9e04bc712c9..9e1ab73ee6e 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -18,6 +18,7 @@
 #include "access/heapam_xlog.h"
 #include "access/htup_details.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "catalog/catalog.h"
 #include "miscadmin.h"
@@ -94,11 +95,11 @@ heap_page_prune_opt(Relation relation, Buffer buffer)
 	Size		minfree;
 
 	/*
-	 * We can't write WAL in recovery mode, so there's no point trying to
+	 * We can't write WAL during read-only mode, so there's no point trying to
 	 * clean the page. The primary will likely issue a cleaning WAL record soon
 	 * anyway, so this is no particular loss.
 	 */
-	if (RecoveryInProgress())
+	if (!XLogInsertAllowed())
 		return;
 
 	/*
@@ -232,6 +233,7 @@ heap_page_prune(Relation relation, Buffer buffer,
 	OffsetNumber offnum,
 				maxoff;
 	PruneState	prstate;
+	bool		needwal;
 
 	/*
 	 * Our strategy is to scan the page and make lists of items to change,
@@ -286,6 +288,10 @@ heap_page_prune(Relation relation, Buffer buffer,
 	if (off_loc)
 		*off_loc = InvalidOffsetNumber;
 
+	needwal = RelationNeedsWAL(relation);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Any error while applying the changes is critical */
 	START_CRIT_SECTION();
 
@@ -319,7 +325,7 @@ heap_page_prune(Relation relation, Buffer buffer,
 		/*
 		 * Emit a WAL XLOG_HEAP2_CLEAN record showing what we did
 		 */
-		if (RelationNeedsWAL(relation))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 25f2d5df1b8..8ba1af0be63 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -59,6 +59,7 @@
 #include "access/parallel.h"
 #include "access/transam.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "catalog/storage.h"
@@ -758,6 +759,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 	BlockNumber nblocks,
 				blkno;
 	HeapTupleData tuple;
+	bool		needwal = RelationNeedsWAL(onerel);
 	TransactionId relfrozenxid = onerel->rd_rel->relfrozenxid;
 	TransactionId relminmxid = onerel->rd_rel->relminmxid;
 	BlockNumber empty_pages,
@@ -1200,6 +1202,9 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 			 */
 			if (!PageIsAllVisible(page))
 			{
+				if (needwal)
+					CheckWALPermitted();
+
 				START_CRIT_SECTION();
 
 				/* mark buffer dirty before writing a WAL record */
@@ -1215,7 +1220,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 				 * page has been previously WAL-logged, and if not, do that
 				 * now.
 				 */
-				if (RelationNeedsWAL(onerel) &&
+				if (needwal &&
 					PageGetLSN(page) == InvalidXLogRecPtr)
 					log_newpage_buffer(buf, true);
 
@@ -1481,6 +1486,9 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 		 */
 		if (nfrozen > 0)
 		{
+			if (needwal)
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 
 			MarkBufferDirty(buf);
@@ -1498,7 +1506,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 			}
 
 			/* Now WAL-log freezing if necessary */
-			if (RelationNeedsWAL(onerel))
+			if (needwal)
 			{
 				XLogRecPtr	recptr;
 
@@ -1931,6 +1939,7 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	TransactionId visibility_cutoff_xid;
 	bool		all_frozen;
 	LVSavedErrInfo saved_err_info;
+	bool		needwal = RelationNeedsWAL(onerel);
 
 	pgstat_progress_update_param(PROGRESS_VACUUM_HEAP_BLKS_VACUUMED, blkno);
 
@@ -1938,6 +1947,9 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	update_vacuum_error_info(vacrelstats, &saved_err_info, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
 							 blkno, InvalidOffsetNumber);
 
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	for (; tupindex < dead_tuples->num_tuples; tupindex++)
@@ -1963,7 +1975,7 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	MarkBufferDirty(buffer);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(onerel))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index b1072183bcd..44244363968 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -88,6 +88,7 @@
 
 #include "access/heapam_xlog.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "miscadmin.h"
 #include "port/pg_bitutils.h"
@@ -249,6 +250,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 	uint8		mapOffset = HEAPBLK_TO_OFFSET(heapBlk);
 	Page		page;
 	uint8	   *map;
+	bool		needwal = RelationNeedsWAL(rel);
 
 #ifdef TRACE_VISIBILITYMAP
 	elog(DEBUG1, "vm_set %s %d", RelationGetRelationName(rel), heapBlk);
@@ -270,6 +272,16 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 	map = (uint8 *) PageGetContents(page);
 	LockBuffer(vmBuf, BUFFER_LOCK_EXCLUSIVE);
 
+	/*
+	 * Can reach here from VACUUM or from startup process, so need not have an
+	 * XID.
+	 *
+	 * Recovery in the startup process never have wal prohibit state, skip
+	 * permission check if reach here in the startup process.
+	 */
+	if (needwal)
+		InRecovery ? AssertWALPermitted() : CheckWALPermitted();
+
 	if (flags != (map[mapByte] >> mapOffset & VISIBILITYMAP_VALID_BITS))
 	{
 		START_CRIT_SECTION();
@@ -277,7 +289,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 		map[mapByte] |= (flags << mapOffset);
 		MarkBufferDirty(vmBuf);
 
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			if (XLogRecPtrIsInvalid(recptr))
 			{
@@ -476,6 +488,7 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		Buffer		mapBuffer;
 		Page		page;
 		char	   *map;
+		bool		needwal;
 
 		newnblocks = truncBlock + 1;
 
@@ -489,8 +502,13 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		page = BufferGetPage(mapBuffer);
 		map = PageGetContents(page);
 
+		needwal = (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded());
+
 		LockBuffer(mapBuffer, BUFFER_LOCK_EXCLUSIVE);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -518,7 +536,7 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		 * during recovery.
 		 */
 		MarkBufferDirty(mapBuffer);
-		if (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded())
+		if (needwal)
 			log_newpage_buffer(mapBuffer, false);
 
 		END_CRIT_SECTION();
diff --git a/src/backend/access/nbtree/nbtdedup.c b/src/backend/access/nbtree/nbtdedup.c
index 9e535124c46..8d3c2a72012 100644
--- a/src/backend/access/nbtree/nbtdedup.c
+++ b/src/backend/access/nbtree/nbtdedup.c
@@ -16,6 +16,7 @@
 
 #include "access/nbtree.h"
 #include "access/nbtxlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "utils/rel.h"
 
@@ -227,6 +228,8 @@ _bt_dedup_pass(Relation rel, Buffer buf, Relation heapRel, IndexTuple newitem,
 		nopaque->btpo_flags &= ~BTP_HAS_GARBAGE;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	PageRestoreTempPage(newpage, page);
diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c
index dde43b1415a..652966e679b 100644
--- a/src/backend/access/nbtree/nbtinsert.c
+++ b/src/backend/access/nbtree/nbtinsert.c
@@ -19,6 +19,7 @@
 #include "access/nbtxlog.h"
 #include "access/tableam.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/lmgr.h"
@@ -1230,6 +1231,8 @@ _bt_insertonpg(Relation rel,
 			}
 		}
 
+		AssertWALPermittedHaveXID();
+
 		/* Do the update.  No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -1882,13 +1885,16 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf,
 			ropaque->btpo_flags |= BTP_SPLIT_END;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Right sibling is locked, new siblings are prepared, but original page
 	 * is not updated yet.
 	 *
 	 * NO EREPORT(ERROR) till right sibling is updated.  We can get away with
 	 * not starting the critical section till here because we haven't been
-	 * scribbling on the original page yet; see comments above.
+	 * scribbling on the original page yet; see the comments above for grabbing
+	 * the right sibling.
 	 */
 	START_CRIT_SECTION();
 
@@ -2450,6 +2456,8 @@ _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf)
 	right_item = CopyIndexTuple(item);
 	BTreeTupleSetDownLink(right_item, rbkno);
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till newroot op is logged */
 	START_CRIT_SECTION();
 
diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c
index 848123d9218..7940a626efe 100644
--- a/src/backend/access/nbtree/nbtpage.c
+++ b/src/backend/access/nbtree/nbtpage.c
@@ -26,6 +26,7 @@
 #include "access/nbtxlog.h"
 #include "access/tableam.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
@@ -179,6 +180,7 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	BTMetaPageData *metad;
 	bool		needsRewrite = false;
 	XLogRecPtr	recptr;
+	bool		needwal;
 
 	/* read the metapage and check if it needs rewrite */
 	metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ);
@@ -202,6 +204,10 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	_bt_unlockbuf(rel, metabuf);
 	_bt_lockbuf(rel, metabuf, BT_WRITE);
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* upgrade meta-page if needed */
@@ -214,7 +220,7 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	MarkBufferDirty(metabuf);
 
 	/* write wal record if needed */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_btree_metadata md;
 
@@ -332,6 +338,7 @@ _bt_getroot(Relation rel, int access)
 	if (metad->btm_root == P_NONE)
 	{
 		Page		metapg;
+		bool		needwal;
 
 		/* If access = BT_READ, caller doesn't want us to create root yet */
 		if (access == BT_READ)
@@ -377,6 +384,10 @@ _bt_getroot(Relation rel, int access)
 		/* Get raw page pointer for metapage */
 		metapg = BufferGetPage(metabuf);
 
+		needwal = RelationNeedsWAL(rel);
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO ELOG(ERROR) till meta is updated */
 		START_CRIT_SECTION();
 
@@ -395,7 +406,7 @@ _bt_getroot(Relation rel, int access)
 		MarkBufferDirty(metabuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			xl_btree_newroot xlrec;
 			XLogRecPtr	recptr;
@@ -1131,6 +1142,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 	char	   *updatedbuf = NULL;
 	Size		updatedbuflen = 0;
 	OffsetNumber updatedoffsets[MaxIndexTuplesPerPage];
+	bool		needwal = RelationNeedsWAL(rel);
 
 	/* Shouldn't be called unless there's something to do */
 	Assert(ndeletable > 0 || nupdatable > 0);
@@ -1145,7 +1157,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 	}
 
 	/* XLOG stuff -- allocate and fill buffer before critical section */
-	if (nupdatable > 0 && RelationNeedsWAL(rel))
+	if (nupdatable > 0 && needwal)
 	{
 		Size		offset = 0;
 
@@ -1175,6 +1187,9 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 		}
 	}
 
+	if (needwal)
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -1227,7 +1242,7 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 	MarkBufferDirty(buf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		xl_btree_vacuum xlrec_vacuum;
@@ -1294,6 +1309,8 @@ _bt_delitems_delete(Relation rel, Buffer buf,
 		latestRemovedXid =
 			_bt_xid_horizon(rel, heapRel, page, deletable, ndeletable);
 
+	AssertWALPermittedHaveXID();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -1831,6 +1848,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	OffsetNumber nextoffset;
 	IndexTuple	itup;
 	IndexTupleData trunctuple;
+	bool		needwal;
 
 	page = BufferGetPage(leafbuf);
 	opaque = (BTPageOpaque) PageGetSpecialPointer(page);
@@ -1919,6 +1937,10 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	 */
 	PredicateLockPageCombine(rel, leafblkno, leafrightsib);
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -1970,7 +1992,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	MarkBufferDirty(leafbuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_btree_mark_page_halfdead xlrec;
 		XLogRecPtr	recptr;
@@ -2063,6 +2085,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 	int			targetlevel;
 	IndexTuple	leafhikey;
 	BlockNumber nextchild;
+	bool		needwal;
 
 	page = BufferGetPage(leafbuf);
 	opaque = (BTPageOpaque) PageGetSpecialPointer(page);
@@ -2276,6 +2299,10 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 	 * Here we begin doing the deletion.
 	 */
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -2355,7 +2382,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 		MarkBufferDirty(leafbuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_btree_unlink_page xlrec;
 		xl_btree_metadata xlmeta;
diff --git a/src/backend/access/spgist/spgdoinsert.c b/src/backend/access/spgist/spgdoinsert.c
index 934d65b89f2..3c5a15c5d32 100644
--- a/src/backend/access/spgist/spgdoinsert.c
+++ b/src/backend/access/spgist/spgdoinsert.c
@@ -18,6 +18,7 @@
 #include "access/genam.h"
 #include "access/spgist_private.h"
 #include "access/spgxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -214,6 +215,8 @@ addLeafTuple(Relation index, SpGistState *state, SpGistLeafTuple leafTuple,
 	xlrec.offnumParent = InvalidOffsetNumber;
 	xlrec.nodeI = 0;
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	if (current->offnum == InvalidOffsetNumber ||
@@ -458,6 +461,8 @@ moveLeafs(Relation index, SpGistState *state,
 
 	leafdata = leafptr = palloc(size);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/* copy all the old tuples to new page, unless they're dead */
@@ -1109,6 +1114,8 @@ doPickSplit(Relation index, SpGistState *state,
 
 	leafdata = leafptr = (char *) palloc(totalLeafSizes);
 
+	AssertWALPermittedHaveXID();
+
 	/* Here we begin making the changes to the target pages */
 	START_CRIT_SECTION();
 
@@ -1517,6 +1524,8 @@ spgAddNodeAction(Relation index, SpGistState *state,
 	if (PageGetExactFreeSpace(current->page) >=
 		newInnerTuple->size - innerTuple->size)
 	{
+		AssertWALPermittedHaveXID();
+
 		/*
 		 * We can replace the inner tuple by new version in-place
 		 */
@@ -1603,6 +1612,8 @@ spgAddNodeAction(Relation index, SpGistState *state,
 		else
 			xlrec.parentBlk = 2;
 
+		AssertWALPermittedHaveXID();
+
 		START_CRIT_SECTION();
 
 		/* insert new ... */
@@ -1788,6 +1799,8 @@ spgSplitNodeAction(Relation index, SpGistState *state,
 									&xlrec.newPage);
 	}
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c
index e1c58933f97..3308832b85b 100644
--- a/src/backend/access/spgist/spgvacuum.c
+++ b/src/backend/access/spgist/spgvacuum.c
@@ -19,6 +19,7 @@
 #include "access/spgist_private.h"
 #include "access/spgxlog.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/storage_xlog.h"
 #include "commands/vacuum.h"
@@ -139,6 +140,7 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 	int			nDeletable;
 	OffsetNumber i,
 				max = PageGetMaxOffsetNumber(page);
+	bool		needwal;
 
 	memset(predecessor, 0, sizeof(predecessor));
 	memset(deletable, 0, sizeof(deletable));
@@ -323,6 +325,10 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 	if (nDeletable != xlrec.nDead + xlrec.nPlaceholder + xlrec.nMove)
 		elog(ERROR, "inconsistent counts of deletable tuples");
 
+	needwal = RelationNeedsWAL(index);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Do the updates */
 	START_CRIT_SECTION();
 
@@ -371,7 +377,7 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 
@@ -411,6 +417,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 	OffsetNumber toDelete[MaxIndexTuplesPerPage];
 	OffsetNumber i,
 				max = PageGetMaxOffsetNumber(page);
+	bool		needwal;
 
 	xlrec.nDelete = 0;
 
@@ -447,6 +454,10 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 	if (xlrec.nDelete == 0)
 		return;					/* nothing more to do */
 
+	needwal = RelationNeedsWAL(index);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Do the update */
 	START_CRIT_SECTION();
 
@@ -455,7 +466,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 
@@ -502,6 +513,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer)
 	OffsetNumber itemnos[MaxIndexTuplesPerPage];
 	spgxlogVacuumRedirect xlrec;
 	GlobalVisState *vistest;
+	bool		needwal;
 
 	xlrec.nToPlaceholder = 0;
 	xlrec.newestRedirectXid = InvalidTransactionId;
@@ -509,6 +521,10 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer)
 	/* XXX: providing heap relation would allow more pruning */
 	vistest = GlobalVisTestFor(NULL);
 
+	needwal = RelationNeedsWAL(index);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -584,7 +600,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer)
 	if (hasUpdate)
 		MarkBufferDirty(buffer);
 
-	if (hasUpdate && RelationNeedsWAL(index))
+	if (hasUpdate && needwal)
 	{
 		XLogRecPtr	recptr;
 
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index eb8de7cf329..b2d41925fb4 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -73,6 +73,7 @@
 #include "access/transam.h"
 #include "access/twophase.h"
 #include "access/twophase_rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1162,6 +1163,8 @@ GetNewMultiXactId(int nmembers, MultiXactOffset *offset)
 
 	ExtendMultiXactMember(nextOffset, nmembers);
 
+	CheckWALPermitted();
+
 	/*
 	 * Critical section from here until caller has written the data into the
 	 * just-reserved SLRU space; we don't want to error out with a partly
@@ -2946,7 +2949,7 @@ TruncateMultiXact(MultiXactId newOldestMulti, Oid newOldestMultiDB)
 	mxtruncinfo trunc;
 	MultiXactId earliest;
 
-	Assert(!RecoveryInProgress());
+	Assert(XLogInsertAllowed());
 	Assert(MultiXactState->finishedStartup);
 
 	/*
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 873bf9bad98..5aef62724c6 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -82,6 +82,7 @@
 #include "access/transam.h"
 #include "access/twophase.h"
 #include "access/twophase_rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1106,6 +1107,8 @@ EndPrepare(GlobalTransaction gxact)
 	 */
 	XLogEnsureRecordSpace(0, records.num_chunks);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	MyProc->delayChkpt = true;
@@ -2196,6 +2199,9 @@ RecordTransactionCommitPrepared(TransactionId xid,
 	replorigin = (replorigin_session_origin != InvalidRepOriginId &&
 				  replorigin_session_origin != DoNotReplicateId);
 
+	/* COMMIT PREPARED need not have an XID */
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* See notes in RecordTransactionCommit */
@@ -2286,6 +2292,9 @@ RecordTransactionAbortPrepared(TransactionId xid,
 		elog(PANIC, "cannot abort transaction %u, it was already committed",
 			 xid);
 
+	/* ROLLBACK PREPARED need not have an XID */
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
diff --git a/src/backend/access/transam/varsup.c b/src/backend/access/transam/varsup.c
index a4944faa32e..0c7a2362f25 100644
--- a/src/backend/access/transam/varsup.c
+++ b/src/backend/access/transam/varsup.c
@@ -17,6 +17,7 @@
 #include "access/commit_ts.h"
 #include "access/subtrans.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "commands/dbcommands.h"
@@ -75,6 +76,9 @@ GetNewTransactionId(bool isSubXact)
 	if (RecoveryInProgress())
 		elog(ERROR, "cannot assign TransactionIds during recovery");
 
+	/* do not assign transaction id in read-only mode */
+	CheckWALPermitted();
+
 	LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
 
 	full_xid = ShmemVariableCache->nextXid;
diff --git a/src/backend/access/transam/walprohibit.c b/src/backend/access/transam/walprohibit.c
index 00c8894d806..f2b5bf2871a 100644
--- a/src/backend/access/transam/walprohibit.c
+++ b/src/backend/access/transam/walprohibit.c
@@ -24,6 +24,16 @@
 #include "utils/fmgroids.h"
 #include "utils/fmgrprotos.h"
 
+/*
+ * Assert flag to enforce WAL insert permission check rule before starting a
+ * critical section for the WAL writes.  For this, either of
+ * CheckWALPermitted(), AssertWALPermittedHaveXID(), or AssertWALPermitted()
+ * must be called before starting the critical section.
+ */
+#ifdef USE_ASSERT_CHECKING
+WALPermitCheckState walpermit_checked_state = WALPERMIT_UNCHECKED;
+#endif
+
 /*
  * Shared-memory WAL prohibit state
  */
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 485e441225b..40835a89785 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -26,6 +26,7 @@
 #include "access/subtrans.h"
 #include "access/transam.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1317,6 +1318,8 @@ RecordTransactionCommit(void)
 		/* Tell bufmgr and smgr to prepare for commit */
 		BufmgrCommit();
 
+		AssertWALPermittedHaveXID();
+
 		/*
 		 * Mark ourselves as within our "commit critical section".  This
 		 * forces any concurrent checkpoint to wait until we've updated
@@ -1677,6 +1680,9 @@ RecordTransactionAbort(bool isSubXact)
 		elog(PANIC, "cannot abort transaction %u, it was already committed",
 			 xid);
 
+	/* We'll be reaching here with valid XID only. */
+	AssertWALPermittedHaveXID();
+
 	/* Fetch the data we need for the abort record */
 	nrels = smgrGetPendingDeletes(false, &rels);
 	nchildren = xactGetCommittedChildren(&children);
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 659376bb327..4dac37ab799 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -1026,7 +1026,7 @@ XLogInsertRecord(XLogRecData *rdata,
 
 	/* cross-check on whether we should be here or not */
 	if (!XLogInsertAllowed())
-		elog(ERROR, "cannot make new WAL entries during recovery");
+		elog(ERROR, "cannot make new WAL entries now");
 
 	/*----------
 	 *
@@ -2862,9 +2862,11 @@ XLogFlush(XLogRecPtr record)
 	 * trying to flush the WAL, we should update minRecoveryPoint instead. We
 	 * test XLogInsertAllowed(), not InRecovery, because we need checkpointer
 	 * to act this way too, and because when it tries to write the
-	 * end-of-recovery checkpoint, it should indeed flush.
+	 * end-of-recovery checkpoint, it should indeed flush. Also, WAL prohibit
+	 * state should not restrict WAL flushing. Otherwise, the dirty buffer
+	 * cannot be evicted until WAL has been flushed up to the buffer's LSN.
 	 */
-	if (!XLogInsertAllowed())
+	if (!XLogInsertAllowed() && !IsWALProhibited())
 	{
 		UpdateMinRecoveryPoint(record, false);
 		return;
@@ -8803,6 +8805,8 @@ CreateCheckPoint(int flags)
 	/* sanity check */
 	if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
 		elog(ERROR, "can't create a checkpoint during recovery");
+	if (!XLogInsertAllowed() && !RecoveryInProgress())
+		elog(ERROR, "can't create a checkpoint while system is read only ");
 
 	/*
 	 * Initialize InitXLogInsert working areas before entering the critical
@@ -8832,6 +8836,8 @@ CreateCheckPoint(int flags)
 	MemSet(&CheckpointStats, 0, sizeof(CheckpointStats));
 	CheckpointStats.ckpt_start_t = GetCurrentTimestamp();
 
+	AssertWALPermitted();
+
 	/*
 	 * Use a critical section to force system panic if we have trouble.
 	 */
@@ -9060,6 +9066,8 @@ CreateCheckPoint(int flags)
 	if (!shutdown && XLogStandbyInfoActive())
 		LogStandbySnapshot();
 
+	AssertWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -9217,6 +9225,8 @@ CreateEndOfRecoveryRecord(void)
 
 	LocalSetXLogInsertAllowed();
 
+	AssertWALPermitted();
+
 	START_CRIT_SECTION();
 
 	XLogBeginInsert();
@@ -9875,7 +9885,7 @@ void
 UpdateFullPageWrites(void)
 {
 	XLogCtlInsert *Insert = &XLogCtl->Insert;
-	bool		recoveryInProgress;
+	bool		WALInsertAllowed;
 
 	/*
 	 * Do nothing if full_page_writes has not been changed.
@@ -9889,10 +9899,10 @@ UpdateFullPageWrites(void)
 
 	/*
 	 * Perform this outside critical section so that the WAL insert
-	 * initialization done by RecoveryInProgress() doesn't trigger an
+	 * initialization done by XLogInsertAllowed() doesn't trigger an
 	 * assertion failure.
 	 */
-	recoveryInProgress = RecoveryInProgress();
+	WALInsertAllowed = XLogInsertAllowed();
 
 	START_CRIT_SECTION();
 
@@ -9914,8 +9924,11 @@ UpdateFullPageWrites(void)
 	 * Write an XLOG_FPW_CHANGE record. This allows us to keep track of
 	 * full_page_writes during archive recovery, if required.
 	 */
-	if (XLogStandbyInfoActive() && !recoveryInProgress)
+	if (XLogStandbyInfoActive() && WALInsertAllowed)
 	{
+		/* Assured that WAL permission has been checked */
+		AssertWALPermitted();
+
 		XLogBeginInsert();
 		XLogRegisterData((char *) (&fullPageWrites), sizeof(bool));
 
diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c
index 1f0e4e01e69..710806143d4 100644
--- a/src/backend/access/transam/xloginsert.c
+++ b/src/backend/access/transam/xloginsert.c
@@ -19,6 +19,7 @@
 
 #include "postgres.h"
 
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xlog_internal.h"
@@ -126,9 +127,14 @@ XLogBeginInsert(void)
 	Assert(mainrdata_last == (XLogRecData *) &mainrdata_head);
 	Assert(mainrdata_len == 0);
 
+	/*
+	 * WAL permission must have checked before entering the critical section.
+	 * Otherwise, WAL prohibited error will force system panic.
+	 */
+	Assert(walpermit_checked_state != WALPERMIT_UNCHECKED || !CritSectionCount);
+
 	/* cross-check on whether we should be here or not */
-	if (!XLogInsertAllowed())
-		elog(ERROR, "cannot make new WAL entries during recovery");
+	CheckWALPermitted();
 
 	if (begininsert_called)
 		elog(ERROR, "XLogBeginInsert was already called");
@@ -210,6 +216,9 @@ XLogResetInsertion(void)
 	mainrdata_last = (XLogRecData *) &mainrdata_head;
 	curinsert_flags = 0;
 	begininsert_called = false;
+
+	/* Reset walpermit_checked flag */
+	RESET_WALPERMIT_CHECKED_STATE();
 }
 
 /*
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 632b34af610..b01ad5a966a 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -20,6 +20,7 @@
 #include "access/relation.h"
 #include "access/table.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -378,8 +379,13 @@ fill_seq_with_data(Relation rel, HeapTuple tuple)
 
 	/* check the comment above nextval_internal()'s equivalent call. */
 	if (RelationNeedsWAL(rel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermittedHaveXID();
+	}
+
 	START_CRIT_SECTION();
 
 	MarkBufferDirty(buf);
@@ -766,8 +772,13 @@ nextval_internal(Oid relid, bool check_permissions)
 	 * (Have to do that here, so we're outside the critical section)
 	 */
 	if (logit && RelationNeedsWAL(seqrel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermittedHaveXID();
+	}
+
 	/* ready to change the on-disk (or really, in-buffer) tuple */
 	START_CRIT_SECTION();
 
@@ -977,8 +988,13 @@ do_setval(Oid relid, int64 next, bool iscalled)
 
 	/* check the comment above nextval_internal()'s equivalent call. */
 	if (RelationNeedsWAL(seqrel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermittedHaveXID();
+	}
+
 	/* ready to change the on-disk (or really, in-buffer) tuple */
 	START_CRIT_SECTION();
 
diff --git a/src/backend/commands/variable.c b/src/backend/commands/variable.c
index 484f7ea2c0e..acc34d2ad7c 100644
--- a/src/backend/commands/variable.c
+++ b/src/backend/commands/variable.c
@@ -501,11 +501,14 @@ check_transaction_read_only(bool *newval, void **extra, GucSource source)
 			GUC_check_errmsg("transaction read-write mode must be set before any query");
 			return false;
 		}
-		/* Can't go to r/w mode while recovery is still active */
-		if (RecoveryInProgress())
+		/*
+		 * Can't go to r/w mode while recovery is still active or while in WAL
+		 * prohibit state
+		 */
+		if (!XLogInsertAllowed())
 		{
 			GUC_check_errcode(ERRCODE_FEATURE_NOT_SUPPORTED);
-			GUC_check_errmsg("cannot set transaction read-write mode during recovery");
+			GUC_check_errmsg("cannot set transaction read-write mode while system is read only");
 			return false;
 		}
 	}
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index bd66705a9af..da4b8d502ad 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -945,6 +945,10 @@ RequestCheckpoint(int flags)
 	int			old_failed,
 				old_started;
 
+	/* The checkpoint is allowed in recovery but not in WAL prohibit state */
+	if (!RecoveryInProgress())
+		CheckWALPermitted();
+
 	/*
 	 * If in a standalone backend, just do it ourselves.
 	 */
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index ad0d1a9abc0..a18a22350e2 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -3633,13 +3633,15 @@ MarkBufferDirtyHint(Buffer buffer, bool buffer_std)
 		{
 			/*
 			 * If we must not write WAL, due to a relfilenode-specific
-			 * condition or being in recovery, don't dirty the page.  We can
-			 * set the hint, just not dirty the page as a result so the hint
-			 * is lost when we evict the page or shutdown.
+			 * condition or in general, don't dirty the page.  We can
+			 * set the hint, but must not dirty the page as a result, lest
+			 * we trigger WAL generation. Unless the page is dirtied again
+			 * later, the hint will be lost when the page is evicted, or at
+			 * shutdown.
 			 *
 			 * See src/backend/storage/page/README for longer discussion.
 			 */
-			if (RecoveryInProgress() ||
+			if (!XLogInsertAllowed() ||
 				RelFileNodeSkippingWAL(bufHdr->tag.rnode))
 				return;
 
diff --git a/src/backend/storage/freespace/freespace.c b/src/backend/storage/freespace/freespace.c
index 6a96126b0c2..b05b0fe5f41 100644
--- a/src/backend/storage/freespace/freespace.c
+++ b/src/backend/storage/freespace/freespace.c
@@ -24,6 +24,7 @@
 #include "postgres.h"
 
 #include "access/htup_details.h"
+#include "access/walprohibit.h"
 #include "access/xlogutils.h"
 #include "miscadmin.h"
 #include "storage/freespace.h"
@@ -285,12 +286,19 @@ FreeSpaceMapPrepareTruncateRel(Relation rel, BlockNumber nblocks)
 	 */
 	if (first_removed_slot > 0)
 	{
+		bool needwal;
+
 		buf = fsm_readbuf(rel, first_removed_address, false);
 		if (!BufferIsValid(buf))
 			return InvalidBlockNumber;	/* nothing to do; the FSM was already
 										 * smaller */
 		LockBuffer(buf, BUFFER_LOCK_EXCLUSIVE);
 
+		needwal = (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded());
+
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -305,7 +313,7 @@ FreeSpaceMapPrepareTruncateRel(Relation rel, BlockNumber nblocks)
 		 * during recovery.
 		 */
 		MarkBufferDirty(buf);
-		if (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded())
+		if (needwal)
 			log_newpage_buffer(buf, false);
 
 		END_CRIT_SECTION();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index d86566f4554..f949a290745 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -794,15 +794,15 @@ LockAcquireExtended(const LOCKTAG *locktag,
 	if (lockmode <= 0 || lockmode > lockMethodTable->numLockModes)
 		elog(ERROR, "unrecognized lock mode: %d", lockmode);
 
-	if (RecoveryInProgress() && !InRecovery &&
+	if (!XLogInsertAllowed() && !InRecovery &&
 		(locktag->locktag_type == LOCKTAG_OBJECT ||
 		 locktag->locktag_type == LOCKTAG_RELATION) &&
 		lockmode > RowExclusiveLock)
 		ereport(ERROR,
 				(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
-				 errmsg("cannot acquire lock mode %s on database objects while recovery is in progress",
+				 errmsg("cannot acquire lock mode %s on database objects while system is read only",
 						lockMethodTable->lockModeNames[lockmode]),
-				 errhint("Only RowExclusiveLock or less can be acquired on database objects during recovery.")));
+				 errhint("Only RowExclusiveLock or less can be acquired on database objects while system is read only")));
 
 #ifdef LOCK_DEBUG
 	if (LOCK_DEBUG_ENABLED(locktag))
diff --git a/src/backend/utils/cache/relmapper.c b/src/backend/utils/cache/relmapper.c
index 671fbb0ed5c..90d7599a57c 100644
--- a/src/backend/utils/cache/relmapper.c
+++ b/src/backend/utils/cache/relmapper.c
@@ -43,6 +43,7 @@
 #include <sys/stat.h>
 #include <unistd.h>
 
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -841,6 +842,8 @@ write_relmap_file(bool shared, RelMapFile *newmap,
 		xl_relmap_update xlrec;
 		XLogRecPtr	lsn;
 
+		AssertWALPermittedHaveXID();
+
 		/* now errors are fatal ... */
 		START_CRIT_SECTION();
 
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e33523984..f3ff120601e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -94,12 +94,37 @@ extern PGDLLIMPORT volatile uint32 CritSectionCount;
 /* in tcop/postgres.c */
 extern void ProcessInterrupts(void);
 
+#ifdef USE_ASSERT_CHECKING
+typedef enum
+{
+	WALPERMIT_UNCHECKED,
+	WALPERMIT_CHECKED,
+	WALPERMIT_CHECKED_AND_USED
+} WALPermitCheckState;
+
+/* in access/walprohibit.c */
+extern WALPermitCheckState walpermit_checked_state;
+
+/*
+ * Reset walpermit_checked flag when no longer in the critical section.
+ * Otherwise, marked checked and used.
+ */
+#define RESET_WALPERMIT_CHECKED_STATE() \
+do { \
+	walpermit_checked_state = CritSectionCount ? \
+	WALPERMIT_CHECKED_AND_USED : WALPERMIT_UNCHECKED; \
+} while(0)
+#else
+#define RESET_WALPERMIT_CHECKED_STATE() ((void) 0)
+#endif
+
 #ifndef WIN32
 
 #define CHECK_FOR_INTERRUPTS() \
 do { \
 	if (unlikely(InterruptPending)) \
 		ProcessInterrupts(); \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 #else							/* WIN32 */
 
@@ -109,6 +134,7 @@ do { \
 		pgwin32_dispatch_queued_signals(); \
 	if (unlikely(InterruptPending)) \
 		ProcessInterrupts(); \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 #endif							/* WIN32 */
 
@@ -135,6 +161,7 @@ do { \
 do { \
 	Assert(CritSectionCount > 0); \
 	CritSectionCount--; \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 
 
-- 
2.18.0

From 15720547e8d3a32603027cd10163eb23d6766696 Mon Sep 17 00:00:00 2001
From: Robert Haas <rhaas@postgresql.org>
Date: Wed, 7 Oct 2020 13:04:16 -0400
Subject: [PATCH v11 2/6] Test module for barriers. NOT FOR COMMIT.

---
 contrib/barrier/Makefile         | 23 ++++++++++++
 contrib/barrier/barrier--1.0.sql | 14 +++++++
 contrib/barrier/barrier.c        | 63 ++++++++++++++++++++++++++++++++
 contrib/barrier/barrier.control  |  5 +++
 4 files changed, 105 insertions(+)
 create mode 100644 contrib/barrier/Makefile
 create mode 100644 contrib/barrier/barrier--1.0.sql
 create mode 100644 contrib/barrier/barrier.c
 create mode 100644 contrib/barrier/barrier.control

diff --git a/contrib/barrier/Makefile b/contrib/barrier/Makefile
new file mode 100644
index 00000000000..71f59f6629e
--- /dev/null
+++ b/contrib/barrier/Makefile
@@ -0,0 +1,23 @@
+# contrib/barrier/Makefile
+
+MODULE_big = barrier
+OBJS = \
+	$(WIN32RES) \
+	barrier.o
+
+EXTENSION = barrier
+DATA = barrier--1.0.sql
+PGFILEDESC = "barrier - barrier test code NOT FOR COMMIT"
+
+REGRESS = barrier
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/barrier
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/barrier/barrier--1.0.sql b/contrib/barrier/barrier--1.0.sql
new file mode 100644
index 00000000000..66cae976a96
--- /dev/null
+++ b/contrib/barrier/barrier--1.0.sql
@@ -0,0 +1,14 @@
+/* contrib/barrier/barrier--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION barrier" to load this file. \quit
+
+CREATE FUNCTION emit_barrier(barrier_type text, count integer default 1)
+RETURNS void
+AS 'MODULE_PATHNAME', 'emit_barrier'
+LANGUAGE C STRICT;
+
+CREATE FUNCTION wait_barrier(barrier_type text)
+RETURNS void
+AS 'MODULE_PATHNAME', 'wait_barrier'
+LANGUAGE C STRICT;
diff --git a/contrib/barrier/barrier.c b/contrib/barrier/barrier.c
new file mode 100644
index 00000000000..a0b98439924
--- /dev/null
+++ b/contrib/barrier/barrier.c
@@ -0,0 +1,63 @@
+/*-------------------------------------------------------------------------
+ *
+ * barrier.c
+ *	  emit ProcSignalBarriers for testing purposes
+ *
+ * Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ *
+ *	  contrib/barrier/barrier.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "funcapi.h"
+#include "miscadmin.h"
+#include "storage/procsignal.h"
+#include "utils/builtins.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(emit_barrier);
+PG_FUNCTION_INFO_V1(wait_barrier);
+
+static ProcSignalBarrierType
+get_barrier_type(text *barrier_type)
+{
+	char	   *btype = text_to_cstring(barrier_type);
+
+	if (strcmp(btype, "placeholder") == 0)
+		return PROCSIGNAL_BARRIER_PLACEHOLDER;
+
+	elog(ERROR, "unknown barrier type: \"%s\"", btype);
+}
+
+Datum
+emit_barrier(PG_FUNCTION_ARGS)
+{
+	text	   *barrier_type = PG_GETARG_TEXT_PP(0);
+	int32		count = PG_GETARG_INT32(1);
+	int32		i;
+	ProcSignalBarrierType t = get_barrier_type(barrier_type);
+
+	for (i = 0; i < count; ++i)
+	{
+		CHECK_FOR_INTERRUPTS();
+		EmitProcSignalBarrier(t);
+	}
+
+	PG_RETURN_VOID();
+}
+
+Datum
+wait_barrier(PG_FUNCTION_ARGS)
+{
+	text	   *barrier_type = PG_GETARG_TEXT_PP(0);
+	ProcSignalBarrierType t = get_barrier_type(barrier_type);
+	uint64		generation;
+
+	generation = EmitProcSignalBarrier(t);
+	elog(NOTICE, "waiting for barrier");
+	WaitForProcSignalBarrier(generation);
+
+	PG_RETURN_VOID();
+}
diff --git a/contrib/barrier/barrier.control b/contrib/barrier/barrier.control
new file mode 100644
index 00000000000..425ffc15432
--- /dev/null
+++ b/contrib/barrier/barrier.control
@@ -0,0 +1,5 @@
+# barrier extension
+comment = 'emit ProcSignalBarrier for test purposes'
+default_version = '1.0'
+module_pathname = '$libdir/barrier'
+relocatable = true
-- 
2.18.0

From cf62f0034292f1070ee388cf8dd3fe95c5c1db25 Mon Sep 17 00:00:00 2001
From: Robert Haas <rhaas@postgresql.org>
Date: Tue, 6 Oct 2020 15:26:43 -0400
Subject: [PATCH v11 1/6] Allow for error or refusal while absorbing barriers.

Previously, the per-barrier-type functions tasked with absorbing
them were expected to always succeed and never throw an error.
However, that's a bit inconvenient. Further study has revealed that
there are realistic cases where it might not be possible to absorb
a ProcSignalBarrier without terminating the transaction, or even
the whole backend. Similarly, for some barrier types, there might
be other reasons where it's not reasonably possible to absorb the
barrier at certain points in the code, so provide a way for a
per-barrier-type function to reject absorbing the barrier.

Patch by me, reviewed by Andres Freund.

Discussion: http://postgr.es/m/20200908182005.xya7wetdh3pndzim@alap3.anarazel.de
---
 src/backend/storage/ipc/procsignal.c | 125 ++++++++++++++++++++++++---
 1 file changed, 113 insertions(+), 12 deletions(-)

diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index ffe67acea1c..9c3ebccd3e2 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -18,6 +18,7 @@
 #include <unistd.h>
 
 #include "access/parallel.h"
+#include "port/pg_bitutils.h"
 #include "commands/async.h"
 #include "miscadmin.h"
 #include "pgstat.h"
@@ -87,12 +88,17 @@ typedef struct
 #define BARRIER_SHOULD_CHECK(flags, type) \
 	(((flags) & (((uint32) 1) << (uint32) (type))) != 0)
 
+/* Clear the relevant type bit from the flags. */
+#define BARRIER_CLEAR_BIT(flags, type) \
+	((flags) &= ~(((uint32) 1) << (uint32) (type)))
+
 static ProcSignalHeader *ProcSignal = NULL;
 static volatile ProcSignalSlot *MyProcSignalSlot = NULL;
 
 static bool CheckProcSignal(ProcSignalReason reason);
 static void CleanupProcSignalState(int status, Datum arg);
-static void ProcessBarrierPlaceholder(void);
+static void ResetProcSignalBarrierBits(uint32 flags);
+static bool ProcessBarrierPlaceholder(void);
 
 /*
  * ProcSignalShmemSize
@@ -394,6 +400,12 @@ WaitForProcSignalBarrier(uint64 generation)
 		volatile ProcSignalSlot *slot = &ProcSignal->psh_slot[i];
 		uint64		oldval;
 
+		/*
+		 * It's important that we check only pss_barrierGeneration here and
+		 * not pss_barrierCheckMask. Bits in pss_barrierCheckMask get cleared
+		 * before the barrier is actually absorbed, but pss_barrierGeneration
+		 * is updated only afterward.
+		 */
 		oldval = pg_atomic_read_u64(&slot->pss_barrierGeneration);
 		while (oldval < generation)
 		{
@@ -453,7 +465,7 @@ ProcessProcSignalBarrier(void)
 {
 	uint64		local_gen;
 	uint64		shared_gen;
-	uint32		flags;
+	volatile uint32		flags;
 
 	Assert(MyProcSignalSlot);
 
@@ -482,21 +494,92 @@ ProcessProcSignalBarrier(void)
 	 * read of the barrier generation above happens before we atomically
 	 * extract the flags, and that any subsequent state changes happen
 	 * afterward.
+	 *
+	 * NB: In order to avoid race conditions, we must zero pss_barrierCheckMask
+	 * first and only afterwards try to do barrier processing. If we did it
+	 * in the other order, someone could send us another barrier of some
+	 * type right after we called the barrier-processing function but before
+	 * we cleared the bit. We would have no way of knowing that the bit needs
+	 * to stay set in that case, so the need to call the barrier-processing
+	 * function again would just get forgotten. So instead, we tentatively
+	 * clear all the bits and then put back any for which we don't manage
+	 * to successfully absorb the barrier.
 	 */
 	flags = pg_atomic_exchange_u32(&MyProcSignalSlot->pss_barrierCheckMask, 0);
 
 	/*
-	 * Process each type of barrier. It's important that nothing we call from
-	 * here throws an error, because pss_barrierCheckMask has already been
-	 * cleared. If we jumped out of here before processing all barrier types,
-	 * then we'd forget about the need to do so later.
-	 *
-	 * NB: It ought to be OK to call the barrier-processing functions
-	 * unconditionally, but it's more efficient to call only the ones that
-	 * might need us to do something based on the flags.
+	 * If there are no flags set, then we can skip doing any real work.
+	 * Otherwise, establish a PG_TRY block, so that we don't lose track of
+	 * which types of barrier processing are needed if an ERROR occurs.
 	 */
-	if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_PLACEHOLDER))
-		ProcessBarrierPlaceholder();
+	if (flags != 0)
+	{
+		bool	success = true;
+
+		PG_TRY();
+		{
+			/*
+			 * Process each type of barrier. The barrier-processing functions
+			 * should normally return true, but may return false if the barrier
+			 * can't be absorbed at the current time. This should be rare,
+			 * because it's pretty expensive.  Every single
+			 * CHECK_FOR_INTERRUPTS() will return here until we manage to
+			 * absorb the barrier, and that cost will add up in a hurry.
+			 *
+			 * NB: It ought to be OK to call the barrier-processing functions
+			 * unconditionally, but it's more efficient to call only the ones
+			 * that might need us to do something based on the flags.
+			 */
+			while (flags != 0)
+			{
+				ProcSignalBarrierType	type;
+				bool processed = true;
+
+				type = (ProcSignalBarrierType) pg_rightmost_one_pos32(flags);
+				switch (type)
+				{
+					case PROCSIGNAL_BARRIER_PLACEHOLDER:
+						processed = ProcessBarrierPlaceholder();
+						break;
+				}
+
+				/*
+				 * To avoid an infinite loop, we must always unset the bit
+				 * in flags.
+				 */
+				BARRIER_CLEAR_BIT(flags, type);
+
+				/*
+				 * If we failed to process the barrier, reset the shared bit
+				 * so we try again later, and set a flag so that we don't bump
+				 * our generation.
+				 */
+				if (!processed)
+				{
+					ResetProcSignalBarrierBits(((uint32) 1) << type);
+					success = false;
+				}
+			}
+		}
+		PG_CATCH();
+		{
+			/*
+			 * If an ERROR occurred, we'll need to try again later to handle
+			 * that barrier type and any others that haven't been handled yet
+			 * or weren't successfully absorbed.
+			 */
+			ResetProcSignalBarrierBits(flags);
+			PG_RE_THROW();
+		}
+		PG_END_TRY();
+
+		/*
+		 * If some barrier types were not successfully absorbed, we will have
+		 * to try again later.
+		 */
+		if (!success)
+			return;
+	}
 
 	/*
 	 * State changes related to all types of barriers that might have been
@@ -508,7 +591,20 @@ ProcessProcSignalBarrier(void)
 	pg_atomic_write_u64(&MyProcSignalSlot->pss_barrierGeneration, shared_gen);
 }
 
+/*
+ * If it turns out that we couldn't absorb one or more barrier types, either
+ * because the barrier-processing functions returned false or due to an error,
+ * arrange for processing to be retried later.
+ */
 static void
+ResetProcSignalBarrierBits(uint32 flags)
+{
+	pg_atomic_fetch_or_u32(&MyProcSignalSlot->pss_barrierCheckMask, flags);
+	ProcSignalBarrierPending = true;
+	InterruptPending = true;
+}
+
+static bool
 ProcessBarrierPlaceholder(void)
 {
 	/*
@@ -518,7 +614,12 @@ ProcessBarrierPlaceholder(void)
 	 * appropriately descriptive. Get rid of this function and instead have
 	 * ProcessBarrierSomethingElse. Most likely, that function should live in
 	 * the file pertaining to that subsystem, rather than here.
+	 *
+	 * The return value should be 'true' if the barrier was successfully
+	 * absorbed and 'false' if not. Note that returning 'false' can lead to
+	 * very frequent retries, so try hard to make that an uncommon case.
 	 */
+	return true;
 }
 
 /*
-- 
2.18.0

From efb62b722d624770d4b686dbd7ff8db465118c69 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Tue, 14 Jul 2020 02:30:44 -0400
Subject: [PATCH v11 6/6] WIP - Documentation.

TODOs:

1] Documentation regarding ALTER SYSTEM READ/WRITE
---
 src/backend/access/transam/README | 60 ++++++++++++++++++++++++++++---
 src/backend/storage/page/README   | 12 +++----
 2 files changed, 61 insertions(+), 11 deletions(-)

diff --git a/src/backend/access/transam/README b/src/backend/access/transam/README
index 1edc8180c12..fe7a84f93da 100644
--- a/src/backend/access/transam/README
+++ b/src/backend/access/transam/README
@@ -442,8 +442,8 @@ to be modified.
 2. START_CRIT_SECTION()  (Any error during the next three steps must cause a
 PANIC because the shared buffers will contain unlogged changes, which we
 have to ensure don't get to disk.  Obviously, you should check conditions
-such as whether there's enough free space on the page before you start the
-critical section.)
+such as whether there's WAL write permission and enough free space on the page
+before you start the critical section.)
 
 3. Apply the required changes to the shared buffer(s).
 
@@ -486,6 +486,54 @@ with the incomplete-split flag set, it will finish the interrupted split by
 inserting the key to the parent, before proceeding.
 
 
+Read only system state
+----------------------
+
+The system state when it is not currently possible to insert write-ahead log
+records, either because the system is still in recovery or because the system
+forced to WAL prohibited by executing ALTER SYSTEM READ ONLY.  We have a
+lower-level defense in XLogBeginInsert() and elsewhere to stop us from modifying
+data during recovery when !XLogInsertAllowed(), but if XLogBeginInsert() is
+inside the critical section we must not depend on it to report an error.
+Otherwise, it will cause PANIC as mentioned previously.
+
+We do not reach the point where we try to write WAL during recovery but ALTER
+SYSTEM READ ONLY can be executed anytime by the user to stop WAL writing.  Any
+backends which receive read-only system state transition barrier interrupt need
+to stop WAL writing immediately.  For barrier absorption the backed(s) will kill
+the running transaction which has valid XID indicates that the transaction has
+performed and/or planning WAL write.  The transaction which doesn't acquire
+valid XID yet or operation such VACUUM or CONCURRENT CREATE INDEX which not
+necessary have valid XID for WAL will not be prevented while barrier processing,
+and those might hit the error from XLogBeginInsert() while trying to write WAL
+in read only system state.  To prevent such error from XLogBeginInsert() inside
+the critical section the WAL write permission has to check before
+START_CRIT_SECTION().
+
+To enforce the practice to check WAL permission before entering into critical
+section for the WAL write, we have added an assert check flag that indicates
+permission has been checked before calling XLogBeginInsert().  If not,
+XLogBeginInsert() will have assertion failure.  WAL permission check is not
+mandatory if the XLogBeginInsert() is not inside the critical section where
+throwing the error is acceptable.  To get permission check flag set either
+CheckWALPermitted(), AssertWALPermitted_HaveXID(), or AssertWALPermitted()
+should be called before START_CRIT_SECTION().  This flag automatically resets
+while exiting from the critical section.  The rule to place either of permission
+check routines will be:
+
+	The places where WAL write operation in critical can be expected without
+	having valid XID (e.g vacuum) need to protect by CheckWALPermitted(), so
+	that error can be reported outside before critical section.
+
+	The places where INSERT and UPDATE are expected which are never happened
+	without valid XID can be checked using AssertWALPermitted_HaveXID.  So that
+	non-assert build will not have the checking overhead.
+
+	The places we know that we cannot be reached in the read-only state and may
+	or may not have XID, but need to ensure the permission has been checked on
+	assert enabled build should use AssertWALPermitted().
+
+
 Constructing a WAL record
 -------------------------
 
@@ -531,7 +579,8 @@ Details of the API functions:
 
 void XLogBeginInsert(void)
 
-    Must be called before XLogRegisterBuffer and XLogRegisterData.
+	Must be called before XLogRegisterBuffer and XLogRegisterData.  WAL
+	permission must be check before calling it in a critical section.
 
 void XLogResetInsertion(void)
 
@@ -638,8 +687,9 @@ MarkBufferDirtyHint() to mark the block dirty.
 If the buffer is clean and checksums are in use then MarkBufferDirtyHint()
 inserts an XLOG_FPI_FOR_HINT record to ensure that we take a full page image
 that includes the hint. We do this to avoid a partial page write, when we
-write the dirtied page. WAL is not written during recovery, so we simply skip
-dirtying blocks because of hints when in recovery.
+write the dirtied page. WAL is not written while in read only (i.e. during
+recovery or in WAL prohibit state), so we simply skip dirtying blocks because of
+hints when in recovery.
 
 If you do decide to optimise away a WAL record, then any calls to
 MarkBufferDirty() must be replaced by MarkBufferDirtyHint(),
diff --git a/src/backend/storage/page/README b/src/backend/storage/page/README
index e30d7ac59ad..15f0bb4b7b5 100644
--- a/src/backend/storage/page/README
+++ b/src/backend/storage/page/README
@@ -56,9 +56,9 @@ WAL is a fatal error and prevents further recovery, whereas a checksum failure
 on a normal data block is a hard error but not a critical one for the server,
 even if it is a very bad thing for the user.
 
-New WAL records cannot be written during recovery, so hint bits set during
-recovery must not dirty the page if the buffer is not already dirty, when
-checksums are enabled.  Systems in Hot-Standby mode may benefit from hint bits
-being set, but with checksums enabled, a page cannot be dirtied after setting a
-hint bit (due to the torn page risk). So, it must wait for full-page images
-containing the hint bit updates to arrive from the primary.
+New WAL records cannot be written during recovery or or while in WAL prohibit
+state, so hint bits set during recovery must not dirty the page if the buffer is
+not already dirty, when checksums are enabled.  Systems in Hot-Standby mode may
+benefit from hint bits being set, but with checksums enabled, a page cannot be
+dirtied after setting a hint bit (due to the torn page risk). So, it must wait
+for full-page images containing the hint bit updates to arrive from the primary.
-- 
2.18.0

From e79194f4adb77414ff191683bec2724d57261ea5 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 27 Mar 2020 05:05:38 -0400
Subject: [PATCH v11 3/6] Add alter system read only/write syntax

Note that syntax doesn't have any implementation.
---
 src/backend/nodes/copyfuncs.c    | 12 ++++++++++++
 src/backend/nodes/equalfuncs.c   |  9 +++++++++
 src/backend/nodes/outfuncs.c     | 12 ++++++++++++
 src/backend/nodes/readfuncs.c    | 15 +++++++++++++++
 src/backend/parser/gram.y        | 13 +++++++++++++
 src/backend/tcop/utility.c       | 21 +++++++++++++++++++++
 src/bin/psql/tab-complete.c      |  6 ++++--
 src/include/nodes/nodes.h        |  1 +
 src/include/nodes/parsenodes.h   | 10 ++++++++++
 src/tools/pgindent/typedefs.list |  1 +
 10 files changed, 98 insertions(+), 2 deletions(-)

diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 5a591d0a751..7ba114f6fdc 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -4010,6 +4010,15 @@ _copyAlterSystemStmt(const AlterSystemStmt *from)
 	return newnode;
 }
 
+static AlterSystemWALProhibitState *
+_copyAlterSystemWALProhibitState(const AlterSystemWALProhibitState *from)
+{
+	AlterSystemWALProhibitState *newnode = makeNode(AlterSystemWALProhibitState);
+
+	COPY_SCALAR_FIELD(walprohibited);
+	return newnode;
+}
+
 static CreateSeqStmt *
 _copyCreateSeqStmt(const CreateSeqStmt *from)
 {
@@ -5393,6 +5402,9 @@ copyObjectImpl(const void *from)
 		case T_AlterSystemStmt:
 			retval = _copyAlterSystemStmt(from);
 			break;
+		case T_AlterSystemWALProhibitState:
+			retval = _copyAlterSystemWALProhibitState(from);
+			break;
 		case T_CreateSeqStmt:
 			retval = _copyCreateSeqStmt(from);
 			break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e2895a8985d..269683fb873 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1761,6 +1761,12 @@ _equalAlterSystemStmt(const AlterSystemStmt *a, const AlterSystemStmt *b)
 	return true;
 }
 
+static bool
+_equalAlterSystemWALProhibitState(const AlterSystemWALProhibitState *a, const AlterSystemWALProhibitState *b)
+{
+	COMPARE_SCALAR_FIELD(walprohibited);
+	return true;
+}
 
 static bool
 _equalCreateSeqStmt(const CreateSeqStmt *a, const CreateSeqStmt *b)
@@ -3448,6 +3454,9 @@ equal(const void *a, const void *b)
 		case T_AlterSystemStmt:
 			retval = _equalAlterSystemStmt(a, b);
 			break;
+		case T_AlterSystemWALProhibitState:
+			retval = _equalAlterSystemWALProhibitState(a, b);
+			break;
 		case T_CreateSeqStmt:
 			retval = _equalCreateSeqStmt(a, b);
 			break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index f26498cea2d..69e11e913a0 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -1356,6 +1356,15 @@ _outAlternativeSubPlan(StringInfo str, const AlternativeSubPlan *node)
 	WRITE_NODE_FIELD(subplans);
 }
 
+static void
+_outAlterSystemWALProhibitState(StringInfo str,
+								const AlterSystemWALProhibitState *node)
+{
+	WRITE_NODE_TYPE("ALTERSYSTEMWALPROHIBITSTATE");
+
+	WRITE_BOOL_FIELD(walprohibited);
+}
+
 static void
 _outFieldSelect(StringInfo str, const FieldSelect *node)
 {
@@ -3909,6 +3918,9 @@ outNode(StringInfo str, const void *obj)
 			case T_AlternativeSubPlan:
 				_outAlternativeSubPlan(str, obj);
 				break;
+			case T_AlterSystemWALProhibitState:
+				_outAlterSystemWALProhibitState(str, obj);
+				break;
 			case T_FieldSelect:
 				_outFieldSelect(str, obj);
 				break;
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index ab7b535caae..d5acb438f6d 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -2550,6 +2550,19 @@ _readAlternativeSubPlan(void)
 	READ_DONE();
 }
 
+/*
+ * _readAlterSystemWALProhibitState
+ */
+static AlterSystemWALProhibitState *
+_readAlterSystemWALProhibitState(void)
+{
+	READ_LOCALS(AlterSystemWALProhibitState);
+
+	READ_BOOL_FIELD(walprohibited);
+
+	READ_DONE();
+}
+
 /*
  * _readExtensibleNode
  */
@@ -2872,6 +2885,8 @@ parseNodeString(void)
 		return_value = _readSubPlan();
 	else if (MATCH("ALTERNATIVESUBPLAN", 18))
 		return_value = _readAlternativeSubPlan();
+	else if (MATCH("ALTERSYSTEMWALPROHIBITSTATE", 27))
+		return_value = _readAlterSystemWALProhibitState();
 	else if (MATCH("EXTENSIBLENODE", 14))
 		return_value = _readExtensibleNode();
 	else if (MATCH("PARTITIONBOUNDSPEC", 18))
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index efc9c997541..ceed52fbaae 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -479,6 +479,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
 
 %type <vsetstmt> generic_set set_rest set_rest_more generic_reset reset_rest
 				 SetResetClause FunctionSetResetClause
+%type <boolean> system_readonly_state
 
 %type <node>	TableElement TypedTableElement ConstraintElem TableFuncElement
 %type <node>	columnDef columnOptions
@@ -10202,8 +10203,20 @@ AlterSystemStmt:
 					n->setstmt = $4;
 					$$ = (Node *)n;
 				}
+			| ALTER SYSTEM_P system_readonly_state
+				{
+					AlterSystemWALProhibitState *n = makeNode(AlterSystemWALProhibitState);
+					n->walprohibited = $3;
+					$$ = (Node *)n;
+				}
 		;
 
+system_readonly_state:
+			 READ ONLY
+					{ $$ = true; }
+			| READ WRITE
+					{ $$ = false; }
+		;
 
 /*****************************************************************************
  *
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 81ac9b1cb2d..f68b11d0af1 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -85,6 +85,7 @@ static void ProcessUtilitySlow(ParseState *pstate,
 							   DestReceiver *dest,
 							   QueryCompletion *qc);
 static void ExecDropStmt(DropStmt *stmt, bool isTopLevel);
+static void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
 
 /*
  * CommandIsReadOnly: is an executable query read-only?
@@ -219,6 +220,7 @@ ClassifyUtilityCommandAsReadOnly(Node *parsetree)
 				return COMMAND_IS_NOT_READ_ONLY;
 			}
 
+		case T_AlterSystemWALProhibitState:
 		case T_AlterSystemStmt:
 			{
 				/*
@@ -834,6 +836,11 @@ standard_ProcessUtility(PlannedStmt *pstmt,
 			AlterSystemSetConfigFile((AlterSystemStmt *) parsetree);
 			break;
 
+		case T_AlterSystemWALProhibitState:
+			PreventInTransactionBlock(isTopLevel, "ALTER SYSTEM");
+			AlterSystemSetWALProhibitState((AlterSystemWALProhibitState *) parsetree);
+			break;
+
 		case T_VariableSetStmt:
 			ExecSetVariableStmt((VariableSetStmt *) parsetree, isTopLevel);
 			break;
@@ -2813,6 +2820,7 @@ CreateCommandTag(Node *parsetree)
 			tag = CMDTAG_REFRESH_MATERIALIZED_VIEW;
 			break;
 
+		case T_AlterSystemWALProhibitState:
 		case T_AlterSystemStmt:
 			tag = CMDTAG_ALTER_SYSTEM;
 			break;
@@ -3669,3 +3677,16 @@ GetCommandLogLevel(Node *parsetree)
 
 	return lev;
 }
+
+/*
+ * AlterSystemSetWALProhibitState
+ *
+ * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
+ */
+static void
+AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
+{
+	ereport(ERROR,
+			(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+			 errmsg("ALTER SYSTEM READ { ONLY | WRITE } not implemented")));
+}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 8afc780acc3..d416106e0ab 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1889,9 +1889,11 @@ psql_completion(const char *text, int start, int end)
 	/* ALTER SERVER <name> VERSION <version> */
 	else if (Matches("ALTER", "SERVER", MatchAny, "VERSION", MatchAny))
 		COMPLETE_WITH("OPTIONS");
-	/* ALTER SYSTEM SET, RESET, RESET ALL */
+	/* ALTER SYSTEM READ, SET, RESET, RESET ALL */
 	else if (Matches("ALTER", "SYSTEM"))
-		COMPLETE_WITH("SET", "RESET");
+		COMPLETE_WITH("SET", "RESET", "READ");
+	else if (Matches("ALTER", "SYSTEM", "READ"))
+		COMPLETE_WITH("ONLY", "WRITE");
 	else if (Matches("ALTER", "SYSTEM", "SET|RESET"))
 		COMPLETE_WITH_QUERY(Query_for_list_of_alter_system_set_vars);
 	else if (Matches("ALTER", "SYSTEM", "SET", MatchAny))
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 7ddd8c011bf..7b233925692 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -411,6 +411,7 @@ typedef enum NodeTag
 	T_RefreshMatViewStmt,
 	T_ReplicaIdentityStmt,
 	T_AlterSystemStmt,
+	T_AlterSystemWALProhibitState,
 	T_CreatePolicyStmt,
 	T_AlterPolicyStmt,
 	T_CreateTransformStmt,
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index d1f9ef29ca0..6f7b4f0600f 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -3192,6 +3192,16 @@ typedef struct AlterSystemStmt
 	VariableSetStmt *setstmt;	/* SET subcommand */
 } AlterSystemStmt;
 
+/* ----------------------
+ *		Alter System Read Statement
+ * ----------------------
+ */
+typedef struct AlterSystemWALProhibitState
+{
+	NodeTag		type;
+	bool		walprohibited;
+} AlterSystemWALProhibitState;
+
 /* ----------------------
  *		Cluster Statement (support pbrown's cluster index implementation)
  * ----------------------
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index fde701bfd4d..37bfa69edf0 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -87,6 +87,7 @@ AlterStatsStmt
 AlterSubscriptionStmt
 AlterSubscriptionType
 AlterSystemStmt
+AlterSystemWALProhibitState
 AlterTSConfigType
 AlterTSConfigurationStmt
 AlterTSDictionaryStmt
-- 
2.18.0

From b9304d1f557e02065898d61badae867901556a74 Mon Sep 17 00:00:00 2001
From: Robert Haas <rhaas@postgresql.org>
Date: Tue, 6 Oct 2020 15:26:43 -0400
Subject: [PATCH v12 1/6] Allow for error or refusal while absorbing barriers.

Previously, the per-barrier-type functions tasked with absorbing
them were expected to always succeed and never throw an error.
However, that's a bit inconvenient. Further study has revealed that
there are realistic cases where it might not be possible to absorb
a ProcSignalBarrier without terminating the transaction, or even
the whole backend. Similarly, for some barrier types, there might
be other reasons where it's not reasonably possible to absorb the
barrier at certain points in the code, so provide a way for a
per-barrier-type function to reject absorbing the barrier.

Patch by me, reviewed by Andres Freund.

Discussion: http://postgr.es/m/20200908182005.xya7wetdh3pndzim@alap3.anarazel.de
---
 src/backend/storage/ipc/procsignal.c | 125 ++++++++++++++++++++++++---
 1 file changed, 113 insertions(+), 12 deletions(-)

diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 583efaecff8..c43cdd685b4 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -18,6 +18,7 @@
 #include <unistd.h>
 
 #include "access/parallel.h"
+#include "port/pg_bitutils.h"
 #include "commands/async.h"
 #include "miscadmin.h"
 #include "pgstat.h"
@@ -87,12 +88,17 @@ typedef struct
 #define BARRIER_SHOULD_CHECK(flags, type) \
 	(((flags) & (((uint32) 1) << (uint32) (type))) != 0)
 
+/* Clear the relevant type bit from the flags. */
+#define BARRIER_CLEAR_BIT(flags, type) \
+	((flags) &= ~(((uint32) 1) << (uint32) (type)))
+
 static ProcSignalHeader *ProcSignal = NULL;
 static volatile ProcSignalSlot *MyProcSignalSlot = NULL;
 
 static bool CheckProcSignal(ProcSignalReason reason);
 static void CleanupProcSignalState(int status, Datum arg);
-static void ProcessBarrierPlaceholder(void);
+static void ResetProcSignalBarrierBits(uint32 flags);
+static bool ProcessBarrierPlaceholder(void);
 
 /*
  * ProcSignalShmemSize
@@ -394,6 +400,12 @@ WaitForProcSignalBarrier(uint64 generation)
 		volatile ProcSignalSlot *slot = &ProcSignal->psh_slot[i];
 		uint64		oldval;
 
+		/*
+		 * It's important that we check only pss_barrierGeneration here and
+		 * not pss_barrierCheckMask. Bits in pss_barrierCheckMask get cleared
+		 * before the barrier is actually absorbed, but pss_barrierGeneration
+		 * is updated only afterward.
+		 */
 		oldval = pg_atomic_read_u64(&slot->pss_barrierGeneration);
 		while (oldval < generation)
 		{
@@ -453,7 +465,7 @@ ProcessProcSignalBarrier(void)
 {
 	uint64		local_gen;
 	uint64		shared_gen;
-	uint32		flags;
+	volatile uint32		flags;
 
 	Assert(MyProcSignalSlot);
 
@@ -482,21 +494,92 @@ ProcessProcSignalBarrier(void)
 	 * read of the barrier generation above happens before we atomically
 	 * extract the flags, and that any subsequent state changes happen
 	 * afterward.
+	 *
+	 * NB: In order to avoid race conditions, we must zero pss_barrierCheckMask
+	 * first and only afterwards try to do barrier processing. If we did it
+	 * in the other order, someone could send us another barrier of some
+	 * type right after we called the barrier-processing function but before
+	 * we cleared the bit. We would have no way of knowing that the bit needs
+	 * to stay set in that case, so the need to call the barrier-processing
+	 * function again would just get forgotten. So instead, we tentatively
+	 * clear all the bits and then put back any for which we don't manage
+	 * to successfully absorb the barrier.
 	 */
 	flags = pg_atomic_exchange_u32(&MyProcSignalSlot->pss_barrierCheckMask, 0);
 
 	/*
-	 * Process each type of barrier. It's important that nothing we call from
-	 * here throws an error, because pss_barrierCheckMask has already been
-	 * cleared. If we jumped out of here before processing all barrier types,
-	 * then we'd forget about the need to do so later.
-	 *
-	 * NB: It ought to be OK to call the barrier-processing functions
-	 * unconditionally, but it's more efficient to call only the ones that
-	 * might need us to do something based on the flags.
+	 * If there are no flags set, then we can skip doing any real work.
+	 * Otherwise, establish a PG_TRY block, so that we don't lose track of
+	 * which types of barrier processing are needed if an ERROR occurs.
 	 */
-	if (BARRIER_SHOULD_CHECK(flags, PROCSIGNAL_BARRIER_PLACEHOLDER))
-		ProcessBarrierPlaceholder();
+	if (flags != 0)
+	{
+		bool	success = true;
+
+		PG_TRY();
+		{
+			/*
+			 * Process each type of barrier. The barrier-processing functions
+			 * should normally return true, but may return false if the barrier
+			 * can't be absorbed at the current time. This should be rare,
+			 * because it's pretty expensive.  Every single
+			 * CHECK_FOR_INTERRUPTS() will return here until we manage to
+			 * absorb the barrier, and that cost will add up in a hurry.
+			 *
+			 * NB: It ought to be OK to call the barrier-processing functions
+			 * unconditionally, but it's more efficient to call only the ones
+			 * that might need us to do something based on the flags.
+			 */
+			while (flags != 0)
+			{
+				ProcSignalBarrierType	type;
+				bool processed = true;
+
+				type = (ProcSignalBarrierType) pg_rightmost_one_pos32(flags);
+				switch (type)
+				{
+					case PROCSIGNAL_BARRIER_PLACEHOLDER:
+						processed = ProcessBarrierPlaceholder();
+						break;
+				}
+
+				/*
+				 * To avoid an infinite loop, we must always unset the bit
+				 * in flags.
+				 */
+				BARRIER_CLEAR_BIT(flags, type);
+
+				/*
+				 * If we failed to process the barrier, reset the shared bit
+				 * so we try again later, and set a flag so that we don't bump
+				 * our generation.
+				 */
+				if (!processed)
+				{
+					ResetProcSignalBarrierBits(((uint32) 1) << type);
+					success = false;
+				}
+			}
+		}
+		PG_CATCH();
+		{
+			/*
+			 * If an ERROR occurred, we'll need to try again later to handle
+			 * that barrier type and any others that haven't been handled yet
+			 * or weren't successfully absorbed.
+			 */
+			ResetProcSignalBarrierBits(flags);
+			PG_RE_THROW();
+		}
+		PG_END_TRY();
+
+		/*
+		 * If some barrier types were not successfully absorbed, we will have
+		 * to try again later.
+		 */
+		if (!success)
+			return;
+	}
 
 	/*
 	 * State changes related to all types of barriers that might have been
@@ -508,7 +591,20 @@ ProcessProcSignalBarrier(void)
 	pg_atomic_write_u64(&MyProcSignalSlot->pss_barrierGeneration, shared_gen);
 }
 
+/*
+ * If it turns out that we couldn't absorb one or more barrier types, either
+ * because the barrier-processing functions returned false or due to an error,
+ * arrange for processing to be retried later.
+ */
 static void
+ResetProcSignalBarrierBits(uint32 flags)
+{
+	pg_atomic_fetch_or_u32(&MyProcSignalSlot->pss_barrierCheckMask, flags);
+	ProcSignalBarrierPending = true;
+	InterruptPending = true;
+}
+
+static bool
 ProcessBarrierPlaceholder(void)
 {
 	/*
@@ -518,7 +614,12 @@ ProcessBarrierPlaceholder(void)
 	 * appropriately descriptive. Get rid of this function and instead have
 	 * ProcessBarrierSomethingElse. Most likely, that function should live in
 	 * the file pertaining to that subsystem, rather than here.
+	 *
+	 * The return value should be 'true' if the barrier was successfully
+	 * absorbed and 'false' if not. Note that returning 'false' can lead to
+	 * very frequent retries, so try hard to make that an uncommon case.
 	 */
+	return true;
 }
 
 /*
-- 
2.18.0

From 2d1c0150e8f41ce15bcea3e052714ec0e7730e70 Mon Sep 17 00:00:00 2001
From: Robert Haas <rhaas@postgresql.org>
Date: Wed, 7 Oct 2020 13:04:16 -0400
Subject: [PATCH v12 2/6] Test module for barriers. NOT FOR COMMIT.

---
 contrib/barrier/Makefile         | 23 ++++++++++++
 contrib/barrier/barrier--1.0.sql | 14 +++++++
 contrib/barrier/barrier.c        | 63 ++++++++++++++++++++++++++++++++
 contrib/barrier/barrier.control  |  5 +++
 4 files changed, 105 insertions(+)
 create mode 100644 contrib/barrier/Makefile
 create mode 100644 contrib/barrier/barrier--1.0.sql
 create mode 100644 contrib/barrier/barrier.c
 create mode 100644 contrib/barrier/barrier.control

diff --git a/contrib/barrier/Makefile b/contrib/barrier/Makefile
new file mode 100644
index 00000000000..71f59f6629e
--- /dev/null
+++ b/contrib/barrier/Makefile
@@ -0,0 +1,23 @@
+# contrib/barrier/Makefile
+
+MODULE_big = barrier
+OBJS = \
+	$(WIN32RES) \
+	barrier.o
+
+EXTENSION = barrier
+DATA = barrier--1.0.sql
+PGFILEDESC = "barrier - barrier test code NOT FOR COMMIT"
+
+REGRESS = barrier
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/barrier
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/barrier/barrier--1.0.sql b/contrib/barrier/barrier--1.0.sql
new file mode 100644
index 00000000000..66cae976a96
--- /dev/null
+++ b/contrib/barrier/barrier--1.0.sql
@@ -0,0 +1,14 @@
+/* contrib/barrier/barrier--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION barrier" to load this file. \quit
+
+CREATE FUNCTION emit_barrier(barrier_type text, count integer default 1)
+RETURNS void
+AS 'MODULE_PATHNAME', 'emit_barrier'
+LANGUAGE C STRICT;
+
+CREATE FUNCTION wait_barrier(barrier_type text)
+RETURNS void
+AS 'MODULE_PATHNAME', 'wait_barrier'
+LANGUAGE C STRICT;
diff --git a/contrib/barrier/barrier.c b/contrib/barrier/barrier.c
new file mode 100644
index 00000000000..a0b98439924
--- /dev/null
+++ b/contrib/barrier/barrier.c
@@ -0,0 +1,63 @@
+/*-------------------------------------------------------------------------
+ *
+ * barrier.c
+ *	  emit ProcSignalBarriers for testing purposes
+ *
+ * Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ *
+ *	  contrib/barrier/barrier.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "funcapi.h"
+#include "miscadmin.h"
+#include "storage/procsignal.h"
+#include "utils/builtins.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(emit_barrier);
+PG_FUNCTION_INFO_V1(wait_barrier);
+
+static ProcSignalBarrierType
+get_barrier_type(text *barrier_type)
+{
+	char	   *btype = text_to_cstring(barrier_type);
+
+	if (strcmp(btype, "placeholder") == 0)
+		return PROCSIGNAL_BARRIER_PLACEHOLDER;
+
+	elog(ERROR, "unknown barrier type: \"%s\"", btype);
+}
+
+Datum
+emit_barrier(PG_FUNCTION_ARGS)
+{
+	text	   *barrier_type = PG_GETARG_TEXT_PP(0);
+	int32		count = PG_GETARG_INT32(1);
+	int32		i;
+	ProcSignalBarrierType t = get_barrier_type(barrier_type);
+
+	for (i = 0; i < count; ++i)
+	{
+		CHECK_FOR_INTERRUPTS();
+		EmitProcSignalBarrier(t);
+	}
+
+	PG_RETURN_VOID();
+}
+
+Datum
+wait_barrier(PG_FUNCTION_ARGS)
+{
+	text	   *barrier_type = PG_GETARG_TEXT_PP(0);
+	ProcSignalBarrierType t = get_barrier_type(barrier_type);
+	uint64		generation;
+
+	generation = EmitProcSignalBarrier(t);
+	elog(NOTICE, "waiting for barrier");
+	WaitForProcSignalBarrier(generation);
+
+	PG_RETURN_VOID();
+}
diff --git a/contrib/barrier/barrier.control b/contrib/barrier/barrier.control
new file mode 100644
index 00000000000..425ffc15432
--- /dev/null
+++ b/contrib/barrier/barrier.control
@@ -0,0 +1,5 @@
+# barrier extension
+comment = 'emit ProcSignalBarrier for test purposes'
+default_version = '1.0'
+module_pathname = '$libdir/barrier'
+relocatable = true
-- 
2.18.0

From 1615be569c0f5a3887e0acc38d3f4f17a4e1dd7b Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 19 Jun 2020 06:29:36 -0400
Subject: [PATCH v12 4/6] WIP - Implement ALTER SYSTEM READ ONLY using global
 barrier.

Implementation:

 1. When a user tried to change server state to WAL-Prohibited using
    ALTER SYSTEM READ ONLY command or by calling
    pg_alter_wal_prohibit_state(true) sql function, the current state
    generation to inprogress in shared memory marked and signaled
    checkpointer process.  Checkpointer, noticing that the current state
    generation has WALPROHIBIT_TRANSITION_IN_PROGRESS flag set, does the
    barrier request, and then acknowledges back to the backend who
    requested the state change once the transition has been completed.
    Final state will be updated in control file to make it persistent
    across the system restarts.

 2. When a backend receives the WAL-Prohibited barrier, at that moment if
    it is already in a transaction and the transaction already assigned XID,
    then the backend will be killed by throwing FATAL(XXX: need more discussion
    on this)

 3. Otherwise, if that backend running transaction without valid XID then, we
    don't need to do anything special right now, simply call
    ResetLocalXLogInsertAllowed() so that any future WAL insert in will check
    XLogInsertAllowed() first which set ready only state appropriately.

 4. A new transaction (in an existing or in a new backend) starts as a
    read-only transaction.

 5. Autovacuum launcher as well as checkpointer will not do anything in
    WAL-Prohibited server state until someone wakes us up.  E.g. a backend
    might later on request us to put the system back to read-write.
 5. The Autovacuum launcher, as well as the checkpointer, will not do
    anything while in the WAL-Prohibited server state until someone wakes
    up.  E.g. user might, later on, request us to put the system back to
    read-write by executing ALTER SYSTEM READ WRITE.

 6. At shutdown in WAL-Prohibited mode, we'll skip shutdown checkpoint
    and xlog rotation. Starting up again will perform crash recovery(XXX:
    need some discussion on this as well) but the end of recovery checkpoint
    will be skipped and it will be performed when the system changed to
    WAL-Permitted mode.

 7. ALTER SYSTEM READ ONLY/WRITE is restricted on standby server.

 8. To execute ALTER SYSTEM READ ONLY/WRITE, the user should have execute
    permssion on pg_alter_wal_prohibit_state() function.

 9. Add system_is_read_only GUC show the system state -- will true when system
    is wal prohibited or in recovery.

=====
TODO:
=====
 1. Commented CheckpointLock acquiring code in CreateCheckPoint() to make
    the latest changes to the checkpointer to process wal prohibit state
    change request ASAP.  We might want to depreciate  CheckpointLock
    completely from CreateCheckPoint(). The same discussion is started on
    pg_hackers[1]. Until then this patch mark as WIP.

====
REF:
====
 1. http://postgr.es/m/CAAJ_b97XnBBfYeSREDJorFsyoD1sHgqnNuCi=02mNQBUMnA=FA@mail.gmail.com

wip - add new function call from middle of checkpointer
---
 src/backend/access/transam/Makefile      |   1 +
 src/backend/access/transam/walprohibit.c | 464 +++++++++++++++++++++++
 src/backend/access/transam/xact.c        |  37 +-
 src/backend/access/transam/xlog.c        | 113 +++++-
 src/backend/catalog/system_views.sql     |   2 +
 src/backend/postmaster/autovacuum.c      |   4 +
 src/backend/postmaster/bgwriter.c        |   2 +-
 src/backend/postmaster/checkpointer.c    |  18 +
 src/backend/postmaster/pgstat.c          |   3 +
 src/backend/storage/ipc/ipci.c           |   6 +
 src/backend/storage/ipc/procsignal.c     |  24 +-
 src/backend/storage/sync/sync.c          |  30 +-
 src/backend/tcop/utility.c               |  15 +-
 src/backend/utils/misc/guc.c             |  26 ++
 src/bin/pg_controldata/pg_controldata.c  |   2 +
 src/include/access/walprohibit.h         |  93 +++++
 src/include/access/xlog.h                |   4 +
 src/include/catalog/pg_control.h         |   3 +
 src/include/catalog/pg_proc.dat          |   4 +
 src/include/pgstat.h                     |   1 +
 src/include/postmaster/bgwriter.h        |   2 +
 src/include/storage/procsignal.h         |   7 +-
 src/tools/pgindent/typedefs.list         |   1 +
 23 files changed, 788 insertions(+), 74 deletions(-)
 create mode 100644 src/backend/access/transam/walprohibit.c
 create mode 100644 src/include/access/walprohibit.h

diff --git a/src/backend/access/transam/Makefile b/src/backend/access/transam/Makefile
index 595e02de722..b5322a69954 100644
--- a/src/backend/access/transam/Makefile
+++ b/src/backend/access/transam/Makefile
@@ -26,6 +26,7 @@ OBJS = \
 	twophase.o \
 	twophase_rmgr.o \
 	varsup.o \
+	walprohibit.o \
 	xact.o \
 	xlog.o \
 	xlogarchive.o \
diff --git a/src/backend/access/transam/walprohibit.c b/src/backend/access/transam/walprohibit.c
new file mode 100644
index 00000000000..de1f2f47040
--- /dev/null
+++ b/src/backend/access/transam/walprohibit.c
@@ -0,0 +1,464 @@
+/*-------------------------------------------------------------------------
+ *
+ * walprohibit.c
+ * 		PostgreSQL write-ahead log prohibit states
+ *
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/backend/access/transam/walprohibit.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/walprohibit.h"
+#include "fmgr.h"
+#include "pgstat.h"
+#include "port/atomics.h"
+#include "postmaster/bgwriter.h"
+#include "postmaster/interrupt.h"
+#include "storage/condition_variable.h"
+#include "storage/procsignal.h"
+#include "storage/shmem.h"
+#include "utils/acl.h"
+#include "utils/fmgroids.h"
+#include "utils/fmgrprotos.h"
+
+/*
+ * Shared-memory WAL prohibit state
+ */
+typedef struct WALProhibitStateData
+{
+	/*
+	 * Indicates current WAL prohibit state generation and the last two bits of
+	 * this generation indicates current wal prohibit state.
+	 */
+	pg_atomic_uint32 shared_state_generation;
+
+	/* Signaled when requested WAL prohibit state changes */
+	ConditionVariable walprohibit_cv;
+} WALProhibitStateData;
+
+static WALProhibitStateData *WALProhibitState = NULL;
+
+static void RequestWALProhibitChange(uint32 cur_state_gen);
+static void CompleteWALProhibitChange(uint32 cur_state_gen);
+static uint32 GetWALProhibitStateGen(void);
+static uint32 SetWALProhibitState(bool wal_prohibited, bool is_final_state);
+
+/*
+ * ProcessBarrierWALProhibit()
+ *
+ * Handle WAL prohibit state change request.
+ */
+bool
+ProcessBarrierWALProhibit(void)
+{
+	/*
+	 * Kill off any transactions that have an XID *before* allowing the system
+	 * to go WAL prohibit state.
+	 */
+	if (FullTransactionIdIsValid(GetTopFullTransactionIdIfAny()))
+	{
+		/*
+		 * Should be here only while transiting towards the WAL prohibit state.
+		 */
+		Assert(WALPROHIBIT_CURRENT_STATE(GetWALProhibitStateGen()) ==
+			   WALPROHIBIT_STATE_GOING_READ_ONLY);
+
+		/*
+		 * XXX: Kill off the whole session by throwing FATAL instead of
+		 * killing transaction by throwing ERROR due to following reasons that
+		 * need be thought:
+		 *
+		 * 1. Due to some presents challenges with the wire protocol, we could
+		 * not simply kill of idle transaction.
+		 *
+		 * 2. If we are here in subtransaction then the ERROR will kill the
+		 * current subtransaction only. In the case of invalidations, that
+		 * might be good enough, but for XID assignment it's not, because
+		 * assigning an XID to a subtransaction also causes higher
+		 * sub-transaction levels and the parent transaction to get XIDs.
+		 */
+		ereport(FATAL,
+				(errcode(ERRCODE_ACTIVE_SQL_TRANSACTION),
+				 errmsg("system is now read only"),
+				 errhint("Sessions with open write transactions must be terminated.")));
+	}
+
+	/* Return to "check" state */
+	ResetLocalXLogInsertAllowed();
+
+	return true;
+}
+
+/*
+ * AlterSystemSetWALProhibitState()
+ *
+ * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
+ */
+void
+AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
+{
+	/* Check permission for pg_alter_wal_prohibit_state() */
+	if (pg_proc_aclcheck(F_PG_ALTER_WAL_PROHIBIT_STATE,
+						 GetUserId(), ACL_EXECUTE) != ACLCHECK_OK)
+		ereport(ERROR,
+				(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+				 errmsg("permission denied for command ALTER SYSTEM"),
+				 errhint("Get execute permission for pg_alter_wal_prohibit_state() to this user.")));
+
+	/* Alter WAL prohibit state not allowed during recovery */
+	PreventCommandDuringRecovery("ALTER SYSTEM");
+
+	/* Execute function to alter wal prohibit state */
+	(void) OidFunctionCall1(F_PG_ALTER_WAL_PROHIBIT_STATE,
+							BoolGetDatum(stmt->walprohibited));
+}
+
+/*
+ * pg_alter_wal_prohibit_state()
+ *
+ * SQL callable function to alter system read write state.
+ */
+Datum
+pg_alter_wal_prohibit_state(PG_FUNCTION_ARGS)
+{
+	bool		walprohibited = PG_GETARG_BOOL(0);
+	uint32		cur_state_gen;
+
+	/* Alter WAL prohibit state not allowed during recovery */
+	PreventCommandDuringRecovery("pg_alter_wal_prohibit_state()");
+
+	/*
+	 * It is not a final state since we yet to convey this WAL prohibit state to
+	 * all backend.
+	 */
+	cur_state_gen = SetWALProhibitState(walprohibited, false);
+
+	/* Server is already in requested state */
+	if (!cur_state_gen)
+		PG_RETURN_VOID();
+
+	/*
+	 * Signal the checkpointer to do the actual state transition, and wait for
+	 * the state change to occur.
+	 */
+	RequestWALProhibitChange(cur_state_gen);
+
+	PG_RETURN_VOID();
+}
+
+/*
+ * Is the system still in WAL prohibited state?
+ */
+bool
+IsWALProhibited(void)
+{
+	uint32 		cur_state = WALPROHIBIT_CURRENT_STATE(GetWALProhibitStateGen());
+
+	return (cur_state != WALPROHIBIT_STATE_READ_WRITE &&
+			cur_state != WALPROHIBIT_STATE_GOING_READ_WRITE);
+}
+
+/*
+ * RequestWALProhibitChange()
+ *
+ * Request checkpointer to make the WALProhibitState to read-only.
+ */
+static void
+RequestWALProhibitChange(uint32 cur_state_gen)
+{
+	/* Must not be called from checkpointer */
+	Assert(!AmCheckpointerProcess());
+	Assert(GetWALProhibitStateGen() & WALPROHIBIT_TRANSITION_IN_PROGRESS);
+
+	/*
+	 * If in a standalone backend, just do it ourselves.
+	 */
+	if (!IsPostmasterEnvironment)
+	{
+		CompleteWALProhibitChange(cur_state_gen);
+		return;
+	}
+
+	/* Signal checkpointer process */
+	SendsSignalToCheckpointer(SIGINT);
+
+	/* Wait for the state to change to read-only */
+	ConditionVariablePrepareToSleep(&WALProhibitState->walprohibit_cv);
+	for (;;)
+	{
+		/* We'll be done once wal prohibit state generation changes */
+		if (GetWALProhibitStateGen() != cur_state_gen)
+			break;
+
+		ConditionVariableSleep(&WALProhibitState->walprohibit_cv,
+							   WAIT_EVENT_WALPROHIBIT_STATE_CHANGE);
+	}
+	ConditionVariableCancelSleep();
+}
+
+/*
+ * CompleteWALProhibitChange()
+ *
+ * Complete the requested WAL prohibit state transition.
+ */
+static void
+CompleteWALProhibitChange(uint32 cur_state_gen)
+{
+	uint64		barrier_gen;
+	bool		wal_prohibited;
+
+	/*
+	 * Must be called from checkpointer. Otherwise, it must be single-user
+	 * backend.
+	 */
+	Assert(AmCheckpointerProcess() || !IsPostmasterEnvironment);
+	Assert(cur_state_gen & WALPROHIBIT_TRANSITION_IN_PROGRESS);
+
+	/*
+	 * WAL prohibit state change is initiated. We need to complete the state
+	 * transition by setting requested WAL prohibit state in all backends.
+	 */
+	elog(DEBUG1, "waiting for backends to adopt requested WAL prohibit state change");
+
+	/* Emit global barrier */
+	barrier_gen = EmitProcSignalBarrier(PROCSIGNAL_BARRIER_WALPROHIBIT);
+	WaitForProcSignalBarrier(barrier_gen);
+
+	/*
+	 * Since we do not process barrier for ourself, for now, return to "check"
+	 * state.
+	 */
+	ResetLocalXLogInsertAllowed();
+
+	/* And flush all inserts. */
+	XLogFlush(GetXLogInsertRecPtr());
+
+	wal_prohibited =
+		(WALPROHIBIT_NEXT_STATE(cur_state_gen) == WALPROHIBIT_STATE_READ_ONLY);
+
+	/* Set the final state */
+	(void) SetWALProhibitState(wal_prohibited, true);
+
+	/* Update the control file to make state persistent */
+	SetControlFileWALProhibitFlag(wal_prohibited);
+
+	if (wal_prohibited)
+		ereport(LOG, (errmsg("system is now read only")));
+	else
+	{
+		/*
+		 * Request checkpoint if the end-of-recovery checkpoint has been skipped
+		 * previously.
+		 */
+		if (LastCheckPointIsSkipped())
+		{
+			RequestCheckpoint(CHECKPOINT_IMMEDIATE);
+			SetLastCheckPointSkipped(false);
+		}
+		ereport(LOG, (errmsg("system is now read write")));
+	}
+
+	/* Wake up all backends waiting on this. */
+	ConditionVariableBroadcast(&WALProhibitState->walprohibit_cv);
+}
+
+/*
+ * ProcessWALProhibitStateChangeRequest()
+ *
+ * Checkpointer will complete wal prohibit state change request.
+ */
+void
+ProcessWALProhibitStateChangeRequest(void)
+{
+	uint32		wal_state_gen;
+
+	if (!AmCheckpointerProcess())
+		return;
+
+	while (1)
+	{
+		wal_state_gen = GetWALProhibitStateGen();
+
+		if (wal_state_gen & WALPROHIBIT_TRANSITION_IN_PROGRESS)
+		{
+			CompleteWALProhibitChange(wal_state_gen);
+			continue; /* check changed state */
+		}
+		else if (WALPROHIBIT_CURRENT_STATE(wal_state_gen) ==
+				 WALPROHIBIT_STATE_READ_ONLY)
+		{
+			int			rc;
+
+			/*
+			 * Don't let Checkpointer process do anything until someone wakes it
+			 * up.  For example a backend might later on request us to put the
+			 * system back to read-write wal prohibit sate.
+			 */
+			rc = WaitLatch(MyLatch, WL_LATCH_SET | WL_POSTMASTER_DEATH, -1,
+						   WAIT_EVENT_CHECKPOINTER_MAIN);
+
+			/*
+			 * If the postmaster dies or a shutdown request is received, just
+			 * bail out.
+			 */
+			if (rc & WL_POSTMASTER_DEATH || ShutdownRequestPending)
+				return;
+
+			/* Re-check wal prohibit state */
+			continue;
+		}
+
+		Assert(WALPROHIBIT_CURRENT_STATE(wal_state_gen) ==
+			   WALPROHIBIT_STATE_READ_WRITE);
+
+		break; /* Done */
+	}
+}
+
+/*
+ * GetWALProhibitStateGen()
+ *
+ * Atomically return the current server WAL prohibited state generation.
+ */
+static uint32
+GetWALProhibitStateGen(void)
+{
+	return pg_atomic_read_u32(&WALProhibitState->shared_state_generation);
+}
+
+/*
+ * SetWALProhibitState()
+ *
+ * Increments current shared WAL prohibit state generation concerning to
+ * requested state and returns the same.
+ *
+ * For the transition state request where is_final_state is false if the server
+ * desired transition state is the same as the current state which might have
+ * been requested by some other backend and has been proceeded then the current
+ * wal prohibit generation will be returned so that this backend can wait until
+ * the shared wal prohibited generation change for the final state.  And, if the
+ * server is already completely moved to the requested state then the requester
+ * backend doesn't need to wait, in that case, 0 will be returned.
+ *
+ * The final state can only be requested by the checkpointer or by the
+ * single-user so that there will be no chance that the server is already in the
+ * desired final state.
+ */
+static uint32
+SetWALProhibitState(bool wal_prohibited, bool is_final_state)
+{
+	uint32		new_state;
+	uint32		cur_state;
+	uint32		cur_state_gen;
+	uint32		next_state_gen;
+
+	/* Get the current state */
+	cur_state_gen = GetWALProhibitStateGen();
+	cur_state = WALPROHIBIT_CURRENT_STATE(cur_state_gen);
+
+	/* Compute new state */
+	if (is_final_state)
+	{
+		/*
+		 * Only checkpointer or single-user can set the final wal prohibit
+		 * state.
+		 */
+		Assert(AmCheckpointerProcess() || !IsPostmasterEnvironment);
+
+		new_state = wal_prohibited ?
+			WALPROHIBIT_STATE_READ_ONLY : WALPROHIBIT_STATE_READ_WRITE;
+
+		/*
+		 * There won't be any other process for the final state setting so that
+		 * the next final state will be the desired state.
+		 */
+		Assert(WALPROHIBIT_NEXT_STATE(cur_state) == new_state);
+	}
+	else
+	{
+		new_state = wal_prohibited ?
+			WALPROHIBIT_STATE_GOING_READ_ONLY :
+			WALPROHIBIT_STATE_GOING_READ_WRITE;
+
+		/* Server is already in the requested transition state */
+		if (cur_state == new_state)
+			return cur_state;		/* Wait for state transition completion */
+
+		/* Server is already in requested state */
+		if (WALPROHIBIT_NEXT_STATE(new_state) == cur_state)
+			return 0;		/* No wait is needed */
+
+		/* Prevent concurrent contrary in progress transition state setting */
+		if (cur_state & WALPROHIBIT_TRANSITION_IN_PROGRESS)
+		{
+			if (cur_state & WALPROHIBIT_STATE_READ_ONLY)
+				ereport(ERROR,
+						(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+						 errmsg("system state transition to read only is already in progress"),
+						 errhint("Try after sometime again.")));
+			else
+				ereport(ERROR,
+						(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+						 errmsg("system state transition to read write is already in progress"),
+						 errhint("Try after sometime again.")));
+		}
+	}
+
+	/*
+	 * Update new state generation in share memory only if the state generation
+	 * hasn't changed until now we have checked.
+	 */
+	next_state_gen = cur_state_gen + 1;
+	(void) pg_atomic_compare_exchange_u32(&WALProhibitState->shared_state_generation,
+										  &cur_state_gen, next_state_gen);
+
+	/* To be sure that any later reads of memory happen strictly after this. */
+	pg_memory_barrier();
+
+	return next_state_gen;
+}
+
+/*
+ * WALProhibitStateGenerationInit()
+ *
+ * Initialization of shared wal prohibit state generation.
+ */
+void
+WALProhibitStateGenerationInit(bool wal_prohibited)
+{
+	uint32	new_state;
+
+	Assert(AmStartupProcess() || !IsPostmasterEnvironment);
+
+	new_state = wal_prohibited ?
+		WALPROHIBIT_STATE_READ_ONLY : WALPROHIBIT_STATE_READ_WRITE;
+
+	pg_atomic_init_u32(&WALProhibitState->shared_state_generation, new_state);
+}
+
+/*
+ * WALProhibitStateShmemInit()
+ *
+ * Initialization of shared memory for WAL prohibit state.
+ */
+void
+WALProhibitStateShmemInit(void)
+{
+	bool		found;
+
+	WALProhibitState = (WALProhibitStateData *)
+		ShmemInitStruct("WAL Prohibit State",
+						sizeof(WALProhibitStateData),
+						&found);
+
+	if (found)
+		return;
+
+	/* First time through ... */
+	memset(WALProhibitState, 0, sizeof(WALProhibitStateData));
+	ConditionVariableInit(&WALProhibitState->walprohibit_cv);
+}
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index a2068e3fd45..0de63af6365 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -1962,23 +1962,28 @@ StartTransaction(void)
 	Assert(s->prevSecContext == 0);
 
 	/*
-	 * Make sure we've reset xact state variables
+	 * Reset xact state variables.
 	 *
-	 * If recovery is still in progress, mark this transaction as read-only.
-	 * We have lower level defences in XLogInsert and elsewhere to stop us
-	 * from modifying data during recovery, but this gives the normal
-	 * indication to the user that the transaction is read-only.
-	 */
-	if (RecoveryInProgress())
-	{
-		s->startedInRecovery = true;
-		XactReadOnly = true;
-	}
-	else
-	{
-		s->startedInRecovery = false;
-		XactReadOnly = DefaultXactReadOnly;
-	}
+	 * If it is not currently possible to insert write-ahead log records,
+	 * either because we are still in recovery or because ALTER SYSTEM READ
+	 * ONLY has been executed, force this to be a read-only transaction.
+	 * We have lower level defences in XLogBeginInsert() and elsewhere to stop
+	 * us from modifying data during recovery when !XLogInsertAllowed(), but
+	 * this gives the normal indication to the user that the transaction is
+	 * read-only.
+	 *
+	 * On the other hand, we only need to set the startedInRecovery flag when
+	 * the transaction started during recovery, and not when WAL is otherwise
+	 * prohibited. This information is used by RelationGetIndexScan() to
+	 * decide whether to permit (1) relying on existing killed-tuple markings
+	 * and (2) further killing of index tuples. Even when WAL is prohibited
+	 * on the master, it's still the master, so the former is OK; and since
+	 * killing index tuples doesn't generate WAL, the latter is also OK.
+	 * See comments in RelationGetIndexScan() and MarkBufferDirtyHint().
+	 */
+	XactReadOnly = DefaultXactReadOnly || !XLogInsertAllowed();
+	s->startedInRecovery = RecoveryInProgress();
+
 	XactDeferrable = DefaultXactDeferrable;
 	XactIsoLevel = DefaultXactIsoLevel;
 	forceSyncCommit = false;
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index b18257c1980..e31327ed5c7 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -31,6 +31,7 @@
 #include "access/timeline.h"
 #include "access/transam.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog_internal.h"
 #include "access/xlogarchive.h"
@@ -247,9 +248,10 @@ static bool LocalPromoteIsTriggered = false;
  *		0: unconditionally not allowed to insert XLOG
  *		-1: must check RecoveryInProgress(); disallow until it is false
  * Most processes start with -1 and transition to 1 after seeing that recovery
- * is not in progress.  But we can also force the value for special cases.
- * The coding in XLogInsertAllowed() depends on the first two of these states
- * being numerically the same as bool true and false.
+ * is not in progress or the server state is not a WAL prohibited state.  But
+ * we can also force the value for special cases.  The coding in
+ * XLogInsertAllowed() depends on the first two of these states being
+ * numerically the same as bool true and false.
  */
 static int	LocalXLogInsertAllowed = -1;
 
@@ -734,6 +736,11 @@ typedef struct XLogCtlData
 	 */
 	XLogRecPtr	lastFpwDisableRecPtr;
 
+	/*
+	 * lastCheckPointSkipped indicates if the last checkpoint is skipped.
+	 */
+	bool		lastCheckPointSkipped;
+
 	slock_t		info_lck;		/* locks shared variables shown above */
 } XLogCtlData;
 
@@ -6219,6 +6226,32 @@ SetCurrentChunkStartTime(TimestampTz xtime)
 	SpinLockRelease(&XLogCtl->info_lck);
 }
 
+/*
+ * Set or unset flag to indicating that the last checkpoint has been skipped.
+ */
+void
+SetLastCheckPointSkipped(bool ChkptSkip)
+{
+	SpinLockAcquire(&XLogCtl->info_lck);
+	XLogCtl->lastCheckPointSkipped = ChkptSkip;
+	SpinLockRelease(&XLogCtl->info_lck);
+}
+
+/*
+ * Return value of lastCheckPointSkipped flag.
+ */
+bool
+LastCheckPointIsSkipped(void)
+{
+	bool	ChkptSkipped;
+
+	SpinLockAcquire(&XLogCtl->info_lck);
+	ChkptSkipped = XLogCtl->lastCheckPointSkipped;
+	SpinLockRelease(&XLogCtl->info_lck);
+
+	return ChkptSkipped;
+}
+
 /*
  * Fetch timestamp of latest processed commit/abort record.
  * Startup process maintains an accurate local copy in XLogReceiptTime
@@ -7732,6 +7765,12 @@ StartupXLOG(void)
 	XLogCtl->LogwrtRqst.Write = EndOfLog;
 	XLogCtl->LogwrtRqst.Flush = EndOfLog;
 
+	/*
+	 * Before enabling WAL insertion, initialize WAL prohibit state in shared
+	 * memory that will decide the further WAL insert should be allowed or not.
+	 */
+	WALProhibitStateGenerationInit(ControlFile->wal_prohibited);
+
 	/*
 	 * Update full_page_writes in shared memory and write an XLOG_FPW_CHANGE
 	 * record before resource manager writes cleanup WAL records or checkpoint
@@ -7742,7 +7781,17 @@ StartupXLOG(void)
 	UpdateFullPageWrites();
 	LocalXLogInsertAllowed = -1;
 
-	if (InRecovery)
+	/*
+	 * Skip end-of-recovery checkpoint if the system is in WAL prohibited state.
+	 */
+	if (ControlFile->wal_prohibited && InRecovery)
+	{
+		SetLastCheckPointSkipped(true);
+
+		ereport(LOG,
+				(errmsg("skipping startup checkpoint because the system is read only")));
+	}
+	else if (InRecovery)
 	{
 		/*
 		 * Perform a checkpoint to update all our recovery activity to disk.
@@ -7988,6 +8037,16 @@ StartupXLOG(void)
 		RequestCheckpoint(CHECKPOINT_FORCE);
 }
 
+/* Set ControlFile's WAL prohibit flag */
+void
+SetControlFileWALProhibitFlag(bool wal_prohibited)
+{
+	LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+	ControlFile->wal_prohibited = wal_prohibited;
+	UpdateControlFile();
+	LWLockRelease(ControlFileLock);
+}
+
 /*
  * Checks if recovery has reached a consistent state. When consistency is
  * reached and we have a valid starting standby snapshot, tell postmaster
@@ -8203,9 +8262,9 @@ HotStandbyActiveInReplay(void)
 /*
  * Is this process allowed to insert new WAL records?
  *
- * Ordinarily this is essentially equivalent to !RecoveryInProgress().
- * But we also have provisions for forcing the result "true" or "false"
- * within specific processes regardless of the global state.
+ * Ordinarily this is essentially equivalent to !RecoveryInProgress() and
+ * !IsWALProhibited().  But we also have provisions for forcing the result
+ * "true" or "false" within specific processes regardless of the global state.
  */
 bool
 XLogInsertAllowed(void)
@@ -8224,9 +8283,20 @@ XLogInsertAllowed(void)
 	if (RecoveryInProgress())
 		return false;
 
+	/* Or, in WAL prohibited state */
+	if (IsWALProhibited())
+	{
+		/*
+		 * Set it to "unconditionally false" to avoid checking until it gets
+		 * reset.
+		 */
+		LocalXLogInsertAllowed = 0;
+		return false;
+	}
+
 	/*
-	 * On exit from recovery, reset to "unconditionally true", since there is
-	 * no need to keep checking.
+	 * On exit from recovery or WAL prohibited state, reset to "unconditionally
+	 * true", since there is no need to keep checking.
 	 */
 	LocalXLogInsertAllowed = 1;
 	return true;
@@ -8248,6 +8318,12 @@ LocalSetXLogInsertAllowed(void)
 	InitXLOGAccess();
 }
 
+void
+ResetLocalXLogInsertAllowed(void)
+{
+	LocalXLogInsertAllowed = -1;
+}
+
 /*
  * Subroutine to try to fetch and validate a prior checkpoint record.
  *
@@ -8537,9 +8613,13 @@ ShutdownXLOG(int code, Datum arg)
 	 */
 	WalSndWaitStopping();
 
+	/*
+	 * The restartpoint, checkpoint, or xlog rotation will be performed if the
+	 * WAL writing is permitted.
+	 */
 	if (RecoveryInProgress())
 		CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
-	else
+	else if (XLogInsertAllowed())
 	{
 		/*
 		 * If archiving is enabled, rotate the last XLOG file so that all the
@@ -8552,6 +8632,9 @@ ShutdownXLOG(int code, Datum arg)
 
 		CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
 	}
+	else
+		ereport(LOG,
+			   (errmsg("skipping shutdown checkpoint because the system is read only")));
 }
 
 /*
@@ -8820,7 +8903,11 @@ CreateCheckPoint(int flags)
 	 * only one process that is allowed to issue checkpoints at any given
 	 * time.)
 	 */
-	LWLockAcquire(CheckpointLock, LW_EXCLUSIVE);
+	//======================================================================================
+	// TODO: Tbc, the discussion on this lock requirement is in progress at
+	// http://postgr.es/m/CAAJ_b97XnBBfYeSREDJorFsyoD1sHgqnNuCi=02mNQBUMnA=FA@mail.gmail.com
+	//======================================================================================
+	//LWLockAcquire(CheckpointLock, LW_EXCLUSIVE);
 
 	/*
 	 * Prepare to accumulate statistics.
@@ -8891,7 +8978,7 @@ CreateCheckPoint(int flags)
 		if (last_important_lsn == ControlFile->checkPoint)
 		{
 			WALInsertLockRelease();
-			LWLockRelease(CheckpointLock);
+			//LWLockRelease(CheckpointLock);			TODO: Tbc
 			END_CRIT_SECTION();
 			ereport(DEBUG1,
 					(errmsg("checkpoint skipped because system is idle")));
@@ -9192,7 +9279,7 @@ CreateCheckPoint(int flags)
 									 CheckpointStats.ckpt_segs_removed,
 									 CheckpointStats.ckpt_segs_recycled);
 
-	LWLockRelease(CheckpointLock);
+	//LWLockRelease(CheckpointLock);	TODO: Tbc
 }
 
 /*
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 5d89e77dbe2..f8ae90c03c7 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1518,6 +1518,8 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
 REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
 REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
 
+REVOKE EXECUTE ON FUNCTION pg_alter_wal_prohibit_state(bool) FROM public;
+
 --
 -- We also set up some things as accessible to standard roles.
 --
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 47e60ca5613..516d6cd032a 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -663,6 +663,10 @@ AutoVacLauncherMain(int argc, char *argv[])
 
 		HandleAutoVacLauncherInterrupts();
 
+		/* If the server is read only just go back to sleep. */
+		if (!XLogInsertAllowed())
+			continue;
+
 		/*
 		 * a worker finished, or postmaster signaled failure to start a worker
 		 */
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index 715d5195bb6..5157237731c 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -278,7 +278,7 @@ BackgroundWriterMain(void)
 		 * Checkpointer, when active, is barely ever in its mainloop and thus
 		 * makes it hard to log regularly.
 		 */
-		if (XLogStandbyInfoActive() && !RecoveryInProgress())
+		if (XLogStandbyInfoActive() && XLogInsertAllowed())
 		{
 			TimestampTz timeout = 0;
 			TimestampTz now = GetCurrentTimestamp();
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 54a818bf611..1324d494724 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -39,6 +39,7 @@
 #include <sys/time.h>
 #include <time.h>
 
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xlog_internal.h"
 #include "libpq/pqsignal.h"
@@ -351,6 +352,7 @@ CheckpointerMain(void)
 		 */
 		AbsorbSyncRequests();
 		HandleCheckpointerInterrupts();
+		ProcessWALProhibitStateChangeRequest();
 
 		/*
 		 * Detect a pending checkpoint request by checking whether the flags
@@ -688,6 +690,9 @@ CheckpointWriteDelay(int flags, double progress)
 	if (!AmCheckpointerProcess())
 		return;
 
+	/* Check for wal prohibit state change request */
+	ProcessWALProhibitStateChangeRequest();
+
 	/*
 	 * Perform the usual duties and take a nap, unless we're behind schedule,
 	 * in which case we just try to catch up as quickly as possible.
@@ -1335,3 +1340,16 @@ FirstCallSinceLastCheckpoint(void)
 
 	return FirstCall;
 }
+
+/*
+ * SendsSignalToCheckpointer allows a process to send a signal to the checkpoint process.
+ */
+void
+SendsSignalToCheckpointer(int signum)
+{
+	if (CheckpointerShmem->checkpointer_pid == 0)
+		elog(ERROR, "checkpointer is not running");
+
+	if (kill(CheckpointerShmem->checkpointer_pid, signum) != 0)
+		elog(ERROR, "could not signal checkpointer: %m");
+}
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 3f24a33ef1d..76a440836a2 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -4227,6 +4227,9 @@ pgstat_get_wait_io(WaitEventIO w)
 		case WAIT_EVENT_REPLICATION_SLOT_WRITE:
 			event_name = "ReplicationSlotWrite";
 			break;
+		case WAIT_EVENT_WALPROHIBIT_STATE_CHANGE:
+			event_name = "SystemWALProhibitStateChange";
+			break;
 		case WAIT_EVENT_SLRU_FLUSH_SYNC:
 			event_name = "SLRUFlushSync";
 			break;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index f9bbe97b507..c3c5ec641cf 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -22,6 +22,7 @@
 #include "access/subtrans.h"
 #include "access/syncscan.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "commands/async.h"
 #include "miscadmin.h"
 #include "pgstat.h"
@@ -222,6 +223,11 @@ CreateSharedMemoryAndSemaphores(void)
 	MultiXactShmemInit();
 	InitBufferPool();
 
+	/*
+	 * Set up wal probibit shared state
+	 */
+	WALProhibitStateShmemInit();
+
 	/*
 	 * Set up lock manager
 	 */
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index c43cdd685b4..31383a11d08 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -18,6 +18,7 @@
 #include <unistd.h>
 
 #include "access/parallel.h"
+#include "access/walprohibit.h"
 #include "port/pg_bitutils.h"
 #include "commands/async.h"
 #include "miscadmin.h"
@@ -98,7 +99,6 @@ static volatile ProcSignalSlot *MyProcSignalSlot = NULL;
 static bool CheckProcSignal(ProcSignalReason reason);
 static void CleanupProcSignalState(int status, Datum arg);
 static void ResetProcSignalBarrierBits(uint32 flags);
-static bool ProcessBarrierPlaceholder(void);
 
 /*
  * ProcSignalShmemSize
@@ -538,8 +538,8 @@ ProcessProcSignalBarrier(void)
 				type = (ProcSignalBarrierType) pg_rightmost_one_pos32(flags);
 				switch (type)
 				{
-					case PROCSIGNAL_BARRIER_PLACEHOLDER:
-						processed = ProcessBarrierPlaceholder();
+					case PROCSIGNAL_BARRIER_WALPROHIBIT:
+						processed = ProcessBarrierWALProhibit();
 						break;
 				}
 
@@ -604,24 +604,6 @@ ResetProcSignalBarrierBits(uint32 flags)
 	InterruptPending = true;
 }
 
-static bool
-ProcessBarrierPlaceholder(void)
-{
-	/*
-	 * XXX. This is just a placeholder until the first real user of this
-	 * machinery gets committed. Rename PROCSIGNAL_BARRIER_PLACEHOLDER to
-	 * PROCSIGNAL_BARRIER_SOMETHING_ELSE where SOMETHING_ELSE is something
-	 * appropriately descriptive. Get rid of this function and instead have
-	 * ProcessBarrierSomethingElse. Most likely, that function should live in
-	 * the file pertaining to that subsystem, rather than here.
-	 *
-	 * The return value should be 'true' if the barrier was successfully
-	 * absorbed and 'false' if not. Note that returning 'false' can lead to
-	 * very frequent retries, so try hard to make that an uncommon case.
-	 */
-	return true;
-}
-
 /*
  * CheckProcSignal - check to see if a particular reason has been
  * signaled, and clear the signal flag.  Should be called after receiving
diff --git a/src/backend/storage/sync/sync.c b/src/backend/storage/sync/sync.c
index fe143151cc5..1c7b40563b5 100644
--- a/src/backend/storage/sync/sync.c
+++ b/src/backend/storage/sync/sync.c
@@ -21,6 +21,7 @@
 #include "access/commit_ts.h"
 #include "access/clog.h"
 #include "access/multixact.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xlogutils.h"
 #include "commands/tablespace.h"
@@ -236,10 +237,17 @@ SyncPostCheckpoint(void)
 		pfree(entry);
 
 		/*
-		 * As in ProcessSyncRequests, we don't want to stop absorbing fsync
+		 * As in ProcessSyncRequests, we don't want to stop wal prohibit change
 		 * requests for a long time when there are many deletions to be done.
-		 * We can safely call AbsorbSyncRequests() at this point in the loop
-		 * (note it might try to delete list entries).
+		 * It needs to be check and processed by checkpointer as soon as
+		 * possible.
+		 */
+		ProcessWALProhibitStateChangeRequest();
+
+		/*
+		 * Similarly, we don't want to stop absorbing fsync requests for the
+		 * long time.  We can safely call AbsorbSyncRequests() at this point in
+		 * the loop (note it might try to delete list entries).
 		 */
 		if (--absorb_counter <= 0)
 		{
@@ -278,6 +286,9 @@ ProcessSyncRequests(void)
 	if (!pendingOps)
 		elog(ERROR, "cannot sync without a pendingOps table");
 
+	/* Check for wal prohibit state change request for checkpointer */
+	ProcessWALProhibitStateChangeRequest();
+
 	/*
 	 * If we are in the checkpointer, the sync had better include all fsync
 	 * requests that were queued by backends up to this point.  The tightest
@@ -336,6 +347,13 @@ ProcessSyncRequests(void)
 	{
 		int			failures;
 
+		/*
+		 * Don't want to stop wal prohibit change requests for a long time when
+		 * there are many fsync requests to be processed.  It needs to be check
+		 * and processed by checkpointer as soon as possible.
+		 */
+		ProcessWALProhibitStateChangeRequest();
+
 		/*
 		 * If the entry is new then don't process it this time; it is new.
 		 * Note "continue" bypasses the hash-remove call at the bottom of the
@@ -422,6 +440,12 @@ ProcessSyncRequests(void)
 							 errmsg("could not fsync file \"%s\" but retrying: %m",
 									path)));
 
+				/*
+				 * For the same reason mentioned previously for the wal prohibit
+				 * state change request check.
+				 */
+				ProcessWALProhibitStateChangeRequest();
+
 				/*
 				 * Absorb incoming requests and check to see if a cancel
 				 * arrived for this relation fork.
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index e9cc4a22324..d295a1dc15f 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -19,6 +19,7 @@
 #include "access/htup_details.h"
 #include "access/reloptions.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "catalog/catalog.h"
@@ -86,7 +87,6 @@ static void ProcessUtilitySlow(ParseState *pstate,
 							   DestReceiver *dest,
 							   QueryCompletion *qc);
 static void ExecDropStmt(DropStmt *stmt, bool isTopLevel);
-static void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
 
 /*
  * CommandIsReadOnly: is an executable query read-only?
@@ -3686,16 +3686,3 @@ GetCommandLogLevel(Node *parsetree)
 
 	return lev;
 }
-
-/*
- * AlterSystemSetWALProhibitState
- *
- * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
- */
-static void
-AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
-{
-	ereport(ERROR,
-			(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
-			 errmsg("ALTER SYSTEM READ { ONLY | WRITE } not implemented")));
-}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 17579eeaca9..6d9c23f2d9f 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -227,6 +227,7 @@ static bool check_recovery_target_lsn(char **newval, void **extra, GucSource sou
 static void assign_recovery_target_lsn(const char *newval, void *extra);
 static bool check_primary_slot_name(char **newval, void **extra, GucSource source);
 static bool check_default_with_oids(bool *newval, void **extra, GucSource source);
+static const char *show_system_is_read_only(void);
 
 /* Private functions in guc-file.l that need to be called from guc.c */
 static ConfigVariable *ProcessConfigFileInternal(GucContext context,
@@ -618,6 +619,7 @@ static char *recovery_target_string;
 static char *recovery_target_xid_string;
 static char *recovery_target_name_string;
 static char *recovery_target_lsn_string;
+static bool system_is_read_only;
 
 
 /* should be static, but commands/variable.c needs to get at this */
@@ -2048,6 +2050,18 @@ static struct config_bool ConfigureNamesBool[] =
 		NULL, NULL, NULL
 	},
 
+	{
+		/* Not for general use */
+		{"system_is_read_only", PGC_INTERNAL, WAL,
+			gettext_noop("Shows whether the system is read only."),
+			NULL,
+			GUC_NO_RESET_ALL | GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE
+		},
+		&system_is_read_only,
+		false,
+		NULL, NULL, show_system_is_read_only
+	},
+
 	/* End-of-list marker */
 	{
 		{NULL, 0, 0, NULL, NULL}, NULL, false, NULL, NULL, NULL
@@ -12218,4 +12232,16 @@ check_default_with_oids(bool *newval, void **extra, GucSource source)
 	return true;
 }
 
+/*
+ * NB: The return string should be the same as the _ShowOption() for boolean
+ * type.
+ */
+static const char *
+show_system_is_read_only(void)
+{
+	if (!XLogInsertAllowed())
+		return "on";
+	return "off";
+}
+
 #include "guc-file.c"
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index 3e00ac0f701..922cd9641d8 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -290,6 +290,8 @@ main(int argc, char *argv[])
 		   (uint32) ControlFile->backupEndPoint);
 	printf(_("End-of-backup record required:        %s\n"),
 		   ControlFile->backupEndRequired ? _("yes") : _("no"));
+	printf(_("WAL write prohibited:                 %s\n"),
+		   ControlFile->wal_prohibited ? _("yes") : _("no"));
 	printf(_("wal_level setting:                    %s\n"),
 		   wal_level_str(ControlFile->wal_level));
 	printf(_("wal_log_hints setting:                %s\n"),
diff --git a/src/include/access/walprohibit.h b/src/include/access/walprohibit.h
new file mode 100644
index 00000000000..800542de123
--- /dev/null
+++ b/src/include/access/walprohibit.h
@@ -0,0 +1,93 @@
+/*
+ * walprohibit.h
+ *
+ * PostgreSQL write-ahead log prohibit states
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/include/access/walprohibit.h
+ */
+#ifndef WALPROHIBIT_H
+#define WALPROHIBIT_H
+
+#include "access/xact.h"
+#include "access/xlog.h"
+#include "miscadmin.h"
+#include "nodes/parsenodes.h"
+
+extern bool ProcessBarrierWALProhibit(void);
+extern void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
+extern void MarkCheckPointSkippedInWalProhibitState(void);
+extern void WALProhibitStateGenerationInit(bool wal_prohibited);
+extern void WALProhibitStateShmemInit(void);
+extern bool IsWALProhibited(void);
+extern void ProcessWALProhibitStateChangeRequest(void);
+
+/*
+ * The WAL Prohibit States.
+ *
+ * 	The odd number represents the transition state and whereas the even number
+ * 	represents the final state.  These states can be distinguished by checking
+ * 	the 0th bits aka transition bit.
+ */
+#define	WALPROHIBIT_STATE_READ_WRITE		(uint32) 0	/* WAL permitted */
+#define	WALPROHIBIT_STATE_GOING_READ_ONLY	(uint32) 1
+#define	WALPROHIBIT_STATE_READ_ONLY			(uint32) 2	/* WAL prohibited */
+#define	WALPROHIBIT_STATE_GOING_READ_WRITE	(uint32) 3
+
+/* The transition bit to distinguish states.  */
+#define	WALPROHIBIT_TRANSITION_IN_PROGRESS	((uint32) 1 << 0)
+
+/* Extract last two bits */
+#define	WALPROHIBIT_CURRENT_STATE(stateGeneration)	\
+	((uint32)(stateGeneration) & ((uint32) ((1 << 2) - 1)))
+#define	WALPROHIBIT_NEXT_STATE(stateGeneration)	\
+	WALPROHIBIT_CURRENT_STATE((stateGeneration + 1))
+
+/* Never reaches when WAL is prohibited. */
+static inline void
+AssertWALPermitted(void)
+{
+	/*
+	 * Recovery in the startup process never is in wal prohibited state.
+	 */
+	Assert(InRecovery || XLogInsertAllowed());
+
+#ifdef USE_ASSERT_CHECKING
+	walpermit_checked_state = WALPERMIT_CHECKED;
+#endif
+}
+
+/*
+ * XID-bearing transactions are killed off by "ALTER SYSTEM READ ONLY", so any
+ * part of the code that can only be reached with an XID assigned is never
+ * reached when WAL is prohibited.
+ */
+static inline void
+AssertWALPermittedHaveXID(void)
+{
+	/* Must be performing an INSERT, UPDATE or DELETE, so we'll have an XID */
+	Assert(FullTransactionIdIsValid(GetTopFullTransactionIdIfAny()));
+	AssertWALPermitted();
+}
+
+/*
+ * In opposite to the above assertion if a transaction doesn't have valid XID
+ * (e.g. VACUUM) then it won't be killed while changing the system state to WAL
+ * prohibited.  Therefore, we need to explicitly error out before entering into
+ * the critical section.
+ */
+static inline void
+CheckWALPermitted(void)
+{
+	if (!XLogInsertAllowed())
+		ereport(ERROR,
+				(errcode(ERRCODE_READ_ONLY_SQL_TRANSACTION),
+				 errmsg("system is now read only")));
+
+#ifdef USE_ASSERT_CHECKING
+	walpermit_checked_state = WALPERMIT_CHECKED;
+#endif
+}
+
+#endif							/* WALPROHIBIT_H */
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 75ec1073bd0..d348155b1ac 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -306,6 +306,7 @@ extern RecoveryState GetRecoveryState(void);
 extern bool HotStandbyActive(void);
 extern bool HotStandbyActiveInReplay(void);
 extern bool XLogInsertAllowed(void);
+extern void ResetLocalXLogInsertAllowed(void);
 extern void GetXLogReceiptTime(TimestampTz *rtime, bool *fromStream);
 extern XLogRecPtr GetXLogReplayRecPtr(TimeLineID *replayTLI);
 extern XLogRecPtr GetXLogInsertRecPtr(void);
@@ -314,6 +315,8 @@ extern bool RecoveryIsPaused(void);
 extern void SetRecoveryPause(bool recoveryPause);
 extern TimestampTz GetLatestXTime(void);
 extern TimestampTz GetCurrentChunkReplayStartTime(void);
+extern void SetLastCheckPointSkipped(bool ChkptSkip);
+extern bool LastCheckPointIsSkipped(void);
 
 extern void UpdateControlFile(void);
 extern uint64 GetSystemIdentifier(void);
@@ -325,6 +328,7 @@ extern void XLOGShmemInit(void);
 extern void BootStrapXLOG(void);
 extern void LocalProcessControlFile(bool reset);
 extern void StartupXLOG(void);
+extern void SetControlFileWALProhibitFlag(bool wal_prohibited);
 extern void ShutdownXLOG(int code, Datum arg);
 extern void InitXLOGAccess(void);
 extern void CreateCheckPoint(int flags);
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index e3f48158ce7..f6a1f3b9826 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -182,6 +182,9 @@ typedef struct ControlFileData
 	int			max_locks_per_xact;
 	bool		track_commit_timestamp;
 
+	/* WAL prohibited determines if the WAL insert is allowed or not. */
+	bool		wal_prohibited;
+
 	/*
 	 * This data is used to check for hardware-architecture compatibility of
 	 * the database and the backend executable.  We need not check endianness
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index d27336adcd9..8a54e35b8ff 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11329,6 +11329,10 @@
   proname => 'pg_partition_root', prorettype => 'regclass',
   proargtypes => 'regclass', prosrc => 'pg_partition_root' },
 
+{ oid => '4543', descr => 'alter system read only state',
+  proname => 'pg_alter_wal_prohibit_state', prorettype => 'void',
+  proargtypes => 'bool', prosrc => 'pg_alter_wal_prohibit_state' },
+
 { oid => '4350', descr => 'Unicode normalization',
   proname => 'normalize', prorettype => 'text', proargtypes => 'text text',
   prosrc => 'unicode_normalize_func' },
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index c38b6897101..491cf78a905 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -1033,6 +1033,7 @@ typedef enum
 	WAIT_EVENT_REPLICATION_SLOT_RESTORE_SYNC,
 	WAIT_EVENT_REPLICATION_SLOT_SYNC,
 	WAIT_EVENT_REPLICATION_SLOT_WRITE,
+	WAIT_EVENT_WALPROHIBIT_STATE_CHANGE,
 	WAIT_EVENT_SLRU_FLUSH_SYNC,
 	WAIT_EVENT_SLRU_READ,
 	WAIT_EVENT_SLRU_SYNC,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index c430b1b2366..b29884ba0d6 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -42,4 +42,6 @@ extern void CheckpointerShmemInit(void);
 
 extern bool FirstCallSinceLastCheckpoint(void);
 
+extern void SendsSignalToCheckpointer(int signum);
+
 #endif							/* _BGWRITER_H */
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 4ae7dc33b8e..9e834247871 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -48,12 +48,7 @@ typedef enum
 
 typedef enum
 {
-	/*
-	 * XXX. PROCSIGNAL_BARRIER_PLACEHOLDER should be replaced when the first
-	 * real user of the ProcSignalBarrier mechanism is added. It's just here
-	 * for now because we can't have an empty enum.
-	 */
-	PROCSIGNAL_BARRIER_PLACEHOLDER = 0
+	PROCSIGNAL_BARRIER_WALPROHIBIT = 0
 } ProcSignalBarrierType;
 
 /*
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 4ba4ec1cbdc..8484993d1c1 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2696,6 +2696,7 @@ WALAvailability
 WALInsertLock
 WALInsertLockPadded
 WALOpenSegment
+WALProhibitStateData
 WALReadError
 WALSegmentCloseCB
 WALSegmentContext
-- 
2.18.0

From e6bce84a1be1cf4b18a2e395a19425b7d956bbf9 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Mon, 27 Jul 2020 02:13:36 -0400
Subject: [PATCH v12 5/6] Error or Assert before START_CRIT_SECTION for WAL
 write

Based on the following criteria added the Assert or the Error when system is
prohibited:

 - Added ERROR for the function which can be reachable without valid XID in case
   of VACUUM or CONCURRENT CREATE INDEX.  For that added common static inline
   function CheckWALPermitted().
 - Added ASSERT for the function which cannot be reached without valid XID;
   Assert to ensure XID validation. For that added AssertWALPermitted_HaveXID().

To enforce the rule to have aforesaid assert or error check before entering a
critical section for WAL write, a new walpermit_checked_state assert only flag
is added. If these check is missing then XLogBeginInsert() will have an
assertion if it is in critical section.

If we are not doing WAL insert inside the critical section then the above
checking is not necessary, we can rely on XLogBeginInsert() for that check &
report an error.
---
 contrib/pg_surgery/heap_surgery.c         | 10 +++++--
 src/backend/access/brin/brin.c            |  4 +++
 src/backend/access/brin/brin_pageops.c    | 21 ++++++++++++--
 src/backend/access/brin/brin_revmap.c     | 10 ++++++-
 src/backend/access/gin/ginbtree.c         | 15 ++++++++--
 src/backend/access/gin/gindatapage.c      | 18 ++++++++++--
 src/backend/access/gin/ginfast.c          | 11 ++++++--
 src/backend/access/gin/gininsert.c        |  4 +++
 src/backend/access/gin/ginutil.c          |  9 +++++-
 src/backend/access/gin/ginvacuum.c        | 11 +++++++-
 src/backend/access/gist/gist.c            | 25 +++++++++++++----
 src/backend/access/gist/gistvacuum.c      | 13 +++++++--
 src/backend/access/hash/hash.c            | 19 +++++++++++--
 src/backend/access/hash/hashinsert.c      |  9 +++++-
 src/backend/access/hash/hashovfl.c        | 22 ++++++++++++---
 src/backend/access/hash/hashpage.c        |  9 ++++++
 src/backend/access/heap/heapam.c          | 26 ++++++++++++++++-
 src/backend/access/heap/pruneheap.c       | 12 ++++++--
 src/backend/access/heap/vacuumlazy.c      | 18 ++++++++++--
 src/backend/access/heap/visibilitymap.c   | 22 +++++++++++++--
 src/backend/access/nbtree/nbtdedup.c      |  3 ++
 src/backend/access/nbtree/nbtinsert.c     | 10 ++++++-
 src/backend/access/nbtree/nbtpage.c       | 34 ++++++++++++++++++++---
 src/backend/access/spgist/spgdoinsert.c   | 13 +++++++++
 src/backend/access/spgist/spgvacuum.c     | 22 +++++++++++++--
 src/backend/access/transam/multixact.c    |  5 +++-
 src/backend/access/transam/twophase.c     |  9 ++++++
 src/backend/access/transam/varsup.c       |  4 +++
 src/backend/access/transam/walprohibit.c  | 10 +++++++
 src/backend/access/transam/xact.c         |  6 ++++
 src/backend/access/transam/xlog.c         | 26 ++++++++++++-----
 src/backend/access/transam/xloginsert.c   | 13 +++++++--
 src/backend/commands/sequence.c           | 16 +++++++++++
 src/backend/commands/variable.c           |  9 ++++--
 src/backend/postmaster/checkpointer.c     |  4 +++
 src/backend/storage/buffer/bufmgr.c       | 10 ++++---
 src/backend/storage/freespace/freespace.c | 10 ++++++-
 src/backend/storage/lmgr/lock.c           |  6 ++--
 src/backend/utils/cache/relmapper.c       |  3 ++
 src/include/miscadmin.h                   | 27 ++++++++++++++++++
 40 files changed, 459 insertions(+), 69 deletions(-)

diff --git a/contrib/pg_surgery/heap_surgery.c b/contrib/pg_surgery/heap_surgery.c
index d31e5f31fd4..2938b937f75 100644
--- a/contrib/pg_surgery/heap_surgery.c
+++ b/contrib/pg_surgery/heap_surgery.c
@@ -14,6 +14,7 @@
 
 #include "access/heapam.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "catalog/pg_am_d.h"
 #include "catalog/pg_proc_d.h"
 #include "miscadmin.h"
@@ -89,6 +90,7 @@ heap_force_common(FunctionCallInfo fcinfo, HeapTupleForceOption heap_force_opt)
 	OffsetNumber curr_start_ptr,
 				next_start_ptr;
 	bool		include_this_tid[MaxHeapTuplesPerPage];
+	bool		needwal;
 
 	if (RecoveryInProgress())
 		ereport(ERROR,
@@ -100,6 +102,7 @@ heap_force_common(FunctionCallInfo fcinfo, HeapTupleForceOption heap_force_opt)
 	sanity_check_tid_array(ta, &ntids);
 
 	rel = relation_open(relid, RowExclusiveLock);
+	needwal = RelationNeedsWAL(rel);
 
 	/* Check target relation. */
 	sanity_check_relation(rel);
@@ -217,6 +220,9 @@ heap_force_common(FunctionCallInfo fcinfo, HeapTupleForceOption heap_force_opt)
 		if (heap_force_opt == HEAP_FORCE_KILL && PageIsAllVisible(page))
 			visibilitymap_pin(rel, blkno, &vmbuf);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* No ereport(ERROR) from here until all the changes are logged. */
 		START_CRIT_SECTION();
 
@@ -297,12 +303,12 @@ heap_force_common(FunctionCallInfo fcinfo, HeapTupleForceOption heap_force_opt)
 			MarkBufferDirty(buf);
 
 			/* XLOG stuff */
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 				log_newpage_buffer(buf, true);
 		}
 
 		/* WAL log the VM page if it was modified. */
-		if (did_modify_vm && RelationNeedsWAL(rel))
+		if (did_modify_vm && needwal)
 			log_newpage_buffer(vmbuf, false);
 
 		END_CRIT_SECTION();
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index 27ba596c6e4..17e719e2dfc 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -24,6 +24,7 @@
 #include "access/relscan.h"
 #include "access/table.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/index.h"
 #include "catalog/pg_am.h"
@@ -760,6 +761,9 @@ brinbuildempty(Relation index)
 		ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermittedHaveXID();
+
 	/* Initialize and xlog metabuffer. */
 	START_CRIT_SECTION();
 	brin_metapage_init(BufferGetPage(metabuf), BrinGetPagesPerRange(index),
diff --git a/src/backend/access/brin/brin_pageops.c b/src/backend/access/brin/brin_pageops.c
index df9ffc2fb86..270a881ab12 100644
--- a/src/backend/access/brin/brin_pageops.c
+++ b/src/backend/access/brin/brin_pageops.c
@@ -14,6 +14,7 @@
 #include "access/brin_pageops.h"
 #include "access/brin_revmap.h"
 #include "access/brin_xlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -65,6 +66,7 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 	Buffer		newbuf;
 	BlockNumber newblk = InvalidBlockNumber;
 	bool		extended;
+	bool		needwal = RelationNeedsWAL(idxrel);
 
 	Assert(newsz == MAXALIGN(newsz));
 
@@ -176,13 +178,16 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 	if (((BrinPageFlags(oldpage) & BRIN_EVACUATE_PAGE) == 0) &&
 		brin_can_do_samepage_update(oldbuf, origsz, newsz))
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 		if (!PageIndexTupleOverwrite(oldpage, oldoff, (Item) unconstify(BrinTuple *, newtup), newsz))
 			elog(ERROR, "failed to replace BRIN tuple");
 		MarkBufferDirty(oldbuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(idxrel))
+		if (needwal)
 		{
 			xl_brin_samepage_update xlrec;
 			XLogRecPtr	recptr;
@@ -240,6 +245,9 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 
 		revmapbuf = brinLockRevmapPageForUpdate(revmap, heapBlk);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		/*
@@ -267,7 +275,7 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 		MarkBufferDirty(revmapbuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(idxrel))
+		if (needwal)
 		{
 			xl_brin_update xlrec;
 			XLogRecPtr	recptr;
@@ -351,6 +359,7 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	Buffer		revmapbuf;
 	ItemPointerData tid;
 	bool		extended;
+	bool		needwal;
 
 	Assert(itemsz == MAXALIGN(itemsz));
 
@@ -405,6 +414,10 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	page = BufferGetPage(*buffer);
 	blk = BufferGetBlockNumber(*buffer);
 
+	needwal = RelationNeedsWAL(idxrel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Execute the actual insertion */
 	START_CRIT_SECTION();
 	if (extended)
@@ -424,7 +437,7 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	MarkBufferDirty(revmapbuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(idxrel))
+	if (needwal)
 	{
 		xl_brin_insert xlrec;
 		XLogRecPtr	recptr;
@@ -881,6 +894,8 @@ brin_initialize_empty_new_buffer(Relation idxrel, Buffer buffer)
 			   "brin_initialize_empty_new_buffer: initializing blank page %u",
 			   BufferGetBlockNumber(buffer)));
 
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 	page = BufferGetPage(buffer);
 	brin_page_init(page, BRIN_PAGETYPE_REGULAR);
diff --git a/src/backend/access/brin/brin_revmap.c b/src/backend/access/brin/brin_revmap.c
index bab2a88ee3f..70d846d5121 100644
--- a/src/backend/access/brin/brin_revmap.c
+++ b/src/backend/access/brin/brin_revmap.c
@@ -26,6 +26,7 @@
 #include "access/brin_tuple.h"
 #include "access/brin_xlog.h"
 #include "access/rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -340,6 +341,7 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 	OffsetNumber revmapOffset;
 	OffsetNumber regOffset;
 	ItemId		lp;
+	bool		needwal;
 
 	revmap = brinRevmapInitialize(idxrel, &pagesPerRange, NULL);
 
@@ -404,6 +406,10 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 	 * crashed or aborted summarization; remove them silently.
 	 */
 
+	needwal = RelationNeedsWAL(idxrel);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	ItemPointerSetInvalid(&invalidIptr);
@@ -415,7 +421,7 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 	MarkBufferDirty(regBuf);
 	MarkBufferDirty(revmapBuf);
 
-	if (RelationNeedsWAL(idxrel))
+	if (needwal)
 	{
 		xl_brin_desummarize xlrec;
 		XLogRecPtr	recptr;
@@ -613,6 +619,8 @@ revmap_physical_extend(BrinRevmap *revmap)
 		return;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Ok, we have now locked the metapage and the target block. Re-initialize
 	 * the target block as a revmap page, and update the metapage.
diff --git a/src/backend/access/gin/ginbtree.c b/src/backend/access/gin/ginbtree.c
index 482cf10877c..0d6dacfe84a 100644
--- a/src/backend/access/gin/ginbtree.c
+++ b/src/backend/access/gin/ginbtree.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/predicate.h"
@@ -332,6 +333,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 {
 	Page		page = BufferGetPage(stack->buffer);
 	bool		result;
+	bool		needwal;
 	GinPlaceToPageRC rc;
 	uint16		xlflags = 0;
 	Page		childpage = NULL;
@@ -377,6 +379,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 								 insertdata, updateblkno,
 								 &ptp_workspace,
 								 &newlpage, &newrpage);
+	needwal = RelationNeedsWAL(btree->index) && !btree->isBuild;
 
 	if (rc == GPTP_NO_WORK)
 	{
@@ -385,10 +388,13 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 	}
 	else if (rc == GPTP_INSERT)
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		/* It will fit, perform the insertion */
 		START_CRIT_SECTION();
 
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogBeginInsert();
 			XLogRegisterBuffer(0, stack->buffer, REGBUF_STANDARD);
@@ -409,7 +415,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 			MarkBufferDirty(childbuf);
 		}
 
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 			ginxlogInsert xlrec;
@@ -547,6 +553,9 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 			}
 		}
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/*
 		 * OK, we have the new contents of the left page in a temporary copy
 		 * now (newlpage), and likewise for the new contents of the
@@ -587,7 +596,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 		}
 
 		/* write WAL record */
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gin/gindatapage.c b/src/backend/access/gin/gindatapage.c
index 06c05865435..2b4812fb23a 100644
--- a/src/backend/access/gin/gindatapage.c
+++ b/src/backend/access/gin/gindatapage.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "lib/ilist.h"
 #include "miscadmin.h"
@@ -811,6 +812,7 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 	if (removedsomething)
 	{
 		bool		modified;
+		bool		needwal;
 
 		/*
 		 * Make sure we have a palloc'd copy of all segments, after the first
@@ -835,8 +837,12 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 			}
 		}
 
-		if (RelationNeedsWAL(indexrel))
+		needwal = RelationNeedsWAL(indexrel);
+		if (needwal)
+		{
+			CheckWALPermitted();
 			computeLeafRecompressWALData(leaf);
+		}
 
 		/* Apply changes to page */
 		START_CRIT_SECTION();
@@ -845,7 +851,7 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 
 		MarkBufferDirty(buffer);
 
-		if (RelationNeedsWAL(indexrel))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
@@ -1777,6 +1783,7 @@ createPostingTree(Relation index, ItemPointerData *items, uint32 nitems,
 	int			nrootitems;
 	int			rootsize;
 	bool		is_build = (buildStats != NULL);
+	bool		needwal;
 
 	/* Construct the new root page in memory first. */
 	tmppage = (Page) palloc(BLCKSZ);
@@ -1825,12 +1832,17 @@ createPostingTree(Relation index, ItemPointerData *items, uint32 nitems,
 	 */
 	PredicateLockPageSplit(index, BufferGetBlockNumber(entrybuffer), blkno);
 
+	needwal = RelationNeedsWAL(index) && !is_build;
+
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	PageRestoreTempPage(tmppage, page);
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index) && !is_build)
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogCreatePostingTree data;
diff --git a/src/backend/access/gin/ginfast.c b/src/backend/access/gin/ginfast.c
index e0d99409461..53c156018e7 100644
--- a/src/backend/access/gin/ginfast.c
+++ b/src/backend/access/gin/ginfast.c
@@ -20,6 +20,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
 #include "catalog/pg_am.h"
@@ -68,6 +69,8 @@ writeListPage(Relation index, Buffer buffer,
 	PGAlignedBlock workspace;
 	char	   *ptr;
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	GinInitBuffer(buffer, GIN_LIST);
@@ -548,6 +551,7 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 	Page		metapage;
 	GinMetaPageData *metadata;
 	BlockNumber blknoToDelete;
+	bool		needwal = RelationNeedsWAL(index);
 
 	metapage = BufferGetPage(metabuffer);
 	metadata = GinPageGetMeta(metapage);
@@ -586,8 +590,11 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 		 * prepare the XLogInsert machinery for that before entering the
 		 * critical section.
 		 */
-		if (RelationNeedsWAL(index))
+		if (needwal)
+		{
+			CheckWALPermitted();
 			XLogEnsureRecordSpace(data.ndeleted, 0);
+		}
 
 		START_CRIT_SECTION();
 
@@ -625,7 +632,7 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 			MarkBufferDirty(buffers[i]);
 		}
 
-		if (RelationNeedsWAL(index))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gin/gininsert.c b/src/backend/access/gin/gininsert.c
index 0e8672c9e90..79312e5d2d0 100644
--- a/src/backend/access/gin/gininsert.c
+++ b/src/backend/access/gin/gininsert.c
@@ -17,6 +17,7 @@
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/index.h"
 #include "miscadmin.h"
@@ -447,6 +448,9 @@ ginbuildempty(Relation index)
 		ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(RootBuffer, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermittedHaveXID();
+
 	/* Initialize and xlog metabuffer and root buffer. */
 	START_CRIT_SECTION();
 	GinInitMetabuffer(MetaBuffer);
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index 6b9b04cf429..6c20bc53fc5 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -17,6 +17,7 @@
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
 #include "access/reloptions.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/pg_collation.h"
 #include "catalog/pg_type.h"
@@ -659,12 +660,18 @@ ginUpdateStats(Relation index, const GinStatsData *stats, bool is_build)
 	Buffer		metabuffer;
 	Page		metapage;
 	GinMetaPageData *metadata;
+	bool		needwal;
 
 	metabuffer = ReadBuffer(index, GIN_METAPAGE_BLKNO);
 	LockBuffer(metabuffer, GIN_EXCLUSIVE);
 	metapage = BufferGetPage(metabuffer);
 	metadata = GinPageGetMeta(metapage);
 
+	needwal = RelationNeedsWAL(index) && !is_build;
+
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	metadata->nTotalPages = stats->nTotalPages;
@@ -684,7 +691,7 @@ ginUpdateStats(Relation index, const GinStatsData *stats, bool is_build)
 
 	MarkBufferDirty(metabuffer);
 
-	if (RelationNeedsWAL(index) && !is_build)
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogUpdateMeta data;
diff --git a/src/backend/access/gin/ginvacuum.c b/src/backend/access/gin/ginvacuum.c
index 35b85a9bff0..9e195e4bc6f 100644
--- a/src/backend/access/gin/ginvacuum.c
+++ b/src/backend/access/gin/ginvacuum.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "commands/vacuum.h"
 #include "miscadmin.h"
@@ -136,6 +137,7 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	Page		page,
 				parentPage;
 	BlockNumber rightlink;
+	bool		needwal;
 
 	/*
 	 * This function MUST be called only if someone of parent pages hold
@@ -159,6 +161,10 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	 */
 	PredicateLockPageCombine(gvs->index, deleteBlkno, rightlink);
 
+	needwal = RelationNeedsWAL(gvs->index);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* Unlink the page by changing left sibling's rightlink */
@@ -195,7 +201,7 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	MarkBufferDirty(lBuffer);
 	MarkBufferDirty(dBuffer);
 
-	if (RelationNeedsWAL(gvs->index))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogDeletePage data;
@@ -650,6 +656,9 @@ ginbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 
 		if (resPage)
 		{
+			if (RelationNeedsWAL(gvs.index))
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 			PageRestoreTempPage(resPage, page);
 			MarkBufferDirty(buffer);
diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c
index 992936cfa8e..affbd519328 100644
--- a/src/backend/access/gist/gist.c
+++ b/src/backend/access/gist/gist.c
@@ -16,6 +16,7 @@
 
 #include "access/gist_private.h"
 #include "access/gistscan.h"
+#include "access/walprohibit.h"
 #include "catalog/pg_collation.h"
 #include "commands/vacuum.h"
 #include "miscadmin.h"
@@ -135,6 +136,9 @@ gistbuildempty(Relation index)
 	buffer = ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermittedHaveXID();
+
 	/* Initialize and xlog buffer */
 	START_CRIT_SECTION();
 	GISTInitBuffer(buffer, F_LEAF);
@@ -235,6 +239,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 	XLogRecPtr	recptr;
 	int			i;
 	bool		is_split;
+	bool		needwal = RelationNeedsWAL(rel);
 
 	/*
 	 * Refuse to modify a page that's incompletely split. This should not
@@ -466,9 +471,12 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 		 * insertion for that. NB: The number of pages and data segments
 		 * specified here must match the calculations in gistXLogSplit()!
 		 */
-		if (!is_build && RelationNeedsWAL(rel))
+		if (!is_build && needwal)
 			XLogEnsureRecordSpace(npage, 1 + npage * 2);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		/*
@@ -501,7 +509,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 			recptr = GistBuildLSN;
 		else
 		{
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 				recptr = gistXLogSplit(is_leaf,
 									   dist, oldrlink, oldnsn, leftchildbuf,
 									   markfollowright);
@@ -527,6 +535,9 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 	}
 	else
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		/*
 		 * Enough space.  We always get here if ntup==0.
 		 */
@@ -568,7 +579,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 			recptr = GistBuildLSN;
 		else
 		{
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 			{
 				OffsetNumber ndeloffs = 0,
 							deloffs[1];
@@ -1643,6 +1654,7 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 	OffsetNumber offnum,
 				maxoff;
 	TransactionId latestRemovedXid = InvalidTransactionId;
+	bool		needwal = RelationNeedsWAL(rel);
 
 	Assert(GistPageIsLeaf(page));
 
@@ -1661,13 +1673,16 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 			deletable[ndeletable++] = offnum;
 	}
 
-	if (XLogStandbyInfoActive() && RelationNeedsWAL(rel))
+	if (XLogStandbyInfoActive() && needwal)
 		latestRemovedXid =
 			index_compute_xid_horizon_for_tuples(rel, heapRel, buffer,
 												 deletable, ndeletable);
 
 	if (ndeletable > 0)
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		PageIndexMultiDelete(page, deletable, ndeletable);
@@ -1684,7 +1699,7 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 		MarkBufferDirty(buffer);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gist/gistvacuum.c b/src/backend/access/gist/gistvacuum.c
index 94a7e127639..e0c483171ed 100644
--- a/src/backend/access/gist/gistvacuum.c
+++ b/src/backend/access/gist/gistvacuum.c
@@ -17,6 +17,7 @@
 #include "access/genam.h"
 #include "access/gist_private.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "commands/vacuum.h"
 #include "lib/integerset.h"
 #include "miscadmin.h"
@@ -260,6 +261,7 @@ gistvacuumpage(GistVacState *vstate, BlockNumber blkno, BlockNumber orig_blkno)
 	Buffer		buffer;
 	Page		page;
 	BlockNumber recurse_to;
+	bool		needwal = RelationNeedsWAL(rel);
 
 restart:
 	recurse_to = InvalidBlockNumber;
@@ -341,6 +343,9 @@ restart:
 		 */
 		if (ntodelete > 0)
 		{
+			if (needwal)
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 
 			MarkBufferDirty(buffer);
@@ -348,7 +353,7 @@ restart:
 			PageIndexMultiDelete(page, todelete, ntodelete);
 			GistMarkTuplesDeleted(page);
 
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 			{
 				XLogRecPtr	recptr;
 
@@ -580,6 +585,7 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	IndexTuple	idxtuple;
 	XLogRecPtr	recptr;
 	FullTransactionId txid;
+	bool		needwal = RelationNeedsWAL(info->index);
 
 	/*
 	 * Check that the leaf is still empty and deletable.
@@ -634,6 +640,9 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	 */
 	txid = ReadNextFullTransactionId();
 
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* mark the page as deleted */
@@ -645,7 +654,7 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	MarkBufferDirty(parentBuffer);
 	PageIndexTupleDelete(parentPage, downlink);
 
-	if (RelationNeedsWAL(info->index))
+	if (needwal)
 		recptr = gistXLogPageDelete(leafBuffer, txid, parentBuffer, downlink);
 	else
 		recptr = gistGetFakeLSN(info->index);
diff --git a/src/backend/access/hash/hash.c b/src/backend/access/hash/hash.c
index 0752fb38a92..52b2b36e4a9 100644
--- a/src/backend/access/hash/hash.c
+++ b/src/backend/access/hash/hash.c
@@ -22,6 +22,7 @@
 #include "access/hash_xlog.h"
 #include "access/relscan.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "catalog/index.h"
 #include "commands/progress.h"
 #include "commands/vacuum.h"
@@ -468,6 +469,7 @@ hashbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	Buffer		metabuf = InvalidBuffer;
 	HashMetaPage metap;
 	HashMetaPage cachedmetap;
+	bool		needwal;
 
 	tuples_removed = 0;
 	num_index_tuples = 0;
@@ -574,6 +576,10 @@ loop_top:
 		goto loop_top;
 	}
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Okay, we're really done.  Update tuple count in metapage. */
 	START_CRIT_SECTION();
 
@@ -604,7 +610,7 @@ loop_top:
 	MarkBufferDirty(metabuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_hash_update_meta_page xlrec;
 		XLogRecPtr	recptr;
@@ -691,6 +697,7 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 	Buffer		buf;
 	Bucket		new_bucket PG_USED_FOR_ASSERTS_ONLY = InvalidBucket;
 	bool		bucket_dirty = false;
+	bool		needwal = RelationNeedsWAL(rel);
 
 	blkno = bucket_blkno;
 	buf = bucket_buf;
@@ -789,6 +796,9 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		 */
 		if (ndeletable > 0)
 		{
+			if (needwal)
+				CheckWALPermitted();
+
 			/* No ereport(ERROR) until changes are logged */
 			START_CRIT_SECTION();
 
@@ -810,7 +820,7 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 			MarkBufferDirty(buf);
 
 			/* XLOG stuff */
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 			{
 				xl_hash_delete xlrec;
 				XLogRecPtr	recptr;
@@ -884,6 +894,9 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		page = BufferGetPage(bucket_buf);
 		bucket_opaque = (HashPageOpaque) PageGetSpecialPointer(page);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -891,7 +904,7 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		MarkBufferDirty(bucket_buf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c
index d254a00b6ac..20ff6d051bf 100644
--- a/src/backend/access/hash/hashinsert.c
+++ b/src/backend/access/hash/hashinsert.c
@@ -17,6 +17,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "storage/buf_internals.h"
 #include "storage/lwlock.h"
@@ -193,6 +194,8 @@ restart_insert:
 	 */
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	AssertWALPermittedHaveXID();
+
 	/* Do the update.  No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -360,6 +363,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 	if (ndeletable > 0)
 	{
 		TransactionId latestRemovedXid;
+		bool		needwal = RelationNeedsWAL(rel);
 
 		latestRemovedXid =
 			index_compute_xid_horizon_for_tuples(rel, hrel, buf,
@@ -370,6 +374,9 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 		 */
 		LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -393,7 +400,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 		MarkBufferDirty(metabuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			xl_hash_vacuum_one_page xlrec;
 			XLogRecPtr	recptr;
diff --git a/src/backend/access/hash/hashovfl.c b/src/backend/access/hash/hashovfl.c
index 1ff2e0c18ee..f251d6fc388 100644
--- a/src/backend/access/hash/hashovfl.c
+++ b/src/backend/access/hash/hashovfl.c
@@ -19,6 +19,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "utils/rel.h"
 
@@ -312,6 +313,8 @@ _hash_addovflpage(Relation rel, Buffer metabuf, Buffer buf, bool retain_pin)
 
 found:
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Do the update.  No ereport(ERROR) until changes are logged. We want to
 	 * log the changes for bitmap page and overflow page together to avoid
@@ -510,6 +513,7 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	Buffer		prevbuf = InvalidBuffer;
 	Buffer		nextbuf = InvalidBuffer;
 	bool		update_metap = false;
+	bool		needwal;
 
 	/* Get information from the doomed page */
 	_hash_checkpage(rel, ovflbuf, LH_OVERFLOW_PAGE);
@@ -573,10 +577,15 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	/* Get write-lock on metapage to update firstfree */
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	needwal = RelationNeedsWAL(rel);
+
 	/* This operation needs to log multiple tuples, prepare WAL for that */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 		XLogEnsureRecordSpace(HASH_XLOG_FREE_OVFL_BUFS, 4 + nitups);
 
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -641,7 +650,7 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	}
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_hash_squeeze_page xlrec;
 		XLogRecPtr	recptr;
@@ -922,14 +931,19 @@ readpage:
 
 				if (nitups > 0)
 				{
+					bool		needwal = RelationNeedsWAL(rel);
+
 					Assert(nitups == ndeletable);
 
 					/*
 					 * This operation needs to log multiple tuples, prepare
 					 * WAL for that.
 					 */
-					if (RelationNeedsWAL(rel))
+					if (needwal)
+					{
+						CheckWALPermitted();
 						XLogEnsureRecordSpace(0, 3 + nitups);
+					}
 
 					START_CRIT_SECTION();
 
@@ -947,7 +961,7 @@ readpage:
 					MarkBufferDirty(rbuf);
 
 					/* XLOG stuff */
-					if (RelationNeedsWAL(rel))
+					if (needwal)
 					{
 						XLogRecPtr	recptr;
 						xl_hash_move_page_contents xlrec;
diff --git a/src/backend/access/hash/hashpage.c b/src/backend/access/hash/hashpage.c
index 49a98677876..f161a9a49eb 100644
--- a/src/backend/access/hash/hashpage.c
+++ b/src/backend/access/hash/hashpage.c
@@ -30,6 +30,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "port/pg_bitutils.h"
 #include "storage/lmgr.h"
@@ -816,6 +817,8 @@ restart_expand:
 		goto fail;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Since we are scribbling on the pages in the shared buffers, establish a
 	 * critical section.  Any failure in this next code leaves us with a big
@@ -1172,6 +1175,8 @@ _hash_splitbucket(Relation rel,
 
 				if (PageGetFreeSpaceForMultipleTuples(npage, nitups + 1) < (all_tups_size + itemsz))
 				{
+					AssertWALPermittedHaveXID();
+
 					/*
 					 * Change the shared buffer state in critical section,
 					 * otherwise any error could make it unrecoverable.
@@ -1223,6 +1228,8 @@ _hash_splitbucket(Relation rel,
 		/* Exit loop if no more overflow pages in old bucket */
 		if (!BlockNumberIsValid(oblkno))
 		{
+			AssertWALPermittedHaveXID();
+
 			/*
 			 * Change the shared buffer state in critical section, otherwise
 			 * any error could make it unrecoverable.
@@ -1269,6 +1276,8 @@ _hash_splitbucket(Relation rel,
 	npage = BufferGetPage(bucket_nbuf);
 	nopaque = (HashPageOpaque) PageGetSpecialPointer(npage);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	oopaque->hasho_flag &= ~LH_BUCKET_BEING_SPLIT;
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 5b9cfb26cf7..d97dd657f76 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -47,6 +47,7 @@
 #include "access/transam.h"
 #include "access/valid.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1916,6 +1917,8 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
 	 */
 	CheckForSerializableConflictIn(relation, NULL, InvalidBlockNumber);
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -2190,6 +2193,8 @@ heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples,
 										   &vmbuffer, NULL);
 		page = BufferGetPage(buffer);
 
+		AssertWALPermittedHaveXID();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -2708,6 +2713,8 @@ l1:
 							  xid, LockTupleExclusive, true,
 							  &new_xmax, &new_infomask, &new_infomask2);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -3460,6 +3467,8 @@ l2:
 
 		Assert(HEAP_XMAX_IS_LOCKED_ONLY(infomask_lock_old_tuple));
 
+		AssertWALPermittedHaveXID();
+
 		START_CRIT_SECTION();
 
 		/* Clear obsolete visibility flags ... */
@@ -3633,6 +3642,8 @@ l2:
 										   bms_overlap(modified_attrs, id_attrs),
 										   &old_key_copied);
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -4566,6 +4577,8 @@ failed:
 							  GetCurrentTransactionId(), mode, false,
 							  &xid, &new_infomask, &new_infomask2);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -5357,6 +5370,8 @@ l4:
 								VISIBILITYMAP_ALL_FROZEN))
 			cleared_all_frozen = true;
 
+		AssertWALPermittedHaveXID();
+
 		START_CRIT_SECTION();
 
 		/* ... and set them */
@@ -5515,6 +5530,8 @@ heap_finish_speculative(Relation relation, ItemPointer tid)
 	StaticAssertStmt(MaxOffsetNumber < SpecTokenOffsetNumber,
 					 "invalid speculative token constant");
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -5623,6 +5640,8 @@ heap_abort_speculative(Relation relation, ItemPointer tid)
 		elog(ERROR, "attempted to kill a non-speculative tuple");
 	Assert(!HeapTupleHeaderIsHeapOnly(tp.t_data));
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * No need to check for serializable conflicts here.  There is never a
 	 * need for a combocid, either.  No need to extract replica identity, or
@@ -5743,6 +5762,7 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	HeapTupleHeader htup;
 	uint32		oldlen;
 	uint32		newlen;
+	bool		needwal;
 
 	/*
 	 * For now, we don't allow parallel updates.  Unlike a regular update,
@@ -5773,6 +5793,10 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	if (oldlen != newlen || htup->t_hoff != tuple->t_data->t_hoff)
 		elog(ERROR, "wrong tuple length");
 
+	needwal = RelationNeedsWAL(relation);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -5783,7 +5807,7 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	MarkBufferDirty(buffer);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(relation))
+	if (needwal)
 	{
 		xl_heap_inplace xlrec;
 		XLogRecPtr	recptr;
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index e3a716a2a2f..e93c211da4f 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -18,6 +18,7 @@
 #include "access/heapam_xlog.h"
 #include "access/htup_details.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "catalog/catalog.h"
 #include "miscadmin.h"
@@ -94,11 +95,11 @@ heap_page_prune_opt(Relation relation, Buffer buffer)
 	Size		minfree;
 
 	/*
-	 * We can't write WAL in recovery mode, so there's no point trying to
+	 * We can't write WAL during read-only mode, so there's no point trying to
 	 * clean the page. The primary will likely issue a cleaning WAL record soon
 	 * anyway, so this is no particular loss.
 	 */
-	if (RecoveryInProgress())
+	if (!XLogInsertAllowed())
 		return;
 
 	/*
@@ -232,6 +233,7 @@ heap_page_prune(Relation relation, Buffer buffer,
 	OffsetNumber offnum,
 				maxoff;
 	PruneState	prstate;
+	bool		needwal;
 
 	/*
 	 * Our strategy is to scan the page and make lists of items to change,
@@ -286,6 +288,10 @@ heap_page_prune(Relation relation, Buffer buffer,
 	if (off_loc)
 		*off_loc = InvalidOffsetNumber;
 
+	needwal = RelationNeedsWAL(relation);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Any error while applying the changes is critical */
 	START_CRIT_SECTION();
 
@@ -319,7 +325,7 @@ heap_page_prune(Relation relation, Buffer buffer,
 		/*
 		 * Emit a WAL XLOG_HEAP2_CLEAN record showing what we did
 		 */
-		if (RelationNeedsWAL(relation))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index f3d2265fad7..7370dc1f64d 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -59,6 +59,7 @@
 #include "access/parallel.h"
 #include "access/transam.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "catalog/storage.h"
@@ -758,6 +759,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 	BlockNumber nblocks,
 				blkno;
 	HeapTupleData tuple;
+	bool		needwal = RelationNeedsWAL(onerel);
 	TransactionId relfrozenxid = onerel->rd_rel->relfrozenxid;
 	TransactionId relminmxid = onerel->rd_rel->relminmxid;
 	BlockNumber empty_pages,
@@ -1200,6 +1202,9 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 			 */
 			if (!PageIsAllVisible(page))
 			{
+				if (needwal)
+					CheckWALPermitted();
+
 				START_CRIT_SECTION();
 
 				/* mark buffer dirty before writing a WAL record */
@@ -1215,7 +1220,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 				 * page has been previously WAL-logged, and if not, do that
 				 * now.
 				 */
-				if (RelationNeedsWAL(onerel) &&
+				if (needwal &&
 					PageGetLSN(page) == InvalidXLogRecPtr)
 					log_newpage_buffer(buf, true);
 
@@ -1481,6 +1486,9 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 		 */
 		if (nfrozen > 0)
 		{
+			if (needwal)
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 
 			MarkBufferDirty(buf);
@@ -1498,7 +1506,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 			}
 
 			/* Now WAL-log freezing if necessary */
-			if (RelationNeedsWAL(onerel))
+			if (needwal)
 			{
 				XLogRecPtr	recptr;
 
@@ -1931,6 +1939,7 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	TransactionId visibility_cutoff_xid;
 	bool		all_frozen;
 	LVSavedErrInfo saved_err_info;
+	bool		needwal = RelationNeedsWAL(onerel);
 
 	pgstat_progress_update_param(PROGRESS_VACUUM_HEAP_BLKS_VACUUMED, blkno);
 
@@ -1938,6 +1947,9 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	update_vacuum_error_info(vacrelstats, &saved_err_info, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
 							 blkno, InvalidOffsetNumber);
 
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	for (; tupindex < dead_tuples->num_tuples; tupindex++)
@@ -1963,7 +1975,7 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	MarkBufferDirty(buffer);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(onerel))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index e198df65d82..05f7de15c76 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -88,6 +88,7 @@
 
 #include "access/heapam_xlog.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "miscadmin.h"
 #include "port/pg_bitutils.h"
@@ -249,6 +250,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 	uint8		mapOffset = HEAPBLK_TO_OFFSET(heapBlk);
 	Page		page;
 	uint8	   *map;
+	bool		needwal = RelationNeedsWAL(rel);
 
 #ifdef TRACE_VISIBILITYMAP
 	elog(DEBUG1, "vm_set %s %d", RelationGetRelationName(rel), heapBlk);
@@ -270,6 +272,16 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 	map = (uint8 *) PageGetContents(page);
 	LockBuffer(vmBuf, BUFFER_LOCK_EXCLUSIVE);
 
+	/*
+	 * Can reach here from VACUUM or from startup process, so need not have an
+	 * XID.
+	 *
+	 * Recovery in the startup process never have wal prohibit state, skip
+	 * permission check if reach here in the startup process.
+	 */
+	if (needwal)
+		InRecovery ? AssertWALPermitted() : CheckWALPermitted();
+
 	if (flags != (map[mapByte] >> mapOffset & VISIBILITYMAP_VALID_BITS))
 	{
 		START_CRIT_SECTION();
@@ -277,7 +289,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 		map[mapByte] |= (flags << mapOffset);
 		MarkBufferDirty(vmBuf);
 
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			if (XLogRecPtrIsInvalid(recptr))
 			{
@@ -476,6 +488,7 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		Buffer		mapBuffer;
 		Page		page;
 		char	   *map;
+		bool		needwal;
 
 		newnblocks = truncBlock + 1;
 
@@ -489,8 +502,13 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		page = BufferGetPage(mapBuffer);
 		map = PageGetContents(page);
 
+		needwal = (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded());
+
 		LockBuffer(mapBuffer, BUFFER_LOCK_EXCLUSIVE);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -518,7 +536,7 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		 * during recovery.
 		 */
 		MarkBufferDirty(mapBuffer);
-		if (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded())
+		if (needwal)
 			log_newpage_buffer(mapBuffer, false);
 
 		END_CRIT_SECTION();
diff --git a/src/backend/access/nbtree/nbtdedup.c b/src/backend/access/nbtree/nbtdedup.c
index 854e3b2cf9a..71f5ade8c28 100644
--- a/src/backend/access/nbtree/nbtdedup.c
+++ b/src/backend/access/nbtree/nbtdedup.c
@@ -16,6 +16,7 @@
 
 #include "access/nbtree.h"
 #include "access/nbtxlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "utils/rel.h"
 
@@ -229,6 +230,8 @@ _bt_dedup_pass(Relation rel, Buffer buf, Relation heapRel, IndexTuple newitem,
 		nopaque->btpo_flags &= ~BTP_HAS_GARBAGE;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	PageRestoreTempPage(newpage, page);
diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c
index e3336039125..c840912d116 100644
--- a/src/backend/access/nbtree/nbtinsert.c
+++ b/src/backend/access/nbtree/nbtinsert.c
@@ -18,6 +18,7 @@
 #include "access/nbtree.h"
 #include "access/nbtxlog.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "lib/qunique.h"
 #include "miscadmin.h"
@@ -1249,6 +1250,8 @@ _bt_insertonpg(Relation rel,
 			}
 		}
 
+		AssertWALPermittedHaveXID();
+
 		/* Do the update.  No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -1901,13 +1904,16 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf,
 			ropaque->btpo_flags |= BTP_SPLIT_END;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Right sibling is locked, new siblings are prepared, but original page
 	 * is not updated yet.
 	 *
 	 * NO EREPORT(ERROR) till right sibling is updated.  We can get away with
 	 * not starting the critical section till here because we haven't been
-	 * scribbling on the original page yet; see comments above.
+	 * scribbling on the original page yet; see the comments above for grabbing
+	 * the right sibling.
 	 */
 	START_CRIT_SECTION();
 
@@ -2469,6 +2475,8 @@ _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf)
 	right_item = CopyIndexTuple(item);
 	BTreeTupleSetDownLink(right_item, rbkno);
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till newroot op is logged */
 	START_CRIT_SECTION();
 
diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c
index e230f912c28..9a9e2fc2b3c 100644
--- a/src/backend/access/nbtree/nbtpage.c
+++ b/src/backend/access/nbtree/nbtpage.c
@@ -26,6 +26,7 @@
 #include "access/nbtxlog.h"
 #include "access/tableam.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
@@ -185,6 +186,7 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	BTMetaPageData *metad;
 	bool		needsRewrite = false;
 	XLogRecPtr	recptr;
+	bool		needwal;
 
 	/* read the metapage and check if it needs rewrite */
 	metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ);
@@ -208,6 +210,10 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	_bt_unlockbuf(rel, metabuf);
 	_bt_lockbuf(rel, metabuf, BT_WRITE);
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* upgrade meta-page if needed */
@@ -220,7 +226,7 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	MarkBufferDirty(metabuf);
 
 	/* write wal record if needed */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_btree_metadata md;
 
@@ -338,6 +344,7 @@ _bt_getroot(Relation rel, int access)
 	if (metad->btm_root == P_NONE)
 	{
 		Page		metapg;
+		bool		needwal;
 
 		/* If access = BT_READ, caller doesn't want us to create root yet */
 		if (access == BT_READ)
@@ -383,6 +390,10 @@ _bt_getroot(Relation rel, int access)
 		/* Get raw page pointer for metapage */
 		metapg = BufferGetPage(metabuf);
 
+		needwal = RelationNeedsWAL(rel);
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO ELOG(ERROR) till meta is updated */
 		START_CRIT_SECTION();
 
@@ -401,7 +412,7 @@ _bt_getroot(Relation rel, int access)
 		MarkBufferDirty(metabuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			xl_btree_newroot xlrec;
 			XLogRecPtr	recptr;
@@ -1148,6 +1159,9 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 										 updatedoffsets, &updatedbuflen,
 										 needswal);
 
+	if (needswal)
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -1279,6 +1293,8 @@ _bt_delitems_delete(Relation rel, Buffer buf, TransactionId latestRemovedXid,
 										 updatedoffsets, &updatedbuflen,
 										 needswal);
 
+	AssertWALPermittedHaveXID();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -2065,6 +2081,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	OffsetNumber nextoffset;
 	IndexTuple	itup;
 	IndexTupleData trunctuple;
+	bool		needwal;
 
 	page = BufferGetPage(leafbuf);
 	opaque = (BTPageOpaque) PageGetSpecialPointer(page);
@@ -2153,6 +2170,10 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	 */
 	PredicateLockPageCombine(rel, leafblkno, leafrightsib);
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -2204,7 +2225,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	MarkBufferDirty(leafbuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_btree_mark_page_halfdead xlrec;
 		XLogRecPtr	recptr;
@@ -2297,6 +2318,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 	int			targetlevel;
 	IndexTuple	leafhikey;
 	BlockNumber nextchild;
+	bool		needwal;
 
 	page = BufferGetPage(leafbuf);
 	opaque = (BTPageOpaque) PageGetSpecialPointer(page);
@@ -2511,6 +2533,10 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 	 * Here we begin doing the deletion.
 	 */
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -2590,7 +2616,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 		MarkBufferDirty(leafbuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_btree_unlink_page xlrec;
 		xl_btree_metadata xlmeta;
diff --git a/src/backend/access/spgist/spgdoinsert.c b/src/backend/access/spgist/spgdoinsert.c
index 7bd269fd2a0..1f9dd667262 100644
--- a/src/backend/access/spgist/spgdoinsert.c
+++ b/src/backend/access/spgist/spgdoinsert.c
@@ -18,6 +18,7 @@
 #include "access/genam.h"
 #include "access/spgist_private.h"
 #include "access/spgxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -214,6 +215,8 @@ addLeafTuple(Relation index, SpGistState *state, SpGistLeafTuple leafTuple,
 	xlrec.offnumParent = InvalidOffsetNumber;
 	xlrec.nodeI = 0;
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	if (current->offnum == InvalidOffsetNumber ||
@@ -458,6 +461,8 @@ moveLeafs(Relation index, SpGistState *state,
 
 	leafdata = leafptr = palloc(size);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/* copy all the old tuples to new page, unless they're dead */
@@ -1109,6 +1114,8 @@ doPickSplit(Relation index, SpGistState *state,
 
 	leafdata = leafptr = (char *) palloc(totalLeafSizes);
 
+	AssertWALPermittedHaveXID();
+
 	/* Here we begin making the changes to the target pages */
 	START_CRIT_SECTION();
 
@@ -1517,6 +1524,8 @@ spgAddNodeAction(Relation index, SpGistState *state,
 	if (PageGetExactFreeSpace(current->page) >=
 		newInnerTuple->size - innerTuple->size)
 	{
+		AssertWALPermittedHaveXID();
+
 		/*
 		 * We can replace the inner tuple by new version in-place
 		 */
@@ -1603,6 +1612,8 @@ spgAddNodeAction(Relation index, SpGistState *state,
 		else
 			xlrec.parentBlk = 2;
 
+		AssertWALPermittedHaveXID();
+
 		START_CRIT_SECTION();
 
 		/* insert new ... */
@@ -1788,6 +1799,8 @@ spgSplitNodeAction(Relation index, SpGistState *state,
 									&xlrec.newPage);
 	}
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c
index 0d02a02222e..d689473a713 100644
--- a/src/backend/access/spgist/spgvacuum.c
+++ b/src/backend/access/spgist/spgvacuum.c
@@ -19,6 +19,7 @@
 #include "access/spgist_private.h"
 #include "access/spgxlog.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/storage_xlog.h"
 #include "commands/vacuum.h"
@@ -139,6 +140,7 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 	int			nDeletable;
 	OffsetNumber i,
 				max = PageGetMaxOffsetNumber(page);
+	bool		needwal;
 
 	memset(predecessor, 0, sizeof(predecessor));
 	memset(deletable, 0, sizeof(deletable));
@@ -323,6 +325,10 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 	if (nDeletable != xlrec.nDead + xlrec.nPlaceholder + xlrec.nMove)
 		elog(ERROR, "inconsistent counts of deletable tuples");
 
+	needwal = RelationNeedsWAL(index);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Do the updates */
 	START_CRIT_SECTION();
 
@@ -371,7 +377,7 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 
@@ -411,6 +417,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 	OffsetNumber toDelete[MaxIndexTuplesPerPage];
 	OffsetNumber i,
 				max = PageGetMaxOffsetNumber(page);
+	bool		needwal;
 
 	xlrec.nDelete = 0;
 
@@ -447,6 +454,10 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 	if (xlrec.nDelete == 0)
 		return;					/* nothing more to do */
 
+	needwal = RelationNeedsWAL(index);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Do the update */
 	START_CRIT_SECTION();
 
@@ -455,7 +466,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 
@@ -502,6 +513,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer)
 	OffsetNumber itemnos[MaxIndexTuplesPerPage];
 	spgxlogVacuumRedirect xlrec;
 	GlobalVisState *vistest;
+	bool		needwal;
 
 	xlrec.nToPlaceholder = 0;
 	xlrec.newestRedirectXid = InvalidTransactionId;
@@ -509,6 +521,10 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer)
 	/* XXX: providing heap relation would allow more pruning */
 	vistest = GlobalVisTestFor(NULL);
 
+	needwal = RelationNeedsWAL(index);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -584,7 +600,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer)
 	if (hasUpdate)
 		MarkBufferDirty(buffer);
 
-	if (hasUpdate && RelationNeedsWAL(index))
+	if (hasUpdate && needwal)
 	{
 		XLogRecPtr	recptr;
 
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index 1233448481a..5a665caa87b 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -73,6 +73,7 @@
 #include "access/transam.h"
 #include "access/twophase.h"
 #include "access/twophase_rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1162,6 +1163,8 @@ GetNewMultiXactId(int nmembers, MultiXactOffset *offset)
 
 	ExtendMultiXactMember(nextOffset, nmembers);
 
+	CheckWALPermitted();
+
 	/*
 	 * Critical section from here until caller has written the data into the
 	 * just-reserved SLRU space; we don't want to error out with a partly
@@ -2946,7 +2949,7 @@ TruncateMultiXact(MultiXactId newOldestMulti, Oid newOldestMultiDB)
 	mxtruncinfo trunc;
 	MultiXactId earliest;
 
-	Assert(!RecoveryInProgress());
+	Assert(XLogInsertAllowed());
 	Assert(MultiXactState->finishedStartup);
 
 	/*
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index fc18b778324..91f2b18e367 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -82,6 +82,7 @@
 #include "access/transam.h"
 #include "access/twophase.h"
 #include "access/twophase_rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1106,6 +1107,8 @@ EndPrepare(GlobalTransaction gxact)
 	 */
 	XLogEnsureRecordSpace(0, records.num_chunks);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	MyProc->delayChkpt = true;
@@ -2196,6 +2199,9 @@ RecordTransactionCommitPrepared(TransactionId xid,
 	replorigin = (replorigin_session_origin != InvalidRepOriginId &&
 				  replorigin_session_origin != DoNotReplicateId);
 
+	/* COMMIT PREPARED need not have an XID */
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* See notes in RecordTransactionCommit */
@@ -2286,6 +2292,9 @@ RecordTransactionAbortPrepared(TransactionId xid,
 		elog(PANIC, "cannot abort transaction %u, it was already committed",
 			 xid);
 
+	/* ROLLBACK PREPARED need not have an XID */
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
diff --git a/src/backend/access/transam/varsup.c b/src/backend/access/transam/varsup.c
index 2264c2c849c..15cb9b9a25c 100644
--- a/src/backend/access/transam/varsup.c
+++ b/src/backend/access/transam/varsup.c
@@ -17,6 +17,7 @@
 #include "access/commit_ts.h"
 #include "access/subtrans.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "commands/dbcommands.h"
@@ -75,6 +76,9 @@ GetNewTransactionId(bool isSubXact)
 	if (RecoveryInProgress())
 		elog(ERROR, "cannot assign TransactionIds during recovery");
 
+	/* do not assign transaction id in read-only mode */
+	CheckWALPermitted();
+
 	LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
 
 	full_xid = ShmemVariableCache->nextXid;
diff --git a/src/backend/access/transam/walprohibit.c b/src/backend/access/transam/walprohibit.c
index de1f2f47040..60e368f6bcb 100644
--- a/src/backend/access/transam/walprohibit.c
+++ b/src/backend/access/transam/walprohibit.c
@@ -25,6 +25,16 @@
 #include "utils/fmgroids.h"
 #include "utils/fmgrprotos.h"
 
+/*
+ * Assert flag to enforce WAL insert permission check rule before starting a
+ * critical section for the WAL writes.  For this, either of
+ * CheckWALPermitted(), AssertWALPermittedHaveXID(), or AssertWALPermitted()
+ * must be called before starting the critical section.
+ */
+#ifdef USE_ASSERT_CHECKING
+WALPermitCheckState walpermit_checked_state = WALPERMIT_UNCHECKED;
+#endif
+
 /*
  * Shared-memory WAL prohibit state
  */
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 0de63af6365..cdb18c47b0c 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -26,6 +26,7 @@
 #include "access/subtrans.h"
 #include "access/transam.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1317,6 +1318,8 @@ RecordTransactionCommit(void)
 		/* Tell bufmgr and smgr to prepare for commit */
 		BufmgrCommit();
 
+		AssertWALPermittedHaveXID();
+
 		/*
 		 * Mark ourselves as within our "commit critical section".  This
 		 * forces any concurrent checkpoint to wait until we've updated
@@ -1677,6 +1680,9 @@ RecordTransactionAbort(bool isSubXact)
 		elog(PANIC, "cannot abort transaction %u, it was already committed",
 			 xid);
 
+	/* We'll be reaching here with valid XID only. */
+	AssertWALPermittedHaveXID();
+
 	/* Fetch the data we need for the abort record */
 	nrels = smgrGetPendingDeletes(false, &rels);
 	nchildren = xactGetCommittedChildren(&children);
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index e31327ed5c7..2ba93bfad5e 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -1036,7 +1036,7 @@ XLogInsertRecord(XLogRecData *rdata,
 
 	/* cross-check on whether we should be here or not */
 	if (!XLogInsertAllowed())
-		elog(ERROR, "cannot make new WAL entries during recovery");
+		elog(ERROR, "cannot make new WAL entries now");
 
 	/*----------
 	 *
@@ -2873,9 +2873,11 @@ XLogFlush(XLogRecPtr record)
 	 * trying to flush the WAL, we should update minRecoveryPoint instead. We
 	 * test XLogInsertAllowed(), not InRecovery, because we need checkpointer
 	 * to act this way too, and because when it tries to write the
-	 * end-of-recovery checkpoint, it should indeed flush.
+	 * end-of-recovery checkpoint, it should indeed flush. Also, WAL prohibit
+	 * state should not restrict WAL flushing. Otherwise, the dirty buffer
+	 * cannot be evicted until WAL has been flushed up to the buffer's LSN.
 	 */
-	if (!XLogInsertAllowed())
+	if (!XLogInsertAllowed() && !IsWALProhibited())
 	{
 		UpdateMinRecoveryPoint(record, false);
 		return;
@@ -8886,6 +8888,8 @@ CreateCheckPoint(int flags)
 	/* sanity check */
 	if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
 		elog(ERROR, "can't create a checkpoint during recovery");
+	if (!XLogInsertAllowed() && !RecoveryInProgress())
+		elog(ERROR, "can't create a checkpoint while system is read only ");
 
 	/*
 	 * Initialize InitXLogInsert working areas before entering the critical
@@ -9150,6 +9154,9 @@ CreateCheckPoint(int flags)
 	if (!shutdown && XLogStandbyInfoActive())
 		LogStandbySnapshot();
 
+ 	/* Error out if wal writes are disabled. */
+ 	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -9310,6 +9317,8 @@ CreateEndOfRecoveryRecord(void)
 
 	LocalSetXLogInsertAllowed();
 
+	AssertWALPermitted();
+
 	START_CRIT_SECTION();
 
 	XLogBeginInsert();
@@ -9974,7 +9983,7 @@ void
 UpdateFullPageWrites(void)
 {
 	XLogCtlInsert *Insert = &XLogCtl->Insert;
-	bool		recoveryInProgress;
+	bool		WALInsertAllowed;
 
 	/*
 	 * Do nothing if full_page_writes has not been changed.
@@ -9988,10 +9997,10 @@ UpdateFullPageWrites(void)
 
 	/*
 	 * Perform this outside critical section so that the WAL insert
-	 * initialization done by RecoveryInProgress() doesn't trigger an
+	 * initialization done by XLogInsertAllowed() doesn't trigger an
 	 * assertion failure.
 	 */
-	recoveryInProgress = RecoveryInProgress();
+	WALInsertAllowed = XLogInsertAllowed();
 
 	START_CRIT_SECTION();
 
@@ -10013,8 +10022,11 @@ UpdateFullPageWrites(void)
 	 * Write an XLOG_FPW_CHANGE record. This allows us to keep track of
 	 * full_page_writes during archive recovery, if required.
 	 */
-	if (XLogStandbyInfoActive() && !recoveryInProgress)
+	if (XLogStandbyInfoActive() && WALInsertAllowed)
 	{
+		/* Assured that WAL permission has been checked */
+		AssertWALPermitted();
+
 		XLogBeginInsert();
 		XLogRegisterData((char *) (&fullPageWrites), sizeof(bool));
 
diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c
index 7052dc245ee..a925cc9d5cd 100644
--- a/src/backend/access/transam/xloginsert.c
+++ b/src/backend/access/transam/xloginsert.c
@@ -19,6 +19,7 @@
 
 #include "postgres.h"
 
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xlog_internal.h"
@@ -126,9 +127,14 @@ XLogBeginInsert(void)
 	Assert(mainrdata_last == (XLogRecData *) &mainrdata_head);
 	Assert(mainrdata_len == 0);
 
+	/*
+	 * WAL permission must have checked before entering the critical section.
+	 * Otherwise, WAL prohibited error will force system panic.
+	 */
+	Assert(walpermit_checked_state != WALPERMIT_UNCHECKED || !CritSectionCount);
+
 	/* cross-check on whether we should be here or not */
-	if (!XLogInsertAllowed())
-		elog(ERROR, "cannot make new WAL entries during recovery");
+	CheckWALPermitted();
 
 	if (begininsert_called)
 		elog(ERROR, "XLogBeginInsert was already called");
@@ -210,6 +216,9 @@ XLogResetInsertion(void)
 	mainrdata_last = (XLogRecData *) &mainrdata_head;
 	curinsert_flags = 0;
 	begininsert_called = false;
+
+	/* Reset walpermit_checked flag */
+	RESET_WALPERMIT_CHECKED_STATE();
 }
 
 /*
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 0415df9ccb7..ab108e621d4 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -20,6 +20,7 @@
 #include "access/relation.h"
 #include "access/table.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -378,8 +379,13 @@ fill_seq_with_data(Relation rel, HeapTuple tuple)
 
 	/* check the comment above nextval_internal()'s equivalent call. */
 	if (RelationNeedsWAL(rel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermittedHaveXID();
+	}
+
 	START_CRIT_SECTION();
 
 	MarkBufferDirty(buf);
@@ -766,8 +772,13 @@ nextval_internal(Oid relid, bool check_permissions)
 	 * (Have to do that here, so we're outside the critical section)
 	 */
 	if (logit && RelationNeedsWAL(seqrel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermittedHaveXID();
+	}
+
 	/* ready to change the on-disk (or really, in-buffer) tuple */
 	START_CRIT_SECTION();
 
@@ -977,8 +988,13 @@ do_setval(Oid relid, int64 next, bool iscalled)
 
 	/* check the comment above nextval_internal()'s equivalent call. */
 	if (RelationNeedsWAL(seqrel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermittedHaveXID();
+	}
+
 	/* ready to change the on-disk (or really, in-buffer) tuple */
 	START_CRIT_SECTION();
 
diff --git a/src/backend/commands/variable.c b/src/backend/commands/variable.c
index c5cf08b4237..aa1b0c12000 100644
--- a/src/backend/commands/variable.c
+++ b/src/backend/commands/variable.c
@@ -501,11 +501,14 @@ check_transaction_read_only(bool *newval, void **extra, GucSource source)
 			GUC_check_errmsg("transaction read-write mode must be set before any query");
 			return false;
 		}
-		/* Can't go to r/w mode while recovery is still active */
-		if (RecoveryInProgress())
+		/*
+		 * Can't go to r/w mode while recovery is still active or while in WAL
+		 * prohibit state
+		 */
+		if (!XLogInsertAllowed())
 		{
 			GUC_check_errcode(ERRCODE_FEATURE_NOT_SUPPORTED);
-			GUC_check_errmsg("cannot set transaction read-write mode during recovery");
+			GUC_check_errmsg("cannot set transaction read-write mode while system is read only");
 			return false;
 		}
 	}
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 1324d494724..dcec7d57471 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -924,6 +924,10 @@ RequestCheckpoint(int flags)
 	int			old_failed,
 				old_started;
 
+	/* The checkpoint is allowed in recovery but not in WAL prohibit state */
+	if (!RecoveryInProgress())
+		CheckWALPermitted();
+
 	/*
 	 * If in a standalone backend, just do it ourselves.
 	 */
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 561c212092f..909c3e75107 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -3820,13 +3820,15 @@ MarkBufferDirtyHint(Buffer buffer, bool buffer_std)
 		{
 			/*
 			 * If we must not write WAL, due to a relfilenode-specific
-			 * condition or being in recovery, don't dirty the page.  We can
-			 * set the hint, just not dirty the page as a result so the hint
-			 * is lost when we evict the page or shutdown.
+			 * condition or in general, don't dirty the page.  We can
+			 * set the hint, but must not dirty the page as a result, lest
+			 * we trigger WAL generation. Unless the page is dirtied again
+			 * later, the hint will be lost when the page is evicted, or at
+			 * shutdown.
 			 *
 			 * See src/backend/storage/page/README for longer discussion.
 			 */
-			if (RecoveryInProgress() ||
+			if (!XLogInsertAllowed() ||
 				RelFileNodeSkippingWAL(bufHdr->tag.rnode))
 				return;
 
diff --git a/src/backend/storage/freespace/freespace.c b/src/backend/storage/freespace/freespace.c
index 8c12dda2380..41ba8c59a88 100644
--- a/src/backend/storage/freespace/freespace.c
+++ b/src/backend/storage/freespace/freespace.c
@@ -24,6 +24,7 @@
 #include "postgres.h"
 
 #include "access/htup_details.h"
+#include "access/walprohibit.h"
 #include "access/xlogutils.h"
 #include "miscadmin.h"
 #include "storage/freespace.h"
@@ -285,12 +286,19 @@ FreeSpaceMapPrepareTruncateRel(Relation rel, BlockNumber nblocks)
 	 */
 	if (first_removed_slot > 0)
 	{
+		bool needwal;
+
 		buf = fsm_readbuf(rel, first_removed_address, false);
 		if (!BufferIsValid(buf))
 			return InvalidBlockNumber;	/* nothing to do; the FSM was already
 										 * smaller */
 		LockBuffer(buf, BUFFER_LOCK_EXCLUSIVE);
 
+		needwal = (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded());
+
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -305,7 +313,7 @@ FreeSpaceMapPrepareTruncateRel(Relation rel, BlockNumber nblocks)
 		 * during recovery.
 		 */
 		MarkBufferDirty(buf);
-		if (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded())
+		if (needwal)
 			log_newpage_buffer(buf, false);
 
 		END_CRIT_SECTION();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 20e50247ea4..693601de238 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -793,15 +793,15 @@ LockAcquireExtended(const LOCKTAG *locktag,
 	if (lockmode <= 0 || lockmode > lockMethodTable->numLockModes)
 		elog(ERROR, "unrecognized lock mode: %d", lockmode);
 
-	if (RecoveryInProgress() && !InRecovery &&
+	if (!XLogInsertAllowed() && !InRecovery &&
 		(locktag->locktag_type == LOCKTAG_OBJECT ||
 		 locktag->locktag_type == LOCKTAG_RELATION) &&
 		lockmode > RowExclusiveLock)
 		ereport(ERROR,
 				(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
-				 errmsg("cannot acquire lock mode %s on database objects while recovery is in progress",
+				 errmsg("cannot acquire lock mode %s on database objects while system is read only",
 						lockMethodTable->lockModeNames[lockmode]),
-				 errhint("Only RowExclusiveLock or less can be acquired on database objects during recovery.")));
+				 errhint("Only RowExclusiveLock or less can be acquired on database objects while system is read only")));
 
 #ifdef LOCK_DEBUG
 	if (LOCK_DEBUG_ENABLED(locktag))
diff --git a/src/backend/utils/cache/relmapper.c b/src/backend/utils/cache/relmapper.c
index 424624cf0da..d91e247f3e4 100644
--- a/src/backend/utils/cache/relmapper.c
+++ b/src/backend/utils/cache/relmapper.c
@@ -43,6 +43,7 @@
 #include <sys/stat.h>
 #include <unistd.h>
 
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -841,6 +842,8 @@ write_relmap_file(bool shared, RelMapFile *newmap,
 		xl_relmap_update xlrec;
 		XLogRecPtr	lsn;
 
+		AssertWALPermittedHaveXID();
+
 		/* now errors are fatal ... */
 		START_CRIT_SECTION();
 
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bdc97e3082..2ee57769835 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -95,12 +95,37 @@ extern PGDLLIMPORT volatile uint32 CritSectionCount;
 /* in tcop/postgres.c */
 extern void ProcessInterrupts(void);
 
+#ifdef USE_ASSERT_CHECKING
+typedef enum
+{
+	WALPERMIT_UNCHECKED,
+	WALPERMIT_CHECKED,
+	WALPERMIT_CHECKED_AND_USED
+} WALPermitCheckState;
+
+/* in access/walprohibit.c */
+extern WALPermitCheckState walpermit_checked_state;
+
+/*
+ * Reset walpermit_checked flag when no longer in the critical section.
+ * Otherwise, marked checked and used.
+ */
+#define RESET_WALPERMIT_CHECKED_STATE() \
+do { \
+	walpermit_checked_state = CritSectionCount ? \
+	WALPERMIT_CHECKED_AND_USED : WALPERMIT_UNCHECKED; \
+} while(0)
+#else
+#define RESET_WALPERMIT_CHECKED_STATE() ((void) 0)
+#endif
+
 #ifndef WIN32
 
 #define CHECK_FOR_INTERRUPTS() \
 do { \
 	if (unlikely(InterruptPending)) \
 		ProcessInterrupts(); \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 #else							/* WIN32 */
 
@@ -110,6 +135,7 @@ do { \
 		pgwin32_dispatch_queued_signals(); \
 	if (unlikely(InterruptPending)) \
 		ProcessInterrupts(); \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 #endif							/* WIN32 */
 
@@ -136,6 +162,7 @@ do { \
 do { \
 	Assert(CritSectionCount > 0); \
 	CritSectionCount--; \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 
 
-- 
2.18.0

From 4750542d598361b021801c4d56ecdc7603912ef7 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 27 Mar 2020 05:05:38 -0400
Subject: [PATCH v12 3/6] Add alter system read only/write syntax

Note that syntax doesn't have any implementation.
---
 src/backend/nodes/copyfuncs.c    | 12 ++++++++++++
 src/backend/nodes/equalfuncs.c   |  9 +++++++++
 src/backend/nodes/outfuncs.c     | 12 ++++++++++++
 src/backend/nodes/readfuncs.c    | 15 +++++++++++++++
 src/backend/parser/gram.y        | 13 +++++++++++++
 src/backend/tcop/utility.c       | 21 +++++++++++++++++++++
 src/bin/psql/tab-complete.c      |  6 ++++--
 src/include/nodes/nodes.h        |  1 +
 src/include/nodes/parsenodes.h   | 10 ++++++++++
 src/tools/pgindent/typedefs.list |  1 +
 10 files changed, 98 insertions(+), 2 deletions(-)

diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index ba3ccc712c8..7ce1a3146bf 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -4028,6 +4028,15 @@ _copyAlterSystemStmt(const AlterSystemStmt *from)
 	return newnode;
 }
 
+static AlterSystemWALProhibitState *
+_copyAlterSystemWALProhibitState(const AlterSystemWALProhibitState *from)
+{
+	AlterSystemWALProhibitState *newnode = makeNode(AlterSystemWALProhibitState);
+
+	COPY_SCALAR_FIELD(walprohibited);
+	return newnode;
+}
+
 static CreateSeqStmt *
 _copyCreateSeqStmt(const CreateSeqStmt *from)
 {
@@ -5414,6 +5423,9 @@ copyObjectImpl(const void *from)
 		case T_AlterSystemStmt:
 			retval = _copyAlterSystemStmt(from);
 			break;
+		case T_AlterSystemWALProhibitState:
+			retval = _copyAlterSystemWALProhibitState(from);
+			break;
 		case T_CreateSeqStmt:
 			retval = _copyCreateSeqStmt(from);
 			break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index a2ef853dc2a..fd54fe3088a 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1777,6 +1777,12 @@ _equalAlterSystemStmt(const AlterSystemStmt *a, const AlterSystemStmt *b)
 	return true;
 }
 
+static bool
+_equalAlterSystemWALProhibitState(const AlterSystemWALProhibitState *a, const AlterSystemWALProhibitState *b)
+{
+	COMPARE_SCALAR_FIELD(walprohibited);
+	return true;
+}
 
 static bool
 _equalCreateSeqStmt(const CreateSeqStmt *a, const CreateSeqStmt *b)
@@ -3467,6 +3473,9 @@ equal(const void *a, const void *b)
 		case T_AlterSystemStmt:
 			retval = _equalAlterSystemStmt(a, b);
 			break;
+		case T_AlterSystemWALProhibitState:
+			retval = _equalAlterSystemWALProhibitState(a, b);
+			break;
 		case T_CreateSeqStmt:
 			retval = _equalCreateSeqStmt(a, b);
 			break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 8392be6d44a..47230ad346a 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -1359,6 +1359,15 @@ _outAlternativeSubPlan(StringInfo str, const AlternativeSubPlan *node)
 	WRITE_NODE_FIELD(subplans);
 }
 
+static void
+_outAlterSystemWALProhibitState(StringInfo str,
+								const AlterSystemWALProhibitState *node)
+{
+	WRITE_NODE_TYPE("ALTERSYSTEMWALPROHIBITSTATE");
+
+	WRITE_BOOL_FIELD(walprohibited);
+}
+
 static void
 _outFieldSelect(StringInfo str, const FieldSelect *node)
 {
@@ -3938,6 +3947,9 @@ outNode(StringInfo str, const void *obj)
 			case T_AlternativeSubPlan:
 				_outAlternativeSubPlan(str, obj);
 				break;
+			case T_AlterSystemWALProhibitState:
+				_outAlterSystemWALProhibitState(str, obj);
+				break;
 			case T_FieldSelect:
 				_outFieldSelect(str, obj);
 				break;
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index d2c8d58070b..f5f3c8cff89 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -2553,6 +2553,19 @@ _readAlternativeSubPlan(void)
 	READ_DONE();
 }
 
+/*
+ * _readAlterSystemWALProhibitState
+ */
+static AlterSystemWALProhibitState *
+_readAlterSystemWALProhibitState(void)
+{
+	READ_LOCALS(AlterSystemWALProhibitState);
+
+	READ_BOOL_FIELD(walprohibited);
+
+	READ_DONE();
+}
+
 /*
  * _readExtensibleNode
  */
@@ -2875,6 +2888,8 @@ parseNodeString(void)
 		return_value = _readSubPlan();
 	else if (MATCH("ALTERNATIVESUBPLAN", 18))
 		return_value = _readAlternativeSubPlan();
+	else if (MATCH("ALTERSYSTEMWALPROHIBITSTATE", 27))
+		return_value = _readAlterSystemWALProhibitState();
 	else if (MATCH("EXTENSIBLENODE", 14))
 		return_value = _readExtensibleNode();
 	else if (MATCH("PARTITIONBOUNDSPEC", 18))
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 31c95443a5b..db39f7caa91 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -477,6 +477,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
 
 %type <vsetstmt> generic_set set_rest set_rest_more generic_reset reset_rest
 				 SetResetClause FunctionSetResetClause
+%type <boolean> system_readonly_state
 
 %type <node>	TableElement TypedTableElement ConstraintElem TableFuncElement
 %type <node>	columnDef columnOptions
@@ -10227,8 +10228,20 @@ AlterSystemStmt:
 					n->setstmt = $4;
 					$$ = (Node *)n;
 				}
+			| ALTER SYSTEM_P system_readonly_state
+				{
+					AlterSystemWALProhibitState *n = makeNode(AlterSystemWALProhibitState);
+					n->walprohibited = $3;
+					$$ = (Node *)n;
+				}
 		;
 
+system_readonly_state:
+			 READ ONLY
+					{ $$ = true; }
+			| READ WRITE
+					{ $$ = false; }
+		;
 
 /*****************************************************************************
  *
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 53a511f1da8..e9cc4a22324 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -86,6 +86,7 @@ static void ProcessUtilitySlow(ParseState *pstate,
 							   DestReceiver *dest,
 							   QueryCompletion *qc);
 static void ExecDropStmt(DropStmt *stmt, bool isTopLevel);
+static void AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt);
 
 /*
  * CommandIsReadOnly: is an executable query read-only?
@@ -220,6 +221,7 @@ ClassifyUtilityCommandAsReadOnly(Node *parsetree)
 				return COMMAND_IS_NOT_READ_ONLY;
 			}
 
+		case T_AlterSystemWALProhibitState:
 		case T_AlterSystemStmt:
 			{
 				/*
@@ -835,6 +837,11 @@ standard_ProcessUtility(PlannedStmt *pstmt,
 			AlterSystemSetConfigFile((AlterSystemStmt *) parsetree);
 			break;
 
+		case T_AlterSystemWALProhibitState:
+			PreventInTransactionBlock(isTopLevel, "ALTER SYSTEM");
+			AlterSystemSetWALProhibitState((AlterSystemWALProhibitState *) parsetree);
+			break;
+
 		case T_VariableSetStmt:
 			ExecSetVariableStmt((VariableSetStmt *) parsetree, isTopLevel);
 			break;
@@ -2818,6 +2825,7 @@ CreateCommandTag(Node *parsetree)
 			tag = CMDTAG_REFRESH_MATERIALIZED_VIEW;
 			break;
 
+		case T_AlterSystemWALProhibitState:
 		case T_AlterSystemStmt:
 			tag = CMDTAG_ALTER_SYSTEM;
 			break;
@@ -3678,3 +3686,16 @@ GetCommandLogLevel(Node *parsetree)
 
 	return lev;
 }
+
+/*
+ * AlterSystemSetWALProhibitState
+ *
+ * Execute ALTER SYSTEM READ { ONLY | WRITE } statement.
+ */
+static void
+AlterSystemSetWALProhibitState(AlterSystemWALProhibitState *stmt)
+{
+	ereport(ERROR,
+			(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+			 errmsg("ALTER SYSTEM READ { ONLY | WRITE } not implemented")));
+}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 6abcbea9634..cfa492c8d81 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1894,9 +1894,11 @@ psql_completion(const char *text, int start, int end)
 	/* ALTER SERVER <name> VERSION <version> */
 	else if (Matches("ALTER", "SERVER", MatchAny, "VERSION", MatchAny))
 		COMPLETE_WITH("OPTIONS");
-	/* ALTER SYSTEM SET, RESET, RESET ALL */
+	/* ALTER SYSTEM READ, SET, RESET, RESET ALL */
 	else if (Matches("ALTER", "SYSTEM"))
-		COMPLETE_WITH("SET", "RESET");
+		COMPLETE_WITH("SET", "RESET", "READ");
+	else if (Matches("ALTER", "SYSTEM", "READ"))
+		COMPLETE_WITH("ONLY", "WRITE");
 	else if (Matches("ALTER", "SYSTEM", "SET|RESET"))
 		COMPLETE_WITH_QUERY(Query_for_list_of_alter_system_set_vars);
 	else if (Matches("ALTER", "SYSTEM", "SET", MatchAny))
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index caed683ba92..5972800479f 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -411,6 +411,7 @@ typedef enum NodeTag
 	T_RefreshMatViewStmt,
 	T_ReplicaIdentityStmt,
 	T_AlterSystemStmt,
+	T_AlterSystemWALProhibitState,
 	T_CreatePolicyStmt,
 	T_AlterPolicyStmt,
 	T_CreateTransformStmt,
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index dc2bb40926a..6677f8ac470 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -3211,6 +3211,16 @@ typedef struct AlterSystemStmt
 	VariableSetStmt *setstmt;	/* SET subcommand */
 } AlterSystemStmt;
 
+/* ----------------------
+ *		Alter System Read Statement
+ * ----------------------
+ */
+typedef struct AlterSystemWALProhibitState
+{
+	NodeTag		type;
+	bool		walprohibited;
+} AlterSystemWALProhibitState;
+
 /* ----------------------
  *		Cluster Statement (support pbrown's cluster index implementation)
  * ----------------------
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index fb57b8393f1..4ba4ec1cbdc 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -86,6 +86,7 @@ AlterStatsStmt
 AlterSubscriptionStmt
 AlterSubscriptionType
 AlterSystemStmt
+AlterSystemWALProhibitState
 AlterTSConfigType
 AlterTSConfigurationStmt
 AlterTSDictionaryStmt
-- 
2.18.0

From 3ffc2c05902414b950c44d962d16bdab45716ff8 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Tue, 14 Jul 2020 02:30:44 -0400
Subject: [PATCH v12 6/6] WIP - Documentation.

TODOs:

1] Documentation regarding ALTER SYSTEM READ/WRITE
---
 src/backend/access/transam/README | 60 ++++++++++++++++++++++++++++---
 src/backend/storage/page/README   | 12 +++----
 2 files changed, 61 insertions(+), 11 deletions(-)

diff --git a/src/backend/access/transam/README b/src/backend/access/transam/README
index 1edc8180c12..fe7a84f93da 100644
--- a/src/backend/access/transam/README
+++ b/src/backend/access/transam/README
@@ -442,8 +442,8 @@ to be modified.
 2. START_CRIT_SECTION()  (Any error during the next three steps must cause a
 PANIC because the shared buffers will contain unlogged changes, which we
 have to ensure don't get to disk.  Obviously, you should check conditions
-such as whether there's enough free space on the page before you start the
-critical section.)
+such as whether there's WAL write permission and enough free space on the page
+before you start the critical section.)
 
 3. Apply the required changes to the shared buffer(s).
 
@@ -486,6 +486,54 @@ with the incomplete-split flag set, it will finish the interrupted split by
 inserting the key to the parent, before proceeding.
 
 
+Read only system state
+----------------------
+
+The system state when it is not currently possible to insert write-ahead log
+records, either because the system is still in recovery or because the system
+forced to WAL prohibited by executing ALTER SYSTEM READ ONLY.  We have a
+lower-level defense in XLogBeginInsert() and elsewhere to stop us from modifying
+data during recovery when !XLogInsertAllowed(), but if XLogBeginInsert() is
+inside the critical section we must not depend on it to report an error.
+Otherwise, it will cause PANIC as mentioned previously.
+
+We do not reach the point where we try to write WAL during recovery but ALTER
+SYSTEM READ ONLY can be executed anytime by the user to stop WAL writing.  Any
+backends which receive read-only system state transition barrier interrupt need
+to stop WAL writing immediately.  For barrier absorption the backed(s) will kill
+the running transaction which has valid XID indicates that the transaction has
+performed and/or planning WAL write.  The transaction which doesn't acquire
+valid XID yet or operation such VACUUM or CONCURRENT CREATE INDEX which not
+necessary have valid XID for WAL will not be prevented while barrier processing,
+and those might hit the error from XLogBeginInsert() while trying to write WAL
+in read only system state.  To prevent such error from XLogBeginInsert() inside
+the critical section the WAL write permission has to check before
+START_CRIT_SECTION().
+
+To enforce the practice to check WAL permission before entering into critical
+section for the WAL write, we have added an assert check flag that indicates
+permission has been checked before calling XLogBeginInsert().  If not,
+XLogBeginInsert() will have assertion failure.  WAL permission check is not
+mandatory if the XLogBeginInsert() is not inside the critical section where
+throwing the error is acceptable.  To get permission check flag set either
+CheckWALPermitted(), AssertWALPermitted_HaveXID(), or AssertWALPermitted()
+should be called before START_CRIT_SECTION().  This flag automatically resets
+while exiting from the critical section.  The rule to place either of permission
+check routines will be:
+
+	The places where WAL write operation in critical can be expected without
+	having valid XID (e.g vacuum) need to protect by CheckWALPermitted(), so
+	that error can be reported outside before critical section.
+
+	The places where INSERT and UPDATE are expected which are never happened
+	without valid XID can be checked using AssertWALPermitted_HaveXID.  So that
+	non-assert build will not have the checking overhead.
+
+	The places we know that we cannot be reached in the read-only state and may
+	or may not have XID, but need to ensure the permission has been checked on
+	assert enabled build should use AssertWALPermitted().
+
+
 Constructing a WAL record
 -------------------------
 
@@ -531,7 +579,8 @@ Details of the API functions:
 
 void XLogBeginInsert(void)
 
-    Must be called before XLogRegisterBuffer and XLogRegisterData.
+	Must be called before XLogRegisterBuffer and XLogRegisterData.  WAL
+	permission must be check before calling it in a critical section.
 
 void XLogResetInsertion(void)
 
@@ -638,8 +687,9 @@ MarkBufferDirtyHint() to mark the block dirty.
 If the buffer is clean and checksums are in use then MarkBufferDirtyHint()
 inserts an XLOG_FPI_FOR_HINT record to ensure that we take a full page image
 that includes the hint. We do this to avoid a partial page write, when we
-write the dirtied page. WAL is not written during recovery, so we simply skip
-dirtying blocks because of hints when in recovery.
+write the dirtied page. WAL is not written while in read only (i.e. during
+recovery or in WAL prohibit state), so we simply skip dirtying blocks because of
+hints when in recovery.
 
 If you do decide to optimise away a WAL record, then any calls to
 MarkBufferDirty() must be replaced by MarkBufferDirtyHint(),
diff --git a/src/backend/storage/page/README b/src/backend/storage/page/README
index e30d7ac59ad..15f0bb4b7b5 100644
--- a/src/backend/storage/page/README
+++ b/src/backend/storage/page/README
@@ -56,9 +56,9 @@ WAL is a fatal error and prevents further recovery, whereas a checksum failure
 on a normal data block is a hard error but not a critical one for the server,
 even if it is a very bad thing for the user.
 
-New WAL records cannot be written during recovery, so hint bits set during
-recovery must not dirty the page if the buffer is not already dirty, when
-checksums are enabled.  Systems in Hot-Standby mode may benefit from hint bits
-being set, but with checksums enabled, a page cannot be dirtied after setting a
-hint bit (due to the torn page risk). So, it must wait for full-page images
-containing the hint bit updates to arrive from the primary.
+New WAL records cannot be written during recovery or or while in WAL prohibit
+state, so hint bits set during recovery must not dirty the page if the buffer is
+not already dirty, when checksums are enabled.  Systems in Hot-Standby mode may
+benefit from hint bits being set, but with checksums enabled, a page cannot be
+dirtied after setting a hint bit (due to the torn page risk). So, it must wait
+for full-page images containing the hint bit updates to arrive from the primary.
-- 
2.18.0

From 234b46d195a2c3cd71a8467cec26453a2ee06823 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Fri, 19 Jun 2020 06:29:36 -0400
Subject: [PATCH v13 1/3] WIP - Implement wal prohibit state using global
 barrier.

Implementation:

 1. When a user tried to change server state to WAL-Prohibited
    by calling pg_alter_wal_prohibit_state(true) sql function, the current
    state generation to inprogress in shared memory marked and signaled
    checkpointer process.  Checkpointer, noticing that the current state
    generation has WALPROHIBIT_TRANSITION_IN_PROGRESS flag set, does the
    barrier request, and then acknowledges back to the backend who
    requested the state change once the transition has been completed.
    Final state will be updated in control file to make it persistent
    across the system restarts.

 2. When a backend receives the WAL-Prohibited barrier, at that moment if
    it is already in a transaction and the transaction already assigned XID,
    then the backend will be killed by throwing FATAL(XXX: need more discussion
    on this)

 3. Otherwise, if that backend running transaction without valid XID then, we
    don't need to do anything special right now, simply call
    ResetLocalXLogInsertAllowed() so that any future WAL insert in will check
    XLogInsertAllowed() first which set ready only state appropriately.

 4. A new transaction (in an existing or in a new backend) starts as a
    read-only transaction.

 5. Autovacuum launcher as well as checkpointer will not do anything in
    WAL-Prohibited server state until someone wakes us up.  E.g. a backend
    might later on request us to put the system back to read-write.

 6. At shutdown in WAL-Prohibited mode, we'll skip shutdown checkpoint
    and xlog rotation. Starting up again will perform crash recovery(XXX:
    need some discussion on this as well) but the end of recovery checkpoint
    will be skipped and it will be performed when the system changed to
    WAL-Permitted mode.

 7. altering WAL-Prohibited mode is restricted on standby server.

 8. Add system_is_read_only GUC show the system state -- will true when system
    is wal prohibited or in recovery.

=====
TODO:
=====
 1. Commented CheckpointLock acquiring code in CreateCheckPoint() to make
    the latest changes to the checkpointer to process wal prohibit state
    change request ASAP.  We might want to depreciate  CheckpointLock
    completely from CreateCheckPoint(). The same discussion is started on
    pg_hackers[1]. Until then this patch mark as WIP.

 2. Tbc, skipping recovery checkpoint in StartupXLog() is correct?
    And the code right afterwards will work with that? More thought on Robert's
    comment on StartupXLog() changes[2]

====
REF:
====
 1. http://postgr.es/m/CAAJ_b97XnBBfYeSREDJorFsyoD1sHgqnNuCi=02mNQBUMnA=FA@mail.gmail.com
 2. http://postgr.es/m/CA+TgmoZf2by_6kbY0JntGEmrSkj3DxUTNZDXORNasGdsSmkJjA@mail.gmail.com
---
 src/backend/access/transam/Makefile      |   1 +
 src/backend/access/transam/walprohibit.c | 399 +++++++++++++++++++++++
 src/backend/access/transam/xact.c        |  37 ++-
 src/backend/access/transam/xlog.c        | 109 ++++++-
 src/backend/catalog/system_views.sql     |   2 +
 src/backend/postmaster/autovacuum.c      |   9 +-
 src/backend/postmaster/bgwriter.c        |   2 +-
 src/backend/postmaster/checkpointer.c    |  19 ++
 src/backend/postmaster/pgstat.c          |   6 +
 src/backend/storage/ipc/ipci.c           |   6 +
 src/backend/storage/ipc/procsignal.c     |  24 +-
 src/backend/storage/sync/sync.c          |  30 +-
 src/backend/tcop/utility.c               |   1 +
 src/backend/utils/misc/guc.c             |  26 ++
 src/bin/pg_controldata/pg_controldata.c  |   2 +
 src/include/access/walprohibit.h         |  92 ++++++
 src/include/access/xlog.h                |   4 +
 src/include/catalog/pg_control.h         |   3 +
 src/include/catalog/pg_proc.dat          |   4 +
 src/include/pgstat.h                     |   2 +
 src/include/postmaster/bgwriter.h        |   2 +
 src/include/storage/procsignal.h         |   7 +-
 src/tools/pgindent/typedefs.list         |   1 +
 23 files changed, 725 insertions(+), 63 deletions(-)
 create mode 100644 src/backend/access/transam/walprohibit.c
 create mode 100644 src/include/access/walprohibit.h

diff --git a/src/backend/access/transam/Makefile b/src/backend/access/transam/Makefile
index 595e02de722..b5322a69954 100644
--- a/src/backend/access/transam/Makefile
+++ b/src/backend/access/transam/Makefile
@@ -26,6 +26,7 @@ OBJS = \
 	twophase.o \
 	twophase_rmgr.o \
 	varsup.o \
+	walprohibit.o \
 	xact.o \
 	xlog.o \
 	xlogarchive.o \
diff --git a/src/backend/access/transam/walprohibit.c b/src/backend/access/transam/walprohibit.c
new file mode 100644
index 00000000000..831b8c7d8ea
--- /dev/null
+++ b/src/backend/access/transam/walprohibit.c
@@ -0,0 +1,399 @@
+/*-------------------------------------------------------------------------
+ *
+ * walprohibit.c
+ * 		PostgreSQL write-ahead log prohibit states
+ *
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/backend/access/transam/walprohibit.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/walprohibit.h"
+#include "fmgr.h"
+#include "pgstat.h"
+#include "port/atomics.h"
+#include "postmaster/bgwriter.h"
+#include "postmaster/interrupt.h"
+#include "storage/condition_variable.h"
+#include "storage/procsignal.h"
+#include "storage/shmem.h"
+#include "utils/acl.h"
+#include "utils/fmgroids.h"
+#include "utils/fmgrprotos.h"
+
+/*
+ * Shared-memory WAL prohibit state
+ */
+typedef struct WALProhibitStateData
+{
+	/*
+	 * Indicates current WAL prohibit state generation and the last two bits of
+	 * this generation indicates current wal prohibit state.
+	 */
+	pg_atomic_uint32 shared_state_generation;
+
+	/* Signaled when requested WAL prohibit state changes */
+	ConditionVariable walprohibit_cv;
+} WALProhibitStateData;
+
+static WALProhibitStateData *WALProhibitState = NULL;
+
+static void CompleteWALProhibitChange(uint32 cur_state_gen);
+static uint32 GetWALProhibitStateGen(void);
+
+/*
+ * ProcessBarrierWALProhibit()
+ *
+ * Handle WAL prohibit state change request.
+ */
+bool
+ProcessBarrierWALProhibit(void)
+{
+	/*
+	 * Kill off any transactions that have an XID *before* allowing the system
+	 * to go WAL prohibit state.
+	 */
+	if (FullTransactionIdIsValid(GetTopFullTransactionIdIfAny()))
+	{
+		/*
+		 * Should be here only while transiting towards the WAL prohibit state.
+		 */
+		Assert(WALPROHIBIT_CURRENT_STATE(GetWALProhibitStateGen()) ==
+			   WALPROHIBIT_STATE_GOING_READ_ONLY);
+
+		/*
+		 * XXX: Kill off the whole session by throwing FATAL instead of
+		 * killing transaction by throwing ERROR due to following reasons that
+		 * need be thought:
+		 *
+		 * 1. Due to some presents challenges with the wire protocol, we could
+		 * not simply kill of idle transaction.
+		 *
+		 * 2. If we are here in subtransaction then the ERROR will kill the
+		 * current subtransaction only. In the case of invalidations, that
+		 * might be good enough, but for XID assignment it's not, because
+		 * assigning an XID to a subtransaction also causes higher
+		 * sub-transaction levels and the parent transaction to get XIDs.
+		 */
+		ereport(FATAL,
+				(errcode(ERRCODE_ACTIVE_SQL_TRANSACTION),
+				 errmsg("system is now read only"),
+				 errhint("Sessions with open write transactions must be terminated.")));
+	}
+
+	/* Return to "check" state */
+	ResetLocalXLogInsertAllowed();
+
+	return true;
+}
+
+/*
+ * pg_alter_wal_prohibit_state()
+ *
+ * SQL callable function to alter system read write state.
+ */
+Datum
+pg_alter_wal_prohibit_state(PG_FUNCTION_ARGS)
+{
+	bool		walprohibited = PG_GETARG_BOOL(0);
+	uint32		cur_state_gen;
+	uint32		cur_state;
+	uint32		new_transition_state;
+
+	/* Alter WAL prohibit state not allowed during recovery */
+	PreventCommandDuringRecovery("pg_alter_wal_prohibit_state()");
+
+	/*
+	 * It is not a final state since we yet to convey this WAL prohibit state to
+	 * all backend.
+	 */
+	new_transition_state = walprohibited?
+		WALPROHIBIT_STATE_GOING_READ_ONLY :
+		WALPROHIBIT_STATE_GOING_READ_WRITE;
+
+	cur_state_gen = GetWALProhibitStateGen();
+	cur_state = WALPROHIBIT_CURRENT_STATE(cur_state_gen);
+
+	/* Server is already in requested state */
+	if (WALPROHIBIT_NEXT_STATE(new_transition_state) == cur_state)
+		PG_RETURN_VOID();
+
+	/*
+	 * Increment the state generation counter in share memory if next wal
+	 * prohibit state being change not same the requested. Otherwise, wait for
+	 * transition state completion.
+	 */
+	if (new_transition_state != cur_state)
+	{
+		bool		success PG_USED_FOR_ASSERTS_ONLY;
+		uint32		next_state_gen;
+
+		/* Prevent concurrent contrary in progress transition state setting */
+		if (cur_state_gen & WALPROHIBIT_TRANSITION_IN_PROGRESS)
+		{
+			if (cur_state == WALPROHIBIT_STATE_READ_ONLY)
+				ereport(ERROR,
+						(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+						 errmsg("system state transition to read only is already in progress"),
+						 errhint("Try after sometime again.")));
+			else
+				ereport(ERROR,
+						(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+						 errmsg("system state transition to read write is already in progress"),
+						 errhint("Try after sometime again.")));
+		}
+
+		/* Increment the state generation counter in share memory. */
+		next_state_gen = cur_state_gen + 1;
+		success = pg_atomic_compare_exchange_u32(&WALProhibitState->shared_state_generation,
+												 &cur_state_gen, next_state_gen);
+		Assert(success);
+
+		/* Update our local state generation couter as well */
+		cur_state_gen = next_state_gen;
+
+		/* To be sure that any later reads of memory happen strictly after this. */
+		pg_memory_barrier();
+	}
+
+	/* Now must be in the requested transition state */
+	Assert(WALPROHIBIT_CURRENT_STATE(cur_state_gen) == new_transition_state);
+
+	/*
+	 * If in a standalone backend, just do it ourselves.
+	 */
+	if (!IsPostmasterEnvironment)
+	{
+		CompleteWALProhibitChange(cur_state_gen);
+		PG_RETURN_VOID();
+	}
+
+	/* Signal checkpointer process */
+	if (!SendSignalToCheckpointer(SIGUSR1))
+	{
+		ereport(WARNING,
+				(errmsg("could not change system state now"),
+				 errdetail("Checkpointer might not running."),
+				 errhint("The relaunched checkpointer process will automatically complete the system state change.")));
+		PG_RETURN_VOID();	/* no wait */
+	}
+
+	/* Wait for the state generation counter in shared memory to change. */
+	ConditionVariablePrepareToSleep(&WALProhibitState->walprohibit_cv);
+	for (;;)
+	{
+		/* We'll be done once wal prohibit state generation changes */
+		if (GetWALProhibitStateGen() != cur_state_gen)
+			break;
+
+		ConditionVariableSleep(&WALProhibitState->walprohibit_cv,
+							   WAIT_EVENT_WALPROHIBIT_STATE_CHANGE);
+	}
+	ConditionVariableCancelSleep();
+
+	PG_RETURN_VOID();
+}
+
+/*
+ * Is the system still in WAL prohibited state?
+ */
+bool
+IsWALProhibited(void)
+{
+	uint32 		cur_state = WALPROHIBIT_CURRENT_STATE(GetWALProhibitStateGen());
+
+	return (cur_state != WALPROHIBIT_STATE_READ_WRITE &&
+			cur_state != WALPROHIBIT_STATE_GOING_READ_WRITE);
+}
+
+/*
+ * CompleteWALProhibitChange()
+ *
+ * Complete the requested WAL prohibit state transition.
+ */
+static void
+CompleteWALProhibitChange(uint32 cur_state_gen)
+{
+	uint64		barrier_gen;
+	uint32		new_state;
+	bool		wal_prohibited;
+	bool		success PG_USED_FOR_ASSERTS_ONLY;
+
+	/*
+	 * Must be called from checkpointer. Otherwise, it must be single-user
+	 * backend.
+	 */
+	Assert(AmCheckpointerProcess() || !IsPostmasterEnvironment);
+	Assert(cur_state_gen & WALPROHIBIT_TRANSITION_IN_PROGRESS);
+
+	/*
+	 * WAL prohibit state change is initiated. We need to complete the state
+	 * transition by setting requested WAL prohibit state in all backends.
+	 */
+	elog(DEBUG1, "waiting for backends to adopt requested WAL prohibit state change");
+
+	/* Emit global barrier */
+	barrier_gen = EmitProcSignalBarrier(PROCSIGNAL_BARRIER_WALPROHIBIT);
+	WaitForProcSignalBarrier(barrier_gen);
+
+	/*
+	 * Since we do not process barrier for ourself, for now, return to "check"
+	 * state.
+	 */
+	ResetLocalXLogInsertAllowed();
+
+	/* And flush all inserts. */
+	XLogFlush(GetXLogInsertRecPtr());
+
+
+	/* Set the final state */
+	new_state = WALPROHIBIT_NEXT_STATE(cur_state_gen);
+
+	/*
+	 * There won't be any other process for the final state setting so that
+	 * the next final state will be the desired state.
+	 */
+	Assert((GetWALProhibitStateGen() & WALPROHIBIT_TRANSITION_IN_PROGRESS) &&
+		   (WALPROHIBIT_NEXT_STATE(GetWALProhibitStateGen()) == new_state));
+
+	/* Update new state generation in share memory. */
+	success = pg_atomic_compare_exchange_u32(&WALProhibitState->shared_state_generation,
+											 &cur_state_gen, cur_state_gen + 1);
+	Assert(success);
+
+	/* To be sure that any later reads of memory happen strictly after this. */
+	pg_memory_barrier();
+
+	wal_prohibited = (new_state == WALPROHIBIT_STATE_READ_ONLY);
+
+	/* Update the control file to make state persistent */
+	SetControlFileWALProhibitFlag(wal_prohibited);
+
+	if (wal_prohibited)
+		ereport(LOG, (errmsg("system is now read only")));
+	else
+	{
+		/*
+		 * Request checkpoint if the end-of-recovery checkpoint has been skipped
+		 * previously.
+		 */
+		if (RecoveryCheckpointIsSkipped())
+		{
+			RequestCheckpoint(CHECKPOINT_IMMEDIATE);
+			SetRecoveryCheckpointSkippedFlag(false);
+		}
+		ereport(LOG, (errmsg("system is now read write")));
+	}
+
+	/* Wake up all backends waiting on this. */
+	ConditionVariableBroadcast(&WALProhibitState->walprohibit_cv);
+}
+
+/*
+ * ProcessWALProhibitStateChangeRequest()
+ *
+ * Checkpointer will complete wal prohibit state change request.
+ */
+void
+ProcessWALProhibitStateChangeRequest(void)
+{
+	uint32		wal_state_gen;
+
+	if (!AmCheckpointerProcess())
+		return;
+
+	while (1)
+	{
+		wal_state_gen = GetWALProhibitStateGen();
+
+		if (wal_state_gen & WALPROHIBIT_TRANSITION_IN_PROGRESS)
+		{
+			CompleteWALProhibitChange(wal_state_gen);
+			continue; /* check changed state */
+		}
+		else if (WALPROHIBIT_CURRENT_STATE(wal_state_gen) ==
+				 WALPROHIBIT_STATE_READ_ONLY)
+		{
+			int			rc;
+
+			/*
+			 * Don't let Checkpointer process do anything until someone wakes it
+			 * up.  For example a backend might later on request us to put the
+			 * system back to read-write wal prohibit sate.
+			 */
+			rc = WaitLatch(MyLatch, WL_LATCH_SET | WL_POSTMASTER_DEATH, -1,
+						   WAIT_EVENT_WALPROHIBIT_STATE);
+
+			/*
+			 * If the postmaster dies or a shutdown request is received, just
+			 * bail out.
+			 */
+			if (rc & WL_POSTMASTER_DEATH || ShutdownRequestPending)
+				return;
+
+			/* Re-check wal prohibit state */
+			continue;
+		}
+
+		Assert(WALPROHIBIT_CURRENT_STATE(wal_state_gen) ==
+			   WALPROHIBIT_STATE_READ_WRITE);
+
+		break; /* Done */
+	}
+}
+
+/*
+ * GetWALProhibitStateGen()
+ *
+ * Atomically return the current server WAL prohibited state generation.
+ */
+static uint32
+GetWALProhibitStateGen(void)
+{
+	return pg_atomic_read_u32(&WALProhibitState->shared_state_generation);
+}
+
+/*
+ * WALProhibitStateGenerationInit()
+ *
+ * Initialization of shared wal prohibit state generation.
+ */
+void
+WALProhibitStateGenerationInit(bool wal_prohibited)
+{
+	uint32	new_state;
+
+	Assert(AmStartupProcess() || !IsPostmasterEnvironment);
+
+	new_state = wal_prohibited ?
+		WALPROHIBIT_STATE_READ_ONLY : WALPROHIBIT_STATE_READ_WRITE;
+
+	pg_atomic_init_u32(&WALProhibitState->shared_state_generation, new_state);
+}
+
+/*
+ * WALProhibitStateShmemInit()
+ *
+ * Initialization of shared memory for WAL prohibit state.
+ */
+void
+WALProhibitStateShmemInit(void)
+{
+	bool		found;
+
+	WALProhibitState = (WALProhibitStateData *)
+		ShmemInitStruct("WAL Prohibit State",
+						sizeof(WALProhibitStateData),
+						&found);
+
+	if (!found)
+	{
+		/* First time through ... */
+		memset(WALProhibitState, 0, sizeof(WALProhibitStateData));
+		ConditionVariableInit(&WALProhibitState->walprohibit_cv);
+	}
+}
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index a2068e3fd45..0de63af6365 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -1962,23 +1962,28 @@ StartTransaction(void)
 	Assert(s->prevSecContext == 0);
 
 	/*
-	 * Make sure we've reset xact state variables
+	 * Reset xact state variables.
 	 *
-	 * If recovery is still in progress, mark this transaction as read-only.
-	 * We have lower level defences in XLogInsert and elsewhere to stop us
-	 * from modifying data during recovery, but this gives the normal
-	 * indication to the user that the transaction is read-only.
-	 */
-	if (RecoveryInProgress())
-	{
-		s->startedInRecovery = true;
-		XactReadOnly = true;
-	}
-	else
-	{
-		s->startedInRecovery = false;
-		XactReadOnly = DefaultXactReadOnly;
-	}
+	 * If it is not currently possible to insert write-ahead log records,
+	 * either because we are still in recovery or because ALTER SYSTEM READ
+	 * ONLY has been executed, force this to be a read-only transaction.
+	 * We have lower level defences in XLogBeginInsert() and elsewhere to stop
+	 * us from modifying data during recovery when !XLogInsertAllowed(), but
+	 * this gives the normal indication to the user that the transaction is
+	 * read-only.
+	 *
+	 * On the other hand, we only need to set the startedInRecovery flag when
+	 * the transaction started during recovery, and not when WAL is otherwise
+	 * prohibited. This information is used by RelationGetIndexScan() to
+	 * decide whether to permit (1) relying on existing killed-tuple markings
+	 * and (2) further killing of index tuples. Even when WAL is prohibited
+	 * on the master, it's still the master, so the former is OK; and since
+	 * killing index tuples doesn't generate WAL, the latter is also OK.
+	 * See comments in RelationGetIndexScan() and MarkBufferDirtyHint().
+	 */
+	XactReadOnly = DefaultXactReadOnly || !XLogInsertAllowed();
+	s->startedInRecovery = RecoveryInProgress();
+
 	XactDeferrable = DefaultXactDeferrable;
 	XactIsoLevel = DefaultXactIsoLevel;
 	forceSyncCommit = false;
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 470e113b331..d1f0ce9a3ba 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -31,6 +31,7 @@
 #include "access/timeline.h"
 #include "access/transam.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog_internal.h"
 #include "access/xlogarchive.h"
@@ -247,9 +248,10 @@ static bool LocalPromoteIsTriggered = false;
  *		0: unconditionally not allowed to insert XLOG
  *		-1: must check RecoveryInProgress(); disallow until it is false
  * Most processes start with -1 and transition to 1 after seeing that recovery
- * is not in progress.  But we can also force the value for special cases.
- * The coding in XLogInsertAllowed() depends on the first two of these states
- * being numerically the same as bool true and false.
+ * is not in progress or the server state is not a WAL prohibited state.  But
+ * we can also force the value for special cases.  The coding in
+ * XLogInsertAllowed() depends on the first two of these states being
+ * numerically the same as bool true and false.
  */
 static int	LocalXLogInsertAllowed = -1;
 
@@ -734,6 +736,13 @@ typedef struct XLogCtlData
 	 */
 	XLogRecPtr	lastFpwDisableRecPtr;
 
+	/*
+	 * lastRecoveryCheckpointSkipped indicates if the last recovery checkpoint
+	 * is skipped. Lock protection is not needed since it isn't going to be read
+	 * and/or updated concurrently.
+	 */
+	bool		lastRecoveryCheckpointSkipped;
+
 	slock_t		info_lck;		/* locks shared variables shown above */
 } XLogCtlData;
 
@@ -6220,6 +6229,25 @@ SetCurrentChunkStartTime(TimestampTz xtime)
 	SpinLockRelease(&XLogCtl->info_lck);
 }
 
+/*
+ * Set or unset flag to indicating that the last checkpoint has been skipped.
+ */
+void
+SetRecoveryCheckpointSkippedFlag(bool ChkptSkip)
+{
+	XLogCtl->lastRecoveryCheckpointSkipped = ChkptSkip;
+}
+
+/*
+ * Return value of lastRecoveryCheckpointSkipped flag.
+ */
+bool
+RecoveryCheckpointIsSkipped(void)
+{
+	/* Read the latest value */
+	return ((volatile XLogCtlData *) XLogCtl)->lastRecoveryCheckpointSkipped;
+}
+
 /*
  * Fetch timestamp of latest processed commit/abort record.
  * Startup process maintains an accurate local copy in XLogReceiptTime
@@ -7782,6 +7810,12 @@ StartupXLOG(void)
 	XLogCtl->LogwrtRqst.Write = EndOfLog;
 	XLogCtl->LogwrtRqst.Flush = EndOfLog;
 
+	/*
+	 * Before enabling WAL insertion, initialize WAL prohibit state in shared
+	 * memory that will decide the further WAL insert should be allowed or not.
+	 */
+	WALProhibitStateGenerationInit(ControlFile->wal_prohibited);
+
 	/*
 	 * Update full_page_writes in shared memory and write an XLOG_FPW_CHANGE
 	 * record before resource manager writes cleanup WAL records or checkpoint
@@ -7792,7 +7826,17 @@ StartupXLOG(void)
 	UpdateFullPageWrites();
 	LocalXLogInsertAllowed = -1;
 
-	if (InRecovery)
+	/*
+	 * Skip end-of-recovery checkpoint if the system is in WAL prohibited state.
+	 */
+	if (ControlFile->wal_prohibited && InRecovery)
+	{
+		SetRecoveryCheckpointSkippedFlag(true);
+
+		ereport(LOG,
+				(errmsg("skipping startup checkpoint because the system is read only")));
+	}
+	else if (InRecovery)
 	{
 		/*
 		 * Perform a checkpoint to update all our recovery activity to disk.
@@ -7988,6 +8032,7 @@ StartupXLOG(void)
 	 */
 	LocalSetXLogInsertAllowed();
 	XLogReportParameters();
+	LocalXLogInsertAllowed = -1;
 
 	/*
 	 * Local WAL inserts enabled, so it's time to finish initialization of
@@ -8038,6 +8083,16 @@ StartupXLOG(void)
 		RequestCheckpoint(CHECKPOINT_FORCE);
 }
 
+/* Set ControlFile's WAL prohibit flag */
+void
+SetControlFileWALProhibitFlag(bool wal_prohibited)
+{
+	LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
+	ControlFile->wal_prohibited = wal_prohibited;
+	UpdateControlFile();
+	LWLockRelease(ControlFileLock);
+}
+
 /*
  * Checks if recovery has reached a consistent state. When consistency is
  * reached and we have a valid starting standby snapshot, tell postmaster
@@ -8253,9 +8308,9 @@ HotStandbyActiveInReplay(void)
 /*
  * Is this process allowed to insert new WAL records?
  *
- * Ordinarily this is essentially equivalent to !RecoveryInProgress().
- * But we also have provisions for forcing the result "true" or "false"
- * within specific processes regardless of the global state.
+ * Ordinarily this is essentially equivalent to !RecoveryInProgress() and
+ * !IsWALProhibited().  But we also have provisions for forcing the result
+ * "true" or "false" within specific processes regardless of the global state.
  */
 bool
 XLogInsertAllowed(void)
@@ -8274,9 +8329,20 @@ XLogInsertAllowed(void)
 	if (RecoveryInProgress())
 		return false;
 
+	/* Or, in WAL prohibited state */
+	if (IsWALProhibited())
+	{
+		/*
+		 * Set it to "unconditionally false" to avoid checking until it gets
+		 * reset.
+		 */
+		LocalXLogInsertAllowed = 0;
+		return false;
+	}
+
 	/*
-	 * On exit from recovery, reset to "unconditionally true", since there is
-	 * no need to keep checking.
+	 * On exit from recovery or WAL prohibited state, reset to "unconditionally
+	 * true", since there is no need to keep checking.
 	 */
 	LocalXLogInsertAllowed = 1;
 	return true;
@@ -8298,6 +8364,12 @@ LocalSetXLogInsertAllowed(void)
 	InitXLOGAccess();
 }
 
+void
+ResetLocalXLogInsertAllowed(void)
+{
+	LocalXLogInsertAllowed = -1;
+}
+
 /*
  * Subroutine to try to fetch and validate a prior checkpoint record.
  *
@@ -8587,9 +8659,13 @@ ShutdownXLOG(int code, Datum arg)
 	 */
 	WalSndWaitStopping();
 
+	/*
+	 * The restartpoint, checkpoint, or xlog rotation will be performed if the
+	 * WAL writing is permitted.
+	 */
 	if (RecoveryInProgress())
 		CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
-	else
+	else if (XLogInsertAllowed())
 	{
 		/*
 		 * If archiving is enabled, rotate the last XLOG file so that all the
@@ -8602,6 +8678,9 @@ ShutdownXLOG(int code, Datum arg)
 
 		CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
 	}
+	else
+		ereport(LOG,
+			   (errmsg("skipping shutdown checkpoint because the system is read only")));
 }
 
 /*
@@ -8870,7 +8949,11 @@ CreateCheckPoint(int flags)
 	 * only one process that is allowed to issue checkpoints at any given
 	 * time.)
 	 */
-	LWLockAcquire(CheckpointLock, LW_EXCLUSIVE);
+	//======================================================================================
+	// TODO: Tbc, the discussion on this lock requirement is in progress at
+	// http://postgr.es/m/CAAJ_b97XnBBfYeSREDJorFsyoD1sHgqnNuCi=02mNQBUMnA=FA@mail.gmail.com
+	//======================================================================================
+	//LWLockAcquire(CheckpointLock, LW_EXCLUSIVE);
 
 	/*
 	 * Prepare to accumulate statistics.
@@ -8941,7 +9024,7 @@ CreateCheckPoint(int flags)
 		if (last_important_lsn == ControlFile->checkPoint)
 		{
 			WALInsertLockRelease();
-			LWLockRelease(CheckpointLock);
+			//LWLockRelease(CheckpointLock);			TODO: Tbc
 			END_CRIT_SECTION();
 			ereport(DEBUG1,
 					(errmsg("checkpoint skipped because system is idle")));
@@ -9242,7 +9325,7 @@ CreateCheckPoint(int flags)
 									 CheckpointStats.ckpt_segs_removed,
 									 CheckpointStats.ckpt_segs_recycled);
 
-	LWLockRelease(CheckpointLock);
+	//LWLockRelease(CheckpointLock);	TODO: Tbc
 }
 
 /*
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index fa58afd9d78..b7c83f4b9e4 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1525,6 +1525,8 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
 REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
 REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
 
+REVOKE EXECUTE ON FUNCTION pg_alter_wal_prohibit_state(bool) FROM public;
+
 --
 -- We also set up some things as accessible to standard roles.
 --
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 47e60ca5613..2d14b448eb5 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -700,10 +700,13 @@ AutoVacLauncherMain(int argc, char *argv[])
 
 		/*
 		 * There are some conditions that we need to check before trying to
-		 * start a worker.  First, we need to make sure that there is a worker
-		 * slot available.  Second, we need to make sure that no other worker
-		 * failed while starting up.
+		 * start a worker.  First, the system is not read only i.e. wal writes
+		 * allowed.  Second, we need to make sure that there is a worker slot
+		 * available.  Third, we need to make sure that no other worker failed
+		 * while starting up.
 		 */
+		if (!XLogInsertAllowed())
+			continue;
 
 		current_time = GetCurrentTimestamp();
 		LWLockAcquire(AutovacuumLock, LW_SHARED);
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index 715d5195bb6..5157237731c 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -278,7 +278,7 @@ BackgroundWriterMain(void)
 		 * Checkpointer, when active, is barely ever in its mainloop and thus
 		 * makes it hard to log regularly.
 		 */
-		if (XLogStandbyInfoActive() && !RecoveryInProgress())
+		if (XLogStandbyInfoActive() && XLogInsertAllowed())
 		{
 			TimestampTz timeout = 0;
 			TimestampTz now = GetCurrentTimestamp();
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 54a818bf611..033f8a7bdd9 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -39,6 +39,7 @@
 #include <sys/time.h>
 #include <time.h>
 
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xlog_internal.h"
 #include "libpq/pqsignal.h"
@@ -351,6 +352,7 @@ CheckpointerMain(void)
 		 */
 		AbsorbSyncRequests();
 		HandleCheckpointerInterrupts();
+		ProcessWALProhibitStateChangeRequest();
 
 		/*
 		 * Detect a pending checkpoint request by checking whether the flags
@@ -688,6 +690,9 @@ CheckpointWriteDelay(int flags, double progress)
 	if (!AmCheckpointerProcess())
 		return;
 
+	/* Check for wal prohibit state change request */
+	ProcessWALProhibitStateChangeRequest();
+
 	/*
 	 * Perform the usual duties and take a nap, unless we're behind schedule,
 	 * in which case we just try to catch up as quickly as possible.
@@ -1335,3 +1340,17 @@ FirstCallSinceLastCheckpoint(void)
 
 	return FirstCall;
 }
+
+/*
+ * SendSignalToCheckpointer allows a process to send a signal to the checkpoint process.
+ */
+bool
+SendSignalToCheckpointer(int signum)
+{
+	if (CheckpointerShmem->checkpointer_pid == 0)
+		return false;
+
+	if (kill(CheckpointerShmem->checkpointer_pid, signum) != 0)
+		return false;
+	return true; /* Signaled checkpointer successfully */
+}
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index f75b52719dd..63d52825497 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -4307,6 +4307,12 @@ pgstat_get_wait_io(WaitEventIO w)
 		case WAIT_EVENT_REPLICATION_SLOT_WRITE:
 			event_name = "ReplicationSlotWrite";
 			break;
+		case WAIT_EVENT_WALPROHIBIT_STATE:
+			event_name = "SystemWALProhibitState";
+			break;
+		case WAIT_EVENT_WALPROHIBIT_STATE_CHANGE:
+			event_name = "SystemWALProhibitStateChange";
+			break;
 		case WAIT_EVENT_SLRU_FLUSH_SYNC:
 			event_name = "SLRUFlushSync";
 			break;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index f9bbe97b507..c3c5ec641cf 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -22,6 +22,7 @@
 #include "access/subtrans.h"
 #include "access/syncscan.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "commands/async.h"
 #include "miscadmin.h"
 #include "pgstat.h"
@@ -222,6 +223,11 @@ CreateSharedMemoryAndSemaphores(void)
 	MultiXactShmemInit();
 	InitBufferPool();
 
+	/*
+	 * Set up wal probibit shared state
+	 */
+	WALProhibitStateShmemInit();
+
 	/*
 	 * Set up lock manager
 	 */
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index c43cdd685b4..31383a11d08 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -18,6 +18,7 @@
 #include <unistd.h>
 
 #include "access/parallel.h"
+#include "access/walprohibit.h"
 #include "port/pg_bitutils.h"
 #include "commands/async.h"
 #include "miscadmin.h"
@@ -98,7 +99,6 @@ static volatile ProcSignalSlot *MyProcSignalSlot = NULL;
 static bool CheckProcSignal(ProcSignalReason reason);
 static void CleanupProcSignalState(int status, Datum arg);
 static void ResetProcSignalBarrierBits(uint32 flags);
-static bool ProcessBarrierPlaceholder(void);
 
 /*
  * ProcSignalShmemSize
@@ -538,8 +538,8 @@ ProcessProcSignalBarrier(void)
 				type = (ProcSignalBarrierType) pg_rightmost_one_pos32(flags);
 				switch (type)
 				{
-					case PROCSIGNAL_BARRIER_PLACEHOLDER:
-						processed = ProcessBarrierPlaceholder();
+					case PROCSIGNAL_BARRIER_WALPROHIBIT:
+						processed = ProcessBarrierWALProhibit();
 						break;
 				}
 
@@ -604,24 +604,6 @@ ResetProcSignalBarrierBits(uint32 flags)
 	InterruptPending = true;
 }
 
-static bool
-ProcessBarrierPlaceholder(void)
-{
-	/*
-	 * XXX. This is just a placeholder until the first real user of this
-	 * machinery gets committed. Rename PROCSIGNAL_BARRIER_PLACEHOLDER to
-	 * PROCSIGNAL_BARRIER_SOMETHING_ELSE where SOMETHING_ELSE is something
-	 * appropriately descriptive. Get rid of this function and instead have
-	 * ProcessBarrierSomethingElse. Most likely, that function should live in
-	 * the file pertaining to that subsystem, rather than here.
-	 *
-	 * The return value should be 'true' if the barrier was successfully
-	 * absorbed and 'false' if not. Note that returning 'false' can lead to
-	 * very frequent retries, so try hard to make that an uncommon case.
-	 */
-	return true;
-}
-
 /*
  * CheckProcSignal - check to see if a particular reason has been
  * signaled, and clear the signal flag.  Should be called after receiving
diff --git a/src/backend/storage/sync/sync.c b/src/backend/storage/sync/sync.c
index fe143151cc5..1c7b40563b5 100644
--- a/src/backend/storage/sync/sync.c
+++ b/src/backend/storage/sync/sync.c
@@ -21,6 +21,7 @@
 #include "access/commit_ts.h"
 #include "access/clog.h"
 #include "access/multixact.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xlogutils.h"
 #include "commands/tablespace.h"
@@ -236,10 +237,17 @@ SyncPostCheckpoint(void)
 		pfree(entry);
 
 		/*
-		 * As in ProcessSyncRequests, we don't want to stop absorbing fsync
+		 * As in ProcessSyncRequests, we don't want to stop wal prohibit change
 		 * requests for a long time when there are many deletions to be done.
-		 * We can safely call AbsorbSyncRequests() at this point in the loop
-		 * (note it might try to delete list entries).
+		 * It needs to be check and processed by checkpointer as soon as
+		 * possible.
+		 */
+		ProcessWALProhibitStateChangeRequest();
+
+		/*
+		 * Similarly, we don't want to stop absorbing fsync requests for the
+		 * long time.  We can safely call AbsorbSyncRequests() at this point in
+		 * the loop (note it might try to delete list entries).
 		 */
 		if (--absorb_counter <= 0)
 		{
@@ -278,6 +286,9 @@ ProcessSyncRequests(void)
 	if (!pendingOps)
 		elog(ERROR, "cannot sync without a pendingOps table");
 
+	/* Check for wal prohibit state change request for checkpointer */
+	ProcessWALProhibitStateChangeRequest();
+
 	/*
 	 * If we are in the checkpointer, the sync had better include all fsync
 	 * requests that were queued by backends up to this point.  The tightest
@@ -336,6 +347,13 @@ ProcessSyncRequests(void)
 	{
 		int			failures;
 
+		/*
+		 * Don't want to stop wal prohibit change requests for a long time when
+		 * there are many fsync requests to be processed.  It needs to be check
+		 * and processed by checkpointer as soon as possible.
+		 */
+		ProcessWALProhibitStateChangeRequest();
+
 		/*
 		 * If the entry is new then don't process it this time; it is new.
 		 * Note "continue" bypasses the hash-remove call at the bottom of the
@@ -422,6 +440,12 @@ ProcessSyncRequests(void)
 							 errmsg("could not fsync file \"%s\" but retrying: %m",
 									path)));
 
+				/*
+				 * For the same reason mentioned previously for the wal prohibit
+				 * state change request check.
+				 */
+				ProcessWALProhibitStateChangeRequest();
+
 				/*
 				 * Absorb incoming requests and check to see if a cancel
 				 * arrived for this relation fork.
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 1d81071c357..a5f8ced59e4 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -19,6 +19,7 @@
 #include "access/htup_details.h"
 #include "access/reloptions.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "catalog/catalog.h"
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 17579eeaca9..6d9c23f2d9f 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -227,6 +227,7 @@ static bool check_recovery_target_lsn(char **newval, void **extra, GucSource sou
 static void assign_recovery_target_lsn(const char *newval, void *extra);
 static bool check_primary_slot_name(char **newval, void **extra, GucSource source);
 static bool check_default_with_oids(bool *newval, void **extra, GucSource source);
+static const char *show_system_is_read_only(void);
 
 /* Private functions in guc-file.l that need to be called from guc.c */
 static ConfigVariable *ProcessConfigFileInternal(GucContext context,
@@ -618,6 +619,7 @@ static char *recovery_target_string;
 static char *recovery_target_xid_string;
 static char *recovery_target_name_string;
 static char *recovery_target_lsn_string;
+static bool system_is_read_only;
 
 
 /* should be static, but commands/variable.c needs to get at this */
@@ -2048,6 +2050,18 @@ static struct config_bool ConfigureNamesBool[] =
 		NULL, NULL, NULL
 	},
 
+	{
+		/* Not for general use */
+		{"system_is_read_only", PGC_INTERNAL, WAL,
+			gettext_noop("Shows whether the system is read only."),
+			NULL,
+			GUC_NO_RESET_ALL | GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE
+		},
+		&system_is_read_only,
+		false,
+		NULL, NULL, show_system_is_read_only
+	},
+
 	/* End-of-list marker */
 	{
 		{NULL, 0, 0, NULL, NULL}, NULL, false, NULL, NULL, NULL
@@ -12218,4 +12232,16 @@ check_default_with_oids(bool *newval, void **extra, GucSource source)
 	return true;
 }
 
+/*
+ * NB: The return string should be the same as the _ShowOption() for boolean
+ * type.
+ */
+static const char *
+show_system_is_read_only(void)
+{
+	if (!XLogInsertAllowed())
+		return "on";
+	return "off";
+}
+
 #include "guc-file.c"
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index 3e00ac0f701..922cd9641d8 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -290,6 +290,8 @@ main(int argc, char *argv[])
 		   (uint32) ControlFile->backupEndPoint);
 	printf(_("End-of-backup record required:        %s\n"),
 		   ControlFile->backupEndRequired ? _("yes") : _("no"));
+	printf(_("WAL write prohibited:                 %s\n"),
+		   ControlFile->wal_prohibited ? _("yes") : _("no"));
 	printf(_("wal_level setting:                    %s\n"),
 		   wal_level_str(ControlFile->wal_level));
 	printf(_("wal_log_hints setting:                %s\n"),
diff --git a/src/include/access/walprohibit.h b/src/include/access/walprohibit.h
new file mode 100644
index 00000000000..01785d293db
--- /dev/null
+++ b/src/include/access/walprohibit.h
@@ -0,0 +1,92 @@
+/*
+ * walprohibit.h
+ *
+ * PostgreSQL write-ahead log prohibit states
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/include/access/walprohibit.h
+ */
+#ifndef WALPROHIBIT_H
+#define WALPROHIBIT_H
+
+#include "access/xact.h"
+#include "access/xlog.h"
+#include "miscadmin.h"
+#include "nodes/parsenodes.h"
+
+extern bool ProcessBarrierWALProhibit(void);
+extern void MarkCheckPointSkippedInWalProhibitState(void);
+extern void WALProhibitStateGenerationInit(bool wal_prohibited);
+extern void WALProhibitStateShmemInit(void);
+extern bool IsWALProhibited(void);
+extern void ProcessWALProhibitStateChangeRequest(void);
+
+/*
+ * The WAL Prohibit States.
+ *
+ * 	The odd number represents the transition state and whereas the even number
+ * 	represents the final state.  These states can be distinguished by checking
+ * 	the 0th bits aka transition bit.
+ */
+#define	WALPROHIBIT_STATE_READ_WRITE		(uint32) 0	/* WAL permitted */
+#define	WALPROHIBIT_STATE_GOING_READ_ONLY	(uint32) 1
+#define	WALPROHIBIT_STATE_READ_ONLY			(uint32) 2	/* WAL prohibited */
+#define	WALPROHIBIT_STATE_GOING_READ_WRITE	(uint32) 3
+
+/* The transition bit to distinguish states.  */
+#define	WALPROHIBIT_TRANSITION_IN_PROGRESS	((uint32) 1 << 0)
+
+/* Extract last two bits */
+#define	WALPROHIBIT_CURRENT_STATE(stateGeneration)	\
+	((uint32)(stateGeneration) & ((uint32) ((1 << 2) - 1)))
+#define	WALPROHIBIT_NEXT_STATE(stateGeneration)	\
+	WALPROHIBIT_CURRENT_STATE((stateGeneration + 1))
+
+/* Never reaches when WAL is prohibited. */
+static inline void
+AssertWALPermitted(void)
+{
+	/*
+	 * Recovery in the startup process never is in wal prohibited state.
+	 */
+	Assert(InRecovery || XLogInsertAllowed());
+
+#ifdef USE_ASSERT_CHECKING
+	walpermit_checked_state = WALPERMIT_CHECKED;
+#endif
+}
+
+/*
+ * XID-bearing transactions are killed off by "ALTER SYSTEM READ ONLY", so any
+ * part of the code that can only be reached with an XID assigned is never
+ * reached when WAL is prohibited.
+ */
+static inline void
+AssertWALPermittedHaveXID(void)
+{
+	/* Must be performing an INSERT, UPDATE or DELETE, so we'll have an XID */
+	Assert(FullTransactionIdIsValid(GetTopFullTransactionIdIfAny()));
+	AssertWALPermitted();
+}
+
+/*
+ * In opposite to the above assertion if a transaction doesn't have valid XID
+ * (e.g. VACUUM) then it won't be killed while changing the system state to WAL
+ * prohibited.  Therefore, we need to explicitly error out before entering into
+ * the critical section.
+ */
+static inline void
+CheckWALPermitted(void)
+{
+	if (!XLogInsertAllowed())
+		ereport(ERROR,
+				(errcode(ERRCODE_READ_ONLY_SQL_TRANSACTION),
+				 errmsg("system is now read only")));
+
+#ifdef USE_ASSERT_CHECKING
+	walpermit_checked_state = WALPERMIT_CHECKED;
+#endif
+}
+
+#endif							/* WALPROHIBIT_H */
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 75ec1073bd0..9857ab05c43 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -306,6 +306,7 @@ extern RecoveryState GetRecoveryState(void);
 extern bool HotStandbyActive(void);
 extern bool HotStandbyActiveInReplay(void);
 extern bool XLogInsertAllowed(void);
+extern void ResetLocalXLogInsertAllowed(void);
 extern void GetXLogReceiptTime(TimestampTz *rtime, bool *fromStream);
 extern XLogRecPtr GetXLogReplayRecPtr(TimeLineID *replayTLI);
 extern XLogRecPtr GetXLogInsertRecPtr(void);
@@ -314,6 +315,8 @@ extern bool RecoveryIsPaused(void);
 extern void SetRecoveryPause(bool recoveryPause);
 extern TimestampTz GetLatestXTime(void);
 extern TimestampTz GetCurrentChunkReplayStartTime(void);
+extern void SetRecoveryCheckpointSkippedFlag(bool ChkptSkip);
+extern bool RecoveryCheckpointIsSkipped(void);
 
 extern void UpdateControlFile(void);
 extern uint64 GetSystemIdentifier(void);
@@ -325,6 +328,7 @@ extern void XLOGShmemInit(void);
 extern void BootStrapXLOG(void);
 extern void LocalProcessControlFile(bool reset);
 extern void StartupXLOG(void);
+extern void SetControlFileWALProhibitFlag(bool wal_prohibited);
 extern void ShutdownXLOG(int code, Datum arg);
 extern void InitXLOGAccess(void);
 extern void CreateCheckPoint(int flags);
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index e3f48158ce7..f6a1f3b9826 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -182,6 +182,9 @@ typedef struct ControlFileData
 	int			max_locks_per_xact;
 	bool		track_commit_timestamp;
 
+	/* WAL prohibited determines if the WAL insert is allowed or not. */
+	bool		wal_prohibited;
+
 	/*
 	 * This data is used to check for hardware-architecture compatibility of
 	 * the database and the backend executable.  We need not check endianness
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index b5f52d4e4a3..40bea0f4271 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11367,6 +11367,10 @@
   proname => 'pg_partition_root', prorettype => 'regclass',
   proargtypes => 'regclass', prosrc => 'pg_partition_root' },
 
+{ oid => '4543', descr => 'alter system read write state',
+  proname => 'pg_alter_wal_prohibit_state', prorettype => 'void',
+  proargtypes => 'bool', prosrc => 'pg_alter_wal_prohibit_state' },
+
 { oid => '4350', descr => 'Unicode normalization',
   proname => 'normalize', prorettype => 'text', proargtypes => 'text text',
   prosrc => 'unicode_normalize_func' },
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 724068cf87e..8f4fc4f1e15 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -1067,6 +1067,8 @@ typedef enum
 	WAIT_EVENT_REPLICATION_SLOT_RESTORE_SYNC,
 	WAIT_EVENT_REPLICATION_SLOT_SYNC,
 	WAIT_EVENT_REPLICATION_SLOT_WRITE,
+	WAIT_EVENT_WALPROHIBIT_STATE,
+	WAIT_EVENT_WALPROHIBIT_STATE_CHANGE,
 	WAIT_EVENT_SLRU_FLUSH_SYNC,
 	WAIT_EVENT_SLRU_READ,
 	WAIT_EVENT_SLRU_SYNC,
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index c430b1b2366..bee495f05da 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -42,4 +42,6 @@ extern void CheckpointerShmemInit(void);
 
 extern bool FirstCallSinceLastCheckpoint(void);
 
+extern bool SendSignalToCheckpointer(int signum);
+
 #endif							/* _BGWRITER_H */
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 4ae7dc33b8e..9e834247871 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -48,12 +48,7 @@ typedef enum
 
 typedef enum
 {
-	/*
-	 * XXX. PROCSIGNAL_BARRIER_PLACEHOLDER should be replaced when the first
-	 * real user of the ProcSignalBarrier mechanism is added. It's just here
-	 * for now because we can't have an empty enum.
-	 */
-	PROCSIGNAL_BARRIER_PLACEHOLDER = 0
+	PROCSIGNAL_BARRIER_WALPROHIBIT = 0
 } ProcSignalBarrierType;
 
 /*
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 943142ced8c..f295b05e760 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2697,6 +2697,7 @@ WALAvailability
 WALInsertLock
 WALInsertLockPadded
 WALOpenSegment
+WALProhibitStateData
 WALReadError
 WALSegmentCloseCB
 WALSegmentContext
-- 
2.18.0

From 6e0a961b7e39081ad48f123d29c18c4dc963507c Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Mon, 27 Jul 2020 02:13:36 -0400
Subject: [PATCH v13 2/3] Error or Assert before START_CRIT_SECTION for WAL
 write

Based on the following criteria added the Assert or the Error when system is
prohibited:

 - Added ERROR for the function which can be reachable without valid XID in case
   of VACUUM or CONCURRENT CREATE INDEX.  For that added common static inline
   function CheckWALPermitted().
 - Added ASSERT for the function which cannot be reached without valid XID;
   Assert to ensure XID validation. For that added AssertWALPermitted_HaveXID().

To enforce the rule to have aforesaid assert or error check before entering a
critical section for WAL write, a new walpermit_checked_state assert only flag
is added. If these check is missing then XLogBeginInsert() will have an
assertion if it is in critical section.

If we are not doing WAL insert inside the critical section then the above
checking is not necessary, we can rely on XLogBeginInsert() for that check &
report an error.
---
 contrib/pg_surgery/heap_surgery.c         | 10 +++++--
 src/backend/access/brin/brin.c            |  4 +++
 src/backend/access/brin/brin_pageops.c    | 21 ++++++++++++--
 src/backend/access/brin/brin_revmap.c     | 10 ++++++-
 src/backend/access/gin/ginbtree.c         | 15 ++++++++--
 src/backend/access/gin/gindatapage.c      | 18 ++++++++++--
 src/backend/access/gin/ginfast.c          | 11 ++++++--
 src/backend/access/gin/gininsert.c        |  4 +++
 src/backend/access/gin/ginutil.c          |  9 +++++-
 src/backend/access/gin/ginvacuum.c        | 11 +++++++-
 src/backend/access/gist/gist.c            | 25 +++++++++++++----
 src/backend/access/gist/gistvacuum.c      | 13 +++++++--
 src/backend/access/hash/hash.c            | 19 +++++++++++--
 src/backend/access/hash/hashinsert.c      |  9 +++++-
 src/backend/access/hash/hashovfl.c        | 22 ++++++++++++---
 src/backend/access/hash/hashpage.c        |  9 ++++++
 src/backend/access/heap/heapam.c          | 26 ++++++++++++++++-
 src/backend/access/heap/pruneheap.c       | 12 ++++++--
 src/backend/access/heap/vacuumlazy.c      | 18 ++++++++++--
 src/backend/access/heap/visibilitymap.c   | 22 +++++++++++++--
 src/backend/access/nbtree/nbtdedup.c      |  3 ++
 src/backend/access/nbtree/nbtinsert.c     | 10 ++++++-
 src/backend/access/nbtree/nbtpage.c       | 34 ++++++++++++++++++++---
 src/backend/access/spgist/spgdoinsert.c   | 13 +++++++++
 src/backend/access/spgist/spgvacuum.c     | 22 +++++++++++++--
 src/backend/access/transam/multixact.c    |  5 +++-
 src/backend/access/transam/twophase.c     |  9 ++++++
 src/backend/access/transam/varsup.c       |  4 +++
 src/backend/access/transam/walprohibit.c  | 10 +++++++
 src/backend/access/transam/xact.c         |  6 ++++
 src/backend/access/transam/xlog.c         | 26 ++++++++++++-----
 src/backend/access/transam/xloginsert.c   | 13 +++++++--
 src/backend/commands/sequence.c           | 16 +++++++++++
 src/backend/commands/variable.c           |  9 ++++--
 src/backend/postmaster/checkpointer.c     |  4 +++
 src/backend/storage/buffer/bufmgr.c       | 10 ++++---
 src/backend/storage/freespace/freespace.c | 10 ++++++-
 src/backend/storage/lmgr/lock.c           |  6 ++--
 src/backend/utils/cache/relmapper.c       |  3 ++
 src/include/miscadmin.h                   | 27 ++++++++++++++++++
 40 files changed, 459 insertions(+), 69 deletions(-)

diff --git a/contrib/pg_surgery/heap_surgery.c b/contrib/pg_surgery/heap_surgery.c
index d31e5f31fd4..2938b937f75 100644
--- a/contrib/pg_surgery/heap_surgery.c
+++ b/contrib/pg_surgery/heap_surgery.c
@@ -14,6 +14,7 @@
 
 #include "access/heapam.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "catalog/pg_am_d.h"
 #include "catalog/pg_proc_d.h"
 #include "miscadmin.h"
@@ -89,6 +90,7 @@ heap_force_common(FunctionCallInfo fcinfo, HeapTupleForceOption heap_force_opt)
 	OffsetNumber curr_start_ptr,
 				next_start_ptr;
 	bool		include_this_tid[MaxHeapTuplesPerPage];
+	bool		needwal;
 
 	if (RecoveryInProgress())
 		ereport(ERROR,
@@ -100,6 +102,7 @@ heap_force_common(FunctionCallInfo fcinfo, HeapTupleForceOption heap_force_opt)
 	sanity_check_tid_array(ta, &ntids);
 
 	rel = relation_open(relid, RowExclusiveLock);
+	needwal = RelationNeedsWAL(rel);
 
 	/* Check target relation. */
 	sanity_check_relation(rel);
@@ -217,6 +220,9 @@ heap_force_common(FunctionCallInfo fcinfo, HeapTupleForceOption heap_force_opt)
 		if (heap_force_opt == HEAP_FORCE_KILL && PageIsAllVisible(page))
 			visibilitymap_pin(rel, blkno, &vmbuf);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* No ereport(ERROR) from here until all the changes are logged. */
 		START_CRIT_SECTION();
 
@@ -297,12 +303,12 @@ heap_force_common(FunctionCallInfo fcinfo, HeapTupleForceOption heap_force_opt)
 			MarkBufferDirty(buf);
 
 			/* XLOG stuff */
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 				log_newpage_buffer(buf, true);
 		}
 
 		/* WAL log the VM page if it was modified. */
-		if (did_modify_vm && RelationNeedsWAL(rel))
+		if (did_modify_vm && needwal)
 			log_newpage_buffer(vmbuf, false);
 
 		END_CRIT_SECTION();
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index 27ba596c6e4..17e719e2dfc 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -24,6 +24,7 @@
 #include "access/relscan.h"
 #include "access/table.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/index.h"
 #include "catalog/pg_am.h"
@@ -760,6 +761,9 @@ brinbuildempty(Relation index)
 		ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermittedHaveXID();
+
 	/* Initialize and xlog metabuffer. */
 	START_CRIT_SECTION();
 	brin_metapage_init(BufferGetPage(metabuf), BrinGetPagesPerRange(index),
diff --git a/src/backend/access/brin/brin_pageops.c b/src/backend/access/brin/brin_pageops.c
index df9ffc2fb86..270a881ab12 100644
--- a/src/backend/access/brin/brin_pageops.c
+++ b/src/backend/access/brin/brin_pageops.c
@@ -14,6 +14,7 @@
 #include "access/brin_pageops.h"
 #include "access/brin_revmap.h"
 #include "access/brin_xlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -65,6 +66,7 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 	Buffer		newbuf;
 	BlockNumber newblk = InvalidBlockNumber;
 	bool		extended;
+	bool		needwal = RelationNeedsWAL(idxrel);
 
 	Assert(newsz == MAXALIGN(newsz));
 
@@ -176,13 +178,16 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 	if (((BrinPageFlags(oldpage) & BRIN_EVACUATE_PAGE) == 0) &&
 		brin_can_do_samepage_update(oldbuf, origsz, newsz))
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 		if (!PageIndexTupleOverwrite(oldpage, oldoff, (Item) unconstify(BrinTuple *, newtup), newsz))
 			elog(ERROR, "failed to replace BRIN tuple");
 		MarkBufferDirty(oldbuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(idxrel))
+		if (needwal)
 		{
 			xl_brin_samepage_update xlrec;
 			XLogRecPtr	recptr;
@@ -240,6 +245,9 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 
 		revmapbuf = brinLockRevmapPageForUpdate(revmap, heapBlk);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		/*
@@ -267,7 +275,7 @@ brin_doupdate(Relation idxrel, BlockNumber pagesPerRange,
 		MarkBufferDirty(revmapbuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(idxrel))
+		if (needwal)
 		{
 			xl_brin_update xlrec;
 			XLogRecPtr	recptr;
@@ -351,6 +359,7 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	Buffer		revmapbuf;
 	ItemPointerData tid;
 	bool		extended;
+	bool		needwal;
 
 	Assert(itemsz == MAXALIGN(itemsz));
 
@@ -405,6 +414,10 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	page = BufferGetPage(*buffer);
 	blk = BufferGetBlockNumber(*buffer);
 
+	needwal = RelationNeedsWAL(idxrel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Execute the actual insertion */
 	START_CRIT_SECTION();
 	if (extended)
@@ -424,7 +437,7 @@ brin_doinsert(Relation idxrel, BlockNumber pagesPerRange,
 	MarkBufferDirty(revmapbuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(idxrel))
+	if (needwal)
 	{
 		xl_brin_insert xlrec;
 		XLogRecPtr	recptr;
@@ -881,6 +894,8 @@ brin_initialize_empty_new_buffer(Relation idxrel, Buffer buffer)
 			   "brin_initialize_empty_new_buffer: initializing blank page %u",
 			   BufferGetBlockNumber(buffer)));
 
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 	page = BufferGetPage(buffer);
 	brin_page_init(page, BRIN_PAGETYPE_REGULAR);
diff --git a/src/backend/access/brin/brin_revmap.c b/src/backend/access/brin/brin_revmap.c
index bab2a88ee3f..70d846d5121 100644
--- a/src/backend/access/brin/brin_revmap.c
+++ b/src/backend/access/brin/brin_revmap.c
@@ -26,6 +26,7 @@
 #include "access/brin_tuple.h"
 #include "access/brin_xlog.h"
 #include "access/rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -340,6 +341,7 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 	OffsetNumber revmapOffset;
 	OffsetNumber regOffset;
 	ItemId		lp;
+	bool		needwal;
 
 	revmap = brinRevmapInitialize(idxrel, &pagesPerRange, NULL);
 
@@ -404,6 +406,10 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 	 * crashed or aborted summarization; remove them silently.
 	 */
 
+	needwal = RelationNeedsWAL(idxrel);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	ItemPointerSetInvalid(&invalidIptr);
@@ -415,7 +421,7 @@ brinRevmapDesummarizeRange(Relation idxrel, BlockNumber heapBlk)
 	MarkBufferDirty(regBuf);
 	MarkBufferDirty(revmapBuf);
 
-	if (RelationNeedsWAL(idxrel))
+	if (needwal)
 	{
 		xl_brin_desummarize xlrec;
 		XLogRecPtr	recptr;
@@ -613,6 +619,8 @@ revmap_physical_extend(BrinRevmap *revmap)
 		return;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Ok, we have now locked the metapage and the target block. Re-initialize
 	 * the target block as a revmap page, and update the metapage.
diff --git a/src/backend/access/gin/ginbtree.c b/src/backend/access/gin/ginbtree.c
index 482cf10877c..0d6dacfe84a 100644
--- a/src/backend/access/gin/ginbtree.c
+++ b/src/backend/access/gin/ginbtree.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/predicate.h"
@@ -332,6 +333,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 {
 	Page		page = BufferGetPage(stack->buffer);
 	bool		result;
+	bool		needwal;
 	GinPlaceToPageRC rc;
 	uint16		xlflags = 0;
 	Page		childpage = NULL;
@@ -377,6 +379,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 								 insertdata, updateblkno,
 								 &ptp_workspace,
 								 &newlpage, &newrpage);
+	needwal = RelationNeedsWAL(btree->index) && !btree->isBuild;
 
 	if (rc == GPTP_NO_WORK)
 	{
@@ -385,10 +388,13 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 	}
 	else if (rc == GPTP_INSERT)
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		/* It will fit, perform the insertion */
 		START_CRIT_SECTION();
 
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogBeginInsert();
 			XLogRegisterBuffer(0, stack->buffer, REGBUF_STANDARD);
@@ -409,7 +415,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 			MarkBufferDirty(childbuf);
 		}
 
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 			ginxlogInsert xlrec;
@@ -547,6 +553,9 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 			}
 		}
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/*
 		 * OK, we have the new contents of the left page in a temporary copy
 		 * now (newlpage), and likewise for the new contents of the
@@ -587,7 +596,7 @@ ginPlaceToPage(GinBtree btree, GinBtreeStack *stack,
 		}
 
 		/* write WAL record */
-		if (RelationNeedsWAL(btree->index) && !btree->isBuild)
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gin/gindatapage.c b/src/backend/access/gin/gindatapage.c
index 06c05865435..2b4812fb23a 100644
--- a/src/backend/access/gin/gindatapage.c
+++ b/src/backend/access/gin/gindatapage.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "lib/ilist.h"
 #include "miscadmin.h"
@@ -811,6 +812,7 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 	if (removedsomething)
 	{
 		bool		modified;
+		bool		needwal;
 
 		/*
 		 * Make sure we have a palloc'd copy of all segments, after the first
@@ -835,8 +837,12 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 			}
 		}
 
-		if (RelationNeedsWAL(indexrel))
+		needwal = RelationNeedsWAL(indexrel);
+		if (needwal)
+		{
+			CheckWALPermitted();
 			computeLeafRecompressWALData(leaf);
+		}
 
 		/* Apply changes to page */
 		START_CRIT_SECTION();
@@ -845,7 +851,7 @@ ginVacuumPostingTreeLeaf(Relation indexrel, Buffer buffer, GinVacuumState *gvs)
 
 		MarkBufferDirty(buffer);
 
-		if (RelationNeedsWAL(indexrel))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
@@ -1777,6 +1783,7 @@ createPostingTree(Relation index, ItemPointerData *items, uint32 nitems,
 	int			nrootitems;
 	int			rootsize;
 	bool		is_build = (buildStats != NULL);
+	bool		needwal;
 
 	/* Construct the new root page in memory first. */
 	tmppage = (Page) palloc(BLCKSZ);
@@ -1825,12 +1832,17 @@ createPostingTree(Relation index, ItemPointerData *items, uint32 nitems,
 	 */
 	PredicateLockPageSplit(index, BufferGetBlockNumber(entrybuffer), blkno);
 
+	needwal = RelationNeedsWAL(index) && !is_build;
+
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	PageRestoreTempPage(tmppage, page);
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index) && !is_build)
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogCreatePostingTree data;
diff --git a/src/backend/access/gin/ginfast.c b/src/backend/access/gin/ginfast.c
index e0d99409461..53c156018e7 100644
--- a/src/backend/access/gin/ginfast.c
+++ b/src/backend/access/gin/ginfast.c
@@ -20,6 +20,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
 #include "catalog/pg_am.h"
@@ -68,6 +69,8 @@ writeListPage(Relation index, Buffer buffer,
 	PGAlignedBlock workspace;
 	char	   *ptr;
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	GinInitBuffer(buffer, GIN_LIST);
@@ -548,6 +551,7 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 	Page		metapage;
 	GinMetaPageData *metadata;
 	BlockNumber blknoToDelete;
+	bool		needwal = RelationNeedsWAL(index);
 
 	metapage = BufferGetPage(metabuffer);
 	metadata = GinPageGetMeta(metapage);
@@ -586,8 +590,11 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 		 * prepare the XLogInsert machinery for that before entering the
 		 * critical section.
 		 */
-		if (RelationNeedsWAL(index))
+		if (needwal)
+		{
+			CheckWALPermitted();
 			XLogEnsureRecordSpace(data.ndeleted, 0);
+		}
 
 		START_CRIT_SECTION();
 
@@ -625,7 +632,7 @@ shiftList(Relation index, Buffer metabuffer, BlockNumber newHead,
 			MarkBufferDirty(buffers[i]);
 		}
 
-		if (RelationNeedsWAL(index))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gin/gininsert.c b/src/backend/access/gin/gininsert.c
index 0e8672c9e90..79312e5d2d0 100644
--- a/src/backend/access/gin/gininsert.c
+++ b/src/backend/access/gin/gininsert.c
@@ -17,6 +17,7 @@
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/index.h"
 #include "miscadmin.h"
@@ -447,6 +448,9 @@ ginbuildempty(Relation index)
 		ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(RootBuffer, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermittedHaveXID();
+
 	/* Initialize and xlog metabuffer and root buffer. */
 	START_CRIT_SECTION();
 	GinInitMetabuffer(MetaBuffer);
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index 6b9b04cf429..6c20bc53fc5 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -17,6 +17,7 @@
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
 #include "access/reloptions.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/pg_collation.h"
 #include "catalog/pg_type.h"
@@ -659,12 +660,18 @@ ginUpdateStats(Relation index, const GinStatsData *stats, bool is_build)
 	Buffer		metabuffer;
 	Page		metapage;
 	GinMetaPageData *metadata;
+	bool		needwal;
 
 	metabuffer = ReadBuffer(index, GIN_METAPAGE_BLKNO);
 	LockBuffer(metabuffer, GIN_EXCLUSIVE);
 	metapage = BufferGetPage(metabuffer);
 	metadata = GinPageGetMeta(metapage);
 
+	needwal = RelationNeedsWAL(index) && !is_build;
+
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	metadata->nTotalPages = stats->nTotalPages;
@@ -684,7 +691,7 @@ ginUpdateStats(Relation index, const GinStatsData *stats, bool is_build)
 
 	MarkBufferDirty(metabuffer);
 
-	if (RelationNeedsWAL(index) && !is_build)
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogUpdateMeta data;
diff --git a/src/backend/access/gin/ginvacuum.c b/src/backend/access/gin/ginvacuum.c
index 35b85a9bff0..9e195e4bc6f 100644
--- a/src/backend/access/gin/ginvacuum.c
+++ b/src/backend/access/gin/ginvacuum.c
@@ -16,6 +16,7 @@
 
 #include "access/gin_private.h"
 #include "access/ginxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "commands/vacuum.h"
 #include "miscadmin.h"
@@ -136,6 +137,7 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	Page		page,
 				parentPage;
 	BlockNumber rightlink;
+	bool		needwal;
 
 	/*
 	 * This function MUST be called only if someone of parent pages hold
@@ -159,6 +161,10 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	 */
 	PredicateLockPageCombine(gvs->index, deleteBlkno, rightlink);
 
+	needwal = RelationNeedsWAL(gvs->index);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* Unlink the page by changing left sibling's rightlink */
@@ -195,7 +201,7 @@ ginDeletePage(GinVacuumState *gvs, BlockNumber deleteBlkno, BlockNumber leftBlkn
 	MarkBufferDirty(lBuffer);
 	MarkBufferDirty(dBuffer);
 
-	if (RelationNeedsWAL(gvs->index))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 		ginxlogDeletePage data;
@@ -650,6 +656,9 @@ ginbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 
 		if (resPage)
 		{
+			if (RelationNeedsWAL(gvs.index))
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 			PageRestoreTempPage(resPage, page);
 			MarkBufferDirty(buffer);
diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c
index f203bb594cd..7ad6e0036c3 100644
--- a/src/backend/access/gist/gist.c
+++ b/src/backend/access/gist/gist.c
@@ -16,6 +16,7 @@
 
 #include "access/gist_private.h"
 #include "access/gistscan.h"
+#include "access/walprohibit.h"
 #include "catalog/pg_collation.h"
 #include "commands/vacuum.h"
 #include "miscadmin.h"
@@ -135,6 +136,9 @@ gistbuildempty(Relation index)
 	buffer = ReadBufferExtended(index, INIT_FORKNUM, P_NEW, RBM_NORMAL, NULL);
 	LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
 
+	/* Building indexes will have an XID */
+	AssertWALPermittedHaveXID();
+
 	/* Initialize and xlog buffer */
 	START_CRIT_SECTION();
 	GISTInitBuffer(buffer, F_LEAF);
@@ -235,6 +239,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 	XLogRecPtr	recptr;
 	int			i;
 	bool		is_split;
+	bool		needwal = RelationNeedsWAL(rel);
 
 	/*
 	 * Refuse to modify a page that's incompletely split. This should not
@@ -469,9 +474,12 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 		 * insertion for that. NB: The number of pages and data segments
 		 * specified here must match the calculations in gistXLogSplit()!
 		 */
-		if (!is_build && RelationNeedsWAL(rel))
+		if (!is_build && needwal)
 			XLogEnsureRecordSpace(npage, 1 + npage * 2);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		/*
@@ -504,7 +512,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 			recptr = GistBuildLSN;
 		else
 		{
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 				recptr = gistXLogSplit(is_leaf,
 									   dist, oldrlink, oldnsn, leftchildbuf,
 									   markfollowright);
@@ -530,6 +538,9 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 	}
 	else
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		/*
 		 * Enough space.  We always get here if ntup==0.
 		 */
@@ -571,7 +582,7 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate,
 			recptr = GistBuildLSN;
 		else
 		{
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 			{
 				OffsetNumber ndeloffs = 0,
 							deloffs[1];
@@ -1646,6 +1657,7 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 	OffsetNumber offnum,
 				maxoff;
 	TransactionId latestRemovedXid = InvalidTransactionId;
+	bool		needwal = RelationNeedsWAL(rel);
 
 	Assert(GistPageIsLeaf(page));
 
@@ -1664,13 +1676,16 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 			deletable[ndeletable++] = offnum;
 	}
 
-	if (XLogStandbyInfoActive() && RelationNeedsWAL(rel))
+	if (XLogStandbyInfoActive() && needwal)
 		latestRemovedXid =
 			index_compute_xid_horizon_for_tuples(rel, heapRel, buffer,
 												 deletable, ndeletable);
 
 	if (ndeletable > 0)
 	{
+		if (needwal)
+			CheckWALPermitted();
+
 		START_CRIT_SECTION();
 
 		PageIndexMultiDelete(page, deletable, ndeletable);
@@ -1687,7 +1702,7 @@ gistprunepage(Relation rel, Page page, Buffer buffer, Relation heapRel)
 		MarkBufferDirty(buffer);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/gist/gistvacuum.c b/src/backend/access/gist/gistvacuum.c
index 94a7e127639..e0c483171ed 100644
--- a/src/backend/access/gist/gistvacuum.c
+++ b/src/backend/access/gist/gistvacuum.c
@@ -17,6 +17,7 @@
 #include "access/genam.h"
 #include "access/gist_private.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "commands/vacuum.h"
 #include "lib/integerset.h"
 #include "miscadmin.h"
@@ -260,6 +261,7 @@ gistvacuumpage(GistVacState *vstate, BlockNumber blkno, BlockNumber orig_blkno)
 	Buffer		buffer;
 	Page		page;
 	BlockNumber recurse_to;
+	bool		needwal = RelationNeedsWAL(rel);
 
 restart:
 	recurse_to = InvalidBlockNumber;
@@ -341,6 +343,9 @@ restart:
 		 */
 		if (ntodelete > 0)
 		{
+			if (needwal)
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 
 			MarkBufferDirty(buffer);
@@ -348,7 +353,7 @@ restart:
 			PageIndexMultiDelete(page, todelete, ntodelete);
 			GistMarkTuplesDeleted(page);
 
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 			{
 				XLogRecPtr	recptr;
 
@@ -580,6 +585,7 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	IndexTuple	idxtuple;
 	XLogRecPtr	recptr;
 	FullTransactionId txid;
+	bool		needwal = RelationNeedsWAL(info->index);
 
 	/*
 	 * Check that the leaf is still empty and deletable.
@@ -634,6 +640,9 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	 */
 	txid = ReadNextFullTransactionId();
 
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* mark the page as deleted */
@@ -645,7 +654,7 @@ gistdeletepage(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	MarkBufferDirty(parentBuffer);
 	PageIndexTupleDelete(parentPage, downlink);
 
-	if (RelationNeedsWAL(info->index))
+	if (needwal)
 		recptr = gistXLogPageDelete(leafBuffer, txid, parentBuffer, downlink);
 	else
 		recptr = gistGetFakeLSN(info->index);
diff --git a/src/backend/access/hash/hash.c b/src/backend/access/hash/hash.c
index 0752fb38a92..52b2b36e4a9 100644
--- a/src/backend/access/hash/hash.c
+++ b/src/backend/access/hash/hash.c
@@ -22,6 +22,7 @@
 #include "access/hash_xlog.h"
 #include "access/relscan.h"
 #include "access/tableam.h"
+#include "access/walprohibit.h"
 #include "catalog/index.h"
 #include "commands/progress.h"
 #include "commands/vacuum.h"
@@ -468,6 +469,7 @@ hashbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
 	Buffer		metabuf = InvalidBuffer;
 	HashMetaPage metap;
 	HashMetaPage cachedmetap;
+	bool		needwal;
 
 	tuples_removed = 0;
 	num_index_tuples = 0;
@@ -574,6 +576,10 @@ loop_top:
 		goto loop_top;
 	}
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Okay, we're really done.  Update tuple count in metapage. */
 	START_CRIT_SECTION();
 
@@ -604,7 +610,7 @@ loop_top:
 	MarkBufferDirty(metabuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_hash_update_meta_page xlrec;
 		XLogRecPtr	recptr;
@@ -691,6 +697,7 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 	Buffer		buf;
 	Bucket		new_bucket PG_USED_FOR_ASSERTS_ONLY = InvalidBucket;
 	bool		bucket_dirty = false;
+	bool		needwal = RelationNeedsWAL(rel);
 
 	blkno = bucket_blkno;
 	buf = bucket_buf;
@@ -789,6 +796,9 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		 */
 		if (ndeletable > 0)
 		{
+			if (needwal)
+				CheckWALPermitted();
+
 			/* No ereport(ERROR) until changes are logged */
 			START_CRIT_SECTION();
 
@@ -810,7 +820,7 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 			MarkBufferDirty(buf);
 
 			/* XLOG stuff */
-			if (RelationNeedsWAL(rel))
+			if (needwal)
 			{
 				xl_hash_delete xlrec;
 				XLogRecPtr	recptr;
@@ -884,6 +894,9 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		page = BufferGetPage(bucket_buf);
 		bucket_opaque = (HashPageOpaque) PageGetSpecialPointer(page);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -891,7 +904,7 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 		MarkBufferDirty(bucket_buf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c
index d254a00b6ac..20ff6d051bf 100644
--- a/src/backend/access/hash/hashinsert.c
+++ b/src/backend/access/hash/hashinsert.c
@@ -17,6 +17,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "storage/buf_internals.h"
 #include "storage/lwlock.h"
@@ -193,6 +194,8 @@ restart_insert:
 	 */
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	AssertWALPermittedHaveXID();
+
 	/* Do the update.  No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -360,6 +363,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 	if (ndeletable > 0)
 	{
 		TransactionId latestRemovedXid;
+		bool		needwal = RelationNeedsWAL(rel);
 
 		latestRemovedXid =
 			index_compute_xid_horizon_for_tuples(rel, hrel, buf,
@@ -370,6 +374,9 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 		 */
 		LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -393,7 +400,7 @@ _hash_vacuum_one_page(Relation rel, Relation hrel, Buffer metabuf, Buffer buf)
 		MarkBufferDirty(metabuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			xl_hash_vacuum_one_page xlrec;
 			XLogRecPtr	recptr;
diff --git a/src/backend/access/hash/hashovfl.c b/src/backend/access/hash/hashovfl.c
index 1ff2e0c18ee..f251d6fc388 100644
--- a/src/backend/access/hash/hashovfl.c
+++ b/src/backend/access/hash/hashovfl.c
@@ -19,6 +19,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "utils/rel.h"
 
@@ -312,6 +313,8 @@ _hash_addovflpage(Relation rel, Buffer metabuf, Buffer buf, bool retain_pin)
 
 found:
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Do the update.  No ereport(ERROR) until changes are logged. We want to
 	 * log the changes for bitmap page and overflow page together to avoid
@@ -510,6 +513,7 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	Buffer		prevbuf = InvalidBuffer;
 	Buffer		nextbuf = InvalidBuffer;
 	bool		update_metap = false;
+	bool		needwal;
 
 	/* Get information from the doomed page */
 	_hash_checkpage(rel, ovflbuf, LH_OVERFLOW_PAGE);
@@ -573,10 +577,15 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	/* Get write-lock on metapage to update firstfree */
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
+	needwal = RelationNeedsWAL(rel);
+
 	/* This operation needs to log multiple tuples, prepare WAL for that */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 		XLogEnsureRecordSpace(HASH_XLOG_FREE_OVFL_BUFS, 4 + nitups);
 
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -641,7 +650,7 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	}
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_hash_squeeze_page xlrec;
 		XLogRecPtr	recptr;
@@ -922,14 +931,19 @@ readpage:
 
 				if (nitups > 0)
 				{
+					bool		needwal = RelationNeedsWAL(rel);
+
 					Assert(nitups == ndeletable);
 
 					/*
 					 * This operation needs to log multiple tuples, prepare
 					 * WAL for that.
 					 */
-					if (RelationNeedsWAL(rel))
+					if (needwal)
+					{
+						CheckWALPermitted();
 						XLogEnsureRecordSpace(0, 3 + nitups);
+					}
 
 					START_CRIT_SECTION();
 
@@ -947,7 +961,7 @@ readpage:
 					MarkBufferDirty(rbuf);
 
 					/* XLOG stuff */
-					if (RelationNeedsWAL(rel))
+					if (needwal)
 					{
 						XLogRecPtr	recptr;
 						xl_hash_move_page_contents xlrec;
diff --git a/src/backend/access/hash/hashpage.c b/src/backend/access/hash/hashpage.c
index 49a98677876..f161a9a49eb 100644
--- a/src/backend/access/hash/hashpage.c
+++ b/src/backend/access/hash/hashpage.c
@@ -30,6 +30,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "port/pg_bitutils.h"
 #include "storage/lmgr.h"
@@ -816,6 +817,8 @@ restart_expand:
 		goto fail;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Since we are scribbling on the pages in the shared buffers, establish a
 	 * critical section.  Any failure in this next code leaves us with a big
@@ -1172,6 +1175,8 @@ _hash_splitbucket(Relation rel,
 
 				if (PageGetFreeSpaceForMultipleTuples(npage, nitups + 1) < (all_tups_size + itemsz))
 				{
+					AssertWALPermittedHaveXID();
+
 					/*
 					 * Change the shared buffer state in critical section,
 					 * otherwise any error could make it unrecoverable.
@@ -1223,6 +1228,8 @@ _hash_splitbucket(Relation rel,
 		/* Exit loop if no more overflow pages in old bucket */
 		if (!BlockNumberIsValid(oblkno))
 		{
+			AssertWALPermittedHaveXID();
+
 			/*
 			 * Change the shared buffer state in critical section, otherwise
 			 * any error could make it unrecoverable.
@@ -1269,6 +1276,8 @@ _hash_splitbucket(Relation rel,
 	npage = BufferGetPage(bucket_nbuf);
 	nopaque = (HashPageOpaque) PageGetSpecialPointer(npage);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	oopaque->hasho_flag &= ~LH_BUCKET_BEING_SPLIT;
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index faffbb18658..6aa14c275d2 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -47,6 +47,7 @@
 #include "access/transam.h"
 #include "access/valid.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1916,6 +1917,8 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
 	 */
 	CheckForSerializableConflictIn(relation, NULL, InvalidBlockNumber);
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -2200,6 +2203,8 @@ heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples,
 		if (starting_with_empty_page && (options & HEAP_INSERT_FROZEN))
 			all_frozen_set = true;
 
+		AssertWALPermittedHaveXID();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -2761,6 +2766,8 @@ l1:
 							  xid, LockTupleExclusive, true,
 							  &new_xmax, &new_infomask, &new_infomask2);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -3513,6 +3520,8 @@ l2:
 
 		Assert(HEAP_XMAX_IS_LOCKED_ONLY(infomask_lock_old_tuple));
 
+		AssertWALPermittedHaveXID();
+
 		START_CRIT_SECTION();
 
 		/* Clear obsolete visibility flags ... */
@@ -3686,6 +3695,8 @@ l2:
 										   bms_overlap(modified_attrs, id_attrs),
 										   &old_key_copied);
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -4619,6 +4630,8 @@ failed:
 							  GetCurrentTransactionId(), mode, false,
 							  &xid, &new_infomask, &new_infomask2);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -5410,6 +5423,8 @@ l4:
 								VISIBILITYMAP_ALL_FROZEN))
 			cleared_all_frozen = true;
 
+		AssertWALPermittedHaveXID();
+
 		START_CRIT_SECTION();
 
 		/* ... and set them */
@@ -5568,6 +5583,8 @@ heap_finish_speculative(Relation relation, ItemPointer tid)
 	StaticAssertStmt(MaxOffsetNumber < SpecTokenOffsetNumber,
 					 "invalid speculative token constant");
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -5676,6 +5693,8 @@ heap_abort_speculative(Relation relation, ItemPointer tid)
 		elog(ERROR, "attempted to kill a non-speculative tuple");
 	Assert(!HeapTupleHeaderIsHeapOnly(tp.t_data));
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * No need to check for serializable conflicts here.  There is never a
 	 * need for a combocid, either.  No need to extract replica identity, or
@@ -5796,6 +5815,7 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	HeapTupleHeader htup;
 	uint32		oldlen;
 	uint32		newlen;
+	bool		needwal;
 
 	/*
 	 * For now, we don't allow parallel updates.  Unlike a regular update,
@@ -5826,6 +5846,10 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	if (oldlen != newlen || htup->t_hoff != tuple->t_data->t_hoff)
 		elog(ERROR, "wrong tuple length");
 
+	needwal = RelationNeedsWAL(relation);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* NO EREPORT(ERROR) from here till changes are logged */
 	START_CRIT_SECTION();
 
@@ -5836,7 +5860,7 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
 	MarkBufferDirty(buffer);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(relation))
+	if (needwal)
 	{
 		xl_heap_inplace xlrec;
 		XLogRecPtr	recptr;
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index e3a716a2a2f..e93c211da4f 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -18,6 +18,7 @@
 #include "access/heapam_xlog.h"
 #include "access/htup_details.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "catalog/catalog.h"
 #include "miscadmin.h"
@@ -94,11 +95,11 @@ heap_page_prune_opt(Relation relation, Buffer buffer)
 	Size		minfree;
 
 	/*
-	 * We can't write WAL in recovery mode, so there's no point trying to
+	 * We can't write WAL during read-only mode, so there's no point trying to
 	 * clean the page. The primary will likely issue a cleaning WAL record soon
 	 * anyway, so this is no particular loss.
 	 */
-	if (RecoveryInProgress())
+	if (!XLogInsertAllowed())
 		return;
 
 	/*
@@ -232,6 +233,7 @@ heap_page_prune(Relation relation, Buffer buffer,
 	OffsetNumber offnum,
 				maxoff;
 	PruneState	prstate;
+	bool		needwal;
 
 	/*
 	 * Our strategy is to scan the page and make lists of items to change,
@@ -286,6 +288,10 @@ heap_page_prune(Relation relation, Buffer buffer,
 	if (off_loc)
 		*off_loc = InvalidOffsetNumber;
 
+	needwal = RelationNeedsWAL(relation);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Any error while applying the changes is critical */
 	START_CRIT_SECTION();
 
@@ -319,7 +325,7 @@ heap_page_prune(Relation relation, Buffer buffer,
 		/*
 		 * Emit a WAL XLOG_HEAP2_CLEAN record showing what we did
 		 */
-		if (RelationNeedsWAL(relation))
+		if (needwal)
 		{
 			XLogRecPtr	recptr;
 
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index f3d2265fad7..7370dc1f64d 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -59,6 +59,7 @@
 #include "access/parallel.h"
 #include "access/transam.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "catalog/storage.h"
@@ -758,6 +759,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 	BlockNumber nblocks,
 				blkno;
 	HeapTupleData tuple;
+	bool		needwal = RelationNeedsWAL(onerel);
 	TransactionId relfrozenxid = onerel->rd_rel->relfrozenxid;
 	TransactionId relminmxid = onerel->rd_rel->relminmxid;
 	BlockNumber empty_pages,
@@ -1200,6 +1202,9 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 			 */
 			if (!PageIsAllVisible(page))
 			{
+				if (needwal)
+					CheckWALPermitted();
+
 				START_CRIT_SECTION();
 
 				/* mark buffer dirty before writing a WAL record */
@@ -1215,7 +1220,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 				 * page has been previously WAL-logged, and if not, do that
 				 * now.
 				 */
-				if (RelationNeedsWAL(onerel) &&
+				if (needwal &&
 					PageGetLSN(page) == InvalidXLogRecPtr)
 					log_newpage_buffer(buf, true);
 
@@ -1481,6 +1486,9 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 		 */
 		if (nfrozen > 0)
 		{
+			if (needwal)
+				CheckWALPermitted();
+
 			START_CRIT_SECTION();
 
 			MarkBufferDirty(buf);
@@ -1498,7 +1506,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
 			}
 
 			/* Now WAL-log freezing if necessary */
-			if (RelationNeedsWAL(onerel))
+			if (needwal)
 			{
 				XLogRecPtr	recptr;
 
@@ -1931,6 +1939,7 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	TransactionId visibility_cutoff_xid;
 	bool		all_frozen;
 	LVSavedErrInfo saved_err_info;
+	bool		needwal = RelationNeedsWAL(onerel);
 
 	pgstat_progress_update_param(PROGRESS_VACUUM_HEAP_BLKS_VACUUMED, blkno);
 
@@ -1938,6 +1947,9 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	update_vacuum_error_info(vacrelstats, &saved_err_info, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
 							 blkno, InvalidOffsetNumber);
 
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	for (; tupindex < dead_tuples->num_tuples; tupindex++)
@@ -1963,7 +1975,7 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
 	MarkBufferDirty(buffer);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(onerel))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index e198df65d82..05f7de15c76 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -88,6 +88,7 @@
 
 #include "access/heapam_xlog.h"
 #include "access/visibilitymap.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "miscadmin.h"
 #include "port/pg_bitutils.h"
@@ -249,6 +250,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 	uint8		mapOffset = HEAPBLK_TO_OFFSET(heapBlk);
 	Page		page;
 	uint8	   *map;
+	bool		needwal = RelationNeedsWAL(rel);
 
 #ifdef TRACE_VISIBILITYMAP
 	elog(DEBUG1, "vm_set %s %d", RelationGetRelationName(rel), heapBlk);
@@ -270,6 +272,16 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 	map = (uint8 *) PageGetContents(page);
 	LockBuffer(vmBuf, BUFFER_LOCK_EXCLUSIVE);
 
+	/*
+	 * Can reach here from VACUUM or from startup process, so need not have an
+	 * XID.
+	 *
+	 * Recovery in the startup process never have wal prohibit state, skip
+	 * permission check if reach here in the startup process.
+	 */
+	if (needwal)
+		InRecovery ? AssertWALPermitted() : CheckWALPermitted();
+
 	if (flags != (map[mapByte] >> mapOffset & VISIBILITYMAP_VALID_BITS))
 	{
 		START_CRIT_SECTION();
@@ -277,7 +289,7 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
 		map[mapByte] |= (flags << mapOffset);
 		MarkBufferDirty(vmBuf);
 
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			if (XLogRecPtrIsInvalid(recptr))
 			{
@@ -476,6 +488,7 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		Buffer		mapBuffer;
 		Page		page;
 		char	   *map;
+		bool		needwal;
 
 		newnblocks = truncBlock + 1;
 
@@ -489,8 +502,13 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		page = BufferGetPage(mapBuffer);
 		map = PageGetContents(page);
 
+		needwal = (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded());
+
 		LockBuffer(mapBuffer, BUFFER_LOCK_EXCLUSIVE);
 
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -518,7 +536,7 @@ visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks)
 		 * during recovery.
 		 */
 		MarkBufferDirty(mapBuffer);
-		if (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded())
+		if (needwal)
 			log_newpage_buffer(mapBuffer, false);
 
 		END_CRIT_SECTION();
diff --git a/src/backend/access/nbtree/nbtdedup.c b/src/backend/access/nbtree/nbtdedup.c
index 854e3b2cf9a..71f5ade8c28 100644
--- a/src/backend/access/nbtree/nbtdedup.c
+++ b/src/backend/access/nbtree/nbtdedup.c
@@ -16,6 +16,7 @@
 
 #include "access/nbtree.h"
 #include "access/nbtxlog.h"
+#include "access/walprohibit.h"
 #include "miscadmin.h"
 #include "utils/rel.h"
 
@@ -229,6 +230,8 @@ _bt_dedup_pass(Relation rel, Buffer buf, Relation heapRel, IndexTuple newitem,
 		nopaque->btpo_flags &= ~BTP_HAS_GARBAGE;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	PageRestoreTempPage(newpage, page);
diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c
index e3336039125..c840912d116 100644
--- a/src/backend/access/nbtree/nbtinsert.c
+++ b/src/backend/access/nbtree/nbtinsert.c
@@ -18,6 +18,7 @@
 #include "access/nbtree.h"
 #include "access/nbtxlog.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "lib/qunique.h"
 #include "miscadmin.h"
@@ -1249,6 +1250,8 @@ _bt_insertonpg(Relation rel,
 			}
 		}
 
+		AssertWALPermittedHaveXID();
+
 		/* Do the update.  No ereport(ERROR) until changes are logged */
 		START_CRIT_SECTION();
 
@@ -1901,13 +1904,16 @@ _bt_split(Relation rel, BTScanInsert itup_key, Buffer buf, Buffer cbuf,
 			ropaque->btpo_flags |= BTP_SPLIT_END;
 	}
 
+	AssertWALPermittedHaveXID();
+
 	/*
 	 * Right sibling is locked, new siblings are prepared, but original page
 	 * is not updated yet.
 	 *
 	 * NO EREPORT(ERROR) till right sibling is updated.  We can get away with
 	 * not starting the critical section till here because we haven't been
-	 * scribbling on the original page yet; see comments above.
+	 * scribbling on the original page yet; see the comments above for grabbing
+	 * the right sibling.
 	 */
 	START_CRIT_SECTION();
 
@@ -2469,6 +2475,8 @@ _bt_newroot(Relation rel, Buffer lbuf, Buffer rbuf)
 	right_item = CopyIndexTuple(item);
 	BTreeTupleSetDownLink(right_item, rbkno);
 
+	AssertWALPermittedHaveXID();
+
 	/* NO EREPORT(ERROR) from here till newroot op is logged */
 	START_CRIT_SECTION();
 
diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c
index e230f912c28..9a9e2fc2b3c 100644
--- a/src/backend/access/nbtree/nbtpage.c
+++ b/src/backend/access/nbtree/nbtpage.c
@@ -26,6 +26,7 @@
 #include "access/nbtxlog.h"
 #include "access/tableam.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
@@ -185,6 +186,7 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	BTMetaPageData *metad;
 	bool		needsRewrite = false;
 	XLogRecPtr	recptr;
+	bool		needwal;
 
 	/* read the metapage and check if it needs rewrite */
 	metabuf = _bt_getbuf(rel, BTREE_METAPAGE, BT_READ);
@@ -208,6 +210,10 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	_bt_unlockbuf(rel, metabuf);
 	_bt_lockbuf(rel, metabuf, BT_WRITE);
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* upgrade meta-page if needed */
@@ -220,7 +226,7 @@ _bt_update_meta_cleanup_info(Relation rel, TransactionId oldestBtpoXact,
 	MarkBufferDirty(metabuf);
 
 	/* write wal record if needed */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_btree_metadata md;
 
@@ -338,6 +344,7 @@ _bt_getroot(Relation rel, int access)
 	if (metad->btm_root == P_NONE)
 	{
 		Page		metapg;
+		bool		needwal;
 
 		/* If access = BT_READ, caller doesn't want us to create root yet */
 		if (access == BT_READ)
@@ -383,6 +390,10 @@ _bt_getroot(Relation rel, int access)
 		/* Get raw page pointer for metapage */
 		metapg = BufferGetPage(metabuf);
 
+		needwal = RelationNeedsWAL(rel);
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO ELOG(ERROR) till meta is updated */
 		START_CRIT_SECTION();
 
@@ -401,7 +412,7 @@ _bt_getroot(Relation rel, int access)
 		MarkBufferDirty(metabuf);
 
 		/* XLOG stuff */
-		if (RelationNeedsWAL(rel))
+		if (needwal)
 		{
 			xl_btree_newroot xlrec;
 			XLogRecPtr	recptr;
@@ -1148,6 +1159,9 @@ _bt_delitems_vacuum(Relation rel, Buffer buf,
 										 updatedoffsets, &updatedbuflen,
 										 needswal);
 
+	if (needswal)
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -1279,6 +1293,8 @@ _bt_delitems_delete(Relation rel, Buffer buf, TransactionId latestRemovedXid,
 										 updatedoffsets, &updatedbuflen,
 										 needswal);
 
+	AssertWALPermittedHaveXID();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -2065,6 +2081,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	OffsetNumber nextoffset;
 	IndexTuple	itup;
 	IndexTupleData trunctuple;
+	bool		needwal;
 
 	page = BufferGetPage(leafbuf);
 	opaque = (BTPageOpaque) PageGetSpecialPointer(page);
@@ -2153,6 +2170,10 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	 */
 	PredicateLockPageCombine(rel, leafblkno, leafrightsib);
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -2204,7 +2225,7 @@ _bt_mark_page_halfdead(Relation rel, Buffer leafbuf, BTStack stack)
 	MarkBufferDirty(leafbuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_btree_mark_page_halfdead xlrec;
 		XLogRecPtr	recptr;
@@ -2297,6 +2318,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 	int			targetlevel;
 	IndexTuple	leafhikey;
 	BlockNumber nextchild;
+	bool		needwal;
 
 	page = BufferGetPage(leafbuf);
 	opaque = (BTPageOpaque) PageGetSpecialPointer(page);
@@ -2511,6 +2533,10 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 	 * Here we begin doing the deletion.
 	 */
 
+	needwal = RelationNeedsWAL(rel);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* No ereport(ERROR) until changes are logged */
 	START_CRIT_SECTION();
 
@@ -2590,7 +2616,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno,
 		MarkBufferDirty(leafbuf);
 
 	/* XLOG stuff */
-	if (RelationNeedsWAL(rel))
+	if (needwal)
 	{
 		xl_btree_unlink_page xlrec;
 		xl_btree_metadata xlmeta;
diff --git a/src/backend/access/spgist/spgdoinsert.c b/src/backend/access/spgist/spgdoinsert.c
index 7bd269fd2a0..1f9dd667262 100644
--- a/src/backend/access/spgist/spgdoinsert.c
+++ b/src/backend/access/spgist/spgdoinsert.c
@@ -18,6 +18,7 @@
 #include "access/genam.h"
 #include "access/spgist_private.h"
 #include "access/spgxlog.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
@@ -214,6 +215,8 @@ addLeafTuple(Relation index, SpGistState *state, SpGistLeafTuple leafTuple,
 	xlrec.offnumParent = InvalidOffsetNumber;
 	xlrec.nodeI = 0;
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	if (current->offnum == InvalidOffsetNumber ||
@@ -458,6 +461,8 @@ moveLeafs(Relation index, SpGistState *state,
 
 	leafdata = leafptr = palloc(size);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/* copy all the old tuples to new page, unless they're dead */
@@ -1109,6 +1114,8 @@ doPickSplit(Relation index, SpGistState *state,
 
 	leafdata = leafptr = (char *) palloc(totalLeafSizes);
 
+	AssertWALPermittedHaveXID();
+
 	/* Here we begin making the changes to the target pages */
 	START_CRIT_SECTION();
 
@@ -1517,6 +1524,8 @@ spgAddNodeAction(Relation index, SpGistState *state,
 	if (PageGetExactFreeSpace(current->page) >=
 		newInnerTuple->size - innerTuple->size)
 	{
+		AssertWALPermittedHaveXID();
+
 		/*
 		 * We can replace the inner tuple by new version in-place
 		 */
@@ -1603,6 +1612,8 @@ spgAddNodeAction(Relation index, SpGistState *state,
 		else
 			xlrec.parentBlk = 2;
 
+		AssertWALPermittedHaveXID();
+
 		START_CRIT_SECTION();
 
 		/* insert new ... */
@@ -1788,6 +1799,8 @@ spgSplitNodeAction(Relation index, SpGistState *state,
 									&xlrec.newPage);
 	}
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	/*
diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c
index 0d02a02222e..d689473a713 100644
--- a/src/backend/access/spgist/spgvacuum.c
+++ b/src/backend/access/spgist/spgvacuum.c
@@ -19,6 +19,7 @@
 #include "access/spgist_private.h"
 #include "access/spgxlog.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xloginsert.h"
 #include "catalog/storage_xlog.h"
 #include "commands/vacuum.h"
@@ -139,6 +140,7 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 	int			nDeletable;
 	OffsetNumber i,
 				max = PageGetMaxOffsetNumber(page);
+	bool		needwal;
 
 	memset(predecessor, 0, sizeof(predecessor));
 	memset(deletable, 0, sizeof(deletable));
@@ -323,6 +325,10 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 	if (nDeletable != xlrec.nDead + xlrec.nPlaceholder + xlrec.nMove)
 		elog(ERROR, "inconsistent counts of deletable tuples");
 
+	needwal = RelationNeedsWAL(index);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Do the updates */
 	START_CRIT_SECTION();
 
@@ -371,7 +377,7 @@ vacuumLeafPage(spgBulkDeleteState *bds, Relation index, Buffer buffer,
 
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 
@@ -411,6 +417,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 	OffsetNumber toDelete[MaxIndexTuplesPerPage];
 	OffsetNumber i,
 				max = PageGetMaxOffsetNumber(page);
+	bool		needwal;
 
 	xlrec.nDelete = 0;
 
@@ -447,6 +454,10 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 	if (xlrec.nDelete == 0)
 		return;					/* nothing more to do */
 
+	needwal = RelationNeedsWAL(index);
+	if (needwal)
+		CheckWALPermitted();
+
 	/* Do the update */
 	START_CRIT_SECTION();
 
@@ -455,7 +466,7 @@ vacuumLeafRoot(spgBulkDeleteState *bds, Relation index, Buffer buffer)
 
 	MarkBufferDirty(buffer);
 
-	if (RelationNeedsWAL(index))
+	if (needwal)
 	{
 		XLogRecPtr	recptr;
 
@@ -502,6 +513,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer)
 	OffsetNumber itemnos[MaxIndexTuplesPerPage];
 	spgxlogVacuumRedirect xlrec;
 	GlobalVisState *vistest;
+	bool		needwal;
 
 	xlrec.nToPlaceholder = 0;
 	xlrec.newestRedirectXid = InvalidTransactionId;
@@ -509,6 +521,10 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer)
 	/* XXX: providing heap relation would allow more pruning */
 	vistest = GlobalVisTestFor(NULL);
 
+	needwal = RelationNeedsWAL(index);
+	if (needwal)
+		CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -584,7 +600,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer)
 	if (hasUpdate)
 		MarkBufferDirty(buffer);
 
-	if (hasUpdate && RelationNeedsWAL(index))
+	if (hasUpdate && needwal)
 	{
 		XLogRecPtr	recptr;
 
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index 7dcfa023236..bc0dcddb9b2 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -73,6 +73,7 @@
 #include "access/transam.h"
 #include "access/twophase.h"
 #include "access/twophase_rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1162,6 +1163,8 @@ GetNewMultiXactId(int nmembers, MultiXactOffset *offset)
 
 	ExtendMultiXactMember(nextOffset, nmembers);
 
+	CheckWALPermitted();
+
 	/*
 	 * Critical section from here until caller has written the data into the
 	 * just-reserved SLRU space; we don't want to error out with a partly
@@ -2948,7 +2951,7 @@ TruncateMultiXact(MultiXactId newOldestMulti, Oid newOldestMultiDB)
 	mxtruncinfo trunc;
 	MultiXactId earliest;
 
-	Assert(!RecoveryInProgress());
+	Assert(XLogInsertAllowed());
 	Assert(MultiXactState->finishedStartup);
 
 	/*
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index fc18b778324..91f2b18e367 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -82,6 +82,7 @@
 #include "access/transam.h"
 #include "access/twophase.h"
 #include "access/twophase_rmgr.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1106,6 +1107,8 @@ EndPrepare(GlobalTransaction gxact)
 	 */
 	XLogEnsureRecordSpace(0, records.num_chunks);
 
+	AssertWALPermittedHaveXID();
+
 	START_CRIT_SECTION();
 
 	MyProc->delayChkpt = true;
@@ -2196,6 +2199,9 @@ RecordTransactionCommitPrepared(TransactionId xid,
 	replorigin = (replorigin_session_origin != InvalidRepOriginId &&
 				  replorigin_session_origin != DoNotReplicateId);
 
+	/* COMMIT PREPARED need not have an XID */
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/* See notes in RecordTransactionCommit */
@@ -2286,6 +2292,9 @@ RecordTransactionAbortPrepared(TransactionId xid,
 		elog(PANIC, "cannot abort transaction %u, it was already committed",
 			 xid);
 
+	/* ROLLBACK PREPARED need not have an XID */
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
diff --git a/src/backend/access/transam/varsup.c b/src/backend/access/transam/varsup.c
index 2264c2c849c..15cb9b9a25c 100644
--- a/src/backend/access/transam/varsup.c
+++ b/src/backend/access/transam/varsup.c
@@ -17,6 +17,7 @@
 #include "access/commit_ts.h"
 #include "access/subtrans.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "commands/dbcommands.h"
@@ -75,6 +76,9 @@ GetNewTransactionId(bool isSubXact)
 	if (RecoveryInProgress())
 		elog(ERROR, "cannot assign TransactionIds during recovery");
 
+	/* do not assign transaction id in read-only mode */
+	CheckWALPermitted();
+
 	LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
 
 	full_xid = ShmemVariableCache->nextXid;
diff --git a/src/backend/access/transam/walprohibit.c b/src/backend/access/transam/walprohibit.c
index 831b8c7d8ea..cec1bb2d0ba 100644
--- a/src/backend/access/transam/walprohibit.c
+++ b/src/backend/access/transam/walprohibit.c
@@ -25,6 +25,16 @@
 #include "utils/fmgroids.h"
 #include "utils/fmgrprotos.h"
 
+/*
+ * Assert flag to enforce WAL insert permission check rule before starting a
+ * critical section for the WAL writes.  For this, either of
+ * CheckWALPermitted(), AssertWALPermittedHaveXID(), or AssertWALPermitted()
+ * must be called before starting the critical section.
+ */
+#ifdef USE_ASSERT_CHECKING
+WALPermitCheckState walpermit_checked_state = WALPERMIT_UNCHECKED;
+#endif
+
 /*
  * Shared-memory WAL prohibit state
  */
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 0de63af6365..cdb18c47b0c 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -26,6 +26,7 @@
 #include "access/subtrans.h"
 #include "access/transam.h"
 #include "access/twophase.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -1317,6 +1318,8 @@ RecordTransactionCommit(void)
 		/* Tell bufmgr and smgr to prepare for commit */
 		BufmgrCommit();
 
+		AssertWALPermittedHaveXID();
+
 		/*
 		 * Mark ourselves as within our "commit critical section".  This
 		 * forces any concurrent checkpoint to wait until we've updated
@@ -1677,6 +1680,9 @@ RecordTransactionAbort(bool isSubXact)
 		elog(PANIC, "cannot abort transaction %u, it was already committed",
 			 xid);
 
+	/* We'll be reaching here with valid XID only. */
+	AssertWALPermittedHaveXID();
+
 	/* Fetch the data we need for the abort record */
 	nrels = smgrGetPendingDeletes(false, &rels);
 	nchildren = xactGetCommittedChildren(&children);
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index d1f0ce9a3ba..9d38d1766ce 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -1039,7 +1039,7 @@ XLogInsertRecord(XLogRecData *rdata,
 
 	/* cross-check on whether we should be here or not */
 	if (!XLogInsertAllowed())
-		elog(ERROR, "cannot make new WAL entries during recovery");
+		elog(ERROR, "cannot make new WAL entries now");
 
 	/*----------
 	 *
@@ -2876,9 +2876,11 @@ XLogFlush(XLogRecPtr record)
 	 * trying to flush the WAL, we should update minRecoveryPoint instead. We
 	 * test XLogInsertAllowed(), not InRecovery, because we need checkpointer
 	 * to act this way too, and because when it tries to write the
-	 * end-of-recovery checkpoint, it should indeed flush.
+	 * end-of-recovery checkpoint, it should indeed flush. Also, WAL prohibit
+	 * state should not restrict WAL flushing. Otherwise, the dirty buffer
+	 * cannot be evicted until WAL has been flushed up to the buffer's LSN.
 	 */
-	if (!XLogInsertAllowed())
+	if (!XLogInsertAllowed() && !IsWALProhibited())
 	{
 		UpdateMinRecoveryPoint(record, false);
 		return;
@@ -8932,6 +8934,8 @@ CreateCheckPoint(int flags)
 	/* sanity check */
 	if (RecoveryInProgress() && (flags & CHECKPOINT_END_OF_RECOVERY) == 0)
 		elog(ERROR, "can't create a checkpoint during recovery");
+	if (!XLogInsertAllowed() && !RecoveryInProgress())
+		elog(ERROR, "can't create a checkpoint while system is read only ");
 
 	/*
 	 * Initialize InitXLogInsert working areas before entering the critical
@@ -9196,6 +9200,9 @@ CreateCheckPoint(int flags)
 	if (!shutdown && XLogStandbyInfoActive())
 		LogStandbySnapshot();
 
+	/* Error out if wal writes are disabled. */
+	CheckWALPermitted();
+
 	START_CRIT_SECTION();
 
 	/*
@@ -9356,6 +9363,8 @@ CreateEndOfRecoveryRecord(void)
 
 	LocalSetXLogInsertAllowed();
 
+	AssertWALPermitted();
+
 	START_CRIT_SECTION();
 
 	XLogBeginInsert();
@@ -10020,7 +10029,7 @@ void
 UpdateFullPageWrites(void)
 {
 	XLogCtlInsert *Insert = &XLogCtl->Insert;
-	bool		recoveryInProgress;
+	bool		WALInsertAllowed;
 
 	/*
 	 * Do nothing if full_page_writes has not been changed.
@@ -10034,10 +10043,10 @@ UpdateFullPageWrites(void)
 
 	/*
 	 * Perform this outside critical section so that the WAL insert
-	 * initialization done by RecoveryInProgress() doesn't trigger an
+	 * initialization done by XLogInsertAllowed() doesn't trigger an
 	 * assertion failure.
 	 */
-	recoveryInProgress = RecoveryInProgress();
+	WALInsertAllowed = XLogInsertAllowed();
 
 	START_CRIT_SECTION();
 
@@ -10059,8 +10068,11 @@ UpdateFullPageWrites(void)
 	 * Write an XLOG_FPW_CHANGE record. This allows us to keep track of
 	 * full_page_writes during archive recovery, if required.
 	 */
-	if (XLogStandbyInfoActive() && !recoveryInProgress)
+	if (XLogStandbyInfoActive() && WALInsertAllowed)
 	{
+		/* Assured that WAL permission has been checked */
+		AssertWALPermitted();
+
 		XLogBeginInsert();
 		XLogRegisterData((char *) (&fullPageWrites), sizeof(bool));
 
diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c
index 7052dc245ee..a925cc9d5cd 100644
--- a/src/backend/access/transam/xloginsert.c
+++ b/src/backend/access/transam/xloginsert.c
@@ -19,6 +19,7 @@
 
 #include "postgres.h"
 
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xlog_internal.h"
@@ -126,9 +127,14 @@ XLogBeginInsert(void)
 	Assert(mainrdata_last == (XLogRecData *) &mainrdata_head);
 	Assert(mainrdata_len == 0);
 
+	/*
+	 * WAL permission must have checked before entering the critical section.
+	 * Otherwise, WAL prohibited error will force system panic.
+	 */
+	Assert(walpermit_checked_state != WALPERMIT_UNCHECKED || !CritSectionCount);
+
 	/* cross-check on whether we should be here or not */
-	if (!XLogInsertAllowed())
-		elog(ERROR, "cannot make new WAL entries during recovery");
+	CheckWALPermitted();
 
 	if (begininsert_called)
 		elog(ERROR, "XLogBeginInsert was already called");
@@ -210,6 +216,9 @@ XLogResetInsertion(void)
 	mainrdata_last = (XLogRecData *) &mainrdata_head;
 	curinsert_flags = 0;
 	begininsert_called = false;
+
+	/* Reset walpermit_checked flag */
+	RESET_WALPERMIT_CHECKED_STATE();
 }
 
 /*
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 0415df9ccb7..ab108e621d4 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -20,6 +20,7 @@
 #include "access/relation.h"
 #include "access/table.h"
 #include "access/transam.h"
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -378,8 +379,13 @@ fill_seq_with_data(Relation rel, HeapTuple tuple)
 
 	/* check the comment above nextval_internal()'s equivalent call. */
 	if (RelationNeedsWAL(rel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermittedHaveXID();
+	}
+
 	START_CRIT_SECTION();
 
 	MarkBufferDirty(buf);
@@ -766,8 +772,13 @@ nextval_internal(Oid relid, bool check_permissions)
 	 * (Have to do that here, so we're outside the critical section)
 	 */
 	if (logit && RelationNeedsWAL(seqrel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermittedHaveXID();
+	}
+
 	/* ready to change the on-disk (or really, in-buffer) tuple */
 	START_CRIT_SECTION();
 
@@ -977,8 +988,13 @@ do_setval(Oid relid, int64 next, bool iscalled)
 
 	/* check the comment above nextval_internal()'s equivalent call. */
 	if (RelationNeedsWAL(seqrel))
+	{
 		GetTopTransactionId();
 
+		/* Cannot have valid XID without WAL permission */
+		AssertWALPermittedHaveXID();
+	}
+
 	/* ready to change the on-disk (or really, in-buffer) tuple */
 	START_CRIT_SECTION();
 
diff --git a/src/backend/commands/variable.c b/src/backend/commands/variable.c
index c5cf08b4237..aa1b0c12000 100644
--- a/src/backend/commands/variable.c
+++ b/src/backend/commands/variable.c
@@ -501,11 +501,14 @@ check_transaction_read_only(bool *newval, void **extra, GucSource source)
 			GUC_check_errmsg("transaction read-write mode must be set before any query");
 			return false;
 		}
-		/* Can't go to r/w mode while recovery is still active */
-		if (RecoveryInProgress())
+		/*
+		 * Can't go to r/w mode while recovery is still active or while in WAL
+		 * prohibit state
+		 */
+		if (!XLogInsertAllowed())
 		{
 			GUC_check_errcode(ERRCODE_FEATURE_NOT_SUPPORTED);
-			GUC_check_errmsg("cannot set transaction read-write mode during recovery");
+			GUC_check_errmsg("cannot set transaction read-write mode while system is read only");
 			return false;
 		}
 	}
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 033f8a7bdd9..e4ee43e4a41 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -924,6 +924,10 @@ RequestCheckpoint(int flags)
 	int			old_failed,
 				old_started;
 
+	/* The checkpoint is allowed in recovery but not in WAL prohibit state */
+	if (!RecoveryInProgress())
+		CheckWALPermitted();
+
 	/*
 	 * If in a standalone backend, just do it ourselves.
 	 */
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 561c212092f..909c3e75107 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -3820,13 +3820,15 @@ MarkBufferDirtyHint(Buffer buffer, bool buffer_std)
 		{
 			/*
 			 * If we must not write WAL, due to a relfilenode-specific
-			 * condition or being in recovery, don't dirty the page.  We can
-			 * set the hint, just not dirty the page as a result so the hint
-			 * is lost when we evict the page or shutdown.
+			 * condition or in general, don't dirty the page.  We can
+			 * set the hint, but must not dirty the page as a result, lest
+			 * we trigger WAL generation. Unless the page is dirtied again
+			 * later, the hint will be lost when the page is evicted, or at
+			 * shutdown.
 			 *
 			 * See src/backend/storage/page/README for longer discussion.
 			 */
-			if (RecoveryInProgress() ||
+			if (!XLogInsertAllowed() ||
 				RelFileNodeSkippingWAL(bufHdr->tag.rnode))
 				return;
 
diff --git a/src/backend/storage/freespace/freespace.c b/src/backend/storage/freespace/freespace.c
index 8c12dda2380..41ba8c59a88 100644
--- a/src/backend/storage/freespace/freespace.c
+++ b/src/backend/storage/freespace/freespace.c
@@ -24,6 +24,7 @@
 #include "postgres.h"
 
 #include "access/htup_details.h"
+#include "access/walprohibit.h"
 #include "access/xlogutils.h"
 #include "miscadmin.h"
 #include "storage/freespace.h"
@@ -285,12 +286,19 @@ FreeSpaceMapPrepareTruncateRel(Relation rel, BlockNumber nblocks)
 	 */
 	if (first_removed_slot > 0)
 	{
+		bool needwal;
+
 		buf = fsm_readbuf(rel, first_removed_address, false);
 		if (!BufferIsValid(buf))
 			return InvalidBlockNumber;	/* nothing to do; the FSM was already
 										 * smaller */
 		LockBuffer(buf, BUFFER_LOCK_EXCLUSIVE);
 
+		needwal = (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded());
+
+		if (needwal)
+			CheckWALPermitted();
+
 		/* NO EREPORT(ERROR) from here till changes are logged */
 		START_CRIT_SECTION();
 
@@ -305,7 +313,7 @@ FreeSpaceMapPrepareTruncateRel(Relation rel, BlockNumber nblocks)
 		 * during recovery.
 		 */
 		MarkBufferDirty(buf);
-		if (!InRecovery && RelationNeedsWAL(rel) && XLogHintBitIsNeeded())
+		if (needwal)
 			log_newpage_buffer(buf, false);
 
 		END_CRIT_SECTION();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 20e50247ea4..693601de238 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -793,15 +793,15 @@ LockAcquireExtended(const LOCKTAG *locktag,
 	if (lockmode <= 0 || lockmode > lockMethodTable->numLockModes)
 		elog(ERROR, "unrecognized lock mode: %d", lockmode);
 
-	if (RecoveryInProgress() && !InRecovery &&
+	if (!XLogInsertAllowed() && !InRecovery &&
 		(locktag->locktag_type == LOCKTAG_OBJECT ||
 		 locktag->locktag_type == LOCKTAG_RELATION) &&
 		lockmode > RowExclusiveLock)
 		ereport(ERROR,
 				(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
-				 errmsg("cannot acquire lock mode %s on database objects while recovery is in progress",
+				 errmsg("cannot acquire lock mode %s on database objects while system is read only",
 						lockMethodTable->lockModeNames[lockmode]),
-				 errhint("Only RowExclusiveLock or less can be acquired on database objects during recovery.")));
+				 errhint("Only RowExclusiveLock or less can be acquired on database objects while system is read only")));
 
 #ifdef LOCK_DEBUG
 	if (LOCK_DEBUG_ENABLED(locktag))
diff --git a/src/backend/utils/cache/relmapper.c b/src/backend/utils/cache/relmapper.c
index 424624cf0da..d91e247f3e4 100644
--- a/src/backend/utils/cache/relmapper.c
+++ b/src/backend/utils/cache/relmapper.c
@@ -43,6 +43,7 @@
 #include <sys/stat.h>
 #include <unistd.h>
 
+#include "access/walprohibit.h"
 #include "access/xact.h"
 #include "access/xlog.h"
 #include "access/xloginsert.h"
@@ -841,6 +842,8 @@ write_relmap_file(bool shared, RelMapFile *newmap,
 		xl_relmap_update xlrec;
 		XLogRecPtr	lsn;
 
+		AssertWALPermittedHaveXID();
+
 		/* now errors are fatal ... */
 		START_CRIT_SECTION();
 
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bdc97e3082..2ee57769835 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -95,12 +95,37 @@ extern PGDLLIMPORT volatile uint32 CritSectionCount;
 /* in tcop/postgres.c */
 extern void ProcessInterrupts(void);
 
+#ifdef USE_ASSERT_CHECKING
+typedef enum
+{
+	WALPERMIT_UNCHECKED,
+	WALPERMIT_CHECKED,
+	WALPERMIT_CHECKED_AND_USED
+} WALPermitCheckState;
+
+/* in access/walprohibit.c */
+extern WALPermitCheckState walpermit_checked_state;
+
+/*
+ * Reset walpermit_checked flag when no longer in the critical section.
+ * Otherwise, marked checked and used.
+ */
+#define RESET_WALPERMIT_CHECKED_STATE() \
+do { \
+	walpermit_checked_state = CritSectionCount ? \
+	WALPERMIT_CHECKED_AND_USED : WALPERMIT_UNCHECKED; \
+} while(0)
+#else
+#define RESET_WALPERMIT_CHECKED_STATE() ((void) 0)
+#endif
+
 #ifndef WIN32
 
 #define CHECK_FOR_INTERRUPTS() \
 do { \
 	if (unlikely(InterruptPending)) \
 		ProcessInterrupts(); \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 #else							/* WIN32 */
 
@@ -110,6 +135,7 @@ do { \
 		pgwin32_dispatch_queued_signals(); \
 	if (unlikely(InterruptPending)) \
 		ProcessInterrupts(); \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 #endif							/* WIN32 */
 
@@ -136,6 +162,7 @@ do { \
 do { \
 	Assert(CritSectionCount > 0); \
 	CritSectionCount--; \
+	RESET_WALPERMIT_CHECKED_STATE(); \
 } while(0)
 
 
-- 
2.18.0

From e008b26960c269ddfa5ec192ded4523f09bed100 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Tue, 14 Jul 2020 02:30:44 -0400
Subject: [PATCH v13 3/3] WIP - Documentation.

TODOs:

1] Documentation regarding ALTER SYSTEM READ/WRITE
---
 src/backend/access/transam/README | 60 ++++++++++++++++++++++++++++---
 src/backend/storage/page/README   | 12 +++----
 2 files changed, 61 insertions(+), 11 deletions(-)

diff --git a/src/backend/access/transam/README b/src/backend/access/transam/README
index 1edc8180c12..fe7a84f93da 100644
--- a/src/backend/access/transam/README
+++ b/src/backend/access/transam/README
@@ -442,8 +442,8 @@ to be modified.
 2. START_CRIT_SECTION()  (Any error during the next three steps must cause a
 PANIC because the shared buffers will contain unlogged changes, which we
 have to ensure don't get to disk.  Obviously, you should check conditions
-such as whether there's enough free space on the page before you start the
-critical section.)
+such as whether there's WAL write permission and enough free space on the page
+before you start the critical section.)
 
 3. Apply the required changes to the shared buffer(s).
 
@@ -486,6 +486,54 @@ with the incomplete-split flag set, it will finish the interrupted split by
 inserting the key to the parent, before proceeding.
 
 
+Read only system state
+----------------------
+
+The system state when it is not currently possible to insert write-ahead log
+records, either because the system is still in recovery or because the system
+forced to WAL prohibited by executing ALTER SYSTEM READ ONLY.  We have a
+lower-level defense in XLogBeginInsert() and elsewhere to stop us from modifying
+data during recovery when !XLogInsertAllowed(), but if XLogBeginInsert() is
+inside the critical section we must not depend on it to report an error.
+Otherwise, it will cause PANIC as mentioned previously.
+
+We do not reach the point where we try to write WAL during recovery but ALTER
+SYSTEM READ ONLY can be executed anytime by the user to stop WAL writing.  Any
+backends which receive read-only system state transition barrier interrupt need
+to stop WAL writing immediately.  For barrier absorption the backed(s) will kill
+the running transaction which has valid XID indicates that the transaction has
+performed and/or planning WAL write.  The transaction which doesn't acquire
+valid XID yet or operation such VACUUM or CONCURRENT CREATE INDEX which not
+necessary have valid XID for WAL will not be prevented while barrier processing,
+and those might hit the error from XLogBeginInsert() while trying to write WAL
+in read only system state.  To prevent such error from XLogBeginInsert() inside
+the critical section the WAL write permission has to check before
+START_CRIT_SECTION().
+
+To enforce the practice to check WAL permission before entering into critical
+section for the WAL write, we have added an assert check flag that indicates
+permission has been checked before calling XLogBeginInsert().  If not,
+XLogBeginInsert() will have assertion failure.  WAL permission check is not
+mandatory if the XLogBeginInsert() is not inside the critical section where
+throwing the error is acceptable.  To get permission check flag set either
+CheckWALPermitted(), AssertWALPermitted_HaveXID(), or AssertWALPermitted()
+should be called before START_CRIT_SECTION().  This flag automatically resets
+while exiting from the critical section.  The rule to place either of permission
+check routines will be:
+
+	The places where WAL write operation in critical can be expected without
+	having valid XID (e.g vacuum) need to protect by CheckWALPermitted(), so
+	that error can be reported outside before critical section.
+
+	The places where INSERT and UPDATE are expected which are never happened
+	without valid XID can be checked using AssertWALPermitted_HaveXID.  So that
+	non-assert build will not have the checking overhead.
+
+	The places we know that we cannot be reached in the read-only state and may
+	or may not have XID, but need to ensure the permission has been checked on
+	assert enabled build should use AssertWALPermitted().
+
+
 Constructing a WAL record
 -------------------------
 
@@ -531,7 +579,8 @@ Details of the API functions:
 
 void XLogBeginInsert(void)
 
-    Must be called before XLogRegisterBuffer and XLogRegisterData.
+	Must be called before XLogRegisterBuffer and XLogRegisterData.  WAL
+	permission must be check before calling it in a critical section.
 
 void XLogResetInsertion(void)
 
@@ -638,8 +687,9 @@ MarkBufferDirtyHint() to mark the block dirty.
 If the buffer is clean and checksums are in use then MarkBufferDirtyHint()
 inserts an XLOG_FPI_FOR_HINT record to ensure that we take a full page image
 that includes the hint. We do this to avoid a partial page write, when we
-write the dirtied page. WAL is not written during recovery, so we simply skip
-dirtying blocks because of hints when in recovery.
+write the dirtied page. WAL is not written while in read only (i.e. during
+recovery or in WAL prohibit state), so we simply skip dirtying blocks because of
+hints when in recovery.
 
 If you do decide to optimise away a WAL record, then any calls to
 MarkBufferDirty() must be replaced by MarkBufferDirtyHint(),
diff --git a/src/backend/storage/page/README b/src/backend/storage/page/README
index e30d7ac59ad..15f0bb4b7b5 100644
--- a/src/backend/storage/page/README
+++ b/src/backend/storage/page/README
@@ -56,9 +56,9 @@ WAL is a fatal error and prevents further recovery, whereas a checksum failure
 on a normal data block is a hard error but not a critical one for the server,
 even if it is a very bad thing for the user.
 
-New WAL records cannot be written during recovery, so hint bits set during
-recovery must not dirty the page if the buffer is not already dirty, when
-checksums are enabled.  Systems in Hot-Standby mode may benefit from hint bits
-being set, but with checksums enabled, a page cannot be dirtied after setting a
-hint bit (due to the torn page risk). So, it must wait for full-page images
-containing the hint bit updates to arrive from the primary.
+New WAL records cannot be written during recovery or or while in WAL prohibit
+state, so hint bits set during recovery must not dirty the page if the buffer is
+not already dirty, when checksums are enabled.  Systems in Hot-Standby mode may
+benefit from hint bits being set, but with checksums enabled, a page cannot be
+dirtied after setting a hint bit (due to the torn page risk). So, it must wait
+for full-page images containing the hint bit updates to arrive from the primary.
-- 
2.18.0