[BUG]Invalidate relcache when setting REPLICA IDENTITY

Started by tanghy.fnst@fujitsu.comabout 4 years ago17 messages
#1tanghy.fnst@fujitsu.com
tanghy.fnst@fujitsu.com
1 attachment(s)

Hi

I think I found a bug related to logical replication(REPLICA IDENTITY in specific).
If I change REPLICA IDENTITY after creating publication, the DELETE/UPDATE operations won't be replicated as expected.

For example:
-- publisher
CREATE TABLE tbl(a int, b int);
ALTER TABLE tbl ALTER COLUMN a SET NOT NULL;
CREATE UNIQUE INDEX idx_a ON tbl(a);
ALTER TABLE tbl ALTER COLUMN b SET NOT NULL;
CREATE UNIQUE INDEX idx_b ON tbl(b);
ALTER TABLE tbl REPLICA IDENTITY USING INDEX idx_a;
CREATE PUBLICATION pub FOR TABLE tbl;
ALTER TABLE tbl REPLICA IDENTITY USING INDEX idx_b;
INSERT INTO tbl VALUES (1,1);

-- subscriber
CREATE TABLE tbl(a int, b int);
ALTER TABLE tbl ALTER COLUMN b SET NOT NULL;
CREATE UNIQUE INDEX idx_b ON tbl(b);
ALTER TABLE tbl REPLICA IDENTITY USING INDEX idx_b;
CREATE SUBSCRIPTION sub CONNECTION 'dbname=postgres port=5432' PUBLICATION pub;
SELECT * FROM tbl;

-- publisher
postgres=# UPDATE tbl SET a=-a;
UPDATE 1
postgres=# SELECT * FROM tbl;
a | b
----+---
-1 | 1
(1 row)

-- subscriber
postgres=# SELECT * FROM tbl;
a | b
---+---
1 | 1
(1 row)

(a in subscriber should be -1)

But if I restart a psql client before executing UPDATE operation in publication, it works well.
So I think the problem is: when using "ALTER TABLE ... REPLICA IDENTITY USING INDEX ...", relcahe was not invalidated.

Attach a patch to fix it, I also added a test case for it.(Hou helped me to write/review this patch, which is very kind of him)

FYI: I also tested that same problem could be reproduced on PG10 ~ PG14. (Logical replication is introduced in PG10.)
So I think backpatch is needed here.

Regards
Tang

Attachments:

0001-Invalidate-relcache-when-setting-REPLICA-IDENTITY.patchapplication/octet-stream; name=0001-Invalidate-relcache-when-setting-REPLICA-IDENTITY.patchDownload
From e8cb14d97feac9d4575ded49b976e7b7a54db794 Mon Sep 17 00:00:00 2001
From: tanghy <tanghy.fnst@fujitsu.com>
Date: Wed, 10 Nov 2021 13:46:43 +0800
Subject: [PATCH] Invalidate relcache when setting REPLICA IDENTITY

Author: Haiying Tang, Hou Zhijie
Reviewed-by: Hou Zhijie

---
 src/backend/commands/tablecmds.c           |  1 +
 src/test/subscription/t/001_rep_changes.pl | 51 +++++++++++++++++++++-
 2 files changed, 50 insertions(+), 2 deletions(-)

diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 857cc5ce6e..b77112bd3a 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15488,6 +15488,7 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
 			CatalogTupleUpdate(pg_index, &pg_index_tuple->t_self, pg_index_tuple);
 			InvokeObjectPostAlterHookArg(IndexRelationId, thisIndexOid, 0,
 										 InvalidOid, is_internal);
+			CacheInvalidateRelcache(rel);
 		}
 		heap_freetuple(pg_index_tuple);
 	}
diff --git a/src/test/subscription/t/001_rep_changes.pl b/src/test/subscription/t/001_rep_changes.pl
index 9531d81f19..0be579b509 100644
--- a/src/test/subscription/t/001_rep_changes.pl
+++ b/src/test/subscription/t/001_rep_changes.pl
@@ -6,7 +6,7 @@ use strict;
 use warnings;
 use PostgreSQL::Test::Cluster;
 use PostgreSQL::Test::Utils;
-use Test::More tests => 32;
+use Test::More tests => 34;
 
 # Initialize publisher node
 my $node_publisher = PostgreSQL::Test::Cluster->new('publisher');
@@ -57,6 +57,22 @@ $node_publisher->safe_psql('postgres',
 	"CREATE INDEX idx_no_replidentity_index ON tab_no_replidentity_index(c1)"
 );
 
+# Replicate the changes with replica identity index but not primary key
+$node_publisher->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_a ON tab_replidentity_index(a)"
+);
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+# using index idx_replidentity_index_a
+$node_publisher->safe_psql('postgres',
+	"ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_a;"
+);
+$node_publisher->safe_psql('postgres',
+	"INSERT INTO tab_replidentity_index VALUES(1, 1)");
+
 # Setup structure on subscriber
 $node_subscriber->safe_psql('postgres', "CREATE TABLE tab_notrep (a int)");
 $node_subscriber->safe_psql('postgres', "CREATE TABLE tab_ins (a int)");
@@ -87,13 +103,20 @@ $node_subscriber->safe_psql('postgres',
 	"CREATE INDEX idx_no_replidentity_index ON tab_no_replidentity_index(c1)"
 );
 
+# replication of the table without replica identity index but not primary key
+$node_subscriber->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_subscriber->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+
 # Setup logical replication
 my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
 $node_publisher->safe_psql('postgres', "CREATE PUBLICATION tap_pub");
 $node_publisher->safe_psql('postgres',
 	"CREATE PUBLICATION tap_pub_ins_only WITH (publish = insert)");
 $node_publisher->safe_psql('postgres',
-	"ALTER PUBLICATION tap_pub ADD TABLE tab_rep, tab_full, tab_full2, tab_mixed, tab_include, tab_nothing, tab_full_pk, tab_no_replidentity_index"
+	"ALTER PUBLICATION tap_pub ADD TABLE tab_rep, tab_full, tab_full2, tab_mixed, tab_include, tab_nothing, tab_full_pk, tab_no_replidentity_index, tab_replidentity_index"
 );
 $node_publisher->safe_psql('postgres',
 	"ALTER PUBLICATION tap_pub_ins_only ADD TABLE tab_ins");
@@ -325,6 +348,30 @@ is( $result, qq(1|bar
 2|baz),
 	'update works with REPLICA IDENTITY FULL and a primary key');
 
+# check replication with REPLICA IDENTITY but not primary key
+# using index idx_replidentity_index_b
+$node_subscriber->safe_psql('postgres',
+);
+
+$result = $node_subscriber->safe_psql('postgres',
+	"SELECT * FROM tab_replidentity_index"),
+is( $result, qq(1|1),
+	"check initial data on subscriber");
+
+# using idx_replidentity_index_b and update
+$node_subscriber->safe_psql('postgres', qq[
+	ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b;
+]);
+$node_publisher->safe_psql('postgres', qq[
+	ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b;
+	UPDATE tab_replidentity_index SET a = -a;
+]);
+$node_publisher->wait_for_catchup('tap_sub');
+$result = $node_subscriber->safe_psql('postgres',
+	"SELECT * FROM tab_replidentity_index"),
+is( $result, qq(-1|1),
+	"update works with REPLICA IDENTITY");
+
 # Check that subscriber handles cases where update/delete target tuple
 # is missing.  We have to look for the DEBUG1 log messages about that,
 # so temporarily bump up the log verbosity.
-- 
2.18.4

#2Amit Kapila
amit.kapila16@gmail.com
In reply to: tanghy.fnst@fujitsu.com (#1)
Re: [BUG]Invalidate relcache when setting REPLICA IDENTITY

On Wed, Nov 10, 2021 at 12:42 PM tanghy.fnst@fujitsu.com
<tanghy.fnst@fujitsu.com> wrote:

Hi

I think I found a bug related to logical replication(REPLICA IDENTITY in specific).
If I change REPLICA IDENTITY after creating publication, the DELETE/UPDATE operations won't be replicated as expected.

For example:
-- publisher
CREATE TABLE tbl(a int, b int);
ALTER TABLE tbl ALTER COLUMN a SET NOT NULL;
CREATE UNIQUE INDEX idx_a ON tbl(a);
ALTER TABLE tbl ALTER COLUMN b SET NOT NULL;
CREATE UNIQUE INDEX idx_b ON tbl(b);
ALTER TABLE tbl REPLICA IDENTITY USING INDEX idx_a;
CREATE PUBLICATION pub FOR TABLE tbl;
ALTER TABLE tbl REPLICA IDENTITY USING INDEX idx_b;
INSERT INTO tbl VALUES (1,1);

-- subscriber
CREATE TABLE tbl(a int, b int);
ALTER TABLE tbl ALTER COLUMN b SET NOT NULL;
CREATE UNIQUE INDEX idx_b ON tbl(b);
ALTER TABLE tbl REPLICA IDENTITY USING INDEX idx_b;
CREATE SUBSCRIPTION sub CONNECTION 'dbname=postgres port=5432' PUBLICATION pub;
SELECT * FROM tbl;

-- publisher
postgres=# UPDATE tbl SET a=-a;
UPDATE 1
postgres=# SELECT * FROM tbl;
a | b
----+---
-1 | 1
(1 row)

-- subscriber
postgres=# SELECT * FROM tbl;
a | b
---+---
1 | 1
(1 row)

(a in subscriber should be -1)

I don't understand the purpose of idx_b in the above test case, why is
it required to reproduce the problem?

@@ -15488,6 +15488,7 @@ relation_mark_replica_identity(Relation rel,
char ri_type, Oid indexOid,
CatalogTupleUpdate(pg_index, &pg_index_tuple->t_self, pg_index_tuple);
InvokeObjectPostAlterHookArg(IndexRelationId, thisIndexOid, 0,
InvalidOid, is_internal);
+ CacheInvalidateRelcache(rel);

CatalogTupleUpdate internally calls heap_update which calls
CacheInvalidateHeapTuple(), why is that not sufficient for
invalidation?

--
With Regards,
Amit Kapila.

#3houzj.fnst@fujitsu.com
houzj.fnst@fujitsu.com
In reply to: Amit Kapila (#2)
RE: [BUG]Invalidate relcache when setting REPLICA IDENTITY

On Wed, Nov 10, 2021 7:29 PM Amit Kapila <amit.kapila16@gmail.com> wrote:

On Wed, Nov 10, 2021 at 12:42 PM tanghy.fnst@fujitsu.com
<tanghy.fnst@fujitsu.com> wrote:

Hi

I think I found a bug related to logical replication(REPLICA IDENTITY in

specific).

If I change REPLICA IDENTITY after creating publication, the

DELETE/UPDATE operations won't be replicated as expected.

For example:
-- publisher
CREATE TABLE tbl(a int, b int);
ALTER TABLE tbl ALTER COLUMN a SET NOT NULL; CREATE UNIQUE INDEX

idx_a

ON tbl(a); ALTER TABLE tbl ALTER COLUMN b SET NOT NULL; CREATE UNIQUE
INDEX idx_b ON tbl(b); ALTER TABLE tbl REPLICA IDENTITY USING INDEX
idx_a; CREATE PUBLICATION pub FOR TABLE tbl; ALTER TABLE tbl REPLICA
IDENTITY USING INDEX idx_b; INSERT INTO tbl VALUES (1,1);

-- subscriber
CREATE TABLE tbl(a int, b int);
ALTER TABLE tbl ALTER COLUMN b SET NOT NULL; CREATE UNIQUE INDEX

idx_b

ON tbl(b); ALTER TABLE tbl REPLICA IDENTITY USING INDEX idx_b; CREATE
SUBSCRIPTION sub CONNECTION 'dbname=postgres port=5432'

PUBLICATION

pub; SELECT * FROM tbl;

-- publisher
postgres=# UPDATE tbl SET a=-a;
UPDATE 1
postgres=# SELECT * FROM tbl;
a | b
----+---
-1 | 1
(1 row)

-- subscriber
postgres=# SELECT * FROM tbl;
a | b
---+---
1 | 1
(1 row)

(a in subscriber should be -1)

I don't understand the purpose of idx_b in the above test case, why is it
required to reproduce the problem?
@@ -15488,6 +15488,7 @@ relation_mark_replica_identity(Relation rel, char
ri_type, Oid indexOid,
CatalogTupleUpdate(pg_index, &pg_index_tuple->t_self, pg_index_tuple);
InvokeObjectPostAlterHookArg(IndexRelationId, thisIndexOid, 0,
InvalidOid, is_internal);
+ CacheInvalidateRelcache(rel);

CatalogTupleUpdate internally calls heap_update which calls
CacheInvalidateHeapTuple(), why is that not sufficient for invalidation?

I think it's because the bug happens only when changing REPLICA IDENTITY index
from one(idx_a) to another one(idx_b).

When first time specify the REPLICA IDENTITY index, the code works well because
it will invoke 'CatalogTupleUpdate(pg_class,...' Which will invalidate the
target table's relcache.

if (pg_class_form->relreplident != ri_type)
{
pg_class_form->relreplident = ri_type;
CatalogTupleUpdate(pg_class, &pg_class_tuple->t_self, pg_class_tuple);
}

But when changing REPLICA IDENTITY index, the code only invoke
'CatalogTupleUpdate(pg_index,' which seems only invalidate the index's cache not the
target table.

if (dirty)
{
CatalogTupleUpdate(pg_index, &pg_index_tuple->t_self, pg_index_tuple);
InvokeObjectPostAlterHookArg(IndexRelationId, thisIndexOid, 0,
InvalidOid, is_internal);
}

Best regards,
Hou zj

#4Amit Kapila
amit.kapila16@gmail.com
In reply to: houzj.fnst@fujitsu.com (#3)
Re: [BUG]Invalidate relcache when setting REPLICA IDENTITY

On Thu, Nov 11, 2021 at 7:07 AM houzj.fnst@fujitsu.com
<houzj.fnst@fujitsu.com> wrote:

On Wed, Nov 10, 2021 7:29 PM Amit Kapila <amit.kapila16@gmail.com> wrote:

I don't understand the purpose of idx_b in the above test case, why is it
required to reproduce the problem?
@@ -15488,6 +15488,7 @@ relation_mark_replica_identity(Relation rel, char
ri_type, Oid indexOid,
CatalogTupleUpdate(pg_index, &pg_index_tuple->t_self, pg_index_tuple);
InvokeObjectPostAlterHookArg(IndexRelationId, thisIndexOid, 0,
InvalidOid, is_internal);
+ CacheInvalidateRelcache(rel);

CatalogTupleUpdate internally calls heap_update which calls
CacheInvalidateHeapTuple(), why is that not sufficient for invalidation?

I think it's because the bug happens only when changing REPLICA IDENTITY index
from one(idx_a) to another one(idx_b).

Okay, but do we need to invalidate the rel cache each time the dirty
flag is set? Can't we do it once outside the foreach index loop? I
think we can record whether to do relcache invalidation in a new
boolean variable need_rel_inval or something like that. Also, it is
better if you can add a comment on the lines of: "Invalidate the
relcache for the table so that after we commit all sessions will
refresh the table's replica identity index before attempting any
update on the table.".

--
With Regards,
Amit Kapila.

#5Amit Kapila
amit.kapila16@gmail.com
In reply to: Amit Kapila (#4)
Re: [BUG]Invalidate relcache when setting REPLICA IDENTITY

On Thu, Nov 11, 2021 at 9:37 AM Amit Kapila <amit.kapila16@gmail.com> wrote:

On Thu, Nov 11, 2021 at 7:07 AM houzj.fnst@fujitsu.com
<houzj.fnst@fujitsu.com> wrote:

On Wed, Nov 10, 2021 7:29 PM Amit Kapila <amit.kapila16@gmail.com> wrote:

I don't understand the purpose of idx_b in the above test case, why is it
required to reproduce the problem?
@@ -15488,6 +15488,7 @@ relation_mark_replica_identity(Relation rel, char
ri_type, Oid indexOid,
CatalogTupleUpdate(pg_index, &pg_index_tuple->t_self, pg_index_tuple);
InvokeObjectPostAlterHookArg(IndexRelationId, thisIndexOid, 0,
InvalidOid, is_internal);
+ CacheInvalidateRelcache(rel);

CatalogTupleUpdate internally calls heap_update which calls
CacheInvalidateHeapTuple(), why is that not sufficient for invalidation?

I think it's because the bug happens only when changing REPLICA IDENTITY index
from one(idx_a) to another one(idx_b).

Okay, but do we need to invalidate the rel cache each time the dirty
flag is set? Can't we do it once outside the foreach index loop? I
think we can record whether to do relcache invalidation in a new
boolean variable need_rel_inval or something like that. Also, it is
better if you can add a comment on the lines of: "Invalidate the
relcache for the table so that after we commit all sessions will
refresh the table's replica identity index before attempting any
update on the table.".

Few comments on testcase added by the patch:
==================================
1.
+$node_subscriber->safe_psql('postgres',
+);

It is not clear what is the purpose served by this statement.

2.
+# replication of the table without replica identity index but not primary key
+$node_subscriber->safe_psql('postgres',
+ "CREATE TABLE tab_replidentity_index(a int not null, b int not null)");

I think the part of the comment "but not primary key" is not required.

3. Can we move this test to subscription/t/100_bugs?

--
With Regards,
Amit Kapila.

#6houzj.fnst@fujitsu.com
houzj.fnst@fujitsu.com
In reply to: Amit Kapila (#4)
1 attachment(s)
RE: [BUG]Invalidate relcache when setting REPLICA IDENTITY

On Thur, Nov 11, 2021 12:08 PM Amit Kapila <amit.kapila16@gmail.com> wrote:

On Thu, Nov 11, 2021 at 7:07 AM houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com> wrote:

On Wed, Nov 10, 2021 7:29 PM Amit Kapila <amit.kapila16@gmail.com> wrote:

I don't understand the purpose of idx_b in the above test case, why is it
required to reproduce the problem?
@@ -15488,6 +15488,7 @@ relation_mark_replica_identity(Relation rel,

char

ri_type, Oid indexOid,
CatalogTupleUpdate(pg_index, &pg_index_tuple->t_self,

pg_index_tuple);

InvokeObjectPostAlterHookArg(IndexRelationId, thisIndexOid, 0,
InvalidOid, is_internal);
+ CacheInvalidateRelcache(rel);

CatalogTupleUpdate internally calls heap_update which calls
CacheInvalidateHeapTuple(), why is that not sufficient for invalidation?

I think it's because the bug happens only when changing REPLICA IDENTITY

index

from one(idx_a) to another one(idx_b).

Okay, but do we need to invalidate the rel cache each time the dirty
flag is set? Can't we do it once outside the foreach index loop? I
think we can record whether to do relcache invalidation in a new
boolean variable need_rel_inval or something like that. Also, it is
better if you can add a comment on the lines of: "Invalidate the
relcache for the table so that after we commit all sessions will
refresh the table's replica identity index before attempting any
update on the table.".

I agree.

Attach the new version fix patch which move the invalidation out of the loop
and addressed the comments[1]/messages/by-id/CAA4eK1Ki-kUx52uRER3bZSC6bLc=iix+QnBxs1pZSHHYZOEF1A@mail.gmail.com for the testcases.

[1]: /messages/by-id/CAA4eK1Ki-kUx52uRER3bZSC6bLc=iix+QnBxs1pZSHHYZOEF1A@mail.gmail.com

Best regards,
Hou zj

Attachments:

v2-0001-Invalidate-relcache-when-setting-REPLICA-IDENTITY.patchapplication/octet-stream; name=v2-0001-Invalidate-relcache-when-setting-REPLICA-IDENTITY.patchDownload
From 0e650ef40424ffd36dfbb18d92635d8c5b91c377 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@fujitsu.com>
Date: Thu, 11 Nov 2021 17:31:05 +0800
Subject: [PATCH] Invalidate relcache when setting REPLICA IDENTITY

When changing REPLICA IDENTITY INDEX to another one, the target table's
relcache would not be invalidated which could cause unexpected behaviour when
replicating UPDATE change.

So, Invalidate the relcache in this case to make sure the the bitmap of index
attribute is rebuilt

Author: Tang Haiying, Hou Zhijie
Reviewed-by: Amit Kapila

---
 src/backend/commands/tablecmds.c    | 24 +++++++---
 src/test/subscription/t/100_bugs.pl | 69 ++++++++++++++++++++++++++++-
 2 files changed, 85 insertions(+), 8 deletions(-)

diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 857cc5ce6e..8306fd1eb8 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15412,6 +15412,7 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
 	Form_pg_index pg_index_form;
 
 	ListCell   *index;
+	bool		need_rel_inval = false;
 
 	/*
 	 * Check whether relreplident has changed, and update it if so.
@@ -15488,10 +15489,19 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
 			CatalogTupleUpdate(pg_index, &pg_index_tuple->t_self, pg_index_tuple);
 			InvokeObjectPostAlterHookArg(IndexRelationId, thisIndexOid, 0,
 										 InvalidOid, is_internal);
+			need_rel_inval = true;
 		}
 		heap_freetuple(pg_index_tuple);
 	}
 
+	/*
+	 * Invalidate the relcache for the table, so that after we commit all
+	 * sessions will refresh the table's replica identity index before
+	 * attempting any update on the table.
+	 */
+	if (need_rel_inval)
+		CacheInvalidateRelcache(rel);
+
 	table_close(pg_index, RowExclusiveLock);
 }
 
@@ -17525,12 +17535,12 @@ ATExecAttachPartition(List **wqueue, Relation rel, PartitionCmd *cmd,
 	/*
 	 * If the partition we just attached is partitioned itself, invalidate
 	 * relcache for all descendent partitions too to ensure that their
-	 * rd_partcheck expression trees are rebuilt; partitions already locked
-	 * at the beginning of this function.
+	 * rd_partcheck expression trees are rebuilt; partitions already locked at
+	 * the beginning of this function.
 	 */
 	if (attachrel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
-		ListCell *l;
+		ListCell   *l;
 
 		foreach(l, attachrel_children)
 		{
@@ -18232,13 +18242,13 @@ DetachPartitionFinalize(Relation rel, Relation partRel, bool concurrent,
 	/*
 	 * If the partition we just detached is partitioned itself, invalidate
 	 * relcache for all descendent partitions too to ensure that their
-	 * rd_partcheck expression trees are rebuilt; must lock partitions
-	 * before doing so, using the same lockmode as what partRel has been
-	 * locked with by the caller.
+	 * rd_partcheck expression trees are rebuilt; must lock partitions before
+	 * doing so, using the same lockmode as what partRel has been locked with
+	 * by the caller.
 	 */
 	if (partRel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
-		List   *children;
+		List	   *children;
 
 		children = find_all_inheritors(RelationGetRelid(partRel),
 									   AccessExclusiveLock, NULL);
diff --git a/src/test/subscription/t/100_bugs.pl b/src/test/subscription/t/100_bugs.pl
index 34a60fd9ab..a62b16959e 100644
--- a/src/test/subscription/t/100_bugs.pl
+++ b/src/test/subscription/t/100_bugs.pl
@@ -6,7 +6,7 @@ use strict;
 use warnings;
 use PostgreSQL::Test::Cluster;
 use PostgreSQL::Test::Utils;
-use Test::More tests => 5;
+use Test::More tests => 7;
 
 # Bug #15114
 
@@ -224,3 +224,70 @@ $node_sub->safe_psql('postgres', "DROP TABLE tab1");
 $node_pub->stop('fast');
 $node_pub_sub->stop('fast');
 $node_sub->stop('fast');
+
+# Test the replication of the table after changing REPLICA IDENTITY INDEX
+
+# The bug was that when changing REPLICA IDENTITY INDEX to another one, the
+# target table's relcache would not be invalidated. This could cause unexpected
+# behaviour when replicating UPDATE change.
+$node_publisher = PostgreSQL::Test::Cluster->new('publisher3');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->start;
+
+$node_subscriber = PostgreSQL::Test::Cluster->new('subscriber3');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->start;
+
+$node_publisher->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_a ON tab_replidentity_index(a)"
+);
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+
+# use index idx_replidentity_index_a as REPLICA IDENTITY in publisher.
+$node_publisher->safe_psql('postgres',
+	"ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_a;"
+);
+$node_publisher->safe_psql('postgres',
+	"INSERT INTO tab_replidentity_index VALUES(1, 1)");
+
+$node_subscriber->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_subscriber->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+
+$publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+	"CREATE PUBLICATION tap_pub FOR TABLE tab_replidentity_index");
+$node_subscriber->safe_psql('postgres',
+	"CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub"
+);
+
+$node_publisher->wait_for_catchup('tap_sub');
+
+is($node_subscriber->safe_psql('postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(1|1),
+	"check initial data on subscriber");
+
+# use index idx_replidentity_index_a as REPLICA IDENTITY in subcriber.
+$node_subscriber->safe_psql('postgres', qq[
+	ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b;
+]);
+
+# Set publisher's REPLICA IDENTITY to idx_replidentity_index_b, then UPDATE the table.
+$node_publisher->safe_psql('postgres', qq[
+	ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b;
+	UPDATE tab_replidentity_index SET a = -a;
+]);
+
+$node_publisher->wait_for_catchup('tap_sub');
+is( $node_subscriber->safe_psql('postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(-1|1),
+	"update works with REPLICA IDENTITY");
+
+$node_publisher->stop('fast');
+$node_subscriber->stop('fast');
-- 
2.18.4

#7houzj.fnst@fujitsu.com
houzj.fnst@fujitsu.com
In reply to: houzj.fnst@fujitsu.com (#6)
6 attachment(s)
RE: [BUG]Invalidate relcache when setting REPLICA IDENTITY

On Thursday, November 11, 2021 5:36 PM houzj.fnst@fujitsu.com wrote:

On Thur, Nov 11, 2021 12:08 PM Amit Kapila <amit.kapila16@gmail.com> wrote:

On Thu, Nov 11, 2021 at 7:07 AM houzj.fnst@fujitsu.com

<houzj.fnst@fujitsu.com> wrote:

On Wed, Nov 10, 2021 7:29 PM Amit Kapila <amit.kapila16@gmail.com>

wrote:

I don't understand the purpose of idx_b in the above test case,
why is it required to reproduce the problem?
@@ -15488,6 +15488,7 @@ relation_mark_replica_identity(Relation
rel,

char

ri_type, Oid indexOid,
CatalogTupleUpdate(pg_index, &pg_index_tuple->t_self,

pg_index_tuple);

InvokeObjectPostAlterHookArg(IndexRelationId, thisIndexOid, 0,
InvalidOid, is_internal);
+ CacheInvalidateRelcache(rel);

CatalogTupleUpdate internally calls heap_update which calls
CacheInvalidateHeapTuple(), why is that not sufficient for invalidation?

I think it's because the bug happens only when changing REPLICA
IDENTITY

index

from one(idx_a) to another one(idx_b).

Okay, but do we need to invalidate the rel cache each time the dirty
flag is set? Can't we do it once outside the foreach index loop? I
think we can record whether to do relcache invalidation in a new
boolean variable need_rel_inval or something like that. Also, it is
better if you can add a comment on the lines of: "Invalidate the
relcache for the table so that after we commit all sessions will
refresh the table's replica identity index before attempting any
update on the table.".

I agree.

Attach the new version fix patch which move the invalidation out of the loop and
addressed the comments[1] for the testcases.

Also attach the patches for back branch and remove some unnecessary
changes from pgindent.

Best regards,
Hou zj

Attachments:

PG12-0001-Invalidate-relcache-when-setting-REPLICA-IDENTITY.patchapplication/octet-stream; name=PG12-0001-Invalidate-relcache-when-setting-REPLICA-IDENTITY.patchDownload
From 98dc03ff5f03f1c75e577fdbc0c2dfb44fe01de7 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Thu, 11 Nov 2021 19:34:33 +0800
Subject: [PATCH] Invalidate relcache when setting REPLICA IDENTITY

When changing REPLICA IDENTITY INDEX to another one, the target table's
relcache would not be invalidated which could cause unexpected behaviour when
replicating UPDATE change.

So, Invalidate the relcache in this case to make sure the the bitmap of index
attribute is rebuilt

Author: Tang Haiying, Hou Zhijie
Reviewed-by: Amit Kapila
Backpatch-through: 10, where it was introduced
Discussion: https://www.postgresql.org/message-id/OS0PR01MB61133CA11630DAE45BC6AD95FB939%40OS0PR01MB6113.jpnprd01.prod.outlook.com

---
 src/backend/commands/tablecmds.c    | 10 ++++++
 src/test/subscription/t/100_bugs.pl | 69 ++++++++++++++++++++++++++++++++++++-
 2 files changed, 78 insertions(+), 1 deletion(-)

diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index fe6ee63..4d4df14 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -14335,6 +14335,7 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
 	Form_pg_index pg_index_form;
 
 	ListCell   *index;
+	bool		need_rel_inval = false;
 
 	/*
 	 * Check whether relreplident has changed, and update it if so.
@@ -14411,10 +14412,19 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
 			CatalogTupleUpdate(pg_index, &pg_index_tuple->t_self, pg_index_tuple);
 			InvokeObjectPostAlterHookArg(IndexRelationId, thisIndexOid, 0,
 										 InvalidOid, is_internal);
+			need_rel_inval = true;
 		}
 		heap_freetuple(pg_index_tuple);
 	}
 
+	/*
+	 * Invalidate the relcache for the table, so that after we commit all
+	 * sessions will refresh the table's replica identity index before
+	 * attempting any update on the table.
+	 */
+	if (need_rel_inval)
+		CacheInvalidateRelcache(rel);
+
 	table_close(pg_index, RowExclusiveLock);
 }
 
diff --git a/src/test/subscription/t/100_bugs.pl b/src/test/subscription/t/100_bugs.pl
index 366a7a9..2993a08 100644
--- a/src/test/subscription/t/100_bugs.pl
+++ b/src/test/subscription/t/100_bugs.pl
@@ -3,7 +3,7 @@ use strict;
 use warnings;
 use PostgresNode;
 use TestLib;
-use Test::More tests => 3;
+use Test::More tests => 5;
 
 # Bug #15114
 
@@ -100,3 +100,70 @@ is( $node_publisher->psql(
 );
 
 $node_publisher->stop('fast');
+
+# Test the replication of the table after changing REPLICA IDENTITY INDEX
+
+# The bug was that when changing REPLICA IDENTITY INDEX to another one, the
+# target table's relcache would not be invalidated. This could cause unexpected
+# behaviour when replicating UPDATE change.
+$node_publisher = get_new_node('publisher3');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->start;
+
+$node_subscriber = get_new_node('subscriber3');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->start;
+
+$node_publisher->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_a ON tab_replidentity_index(a)"
+);
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+
+# use index idx_replidentity_index_a as REPLICA IDENTITY in publisher.
+$node_publisher->safe_psql('postgres',
+	"ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_a;"
+);
+$node_publisher->safe_psql('postgres',
+	"INSERT INTO tab_replidentity_index VALUES(1, 1)");
+
+$node_subscriber->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_subscriber->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+
+$publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+	"CREATE PUBLICATION tap_pub FOR TABLE tab_replidentity_index");
+$node_subscriber->safe_psql('postgres',
+	"CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub"
+);
+
+$node_publisher->wait_for_catchup('tap_sub');
+
+is($node_subscriber->safe_psql('postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(1|1),
+	"check initial data on subscriber");
+
+# use index idx_replidentity_index_a as REPLICA IDENTITY in subcriber.
+$node_subscriber->safe_psql('postgres', qq[
+	ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b;
+]);
+
+# Set publisher's REPLICA IDENTITY to idx_replidentity_index_b, then UPDATE the table.
+$node_publisher->safe_psql('postgres', qq[
+	ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b;
+	UPDATE tab_replidentity_index SET a = -a;
+]);
+
+$node_publisher->wait_for_catchup('tap_sub');
+is( $node_subscriber->safe_psql('postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(-1|1),
+	"update works with REPLICA IDENTITY");
+
+$node_publisher->stop('fast');
+$node_subscriber->stop('fast');
-- 
2.7.2.windows.1

PG11-0001-Invalidate-relcache-when-setting-REPLICA-IDENTITY.patchapplication/octet-stream; name=PG11-0001-Invalidate-relcache-when-setting-REPLICA-IDENTITY.patchDownload
From ad89eb1182d8260fada07ee32a5375f7d5e2422e Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Thu, 11 Nov 2021 19:44:14 +0800
Subject: [PATCH] Invalidate relcache when setting REPLICA IDENTITY

When changing REPLICA IDENTITY INDEX to another one, the target table's
relcache would not be invalidated which could cause unexpected behaviour when
replicating UPDATE change.

So, Invalidate the relcache in this case to make sure the the bitmap of index
attribute is rebuilt

Author: Tang Haiying, Hou Zhijie
Reviewed-by: Amit Kapila
Backpatch-through: 10, where it was introduced
Discussion: https://www.postgresql.org/message-id/OS0PR01MB61133CA11630DAE45BC6AD95FB939%40OS0PR01MB6113.jpnprd01.prod.outlook.com

---
 src/backend/commands/tablecmds.c    | 10 ++++++
 src/test/subscription/t/100_bugs.pl | 69 ++++++++++++++++++++++++++++++++++++-
 2 files changed, 78 insertions(+), 1 deletion(-)

diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 62de9ba..3d65eb0 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -13322,6 +13322,7 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
 	Form_pg_index pg_index_form;
 
 	ListCell   *index;
+	bool		need_rel_inval = false;
 
 	/*
 	 * Check whether relreplident has changed, and update it if so.
@@ -13398,10 +13399,19 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
 			CatalogTupleUpdate(pg_index, &pg_index_tuple->t_self, pg_index_tuple);
 			InvokeObjectPostAlterHookArg(IndexRelationId, thisIndexOid, 0,
 										 InvalidOid, is_internal);
+			need_rel_inval = true;
 		}
 		heap_freetuple(pg_index_tuple);
 	}
 
+	/*
+	 * Invalidate the relcache for the table, so that after we commit all
+	 * sessions will refresh the table's replica identity index before
+	 * attempting any update on the table.
+	 */
+	if (need_rel_inval)
+		CacheInvalidateRelcache(rel);
+
 	heap_close(pg_index, RowExclusiveLock);
 }
 
diff --git a/src/test/subscription/t/100_bugs.pl b/src/test/subscription/t/100_bugs.pl
index 9ef42c3..79a921c 100644
--- a/src/test/subscription/t/100_bugs.pl
+++ b/src/test/subscription/t/100_bugs.pl
@@ -3,7 +3,7 @@ use strict;
 use warnings;
 use PostgresNode;
 use TestLib;
-use Test::More tests => 3;
+use Test::More tests => 5;
 
 # Bug #15114
 
@@ -98,3 +98,70 @@ is( $node_publisher->psql(
 );
 
 $node_publisher->stop('fast');
+
+# Test the replication of the table after changing REPLICA IDENTITY INDEX
+
+# The bug was that when changing REPLICA IDENTITY INDEX to another one, the
+# target table's relcache would not be invalidated. This could cause unexpected
+# behaviour when replicating UPDATE change.
+$node_publisher = get_new_node('publisher3');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->start;
+
+$node_subscriber = get_new_node('subscriber3');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->start;
+
+$node_publisher->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_a ON tab_replidentity_index(a)"
+);
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+
+# use index idx_replidentity_index_a as REPLICA IDENTITY in publisher.
+$node_publisher->safe_psql('postgres',
+	"ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_a;"
+);
+$node_publisher->safe_psql('postgres',
+	"INSERT INTO tab_replidentity_index VALUES(1, 1)");
+
+$node_subscriber->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_subscriber->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+
+$publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+	"CREATE PUBLICATION tap_pub FOR TABLE tab_replidentity_index");
+$node_subscriber->safe_psql('postgres',
+	"CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr application_name=tap_sub' PUBLICATION tap_pub"
+);
+
+$node_publisher->wait_for_catchup('tap_sub');
+
+is($node_subscriber->safe_psql('postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(1|1),
+	"check initial data on subscriber");
+
+# use index idx_replidentity_index_a as REPLICA IDENTITY in subcriber.
+$node_subscriber->safe_psql('postgres', qq[
+	ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b;
+]);
+
+# Set publisher's REPLICA IDENTITY to idx_replidentity_index_b, then UPDATE the table.
+$node_publisher->safe_psql('postgres', qq[
+	ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b;
+	UPDATE tab_replidentity_index SET a = -a;
+]);
+
+$node_publisher->wait_for_catchup('tap_sub');
+is( $node_subscriber->safe_psql('postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(-1|1),
+	"update works with REPLICA IDENTITY");
+
+$node_publisher->stop('fast');
+$node_subscriber->stop('fast');
-- 
2.7.2.windows.1

PG10-0001-Invalidate-relcache-when-setting-REPLICA-IDENTITY.patchapplication/octet-stream; name=PG10-0001-Invalidate-relcache-when-setting-REPLICA-IDENTITY.patchDownload
From b68cfe18a10f2003e9b3c4fbe3c339b464bd2fba Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Thu, 11 Nov 2021 19:30:14 +0800
Subject: [PATCH] Invalidate relcache when setting REPLICA IDENTITY

When changing REPLICA IDENTITY INDEX to another one, the target table's
relcache would not be invalidated which could cause unexpected behaviour when
replicating UPDATE change.

So, Invalidate the relcache in this case to make sure the the bitmap of index
attribute is rebuilt

Author: Tang Haiying, Hou Zhijie
Reviewed-by: Amit Kapila
Backpatch-through: 10, where it was introduced
Discussion: https://www.postgresql.org/message-id/OS0PR01MB61133CA11630DAE45BC6AD95FB939%40OS0PR01MB6113.jpnprd01.prod.outlook.com

---
 src/backend/commands/tablecmds.c    | 10 ++++++
 src/test/subscription/t/100_bugs.pl | 69 ++++++++++++++++++++++++++++++++++++-
 2 files changed, 78 insertions(+), 1 deletion(-)

diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 7163ad5..ba2adf3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -12124,6 +12124,7 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
 	Form_pg_index pg_index_form;
 
 	ListCell   *index;
+	bool		need_rel_inval = false;
 
 	/*
 	 * Check whether relreplident has changed, and update it if so.
@@ -12200,10 +12201,19 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
 			CatalogTupleUpdate(pg_index, &pg_index_tuple->t_self, pg_index_tuple);
 			InvokeObjectPostAlterHookArg(IndexRelationId, thisIndexOid, 0,
 										 InvalidOid, is_internal);
+			need_rel_inval = true;
 		}
 		heap_freetuple(pg_index_tuple);
 	}
 
+	/*
+	 * Invalidate the relcache for the table, so that after we commit all
+	 * sessions will refresh the table's replica identity index before
+	 * attempting any update on the table.
+	 */
+	if (need_rel_inval)
+		CacheInvalidateRelcache(rel);
+
 	heap_close(pg_index, RowExclusiveLock);
 }
 
diff --git a/src/test/subscription/t/100_bugs.pl b/src/test/subscription/t/100_bugs.pl
index b675f16..7f1ce06 100644
--- a/src/test/subscription/t/100_bugs.pl
+++ b/src/test/subscription/t/100_bugs.pl
@@ -3,7 +3,7 @@ use strict;
 use warnings;
 use PostgresNode;
 use TestLib;
-use Test::More tests => 3;
+use Test::More tests => 5;
 
 sub wait_for_caught_up
 {
@@ -107,3 +107,70 @@ is( $node_publisher->psql(
 );
 
 $node_publisher->stop('fast');
+
+# Test the replication of the table after changing REPLICA IDENTITY INDEX
+
+# The bug was that when changing REPLICA IDENTITY INDEX to another one, the
+# target table's relcache would not be invalidated. This could cause unexpected
+# behaviour when replicating UPDATE change.
+$node_publisher = get_new_node('publisher3');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->start;
+
+$node_subscriber = get_new_node('subscriber3');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->start;
+
+$node_publisher->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_a ON tab_replidentity_index(a)"
+);
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+
+# use index idx_replidentity_index_a as REPLICA IDENTITY in publisher.
+$node_publisher->safe_psql('postgres',
+	"ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_a;"
+);
+$node_publisher->safe_psql('postgres',
+	"INSERT INTO tab_replidentity_index VALUES(1, 1)");
+
+$node_subscriber->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_subscriber->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+
+$publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+	"CREATE PUBLICATION tap_pub FOR TABLE tab_replidentity_index");
+$node_subscriber->safe_psql('postgres',
+	"CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr application_name=tap_sub' PUBLICATION tap_pub"
+);
+
+wait_for_caught_up($node_publisher, 'tap_sub');
+
+is($node_subscriber->safe_psql('postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(1|1),
+	"check initial data on subscriber");
+
+# use index idx_replidentity_index_a as REPLICA IDENTITY in subcriber.
+$node_subscriber->safe_psql('postgres', qq[
+	ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b;
+]);
+
+# Set publisher's REPLICA IDENTITY to idx_replidentity_index_b, then UPDATE the table.
+$node_publisher->safe_psql('postgres', qq[
+	ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b;
+	UPDATE tab_replidentity_index SET a = -a;
+]);
+
+wait_for_caught_up($node_publisher, 'tap_sub');
+is( $node_subscriber->safe_psql('postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(-1|1),
+	"update works with REPLICA IDENTITY");
+
+$node_publisher->stop('fast');
+$node_subscriber->stop('fast');
-- 
2.7.2.windows.1

PG14-0001-Invalidate-relcache-when-setting-REPLICA-IDENTITY.patchapplication/octet-stream; name=PG14-0001-Invalidate-relcache-when-setting-REPLICA-IDENTITY.patchDownload
From 7b4ba6123579d7f50e6e586b447389c851ed0e0d Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Thu, 11 Nov 2021 18:27:11 +0800
Subject: [PATCH] Invalidate relcache when setting REPLICA IDENTITY

When changing REPLICA IDENTITY INDEX to another one, the target table's
relcache would not be invalidated which could cause unexpected behaviour when
replicating UPDATE change.

So, Invalidate the relcache in this case to make sure the the bitmap of index
attribute is rebuilt

Author: Tang Haiying, Hou Zhijie
Reviewed-by: Amit Kapila
Backpatch-through: 10, where it was introduced
Discussion: https://www.postgresql.org/message-id/OS0PR01MB61133CA11630DAE45BC6AD95FB939%40OS0PR01MB6113.jpnprd01.prod.outlook.com

---
 src/backend/commands/tablecmds.c    | 10 ++++++
 src/test/subscription/t/100_bugs.pl | 69 ++++++++++++++++++++++++++++++++++++-
 2 files changed, 78 insertions(+), 1 deletion(-)

diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 0d34f70..8d38382 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15268,6 +15268,7 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
 	Form_pg_index pg_index_form;
 
 	ListCell   *index;
+	bool		need_rel_inval = false;
 
 	/*
 	 * Check whether relreplident has changed, and update it if so.
@@ -15344,10 +15345,19 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
 			CatalogTupleUpdate(pg_index, &pg_index_tuple->t_self, pg_index_tuple);
 			InvokeObjectPostAlterHookArg(IndexRelationId, thisIndexOid, 0,
 										 InvalidOid, is_internal);
+			need_rel_inval = true;
 		}
 		heap_freetuple(pg_index_tuple);
 	}
 
+	/*
+	 * Invalidate the relcache for the table, so that after we commit all
+	 * sessions will refresh the table's replica identity index before
+	 * attempting any update on the table.
+	 */
+	if (need_rel_inval)
+		CacheInvalidateRelcache(rel);
+
 	table_close(pg_index, RowExclusiveLock);
 }
 
diff --git a/src/test/subscription/t/100_bugs.pl b/src/test/subscription/t/100_bugs.pl
index 21eabce..9ebdd58 100644
--- a/src/test/subscription/t/100_bugs.pl
+++ b/src/test/subscription/t/100_bugs.pl
@@ -6,7 +6,7 @@ use strict;
 use warnings;
 use PostgresNode;
 use TestLib;
-use Test::More tests => 5;
+use Test::More tests => 7;
 
 # Bug #15114
 
@@ -224,3 +224,70 @@ $node_sub->safe_psql('postgres', "DROP TABLE tab1");
 $node_pub->stop('fast');
 $node_pub_sub->stop('fast');
 $node_sub->stop('fast');
+
+# Test the replication of the table after changing REPLICA IDENTITY INDEX
+
+# The bug was that when changing REPLICA IDENTITY INDEX to another one, the
+# target table's relcache would not be invalidated. This could cause unexpected
+# behaviour when replicating UPDATE change.
+$node_publisher = get_new_node('publisher3');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->start;
+
+$node_subscriber = get_new_node('subscriber3');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->start;
+
+$node_publisher->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_a ON tab_replidentity_index(a)"
+);
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+
+# use index idx_replidentity_index_a as REPLICA IDENTITY in publisher.
+$node_publisher->safe_psql('postgres',
+	"ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_a;"
+);
+$node_publisher->safe_psql('postgres',
+	"INSERT INTO tab_replidentity_index VALUES(1, 1)");
+
+$node_subscriber->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_subscriber->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+
+$publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+	"CREATE PUBLICATION tap_pub FOR TABLE tab_replidentity_index");
+$node_subscriber->safe_psql('postgres',
+	"CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub"
+);
+
+$node_publisher->wait_for_catchup('tap_sub');
+
+is($node_subscriber->safe_psql('postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(1|1),
+	"check initial data on subscriber");
+
+# use index idx_replidentity_index_a as REPLICA IDENTITY in subcriber.
+$node_subscriber->safe_psql('postgres', qq[
+	ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b;
+]);
+
+# Set publisher's REPLICA IDENTITY to idx_replidentity_index_b, then UPDATE the table.
+$node_publisher->safe_psql('postgres', qq[
+	ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b;
+	UPDATE tab_replidentity_index SET a = -a;
+]);
+
+$node_publisher->wait_for_catchup('tap_sub');
+is( $node_subscriber->safe_psql('postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(-1|1),
+	"update works with REPLICA IDENTITY");
+
+$node_publisher->stop('fast');
+$node_subscriber->stop('fast');
-- 
2.7.2.windows.1

PG13-0001-Invalidate-relcache-when-setting-REPLICA-IDENTITY.patchapplication/octet-stream; name=PG13-0001-Invalidate-relcache-when-setting-REPLICA-IDENTITY.patchDownload
From 4a5b61e2977e53f9fe316cd7c013c4a12f514fba Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Thu, 11 Nov 2021 18:31:08 +0800
Subject: [PATCH] Invalidate relcache when setting REPLICA IDENTITY

When changing REPLICA IDENTITY INDEX to another one, the target table's
relcache would not be invalidated which could cause unexpected behaviour when
replicating UPDATE change.

So, Invalidate the relcache in this case to make sure the the bitmap of index
attribute is rebuilt

Author: Tang Haiying, Hou Zhijie
Reviewed-by: Amit Kapila
Backpatch-through: 10, where it was introduced
Discussion: https://www.postgresql.org/message-id/OS0PR01MB61133CA11630DAE45BC6AD95FB939%40OS0PR01MB6113.jpnprd01.prod.outlook.com

---
 src/backend/commands/tablecmds.c    | 10 ++++++
 src/test/subscription/t/100_bugs.pl | 69 ++++++++++++++++++++++++++++++++++++-
 2 files changed, 78 insertions(+), 1 deletion(-)

diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index d492f27..067494e 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -14815,6 +14815,7 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
 	Form_pg_index pg_index_form;
 
 	ListCell   *index;
+	bool		need_rel_inval = false;
 
 	/*
 	 * Check whether relreplident has changed, and update it if so.
@@ -14891,10 +14892,19 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
 			CatalogTupleUpdate(pg_index, &pg_index_tuple->t_self, pg_index_tuple);
 			InvokeObjectPostAlterHookArg(IndexRelationId, thisIndexOid, 0,
 										 InvalidOid, is_internal);
+			need_rel_inval = true;
 		}
 		heap_freetuple(pg_index_tuple);
 	}
 
+	/*
+	 * Invalidate the relcache for the table, so that after we commit all
+	 * sessions will refresh the table's replica identity index before
+	 * attempting any update on the table.
+	 */
+	if (need_rel_inval)
+		CacheInvalidateRelcache(rel);
+
 	table_close(pg_index, RowExclusiveLock);
 }
 
diff --git a/src/test/subscription/t/100_bugs.pl b/src/test/subscription/t/100_bugs.pl
index d1e407a..a7a4e6a 100644
--- a/src/test/subscription/t/100_bugs.pl
+++ b/src/test/subscription/t/100_bugs.pl
@@ -3,7 +3,7 @@ use strict;
 use warnings;
 use PostgresNode;
 use TestLib;
-use Test::More tests => 5;
+use Test::More tests => 7;
 
 # Bug #15114
 
@@ -153,3 +153,70 @@ is($node_twoways->safe_psql('d2', "SELECT count(f) FROM t"),
 	$rows * 2, "2x$rows rows in t");
 is($node_twoways->safe_psql('d2', "SELECT count(f) FROM t2"),
 	$rows * 2, "2x$rows rows in t2");
+
+# Test the replication of the table after changing REPLICA IDENTITY INDEX
+
+# The bug was that when changing REPLICA IDENTITY INDEX to another one, the
+# target table's relcache would not be invalidated. This could cause unexpected
+# behaviour when replicating UPDATE change.
+$node_publisher = get_new_node('publisher3');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->start;
+
+$node_subscriber = get_new_node('subscriber3');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->start;
+
+$node_publisher->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_a ON tab_replidentity_index(a)"
+);
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+
+# use index idx_replidentity_index_a as REPLICA IDENTITY in publisher.
+$node_publisher->safe_psql('postgres',
+	"ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_a;"
+);
+$node_publisher->safe_psql('postgres',
+	"INSERT INTO tab_replidentity_index VALUES(1, 1)");
+
+$node_subscriber->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_subscriber->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+
+$publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+	"CREATE PUBLICATION tap_pub FOR TABLE tab_replidentity_index");
+$node_subscriber->safe_psql('postgres',
+	"CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub"
+);
+
+$node_publisher->wait_for_catchup('tap_sub');
+
+is($node_subscriber->safe_psql('postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(1|1),
+	"check initial data on subscriber");
+
+# use index idx_replidentity_index_a as REPLICA IDENTITY in subcriber.
+$node_subscriber->safe_psql('postgres', qq[
+	ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b;
+]);
+
+# Set publisher's REPLICA IDENTITY to idx_replidentity_index_b, then UPDATE the table.
+$node_publisher->safe_psql('postgres', qq[
+	ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b;
+	UPDATE tab_replidentity_index SET a = -a;
+]);
+
+$node_publisher->wait_for_catchup('tap_sub');
+is( $node_subscriber->safe_psql('postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(-1|1),
+	"update works with REPLICA IDENTITY");
+
+$node_publisher->stop('fast');
+$node_subscriber->stop('fast');
-- 
2.7.2.windows.1

v2-HEAD-0001-Invalidate-relcache-when-setting-REPLICA-IDENTITY.patchapplication/octet-stream; name=v2-HEAD-0001-Invalidate-relcache-when-setting-REPLICA-IDENTITY.patchDownload
From 31e0a1be14084f93534f76c56d4e91e8710e19b1 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Thu, 11 Nov 2021 18:21:37 +0800
Subject: [PATCH] Invalidate relcache when setting REPLICA IDENTITY

When changing REPLICA IDENTITY INDEX to another one, the target table's
relcache would not be invalidated which could cause unexpected behaviour when
replicating UPDATE change.

So, Invalidate the relcache in this case to make sure the the bitmap of index
attribute is rebuilt

Author: Tang Haiying, Hou Zhijie
Reviewed-by: Amit Kapila
Backpatch-through: 10, where it was introduced
Discussion: https://www.postgresql.org/message-id/OS0PR01MB61133CA11630DAE45BC6AD95FB939%40OS0PR01MB6113.jpnprd01.prod.outlook.com

---
 src/backend/commands/tablecmds.c    | 10 ++++++
 src/test/subscription/t/100_bugs.pl | 69 ++++++++++++++++++++++++++++++++++++-
 2 files changed, 78 insertions(+), 1 deletion(-)

diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 857cc5c..55c1356 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15412,6 +15412,7 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
 	Form_pg_index pg_index_form;
 
 	ListCell   *index;
+	bool		need_rel_inval = false;
 
 	/*
 	 * Check whether relreplident has changed, and update it if so.
@@ -15488,10 +15489,19 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
 			CatalogTupleUpdate(pg_index, &pg_index_tuple->t_self, pg_index_tuple);
 			InvokeObjectPostAlterHookArg(IndexRelationId, thisIndexOid, 0,
 										 InvalidOid, is_internal);
+			need_rel_inval = true;
 		}
 		heap_freetuple(pg_index_tuple);
 	}
 
+	/*
+	 * Invalidate the relcache for the table, so that after we commit all
+	 * sessions will refresh the table's replica identity index before
+	 * attempting any update on the table.
+	 */
+	if (need_rel_inval)
+		CacheInvalidateRelcache(rel);
+
 	table_close(pg_index, RowExclusiveLock);
 }
 
diff --git a/src/test/subscription/t/100_bugs.pl b/src/test/subscription/t/100_bugs.pl
index 34a60fd..a62b169 100644
--- a/src/test/subscription/t/100_bugs.pl
+++ b/src/test/subscription/t/100_bugs.pl
@@ -6,7 +6,7 @@ use strict;
 use warnings;
 use PostgreSQL::Test::Cluster;
 use PostgreSQL::Test::Utils;
-use Test::More tests => 5;
+use Test::More tests => 7;
 
 # Bug #15114
 
@@ -224,3 +224,70 @@ $node_sub->safe_psql('postgres', "DROP TABLE tab1");
 $node_pub->stop('fast');
 $node_pub_sub->stop('fast');
 $node_sub->stop('fast');
+
+# Test the replication of the table after changing REPLICA IDENTITY INDEX
+
+# The bug was that when changing REPLICA IDENTITY INDEX to another one, the
+# target table's relcache would not be invalidated. This could cause unexpected
+# behaviour when replicating UPDATE change.
+$node_publisher = PostgreSQL::Test::Cluster->new('publisher3');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->start;
+
+$node_subscriber = PostgreSQL::Test::Cluster->new('subscriber3');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->start;
+
+$node_publisher->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_a ON tab_replidentity_index(a)"
+);
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+
+# use index idx_replidentity_index_a as REPLICA IDENTITY in publisher.
+$node_publisher->safe_psql('postgres',
+	"ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_a;"
+);
+$node_publisher->safe_psql('postgres',
+	"INSERT INTO tab_replidentity_index VALUES(1, 1)");
+
+$node_subscriber->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_subscriber->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+
+$publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+	"CREATE PUBLICATION tap_pub FOR TABLE tab_replidentity_index");
+$node_subscriber->safe_psql('postgres',
+	"CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub"
+);
+
+$node_publisher->wait_for_catchup('tap_sub');
+
+is($node_subscriber->safe_psql('postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(1|1),
+	"check initial data on subscriber");
+
+# use index idx_replidentity_index_a as REPLICA IDENTITY in subcriber.
+$node_subscriber->safe_psql('postgres', qq[
+	ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b;
+]);
+
+# Set publisher's REPLICA IDENTITY to idx_replidentity_index_b, then UPDATE the table.
+$node_publisher->safe_psql('postgres', qq[
+	ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b;
+	UPDATE tab_replidentity_index SET a = -a;
+]);
+
+$node_publisher->wait_for_catchup('tap_sub');
+is( $node_subscriber->safe_psql('postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(-1|1),
+	"update works with REPLICA IDENTITY");
+
+$node_publisher->stop('fast');
+$node_subscriber->stop('fast');
-- 
2.7.2.windows.1

#8Euler Taveira
euler@eulerto.com
In reply to: houzj.fnst@fujitsu.com (#7)
1 attachment(s)
Re: [BUG]Invalidate relcache when setting REPLICA IDENTITY

On Thu, Nov 11, 2021, at 9:01 AM, houzj.fnst@fujitsu.com wrote:

Also attach the patches for back branch and remove some unnecessary
changes from pgindent.

I reviewed your patch and I think the fix could be simplified by

if (OidIsValid(indexOid))
CacheInvalidateRelcache(rel);

If indexOid is valid it is a REPLICA IDENTITY INDEX. A few lines above there is
a check for a valid indexOid that makes sure the index is already marked as a
replica identity; if so, it bail out. If it is not, the relation should be
invalidated. Am I missing something?

I also modified your test case to include a DELETE command, wait the initial
table sync to avoid failing a subsequent test and improve some comments.

--
Euler Taveira
EDB https://www.enterprisedb.com/

Attachments:

v3-0001-Invalidate-relcache-entry-when-changing-REPLICA-I.patchtext/x-patch; name=v3-0001-Invalidate-relcache-entry-when-changing-REPLICA-I.patchDownload
From e81e0bb6c6796e7fd692b7d6643db8fb6770d9a1 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Thu, 11 Nov 2021 18:21:37 +0800
Subject: [PATCH v3] Invalidate relcache entry when changing REPLICA IDENTITY
 INDEX

When changing REPLICA IDENTITY INDEX to another one, the table relcache
entry was not invalidated which cause unexpected behaviour when
replicating UPDATE or DELETE changes.

If we are replacing indexes on the REPLICA IDENTITY, invalidate the
relcache entry to make sure the index Bitmapset is rebuild.
---
 src/backend/commands/tablecmds.c    |  9 ++++
 src/test/subscription/t/100_bugs.pl | 80 ++++++++++++++++++++++++++++-
 2 files changed, 88 insertions(+), 1 deletion(-)

diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 857cc5ce6e..97a05da4b1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15492,6 +15492,15 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
 		heap_freetuple(pg_index_tuple);
 	}
 
+	/*
+	 * If the correct index was not marked as replica identity, table relcache
+	 * should be invalidated. Hence, all sessions will refresh the table
+	 * replica identity before any UPDATE or DELETE on the table that was
+	 * invalidated.
+	 */
+	if (OidIsValid(indexOid))
+		CacheInvalidateRelcache(rel);
+
 	table_close(pg_index, RowExclusiveLock);
 }
 
diff --git a/src/test/subscription/t/100_bugs.pl b/src/test/subscription/t/100_bugs.pl
index 34a60fd9ab..00de20e9b5 100644
--- a/src/test/subscription/t/100_bugs.pl
+++ b/src/test/subscription/t/100_bugs.pl
@@ -6,7 +6,7 @@ use strict;
 use warnings;
 use PostgreSQL::Test::Cluster;
 use PostgreSQL::Test::Utils;
-use Test::More tests => 5;
+use Test::More tests => 7;
 
 # Bug #15114
 
@@ -224,3 +224,81 @@ $node_sub->safe_psql('postgres', "DROP TABLE tab1");
 $node_pub->stop('fast');
 $node_pub_sub->stop('fast');
 $node_sub->stop('fast');
+
+# https://postgr.es/m/OS0PR01MB61133CA11630DAE45BC6AD95FB939%40OS0PR01MB6113.jpnprd01.prod.outlook.com'
+
+# The bug was that when changing the REPLICA IDENTITY INDEX to another one, the
+# target table relcache would not be invalidated. This could cause unexpected
+# behaviour when replicating UPDATE/DELETE change.
+$node_publisher = PostgreSQL::Test::Cluster->new('publisher3');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->start;
+
+$node_subscriber = PostgreSQL::Test::Cluster->new('subscriber3');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->start;
+
+$node_publisher->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_a ON tab_replidentity_index(a)"
+);
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+
+# use index idx_replidentity_index_a as REPLICA IDENTITY on publisher.
+$node_publisher->safe_psql('postgres',
+	"ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_a"
+);
+
+$node_publisher->safe_psql('postgres',
+	"INSERT INTO tab_replidentity_index VALUES(1, 1),(2, 2)");
+
+$node_subscriber->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_subscriber->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_a ON tab_replidentity_index(a)"
+);
+$node_subscriber->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+# use index idx_replidentity_index_b as REPLICA IDENTITY on subscriber because
+# it reflects the future scenario we are testing: changing REPLICA IDENTITY
+# INDEX.
+$node_subscriber->safe_psql('postgres',
+	"ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b"
+);
+
+$publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+	"CREATE PUBLICATION tap_pub FOR TABLE tab_replidentity_index");
+$node_subscriber->safe_psql('postgres',
+	"CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub"
+);
+
+$node_publisher->wait_for_catchup('tap_sub');
+
+# Also wait for initial table sync to finish
+$node_subscriber->poll_query_until('postgres', $synced_query)
+  or die "Timed out while waiting for subscriber to synchronize data";
+
+is($node_subscriber->safe_psql('postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(1|1
+2|2),
+	"check initial data on subscriber");
+
+# Set REPLICA IDENTITY to idx_replidentity_index_b on publisher, then run UPDATE and DELETE.
+$node_publisher->safe_psql('postgres', qq[
+	ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b;
+	UPDATE tab_replidentity_index SET a = -a WHERE a = 1;
+	DELETE FROM tab_replidentity_index WHERE a = 2;
+]);
+
+$node_publisher->wait_for_catchup('tap_sub');
+is( $node_subscriber->safe_psql('postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(-1|1),
+	"update works with REPLICA IDENTITY");
+
+$node_publisher->stop('fast');
+$node_subscriber->stop('fast');
-- 
2.20.1

#9houzj.fnst@fujitsu.com
houzj.fnst@fujitsu.com
In reply to: Euler Taveira (#8)
1 attachment(s)
RE: [BUG]Invalidate relcache when setting REPLICA IDENTITY

On Friday, November 12, 2021 8:15 AM Euler Taveira <euler@eulerto.com> wrote:

I reviewed your patch and I think the fix could be simplified by

if (OidIsValid(indexOid))
CacheInvalidateRelcache(rel);

If indexOid is valid it is a REPLICA IDENTITY INDEX. A few lines above there is
a check for a valid indexOid that makes sure the index is already marked as a
replica identity; if so, it bail out. If it is not, the relation should be
invalidated. Am I missing something?

Thanks for reviewing !
But I am not sure it's better to simplify the code like
"if (OidIsValid(indexOid)) CacheInvalidate". After this change, it will
also invalidate the relcache when changing REPLICA IDENTITY from
DEFAULT/FULL/NOTHING to USING INDEX which I think is unnecessary, because the
invalidate for these cases was already handled by
"CatalogTupleUpdate(pg_class".

So, I still think the flag need_rel_inval is needed.

I also modified your test case to include a DELETE command, wait the initial
table sync to avoid failing a subsequent test and improve some comments.

Thanks!

Attach the patch which changed the code use need_rel_inval flag.
Also ran pgperltidy for the testcases. I will post patches for backbranch
if there are no more comments.

Best regards,
Hou zj

Attachments:

v4-0001-Invalidate-relcache-entry-when-changing-REPLICA-IDEN.patchapplication/octet-stream; name=v4-0001-Invalidate-relcache-entry-when-changing-REPLICA-IDEN.patchDownload
From 286dbcf844c4842c03226953ac6b72c721de7144 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Fri, 12 Nov 2021 10:30:30 +0800
Subject: [PATCH] Invalidate relcache entry when changing REPLICA IDENTITY
 INDEX

When changing REPLICA IDENTITY INDEX to another one, the table relcache
entry was not invalidated which cause unexpected behaviour when
replicating UPDATE or DELETE changes.

If we are replacing indexes on the REPLICA IDENTITY, invalidate the
relcache entry to make sure the index Bitmapset is rebuild.
---
 src/backend/commands/tablecmds.c    | 11 +++++
 src/test/subscription/t/100_bugs.pl | 83 ++++++++++++++++++++++++++++++++++++-
 2 files changed, 93 insertions(+), 1 deletion(-)

diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 857cc5c..dc94d21 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15412,6 +15412,7 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
 	Form_pg_index pg_index_form;
 
 	ListCell   *index;
+	bool		need_rel_inval = false;
 
 	/*
 	 * Check whether relreplident has changed, and update it if so.
@@ -15488,10 +15489,20 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
 			CatalogTupleUpdate(pg_index, &pg_index_tuple->t_self, pg_index_tuple);
 			InvokeObjectPostAlterHookArg(IndexRelationId, thisIndexOid, 0,
 										 InvalidOid, is_internal);
+			need_rel_inval = true;
 		}
 		heap_freetuple(pg_index_tuple);
 	}
 
+	/*
+	 * If the correct index was not marked as replica identity, table relcache
+	 * should be invalidated. Hence, all sessions will refresh the table
+	 * replica identity before any UPDATE or DELETE on the table that was
+	 * invalidated.
+	 */
+	if (need_rel_inval)
+		CacheInvalidateRelcache(rel);
+
 	table_close(pg_index, RowExclusiveLock);
 }
 
diff --git a/src/test/subscription/t/100_bugs.pl b/src/test/subscription/t/100_bugs.pl
index 34a60fd..675a7c0 100644
--- a/src/test/subscription/t/100_bugs.pl
+++ b/src/test/subscription/t/100_bugs.pl
@@ -6,7 +6,7 @@ use strict;
 use warnings;
 use PostgreSQL::Test::Cluster;
 use PostgreSQL::Test::Utils;
-use Test::More tests => 5;
+use Test::More tests => 7;
 
 # Bug #15114
 
@@ -224,3 +224,84 @@ $node_sub->safe_psql('postgres', "DROP TABLE tab1");
 $node_pub->stop('fast');
 $node_pub_sub->stop('fast');
 $node_sub->stop('fast');
+
+# https://postgr.es/m/OS0PR01MB61133CA11630DAE45BC6AD95FB939%40OS0PR01MB6113.jpnprd01.prod.outlook.com
+
+# The bug was that when changing the REPLICA IDENTITY INDEX to another one, the
+# target table relcache would not be invalidated. This could cause unexpected
+# behaviour when replicating UPDATE/DELETE change.
+$node_publisher = PostgreSQL::Test::Cluster->new('publisher3');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->start;
+
+$node_subscriber = PostgreSQL::Test::Cluster->new('subscriber3');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->start;
+
+$node_publisher->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_a ON tab_replidentity_index(a)"
+);
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+
+# use index idx_replidentity_index_a as REPLICA IDENTITY on publisher.
+$node_publisher->safe_psql('postgres',
+	"ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_a"
+);
+
+$node_publisher->safe_psql('postgres',
+	"INSERT INTO tab_replidentity_index VALUES(1, 1),(2, 2)");
+
+$node_subscriber->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_subscriber->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_a ON tab_replidentity_index(a)"
+);
+$node_subscriber->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+# use index idx_replidentity_index_b as REPLICA IDENTITY on subscriber because
+# it reflects the future scenario we are testing: changing REPLICA IDENTITY
+# INDEX.
+$node_subscriber->safe_psql('postgres',
+	"ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b"
+);
+
+$publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+	"CREATE PUBLICATION tap_pub FOR TABLE tab_replidentity_index");
+$node_subscriber->safe_psql('postgres',
+	"CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub"
+);
+
+$node_publisher->wait_for_catchup('tap_sub');
+
+# Also wait for initial table sync to finish
+$node_subscriber->poll_query_until('postgres', $synced_query)
+  or die "Timed out while waiting for subscriber to synchronize data";
+
+is( $node_subscriber->safe_psql(
+		'postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(1|1
+2|2),
+	"check initial data on subscriber");
+
+# Set REPLICA IDENTITY to idx_replidentity_index_b on publisher, then run UPDATE and DELETE.
+$node_publisher->safe_psql(
+	'postgres', qq[
+	ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b;
+	UPDATE tab_replidentity_index SET a = -a WHERE a = 1;
+	DELETE FROM tab_replidentity_index WHERE a = 2;
+]);
+
+$node_publisher->wait_for_catchup('tap_sub');
+is( $node_subscriber->safe_psql(
+		'postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(-1|1),
+	"update works with REPLICA IDENTITY");
+
+$node_publisher->stop('fast');
+$node_subscriber->stop('fast');
-- 
2.7.2.windows.1

#10houzj.fnst@fujitsu.com
houzj.fnst@fujitsu.com
In reply to: houzj.fnst@fujitsu.com (#9)
1 attachment(s)
RE: [BUG]Invalidate relcache when setting REPLICA IDENTITY

On Friday, November 12, 2021 10:46 AM I wrote:

On Friday, November 12, 2021 8:15 AM Euler Taveira <euler@eulerto.com>
wrote:

I reviewed your patch and I think the fix could be simplified by

if (OidIsValid(indexOid))
CacheInvalidateRelcache(rel);

If indexOid is valid it is a REPLICA IDENTITY INDEX. A few lines above
there is a check for a valid indexOid that makes sure the index is
already marked as a replica identity; if so, it bail out. If it is
not, the relation should be invalidated. Am I missing something?

Thanks for reviewing !
But I am not sure it's better to simplify the code like "if (OidIsValid(indexOid))
CacheInvalidate".

Oh, I got confused with the logic in relation_mark_replica_identity, sorry for that.
I now realize that you are right, we can just check "if (OidIsValid(indexOid))" here
to simplify the code.

Attach the new version patch which simplify the check as pointed out by Euler.

Best regards,
Hou zj

Attachments:

v5-0001-Invalidate-relcache-entry-when-changing-REPLICA-IDEN.patchapplication/octet-stream; name=v5-0001-Invalidate-relcache-entry-when-changing-REPLICA-IDEN.patchDownload
From b572d073e39797cd29d191739445f18304cd2edd Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Fri, 12 Nov 2021 12:51:42 +0800
Subject: [PATCH] Invalidate relcache entry when changing REPLICA IDENTITY
 INDEX

When changing REPLICA IDENTITY INDEX to another one, the table relcache
entry was not invalidated which cause unexpected behaviour when
replicating UPDATE or DELETE changes.

If we are replacing indexes on the REPLICA IDENTITY, invalidate the
relcache entry to make sure the index Bitmapset is rebuild.
---
 src/backend/commands/tablecmds.c    |  9 ++++
 src/test/subscription/t/100_bugs.pl | 83 ++++++++++++++++++++++++++++++++++++-
 2 files changed, 91 insertions(+), 1 deletion(-)

diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 857cc5c..97a05da 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15492,6 +15492,15 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
 		heap_freetuple(pg_index_tuple);
 	}
 
+	/*
+	 * If the correct index was not marked as replica identity, table relcache
+	 * should be invalidated. Hence, all sessions will refresh the table
+	 * replica identity before any UPDATE or DELETE on the table that was
+	 * invalidated.
+	 */
+	if (OidIsValid(indexOid))
+		CacheInvalidateRelcache(rel);
+
 	table_close(pg_index, RowExclusiveLock);
 }
 
diff --git a/src/test/subscription/t/100_bugs.pl b/src/test/subscription/t/100_bugs.pl
index 34a60fd..675a7c0 100644
--- a/src/test/subscription/t/100_bugs.pl
+++ b/src/test/subscription/t/100_bugs.pl
@@ -6,7 +6,7 @@ use strict;
 use warnings;
 use PostgreSQL::Test::Cluster;
 use PostgreSQL::Test::Utils;
-use Test::More tests => 5;
+use Test::More tests => 7;
 
 # Bug #15114
 
@@ -224,3 +224,84 @@ $node_sub->safe_psql('postgres', "DROP TABLE tab1");
 $node_pub->stop('fast');
 $node_pub_sub->stop('fast');
 $node_sub->stop('fast');
+
+# https://postgr.es/m/OS0PR01MB61133CA11630DAE45BC6AD95FB939%40OS0PR01MB6113.jpnprd01.prod.outlook.com
+
+# The bug was that when changing the REPLICA IDENTITY INDEX to another one, the
+# target table relcache would not be invalidated. This could cause unexpected
+# behaviour when replicating UPDATE/DELETE change.
+$node_publisher = PostgreSQL::Test::Cluster->new('publisher3');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->start;
+
+$node_subscriber = PostgreSQL::Test::Cluster->new('subscriber3');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->start;
+
+$node_publisher->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_a ON tab_replidentity_index(a)"
+);
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+
+# use index idx_replidentity_index_a as REPLICA IDENTITY on publisher.
+$node_publisher->safe_psql('postgres',
+	"ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_a"
+);
+
+$node_publisher->safe_psql('postgres',
+	"INSERT INTO tab_replidentity_index VALUES(1, 1),(2, 2)");
+
+$node_subscriber->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_subscriber->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_a ON tab_replidentity_index(a)"
+);
+$node_subscriber->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+# use index idx_replidentity_index_b as REPLICA IDENTITY on subscriber because
+# it reflects the future scenario we are testing: changing REPLICA IDENTITY
+# INDEX.
+$node_subscriber->safe_psql('postgres',
+	"ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b"
+);
+
+$publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+	"CREATE PUBLICATION tap_pub FOR TABLE tab_replidentity_index");
+$node_subscriber->safe_psql('postgres',
+	"CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub"
+);
+
+$node_publisher->wait_for_catchup('tap_sub');
+
+# Also wait for initial table sync to finish
+$node_subscriber->poll_query_until('postgres', $synced_query)
+  or die "Timed out while waiting for subscriber to synchronize data";
+
+is( $node_subscriber->safe_psql(
+		'postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(1|1
+2|2),
+	"check initial data on subscriber");
+
+# Set REPLICA IDENTITY to idx_replidentity_index_b on publisher, then run UPDATE and DELETE.
+$node_publisher->safe_psql(
+	'postgres', qq[
+	ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b;
+	UPDATE tab_replidentity_index SET a = -a WHERE a = 1;
+	DELETE FROM tab_replidentity_index WHERE a = 2;
+]);
+
+$node_publisher->wait_for_catchup('tap_sub');
+is( $node_subscriber->safe_psql(
+		'postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(-1|1),
+	"update works with REPLICA IDENTITY");
+
+$node_publisher->stop('fast');
+$node_subscriber->stop('fast');
-- 
2.7.2.windows.1

#11Amit Kapila
amit.kapila16@gmail.com
In reply to: houzj.fnst@fujitsu.com (#10)
Re: [BUG]Invalidate relcache when setting REPLICA IDENTITY

On Fri, Nov 12, 2021 at 10:27 AM houzj.fnst@fujitsu.com
<houzj.fnst@fujitsu.com> wrote:

On Friday, November 12, 2021 10:46 AM I wrote:

On Friday, November 12, 2021 8:15 AM Euler Taveira <euler@eulerto.com>
wrote:

I reviewed your patch and I think the fix could be simplified by

if (OidIsValid(indexOid))
CacheInvalidateRelcache(rel);

If indexOid is valid it is a REPLICA IDENTITY INDEX. A few lines above
there is a check for a valid indexOid that makes sure the index is
already marked as a replica identity; if so, it bail out. If it is
not, the relation should be invalidated. Am I missing something?

Thanks for reviewing !
But I am not sure it's better to simplify the code like "if (OidIsValid(indexOid))
CacheInvalidate".

Oh, I got confused with the logic in relation_mark_replica_identity, sorry for that.
I now realize that you are right, we can just check "if (OidIsValid(indexOid))" here
to simplify the code.

But won't that generate invalidation for the rel twice in the case
(change Replica Identity from Nothing to some index) you mentioned in
the previous email?

--
With Regards,
Amit Kapila.

#12Amit Kapila
amit.kapila16@gmail.com
In reply to: Amit Kapila (#11)
Re: [BUG]Invalidate relcache when setting REPLICA IDENTITY

On Fri, Nov 12, 2021 at 10:50 AM Amit Kapila <amit.kapila16@gmail.com> wrote:

On Fri, Nov 12, 2021 at 10:27 AM houzj.fnst@fujitsu.com
<houzj.fnst@fujitsu.com> wrote:

On Friday, November 12, 2021 10:46 AM I wrote:

On Friday, November 12, 2021 8:15 AM Euler Taveira <euler@eulerto.com>
wrote:

I reviewed your patch and I think the fix could be simplified by

if (OidIsValid(indexOid))
CacheInvalidateRelcache(rel);

If indexOid is valid it is a REPLICA IDENTITY INDEX. A few lines above
there is a check for a valid indexOid that makes sure the index is
already marked as a replica identity; if so, it bail out. If it is
not, the relation should be invalidated. Am I missing something?

Thanks for reviewing !
But I am not sure it's better to simplify the code like "if (OidIsValid(indexOid))
CacheInvalidate".

Oh, I got confused with the logic in relation_mark_replica_identity, sorry for that.
I now realize that you are right, we can just check "if (OidIsValid(indexOid))" here
to simplify the code.

But won't that generate invalidation for the rel twice in the case
(change Replica Identity from Nothing to some index) you mentioned in
the previous email?

Oh, I see the point. I think this is okay because
AddRelcacheInvalidationMessage doesn't allow to add duplicate rel
invalidation. If that is the case I wonder why not simply register
invalidation without any check in the for loop as was the case with
Tang's original patch?

--
With Regards,
Amit Kapila.

#13houzj.fnst@fujitsu.com
houzj.fnst@fujitsu.com
In reply to: Amit Kapila (#12)
1 attachment(s)
RE: [BUG]Invalidate relcache when setting REPLICA IDENTITY

On Fri, Nov 12, 2021 1:33 PM Amit Kapila <amit.kapila16@gmail.com> wrote:

On Fri, Nov 12, 2021 at 10:50 AM Amit Kapila <amit.kapila16@gmail.com> wrote:

But won't that generate invalidation for the rel twice in the case
(change Replica Identity from Nothing to some index) you mentioned in
the previous email?

Oh, I see the point. I think this is okay because
AddRelcacheInvalidationMessage doesn't allow to add duplicate rel
invalidation. If that is the case I wonder why not simply register
invalidation without any check in the for loop as was the case with
Tang's original patch?

OK, I also think the code in Tang's original patch is fine.
Attach the patch which register invalidation without any check in the for loop.

Best regards,
Hou zj

Attachments:

v6-0001-Invalidate-relcache-entry-when-changing-REPLICA-IDEN.patchapplication/octet-stream; name=v6-0001-Invalidate-relcache-entry-when-changing-REPLICA-IDEN.patchDownload
From 306ea6cd871c13e3d1f5bcd210552446a0167b80 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Fri, 12 Nov 2021 13:58:40 +0800
Subject: [PATCH] Invalidate relcache entry when changing REPLICA IDENTITY
 INDEX

When changing REPLICA IDENTITY INDEX to another one, the table relcache
entry was not invalidated which cause unexpected behaviour when
replicating UPDATE or DELETE changes.

If we are replacing indexes on the REPLICA IDENTITY, invalidate the
relcache entry to make sure the index Bitmapset is rebuild.
---
 src/backend/commands/tablecmds.c    |  8 ++++
 src/test/subscription/t/100_bugs.pl | 83 ++++++++++++++++++++++++++++++++++++-
 2 files changed, 90 insertions(+), 1 deletion(-)

diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 857cc5c..30f7cf8 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15488,6 +15488,14 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
 			CatalogTupleUpdate(pg_index, &pg_index_tuple->t_self, pg_index_tuple);
 			InvokeObjectPostAlterHookArg(IndexRelationId, thisIndexOid, 0,
 										 InvalidOid, is_internal);
+
+			/*
+			 * If the correct index was not marked as replica identity, table relcache
+			 * should be invalidated. Hence, all sessions will refresh the table
+			 * replica identity before any UPDATE or DELETE on the table that was
+			 * invalidated.
+			 */
+			CacheInvalidateRelcache(rel);
 		}
 		heap_freetuple(pg_index_tuple);
 	}
diff --git a/src/test/subscription/t/100_bugs.pl b/src/test/subscription/t/100_bugs.pl
index 34a60fd..675a7c0 100644
--- a/src/test/subscription/t/100_bugs.pl
+++ b/src/test/subscription/t/100_bugs.pl
@@ -6,7 +6,7 @@ use strict;
 use warnings;
 use PostgreSQL::Test::Cluster;
 use PostgreSQL::Test::Utils;
-use Test::More tests => 5;
+use Test::More tests => 7;
 
 # Bug #15114
 
@@ -224,3 +224,84 @@ $node_sub->safe_psql('postgres', "DROP TABLE tab1");
 $node_pub->stop('fast');
 $node_pub_sub->stop('fast');
 $node_sub->stop('fast');
+
+# https://postgr.es/m/OS0PR01MB61133CA11630DAE45BC6AD95FB939%40OS0PR01MB6113.jpnprd01.prod.outlook.com
+
+# The bug was that when changing the REPLICA IDENTITY INDEX to another one, the
+# target table relcache would not be invalidated. This could cause unexpected
+# behaviour when replicating UPDATE/DELETE change.
+$node_publisher = PostgreSQL::Test::Cluster->new('publisher3');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->start;
+
+$node_subscriber = PostgreSQL::Test::Cluster->new('subscriber3');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->start;
+
+$node_publisher->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_a ON tab_replidentity_index(a)"
+);
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+
+# use index idx_replidentity_index_a as REPLICA IDENTITY on publisher.
+$node_publisher->safe_psql('postgres',
+	"ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_a"
+);
+
+$node_publisher->safe_psql('postgres',
+	"INSERT INTO tab_replidentity_index VALUES(1, 1),(2, 2)");
+
+$node_subscriber->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_subscriber->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_a ON tab_replidentity_index(a)"
+);
+$node_subscriber->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+# use index idx_replidentity_index_b as REPLICA IDENTITY on subscriber because
+# it reflects the future scenario we are testing: changing REPLICA IDENTITY
+# INDEX.
+$node_subscriber->safe_psql('postgres',
+	"ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b"
+);
+
+$publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+	"CREATE PUBLICATION tap_pub FOR TABLE tab_replidentity_index");
+$node_subscriber->safe_psql('postgres',
+	"CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub"
+);
+
+$node_publisher->wait_for_catchup('tap_sub');
+
+# Also wait for initial table sync to finish
+$node_subscriber->poll_query_until('postgres', $synced_query)
+  or die "Timed out while waiting for subscriber to synchronize data";
+
+is( $node_subscriber->safe_psql(
+		'postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(1|1
+2|2),
+	"check initial data on subscriber");
+
+# Set REPLICA IDENTITY to idx_replidentity_index_b on publisher, then run UPDATE and DELETE.
+$node_publisher->safe_psql(
+	'postgres', qq[
+	ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b;
+	UPDATE tab_replidentity_index SET a = -a WHERE a = 1;
+	DELETE FROM tab_replidentity_index WHERE a = 2;
+]);
+
+$node_publisher->wait_for_catchup('tap_sub');
+is( $node_subscriber->safe_psql(
+		'postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(-1|1),
+	"update works with REPLICA IDENTITY");
+
+$node_publisher->stop('fast');
+$node_subscriber->stop('fast');
-- 
2.7.2.windows.1

#14Euler Taveira
euler@eulerto.com
In reply to: houzj.fnst@fujitsu.com (#13)
Re: [BUG]Invalidate relcache when setting REPLICA IDENTITY

On Fri, Nov 12, 2021, at 3:10 AM, houzj.fnst@fujitsu.com wrote:

On Fri, Nov 12, 2021 1:33 PM Amit Kapila <amit.kapila16@gmail.com> wrote:

On Fri, Nov 12, 2021 at 10:50 AM Amit Kapila <amit.kapila16@gmail.com> wrote:

But won't that generate invalidation for the rel twice in the case
(change Replica Identity from Nothing to some index) you mentioned in
the previous email?

Oh, I see the point. I think this is okay because
AddRelcacheInvalidationMessage doesn't allow to add duplicate rel
invalidation. If that is the case I wonder why not simply register
invalidation without any check in the for loop as was the case with
Tang's original patch?

OK, I also think the code in Tang's original patch is fine.
Attach the patch which register invalidation without any check in the for loop.

WFM.

--
Euler Taveira
EDB https://www.enterprisedb.com/

#15Amit Kapila
amit.kapila16@gmail.com
In reply to: Euler Taveira (#14)
1 attachment(s)
Re: [BUG]Invalidate relcache when setting REPLICA IDENTITY

On Sat, Nov 13, 2021 at 12:47 AM Euler Taveira <euler@eulerto.com> wrote:

On Fri, Nov 12, 2021, at 3:10 AM, houzj.fnst@fujitsu.com wrote:

On Fri, Nov 12, 2021 1:33 PM Amit Kapila <amit.kapila16@gmail.com> wrote:

On Fri, Nov 12, 2021 at 10:50 AM Amit Kapila <amit.kapila16@gmail.com> wrote:

But won't that generate invalidation for the rel twice in the case
(change Replica Identity from Nothing to some index) you mentioned in
the previous email?

Oh, I see the point. I think this is okay because
AddRelcacheInvalidationMessage doesn't allow to add duplicate rel
invalidation. If that is the case I wonder why not simply register
invalidation without any check in the for loop as was the case with
Tang's original patch?

OK, I also think the code in Tang's original patch is fine.
Attach the patch which register invalidation without any check in the for loop.

WFM.

The patch looks mostly good to me. I have slightly tweaked the
comments in the code (as per my previous suggestion) and test. Also, I
have slightly modified the commit message. If the attached looks good
to you then kindly prepare patches for back-branches.

--
With Regards,
Amit Kapila.

Attachments:

v7-0001-Invalidate-relcache-when-changing-REPLICA-IDENTIT.patchapplication/octet-stream; name=v7-0001-Invalidate-relcache-when-changing-REPLICA-IDENTIT.patchDownload
From 769107fc8818cdd48a380176c2fa69b5ac359573 Mon Sep 17 00:00:00 2001
From: Amit Kapila <akapila@postgresql.org>
Date: Sat, 13 Nov 2021 15:55:30 +0530
Subject: [PATCH v7] Invalidate relcache when changing REPLICA IDENTITY index.

When changing REPLICA IDENTITY INDEX to another one, the target table's
relcache was not being invalidated. This leads to skipping update/delete
operations during apply on the subscriber side as the columns required to
search corresponding rows won't get logged.

Author: Tang Haiying, Hou Zhijie
Reviewed-by: Euler Taveira, Amit Kapila
Backpatch-through: 10
Discussion: https://postgr.es/m/OS0PR01MB61133CA11630DAE45BC6AD95FB939@OS0PR01MB6113.jpnprd01.prod.outlook.com
---
 src/backend/commands/tablecmds.c    |  6 +++
 src/test/subscription/t/100_bugs.pl | 84 ++++++++++++++++++++++++++++-
 2 files changed, 89 insertions(+), 1 deletion(-)

diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 857cc5ce6e..d675d261f7 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15488,6 +15488,12 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
 			CatalogTupleUpdate(pg_index, &pg_index_tuple->t_self, pg_index_tuple);
 			InvokeObjectPostAlterHookArg(IndexRelationId, thisIndexOid, 0,
 										 InvalidOid, is_internal);
+			/*
+			 * Invalidate the relcache for the table, so that after we commit
+			 * all sessions will refresh the table's replica identity index
+			 * before attempting any UPDATE or DELETE on the table.
+			 */
+			CacheInvalidateRelcache(rel);
 		}
 		heap_freetuple(pg_index_tuple);
 	}
diff --git a/src/test/subscription/t/100_bugs.pl b/src/test/subscription/t/100_bugs.pl
index 34a60fd9ab..6574e500ff 100644
--- a/src/test/subscription/t/100_bugs.pl
+++ b/src/test/subscription/t/100_bugs.pl
@@ -6,7 +6,7 @@ use strict;
 use warnings;
 use PostgreSQL::Test::Cluster;
 use PostgreSQL::Test::Utils;
-use Test::More tests => 5;
+use Test::More tests => 7;
 
 # Bug #15114
 
@@ -224,3 +224,85 @@ $node_sub->safe_psql('postgres', "DROP TABLE tab1");
 $node_pub->stop('fast');
 $node_pub_sub->stop('fast');
 $node_sub->stop('fast');
+
+# https://postgr.es/m/OS0PR01MB61133CA11630DAE45BC6AD95FB939%40OS0PR01MB6113.jpnprd01.prod.outlook.com
+
+# The bug was that when changing the REPLICA IDENTITY INDEX to another one, the
+# target table's relcache was not being invalidated. This leads to skipping
+# UPDATE/DELETE operations during apply on the subscriber side as the columns
+# required to search corresponding rows won't get logged.
+$node_publisher = PostgreSQL::Test::Cluster->new('publisher3');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->start;
+
+$node_subscriber = PostgreSQL::Test::Cluster->new('subscriber3');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->start;
+
+$node_publisher->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_a ON tab_replidentity_index(a)"
+);
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+
+# use index idx_replidentity_index_a as REPLICA IDENTITY on publisher.
+$node_publisher->safe_psql('postgres',
+	"ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_a"
+);
+
+$node_publisher->safe_psql('postgres',
+	"INSERT INTO tab_replidentity_index VALUES(1, 1),(2, 2)");
+
+$node_subscriber->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_subscriber->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_a ON tab_replidentity_index(a)"
+);
+$node_subscriber->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+# use index idx_replidentity_index_b as REPLICA IDENTITY on subscriber because
+# it reflects the future scenario we are testing: changing REPLICA IDENTITY
+# INDEX.
+$node_subscriber->safe_psql('postgres',
+	"ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b"
+);
+
+$publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+	"CREATE PUBLICATION tap_pub FOR TABLE tab_replidentity_index");
+$node_subscriber->safe_psql('postgres',
+	"CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub"
+);
+
+$node_publisher->wait_for_catchup('tap_sub');
+
+# Also wait for initial table sync to finish
+$node_subscriber->poll_query_until('postgres', $synced_query)
+  or die "Timed out while waiting for subscriber to synchronize data";
+
+is( $node_subscriber->safe_psql(
+		'postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(1|1
+2|2),
+	"check initial data on subscriber");
+
+# Set REPLICA IDENTITY to idx_replidentity_index_b on publisher, then run UPDATE and DELETE.
+$node_publisher->safe_psql(
+	'postgres', qq[
+	ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b;
+	UPDATE tab_replidentity_index SET a = -a WHERE a = 1;
+	DELETE FROM tab_replidentity_index WHERE a = 2;
+]);
+
+$node_publisher->wait_for_catchup('tap_sub');
+is( $node_subscriber->safe_psql(
+		'postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(-1|1),
+	"update works with REPLICA IDENTITY");
+
+$node_publisher->stop('fast');
+$node_subscriber->stop('fast');
-- 
2.28.0.windows.1

#16houzj.fnst@fujitsu.com
houzj.fnst@fujitsu.com
In reply to: Amit Kapila (#15)
6 attachment(s)
RE: [BUG]Invalidate relcache when setting REPLICA IDENTITY

On Sat, Nov 13, 2021 6:50 PM Amit Kapila <amit.kapila16@gmail.com> wrote:

The patch looks mostly good to me. I have slightly tweaked the comments in
the code (as per my previous suggestion) and test. Also, I have slightly
modified the commit message. If the attached looks good to you then kindly
prepare patches for back-branches.

Thanks for reviewing, the latest patch looks good to me.
Attach the patches for back branch (HEAD ~ v10).

Best regards,
Hou zj

Attachments:

v8-HEAD-0001-Invalidate-relcache-when-changing-REPLICA-IDENTIT.patchapplication/octet-stream; name=v8-HEAD-0001-Invalidate-relcache-when-changing-REPLICA-IDENTIT.patchDownload
From 769107fc8818cdd48a380176c2fa69b5ac359573 Mon Sep 17 00:00:00 2001
From: Amit Kapila <akapila@postgresql.org>
Date: Sat, 13 Nov 2021 15:55:30 +0530
Subject: [PATCH v7] Invalidate relcache when changing REPLICA IDENTITY index.

When changing REPLICA IDENTITY INDEX to another one, the target table's
relcache was not being invalidated. This leads to skipping update/delete
operations during apply on the subscriber side as the columns required to
search corresponding rows won't get logged.

Author: Tang Haiying, Hou Zhijie
Reviewed-by: Euler Taveira, Amit Kapila
Backpatch-through: 10
Discussion: https://postgr.es/m/OS0PR01MB61133CA11630DAE45BC6AD95FB939@OS0PR01MB6113.jpnprd01.prod.outlook.com
---
 src/backend/commands/tablecmds.c    |  6 +++
 src/test/subscription/t/100_bugs.pl | 84 ++++++++++++++++++++++++++++-
 2 files changed, 89 insertions(+), 1 deletion(-)

diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 857cc5ce6e..d675d261f7 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15488,6 +15488,12 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
 			CatalogTupleUpdate(pg_index, &pg_index_tuple->t_self, pg_index_tuple);
 			InvokeObjectPostAlterHookArg(IndexRelationId, thisIndexOid, 0,
 										 InvalidOid, is_internal);
+			/*
+			 * Invalidate the relcache for the table, so that after we commit
+			 * all sessions will refresh the table's replica identity index
+			 * before attempting any UPDATE or DELETE on the table.
+			 */
+			CacheInvalidateRelcache(rel);
 		}
 		heap_freetuple(pg_index_tuple);
 	}
diff --git a/src/test/subscription/t/100_bugs.pl b/src/test/subscription/t/100_bugs.pl
index 34a60fd9ab..6574e500ff 100644
--- a/src/test/subscription/t/100_bugs.pl
+++ b/src/test/subscription/t/100_bugs.pl
@@ -6,7 +6,7 @@ use strict;
 use warnings;
 use PostgreSQL::Test::Cluster;
 use PostgreSQL::Test::Utils;
-use Test::More tests => 5;
+use Test::More tests => 7;
 
 # Bug #15114
 
@@ -224,3 +224,85 @@ $node_sub->safe_psql('postgres', "DROP TABLE tab1");
 $node_pub->stop('fast');
 $node_pub_sub->stop('fast');
 $node_sub->stop('fast');
+
+# https://postgr.es/m/OS0PR01MB61133CA11630DAE45BC6AD95FB939%40OS0PR01MB6113.jpnprd01.prod.outlook.com
+
+# The bug was that when changing the REPLICA IDENTITY INDEX to another one, the
+# target table's relcache was not being invalidated. This leads to skipping
+# UPDATE/DELETE operations during apply on the subscriber side as the columns
+# required to search corresponding rows won't get logged.
+$node_publisher = PostgreSQL::Test::Cluster->new('publisher3');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->start;
+
+$node_subscriber = PostgreSQL::Test::Cluster->new('subscriber3');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->start;
+
+$node_publisher->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_a ON tab_replidentity_index(a)"
+);
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+
+# use index idx_replidentity_index_a as REPLICA IDENTITY on publisher.
+$node_publisher->safe_psql('postgres',
+	"ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_a"
+);
+
+$node_publisher->safe_psql('postgres',
+	"INSERT INTO tab_replidentity_index VALUES(1, 1),(2, 2)");
+
+$node_subscriber->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_subscriber->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_a ON tab_replidentity_index(a)"
+);
+$node_subscriber->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+# use index idx_replidentity_index_b as REPLICA IDENTITY on subscriber because
+# it reflects the future scenario we are testing: changing REPLICA IDENTITY
+# INDEX.
+$node_subscriber->safe_psql('postgres',
+	"ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b"
+);
+
+$publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+	"CREATE PUBLICATION tap_pub FOR TABLE tab_replidentity_index");
+$node_subscriber->safe_psql('postgres',
+	"CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub"
+);
+
+$node_publisher->wait_for_catchup('tap_sub');
+
+# Also wait for initial table sync to finish
+$node_subscriber->poll_query_until('postgres', $synced_query)
+  or die "Timed out while waiting for subscriber to synchronize data";
+
+is( $node_subscriber->safe_psql(
+		'postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(1|1
+2|2),
+	"check initial data on subscriber");
+
+# Set REPLICA IDENTITY to idx_replidentity_index_b on publisher, then run UPDATE and DELETE.
+$node_publisher->safe_psql(
+	'postgres', qq[
+	ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b;
+	UPDATE tab_replidentity_index SET a = -a WHERE a = 1;
+	DELETE FROM tab_replidentity_index WHERE a = 2;
+]);
+
+$node_publisher->wait_for_catchup('tap_sub');
+is( $node_subscriber->safe_psql(
+		'postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(-1|1),
+	"update works with REPLICA IDENTITY");
+
+$node_publisher->stop('fast');
+$node_subscriber->stop('fast');
-- 
2.28.0.windows.1

v8-PG10-0001-Invalidate-relcache-when-changing-REPLICA-IDENTITY-i.patchapplication/octet-stream; name=v8-PG10-0001-Invalidate-relcache-when-changing-REPLICA-IDENTITY-i.patchDownload
From e2b185a409ae9f07e8ca4575a2382663a085f975 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Mon, 15 Nov 2021 09:23:41 +0800
Subject: [PATCH] Invalidate relcache when changing REPLICA IDENTITY index.

When changing REPLICA IDENTITY INDEX to another one, the target table's
relcache was not being invalidated. This leads to skipping update/delete
operations during apply on the subscriber side as the columns required to
search corresponding rows won't get logged.

Author: Tang Haiying, Hou Zhijie
Reviewed-by: Euler Taveira, Amit Kapila
Backpatch-through: 10
Discussion: https://postgr.es/m/OS0PR01MB61133CA11630DAE45BC6AD95FB939@OS0PR01MB6113.jpnprd01.prod.outlook.com
---
 src/backend/commands/tablecmds.c    |  6 +++
 src/test/subscription/t/100_bugs.pl | 86 ++++++++++++++++++++++++++++++++++++-
 2 files changed, 91 insertions(+), 1 deletion(-)

diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 7163ad5..c35bf67 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -12200,6 +12200,12 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
 			CatalogTupleUpdate(pg_index, &pg_index_tuple->t_self, pg_index_tuple);
 			InvokeObjectPostAlterHookArg(IndexRelationId, thisIndexOid, 0,
 										 InvalidOid, is_internal);
+			/*
+			 * Invalidate the relcache for the table, so that after we commit
+			 * all sessions will refresh the table's replica identity index
+			 * before attempting any UPDATE or DELETE on the table.
+			 */
+			CacheInvalidateRelcache(rel);
 		}
 		heap_freetuple(pg_index_tuple);
 	}
diff --git a/src/test/subscription/t/100_bugs.pl b/src/test/subscription/t/100_bugs.pl
index b675f16..809c9ff 100644
--- a/src/test/subscription/t/100_bugs.pl
+++ b/src/test/subscription/t/100_bugs.pl
@@ -3,7 +3,7 @@ use strict;
 use warnings;
 use PostgresNode;
 use TestLib;
-use Test::More tests => 3;
+use Test::More tests => 5;
 
 sub wait_for_caught_up
 {
@@ -107,3 +107,87 @@ is( $node_publisher->psql(
 );
 
 $node_publisher->stop('fast');
+
+# https://postgr.es/m/OS0PR01MB61133CA11630DAE45BC6AD95FB939%40OS0PR01MB6113.jpnprd01.prod.outlook.com
+
+# The bug was that when changing the REPLICA IDENTITY INDEX to another one, the
+# target table's relcache was not being invalidated. This leads to skipping
+# UPDATE/DELETE operations during apply on the subscriber side as the columns
+# required to search corresponding rows won't get logged.
+$node_publisher = get_new_node('publisher3');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->start;
+
+$node_subscriber = get_new_node('subscriber3');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->start;
+
+$node_publisher->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_a ON tab_replidentity_index(a)"
+);
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+
+# use index idx_replidentity_index_a as REPLICA IDENTITY on publisher.
+$node_publisher->safe_psql('postgres',
+	"ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_a"
+);
+
+$node_publisher->safe_psql('postgres',
+	"INSERT INTO tab_replidentity_index VALUES(1, 1),(2, 2)");
+
+$node_subscriber->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_subscriber->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_a ON tab_replidentity_index(a)"
+);
+$node_subscriber->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+# use index idx_replidentity_index_b as REPLICA IDENTITY on subscriber because
+# it reflects the future scenario we are testing: changing REPLICA IDENTITY
+# INDEX.
+$node_subscriber->safe_psql('postgres',
+	"ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b"
+);
+
+$publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+	"CREATE PUBLICATION tap_pub FOR TABLE tab_replidentity_index");
+$node_subscriber->safe_psql('postgres',
+	"CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr application_name=tap_sub' PUBLICATION tap_pub"
+);
+
+wait_for_caught_up($node_publisher, 'tap_sub');
+
+# Also wait for initial table sync to finish
+my $synced_query =
+"SELECT count(1) = 0 FROM pg_subscription_rel WHERE srsubstate NOT IN ('s', 'r');";
+$node_subscriber->poll_query_until('postgres', $synced_query)
+  or die "Timed out while waiting for subscriber to synchronize data";
+
+is( $node_subscriber->safe_psql(
+		'postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(1|1
+2|2),
+	"check initial data on subscriber");
+
+# Set REPLICA IDENTITY to idx_replidentity_index_b on publisher, then run UPDATE and DELETE.
+$node_publisher->safe_psql(
+	'postgres', qq[
+	ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b;
+	UPDATE tab_replidentity_index SET a = -a WHERE a = 1;
+	DELETE FROM tab_replidentity_index WHERE a = 2;
+]);
+
+wait_for_caught_up($node_publisher, 'tap_sub');
+is( $node_subscriber->safe_psql(
+		'postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(-1|1),
+	"update works with REPLICA IDENTITY");
+
+$node_publisher->stop('fast');
+$node_subscriber->stop('fast');
-- 
2.7.2.windows.1

v8-PG12-0001-Invalidate-relcache-when-changing-REPLICA-IDENTITY-i.patchapplication/octet-stream; name=v8-PG12-0001-Invalidate-relcache-when-changing-REPLICA-IDENTITY-i.patchDownload
From 3a86efe7f6defe2f021db203f2beab54b47fa21d Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Mon, 15 Nov 2021 09:27:47 +0800
Subject: [PATCH] Invalidate relcache when changing REPLICA IDENTITY index.

When changing REPLICA IDENTITY INDEX to another one, the target table's
relcache was not being invalidated. This leads to skipping update/delete
operations during apply on the subscriber side as the columns required to
search corresponding rows won't get logged.

Author: Tang Haiying, Hou Zhijie
Reviewed-by: Euler Taveira, Amit Kapila
Backpatch-through: 10
Discussion: https://postgr.es/m/OS0PR01MB61133CA11630DAE45BC6AD95FB939@OS0PR01MB6113.jpnprd01.prod.outlook.com
---
 src/backend/commands/tablecmds.c    |  6 +++
 src/test/subscription/t/100_bugs.pl | 86 ++++++++++++++++++++++++++++++++++++-
 2 files changed, 91 insertions(+), 1 deletion(-)

diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index fe6ee63..dc0001a 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -14411,6 +14411,12 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
 			CatalogTupleUpdate(pg_index, &pg_index_tuple->t_self, pg_index_tuple);
 			InvokeObjectPostAlterHookArg(IndexRelationId, thisIndexOid, 0,
 										 InvalidOid, is_internal);
+			/*
+			 * Invalidate the relcache for the table, so that after we commit
+			 * all sessions will refresh the table's replica identity index
+			 * before attempting any UPDATE or DELETE on the table.
+			 */
+			CacheInvalidateRelcache(rel);
 		}
 		heap_freetuple(pg_index_tuple);
 	}
diff --git a/src/test/subscription/t/100_bugs.pl b/src/test/subscription/t/100_bugs.pl
index 366a7a9..f387b19 100644
--- a/src/test/subscription/t/100_bugs.pl
+++ b/src/test/subscription/t/100_bugs.pl
@@ -3,7 +3,7 @@ use strict;
 use warnings;
 use PostgresNode;
 use TestLib;
-use Test::More tests => 3;
+use Test::More tests => 5;
 
 # Bug #15114
 
@@ -100,3 +100,87 @@ is( $node_publisher->psql(
 );
 
 $node_publisher->stop('fast');
+
+# https://postgr.es/m/OS0PR01MB61133CA11630DAE45BC6AD95FB939%40OS0PR01MB6113.jpnprd01.prod.outlook.com
+
+# The bug was that when changing the REPLICA IDENTITY INDEX to another one, the
+# target table's relcache was not being invalidated. This leads to skipping
+# UPDATE/DELETE operations during apply on the subscriber side as the columns
+# required to search corresponding rows won't get logged.
+$node_publisher = get_new_node('publisher3');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->start;
+
+$node_subscriber = get_new_node('subscriber3');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->start;
+
+$node_publisher->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_a ON tab_replidentity_index(a)"
+);
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+
+# use index idx_replidentity_index_a as REPLICA IDENTITY on publisher.
+$node_publisher->safe_psql('postgres',
+	"ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_a"
+);
+
+$node_publisher->safe_psql('postgres',
+	"INSERT INTO tab_replidentity_index VALUES(1, 1),(2, 2)");
+
+$node_subscriber->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_subscriber->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_a ON tab_replidentity_index(a)"
+);
+$node_subscriber->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+# use index idx_replidentity_index_b as REPLICA IDENTITY on subscriber because
+# it reflects the future scenario we are testing: changing REPLICA IDENTITY
+# INDEX.
+$node_subscriber->safe_psql('postgres',
+	"ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b"
+);
+
+$publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+	"CREATE PUBLICATION tap_pub FOR TABLE tab_replidentity_index");
+$node_subscriber->safe_psql('postgres',
+	"CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub"
+);
+
+$node_publisher->wait_for_catchup('tap_sub');
+
+# Also wait for initial table sync to finish
+my $synced_query =
+  "SELECT count(1) = 0 FROM pg_subscription_rel WHERE srsubstate NOT IN ('s', 'r');";
+$node_subscriber->poll_query_until('postgres', $synced_query)
+  or die "Timed out while waiting for subscriber to synchronize data";
+
+is( $node_subscriber->safe_psql(
+		'postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(1|1
+2|2),
+	"check initial data on subscriber");
+
+# Set REPLICA IDENTITY to idx_replidentity_index_b on publisher, then run UPDATE and DELETE.
+$node_publisher->safe_psql(
+	'postgres', qq[
+	ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b;
+	UPDATE tab_replidentity_index SET a = -a WHERE a = 1;
+	DELETE FROM tab_replidentity_index WHERE a = 2;
+]);
+
+$node_publisher->wait_for_catchup('tap_sub');
+is( $node_subscriber->safe_psql(
+		'postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(-1|1),
+	"update works with REPLICA IDENTITY");
+
+$node_publisher->stop('fast');
+$node_subscriber->stop('fast');
-- 
2.7.2.windows.1

v8-PG11-0001-Invalidate-relcache-when-changing-REPLICA-IDENTITY-i.patchapplication/octet-stream; name=v8-PG11-0001-Invalidate-relcache-when-changing-REPLICA-IDENTITY-i.patchDownload
From 1b0e43d404800ef022c874201eca6573ea8561b9 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Mon, 15 Nov 2021 09:25:44 +0800
Subject: [PATCH] Invalidate relcache when changing REPLICA IDENTITY index.

When changing REPLICA IDENTITY INDEX to another one, the target table's
relcache was not being invalidated. This leads to skipping update/delete
operations during apply on the subscriber side as the columns required to
search corresponding rows won't get logged.

Author: Tang Haiying, Hou Zhijie
Reviewed-by: Euler Taveira, Amit Kapila
Backpatch-through: 10
Discussion: https://postgr.es/m/OS0PR01MB61133CA11630DAE45BC6AD95FB939@OS0PR01MB6113.jpnprd01.prod.outlook.com
---
 src/backend/commands/tablecmds.c    |  6 +++
 src/test/subscription/t/100_bugs.pl | 86 ++++++++++++++++++++++++++++++++++++-
 2 files changed, 91 insertions(+), 1 deletion(-)

diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 62de9ba..49835f28 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -13398,6 +13398,12 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
 			CatalogTupleUpdate(pg_index, &pg_index_tuple->t_self, pg_index_tuple);
 			InvokeObjectPostAlterHookArg(IndexRelationId, thisIndexOid, 0,
 										 InvalidOid, is_internal);
+			/*
+			 * Invalidate the relcache for the table, so that after we commit
+			 * all sessions will refresh the table's replica identity index
+			 * before attempting any UPDATE or DELETE on the table.
+			 */
+			CacheInvalidateRelcache(rel);
 		}
 		heap_freetuple(pg_index_tuple);
 	}
diff --git a/src/test/subscription/t/100_bugs.pl b/src/test/subscription/t/100_bugs.pl
index 9ef42c3..bd04fa5 100644
--- a/src/test/subscription/t/100_bugs.pl
+++ b/src/test/subscription/t/100_bugs.pl
@@ -3,7 +3,7 @@ use strict;
 use warnings;
 use PostgresNode;
 use TestLib;
-use Test::More tests => 3;
+use Test::More tests => 5;
 
 # Bug #15114
 
@@ -98,3 +98,87 @@ is( $node_publisher->psql(
 );
 
 $node_publisher->stop('fast');
+
+# https://postgr.es/m/OS0PR01MB61133CA11630DAE45BC6AD95FB939%40OS0PR01MB6113.jpnprd01.prod.outlook.com
+
+# The bug was that when changing the REPLICA IDENTITY INDEX to another one, the
+# target table's relcache was not being invalidated. This leads to skipping
+# UPDATE/DELETE operations during apply on the subscriber side as the columns
+# required to search corresponding rows won't get logged.
+$node_publisher = get_new_node('publisher3');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->start;
+
+$node_subscriber = get_new_node('subscriber3');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->start;
+
+$node_publisher->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_a ON tab_replidentity_index(a)"
+);
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+
+# use index idx_replidentity_index_a as REPLICA IDENTITY on publisher.
+$node_publisher->safe_psql('postgres',
+	"ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_a"
+);
+
+$node_publisher->safe_psql('postgres',
+	"INSERT INTO tab_replidentity_index VALUES(1, 1),(2, 2)");
+
+$node_subscriber->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_subscriber->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_a ON tab_replidentity_index(a)"
+);
+$node_subscriber->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+# use index idx_replidentity_index_b as REPLICA IDENTITY on subscriber because
+# it reflects the future scenario we are testing: changing REPLICA IDENTITY
+# INDEX.
+$node_subscriber->safe_psql('postgres',
+	"ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b"
+);
+
+$publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+	"CREATE PUBLICATION tap_pub FOR TABLE tab_replidentity_index");
+$node_subscriber->safe_psql('postgres',
+	"CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr application_name=tap_sub' PUBLICATION tap_pub"
+);
+
+$node_publisher->wait_for_catchup('tap_sub');
+
+# Also wait for initial table sync to finish
+my $synced_query =
+  "SELECT count(1) = 0 FROM pg_subscription_rel WHERE srsubstate NOT IN ('s', 'r');";
+$node_subscriber->poll_query_until('postgres', $synced_query)
+  or die "Timed out while waiting for subscriber to synchronize data";
+
+is( $node_subscriber->safe_psql(
+		'postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(1|1
+2|2),
+	"check initial data on subscriber");
+
+# Set REPLICA IDENTITY to idx_replidentity_index_b on publisher, then run UPDATE and DELETE.
+$node_publisher->safe_psql(
+	'postgres', qq[
+	ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b;
+	UPDATE tab_replidentity_index SET a = -a WHERE a = 1;
+	DELETE FROM tab_replidentity_index WHERE a = 2;
+]);
+
+$node_publisher->wait_for_catchup('tap_sub');
+is( $node_subscriber->safe_psql(
+		'postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(-1|1),
+	"update works with REPLICA IDENTITY");
+
+$node_publisher->stop('fast');
+$node_subscriber->stop('fast');
-- 
2.7.2.windows.1

v8-PG13-0001-Invalidate-relcache-when-changing-REPLICA-IDENTITY-i.patchapplication/octet-stream; name=v8-PG13-0001-Invalidate-relcache-when-changing-REPLICA-IDENTITY-i.patchDownload
From a1667916a5f0d2c3c7a7d61d6a4a8772607ac488 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Mon, 15 Nov 2021 08:48:07 +0800
Subject: [PATCH] Invalidate relcache when changing REPLICA IDENTITY index.

When changing REPLICA IDENTITY INDEX to another one, the target table's
relcache was not being invalidated. This leads to skipping update/delete
operations during apply on the subscriber side as the columns required to
search corresponding rows won't get logged.

Author: Tang Haiying, Hou Zhijie
Reviewed-by: Euler Taveira, Amit Kapila
Backpatch-through: 10
Discussion: https://postgr.es/m/OS0PR01MB61133CA11630DAE45BC6AD95FB939@OS0PR01MB6113.jpnprd01.prod.outlook.com
---
 src/backend/commands/tablecmds.c    |  6 +++
 src/test/subscription/t/100_bugs.pl | 84 ++++++++++++++++++++++++++++++++++++-
 2 files changed, 89 insertions(+), 1 deletion(-)

diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index d492f27..8c6b684 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -14891,6 +14891,12 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
 			CatalogTupleUpdate(pg_index, &pg_index_tuple->t_self, pg_index_tuple);
 			InvokeObjectPostAlterHookArg(IndexRelationId, thisIndexOid, 0,
 										 InvalidOid, is_internal);
+			/*
+			 * Invalidate the relcache for the table, so that after we commit
+			 * all sessions will refresh the table's replica identity index
+			 * before attempting any UPDATE or DELETE on the table.
+			 */
+			CacheInvalidateRelcache(rel);
 		}
 		heap_freetuple(pg_index_tuple);
 	}
diff --git a/src/test/subscription/t/100_bugs.pl b/src/test/subscription/t/100_bugs.pl
index d1e407a..be37c35 100644
--- a/src/test/subscription/t/100_bugs.pl
+++ b/src/test/subscription/t/100_bugs.pl
@@ -3,7 +3,7 @@ use strict;
 use warnings;
 use PostgresNode;
 use TestLib;
-use Test::More tests => 5;
+use Test::More tests => 7;
 
 # Bug #15114
 
@@ -153,3 +153,85 @@ is($node_twoways->safe_psql('d2', "SELECT count(f) FROM t"),
 	$rows * 2, "2x$rows rows in t");
 is($node_twoways->safe_psql('d2', "SELECT count(f) FROM t2"),
 	$rows * 2, "2x$rows rows in t2");
+
+# https://postgr.es/m/OS0PR01MB61133CA11630DAE45BC6AD95FB939%40OS0PR01MB6113.jpnprd01.prod.outlook.com
+
+# The bug was that when changing the REPLICA IDENTITY INDEX to another one, the
+# target table's relcache was not being invalidated. This leads to skipping
+# UPDATE/DELETE operations during apply on the subscriber side as the columns
+# required to search corresponding rows won't get logged.
+$node_publisher = get_new_node('publisher3');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->start;
+
+$node_subscriber = get_new_node('subscriber3');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->start;
+
+$node_publisher->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_a ON tab_replidentity_index(a)"
+);
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+
+# use index idx_replidentity_index_a as REPLICA IDENTITY on publisher.
+$node_publisher->safe_psql('postgres',
+	"ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_a"
+);
+
+$node_publisher->safe_psql('postgres',
+	"INSERT INTO tab_replidentity_index VALUES(1, 1),(2, 2)");
+
+$node_subscriber->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_subscriber->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_a ON tab_replidentity_index(a)"
+);
+$node_subscriber->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+# use index idx_replidentity_index_b as REPLICA IDENTITY on subscriber because
+# it reflects the future scenario we are testing: changing REPLICA IDENTITY
+# INDEX.
+$node_subscriber->safe_psql('postgres',
+	"ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b"
+);
+
+$publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+	"CREATE PUBLICATION tap_pub FOR TABLE tab_replidentity_index");
+$node_subscriber->safe_psql('postgres',
+	"CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub"
+);
+
+$node_publisher->wait_for_catchup('tap_sub');
+
+# Also wait for initial table sync to finish
+$node_subscriber->poll_query_until('postgres', $synced_query)
+  or die "Timed out while waiting for subscriber to synchronize data";
+
+is( $node_subscriber->safe_psql(
+		'postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(1|1
+2|2),
+	"check initial data on subscriber");
+
+# Set REPLICA IDENTITY to idx_replidentity_index_b on publisher, then run UPDATE and DELETE.
+$node_publisher->safe_psql(
+	'postgres', qq[
+	ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b;
+	UPDATE tab_replidentity_index SET a = -a WHERE a = 1;
+	DELETE FROM tab_replidentity_index WHERE a = 2;
+]);
+
+$node_publisher->wait_for_catchup('tap_sub');
+is( $node_subscriber->safe_psql(
+		'postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(-1|1),
+	"update works with REPLICA IDENTITY");
+
+$node_publisher->stop('fast');
+$node_subscriber->stop('fast');
-- 
2.7.2.windows.1

v8-PG14-0001-Invalidate-relcache-when-changing-REPLICA-IDENTITY-i.patchapplication/octet-stream; name=v8-PG14-0001-Invalidate-relcache-when-changing-REPLICA-IDENTITY-i.patchDownload
From 10008a6454a4d176db1c2ba7f2619b511cae47c5 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Mon, 15 Nov 2021 08:43:10 +0800
Subject: [PATCH] Invalidate relcache when changing REPLICA IDENTITY index.

When changing REPLICA IDENTITY INDEX to another one, the target table's
relcache was not being invalidated. This leads to skipping update/delete
operations during apply on the subscriber side as the columns required to
search corresponding rows won't get logged.

Author: Tang Haiying, Hou Zhijie
Reviewed-by: Euler Taveira, Amit Kapila
Backpatch-through: 10
Discussion: https://postgr.es/m/OS0PR01MB61133CA11630DAE45BC6AD95FB939@OS0PR01MB6113.jpnprd01.prod.outlook.com
---
 src/backend/commands/tablecmds.c    |  6 +++
 src/test/subscription/t/100_bugs.pl | 84 ++++++++++++++++++++++++++++++++++++-
 2 files changed, 89 insertions(+), 1 deletion(-)

diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 0d34f70..e82a6d0 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15344,6 +15344,12 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
 			CatalogTupleUpdate(pg_index, &pg_index_tuple->t_self, pg_index_tuple);
 			InvokeObjectPostAlterHookArg(IndexRelationId, thisIndexOid, 0,
 										 InvalidOid, is_internal);
+			/*
+			 * Invalidate the relcache for the table, so that after we commit
+			 * all sessions will refresh the table's replica identity index
+			 * before attempting any UPDATE or DELETE on the table.
+			 */
+			CacheInvalidateRelcache(rel);
 		}
 		heap_freetuple(pg_index_tuple);
 	}
diff --git a/src/test/subscription/t/100_bugs.pl b/src/test/subscription/t/100_bugs.pl
index 21eabce..a981d2b 100644
--- a/src/test/subscription/t/100_bugs.pl
+++ b/src/test/subscription/t/100_bugs.pl
@@ -6,7 +6,7 @@ use strict;
 use warnings;
 use PostgresNode;
 use TestLib;
-use Test::More tests => 5;
+use Test::More tests => 7;
 
 # Bug #15114
 
@@ -224,3 +224,85 @@ $node_sub->safe_psql('postgres', "DROP TABLE tab1");
 $node_pub->stop('fast');
 $node_pub_sub->stop('fast');
 $node_sub->stop('fast');
+
+# https://postgr.es/m/OS0PR01MB61133CA11630DAE45BC6AD95FB939%40OS0PR01MB6113.jpnprd01.prod.outlook.com
+
+# The bug was that when changing the REPLICA IDENTITY INDEX to another one, the
+# target table's relcache was not being invalidated. This leads to skipping
+# UPDATE/DELETE operations during apply on the subscriber side as the columns
+# required to search corresponding rows won't get logged.
+$node_publisher = get_new_node('publisher3');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->start;
+
+$node_subscriber = get_new_node('subscriber3');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->start;
+
+$node_publisher->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_a ON tab_replidentity_index(a)"
+);
+$node_publisher->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+
+# use index idx_replidentity_index_a as REPLICA IDENTITY on publisher.
+$node_publisher->safe_psql('postgres',
+	"ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_a"
+);
+
+$node_publisher->safe_psql('postgres',
+	"INSERT INTO tab_replidentity_index VALUES(1, 1),(2, 2)");
+
+$node_subscriber->safe_psql('postgres',
+	"CREATE TABLE tab_replidentity_index(a int not null, b int not null)");
+$node_subscriber->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_a ON tab_replidentity_index(a)"
+);
+$node_subscriber->safe_psql('postgres',
+	"CREATE UNIQUE INDEX idx_replidentity_index_b ON tab_replidentity_index(b)"
+);
+# use index idx_replidentity_index_b as REPLICA IDENTITY on subscriber because
+# it reflects the future scenario we are testing: changing REPLICA IDENTITY
+# INDEX.
+$node_subscriber->safe_psql('postgres',
+	"ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b"
+);
+
+$publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+	"CREATE PUBLICATION tap_pub FOR TABLE tab_replidentity_index");
+$node_subscriber->safe_psql('postgres',
+	"CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub"
+);
+
+$node_publisher->wait_for_catchup('tap_sub');
+
+# Also wait for initial table sync to finish
+$node_subscriber->poll_query_until('postgres', $synced_query)
+  or die "Timed out while waiting for subscriber to synchronize data";
+
+is( $node_subscriber->safe_psql(
+		'postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(1|1
+2|2),
+	"check initial data on subscriber");
+
+# Set REPLICA IDENTITY to idx_replidentity_index_b on publisher, then run UPDATE and DELETE.
+$node_publisher->safe_psql(
+	'postgres', qq[
+	ALTER TABLE tab_replidentity_index REPLICA IDENTITY USING INDEX idx_replidentity_index_b;
+	UPDATE tab_replidentity_index SET a = -a WHERE a = 1;
+	DELETE FROM tab_replidentity_index WHERE a = 2;
+]);
+
+$node_publisher->wait_for_catchup('tap_sub');
+is( $node_subscriber->safe_psql(
+		'postgres', "SELECT * FROM tab_replidentity_index"),
+	qq(-1|1),
+	"update works with REPLICA IDENTITY");
+
+$node_publisher->stop('fast');
+$node_subscriber->stop('fast');
-- 
2.7.2.windows.1

#17Amit Kapila
amit.kapila16@gmail.com
In reply to: houzj.fnst@fujitsu.com (#16)
Re: [BUG]Invalidate relcache when setting REPLICA IDENTITY

On Mon, Nov 15, 2021 at 7:20 AM houzj.fnst@fujitsu.com
<houzj.fnst@fujitsu.com> wrote:

On Sat, Nov 13, 2021 6:50 PM Amit Kapila <amit.kapila16@gmail.com> wrote:

The patch looks mostly good to me. I have slightly tweaked the comments in
the code (as per my previous suggestion) and test. Also, I have slightly
modified the commit message. If the attached looks good to you then kindly
prepare patches for back-branches.

Thanks for reviewing, the latest patch looks good to me.
Attach the patches for back branch (HEAD ~ v10).

Pushed.

--
With Regards,
Amit Kapila.