More tests with USING INDEX replident and dropped indexes
Hi all,
While working on some other logical decoding patch recently, I bumped
into the fact that we have special handling for the case of REPLICA
IDENTITY USING INDEX when the dependent index is dropped, where the
code handles that case as an equivalent of NOTHING.
Attached is a patch to add more coverage for that. Any thoughts?
--
Michael
Attachments:
logidec-index-v1.patchtext/x-diff; charset=us-asciiDownload
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index d79cd316b7..fb4a7d7689 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -565,6 +565,36 @@ UPDATE table_with_unique_not_null SET data = 3 WHERE data = 2;
UPDATE table_with_unique_not_null SET id = -id;
UPDATE table_with_unique_not_null SET id = -id;
DELETE FROM table_with_unique_not_null WHERE data = 3;
+-- check tables with dropped indexes used in REPLICA IDENTITY
+-- with primary key
+CREATE TABLE table_dropped_index_with_pk (a int PRIMARY KEY, b int, c int);
+CREATE UNIQUE INDEX table_dropped_index_with_pk_idx
+ ON table_dropped_index_with_pk(a);
+ALTER TABLE table_dropped_index_with_pk REPLICA IDENTITY
+ USING INDEX table_dropped_index_with_pk_idx;
+DROP INDEX table_dropped_index_with_pk_idx;
+INSERT INTO table_dropped_index_with_pk VALUES (1,1,1), (2,2,2), (3,3,3);
+UPDATE table_dropped_index_with_pk SET a = 4 WHERE a = 1;
+UPDATE table_dropped_index_with_pk SET b = 5 WHERE a = 2;
+UPDATE table_dropped_index_with_pk SET b = 6, c = 7 WHERE a = 3;
+DELETE FROM table_dropped_index_with_pk WHERE b = 1;
+DELETE FROM table_dropped_index_with_pk WHERE a = 3;
+DROP TABLE table_dropped_index_with_pk;
+-- without primary key
+CREATE TABLE table_dropped_index_no_pk (a int NOT NULL, b int, c int);
+CREATE UNIQUE INDEX table_dropped_index_no_pk_idx
+ ON table_dropped_index_no_pk(a);
+ALTER TABLE table_dropped_index_no_pk REPLICA IDENTITY
+ USING INDEX table_dropped_index_no_pk_idx;
+INSERT INTO table_dropped_index_no_pk VALUES (1,1,1), (2,2,2), (3,3,3);
+DROP INDEX table_dropped_index_no_pk_idx;
+INSERT INTO table_dropped_index_no_pk VALUES (1,1,1), (2,2,2), (3,3,3);
+UPDATE table_dropped_index_no_pk SET a = 4 WHERE a = 1;
+UPDATE table_dropped_index_no_pk SET b = 5 WHERE a = 2;
+UPDATE table_dropped_index_no_pk SET b = 6, c = 7 WHERE a = 3;
+DELETE FROM table_dropped_index_no_pk WHERE b = 1;
+DELETE FROM table_dropped_index_no_pk WHERE a = 3;
+DROP TABLE table_dropped_index_no_pk;
-- check toast support
BEGIN;
CREATE SEQUENCE toasttable_rand_seq START 79 INCREMENT 1499; -- portable "random"
@@ -682,6 +712,56 @@ SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
table public.table_with_unique_not_null: DELETE: id[integer]:4
COMMIT
BEGIN
+ table public.table_dropped_index_with_pk: INSERT: a[integer]:1 b[integer]:1 c[integer]:1
+ table public.table_dropped_index_with_pk: INSERT: a[integer]:2 b[integer]:2 c[integer]:2
+ table public.table_dropped_index_with_pk: INSERT: a[integer]:3 b[integer]:3 c[integer]:3
+ COMMIT
+ BEGIN
+ table public.table_dropped_index_with_pk: UPDATE: a[integer]:4 b[integer]:1 c[integer]:1
+ COMMIT
+ BEGIN
+ table public.table_dropped_index_with_pk: UPDATE: a[integer]:2 b[integer]:5 c[integer]:2
+ COMMIT
+ BEGIN
+ table public.table_dropped_index_with_pk: UPDATE: a[integer]:3 b[integer]:6 c[integer]:7
+ COMMIT
+ BEGIN
+ table public.table_dropped_index_with_pk: DELETE: (no-tuple-data)
+ COMMIT
+ BEGIN
+ table public.table_dropped_index_with_pk: DELETE: (no-tuple-data)
+ COMMIT
+ BEGIN
+ table public.table_dropped_index_no_pk: INSERT: a[integer]:1 b[integer]:1 c[integer]:1
+ table public.table_dropped_index_no_pk: INSERT: a[integer]:2 b[integer]:2 c[integer]:2
+ table public.table_dropped_index_no_pk: INSERT: a[integer]:3 b[integer]:3 c[integer]:3
+ COMMIT
+ BEGIN
+ table public.table_dropped_index_no_pk: INSERT: a[integer]:1 b[integer]:1 c[integer]:1
+ table public.table_dropped_index_no_pk: INSERT: a[integer]:2 b[integer]:2 c[integer]:2
+ table public.table_dropped_index_no_pk: INSERT: a[integer]:3 b[integer]:3 c[integer]:3
+ COMMIT
+ BEGIN
+ table public.table_dropped_index_no_pk: UPDATE: a[integer]:4 b[integer]:1 c[integer]:1
+ table public.table_dropped_index_no_pk: UPDATE: a[integer]:4 b[integer]:1 c[integer]:1
+ COMMIT
+ BEGIN
+ table public.table_dropped_index_no_pk: UPDATE: a[integer]:2 b[integer]:5 c[integer]:2
+ table public.table_dropped_index_no_pk: UPDATE: a[integer]:2 b[integer]:5 c[integer]:2
+ COMMIT
+ BEGIN
+ table public.table_dropped_index_no_pk: UPDATE: a[integer]:3 b[integer]:6 c[integer]:7
+ table public.table_dropped_index_no_pk: UPDATE: a[integer]:3 b[integer]:6 c[integer]:7
+ COMMIT
+ BEGIN
+ table public.table_dropped_index_no_pk: DELETE: (no-tuple-data)
+ table public.table_dropped_index_no_pk: DELETE: (no-tuple-data)
+ COMMIT
+ BEGIN
+ table public.table_dropped_index_no_pk: DELETE: (no-tuple-data)
+ table public.table_dropped_index_no_pk: DELETE: (no-tuple-data)
+ COMMIT
+ BEGIN
table public.toasttable: INSERT: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
BEGIN
@@ -690,7 +770,7 @@ SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
BEGIN
table public.toasttable: UPDATE: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
-(103 rows)
+(153 rows)
INSERT INTO toasttable(toasted_col1) SELECT string_agg(g.i::text, '') FROM generate_series(1, 2000) g(i);
-- update of second column, first column unchanged
diff --git a/contrib/test_decoding/sql/ddl.sql b/contrib/test_decoding/sql/ddl.sql
index 2c4823e578..ac26048165 100644
--- a/contrib/test_decoding/sql/ddl.sql
+++ b/contrib/test_decoding/sql/ddl.sql
@@ -345,6 +345,38 @@ UPDATE table_with_unique_not_null SET id = -id;
UPDATE table_with_unique_not_null SET id = -id;
DELETE FROM table_with_unique_not_null WHERE data = 3;
+-- check tables with dropped indexes used in REPLICA IDENTITY
+-- with primary key
+CREATE TABLE table_dropped_index_with_pk (a int PRIMARY KEY, b int, c int);
+CREATE UNIQUE INDEX table_dropped_index_with_pk_idx
+ ON table_dropped_index_with_pk(a);
+ALTER TABLE table_dropped_index_with_pk REPLICA IDENTITY
+ USING INDEX table_dropped_index_with_pk_idx;
+DROP INDEX table_dropped_index_with_pk_idx;
+INSERT INTO table_dropped_index_with_pk VALUES (1,1,1), (2,2,2), (3,3,3);
+UPDATE table_dropped_index_with_pk SET a = 4 WHERE a = 1;
+UPDATE table_dropped_index_with_pk SET b = 5 WHERE a = 2;
+UPDATE table_dropped_index_with_pk SET b = 6, c = 7 WHERE a = 3;
+DELETE FROM table_dropped_index_with_pk WHERE b = 1;
+DELETE FROM table_dropped_index_with_pk WHERE a = 3;
+DROP TABLE table_dropped_index_with_pk;
+
+-- without primary key
+CREATE TABLE table_dropped_index_no_pk (a int NOT NULL, b int, c int);
+CREATE UNIQUE INDEX table_dropped_index_no_pk_idx
+ ON table_dropped_index_no_pk(a);
+ALTER TABLE table_dropped_index_no_pk REPLICA IDENTITY
+ USING INDEX table_dropped_index_no_pk_idx;
+INSERT INTO table_dropped_index_no_pk VALUES (1,1,1), (2,2,2), (3,3,3);
+DROP INDEX table_dropped_index_no_pk_idx;
+INSERT INTO table_dropped_index_no_pk VALUES (1,1,1), (2,2,2), (3,3,3);
+UPDATE table_dropped_index_no_pk SET a = 4 WHERE a = 1;
+UPDATE table_dropped_index_no_pk SET b = 5 WHERE a = 2;
+UPDATE table_dropped_index_no_pk SET b = 6, c = 7 WHERE a = 3;
+DELETE FROM table_dropped_index_no_pk WHERE b = 1;
+DELETE FROM table_dropped_index_no_pk WHERE a = 3;
+DROP TABLE table_dropped_index_no_pk;
+
-- check toast support
BEGIN;
CREATE SEQUENCE toasttable_rand_seq START 79 INCREMENT 1499; -- portable "random"
On Fri, 22 May 2020 at 12:50, Michael Paquier <michael@paquier.xyz> wrote:
Hi all,
While working on some other logical decoding patch recently, I bumped
into the fact that we have special handling for the case of REPLICA
IDENTITY USING INDEX when the dependent index is dropped, where the
code handles that case as an equivalent of NOTHING.Attached is a patch to add more coverage for that. Any thoughts?
How about avoiding such an inconsistent situation? In that case,
replica identity works as NOTHING, but pg_class.relreplident is still
‘i’, confusing users. It seems to me that dropping an index specified
by REPLICA IDENTITY USING INDEX is not a valid operation.
Regards,
--
Masahiko Sawada http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Tue, Jun 02, 2020 at 04:46:55PM +0900, Masahiko Sawada wrote:
How about avoiding such an inconsistent situation? In that case,
replica identity works as NOTHING, but pg_class.relreplident is still
‘i’, confusing users. It seems to me that dropping an index specified
by REPLICA IDENTITY USING INDEX is not a valid operation.
This looks first like complicating RemoveRelations() or the internal
object removal APIs with a dedicated lookup at this index's pg_index
tuple, but you could just put that in index_drop when REINDEX
CONCURRENTLY is not used. Still, I am not sure if it is worth
complicating those code paths. It would be better to get more
opinions about that first.
--
Michael
On Wed, 3 Jun 2020 at 03:14, Michael Paquier <michael@paquier.xyz> wrote:
On Tue, Jun 02, 2020 at 04:46:55PM +0900, Masahiko Sawada wrote:
How about avoiding such an inconsistent situation? In that case,
replica identity works as NOTHING, but pg_class.relreplident is still
‘i’, confusing users. It seems to me that dropping an index specified
by REPLICA IDENTITY USING INDEX is not a valid operation.This looks first like complicating RemoveRelations() or the internal
object removal APIs with a dedicated lookup at this index's pg_index
tuple, but you could just put that in index_drop when REINDEX
CONCURRENTLY is not used. Still, I am not sure if it is worth
complicating those code paths. It would be better to get more
opinions about that first.
Consistency is a good goal. Why don't we clear the relreplident from the
relation while dropping the index? relation_mark_replica_identity() already
does that but do other things too. Let's move the first code block from
relation_mark_replica_identity to another function and call this new
function
while dropping the index.
--
Euler Taveira http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Wed, Jun 03, 2020 at 12:08:56PM -0300, Euler Taveira wrote:
Consistency is a good goal. Why don't we clear the relreplident from the
relation while dropping the index? relation_mark_replica_identity() already
does that but do other things too. Let's move the first code block from
relation_mark_replica_identity to another function and call this new
function
while dropping the index.
I have looked at your suggestion, and finished with the attached.
There are a couple of things to be aware of:
- For DROP INDEX CONCURRENTLY, pg_class.relreplident of the parent
table is set in the last transaction doing the drop. It would be
tempting to reset the flag in the same transaction as the one marking
the index as invalid, but there could be a point in reindexing the
invalid index whose drop has failed, and this adds more complexity to
the patch.
- relreplident is switched to REPLICA_IDENTITY_NOTHING, which is the
behavior we have now after an index is dropped, even if there is a
primary key.
- The CCI done even if ri_type is not updated in index_drop() may look
useless, but the CCI will happen in any case as switching the replica
identity of a table to NOTHING resets pg_index.indisreplident for an
index previously used.
Thoughts?
--
Michael
Attachments:
logidec-index-v2.patchtext/x-diff; charset=us-asciiDownload
diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h
index c1581ad178..b163a82c52 100644
--- a/src/include/commands/tablecmds.h
+++ b/src/include/commands/tablecmds.h
@@ -61,6 +61,8 @@ extern void ExecuteTruncateGuts(List *explicit_rels, List *relids, List *relids_
extern void SetRelationHasSubclass(Oid relationId, bool relhassubclass);
+extern void SetRelationReplIdent(Oid relationId, char ri_type);
+
extern ObjectAddress renameatt(RenameStmt *stmt);
extern ObjectAddress RenameConstraint(RenameStmt *stmt);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..d344a070ce 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -2195,6 +2195,18 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
if (userIndexRelation->rd_rel->relkind != RELKIND_PARTITIONED_INDEX)
RelationDropStorage(userIndexRelation);
+ /*
+ * If the index to be dropped is marked as indisreplident, reset
+ * pg_class.relreplident for the parent table.
+ */
+ if (userIndexRelation->rd_index->indisreplident)
+ {
+ SetRelationReplIdent(heapId, REPLICA_IDENTITY_NOTHING);
+
+ /* make the update visible */
+ CommandCounterIncrement();
+ }
+
/*
* Close and flush the index's relcache entry, to ensure relcache doesn't
* try to rebuild it while we're deleting catalog entries. We keep the
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index cd989c95e5..e11e8baf89 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -2976,6 +2976,41 @@ SetRelationHasSubclass(Oid relationId, bool relhassubclass)
table_close(relationRelation, RowExclusiveLock);
}
+/*
+ * SetRelationReplIdent
+ * Set the value of the relation's relreplident field in pg_class.
+ *
+ * NOTE: caller must be holding an appropriate lock on the relation.
+ * ShareUpdateExclusiveLock is sufficient.
+ */
+void
+SetRelationReplIdent(Oid relationId, char ri_type)
+{
+ Relation relationRelation;
+ HeapTuple tuple;
+ Form_pg_class classtuple;
+
+ /*
+ * Fetch a modifiable copy of the tuple, modify it, update pg_class.
+ */
+ relationRelation = table_open(RelationRelationId, RowExclusiveLock);
+ tuple = SearchSysCacheCopy1(RELOID,
+ ObjectIdGetDatum(relationId));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for relation \"%s\"",
+ get_rel_name(relationId));
+ classtuple = (Form_pg_class) GETSTRUCT(tuple);
+
+ if (classtuple->relreplident != ri_type)
+ {
+ classtuple->relreplident = ri_type;
+ CatalogTupleUpdate(relationRelation, &tuple->t_self, tuple);
+ }
+
+ heap_freetuple(tuple);
+ table_close(relationRelation, RowExclusiveLock);
+}
+
/*
* renameatt_check - basic sanity checks before attribute rename
*/
@@ -14384,31 +14419,12 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
bool is_internal)
{
Relation pg_index;
- Relation pg_class;
- HeapTuple pg_class_tuple;
HeapTuple pg_index_tuple;
- Form_pg_class pg_class_form;
Form_pg_index pg_index_form;
-
ListCell *index;
- /*
- * Check whether relreplident has changed, and update it if so.
- */
- pg_class = table_open(RelationRelationId, RowExclusiveLock);
- pg_class_tuple = SearchSysCacheCopy1(RELOID,
- ObjectIdGetDatum(RelationGetRelid(rel)));
- if (!HeapTupleIsValid(pg_class_tuple))
- elog(ERROR, "cache lookup failed for relation \"%s\"",
- RelationGetRelationName(rel));
- pg_class_form = (Form_pg_class) GETSTRUCT(pg_class_tuple);
- if (pg_class_form->relreplident != ri_type)
- {
- pg_class_form->relreplident = ri_type;
- CatalogTupleUpdate(pg_class, &pg_class_tuple->t_self, pg_class_tuple);
- }
- table_close(pg_class, RowExclusiveLock);
- heap_freetuple(pg_class_tuple);
+ /* update relreplident if necessary */
+ SetRelationReplIdent(RelationGetRelid(rel), ri_type);
/*
* Check whether the correct index is marked indisreplident; if so, we're
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index fc329c5cff..6e293c9cca 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2144,8 +2144,7 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<literal>n</literal> = nothing,
<literal>f</literal> = all columns,
<literal>i</literal> = index with
- <structfield>indisreplident</structfield> set (same as nothing if the
- index used has been dropped)
+ <structfield>indisreplident</structfield> set
</para></entry>
</row>
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index d79cd316b7..3e7228a937 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -565,6 +565,50 @@ UPDATE table_with_unique_not_null SET data = 3 WHERE data = 2;
UPDATE table_with_unique_not_null SET id = -id;
UPDATE table_with_unique_not_null SET id = -id;
DELETE FROM table_with_unique_not_null WHERE data = 3;
+-- check tables with dropped indexes used in REPLICA IDENTITY
+-- with primary key
+CREATE TABLE table_dropped_index_with_pk (a int PRIMARY KEY, b int, c int);
+CREATE UNIQUE INDEX table_dropped_index_with_pk_idx
+ ON table_dropped_index_with_pk(a);
+ALTER TABLE table_dropped_index_with_pk REPLICA IDENTITY
+ USING INDEX table_dropped_index_with_pk_idx;
+DROP INDEX table_dropped_index_with_pk_idx;
+SELECT relreplident FROM pg_class
+ WHERE oid = 'table_dropped_index_with_pk'::regclass;
+ relreplident
+--------------
+ n
+(1 row)
+
+INSERT INTO table_dropped_index_with_pk VALUES (1,1,1), (2,2,2), (3,3,3);
+UPDATE table_dropped_index_with_pk SET a = 4 WHERE a = 1;
+UPDATE table_dropped_index_with_pk SET b = 5 WHERE a = 2;
+UPDATE table_dropped_index_with_pk SET b = 6, c = 7 WHERE a = 3;
+DELETE FROM table_dropped_index_with_pk WHERE b = 1;
+DELETE FROM table_dropped_index_with_pk WHERE a = 3;
+DROP TABLE table_dropped_index_with_pk;
+-- without primary key
+CREATE TABLE table_dropped_index_no_pk (a int NOT NULL, b int, c int);
+CREATE UNIQUE INDEX table_dropped_index_no_pk_idx
+ ON table_dropped_index_no_pk(a);
+ALTER TABLE table_dropped_index_no_pk REPLICA IDENTITY
+ USING INDEX table_dropped_index_no_pk_idx;
+INSERT INTO table_dropped_index_no_pk VALUES (1,1,1), (2,2,2), (3,3,3);
+DROP INDEX table_dropped_index_no_pk_idx;
+SELECT relreplident FROM pg_class
+ WHERE oid = 'table_dropped_index_no_pk'::regclass;
+ relreplident
+--------------
+ n
+(1 row)
+
+INSERT INTO table_dropped_index_no_pk VALUES (1,1,1), (2,2,2), (3,3,3);
+UPDATE table_dropped_index_no_pk SET a = 4 WHERE a = 1;
+UPDATE table_dropped_index_no_pk SET b = 5 WHERE a = 2;
+UPDATE table_dropped_index_no_pk SET b = 6, c = 7 WHERE a = 3;
+DELETE FROM table_dropped_index_no_pk WHERE b = 1;
+DELETE FROM table_dropped_index_no_pk WHERE a = 3;
+DROP TABLE table_dropped_index_no_pk;
-- check toast support
BEGIN;
CREATE SEQUENCE toasttable_rand_seq START 79 INCREMENT 1499; -- portable "random"
@@ -682,6 +726,56 @@ SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
table public.table_with_unique_not_null: DELETE: id[integer]:4
COMMIT
BEGIN
+ table public.table_dropped_index_with_pk: INSERT: a[integer]:1 b[integer]:1 c[integer]:1
+ table public.table_dropped_index_with_pk: INSERT: a[integer]:2 b[integer]:2 c[integer]:2
+ table public.table_dropped_index_with_pk: INSERT: a[integer]:3 b[integer]:3 c[integer]:3
+ COMMIT
+ BEGIN
+ table public.table_dropped_index_with_pk: UPDATE: a[integer]:4 b[integer]:1 c[integer]:1
+ COMMIT
+ BEGIN
+ table public.table_dropped_index_with_pk: UPDATE: a[integer]:2 b[integer]:5 c[integer]:2
+ COMMIT
+ BEGIN
+ table public.table_dropped_index_with_pk: UPDATE: a[integer]:3 b[integer]:6 c[integer]:7
+ COMMIT
+ BEGIN
+ table public.table_dropped_index_with_pk: DELETE: (no-tuple-data)
+ COMMIT
+ BEGIN
+ table public.table_dropped_index_with_pk: DELETE: (no-tuple-data)
+ COMMIT
+ BEGIN
+ table public.table_dropped_index_no_pk: INSERT: a[integer]:1 b[integer]:1 c[integer]:1
+ table public.table_dropped_index_no_pk: INSERT: a[integer]:2 b[integer]:2 c[integer]:2
+ table public.table_dropped_index_no_pk: INSERT: a[integer]:3 b[integer]:3 c[integer]:3
+ COMMIT
+ BEGIN
+ table public.table_dropped_index_no_pk: INSERT: a[integer]:1 b[integer]:1 c[integer]:1
+ table public.table_dropped_index_no_pk: INSERT: a[integer]:2 b[integer]:2 c[integer]:2
+ table public.table_dropped_index_no_pk: INSERT: a[integer]:3 b[integer]:3 c[integer]:3
+ COMMIT
+ BEGIN
+ table public.table_dropped_index_no_pk: UPDATE: a[integer]:4 b[integer]:1 c[integer]:1
+ table public.table_dropped_index_no_pk: UPDATE: a[integer]:4 b[integer]:1 c[integer]:1
+ COMMIT
+ BEGIN
+ table public.table_dropped_index_no_pk: UPDATE: a[integer]:2 b[integer]:5 c[integer]:2
+ table public.table_dropped_index_no_pk: UPDATE: a[integer]:2 b[integer]:5 c[integer]:2
+ COMMIT
+ BEGIN
+ table public.table_dropped_index_no_pk: UPDATE: a[integer]:3 b[integer]:6 c[integer]:7
+ table public.table_dropped_index_no_pk: UPDATE: a[integer]:3 b[integer]:6 c[integer]:7
+ COMMIT
+ BEGIN
+ table public.table_dropped_index_no_pk: DELETE: (no-tuple-data)
+ table public.table_dropped_index_no_pk: DELETE: (no-tuple-data)
+ COMMIT
+ BEGIN
+ table public.table_dropped_index_no_pk: DELETE: (no-tuple-data)
+ table public.table_dropped_index_no_pk: DELETE: (no-tuple-data)
+ COMMIT
+ BEGIN
table public.toasttable: INSERT: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
BEGIN
@@ -690,7 +784,7 @@ SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
BEGIN
table public.toasttable: UPDATE: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
-(103 rows)
+(153 rows)
INSERT INTO toasttable(toasted_col1) SELECT string_agg(g.i::text, '') FROM generate_series(1, 2000) g(i);
-- update of second column, first column unchanged
diff --git a/contrib/test_decoding/sql/ddl.sql b/contrib/test_decoding/sql/ddl.sql
index 2c4823e578..cbb6966e34 100644
--- a/contrib/test_decoding/sql/ddl.sql
+++ b/contrib/test_decoding/sql/ddl.sql
@@ -345,6 +345,42 @@ UPDATE table_with_unique_not_null SET id = -id;
UPDATE table_with_unique_not_null SET id = -id;
DELETE FROM table_with_unique_not_null WHERE data = 3;
+-- check tables with dropped indexes used in REPLICA IDENTITY
+-- with primary key
+CREATE TABLE table_dropped_index_with_pk (a int PRIMARY KEY, b int, c int);
+CREATE UNIQUE INDEX table_dropped_index_with_pk_idx
+ ON table_dropped_index_with_pk(a);
+ALTER TABLE table_dropped_index_with_pk REPLICA IDENTITY
+ USING INDEX table_dropped_index_with_pk_idx;
+DROP INDEX table_dropped_index_with_pk_idx;
+SELECT relreplident FROM pg_class
+ WHERE oid = 'table_dropped_index_with_pk'::regclass;
+INSERT INTO table_dropped_index_with_pk VALUES (1,1,1), (2,2,2), (3,3,3);
+UPDATE table_dropped_index_with_pk SET a = 4 WHERE a = 1;
+UPDATE table_dropped_index_with_pk SET b = 5 WHERE a = 2;
+UPDATE table_dropped_index_with_pk SET b = 6, c = 7 WHERE a = 3;
+DELETE FROM table_dropped_index_with_pk WHERE b = 1;
+DELETE FROM table_dropped_index_with_pk WHERE a = 3;
+DROP TABLE table_dropped_index_with_pk;
+
+-- without primary key
+CREATE TABLE table_dropped_index_no_pk (a int NOT NULL, b int, c int);
+CREATE UNIQUE INDEX table_dropped_index_no_pk_idx
+ ON table_dropped_index_no_pk(a);
+ALTER TABLE table_dropped_index_no_pk REPLICA IDENTITY
+ USING INDEX table_dropped_index_no_pk_idx;
+INSERT INTO table_dropped_index_no_pk VALUES (1,1,1), (2,2,2), (3,3,3);
+DROP INDEX table_dropped_index_no_pk_idx;
+SELECT relreplident FROM pg_class
+ WHERE oid = 'table_dropped_index_no_pk'::regclass;
+INSERT INTO table_dropped_index_no_pk VALUES (1,1,1), (2,2,2), (3,3,3);
+UPDATE table_dropped_index_no_pk SET a = 4 WHERE a = 1;
+UPDATE table_dropped_index_no_pk SET b = 5 WHERE a = 2;
+UPDATE table_dropped_index_no_pk SET b = 6, c = 7 WHERE a = 3;
+DELETE FROM table_dropped_index_no_pk WHERE b = 1;
+DELETE FROM table_dropped_index_no_pk WHERE a = 3;
+DROP TABLE table_dropped_index_no_pk;
+
-- check toast support
BEGIN;
CREATE SEQUENCE toasttable_rand_seq START 79 INCREMENT 1499; -- portable "random"
Hi,
I couldn't test the patch as it does not apply cleanly on master.
Please find below some review comments:
1. Would it better to throw a warning at the time of dropping the
REPLICA IDENTITY
index that it would also� drop the REPLICA IDENTITY of the parent table?
2. CCI is used after calling SetRelationReplIdent from index_drop() but not
after following call
relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
��������������������������������������������������������� bool is_internal)
+������ /* update relreplident if necessary */
+������ SetRelationReplIdent(RelationGetRelid(rel), ri_type);
3.�� I think the former variable names `pg_class_tuple` and
`pg_class_form`provided better clarity
�+������ HeapTuple tuple;
�+������ Form_pg_class classtuple;
- relreplident is switched to REPLICA_IDENTITY_NOTHING, which is the
behavior we have now after an index is dropped, even if there is a
primary key.
4. I am not aware of the reason of the current behavior, however it
seems better
to switch to REPLICA_IDENTITY_DEFAULT. Can't think of a reason why user
would not be
fine with using primary key as replica identity when replica identity
index is dropped
Thank you,
On Wed, Aug 19, 2020 at 05:33:36PM +0530, Rahila Syed wrote:
I couldn't test the patch as it does not apply cleanly on master.
The CF bot is green, and I can apply v2 cleanly FWIW:
http://commitfest.cputube.org/michael-paquier.html
Please find below some review comments:
1. Would it better to throw a warning at the time of dropping the REPLICA
IDENTITY index that it would also drop the REPLICA IDENTITY of the
parent table?
Not sure that's worth it. Updating relreplident is a matter of
consistency.
2. CCI is used after calling SetRelationReplIdent from index_drop() but not
after following callrelation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
bool is_internal)+ /* update relreplident if necessary */ + SetRelationReplIdent(RelationGetRelid(rel), ri_type);
Yes, not having a CCI here is the intention, because it is not
necessary. That's not the case in index_drop() where the pg_class
entry of the parent table gets updated afterwards.
3. I think the former variable names `pg_class_tuple` and
`pg_class_form`provided better clarity
+ HeapTuple tuple;+ Form_pg_class classtuple;
This is chosen to be consistent with SetRelationHasSubclass().
4. I am not aware of the reason of the current behavior, however it seems
betterto switch to REPLICA_IDENTITY_DEFAULT. Can't think of a reason why user
would not be fine with using primary key as replica identity when
replica identity index is dropped
Using NOTHING is more consistent with the current behavior we have
since 9.4 now. There could be an argument for switching back to the
default, but that could be surprising to change an old behavior.
--
Michael
Hi,
I couldn't test the patch as it does not apply cleanly on master.
The CF bot is green, and I can apply v2 cleanly FWIW:
http://commitfest.cputube.org/michael-paquier.html
Sorry, I didn't apply correctly.� The� tests pass for me. In addition, I
tested
with partitioned tables.� It works as expected and makes the REPLICA
IDENTITY
'n' for the partitions as well when an index on a partitioned table is
dropped.
Thank you,
On Tue, Aug 25, 2020 at 08:59:37PM +0530, Rahila wrote:
Sorry, I didn't apply correctly. The tests pass for me. In addition, I
tested with partitioned tables. It works as expected and makes the REPLICA
IDENTITY 'n' for the partitions as well when an index on a partitioned
table is dropped.
Indeed. I have added a test for that.
While looking at this patch again, I noticed that the new tests for
contrib/test_decoding/ and the improvements for relreplident are
really two different bullet points, and that the new tests should not
be impacted as long as we switch to NOTHING the replica identity once
its index is dropped. So, for now, I have applied the new decoding
tests with a first commit, and attached is an updated patch which
includes tests in the main regression test suite for
replica_identity.sql, which is more consistent with the rest as that's
the main place where we look at the state of pg_class.relreplident.
I'll go through this part again tomorrow, it is late here.
--
Michael
Attachments:
v3-0001-Reset-pg_class.relreplident-when-associated-repli.patchtext/x-diff; charset=us-asciiDownload
From 277d9b53e2be9098e72167b5de2a52c40e6e8542 Mon Sep 17 00:00:00 2001
From: Michael Paquier <michael@paquier.xyz>
Date: Wed, 26 Aug 2020 20:59:34 +0900
Subject: [PATCH v3] Reset pg_class.relreplident when associated replica index
is dropped
When an index associated to REPLICA IDENTITY USING INDEX gets dropped,
relreplident gets set to 'i', which can be confusing as the catalogs
have a state inconsistent with pg_index that lacks an associated entry
marked with indisreplident. This changes the logic so as such an index
dropped leads to 'n' to be set for the parent table, equivalent to
REPLICA IDENTITY NOTHING.
Author: Michael Paquier
Reviewed-by: Masahiko Sawada, Rahila Syed, Euler Taveira
Discussion: https://postgr.es/m/20200522035028.GO2355@paquier.xyz
---
src/include/commands/tablecmds.h | 2 +
src/backend/catalog/index.c | 12 ++++
src/backend/commands/tablecmds.c | 58 ++++++++++++-------
.../regress/expected/replica_identity.out | 30 ++++++++++
src/test/regress/sql/replica_identity.sql | 22 +++++++
doc/src/sgml/catalogs.sgml | 3 +-
6 files changed, 104 insertions(+), 23 deletions(-)
diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h
index c1581ad178..b163a82c52 100644
--- a/src/include/commands/tablecmds.h
+++ b/src/include/commands/tablecmds.h
@@ -61,6 +61,8 @@ extern void ExecuteTruncateGuts(List *explicit_rels, List *relids, List *relids_
extern void SetRelationHasSubclass(Oid relationId, bool relhassubclass);
+extern void SetRelationReplIdent(Oid relationId, char ri_type);
+
extern ObjectAddress renameatt(RenameStmt *stmt);
extern ObjectAddress RenameConstraint(RenameStmt *stmt);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..d344a070ce 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -2195,6 +2195,18 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
if (userIndexRelation->rd_rel->relkind != RELKIND_PARTITIONED_INDEX)
RelationDropStorage(userIndexRelation);
+ /*
+ * If the index to be dropped is marked as indisreplident, reset
+ * pg_class.relreplident for the parent table.
+ */
+ if (userIndexRelation->rd_index->indisreplident)
+ {
+ SetRelationReplIdent(heapId, REPLICA_IDENTITY_NOTHING);
+
+ /* make the update visible */
+ CommandCounterIncrement();
+ }
+
/*
* Close and flush the index's relcache entry, to ensure relcache doesn't
* try to rebuild it while we're deleting catalog entries. We keep the
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index d2b15a3387..968cb41f98 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -3020,6 +3020,41 @@ SetRelationHasSubclass(Oid relationId, bool relhassubclass)
table_close(relationRelation, RowExclusiveLock);
}
+/*
+ * SetRelationReplIdent
+ * Set the value of the relation's relreplident field in pg_class.
+ *
+ * NOTE: caller must be holding an appropriate lock on the relation.
+ * ShareUpdateExclusiveLock is sufficient.
+ */
+void
+SetRelationReplIdent(Oid relationId, char ri_type)
+{
+ Relation relationRelation;
+ HeapTuple tuple;
+ Form_pg_class classtuple;
+
+ /*
+ * Fetch a modifiable copy of the tuple, modify it, update pg_class.
+ */
+ relationRelation = table_open(RelationRelationId, RowExclusiveLock);
+ tuple = SearchSysCacheCopy1(RELOID,
+ ObjectIdGetDatum(relationId));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for relation \"%s\"",
+ get_rel_name(relationId));
+ classtuple = (Form_pg_class) GETSTRUCT(tuple);
+
+ if (classtuple->relreplident != ri_type)
+ {
+ classtuple->relreplident = ri_type;
+ CatalogTupleUpdate(relationRelation, &tuple->t_self, tuple);
+ }
+
+ heap_freetuple(tuple);
+ table_close(relationRelation, RowExclusiveLock);
+}
+
/*
* renameatt_check - basic sanity checks before attribute rename
*/
@@ -14487,31 +14522,12 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
bool is_internal)
{
Relation pg_index;
- Relation pg_class;
- HeapTuple pg_class_tuple;
HeapTuple pg_index_tuple;
- Form_pg_class pg_class_form;
Form_pg_index pg_index_form;
-
ListCell *index;
- /*
- * Check whether relreplident has changed, and update it if so.
- */
- pg_class = table_open(RelationRelationId, RowExclusiveLock);
- pg_class_tuple = SearchSysCacheCopy1(RELOID,
- ObjectIdGetDatum(RelationGetRelid(rel)));
- if (!HeapTupleIsValid(pg_class_tuple))
- elog(ERROR, "cache lookup failed for relation \"%s\"",
- RelationGetRelationName(rel));
- pg_class_form = (Form_pg_class) GETSTRUCT(pg_class_tuple);
- if (pg_class_form->relreplident != ri_type)
- {
- pg_class_form->relreplident = ri_type;
- CatalogTupleUpdate(pg_class, &pg_class_tuple->t_self, pg_class_tuple);
- }
- table_close(pg_class, RowExclusiveLock);
- heap_freetuple(pg_class_tuple);
+ /* update relreplident if necessary */
+ SetRelationReplIdent(RelationGetRelid(rel), ri_type);
/*
* Check whether the correct index is marked indisreplident; if so, we're
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 79002197a7..340f8615f5 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -227,3 +227,33 @@ DROP TABLE test_replica_identity;
DROP TABLE test_replica_identity2;
DROP TABLE test_replica_identity3;
DROP TABLE test_replica_identity_othertable;
+--
+-- Test that DROP INDEX on a index used for REPLICA IDENTITY sets
+-- relreplident back to NOTHING.
+--
+CREATE TABLE test_replica_identity_4 (id int NOT NULL);
+CREATE UNIQUE INDEX test_replica_index_4 ON test_replica_identity_4(id);
+ALTER TABLE test_replica_identity_4 REPLICA IDENTITY
+ USING INDEX test_replica_index_4;
+DROP INDEX test_replica_index_4;
+SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity_4'::regclass;
+ relreplident
+--------------
+ n
+(1 row)
+
+DROP TABLE test_replica_identity_4;
+-- Partitioned table
+CREATE TABLE test_replica_part (id int NOT NULL)
+ PARTITION BY RANGE (id);
+CREATE UNIQUE INDEX test_replica_part_index ON test_replica_part(id);
+ALTER TABLE test_replica_part REPLICA IDENTITY
+ USING INDEX test_replica_part_index;
+DROP INDEX test_replica_part_index;
+SELECT relreplident FROM pg_class WHERE oid = 'test_replica_part'::regclass;
+ relreplident
+--------------
+ n
+(1 row)
+
+DROP TABLE test_replica_part;
diff --git a/src/test/regress/sql/replica_identity.sql b/src/test/regress/sql/replica_identity.sql
index a5ac8f5567..18220a595b 100644
--- a/src/test/regress/sql/replica_identity.sql
+++ b/src/test/regress/sql/replica_identity.sql
@@ -98,3 +98,25 @@ DROP TABLE test_replica_identity;
DROP TABLE test_replica_identity2;
DROP TABLE test_replica_identity3;
DROP TABLE test_replica_identity_othertable;
+
+--
+-- Test that DROP INDEX on a index used for REPLICA IDENTITY sets
+-- relreplident back to NOTHING.
+--
+
+CREATE TABLE test_replica_identity_4 (id int NOT NULL);
+CREATE UNIQUE INDEX test_replica_index_4 ON test_replica_identity_4(id);
+ALTER TABLE test_replica_identity_4 REPLICA IDENTITY
+ USING INDEX test_replica_index_4;
+DROP INDEX test_replica_index_4;
+SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity_4'::regclass;
+DROP TABLE test_replica_identity_4;
+-- Partitioned table
+CREATE TABLE test_replica_part (id int NOT NULL)
+ PARTITION BY RANGE (id);
+CREATE UNIQUE INDEX test_replica_part_index ON test_replica_part(id);
+ALTER TABLE test_replica_part REPLICA IDENTITY
+ USING INDEX test_replica_part_index;
+DROP INDEX test_replica_part_index;
+SELECT relreplident FROM pg_class WHERE oid = 'test_replica_part'::regclass;
+DROP TABLE test_replica_part;
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 9fe260ecff..601b3d448c 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2144,8 +2144,7 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<literal>n</literal> = nothing,
<literal>f</literal> = all columns,
<literal>i</literal> = index with
- <structfield>indisreplident</structfield> set (same as nothing if the
- index used has been dropped)
+ <structfield>indisreplident</structfield> set
</para></entry>
</row>
--
2.28.0
On Wed, Aug 26, 2020 at 09:08:51PM +0900, Michael Paquier wrote:
I'll go through this part again tomorrow, it is late here.
There is actually a huge gotcha here that exists since replica
identities exist: relation_mark_replica_identity() only considers
valid indexes! So, on HEAD, if DROP INDEX CONCURRENTLY fails in the
second transaction doing the physical drop, then we would finish with
a catalog entry that has indisreplident and !indisinvalid. If you
reindex it afterwards and switch the index back to be valid, there
can be more than one valid index marked with indisreplident, which
messes up with the logic in tablecmds.c because we use
RelationGetIndexList(), that discards invalid indexes. This case is
actually rather similar to the handling of indisclustered.
So we have a set of two issues here:
1) indisreplident should be switched to false when we clear the valid
flag of an index for INDEX_DROP_CLEAR_VALID. This issue exists since
9.4.
2) To never finish in a state where we have REPLICA IDENTITY INDEX
without an index marked as indisreplident, we need to reset the
replident of the parent table in the same transaction as the one
clearing indisvalid for the concurrent case. That's a problem of the
patch proposed upthread.
Attached is a patch for 1) and 2) grouped together, to ease review for
now. I think that we had better fix 1) separately though, so I am
going to start a new thread about that with a separate patch as the
current thread is misleading.
--
Michael
Attachments:
v4-0001-Reset-pg_class.relreplident-when-associated-repli.patchtext/x-diff; charset=us-asciiDownload
From 8893acae8b4d14b1cccb3bdcf8c23e46215d2060 Mon Sep 17 00:00:00 2001
From: Michael Paquier <michael@paquier.xyz>
Date: Thu, 27 Aug 2020 11:27:04 +0900
Subject: [PATCH v4] Reset pg_class.relreplident when associated replica index
is dropped
When an index associated to REPLICA IDENTITY USING INDEX gets dropped,
relreplident gets set to 'i', which can be confusing as the catalogs
have a state inconsistent with pg_index that lacks an associated entry
marked with indisreplident. This changes the logic so as such an index
dropped leads to 'n' to be set for the parent table, equivalent to
REPLICA IDENTITY NOTHING.
Author: Michael Paquier
Reviewed-by: Masahiko Sawada, Rahila Syed, Euler Taveira
Discussion: https://postgr.es/m/20200522035028.GO2355@paquier.xyz
---
src/include/commands/tablecmds.h | 2 +
src/backend/catalog/index.c | 43 +++++++++++++-
src/backend/commands/tablecmds.c | 58 ++++++++++++-------
.../regress/expected/replica_identity.out | 30 ++++++++++
src/test/regress/sql/replica_identity.sql | 22 +++++++
doc/src/sgml/catalogs.sgml | 3 +-
6 files changed, 132 insertions(+), 26 deletions(-)
diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h
index c1581ad178..b163a82c52 100644
--- a/src/include/commands/tablecmds.h
+++ b/src/include/commands/tablecmds.h
@@ -61,6 +61,8 @@ extern void ExecuteTruncateGuts(List *explicit_rels, List *relids, List *relids_
extern void SetRelationHasSubclass(Oid relationId, bool relhassubclass);
+extern void SetRelationReplIdent(Oid relationId, char ri_type);
+
extern ObjectAddress renameatt(RenameStmt *stmt);
extern ObjectAddress RenameConstraint(RenameStmt *stmt);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..ccdda7561a 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -2005,6 +2005,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
Relation indexRelation;
HeapTuple tuple;
bool hasexprs;
+ bool resetreplident;
LockRelId heaprelid,
indexrelid;
LOCKTAG heaplocktag;
@@ -2048,6 +2049,15 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
CheckTableNotInUse(userIndexRelation, "DROP INDEX");
+ /*
+ * Check if the REPLICA IDENTITY of the parent relation needs to be
+ * reset. This should be done only if the index to drop here is
+ * marked as used in a replica identity. We cannot rely on the
+ * contents of pg_index for the index either as a concurrent drop
+ * may have changed indisreplident already.
+ */
+ resetreplident = userIndexRelation->rd_index->indisreplident;
+
/*
* Drop Index Concurrently is more or less the reverse process of Create
* Index Concurrently.
@@ -2159,10 +2169,32 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
/* Finish invalidation of index and mark it as dead */
index_concurrently_set_dead(heapId, indexId);
+ }
+ /*
+ * If the index to be dropped was marked as indisreplident (it could
+ * have been updated when clearing indisvalid previously), reset
+ * pg_class.relreplident for the parent table. Note that this part
+ * needs to be done in the same transaction as the one marking the
+ * index as invalid, so as we never finish with the case where the
+ * parent's replica identity is based on an index, without any index
+ * marked with indisreplident.
+ */
+ if (resetreplident)
+ {
+ SetRelationReplIdent(heapId, REPLICA_IDENTITY_NOTHING);
+
+ /* make the update visible, mandatory for the non-concurrent case */
+ CommandCounterIncrement();
+ }
+
+ /* continue the concurrent process */
+ if (concurrent)
+ {
/*
- * Again, commit the transaction to make the pg_index update visible
- * to other sessions.
+ * Again, commit the transaction to make the pg_index and pg_class
+ * (in the case where indisreplident is set) updates are visible to
+ * other sessions.
*/
CommitTransactionCommand();
StartTransactionCommand();
@@ -3349,10 +3381,13 @@ index_set_state_flags(Oid indexId, IndexStateFlagsAction action)
* CONCURRENTLY that failed partway through.)
*
* Note: the CLUSTER logic assumes that indisclustered cannot be
- * set on any invalid index, so clear that flag too.
+ * set on any invalid index, so clear that flag too. Similarly,
+ * ALTER TABLE assumes that indisreplident cannot be set for
+ * invalid indexes.
*/
indexForm->indisvalid = false;
indexForm->indisclustered = false;
+ indexForm->indisreplident = false;
break;
case INDEX_DROP_SET_DEAD:
@@ -3364,6 +3399,8 @@ index_set_state_flags(Oid indexId, IndexStateFlagsAction action)
* the index at all.
*/
Assert(!indexForm->indisvalid);
+ Assert(!indexForm->indisclustered);
+ Assert(!indexForm->indisreplident);
indexForm->indisready = false;
indexForm->indislive = false;
break;
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index d2b15a3387..968cb41f98 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -3020,6 +3020,41 @@ SetRelationHasSubclass(Oid relationId, bool relhassubclass)
table_close(relationRelation, RowExclusiveLock);
}
+/*
+ * SetRelationReplIdent
+ * Set the value of the relation's relreplident field in pg_class.
+ *
+ * NOTE: caller must be holding an appropriate lock on the relation.
+ * ShareUpdateExclusiveLock is sufficient.
+ */
+void
+SetRelationReplIdent(Oid relationId, char ri_type)
+{
+ Relation relationRelation;
+ HeapTuple tuple;
+ Form_pg_class classtuple;
+
+ /*
+ * Fetch a modifiable copy of the tuple, modify it, update pg_class.
+ */
+ relationRelation = table_open(RelationRelationId, RowExclusiveLock);
+ tuple = SearchSysCacheCopy1(RELOID,
+ ObjectIdGetDatum(relationId));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for relation \"%s\"",
+ get_rel_name(relationId));
+ classtuple = (Form_pg_class) GETSTRUCT(tuple);
+
+ if (classtuple->relreplident != ri_type)
+ {
+ classtuple->relreplident = ri_type;
+ CatalogTupleUpdate(relationRelation, &tuple->t_self, tuple);
+ }
+
+ heap_freetuple(tuple);
+ table_close(relationRelation, RowExclusiveLock);
+}
+
/*
* renameatt_check - basic sanity checks before attribute rename
*/
@@ -14487,31 +14522,12 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
bool is_internal)
{
Relation pg_index;
- Relation pg_class;
- HeapTuple pg_class_tuple;
HeapTuple pg_index_tuple;
- Form_pg_class pg_class_form;
Form_pg_index pg_index_form;
-
ListCell *index;
- /*
- * Check whether relreplident has changed, and update it if so.
- */
- pg_class = table_open(RelationRelationId, RowExclusiveLock);
- pg_class_tuple = SearchSysCacheCopy1(RELOID,
- ObjectIdGetDatum(RelationGetRelid(rel)));
- if (!HeapTupleIsValid(pg_class_tuple))
- elog(ERROR, "cache lookup failed for relation \"%s\"",
- RelationGetRelationName(rel));
- pg_class_form = (Form_pg_class) GETSTRUCT(pg_class_tuple);
- if (pg_class_form->relreplident != ri_type)
- {
- pg_class_form->relreplident = ri_type;
- CatalogTupleUpdate(pg_class, &pg_class_tuple->t_self, pg_class_tuple);
- }
- table_close(pg_class, RowExclusiveLock);
- heap_freetuple(pg_class_tuple);
+ /* update relreplident if necessary */
+ SetRelationReplIdent(RelationGetRelid(rel), ri_type);
/*
* Check whether the correct index is marked indisreplident; if so, we're
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 79002197a7..86053c5bb4 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -227,3 +227,33 @@ DROP TABLE test_replica_identity;
DROP TABLE test_replica_identity2;
DROP TABLE test_replica_identity3;
DROP TABLE test_replica_identity_othertable;
+--
+-- Test that DROP INDEX on a index used for REPLICA IDENTITY sets
+-- relreplident back to NOTHING.
+--
+CREATE TABLE test_replica_identity_4 (id int NOT NULL);
+CREATE UNIQUE INDEX test_replica_index_4 ON test_replica_identity_4(id);
+ALTER TABLE test_replica_identity_4 REPLICA IDENTITY
+ USING INDEX test_replica_index_4;
+DROP INDEX CONCURRENTLY test_replica_index_4;
+SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity_4'::regclass;
+ relreplident
+--------------
+ n
+(1 row)
+
+DROP TABLE test_replica_identity_4;
+-- Partitioned table
+CREATE TABLE test_replica_part (id int NOT NULL)
+ PARTITION BY RANGE (id);
+CREATE UNIQUE INDEX test_replica_part_index ON test_replica_part(id);
+ALTER TABLE test_replica_part REPLICA IDENTITY
+ USING INDEX test_replica_part_index;
+DROP INDEX test_replica_part_index;
+SELECT relreplident FROM pg_class WHERE oid = 'test_replica_part'::regclass;
+ relreplident
+--------------
+ n
+(1 row)
+
+DROP TABLE test_replica_part;
diff --git a/src/test/regress/sql/replica_identity.sql b/src/test/regress/sql/replica_identity.sql
index a5ac8f5567..6dd34dc39d 100644
--- a/src/test/regress/sql/replica_identity.sql
+++ b/src/test/regress/sql/replica_identity.sql
@@ -98,3 +98,25 @@ DROP TABLE test_replica_identity;
DROP TABLE test_replica_identity2;
DROP TABLE test_replica_identity3;
DROP TABLE test_replica_identity_othertable;
+
+--
+-- Test that DROP INDEX on a index used for REPLICA IDENTITY sets
+-- relreplident back to NOTHING.
+--
+
+CREATE TABLE test_replica_identity_4 (id int NOT NULL);
+CREATE UNIQUE INDEX test_replica_index_4 ON test_replica_identity_4(id);
+ALTER TABLE test_replica_identity_4 REPLICA IDENTITY
+ USING INDEX test_replica_index_4;
+DROP INDEX CONCURRENTLY test_replica_index_4;
+SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity_4'::regclass;
+DROP TABLE test_replica_identity_4;
+-- Partitioned table
+CREATE TABLE test_replica_part (id int NOT NULL)
+ PARTITION BY RANGE (id);
+CREATE UNIQUE INDEX test_replica_part_index ON test_replica_part(id);
+ALTER TABLE test_replica_part REPLICA IDENTITY
+ USING INDEX test_replica_part_index;
+DROP INDEX test_replica_part_index;
+SELECT relreplident FROM pg_class WHERE oid = 'test_replica_part'::regclass;
+DROP TABLE test_replica_part;
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 9fe260ecff..601b3d448c 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2144,8 +2144,7 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<literal>n</literal> = nothing,
<literal>f</literal> = all columns,
<literal>i</literal> = index with
- <structfield>indisreplident</structfield> set (same as nothing if the
- index used has been dropped)
+ <structfield>indisreplident</structfield> set
</para></entry>
</row>
--
2.28.0
On Thu, Aug 27, 2020 at 11:28:35AM +0900, Michael Paquier wrote:
Attached is a patch for 1) and 2) grouped together, to ease review for
now. I think that we had better fix 1) separately though, so I am
going to start a new thread about that with a separate patch as the
current thread is misleading.
A fix for consistency problem with indisreplident and invalid indexes
has been committed as of 9511fb37. Attached is a rebased patch, where
I noticed two incorrect things:
- The comment of REPLICA_IDENTITY_INDEX is originally incorrect. If
the index is dropped, the replica index switches back to nothing.
- relreplident was getting reset one transaction too late, when the
old index is marked as dead.
--
Michael
Attachments:
v5-0001-Reset-pg_class.relreplident-when-associated-repli.patchtext/x-diff; charset=us-asciiDownload
From fa7585609a614bdba27bc2e3199379d04fc0eabd Mon Sep 17 00:00:00 2001
From: Michael Paquier <michael@paquier.xyz>
Date: Sun, 30 Aug 2020 16:02:32 +0900
Subject: [PATCH v5] Reset pg_class.relreplident when associated replica index
is dropped
When an index associated to REPLICA IDENTITY USING INDEX gets dropped,
relreplident gets set to 'i', which can be confusing as the catalogs
have a state inconsistent with pg_index that lacks an associated entry
marked with indisreplident. This changes the logic so as such an index
dropped leads to 'n' to be set for the parent table, equivalent to
REPLICA IDENTITY NOTHING.
Author: Michael Paquier
Reviewed-by: Masahiko Sawada, Rahila Syed, Euler Taveira
Discussion: https://postgr.es/m/20200522035028.GO2355@paquier.xyz
---
src/include/catalog/pg_class.h | 6 +-
src/include/commands/tablecmds.h | 2 +
src/backend/catalog/index.c | 37 +++++++++++-
src/backend/commands/tablecmds.c | 58 ++++++++++++-------
.../regress/expected/replica_identity.out | 30 ++++++++++
src/test/regress/sql/replica_identity.sql | 22 +++++++
doc/src/sgml/catalogs.sgml | 3 +-
7 files changed, 128 insertions(+), 30 deletions(-)
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index 78b33b2a7f..4f9ce17651 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -175,11 +175,7 @@ typedef FormData_pg_class *Form_pg_class;
#define REPLICA_IDENTITY_NOTHING 'n'
/* all columns are logged as replica identity */
#define REPLICA_IDENTITY_FULL 'f'
-/*
- * an explicitly chosen candidate key's columns are used as replica identity.
- * Note this will still be set if the index has been dropped; in that case it
- * has the same meaning as 'd'.
- */
+/* an explicitly-chosen candidate key's columns are used as replica identity */
#define REPLICA_IDENTITY_INDEX 'i'
/*
diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h
index c1581ad178..b163a82c52 100644
--- a/src/include/commands/tablecmds.h
+++ b/src/include/commands/tablecmds.h
@@ -61,6 +61,8 @@ extern void ExecuteTruncateGuts(List *explicit_rels, List *relids, List *relids_
extern void SetRelationHasSubclass(Oid relationId, bool relhassubclass);
+extern void SetRelationReplIdent(Oid relationId, char ri_type);
+
extern ObjectAddress renameatt(RenameStmt *stmt);
extern ObjectAddress RenameConstraint(RenameStmt *stmt);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 62e487bb4c..61d05495b4 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -2005,6 +2005,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
Relation indexRelation;
HeapTuple tuple;
bool hasexprs;
+ bool resetreplident;
LockRelId heaprelid,
indexrelid;
LOCKTAG heaplocktag;
@@ -2048,6 +2049,15 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
CheckTableNotInUse(userIndexRelation, "DROP INDEX");
+ /*
+ * Check if the REPLICA IDENTITY of the parent relation needs to be
+ * reset. This should be done only if the index to drop here is
+ * marked as used in a replica identity. We cannot rely on the
+ * contents of pg_index for the index as a concurrent drop would have
+ * reset indisreplident already, so save the existing value first.
+ */
+ resetreplident = userIndexRelation->rd_index->indisreplident;
+
/*
* Drop Index Concurrently is more or less the reverse process of Create
* Index Concurrently.
@@ -2132,7 +2142,29 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
LockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
LockRelationIdForSession(&indexrelid, ShareUpdateExclusiveLock);
+ }
+ /*
+ * If the index to be dropped was marked as indisreplident (note that it
+ * would have been reset when clearing indisvalid previously), reset
+ * pg_class.relreplident for the parent table. Note that this part
+ * needs to be done in the same transaction as the one marking the
+ * index as invalid, so as we never finish with the case where the
+ * parent relation uses REPLICA_IDENTITY_INDEX, without any index marked
+ * with indisreplident.
+ */
+ if (resetreplident)
+ {
+ SetRelationReplIdent(heapId, REPLICA_IDENTITY_NOTHING);
+
+ /* make the update visible, only for the non-concurrent case */
+ if (!concurrent)
+ CommandCounterIncrement();
+ }
+
+ /* continue the concurrent process */
+ if (concurrent)
+ {
PopActiveSnapshot();
CommitTransactionCommand();
StartTransactionCommand();
@@ -2161,8 +2193,9 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
index_concurrently_set_dead(heapId, indexId);
/*
- * Again, commit the transaction to make the pg_index update visible
- * to other sessions.
+ * Again, commit the transaction to make the pg_index and pg_class
+ * (in the case where indisreplident is set) updates are visible to
+ * other sessions.
*/
CommitTransactionCommand();
StartTransactionCommand();
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index d2b15a3387..968cb41f98 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -3020,6 +3020,41 @@ SetRelationHasSubclass(Oid relationId, bool relhassubclass)
table_close(relationRelation, RowExclusiveLock);
}
+/*
+ * SetRelationReplIdent
+ * Set the value of the relation's relreplident field in pg_class.
+ *
+ * NOTE: caller must be holding an appropriate lock on the relation.
+ * ShareUpdateExclusiveLock is sufficient.
+ */
+void
+SetRelationReplIdent(Oid relationId, char ri_type)
+{
+ Relation relationRelation;
+ HeapTuple tuple;
+ Form_pg_class classtuple;
+
+ /*
+ * Fetch a modifiable copy of the tuple, modify it, update pg_class.
+ */
+ relationRelation = table_open(RelationRelationId, RowExclusiveLock);
+ tuple = SearchSysCacheCopy1(RELOID,
+ ObjectIdGetDatum(relationId));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for relation \"%s\"",
+ get_rel_name(relationId));
+ classtuple = (Form_pg_class) GETSTRUCT(tuple);
+
+ if (classtuple->relreplident != ri_type)
+ {
+ classtuple->relreplident = ri_type;
+ CatalogTupleUpdate(relationRelation, &tuple->t_self, tuple);
+ }
+
+ heap_freetuple(tuple);
+ table_close(relationRelation, RowExclusiveLock);
+}
+
/*
* renameatt_check - basic sanity checks before attribute rename
*/
@@ -14487,31 +14522,12 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
bool is_internal)
{
Relation pg_index;
- Relation pg_class;
- HeapTuple pg_class_tuple;
HeapTuple pg_index_tuple;
- Form_pg_class pg_class_form;
Form_pg_index pg_index_form;
-
ListCell *index;
- /*
- * Check whether relreplident has changed, and update it if so.
- */
- pg_class = table_open(RelationRelationId, RowExclusiveLock);
- pg_class_tuple = SearchSysCacheCopy1(RELOID,
- ObjectIdGetDatum(RelationGetRelid(rel)));
- if (!HeapTupleIsValid(pg_class_tuple))
- elog(ERROR, "cache lookup failed for relation \"%s\"",
- RelationGetRelationName(rel));
- pg_class_form = (Form_pg_class) GETSTRUCT(pg_class_tuple);
- if (pg_class_form->relreplident != ri_type)
- {
- pg_class_form->relreplident = ri_type;
- CatalogTupleUpdate(pg_class, &pg_class_tuple->t_self, pg_class_tuple);
- }
- table_close(pg_class, RowExclusiveLock);
- heap_freetuple(pg_class_tuple);
+ /* update relreplident if necessary */
+ SetRelationReplIdent(RelationGetRelid(rel), ri_type);
/*
* Check whether the correct index is marked indisreplident; if so, we're
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 79002197a7..86053c5bb4 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -227,3 +227,33 @@ DROP TABLE test_replica_identity;
DROP TABLE test_replica_identity2;
DROP TABLE test_replica_identity3;
DROP TABLE test_replica_identity_othertable;
+--
+-- Test that DROP INDEX on a index used for REPLICA IDENTITY sets
+-- relreplident back to NOTHING.
+--
+CREATE TABLE test_replica_identity_4 (id int NOT NULL);
+CREATE UNIQUE INDEX test_replica_index_4 ON test_replica_identity_4(id);
+ALTER TABLE test_replica_identity_4 REPLICA IDENTITY
+ USING INDEX test_replica_index_4;
+DROP INDEX CONCURRENTLY test_replica_index_4;
+SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity_4'::regclass;
+ relreplident
+--------------
+ n
+(1 row)
+
+DROP TABLE test_replica_identity_4;
+-- Partitioned table
+CREATE TABLE test_replica_part (id int NOT NULL)
+ PARTITION BY RANGE (id);
+CREATE UNIQUE INDEX test_replica_part_index ON test_replica_part(id);
+ALTER TABLE test_replica_part REPLICA IDENTITY
+ USING INDEX test_replica_part_index;
+DROP INDEX test_replica_part_index;
+SELECT relreplident FROM pg_class WHERE oid = 'test_replica_part'::regclass;
+ relreplident
+--------------
+ n
+(1 row)
+
+DROP TABLE test_replica_part;
diff --git a/src/test/regress/sql/replica_identity.sql b/src/test/regress/sql/replica_identity.sql
index a5ac8f5567..6dd34dc39d 100644
--- a/src/test/regress/sql/replica_identity.sql
+++ b/src/test/regress/sql/replica_identity.sql
@@ -98,3 +98,25 @@ DROP TABLE test_replica_identity;
DROP TABLE test_replica_identity2;
DROP TABLE test_replica_identity3;
DROP TABLE test_replica_identity_othertable;
+
+--
+-- Test that DROP INDEX on a index used for REPLICA IDENTITY sets
+-- relreplident back to NOTHING.
+--
+
+CREATE TABLE test_replica_identity_4 (id int NOT NULL);
+CREATE UNIQUE INDEX test_replica_index_4 ON test_replica_identity_4(id);
+ALTER TABLE test_replica_identity_4 REPLICA IDENTITY
+ USING INDEX test_replica_index_4;
+DROP INDEX CONCURRENTLY test_replica_index_4;
+SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity_4'::regclass;
+DROP TABLE test_replica_identity_4;
+-- Partitioned table
+CREATE TABLE test_replica_part (id int NOT NULL)
+ PARTITION BY RANGE (id);
+CREATE UNIQUE INDEX test_replica_part_index ON test_replica_part(id);
+ALTER TABLE test_replica_part REPLICA IDENTITY
+ USING INDEX test_replica_part_index;
+DROP INDEX test_replica_part_index;
+SELECT relreplident FROM pg_class WHERE oid = 'test_replica_part'::regclass;
+DROP TABLE test_replica_part;
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 9fe260ecff..601b3d448c 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2144,8 +2144,7 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<literal>n</literal> = nothing,
<literal>f</literal> = all columns,
<literal>i</literal> = index with
- <structfield>indisreplident</structfield> set (same as nothing if the
- index used has been dropped)
+ <structfield>indisreplident</structfield> set
</para></entry>
</row>
--
2.28.0
Hi Michael,
Thanks for updating the patch.
Please see following comments,
1.
������� */
�������� if (resetreplident)
�������� {
��������������� SetRelationReplIdent(heapId, REPLICA_IDENTITY_NOTHING);
��������������� /* make the update visible, only for the non-concurrent
case */
��������������� if (!concurrent)
��������������������� CommandCounterIncrement();
��� � }
�������� /* continue the concurrent process */
�������� if (concurrent)
�������� {
���������������� PopActiveSnapshot();
���������������� CommitTransactionCommand();
���������������� StartTransactionCommand();
Now, the relreplident is being set in the transaction previous to
the one marking index as invalid for concurrent drop. Won't
it cause issues with relreplident cleared but index not dropped,
if drop index concurrently fails in the small window after
commit transaction in above snippet and before the start transaction above?
Although, it seems like a really small window.
2.� I have following suggestion for the test.
To the existing partitioned table test, can we add a test to demonstrate
that relreplident is set to 'n' for even the individual partitions.
I mean explicitly setting replica identity index for individual partitions
ALTER TABLE part1 REPLICA IDENTITY
USING INDEX test_replica_part_index_part_1;
and checking for relreplident for individual partition after parent
index is dropped.
SELECT relreplident FROM pg_class WHERE oid = 'part1'::regclass;
3.
� +* Again, commit the transaction to make the pg_index and pg_class
� +��������������� * (in the case where indisreplident is set) updates
are visible to
� +��������������� * other sessions.
Typo, s/updates are visible/updates visible
4. I am wondering with the recent change to move the SetRelationRepldent
together with invalidating index, whether your initial concern stated as
follows
becomes valid.
- For DROP INDEX CONCURRENTLY, pg_class.relreplident of the parent
table is set in the last transaction doing the drop.� It would be
tempting to reset the flag in the same transaction as the one marking
the index as invalid, but there could be a point in reindexing the
invalid index whose drop has failed, and this addsmorecomplexity to
the patch.
I will try to test that.
Thank you,
Rahila Syed
On Mon, Aug 31, 2020 at 10:36:13AM +0530, Rahila wrote:
Now, the relreplident is being set in the transaction previous to
the one marking index as invalid for concurrent drop. Won't
it cause issues with relreplident cleared but index not dropped,
if drop index concurrently fails in the small window after
commit transaction in above snippet and before the start transaction above?
Argh. I missed your point that this stuff uses heap_inplace_update(),
so the update of indisvalid flag is not transactional. The thing is
that we won't be able to update the flag consistently with
relreplident except if we switch index_set_state_flags() to use a
transactional operation here. So, with the current state of affairs,
it looks like we could just call SetRelationReplIdent() in the
last transaction, after the transaction marking the index as dead has
committed (see the top of index_set_state_flags() saying that this
should be the last step of a transaction), but that leaves a window
open as you say.
On the other hand, it looks appealing to make index_set_state_flags()
transactional. This would solve the consistency problem, and looking
at the code scanning pg_index, I don't see a reason why we could not
do that. What do you think?
To the existing partitioned table test, can we add a test to demonstrate
that relreplident is set to 'n' for even the individual partitions.
Makes sense. It is still important to test the case where a
partitioned table without leaves is correctly reset IMO.
Typo, s/updates are visible/updates visible
Thanks.
- For DROP INDEX CONCURRENTLY, pg_class.relreplident of the parent
table is set in the last transaction doing the drop. It would be
tempting to reset the flag in the same transaction as the one marking
the index as invalid, but there could be a point in reindexing the
invalid index whose drop has failed, and this adds morecomplexity to
the patch.
That's a point I studied, but it makes actually more sense to just
reset the flag once the index is marked as invalid, similarly to
indisclustered. Reindexing an invalid index can have value in some
cases, but we would have much more problems with the relcache if we
finish with two indexes marked as indreplident :(
--
Michael
On the other hand, it looks appealing to make index_set_state_flags()
transactional. This would solve the consistency problem, and looking
at the code scanning pg_index, I don't see a reason why we could not
do that. What do you think?
TBH, I am not sure. I think it is a reasonable change. It is even
indicated in the
comment above index_set_state_flags() that it can be made transactional.
At the same time, probably we can just go ahead with current
inconsistent update of relisreplident and indisvalid flags. Can't see what
will break with that.
Thank you,
--
Rahila Syed
Performance Engineer
2ndQuadrant
http://www.2ndQuadrant.com <http://www.2ndquadrant.com/>
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Wed, Sep 02, 2020 at 09:27:33AM +0530, Rahila Syed wrote:
TBH, I am not sure. I think it is a reasonable change. It is even
indicated in the
comment above index_set_state_flags() that it can be made transactional.
At the same time, probably we can just go ahead with current
inconsistent update of relisreplident and indisvalid flags. Can't see what
will break with that.
Yeah, that's true as well. Still, I would like to see first if people
are fine with changing this code path to be transactional. This way,
we will have zero history in the tree where there was a risk for an
inconsistent window.
--
Michael
On Wed, Sep 02, 2020 at 03:00:44PM +0900, Michael Paquier wrote:
Yeah, that's true as well. Still, I would like to see first if people
are fine with changing this code path to be transactional. This way,
we will have zero history in the tree where there was a risk for an
inconsistent window.
So, I have begun a thread about that part with a dedicated CF entry:
https://commitfest.postgresql.org/30/2725/
While considering more this point, I think that making this code path
transactional is the correct way to go, as a first step. Also, as
this thread is about adding more tests and that this has been done
with fe7fd4e9, I am marking the CF entry as committed. Let's discuss
the reset of relreplident for the parent relation when its replica
identity index is dropped once the transactional part is fully
discussed, on a new thread.
--
Michael