[PATCH] Logical decoding of TRUNCATE
Started by Marco Nenciariniabout 8 years ago49 messages
Hi,
This patch implements support for TRUNCATE statements
in logical replication. The work has mainly done by Simon Riggs then
finished by me. Tests are written by me.
TRUNCATE is treated as a form of DELETE for the purpose of deciding
whether to publish, or not.
The "TRUNCATE behavior when session_replication_role = replica"[1]https://commitfest.postgresql.org/16/1447/ patch
is required from this patch to work correctly with tables referenced by
foreign keys.
[1]: https://commitfest.postgresql.org/16/1447/
Regards,
Marco
--
Marco Nenciarini - 2ndQuadrant Italy
PostgreSQL Training, Services and Support
marco.nenciarini@2ndQuadrant.it | www.2ndQuadrant.it
Attachments:
logical_decode_truncate.v10.patchtext/plain; charset=UTF-8; name=logical_decode_truncate.v10.patch; x-mac-creator=0; x-mac-type=0Download
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 1e22c1eefc..d1feea4909 100644
*** a/contrib/test_decoding/expected/ddl.out
--- b/contrib/test_decoding/expected/ddl.out
***************
*** 543,548 **** UPDATE table_with_unique_not_null SET data = 3 WHERE data = 2;
--- 543,550 ----
UPDATE table_with_unique_not_null SET id = -id;
UPDATE table_with_unique_not_null SET id = -id;
DELETE FROM table_with_unique_not_null WHERE data = 3;
+ TRUNCATE table_with_unique_not_null;
+ TRUNCATE table_with_unique_not_null, table_with_unique_not_null RESTART IDENTITY CASCADE;
-- check toast support
BEGIN;
CREATE SEQUENCE toasttable_rand_seq START 79 INCREMENT 1499; -- portable "random"
***************
*** 660,665 **** SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
--- 662,673 ----
table public.table_with_unique_not_null: DELETE: id[integer]:4
COMMIT
BEGIN
+ table public.table_with_unique_not_null: TRUNCATE: (no-flags)
+ COMMIT
+ BEGIN
+ table public.table_with_unique_not_null: TRUNCATE: restart_seqs cascade
+ COMMIT
+ BEGIN
table public.toasttable: INSERT: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
BEGIN
***************
*** 668,674 **** SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
BEGIN
table public.toasttable: UPDATE: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
! (103 rows)
INSERT INTO toasttable(toasted_col1) SELECT string_agg(g.i::text, '') FROM generate_series(1, 2000) g(i);
-- update of second column, first column unchanged
--- 676,682 ----
BEGIN
table public.toasttable: UPDATE: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
! (109 rows)
INSERT INTO toasttable(toasted_col1) SELECT string_agg(g.i::text, '') FROM generate_series(1, 2000) g(i);
-- update of second column, first column unchanged
diff --git a/contrib/test_decoding/sql/ddl.sqindex 057dae056b..0fb4102db8 100644
*** a/contrib/test_decoding/sql/ddl.sql
--- b/contrib/test_decoding/sql/ddl.sql
***************
*** 333,338 **** UPDATE table_with_unique_not_null SET id = -id;
--- 333,341 ----
UPDATE table_with_unique_not_null SET id = -id;
DELETE FROM table_with_unique_not_null WHERE data = 3;
+ TRUNCATE table_with_unique_not_null;
+ TRUNCATE table_with_unique_not_null, table_with_unique_not_null RESTART IDENTITY CASCADE;
+
-- check toast support
BEGIN;
CREATE SEQUENCE toasttable_rand_seq START 79 INCREMENT 1499; -- portable "random"
diff --git a/contrib/test_decoding/test_index 135b3b7638..f99f91b9b0 100644
*** a/contrib/test_decoding/test_decoding.c
--- b/contrib/test_decoding/test_decoding.c
***************
*** 466,471 **** pg_decode_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
--- 466,485 ----
&change->data.tp.oldtuple->tuple,
true);
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ appendStringInfoString(ctx->out, " TRUNCATE:");
+
+ if (change->data.truncate_msg.restart_seqs
+ || change->data.truncate_msg.cascade)
+ {
+ if (change->data.truncate_msg.restart_seqs)
+ appendStringInfo(ctx->out, " restart_seqs");
+ if (change->data.truncate_msg.cascade)
+ appendStringInfo(ctx->out, " cascade");
+ }
+ else
+ appendStringInfoString(ctx->out, " (no-flags)");
+ break;
default:
Assert(false);
}
diff --git a/doc/src/sgml/logical-replicatioindex 75551d8ee1..6b64c8ef76 100644
*** a/doc/src/sgml/logical-replication.sgml
--- b/doc/src/sgml/logical-replication.sgml
***************
*** 111,116 ****
--- 111,118 ----
any combination of <command>INSERT</command>, <command>UPDATE</command>, and
<command>DELETE</command>, similar to how triggers are fired by
particular event types. By default, all operation types are replicated.
+ <command>TRUNCATE</command> is treated as a form of <command>DELETE</command>
+ for the purpose of deciding whether to publish, or not.
</para>
<para>
***************
*** 364,378 ****
</para>
</listitem>
- <listitem>
- <para>
- <command>TRUNCATE</command> commands are not replicated. This can, of
- course, be worked around by using <command>DELETE</command> instead. To
- avoid accidental <command>TRUNCATE</command> invocations, you can revoke
- the <literal>TRUNCATE</literal> privilege from tables.
- </para>
- </listitem>
-
<listitem>
<para>
Large objects (see <xref linkend="largeobjects"/>) are not replicated.
--- 366,371 ----
diff --git a/doc/src/sgml/protocol.sgml b/doindex 8174e3defa..88cbecfbdd 100644
*** a/doc/src/sgml/protocol.sgml
--- b/doc/src/sgml/protocol.sgml
***************
*** 6828,6833 **** TupleData
--- 6828,6874 ----
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ Truncate
+ </term>
+ <listitem>
+ <para>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ Byte1('T')
+ </term>
+ <listitem>
+ <para>
+ Identifies the message as a truncate message.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ Int32
+ </term>
+ <listitem>
+ <para>
+ ID of the relation corresponding to the ID in the relation
+ message.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ Int8
+ </term>
+ <listitem>
+ <para>
+ Option flags for truncate. Currently can be 0 for no flags,
+ 1 for CASCADE, 2 for RESTART IDENTITY and 3 if both are set.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</para>
</listitem>
diff --git a/doc/src/sgml/ref/creindex bfe12d5f41..c5032938a7 100644
*** a/doc/src/sgml/ref/create_publication.sgml
--- b/doc/src/sgml/ref/create_publication.sgml
***************
*** 111,116 **** CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
--- 111,121 ----
and so the default value for this option is
<literal>'insert, update, delete'</literal>.
</para>
+ <para>
+ <command>TRUNCATE</command> is treated as a form of
+ <command>DELETE</command> for the purpose of deciding whether
+ to publish, or not.
+ </para>
</listitem>
</varlistentry>
</variablelist>
***************
*** 168,175 **** CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
</para>
<para>
! <command>TRUNCATE</command> and <acronym>DDL</acronym> operations
! are not published.
</para>
</refsect1>
--- 173,179 ----
</para>
<para>
! <acronym>DDL</acronym> operations are not published.
</para>
</refsect1>
diff --git a/src/backend/access/heap/heapam.c bindex 54f1100ffd..2e1cea373d 100644
*** a/src/backend/access/heap/heapam.c
--- b/src/backend/access/heap/heapam.c
***************
*** 9134,9139 **** heap_redo(XLogReaderState *record)
--- 9134,9146 ----
case XLOG_HEAP_UPDATE:
heap_xlog_update(record, false);
break;
+ case XLOG_HEAP_TRUNCATE:
+ /*
+ * TRUNCATE is a no-op because the actions are already logged
+ * as SMGR WAL records. TRUNCATE WAL record only exists to allow
+ * it to be logically decoded precisely.
+ */
+ break;
case XLOG_HEAP_HOT_UPDATE:
heap_xlog_update(record, true);
break;
diff --git a/src/backend/access/rmgrdesindex 44d2d6333f..a44996379b 100644
*** a/src/backend/access/rmgrdesc/heapdesc.c
--- b/src/backend/access/rmgrdesc/heapdesc.c
***************
*** 75,80 **** heap_desc(StringInfo buf, XLogReaderState *record)
--- 75,93 ----
xlrec->new_offnum,
xlrec->new_xmax);
}
+ else if (info == XLOG_HEAP_TRUNCATE)
+ {
+ xl_heap_truncate *xlrec = (xl_heap_truncate *) rec;
+
+ if (xlrec->flags & XLH_TRUNCATE_CASCADE)
+ appendStringInfo(buf, "cascade ");
+ if (xlrec->flags & XLH_TRUNCATE_RESTART_SEQS)
+ appendStringInfo(buf, "restart_seqs ");
+ appendStringInfo(buf, "nrelids %u nseqrelids %u",
+ xlrec->nrelids,
+ xlrec->nseqrelids);
+ /* Skip the list of relids and seqrelids */
+ }
else if (info == XLOG_HEAP_CONFIRM)
{
xl_heap_confirm *xlrec = (xl_heap_confirm *) rec;
***************
*** 186,191 **** heap_identify(uint8 info)
--- 199,207 ----
case XLOG_HEAP_HOT_UPDATE | XLOG_HEAP_INIT_PAGE:
id = "HOT_UPDATE+INIT";
break;
+ case XLOG_HEAP_TRUNCATE:
+ id = "TRUNCATE";
+ break;
case XLOG_HEAP_CONFIRM:
id = "HEAP_CONFIRM";
break;
diff --git a/src/backend/commands/tablecmds.cindex 296807849f..e5f7cb320f 100644
*** a/src/backend/commands/tablecmds.c
--- b/src/backend/commands/tablecmds.c
***************
*** 16,21 ****
--- 16,22 ----
#include "access/genam.h"
#include "access/heapam.h"
+ #include "access/heapam_xlog.h"
#include "access/multixact.h"
#include "access/reloptions.h"
#include "access/relscan.h"
***************
*** 1249,1259 **** ExecuteTruncate(TruncateStmt *stmt)
{
List *rels = NIL;
List *relids = NIL;
! List *seq_relids = NIL;
! EState *estate;
! ResultRelInfo *resultRelInfos;
! ResultRelInfo *resultRelInfo;
! SubTransactionId mySubid;
ListCell *cell;
/*
--- 1250,1256 ----
{
List *rels = NIL;
List *relids = NIL;
! List *relids_logged = NIL;
ListCell *cell;
/*
***************
*** 1277,1282 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1274,1282 ----
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, myrelid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, myrelid);
if (recurse)
{
***************
*** 1297,1302 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1297,1305 ----
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, childrelid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, childrelid);
}
}
else if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
***************
*** 1306,1312 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1309,1337 ----
errhint("Do not specify the ONLY keyword, or use truncate only on the partitions directly.")));
}
+ ExecuteTruncateGuts(rels, relids, relids_logged,
+ stmt->behavior, stmt->restart_seqs);
+ }
+
+ void
+ ExecuteTruncateGuts(List *rels, List *relids, List *relids_logged,
+ DropBehavior behavior, bool restart_seqs)
+ {
+ List *seq_relids = NIL;
+ EState *estate;
+ ResultRelInfo *resultRelInfos;
+ ResultRelInfo *resultRelInfo;
+ SubTransactionId mySubid;
+ ListCell *cell;
+ List *seq_relids_logged = NIL;
+ uint32 nrelids = 0;
+ uint32 nseqrelids = 0;
+ uint32 maxrelids = 2;
+ Oid *logrelids = NULL;
+
/*
+ * Open, exclusive-lock, and check all the explicitly-specified relations
+ *
* In CASCADE mode, suck in all referencing relations as well. This
* requires multiple iterations to find indirectly-dependent relations. At
* each phase, we need to exclusive-lock new rels before looking for their
***************
*** 1314,1320 **** ExecuteTruncate(TruncateStmt *stmt)
* soon as we open it, to avoid a faux pas such as holding lock for a long
* time on a rel we have no permissions for.
*/
! if (stmt->behavior == DROP_CASCADE)
{
for (;;)
{
--- 1339,1345 ----
* soon as we open it, to avoid a faux pas such as holding lock for a long
* time on a rel we have no permissions for.
*/
! if (behavior == DROP_CASCADE)
{
for (;;)
{
***************
*** 1336,1341 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1361,1369 ----
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, relid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, relid);
}
}
}
***************
*** 1355,1361 **** ExecuteTruncate(TruncateStmt *stmt)
#ifdef USE_ASSERT_CHECKING
heap_truncate_check_FKs(rels, false);
#else
! if (stmt->behavior == DROP_RESTRICT)
heap_truncate_check_FKs(rels, false);
#endif
}
--- 1383,1389 ----
#ifdef USE_ASSERT_CHECKING
heap_truncate_check_FKs(rels, false);
#else
! if (behavior == DROP_RESTRICT)
heap_truncate_check_FKs(rels, false);
#endif
}
***************
*** 1366,1372 **** ExecuteTruncate(TruncateStmt *stmt)
* We want to do this early since it's pointless to do all the truncation
* work only to fail on sequence permissions.
*/
! if (stmt->restart_seqs)
{
foreach(cell, rels)
{
--- 1394,1400 ----
* We want to do this early since it's pointless to do all the truncation
* work only to fail on sequence permissions.
*/
! if (restart_seqs)
{
foreach(cell, rels)
{
***************
*** 1387,1392 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1415,1424 ----
RelationGetRelationName(seq_rel));
seq_relids = lappend_oid(seq_relids, seq_relid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(seq_rel))
+ seq_relids_logged = lappend_oid(seq_relids_logged,
+ seq_relid);
relation_close(seq_rel, NoLock);
}
***************
*** 1521,1526 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1553,1612 ----
ResetSequence(seq_relid);
}
+ /*
+ * Write a WAL record to allow this set of actions to be logically decoded.
+ * We could optimize this away when !RelationIsLogicallyLogged(rel)
+ * but that doesn't save much space or time.
+ *
+ * Assemble an array of relids, then an array of seqrelids so we can write
+ * a single WAL record for the whole action.
+ */
+ logrelids = palloc(maxrelids * sizeof(Oid));
+ foreach (cell, relids_logged)
+ {
+ nrelids++;
+ if (nrelids > maxrelids)
+ {
+ maxrelids *= 2;
+ logrelids = repalloc(logrelids, maxrelids * sizeof(Oid));
+ }
+ logrelids[nrelids - 1] = lfirst_oid(cell);
+ }
+
+ foreach (cell, seq_relids_logged)
+ {
+ nseqrelids++;
+ if ((nrelids + nseqrelids) > maxrelids)
+ {
+ maxrelids *= 2;
+ logrelids = repalloc(logrelids, maxrelids * sizeof(Oid));
+ }
+ logrelids[nrelids + nseqrelids - 1] = lfirst_oid(cell);
+ }
+
+ if ((nrelids + nseqrelids) > 0)
+ {
+ xl_heap_truncate xlrec;
+
+ xlrec.dbId = MyDatabaseId;
+ xlrec.nrelids = nrelids;
+ xlrec.nseqrelids = nseqrelids;
+ xlrec.flags = 0;
+ if (behavior == DROP_CASCADE)
+ xlrec.flags |= XLH_TRUNCATE_CASCADE;
+ if (restart_seqs)
+ xlrec.flags |= XLH_TRUNCATE_RESTART_SEQS;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfHeapTruncate);
+ XLogRegisterData((char *) logrelids,
+ (nrelids + nseqrelids) * sizeof(Oid));
+
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ (void) XLogInsert(RM_HEAP_ID, XLOG_HEAP_TRUNCATE);
+ }
+
/*
* Process all AFTER STATEMENT TRUNCATE triggers.
*/
diff --git a/src/backend/replication/loindex 486fd0c988..e06dcfe62d 100644
*** a/src/backend/replication/logical/decode.c
--- b/src/backend/replication/logical/decode.c
***************
*** 65,70 **** static void DecodeLogicalMsgOp(LogicalDecodingContext *ctx, XLogRecordBuffer *bu
--- 65,71 ----
static void DecodeInsert(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeUpdate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeDelete(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
+ static void DecodeTruncate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeMultiInsert(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeSpecConfirm(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
***************
*** 435,440 **** DecodeHeapOp(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
--- 436,446 ----
DecodeDelete(ctx, buf);
break;
+ case XLOG_HEAP_TRUNCATE:
+ if (SnapBuildProcessChange(builder, xid, buf->origptr))
+ DecodeTruncate(ctx, buf);
+ break;
+
case XLOG_HEAP_INPLACE:
/*
***************
*** 805,810 **** DecodeDelete(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
--- 811,856 ----
ReorderBufferQueueChange(ctx->reorder, XLogRecGetXid(r), buf->origptr, change);
}
+ /*
+ * Parse XLOG_HEAP_TRUNCATE from wal
+ */
+ static void
+ DecodeTruncate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+ {
+ XLogReaderState *r = buf->record;
+ xl_heap_truncate *xlrec;
+ ReorderBufferChange *change;
+ int i;
+
+ xlrec = (xl_heap_truncate *) XLogRecGetData(r);
+
+ /* only interested in our database */
+ if (xlrec->dbId != ctx->slot->data.database)
+ return;
+
+ /* output plugin doesn't look for this origin, no need to queue */
+ if (FilterByOrigin(ctx, XLogRecGetOrigin(r)))
+ return;
+
+ change = ReorderBufferGetChange(ctx->reorder);
+ change->action = REORDER_BUFFER_CHANGE_TRUNCATE;
+ change->origin_id = XLogRecGetOrigin(r);
+ if (xlrec->flags & XLH_TRUNCATE_CASCADE)
+ change->data.truncate_msg.cascade = true;
+ if (xlrec->flags & XLH_TRUNCATE_RESTART_SEQS)
+ change->data.truncate_msg.restart_seqs = true;
+
+ /*
+ * Queue up one change per relation, ignoring sequences for now
+ */
+ for (i = 0; i < xlrec->nrelids; i++)
+ {
+ change->data.truncate_msg.relid = xlrec->relids[i];
+ ReorderBufferQueueChange(ctx->reorder, XLogRecGetXid(r),
+ buf->origptr, change);
+ }
+ }
+
/*
* Decode XLOG_HEAP2_MULTI_INSERT_insert record into multiple tuplebufs.
*
diff --git a/src/backend/replication/logical/prindex 9b126b2957..8ffa486e8c 100644
*** a/src/backend/replication/logical/proto.c
--- b/src/backend/replication/logical/proto.c
***************
*** 26,31 ****
--- 26,34 ----
*/
#define LOGICALREP_IS_REPLICA_IDENTITY 1
+ #define TRUNCATE_CASCADE (1<<0)
+ #define TRUNCATE_RESTART_SEQS (1<<1)
+
static void logicalrep_write_attrs(StringInfo out, Relation rel);
static void logicalrep_write_tuple(StringInfo out, Relation rel,
HeapTuple tuple);
***************
*** 292,297 **** logicalrep_read_delete(StringInfo in, LogicalRepTupleData *oldtup)
--- 295,342 ----
return relid;
}
+ /*
+ * Write TRUNCATE to the output stream.
+ */
+ void
+ logicalrep_write_truncate(StringInfo out, Relation rel,
+ bool cascade, bool restart_seqs)
+ {
+ uint8 flags = 0;
+
+ pq_sendbyte(out, 'T'); /* action TRUNCATE */
+
+ /* use Oid as relation identifier */
+ pq_sendint32(out, RelationGetRelid(rel));
+
+ /* encode and send truncate flags */
+ if (cascade)
+ flags |= TRUNCATE_CASCADE;
+ if (restart_seqs)
+ flags |= TRUNCATE_RESTART_SEQS;
+ pq_sendint8(out, flags);
+ }
+
+ /*
+ * Read TRUNCATE from stream.
+ */
+ LogicalRepRelId
+ logicalrep_read_truncate(StringInfo in, bool *cascade, bool *restart_seqs)
+ {
+ LogicalRepRelId relid;
+ uint8 flags;
+
+ /* read the relation id */
+ relid = pq_getmsgint(in, 4);
+
+ /* read and decode truncate flags */
+ flags = pq_getmsgint(in, 1);
+ *cascade = (flags & TRUNCATE_CASCADE) > 0;
+ *restart_seqs = (flags & TRUNCATE_RESTART_SEQS) > 0;
+
+ return relid;
+ }
+
/*
* Write relation description to the output stream.
*/
diff --git a/src/backend/replication/logical/rindex 5ac391dbda..4eefa53681 100644
*** a/src/backend/replication/logical/reorderbuffer.c
--- b/src/backend/replication/logical/reorderbuffer.c
***************
*** 403,408 **** ReorderBufferReturnChange(ReorderBuffer *rb, ReorderBufferChange *change)
--- 403,410 ----
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ break;
}
pfree(change);
***************
*** 1342,1347 **** ReorderBufferCommit(ReorderBuffer *rb, TransactionId xid,
--- 1344,1356 ----
switch (change->action)
{
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ reloid = change->data.truncate_msg.relid;
+ relation = RelationIdGetRelation(reloid);
+ rb->apply_change(rb, txn, relation, change);
+ RelationClose(relation);
+ break;
+
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
/*
***************
*** 2224,2229 **** ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
--- 2233,2239 ----
}
break;
}
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
***************
*** 2509,2514 **** ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
--- 2519,2525 ----
break;
}
/* the base struct contains all the data, easy peasy */
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
diff --git a/src/backend/replication/logical/worker.c index fa5d9bb120..17e5b4b150 100644
*** a/src/backend/replication/logical/worker.c
--- b/src/backend/replication/logical/worker.c
***************
*** 869,874 **** apply_handle_delete(StringInfo s)
--- 869,922 ----
CommandCounterIncrement();
}
+ /*
+ * Handle TRUNCATE message.
+ *
+ * TODO: FDW support
+ */
+ static void
+ apply_handle_truncate(StringInfo s)
+ {
+ LogicalRepRelMapEntry *rel;
+ LogicalRepRelId relid;
+ bool cascade = false;
+ bool restart_seqs = false;
+ List *rels = NIL;
+ List *relids = NIL;
+
+ ensure_transaction();
+
+ relid = logicalrep_read_truncate(s, &cascade, &restart_seqs);
+ rel = logicalrep_rel_open(relid, RowExclusiveLock);
+ if (!should_apply_changes_for_rel(rel))
+ {
+ /*
+ * The relation can't become interesting in the middle of the
+ * transaction so it's safe to unlock it.
+ */
+ logicalrep_rel_close(rel, RowExclusiveLock);
+ return;
+ }
+
+ /* Check if we can do the truncate. */
+ check_relation_updatable(rel);
+
+ rels = lappend(rels, rel->localrel);
+ relids = lappend_oid(relids, rel->localreloid);
+
+ ExecuteTruncateGuts(rels, relids, NULL,
+ cascade ? DROP_CASCADE : DROP_RESTRICT,
+ restart_seqs);
+
+ /* logicalrep_rel_close call not needed, because ExecuteTruncateGuts
+ * already closes the relations. Setting localrel to NULL in the map entry
+ * is still needed.
+ */
+ rel->localrel = NULL;
+
+ CommandCounterIncrement();
+ }
+
/*
* Logical replication protocol message dispatcher.
***************
*** 900,905 **** apply_dispatch(StringInfo s)
--- 948,957 ----
case 'D':
apply_handle_delete(s);
break;
+ /* TRUNCATE */
+ case 'T':
+ apply_handle_truncate(s);
+ break;
/* RELATION */
case 'R':
apply_handle_relation(s);
diff --git a/src/backend/replication/pgoutput/pindex 550b156e2d..5bd5af98c5 100644
*** a/src/backend/replication/pgoutput/pgoutput.c
--- b/src/backend/replication/pgoutput/pgoutput.c
***************
*** 275,280 **** pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
--- 275,281 ----
if (!relentry->pubactions.pubupdate)
return;
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
case REORDER_BUFFER_CHANGE_DELETE:
if (!relentry->pubactions.pubdelete)
return;
***************
*** 352,357 **** pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
--- 353,367 ----
else
elog(DEBUG1, "didn't send DELETE change because of missing oldtuple");
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ {
+ OutputPluginPrepareWrite(ctx, true);
+ logicalrep_write_truncate(ctx->out, relation,
+ change->data.truncate_msg.cascade,
+ change->data.truncate_msg.restart_seqs);
+ OutputPluginWrite(ctx, true);
+ }
+ break;
default:
Assert(false);
}
diff --git a/src/include/access/heapam_xlog.h b/srindex 38f7f63984..88b5791e7c 100644
*** a/src/include/access/heapam_xlog.h
--- b/src/include/access/heapam_xlog.h
***************
*** 32,38 ****
#define XLOG_HEAP_INSERT 0x00
#define XLOG_HEAP_DELETE 0x10
#define XLOG_HEAP_UPDATE 0x20
! /* 0x030 is free, was XLOG_HEAP_MOVE */
#define XLOG_HEAP_HOT_UPDATE 0x40
#define XLOG_HEAP_CONFIRM 0x50
#define XLOG_HEAP_LOCK 0x60
--- 32,38 ----
#define XLOG_HEAP_INSERT 0x00
#define XLOG_HEAP_DELETE 0x10
#define XLOG_HEAP_UPDATE 0x20
! #define XLOG_HEAP_TRUNCATE 0x30
#define XLOG_HEAP_HOT_UPDATE 0x40
#define XLOG_HEAP_CONFIRM 0x50
#define XLOG_HEAP_LOCK 0x60
***************
*** 109,114 **** typedef struct xl_heap_delete
--- 109,136 ----
#define SizeOfHeapDelete (offsetof(xl_heap_delete, flags) + sizeof(uint8))
+ /*
+ * xl_heap_delete flag values, 8 bits are available.
+ */
+ #define XLH_TRUNCATE_CASCADE (1<<0)
+ #define XLH_TRUNCATE_RESTART_SEQS (1<<1)
+
+ /*
+ * For truncate we list all truncated relids in an array, followed by all
+ * sequence relids that need to be restarted, if any.
+ * All rels are always within the same database, so we just list dbid once.
+ */
+ typedef struct xl_heap_truncate
+ {
+ Oid dbId;
+ uint32 nrelids;
+ uint32 nseqrelids;
+ uint8 flags;
+ Oid relids[FLEXIBLE_ARRAY_MEMBER];
+ } xl_heap_truncate;
+
+ #define SizeOfHeapTruncate (offsetof(xl_heap_truncate, relids))
+
/*
* We don't store the whole fixed part (HeapTupleHeaderData) of an inserted
* or updated tuple in WAL; we can save a few bytes by reconstructing the
diff --git a/src/include/executor/execuindex 2cc74da0ba..027def882e 100644
*** a/src/include/executor/executor.h
--- b/src/include/executor/executor.h
***************
*** 553,556 **** extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
--- 553,562 ----
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
+ /*
+ * prototype for tablecmds.c accessible for logical apply
+ */
+ void ExecuteTruncateGuts(List *rels, List *relids, List *relids_logged,
+ DropBehavior behavior, bool restart_seqs);
+
#endif /* EXECUTOR_H */
diff --git a/src/include/replication/lindex a9736e1bf6..bd5e579980 100644
*** a/src/include/replication/logicalproto.h
--- b/src/include/replication/logicalproto.h
***************
*** 98,103 **** extern void logicalrep_write_delete(StringInfo out, Relation rel,
--- 98,107 ----
HeapTuple oldtuple);
extern LogicalRepRelId logicalrep_read_delete(StringInfo in,
LogicalRepTupleData *oldtup);
+ extern void logicalrep_write_truncate(StringInfo out, Relation rel,
+ bool cascade, bool restart_seqs);
+ extern LogicalRepRelId logicalrep_read_truncate(StringInfo in,
+ bool *cascade, bool *restart_seqs);
extern void logicalrep_write_rel(StringInfo out, Relation rel);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
extern void logicalrep_write_typ(StringInfo out, Oid typoid);
diff --git a/src/include/replication/reorderbindex b18ce5a9df..5a29e1ab5b 100644
*** a/src/include/replication/reorderbuffer.h
--- b/src/include/replication/reorderbuffer.h
***************
*** 59,65 **** enum ReorderBufferChangeType
REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID,
REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID,
REORDER_BUFFER_CHANGE_INTERNAL_SPEC_INSERT,
! REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM
};
/*
--- 59,66 ----
REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID,
REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID,
REORDER_BUFFER_CHANGE_INTERNAL_SPEC_INSERT,
! REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM,
! REORDER_BUFFER_CHANGE_TRUNCATE
};
/*
***************
*** 128,134 **** typedef struct ReorderBufferChange
CommandId cmax;
CommandId combocid;
} tuplecid;
! } data;
/*
* While in use this is how a change is linked into a transactions,
--- 129,146 ----
CommandId cmax;
CommandId combocid;
} tuplecid;
!
! /*
! * Truncate data for REORDER_BUFFER_CHANGE_TRUNCATE representing
! * one relation to be truncated.
! */
! struct
! {
! Oid relid;
! bool cascade;
! bool restart_seqs;
! } truncate_msg;
! } data;
/*
* While in use this is how a change is linked into a transactions,
diff --git a/src/test/subscription/t/001_rep_cindex 0136c79d4b..974e88821c 100644
*** a/src/test/subscription/t/001_rep_changes.pl
--- b/src/test/subscription/t/001_rep_changes.pl
***************
*** 3,9 **** use strict;
use warnings;
use PostgresNode;
use TestLib;
! use Test::More tests => 16;
# Initialize publisher node
my $node_publisher = get_new_node('publisher');
--- 3,9 ----
use warnings;
use PostgresNode;
use TestLib;
! use Test::More tests => 24;
# Initialize publisher node
my $node_publisher = get_new_node('publisher');
***************
*** 189,199 **** $result = $node_subscriber->safe_psql('postgres',
is($result, qq(20|-20|-1),
'check changes skipped after subscription publication change');
# check alter publication (relcache invalidation etc)
$node_publisher->safe_psql('postgres',
"ALTER PUBLICATION tap_pub_ins_only SET (publish = 'insert, delete')");
$node_publisher->safe_psql('postgres',
! "ALTER PUBLICATION tap_pub_ins_only ADD TABLE tab_full");
$node_publisher->safe_psql('postgres', "DELETE FROM tab_ins WHERE a > 0");
$node_subscriber->safe_psql('postgres',
"ALTER SUBSCRIPTION tap_sub REFRESH PUBLICATION WITH (copy_data = false)"
--- 189,210 ----
is($result, qq(20|-20|-1),
'check changes skipped after subscription publication change');
+ # truncate should not be replicated
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep");
+
+ $node_publisher->poll_query_until('postgres', $caughtup_query)
+ or die "Timed out while waiting for subscriber to catch up";
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_rep");
+ is($result, qq(20|-20|-1),
+ 'check changes skipped after subscription publication change');
+
# check alter publication (relcache invalidation etc)
$node_publisher->safe_psql('postgres',
"ALTER PUBLICATION tap_pub_ins_only SET (publish = 'insert, delete')");
$node_publisher->safe_psql('postgres',
! "ALTER PUBLICATION tap_pub_ins_only ADD TABLE tab_full, tab_rep");
$node_publisher->safe_psql('postgres', "DELETE FROM tab_ins WHERE a > 0");
$node_subscriber->safe_psql('postgres',
"ALTER SUBSCRIPTION tap_sub REFRESH PUBLICATION WITH (copy_data = false)"
***************
*** 213,218 **** $result = $node_subscriber->safe_psql('postgres',
--- 224,295 ----
"SELECT count(*), min(a), max(a) FROM tab_full");
is($result, qq(21|0|100), 'check replicated insert after alter publication');
+ # add a sequence and a foreign key to check cascade and restart
+ # identity truncate options
+ $node_subscriber->safe_psql('postgres',
+ "CREATE TABLE tab_notrep_fk (a int REFERENCES tab_rep(a))"
+ );
+ $node_subscriber->safe_psql('postgres', "INSERT INTO tab_notrep_fk VALUES (-1)");
+ $node_subscriber->safe_psql('postgres',
+ "CREATE SEQUENCE seq_notrep OWNED BY tab_rep.a"
+ );
+ $node_subscriber->safe_psql('postgres',
+ "ALTER SEQUENCE seq_notrep START 101"
+ );
+
+ # truncate should now be replicated
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep");
+
+ $node_publisher->poll_query_until('postgres', $caughtup_query)
+ or die "Timed out while waiting for subscriber to catch up";
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_rep");
+ is($result, qq(0||),
+ 'check replicated truncate after alter publication');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_notrep_fk");
+ is($result, qq(1|-1|-1),
+ 'check replicated truncate does not cascade');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq_notrep')");
+ is($result, qq(1),
+ 'check replicated truncate does not restart identities');
+
+ # should restart identity
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep RESTART IDENTITY");
+
+ $node_publisher->poll_query_until('postgres', $caughtup_query)
+ or die "Timed out while waiting for subscriber to catch up";
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_notrep_fk");
+ is($result, qq(1|-1|-1),
+ 'check replicated truncate does not cascade');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq_notrep')");
+ is($result, qq(101),
+ 'check replicated truncate restart identities');
+
+ # should cascade
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep CASCADE");
+
+ $node_publisher->poll_query_until('postgres', $caughtup_query)
+ or die "Timed out while waiting for subscriber to catch up";
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_notrep_fk");
+ is($result, qq(0||),
+ 'check replicated truncate cascade');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq_notrep')");
+ is($result, qq(102),
+ 'check replicated truncate does not restart identities');
+
# check restart on rename
$oldpid = $node_publisher->safe_psql('postgres',
"SELECT pid FROM pg_stat_replication WHERE application_name = '$appname';"
Re: [PATCH] Logical decoding of TRUNCATE
Hi,
On 2017-12-29 14:15:22 +0100, Marco Nenciarini wrote:
This patch implements support for TRUNCATE statements
in logical replication. The work has mainly done by Simon Riggs then
finished by me. Tests are written by me.TRUNCATE is treated as a form of DELETE for the purpose of deciding
whether to publish, or not.
It'd be good if you explained exactly what the chosen behaviour is, and
why that's the right behaviour in comparison to other alternatives.
- Andres
Re: [PATCH] Logical decoding of TRUNCATE
Hi,
Il 29/12/17 20:55, Andres Freund ha scritto:
Hi,
On 2017-12-29 14:15:22 +0100, Marco Nenciarini wrote:
This patch implements support for TRUNCATE statements
in logical replication. The work has mainly done by Simon Riggs then
finished by me. Tests are written by me.TRUNCATE is treated as a form of DELETE for the purpose of deciding
whether to publish, or not.It'd be good if you explained exactly what the chosen behaviour is, and
why that's the right behaviour in comparison to other alternatives.
here is the description of how the patch works:
* A new WAL record type has been added (XLOG_HEAP_TRUNCATE) to allow a
precise logical decoding of the TRUNCATE operation. It contains the
TRUNCATE flags and the list of the involved tables and sequences. It is
treated as a NO-OP during the WAL reply as all the physical operations
are logged in SMGR WAL records.
* A new TRUNCATE message has been added to the logical protocol. It
carries the information about the truncation of one single table
specifying if CASCADE or RESTART IDENTITY has been specified.
* The ExecuteTruncateGuts function has been extracted from
ExecuteTruncate. This function is used by the logical apply process to
replicate the TRUNCATE operations, honouring the eventual CASCADE and
RESTART IDENTITY flag.
Regards,
Marco
--
Marco Nenciarini - 2ndQuadrant Italy
PostgreSQL Training, Services and Support
marco.nenciarini@2ndQuadrant.it | www.2ndQuadrant.it
Re: [PATCH] Logical decoding of TRUNCATE
On 29 December 2017 at 19:55, Andres Freund <andres@anarazel.de> wrote:
Hi,
On 2017-12-29 14:15:22 +0100, Marco Nenciarini wrote:
This patch implements support for TRUNCATE statements
in logical replication. The work has mainly done by Simon Riggs then
finished by me. Tests are written by me.TRUNCATE is treated as a form of DELETE for the purpose of deciding
whether to publish, or not.It'd be good if you explained exactly what the chosen behaviour is, and
why that's the right behaviour in comparison to other alternatives.
At present the patch treats TRUNCATE as if it were a DELETE
which means that
CREATE PUBLICATION insert_only FOR TABLE mydata WITH (publish = 'insert');
will not publish truncates before or after this patch
CREATE PUBLICATION insert_only FOR TABLE mydata;
will now publish TRUNCATEs, although they were ignored in PG10
so PG10 publications will act differently
I had regarded it as a missing piece, but some may view that is a
behaviour change in PG11
Alternatively, we could also use WITH (publish = 'truncate') as a
separate decision.
That is an easy change if we wish it.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [PATCH] Logical decoding of TRUNCATE
Hi,
I've found some SGML errors in the version v10 of the patch. I've fixed
it in version v11 that is attached.
Regards,
Marco
--
Marco Nenciarini - 2ndQuadrant Italy
PostgreSQL Training, Services and Support
marco.nenciarini@2ndQuadrant.it | www.2ndQuadrant.it
Attachments:
logical_decode_truncate.v11.patchtext/plain; charset=UTF-8; name=logical_decode_truncate.v11.patch; x-mac-creator=0; x-mac-type=0Download
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 1e22c1eefc..d1feea4909 100644
*** a/contrib/test_decoding/expected/ddl.out
--- b/contrib/test_decoding/expected/ddl.out
***************
*** 543,548 **** UPDATE table_with_unique_not_null SET data = 3 WHERE data = 2;
--- 543,550 ----
UPDATE table_with_unique_not_null SET id = -id;
UPDATE table_with_unique_not_null SET id = -id;
DELETE FROM table_with_unique_not_null WHERE data = 3;
+ TRUNCATE table_with_unique_not_null;
+ TRUNCATE table_with_unique_not_null, table_with_unique_not_null RESTART IDENTITY CASCADE;
-- check toast support
BEGIN;
CREATE SEQUENCE toasttable_rand_seq START 79 INCREMENT 1499; -- portable "random"
***************
*** 660,665 **** SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
--- 662,673 ----
table public.table_with_unique_not_null: DELETE: id[integer]:4
COMMIT
BEGIN
+ table public.table_with_unique_not_null: TRUNCATE: (no-flags)
+ COMMIT
+ BEGIN
+ table public.table_with_unique_not_null: TRUNCATE: restart_seqs cascade
+ COMMIT
+ BEGIN
table public.toasttable: INSERT: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
BEGIN
***************
*** 668,674 **** SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
BEGIN
table public.toasttable: UPDATE: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
! (103 rows)
INSERT INTO toasttable(toasted_col1) SELECT string_agg(g.i::text, '') FROM generate_series(1, 2000) g(i);
-- update of second column, first column unchanged
--- 676,682 ----
BEGIN
table public.toasttable: UPDATE: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
! (109 rows)
INSERT INTO toasttable(toasted_col1) SELECT string_agg(g.i::text, '') FROM generate_series(1, 2000) g(i);
-- update of second column, first column unchanged
diff --git a/contrib/test_decoding/sql/ddl.sqindex 057dae056b..0fb4102db8 100644
*** a/contrib/test_decoding/sql/ddl.sql
--- b/contrib/test_decoding/sql/ddl.sql
***************
*** 333,338 **** UPDATE table_with_unique_not_null SET id = -id;
--- 333,341 ----
UPDATE table_with_unique_not_null SET id = -id;
DELETE FROM table_with_unique_not_null WHERE data = 3;
+ TRUNCATE table_with_unique_not_null;
+ TRUNCATE table_with_unique_not_null, table_with_unique_not_null RESTART IDENTITY CASCADE;
+
-- check toast support
BEGIN;
CREATE SEQUENCE toasttable_rand_seq START 79 INCREMENT 1499; -- portable "random"
diff --git a/contrib/test_decoding/test_index 0f18afa852..55c4593ed5 100644
*** a/contrib/test_decoding/test_decoding.c
--- b/contrib/test_decoding/test_decoding.c
***************
*** 466,471 **** pg_decode_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
--- 466,485 ----
&change->data.tp.oldtuple->tuple,
true);
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ appendStringInfoString(ctx->out, " TRUNCATE:");
+
+ if (change->data.truncate_msg.restart_seqs
+ || change->data.truncate_msg.cascade)
+ {
+ if (change->data.truncate_msg.restart_seqs)
+ appendStringInfo(ctx->out, " restart_seqs");
+ if (change->data.truncate_msg.cascade)
+ appendStringInfo(ctx->out, " cascade");
+ }
+ else
+ appendStringInfoString(ctx->out, " (no-flags)");
+ break;
default:
Assert(false);
}
diff --git a/doc/src/sgml/logical-replicatioindex 75551d8ee1..6b64c8ef76 100644
*** a/doc/src/sgml/logical-replication.sgml
--- b/doc/src/sgml/logical-replication.sgml
***************
*** 111,116 ****
--- 111,118 ----
any combination of <command>INSERT</command>, <command>UPDATE</command>, and
<command>DELETE</command>, similar to how triggers are fired by
particular event types. By default, all operation types are replicated.
+ <command>TRUNCATE</command> is treated as a form of <command>DELETE</command>
+ for the purpose of deciding whether to publish, or not.
</para>
<para>
***************
*** 364,378 ****
</para>
</listitem>
- <listitem>
- <para>
- <command>TRUNCATE</command> commands are not replicated. This can, of
- course, be worked around by using <command>DELETE</command> instead. To
- avoid accidental <command>TRUNCATE</command> invocations, you can revoke
- the <literal>TRUNCATE</literal> privilege from tables.
- </para>
- </listitem>
-
<listitem>
<para>
Large objects (see <xref linkend="largeobjects"/>) are not replicated.
--- 366,371 ----
diff --git a/doc/src/sgml/protocol.sgml b/doindex 4c5ed1e6d6..d2a0d7e52e 100644
*** a/doc/src/sgml/protocol.sgml
--- b/doc/src/sgml/protocol.sgml
***************
*** 6838,6843 **** TupleData
--- 6838,6889 ----
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ Truncate
+ </term>
+ <listitem>
+ <para>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ Byte1('T')
+ </term>
+ <listitem>
+ <para>
+ Identifies the message as a truncate message.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ Int32
+ </term>
+ <listitem>
+ <para>
+ ID of the relation corresponding to the ID in the relation
+ message.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ Int8
+ </term>
+ <listitem>
+ <para>
+ Option flags for truncate. Currently can be 0 for no flags,
+ 1 for CASCADE, 2 for RESTART IDENTITY and 3 if both are set.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect1>
diff --git a/doc/src/sgml/ref/creindex bfe12d5f41..c5032938a7 100644
*** a/doc/src/sgml/ref/create_publication.sgml
--- b/doc/src/sgml/ref/create_publication.sgml
***************
*** 111,116 **** CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
--- 111,121 ----
and so the default value for this option is
<literal>'insert, update, delete'</literal>.
</para>
+ <para>
+ <command>TRUNCATE</command> is treated as a form of
+ <command>DELETE</command> for the purpose of deciding whether
+ to publish, or not.
+ </para>
</listitem>
</varlistentry>
</variablelist>
***************
*** 168,175 **** CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
</para>
<para>
! <command>TRUNCATE</command> and <acronym>DDL</acronym> operations
! are not published.
</para>
</refsect1>
--- 173,179 ----
</para>
<para>
! <acronym>DDL</acronym> operations are not published.
</para>
</refsect1>
diff --git a/src/backend/access/heap/heapam.c bindex dbc8f2d6c7..f4c6eaf57f 100644
*** a/src/backend/access/heap/heapam.c
--- b/src/backend/access/heap/heapam.c
***************
*** 9134,9139 **** heap_redo(XLogReaderState *record)
--- 9134,9146 ----
case XLOG_HEAP_UPDATE:
heap_xlog_update(record, false);
break;
+ case XLOG_HEAP_TRUNCATE:
+ /*
+ * TRUNCATE is a no-op because the actions are already logged
+ * as SMGR WAL records. TRUNCATE WAL record only exists to allow
+ * it to be logically decoded precisely.
+ */
+ break;
case XLOG_HEAP_HOT_UPDATE:
heap_xlog_update(record, true);
break;
diff --git a/src/backend/access/rmgrdesindex b00c071cb6..15550cdff0 100644
*** a/src/backend/access/rmgrdesc/heapdesc.c
--- b/src/backend/access/rmgrdesc/heapdesc.c
***************
*** 75,80 **** heap_desc(StringInfo buf, XLogReaderState *record)
--- 75,93 ----
xlrec->new_offnum,
xlrec->new_xmax);
}
+ else if (info == XLOG_HEAP_TRUNCATE)
+ {
+ xl_heap_truncate *xlrec = (xl_heap_truncate *) rec;
+
+ if (xlrec->flags & XLH_TRUNCATE_CASCADE)
+ appendStringInfo(buf, "cascade ");
+ if (xlrec->flags & XLH_TRUNCATE_RESTART_SEQS)
+ appendStringInfo(buf, "restart_seqs ");
+ appendStringInfo(buf, "nrelids %u nseqrelids %u",
+ xlrec->nrelids,
+ xlrec->nseqrelids);
+ /* Skip the list of relids and seqrelids */
+ }
else if (info == XLOG_HEAP_CONFIRM)
{
xl_heap_confirm *xlrec = (xl_heap_confirm *) rec;
***************
*** 186,191 **** heap_identify(uint8 info)
--- 199,207 ----
case XLOG_HEAP_HOT_UPDATE | XLOG_HEAP_INIT_PAGE:
id = "HOT_UPDATE+INIT";
break;
+ case XLOG_HEAP_TRUNCATE:
+ id = "TRUNCATE";
+ break;
case XLOG_HEAP_CONFIRM:
id = "HEAP_CONFIRM";
break;
diff --git a/src/backend/commands/tablecmds.cindex 180ebd0717..8355c4c5c7 100644
*** a/src/backend/commands/tablecmds.c
--- b/src/backend/commands/tablecmds.c
***************
*** 16,21 ****
--- 16,22 ----
#include "access/genam.h"
#include "access/heapam.h"
+ #include "access/heapam_xlog.h"
#include "access/multixact.h"
#include "access/reloptions.h"
#include "access/relscan.h"
***************
*** 1248,1258 **** ExecuteTruncate(TruncateStmt *stmt)
{
List *rels = NIL;
List *relids = NIL;
! List *seq_relids = NIL;
! EState *estate;
! ResultRelInfo *resultRelInfos;
! ResultRelInfo *resultRelInfo;
! SubTransactionId mySubid;
ListCell *cell;
/*
--- 1249,1255 ----
{
List *rels = NIL;
List *relids = NIL;
! List *relids_logged = NIL;
ListCell *cell;
/*
***************
*** 1276,1281 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1273,1281 ----
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, myrelid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, myrelid);
if (recurse)
{
***************
*** 1296,1301 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1296,1304 ----
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, childrelid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, childrelid);
}
}
else if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
***************
*** 1305,1311 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1308,1336 ----
errhint("Do not specify the ONLY keyword, or use truncate only on the partitions directly.")));
}
+ ExecuteTruncateGuts(rels, relids, relids_logged,
+ stmt->behavior, stmt->restart_seqs);
+ }
+
+ void
+ ExecuteTruncateGuts(List *rels, List *relids, List *relids_logged,
+ DropBehavior behavior, bool restart_seqs)
+ {
+ List *seq_relids = NIL;
+ EState *estate;
+ ResultRelInfo *resultRelInfos;
+ ResultRelInfo *resultRelInfo;
+ SubTransactionId mySubid;
+ ListCell *cell;
+ List *seq_relids_logged = NIL;
+ uint32 nrelids = 0;
+ uint32 nseqrelids = 0;
+ uint32 maxrelids = 2;
+ Oid *logrelids = NULL;
+
/*
+ * Open, exclusive-lock, and check all the explicitly-specified relations
+ *
* In CASCADE mode, suck in all referencing relations as well. This
* requires multiple iterations to find indirectly-dependent relations. At
* each phase, we need to exclusive-lock new rels before looking for their
***************
*** 1313,1319 **** ExecuteTruncate(TruncateStmt *stmt)
* soon as we open it, to avoid a faux pas such as holding lock for a long
* time on a rel we have no permissions for.
*/
! if (stmt->behavior == DROP_CASCADE)
{
for (;;)
{
--- 1338,1344 ----
* soon as we open it, to avoid a faux pas such as holding lock for a long
* time on a rel we have no permissions for.
*/
! if (behavior == DROP_CASCADE)
{
for (;;)
{
***************
*** 1335,1340 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1360,1368 ----
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, relid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, relid);
}
}
}
***************
*** 1354,1360 **** ExecuteTruncate(TruncateStmt *stmt)
#ifdef USE_ASSERT_CHECKING
heap_truncate_check_FKs(rels, false);
#else
! if (stmt->behavior == DROP_RESTRICT)
heap_truncate_check_FKs(rels, false);
#endif
}
--- 1382,1388 ----
#ifdef USE_ASSERT_CHECKING
heap_truncate_check_FKs(rels, false);
#else
! if (behavior == DROP_RESTRICT)
heap_truncate_check_FKs(rels, false);
#endif
}
***************
*** 1365,1371 **** ExecuteTruncate(TruncateStmt *stmt)
* We want to do this early since it's pointless to do all the truncation
* work only to fail on sequence permissions.
*/
! if (stmt->restart_seqs)
{
foreach(cell, rels)
{
--- 1393,1399 ----
* We want to do this early since it's pointless to do all the truncation
* work only to fail on sequence permissions.
*/
! if (restart_seqs)
{
foreach(cell, rels)
{
***************
*** 1386,1391 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1414,1423 ----
RelationGetRelationName(seq_rel));
seq_relids = lappend_oid(seq_relids, seq_relid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(seq_rel))
+ seq_relids_logged = lappend_oid(seq_relids_logged,
+ seq_relid);
relation_close(seq_rel, NoLock);
}
***************
*** 1520,1525 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1552,1611 ----
ResetSequence(seq_relid);
}
+ /*
+ * Write a WAL record to allow this set of actions to be logically decoded.
+ * We could optimize this away when !RelationIsLogicallyLogged(rel)
+ * but that doesn't save much space or time.
+ *
+ * Assemble an array of relids, then an array of seqrelids so we can write
+ * a single WAL record for the whole action.
+ */
+ logrelids = palloc(maxrelids * sizeof(Oid));
+ foreach (cell, relids_logged)
+ {
+ nrelids++;
+ if (nrelids > maxrelids)
+ {
+ maxrelids *= 2;
+ logrelids = repalloc(logrelids, maxrelids * sizeof(Oid));
+ }
+ logrelids[nrelids - 1] = lfirst_oid(cell);
+ }
+
+ foreach (cell, seq_relids_logged)
+ {
+ nseqrelids++;
+ if ((nrelids + nseqrelids) > maxrelids)
+ {
+ maxrelids *= 2;
+ logrelids = repalloc(logrelids, maxrelids * sizeof(Oid));
+ }
+ logrelids[nrelids + nseqrelids - 1] = lfirst_oid(cell);
+ }
+
+ if ((nrelids + nseqrelids) > 0)
+ {
+ xl_heap_truncate xlrec;
+
+ xlrec.dbId = MyDatabaseId;
+ xlrec.nrelids = nrelids;
+ xlrec.nseqrelids = nseqrelids;
+ xlrec.flags = 0;
+ if (behavior == DROP_CASCADE)
+ xlrec.flags |= XLH_TRUNCATE_CASCADE;
+ if (restart_seqs)
+ xlrec.flags |= XLH_TRUNCATE_RESTART_SEQS;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfHeapTruncate);
+ XLogRegisterData((char *) logrelids,
+ (nrelids + nseqrelids) * sizeof(Oid));
+
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ (void) XLogInsert(RM_HEAP_ID, XLOG_HEAP_TRUNCATE);
+ }
+
/*
* Process all AFTER STATEMENT TRUNCATE triggers.
*/
diff --git a/src/backend/replication/loindex 537eba7875..d8408e154c 100644
*** a/src/backend/replication/logical/decode.c
--- b/src/backend/replication/logical/decode.c
***************
*** 65,70 **** static void DecodeLogicalMsgOp(LogicalDecodingContext *ctx, XLogRecordBuffer *bu
--- 65,71 ----
static void DecodeInsert(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeUpdate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeDelete(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
+ static void DecodeTruncate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeMultiInsert(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeSpecConfirm(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
***************
*** 435,440 **** DecodeHeapOp(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
--- 436,446 ----
DecodeDelete(ctx, buf);
break;
+ case XLOG_HEAP_TRUNCATE:
+ if (SnapBuildProcessChange(builder, xid, buf->origptr))
+ DecodeTruncate(ctx, buf);
+ break;
+
case XLOG_HEAP_INPLACE:
/*
***************
*** 805,810 **** DecodeDelete(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
--- 811,856 ----
ReorderBufferQueueChange(ctx->reorder, XLogRecGetXid(r), buf->origptr, change);
}
+ /*
+ * Parse XLOG_HEAP_TRUNCATE from wal
+ */
+ static void
+ DecodeTruncate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+ {
+ XLogReaderState *r = buf->record;
+ xl_heap_truncate *xlrec;
+ ReorderBufferChange *change;
+ int i;
+
+ xlrec = (xl_heap_truncate *) XLogRecGetData(r);
+
+ /* only interested in our database */
+ if (xlrec->dbId != ctx->slot->data.database)
+ return;
+
+ /* output plugin doesn't look for this origin, no need to queue */
+ if (FilterByOrigin(ctx, XLogRecGetOrigin(r)))
+ return;
+
+ change = ReorderBufferGetChange(ctx->reorder);
+ change->action = REORDER_BUFFER_CHANGE_TRUNCATE;
+ change->origin_id = XLogRecGetOrigin(r);
+ if (xlrec->flags & XLH_TRUNCATE_CASCADE)
+ change->data.truncate_msg.cascade = true;
+ if (xlrec->flags & XLH_TRUNCATE_RESTART_SEQS)
+ change->data.truncate_msg.restart_seqs = true;
+
+ /*
+ * Queue up one change per relation, ignoring sequences for now
+ */
+ for (i = 0; i < xlrec->nrelids; i++)
+ {
+ change->data.truncate_msg.relid = xlrec->relids[i];
+ ReorderBufferQueueChange(ctx->reorder, XLogRecGetXid(r),
+ buf->origptr, change);
+ }
+ }
+
/*
* Decode XLOG_HEAP2_MULTI_INSERT_insert record into multiple tuplebufs.
*
diff --git a/src/backend/replication/logical/prindex 948343e4ae..2fa6f8393d 100644
*** a/src/backend/replication/logical/proto.c
--- b/src/backend/replication/logical/proto.c
***************
*** 26,31 ****
--- 26,34 ----
*/
#define LOGICALREP_IS_REPLICA_IDENTITY 1
+ #define TRUNCATE_CASCADE (1<<0)
+ #define TRUNCATE_RESTART_SEQS (1<<1)
+
static void logicalrep_write_attrs(StringInfo out, Relation rel);
static void logicalrep_write_tuple(StringInfo out, Relation rel,
HeapTuple tuple);
***************
*** 292,297 **** logicalrep_read_delete(StringInfo in, LogicalRepTupleData *oldtup)
--- 295,342 ----
return relid;
}
+ /*
+ * Write TRUNCATE to the output stream.
+ */
+ void
+ logicalrep_write_truncate(StringInfo out, Relation rel,
+ bool cascade, bool restart_seqs)
+ {
+ uint8 flags = 0;
+
+ pq_sendbyte(out, 'T'); /* action TRUNCATE */
+
+ /* use Oid as relation identifier */
+ pq_sendint32(out, RelationGetRelid(rel));
+
+ /* encode and send truncate flags */
+ if (cascade)
+ flags |= TRUNCATE_CASCADE;
+ if (restart_seqs)
+ flags |= TRUNCATE_RESTART_SEQS;
+ pq_sendint8(out, flags);
+ }
+
+ /*
+ * Read TRUNCATE from stream.
+ */
+ LogicalRepRelId
+ logicalrep_read_truncate(StringInfo in, bool *cascade, bool *restart_seqs)
+ {
+ LogicalRepRelId relid;
+ uint8 flags;
+
+ /* read the relation id */
+ relid = pq_getmsgint(in, 4);
+
+ /* read and decode truncate flags */
+ flags = pq_getmsgint(in, 1);
+ *cascade = (flags & TRUNCATE_CASCADE) > 0;
+ *restart_seqs = (flags & TRUNCATE_RESTART_SEQS) > 0;
+
+ return relid;
+ }
+
/*
* Write relation description to the output stream.
*/
diff --git a/src/backend/replication/logical/rindex 1208da2972..fc7350358a 100644
*** a/src/backend/replication/logical/reorderbuffer.c
--- b/src/backend/replication/logical/reorderbuffer.c
***************
*** 403,408 **** ReorderBufferReturnChange(ReorderBuffer *rb, ReorderBufferChange *change)
--- 403,410 ----
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ break;
}
pfree(change);
***************
*** 1342,1347 **** ReorderBufferCommit(ReorderBuffer *rb, TransactionId xid,
--- 1344,1356 ----
switch (change->action)
{
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ reloid = change->data.truncate_msg.relid;
+ relation = RelationIdGetRelation(reloid);
+ rb->apply_change(rb, txn, relation, change);
+ RelationClose(relation);
+ break;
+
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
/*
***************
*** 2239,2244 **** ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
--- 2248,2254 ----
}
break;
}
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
***************
*** 2524,2529 **** ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
--- 2534,2540 ----
break;
}
/* the base struct contains all the data, easy peasy */
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
diff --git a/src/backend/replication/logical/worker.c index 83c69092ae..a31b859cb8 100644
*** a/src/backend/replication/logical/worker.c
--- b/src/backend/replication/logical/worker.c
***************
*** 869,874 **** apply_handle_delete(StringInfo s)
--- 869,922 ----
CommandCounterIncrement();
}
+ /*
+ * Handle TRUNCATE message.
+ *
+ * TODO: FDW support
+ */
+ static void
+ apply_handle_truncate(StringInfo s)
+ {
+ LogicalRepRelMapEntry *rel;
+ LogicalRepRelId relid;
+ bool cascade = false;
+ bool restart_seqs = false;
+ List *rels = NIL;
+ List *relids = NIL;
+
+ ensure_transaction();
+
+ relid = logicalrep_read_truncate(s, &cascade, &restart_seqs);
+ rel = logicalrep_rel_open(relid, RowExclusiveLock);
+ if (!should_apply_changes_for_rel(rel))
+ {
+ /*
+ * The relation can't become interesting in the middle of the
+ * transaction so it's safe to unlock it.
+ */
+ logicalrep_rel_close(rel, RowExclusiveLock);
+ return;
+ }
+
+ /* Check if we can do the truncate. */
+ check_relation_updatable(rel);
+
+ rels = lappend(rels, rel->localrel);
+ relids = lappend_oid(relids, rel->localreloid);
+
+ ExecuteTruncateGuts(rels, relids, NULL,
+ cascade ? DROP_CASCADE : DROP_RESTRICT,
+ restart_seqs);
+
+ /* logicalrep_rel_close call not needed, because ExecuteTruncateGuts
+ * already closes the relations. Setting localrel to NULL in the map entry
+ * is still needed.
+ */
+ rel->localrel = NULL;
+
+ CommandCounterIncrement();
+ }
+
/*
* Logical replication protocol message dispatcher.
***************
*** 900,905 **** apply_dispatch(StringInfo s)
--- 948,957 ----
case 'D':
apply_handle_delete(s);
break;
+ /* TRUNCATE */
+ case 'T':
+ apply_handle_truncate(s);
+ break;
/* RELATION */
case 'R':
apply_handle_relation(s);
diff --git a/src/backend/replication/pgoutput/pindex 40a1ef3c1d..7fbd0a3dca 100644
*** a/src/backend/replication/pgoutput/pgoutput.c
--- b/src/backend/replication/pgoutput/pgoutput.c
***************
*** 275,280 **** pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
--- 275,281 ----
if (!relentry->pubactions.pubupdate)
return;
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
case REORDER_BUFFER_CHANGE_DELETE:
if (!relentry->pubactions.pubdelete)
return;
***************
*** 352,357 **** pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
--- 353,367 ----
else
elog(DEBUG1, "didn't send DELETE change because of missing oldtuple");
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ {
+ OutputPluginPrepareWrite(ctx, true);
+ logicalrep_write_truncate(ctx->out, relation,
+ change->data.truncate_msg.cascade,
+ change->data.truncate_msg.restart_seqs);
+ OutputPluginWrite(ctx, true);
+ }
+ break;
default:
Assert(false);
}
diff --git a/src/include/access/heapam_xlog.h b/srindex 700e25c36a..c56b200f7d 100644
*** a/src/include/access/heapam_xlog.h
--- b/src/include/access/heapam_xlog.h
***************
*** 32,38 ****
#define XLOG_HEAP_INSERT 0x00
#define XLOG_HEAP_DELETE 0x10
#define XLOG_HEAP_UPDATE 0x20
! /* 0x030 is free, was XLOG_HEAP_MOVE */
#define XLOG_HEAP_HOT_UPDATE 0x40
#define XLOG_HEAP_CONFIRM 0x50
#define XLOG_HEAP_LOCK 0x60
--- 32,38 ----
#define XLOG_HEAP_INSERT 0x00
#define XLOG_HEAP_DELETE 0x10
#define XLOG_HEAP_UPDATE 0x20
! #define XLOG_HEAP_TRUNCATE 0x30
#define XLOG_HEAP_HOT_UPDATE 0x40
#define XLOG_HEAP_CONFIRM 0x50
#define XLOG_HEAP_LOCK 0x60
***************
*** 109,114 **** typedef struct xl_heap_delete
--- 109,136 ----
#define SizeOfHeapDelete (offsetof(xl_heap_delete, flags) + sizeof(uint8))
+ /*
+ * xl_heap_delete flag values, 8 bits are available.
+ */
+ #define XLH_TRUNCATE_CASCADE (1<<0)
+ #define XLH_TRUNCATE_RESTART_SEQS (1<<1)
+
+ /*
+ * For truncate we list all truncated relids in an array, followed by all
+ * sequence relids that need to be restarted, if any.
+ * All rels are always within the same database, so we just list dbid once.
+ */
+ typedef struct xl_heap_truncate
+ {
+ Oid dbId;
+ uint32 nrelids;
+ uint32 nseqrelids;
+ uint8 flags;
+ Oid relids[FLEXIBLE_ARRAY_MEMBER];
+ } xl_heap_truncate;
+
+ #define SizeOfHeapTruncate (offsetof(xl_heap_truncate, relids))
+
/*
* We don't store the whole fixed part (HeapTupleHeaderData) of an inserted
* or updated tuple in WAL; we can save a few bytes by reconstructing the
diff --git a/src/include/executor/execuindex e6569e1038..581932d83d 100644
*** a/src/include/executor/executor.h
--- b/src/include/executor/executor.h
***************
*** 553,556 **** extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
--- 553,562 ----
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
+ /*
+ * prototype for tablecmds.c accessible for logical apply
+ */
+ void ExecuteTruncateGuts(List *rels, List *relids, List *relids_logged,
+ DropBehavior behavior, bool restart_seqs);
+
#endif /* EXECUTOR_H */
diff --git a/src/include/replication/lindex 0eb21057c5..b1abb9e36f 100644
*** a/src/include/replication/logicalproto.h
--- b/src/include/replication/logicalproto.h
***************
*** 98,103 **** extern void logicalrep_write_delete(StringInfo out, Relation rel,
--- 98,107 ----
HeapTuple oldtuple);
extern LogicalRepRelId logicalrep_read_delete(StringInfo in,
LogicalRepTupleData *oldtup);
+ extern void logicalrep_write_truncate(StringInfo out, Relation rel,
+ bool cascade, bool restart_seqs);
+ extern LogicalRepRelId logicalrep_read_truncate(StringInfo in,
+ bool *cascade, bool *restart_seqs);
extern void logicalrep_write_rel(StringInfo out, Relation rel);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
extern void logicalrep_write_typ(StringInfo out, Oid typoid);
diff --git a/src/include/replication/reorderbindex 0970abca52..e8439fac49 100644
*** a/src/include/replication/reorderbuffer.h
--- b/src/include/replication/reorderbuffer.h
***************
*** 59,65 **** enum ReorderBufferChangeType
REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID,
REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID,
REORDER_BUFFER_CHANGE_INTERNAL_SPEC_INSERT,
! REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM
};
/*
--- 59,66 ----
REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID,
REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID,
REORDER_BUFFER_CHANGE_INTERNAL_SPEC_INSERT,
! REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM,
! REORDER_BUFFER_CHANGE_TRUNCATE
};
/*
***************
*** 128,134 **** typedef struct ReorderBufferChange
CommandId cmax;
CommandId combocid;
} tuplecid;
! } data;
/*
* While in use this is how a change is linked into a transactions,
--- 129,146 ----
CommandId cmax;
CommandId combocid;
} tuplecid;
!
! /*
! * Truncate data for REORDER_BUFFER_CHANGE_TRUNCATE representing
! * one relation to be truncated.
! */
! struct
! {
! Oid relid;
! bool cascade;
! bool restart_seqs;
! } truncate_msg;
! } data;
/*
* While in use this is how a change is linked into a transactions,
diff --git a/src/test/subscription/t/001_rep_cindex 0136c79d4b..974e88821c 100644
*** a/src/test/subscription/t/001_rep_changes.pl
--- b/src/test/subscription/t/001_rep_changes.pl
***************
*** 3,9 **** use strict;
use warnings;
use PostgresNode;
use TestLib;
! use Test::More tests => 16;
# Initialize publisher node
my $node_publisher = get_new_node('publisher');
--- 3,9 ----
use warnings;
use PostgresNode;
use TestLib;
! use Test::More tests => 24;
# Initialize publisher node
my $node_publisher = get_new_node('publisher');
***************
*** 189,199 **** $result = $node_subscriber->safe_psql('postgres',
is($result, qq(20|-20|-1),
'check changes skipped after subscription publication change');
# check alter publication (relcache invalidation etc)
$node_publisher->safe_psql('postgres',
"ALTER PUBLICATION tap_pub_ins_only SET (publish = 'insert, delete')");
$node_publisher->safe_psql('postgres',
! "ALTER PUBLICATION tap_pub_ins_only ADD TABLE tab_full");
$node_publisher->safe_psql('postgres', "DELETE FROM tab_ins WHERE a > 0");
$node_subscriber->safe_psql('postgres',
"ALTER SUBSCRIPTION tap_sub REFRESH PUBLICATION WITH (copy_data = false)"
--- 189,210 ----
is($result, qq(20|-20|-1),
'check changes skipped after subscription publication change');
+ # truncate should not be replicated
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep");
+
+ $node_publisher->poll_query_until('postgres', $caughtup_query)
+ or die "Timed out while waiting for subscriber to catch up";
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_rep");
+ is($result, qq(20|-20|-1),
+ 'check changes skipped after subscription publication change');
+
# check alter publication (relcache invalidation etc)
$node_publisher->safe_psql('postgres',
"ALTER PUBLICATION tap_pub_ins_only SET (publish = 'insert, delete')");
$node_publisher->safe_psql('postgres',
! "ALTER PUBLICATION tap_pub_ins_only ADD TABLE tab_full, tab_rep");
$node_publisher->safe_psql('postgres', "DELETE FROM tab_ins WHERE a > 0");
$node_subscriber->safe_psql('postgres',
"ALTER SUBSCRIPTION tap_sub REFRESH PUBLICATION WITH (copy_data = false)"
***************
*** 213,218 **** $result = $node_subscriber->safe_psql('postgres',
--- 224,295 ----
"SELECT count(*), min(a), max(a) FROM tab_full");
is($result, qq(21|0|100), 'check replicated insert after alter publication');
+ # add a sequence and a foreign key to check cascade and restart
+ # identity truncate options
+ $node_subscriber->safe_psql('postgres',
+ "CREATE TABLE tab_notrep_fk (a int REFERENCES tab_rep(a))"
+ );
+ $node_subscriber->safe_psql('postgres', "INSERT INTO tab_notrep_fk VALUES (-1)");
+ $node_subscriber->safe_psql('postgres',
+ "CREATE SEQUENCE seq_notrep OWNED BY tab_rep.a"
+ );
+ $node_subscriber->safe_psql('postgres',
+ "ALTER SEQUENCE seq_notrep START 101"
+ );
+
+ # truncate should now be replicated
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep");
+
+ $node_publisher->poll_query_until('postgres', $caughtup_query)
+ or die "Timed out while waiting for subscriber to catch up";
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_rep");
+ is($result, qq(0||),
+ 'check replicated truncate after alter publication');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_notrep_fk");
+ is($result, qq(1|-1|-1),
+ 'check replicated truncate does not cascade');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq_notrep')");
+ is($result, qq(1),
+ 'check replicated truncate does not restart identities');
+
+ # should restart identity
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep RESTART IDENTITY");
+
+ $node_publisher->poll_query_until('postgres', $caughtup_query)
+ or die "Timed out while waiting for subscriber to catch up";
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_notrep_fk");
+ is($result, qq(1|-1|-1),
+ 'check replicated truncate does not cascade');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq_notrep')");
+ is($result, qq(101),
+ 'check replicated truncate restart identities');
+
+ # should cascade
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep CASCADE");
+
+ $node_publisher->poll_query_until('postgres', $caughtup_query)
+ or die "Timed out while waiting for subscriber to catch up";
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_notrep_fk");
+ is($result, qq(0||),
+ 'check replicated truncate cascade');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq_notrep')");
+ is($result, qq(102),
+ 'check replicated truncate does not restart identities');
+
# check restart on rename
$oldpid = $node_publisher->safe_psql('postgres',
"SELECT pid FROM pg_stat_replication WHERE application_name = '$appname';"
Re: [PATCH] Logical decoding of TRUNCATE
Patch rebased on the current master.
Regards,
Marco
--
Marco Nenciarini - 2ndQuadrant Italy
PostgreSQL Training, Services and Support
marco.nenciarini@2ndQuadrant.it | www.2ndQuadrant.it
Attachments:
logical_decode_truncate.v12.patchtext/plain; charset=UTF-8; name=logical_decode_truncate.v12.patch; x-mac-creator=0; x-mac-type=0Download
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 1e22c1eefc..d1feea4909 100644
*** a/contrib/test_decoding/expected/ddl.out
--- b/contrib/test_decoding/expected/ddl.out
***************
*** 543,548 **** UPDATE table_with_unique_not_null SET data = 3 WHERE data = 2;
--- 543,550 ----
UPDATE table_with_unique_not_null SET id = -id;
UPDATE table_with_unique_not_null SET id = -id;
DELETE FROM table_with_unique_not_null WHERE data = 3;
+ TRUNCATE table_with_unique_not_null;
+ TRUNCATE table_with_unique_not_null, table_with_unique_not_null RESTART IDENTITY CASCADE;
-- check toast support
BEGIN;
CREATE SEQUENCE toasttable_rand_seq START 79 INCREMENT 1499; -- portable "random"
***************
*** 660,665 **** SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
--- 662,673 ----
table public.table_with_unique_not_null: DELETE: id[integer]:4
COMMIT
BEGIN
+ table public.table_with_unique_not_null: TRUNCATE: (no-flags)
+ COMMIT
+ BEGIN
+ table public.table_with_unique_not_null: TRUNCATE: restart_seqs cascade
+ COMMIT
+ BEGIN
table public.toasttable: INSERT: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
BEGIN
***************
*** 668,674 **** SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
BEGIN
table public.toasttable: UPDATE: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
! (103 rows)
INSERT INTO toasttable(toasted_col1) SELECT string_agg(g.i::text, '') FROM generate_series(1, 2000) g(i);
-- update of second column, first column unchanged
--- 676,682 ----
BEGIN
table public.toasttable: UPDATE: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
! (109 rows)
INSERT INTO toasttable(toasted_col1) SELECT string_agg(g.i::text, '') FROM generate_series(1, 2000) g(i);
-- update of second column, first column unchanged
diff --git a/contrib/test_decoding/sql/ddl.sqindex 057dae056b..0fb4102db8 100644
*** a/contrib/test_decoding/sql/ddl.sql
--- b/contrib/test_decoding/sql/ddl.sql
***************
*** 333,338 **** UPDATE table_with_unique_not_null SET id = -id;
--- 333,341 ----
UPDATE table_with_unique_not_null SET id = -id;
DELETE FROM table_with_unique_not_null WHERE data = 3;
+ TRUNCATE table_with_unique_not_null;
+ TRUNCATE table_with_unique_not_null, table_with_unique_not_null RESTART IDENTITY CASCADE;
+
-- check toast support
BEGIN;
CREATE SEQUENCE toasttable_rand_seq START 79 INCREMENT 1499; -- portable "random"
diff --git a/contrib/test_decoding/test_index 0f18afa852..55c4593ed5 100644
*** a/contrib/test_decoding/test_decoding.c
--- b/contrib/test_decoding/test_decoding.c
***************
*** 466,471 **** pg_decode_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
--- 466,485 ----
&change->data.tp.oldtuple->tuple,
true);
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ appendStringInfoString(ctx->out, " TRUNCATE:");
+
+ if (change->data.truncate_msg.restart_seqs
+ || change->data.truncate_msg.cascade)
+ {
+ if (change->data.truncate_msg.restart_seqs)
+ appendStringInfo(ctx->out, " restart_seqs");
+ if (change->data.truncate_msg.cascade)
+ appendStringInfo(ctx->out, " cascade");
+ }
+ else
+ appendStringInfoString(ctx->out, " (no-flags)");
+ break;
default:
Assert(false);
}
diff --git a/doc/src/sgml/logical-replicatioindex 75551d8ee1..6b64c8ef76 100644
*** a/doc/src/sgml/logical-replication.sgml
--- b/doc/src/sgml/logical-replication.sgml
***************
*** 111,116 ****
--- 111,118 ----
any combination of <command>INSERT</command>, <command>UPDATE</command>, and
<command>DELETE</command>, similar to how triggers are fired by
particular event types. By default, all operation types are replicated.
+ <command>TRUNCATE</command> is treated as a form of <command>DELETE</command>
+ for the purpose of deciding whether to publish, or not.
</para>
<para>
***************
*** 364,378 ****
</para>
</listitem>
- <listitem>
- <para>
- <command>TRUNCATE</command> commands are not replicated. This can, of
- course, be worked around by using <command>DELETE</command> instead. To
- avoid accidental <command>TRUNCATE</command> invocations, you can revoke
- the <literal>TRUNCATE</literal> privilege from tables.
- </para>
- </listitem>
-
<listitem>
<para>
Large objects (see <xref linkend="largeobjects"/>) are not replicated.
--- 366,371 ----
diff --git a/doc/src/sgml/protocol.sgml b/doindex 4c5ed1e6d6..d2a0d7e52e 100644
*** a/doc/src/sgml/protocol.sgml
--- b/doc/src/sgml/protocol.sgml
***************
*** 6838,6843 **** TupleData
--- 6838,6889 ----
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ Truncate
+ </term>
+ <listitem>
+ <para>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ Byte1('T')
+ </term>
+ <listitem>
+ <para>
+ Identifies the message as a truncate message.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ Int32
+ </term>
+ <listitem>
+ <para>
+ ID of the relation corresponding to the ID in the relation
+ message.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ Int8
+ </term>
+ <listitem>
+ <para>
+ Option flags for truncate. Currently can be 0 for no flags,
+ 1 for CASCADE, 2 for RESTART IDENTITY and 3 if both are set.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect1>
diff --git a/doc/src/sgml/ref/creindex bfe12d5f41..c5032938a7 100644
*** a/doc/src/sgml/ref/create_publication.sgml
--- b/doc/src/sgml/ref/create_publication.sgml
***************
*** 111,116 **** CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
--- 111,121 ----
and so the default value for this option is
<literal>'insert, update, delete'</literal>.
</para>
+ <para>
+ <command>TRUNCATE</command> is treated as a form of
+ <command>DELETE</command> for the purpose of deciding whether
+ to publish, or not.
+ </para>
</listitem>
</varlistentry>
</variablelist>
***************
*** 168,175 **** CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
</para>
<para>
! <command>TRUNCATE</command> and <acronym>DDL</acronym> operations
! are not published.
</para>
</refsect1>
--- 173,179 ----
</para>
<para>
! <acronym>DDL</acronym> operations are not published.
</para>
</refsect1>
diff --git a/src/backend/access/heap/heapam.c bindex dbc8f2d6c7..f4c6eaf57f 100644
*** a/src/backend/access/heap/heapam.c
--- b/src/backend/access/heap/heapam.c
***************
*** 9134,9139 **** heap_redo(XLogReaderState *record)
--- 9134,9146 ----
case XLOG_HEAP_UPDATE:
heap_xlog_update(record, false);
break;
+ case XLOG_HEAP_TRUNCATE:
+ /*
+ * TRUNCATE is a no-op because the actions are already logged
+ * as SMGR WAL records. TRUNCATE WAL record only exists to allow
+ * it to be logically decoded precisely.
+ */
+ break;
case XLOG_HEAP_HOT_UPDATE:
heap_xlog_update(record, true);
break;
diff --git a/src/backend/access/rmgrdesindex b00c071cb6..15550cdff0 100644
*** a/src/backend/access/rmgrdesc/heapdesc.c
--- b/src/backend/access/rmgrdesc/heapdesc.c
***************
*** 75,80 **** heap_desc(StringInfo buf, XLogReaderState *record)
--- 75,93 ----
xlrec->new_offnum,
xlrec->new_xmax);
}
+ else if (info == XLOG_HEAP_TRUNCATE)
+ {
+ xl_heap_truncate *xlrec = (xl_heap_truncate *) rec;
+
+ if (xlrec->flags & XLH_TRUNCATE_CASCADE)
+ appendStringInfo(buf, "cascade ");
+ if (xlrec->flags & XLH_TRUNCATE_RESTART_SEQS)
+ appendStringInfo(buf, "restart_seqs ");
+ appendStringInfo(buf, "nrelids %u nseqrelids %u",
+ xlrec->nrelids,
+ xlrec->nseqrelids);
+ /* Skip the list of relids and seqrelids */
+ }
else if (info == XLOG_HEAP_CONFIRM)
{
xl_heap_confirm *xlrec = (xl_heap_confirm *) rec;
***************
*** 186,191 **** heap_identify(uint8 info)
--- 199,207 ----
case XLOG_HEAP_HOT_UPDATE | XLOG_HEAP_INIT_PAGE:
id = "HOT_UPDATE+INIT";
break;
+ case XLOG_HEAP_TRUNCATE:
+ id = "TRUNCATE";
+ break;
case XLOG_HEAP_CONFIRM:
id = "HEAP_CONFIRM";
break;
diff --git a/src/backend/commands/tablecmds.cindex 180ebd0717..8355c4c5c7 100644
*** a/src/backend/commands/tablecmds.c
--- b/src/backend/commands/tablecmds.c
***************
*** 16,21 ****
--- 16,22 ----
#include "access/genam.h"
#include "access/heapam.h"
+ #include "access/heapam_xlog.h"
#include "access/multixact.h"
#include "access/reloptions.h"
#include "access/relscan.h"
***************
*** 1248,1258 **** ExecuteTruncate(TruncateStmt *stmt)
{
List *rels = NIL;
List *relids = NIL;
! List *seq_relids = NIL;
! EState *estate;
! ResultRelInfo *resultRelInfos;
! ResultRelInfo *resultRelInfo;
! SubTransactionId mySubid;
ListCell *cell;
/*
--- 1249,1255 ----
{
List *rels = NIL;
List *relids = NIL;
! List *relids_logged = NIL;
ListCell *cell;
/*
***************
*** 1276,1281 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1273,1281 ----
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, myrelid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, myrelid);
if (recurse)
{
***************
*** 1296,1301 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1296,1304 ----
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, childrelid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, childrelid);
}
}
else if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
***************
*** 1305,1311 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1308,1336 ----
errhint("Do not specify the ONLY keyword, or use truncate only on the partitions directly.")));
}
+ ExecuteTruncateGuts(rels, relids, relids_logged,
+ stmt->behavior, stmt->restart_seqs);
+ }
+
+ void
+ ExecuteTruncateGuts(List *rels, List *relids, List *relids_logged,
+ DropBehavior behavior, bool restart_seqs)
+ {
+ List *seq_relids = NIL;
+ EState *estate;
+ ResultRelInfo *resultRelInfos;
+ ResultRelInfo *resultRelInfo;
+ SubTransactionId mySubid;
+ ListCell *cell;
+ List *seq_relids_logged = NIL;
+ uint32 nrelids = 0;
+ uint32 nseqrelids = 0;
+ uint32 maxrelids = 2;
+ Oid *logrelids = NULL;
+
/*
+ * Open, exclusive-lock, and check all the explicitly-specified relations
+ *
* In CASCADE mode, suck in all referencing relations as well. This
* requires multiple iterations to find indirectly-dependent relations. At
* each phase, we need to exclusive-lock new rels before looking for their
***************
*** 1313,1319 **** ExecuteTruncate(TruncateStmt *stmt)
* soon as we open it, to avoid a faux pas such as holding lock for a long
* time on a rel we have no permissions for.
*/
! if (stmt->behavior == DROP_CASCADE)
{
for (;;)
{
--- 1338,1344 ----
* soon as we open it, to avoid a faux pas such as holding lock for a long
* time on a rel we have no permissions for.
*/
! if (behavior == DROP_CASCADE)
{
for (;;)
{
***************
*** 1335,1340 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1360,1368 ----
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, relid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, relid);
}
}
}
***************
*** 1354,1360 **** ExecuteTruncate(TruncateStmt *stmt)
#ifdef USE_ASSERT_CHECKING
heap_truncate_check_FKs(rels, false);
#else
! if (stmt->behavior == DROP_RESTRICT)
heap_truncate_check_FKs(rels, false);
#endif
}
--- 1382,1388 ----
#ifdef USE_ASSERT_CHECKING
heap_truncate_check_FKs(rels, false);
#else
! if (behavior == DROP_RESTRICT)
heap_truncate_check_FKs(rels, false);
#endif
}
***************
*** 1365,1371 **** ExecuteTruncate(TruncateStmt *stmt)
* We want to do this early since it's pointless to do all the truncation
* work only to fail on sequence permissions.
*/
! if (stmt->restart_seqs)
{
foreach(cell, rels)
{
--- 1393,1399 ----
* We want to do this early since it's pointless to do all the truncation
* work only to fail on sequence permissions.
*/
! if (restart_seqs)
{
foreach(cell, rels)
{
***************
*** 1386,1391 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1414,1423 ----
RelationGetRelationName(seq_rel));
seq_relids = lappend_oid(seq_relids, seq_relid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(seq_rel))
+ seq_relids_logged = lappend_oid(seq_relids_logged,
+ seq_relid);
relation_close(seq_rel, NoLock);
}
***************
*** 1520,1525 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1552,1611 ----
ResetSequence(seq_relid);
}
+ /*
+ * Write a WAL record to allow this set of actions to be logically decoded.
+ * We could optimize this away when !RelationIsLogicallyLogged(rel)
+ * but that doesn't save much space or time.
+ *
+ * Assemble an array of relids, then an array of seqrelids so we can write
+ * a single WAL record for the whole action.
+ */
+ logrelids = palloc(maxrelids * sizeof(Oid));
+ foreach (cell, relids_logged)
+ {
+ nrelids++;
+ if (nrelids > maxrelids)
+ {
+ maxrelids *= 2;
+ logrelids = repalloc(logrelids, maxrelids * sizeof(Oid));
+ }
+ logrelids[nrelids - 1] = lfirst_oid(cell);
+ }
+
+ foreach (cell, seq_relids_logged)
+ {
+ nseqrelids++;
+ if ((nrelids + nseqrelids) > maxrelids)
+ {
+ maxrelids *= 2;
+ logrelids = repalloc(logrelids, maxrelids * sizeof(Oid));
+ }
+ logrelids[nrelids + nseqrelids - 1] = lfirst_oid(cell);
+ }
+
+ if ((nrelids + nseqrelids) > 0)
+ {
+ xl_heap_truncate xlrec;
+
+ xlrec.dbId = MyDatabaseId;
+ xlrec.nrelids = nrelids;
+ xlrec.nseqrelids = nseqrelids;
+ xlrec.flags = 0;
+ if (behavior == DROP_CASCADE)
+ xlrec.flags |= XLH_TRUNCATE_CASCADE;
+ if (restart_seqs)
+ xlrec.flags |= XLH_TRUNCATE_RESTART_SEQS;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfHeapTruncate);
+ XLogRegisterData((char *) logrelids,
+ (nrelids + nseqrelids) * sizeof(Oid));
+
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ (void) XLogInsert(RM_HEAP_ID, XLOG_HEAP_TRUNCATE);
+ }
+
/*
* Process all AFTER STATEMENT TRUNCATE triggers.
*/
diff --git a/src/backend/replication/loindex 537eba7875..d8408e154c 100644
*** a/src/backend/replication/logical/decode.c
--- b/src/backend/replication/logical/decode.c
***************
*** 65,70 **** static void DecodeLogicalMsgOp(LogicalDecodingContext *ctx, XLogRecordBuffer *bu
--- 65,71 ----
static void DecodeInsert(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeUpdate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeDelete(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
+ static void DecodeTruncate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeMultiInsert(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeSpecConfirm(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
***************
*** 435,440 **** DecodeHeapOp(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
--- 436,446 ----
DecodeDelete(ctx, buf);
break;
+ case XLOG_HEAP_TRUNCATE:
+ if (SnapBuildProcessChange(builder, xid, buf->origptr))
+ DecodeTruncate(ctx, buf);
+ break;
+
case XLOG_HEAP_INPLACE:
/*
***************
*** 805,810 **** DecodeDelete(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
--- 811,856 ----
ReorderBufferQueueChange(ctx->reorder, XLogRecGetXid(r), buf->origptr, change);
}
+ /*
+ * Parse XLOG_HEAP_TRUNCATE from wal
+ */
+ static void
+ DecodeTruncate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+ {
+ XLogReaderState *r = buf->record;
+ xl_heap_truncate *xlrec;
+ ReorderBufferChange *change;
+ int i;
+
+ xlrec = (xl_heap_truncate *) XLogRecGetData(r);
+
+ /* only interested in our database */
+ if (xlrec->dbId != ctx->slot->data.database)
+ return;
+
+ /* output plugin doesn't look for this origin, no need to queue */
+ if (FilterByOrigin(ctx, XLogRecGetOrigin(r)))
+ return;
+
+ change = ReorderBufferGetChange(ctx->reorder);
+ change->action = REORDER_BUFFER_CHANGE_TRUNCATE;
+ change->origin_id = XLogRecGetOrigin(r);
+ if (xlrec->flags & XLH_TRUNCATE_CASCADE)
+ change->data.truncate_msg.cascade = true;
+ if (xlrec->flags & XLH_TRUNCATE_RESTART_SEQS)
+ change->data.truncate_msg.restart_seqs = true;
+
+ /*
+ * Queue up one change per relation, ignoring sequences for now
+ */
+ for (i = 0; i < xlrec->nrelids; i++)
+ {
+ change->data.truncate_msg.relid = xlrec->relids[i];
+ ReorderBufferQueueChange(ctx->reorder, XLogRecGetXid(r),
+ buf->origptr, change);
+ }
+ }
+
/*
* Decode XLOG_HEAP2_MULTI_INSERT_insert record into multiple tuplebufs.
*
diff --git a/src/backend/replication/logical/prindex 948343e4ae..2fa6f8393d 100644
*** a/src/backend/replication/logical/proto.c
--- b/src/backend/replication/logical/proto.c
***************
*** 26,31 ****
--- 26,34 ----
*/
#define LOGICALREP_IS_REPLICA_IDENTITY 1
+ #define TRUNCATE_CASCADE (1<<0)
+ #define TRUNCATE_RESTART_SEQS (1<<1)
+
static void logicalrep_write_attrs(StringInfo out, Relation rel);
static void logicalrep_write_tuple(StringInfo out, Relation rel,
HeapTuple tuple);
***************
*** 292,297 **** logicalrep_read_delete(StringInfo in, LogicalRepTupleData *oldtup)
--- 295,342 ----
return relid;
}
+ /*
+ * Write TRUNCATE to the output stream.
+ */
+ void
+ logicalrep_write_truncate(StringInfo out, Relation rel,
+ bool cascade, bool restart_seqs)
+ {
+ uint8 flags = 0;
+
+ pq_sendbyte(out, 'T'); /* action TRUNCATE */
+
+ /* use Oid as relation identifier */
+ pq_sendint32(out, RelationGetRelid(rel));
+
+ /* encode and send truncate flags */
+ if (cascade)
+ flags |= TRUNCATE_CASCADE;
+ if (restart_seqs)
+ flags |= TRUNCATE_RESTART_SEQS;
+ pq_sendint8(out, flags);
+ }
+
+ /*
+ * Read TRUNCATE from stream.
+ */
+ LogicalRepRelId
+ logicalrep_read_truncate(StringInfo in, bool *cascade, bool *restart_seqs)
+ {
+ LogicalRepRelId relid;
+ uint8 flags;
+
+ /* read the relation id */
+ relid = pq_getmsgint(in, 4);
+
+ /* read and decode truncate flags */
+ flags = pq_getmsgint(in, 1);
+ *cascade = (flags & TRUNCATE_CASCADE) > 0;
+ *restart_seqs = (flags & TRUNCATE_RESTART_SEQS) > 0;
+
+ return relid;
+ }
+
/*
* Write relation description to the output stream.
*/
diff --git a/src/backend/replication/logical/rindex 1208da2972..fc7350358a 100644
*** a/src/backend/replication/logical/reorderbuffer.c
--- b/src/backend/replication/logical/reorderbuffer.c
***************
*** 403,408 **** ReorderBufferReturnChange(ReorderBuffer *rb, ReorderBufferChange *change)
--- 403,410 ----
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ break;
}
pfree(change);
***************
*** 1342,1347 **** ReorderBufferCommit(ReorderBuffer *rb, TransactionId xid,
--- 1344,1356 ----
switch (change->action)
{
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ reloid = change->data.truncate_msg.relid;
+ relation = RelationIdGetRelation(reloid);
+ rb->apply_change(rb, txn, relation, change);
+ RelationClose(relation);
+ break;
+
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
/*
***************
*** 2239,2244 **** ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
--- 2248,2254 ----
}
break;
}
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
***************
*** 2524,2529 **** ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
--- 2534,2540 ----
break;
}
/* the base struct contains all the data, easy peasy */
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
diff --git a/src/backend/replication/logical/worker.c index 83c69092ae..a31b859cb8 100644
*** a/src/backend/replication/logical/worker.c
--- b/src/backend/replication/logical/worker.c
***************
*** 869,874 **** apply_handle_delete(StringInfo s)
--- 869,922 ----
CommandCounterIncrement();
}
+ /*
+ * Handle TRUNCATE message.
+ *
+ * TODO: FDW support
+ */
+ static void
+ apply_handle_truncate(StringInfo s)
+ {
+ LogicalRepRelMapEntry *rel;
+ LogicalRepRelId relid;
+ bool cascade = false;
+ bool restart_seqs = false;
+ List *rels = NIL;
+ List *relids = NIL;
+
+ ensure_transaction();
+
+ relid = logicalrep_read_truncate(s, &cascade, &restart_seqs);
+ rel = logicalrep_rel_open(relid, RowExclusiveLock);
+ if (!should_apply_changes_for_rel(rel))
+ {
+ /*
+ * The relation can't become interesting in the middle of the
+ * transaction so it's safe to unlock it.
+ */
+ logicalrep_rel_close(rel, RowExclusiveLock);
+ return;
+ }
+
+ /* Check if we can do the truncate. */
+ check_relation_updatable(rel);
+
+ rels = lappend(rels, rel->localrel);
+ relids = lappend_oid(relids, rel->localreloid);
+
+ ExecuteTruncateGuts(rels, relids, NULL,
+ cascade ? DROP_CASCADE : DROP_RESTRICT,
+ restart_seqs);
+
+ /* logicalrep_rel_close call not needed, because ExecuteTruncateGuts
+ * already closes the relations. Setting localrel to NULL in the map entry
+ * is still needed.
+ */
+ rel->localrel = NULL;
+
+ CommandCounterIncrement();
+ }
+
/*
* Logical replication protocol message dispatcher.
***************
*** 900,905 **** apply_dispatch(StringInfo s)
--- 948,957 ----
case 'D':
apply_handle_delete(s);
break;
+ /* TRUNCATE */
+ case 'T':
+ apply_handle_truncate(s);
+ break;
/* RELATION */
case 'R':
apply_handle_relation(s);
diff --git a/src/backend/replication/pgoutput/pindex 40a1ef3c1d..7fbd0a3dca 100644
*** a/src/backend/replication/pgoutput/pgoutput.c
--- b/src/backend/replication/pgoutput/pgoutput.c
***************
*** 275,280 **** pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
--- 275,281 ----
if (!relentry->pubactions.pubupdate)
return;
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
case REORDER_BUFFER_CHANGE_DELETE:
if (!relentry->pubactions.pubdelete)
return;
***************
*** 352,357 **** pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
--- 353,367 ----
else
elog(DEBUG1, "didn't send DELETE change because of missing oldtuple");
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ {
+ OutputPluginPrepareWrite(ctx, true);
+ logicalrep_write_truncate(ctx->out, relation,
+ change->data.truncate_msg.cascade,
+ change->data.truncate_msg.restart_seqs);
+ OutputPluginWrite(ctx, true);
+ }
+ break;
default:
Assert(false);
}
diff --git a/src/include/access/heapam_xlog.h b/srindex 700e25c36a..c56b200f7d 100644
*** a/src/include/access/heapam_xlog.h
--- b/src/include/access/heapam_xlog.h
***************
*** 32,38 ****
#define XLOG_HEAP_INSERT 0x00
#define XLOG_HEAP_DELETE 0x10
#define XLOG_HEAP_UPDATE 0x20
! /* 0x030 is free, was XLOG_HEAP_MOVE */
#define XLOG_HEAP_HOT_UPDATE 0x40
#define XLOG_HEAP_CONFIRM 0x50
#define XLOG_HEAP_LOCK 0x60
--- 32,38 ----
#define XLOG_HEAP_INSERT 0x00
#define XLOG_HEAP_DELETE 0x10
#define XLOG_HEAP_UPDATE 0x20
! #define XLOG_HEAP_TRUNCATE 0x30
#define XLOG_HEAP_HOT_UPDATE 0x40
#define XLOG_HEAP_CONFIRM 0x50
#define XLOG_HEAP_LOCK 0x60
***************
*** 109,114 **** typedef struct xl_heap_delete
--- 109,136 ----
#define SizeOfHeapDelete (offsetof(xl_heap_delete, flags) + sizeof(uint8))
+ /*
+ * xl_heap_delete flag values, 8 bits are available.
+ */
+ #define XLH_TRUNCATE_CASCADE (1<<0)
+ #define XLH_TRUNCATE_RESTART_SEQS (1<<1)
+
+ /*
+ * For truncate we list all truncated relids in an array, followed by all
+ * sequence relids that need to be restarted, if any.
+ * All rels are always within the same database, so we just list dbid once.
+ */
+ typedef struct xl_heap_truncate
+ {
+ Oid dbId;
+ uint32 nrelids;
+ uint32 nseqrelids;
+ uint8 flags;
+ Oid relids[FLEXIBLE_ARRAY_MEMBER];
+ } xl_heap_truncate;
+
+ #define SizeOfHeapTruncate (offsetof(xl_heap_truncate, relids))
+
/*
* We don't store the whole fixed part (HeapTupleHeaderData) of an inserted
* or updated tuple in WAL; we can save a few bytes by reconstructing the
diff --git a/src/include/executor/execuindex a782fae0f8..62ed488442 100644
*** a/src/include/executor/executor.h
--- b/src/include/executor/executor.h
***************
*** 556,559 **** extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
--- 556,565 ----
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
+ /*
+ * prototype for tablecmds.c accessible for logical apply
+ */
+ void ExecuteTruncateGuts(List *rels, List *relids, List *relids_logged,
+ DropBehavior behavior, bool restart_seqs);
+
#endif /* EXECUTOR_H */
diff --git a/src/include/replication/lindex 0eb21057c5..b1abb9e36f 100644
*** a/src/include/replication/logicalproto.h
--- b/src/include/replication/logicalproto.h
***************
*** 98,103 **** extern void logicalrep_write_delete(StringInfo out, Relation rel,
--- 98,107 ----
HeapTuple oldtuple);
extern LogicalRepRelId logicalrep_read_delete(StringInfo in,
LogicalRepTupleData *oldtup);
+ extern void logicalrep_write_truncate(StringInfo out, Relation rel,
+ bool cascade, bool restart_seqs);
+ extern LogicalRepRelId logicalrep_read_truncate(StringInfo in,
+ bool *cascade, bool *restart_seqs);
extern void logicalrep_write_rel(StringInfo out, Relation rel);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
extern void logicalrep_write_typ(StringInfo out, Oid typoid);
diff --git a/src/include/replication/reorderbindex 0970abca52..e8439fac49 100644
*** a/src/include/replication/reorderbuffer.h
--- b/src/include/replication/reorderbuffer.h
***************
*** 59,65 **** enum ReorderBufferChangeType
REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID,
REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID,
REORDER_BUFFER_CHANGE_INTERNAL_SPEC_INSERT,
! REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM
};
/*
--- 59,66 ----
REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID,
REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID,
REORDER_BUFFER_CHANGE_INTERNAL_SPEC_INSERT,
! REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM,
! REORDER_BUFFER_CHANGE_TRUNCATE
};
/*
***************
*** 128,134 **** typedef struct ReorderBufferChange
CommandId cmax;
CommandId combocid;
} tuplecid;
! } data;
/*
* While in use this is how a change is linked into a transactions,
--- 129,146 ----
CommandId cmax;
CommandId combocid;
} tuplecid;
!
! /*
! * Truncate data for REORDER_BUFFER_CHANGE_TRUNCATE representing
! * one relation to be truncated.
! */
! struct
! {
! Oid relid;
! bool cascade;
! bool restart_seqs;
! } truncate_msg;
! } data;
/*
* While in use this is how a change is linked into a transactions,
diff --git a/src/test/subscription/t/001_rep_cindex 0136c79d4b..974e88821c 100644
*** a/src/test/subscription/t/001_rep_changes.pl
--- b/src/test/subscription/t/001_rep_changes.pl
***************
*** 3,9 **** use strict;
use warnings;
use PostgresNode;
use TestLib;
! use Test::More tests => 16;
# Initialize publisher node
my $node_publisher = get_new_node('publisher');
--- 3,9 ----
use warnings;
use PostgresNode;
use TestLib;
! use Test::More tests => 24;
# Initialize publisher node
my $node_publisher = get_new_node('publisher');
***************
*** 189,199 **** $result = $node_subscriber->safe_psql('postgres',
is($result, qq(20|-20|-1),
'check changes skipped after subscription publication change');
# check alter publication (relcache invalidation etc)
$node_publisher->safe_psql('postgres',
"ALTER PUBLICATION tap_pub_ins_only SET (publish = 'insert, delete')");
$node_publisher->safe_psql('postgres',
! "ALTER PUBLICATION tap_pub_ins_only ADD TABLE tab_full");
$node_publisher->safe_psql('postgres', "DELETE FROM tab_ins WHERE a > 0");
$node_subscriber->safe_psql('postgres',
"ALTER SUBSCRIPTION tap_sub REFRESH PUBLICATION WITH (copy_data = false)"
--- 189,210 ----
is($result, qq(20|-20|-1),
'check changes skipped after subscription publication change');
+ # truncate should not be replicated
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep");
+
+ $node_publisher->poll_query_until('postgres', $caughtup_query)
+ or die "Timed out while waiting for subscriber to catch up";
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_rep");
+ is($result, qq(20|-20|-1),
+ 'check changes skipped after subscription publication change');
+
# check alter publication (relcache invalidation etc)
$node_publisher->safe_psql('postgres',
"ALTER PUBLICATION tap_pub_ins_only SET (publish = 'insert, delete')");
$node_publisher->safe_psql('postgres',
! "ALTER PUBLICATION tap_pub_ins_only ADD TABLE tab_full, tab_rep");
$node_publisher->safe_psql('postgres', "DELETE FROM tab_ins WHERE a > 0");
$node_subscriber->safe_psql('postgres',
"ALTER SUBSCRIPTION tap_sub REFRESH PUBLICATION WITH (copy_data = false)"
***************
*** 213,218 **** $result = $node_subscriber->safe_psql('postgres',
--- 224,295 ----
"SELECT count(*), min(a), max(a) FROM tab_full");
is($result, qq(21|0|100), 'check replicated insert after alter publication');
+ # add a sequence and a foreign key to check cascade and restart
+ # identity truncate options
+ $node_subscriber->safe_psql('postgres',
+ "CREATE TABLE tab_notrep_fk (a int REFERENCES tab_rep(a))"
+ );
+ $node_subscriber->safe_psql('postgres', "INSERT INTO tab_notrep_fk VALUES (-1)");
+ $node_subscriber->safe_psql('postgres',
+ "CREATE SEQUENCE seq_notrep OWNED BY tab_rep.a"
+ );
+ $node_subscriber->safe_psql('postgres',
+ "ALTER SEQUENCE seq_notrep START 101"
+ );
+
+ # truncate should now be replicated
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep");
+
+ $node_publisher->poll_query_until('postgres', $caughtup_query)
+ or die "Timed out while waiting for subscriber to catch up";
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_rep");
+ is($result, qq(0||),
+ 'check replicated truncate after alter publication');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_notrep_fk");
+ is($result, qq(1|-1|-1),
+ 'check replicated truncate does not cascade');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq_notrep')");
+ is($result, qq(1),
+ 'check replicated truncate does not restart identities');
+
+ # should restart identity
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep RESTART IDENTITY");
+
+ $node_publisher->poll_query_until('postgres', $caughtup_query)
+ or die "Timed out while waiting for subscriber to catch up";
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_notrep_fk");
+ is($result, qq(1|-1|-1),
+ 'check replicated truncate does not cascade');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq_notrep')");
+ is($result, qq(101),
+ 'check replicated truncate restart identities');
+
+ # should cascade
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep CASCADE");
+
+ $node_publisher->poll_query_until('postgres', $caughtup_query)
+ or die "Timed out while waiting for subscriber to catch up";
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_notrep_fk");
+ is($result, qq(0||),
+ 'check replicated truncate cascade');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq_notrep')");
+ is($result, qq(102),
+ 'check replicated truncate does not restart identities');
+
# check restart on rename
$oldpid = $node_publisher->safe_psql('postgres',
"SELECT pid FROM pg_stat_replication WHERE application_name = '$appname';"
Re: [PATCH] Logical decoding of TRUNCATE
Attached here there is the complete list of patches required to pass all
the tests. The 0001 patch is discussed in a separate thread, but I've
posted it also here to ease the review of the 0002.
Regards,
Marco
--
Marco Nenciarini - 2ndQuadrant Italy
PostgreSQL Training, Services and Support
marco.nenciarini@2ndQuadrant.it | www.2ndQuadrant.it
Attachments:
0001-truncate_ignore_fks_role_replica.v2.patchtext/plain; charset=UTF-8; name=0001-truncate_ignore_fks_role_replica.v2.patch; x-mac-creator=0; x-mac-type=0Download
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index e4a01699e4..aff56891e6 100644
*** a/doc/src/sgml/config.sgml
--- b/doc/src/sgml/config.sgml
***************
*** 6502,6508 **** COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
<listitem>
<para>
Controls firing of replication-related triggers and rules for the
! current session. Setting this variable requires
superuser privilege and results in discarding any previously cached
query plans. Possible values are <literal>origin</literal> (the default),
<literal>replica</literal> and <literal>local</literal>.
--- 6502,6510 ----
<listitem>
<para>
Controls firing of replication-related triggers and rules for the
! current session. When set to <literal>replica</literal> it also
! disables all the foreign key checks, which can leave the data in an
! inconsistent state if improperly used. Setting this variable requires
superuser privilege and results in discarding any previously cached
query plans. Possible values are <literal>origin</literal> (the default),
<literal>replica</literal> and <literal>local</literal>.
diff --git a/src/backend/commanindex d979ce266d..296807849f 100644
*** a/src/backend/commands/tablecmds.c
--- b/src/backend/commands/tablecmds.c
***************
*** 1341,1356 **** ExecuteTruncate(TruncateStmt *stmt)
}
/*
! * Check foreign key references. In CASCADE mode, this should be
! * unnecessary since we just pulled in all the references; but as a
! * cross-check, do it anyway if in an Assert-enabled build.
*/
#ifdef USE_ASSERT_CHECKING
- heap_truncate_check_FKs(rels, false);
- #else
- if (stmt->behavior == DROP_RESTRICT)
heap_truncate_check_FKs(rels, false);
#endif
/*
* If we are asked to restart sequences, find all the sequences, lock them
--- 1341,1364 ----
}
/*
! * Suppress foreign key references check if session replication role is
! * set to REPLICA.
*/
+ if (SessionReplicationRole != SESSION_REPLICATION_ROLE_REPLICA)
+ {
+
+ /*
+ * Check foreign key references. In CASCADE mode, this should be
+ * unnecessary since we just pulled in all the references; but as a
+ * cross-check, do it anyway if in an Assert-enabled build.
+ */
#ifdef USE_ASSERT_CHECKING
heap_truncate_check_FKs(rels, false);
+ #else
+ if (stmt->behavior == DROP_RESTRICT)
+ heap_truncate_check_FKs(rels, false);
#endif
+ }
/*
* If we are asked to restart sequences, find all the sequences, lock them
0002-logical_decode_truncate.v12.patchtext/plain; charset=UTF-8; name=0002-logical_decode_truncate.v12.patch; x-mac-creator=0; x-mac-type=0Download
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 1e22c1eefc..d1feea4909 100644
*** a/contrib/test_decoding/expected/ddl.out
--- b/contrib/test_decoding/expected/ddl.out
***************
*** 543,548 **** UPDATE table_with_unique_not_null SET data = 3 WHERE data = 2;
--- 543,550 ----
UPDATE table_with_unique_not_null SET id = -id;
UPDATE table_with_unique_not_null SET id = -id;
DELETE FROM table_with_unique_not_null WHERE data = 3;
+ TRUNCATE table_with_unique_not_null;
+ TRUNCATE table_with_unique_not_null, table_with_unique_not_null RESTART IDENTITY CASCADE;
-- check toast support
BEGIN;
CREATE SEQUENCE toasttable_rand_seq START 79 INCREMENT 1499; -- portable "random"
***************
*** 660,665 **** SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
--- 662,673 ----
table public.table_with_unique_not_null: DELETE: id[integer]:4
COMMIT
BEGIN
+ table public.table_with_unique_not_null: TRUNCATE: (no-flags)
+ COMMIT
+ BEGIN
+ table public.table_with_unique_not_null: TRUNCATE: restart_seqs cascade
+ COMMIT
+ BEGIN
table public.toasttable: INSERT: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
BEGIN
***************
*** 668,674 **** SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
BEGIN
table public.toasttable: UPDATE: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
! (103 rows)
INSERT INTO toasttable(toasted_col1) SELECT string_agg(g.i::text, '') FROM generate_series(1, 2000) g(i);
-- update of second column, first column unchanged
--- 676,682 ----
BEGIN
table public.toasttable: UPDATE: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
! (109 rows)
INSERT INTO toasttable(toasted_col1) SELECT string_agg(g.i::text, '') FROM generate_series(1, 2000) g(i);
-- update of second column, first column unchanged
diff --git a/contrib/test_decoding/sql/ddl.sqindex 057dae056b..0fb4102db8 100644
*** a/contrib/test_decoding/sql/ddl.sql
--- b/contrib/test_decoding/sql/ddl.sql
***************
*** 333,338 **** UPDATE table_with_unique_not_null SET id = -id;
--- 333,341 ----
UPDATE table_with_unique_not_null SET id = -id;
DELETE FROM table_with_unique_not_null WHERE data = 3;
+ TRUNCATE table_with_unique_not_null;
+ TRUNCATE table_with_unique_not_null, table_with_unique_not_null RESTART IDENTITY CASCADE;
+
-- check toast support
BEGIN;
CREATE SEQUENCE toasttable_rand_seq START 79 INCREMENT 1499; -- portable "random"
diff --git a/contrib/test_decoding/test_index 0f18afa852..55c4593ed5 100644
*** a/contrib/test_decoding/test_decoding.c
--- b/contrib/test_decoding/test_decoding.c
***************
*** 466,471 **** pg_decode_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
--- 466,485 ----
&change->data.tp.oldtuple->tuple,
true);
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ appendStringInfoString(ctx->out, " TRUNCATE:");
+
+ if (change->data.truncate_msg.restart_seqs
+ || change->data.truncate_msg.cascade)
+ {
+ if (change->data.truncate_msg.restart_seqs)
+ appendStringInfo(ctx->out, " restart_seqs");
+ if (change->data.truncate_msg.cascade)
+ appendStringInfo(ctx->out, " cascade");
+ }
+ else
+ appendStringInfoString(ctx->out, " (no-flags)");
+ break;
default:
Assert(false);
}
diff --git a/doc/src/sgml/logical-replicatioindex 75551d8ee1..6b64c8ef76 100644
*** a/doc/src/sgml/logical-replication.sgml
--- b/doc/src/sgml/logical-replication.sgml
***************
*** 111,116 ****
--- 111,118 ----
any combination of <command>INSERT</command>, <command>UPDATE</command>, and
<command>DELETE</command>, similar to how triggers are fired by
particular event types. By default, all operation types are replicated.
+ <command>TRUNCATE</command> is treated as a form of <command>DELETE</command>
+ for the purpose of deciding whether to publish, or not.
</para>
<para>
***************
*** 364,378 ****
</para>
</listitem>
- <listitem>
- <para>
- <command>TRUNCATE</command> commands are not replicated. This can, of
- course, be worked around by using <command>DELETE</command> instead. To
- avoid accidental <command>TRUNCATE</command> invocations, you can revoke
- the <literal>TRUNCATE</literal> privilege from tables.
- </para>
- </listitem>
-
<listitem>
<para>
Large objects (see <xref linkend="largeobjects"/>) are not replicated.
--- 366,371 ----
diff --git a/doc/src/sgml/protocol.sgml b/doindex 4c5ed1e6d6..d2a0d7e52e 100644
*** a/doc/src/sgml/protocol.sgml
--- b/doc/src/sgml/protocol.sgml
***************
*** 6838,6843 **** TupleData
--- 6838,6889 ----
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ Truncate
+ </term>
+ <listitem>
+ <para>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ Byte1('T')
+ </term>
+ <listitem>
+ <para>
+ Identifies the message as a truncate message.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ Int32
+ </term>
+ <listitem>
+ <para>
+ ID of the relation corresponding to the ID in the relation
+ message.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ Int8
+ </term>
+ <listitem>
+ <para>
+ Option flags for truncate. Currently can be 0 for no flags,
+ 1 for CASCADE, 2 for RESTART IDENTITY and 3 if both are set.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect1>
diff --git a/doc/src/sgml/ref/creindex bfe12d5f41..c5032938a7 100644
*** a/doc/src/sgml/ref/create_publication.sgml
--- b/doc/src/sgml/ref/create_publication.sgml
***************
*** 111,116 **** CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
--- 111,121 ----
and so the default value for this option is
<literal>'insert, update, delete'</literal>.
</para>
+ <para>
+ <command>TRUNCATE</command> is treated as a form of
+ <command>DELETE</command> for the purpose of deciding whether
+ to publish, or not.
+ </para>
</listitem>
</varlistentry>
</variablelist>
***************
*** 168,175 **** CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
</para>
<para>
! <command>TRUNCATE</command> and <acronym>DDL</acronym> operations
! are not published.
</para>
</refsect1>
--- 173,179 ----
</para>
<para>
! <acronym>DDL</acronym> operations are not published.
</para>
</refsect1>
diff --git a/src/backend/access/heap/heapam.c bindex dbc8f2d6c7..f4c6eaf57f 100644
*** a/src/backend/access/heap/heapam.c
--- b/src/backend/access/heap/heapam.c
***************
*** 9134,9139 **** heap_redo(XLogReaderState *record)
--- 9134,9146 ----
case XLOG_HEAP_UPDATE:
heap_xlog_update(record, false);
break;
+ case XLOG_HEAP_TRUNCATE:
+ /*
+ * TRUNCATE is a no-op because the actions are already logged
+ * as SMGR WAL records. TRUNCATE WAL record only exists to allow
+ * it to be logically decoded precisely.
+ */
+ break;
case XLOG_HEAP_HOT_UPDATE:
heap_xlog_update(record, true);
break;
diff --git a/src/backend/access/rmgrdesindex b00c071cb6..15550cdff0 100644
*** a/src/backend/access/rmgrdesc/heapdesc.c
--- b/src/backend/access/rmgrdesc/heapdesc.c
***************
*** 75,80 **** heap_desc(StringInfo buf, XLogReaderState *record)
--- 75,93 ----
xlrec->new_offnum,
xlrec->new_xmax);
}
+ else if (info == XLOG_HEAP_TRUNCATE)
+ {
+ xl_heap_truncate *xlrec = (xl_heap_truncate *) rec;
+
+ if (xlrec->flags & XLH_TRUNCATE_CASCADE)
+ appendStringInfo(buf, "cascade ");
+ if (xlrec->flags & XLH_TRUNCATE_RESTART_SEQS)
+ appendStringInfo(buf, "restart_seqs ");
+ appendStringInfo(buf, "nrelids %u nseqrelids %u",
+ xlrec->nrelids,
+ xlrec->nseqrelids);
+ /* Skip the list of relids and seqrelids */
+ }
else if (info == XLOG_HEAP_CONFIRM)
{
xl_heap_confirm *xlrec = (xl_heap_confirm *) rec;
***************
*** 186,191 **** heap_identify(uint8 info)
--- 199,207 ----
case XLOG_HEAP_HOT_UPDATE | XLOG_HEAP_INIT_PAGE:
id = "HOT_UPDATE+INIT";
break;
+ case XLOG_HEAP_TRUNCATE:
+ id = "TRUNCATE";
+ break;
case XLOG_HEAP_CONFIRM:
id = "HEAP_CONFIRM";
break;
diff --git a/src/backend/commands/tablecmds.cindex 180ebd0717..8355c4c5c7 100644
*** a/src/backend/commands/tablecmds.c
--- b/src/backend/commands/tablecmds.c
***************
*** 16,21 ****
--- 16,22 ----
#include "access/genam.h"
#include "access/heapam.h"
+ #include "access/heapam_xlog.h"
#include "access/multixact.h"
#include "access/reloptions.h"
#include "access/relscan.h"
***************
*** 1248,1258 **** ExecuteTruncate(TruncateStmt *stmt)
{
List *rels = NIL;
List *relids = NIL;
! List *seq_relids = NIL;
! EState *estate;
! ResultRelInfo *resultRelInfos;
! ResultRelInfo *resultRelInfo;
! SubTransactionId mySubid;
ListCell *cell;
/*
--- 1249,1255 ----
{
List *rels = NIL;
List *relids = NIL;
! List *relids_logged = NIL;
ListCell *cell;
/*
***************
*** 1276,1281 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1273,1281 ----
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, myrelid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, myrelid);
if (recurse)
{
***************
*** 1296,1301 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1296,1304 ----
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, childrelid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, childrelid);
}
}
else if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
***************
*** 1305,1311 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1308,1336 ----
errhint("Do not specify the ONLY keyword, or use truncate only on the partitions directly.")));
}
+ ExecuteTruncateGuts(rels, relids, relids_logged,
+ stmt->behavior, stmt->restart_seqs);
+ }
+
+ void
+ ExecuteTruncateGuts(List *rels, List *relids, List *relids_logged,
+ DropBehavior behavior, bool restart_seqs)
+ {
+ List *seq_relids = NIL;
+ EState *estate;
+ ResultRelInfo *resultRelInfos;
+ ResultRelInfo *resultRelInfo;
+ SubTransactionId mySubid;
+ ListCell *cell;
+ List *seq_relids_logged = NIL;
+ uint32 nrelids = 0;
+ uint32 nseqrelids = 0;
+ uint32 maxrelids = 2;
+ Oid *logrelids = NULL;
+
/*
+ * Open, exclusive-lock, and check all the explicitly-specified relations
+ *
* In CASCADE mode, suck in all referencing relations as well. This
* requires multiple iterations to find indirectly-dependent relations. At
* each phase, we need to exclusive-lock new rels before looking for their
***************
*** 1313,1319 **** ExecuteTruncate(TruncateStmt *stmt)
* soon as we open it, to avoid a faux pas such as holding lock for a long
* time on a rel we have no permissions for.
*/
! if (stmt->behavior == DROP_CASCADE)
{
for (;;)
{
--- 1338,1344 ----
* soon as we open it, to avoid a faux pas such as holding lock for a long
* time on a rel we have no permissions for.
*/
! if (behavior == DROP_CASCADE)
{
for (;;)
{
***************
*** 1335,1340 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1360,1368 ----
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, relid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, relid);
}
}
}
***************
*** 1354,1360 **** ExecuteTruncate(TruncateStmt *stmt)
#ifdef USE_ASSERT_CHECKING
heap_truncate_check_FKs(rels, false);
#else
! if (stmt->behavior == DROP_RESTRICT)
heap_truncate_check_FKs(rels, false);
#endif
}
--- 1382,1388 ----
#ifdef USE_ASSERT_CHECKING
heap_truncate_check_FKs(rels, false);
#else
! if (behavior == DROP_RESTRICT)
heap_truncate_check_FKs(rels, false);
#endif
}
***************
*** 1365,1371 **** ExecuteTruncate(TruncateStmt *stmt)
* We want to do this early since it's pointless to do all the truncation
* work only to fail on sequence permissions.
*/
! if (stmt->restart_seqs)
{
foreach(cell, rels)
{
--- 1393,1399 ----
* We want to do this early since it's pointless to do all the truncation
* work only to fail on sequence permissions.
*/
! if (restart_seqs)
{
foreach(cell, rels)
{
***************
*** 1386,1391 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1414,1423 ----
RelationGetRelationName(seq_rel));
seq_relids = lappend_oid(seq_relids, seq_relid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(seq_rel))
+ seq_relids_logged = lappend_oid(seq_relids_logged,
+ seq_relid);
relation_close(seq_rel, NoLock);
}
***************
*** 1520,1525 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1552,1611 ----
ResetSequence(seq_relid);
}
+ /*
+ * Write a WAL record to allow this set of actions to be logically decoded.
+ * We could optimize this away when !RelationIsLogicallyLogged(rel)
+ * but that doesn't save much space or time.
+ *
+ * Assemble an array of relids, then an array of seqrelids so we can write
+ * a single WAL record for the whole action.
+ */
+ logrelids = palloc(maxrelids * sizeof(Oid));
+ foreach (cell, relids_logged)
+ {
+ nrelids++;
+ if (nrelids > maxrelids)
+ {
+ maxrelids *= 2;
+ logrelids = repalloc(logrelids, maxrelids * sizeof(Oid));
+ }
+ logrelids[nrelids - 1] = lfirst_oid(cell);
+ }
+
+ foreach (cell, seq_relids_logged)
+ {
+ nseqrelids++;
+ if ((nrelids + nseqrelids) > maxrelids)
+ {
+ maxrelids *= 2;
+ logrelids = repalloc(logrelids, maxrelids * sizeof(Oid));
+ }
+ logrelids[nrelids + nseqrelids - 1] = lfirst_oid(cell);
+ }
+
+ if ((nrelids + nseqrelids) > 0)
+ {
+ xl_heap_truncate xlrec;
+
+ xlrec.dbId = MyDatabaseId;
+ xlrec.nrelids = nrelids;
+ xlrec.nseqrelids = nseqrelids;
+ xlrec.flags = 0;
+ if (behavior == DROP_CASCADE)
+ xlrec.flags |= XLH_TRUNCATE_CASCADE;
+ if (restart_seqs)
+ xlrec.flags |= XLH_TRUNCATE_RESTART_SEQS;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfHeapTruncate);
+ XLogRegisterData((char *) logrelids,
+ (nrelids + nseqrelids) * sizeof(Oid));
+
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ (void) XLogInsert(RM_HEAP_ID, XLOG_HEAP_TRUNCATE);
+ }
+
/*
* Process all AFTER STATEMENT TRUNCATE triggers.
*/
diff --git a/src/backend/replication/loindex 537eba7875..d8408e154c 100644
*** a/src/backend/replication/logical/decode.c
--- b/src/backend/replication/logical/decode.c
***************
*** 65,70 **** static void DecodeLogicalMsgOp(LogicalDecodingContext *ctx, XLogRecordBuffer *bu
--- 65,71 ----
static void DecodeInsert(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeUpdate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeDelete(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
+ static void DecodeTruncate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeMultiInsert(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeSpecConfirm(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
***************
*** 435,440 **** DecodeHeapOp(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
--- 436,446 ----
DecodeDelete(ctx, buf);
break;
+ case XLOG_HEAP_TRUNCATE:
+ if (SnapBuildProcessChange(builder, xid, buf->origptr))
+ DecodeTruncate(ctx, buf);
+ break;
+
case XLOG_HEAP_INPLACE:
/*
***************
*** 805,810 **** DecodeDelete(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
--- 811,856 ----
ReorderBufferQueueChange(ctx->reorder, XLogRecGetXid(r), buf->origptr, change);
}
+ /*
+ * Parse XLOG_HEAP_TRUNCATE from wal
+ */
+ static void
+ DecodeTruncate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+ {
+ XLogReaderState *r = buf->record;
+ xl_heap_truncate *xlrec;
+ ReorderBufferChange *change;
+ int i;
+
+ xlrec = (xl_heap_truncate *) XLogRecGetData(r);
+
+ /* only interested in our database */
+ if (xlrec->dbId != ctx->slot->data.database)
+ return;
+
+ /* output plugin doesn't look for this origin, no need to queue */
+ if (FilterByOrigin(ctx, XLogRecGetOrigin(r)))
+ return;
+
+ change = ReorderBufferGetChange(ctx->reorder);
+ change->action = REORDER_BUFFER_CHANGE_TRUNCATE;
+ change->origin_id = XLogRecGetOrigin(r);
+ if (xlrec->flags & XLH_TRUNCATE_CASCADE)
+ change->data.truncate_msg.cascade = true;
+ if (xlrec->flags & XLH_TRUNCATE_RESTART_SEQS)
+ change->data.truncate_msg.restart_seqs = true;
+
+ /*
+ * Queue up one change per relation, ignoring sequences for now
+ */
+ for (i = 0; i < xlrec->nrelids; i++)
+ {
+ change->data.truncate_msg.relid = xlrec->relids[i];
+ ReorderBufferQueueChange(ctx->reorder, XLogRecGetXid(r),
+ buf->origptr, change);
+ }
+ }
+
/*
* Decode XLOG_HEAP2_MULTI_INSERT_insert record into multiple tuplebufs.
*
diff --git a/src/backend/replication/logical/prindex 948343e4ae..2fa6f8393d 100644
*** a/src/backend/replication/logical/proto.c
--- b/src/backend/replication/logical/proto.c
***************
*** 26,31 ****
--- 26,34 ----
*/
#define LOGICALREP_IS_REPLICA_IDENTITY 1
+ #define TRUNCATE_CASCADE (1<<0)
+ #define TRUNCATE_RESTART_SEQS (1<<1)
+
static void logicalrep_write_attrs(StringInfo out, Relation rel);
static void logicalrep_write_tuple(StringInfo out, Relation rel,
HeapTuple tuple);
***************
*** 292,297 **** logicalrep_read_delete(StringInfo in, LogicalRepTupleData *oldtup)
--- 295,342 ----
return relid;
}
+ /*
+ * Write TRUNCATE to the output stream.
+ */
+ void
+ logicalrep_write_truncate(StringInfo out, Relation rel,
+ bool cascade, bool restart_seqs)
+ {
+ uint8 flags = 0;
+
+ pq_sendbyte(out, 'T'); /* action TRUNCATE */
+
+ /* use Oid as relation identifier */
+ pq_sendint32(out, RelationGetRelid(rel));
+
+ /* encode and send truncate flags */
+ if (cascade)
+ flags |= TRUNCATE_CASCADE;
+ if (restart_seqs)
+ flags |= TRUNCATE_RESTART_SEQS;
+ pq_sendint8(out, flags);
+ }
+
+ /*
+ * Read TRUNCATE from stream.
+ */
+ LogicalRepRelId
+ logicalrep_read_truncate(StringInfo in, bool *cascade, bool *restart_seqs)
+ {
+ LogicalRepRelId relid;
+ uint8 flags;
+
+ /* read the relation id */
+ relid = pq_getmsgint(in, 4);
+
+ /* read and decode truncate flags */
+ flags = pq_getmsgint(in, 1);
+ *cascade = (flags & TRUNCATE_CASCADE) > 0;
+ *restart_seqs = (flags & TRUNCATE_RESTART_SEQS) > 0;
+
+ return relid;
+ }
+
/*
* Write relation description to the output stream.
*/
diff --git a/src/backend/replication/logical/rindex 1208da2972..fc7350358a 100644
*** a/src/backend/replication/logical/reorderbuffer.c
--- b/src/backend/replication/logical/reorderbuffer.c
***************
*** 403,408 **** ReorderBufferReturnChange(ReorderBuffer *rb, ReorderBufferChange *change)
--- 403,410 ----
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ break;
}
pfree(change);
***************
*** 1342,1347 **** ReorderBufferCommit(ReorderBuffer *rb, TransactionId xid,
--- 1344,1356 ----
switch (change->action)
{
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ reloid = change->data.truncate_msg.relid;
+ relation = RelationIdGetRelation(reloid);
+ rb->apply_change(rb, txn, relation, change);
+ RelationClose(relation);
+ break;
+
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
/*
***************
*** 2239,2244 **** ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
--- 2248,2254 ----
}
break;
}
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
***************
*** 2524,2529 **** ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
--- 2534,2540 ----
break;
}
/* the base struct contains all the data, easy peasy */
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
diff --git a/src/backend/replication/logical/worker.c index 83c69092ae..a31b859cb8 100644
*** a/src/backend/replication/logical/worker.c
--- b/src/backend/replication/logical/worker.c
***************
*** 869,874 **** apply_handle_delete(StringInfo s)
--- 869,922 ----
CommandCounterIncrement();
}
+ /*
+ * Handle TRUNCATE message.
+ *
+ * TODO: FDW support
+ */
+ static void
+ apply_handle_truncate(StringInfo s)
+ {
+ LogicalRepRelMapEntry *rel;
+ LogicalRepRelId relid;
+ bool cascade = false;
+ bool restart_seqs = false;
+ List *rels = NIL;
+ List *relids = NIL;
+
+ ensure_transaction();
+
+ relid = logicalrep_read_truncate(s, &cascade, &restart_seqs);
+ rel = logicalrep_rel_open(relid, RowExclusiveLock);
+ if (!should_apply_changes_for_rel(rel))
+ {
+ /*
+ * The relation can't become interesting in the middle of the
+ * transaction so it's safe to unlock it.
+ */
+ logicalrep_rel_close(rel, RowExclusiveLock);
+ return;
+ }
+
+ /* Check if we can do the truncate. */
+ check_relation_updatable(rel);
+
+ rels = lappend(rels, rel->localrel);
+ relids = lappend_oid(relids, rel->localreloid);
+
+ ExecuteTruncateGuts(rels, relids, NULL,
+ cascade ? DROP_CASCADE : DROP_RESTRICT,
+ restart_seqs);
+
+ /* logicalrep_rel_close call not needed, because ExecuteTruncateGuts
+ * already closes the relations. Setting localrel to NULL in the map entry
+ * is still needed.
+ */
+ rel->localrel = NULL;
+
+ CommandCounterIncrement();
+ }
+
/*
* Logical replication protocol message dispatcher.
***************
*** 900,905 **** apply_dispatch(StringInfo s)
--- 948,957 ----
case 'D':
apply_handle_delete(s);
break;
+ /* TRUNCATE */
+ case 'T':
+ apply_handle_truncate(s);
+ break;
/* RELATION */
case 'R':
apply_handle_relation(s);
diff --git a/src/backend/replication/pgoutput/pindex 40a1ef3c1d..7fbd0a3dca 100644
*** a/src/backend/replication/pgoutput/pgoutput.c
--- b/src/backend/replication/pgoutput/pgoutput.c
***************
*** 275,280 **** pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
--- 275,281 ----
if (!relentry->pubactions.pubupdate)
return;
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
case REORDER_BUFFER_CHANGE_DELETE:
if (!relentry->pubactions.pubdelete)
return;
***************
*** 352,357 **** pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
--- 353,367 ----
else
elog(DEBUG1, "didn't send DELETE change because of missing oldtuple");
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ {
+ OutputPluginPrepareWrite(ctx, true);
+ logicalrep_write_truncate(ctx->out, relation,
+ change->data.truncate_msg.cascade,
+ change->data.truncate_msg.restart_seqs);
+ OutputPluginWrite(ctx, true);
+ }
+ break;
default:
Assert(false);
}
diff --git a/src/include/access/heapam_xlog.h b/srindex 700e25c36a..c56b200f7d 100644
*** a/src/include/access/heapam_xlog.h
--- b/src/include/access/heapam_xlog.h
***************
*** 32,38 ****
#define XLOG_HEAP_INSERT 0x00
#define XLOG_HEAP_DELETE 0x10
#define XLOG_HEAP_UPDATE 0x20
! /* 0x030 is free, was XLOG_HEAP_MOVE */
#define XLOG_HEAP_HOT_UPDATE 0x40
#define XLOG_HEAP_CONFIRM 0x50
#define XLOG_HEAP_LOCK 0x60
--- 32,38 ----
#define XLOG_HEAP_INSERT 0x00
#define XLOG_HEAP_DELETE 0x10
#define XLOG_HEAP_UPDATE 0x20
! #define XLOG_HEAP_TRUNCATE 0x30
#define XLOG_HEAP_HOT_UPDATE 0x40
#define XLOG_HEAP_CONFIRM 0x50
#define XLOG_HEAP_LOCK 0x60
***************
*** 109,114 **** typedef struct xl_heap_delete
--- 109,136 ----
#define SizeOfHeapDelete (offsetof(xl_heap_delete, flags) + sizeof(uint8))
+ /*
+ * xl_heap_delete flag values, 8 bits are available.
+ */
+ #define XLH_TRUNCATE_CASCADE (1<<0)
+ #define XLH_TRUNCATE_RESTART_SEQS (1<<1)
+
+ /*
+ * For truncate we list all truncated relids in an array, followed by all
+ * sequence relids that need to be restarted, if any.
+ * All rels are always within the same database, so we just list dbid once.
+ */
+ typedef struct xl_heap_truncate
+ {
+ Oid dbId;
+ uint32 nrelids;
+ uint32 nseqrelids;
+ uint8 flags;
+ Oid relids[FLEXIBLE_ARRAY_MEMBER];
+ } xl_heap_truncate;
+
+ #define SizeOfHeapTruncate (offsetof(xl_heap_truncate, relids))
+
/*
* We don't store the whole fixed part (HeapTupleHeaderData) of an inserted
* or updated tuple in WAL; we can save a few bytes by reconstructing the
diff --git a/src/include/executor/execuindex a782fae0f8..62ed488442 100644
*** a/src/include/executor/executor.h
--- b/src/include/executor/executor.h
***************
*** 556,559 **** extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
--- 556,565 ----
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
+ /*
+ * prototype for tablecmds.c accessible for logical apply
+ */
+ void ExecuteTruncateGuts(List *rels, List *relids, List *relids_logged,
+ DropBehavior behavior, bool restart_seqs);
+
#endif /* EXECUTOR_H */
diff --git a/src/include/replication/lindex 0eb21057c5..b1abb9e36f 100644
*** a/src/include/replication/logicalproto.h
--- b/src/include/replication/logicalproto.h
***************
*** 98,103 **** extern void logicalrep_write_delete(StringInfo out, Relation rel,
--- 98,107 ----
HeapTuple oldtuple);
extern LogicalRepRelId logicalrep_read_delete(StringInfo in,
LogicalRepTupleData *oldtup);
+ extern void logicalrep_write_truncate(StringInfo out, Relation rel,
+ bool cascade, bool restart_seqs);
+ extern LogicalRepRelId logicalrep_read_truncate(StringInfo in,
+ bool *cascade, bool *restart_seqs);
extern void logicalrep_write_rel(StringInfo out, Relation rel);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
extern void logicalrep_write_typ(StringInfo out, Oid typoid);
diff --git a/src/include/replication/reorderbindex 0970abca52..e8439fac49 100644
*** a/src/include/replication/reorderbuffer.h
--- b/src/include/replication/reorderbuffer.h
***************
*** 59,65 **** enum ReorderBufferChangeType
REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID,
REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID,
REORDER_BUFFER_CHANGE_INTERNAL_SPEC_INSERT,
! REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM
};
/*
--- 59,66 ----
REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID,
REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID,
REORDER_BUFFER_CHANGE_INTERNAL_SPEC_INSERT,
! REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM,
! REORDER_BUFFER_CHANGE_TRUNCATE
};
/*
***************
*** 128,134 **** typedef struct ReorderBufferChange
CommandId cmax;
CommandId combocid;
} tuplecid;
! } data;
/*
* While in use this is how a change is linked into a transactions,
--- 129,146 ----
CommandId cmax;
CommandId combocid;
} tuplecid;
!
! /*
! * Truncate data for REORDER_BUFFER_CHANGE_TRUNCATE representing
! * one relation to be truncated.
! */
! struct
! {
! Oid relid;
! bool cascade;
! bool restart_seqs;
! } truncate_msg;
! } data;
/*
* While in use this is how a change is linked into a transactions,
diff --git a/src/test/subscription/t/001_rep_cindex 0136c79d4b..974e88821c 100644
*** a/src/test/subscription/t/001_rep_changes.pl
--- b/src/test/subscription/t/001_rep_changes.pl
***************
*** 3,9 **** use strict;
use warnings;
use PostgresNode;
use TestLib;
! use Test::More tests => 16;
# Initialize publisher node
my $node_publisher = get_new_node('publisher');
--- 3,9 ----
use warnings;
use PostgresNode;
use TestLib;
! use Test::More tests => 24;
# Initialize publisher node
my $node_publisher = get_new_node('publisher');
***************
*** 189,199 **** $result = $node_subscriber->safe_psql('postgres',
is($result, qq(20|-20|-1),
'check changes skipped after subscription publication change');
# check alter publication (relcache invalidation etc)
$node_publisher->safe_psql('postgres',
"ALTER PUBLICATION tap_pub_ins_only SET (publish = 'insert, delete')");
$node_publisher->safe_psql('postgres',
! "ALTER PUBLICATION tap_pub_ins_only ADD TABLE tab_full");
$node_publisher->safe_psql('postgres', "DELETE FROM tab_ins WHERE a > 0");
$node_subscriber->safe_psql('postgres',
"ALTER SUBSCRIPTION tap_sub REFRESH PUBLICATION WITH (copy_data = false)"
--- 189,210 ----
is($result, qq(20|-20|-1),
'check changes skipped after subscription publication change');
+ # truncate should not be replicated
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep");
+
+ $node_publisher->poll_query_until('postgres', $caughtup_query)
+ or die "Timed out while waiting for subscriber to catch up";
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_rep");
+ is($result, qq(20|-20|-1),
+ 'check changes skipped after subscription publication change');
+
# check alter publication (relcache invalidation etc)
$node_publisher->safe_psql('postgres',
"ALTER PUBLICATION tap_pub_ins_only SET (publish = 'insert, delete')");
$node_publisher->safe_psql('postgres',
! "ALTER PUBLICATION tap_pub_ins_only ADD TABLE tab_full, tab_rep");
$node_publisher->safe_psql('postgres', "DELETE FROM tab_ins WHERE a > 0");
$node_subscriber->safe_psql('postgres',
"ALTER SUBSCRIPTION tap_sub REFRESH PUBLICATION WITH (copy_data = false)"
***************
*** 213,218 **** $result = $node_subscriber->safe_psql('postgres',
--- 224,295 ----
"SELECT count(*), min(a), max(a) FROM tab_full");
is($result, qq(21|0|100), 'check replicated insert after alter publication');
+ # add a sequence and a foreign key to check cascade and restart
+ # identity truncate options
+ $node_subscriber->safe_psql('postgres',
+ "CREATE TABLE tab_notrep_fk (a int REFERENCES tab_rep(a))"
+ );
+ $node_subscriber->safe_psql('postgres', "INSERT INTO tab_notrep_fk VALUES (-1)");
+ $node_subscriber->safe_psql('postgres',
+ "CREATE SEQUENCE seq_notrep OWNED BY tab_rep.a"
+ );
+ $node_subscriber->safe_psql('postgres',
+ "ALTER SEQUENCE seq_notrep START 101"
+ );
+
+ # truncate should now be replicated
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep");
+
+ $node_publisher->poll_query_until('postgres', $caughtup_query)
+ or die "Timed out while waiting for subscriber to catch up";
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_rep");
+ is($result, qq(0||),
+ 'check replicated truncate after alter publication');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_notrep_fk");
+ is($result, qq(1|-1|-1),
+ 'check replicated truncate does not cascade');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq_notrep')");
+ is($result, qq(1),
+ 'check replicated truncate does not restart identities');
+
+ # should restart identity
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep RESTART IDENTITY");
+
+ $node_publisher->poll_query_until('postgres', $caughtup_query)
+ or die "Timed out while waiting for subscriber to catch up";
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_notrep_fk");
+ is($result, qq(1|-1|-1),
+ 'check replicated truncate does not cascade');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq_notrep')");
+ is($result, qq(101),
+ 'check replicated truncate restart identities');
+
+ # should cascade
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep CASCADE");
+
+ $node_publisher->poll_query_until('postgres', $caughtup_query)
+ or die "Timed out while waiting for subscriber to catch up";
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_notrep_fk");
+ is($result, qq(0||),
+ 'check replicated truncate cascade');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq_notrep')");
+ is($result, qq(102),
+ 'check replicated truncate does not restart identities');
+
# check restart on rename
$oldpid = $node_publisher->safe_psql('postgres',
"SELECT pid FROM pg_stat_replication WHERE application_name = '$appname';"
Re: [PATCH] Logical decoding of TRUNCATE
Hi all,
After commit bbd3363e128daec0e70952c1bb2f12ab1f6f1292 that refactor
subscription tests to use PostgresNode's wait_for_catchup, the patch
needs to be updated to use wait_for_catchup.
Attached there is the updated patch. is discussed in a separate thread
https://commitfest.postgresql.org/16/1447/, but I've posted it also here
to ease the review of the 0002.
Regards,
Marco
--
Marco Nenciarini - 2ndQuadrant Italy
PostgreSQL Training, Services and Support
marco.nenciarini@2ndQuadrant.it | www.2ndQuadrant.it
Attachments:
0001-truncate_ignore_fks_role_replica.v2.patchtext/plain; charset=UTF-8; name=0001-truncate_ignore_fks_role_replica.v2.patch; x-mac-creator=0; x-mac-type=0Download
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index e4a01699e4..aff56891e6 100644
*** a/doc/src/sgml/config.sgml
--- b/doc/src/sgml/config.sgml
***************
*** 6502,6508 **** COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
<listitem>
<para>
Controls firing of replication-related triggers and rules for the
! current session. Setting this variable requires
superuser privilege and results in discarding any previously cached
query plans. Possible values are <literal>origin</literal> (the default),
<literal>replica</literal> and <literal>local</literal>.
--- 6502,6510 ----
<listitem>
<para>
Controls firing of replication-related triggers and rules for the
! current session. When set to <literal>replica</literal> it also
! disables all the foreign key checks, which can leave the data in an
! inconsistent state if improperly used. Setting this variable requires
superuser privilege and results in discarding any previously cached
query plans. Possible values are <literal>origin</literal> (the default),
<literal>replica</literal> and <literal>local</literal>.
diff --git a/src/backend/commanindex f2a928b823..180ebd0717 100644
*** a/src/backend/commands/tablecmds.c
--- b/src/backend/commands/tablecmds.c
***************
*** 1340,1355 **** ExecuteTruncate(TruncateStmt *stmt)
}
/*
! * Check foreign key references. In CASCADE mode, this should be
! * unnecessary since we just pulled in all the references; but as a
! * cross-check, do it anyway if in an Assert-enabled build.
*/
#ifdef USE_ASSERT_CHECKING
- heap_truncate_check_FKs(rels, false);
- #else
- if (stmt->behavior == DROP_RESTRICT)
heap_truncate_check_FKs(rels, false);
#endif
/*
* If we are asked to restart sequences, find all the sequences, lock them
--- 1340,1363 ----
}
/*
! * Suppress foreign key references check if session replication role is
! * set to REPLICA.
*/
+ if (SessionReplicationRole != SESSION_REPLICATION_ROLE_REPLICA)
+ {
+
+ /*
+ * Check foreign key references. In CASCADE mode, this should be
+ * unnecessary since we just pulled in all the references; but as a
+ * cross-check, do it anyway if in an Assert-enabled build.
+ */
#ifdef USE_ASSERT_CHECKING
heap_truncate_check_FKs(rels, false);
+ #else
+ if (stmt->behavior == DROP_RESTRICT)
+ heap_truncate_check_FKs(rels, false);
#endif
+ }
/*
* If we are asked to restart sequences, find all the sequences, lock them
0002-logical_decode_truncate.v13.patchtext/plain; charset=UTF-8; name=0002-logical_decode_truncate.v13.patch; x-mac-creator=0; x-mac-type=0Download
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 1e22c1eefc..d1feea4909 100644
*** a/contrib/test_decoding/expected/ddl.out
--- b/contrib/test_decoding/expected/ddl.out
***************
*** 543,548 **** UPDATE table_with_unique_not_null SET data = 3 WHERE data = 2;
--- 543,550 ----
UPDATE table_with_unique_not_null SET id = -id;
UPDATE table_with_unique_not_null SET id = -id;
DELETE FROM table_with_unique_not_null WHERE data = 3;
+ TRUNCATE table_with_unique_not_null;
+ TRUNCATE table_with_unique_not_null, table_with_unique_not_null RESTART IDENTITY CASCADE;
-- check toast support
BEGIN;
CREATE SEQUENCE toasttable_rand_seq START 79 INCREMENT 1499; -- portable "random"
***************
*** 660,665 **** SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
--- 662,673 ----
table public.table_with_unique_not_null: DELETE: id[integer]:4
COMMIT
BEGIN
+ table public.table_with_unique_not_null: TRUNCATE: (no-flags)
+ COMMIT
+ BEGIN
+ table public.table_with_unique_not_null: TRUNCATE: restart_seqs cascade
+ COMMIT
+ BEGIN
table public.toasttable: INSERT: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
BEGIN
***************
*** 668,674 **** SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
BEGIN
table public.toasttable: UPDATE: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
! (103 rows)
INSERT INTO toasttable(toasted_col1) SELECT string_agg(g.i::text, '') FROM generate_series(1, 2000) g(i);
-- update of second column, first column unchanged
--- 676,682 ----
BEGIN
table public.toasttable: UPDATE: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
! (109 rows)
INSERT INTO toasttable(toasted_col1) SELECT string_agg(g.i::text, '') FROM generate_series(1, 2000) g(i);
-- update of second column, first column unchanged
diff --git a/contrib/test_decoding/sql/ddl.sqindex 057dae056b..0fb4102db8 100644
*** a/contrib/test_decoding/sql/ddl.sql
--- b/contrib/test_decoding/sql/ddl.sql
***************
*** 333,338 **** UPDATE table_with_unique_not_null SET id = -id;
--- 333,341 ----
UPDATE table_with_unique_not_null SET id = -id;
DELETE FROM table_with_unique_not_null WHERE data = 3;
+ TRUNCATE table_with_unique_not_null;
+ TRUNCATE table_with_unique_not_null, table_with_unique_not_null RESTART IDENTITY CASCADE;
+
-- check toast support
BEGIN;
CREATE SEQUENCE toasttable_rand_seq START 79 INCREMENT 1499; -- portable "random"
diff --git a/contrib/test_decoding/test_index 0f18afa852..55c4593ed5 100644
*** a/contrib/test_decoding/test_decoding.c
--- b/contrib/test_decoding/test_decoding.c
***************
*** 466,471 **** pg_decode_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
--- 466,485 ----
&change->data.tp.oldtuple->tuple,
true);
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ appendStringInfoString(ctx->out, " TRUNCATE:");
+
+ if (change->data.truncate_msg.restart_seqs
+ || change->data.truncate_msg.cascade)
+ {
+ if (change->data.truncate_msg.restart_seqs)
+ appendStringInfo(ctx->out, " restart_seqs");
+ if (change->data.truncate_msg.cascade)
+ appendStringInfo(ctx->out, " cascade");
+ }
+ else
+ appendStringInfoString(ctx->out, " (no-flags)");
+ break;
default:
Assert(false);
}
diff --git a/doc/src/sgml/logical-replicatioindex 75551d8ee1..6b64c8ef76 100644
*** a/doc/src/sgml/logical-replication.sgml
--- b/doc/src/sgml/logical-replication.sgml
***************
*** 111,116 ****
--- 111,118 ----
any combination of <command>INSERT</command>, <command>UPDATE</command>, and
<command>DELETE</command>, similar to how triggers are fired by
particular event types. By default, all operation types are replicated.
+ <command>TRUNCATE</command> is treated as a form of <command>DELETE</command>
+ for the purpose of deciding whether to publish, or not.
</para>
<para>
***************
*** 364,378 ****
</para>
</listitem>
- <listitem>
- <para>
- <command>TRUNCATE</command> commands are not replicated. This can, of
- course, be worked around by using <command>DELETE</command> instead. To
- avoid accidental <command>TRUNCATE</command> invocations, you can revoke
- the <literal>TRUNCATE</literal> privilege from tables.
- </para>
- </listitem>
-
<listitem>
<para>
Large objects (see <xref linkend="largeobjects"/>) are not replicated.
--- 366,371 ----
diff --git a/doc/src/sgml/protocol.sgml b/doindex 4c5ed1e6d6..d2a0d7e52e 100644
*** a/doc/src/sgml/protocol.sgml
--- b/doc/src/sgml/protocol.sgml
***************
*** 6838,6843 **** TupleData
--- 6838,6889 ----
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ Truncate
+ </term>
+ <listitem>
+ <para>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ Byte1('T')
+ </term>
+ <listitem>
+ <para>
+ Identifies the message as a truncate message.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ Int32
+ </term>
+ <listitem>
+ <para>
+ ID of the relation corresponding to the ID in the relation
+ message.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ Int8
+ </term>
+ <listitem>
+ <para>
+ Option flags for truncate. Currently can be 0 for no flags,
+ 1 for CASCADE, 2 for RESTART IDENTITY and 3 if both are set.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect1>
diff --git a/doc/src/sgml/ref/creindex bfe12d5f41..c5032938a7 100644
*** a/doc/src/sgml/ref/create_publication.sgml
--- b/doc/src/sgml/ref/create_publication.sgml
***************
*** 111,116 **** CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
--- 111,121 ----
and so the default value for this option is
<literal>'insert, update, delete'</literal>.
</para>
+ <para>
+ <command>TRUNCATE</command> is treated as a form of
+ <command>DELETE</command> for the purpose of deciding whether
+ to publish, or not.
+ </para>
</listitem>
</varlistentry>
</variablelist>
***************
*** 168,175 **** CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
</para>
<para>
! <command>TRUNCATE</command> and <acronym>DDL</acronym> operations
! are not published.
</para>
</refsect1>
--- 173,179 ----
</para>
<para>
! <acronym>DDL</acronym> operations are not published.
</para>
</refsect1>
diff --git a/src/backend/access/heap/heapam.c bindex dbc8f2d6c7..f4c6eaf57f 100644
*** a/src/backend/access/heap/heapam.c
--- b/src/backend/access/heap/heapam.c
***************
*** 9134,9139 **** heap_redo(XLogReaderState *record)
--- 9134,9146 ----
case XLOG_HEAP_UPDATE:
heap_xlog_update(record, false);
break;
+ case XLOG_HEAP_TRUNCATE:
+ /*
+ * TRUNCATE is a no-op because the actions are already logged
+ * as SMGR WAL records. TRUNCATE WAL record only exists to allow
+ * it to be logically decoded precisely.
+ */
+ break;
case XLOG_HEAP_HOT_UPDATE:
heap_xlog_update(record, true);
break;
diff --git a/src/backend/access/rmgrdesindex b00c071cb6..15550cdff0 100644
*** a/src/backend/access/rmgrdesc/heapdesc.c
--- b/src/backend/access/rmgrdesc/heapdesc.c
***************
*** 75,80 **** heap_desc(StringInfo buf, XLogReaderState *record)
--- 75,93 ----
xlrec->new_offnum,
xlrec->new_xmax);
}
+ else if (info == XLOG_HEAP_TRUNCATE)
+ {
+ xl_heap_truncate *xlrec = (xl_heap_truncate *) rec;
+
+ if (xlrec->flags & XLH_TRUNCATE_CASCADE)
+ appendStringInfo(buf, "cascade ");
+ if (xlrec->flags & XLH_TRUNCATE_RESTART_SEQS)
+ appendStringInfo(buf, "restart_seqs ");
+ appendStringInfo(buf, "nrelids %u nseqrelids %u",
+ xlrec->nrelids,
+ xlrec->nseqrelids);
+ /* Skip the list of relids and seqrelids */
+ }
else if (info == XLOG_HEAP_CONFIRM)
{
xl_heap_confirm *xlrec = (xl_heap_confirm *) rec;
***************
*** 186,191 **** heap_identify(uint8 info)
--- 199,207 ----
case XLOG_HEAP_HOT_UPDATE | XLOG_HEAP_INIT_PAGE:
id = "HOT_UPDATE+INIT";
break;
+ case XLOG_HEAP_TRUNCATE:
+ id = "TRUNCATE";
+ break;
case XLOG_HEAP_CONFIRM:
id = "HEAP_CONFIRM";
break;
diff --git a/src/backend/commands/tablecmds.cindex 180ebd0717..8355c4c5c7 100644
*** a/src/backend/commands/tablecmds.c
--- b/src/backend/commands/tablecmds.c
***************
*** 16,21 ****
--- 16,22 ----
#include "access/genam.h"
#include "access/heapam.h"
+ #include "access/heapam_xlog.h"
#include "access/multixact.h"
#include "access/reloptions.h"
#include "access/relscan.h"
***************
*** 1248,1258 **** ExecuteTruncate(TruncateStmt *stmt)
{
List *rels = NIL;
List *relids = NIL;
! List *seq_relids = NIL;
! EState *estate;
! ResultRelInfo *resultRelInfos;
! ResultRelInfo *resultRelInfo;
! SubTransactionId mySubid;
ListCell *cell;
/*
--- 1249,1255 ----
{
List *rels = NIL;
List *relids = NIL;
! List *relids_logged = NIL;
ListCell *cell;
/*
***************
*** 1276,1281 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1273,1281 ----
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, myrelid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, myrelid);
if (recurse)
{
***************
*** 1296,1301 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1296,1304 ----
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, childrelid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, childrelid);
}
}
else if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
***************
*** 1305,1311 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1308,1336 ----
errhint("Do not specify the ONLY keyword, or use truncate only on the partitions directly.")));
}
+ ExecuteTruncateGuts(rels, relids, relids_logged,
+ stmt->behavior, stmt->restart_seqs);
+ }
+
+ void
+ ExecuteTruncateGuts(List *rels, List *relids, List *relids_logged,
+ DropBehavior behavior, bool restart_seqs)
+ {
+ List *seq_relids = NIL;
+ EState *estate;
+ ResultRelInfo *resultRelInfos;
+ ResultRelInfo *resultRelInfo;
+ SubTransactionId mySubid;
+ ListCell *cell;
+ List *seq_relids_logged = NIL;
+ uint32 nrelids = 0;
+ uint32 nseqrelids = 0;
+ uint32 maxrelids = 2;
+ Oid *logrelids = NULL;
+
/*
+ * Open, exclusive-lock, and check all the explicitly-specified relations
+ *
* In CASCADE mode, suck in all referencing relations as well. This
* requires multiple iterations to find indirectly-dependent relations. At
* each phase, we need to exclusive-lock new rels before looking for their
***************
*** 1313,1319 **** ExecuteTruncate(TruncateStmt *stmt)
* soon as we open it, to avoid a faux pas such as holding lock for a long
* time on a rel we have no permissions for.
*/
! if (stmt->behavior == DROP_CASCADE)
{
for (;;)
{
--- 1338,1344 ----
* soon as we open it, to avoid a faux pas such as holding lock for a long
* time on a rel we have no permissions for.
*/
! if (behavior == DROP_CASCADE)
{
for (;;)
{
***************
*** 1335,1340 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1360,1368 ----
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, relid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, relid);
}
}
}
***************
*** 1354,1360 **** ExecuteTruncate(TruncateStmt *stmt)
#ifdef USE_ASSERT_CHECKING
heap_truncate_check_FKs(rels, false);
#else
! if (stmt->behavior == DROP_RESTRICT)
heap_truncate_check_FKs(rels, false);
#endif
}
--- 1382,1388 ----
#ifdef USE_ASSERT_CHECKING
heap_truncate_check_FKs(rels, false);
#else
! if (behavior == DROP_RESTRICT)
heap_truncate_check_FKs(rels, false);
#endif
}
***************
*** 1365,1371 **** ExecuteTruncate(TruncateStmt *stmt)
* We want to do this early since it's pointless to do all the truncation
* work only to fail on sequence permissions.
*/
! if (stmt->restart_seqs)
{
foreach(cell, rels)
{
--- 1393,1399 ----
* We want to do this early since it's pointless to do all the truncation
* work only to fail on sequence permissions.
*/
! if (restart_seqs)
{
foreach(cell, rels)
{
***************
*** 1386,1391 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1414,1423 ----
RelationGetRelationName(seq_rel));
seq_relids = lappend_oid(seq_relids, seq_relid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(seq_rel))
+ seq_relids_logged = lappend_oid(seq_relids_logged,
+ seq_relid);
relation_close(seq_rel, NoLock);
}
***************
*** 1520,1525 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1552,1611 ----
ResetSequence(seq_relid);
}
+ /*
+ * Write a WAL record to allow this set of actions to be logically decoded.
+ * We could optimize this away when !RelationIsLogicallyLogged(rel)
+ * but that doesn't save much space or time.
+ *
+ * Assemble an array of relids, then an array of seqrelids so we can write
+ * a single WAL record for the whole action.
+ */
+ logrelids = palloc(maxrelids * sizeof(Oid));
+ foreach (cell, relids_logged)
+ {
+ nrelids++;
+ if (nrelids > maxrelids)
+ {
+ maxrelids *= 2;
+ logrelids = repalloc(logrelids, maxrelids * sizeof(Oid));
+ }
+ logrelids[nrelids - 1] = lfirst_oid(cell);
+ }
+
+ foreach (cell, seq_relids_logged)
+ {
+ nseqrelids++;
+ if ((nrelids + nseqrelids) > maxrelids)
+ {
+ maxrelids *= 2;
+ logrelids = repalloc(logrelids, maxrelids * sizeof(Oid));
+ }
+ logrelids[nrelids + nseqrelids - 1] = lfirst_oid(cell);
+ }
+
+ if ((nrelids + nseqrelids) > 0)
+ {
+ xl_heap_truncate xlrec;
+
+ xlrec.dbId = MyDatabaseId;
+ xlrec.nrelids = nrelids;
+ xlrec.nseqrelids = nseqrelids;
+ xlrec.flags = 0;
+ if (behavior == DROP_CASCADE)
+ xlrec.flags |= XLH_TRUNCATE_CASCADE;
+ if (restart_seqs)
+ xlrec.flags |= XLH_TRUNCATE_RESTART_SEQS;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfHeapTruncate);
+ XLogRegisterData((char *) logrelids,
+ (nrelids + nseqrelids) * sizeof(Oid));
+
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ (void) XLogInsert(RM_HEAP_ID, XLOG_HEAP_TRUNCATE);
+ }
+
/*
* Process all AFTER STATEMENT TRUNCATE triggers.
*/
diff --git a/src/backend/replication/loindex 537eba7875..d8408e154c 100644
*** a/src/backend/replication/logical/decode.c
--- b/src/backend/replication/logical/decode.c
***************
*** 65,70 **** static void DecodeLogicalMsgOp(LogicalDecodingContext *ctx, XLogRecordBuffer *bu
--- 65,71 ----
static void DecodeInsert(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeUpdate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeDelete(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
+ static void DecodeTruncate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeMultiInsert(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeSpecConfirm(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
***************
*** 435,440 **** DecodeHeapOp(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
--- 436,446 ----
DecodeDelete(ctx, buf);
break;
+ case XLOG_HEAP_TRUNCATE:
+ if (SnapBuildProcessChange(builder, xid, buf->origptr))
+ DecodeTruncate(ctx, buf);
+ break;
+
case XLOG_HEAP_INPLACE:
/*
***************
*** 805,810 **** DecodeDelete(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
--- 811,856 ----
ReorderBufferQueueChange(ctx->reorder, XLogRecGetXid(r), buf->origptr, change);
}
+ /*
+ * Parse XLOG_HEAP_TRUNCATE from wal
+ */
+ static void
+ DecodeTruncate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+ {
+ XLogReaderState *r = buf->record;
+ xl_heap_truncate *xlrec;
+ ReorderBufferChange *change;
+ int i;
+
+ xlrec = (xl_heap_truncate *) XLogRecGetData(r);
+
+ /* only interested in our database */
+ if (xlrec->dbId != ctx->slot->data.database)
+ return;
+
+ /* output plugin doesn't look for this origin, no need to queue */
+ if (FilterByOrigin(ctx, XLogRecGetOrigin(r)))
+ return;
+
+ change = ReorderBufferGetChange(ctx->reorder);
+ change->action = REORDER_BUFFER_CHANGE_TRUNCATE;
+ change->origin_id = XLogRecGetOrigin(r);
+ if (xlrec->flags & XLH_TRUNCATE_CASCADE)
+ change->data.truncate_msg.cascade = true;
+ if (xlrec->flags & XLH_TRUNCATE_RESTART_SEQS)
+ change->data.truncate_msg.restart_seqs = true;
+
+ /*
+ * Queue up one change per relation, ignoring sequences for now
+ */
+ for (i = 0; i < xlrec->nrelids; i++)
+ {
+ change->data.truncate_msg.relid = xlrec->relids[i];
+ ReorderBufferQueueChange(ctx->reorder, XLogRecGetXid(r),
+ buf->origptr, change);
+ }
+ }
+
/*
* Decode XLOG_HEAP2_MULTI_INSERT_insert record into multiple tuplebufs.
*
diff --git a/src/backend/replication/logical/prindex 948343e4ae..2fa6f8393d 100644
*** a/src/backend/replication/logical/proto.c
--- b/src/backend/replication/logical/proto.c
***************
*** 26,31 ****
--- 26,34 ----
*/
#define LOGICALREP_IS_REPLICA_IDENTITY 1
+ #define TRUNCATE_CASCADE (1<<0)
+ #define TRUNCATE_RESTART_SEQS (1<<1)
+
static void logicalrep_write_attrs(StringInfo out, Relation rel);
static void logicalrep_write_tuple(StringInfo out, Relation rel,
HeapTuple tuple);
***************
*** 292,297 **** logicalrep_read_delete(StringInfo in, LogicalRepTupleData *oldtup)
--- 295,342 ----
return relid;
}
+ /*
+ * Write TRUNCATE to the output stream.
+ */
+ void
+ logicalrep_write_truncate(StringInfo out, Relation rel,
+ bool cascade, bool restart_seqs)
+ {
+ uint8 flags = 0;
+
+ pq_sendbyte(out, 'T'); /* action TRUNCATE */
+
+ /* use Oid as relation identifier */
+ pq_sendint32(out, RelationGetRelid(rel));
+
+ /* encode and send truncate flags */
+ if (cascade)
+ flags |= TRUNCATE_CASCADE;
+ if (restart_seqs)
+ flags |= TRUNCATE_RESTART_SEQS;
+ pq_sendint8(out, flags);
+ }
+
+ /*
+ * Read TRUNCATE from stream.
+ */
+ LogicalRepRelId
+ logicalrep_read_truncate(StringInfo in, bool *cascade, bool *restart_seqs)
+ {
+ LogicalRepRelId relid;
+ uint8 flags;
+
+ /* read the relation id */
+ relid = pq_getmsgint(in, 4);
+
+ /* read and decode truncate flags */
+ flags = pq_getmsgint(in, 1);
+ *cascade = (flags & TRUNCATE_CASCADE) > 0;
+ *restart_seqs = (flags & TRUNCATE_RESTART_SEQS) > 0;
+
+ return relid;
+ }
+
/*
* Write relation description to the output stream.
*/
diff --git a/src/backend/replication/logical/rindex c72a611a39..f1d65c1457 100644
*** a/src/backend/replication/logical/reorderbuffer.c
--- b/src/backend/replication/logical/reorderbuffer.c
***************
*** 403,408 **** ReorderBufferReturnChange(ReorderBuffer *rb, ReorderBufferChange *change)
--- 403,410 ----
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ break;
}
pfree(change);
***************
*** 1342,1347 **** ReorderBufferCommit(ReorderBuffer *rb, TransactionId xid,
--- 1344,1356 ----
switch (change->action)
{
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ reloid = change->data.truncate_msg.relid;
+ relation = RelationIdGetRelation(reloid);
+ rb->apply_change(rb, txn, relation, change);
+ RelationClose(relation);
+ break;
+
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
/*
***************
*** 2239,2244 **** ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
--- 2248,2254 ----
}
break;
}
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
***************
*** 2524,2529 **** ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
--- 2534,2540 ----
break;
}
/* the base struct contains all the data, easy peasy */
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
diff --git a/src/backend/replication/logical/worker.c index 83c69092ae..a31b859cb8 100644
*** a/src/backend/replication/logical/worker.c
--- b/src/backend/replication/logical/worker.c
***************
*** 869,874 **** apply_handle_delete(StringInfo s)
--- 869,922 ----
CommandCounterIncrement();
}
+ /*
+ * Handle TRUNCATE message.
+ *
+ * TODO: FDW support
+ */
+ static void
+ apply_handle_truncate(StringInfo s)
+ {
+ LogicalRepRelMapEntry *rel;
+ LogicalRepRelId relid;
+ bool cascade = false;
+ bool restart_seqs = false;
+ List *rels = NIL;
+ List *relids = NIL;
+
+ ensure_transaction();
+
+ relid = logicalrep_read_truncate(s, &cascade, &restart_seqs);
+ rel = logicalrep_rel_open(relid, RowExclusiveLock);
+ if (!should_apply_changes_for_rel(rel))
+ {
+ /*
+ * The relation can't become interesting in the middle of the
+ * transaction so it's safe to unlock it.
+ */
+ logicalrep_rel_close(rel, RowExclusiveLock);
+ return;
+ }
+
+ /* Check if we can do the truncate. */
+ check_relation_updatable(rel);
+
+ rels = lappend(rels, rel->localrel);
+ relids = lappend_oid(relids, rel->localreloid);
+
+ ExecuteTruncateGuts(rels, relids, NULL,
+ cascade ? DROP_CASCADE : DROP_RESTRICT,
+ restart_seqs);
+
+ /* logicalrep_rel_close call not needed, because ExecuteTruncateGuts
+ * already closes the relations. Setting localrel to NULL in the map entry
+ * is still needed.
+ */
+ rel->localrel = NULL;
+
+ CommandCounterIncrement();
+ }
+
/*
* Logical replication protocol message dispatcher.
***************
*** 900,905 **** apply_dispatch(StringInfo s)
--- 948,957 ----
case 'D':
apply_handle_delete(s);
break;
+ /* TRUNCATE */
+ case 'T':
+ apply_handle_truncate(s);
+ break;
/* RELATION */
case 'R':
apply_handle_relation(s);
diff --git a/src/backend/replication/pgoutput/pindex 40a1ef3c1d..7fbd0a3dca 100644
*** a/src/backend/replication/pgoutput/pgoutput.c
--- b/src/backend/replication/pgoutput/pgoutput.c
***************
*** 275,280 **** pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
--- 275,281 ----
if (!relentry->pubactions.pubupdate)
return;
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
case REORDER_BUFFER_CHANGE_DELETE:
if (!relentry->pubactions.pubdelete)
return;
***************
*** 352,357 **** pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
--- 353,367 ----
else
elog(DEBUG1, "didn't send DELETE change because of missing oldtuple");
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ {
+ OutputPluginPrepareWrite(ctx, true);
+ logicalrep_write_truncate(ctx->out, relation,
+ change->data.truncate_msg.cascade,
+ change->data.truncate_msg.restart_seqs);
+ OutputPluginWrite(ctx, true);
+ }
+ break;
default:
Assert(false);
}
diff --git a/src/include/access/heapam_xlog.h b/srindex 700e25c36a..c56b200f7d 100644
*** a/src/include/access/heapam_xlog.h
--- b/src/include/access/heapam_xlog.h
***************
*** 32,38 ****
#define XLOG_HEAP_INSERT 0x00
#define XLOG_HEAP_DELETE 0x10
#define XLOG_HEAP_UPDATE 0x20
! /* 0x030 is free, was XLOG_HEAP_MOVE */
#define XLOG_HEAP_HOT_UPDATE 0x40
#define XLOG_HEAP_CONFIRM 0x50
#define XLOG_HEAP_LOCK 0x60
--- 32,38 ----
#define XLOG_HEAP_INSERT 0x00
#define XLOG_HEAP_DELETE 0x10
#define XLOG_HEAP_UPDATE 0x20
! #define XLOG_HEAP_TRUNCATE 0x30
#define XLOG_HEAP_HOT_UPDATE 0x40
#define XLOG_HEAP_CONFIRM 0x50
#define XLOG_HEAP_LOCK 0x60
***************
*** 109,114 **** typedef struct xl_heap_delete
--- 109,136 ----
#define SizeOfHeapDelete (offsetof(xl_heap_delete, flags) + sizeof(uint8))
+ /*
+ * xl_heap_delete flag values, 8 bits are available.
+ */
+ #define XLH_TRUNCATE_CASCADE (1<<0)
+ #define XLH_TRUNCATE_RESTART_SEQS (1<<1)
+
+ /*
+ * For truncate we list all truncated relids in an array, followed by all
+ * sequence relids that need to be restarted, if any.
+ * All rels are always within the same database, so we just list dbid once.
+ */
+ typedef struct xl_heap_truncate
+ {
+ Oid dbId;
+ uint32 nrelids;
+ uint32 nseqrelids;
+ uint8 flags;
+ Oid relids[FLEXIBLE_ARRAY_MEMBER];
+ } xl_heap_truncate;
+
+ #define SizeOfHeapTruncate (offsetof(xl_heap_truncate, relids))
+
/*
* We don't store the whole fixed part (HeapTupleHeaderData) of an inserted
* or updated tuple in WAL; we can save a few bytes by reconstructing the
diff --git a/src/include/executor/execuindex 6545a80222..d5ad93a0a1 100644
*** a/src/include/executor/executor.h
--- b/src/include/executor/executor.h
***************
*** 558,561 **** extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
--- 558,567 ----
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
+ /*
+ * prototype for tablecmds.c accessible for logical apply
+ */
+ void ExecuteTruncateGuts(List *rels, List *relids, List *relids_logged,
+ DropBehavior behavior, bool restart_seqs);
+
#endif /* EXECUTOR_H */
diff --git a/src/include/replication/lindex 0eb21057c5..b1abb9e36f 100644
*** a/src/include/replication/logicalproto.h
--- b/src/include/replication/logicalproto.h
***************
*** 98,103 **** extern void logicalrep_write_delete(StringInfo out, Relation rel,
--- 98,107 ----
HeapTuple oldtuple);
extern LogicalRepRelId logicalrep_read_delete(StringInfo in,
LogicalRepTupleData *oldtup);
+ extern void logicalrep_write_truncate(StringInfo out, Relation rel,
+ bool cascade, bool restart_seqs);
+ extern LogicalRepRelId logicalrep_read_truncate(StringInfo in,
+ bool *cascade, bool *restart_seqs);
extern void logicalrep_write_rel(StringInfo out, Relation rel);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
extern void logicalrep_write_typ(StringInfo out, Oid typoid);
diff --git a/src/include/replication/reorderbindex 0970abca52..e8439fac49 100644
*** a/src/include/replication/reorderbuffer.h
--- b/src/include/replication/reorderbuffer.h
***************
*** 59,65 **** enum ReorderBufferChangeType
REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID,
REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID,
REORDER_BUFFER_CHANGE_INTERNAL_SPEC_INSERT,
! REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM
};
/*
--- 59,66 ----
REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID,
REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID,
REORDER_BUFFER_CHANGE_INTERNAL_SPEC_INSERT,
! REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM,
! REORDER_BUFFER_CHANGE_TRUNCATE
};
/*
***************
*** 128,134 **** typedef struct ReorderBufferChange
CommandId cmax;
CommandId combocid;
} tuplecid;
! } data;
/*
* While in use this is how a change is linked into a transactions,
--- 129,146 ----
CommandId cmax;
CommandId combocid;
} tuplecid;
!
! /*
! * Truncate data for REORDER_BUFFER_CHANGE_TRUNCATE representing
! * one relation to be truncated.
! */
! struct
! {
! Oid relid;
! bool cascade;
! bool restart_seqs;
! } truncate_msg;
! } data;
/*
* While in use this is how a change is linked into a transactions,
diff --git a/src/test/subscription/t/001_rep_cindex e0104cd8d0..7493b3f14a 100644
*** a/src/test/subscription/t/001_rep_changes.pl
--- b/src/test/subscription/t/001_rep_changes.pl
***************
*** 3,9 **** use strict;
use warnings;
use PostgresNode;
use TestLib;
! use Test::More tests => 16;
# Initialize publisher node
my $node_publisher = get_new_node('publisher');
--- 3,9 ----
use warnings;
use PostgresNode;
use TestLib;
! use Test::More tests => 24;
# Initialize publisher node
my $node_publisher = get_new_node('publisher');
***************
*** 176,181 **** $result = $node_subscriber->safe_psql('postgres',
--- 176,191 ----
is($result, qq(1152|1|1100),
'check replicated inserts after subscription publication change');
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_rep");
+ is($result, qq(20|-20|-1),
+ 'check changes skipped after subscription publication change');
+
+ # truncate should not be replicated
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep");
+
+ $node_publisher->wait_for_catchup($appname);
+
$result = $node_subscriber->safe_psql('postgres',
"SELECT count(*), min(a), max(a) FROM tab_rep");
is($result, qq(20|-20|-1),
***************
*** 185,191 **** is($result, qq(20|-20|-1),
$node_publisher->safe_psql('postgres',
"ALTER PUBLICATION tap_pub_ins_only SET (publish = 'insert, delete')");
$node_publisher->safe_psql('postgres',
! "ALTER PUBLICATION tap_pub_ins_only ADD TABLE tab_full");
$node_publisher->safe_psql('postgres', "DELETE FROM tab_ins WHERE a > 0");
$node_subscriber->safe_psql('postgres',
"ALTER SUBSCRIPTION tap_sub REFRESH PUBLICATION WITH (copy_data = false)"
--- 195,201 ----
$node_publisher->safe_psql('postgres',
"ALTER PUBLICATION tap_pub_ins_only SET (publish = 'insert, delete')");
$node_publisher->safe_psql('postgres',
! "ALTER PUBLICATION tap_pub_ins_only ADD TABLE tab_full, tab_rep");
$node_publisher->safe_psql('postgres', "DELETE FROM tab_ins WHERE a > 0");
$node_subscriber->safe_psql('postgres',
"ALTER SUBSCRIPTION tap_sub REFRESH PUBLICATION WITH (copy_data = false)"
***************
*** 204,209 **** $result = $node_subscriber->safe_psql('postgres',
--- 214,282 ----
"SELECT count(*), min(a), max(a) FROM tab_full");
is($result, qq(21|0|100), 'check replicated insert after alter publication');
+ # add a sequence and a foreign key to check cascade and restart
+ # identity truncate options
+ $node_subscriber->safe_psql('postgres',
+ "CREATE TABLE tab_notrep_fk (a int REFERENCES tab_rep(a))"
+ );
+ $node_subscriber->safe_psql('postgres', "INSERT INTO tab_notrep_fk VALUES (-1)");
+ $node_subscriber->safe_psql('postgres',
+ "CREATE SEQUENCE seq_notrep OWNED BY tab_rep.a"
+ );
+ $node_subscriber->safe_psql('postgres',
+ "ALTER SEQUENCE seq_notrep START 101"
+ );
+
+ # truncate should now be replicated
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep");
+
+ $node_publisher->wait_for_catchup($appname);
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_rep");
+ is($result, qq(0||),
+ 'check replicated truncate after alter publication');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_notrep_fk");
+ is($result, qq(1|-1|-1),
+ 'check replicated truncate does not cascade');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq_notrep')");
+ is($result, qq(1),
+ 'check replicated truncate does not restart identities');
+
+ # should restart identity
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep RESTART IDENTITY");
+
+ $node_publisher->wait_for_catchup($appname);
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_notrep_fk");
+ is($result, qq(1|-1|-1),
+ 'check replicated truncate does not cascade');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq_notrep')");
+ is($result, qq(101),
+ 'check replicated truncate restart identities');
+
+ # should cascade
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep CASCADE");
+
+ $node_publisher->wait_for_catchup($appname);
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_notrep_fk");
+ is($result, qq(0||),
+ 'check replicated truncate cascade');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq_notrep')");
+ is($result, qq(102),
+ 'check replicated truncate does not restart identities');
+
# check restart on rename
$oldpid = $node_publisher->safe_psql('postgres',
"SELECT pid FROM pg_stat_replication WHERE application_name = '$appname';"
Re: [PATCH] Logical decoding of TRUNCATE
Hi,
I reviewed 0001 in its own thread.
So I think that we generally want this patch and I think the design
decisions are right. Namely:
TRUNCATE being treated as DELETE in terms of DML filtering makes sense
to me as it is basically bulk delete, needs to be mentioned in release
notes though.
Adding special message to protocol is appropriate as truncate is more
DML than DDL in sense of manipulating data so it should be replicated
separately from other DDL
Processing relations that were truncated when CASCADE is used separately
is needed because we allow relations to be filtered by logical replication
I see the patch adds new xlog record which is perhaps not ideal but the
current one seems utterly unsuitable for decoding so I guess it's okay,
especially when it's only added for wal_level = logical which it is.
Also TRUNCATE is not exactly high tps operation.
Things I am less convinced about:
The patch will cascade truncation on downstream if cascade was specified
on the upstream, that can potentially be dangerous and we either should
not do it and only truncate the tables which were truncated upstream
(but without restricting because of FKs), leaving the data inconsistent
on downstream (like we do already with DELETE or UPDATE). Or maybe make
it into either subscription or publication option so that user can chose
the behaviour here as I am sure some people will want it to cascade (but
the default should still IMHO be to not cascade as that's safer).
+ /* logicalrep_rel_close call not needed, because ExecuteTruncateGuts + * already closes the relations. Setting localrel to NULL in the map entry + * is still needed. + */ + rel->localrel = NULL;
This is somewhat ugly. Perhaps the ExecuteTruncateGuts should track
which relations it opened and only close those and the rest should be
closed by caller? That should also remove the other ugly part which is
that the ExecuteTruncateGuts modifies the input list. What if caller
wanted to use those relations it sent as parameter later?
--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Re: [PATCH] Logical decoding of TRUNCATE
On 17 January 2018 at 17:07, Petr Jelinek <petr.jelinek@2ndquadrant.com> wrote:
Things I am less convinced about:
The patch will cascade truncation on downstream if cascade was specified
on the upstream, that can potentially be dangerous and we either should
not do it and only truncate the tables which were truncated upstream
(but without restricting because of FKs), leaving the data inconsistent
on downstream (like we do already with DELETE or UPDATE). Or maybe make
it into either subscription or publication option so that user can chose
the behaviour here as I am sure some people will want it to cascade (but
the default should still IMHO be to not cascade as that's safer).
I agree the default should be to NOT cascade.
If someone wants cascading as a publication option, that can be added later.
+ /* logicalrep_rel_close call not needed, because ExecuteTruncateGuts + * already closes the relations. Setting localrel to NULL in the map entry + * is still needed. + */ + rel->localrel = NULL;This is somewhat ugly. Perhaps the ExecuteTruncateGuts should track
which relations it opened and only close those and the rest should be
closed by caller? That should also remove the other ugly part which is
that the ExecuteTruncateGuts modifies the input list. What if caller
wanted to use those relations it sent as parameter later?
Agreed
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [PATCH] Logical decoding of TRUNCATE
Hi All,
Il 18/01/18 17:48, Simon Riggs ha scritto:
On 17 January 2018 at 17:07, Petr Jelinek <petr.jelinek@2ndquadrant.com> wrote:
Things I am less convinced about:
The patch will cascade truncation on downstream if cascade was specified
on the upstream, that can potentially be dangerous and we either should
not do it and only truncate the tables which were truncated upstream
(but without restricting because of FKs), leaving the data inconsistent
on downstream (like we do already with DELETE or UPDATE). Or maybe make
it into either subscription or publication option so that user can chose
the behaviour here as I am sure some people will want it to cascade (but
the default should still IMHO be to not cascade as that's safer).I agree the default should be to NOT cascade.
If someone wants cascading as a publication option, that can be added later.
I agree that not replicating the CASCADE option is the best option
according to POLA principle.
+ /* logicalrep_rel_close call not needed, because ExecuteTruncateGuts + * already closes the relations. Setting localrel to NULL in the map entry + * is still needed. + */ + rel->localrel = NULL;This is somewhat ugly. Perhaps the ExecuteTruncateGuts should track
which relations it opened and only close those and the rest should be
closed by caller? That should also remove the other ugly part which is
that the ExecuteTruncateGuts modifies the input list. What if caller
wanted to use those relations it sent as parameter later?Agreed
Attached a new version of the patch addressing these issues.
Regards,
Marco
--
Marco Nenciarini - 2ndQuadrant Italy
PostgreSQL Training, Services and Support
marco.nenciarini@2ndQuadrant.it | www.2ndQuadrant.it
Attachments:
0001-truncate_ignore_fks_role_replica.v3.patchtext/plain; charset=UTF-8; name=0001-truncate_ignore_fks_role_replica.v3.patch; x-mac-creator=0; x-mac-type=0Download
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index f2a928b823..180ebd0717 100644
*** a/src/backend/commands/tablecmds.c
--- b/src/backend/commands/tablecmds.c
***************
*** 1340,1355 **** ExecuteTruncate(TruncateStmt *stmt)
}
/*
! * Check foreign key references. In CASCADE mode, this should be
! * unnecessary since we just pulled in all the references; but as a
! * cross-check, do it anyway if in an Assert-enabled build.
*/
#ifdef USE_ASSERT_CHECKING
- heap_truncate_check_FKs(rels, false);
- #else
- if (stmt->behavior == DROP_RESTRICT)
heap_truncate_check_FKs(rels, false);
#endif
/*
* If we are asked to restart sequences, find all the sequences, lock them
--- 1340,1363 ----
}
/*
! * Suppress foreign key references check if session replication role is
! * set to REPLICA.
*/
+ if (SessionReplicationRole != SESSION_REPLICATION_ROLE_REPLICA)
+ {
+
+ /*
+ * Check foreign key references. In CASCADE mode, this should be
+ * unnecessary since we just pulled in all the references; but as a
+ * cross-check, do it anyway if in an Assert-enabled build.
+ */
#ifdef USE_ASSERT_CHECKING
heap_truncate_check_FKs(rels, false);
+ #else
+ if (stmt->behavior == DROP_RESTRICT)
+ heap_truncate_check_FKs(rels, false);
#endif
+ }
/*
* If we are asked to restart sequences, find all the sequences, lock them
0002-logical_decode_truncate.v14.patchtext/plain; charset=UTF-8; name=0002-logical_decode_truncate.v14.patch; x-mac-creator=0; x-mac-type=0Download
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 1e22c1eefc..d1feea4909 100644
*** a/contrib/test_decoding/expected/ddl.out
--- b/contrib/test_decoding/expected/ddl.out
***************
*** 543,548 **** UPDATE table_with_unique_not_null SET data = 3 WHERE data = 2;
--- 543,550 ----
UPDATE table_with_unique_not_null SET id = -id;
UPDATE table_with_unique_not_null SET id = -id;
DELETE FROM table_with_unique_not_null WHERE data = 3;
+ TRUNCATE table_with_unique_not_null;
+ TRUNCATE table_with_unique_not_null, table_with_unique_not_null RESTART IDENTITY CASCADE;
-- check toast support
BEGIN;
CREATE SEQUENCE toasttable_rand_seq START 79 INCREMENT 1499; -- portable "random"
***************
*** 660,665 **** SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
--- 662,673 ----
table public.table_with_unique_not_null: DELETE: id[integer]:4
COMMIT
BEGIN
+ table public.table_with_unique_not_null: TRUNCATE: (no-flags)
+ COMMIT
+ BEGIN
+ table public.table_with_unique_not_null: TRUNCATE: restart_seqs cascade
+ COMMIT
+ BEGIN
table public.toasttable: INSERT: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
BEGIN
***************
*** 668,674 **** SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
BEGIN
table public.toasttable: UPDATE: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
! (103 rows)
INSERT INTO toasttable(toasted_col1) SELECT string_agg(g.i::text, '') FROM generate_series(1, 2000) g(i);
-- update of second column, first column unchanged
--- 676,682 ----
BEGIN
table public.toasttable: UPDATE: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
! (109 rows)
INSERT INTO toasttable(toasted_col1) SELECT string_agg(g.i::text, '') FROM generate_series(1, 2000) g(i);
-- update of second column, first column unchanged
diff --git a/contrib/test_decoding/sql/ddl.sqindex 057dae056b..0fb4102db8 100644
*** a/contrib/test_decoding/sql/ddl.sql
--- b/contrib/test_decoding/sql/ddl.sql
***************
*** 333,338 **** UPDATE table_with_unique_not_null SET id = -id;
--- 333,341 ----
UPDATE table_with_unique_not_null SET id = -id;
DELETE FROM table_with_unique_not_null WHERE data = 3;
+ TRUNCATE table_with_unique_not_null;
+ TRUNCATE table_with_unique_not_null, table_with_unique_not_null RESTART IDENTITY CASCADE;
+
-- check toast support
BEGIN;
CREATE SEQUENCE toasttable_rand_seq START 79 INCREMENT 1499; -- portable "random"
diff --git a/contrib/test_decoding/test_index 0f18afa852..55c4593ed5 100644
*** a/contrib/test_decoding/test_decoding.c
--- b/contrib/test_decoding/test_decoding.c
***************
*** 466,471 **** pg_decode_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
--- 466,485 ----
&change->data.tp.oldtuple->tuple,
true);
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ appendStringInfoString(ctx->out, " TRUNCATE:");
+
+ if (change->data.truncate_msg.restart_seqs
+ || change->data.truncate_msg.cascade)
+ {
+ if (change->data.truncate_msg.restart_seqs)
+ appendStringInfo(ctx->out, " restart_seqs");
+ if (change->data.truncate_msg.cascade)
+ appendStringInfo(ctx->out, " cascade");
+ }
+ else
+ appendStringInfoString(ctx->out, " (no-flags)");
+ break;
default:
Assert(false);
}
diff --git a/doc/src/sgml/logical-replicatioindex 75551d8ee1..6b64c8ef76 100644
*** a/doc/src/sgml/logical-replication.sgml
--- b/doc/src/sgml/logical-replication.sgml
***************
*** 111,116 ****
--- 111,118 ----
any combination of <command>INSERT</command>, <command>UPDATE</command>, and
<command>DELETE</command>, similar to how triggers are fired by
particular event types. By default, all operation types are replicated.
+ <command>TRUNCATE</command> is treated as a form of <command>DELETE</command>
+ for the purpose of deciding whether to publish, or not.
</para>
<para>
***************
*** 364,378 ****
</para>
</listitem>
- <listitem>
- <para>
- <command>TRUNCATE</command> commands are not replicated. This can, of
- course, be worked around by using <command>DELETE</command> instead. To
- avoid accidental <command>TRUNCATE</command> invocations, you can revoke
- the <literal>TRUNCATE</literal> privilege from tables.
- </para>
- </listitem>
-
<listitem>
<para>
Large objects (see <xref linkend="largeobjects"/>) are not replicated.
--- 366,371 ----
diff --git a/doc/src/sgml/protocol.sgml b/doindex 4c5ed1e6d6..d2a0d7e52e 100644
*** a/doc/src/sgml/protocol.sgml
--- b/doc/src/sgml/protocol.sgml
***************
*** 6838,6843 **** TupleData
--- 6838,6889 ----
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ Truncate
+ </term>
+ <listitem>
+ <para>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ Byte1('T')
+ </term>
+ <listitem>
+ <para>
+ Identifies the message as a truncate message.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ Int32
+ </term>
+ <listitem>
+ <para>
+ ID of the relation corresponding to the ID in the relation
+ message.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ Int8
+ </term>
+ <listitem>
+ <para>
+ Option flags for truncate. Currently can be 0 for no flags,
+ 1 for CASCADE, 2 for RESTART IDENTITY and 3 if both are set.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect1>
diff --git a/doc/src/sgml/ref/creindex bfe12d5f41..c5032938a7 100644
*** a/doc/src/sgml/ref/create_publication.sgml
--- b/doc/src/sgml/ref/create_publication.sgml
***************
*** 111,116 **** CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
--- 111,121 ----
and so the default value for this option is
<literal>'insert, update, delete'</literal>.
</para>
+ <para>
+ <command>TRUNCATE</command> is treated as a form of
+ <command>DELETE</command> for the purpose of deciding whether
+ to publish, or not.
+ </para>
</listitem>
</varlistentry>
</variablelist>
***************
*** 168,175 **** CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
</para>
<para>
! <command>TRUNCATE</command> and <acronym>DDL</acronym> operations
! are not published.
</para>
</refsect1>
--- 173,179 ----
</para>
<para>
! <acronym>DDL</acronym> operations are not published.
</para>
</refsect1>
diff --git a/src/backend/access/heap/heapam.c bindex dbc8f2d6c7..f4c6eaf57f 100644
*** a/src/backend/access/heap/heapam.c
--- b/src/backend/access/heap/heapam.c
***************
*** 9134,9139 **** heap_redo(XLogReaderState *record)
--- 9134,9146 ----
case XLOG_HEAP_UPDATE:
heap_xlog_update(record, false);
break;
+ case XLOG_HEAP_TRUNCATE:
+ /*
+ * TRUNCATE is a no-op because the actions are already logged
+ * as SMGR WAL records. TRUNCATE WAL record only exists to allow
+ * it to be logically decoded precisely.
+ */
+ break;
case XLOG_HEAP_HOT_UPDATE:
heap_xlog_update(record, true);
break;
diff --git a/src/backend/access/rmgrdesindex b00c071cb6..15550cdff0 100644
*** a/src/backend/access/rmgrdesc/heapdesc.c
--- b/src/backend/access/rmgrdesc/heapdesc.c
***************
*** 75,80 **** heap_desc(StringInfo buf, XLogReaderState *record)
--- 75,93 ----
xlrec->new_offnum,
xlrec->new_xmax);
}
+ else if (info == XLOG_HEAP_TRUNCATE)
+ {
+ xl_heap_truncate *xlrec = (xl_heap_truncate *) rec;
+
+ if (xlrec->flags & XLH_TRUNCATE_CASCADE)
+ appendStringInfo(buf, "cascade ");
+ if (xlrec->flags & XLH_TRUNCATE_RESTART_SEQS)
+ appendStringInfo(buf, "restart_seqs ");
+ appendStringInfo(buf, "nrelids %u nseqrelids %u",
+ xlrec->nrelids,
+ xlrec->nseqrelids);
+ /* Skip the list of relids and seqrelids */
+ }
else if (info == XLOG_HEAP_CONFIRM)
{
xl_heap_confirm *xlrec = (xl_heap_confirm *) rec;
***************
*** 186,191 **** heap_identify(uint8 info)
--- 199,207 ----
case XLOG_HEAP_HOT_UPDATE | XLOG_HEAP_INIT_PAGE:
id = "HOT_UPDATE+INIT";
break;
+ case XLOG_HEAP_TRUNCATE:
+ id = "TRUNCATE";
+ break;
case XLOG_HEAP_CONFIRM:
id = "HEAP_CONFIRM";
break;
diff --git a/src/backend/commands/tablecmds.cindex 180ebd0717..5c498723ad 100644
*** a/src/backend/commands/tablecmds.c
--- b/src/backend/commands/tablecmds.c
***************
*** 16,21 ****
--- 16,22 ----
#include "access/genam.h"
#include "access/heapam.h"
+ #include "access/heapam_xlog.h"
#include "access/multixact.h"
#include "access/reloptions.h"
#include "access/relscan.h"
***************
*** 1248,1258 **** ExecuteTruncate(TruncateStmt *stmt)
{
List *rels = NIL;
List *relids = NIL;
! List *seq_relids = NIL;
! EState *estate;
! ResultRelInfo *resultRelInfos;
! ResultRelInfo *resultRelInfo;
! SubTransactionId mySubid;
ListCell *cell;
/*
--- 1249,1255 ----
{
List *rels = NIL;
List *relids = NIL;
! List *relids_logged = NIL;
ListCell *cell;
/*
***************
*** 1276,1281 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1273,1281 ----
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, myrelid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, myrelid);
if (recurse)
{
***************
*** 1296,1301 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1296,1304 ----
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, childrelid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, childrelid);
}
}
else if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
***************
*** 1305,1311 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1308,1336 ----
errhint("Do not specify the ONLY keyword, or use truncate only on the partitions directly.")));
}
+ ExecuteTruncateGuts(rels, relids, relids_logged,
+ stmt->behavior, stmt->restart_seqs, true);
+ }
+
+ void
+ ExecuteTruncateGuts(List *rels, List *relids, List *relids_logged,
+ DropBehavior behavior, bool restart_seqs, bool closerels)
+ {
+ List *seq_relids = NIL;
+ EState *estate;
+ ResultRelInfo *resultRelInfos;
+ ResultRelInfo *resultRelInfo;
+ SubTransactionId mySubid;
+ ListCell *cell;
+ List *seq_relids_logged = NIL;
+ uint32 nrelids = 0;
+ uint32 nseqrelids = 0;
+ uint32 maxrelids = 2;
+ Oid *logrelids = NULL;
+
/*
+ * Open, exclusive-lock, and check all the explicitly-specified relations
+ *
* In CASCADE mode, suck in all referencing relations as well. This
* requires multiple iterations to find indirectly-dependent relations. At
* each phase, we need to exclusive-lock new rels before looking for their
***************
*** 1313,1319 **** ExecuteTruncate(TruncateStmt *stmt)
* soon as we open it, to avoid a faux pas such as holding lock for a long
* time on a rel we have no permissions for.
*/
! if (stmt->behavior == DROP_CASCADE)
{
for (;;)
{
--- 1338,1344 ----
* soon as we open it, to avoid a faux pas such as holding lock for a long
* time on a rel we have no permissions for.
*/
! if (behavior == DROP_CASCADE)
{
for (;;)
{
***************
*** 1335,1340 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1360,1368 ----
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, relid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, relid);
}
}
}
***************
*** 1354,1360 **** ExecuteTruncate(TruncateStmt *stmt)
#ifdef USE_ASSERT_CHECKING
heap_truncate_check_FKs(rels, false);
#else
! if (stmt->behavior == DROP_RESTRICT)
heap_truncate_check_FKs(rels, false);
#endif
}
--- 1382,1388 ----
#ifdef USE_ASSERT_CHECKING
heap_truncate_check_FKs(rels, false);
#else
! if (behavior == DROP_RESTRICT)
heap_truncate_check_FKs(rels, false);
#endif
}
***************
*** 1365,1371 **** ExecuteTruncate(TruncateStmt *stmt)
* We want to do this early since it's pointless to do all the truncation
* work only to fail on sequence permissions.
*/
! if (stmt->restart_seqs)
{
foreach(cell, rels)
{
--- 1393,1399 ----
* We want to do this early since it's pointless to do all the truncation
* work only to fail on sequence permissions.
*/
! if (restart_seqs)
{
foreach(cell, rels)
{
***************
*** 1386,1391 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1414,1423 ----
RelationGetRelationName(seq_rel));
seq_relids = lappend_oid(seq_relids, seq_relid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(seq_rel))
+ seq_relids_logged = lappend_oid(seq_relids_logged,
+ seq_relid);
relation_close(seq_rel, NoLock);
}
***************
*** 1520,1525 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1552,1611 ----
ResetSequence(seq_relid);
}
+ /*
+ * Write a WAL record to allow this set of actions to be logically decoded.
+ * We could optimize this away when !RelationIsLogicallyLogged(rel)
+ * but that doesn't save much space or time.
+ *
+ * Assemble an array of relids, then an array of seqrelids so we can write
+ * a single WAL record for the whole action.
+ */
+ logrelids = palloc(maxrelids * sizeof(Oid));
+ foreach (cell, relids_logged)
+ {
+ nrelids++;
+ if (nrelids > maxrelids)
+ {
+ maxrelids *= 2;
+ logrelids = repalloc(logrelids, maxrelids * sizeof(Oid));
+ }
+ logrelids[nrelids - 1] = lfirst_oid(cell);
+ }
+
+ foreach (cell, seq_relids_logged)
+ {
+ nseqrelids++;
+ if ((nrelids + nseqrelids) > maxrelids)
+ {
+ maxrelids *= 2;
+ logrelids = repalloc(logrelids, maxrelids * sizeof(Oid));
+ }
+ logrelids[nrelids + nseqrelids - 1] = lfirst_oid(cell);
+ }
+
+ if ((nrelids + nseqrelids) > 0)
+ {
+ xl_heap_truncate xlrec;
+
+ xlrec.dbId = MyDatabaseId;
+ xlrec.nrelids = nrelids;
+ xlrec.nseqrelids = nseqrelids;
+ xlrec.flags = 0;
+ if (behavior == DROP_CASCADE)
+ xlrec.flags |= XLH_TRUNCATE_CASCADE;
+ if (restart_seqs)
+ xlrec.flags |= XLH_TRUNCATE_RESTART_SEQS;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfHeapTruncate);
+ XLogRegisterData((char *) logrelids,
+ (nrelids + nseqrelids) * sizeof(Oid));
+
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ (void) XLogInsert(RM_HEAP_ID, XLOG_HEAP_TRUNCATE);
+ }
+
/*
* Process all AFTER STATEMENT TRUNCATE triggers.
*/
***************
*** 1538,1548 **** ExecuteTruncate(TruncateStmt *stmt)
FreeExecutorState(estate);
/* And close the rels (can't do this while EState still holds refs) */
! foreach(cell, rels)
{
! Relation rel = (Relation) lfirst(cell);
! heap_close(rel, NoLock);
}
}
--- 1624,1637 ----
FreeExecutorState(estate);
/* And close the rels (can't do this while EState still holds refs) */
! if (closerels)
{
! foreach(cell, rels)
! {
! Relation rel = (Relation) lfirst(cell);
! heap_close(rel, NoLock);
! }
}
}
diff --git a/src/backend/replication/loindex 6eb0d5527e..ef5b937807 100644
*** a/src/backend/replication/logical/decode.c
--- b/src/backend/replication/logical/decode.c
***************
*** 65,70 **** static void DecodeLogicalMsgOp(LogicalDecodingContext *ctx, XLogRecordBuffer *bu
--- 65,71 ----
static void DecodeInsert(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeUpdate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeDelete(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
+ static void DecodeTruncate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeMultiInsert(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeSpecConfirm(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
***************
*** 449,454 **** DecodeHeapOp(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
--- 450,460 ----
DecodeDelete(ctx, buf);
break;
+ case XLOG_HEAP_TRUNCATE:
+ if (SnapBuildProcessChange(builder, xid, buf->origptr))
+ DecodeTruncate(ctx, buf);
+ break;
+
case XLOG_HEAP_INPLACE:
/*
***************
*** 825,830 **** DecodeDelete(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
--- 831,876 ----
ReorderBufferQueueChange(ctx->reorder, XLogRecGetXid(r), buf->origptr, change);
}
+ /*
+ * Parse XLOG_HEAP_TRUNCATE from wal
+ */
+ static void
+ DecodeTruncate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+ {
+ XLogReaderState *r = buf->record;
+ xl_heap_truncate *xlrec;
+ ReorderBufferChange *change;
+ int i;
+
+ xlrec = (xl_heap_truncate *) XLogRecGetData(r);
+
+ /* only interested in our database */
+ if (xlrec->dbId != ctx->slot->data.database)
+ return;
+
+ /* output plugin doesn't look for this origin, no need to queue */
+ if (FilterByOrigin(ctx, XLogRecGetOrigin(r)))
+ return;
+
+ change = ReorderBufferGetChange(ctx->reorder);
+ change->action = REORDER_BUFFER_CHANGE_TRUNCATE;
+ change->origin_id = XLogRecGetOrigin(r);
+ if (xlrec->flags & XLH_TRUNCATE_CASCADE)
+ change->data.truncate_msg.cascade = true;
+ if (xlrec->flags & XLH_TRUNCATE_RESTART_SEQS)
+ change->data.truncate_msg.restart_seqs = true;
+
+ /*
+ * Queue up one change per relation, ignoring sequences for now
+ */
+ for (i = 0; i < xlrec->nrelids; i++)
+ {
+ change->data.truncate_msg.relid = xlrec->relids[i];
+ ReorderBufferQueueChange(ctx->reorder, XLogRecGetXid(r),
+ buf->origptr, change);
+ }
+ }
+
/*
* Decode XLOG_HEAP2_MULTI_INSERT_insert record into multiple tuplebufs.
*
diff --git a/src/backend/replication/logical/prindex 948343e4ae..2fa6f8393d 100644
*** a/src/backend/replication/logical/proto.c
--- b/src/backend/replication/logical/proto.c
***************
*** 26,31 ****
--- 26,34 ----
*/
#define LOGICALREP_IS_REPLICA_IDENTITY 1
+ #define TRUNCATE_CASCADE (1<<0)
+ #define TRUNCATE_RESTART_SEQS (1<<1)
+
static void logicalrep_write_attrs(StringInfo out, Relation rel);
static void logicalrep_write_tuple(StringInfo out, Relation rel,
HeapTuple tuple);
***************
*** 292,297 **** logicalrep_read_delete(StringInfo in, LogicalRepTupleData *oldtup)
--- 295,342 ----
return relid;
}
+ /*
+ * Write TRUNCATE to the output stream.
+ */
+ void
+ logicalrep_write_truncate(StringInfo out, Relation rel,
+ bool cascade, bool restart_seqs)
+ {
+ uint8 flags = 0;
+
+ pq_sendbyte(out, 'T'); /* action TRUNCATE */
+
+ /* use Oid as relation identifier */
+ pq_sendint32(out, RelationGetRelid(rel));
+
+ /* encode and send truncate flags */
+ if (cascade)
+ flags |= TRUNCATE_CASCADE;
+ if (restart_seqs)
+ flags |= TRUNCATE_RESTART_SEQS;
+ pq_sendint8(out, flags);
+ }
+
+ /*
+ * Read TRUNCATE from stream.
+ */
+ LogicalRepRelId
+ logicalrep_read_truncate(StringInfo in, bool *cascade, bool *restart_seqs)
+ {
+ LogicalRepRelId relid;
+ uint8 flags;
+
+ /* read the relation id */
+ relid = pq_getmsgint(in, 4);
+
+ /* read and decode truncate flags */
+ flags = pq_getmsgint(in, 1);
+ *cascade = (flags & TRUNCATE_CASCADE) > 0;
+ *restart_seqs = (flags & TRUNCATE_RESTART_SEQS) > 0;
+
+ return relid;
+ }
+
/*
* Write relation description to the output stream.
*/
diff --git a/src/backend/replication/logical/rindex c72a611a39..f1d65c1457 100644
*** a/src/backend/replication/logical/reorderbuffer.c
--- b/src/backend/replication/logical/reorderbuffer.c
***************
*** 403,408 **** ReorderBufferReturnChange(ReorderBuffer *rb, ReorderBufferChange *change)
--- 403,410 ----
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ break;
}
pfree(change);
***************
*** 1342,1347 **** ReorderBufferCommit(ReorderBuffer *rb, TransactionId xid,
--- 1344,1356 ----
switch (change->action)
{
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ reloid = change->data.truncate_msg.relid;
+ relation = RelationIdGetRelation(reloid);
+ rb->apply_change(rb, txn, relation, change);
+ RelationClose(relation);
+ break;
+
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
/*
***************
*** 2239,2244 **** ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
--- 2248,2254 ----
}
break;
}
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
***************
*** 2524,2529 **** ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
--- 2534,2540 ----
break;
}
/* the base struct contains all the data, easy peasy */
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
diff --git a/src/backend/replication/logical/worker.c index 83c69092ae..7177c543c2 100644
*** a/src/backend/replication/logical/worker.c
--- b/src/backend/replication/logical/worker.c
***************
*** 869,874 **** apply_handle_delete(StringInfo s)
--- 869,925 ----
CommandCounterIncrement();
}
+ /*
+ * Handle TRUNCATE message.
+ *
+ * TODO: FDW support
+ */
+ static void
+ apply_handle_truncate(StringInfo s)
+ {
+ LogicalRepRelMapEntry *rel;
+ LogicalRepRelId relid;
+ bool cascade = false;
+ bool restart_seqs = false;
+ List *rels = NIL;
+ List *relids = NIL;
+
+ ensure_transaction();
+
+ relid = logicalrep_read_truncate(s, &cascade, &restart_seqs);
+ rel = logicalrep_rel_open(relid, RowExclusiveLock);
+ if (!should_apply_changes_for_rel(rel))
+ {
+ /*
+ * The relation can't become interesting in the middle of the
+ * transaction so it's safe to unlock it.
+ */
+ logicalrep_rel_close(rel, RowExclusiveLock);
+ return;
+ }
+
+ /* Check if we can do the truncate. */
+ check_relation_updatable(rel);
+
+ rels = lappend(rels, rel->localrel);
+ relids = lappend_oid(relids, rel->localreloid);
+
+ /*
+ * Even if we used CASCADE on the upstream master we explicitly
+ * default to replaying changes without further cascading.
+ * This might be later changeable with a user specified option.
+ */
+ cascade = false;
+
+ ExecuteTruncateGuts(rels, relids, NULL,
+ cascade ? DROP_CASCADE : DROP_RESTRICT,
+ restart_seqs, false);
+
+ logicalrep_rel_close(rel, NoLock);
+
+ CommandCounterIncrement();
+ }
+
/*
* Logical replication protocol message dispatcher.
***************
*** 900,905 **** apply_dispatch(StringInfo s)
--- 951,960 ----
case 'D':
apply_handle_delete(s);
break;
+ /* TRUNCATE */
+ case 'T':
+ apply_handle_truncate(s);
+ break;
/* RELATION */
case 'R':
apply_handle_relation(s);
diff --git a/src/backend/replication/pgoutput/pindex 40a1ef3c1d..7fbd0a3dca 100644
*** a/src/backend/replication/pgoutput/pgoutput.c
--- b/src/backend/replication/pgoutput/pgoutput.c
***************
*** 275,280 **** pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
--- 275,281 ----
if (!relentry->pubactions.pubupdate)
return;
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
case REORDER_BUFFER_CHANGE_DELETE:
if (!relentry->pubactions.pubdelete)
return;
***************
*** 352,357 **** pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
--- 353,367 ----
else
elog(DEBUG1, "didn't send DELETE change because of missing oldtuple");
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ {
+ OutputPluginPrepareWrite(ctx, true);
+ logicalrep_write_truncate(ctx->out, relation,
+ change->data.truncate_msg.cascade,
+ change->data.truncate_msg.restart_seqs);
+ OutputPluginWrite(ctx, true);
+ }
+ break;
default:
Assert(false);
}
diff --git a/src/include/access/heapam_xlog.h b/srindex 700e25c36a..c56b200f7d 100644
*** a/src/include/access/heapam_xlog.h
--- b/src/include/access/heapam_xlog.h
***************
*** 32,38 ****
#define XLOG_HEAP_INSERT 0x00
#define XLOG_HEAP_DELETE 0x10
#define XLOG_HEAP_UPDATE 0x20
! /* 0x030 is free, was XLOG_HEAP_MOVE */
#define XLOG_HEAP_HOT_UPDATE 0x40
#define XLOG_HEAP_CONFIRM 0x50
#define XLOG_HEAP_LOCK 0x60
--- 32,38 ----
#define XLOG_HEAP_INSERT 0x00
#define XLOG_HEAP_DELETE 0x10
#define XLOG_HEAP_UPDATE 0x20
! #define XLOG_HEAP_TRUNCATE 0x30
#define XLOG_HEAP_HOT_UPDATE 0x40
#define XLOG_HEAP_CONFIRM 0x50
#define XLOG_HEAP_LOCK 0x60
***************
*** 109,114 **** typedef struct xl_heap_delete
--- 109,136 ----
#define SizeOfHeapDelete (offsetof(xl_heap_delete, flags) + sizeof(uint8))
+ /*
+ * xl_heap_delete flag values, 8 bits are available.
+ */
+ #define XLH_TRUNCATE_CASCADE (1<<0)
+ #define XLH_TRUNCATE_RESTART_SEQS (1<<1)
+
+ /*
+ * For truncate we list all truncated relids in an array, followed by all
+ * sequence relids that need to be restarted, if any.
+ * All rels are always within the same database, so we just list dbid once.
+ */
+ typedef struct xl_heap_truncate
+ {
+ Oid dbId;
+ uint32 nrelids;
+ uint32 nseqrelids;
+ uint8 flags;
+ Oid relids[FLEXIBLE_ARRAY_MEMBER];
+ } xl_heap_truncate;
+
+ #define SizeOfHeapTruncate (offsetof(xl_heap_truncate, relids))
+
/*
* We don't store the whole fixed part (HeapTupleHeaderData) of an inserted
* or updated tuple in WAL; we can save a few bytes by reconstructing the
diff --git a/src/include/executor/execuindex 6545a80222..8daae6efef 100644
*** a/src/include/executor/executor.h
--- b/src/include/executor/executor.h
***************
*** 558,561 **** extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
--- 558,567 ----
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
+ /*
+ * prototype for tablecmds.c accessible for logical apply
+ */
+ void ExecuteTruncateGuts(List *rels, List *relids, List *relids_logged,
+ DropBehavior behavior, bool restart_seqs, bool closerels);
+
#endif /* EXECUTOR_H */
diff --git a/src/include/replication/lindex 0eb21057c5..b1abb9e36f 100644
*** a/src/include/replication/logicalproto.h
--- b/src/include/replication/logicalproto.h
***************
*** 98,103 **** extern void logicalrep_write_delete(StringInfo out, Relation rel,
--- 98,107 ----
HeapTuple oldtuple);
extern LogicalRepRelId logicalrep_read_delete(StringInfo in,
LogicalRepTupleData *oldtup);
+ extern void logicalrep_write_truncate(StringInfo out, Relation rel,
+ bool cascade, bool restart_seqs);
+ extern LogicalRepRelId logicalrep_read_truncate(StringInfo in,
+ bool *cascade, bool *restart_seqs);
extern void logicalrep_write_rel(StringInfo out, Relation rel);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
extern void logicalrep_write_typ(StringInfo out, Oid typoid);
diff --git a/src/include/replication/reorderbindex 0970abca52..e8439fac49 100644
*** a/src/include/replication/reorderbuffer.h
--- b/src/include/replication/reorderbuffer.h
***************
*** 59,65 **** enum ReorderBufferChangeType
REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID,
REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID,
REORDER_BUFFER_CHANGE_INTERNAL_SPEC_INSERT,
! REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM
};
/*
--- 59,66 ----
REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID,
REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID,
REORDER_BUFFER_CHANGE_INTERNAL_SPEC_INSERT,
! REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM,
! REORDER_BUFFER_CHANGE_TRUNCATE
};
/*
***************
*** 128,134 **** typedef struct ReorderBufferChange
CommandId cmax;
CommandId combocid;
} tuplecid;
! } data;
/*
* While in use this is how a change is linked into a transactions,
--- 129,146 ----
CommandId cmax;
CommandId combocid;
} tuplecid;
!
! /*
! * Truncate data for REORDER_BUFFER_CHANGE_TRUNCATE representing
! * one relation to be truncated.
! */
! struct
! {
! Oid relid;
! bool cascade;
! bool restart_seqs;
! } truncate_msg;
! } data;
/*
* While in use this is how a change is linked into a transactions,
diff --git a/src/test/subscription/t/001_rep_cindex e0104cd8d0..96a6071b3a 100644
*** a/src/test/subscription/t/001_rep_changes.pl
--- b/src/test/subscription/t/001_rep_changes.pl
***************
*** 3,9 **** use strict;
use warnings;
use PostgresNode;
use TestLib;
! use Test::More tests => 16;
# Initialize publisher node
my $node_publisher = get_new_node('publisher');
--- 3,9 ----
use warnings;
use PostgresNode;
use TestLib;
! use Test::More tests => 24;
# Initialize publisher node
my $node_publisher = get_new_node('publisher');
***************
*** 176,181 **** $result = $node_subscriber->safe_psql('postgres',
--- 176,191 ----
is($result, qq(1152|1|1100),
'check replicated inserts after subscription publication change');
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_rep");
+ is($result, qq(20|-20|-1),
+ 'check changes skipped after subscription publication change');
+
+ # truncate should not be replicated
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep");
+
+ $node_publisher->wait_for_catchup($appname);
+
$result = $node_subscriber->safe_psql('postgres',
"SELECT count(*), min(a), max(a) FROM tab_rep");
is($result, qq(20|-20|-1),
***************
*** 185,191 **** is($result, qq(20|-20|-1),
$node_publisher->safe_psql('postgres',
"ALTER PUBLICATION tap_pub_ins_only SET (publish = 'insert, delete')");
$node_publisher->safe_psql('postgres',
! "ALTER PUBLICATION tap_pub_ins_only ADD TABLE tab_full");
$node_publisher->safe_psql('postgres', "DELETE FROM tab_ins WHERE a > 0");
$node_subscriber->safe_psql('postgres',
"ALTER SUBSCRIPTION tap_sub REFRESH PUBLICATION WITH (copy_data = false)"
--- 195,201 ----
$node_publisher->safe_psql('postgres',
"ALTER PUBLICATION tap_pub_ins_only SET (publish = 'insert, delete')");
$node_publisher->safe_psql('postgres',
! "ALTER PUBLICATION tap_pub_ins_only ADD TABLE tab_full, tab_rep");
$node_publisher->safe_psql('postgres', "DELETE FROM tab_ins WHERE a > 0");
$node_subscriber->safe_psql('postgres',
"ALTER SUBSCRIPTION tap_sub REFRESH PUBLICATION WITH (copy_data = false)"
***************
*** 204,209 **** $result = $node_subscriber->safe_psql('postgres',
--- 214,282 ----
"SELECT count(*), min(a), max(a) FROM tab_full");
is($result, qq(21|0|100), 'check replicated insert after alter publication');
+ # add a sequence and a foreign key to check cascade and restart
+ # identity truncate options
+ $node_subscriber->safe_psql('postgres',
+ "CREATE TABLE tab_notrep_fk (a int REFERENCES tab_rep(a))"
+ );
+ $node_subscriber->safe_psql('postgres', "INSERT INTO tab_notrep_fk VALUES (-1)");
+ $node_subscriber->safe_psql('postgres',
+ "CREATE SEQUENCE seq_notrep OWNED BY tab_rep.a"
+ );
+ $node_subscriber->safe_psql('postgres',
+ "ALTER SEQUENCE seq_notrep START 101"
+ );
+
+ # truncate should now be replicated
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep");
+
+ $node_publisher->wait_for_catchup($appname);
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_rep");
+ is($result, qq(0||),
+ 'check replicated truncate after alter publication');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_notrep_fk");
+ is($result, qq(1|-1|-1),
+ 'check replicated truncate does not cascade');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq_notrep')");
+ is($result, qq(1),
+ 'check replicated truncate does not restart identities');
+
+ # should restart identity
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep RESTART IDENTITY");
+
+ $node_publisher->wait_for_catchup($appname);
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_notrep_fk");
+ is($result, qq(1|-1|-1),
+ 'check replicated truncate does not cascade');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq_notrep')");
+ is($result, qq(101),
+ 'check replicated truncate restart identities');
+
+ # should not cascade on replica
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep CASCADE");
+
+ $node_publisher->wait_for_catchup($appname);
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_notrep_fk");
+ is($result, qq(1|-1|-1),
+ 'check replicated truncate does not cascade on replica');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq_notrep')");
+ is($result, qq(102),
+ 'check replicated truncate does not restart identities');
+
# check restart on rename
$oldpid = $node_publisher->safe_psql('postgres',
"SELECT pid FROM pg_stat_replication WHERE application_name = '$appname';"
Re: [PATCH] Logical decoding of TRUNCATE
On 19/01/18 12:37, Marco Nenciarini wrote:
Hi All,
Il 18/01/18 17:48, Simon Riggs ha scritto:
On 17 January 2018 at 17:07, Petr Jelinek <petr.jelinek@2ndquadrant.com> wrote:
Things I am less convinced about:
The patch will cascade truncation on downstream if cascade was specified
on the upstream, that can potentially be dangerous and we either should
not do it and only truncate the tables which were truncated upstream
(but without restricting because of FKs), leaving the data inconsistent
on downstream (like we do already with DELETE or UPDATE). Or maybe make
it into either subscription or publication option so that user can chose
the behaviour here as I am sure some people will want it to cascade (but
the default should still IMHO be to not cascade as that's safer).I agree the default should be to NOT cascade.
If someone wants cascading as a publication option, that can be added later.
I agree that not replicating the CASCADE option is the best option
according to POLA principle.+ /* logicalrep_rel_close call not needed, because ExecuteTruncateGuts + * already closes the relations. Setting localrel to NULL in the map entry + * is still needed. + */ + rel->localrel = NULL;This is somewhat ugly. Perhaps the ExecuteTruncateGuts should track
which relations it opened and only close those and the rest should be
closed by caller? That should also remove the other ugly part which is
that the ExecuteTruncateGuts modifies the input list. What if caller
wanted to use those relations it sent as parameter later?Agreed
Attached a new version of the patch addressing these issues.
Besides the small thing I wrote for the 0001 in the other thread I am
pretty much happy with this now.
--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Re: [PATCH] Logical decoding of TRUNCATE
On 22/01/18 19:45, Petr Jelinek wrote:
On 19/01/18 12:37, Marco Nenciarini wrote:
Hi All,
+ /* logicalrep_rel_close call not needed, because ExecuteTruncateGuts + * already closes the relations. Setting localrel to NULL in the map entry + * is still needed. + */ + rel->localrel = NULL;This is somewhat ugly. Perhaps the ExecuteTruncateGuts should track
which relations it opened and only close those and the rest should be
closed by caller? That should also remove the other ugly part which is
that the ExecuteTruncateGuts modifies the input list. What if caller
wanted to use those relations it sent as parameter later?Agreed
Attached a new version of the patch addressing these issues.
Besides the small thing I wrote for the 0001 in the other thread I am
pretty much happy with this now.
Actually on second look, I don't like the new boolean parameter much.
I'd rather we didn't touch the input list and always close only
relations opened inside the ExecuteTruncateGuts().
It may mean more list(s) but the current interface is still not clean.
--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Re: [PATCH] Logical decoding of TRUNCATE
Il 22/01/18 23:18, Petr Jelinek ha scritto:
On 22/01/18 19:45, Petr Jelinek wrote:
Actually on second look, I don't like the new boolean parameter much.
I'd rather we didn't touch the input list and always close only
relations opened inside the ExecuteTruncateGuts().It may mean more list(s) but the current interface is still not clean.
Now ExecuteTruncateGuts unconditionally closes the relations that it
opens. The caller has now always the responsibility to close the
relations passed with the explicit_rels list.
Version 15 attached.
Regards,
Marco
--
Marco Nenciarini - 2ndQuadrant Italy
PostgreSQL Training, Services and Support
marco.nenciarini@2ndQuadrant.it | www.2ndQuadrant.it
Attachments:
0001-truncate_ignore_fks_role_replica.v4.patchtext/plain; charset=UTF-8; name=0001-truncate_ignore_fks_role_replica.v4.patch; x-mac-creator=0; x-mac-type=0Download
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 2e768dd5e4..bdce4164d6 100644
*** a/src/backend/commands/tablecmds.c
--- b/src/backend/commands/tablecmds.c
***************
*** 1404,1419 **** ExecuteTruncate(TruncateStmt *stmt)
}
/*
! * Check foreign key references. In CASCADE mode, this should be
! * unnecessary since we just pulled in all the references; but as a
! * cross-check, do it anyway if in an Assert-enabled build.
*/
#ifdef USE_ASSERT_CHECKING
- heap_truncate_check_FKs(rels, false);
- #else
- if (stmt->behavior == DROP_RESTRICT)
heap_truncate_check_FKs(rels, false);
#endif
/*
* If we are asked to restart sequences, find all the sequences, lock them
--- 1404,1427 ----
}
/*
! * Suppress foreign key references check if session replication role is
! * set to REPLICA.
*/
+ if (SessionReplicationRole != SESSION_REPLICATION_ROLE_REPLICA)
+ {
+
+ /*
+ * Check foreign key references. In CASCADE mode, this should be
+ * unnecessary since we just pulled in all the references; but as a
+ * cross-check, do it anyway if in an Assert-enabled build.
+ */
#ifdef USE_ASSERT_CHECKING
heap_truncate_check_FKs(rels, false);
+ #else
+ if (stmt->behavior == DROP_RESTRICT)
+ heap_truncate_check_FKs(rels, false);
#endif
+ }
/*
* If we are asked to restart sequences, find all the sequences, lock them
diff --git a/src/test/regress/expected/index d967e8dd21..86748430c5 100644
*** a/src/test/regress/expected/truncate.out
--- b/src/test/regress/expected/truncate.out
***************
*** 68,73 **** HINT: Truncate table "trunc_b" at the same time, or use TRUNCATE ... CASCADE.
--- 68,77 ----
TRUNCATE TABLE truncate_a CASCADE; -- ok
NOTICE: truncate cascades to table "trunc_b"
NOTICE: truncate cascades to table "trunc_e"
+ -- Ignore foreign-key checks with session_replication_role = replica
+ SET session_replication_role = replica;
+ TRUNCATE TABLE truncate_a; -- ok
+ RESET session_replication_role;
-- circular references
ALTER TABLE truncate_a ADD FOREIGN KEY (col1) REFERENCES trunc_c;
-- Add some data to verify that truncating actually works ...
diff --git a/src/test/regress/sql/truncate.sqindex fbd1d1a8a5..0d0a3705d2 100644
*** a/src/test/regress/sql/truncate.sql
--- b/src/test/regress/sql/truncate.sql
***************
*** 33,38 **** TRUNCATE TABLE trunc_c,trunc_d,trunc_e,truncate_a,trunc_b; -- ok
--- 33,43 ----
TRUNCATE TABLE truncate_a RESTRICT; -- fail
TRUNCATE TABLE truncate_a CASCADE; -- ok
+ -- Ignore foreign-key checks with session_replication_role = replica
+ SET session_replication_role = replica;
+ TRUNCATE TABLE truncate_a; -- ok
+ RESET session_replication_role;
+
-- circular references
ALTER TABLE truncate_a ADD FOREIGN KEY (col1) REFERENCES trunc_c;
0002-logical_decode_truncate.v15.patchtext/plain; charset=UTF-8; name=0002-logical_decode_truncate.v15.patch; x-mac-creator=0; x-mac-type=0Download
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 1e22c1eefc..d1feea4909 100644
*** a/contrib/test_decoding/expected/ddl.out
--- b/contrib/test_decoding/expected/ddl.out
***************
*** 543,548 **** UPDATE table_with_unique_not_null SET data = 3 WHERE data = 2;
--- 543,550 ----
UPDATE table_with_unique_not_null SET id = -id;
UPDATE table_with_unique_not_null SET id = -id;
DELETE FROM table_with_unique_not_null WHERE data = 3;
+ TRUNCATE table_with_unique_not_null;
+ TRUNCATE table_with_unique_not_null, table_with_unique_not_null RESTART IDENTITY CASCADE;
-- check toast support
BEGIN;
CREATE SEQUENCE toasttable_rand_seq START 79 INCREMENT 1499; -- portable "random"
***************
*** 660,665 **** SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
--- 662,673 ----
table public.table_with_unique_not_null: DELETE: id[integer]:4
COMMIT
BEGIN
+ table public.table_with_unique_not_null: TRUNCATE: (no-flags)
+ COMMIT
+ BEGIN
+ table public.table_with_unique_not_null: TRUNCATE: restart_seqs cascade
+ COMMIT
+ BEGIN
table public.toasttable: INSERT: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
BEGIN
***************
*** 668,674 **** SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
BEGIN
table public.toasttable: UPDATE: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
! (103 rows)
INSERT INTO toasttable(toasted_col1) SELECT string_agg(g.i::text, '') FROM generate_series(1, 2000) g(i);
-- update of second column, first column unchanged
--- 676,682 ----
BEGIN
table public.toasttable: UPDATE: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
! (109 rows)
INSERT INTO toasttable(toasted_col1) SELECT string_agg(g.i::text, '') FROM generate_series(1, 2000) g(i);
-- update of second column, first column unchanged
diff --git a/contrib/test_decoding/sql/ddl.sqindex 057dae056b..0fb4102db8 100644
*** a/contrib/test_decoding/sql/ddl.sql
--- b/contrib/test_decoding/sql/ddl.sql
***************
*** 333,338 **** UPDATE table_with_unique_not_null SET id = -id;
--- 333,341 ----
UPDATE table_with_unique_not_null SET id = -id;
DELETE FROM table_with_unique_not_null WHERE data = 3;
+ TRUNCATE table_with_unique_not_null;
+ TRUNCATE table_with_unique_not_null, table_with_unique_not_null RESTART IDENTITY CASCADE;
+
-- check toast support
BEGIN;
CREATE SEQUENCE toasttable_rand_seq START 79 INCREMENT 1499; -- portable "random"
diff --git a/contrib/test_decoding/test_index 0f18afa852..55c4593ed5 100644
*** a/contrib/test_decoding/test_decoding.c
--- b/contrib/test_decoding/test_decoding.c
***************
*** 466,471 **** pg_decode_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
--- 466,485 ----
&change->data.tp.oldtuple->tuple,
true);
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ appendStringInfoString(ctx->out, " TRUNCATE:");
+
+ if (change->data.truncate_msg.restart_seqs
+ || change->data.truncate_msg.cascade)
+ {
+ if (change->data.truncate_msg.restart_seqs)
+ appendStringInfo(ctx->out, " restart_seqs");
+ if (change->data.truncate_msg.cascade)
+ appendStringInfo(ctx->out, " cascade");
+ }
+ else
+ appendStringInfoString(ctx->out, " (no-flags)");
+ break;
default:
Assert(false);
}
diff --git a/doc/src/sgml/logical-replicatioindex 75551d8ee1..6b64c8ef76 100644
*** a/doc/src/sgml/logical-replication.sgml
--- b/doc/src/sgml/logical-replication.sgml
***************
*** 111,116 ****
--- 111,118 ----
any combination of <command>INSERT</command>, <command>UPDATE</command>, and
<command>DELETE</command>, similar to how triggers are fired by
particular event types. By default, all operation types are replicated.
+ <command>TRUNCATE</command> is treated as a form of <command>DELETE</command>
+ for the purpose of deciding whether to publish, or not.
</para>
<para>
***************
*** 364,378 ****
</para>
</listitem>
- <listitem>
- <para>
- <command>TRUNCATE</command> commands are not replicated. This can, of
- course, be worked around by using <command>DELETE</command> instead. To
- avoid accidental <command>TRUNCATE</command> invocations, you can revoke
- the <literal>TRUNCATE</literal> privilege from tables.
- </para>
- </listitem>
-
<listitem>
<para>
Large objects (see <xref linkend="largeobjects"/>) are not replicated.
--- 366,371 ----
diff --git a/doc/src/sgml/protocol.sgml b/doindex 4c5ed1e6d6..d2a0d7e52e 100644
*** a/doc/src/sgml/protocol.sgml
--- b/doc/src/sgml/protocol.sgml
***************
*** 6838,6843 **** TupleData
--- 6838,6889 ----
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ Truncate
+ </term>
+ <listitem>
+ <para>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ Byte1('T')
+ </term>
+ <listitem>
+ <para>
+ Identifies the message as a truncate message.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ Int32
+ </term>
+ <listitem>
+ <para>
+ ID of the relation corresponding to the ID in the relation
+ message.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ Int8
+ </term>
+ <listitem>
+ <para>
+ Option flags for truncate. Currently can be 0 for no flags,
+ 1 for CASCADE, 2 for RESTART IDENTITY and 3 if both are set.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect1>
diff --git a/doc/src/sgml/ref/creindex bfe12d5f41..c5032938a7 100644
*** a/doc/src/sgml/ref/create_publication.sgml
--- b/doc/src/sgml/ref/create_publication.sgml
***************
*** 111,116 **** CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
--- 111,121 ----
and so the default value for this option is
<literal>'insert, update, delete'</literal>.
</para>
+ <para>
+ <command>TRUNCATE</command> is treated as a form of
+ <command>DELETE</command> for the purpose of deciding whether
+ to publish, or not.
+ </para>
</listitem>
</varlistentry>
</variablelist>
***************
*** 168,175 **** CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
</para>
<para>
! <command>TRUNCATE</command> and <acronym>DDL</acronym> operations
! are not published.
</para>
</refsect1>
--- 173,179 ----
</para>
<para>
! <acronym>DDL</acronym> operations are not published.
</para>
</refsect1>
diff --git a/src/backend/access/heap/heapam.c bindex be263850cd..f6fa99d48e 100644
*** a/src/backend/access/heap/heapam.c
--- b/src/backend/access/heap/heapam.c
***************
*** 9137,9142 **** heap_redo(XLogReaderState *record)
--- 9137,9149 ----
case XLOG_HEAP_UPDATE:
heap_xlog_update(record, false);
break;
+ case XLOG_HEAP_TRUNCATE:
+ /*
+ * TRUNCATE is a no-op because the actions are already logged
+ * as SMGR WAL records. TRUNCATE WAL record only exists to allow
+ * it to be logically decoded precisely.
+ */
+ break;
case XLOG_HEAP_HOT_UPDATE:
heap_xlog_update(record, true);
break;
diff --git a/src/backend/access/rmgrdesindex b00c071cb6..15550cdff0 100644
*** a/src/backend/access/rmgrdesc/heapdesc.c
--- b/src/backend/access/rmgrdesc/heapdesc.c
***************
*** 75,80 **** heap_desc(StringInfo buf, XLogReaderState *record)
--- 75,93 ----
xlrec->new_offnum,
xlrec->new_xmax);
}
+ else if (info == XLOG_HEAP_TRUNCATE)
+ {
+ xl_heap_truncate *xlrec = (xl_heap_truncate *) rec;
+
+ if (xlrec->flags & XLH_TRUNCATE_CASCADE)
+ appendStringInfo(buf, "cascade ");
+ if (xlrec->flags & XLH_TRUNCATE_RESTART_SEQS)
+ appendStringInfo(buf, "restart_seqs ");
+ appendStringInfo(buf, "nrelids %u nseqrelids %u",
+ xlrec->nrelids,
+ xlrec->nseqrelids);
+ /* Skip the list of relids and seqrelids */
+ }
else if (info == XLOG_HEAP_CONFIRM)
{
xl_heap_confirm *xlrec = (xl_heap_confirm *) rec;
***************
*** 186,191 **** heap_identify(uint8 info)
--- 199,207 ----
case XLOG_HEAP_HOT_UPDATE | XLOG_HEAP_INIT_PAGE:
id = "HOT_UPDATE+INIT";
break;
+ case XLOG_HEAP_TRUNCATE:
+ id = "TRUNCATE";
+ break;
case XLOG_HEAP_CONFIRM:
id = "HEAP_CONFIRM";
break;
diff --git a/src/backend/commands/tablecmds.cindex bdce4164d6..61f700d8b2 100644
*** a/src/backend/commands/tablecmds.c
--- b/src/backend/commands/tablecmds.c
***************
*** 16,21 ****
--- 16,22 ----
#include "access/genam.h"
#include "access/heapam.h"
+ #include "access/heapam_xlog.h"
#include "access/multixact.h"
#include "access/reloptions.h"
#include "access/relscan.h"
***************
*** 1312,1322 **** ExecuteTruncate(TruncateStmt *stmt)
{
List *rels = NIL;
List *relids = NIL;
! List *seq_relids = NIL;
! EState *estate;
! ResultRelInfo *resultRelInfos;
! ResultRelInfo *resultRelInfo;
! SubTransactionId mySubid;
ListCell *cell;
/*
--- 1313,1319 ----
{
List *rels = NIL;
List *relids = NIL;
! List *relids_logged = NIL;
ListCell *cell;
/*
***************
*** 1340,1345 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1337,1345 ----
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, myrelid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, myrelid);
if (recurse)
{
***************
*** 1360,1365 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1360,1368 ----
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, childrelid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, childrelid);
}
}
else if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
***************
*** 1369,1375 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1372,1410 ----
errhint("Do not specify the ONLY keyword, or use truncate only on the partitions directly.")));
}
+ ExecuteTruncateGuts(rels, relids, relids_logged,
+ stmt->behavior, stmt->restart_seqs);
+
+ /* And close the rels */
+ foreach(cell, rels)
+ {
+ Relation rel = (Relation) lfirst(cell);
+
+ heap_close(rel, NoLock);
+ }
+
+ }
+
+ void
+ ExecuteTruncateGuts(List *explicit_rels, List *relids, List *relids_logged,
+ DropBehavior behavior, bool restart_seqs)
+ {
+ List *rels = list_copy(explicit_rels);
+ List *seq_relids = NIL;
+ EState *estate;
+ ResultRelInfo *resultRelInfos;
+ ResultRelInfo *resultRelInfo;
+ SubTransactionId mySubid;
+ ListCell *cell;
+ List *seq_relids_logged = NIL;
+ uint32 nrelids = 0;
+ uint32 nseqrelids = 0;
+ uint32 maxrelids = 2;
+ Oid *logrelids = NULL;
+
/*
+ * Open, exclusive-lock, and check all the explicitly-specified relations
+ *
* In CASCADE mode, suck in all referencing relations as well. This
* requires multiple iterations to find indirectly-dependent relations. At
* each phase, we need to exclusive-lock new rels before looking for their
***************
*** 1377,1383 **** ExecuteTruncate(TruncateStmt *stmt)
* soon as we open it, to avoid a faux pas such as holding lock for a long
* time on a rel we have no permissions for.
*/
! if (stmt->behavior == DROP_CASCADE)
{
for (;;)
{
--- 1412,1418 ----
* soon as we open it, to avoid a faux pas such as holding lock for a long
* time on a rel we have no permissions for.
*/
! if (behavior == DROP_CASCADE)
{
for (;;)
{
***************
*** 1399,1404 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1434,1442 ----
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, relid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, relid);
}
}
}
***************
*** 1418,1424 **** ExecuteTruncate(TruncateStmt *stmt)
#ifdef USE_ASSERT_CHECKING
heap_truncate_check_FKs(rels, false);
#else
! if (stmt->behavior == DROP_RESTRICT)
heap_truncate_check_FKs(rels, false);
#endif
}
--- 1456,1462 ----
#ifdef USE_ASSERT_CHECKING
heap_truncate_check_FKs(rels, false);
#else
! if (behavior == DROP_RESTRICT)
heap_truncate_check_FKs(rels, false);
#endif
}
***************
*** 1429,1435 **** ExecuteTruncate(TruncateStmt *stmt)
* We want to do this early since it's pointless to do all the truncation
* work only to fail on sequence permissions.
*/
! if (stmt->restart_seqs)
{
foreach(cell, rels)
{
--- 1467,1473 ----
* We want to do this early since it's pointless to do all the truncation
* work only to fail on sequence permissions.
*/
! if (restart_seqs)
{
foreach(cell, rels)
{
***************
*** 1450,1455 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1488,1497 ----
RelationGetRelationName(seq_rel));
seq_relids = lappend_oid(seq_relids, seq_relid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(seq_rel))
+ seq_relids_logged = lappend_oid(seq_relids_logged,
+ seq_relid);
relation_close(seq_rel, NoLock);
}
***************
*** 1584,1589 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1626,1685 ----
ResetSequence(seq_relid);
}
+ /*
+ * Write a WAL record to allow this set of actions to be logically decoded.
+ * We could optimize this away when !RelationIsLogicallyLogged(rel)
+ * but that doesn't save much space or time.
+ *
+ * Assemble an array of relids, then an array of seqrelids so we can write
+ * a single WAL record for the whole action.
+ */
+ logrelids = palloc(maxrelids * sizeof(Oid));
+ foreach (cell, relids_logged)
+ {
+ nrelids++;
+ if (nrelids > maxrelids)
+ {
+ maxrelids *= 2;
+ logrelids = repalloc(logrelids, maxrelids * sizeof(Oid));
+ }
+ logrelids[nrelids - 1] = lfirst_oid(cell);
+ }
+
+ foreach (cell, seq_relids_logged)
+ {
+ nseqrelids++;
+ if ((nrelids + nseqrelids) > maxrelids)
+ {
+ maxrelids *= 2;
+ logrelids = repalloc(logrelids, maxrelids * sizeof(Oid));
+ }
+ logrelids[nrelids + nseqrelids - 1] = lfirst_oid(cell);
+ }
+
+ if ((nrelids + nseqrelids) > 0)
+ {
+ xl_heap_truncate xlrec;
+
+ xlrec.dbId = MyDatabaseId;
+ xlrec.nrelids = nrelids;
+ xlrec.nseqrelids = nseqrelids;
+ xlrec.flags = 0;
+ if (behavior == DROP_CASCADE)
+ xlrec.flags |= XLH_TRUNCATE_CASCADE;
+ if (restart_seqs)
+ xlrec.flags |= XLH_TRUNCATE_RESTART_SEQS;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfHeapTruncate);
+ XLogRegisterData((char *) logrelids,
+ (nrelids + nseqrelids) * sizeof(Oid));
+
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ (void) XLogInsert(RM_HEAP_ID, XLOG_HEAP_TRUNCATE);
+ }
+
/*
* Process all AFTER STATEMENT TRUNCATE triggers.
*/
***************
*** 1601,1607 **** ExecuteTruncate(TruncateStmt *stmt)
/* We can clean up the EState now */
FreeExecutorState(estate);
! /* And close the rels (can't do this while EState still holds refs) */
foreach(cell, rels)
{
Relation rel = (Relation) lfirst(cell);
--- 1697,1706 ----
/* We can clean up the EState now */
FreeExecutorState(estate);
! /* And close the eventual rels opened by CASCADE
! * (can't do this while EState still holds refs)
! */
! rels = list_difference_ptr(rels, explicit_rels);
foreach(cell, rels)
{
Relation rel = (Relation) lfirst(cell);
diff --git a/src/backend/replication/loindex 6eb0d5527e..ef5b937807 100644
*** a/src/backend/replication/logical/decode.c
--- b/src/backend/replication/logical/decode.c
***************
*** 65,70 **** static void DecodeLogicalMsgOp(LogicalDecodingContext *ctx, XLogRecordBuffer *bu
--- 65,71 ----
static void DecodeInsert(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeUpdate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeDelete(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
+ static void DecodeTruncate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeMultiInsert(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeSpecConfirm(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
***************
*** 449,454 **** DecodeHeapOp(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
--- 450,460 ----
DecodeDelete(ctx, buf);
break;
+ case XLOG_HEAP_TRUNCATE:
+ if (SnapBuildProcessChange(builder, xid, buf->origptr))
+ DecodeTruncate(ctx, buf);
+ break;
+
case XLOG_HEAP_INPLACE:
/*
***************
*** 825,830 **** DecodeDelete(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
--- 831,876 ----
ReorderBufferQueueChange(ctx->reorder, XLogRecGetXid(r), buf->origptr, change);
}
+ /*
+ * Parse XLOG_HEAP_TRUNCATE from wal
+ */
+ static void
+ DecodeTruncate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+ {
+ XLogReaderState *r = buf->record;
+ xl_heap_truncate *xlrec;
+ ReorderBufferChange *change;
+ int i;
+
+ xlrec = (xl_heap_truncate *) XLogRecGetData(r);
+
+ /* only interested in our database */
+ if (xlrec->dbId != ctx->slot->data.database)
+ return;
+
+ /* output plugin doesn't look for this origin, no need to queue */
+ if (FilterByOrigin(ctx, XLogRecGetOrigin(r)))
+ return;
+
+ change = ReorderBufferGetChange(ctx->reorder);
+ change->action = REORDER_BUFFER_CHANGE_TRUNCATE;
+ change->origin_id = XLogRecGetOrigin(r);
+ if (xlrec->flags & XLH_TRUNCATE_CASCADE)
+ change->data.truncate_msg.cascade = true;
+ if (xlrec->flags & XLH_TRUNCATE_RESTART_SEQS)
+ change->data.truncate_msg.restart_seqs = true;
+
+ /*
+ * Queue up one change per relation, ignoring sequences for now
+ */
+ for (i = 0; i < xlrec->nrelids; i++)
+ {
+ change->data.truncate_msg.relid = xlrec->relids[i];
+ ReorderBufferQueueChange(ctx->reorder, XLogRecGetXid(r),
+ buf->origptr, change);
+ }
+ }
+
/*
* Decode XLOG_HEAP2_MULTI_INSERT_insert record into multiple tuplebufs.
*
diff --git a/src/backend/replication/logical/prindex 948343e4ae..2fa6f8393d 100644
*** a/src/backend/replication/logical/proto.c
--- b/src/backend/replication/logical/proto.c
***************
*** 26,31 ****
--- 26,34 ----
*/
#define LOGICALREP_IS_REPLICA_IDENTITY 1
+ #define TRUNCATE_CASCADE (1<<0)
+ #define TRUNCATE_RESTART_SEQS (1<<1)
+
static void logicalrep_write_attrs(StringInfo out, Relation rel);
static void logicalrep_write_tuple(StringInfo out, Relation rel,
HeapTuple tuple);
***************
*** 292,297 **** logicalrep_read_delete(StringInfo in, LogicalRepTupleData *oldtup)
--- 295,342 ----
return relid;
}
+ /*
+ * Write TRUNCATE to the output stream.
+ */
+ void
+ logicalrep_write_truncate(StringInfo out, Relation rel,
+ bool cascade, bool restart_seqs)
+ {
+ uint8 flags = 0;
+
+ pq_sendbyte(out, 'T'); /* action TRUNCATE */
+
+ /* use Oid as relation identifier */
+ pq_sendint32(out, RelationGetRelid(rel));
+
+ /* encode and send truncate flags */
+ if (cascade)
+ flags |= TRUNCATE_CASCADE;
+ if (restart_seqs)
+ flags |= TRUNCATE_RESTART_SEQS;
+ pq_sendint8(out, flags);
+ }
+
+ /*
+ * Read TRUNCATE from stream.
+ */
+ LogicalRepRelId
+ logicalrep_read_truncate(StringInfo in, bool *cascade, bool *restart_seqs)
+ {
+ LogicalRepRelId relid;
+ uint8 flags;
+
+ /* read the relation id */
+ relid = pq_getmsgint(in, 4);
+
+ /* read and decode truncate flags */
+ flags = pq_getmsgint(in, 1);
+ *cascade = (flags & TRUNCATE_CASCADE) > 0;
+ *restart_seqs = (flags & TRUNCATE_RESTART_SEQS) > 0;
+
+ return relid;
+ }
+
/*
* Write relation description to the output stream.
*/
diff --git a/src/backend/replication/logical/rindex c72a611a39..f1d65c1457 100644
*** a/src/backend/replication/logical/reorderbuffer.c
--- b/src/backend/replication/logical/reorderbuffer.c
***************
*** 403,408 **** ReorderBufferReturnChange(ReorderBuffer *rb, ReorderBufferChange *change)
--- 403,410 ----
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ break;
}
pfree(change);
***************
*** 1342,1347 **** ReorderBufferCommit(ReorderBuffer *rb, TransactionId xid,
--- 1344,1356 ----
switch (change->action)
{
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ reloid = change->data.truncate_msg.relid;
+ relation = RelationIdGetRelation(reloid);
+ rb->apply_change(rb, txn, relation, change);
+ RelationClose(relation);
+ break;
+
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
/*
***************
*** 2239,2244 **** ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
--- 2248,2254 ----
}
break;
}
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
***************
*** 2524,2529 **** ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
--- 2534,2540 ----
break;
}
/* the base struct contains all the data, easy peasy */
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
diff --git a/src/backend/replication/logical/worker.c index 83c69092ae..12add0b4b9 100644
*** a/src/backend/replication/logical/worker.c
--- b/src/backend/replication/logical/worker.c
***************
*** 869,874 **** apply_handle_delete(StringInfo s)
--- 869,925 ----
CommandCounterIncrement();
}
+ /*
+ * Handle TRUNCATE message.
+ *
+ * TODO: FDW support
+ */
+ static void
+ apply_handle_truncate(StringInfo s)
+ {
+ LogicalRepRelMapEntry *rel;
+ LogicalRepRelId relid;
+ bool cascade = false;
+ bool restart_seqs = false;
+ List *rels = NIL;
+ List *relids = NIL;
+
+ ensure_transaction();
+
+ relid = logicalrep_read_truncate(s, &cascade, &restart_seqs);
+ rel = logicalrep_rel_open(relid, RowExclusiveLock);
+ if (!should_apply_changes_for_rel(rel))
+ {
+ /*
+ * The relation can't become interesting in the middle of the
+ * transaction so it's safe to unlock it.
+ */
+ logicalrep_rel_close(rel, RowExclusiveLock);
+ return;
+ }
+
+ /* Check if we can do the truncate. */
+ check_relation_updatable(rel);
+
+ rels = lappend(rels, rel->localrel);
+ relids = lappend_oid(relids, rel->localreloid);
+
+ /*
+ * Even if we used CASCADE on the upstream master we explicitly
+ * default to replaying changes without further cascading.
+ * This might be later changeable with a user specified option.
+ */
+ cascade = false;
+
+ ExecuteTruncateGuts(rels, relids, NULL,
+ cascade ? DROP_CASCADE : DROP_RESTRICT,
+ restart_seqs);
+
+ logicalrep_rel_close(rel, NoLock);
+
+ CommandCounterIncrement();
+ }
+
/*
* Logical replication protocol message dispatcher.
***************
*** 900,905 **** apply_dispatch(StringInfo s)
--- 951,960 ----
case 'D':
apply_handle_delete(s);
break;
+ /* TRUNCATE */
+ case 'T':
+ apply_handle_truncate(s);
+ break;
/* RELATION */
case 'R':
apply_handle_relation(s);
diff --git a/src/backend/replication/pgoutput/pindex 40a1ef3c1d..7fbd0a3dca 100644
*** a/src/backend/replication/pgoutput/pgoutput.c
--- b/src/backend/replication/pgoutput/pgoutput.c
***************
*** 275,280 **** pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
--- 275,281 ----
if (!relentry->pubactions.pubupdate)
return;
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
case REORDER_BUFFER_CHANGE_DELETE:
if (!relentry->pubactions.pubdelete)
return;
***************
*** 352,357 **** pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
--- 353,367 ----
else
elog(DEBUG1, "didn't send DELETE change because of missing oldtuple");
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ {
+ OutputPluginPrepareWrite(ctx, true);
+ logicalrep_write_truncate(ctx->out, relation,
+ change->data.truncate_msg.cascade,
+ change->data.truncate_msg.restart_seqs);
+ OutputPluginWrite(ctx, true);
+ }
+ break;
default:
Assert(false);
}
diff --git a/src/include/access/heapam_xlog.h b/srindex 700e25c36a..c56b200f7d 100644
*** a/src/include/access/heapam_xlog.h
--- b/src/include/access/heapam_xlog.h
***************
*** 32,38 ****
#define XLOG_HEAP_INSERT 0x00
#define XLOG_HEAP_DELETE 0x10
#define XLOG_HEAP_UPDATE 0x20
! /* 0x030 is free, was XLOG_HEAP_MOVE */
#define XLOG_HEAP_HOT_UPDATE 0x40
#define XLOG_HEAP_CONFIRM 0x50
#define XLOG_HEAP_LOCK 0x60
--- 32,38 ----
#define XLOG_HEAP_INSERT 0x00
#define XLOG_HEAP_DELETE 0x10
#define XLOG_HEAP_UPDATE 0x20
! #define XLOG_HEAP_TRUNCATE 0x30
#define XLOG_HEAP_HOT_UPDATE 0x40
#define XLOG_HEAP_CONFIRM 0x50
#define XLOG_HEAP_LOCK 0x60
***************
*** 109,114 **** typedef struct xl_heap_delete
--- 109,136 ----
#define SizeOfHeapDelete (offsetof(xl_heap_delete, flags) + sizeof(uint8))
+ /*
+ * xl_heap_delete flag values, 8 bits are available.
+ */
+ #define XLH_TRUNCATE_CASCADE (1<<0)
+ #define XLH_TRUNCATE_RESTART_SEQS (1<<1)
+
+ /*
+ * For truncate we list all truncated relids in an array, followed by all
+ * sequence relids that need to be restarted, if any.
+ * All rels are always within the same database, so we just list dbid once.
+ */
+ typedef struct xl_heap_truncate
+ {
+ Oid dbId;
+ uint32 nrelids;
+ uint32 nseqrelids;
+ uint8 flags;
+ Oid relids[FLEXIBLE_ARRAY_MEMBER];
+ } xl_heap_truncate;
+
+ #define SizeOfHeapTruncate (offsetof(xl_heap_truncate, relids))
+
/*
* We don't store the whole fixed part (HeapTupleHeaderData) of an inserted
* or updated tuple in WAL; we can save a few bytes by reconstructing the
diff --git a/src/include/executor/execuindex 6545a80222..8cb12a20e9 100644
*** a/src/include/executor/executor.h
--- b/src/include/executor/executor.h
***************
*** 558,561 **** extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
--- 558,567 ----
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
+ /*
+ * prototype for tablecmds.c accessible for logical apply
+ */
+ void ExecuteTruncateGuts(List *explicit_rels, List *relids, List *relids_logged,
+ DropBehavior behavior, bool restart_seqs);
+
#endif /* EXECUTOR_H */
diff --git a/src/include/replication/lindex 0eb21057c5..b1abb9e36f 100644
*** a/src/include/replication/logicalproto.h
--- b/src/include/replication/logicalproto.h
***************
*** 98,103 **** extern void logicalrep_write_delete(StringInfo out, Relation rel,
--- 98,107 ----
HeapTuple oldtuple);
extern LogicalRepRelId logicalrep_read_delete(StringInfo in,
LogicalRepTupleData *oldtup);
+ extern void logicalrep_write_truncate(StringInfo out, Relation rel,
+ bool cascade, bool restart_seqs);
+ extern LogicalRepRelId logicalrep_read_truncate(StringInfo in,
+ bool *cascade, bool *restart_seqs);
extern void logicalrep_write_rel(StringInfo out, Relation rel);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
extern void logicalrep_write_typ(StringInfo out, Oid typoid);
diff --git a/src/include/replication/reorderbindex 0970abca52..e8439fac49 100644
*** a/src/include/replication/reorderbuffer.h
--- b/src/include/replication/reorderbuffer.h
***************
*** 59,65 **** enum ReorderBufferChangeType
REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID,
REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID,
REORDER_BUFFER_CHANGE_INTERNAL_SPEC_INSERT,
! REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM
};
/*
--- 59,66 ----
REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID,
REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID,
REORDER_BUFFER_CHANGE_INTERNAL_SPEC_INSERT,
! REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM,
! REORDER_BUFFER_CHANGE_TRUNCATE
};
/*
***************
*** 128,134 **** typedef struct ReorderBufferChange
CommandId cmax;
CommandId combocid;
} tuplecid;
! } data;
/*
* While in use this is how a change is linked into a transactions,
--- 129,146 ----
CommandId cmax;
CommandId combocid;
} tuplecid;
!
! /*
! * Truncate data for REORDER_BUFFER_CHANGE_TRUNCATE representing
! * one relation to be truncated.
! */
! struct
! {
! Oid relid;
! bool cascade;
! bool restart_seqs;
! } truncate_msg;
! } data;
/*
* While in use this is how a change is linked into a transactions,
diff --git a/src/test/subscription/t/001_rep_cindex e0104cd8d0..96a6071b3a 100644
*** a/src/test/subscription/t/001_rep_changes.pl
--- b/src/test/subscription/t/001_rep_changes.pl
***************
*** 3,9 **** use strict;
use warnings;
use PostgresNode;
use TestLib;
! use Test::More tests => 16;
# Initialize publisher node
my $node_publisher = get_new_node('publisher');
--- 3,9 ----
use warnings;
use PostgresNode;
use TestLib;
! use Test::More tests => 24;
# Initialize publisher node
my $node_publisher = get_new_node('publisher');
***************
*** 176,181 **** $result = $node_subscriber->safe_psql('postgres',
--- 176,191 ----
is($result, qq(1152|1|1100),
'check replicated inserts after subscription publication change');
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_rep");
+ is($result, qq(20|-20|-1),
+ 'check changes skipped after subscription publication change');
+
+ # truncate should not be replicated
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep");
+
+ $node_publisher->wait_for_catchup($appname);
+
$result = $node_subscriber->safe_psql('postgres',
"SELECT count(*), min(a), max(a) FROM tab_rep");
is($result, qq(20|-20|-1),
***************
*** 185,191 **** is($result, qq(20|-20|-1),
$node_publisher->safe_psql('postgres',
"ALTER PUBLICATION tap_pub_ins_only SET (publish = 'insert, delete')");
$node_publisher->safe_psql('postgres',
! "ALTER PUBLICATION tap_pub_ins_only ADD TABLE tab_full");
$node_publisher->safe_psql('postgres', "DELETE FROM tab_ins WHERE a > 0");
$node_subscriber->safe_psql('postgres',
"ALTER SUBSCRIPTION tap_sub REFRESH PUBLICATION WITH (copy_data = false)"
--- 195,201 ----
$node_publisher->safe_psql('postgres',
"ALTER PUBLICATION tap_pub_ins_only SET (publish = 'insert, delete')");
$node_publisher->safe_psql('postgres',
! "ALTER PUBLICATION tap_pub_ins_only ADD TABLE tab_full, tab_rep");
$node_publisher->safe_psql('postgres', "DELETE FROM tab_ins WHERE a > 0");
$node_subscriber->safe_psql('postgres',
"ALTER SUBSCRIPTION tap_sub REFRESH PUBLICATION WITH (copy_data = false)"
***************
*** 204,209 **** $result = $node_subscriber->safe_psql('postgres',
--- 214,282 ----
"SELECT count(*), min(a), max(a) FROM tab_full");
is($result, qq(21|0|100), 'check replicated insert after alter publication');
+ # add a sequence and a foreign key to check cascade and restart
+ # identity truncate options
+ $node_subscriber->safe_psql('postgres',
+ "CREATE TABLE tab_notrep_fk (a int REFERENCES tab_rep(a))"
+ );
+ $node_subscriber->safe_psql('postgres', "INSERT INTO tab_notrep_fk VALUES (-1)");
+ $node_subscriber->safe_psql('postgres',
+ "CREATE SEQUENCE seq_notrep OWNED BY tab_rep.a"
+ );
+ $node_subscriber->safe_psql('postgres',
+ "ALTER SEQUENCE seq_notrep START 101"
+ );
+
+ # truncate should now be replicated
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep");
+
+ $node_publisher->wait_for_catchup($appname);
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_rep");
+ is($result, qq(0||),
+ 'check replicated truncate after alter publication');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_notrep_fk");
+ is($result, qq(1|-1|-1),
+ 'check replicated truncate does not cascade');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq_notrep')");
+ is($result, qq(1),
+ 'check replicated truncate does not restart identities');
+
+ # should restart identity
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep RESTART IDENTITY");
+
+ $node_publisher->wait_for_catchup($appname);
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_notrep_fk");
+ is($result, qq(1|-1|-1),
+ 'check replicated truncate does not cascade');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq_notrep')");
+ is($result, qq(101),
+ 'check replicated truncate restart identities');
+
+ # should not cascade on replica
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep CASCADE");
+
+ $node_publisher->wait_for_catchup($appname);
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_notrep_fk");
+ is($result, qq(1|-1|-1),
+ 'check replicated truncate does not cascade on replica');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq_notrep')");
+ is($result, qq(102),
+ 'check replicated truncate does not restart identities');
+
# check restart on rename
$oldpid = $node_publisher->safe_psql('postgres',
"SELECT pid FROM pg_stat_replication WHERE application_name = '$appname';"
Re: [PATCH] Logical decoding of TRUNCATE
Hi,
On 23/01/18 15:38, Marco Nenciarini wrote:
Il 22/01/18 23:18, Petr Jelinek ha scritto:
On 22/01/18 19:45, Petr Jelinek wrote:
Actually on second look, I don't like the new boolean parameter much.
I'd rather we didn't touch the input list and always close only
relations opened inside the ExecuteTruncateGuts().It may mean more list(s) but the current interface is still not clean.
Now ExecuteTruncateGuts unconditionally closes the relations that it
opens. The caller has now always the responsibility to close the
relations passed with the explicit_rels list.
This looks good.
Version 15 attached.
I see you still do CASCADE on the subscriber though.
--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Re: [PATCH] Logical decoding of TRUNCATE
Il 23/01/18 18:13, Petr Jelinek ha scritto:
Hi,
On 23/01/18 15:38, Marco Nenciarini wrote:
Il 22/01/18 23:18, Petr Jelinek ha scritto:
On 22/01/18 19:45, Petr Jelinek wrote:
Actually on second look, I don't like the new boolean parameter much.
I'd rather we didn't touch the input list and always close only
relations opened inside the ExecuteTruncateGuts().It may mean more list(s) but the current interface is still not clean.
Now ExecuteTruncateGuts unconditionally closes the relations that it
opens. The caller has now always the responsibility to close the
relations passed with the explicit_rels list.This looks good.
Version 15 attached.
I see you still do CASCADE on the subscriber though.
No it doesn't. The following code in worker.c prevents that.
+ /*
+ * Even if we used CASCADE on the upstream master we explicitly
+ * default to replaying changes without further cascading.
+ * This might be later changeable with a user specified option.
+ */
+ cascade = false;There is also a test that check it works as intended:
+ # should not cascade on replica
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep CASCADE");
+
+ $node_publisher->wait_for_catchup($appname);
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_notrep_fk");
+ is($result, qq(1|-1|-1),
+ 'check replicated truncate does not cascade on replica');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq_notrep')");
+ is($result, qq(102),
+ 'check replicated truncate does not restart identities');
+Regards,
Marco
--
Marco Nenciarini - 2ndQuadrant Italy
PostgreSQL Training, Services and Support
marco.nenciarini@2ndQuadrant.it | www.2ndQuadrant.it
Re: [PATCH] Logical decoding of TRUNCATE
On 23/01/18 18:19, Marco Nenciarini wrote:
Il 23/01/18 18:13, Petr Jelinek ha scritto:
Hi,
On 23/01/18 15:38, Marco Nenciarini wrote:
Il 22/01/18 23:18, Petr Jelinek ha scritto:
On 22/01/18 19:45, Petr Jelinek wrote:
Actually on second look, I don't like the new boolean parameter much.
I'd rather we didn't touch the input list and always close only
relations opened inside the ExecuteTruncateGuts().It may mean more list(s) but the current interface is still not clean.
Now ExecuteTruncateGuts unconditionally closes the relations that it
opens. The caller has now always the responsibility to close the
relations passed with the explicit_rels list.This looks good.
Version 15 attached.
I see you still do CASCADE on the subscriber though.
No it doesn't. The following code in worker.c prevents that.
+ /* + * Even if we used CASCADE on the upstream master we explicitly + * default to replaying changes without further cascading. + * This might be later changeable with a user specified option. + */ + cascade = false;
Ah, that's pretty ugly, why don't we just use DROP_RESTRICT always
instead of this (keeping the comment). Unless you plan to make it option
as part of this patch, the current coding is confusing.
--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Re: [PATCH] Logical decoding of TRUNCATE
Il 23/01/18 18:25, Petr Jelinek ha scritto:
On 23/01/18 18:19, Marco Nenciarini wrote:
Il 23/01/18 18:13, Petr Jelinek ha scritto:
Hi,
On 23/01/18 15:38, Marco Nenciarini wrote:
Il 22/01/18 23:18, Petr Jelinek ha scritto:
On 22/01/18 19:45, Petr Jelinek wrote:
Actually on second look, I don't like the new boolean parameter much.
I'd rather we didn't touch the input list and always close only
relations opened inside the ExecuteTruncateGuts().It may mean more list(s) but the current interface is still not clean.
Now ExecuteTruncateGuts unconditionally closes the relations that it
opens. The caller has now always the responsibility to close the
relations passed with the explicit_rels list.This looks good.
Version 15 attached.
I see you still do CASCADE on the subscriber though.
No it doesn't. The following code in worker.c prevents that.
+ /* + * Even if we used CASCADE on the upstream master we explicitly + * default to replaying changes without further cascading. + * This might be later changeable with a user specified option. + */ + cascade = false;Ah, that's pretty ugly, why don't we just use DROP_RESTRICT always
instead of this (keeping the comment). Unless you plan to make it option
as part of this patch, the current coding is confusing.
Ok, Removed.
Version 16 attached.
Regards,
Marco
--
Marco Nenciarini - 2ndQuadrant Italy
PostgreSQL Training, Services and Support
marco.nenciarini@2ndQuadrant.it | www.2ndQuadrant.it
Attachments:
0001-truncate_ignore_fks_role_replica.v4.patchtext/plain; charset=UTF-8; name=0001-truncate_ignore_fks_role_replica.v4.patch; x-mac-creator=0; x-mac-type=0Download
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 2e768dd5e4..bdce4164d6 100644
*** a/src/backend/commands/tablecmds.c
--- b/src/backend/commands/tablecmds.c
***************
*** 1404,1419 **** ExecuteTruncate(TruncateStmt *stmt)
}
/*
! * Check foreign key references. In CASCADE mode, this should be
! * unnecessary since we just pulled in all the references; but as a
! * cross-check, do it anyway if in an Assert-enabled build.
*/
#ifdef USE_ASSERT_CHECKING
- heap_truncate_check_FKs(rels, false);
- #else
- if (stmt->behavior == DROP_RESTRICT)
heap_truncate_check_FKs(rels, false);
#endif
/*
* If we are asked to restart sequences, find all the sequences, lock them
--- 1404,1427 ----
}
/*
! * Suppress foreign key references check if session replication role is
! * set to REPLICA.
*/
+ if (SessionReplicationRole != SESSION_REPLICATION_ROLE_REPLICA)
+ {
+
+ /*
+ * Check foreign key references. In CASCADE mode, this should be
+ * unnecessary since we just pulled in all the references; but as a
+ * cross-check, do it anyway if in an Assert-enabled build.
+ */
#ifdef USE_ASSERT_CHECKING
heap_truncate_check_FKs(rels, false);
+ #else
+ if (stmt->behavior == DROP_RESTRICT)
+ heap_truncate_check_FKs(rels, false);
#endif
+ }
/*
* If we are asked to restart sequences, find all the sequences, lock them
diff --git a/src/test/regress/expected/index d967e8dd21..86748430c5 100644
*** a/src/test/regress/expected/truncate.out
--- b/src/test/regress/expected/truncate.out
***************
*** 68,73 **** HINT: Truncate table "trunc_b" at the same time, or use TRUNCATE ... CASCADE.
--- 68,77 ----
TRUNCATE TABLE truncate_a CASCADE; -- ok
NOTICE: truncate cascades to table "trunc_b"
NOTICE: truncate cascades to table "trunc_e"
+ -- Ignore foreign-key checks with session_replication_role = replica
+ SET session_replication_role = replica;
+ TRUNCATE TABLE truncate_a; -- ok
+ RESET session_replication_role;
-- circular references
ALTER TABLE truncate_a ADD FOREIGN KEY (col1) REFERENCES trunc_c;
-- Add some data to verify that truncating actually works ...
diff --git a/src/test/regress/sql/truncate.sqindex fbd1d1a8a5..0d0a3705d2 100644
*** a/src/test/regress/sql/truncate.sql
--- b/src/test/regress/sql/truncate.sql
***************
*** 33,38 **** TRUNCATE TABLE trunc_c,trunc_d,trunc_e,truncate_a,trunc_b; -- ok
--- 33,43 ----
TRUNCATE TABLE truncate_a RESTRICT; -- fail
TRUNCATE TABLE truncate_a CASCADE; -- ok
+ -- Ignore foreign-key checks with session_replication_role = replica
+ SET session_replication_role = replica;
+ TRUNCATE TABLE truncate_a; -- ok
+ RESET session_replication_role;
+
-- circular references
ALTER TABLE truncate_a ADD FOREIGN KEY (col1) REFERENCES trunc_c;
0002-logical_decode_truncate.v16.patchtext/plain; charset=UTF-8; name=0002-logical_decode_truncate.v16.patch; x-mac-creator=0; x-mac-type=0Download
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 1e22c1eefc..d1feea4909 100644
*** a/contrib/test_decoding/expected/ddl.out
--- b/contrib/test_decoding/expected/ddl.out
***************
*** 543,548 **** UPDATE table_with_unique_not_null SET data = 3 WHERE data = 2;
--- 543,550 ----
UPDATE table_with_unique_not_null SET id = -id;
UPDATE table_with_unique_not_null SET id = -id;
DELETE FROM table_with_unique_not_null WHERE data = 3;
+ TRUNCATE table_with_unique_not_null;
+ TRUNCATE table_with_unique_not_null, table_with_unique_not_null RESTART IDENTITY CASCADE;
-- check toast support
BEGIN;
CREATE SEQUENCE toasttable_rand_seq START 79 INCREMENT 1499; -- portable "random"
***************
*** 660,665 **** SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
--- 662,673 ----
table public.table_with_unique_not_null: DELETE: id[integer]:4
COMMIT
BEGIN
+ table public.table_with_unique_not_null: TRUNCATE: (no-flags)
+ COMMIT
+ BEGIN
+ table public.table_with_unique_not_null: TRUNCATE: restart_seqs cascade
+ COMMIT
+ BEGIN
table public.toasttable: INSERT: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
BEGIN
***************
*** 668,674 **** SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
BEGIN
table public.toasttable: UPDATE: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
! (103 rows)
INSERT INTO toasttable(toasted_col1) SELECT string_agg(g.i::text, '') FROM generate_series(1, 2000) g(i);
-- update of second column, first column unchanged
--- 676,682 ----
BEGIN
table public.toasttable: UPDATE: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
! (109 rows)
INSERT INTO toasttable(toasted_col1) SELECT string_agg(g.i::text, '') FROM generate_series(1, 2000) g(i);
-- update of second column, first column unchanged
diff --git a/contrib/test_decoding/sql/ddl.sqindex 057dae056b..0fb4102db8 100644
*** a/contrib/test_decoding/sql/ddl.sql
--- b/contrib/test_decoding/sql/ddl.sql
***************
*** 333,338 **** UPDATE table_with_unique_not_null SET id = -id;
--- 333,341 ----
UPDATE table_with_unique_not_null SET id = -id;
DELETE FROM table_with_unique_not_null WHERE data = 3;
+ TRUNCATE table_with_unique_not_null;
+ TRUNCATE table_with_unique_not_null, table_with_unique_not_null RESTART IDENTITY CASCADE;
+
-- check toast support
BEGIN;
CREATE SEQUENCE toasttable_rand_seq START 79 INCREMENT 1499; -- portable "random"
diff --git a/contrib/test_decoding/test_index 0f18afa852..55c4593ed5 100644
*** a/contrib/test_decoding/test_decoding.c
--- b/contrib/test_decoding/test_decoding.c
***************
*** 466,471 **** pg_decode_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
--- 466,485 ----
&change->data.tp.oldtuple->tuple,
true);
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ appendStringInfoString(ctx->out, " TRUNCATE:");
+
+ if (change->data.truncate_msg.restart_seqs
+ || change->data.truncate_msg.cascade)
+ {
+ if (change->data.truncate_msg.restart_seqs)
+ appendStringInfo(ctx->out, " restart_seqs");
+ if (change->data.truncate_msg.cascade)
+ appendStringInfo(ctx->out, " cascade");
+ }
+ else
+ appendStringInfoString(ctx->out, " (no-flags)");
+ break;
default:
Assert(false);
}
diff --git a/doc/src/sgml/logical-replicatioindex 75551d8ee1..6b64c8ef76 100644
*** a/doc/src/sgml/logical-replication.sgml
--- b/doc/src/sgml/logical-replication.sgml
***************
*** 111,116 ****
--- 111,118 ----
any combination of <command>INSERT</command>, <command>UPDATE</command>, and
<command>DELETE</command>, similar to how triggers are fired by
particular event types. By default, all operation types are replicated.
+ <command>TRUNCATE</command> is treated as a form of <command>DELETE</command>
+ for the purpose of deciding whether to publish, or not.
</para>
<para>
***************
*** 364,378 ****
</para>
</listitem>
- <listitem>
- <para>
- <command>TRUNCATE</command> commands are not replicated. This can, of
- course, be worked around by using <command>DELETE</command> instead. To
- avoid accidental <command>TRUNCATE</command> invocations, you can revoke
- the <literal>TRUNCATE</literal> privilege from tables.
- </para>
- </listitem>
-
<listitem>
<para>
Large objects (see <xref linkend="largeobjects"/>) are not replicated.
--- 366,371 ----
diff --git a/doc/src/sgml/protocol.sgml b/doindex 4c5ed1e6d6..d2a0d7e52e 100644
*** a/doc/src/sgml/protocol.sgml
--- b/doc/src/sgml/protocol.sgml
***************
*** 6838,6843 **** TupleData
--- 6838,6889 ----
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ Truncate
+ </term>
+ <listitem>
+ <para>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ Byte1('T')
+ </term>
+ <listitem>
+ <para>
+ Identifies the message as a truncate message.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ Int32
+ </term>
+ <listitem>
+ <para>
+ ID of the relation corresponding to the ID in the relation
+ message.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ Int8
+ </term>
+ <listitem>
+ <para>
+ Option flags for truncate. Currently can be 0 for no flags,
+ 1 for CASCADE, 2 for RESTART IDENTITY and 3 if both are set.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect1>
diff --git a/doc/src/sgml/ref/creindex bfe12d5f41..c5032938a7 100644
*** a/doc/src/sgml/ref/create_publication.sgml
--- b/doc/src/sgml/ref/create_publication.sgml
***************
*** 111,116 **** CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
--- 111,121 ----
and so the default value for this option is
<literal>'insert, update, delete'</literal>.
</para>
+ <para>
+ <command>TRUNCATE</command> is treated as a form of
+ <command>DELETE</command> for the purpose of deciding whether
+ to publish, or not.
+ </para>
</listitem>
</varlistentry>
</variablelist>
***************
*** 168,175 **** CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
</para>
<para>
! <command>TRUNCATE</command> and <acronym>DDL</acronym> operations
! are not published.
</para>
</refsect1>
--- 173,179 ----
</para>
<para>
! <acronym>DDL</acronym> operations are not published.
</para>
</refsect1>
diff --git a/src/backend/access/heap/heapam.c bindex be263850cd..f6fa99d48e 100644
*** a/src/backend/access/heap/heapam.c
--- b/src/backend/access/heap/heapam.c
***************
*** 9137,9142 **** heap_redo(XLogReaderState *record)
--- 9137,9149 ----
case XLOG_HEAP_UPDATE:
heap_xlog_update(record, false);
break;
+ case XLOG_HEAP_TRUNCATE:
+ /*
+ * TRUNCATE is a no-op because the actions are already logged
+ * as SMGR WAL records. TRUNCATE WAL record only exists to allow
+ * it to be logically decoded precisely.
+ */
+ break;
case XLOG_HEAP_HOT_UPDATE:
heap_xlog_update(record, true);
break;
diff --git a/src/backend/access/rmgrdesindex b00c071cb6..15550cdff0 100644
*** a/src/backend/access/rmgrdesc/heapdesc.c
--- b/src/backend/access/rmgrdesc/heapdesc.c
***************
*** 75,80 **** heap_desc(StringInfo buf, XLogReaderState *record)
--- 75,93 ----
xlrec->new_offnum,
xlrec->new_xmax);
}
+ else if (info == XLOG_HEAP_TRUNCATE)
+ {
+ xl_heap_truncate *xlrec = (xl_heap_truncate *) rec;
+
+ if (xlrec->flags & XLH_TRUNCATE_CASCADE)
+ appendStringInfo(buf, "cascade ");
+ if (xlrec->flags & XLH_TRUNCATE_RESTART_SEQS)
+ appendStringInfo(buf, "restart_seqs ");
+ appendStringInfo(buf, "nrelids %u nseqrelids %u",
+ xlrec->nrelids,
+ xlrec->nseqrelids);
+ /* Skip the list of relids and seqrelids */
+ }
else if (info == XLOG_HEAP_CONFIRM)
{
xl_heap_confirm *xlrec = (xl_heap_confirm *) rec;
***************
*** 186,191 **** heap_identify(uint8 info)
--- 199,207 ----
case XLOG_HEAP_HOT_UPDATE | XLOG_HEAP_INIT_PAGE:
id = "HOT_UPDATE+INIT";
break;
+ case XLOG_HEAP_TRUNCATE:
+ id = "TRUNCATE";
+ break;
case XLOG_HEAP_CONFIRM:
id = "HEAP_CONFIRM";
break;
diff --git a/src/backend/commands/tablecmds.cindex bdce4164d6..61f700d8b2 100644
*** a/src/backend/commands/tablecmds.c
--- b/src/backend/commands/tablecmds.c
***************
*** 16,21 ****
--- 16,22 ----
#include "access/genam.h"
#include "access/heapam.h"
+ #include "access/heapam_xlog.h"
#include "access/multixact.h"
#include "access/reloptions.h"
#include "access/relscan.h"
***************
*** 1312,1322 **** ExecuteTruncate(TruncateStmt *stmt)
{
List *rels = NIL;
List *relids = NIL;
! List *seq_relids = NIL;
! EState *estate;
! ResultRelInfo *resultRelInfos;
! ResultRelInfo *resultRelInfo;
! SubTransactionId mySubid;
ListCell *cell;
/*
--- 1313,1319 ----
{
List *rels = NIL;
List *relids = NIL;
! List *relids_logged = NIL;
ListCell *cell;
/*
***************
*** 1340,1345 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1337,1345 ----
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, myrelid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, myrelid);
if (recurse)
{
***************
*** 1360,1365 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1360,1368 ----
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, childrelid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, childrelid);
}
}
else if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
***************
*** 1369,1375 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1372,1410 ----
errhint("Do not specify the ONLY keyword, or use truncate only on the partitions directly.")));
}
+ ExecuteTruncateGuts(rels, relids, relids_logged,
+ stmt->behavior, stmt->restart_seqs);
+
+ /* And close the rels */
+ foreach(cell, rels)
+ {
+ Relation rel = (Relation) lfirst(cell);
+
+ heap_close(rel, NoLock);
+ }
+
+ }
+
+ void
+ ExecuteTruncateGuts(List *explicit_rels, List *relids, List *relids_logged,
+ DropBehavior behavior, bool restart_seqs)
+ {
+ List *rels = list_copy(explicit_rels);
+ List *seq_relids = NIL;
+ EState *estate;
+ ResultRelInfo *resultRelInfos;
+ ResultRelInfo *resultRelInfo;
+ SubTransactionId mySubid;
+ ListCell *cell;
+ List *seq_relids_logged = NIL;
+ uint32 nrelids = 0;
+ uint32 nseqrelids = 0;
+ uint32 maxrelids = 2;
+ Oid *logrelids = NULL;
+
/*
+ * Open, exclusive-lock, and check all the explicitly-specified relations
+ *
* In CASCADE mode, suck in all referencing relations as well. This
* requires multiple iterations to find indirectly-dependent relations. At
* each phase, we need to exclusive-lock new rels before looking for their
***************
*** 1377,1383 **** ExecuteTruncate(TruncateStmt *stmt)
* soon as we open it, to avoid a faux pas such as holding lock for a long
* time on a rel we have no permissions for.
*/
! if (stmt->behavior == DROP_CASCADE)
{
for (;;)
{
--- 1412,1418 ----
* soon as we open it, to avoid a faux pas such as holding lock for a long
* time on a rel we have no permissions for.
*/
! if (behavior == DROP_CASCADE)
{
for (;;)
{
***************
*** 1399,1404 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1434,1442 ----
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, relid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, relid);
}
}
}
***************
*** 1418,1424 **** ExecuteTruncate(TruncateStmt *stmt)
#ifdef USE_ASSERT_CHECKING
heap_truncate_check_FKs(rels, false);
#else
! if (stmt->behavior == DROP_RESTRICT)
heap_truncate_check_FKs(rels, false);
#endif
}
--- 1456,1462 ----
#ifdef USE_ASSERT_CHECKING
heap_truncate_check_FKs(rels, false);
#else
! if (behavior == DROP_RESTRICT)
heap_truncate_check_FKs(rels, false);
#endif
}
***************
*** 1429,1435 **** ExecuteTruncate(TruncateStmt *stmt)
* We want to do this early since it's pointless to do all the truncation
* work only to fail on sequence permissions.
*/
! if (stmt->restart_seqs)
{
foreach(cell, rels)
{
--- 1467,1473 ----
* We want to do this early since it's pointless to do all the truncation
* work only to fail on sequence permissions.
*/
! if (restart_seqs)
{
foreach(cell, rels)
{
***************
*** 1450,1455 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1488,1497 ----
RelationGetRelationName(seq_rel));
seq_relids = lappend_oid(seq_relids, seq_relid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(seq_rel))
+ seq_relids_logged = lappend_oid(seq_relids_logged,
+ seq_relid);
relation_close(seq_rel, NoLock);
}
***************
*** 1584,1589 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1626,1685 ----
ResetSequence(seq_relid);
}
+ /*
+ * Write a WAL record to allow this set of actions to be logically decoded.
+ * We could optimize this away when !RelationIsLogicallyLogged(rel)
+ * but that doesn't save much space or time.
+ *
+ * Assemble an array of relids, then an array of seqrelids so we can write
+ * a single WAL record for the whole action.
+ */
+ logrelids = palloc(maxrelids * sizeof(Oid));
+ foreach (cell, relids_logged)
+ {
+ nrelids++;
+ if (nrelids > maxrelids)
+ {
+ maxrelids *= 2;
+ logrelids = repalloc(logrelids, maxrelids * sizeof(Oid));
+ }
+ logrelids[nrelids - 1] = lfirst_oid(cell);
+ }
+
+ foreach (cell, seq_relids_logged)
+ {
+ nseqrelids++;
+ if ((nrelids + nseqrelids) > maxrelids)
+ {
+ maxrelids *= 2;
+ logrelids = repalloc(logrelids, maxrelids * sizeof(Oid));
+ }
+ logrelids[nrelids + nseqrelids - 1] = lfirst_oid(cell);
+ }
+
+ if ((nrelids + nseqrelids) > 0)
+ {
+ xl_heap_truncate xlrec;
+
+ xlrec.dbId = MyDatabaseId;
+ xlrec.nrelids = nrelids;
+ xlrec.nseqrelids = nseqrelids;
+ xlrec.flags = 0;
+ if (behavior == DROP_CASCADE)
+ xlrec.flags |= XLH_TRUNCATE_CASCADE;
+ if (restart_seqs)
+ xlrec.flags |= XLH_TRUNCATE_RESTART_SEQS;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfHeapTruncate);
+ XLogRegisterData((char *) logrelids,
+ (nrelids + nseqrelids) * sizeof(Oid));
+
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ (void) XLogInsert(RM_HEAP_ID, XLOG_HEAP_TRUNCATE);
+ }
+
/*
* Process all AFTER STATEMENT TRUNCATE triggers.
*/
***************
*** 1601,1607 **** ExecuteTruncate(TruncateStmt *stmt)
/* We can clean up the EState now */
FreeExecutorState(estate);
! /* And close the rels (can't do this while EState still holds refs) */
foreach(cell, rels)
{
Relation rel = (Relation) lfirst(cell);
--- 1697,1706 ----
/* We can clean up the EState now */
FreeExecutorState(estate);
! /* And close the eventual rels opened by CASCADE
! * (can't do this while EState still holds refs)
! */
! rels = list_difference_ptr(rels, explicit_rels);
foreach(cell, rels)
{
Relation rel = (Relation) lfirst(cell);
diff --git a/src/backend/replication/loindex 6eb0d5527e..ef5b937807 100644
*** a/src/backend/replication/logical/decode.c
--- b/src/backend/replication/logical/decode.c
***************
*** 65,70 **** static void DecodeLogicalMsgOp(LogicalDecodingContext *ctx, XLogRecordBuffer *bu
--- 65,71 ----
static void DecodeInsert(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeUpdate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeDelete(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
+ static void DecodeTruncate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeMultiInsert(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeSpecConfirm(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
***************
*** 449,454 **** DecodeHeapOp(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
--- 450,460 ----
DecodeDelete(ctx, buf);
break;
+ case XLOG_HEAP_TRUNCATE:
+ if (SnapBuildProcessChange(builder, xid, buf->origptr))
+ DecodeTruncate(ctx, buf);
+ break;
+
case XLOG_HEAP_INPLACE:
/*
***************
*** 825,830 **** DecodeDelete(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
--- 831,876 ----
ReorderBufferQueueChange(ctx->reorder, XLogRecGetXid(r), buf->origptr, change);
}
+ /*
+ * Parse XLOG_HEAP_TRUNCATE from wal
+ */
+ static void
+ DecodeTruncate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+ {
+ XLogReaderState *r = buf->record;
+ xl_heap_truncate *xlrec;
+ ReorderBufferChange *change;
+ int i;
+
+ xlrec = (xl_heap_truncate *) XLogRecGetData(r);
+
+ /* only interested in our database */
+ if (xlrec->dbId != ctx->slot->data.database)
+ return;
+
+ /* output plugin doesn't look for this origin, no need to queue */
+ if (FilterByOrigin(ctx, XLogRecGetOrigin(r)))
+ return;
+
+ change = ReorderBufferGetChange(ctx->reorder);
+ change->action = REORDER_BUFFER_CHANGE_TRUNCATE;
+ change->origin_id = XLogRecGetOrigin(r);
+ if (xlrec->flags & XLH_TRUNCATE_CASCADE)
+ change->data.truncate_msg.cascade = true;
+ if (xlrec->flags & XLH_TRUNCATE_RESTART_SEQS)
+ change->data.truncate_msg.restart_seqs = true;
+
+ /*
+ * Queue up one change per relation, ignoring sequences for now
+ */
+ for (i = 0; i < xlrec->nrelids; i++)
+ {
+ change->data.truncate_msg.relid = xlrec->relids[i];
+ ReorderBufferQueueChange(ctx->reorder, XLogRecGetXid(r),
+ buf->origptr, change);
+ }
+ }
+
/*
* Decode XLOG_HEAP2_MULTI_INSERT_insert record into multiple tuplebufs.
*
diff --git a/src/backend/replication/logical/prindex 948343e4ae..2fa6f8393d 100644
*** a/src/backend/replication/logical/proto.c
--- b/src/backend/replication/logical/proto.c
***************
*** 26,31 ****
--- 26,34 ----
*/
#define LOGICALREP_IS_REPLICA_IDENTITY 1
+ #define TRUNCATE_CASCADE (1<<0)
+ #define TRUNCATE_RESTART_SEQS (1<<1)
+
static void logicalrep_write_attrs(StringInfo out, Relation rel);
static void logicalrep_write_tuple(StringInfo out, Relation rel,
HeapTuple tuple);
***************
*** 292,297 **** logicalrep_read_delete(StringInfo in, LogicalRepTupleData *oldtup)
--- 295,342 ----
return relid;
}
+ /*
+ * Write TRUNCATE to the output stream.
+ */
+ void
+ logicalrep_write_truncate(StringInfo out, Relation rel,
+ bool cascade, bool restart_seqs)
+ {
+ uint8 flags = 0;
+
+ pq_sendbyte(out, 'T'); /* action TRUNCATE */
+
+ /* use Oid as relation identifier */
+ pq_sendint32(out, RelationGetRelid(rel));
+
+ /* encode and send truncate flags */
+ if (cascade)
+ flags |= TRUNCATE_CASCADE;
+ if (restart_seqs)
+ flags |= TRUNCATE_RESTART_SEQS;
+ pq_sendint8(out, flags);
+ }
+
+ /*
+ * Read TRUNCATE from stream.
+ */
+ LogicalRepRelId
+ logicalrep_read_truncate(StringInfo in, bool *cascade, bool *restart_seqs)
+ {
+ LogicalRepRelId relid;
+ uint8 flags;
+
+ /* read the relation id */
+ relid = pq_getmsgint(in, 4);
+
+ /* read and decode truncate flags */
+ flags = pq_getmsgint(in, 1);
+ *cascade = (flags & TRUNCATE_CASCADE) > 0;
+ *restart_seqs = (flags & TRUNCATE_RESTART_SEQS) > 0;
+
+ return relid;
+ }
+
/*
* Write relation description to the output stream.
*/
diff --git a/src/backend/replication/logical/rindex c72a611a39..f1d65c1457 100644
*** a/src/backend/replication/logical/reorderbuffer.c
--- b/src/backend/replication/logical/reorderbuffer.c
***************
*** 403,408 **** ReorderBufferReturnChange(ReorderBuffer *rb, ReorderBufferChange *change)
--- 403,410 ----
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ break;
}
pfree(change);
***************
*** 1342,1347 **** ReorderBufferCommit(ReorderBuffer *rb, TransactionId xid,
--- 1344,1356 ----
switch (change->action)
{
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ reloid = change->data.truncate_msg.relid;
+ relation = RelationIdGetRelation(reloid);
+ rb->apply_change(rb, txn, relation, change);
+ RelationClose(relation);
+ break;
+
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
/*
***************
*** 2239,2244 **** ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
--- 2248,2254 ----
}
break;
}
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
***************
*** 2524,2529 **** ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
--- 2534,2540 ----
break;
}
/* the base struct contains all the data, easy peasy */
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
diff --git a/src/backend/replication/logical/worker.c index 83c69092ae..07fa735b27 100644
*** a/src/backend/replication/logical/worker.c
--- b/src/backend/replication/logical/worker.c
***************
*** 869,874 **** apply_handle_delete(StringInfo s)
--- 869,920 ----
CommandCounterIncrement();
}
+ /*
+ * Handle TRUNCATE message.
+ *
+ * TODO: FDW support
+ */
+ static void
+ apply_handle_truncate(StringInfo s)
+ {
+ LogicalRepRelMapEntry *rel;
+ LogicalRepRelId relid;
+ bool restart_seqs = false;
+ List *rels = NIL;
+ List *relids = NIL;
+
+ ensure_transaction();
+
+ relid = logicalrep_read_truncate(s, &cascade, &restart_seqs);
+ rel = logicalrep_rel_open(relid, RowExclusiveLock);
+ if (!should_apply_changes_for_rel(rel))
+ {
+ /*
+ * The relation can't become interesting in the middle of the
+ * transaction so it's safe to unlock it.
+ */
+ logicalrep_rel_close(rel, RowExclusiveLock);
+ return;
+ }
+
+ /* Check if we can do the truncate. */
+ check_relation_updatable(rel);
+
+ rels = lappend(rels, rel->localrel);
+ relids = lappend_oid(relids, rel->localreloid);
+
+ /*
+ * Even if we used CASCADE on the upstream master we explicitly
+ * default to replaying changes without further cascading.
+ * This might be later changeable with a user specified option.
+ */
+ ExecuteTruncateGuts(rels, relids, NULL, DROP_RESTRICT, restart_seqs);
+
+ logicalrep_rel_close(rel, NoLock);
+
+ CommandCounterIncrement();
+ }
+
/*
* Logical replication protocol message dispatcher.
***************
*** 900,905 **** apply_dispatch(StringInfo s)
--- 946,955 ----
case 'D':
apply_handle_delete(s);
break;
+ /* TRUNCATE */
+ case 'T':
+ apply_handle_truncate(s);
+ break;
/* RELATION */
case 'R':
apply_handle_relation(s);
diff --git a/src/backend/replication/pgoutput/pindex 40a1ef3c1d..7fbd0a3dca 100644
*** a/src/backend/replication/pgoutput/pgoutput.c
--- b/src/backend/replication/pgoutput/pgoutput.c
***************
*** 275,280 **** pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
--- 275,281 ----
if (!relentry->pubactions.pubupdate)
return;
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
case REORDER_BUFFER_CHANGE_DELETE:
if (!relentry->pubactions.pubdelete)
return;
***************
*** 352,357 **** pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
--- 353,367 ----
else
elog(DEBUG1, "didn't send DELETE change because of missing oldtuple");
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ {
+ OutputPluginPrepareWrite(ctx, true);
+ logicalrep_write_truncate(ctx->out, relation,
+ change->data.truncate_msg.cascade,
+ change->data.truncate_msg.restart_seqs);
+ OutputPluginWrite(ctx, true);
+ }
+ break;
default:
Assert(false);
}
diff --git a/src/include/access/heapam_xlog.h b/srindex 700e25c36a..c56b200f7d 100644
*** a/src/include/access/heapam_xlog.h
--- b/src/include/access/heapam_xlog.h
***************
*** 32,38 ****
#define XLOG_HEAP_INSERT 0x00
#define XLOG_HEAP_DELETE 0x10
#define XLOG_HEAP_UPDATE 0x20
! /* 0x030 is free, was XLOG_HEAP_MOVE */
#define XLOG_HEAP_HOT_UPDATE 0x40
#define XLOG_HEAP_CONFIRM 0x50
#define XLOG_HEAP_LOCK 0x60
--- 32,38 ----
#define XLOG_HEAP_INSERT 0x00
#define XLOG_HEAP_DELETE 0x10
#define XLOG_HEAP_UPDATE 0x20
! #define XLOG_HEAP_TRUNCATE 0x30
#define XLOG_HEAP_HOT_UPDATE 0x40
#define XLOG_HEAP_CONFIRM 0x50
#define XLOG_HEAP_LOCK 0x60
***************
*** 109,114 **** typedef struct xl_heap_delete
--- 109,136 ----
#define SizeOfHeapDelete (offsetof(xl_heap_delete, flags) + sizeof(uint8))
+ /*
+ * xl_heap_delete flag values, 8 bits are available.
+ */
+ #define XLH_TRUNCATE_CASCADE (1<<0)
+ #define XLH_TRUNCATE_RESTART_SEQS (1<<1)
+
+ /*
+ * For truncate we list all truncated relids in an array, followed by all
+ * sequence relids that need to be restarted, if any.
+ * All rels are always within the same database, so we just list dbid once.
+ */
+ typedef struct xl_heap_truncate
+ {
+ Oid dbId;
+ uint32 nrelids;
+ uint32 nseqrelids;
+ uint8 flags;
+ Oid relids[FLEXIBLE_ARRAY_MEMBER];
+ } xl_heap_truncate;
+
+ #define SizeOfHeapTruncate (offsetof(xl_heap_truncate, relids))
+
/*
* We don't store the whole fixed part (HeapTupleHeaderData) of an inserted
* or updated tuple in WAL; we can save a few bytes by reconstructing the
diff --git a/src/include/executor/execuindex 6545a80222..8cb12a20e9 100644
*** a/src/include/executor/executor.h
--- b/src/include/executor/executor.h
***************
*** 558,561 **** extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
--- 558,567 ----
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
+ /*
+ * prototype for tablecmds.c accessible for logical apply
+ */
+ void ExecuteTruncateGuts(List *explicit_rels, List *relids, List *relids_logged,
+ DropBehavior behavior, bool restart_seqs);
+
#endif /* EXECUTOR_H */
diff --git a/src/include/replication/lindex 0eb21057c5..b1abb9e36f 100644
*** a/src/include/replication/logicalproto.h
--- b/src/include/replication/logicalproto.h
***************
*** 98,103 **** extern void logicalrep_write_delete(StringInfo out, Relation rel,
--- 98,107 ----
HeapTuple oldtuple);
extern LogicalRepRelId logicalrep_read_delete(StringInfo in,
LogicalRepTupleData *oldtup);
+ extern void logicalrep_write_truncate(StringInfo out, Relation rel,
+ bool cascade, bool restart_seqs);
+ extern LogicalRepRelId logicalrep_read_truncate(StringInfo in,
+ bool *cascade, bool *restart_seqs);
extern void logicalrep_write_rel(StringInfo out, Relation rel);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
extern void logicalrep_write_typ(StringInfo out, Oid typoid);
diff --git a/src/include/replication/reorderbindex 0970abca52..e8439fac49 100644
*** a/src/include/replication/reorderbuffer.h
--- b/src/include/replication/reorderbuffer.h
***************
*** 59,65 **** enum ReorderBufferChangeType
REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID,
REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID,
REORDER_BUFFER_CHANGE_INTERNAL_SPEC_INSERT,
! REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM
};
/*
--- 59,66 ----
REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID,
REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID,
REORDER_BUFFER_CHANGE_INTERNAL_SPEC_INSERT,
! REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM,
! REORDER_BUFFER_CHANGE_TRUNCATE
};
/*
***************
*** 128,134 **** typedef struct ReorderBufferChange
CommandId cmax;
CommandId combocid;
} tuplecid;
! } data;
/*
* While in use this is how a change is linked into a transactions,
--- 129,146 ----
CommandId cmax;
CommandId combocid;
} tuplecid;
!
! /*
! * Truncate data for REORDER_BUFFER_CHANGE_TRUNCATE representing
! * one relation to be truncated.
! */
! struct
! {
! Oid relid;
! bool cascade;
! bool restart_seqs;
! } truncate_msg;
! } data;
/*
* While in use this is how a change is linked into a transactions,
diff --git a/src/test/subscription/t/001_rep_cindex e0104cd8d0..96a6071b3a 100644
*** a/src/test/subscription/t/001_rep_changes.pl
--- b/src/test/subscription/t/001_rep_changes.pl
***************
*** 3,9 **** use strict;
use warnings;
use PostgresNode;
use TestLib;
! use Test::More tests => 16;
# Initialize publisher node
my $node_publisher = get_new_node('publisher');
--- 3,9 ----
use warnings;
use PostgresNode;
use TestLib;
! use Test::More tests => 24;
# Initialize publisher node
my $node_publisher = get_new_node('publisher');
***************
*** 176,181 **** $result = $node_subscriber->safe_psql('postgres',
--- 176,191 ----
is($result, qq(1152|1|1100),
'check replicated inserts after subscription publication change');
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_rep");
+ is($result, qq(20|-20|-1),
+ 'check changes skipped after subscription publication change');
+
+ # truncate should not be replicated
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep");
+
+ $node_publisher->wait_for_catchup($appname);
+
$result = $node_subscriber->safe_psql('postgres',
"SELECT count(*), min(a), max(a) FROM tab_rep");
is($result, qq(20|-20|-1),
***************
*** 185,191 **** is($result, qq(20|-20|-1),
$node_publisher->safe_psql('postgres',
"ALTER PUBLICATION tap_pub_ins_only SET (publish = 'insert, delete')");
$node_publisher->safe_psql('postgres',
! "ALTER PUBLICATION tap_pub_ins_only ADD TABLE tab_full");
$node_publisher->safe_psql('postgres', "DELETE FROM tab_ins WHERE a > 0");
$node_subscriber->safe_psql('postgres',
"ALTER SUBSCRIPTION tap_sub REFRESH PUBLICATION WITH (copy_data = false)"
--- 195,201 ----
$node_publisher->safe_psql('postgres',
"ALTER PUBLICATION tap_pub_ins_only SET (publish = 'insert, delete')");
$node_publisher->safe_psql('postgres',
! "ALTER PUBLICATION tap_pub_ins_only ADD TABLE tab_full, tab_rep");
$node_publisher->safe_psql('postgres', "DELETE FROM tab_ins WHERE a > 0");
$node_subscriber->safe_psql('postgres',
"ALTER SUBSCRIPTION tap_sub REFRESH PUBLICATION WITH (copy_data = false)"
***************
*** 204,209 **** $result = $node_subscriber->safe_psql('postgres',
--- 214,282 ----
"SELECT count(*), min(a), max(a) FROM tab_full");
is($result, qq(21|0|100), 'check replicated insert after alter publication');
+ # add a sequence and a foreign key to check cascade and restart
+ # identity truncate options
+ $node_subscriber->safe_psql('postgres',
+ "CREATE TABLE tab_notrep_fk (a int REFERENCES tab_rep(a))"
+ );
+ $node_subscriber->safe_psql('postgres', "INSERT INTO tab_notrep_fk VALUES (-1)");
+ $node_subscriber->safe_psql('postgres',
+ "CREATE SEQUENCE seq_notrep OWNED BY tab_rep.a"
+ );
+ $node_subscriber->safe_psql('postgres',
+ "ALTER SEQUENCE seq_notrep START 101"
+ );
+
+ # truncate should now be replicated
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep");
+
+ $node_publisher->wait_for_catchup($appname);
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_rep");
+ is($result, qq(0||),
+ 'check replicated truncate after alter publication');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_notrep_fk");
+ is($result, qq(1|-1|-1),
+ 'check replicated truncate does not cascade');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq_notrep')");
+ is($result, qq(1),
+ 'check replicated truncate does not restart identities');
+
+ # should restart identity
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep RESTART IDENTITY");
+
+ $node_publisher->wait_for_catchup($appname);
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_notrep_fk");
+ is($result, qq(1|-1|-1),
+ 'check replicated truncate does not cascade');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq_notrep')");
+ is($result, qq(101),
+ 'check replicated truncate restart identities');
+
+ # should not cascade on replica
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep CASCADE");
+
+ $node_publisher->wait_for_catchup($appname);
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_notrep_fk");
+ is($result, qq(1|-1|-1),
+ 'check replicated truncate does not cascade on replica');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq_notrep')");
+ is($result, qq(102),
+ 'check replicated truncate does not restart identities');
+
# check restart on rename
$oldpid = $node_publisher->safe_psql('postgres',
"SELECT pid FROM pg_stat_replication WHERE application_name = '$appname';"
Re: [PATCH] Logical decoding of TRUNCATE
Hi,
On 23/01/18 18:47, Marco Nenciarini wrote:
Il 23/01/18 18:25, Petr Jelinek ha scritto:
On 23/01/18 18:19, Marco Nenciarini wrote:
Il 23/01/18 18:13, Petr Jelinek ha scritto:
Hi,
On 23/01/18 15:38, Marco Nenciarini wrote:
Il 22/01/18 23:18, Petr Jelinek ha scritto:
On 22/01/18 19:45, Petr Jelinek wrote:
Actually on second look, I don't like the new boolean parameter much.
I'd rather we didn't touch the input list and always close only
relations opened inside the ExecuteTruncateGuts().It may mean more list(s) but the current interface is still not clean.
Now ExecuteTruncateGuts unconditionally closes the relations that it
opens. The caller has now always the responsibility to close the
relations passed with the explicit_rels list.This looks good.
Version 15 attached.
I see you still do CASCADE on the subscriber though.
No it doesn't. The following code in worker.c prevents that.
+ /* + * Even if we used CASCADE on the upstream master we explicitly + * default to replaying changes without further cascading. + * This might be later changeable with a user specified option. + */ + cascade = false;Ah, that's pretty ugly, why don't we just use DROP_RESTRICT always
instead of this (keeping the comment). Unless you plan to make it option
as part of this patch, the current coding is confusing.Ok, Removed.
Great, thanks, I think this is ready now so marking as such.
--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Re: [PATCH] Logical decoding of TRUNCATE
I wonder whether this should be dealing with sequences at all. We are
not currently publishing any information about sequences, so it seems
weird to do it only here. Also, I'd imagine that if we ever get to
publishing sequence events, then the sequence restarts would be
published as independent events.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [PATCH] Logical decoding of TRUNCATE
On 24/01/18 13:50, Peter Eisentraut wrote:
I wonder whether this should be dealing with sequences at all. We are
not currently publishing any information about sequences, so it seems
weird to do it only here. Also, I'd imagine that if we ever get to
publishing sequence events, then the sequence restarts would be
published as independent events.
That depends on if we consider this to be part of sequence handling or
truncate statement replication handling. It's true that if we had
sequence replication, the restart would get propagated that way anyway.
OTOH, if we'll want to add option in the future to propagate cascade (to
any additional tables on downstream), it may need to reset sequences
which are not even present upstream and as such would not be handled by
sequence replication.
--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Re: [PATCH] Logical decoding of TRUNCATE
On Wed, Jan 24, 2018 at 6:47 AM, Marco Nenciarini
<marco.nenciarini@2ndquadrant.it> wrote:
Version 16 attached.
Hi Marco,
FYI this version doesn't compile:
worker.c: In function ‘apply_handle_truncate’:
worker.c:888:39: error: ‘cascade’ undeclared (first use in this function)
relid = logicalrep_read_truncate(s, &cascade, &restart_seqs);
^
--
Thomas Munro
http://www.enterprisedb.com
Re: [PATCH] Logical decoding of TRUNCATE
On 25/01/18 08:26, Thomas Munro wrote:
On Wed, Jan 24, 2018 at 6:47 AM, Marco Nenciarini
<marco.nenciarini@2ndquadrant.it> wrote:Version 16 attached.
Hi Marco,
FYI this version doesn't compile:
worker.c: In function ‘apply_handle_truncate’:
worker.c:888:39: error: ‘cascade’ undeclared (first use in this function)
relid = logicalrep_read_truncate(s, &cascade, &restart_seqs);
^
Indeed.
We also found one more issue - the truncate done by logical replication
worker is not logically logged which would break cascading.
I expect Marco to send new version shortly with both of these fixed.
--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Re: [PATCH] Logical decoding of TRUNCATE
Il 25/01/18 13:18, Petr Jelinek ha scritto:
On 25/01/18 08:26, Thomas Munro wrote:
On Wed, Jan 24, 2018 at 6:47 AM, Marco Nenciarini
<marco.nenciarini@2ndquadrant.it> wrote:Version 16 attached.
Hi Marco,
FYI this version doesn't compile:
worker.c: In function ‘apply_handle_truncate’:
worker.c:888:39: error: ‘cascade’ undeclared (first use in this function)
relid = logicalrep_read_truncate(s, &cascade, &restart_seqs);
^
Fixed.
Indeed.
We also found one more issue - the truncate done by logical replication
worker is not logically logged which would break cascading.
Fixed.
I expect Marco to send new version shortly with both of these fixed.
New patch v17 attached.
Regards,
Marco
--
Marco Nenciarini - 2ndQuadrant Italy
PostgreSQL Training, Services and Support
marco.nenciarini@2ndQuadrant.it | www.2ndQuadrant.it
Attachments:
0001-truncate_ignore_fks_role_replica.v4.patchtext/plain; charset=UTF-8; name=0001-truncate_ignore_fks_role_replica.v4.patch; x-mac-creator=0; x-mac-type=0Download
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 2e768dd5e4..bdce4164d6 100644
*** a/src/backend/commands/tablecmds.c
--- b/src/backend/commands/tablecmds.c
***************
*** 1404,1419 **** ExecuteTruncate(TruncateStmt *stmt)
}
/*
! * Check foreign key references. In CASCADE mode, this should be
! * unnecessary since we just pulled in all the references; but as a
! * cross-check, do it anyway if in an Assert-enabled build.
*/
#ifdef USE_ASSERT_CHECKING
- heap_truncate_check_FKs(rels, false);
- #else
- if (stmt->behavior == DROP_RESTRICT)
heap_truncate_check_FKs(rels, false);
#endif
/*
* If we are asked to restart sequences, find all the sequences, lock them
--- 1404,1427 ----
}
/*
! * Suppress foreign key references check if session replication role is
! * set to REPLICA.
*/
+ if (SessionReplicationRole != SESSION_REPLICATION_ROLE_REPLICA)
+ {
+
+ /*
+ * Check foreign key references. In CASCADE mode, this should be
+ * unnecessary since we just pulled in all the references; but as a
+ * cross-check, do it anyway if in an Assert-enabled build.
+ */
#ifdef USE_ASSERT_CHECKING
heap_truncate_check_FKs(rels, false);
+ #else
+ if (stmt->behavior == DROP_RESTRICT)
+ heap_truncate_check_FKs(rels, false);
#endif
+ }
/*
* If we are asked to restart sequences, find all the sequences, lock them
diff --git a/src/test/regress/expected/index d967e8dd21..86748430c5 100644
*** a/src/test/regress/expected/truncate.out
--- b/src/test/regress/expected/truncate.out
***************
*** 68,73 **** HINT: Truncate table "trunc_b" at the same time, or use TRUNCATE ... CASCADE.
--- 68,77 ----
TRUNCATE TABLE truncate_a CASCADE; -- ok
NOTICE: truncate cascades to table "trunc_b"
NOTICE: truncate cascades to table "trunc_e"
+ -- Ignore foreign-key checks with session_replication_role = replica
+ SET session_replication_role = replica;
+ TRUNCATE TABLE truncate_a; -- ok
+ RESET session_replication_role;
-- circular references
ALTER TABLE truncate_a ADD FOREIGN KEY (col1) REFERENCES trunc_c;
-- Add some data to verify that truncating actually works ...
diff --git a/src/test/regress/sql/truncate.sqindex fbd1d1a8a5..0d0a3705d2 100644
*** a/src/test/regress/sql/truncate.sql
--- b/src/test/regress/sql/truncate.sql
***************
*** 33,38 **** TRUNCATE TABLE trunc_c,trunc_d,trunc_e,truncate_a,trunc_b; -- ok
--- 33,43 ----
TRUNCATE TABLE truncate_a RESTRICT; -- fail
TRUNCATE TABLE truncate_a CASCADE; -- ok
+ -- Ignore foreign-key checks with session_replication_role = replica
+ SET session_replication_role = replica;
+ TRUNCATE TABLE truncate_a; -- ok
+ RESET session_replication_role;
+
-- circular references
ALTER TABLE truncate_a ADD FOREIGN KEY (col1) REFERENCES trunc_c;
0002-logical_decode_truncate.v17.patchtext/plain; charset=UTF-8; name=0002-logical_decode_truncate.v17.patch; x-mac-creator=0; x-mac-type=0Download
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 1e22c1eefc..d1feea4909 100644
*** a/contrib/test_decoding/expected/ddl.out
--- b/contrib/test_decoding/expected/ddl.out
***************
*** 543,548 **** UPDATE table_with_unique_not_null SET data = 3 WHERE data = 2;
--- 543,550 ----
UPDATE table_with_unique_not_null SET id = -id;
UPDATE table_with_unique_not_null SET id = -id;
DELETE FROM table_with_unique_not_null WHERE data = 3;
+ TRUNCATE table_with_unique_not_null;
+ TRUNCATE table_with_unique_not_null, table_with_unique_not_null RESTART IDENTITY CASCADE;
-- check toast support
BEGIN;
CREATE SEQUENCE toasttable_rand_seq START 79 INCREMENT 1499; -- portable "random"
***************
*** 660,665 **** SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
--- 662,673 ----
table public.table_with_unique_not_null: DELETE: id[integer]:4
COMMIT
BEGIN
+ table public.table_with_unique_not_null: TRUNCATE: (no-flags)
+ COMMIT
+ BEGIN
+ table public.table_with_unique_not_null: TRUNCATE: restart_seqs cascade
+ COMMIT
+ BEGIN
table public.toasttable: INSERT: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
BEGIN
***************
*** 668,674 **** SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
BEGIN
table public.toasttable: UPDATE: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
! (103 rows)
INSERT INTO toasttable(toasted_col1) SELECT string_agg(g.i::text, '') FROM generate_series(1, 2000) g(i);
-- update of second column, first column unchanged
--- 676,682 ----
BEGIN
table public.toasttable: UPDATE: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
! (109 rows)
INSERT INTO toasttable(toasted_col1) SELECT string_agg(g.i::text, '') FROM generate_series(1, 2000) g(i);
-- update of second column, first column unchanged
diff --git a/contrib/test_decoding/sql/ddl.sqindex 057dae056b..0fb4102db8 100644
*** a/contrib/test_decoding/sql/ddl.sql
--- b/contrib/test_decoding/sql/ddl.sql
***************
*** 333,338 **** UPDATE table_with_unique_not_null SET id = -id;
--- 333,341 ----
UPDATE table_with_unique_not_null SET id = -id;
DELETE FROM table_with_unique_not_null WHERE data = 3;
+ TRUNCATE table_with_unique_not_null;
+ TRUNCATE table_with_unique_not_null, table_with_unique_not_null RESTART IDENTITY CASCADE;
+
-- check toast support
BEGIN;
CREATE SEQUENCE toasttable_rand_seq START 79 INCREMENT 1499; -- portable "random"
diff --git a/contrib/test_decoding/test_index 0f18afa852..55c4593ed5 100644
*** a/contrib/test_decoding/test_decoding.c
--- b/contrib/test_decoding/test_decoding.c
***************
*** 466,471 **** pg_decode_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
--- 466,485 ----
&change->data.tp.oldtuple->tuple,
true);
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ appendStringInfoString(ctx->out, " TRUNCATE:");
+
+ if (change->data.truncate_msg.restart_seqs
+ || change->data.truncate_msg.cascade)
+ {
+ if (change->data.truncate_msg.restart_seqs)
+ appendStringInfo(ctx->out, " restart_seqs");
+ if (change->data.truncate_msg.cascade)
+ appendStringInfo(ctx->out, " cascade");
+ }
+ else
+ appendStringInfoString(ctx->out, " (no-flags)");
+ break;
default:
Assert(false);
}
diff --git a/doc/src/sgml/logical-replicatioindex 75551d8ee1..6b64c8ef76 100644
*** a/doc/src/sgml/logical-replication.sgml
--- b/doc/src/sgml/logical-replication.sgml
***************
*** 111,116 ****
--- 111,118 ----
any combination of <command>INSERT</command>, <command>UPDATE</command>, and
<command>DELETE</command>, similar to how triggers are fired by
particular event types. By default, all operation types are replicated.
+ <command>TRUNCATE</command> is treated as a form of <command>DELETE</command>
+ for the purpose of deciding whether to publish, or not.
</para>
<para>
***************
*** 364,378 ****
</para>
</listitem>
- <listitem>
- <para>
- <command>TRUNCATE</command> commands are not replicated. This can, of
- course, be worked around by using <command>DELETE</command> instead. To
- avoid accidental <command>TRUNCATE</command> invocations, you can revoke
- the <literal>TRUNCATE</literal> privilege from tables.
- </para>
- </listitem>
-
<listitem>
<para>
Large objects (see <xref linkend="largeobjects"/>) are not replicated.
--- 366,371 ----
diff --git a/doc/src/sgml/protocol.sgml b/doindex 4c5ed1e6d6..d2a0d7e52e 100644
*** a/doc/src/sgml/protocol.sgml
--- b/doc/src/sgml/protocol.sgml
***************
*** 6838,6843 **** TupleData
--- 6838,6889 ----
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ Truncate
+ </term>
+ <listitem>
+ <para>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ Byte1('T')
+ </term>
+ <listitem>
+ <para>
+ Identifies the message as a truncate message.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ Int32
+ </term>
+ <listitem>
+ <para>
+ ID of the relation corresponding to the ID in the relation
+ message.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ Int8
+ </term>
+ <listitem>
+ <para>
+ Option flags for truncate. Currently can be 0 for no flags,
+ 1 for CASCADE, 2 for RESTART IDENTITY and 3 if both are set.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect1>
diff --git a/doc/src/sgml/ref/creindex bfe12d5f41..c5032938a7 100644
*** a/doc/src/sgml/ref/create_publication.sgml
--- b/doc/src/sgml/ref/create_publication.sgml
***************
*** 111,116 **** CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
--- 111,121 ----
and so the default value for this option is
<literal>'insert, update, delete'</literal>.
</para>
+ <para>
+ <command>TRUNCATE</command> is treated as a form of
+ <command>DELETE</command> for the purpose of deciding whether
+ to publish, or not.
+ </para>
</listitem>
</varlistentry>
</variablelist>
***************
*** 168,175 **** CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
</para>
<para>
! <command>TRUNCATE</command> and <acronym>DDL</acronym> operations
! are not published.
</para>
</refsect1>
--- 173,179 ----
</para>
<para>
! <acronym>DDL</acronym> operations are not published.
</para>
</refsect1>
diff --git a/src/backend/access/heap/heapam.c bindex be263850cd..f6fa99d48e 100644
*** a/src/backend/access/heap/heapam.c
--- b/src/backend/access/heap/heapam.c
***************
*** 9137,9142 **** heap_redo(XLogReaderState *record)
--- 9137,9149 ----
case XLOG_HEAP_UPDATE:
heap_xlog_update(record, false);
break;
+ case XLOG_HEAP_TRUNCATE:
+ /*
+ * TRUNCATE is a no-op because the actions are already logged
+ * as SMGR WAL records. TRUNCATE WAL record only exists to allow
+ * it to be logically decoded precisely.
+ */
+ break;
case XLOG_HEAP_HOT_UPDATE:
heap_xlog_update(record, true);
break;
diff --git a/src/backend/access/rmgrdesindex b00c071cb6..15550cdff0 100644
*** a/src/backend/access/rmgrdesc/heapdesc.c
--- b/src/backend/access/rmgrdesc/heapdesc.c
***************
*** 75,80 **** heap_desc(StringInfo buf, XLogReaderState *record)
--- 75,93 ----
xlrec->new_offnum,
xlrec->new_xmax);
}
+ else if (info == XLOG_HEAP_TRUNCATE)
+ {
+ xl_heap_truncate *xlrec = (xl_heap_truncate *) rec;
+
+ if (xlrec->flags & XLH_TRUNCATE_CASCADE)
+ appendStringInfo(buf, "cascade ");
+ if (xlrec->flags & XLH_TRUNCATE_RESTART_SEQS)
+ appendStringInfo(buf, "restart_seqs ");
+ appendStringInfo(buf, "nrelids %u nseqrelids %u",
+ xlrec->nrelids,
+ xlrec->nseqrelids);
+ /* Skip the list of relids and seqrelids */
+ }
else if (info == XLOG_HEAP_CONFIRM)
{
xl_heap_confirm *xlrec = (xl_heap_confirm *) rec;
***************
*** 186,191 **** heap_identify(uint8 info)
--- 199,207 ----
case XLOG_HEAP_HOT_UPDATE | XLOG_HEAP_INIT_PAGE:
id = "HOT_UPDATE+INIT";
break;
+ case XLOG_HEAP_TRUNCATE:
+ id = "TRUNCATE";
+ break;
case XLOG_HEAP_CONFIRM:
id = "HEAP_CONFIRM";
break;
diff --git a/src/backend/commands/tablecmds.cindex bdce4164d6..61f700d8b2 100644
*** a/src/backend/commands/tablecmds.c
--- b/src/backend/commands/tablecmds.c
***************
*** 16,21 ****
--- 16,22 ----
#include "access/genam.h"
#include "access/heapam.h"
+ #include "access/heapam_xlog.h"
#include "access/multixact.h"
#include "access/reloptions.h"
#include "access/relscan.h"
***************
*** 1312,1322 **** ExecuteTruncate(TruncateStmt *stmt)
{
List *rels = NIL;
List *relids = NIL;
! List *seq_relids = NIL;
! EState *estate;
! ResultRelInfo *resultRelInfos;
! ResultRelInfo *resultRelInfo;
! SubTransactionId mySubid;
ListCell *cell;
/*
--- 1313,1319 ----
{
List *rels = NIL;
List *relids = NIL;
! List *relids_logged = NIL;
ListCell *cell;
/*
***************
*** 1340,1345 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1337,1345 ----
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, myrelid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, myrelid);
if (recurse)
{
***************
*** 1360,1365 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1360,1368 ----
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, childrelid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, childrelid);
}
}
else if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
***************
*** 1369,1375 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1372,1410 ----
errhint("Do not specify the ONLY keyword, or use truncate only on the partitions directly.")));
}
+ ExecuteTruncateGuts(rels, relids, relids_logged,
+ stmt->behavior, stmt->restart_seqs);
+
+ /* And close the rels */
+ foreach(cell, rels)
+ {
+ Relation rel = (Relation) lfirst(cell);
+
+ heap_close(rel, NoLock);
+ }
+
+ }
+
+ void
+ ExecuteTruncateGuts(List *explicit_rels, List *relids, List *relids_logged,
+ DropBehavior behavior, bool restart_seqs)
+ {
+ List *rels = list_copy(explicit_rels);
+ List *seq_relids = NIL;
+ EState *estate;
+ ResultRelInfo *resultRelInfos;
+ ResultRelInfo *resultRelInfo;
+ SubTransactionId mySubid;
+ ListCell *cell;
+ List *seq_relids_logged = NIL;
+ uint32 nrelids = 0;
+ uint32 nseqrelids = 0;
+ uint32 maxrelids = 2;
+ Oid *logrelids = NULL;
+
/*
+ * Open, exclusive-lock, and check all the explicitly-specified relations
+ *
* In CASCADE mode, suck in all referencing relations as well. This
* requires multiple iterations to find indirectly-dependent relations. At
* each phase, we need to exclusive-lock new rels before looking for their
***************
*** 1377,1383 **** ExecuteTruncate(TruncateStmt *stmt)
* soon as we open it, to avoid a faux pas such as holding lock for a long
* time on a rel we have no permissions for.
*/
! if (stmt->behavior == DROP_CASCADE)
{
for (;;)
{
--- 1412,1418 ----
* soon as we open it, to avoid a faux pas such as holding lock for a long
* time on a rel we have no permissions for.
*/
! if (behavior == DROP_CASCADE)
{
for (;;)
{
***************
*** 1399,1404 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1434,1442 ----
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, relid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, relid);
}
}
}
***************
*** 1418,1424 **** ExecuteTruncate(TruncateStmt *stmt)
#ifdef USE_ASSERT_CHECKING
heap_truncate_check_FKs(rels, false);
#else
! if (stmt->behavior == DROP_RESTRICT)
heap_truncate_check_FKs(rels, false);
#endif
}
--- 1456,1462 ----
#ifdef USE_ASSERT_CHECKING
heap_truncate_check_FKs(rels, false);
#else
! if (behavior == DROP_RESTRICT)
heap_truncate_check_FKs(rels, false);
#endif
}
***************
*** 1429,1435 **** ExecuteTruncate(TruncateStmt *stmt)
* We want to do this early since it's pointless to do all the truncation
* work only to fail on sequence permissions.
*/
! if (stmt->restart_seqs)
{
foreach(cell, rels)
{
--- 1467,1473 ----
* We want to do this early since it's pointless to do all the truncation
* work only to fail on sequence permissions.
*/
! if (restart_seqs)
{
foreach(cell, rels)
{
***************
*** 1450,1455 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1488,1497 ----
RelationGetRelationName(seq_rel));
seq_relids = lappend_oid(seq_relids, seq_relid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(seq_rel))
+ seq_relids_logged = lappend_oid(seq_relids_logged,
+ seq_relid);
relation_close(seq_rel, NoLock);
}
***************
*** 1584,1589 **** ExecuteTruncate(TruncateStmt *stmt)
--- 1626,1685 ----
ResetSequence(seq_relid);
}
+ /*
+ * Write a WAL record to allow this set of actions to be logically decoded.
+ * We could optimize this away when !RelationIsLogicallyLogged(rel)
+ * but that doesn't save much space or time.
+ *
+ * Assemble an array of relids, then an array of seqrelids so we can write
+ * a single WAL record for the whole action.
+ */
+ logrelids = palloc(maxrelids * sizeof(Oid));
+ foreach (cell, relids_logged)
+ {
+ nrelids++;
+ if (nrelids > maxrelids)
+ {
+ maxrelids *= 2;
+ logrelids = repalloc(logrelids, maxrelids * sizeof(Oid));
+ }
+ logrelids[nrelids - 1] = lfirst_oid(cell);
+ }
+
+ foreach (cell, seq_relids_logged)
+ {
+ nseqrelids++;
+ if ((nrelids + nseqrelids) > maxrelids)
+ {
+ maxrelids *= 2;
+ logrelids = repalloc(logrelids, maxrelids * sizeof(Oid));
+ }
+ logrelids[nrelids + nseqrelids - 1] = lfirst_oid(cell);
+ }
+
+ if ((nrelids + nseqrelids) > 0)
+ {
+ xl_heap_truncate xlrec;
+
+ xlrec.dbId = MyDatabaseId;
+ xlrec.nrelids = nrelids;
+ xlrec.nseqrelids = nseqrelids;
+ xlrec.flags = 0;
+ if (behavior == DROP_CASCADE)
+ xlrec.flags |= XLH_TRUNCATE_CASCADE;
+ if (restart_seqs)
+ xlrec.flags |= XLH_TRUNCATE_RESTART_SEQS;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfHeapTruncate);
+ XLogRegisterData((char *) logrelids,
+ (nrelids + nseqrelids) * sizeof(Oid));
+
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ (void) XLogInsert(RM_HEAP_ID, XLOG_HEAP_TRUNCATE);
+ }
+
/*
* Process all AFTER STATEMENT TRUNCATE triggers.
*/
***************
*** 1601,1607 **** ExecuteTruncate(TruncateStmt *stmt)
/* We can clean up the EState now */
FreeExecutorState(estate);
! /* And close the rels (can't do this while EState still holds refs) */
foreach(cell, rels)
{
Relation rel = (Relation) lfirst(cell);
--- 1697,1706 ----
/* We can clean up the EState now */
FreeExecutorState(estate);
! /* And close the eventual rels opened by CASCADE
! * (can't do this while EState still holds refs)
! */
! rels = list_difference_ptr(rels, explicit_rels);
foreach(cell, rels)
{
Relation rel = (Relation) lfirst(cell);
diff --git a/src/backend/replication/loindex 6eb0d5527e..ef5b937807 100644
*** a/src/backend/replication/logical/decode.c
--- b/src/backend/replication/logical/decode.c
***************
*** 65,70 **** static void DecodeLogicalMsgOp(LogicalDecodingContext *ctx, XLogRecordBuffer *bu
--- 65,71 ----
static void DecodeInsert(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeUpdate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeDelete(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
+ static void DecodeTruncate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeMultiInsert(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeSpecConfirm(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
***************
*** 449,454 **** DecodeHeapOp(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
--- 450,460 ----
DecodeDelete(ctx, buf);
break;
+ case XLOG_HEAP_TRUNCATE:
+ if (SnapBuildProcessChange(builder, xid, buf->origptr))
+ DecodeTruncate(ctx, buf);
+ break;
+
case XLOG_HEAP_INPLACE:
/*
***************
*** 825,830 **** DecodeDelete(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
--- 831,876 ----
ReorderBufferQueueChange(ctx->reorder, XLogRecGetXid(r), buf->origptr, change);
}
+ /*
+ * Parse XLOG_HEAP_TRUNCATE from wal
+ */
+ static void
+ DecodeTruncate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+ {
+ XLogReaderState *r = buf->record;
+ xl_heap_truncate *xlrec;
+ ReorderBufferChange *change;
+ int i;
+
+ xlrec = (xl_heap_truncate *) XLogRecGetData(r);
+
+ /* only interested in our database */
+ if (xlrec->dbId != ctx->slot->data.database)
+ return;
+
+ /* output plugin doesn't look for this origin, no need to queue */
+ if (FilterByOrigin(ctx, XLogRecGetOrigin(r)))
+ return;
+
+ change = ReorderBufferGetChange(ctx->reorder);
+ change->action = REORDER_BUFFER_CHANGE_TRUNCATE;
+ change->origin_id = XLogRecGetOrigin(r);
+ if (xlrec->flags & XLH_TRUNCATE_CASCADE)
+ change->data.truncate_msg.cascade = true;
+ if (xlrec->flags & XLH_TRUNCATE_RESTART_SEQS)
+ change->data.truncate_msg.restart_seqs = true;
+
+ /*
+ * Queue up one change per relation, ignoring sequences for now
+ */
+ for (i = 0; i < xlrec->nrelids; i++)
+ {
+ change->data.truncate_msg.relid = xlrec->relids[i];
+ ReorderBufferQueueChange(ctx->reorder, XLogRecGetXid(r),
+ buf->origptr, change);
+ }
+ }
+
/*
* Decode XLOG_HEAP2_MULTI_INSERT_insert record into multiple tuplebufs.
*
diff --git a/src/backend/replication/logical/prindex 948343e4ae..2fa6f8393d 100644
*** a/src/backend/replication/logical/proto.c
--- b/src/backend/replication/logical/proto.c
***************
*** 26,31 ****
--- 26,34 ----
*/
#define LOGICALREP_IS_REPLICA_IDENTITY 1
+ #define TRUNCATE_CASCADE (1<<0)
+ #define TRUNCATE_RESTART_SEQS (1<<1)
+
static void logicalrep_write_attrs(StringInfo out, Relation rel);
static void logicalrep_write_tuple(StringInfo out, Relation rel,
HeapTuple tuple);
***************
*** 292,297 **** logicalrep_read_delete(StringInfo in, LogicalRepTupleData *oldtup)
--- 295,342 ----
return relid;
}
+ /*
+ * Write TRUNCATE to the output stream.
+ */
+ void
+ logicalrep_write_truncate(StringInfo out, Relation rel,
+ bool cascade, bool restart_seqs)
+ {
+ uint8 flags = 0;
+
+ pq_sendbyte(out, 'T'); /* action TRUNCATE */
+
+ /* use Oid as relation identifier */
+ pq_sendint32(out, RelationGetRelid(rel));
+
+ /* encode and send truncate flags */
+ if (cascade)
+ flags |= TRUNCATE_CASCADE;
+ if (restart_seqs)
+ flags |= TRUNCATE_RESTART_SEQS;
+ pq_sendint8(out, flags);
+ }
+
+ /*
+ * Read TRUNCATE from stream.
+ */
+ LogicalRepRelId
+ logicalrep_read_truncate(StringInfo in, bool *cascade, bool *restart_seqs)
+ {
+ LogicalRepRelId relid;
+ uint8 flags;
+
+ /* read the relation id */
+ relid = pq_getmsgint(in, 4);
+
+ /* read and decode truncate flags */
+ flags = pq_getmsgint(in, 1);
+ *cascade = (flags & TRUNCATE_CASCADE) > 0;
+ *restart_seqs = (flags & TRUNCATE_RESTART_SEQS) > 0;
+
+ return relid;
+ }
+
/*
* Write relation description to the output stream.
*/
diff --git a/src/backend/replication/logical/rindex c72a611a39..f1d65c1457 100644
*** a/src/backend/replication/logical/reorderbuffer.c
--- b/src/backend/replication/logical/reorderbuffer.c
***************
*** 403,408 **** ReorderBufferReturnChange(ReorderBuffer *rb, ReorderBufferChange *change)
--- 403,410 ----
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ break;
}
pfree(change);
***************
*** 1342,1347 **** ReorderBufferCommit(ReorderBuffer *rb, TransactionId xid,
--- 1344,1356 ----
switch (change->action)
{
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ reloid = change->data.truncate_msg.relid;
+ relation = RelationIdGetRelation(reloid);
+ rb->apply_change(rb, txn, relation, change);
+ RelationClose(relation);
+ break;
+
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
/*
***************
*** 2239,2244 **** ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
--- 2248,2254 ----
}
break;
}
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
***************
*** 2524,2529 **** ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
--- 2534,2540 ----
break;
}
/* the base struct contains all the data, easy peasy */
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
diff --git a/src/backend/replication/logical/worker.c index 83c69092ae..d6d08c4ed8 100644
*** a/src/backend/replication/logical/worker.c
--- b/src/backend/replication/logical/worker.c
***************
*** 30,35 ****
--- 30,36 ----
#include "access/xact.h"
#include "access/xlog_internal.h"
+ #include "catalog/catalog.h"
#include "catalog/namespace.h"
#include "catalog/pg_subscription.h"
#include "catalog/pg_subscription_rel.h"
***************
*** 83,88 ****
--- 84,90 ----
#include "utils/inval.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
+ #include "utils/rel.h"
#include "utils/timeout.h"
#include "utils/tqual.h"
#include "utils/syscache.h"
***************
*** 869,874 **** apply_handle_delete(StringInfo s)
--- 871,926 ----
CommandCounterIncrement();
}
+ /*
+ * Handle TRUNCATE message.
+ *
+ * TODO: FDW support
+ */
+ static void
+ apply_handle_truncate(StringInfo s)
+ {
+ LogicalRepRelMapEntry *rel;
+ LogicalRepRelId relid;
+ bool cascade = false;
+ bool restart_seqs = false;
+ List *rels = NIL;
+ List *relids = NIL;
+ List *relids_logged = NIL;
+
+ ensure_transaction();
+
+ relid = logicalrep_read_truncate(s, &cascade, &restart_seqs);
+ rel = logicalrep_rel_open(relid, RowExclusiveLock);
+ if (!should_apply_changes_for_rel(rel))
+ {
+ /*
+ * The relation can't become interesting in the middle of the
+ * transaction so it's safe to unlock it.
+ */
+ logicalrep_rel_close(rel, RowExclusiveLock);
+ return;
+ }
+
+ /* Check if we can do the truncate. */
+ check_relation_updatable(rel);
+
+ rels = lappend(rels, rel->localrel);
+ relids = lappend_oid(relids, rel->localreloid);
+ if (RelationIsLogicallyLogged(rel->localrel))
+ relids_logged = lappend_oid(relids, rel->localreloid);
+
+ /*
+ * Even if we used CASCADE on the upstream master we explicitly
+ * default to replaying changes without further cascading.
+ * This might be later changeable with a user specified option.
+ */
+ ExecuteTruncateGuts(rels, relids, relids_logged, DROP_RESTRICT, restart_seqs);
+
+ logicalrep_rel_close(rel, NoLock);
+
+ CommandCounterIncrement();
+ }
+
/*
* Logical replication protocol message dispatcher.
***************
*** 900,905 **** apply_dispatch(StringInfo s)
--- 952,961 ----
case 'D':
apply_handle_delete(s);
break;
+ /* TRUNCATE */
+ case 'T':
+ apply_handle_truncate(s);
+ break;
/* RELATION */
case 'R':
apply_handle_relation(s);
diff --git a/src/backend/replication/pgoutput/pindex 40a1ef3c1d..7fbd0a3dca 100644
*** a/src/backend/replication/pgoutput/pgoutput.c
--- b/src/backend/replication/pgoutput/pgoutput.c
***************
*** 275,280 **** pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
--- 275,281 ----
if (!relentry->pubactions.pubupdate)
return;
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
case REORDER_BUFFER_CHANGE_DELETE:
if (!relentry->pubactions.pubdelete)
return;
***************
*** 352,357 **** pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
--- 353,367 ----
else
elog(DEBUG1, "didn't send DELETE change because of missing oldtuple");
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ {
+ OutputPluginPrepareWrite(ctx, true);
+ logicalrep_write_truncate(ctx->out, relation,
+ change->data.truncate_msg.cascade,
+ change->data.truncate_msg.restart_seqs);
+ OutputPluginWrite(ctx, true);
+ }
+ break;
default:
Assert(false);
}
diff --git a/src/include/access/heapam_xlog.h b/srindex 700e25c36a..c56b200f7d 100644
*** a/src/include/access/heapam_xlog.h
--- b/src/include/access/heapam_xlog.h
***************
*** 32,38 ****
#define XLOG_HEAP_INSERT 0x00
#define XLOG_HEAP_DELETE 0x10
#define XLOG_HEAP_UPDATE 0x20
! /* 0x030 is free, was XLOG_HEAP_MOVE */
#define XLOG_HEAP_HOT_UPDATE 0x40
#define XLOG_HEAP_CONFIRM 0x50
#define XLOG_HEAP_LOCK 0x60
--- 32,38 ----
#define XLOG_HEAP_INSERT 0x00
#define XLOG_HEAP_DELETE 0x10
#define XLOG_HEAP_UPDATE 0x20
! #define XLOG_HEAP_TRUNCATE 0x30
#define XLOG_HEAP_HOT_UPDATE 0x40
#define XLOG_HEAP_CONFIRM 0x50
#define XLOG_HEAP_LOCK 0x60
***************
*** 109,114 **** typedef struct xl_heap_delete
--- 109,136 ----
#define SizeOfHeapDelete (offsetof(xl_heap_delete, flags) + sizeof(uint8))
+ /*
+ * xl_heap_delete flag values, 8 bits are available.
+ */
+ #define XLH_TRUNCATE_CASCADE (1<<0)
+ #define XLH_TRUNCATE_RESTART_SEQS (1<<1)
+
+ /*
+ * For truncate we list all truncated relids in an array, followed by all
+ * sequence relids that need to be restarted, if any.
+ * All rels are always within the same database, so we just list dbid once.
+ */
+ typedef struct xl_heap_truncate
+ {
+ Oid dbId;
+ uint32 nrelids;
+ uint32 nseqrelids;
+ uint8 flags;
+ Oid relids[FLEXIBLE_ARRAY_MEMBER];
+ } xl_heap_truncate;
+
+ #define SizeOfHeapTruncate (offsetof(xl_heap_truncate, relids))
+
/*
* We don't store the whole fixed part (HeapTupleHeaderData) of an inserted
* or updated tuple in WAL; we can save a few bytes by reconstructing the
diff --git a/src/include/executor/execuindex 6545a80222..8cb12a20e9 100644
*** a/src/include/executor/executor.h
--- b/src/include/executor/executor.h
***************
*** 558,561 **** extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
--- 558,567 ----
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
+ /*
+ * prototype for tablecmds.c accessible for logical apply
+ */
+ void ExecuteTruncateGuts(List *explicit_rels, List *relids, List *relids_logged,
+ DropBehavior behavior, bool restart_seqs);
+
#endif /* EXECUTOR_H */
diff --git a/src/include/replication/lindex 0eb21057c5..b1abb9e36f 100644
*** a/src/include/replication/logicalproto.h
--- b/src/include/replication/logicalproto.h
***************
*** 98,103 **** extern void logicalrep_write_delete(StringInfo out, Relation rel,
--- 98,107 ----
HeapTuple oldtuple);
extern LogicalRepRelId logicalrep_read_delete(StringInfo in,
LogicalRepTupleData *oldtup);
+ extern void logicalrep_write_truncate(StringInfo out, Relation rel,
+ bool cascade, bool restart_seqs);
+ extern LogicalRepRelId logicalrep_read_truncate(StringInfo in,
+ bool *cascade, bool *restart_seqs);
extern void logicalrep_write_rel(StringInfo out, Relation rel);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
extern void logicalrep_write_typ(StringInfo out, Oid typoid);
diff --git a/src/include/replication/reorderbindex 0970abca52..e8439fac49 100644
*** a/src/include/replication/reorderbuffer.h
--- b/src/include/replication/reorderbuffer.h
***************
*** 59,65 **** enum ReorderBufferChangeType
REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID,
REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID,
REORDER_BUFFER_CHANGE_INTERNAL_SPEC_INSERT,
! REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM
};
/*
--- 59,66 ----
REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID,
REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID,
REORDER_BUFFER_CHANGE_INTERNAL_SPEC_INSERT,
! REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM,
! REORDER_BUFFER_CHANGE_TRUNCATE
};
/*
***************
*** 128,134 **** typedef struct ReorderBufferChange
CommandId cmax;
CommandId combocid;
} tuplecid;
! } data;
/*
* While in use this is how a change is linked into a transactions,
--- 129,146 ----
CommandId cmax;
CommandId combocid;
} tuplecid;
!
! /*
! * Truncate data for REORDER_BUFFER_CHANGE_TRUNCATE representing
! * one relation to be truncated.
! */
! struct
! {
! Oid relid;
! bool cascade;
! bool restart_seqs;
! } truncate_msg;
! } data;
/*
* While in use this is how a change is linked into a transactions,
diff --git a/src/test/subscription/t/001_rep_cindex e0104cd8d0..96a6071b3a 100644
*** a/src/test/subscription/t/001_rep_changes.pl
--- b/src/test/subscription/t/001_rep_changes.pl
***************
*** 3,9 **** use strict;
use warnings;
use PostgresNode;
use TestLib;
! use Test::More tests => 16;
# Initialize publisher node
my $node_publisher = get_new_node('publisher');
--- 3,9 ----
use warnings;
use PostgresNode;
use TestLib;
! use Test::More tests => 24;
# Initialize publisher node
my $node_publisher = get_new_node('publisher');
***************
*** 176,181 **** $result = $node_subscriber->safe_psql('postgres',
--- 176,191 ----
is($result, qq(1152|1|1100),
'check replicated inserts after subscription publication change');
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_rep");
+ is($result, qq(20|-20|-1),
+ 'check changes skipped after subscription publication change');
+
+ # truncate should not be replicated
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep");
+
+ $node_publisher->wait_for_catchup($appname);
+
$result = $node_subscriber->safe_psql('postgres',
"SELECT count(*), min(a), max(a) FROM tab_rep");
is($result, qq(20|-20|-1),
***************
*** 185,191 **** is($result, qq(20|-20|-1),
$node_publisher->safe_psql('postgres',
"ALTER PUBLICATION tap_pub_ins_only SET (publish = 'insert, delete')");
$node_publisher->safe_psql('postgres',
! "ALTER PUBLICATION tap_pub_ins_only ADD TABLE tab_full");
$node_publisher->safe_psql('postgres', "DELETE FROM tab_ins WHERE a > 0");
$node_subscriber->safe_psql('postgres',
"ALTER SUBSCRIPTION tap_sub REFRESH PUBLICATION WITH (copy_data = false)"
--- 195,201 ----
$node_publisher->safe_psql('postgres',
"ALTER PUBLICATION tap_pub_ins_only SET (publish = 'insert, delete')");
$node_publisher->safe_psql('postgres',
! "ALTER PUBLICATION tap_pub_ins_only ADD TABLE tab_full, tab_rep");
$node_publisher->safe_psql('postgres', "DELETE FROM tab_ins WHERE a > 0");
$node_subscriber->safe_psql('postgres',
"ALTER SUBSCRIPTION tap_sub REFRESH PUBLICATION WITH (copy_data = false)"
***************
*** 204,209 **** $result = $node_subscriber->safe_psql('postgres',
--- 214,282 ----
"SELECT count(*), min(a), max(a) FROM tab_full");
is($result, qq(21|0|100), 'check replicated insert after alter publication');
+ # add a sequence and a foreign key to check cascade and restart
+ # identity truncate options
+ $node_subscriber->safe_psql('postgres',
+ "CREATE TABLE tab_notrep_fk (a int REFERENCES tab_rep(a))"
+ );
+ $node_subscriber->safe_psql('postgres', "INSERT INTO tab_notrep_fk VALUES (-1)");
+ $node_subscriber->safe_psql('postgres',
+ "CREATE SEQUENCE seq_notrep OWNED BY tab_rep.a"
+ );
+ $node_subscriber->safe_psql('postgres',
+ "ALTER SEQUENCE seq_notrep START 101"
+ );
+
+ # truncate should now be replicated
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep");
+
+ $node_publisher->wait_for_catchup($appname);
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_rep");
+ is($result, qq(0||),
+ 'check replicated truncate after alter publication');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_notrep_fk");
+ is($result, qq(1|-1|-1),
+ 'check replicated truncate does not cascade');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq_notrep')");
+ is($result, qq(1),
+ 'check replicated truncate does not restart identities');
+
+ # should restart identity
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep RESTART IDENTITY");
+
+ $node_publisher->wait_for_catchup($appname);
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_notrep_fk");
+ is($result, qq(1|-1|-1),
+ 'check replicated truncate does not cascade');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq_notrep')");
+ is($result, qq(101),
+ 'check replicated truncate restart identities');
+
+ # should not cascade on replica
+ $node_publisher->safe_psql('postgres', "TRUNCATE tab_rep CASCADE");
+
+ $node_publisher->wait_for_catchup($appname);
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab_notrep_fk");
+ is($result, qq(1|-1|-1),
+ 'check replicated truncate does not cascade on replica');
+
+ $result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq_notrep')");
+ is($result, qq(102),
+ 'check replicated truncate does not restart identities');
+
# check restart on rename
$oldpid = $node_publisher->safe_psql('postgres',
"SELECT pid FROM pg_stat_replication WHERE application_name = '$appname';"
Logical decoding of TRUNCATE vs DELETE
Seeing that patch development is coming along at a nice pace, it seems a
good time to discuss this question.
On 1/4/18 03:24, Simon Riggs wrote:
At present the patch treats TRUNCATE as if it were a DELETE
...
CREATE PUBLICATION insert_only FOR TABLE mydata;
will now publish TRUNCATEs, although they were ignored in PG10
so PG10 publications will act differentlyI had regarded it as a missing piece, but some may view that is a
behaviour change in PG11
My understanding of this design is that there's no possible way to make
a PG11 database behave like a PG10 database did. For example, if someone
has a data warehouse with a single table that's subscribed to publishers
on multiple source databases, they would certainly not want truncate SQL
replicated. If they just upgrade their database without reading all the
release notes (not that anyone would ever do that), they might get a
surprise data loss in the warehouse. Of course I wouldn't suggest using
truncate on their sources - but people may still do it.
Alternatively, we could also use WITH (publish = 'truncate') as a
separate decision.That is an easy change if we wish it.
Of course the user could simply not _allow_ truncate commands on the
source tables via permissions and/or triggers; that seems a little more
"correct" to me from a DB design perspective. But I still like Simon's
idea here of using WITH as a separate decision. I'm sure that there will
be users somewhere who build systems based on our PG10 design - this at
least doesn't completely leave them out to dry, and I don't see any big
downsides to having the separate decision for truncate.
In addition, this seems a little more consistent. In other places that
comes to mind (e.g. triggers and privileges), truncate is treated
distinctly from delete. Makes sense to me to continue that convention.
-Jeremy
--
Jeremy Schneider
Database Engineer
Amazon Web Services
+1 312-725-9249 (m)
schnjere@amazon.com
Re: [PATCH] Logical decoding of TRUNCATE
On Wed, Jan 17, 2018 at 12:07 PM, Petr Jelinek
<petr.jelinek@2ndquadrant.com> wrote:
The patch will cascade truncation on downstream if cascade was specified
on the upstream, that can potentially be dangerous and we either should
not do it and only truncate the tables which were truncated upstream
(but without restricting because of FKs), leaving the data inconsistent
on downstream (like we do already with DELETE or UPDATE). Or maybe make
it into either subscription or publication option so that user can chose
the behaviour here as I am sure some people will want it to cascade (but
the default should still IMHO be to not cascade as that's safer).
Maybe I'm not understanding what is being proposed here, but it sounds
like you're saying that if somebody removes a bunch of data on the
logical master, replication will remove only some of it from the
servers to which the change is replicated. That seems crazy. Then
replication can't be counted on to produce a replica.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [PATCH] Logical decoding of TRUNCATE
On 26/01/18 02:34, Robert Haas wrote:
On Wed, Jan 17, 2018 at 12:07 PM, Petr Jelinek
<petr.jelinek@2ndquadrant.com> wrote:The patch will cascade truncation on downstream if cascade was specified
on the upstream, that can potentially be dangerous and we either should
not do it and only truncate the tables which were truncated upstream
(but without restricting because of FKs), leaving the data inconsistent
on downstream (like we do already with DELETE or UPDATE). Or maybe make
it into either subscription or publication option so that user can chose
the behaviour here as I am sure some people will want it to cascade (but
the default should still IMHO be to not cascade as that's safer).Maybe I'm not understanding what is being proposed here, but it sounds
like you're saying that if somebody removes a bunch of data on the
logical master, replication will remove only some of it from the
servers to which the change is replicated. That seems crazy. Then
replication can't be counted on to produce a replica.
No, I was talking about extra tables that might be present on downstream
which weren't truncated on upstream.
--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Re: [PATCH] Logical decoding of TRUNCATE
On Thu, Jan 25, 2018 at 8:36 PM, Petr Jelinek
<petr.jelinek@2ndquadrant.com> wrote:
On 26/01/18 02:34, Robert Haas wrote:
On Wed, Jan 17, 2018 at 12:07 PM, Petr Jelinek
<petr.jelinek@2ndquadrant.com> wrote:The patch will cascade truncation on downstream if cascade was specified
on the upstream, that can potentially be dangerous and we either should
not do it and only truncate the tables which were truncated upstream
(but without restricting because of FKs), leaving the data inconsistent
on downstream (like we do already with DELETE or UPDATE). Or maybe make
it into either subscription or publication option so that user can chose
the behaviour here as I am sure some people will want it to cascade (but
the default should still IMHO be to not cascade as that's safer).Maybe I'm not understanding what is being proposed here, but it sounds
like you're saying that if somebody removes a bunch of data on the
logical master, replication will remove only some of it from the
servers to which the change is replicated. That seems crazy. Then
replication can't be counted on to produce a replica.No, I was talking about extra tables that might be present on downstream
which weren't truncated on upstream.
Oh. That's different. :-)
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [PATCH] Logical decoding of TRUNCATE
On 1/24/18 07:53, Petr Jelinek wrote:
That depends on if we consider this to be part of sequence handling or
truncate statement replication handling. It's true that if we had
sequence replication, the restart would get propagated that way anyway.
OTOH, if we'll want to add option in the future to propagate cascade (to
any additional tables on downstream), it may need to reset sequences
which are not even present upstream and as such would not be handled by
sequence replication.
Logical replication, as it currently is designed, replicates discrete
actions, not commands. A delete command that deletes five rows and
cascades to delete three more rows somewhere else ends up being
replicated as eight changes. So I think a TRUNCATE command that
cascades to four tables and resets six sequences should end up being
replicated as ten changes. (Since we currently don't handle sequences
at all, we'll omit the six sequence changes for now.)
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [PATCH] Logical decoding of TRUNCATE
On 26/01/18 03:44, Peter Eisentraut wrote:
On 1/24/18 07:53, Petr Jelinek wrote:
That depends on if we consider this to be part of sequence handling or
truncate statement replication handling. It's true that if we had
sequence replication, the restart would get propagated that way anyway.
OTOH, if we'll want to add option in the future to propagate cascade (to
any additional tables on downstream), it may need to reset sequences
which are not even present upstream and as such would not be handled by
sequence replication.Logical replication, as it currently is designed, replicates discrete
actions, not commands. A delete command that deletes five rows and
cascades to delete three more rows somewhere else ends up being
replicated as eight changes. So I think a TRUNCATE command that
cascades to four tables and resets six sequences should end up being
replicated as ten changes. (Since we currently don't handle sequences
at all, we'll omit the six sequence changes for now.)
Well, that depends, I think there are two separate problems a) decoding
b) replication.
I think it's useful for plugins to know if reset sequence or cascade was
specified in the command so I think it should be decoded. Some plugins
will definitely want to forward that info.
But it's true that since we don't handle sequences in logical
replication, the sequence reset does not need to be replicated there.
--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Re: [PATCH] Logical decoding of TRUNCATE
On 2018-01-25 14:21:15 +0100, Marco Nenciarini wrote:
+ if (SessionReplicationRole != SESSION_REPLICATION_ROLE_REPLICA) + { + + /* + * Check foreign key references. In CASCADE mode, this should be + * unnecessary since we just pulled in all the references; but as a + * cross-check, do it anyway if in an Assert-enabled build. + */ #ifdef USE_ASSERT_CHECKING heap_truncate_check_FKs(rels, false); + #else + if (stmt->behavior == DROP_RESTRICT) + heap_truncate_check_FKs(rels, false); #endif + }
That *can't* be right.
+ case REORDER_BUFFER_CHANGE_TRUNCATE: + appendStringInfoString(ctx->out, " TRUNCATE:"); + + if (change->data.truncate_msg.restart_seqs + || change->data.truncate_msg.cascade) + { + if (change->data.truncate_msg.restart_seqs) + appendStringInfo(ctx->out, " restart_seqs"); + if (change->data.truncate_msg.cascade) + appendStringInfo(ctx->out, " cascade"); + } + else + appendStringInfoString(ctx->out, " (no-flags)"); + break; default: Assert(false); }
I know this has been discussed in the thread already, but it really
strikes me as wrong to basically do some mini DDL replication feature
via per-command WAL records.
*************** *** 111,116 **** CREATE PUBLICATION <replaceable class="parameter">name</replaceable> --- 111,121 ---- and so the default value for this option is <literal>'insert, update, delete'</literal>. </para> + <para> + <command>TRUNCATE</command> is treated as a form of + <command>DELETE</command> for the purpose of deciding whether + to publish, or not. + </para> </listitem> </varlistentry> </variablelist>
Why is this a good idea?
Hm, it seems logicaldecoding.sgml hasn't been updated?
+ void + ExecuteTruncateGuts(List *explicit_rels, List *relids, List *relids_logged, + DropBehavior behavior, bool restart_seqs) + { + List *rels = list_copy(explicit_rels);
Why is this copied?
+ * Write a WAL record to allow this set of actions to be logically decoded. + * We could optimize this away when !RelationIsLogicallyLogged(rel) + * but that doesn't save much space or time.
What you're saying isn't that you're not logging anything, but that
you're allocating the header regardless? Because this certainly sounds
like you unconditionally log a WAL record.
+ * Assemble an array of relids, then an array of seqrelids so we can write + * a single WAL record for the whole action. + */ + logrelids = palloc(maxrelids * sizeof(Oid)); + foreach (cell, relids_logged) + { + nrelids++; + if (nrelids > maxrelids) + { + maxrelids *= 2; + logrelids = repalloc(logrelids, maxrelids * sizeof(Oid)); + } + logrelids[nrelids - 1] = lfirst_oid(cell); + } + + foreach (cell, seq_relids_logged) + { + nseqrelids++; + if ((nrelids + nseqrelids) > maxrelids) + { + maxrelids *= 2; + logrelids = repalloc(logrelids, maxrelids * sizeof(Oid)); + } + logrelids[nrelids + nseqrelids - 1] = lfirst_oid(cell); + }
I'm confused. Why do we need the resizing here, when we know the max
upfront?
+ /* + * For truncate we list all truncated relids in an array, followed by all + * sequence relids that need to be restarted, if any. + * All rels are always within the same database, so we just list dbid once. + */ + typedef struct xl_heap_truncate + { + Oid dbId; + uint32 nrelids; + uint32 nseqrelids; + uint8 flags; + Oid relids[FLEXIBLE_ARRAY_MEMBER]; + } xl_heap_truncate;
Given that the space is used anyway due to padding, I'd just make flags
32bit.
Greetings,
Andres Freund
Re: [PATCH] Logical decoding of TRUNCATE
On 1 April 2018 at 21:01, Andres Freund <andres@anarazel.de> wrote:
*************** *** 111,116 **** CREATE PUBLICATION <replaceable class="parameter">name</replaceable> --- 111,121 ---- and so the default value for this option is <literal>'insert, update, delete'</literal>. </para> + <para> + <command>TRUNCATE</command> is treated as a form of + <command>DELETE</command> for the purpose of deciding whether + to publish, or not. + </para> </listitem> </varlistentry> </variablelist>Why is this a good idea?
TRUNCATE removes rows, just as DELETE does, so anybody that wants to
publish the removal of rows will be interested in this.
This avoids unnecessary overcomplication of the existing interface.
+ * Write a WAL record to allow this set of actions to be logically decoded. + * We could optimize this away when !RelationIsLogicallyLogged(rel) + * but that doesn't save much space or time.What you're saying isn't that you're not logging anything, but that
you're allocating the header regardless? Because this certainly sounds
like you unconditionally log a WAL record.
It says that, yes, my idea - as explained.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [PATCH] Logical decoding of TRUNCATE
On 4/1/18 16:01, Andres Freund wrote:
On 2018-01-25 14:21:15 +0100, Marco Nenciarini wrote:
+ if (SessionReplicationRole != SESSION_REPLICATION_ROLE_REPLICA) + { + + /* + * Check foreign key references. In CASCADE mode, this should be + * unnecessary since we just pulled in all the references; but as a + * cross-check, do it anyway if in an Assert-enabled build. + */ #ifdef USE_ASSERT_CHECKING heap_truncate_check_FKs(rels, false); + #else + if (stmt->behavior == DROP_RESTRICT) + heap_truncate_check_FKs(rels, false); #endif + }That *can't* be right.
You mean the part that ignores this in session_replication_role =
replica? I don't like that either. And it's also not clear why it's
needed for this feature.
+ case REORDER_BUFFER_CHANGE_TRUNCATE: + appendStringInfoString(ctx->out, " TRUNCATE:"); + + if (change->data.truncate_msg.restart_seqs + || change->data.truncate_msg.cascade) + { + if (change->data.truncate_msg.restart_seqs) + appendStringInfo(ctx->out, " restart_seqs"); + if (change->data.truncate_msg.cascade) + appendStringInfo(ctx->out, " cascade"); + } + else + appendStringInfoString(ctx->out, " (no-flags)"); + break; default: Assert(false); }I know this has been discussed in the thread already, but it really
strikes me as wrong to basically do some mini DDL replication feature
via per-command WAL records.
The problem is that the interaction of TRUNCATE and foreign keys is a mess.
Let's say I have a provider database with table1, table2, and table3,
all connected by foreign keys, and a replica database with table1,
table2, and table4, all connected by foreign keys. I run TRUNCATE
table1 CASCADE on the provider. What should replication do?
The proposed patch applies the TRUNCATE table1 CASCADE on the replica,
which ends up truncating table1, table2, and table4, which is different
from what happened on the origin.
An alternative might be to make the replication protocol message say "I
truncated table1, table2" (let's say table3 is not replicated). (This
sounds more like logical replication rather than DDL replication.) That
will then fail to apply on the replica, because it requires that you
include all tables connected by foreign keys.
We could then do what we do in the DELETE case and just ignore foreign
keys altogether, which is obviously bad and not a forward-looking behavior.
Or we could do what we *should* be doing in the DELETE case and apply
cascading actions on the replica to table4, but that kind of row-by-row
processing is what we want to avoid by using TRUNCATE in the first place.
None of these solutions are good. I don't have any other ideas, though.
*************** *** 111,116 **** CREATE PUBLICATION <replaceable class="parameter">name</replaceable> --- 111,121 ---- and so the default value for this option is <literal>'insert, update, delete'</literal>. </para> + <para> + <command>TRUNCATE</command> is treated as a form of + <command>DELETE</command> for the purpose of deciding whether + to publish, or not. + </para> </listitem> </varlistentry> </variablelist>Why is this a good idea?
I think it seemed like a good idea at the time, so to speak, but several
people have argued against it, so we should probably change this in the
final version.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [PATCH] Logical decoding of TRUNCATE
On 2 April 2018 at 19:30, Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:
*************** *** 111,116 **** CREATE PUBLICATION <replaceable class="parameter">name</replaceable> --- 111,121 ---- and so the default value for this option is <literal>'insert, update, delete'</literal>. </para> + <para> + <command>TRUNCATE</command> is treated as a form of + <command>DELETE</command> for the purpose of deciding whether + to publish, or not. + </para> </listitem> </varlistentry> </variablelist>Why is this a good idea?
I think it seemed like a good idea at the time, so to speak, but several
people have argued against it, so we should probably change this in the
final version.
Who has argued against it?
I see that Andres has asked twice about it and been answered twice,
but expressed no opinion.
Petr has said he thinks that's right.
Nobody else has commented.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [PATCH] Logical decoding of TRUNCATE
Hi,
On 2018-04-02 14:30:50 -0400, Peter Eisentraut wrote:
On 4/1/18 16:01, Andres Freund wrote:
On 2018-01-25 14:21:15 +0100, Marco Nenciarini wrote:
+ if (SessionReplicationRole != SESSION_REPLICATION_ROLE_REPLICA) + { + + /* + * Check foreign key references. In CASCADE mode, this should be + * unnecessary since we just pulled in all the references; but as a + * cross-check, do it anyway if in an Assert-enabled build. + */ #ifdef USE_ASSERT_CHECKING heap_truncate_check_FKs(rels, false); + #else + if (stmt->behavior == DROP_RESTRICT) + heap_truncate_check_FKs(rels, false); #endif + }That *can't* be right.
You mean the part that ignores this in session_replication_role =
replica? I don't like that either. And it's also not clear why it's
needed for this feature.
I primarily the way the code is written. For me code differing like this
between USE_ASSERT_CHECKING and not seems like a recipe for disaster.
And yea, I'm not sure why the session_replication_role bit is here either.
I know this has been discussed in the thread already, but it really
strikes me as wrong to basically do some mini DDL replication feature
via per-command WAL records.The problem is that the interaction of TRUNCATE and foreign keys is a mess.
Let's say I have a provider database with table1, table2, and table3,
all connected by foreign keys, and a replica database with table1,
table2, and table4, all connected by foreign keys. I run TRUNCATE
table1 CASCADE on the provider. What should replication do?The proposed patch applies the TRUNCATE table1 CASCADE on the replica,
which ends up truncating table1, table2, and table4, which is different
from what happened on the origin.
I actually think that's a fairly sane behaviour because it allows you to
have different schemas on both sides and still deal in a reasonably sane
manner. What I'm concerned about is more that we're developing a one-of
DDL replication feature w/ corresponding bespoke WAL-logging.
An alternative might be to make the replication protocol message say "I
truncated table1, table2" (let's say table3 is not replicated). (This
sounds more like logical replication rather than DDL replication.) That
will then fail to apply on the replica, because it requires that you
include all tables connected by foreign keys.
And entirely fails if there's schema differences.
We could then do what we do in the DELETE case and just ignore foreign
keys altogether, which is obviously bad and not a forward-looking behavior.Or we could do what we *should* be doing in the DELETE case and apply
cascading actions on the replica to table4, but that kind of row-by-row
processing is what we want to avoid by using TRUNCATE in the first place.
Well, you could also queue a re-check at the end of the processed
message, doing a bulk re-check at the end. But that's obviously more
work.
+ <para> + <command>TRUNCATE</command> is treated as a form of + <command>DELETE</command> for the purpose of deciding whether + to publish, or not. + </para> </listitem> </varlistentry> </variablelist>Why is this a good idea?
I think it seemed like a good idea at the time, so to speak, but several
people have argued against it, so we should probably change this in the
final version.
I think it's e.g. perfectly reasonable to want OLTP changes to be
replicated, but not to truncate historical data. So for me those actions
should be separate...
Greetings,
Andres Freund
Re: [PATCH] Logical decoding of TRUNCATE
Hi,
On 2018-04-02 07:39:57 +0100, Simon Riggs wrote:
On 1 April 2018 at 21:01, Andres Freund <andres@anarazel.de> wrote:
*************** *** 111,116 **** CREATE PUBLICATION <replaceable class="parameter">name</replaceable> --- 111,121 ---- and so the default value for this option is <literal>'insert, update, delete'</literal>. </para> + <para> + <command>TRUNCATE</command> is treated as a form of + <command>DELETE</command> for the purpose of deciding whether + to publish, or not. + </para> </listitem> </varlistentry> </variablelist>Why is this a good idea?
TRUNCATE removes rows, just as DELETE does, so anybody that wants to
publish the removal of rows will be interested in this.
I'm not convinced. I think it's perfectly reasonable to want to truncate
away data on the live node, but maintain it on an archival node. In
that case one commonly wants to continue to maintain OLTP changes (hence
DELETE), but not the bulk truncation. I think there's a reasonable
counter-argument in that this isn't fine grained enough.
+ * Write a WAL record to allow this set of actions to be logically decoded. + * We could optimize this away when !RelationIsLogicallyLogged(rel) + * but that doesn't save much space or time.What you're saying isn't that you're not logging anything, but that
you're allocating the header regardless? Because this certainly sounds
like you unconditionally log a WAL record.It says that, yes, my idea - as explained.
My complaint is that the comment and the actual implementation
differ. The above sounds like it's unconditionally WAL logging, but:
+ /*
+ * Write a WAL record to allow this set of actions to be logically decoded.
+ * We could optimize this away when !RelationIsLogicallyLogged(rel)
+ * but that doesn't save much space or time.
+ *
+ * Assemble an array of relids, then an array of seqrelids so we can write
+ * a single WAL record for the whole action.
+ */
+ logrelids = palloc(maxrelids * sizeof(Oid));
+ foreach (cell, relids_logged)
+ {
+ nrelids++;
+ if (nrelids > maxrelids)
+ {
+ maxrelids *= 2;
+ logrelids = repalloc(logrelids, maxrelids * sizeof(Oid));
+ }
+ logrelids[nrelids - 1] = lfirst_oid(cell);
+ }
+
+ foreach (cell, seq_relids_logged)
+ {
+ nseqrelids++;
+ if ((nrelids + nseqrelids) > maxrelids)
+ {
+ maxrelids *= 2;
+ logrelids = repalloc(logrelids, maxrelids * sizeof(Oid));
+ }
+ logrelids[nrelids + nseqrelids - 1] = lfirst_oid(cell);
+ }
+
+ if ((nrelids + nseqrelids) > 0)
+ {
+ xl_heap_truncate xlrec;
+
+ xlrec.dbId = MyDatabaseId;
+ xlrec.nrelids = nrelids;
+ xlrec.nseqrelids = nseqrelids;
+ xlrec.flags = 0;
+ if (behavior == DROP_CASCADE)
+ xlrec.flags |= XLH_TRUNCATE_CASCADE;
+ if (restart_seqs)
+ xlrec.flags |= XLH_TRUNCATE_RESTART_SEQS;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfHeapTruncate);
+ XLogRegisterData((char *) logrelids,
+ (nrelids + nseqrelids) * sizeof(Oid));
+
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ (void) XLogInsert(RM_HEAP_ID, XLOG_HEAP_TRUNCATE);
+ }Note that the XLogInsert() is in an if branch that's only executed if
there's either logged relids or sequence relids.
I think logrelids should be allocated at the max size immediately, and
the comment rewritten to explicitly only talk about the allocation,
rather than sounding like it's about WAL logging as well.
Greetings,
Andres Freund
Re: [PATCH] Logical decoding of TRUNCATE
On Apr 2, 2018, at 11:50, Andres Freund <andres@anarazel.de> wrote:
I'm not convinced. I think it's perfectly reasonable to want to truncate
away data on the live node, but maintain it on an archival node.
+1. We've had at least one specific request for this, in maintaining a data warehouse from an OLTP system.
--
-- Christophe Pettus
xof@thebuild.com
Re: [PATCH] Logical decoding of TRUNCATE
Here is an updated patch that addresses some of your concerns.
I've split it up into the decoding support and the pgoutput replication
support.
The problem I see now is that when we WAL-log a truncate, we include all
the relids in one record. That seems useful. But during decoding we
split this into one change per relid. So at the receiving end, truncate
can only process one relation at a time, which will fail if there are
foreign keys involved. The solution that had been proposed here was to
ignore foreign keys on the subscriber, which is clearly problematic.
I wonder why this approach was chosen. If we go through the trouble of
WAL-logging all the relations together, why not present them to the
output plugin as one so that they can be applied together? This will
clearly make questions of filtering and replication set membership and
so on more difficult, but that's just the nature of the thing. If you
are connecting tables by foreign keys and only replicate some of them,
then that's going to have confusing effects no matter what you do.
(That's perhaps another reason why having the option of replicating
truncate separately from delete could be useful.)
I'm going to try to hack up an alternative approach and see how it works
out.
On 4/1/18 16:01, Andres Freund wrote:
On 2018-01-25 14:21:15 +0100, Marco Nenciarini wrote:
+ if (SessionReplicationRole != SESSION_REPLICATION_ROLE_REPLICA) + { + + /* + * Check foreign key references. In CASCADE mode, this should be + * unnecessary since we just pulled in all the references; but as a + * cross-check, do it anyway if in an Assert-enabled build. + */ #ifdef USE_ASSERT_CHECKING heap_truncate_check_FKs(rels, false); + #else + if (stmt->behavior == DROP_RESTRICT) + heap_truncate_check_FKs(rels, false); #endif + }That *can't* be right.
This is actually existing code that just looks funny in the diff after
being indented.
But I'd rather skip this patch altogether and find a different solution;
see above.
I know this has been discussed in the thread already, but it really
strikes me as wrong to basically do some mini DDL replication feature
via per-command WAL records.
I think TRUNCATE is special in some ways and it's reasonable to treat it
specially. Real DDL is being worked on in the 2PC decoding thread.
What we are discussing here isn't going to be applicable there and vice
versa, I think. In fact, one of the reasons for this effort is that in
BDR TRUNCATE is currently handled like a DDL command, which doesn't work
well.
+ <para> + <command>TRUNCATE</command> is treated as a form of + <command>DELETE</command> for the purpose of deciding whether + to publish, or not. + </para> </listitem> </varlistentry> </variablelist>Why is this a good idea?
I have changed this by making this a separate property.
Hm, it seems logicaldecoding.sgml hasn't been updated?
I re-checked but didn't find anything suitable to update.
+ void + ExecuteTruncateGuts(List *explicit_rels, List *relids, List *relids_logged, + DropBehavior behavior, bool restart_seqs) + { + List *rels = list_copy(explicit_rels);Why is this copied?
Because it is overwritten later. I have moved it down a bit to make
that a bit clearer.
+ * Write a WAL record to allow this set of actions to be logically decoded. + * We could optimize this away when !RelationIsLogicallyLogged(rel) + * but that doesn't save much space or time.What you're saying isn't that you're not logging anything, but that
you're allocating the header regardless? Because this certainly sounds
like you unconditionally log a WAL record.
I'm confused. Why do we need the resizing here, when we know the max
upfront?
I have rewritten this a bit and removed the logging of the sequence
relids, which isn't used anywhere after, to make the code a bit simpler.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
v18-0001-Logical-decoding-of-TRUNCATE.patchtext/plain; charset=UTF-8; name=v18-0001-Logical-decoding-of-TRUNCATE.patch; x-mac-creator=0; x-mac-type=0Download
From 72577e567725526543e5493e235c1059610f3467 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <peter_e@gmx.net>
Date: Tue, 3 Apr 2018 22:12:00 -0400
Subject: [PATCH v18 1/2] Logical decoding of TRUNCATE
---
contrib/test_decoding/expected/ddl.out | 10 ++-
contrib/test_decoding/sql/ddl.sql | 3 +
contrib/test_decoding/test_decoding.c | 14 +++
src/backend/access/heap/heapam.c | 7 ++
src/backend/access/rmgrdesc/heapdesc.c | 14 +++
src/backend/commands/tablecmds.c | 87 +++++++++++++++++--
src/backend/replication/logical/decode.c | 46 ++++++++++
.../replication/logical/reorderbuffer.c | 10 +++
src/include/access/heapam_xlog.h | 23 ++++-
src/include/commands/tablecmds.h | 2 +
src/include/replication/reorderbuffer.h | 16 +++-
11 files changed, 219 insertions(+), 13 deletions(-)
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index b7c76469fc..89c9dc5466 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -547,6 +547,8 @@ UPDATE table_with_unique_not_null SET data = 3 WHERE data = 2;
UPDATE table_with_unique_not_null SET id = -id;
UPDATE table_with_unique_not_null SET id = -id;
DELETE FROM table_with_unique_not_null WHERE data = 3;
+TRUNCATE table_with_unique_not_null;
+TRUNCATE table_with_unique_not_null, table_with_unique_not_null RESTART IDENTITY CASCADE;
-- check toast support
BEGIN;
CREATE SEQUENCE toasttable_rand_seq START 79 INCREMENT 1499; -- portable "random"
@@ -664,6 +666,12 @@ SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
table public.table_with_unique_not_null: DELETE: id[integer]:4
COMMIT
BEGIN
+ table public.table_with_unique_not_null: TRUNCATE: (no-flags)
+ COMMIT
+ BEGIN
+ table public.table_with_unique_not_null: TRUNCATE: restart_seqs cascade
+ COMMIT
+ BEGIN
table public.toasttable: INSERT: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
BEGIN
@@ -672,7 +680,7 @@ SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
BEGIN
table public.toasttable: UPDATE: id[integer]:1 toasted_col1[text]:'12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000' rand1[double precision]:79 toasted_col2[text]:null rand2[double precision]:1578
COMMIT
-(103 rows)
+(109 rows)
INSERT INTO toasttable(toasted_col1) SELECT string_agg(g.i::text, '') FROM generate_series(1, 2000) g(i);
-- update of second column, first column unchanged
diff --git a/contrib/test_decoding/sql/ddl.sql b/contrib/test_decoding/sql/ddl.sql
index c4b10a4cf9..2724d36b79 100644
--- a/contrib/test_decoding/sql/ddl.sql
+++ b/contrib/test_decoding/sql/ddl.sql
@@ -332,6 +332,9 @@ CREATE TABLE table_with_unique_not_null(id serial unique, data int);
UPDATE table_with_unique_not_null SET id = -id;
DELETE FROM table_with_unique_not_null WHERE data = 3;
+TRUNCATE table_with_unique_not_null;
+TRUNCATE table_with_unique_not_null, table_with_unique_not_null RESTART IDENTITY CASCADE;
+
-- check toast support
BEGIN;
CREATE SEQUENCE toasttable_rand_seq START 79 INCREMENT 1499; -- portable "random"
diff --git a/contrib/test_decoding/test_decoding.c b/contrib/test_decoding/test_decoding.c
index a94aeeae29..8c18821886 100644
--- a/contrib/test_decoding/test_decoding.c
+++ b/contrib/test_decoding/test_decoding.c
@@ -470,6 +470,20 @@ pg_decode_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
&change->data.tp.oldtuple->tuple,
true);
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ appendStringInfoString(ctx->out, " TRUNCATE:");
+
+ if (change->data.truncate_msg.restart_seqs
+ || change->data.truncate_msg.cascade)
+ {
+ if (change->data.truncate_msg.restart_seqs)
+ appendStringInfo(ctx->out, " restart_seqs");
+ if (change->data.truncate_msg.cascade)
+ appendStringInfo(ctx->out, " cascade");
+ }
+ else
+ appendStringInfoString(ctx->out, " (no-flags)");
+ break;
default:
Assert(false);
}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index d7279248e7..8fc36fa83d 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -9256,6 +9256,13 @@ heap_redo(XLogReaderState *record)
case XLOG_HEAP_UPDATE:
heap_xlog_update(record, false);
break;
+ case XLOG_HEAP_TRUNCATE:
+ /*
+ * TRUNCATE is a no-op because the actions are already logged as
+ * SMGR WAL records. TRUNCATE WAL record only exists for logical
+ * decoding.
+ */
+ break;
case XLOG_HEAP_HOT_UPDATE:
heap_xlog_update(record, true);
break;
diff --git a/src/backend/access/rmgrdesc/heapdesc.c b/src/backend/access/rmgrdesc/heapdesc.c
index b00c071cb6..349feb6510 100644
--- a/src/backend/access/rmgrdesc/heapdesc.c
+++ b/src/backend/access/rmgrdesc/heapdesc.c
@@ -75,6 +75,17 @@ heap_desc(StringInfo buf, XLogReaderState *record)
xlrec->new_offnum,
xlrec->new_xmax);
}
+ else if (info == XLOG_HEAP_TRUNCATE)
+ {
+ xl_heap_truncate *xlrec = (xl_heap_truncate *) rec;
+
+ if (xlrec->flags & XLH_TRUNCATE_CASCADE)
+ appendStringInfo(buf, "cascade ");
+ if (xlrec->flags & XLH_TRUNCATE_RESTART_SEQS)
+ appendStringInfo(buf, "restart_seqs ");
+ appendStringInfo(buf, "nrelids %u", xlrec->nrelids);
+ /* skip the list of relids */
+ }
else if (info == XLOG_HEAP_CONFIRM)
{
xl_heap_confirm *xlrec = (xl_heap_confirm *) rec;
@@ -186,6 +197,9 @@ heap_identify(uint8 info)
case XLOG_HEAP_HOT_UPDATE | XLOG_HEAP_INIT_PAGE:
id = "HOT_UPDATE+INIT";
break;
+ case XLOG_HEAP_TRUNCATE:
+ id = "TRUNCATE";
+ break;
case XLOG_HEAP_CONFIRM:
id = "HEAP_CONFIRM";
break;
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index c8da82217d..840373f641 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -16,6 +16,7 @@
#include "access/genam.h"
#include "access/heapam.h"
+#include "access/heapam_xlog.h"
#include "access/multixact.h"
#include "access/reloptions.h"
#include "access/relscan.h"
@@ -1316,11 +1317,7 @@ ExecuteTruncate(TruncateStmt *stmt)
{
List *rels = NIL;
List *relids = NIL;
- List *seq_relids = NIL;
- EState *estate;
- ResultRelInfo *resultRelInfos;
- ResultRelInfo *resultRelInfo;
- SubTransactionId mySubid;
+ List *relids_logged = NIL;
ListCell *cell;
/*
@@ -1344,6 +1341,9 @@ ExecuteTruncate(TruncateStmt *stmt)
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, myrelid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, myrelid);
if (recurse)
{
@@ -1364,6 +1364,9 @@ ExecuteTruncate(TruncateStmt *stmt)
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, childrelid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, childrelid);
}
}
else if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
@@ -1373,7 +1376,34 @@ ExecuteTruncate(TruncateStmt *stmt)
errhint("Do not specify the ONLY keyword, or use TRUNCATE ONLY on the partitions directly.")));
}
+ ExecuteTruncateGuts(rels, relids, relids_logged,
+ stmt->behavior, stmt->restart_seqs);
+
+ /* And close the rels */
+ foreach(cell, rels)
+ {
+ Relation rel = (Relation) lfirst(cell);
+
+ heap_close(rel, NoLock);
+ }
+}
+
+void
+ExecuteTruncateGuts(List *explicit_rels, List *relids, List *relids_logged,
+ DropBehavior behavior, bool restart_seqs)
+{
+ List *rels;
+ List *seq_relids = NIL;
+ EState *estate;
+ ResultRelInfo *resultRelInfos;
+ ResultRelInfo *resultRelInfo;
+ SubTransactionId mySubid;
+ ListCell *cell;
+ Oid *logrelids;
+
/*
+ * Open, exclusive-lock, and check all the explicitly-specified relations
+ *
* In CASCADE mode, suck in all referencing relations as well. This
* requires multiple iterations to find indirectly-dependent relations. At
* each phase, we need to exclusive-lock new rels before looking for their
@@ -1381,7 +1411,8 @@ ExecuteTruncate(TruncateStmt *stmt)
* soon as we open it, to avoid a faux pas such as holding lock for a long
* time on a rel we have no permissions for.
*/
- if (stmt->behavior == DROP_CASCADE)
+ rels = list_copy(explicit_rels);
+ if (behavior == DROP_CASCADE)
{
for (;;)
{
@@ -1403,6 +1434,9 @@ ExecuteTruncate(TruncateStmt *stmt)
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, relid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, relid);
}
}
}
@@ -1415,7 +1449,7 @@ ExecuteTruncate(TruncateStmt *stmt)
#ifdef USE_ASSERT_CHECKING
heap_truncate_check_FKs(rels, false);
#else
- if (stmt->behavior == DROP_RESTRICT)
+ if (behavior == DROP_RESTRICT)
heap_truncate_check_FKs(rels, false);
#endif
@@ -1425,7 +1459,7 @@ ExecuteTruncate(TruncateStmt *stmt)
* We want to do this early since it's pointless to do all the truncation
* work only to fail on sequence permissions.
*/
- if (stmt->restart_seqs)
+ if (restart_seqs)
{
foreach(cell, rels)
{
@@ -1580,6 +1614,38 @@ ExecuteTruncate(TruncateStmt *stmt)
ResetSequence(seq_relid);
}
+ /*
+ * Write a WAL record to allow this set of actions to be logically decoded.
+ *
+ * Assemble an array of relids so we can write a single WAL record for the
+ * whole action.
+ */
+ if (list_length(relids_logged) > 0)
+ {
+ xl_heap_truncate xlrec;
+ int i = 0;
+
+ logrelids = palloc(list_length(relids_logged) * sizeof(Oid));
+ foreach (cell, relids_logged)
+ logrelids[i++] = lfirst_oid(cell);
+
+ xlrec.dbId = MyDatabaseId;
+ xlrec.nrelids = list_length(relids_logged);
+ xlrec.flags = 0;
+ if (behavior == DROP_CASCADE)
+ xlrec.flags |= XLH_TRUNCATE_CASCADE;
+ if (restart_seqs)
+ xlrec.flags |= XLH_TRUNCATE_RESTART_SEQS;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfHeapTruncate);
+ XLogRegisterData((char *) logrelids, list_length(relids_logged) * sizeof(Oid));
+
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ (void) XLogInsert(RM_HEAP_ID, XLOG_HEAP_TRUNCATE);
+ }
+
/*
* Process all AFTER STATEMENT TRUNCATE triggers.
*/
@@ -1597,7 +1663,10 @@ ExecuteTruncate(TruncateStmt *stmt)
/* We can clean up the EState now */
FreeExecutorState(estate);
- /* And close the rels (can't do this while EState still holds refs) */
+ /* And close the eventual rels opened by CASCADE
+ * (can't do this while EState still holds refs)
+ */
+ rels = list_difference_ptr(rels, explicit_rels);
foreach(cell, rels)
{
Relation rel = (Relation) lfirst(cell);
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index 6eb0d5527e..fb02db6857 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -65,6 +65,7 @@ static void DecodeLogicalMsgOp(LogicalDecodingContext *ctx, XLogRecordBuffer *bu
static void DecodeInsert(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeUpdate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeDelete(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
+static void DecodeTruncate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeMultiInsert(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeSpecConfirm(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
@@ -449,6 +450,11 @@ DecodeHeapOp(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
DecodeDelete(ctx, buf);
break;
+ case XLOG_HEAP_TRUNCATE:
+ if (SnapBuildProcessChange(builder, xid, buf->origptr))
+ DecodeTruncate(ctx, buf);
+ break;
+
case XLOG_HEAP_INPLACE:
/*
@@ -825,6 +831,46 @@ DecodeDelete(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
ReorderBufferQueueChange(ctx->reorder, XLogRecGetXid(r), buf->origptr, change);
}
+/*
+ * Parse XLOG_HEAP_TRUNCATE from wal
+ */
+static void
+DecodeTruncate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+{
+ XLogReaderState *r = buf->record;
+ xl_heap_truncate *xlrec;
+ ReorderBufferChange *change;
+ int i;
+
+ xlrec = (xl_heap_truncate *) XLogRecGetData(r);
+
+ /* only interested in our database */
+ if (xlrec->dbId != ctx->slot->data.database)
+ return;
+
+ /* output plugin doesn't look for this origin, no need to queue */
+ if (FilterByOrigin(ctx, XLogRecGetOrigin(r)))
+ return;
+
+ change = ReorderBufferGetChange(ctx->reorder);
+ change->action = REORDER_BUFFER_CHANGE_TRUNCATE;
+ change->origin_id = XLogRecGetOrigin(r);
+ if (xlrec->flags & XLH_TRUNCATE_CASCADE)
+ change->data.truncate_msg.cascade = true;
+ if (xlrec->flags & XLH_TRUNCATE_RESTART_SEQS)
+ change->data.truncate_msg.restart_seqs = true;
+
+ /*
+ * Queue up one change per relation
+ */
+ for (i = 0; i < xlrec->nrelids; i++)
+ {
+ change->data.truncate_msg.relid = xlrec->relids[i];
+ ReorderBufferQueueChange(ctx->reorder, XLogRecGetXid(r),
+ buf->origptr, change);
+ }
+}
+
/*
* Decode XLOG_HEAP2_MULTI_INSERT_insert record into multiple tuplebufs.
*
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index b4016ed52b..0869e30bae 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -415,6 +415,7 @@ ReorderBufferReturnChange(ReorderBuffer *rb, ReorderBufferChange *change)
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
break;
}
@@ -1355,6 +1356,13 @@ ReorderBufferCommit(ReorderBuffer *rb, TransactionId xid,
switch (change->action)
{
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ reloid = change->data.truncate_msg.relid;
+ relation = RelationIdGetRelation(reloid);
+ rb->apply_change(rb, txn, relation, change);
+ RelationClose(relation);
+ break;
+
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
/*
@@ -2255,6 +2263,7 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
}
break;
}
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
@@ -2534,6 +2543,7 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
/* the base struct contains all the data, easy peasy */
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h
index 700e25c36a..0052e4c569 100644
--- a/src/include/access/heapam_xlog.h
+++ b/src/include/access/heapam_xlog.h
@@ -32,7 +32,7 @@
#define XLOG_HEAP_INSERT 0x00
#define XLOG_HEAP_DELETE 0x10
#define XLOG_HEAP_UPDATE 0x20
-/* 0x030 is free, was XLOG_HEAP_MOVE */
+#define XLOG_HEAP_TRUNCATE 0x30
#define XLOG_HEAP_HOT_UPDATE 0x40
#define XLOG_HEAP_CONFIRM 0x50
#define XLOG_HEAP_LOCK 0x60
@@ -109,6 +109,27 @@ typedef struct xl_heap_delete
#define SizeOfHeapDelete (offsetof(xl_heap_delete, flags) + sizeof(uint8))
+/*
+ * xl_heap_delete flag values, 8 bits are available.
+ */
+#define XLH_TRUNCATE_CASCADE (1<<0)
+#define XLH_TRUNCATE_RESTART_SEQS (1<<1)
+
+/*
+ * For truncate we list all truncated relids in an array, followed by all
+ * sequence relids that need to be restarted, if any.
+ * All rels are always within the same database, so we just list dbid once.
+ */
+typedef struct xl_heap_truncate
+{
+ Oid dbId;
+ uint32 nrelids;
+ uint8 flags;
+ Oid relids[FLEXIBLE_ARRAY_MEMBER];
+} xl_heap_truncate;
+
+#define SizeOfHeapTruncate (offsetof(xl_heap_truncate, relids))
+
/*
* We don't store the whole fixed part (HeapTupleHeaderData) of an inserted
* or updated tuple in WAL; we can save a few bytes by reconstructing the
diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h
index 06e5180a30..ceabcc14d8 100644
--- a/src/include/commands/tablecmds.h
+++ b/src/include/commands/tablecmds.h
@@ -54,6 +54,8 @@ extern void AlterRelationNamespaceInternal(Relation classRel, Oid relOid,
extern void CheckTableNotInUse(Relation rel, const char *stmt);
extern void ExecuteTruncate(TruncateStmt *stmt);
+extern void ExecuteTruncateGuts(List *explicit_rels, List *relids, List *relids_logged,
+ DropBehavior behavior, bool restart_seqs);
extern void SetRelationHasSubclass(Oid relationId, bool relhassubclass);
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index aa430c843c..7cb3b0f26a 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -59,7 +59,8 @@ enum ReorderBufferChangeType
REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID,
REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID,
REORDER_BUFFER_CHANGE_INTERNAL_SPEC_INSERT,
- REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM
+ REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM,
+ REORDER_BUFFER_CHANGE_TRUNCATE
};
/*
@@ -128,7 +129,18 @@ typedef struct ReorderBufferChange
CommandId cmax;
CommandId combocid;
} tuplecid;
- } data;
+
+ /*
+ * Truncate data for REORDER_BUFFER_CHANGE_TRUNCATE representing
+ * one relation to be truncated.
+ */
+ struct
+ {
+ Oid relid;
+ bool cascade;
+ bool restart_seqs;
+ } truncate_msg;
+ } data;
/*
* While in use this is how a change is linked into a transactions,
base-commit: ed69864350a59c51c8570900601ebd335956b638
--
2.17.0
v18-0002-Logical-replication-support-for-TRUNCATE.patchtext/plain; charset=UTF-8; name=v18-0002-Logical-replication-support-for-TRUNCATE.patch; x-mac-creator=0; x-mac-type=0Download
From e2773f54d42fb7ee41562d720861e5ede9fecc6f Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <peter_e@gmx.net>
Date: Tue, 3 Apr 2018 22:13:07 -0400
Subject: [PATCH v18 2/2] Logical replication support for TRUNCATE
---
doc/src/sgml/catalogs.sgml | 8 ++
doc/src/sgml/logical-replication.sgml | 13 +--
doc/src/sgml/protocol.sgml | 46 +++++++++
doc/src/sgml/ref/create_publication.sgml | 10 +-
src/backend/catalog/pg_publication.c | 1 +
src/backend/commands/publicationcmds.c | 20 +++-
src/backend/replication/logical/proto.c | 45 +++++++++
src/backend/replication/logical/worker.c | 57 +++++++++++
src/backend/replication/pgoutput/pgoutput.c | 16 ++-
src/backend/utils/cache/relcache.c | 3 +-
src/bin/pg_dump/pg_dump.c | 33 ++++--
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 2 +-
src/bin/psql/describe.c | 26 ++++-
src/include/catalog/pg_publication.h | 7 +-
src/include/replication/logicalproto.h | 4 +
src/test/regress/expected/publication.out | 84 ++++++++--------
src/test/subscription/t/010_truncate.pl | 106 ++++++++++++++++++++
18 files changed, 406 insertions(+), 76 deletions(-)
create mode 100644 src/test/subscription/t/010_truncate.pl
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index d6a9d8c580..e8efa13e8d 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -5518,6 +5518,14 @@ <title><structname>pg_publication</structname> Columns</title>
<entry>If true, <command>DELETE</command> operations are replicated for
tables in the publication.</entry>
</row>
+
+ <row>
+ <entry><structfield>pubtruncate</structfield></entry>
+ <entry><type>bool</type></entry>
+ <entry></entry>
+ <entry>If true, <command>TRUNCATE</command> operations are replicated for
+ tables in the publication.</entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 75551d8ee1..151e773fc2 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -108,8 +108,8 @@ <title>Publication</title>
<para>
Publications can choose to limit the changes they produce to
- any combination of <command>INSERT</command>, <command>UPDATE</command>, and
- <command>DELETE</command>, similar to how triggers are fired by
+ any combination of <command>INSERT</command>, <command>UPDATE</command>,
+ <command>DELETE</command>, and <command>TRUNCATE</command>, similar to how triggers are fired by
particular event types. By default, all operation types are replicated.
</para>
@@ -364,15 +364,6 @@ <title>Restrictions</title>
</para>
</listitem>
- <listitem>
- <para>
- <command>TRUNCATE</command> commands are not replicated. This can, of
- course, be worked around by using <command>DELETE</command> instead. To
- avoid accidental <command>TRUNCATE</command> invocations, you can revoke
- the <literal>TRUNCATE</literal> privilege from tables.
- </para>
- </listitem>
-
<listitem>
<para>
Large objects (see <xref linkend="largeobjects"/>) are not replicated.
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index 8c488506fa..83fe7804d0 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -6763,6 +6763,52 @@ <title>Logical Replication Message Formats</title>
</listitem>
</varlistentry>
+<varlistentry>
+<term>
+Truncate
+</term>
+<listitem>
+<para>
+
+<variablelist>
+<varlistentry>
+<term>
+ Byte1('T')
+</term>
+<listitem>
+<para>
+ Identifies the message as a truncate message.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+ Int32
+</term>
+<listitem>
+<para>
+ ID of the relation corresponding to the ID in the relation
+ message.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+ Int8
+</term>
+<listitem>
+<para>
+ Option bits for <command>TRUNCATE</command>:
+ 1 for <literal>CASCADE</literal>, 2 for <literal>RESTART IDENTITY</literal>
+</para>
+</listitem>
+</varlistentry>
+
+</variablelist>
+</para>
+</listitem>
+</varlistentry>
+
</variablelist>
<para>
diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml
index bfe12d5f41..99f87ca393 100644
--- a/doc/src/sgml/ref/create_publication.sgml
+++ b/doc/src/sgml/ref/create_publication.sgml
@@ -106,10 +106,11 @@ <title>Parameters</title>
This parameter determines which DML operations will be published by
the new publication to the subscribers. The value is
comma-separated list of operations. The allowed operations are
- <literal>insert</literal>, <literal>update</literal>, and
- <literal>delete</literal>. The default is to publish all actions,
+ <literal>insert</literal>, <literal>update</literal>,
+ <literal>delete</literal>, and <literal>truncate</literal>.
+ The default is to publish all actions,
and so the default value for this option is
- <literal>'insert, update, delete'</literal>.
+ <literal>'insert, update, delete, truncate'</literal>.
</para>
</listitem>
</varlistentry>
@@ -168,8 +169,7 @@ <title>Notes</title>
</para>
<para>
- <command>TRUNCATE</command> and <acronym>DDL</acronym> operations
- are not published.
+ <acronym>DDL</acronym> operations are not published.
</para>
</refsect1>
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index ba18258ebb..ec3bd1d22d 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -376,6 +376,7 @@ GetPublication(Oid pubid)
pub->pubactions.pubinsert = pubform->pubinsert;
pub->pubactions.pubupdate = pubform->pubupdate;
pub->pubactions.pubdelete = pubform->pubdelete;
+ pub->pubactions.pubtruncate = pubform->pubtruncate;
ReleaseSysCache(tup);
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 9c5aa9ebc2..29992d4a0e 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -62,7 +62,8 @@ parse_publication_options(List *options,
bool *publish_given,
bool *publish_insert,
bool *publish_update,
- bool *publish_delete)
+ bool *publish_delete,
+ bool *publish_truncate)
{
ListCell *lc;
@@ -72,6 +73,7 @@ parse_publication_options(List *options,
*publish_insert = true;
*publish_update = true;
*publish_delete = true;
+ *publish_truncate = true;
/* Parse options */
foreach(lc, options)
@@ -96,6 +98,7 @@ parse_publication_options(List *options,
*publish_insert = false;
*publish_update = false;
*publish_delete = false;
+ *publish_truncate = false;
*publish_given = true;
publish = defGetString(defel);
@@ -116,6 +119,8 @@ parse_publication_options(List *options,
*publish_update = true;
else if (strcmp(publish_opt, "delete") == 0)
*publish_delete = true;
+ else if (strcmp(publish_opt, "truncate") == 0)
+ *publish_truncate = true;
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -145,6 +150,7 @@ CreatePublication(CreatePublicationStmt *stmt)
bool publish_insert;
bool publish_update;
bool publish_delete;
+ bool publish_truncate;
AclResult aclresult;
/* must have CREATE privilege on database */
@@ -181,7 +187,8 @@ CreatePublication(CreatePublicationStmt *stmt)
parse_publication_options(stmt->options,
&publish_given, &publish_insert,
- &publish_update, &publish_delete);
+ &publish_update, &publish_delete,
+ &publish_truncate);
values[Anum_pg_publication_puballtables - 1] =
BoolGetDatum(stmt->for_all_tables);
@@ -191,6 +198,8 @@ CreatePublication(CreatePublicationStmt *stmt)
BoolGetDatum(publish_update);
values[Anum_pg_publication_pubdelete - 1] =
BoolGetDatum(publish_delete);
+ values[Anum_pg_publication_pubtruncate - 1] =
+ BoolGetDatum(publish_truncate);
tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
@@ -237,11 +246,13 @@ AlterPublicationOptions(AlterPublicationStmt *stmt, Relation rel,
bool publish_insert;
bool publish_update;
bool publish_delete;
+ bool publish_truncate;
ObjectAddress obj;
parse_publication_options(stmt->options,
&publish_given, &publish_insert,
- &publish_update, &publish_delete);
+ &publish_update, &publish_delete,
+ &publish_truncate);
/* Everything ok, form a new tuple. */
memset(values, 0, sizeof(values));
@@ -258,6 +269,9 @@ AlterPublicationOptions(AlterPublicationStmt *stmt, Relation rel,
values[Anum_pg_publication_pubdelete - 1] = BoolGetDatum(publish_delete);
replaces[Anum_pg_publication_pubdelete - 1] = true;
+
+ values[Anum_pg_publication_pubtruncate - 1] = BoolGetDatum(publish_truncate);
+ replaces[Anum_pg_publication_pubtruncate - 1] = true;
}
tup = heap_modify_tuple(tup, RelationGetDescr(rel), values, nulls,
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index 948343e4ae..2fa6f8393d 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -26,6 +26,9 @@
*/
#define LOGICALREP_IS_REPLICA_IDENTITY 1
+#define TRUNCATE_CASCADE (1<<0)
+#define TRUNCATE_RESTART_SEQS (1<<1)
+
static void logicalrep_write_attrs(StringInfo out, Relation rel);
static void logicalrep_write_tuple(StringInfo out, Relation rel,
HeapTuple tuple);
@@ -292,6 +295,48 @@ logicalrep_read_delete(StringInfo in, LogicalRepTupleData *oldtup)
return relid;
}
+/*
+ * Write TRUNCATE to the output stream.
+ */
+void
+logicalrep_write_truncate(StringInfo out, Relation rel,
+ bool cascade, bool restart_seqs)
+{
+ uint8 flags = 0;
+
+ pq_sendbyte(out, 'T'); /* action TRUNCATE */
+
+ /* use Oid as relation identifier */
+ pq_sendint32(out, RelationGetRelid(rel));
+
+ /* encode and send truncate flags */
+ if (cascade)
+ flags |= TRUNCATE_CASCADE;
+ if (restart_seqs)
+ flags |= TRUNCATE_RESTART_SEQS;
+ pq_sendint8(out, flags);
+}
+
+/*
+ * Read TRUNCATE from stream.
+ */
+LogicalRepRelId
+logicalrep_read_truncate(StringInfo in, bool *cascade, bool *restart_seqs)
+{
+ LogicalRepRelId relid;
+ uint8 flags;
+
+ /* read the relation id */
+ relid = pq_getmsgint(in, 4);
+
+ /* read and decode truncate flags */
+ flags = pq_getmsgint(in, 1);
+ *cascade = (flags & TRUNCATE_CASCADE) > 0;
+ *restart_seqs = (flags & TRUNCATE_RESTART_SEQS) > 0;
+
+ return relid;
+}
+
/*
* Write relation description to the output stream.
*/
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index fdace7eea2..7c9c24ae9f 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -30,10 +30,12 @@
#include "access/xact.h"
#include "access/xlog_internal.h"
+#include "catalog/catalog.h"
#include "catalog/namespace.h"
#include "catalog/pg_subscription.h"
#include "catalog/pg_subscription_rel.h"
+#include "commands/tablecmds.h"
#include "commands/trigger.h"
#include "executor/executor.h"
@@ -83,6 +85,7 @@
#include "utils/inval.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
+#include "utils/rel.h"
#include "utils/timeout.h"
#include "utils/tqual.h"
#include "utils/syscache.h"
@@ -888,6 +891,56 @@ apply_handle_delete(StringInfo s)
CommandCounterIncrement();
}
+/*
+ * Handle TRUNCATE message.
+ *
+ * TODO: FDW support
+ */
+static void
+apply_handle_truncate(StringInfo s)
+{
+ LogicalRepRelMapEntry *rel;
+ LogicalRepRelId relid;
+ bool cascade = false;
+ bool restart_seqs = false;
+ List *rels = NIL;
+ List *relids = NIL;
+ List *relids_logged = NIL;
+
+ ensure_transaction();
+
+ relid = logicalrep_read_truncate(s, &cascade, &restart_seqs);
+ rel = logicalrep_rel_open(relid, RowExclusiveLock);
+ if (!should_apply_changes_for_rel(rel))
+ {
+ /*
+ * The relation can't become interesting in the middle of the
+ * transaction so it's safe to unlock it.
+ */
+ logicalrep_rel_close(rel, RowExclusiveLock);
+ return;
+ }
+
+ /* Check if we can do the truncate. */
+ check_relation_updatable(rel);
+
+ rels = lappend(rels, rel->localrel);
+ relids = lappend_oid(relids, rel->localreloid);
+ if (RelationIsLogicallyLogged(rel->localrel))
+ relids_logged = lappend_oid(relids, rel->localreloid);
+
+ /*
+ * Even if we used CASCADE on the upstream master we explicitly
+ * default to replaying changes without further cascading.
+ * This might be later changeable with a user specified option.
+ */
+ ExecuteTruncateGuts(rels, relids, relids_logged, DROP_RESTRICT, restart_seqs);
+
+ logicalrep_rel_close(rel, NoLock);
+
+ CommandCounterIncrement();
+}
+
/*
* Logical replication protocol message dispatcher.
@@ -919,6 +972,10 @@ apply_dispatch(StringInfo s)
case 'D':
apply_handle_delete(s);
break;
+ /* TRUNCATE */
+ case 'T':
+ apply_handle_truncate(s);
+ break;
/* RELATION */
case 'R':
apply_handle_relation(s);
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index aa9cf5b54e..27d28259f2 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -281,6 +281,10 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
if (!relentry->pubactions.pubdelete)
return;
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ if (!relentry->pubactions.pubtruncate)
+ return;
+ break;
default:
Assert(false);
}
@@ -354,6 +358,13 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
else
elog(DEBUG1, "didn't send DELETE change because of missing oldtuple");
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ OutputPluginPrepareWrite(ctx, true);
+ logicalrep_write_truncate(ctx->out, relation,
+ change->data.truncate_msg.cascade,
+ change->data.truncate_msg.restart_seqs);
+ OutputPluginWrite(ctx, true);
+ break;
default:
Assert(false);
}
@@ -504,7 +515,7 @@ get_rel_sync_entry(PGOutputData *data, Oid relid)
* we only need to consider ones that the subscriber requested.
*/
entry->pubactions.pubinsert = entry->pubactions.pubupdate =
- entry->pubactions.pubdelete = false;
+ entry->pubactions.pubdelete = entry->pubactions.pubtruncate = false;
foreach(lc, data->publications)
{
@@ -515,10 +526,11 @@ get_rel_sync_entry(PGOutputData *data, Oid relid)
entry->pubactions.pubinsert |= pub->pubactions.pubinsert;
entry->pubactions.pubupdate |= pub->pubactions.pubupdate;
entry->pubactions.pubdelete |= pub->pubactions.pubdelete;
+ entry->pubactions.pubtruncate |= pub->pubactions.pubtruncate;
}
if (entry->pubactions.pubinsert && entry->pubactions.pubupdate &&
- entry->pubactions.pubdelete)
+ entry->pubactions.pubdelete && entry->pubactions.pubtruncate)
break;
}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 69a2114a10..6a67c185b0 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -5339,6 +5339,7 @@ GetRelationPublicationActions(Relation relation)
pubactions->pubinsert |= pubform->pubinsert;
pubactions->pubupdate |= pubform->pubupdate;
pubactions->pubdelete |= pubform->pubdelete;
+ pubactions->pubtruncate |= pubform->pubtruncate;
ReleaseSysCache(tup);
@@ -5347,7 +5348,7 @@ GetRelationPublicationActions(Relation relation)
* other publications.
*/
if (pubactions->pubinsert && pubactions->pubupdate &&
- pubactions->pubdelete)
+ pubactions->pubdelete && pubactions->pubtruncate)
break;
}
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index b8d65a9ee3..f5286f890e 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -3712,6 +3712,7 @@ getPublications(Archive *fout)
int i_pubinsert;
int i_pubupdate;
int i_pubdelete;
+ int i_pubtruncate;
int i,
ntups;
@@ -3723,12 +3724,20 @@ getPublications(Archive *fout)
resetPQExpBuffer(query);
/* Get the publications. */
- appendPQExpBuffer(query,
- "SELECT p.tableoid, p.oid, p.pubname, "
- "(%s p.pubowner) AS rolname, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete "
- "FROM pg_publication p",
- username_subquery);
+ if (fout->remoteVersion >= 110000)
+ appendPQExpBuffer(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "(%s p.pubowner) AS rolname, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate "
+ "FROM pg_publication p",
+ username_subquery);
+ else
+ appendPQExpBuffer(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "(%s p.pubowner) AS rolname, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate "
+ "FROM pg_publication p",
+ username_subquery);
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -3742,6 +3751,7 @@ getPublications(Archive *fout)
i_pubinsert = PQfnumber(res, "pubinsert");
i_pubupdate = PQfnumber(res, "pubupdate");
i_pubdelete = PQfnumber(res, "pubdelete");
+ i_pubtruncate = PQfnumber(res, "pubtruncate");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -3762,6 +3772,8 @@ getPublications(Archive *fout)
(strcmp(PQgetvalue(res, i, i_pubupdate), "t") == 0);
pubinfo[i].pubdelete =
(strcmp(PQgetvalue(res, i, i_pubdelete), "t") == 0);
+ pubinfo[i].pubtruncate =
+ (strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
if (strlen(pubinfo[i].rolname) == 0)
write_msg(NULL, "WARNING: owner of publication \"%s\" appears to be invalid\n",
@@ -3829,6 +3841,15 @@ dumpPublication(Archive *fout, PublicationInfo *pubinfo)
first = false;
}
+ if (pubinfo->pubtruncate)
+ {
+ if (!first)
+ appendPQExpBufferStr(query, ", ");
+
+ appendPQExpBufferStr(query, "truncate");
+ first = false;
+ }
+
appendPQExpBufferStr(query, "');\n");
ArchiveEntry(fout, pubinfo->dobj.catId, pubinfo->dobj.dumpId,
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index a4d6d926a8..c2314758de 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -595,6 +595,7 @@ typedef struct _PublicationInfo
bool pubinsert;
bool pubupdate;
bool pubdelete;
+ bool pubtruncate;
} PublicationInfo;
/*
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 1bea6ae81d..c14664ddaf 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -4401,7 +4401,7 @@
create_order => 50,
create_sql => 'CREATE PUBLICATION pub1;',
regexp => qr/^
- \QCREATE PUBLICATION pub1 WITH (publish = 'insert, update, delete');\E
+ \QCREATE PUBLICATION pub1 WITH (publish = 'insert, update, delete, truncate');\E
/xm,
like => {
binary_upgrade => 1,
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 0c3be1f504..75a1e42cee 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -5187,7 +5187,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -5207,13 +5207,17 @@ listPublications(const char *pattern)
" puballtables AS \"%s\",\n"
" pubinsert AS \"%s\",\n"
" pubupdate AS \"%s\",\n"
- " pubdelete AS \"%s\"\n",
+ " pubdelete AS \"%s\"",
gettext_noop("Name"),
gettext_noop("Owner"),
gettext_noop("All tables"),
gettext_noop("Inserts"),
gettext_noop("Updates"),
gettext_noop("Deletes"));
+ if (pset.sversion >= 110000)
+ appendPQExpBuffer(&buf,
+ ",\n pubtruncate AS \"%s\"",
+ gettext_noop("Truncates"));
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -5254,6 +5258,7 @@ describePublications(const char *pattern)
PQExpBufferData buf;
int i;
PGresult *res;
+ bool has_pubtruncate;
if (pset.sversion < 100000)
{
@@ -5265,13 +5270,19 @@ describePublications(const char *pattern)
return true;
}
+ has_pubtruncate = (pset.sversion >= 110000);
+
initPQExpBuffer(&buf);
printfPQExpBuffer(&buf,
"SELECT oid, pubname,\n"
" pg_catalog.pg_get_userbyid(pubowner) AS owner,\n"
- " puballtables, pubinsert, pubupdate, pubdelete\n"
- "FROM pg_catalog.pg_publication\n");
+ " puballtables, pubinsert, pubupdate, pubdelete");
+ if (has_pubtruncate)
+ appendPQExpBuffer(&buf,
+ ", pubtruncate");
+ appendPQExpBuffer(&buf,
+ "\nFROM pg_catalog.pg_publication\n");
processSQLNamePattern(pset.db, &buf, pattern, false, false,
NULL, "pubname", NULL,
@@ -5317,6 +5328,9 @@ describePublications(const char *pattern)
printTableOpt myopt = pset.popt.topt;
printTableContent cont;
+ if (has_pubtruncate)
+ ncols++;
+
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
printTableInit(&cont, &myopt, title.data, ncols, nrows);
@@ -5326,12 +5340,16 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Inserts"), true, align);
printTableAddHeader(&cont, gettext_noop("Updates"), true, align);
printTableAddHeader(&cont, gettext_noop("Deletes"), true, align);
+ if (has_pubtruncate)
+ printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 4), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 5), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 6), false, false);
+ if (has_pubtruncate)
+ printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (!puballtables)
{
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index 37e77b8be7..b643c489cd 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -49,6 +49,9 @@ CATALOG(pg_publication,6104)
/* true if deletes are published */
bool pubdelete;
+ /* true if truncates are published */
+ bool pubtruncate;
+
} FormData_pg_publication;
/* ----------------
@@ -63,19 +66,21 @@ typedef FormData_pg_publication *Form_pg_publication;
* ----------------
*/
-#define Natts_pg_publication 6
+#define Natts_pg_publication 7
#define Anum_pg_publication_pubname 1
#define Anum_pg_publication_pubowner 2
#define Anum_pg_publication_puballtables 3
#define Anum_pg_publication_pubinsert 4
#define Anum_pg_publication_pubupdate 5
#define Anum_pg_publication_pubdelete 6
+#define Anum_pg_publication_pubtruncate 7
typedef struct PublicationActions
{
bool pubinsert;
bool pubupdate;
bool pubdelete;
+ bool pubtruncate;
} PublicationActions;
typedef struct Publication
diff --git a/src/include/replication/logicalproto.h b/src/include/replication/logicalproto.h
index 116f16f42d..f635ed82c1 100644
--- a/src/include/replication/logicalproto.h
+++ b/src/include/replication/logicalproto.h
@@ -97,6 +97,10 @@ extern void logicalrep_write_delete(StringInfo out, Relation rel,
HeapTuple oldtuple);
extern LogicalRepRelId logicalrep_read_delete(StringInfo in,
LogicalRepTupleData *oldtup);
+extern void logicalrep_write_truncate(StringInfo out, Relation rel,
+ bool cascade, bool restart_seqs);
+extern LogicalRepRelId logicalrep_read_truncate(StringInfo in,
+ bool *cascade, bool *restart_seqs);
extern void logicalrep_write_rel(StringInfo out, Relation rel);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
extern void logicalrep_write_typ(StringInfo out, Oid typoid);
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 0c86c647bc..afbbdd543d 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -21,20 +21,20 @@ ERROR: unrecognized publication parameter: foo
CREATE PUBLICATION testpub_xxx WITH (publish = 'cluster, vacuum');
ERROR: unrecognized "publish" value: "cluster"
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes
---------------------+--------------------------+------------+---------+---------+---------
- testpib_ins_trunct | regress_publication_user | f | t | f | f
- testpub_default | regress_publication_user | f | f | t | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates
+--------------------+--------------------------+------------+---------+---------+---------+-----------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes
---------------------+--------------------------+------------+---------+---------+---------
- testpib_ins_trunct | regress_publication_user | f | t | f | f
- testpub_default | regress_publication_user | f | t | t | t
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates
+--------------------+--------------------------+------------+---------+---------+---------+-----------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f
(2 rows)
--- adding tables
@@ -76,10 +76,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes
---------------------------+------------+---------+---------+---------
- regress_publication_user | t | t | t | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates
+--------------------------+------------+---------+---------+---------+-----------
+ regress_publication_user | t | t | t | f | f
(1 row)
DROP TABLE testpub_tbl2;
@@ -89,19 +89,19 @@ CREATE TABLE testpub_tbl3a (b text) INHERITS (testpub_tbl3);
CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes
---------------------------+------------+---------+---------+---------
- regress_publication_user | f | t | t | t
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates
+--------------------------+------------+---------+---------+---------+-----------
+ regress_publication_user | f | t | t | t | t
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes
---------------------------+------------+---------+---------+---------
- regress_publication_user | f | t | t | t
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates
+--------------------------+------------+---------+---------+---------+-----------
+ regress_publication_user | f | t | t | t | t
Tables:
"public.testpub_tbl3"
@@ -119,10 +119,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes
---------------------------+------------+---------+---------+---------
- regress_publication_user | f | t | t | t
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates
+--------------------------+------------+---------+---------+---------+-----------
+ regress_publication_user | f | t | t | t | t
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -165,10 +165,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes
---------------------------+------------+---------+---------+---------
- regress_publication_user | f | t | t | t
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates
+--------------------------+------------+---------+---------+---------+-----------
+ regress_publication_user | f | t | t | t | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -210,10 +210,10 @@ DROP TABLE testpub_parted;
DROP VIEW testpub_view;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes
---------------------------+------------+---------+---------+---------
- regress_publication_user | f | t | t | t
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates
+--------------------------+------------+---------+---------+---------+-----------
+ regress_publication_user | f | t | t | t | f
(1 row)
-- fail - must be owner of publication
@@ -223,20 +223,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes
--------------+--------------------------+------------+---------+---------+---------
- testpub_foo | regress_publication_user | f | t | t | t
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates
+-------------+--------------------------+------------+---------+---------+---------+-----------
+ testpub_foo | regress_publication_user | f | t | t | t | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes
------------------+---------------------------+------------+---------+---------+---------
- testpub_default | regress_publication_user2 | f | t | t | t
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates
+-----------------+---------------------------+------------+---------+---------+---------+-----------
+ testpub_default | regress_publication_user2 | f | t | t | t | f
(1 row)
DROP PUBLICATION testpub_default;
diff --git a/src/test/subscription/t/010_truncate.pl b/src/test/subscription/t/010_truncate.pl
new file mode 100644
index 0000000000..864135f007
--- /dev/null
+++ b/src/test/subscription/t/010_truncate.pl
@@ -0,0 +1,106 @@
+# Test TRUNCATE
+use strict;
+use warnings;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 5;
+
+# setup
+
+my $node_publisher = get_new_node('publisher');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->start;
+
+my $node_subscriber = get_new_node('subscriber');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->start;
+
+my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+
+$node_publisher->safe_psql('postgres',
+ "CREATE TABLE tab1 (a int PRIMARY KEY)");
+
+$node_subscriber->safe_psql('postgres',
+ "CREATE TABLE tab1 (a int PRIMARY KEY)");
+
+$node_publisher->safe_psql('postgres',
+ "CREATE TABLE tab2 (a int PRIMARY KEY)");
+
+$node_subscriber->safe_psql('postgres',
+ "CREATE TABLE tab2 (a int PRIMARY KEY)");
+
+$node_subscriber->safe_psql('postgres',
+ "CREATE SEQUENCE seq1 OWNED BY tab1.a"
+);
+$node_subscriber->safe_psql('postgres',
+ "ALTER SEQUENCE seq1 START 101"
+);
+
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION pub1 FOR TABLE tab1");
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION pub2 FOR TABLE tab2 WITH (publish = insert)");
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr application_name=sub1' PUBLICATION pub1");
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION sub2 CONNECTION '$publisher_connstr application_name=sub2' PUBLICATION pub2");
+
+$node_publisher->wait_for_catchup('sub1');
+
+# insert data to truncate
+
+$node_subscriber->safe_psql('postgres', "INSERT INTO tab1 VALUES (1), (2), (3)");
+
+$node_publisher->wait_for_catchup('sub1');
+
+# truncate and check
+
+$node_publisher->safe_psql('postgres', "TRUNCATE tab1");
+
+$node_publisher->wait_for_catchup('sub1');
+
+my $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab1");
+is($result, qq(0||),
+ 'truncate replicated');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq1')");
+is($result, qq(1),
+ 'sequence not restarted');
+
+# truncate with restart identity
+
+$node_publisher->safe_psql('postgres', "TRUNCATE tab1 RESTART IDENTITY");
+
+$node_publisher->wait_for_catchup('sub1');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq1')");
+is($result, qq(101),
+ 'truncate restarted identities');
+
+# test publication that does not replicate truncate
+
+$node_subscriber->safe_psql('postgres', "INSERT INTO tab2 VALUES (1), (2), (3)");
+
+$node_publisher->safe_psql('postgres', "TRUNCATE tab2");
+
+$node_publisher->wait_for_catchup('sub2');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab2");
+is($result, qq(3|1|3),
+ 'truncate not replicated');
+
+$node_publisher->safe_psql('postgres',
+ "ALTER PUBLICATION pub2 SET (publish = 'insert, truncate')");
+
+$node_publisher->safe_psql('postgres', "TRUNCATE tab2");
+
+$node_publisher->wait_for_catchup('sub2');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab2");
+is($result, qq(0||),
+ 'truncate replicated after publication change');
--
2.17.0
Re: [PATCH] Logical decoding of TRUNCATE
On 4 April 2018 at 03:31, Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:
I wonder why this approach was chosen.
I looked at coding it that way and it seemed worse than the way chosen.
I'm going to try to hack up an alternative approach and see how it works
out.
If you add a new filter for TRUNCATE it will mean compatibility
problems and the need for pg_dump support.
Need note in release notes to explain that people will need to add
TRUNCATE filter to their existing publications. I was hoping to have
that picked up automatically, which can't happen if we have an
explicit filter for it.
I know this has been discussed in the thread already, but it really
strikes me as wrong to basically do some mini DDL replication feature
via per-command WAL records.
Don't really understand that comment.
I think TRUNCATE is special in some ways and it's reasonable to treat it
specially. Real DDL is being worked on in the 2PC decoding thread.
What we are discussing here isn't going to be applicable there and vice
versa, I think. In fact, one of the reasons for this effort is that in
BDR TRUNCATE is currently handled like a DDL command, which doesn't work
well.
Agreed
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [PATCH] Logical decoding of TRUNCATE
On 4/3/18 22:31, Peter Eisentraut wrote:
The problem I see now is that when we WAL-log a truncate, we include all
the relids in one record. That seems useful. But during decoding we
split this into one change per relid. So at the receiving end, truncate
can only process one relation at a time, which will fail if there are
foreign keys involved. The solution that had been proposed here was to
ignore foreign keys on the subscriber, which is clearly problematic.
I'm going to try to hack up an alternative approach and see how it works
out.
Done here. I added a separate callback for decoding truncates, which
receives all the relations at once. That information can then be
shipped off together and applied together on the receiving side. So
foreign keys are not a problem anymore. This ended up being a bit
larger than the original patch, but I think it's sound behavior and
future-proof.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
v19-0001-Logical-decoding-of-TRUNCATE.patchtext/plain; charset=UTF-8; name=v19-0001-Logical-decoding-of-TRUNCATE.patch; x-mac-creator=0; x-mac-type=0Download
From df101e32c358ac9243285d4e8f125803988e5508 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <peter_e@gmx.net>
Date: Thu, 5 Apr 2018 11:46:41 -0400
Subject: [PATCH v19 1/2] Logical decoding of TRUNCATE
---
contrib/test_decoding/Makefile | 2 +-
contrib/test_decoding/expected/truncate.out | 25 ++++++
contrib/test_decoding/sql/truncate.sql | 10 +++
contrib/test_decoding/test_decoding.c | 61 +++++++++++++
doc/src/sgml/logicaldecoding.sgml | 27 +++++-
src/backend/access/heap/heapam.c | 7 ++
src/backend/access/rmgrdesc/heapdesc.c | 14 +++
src/backend/commands/tablecmds.c | 87 +++++++++++++++++--
src/backend/replication/logical/decode.c | 41 +++++++++
src/backend/replication/logical/logical.c | 43 +++++++++
.../replication/logical/reorderbuffer.c | 35 ++++++++
src/include/access/heapam_xlog.h | 23 ++++-
src/include/commands/tablecmds.h | 2 +
src/include/replication/output_plugin.h | 10 +++
src/include/replication/reorderbuffer.h | 24 ++++-
15 files changed, 398 insertions(+), 13 deletions(-)
create mode 100644 contrib/test_decoding/expected/truncate.out
create mode 100644 contrib/test_decoding/sql/truncate.sql
diff --git a/contrib/test_decoding/Makefile b/contrib/test_decoding/Makefile
index 6c18189d9d..1d601d8144 100644
--- a/contrib/test_decoding/Makefile
+++ b/contrib/test_decoding/Makefile
@@ -39,7 +39,7 @@ submake-test_decoding:
REGRESSCHECKS=ddl xact rewrite toast permissions decoding_in_xact \
decoding_into_rel binary prepared replorigin time messages \
- spill slot
+ spill slot truncate
regresscheck: | submake-regress submake-test_decoding temp-install
$(pg_regress_check) \
diff --git a/contrib/test_decoding/expected/truncate.out b/contrib/test_decoding/expected/truncate.out
new file mode 100644
index 0000000000..be85178206
--- /dev/null
+++ b/contrib/test_decoding/expected/truncate.out
@@ -0,0 +1,25 @@
+SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_decoding');
+ ?column?
+----------
+ init
+(1 row)
+
+CREATE TABLE tab1 (id serial unique, data int);
+CREATE TABLE tab2 (a int primary key, b int);
+TRUNCATE tab1;
+TRUNCATE tab1, tab1 RESTART IDENTITY CASCADE;
+TRUNCATE tab1, tab2;
+SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
+ data
+------------------------------------------------------
+ BEGIN
+ table public.tab1: TRUNCATE: (no-flags)
+ COMMIT
+ BEGIN
+ table public.tab1: TRUNCATE: restart_seqs cascade
+ COMMIT
+ BEGIN
+ table public.tab1, public.tab2: TRUNCATE: (no-flags)
+ COMMIT
+(9 rows)
+
diff --git a/contrib/test_decoding/sql/truncate.sql b/contrib/test_decoding/sql/truncate.sql
new file mode 100644
index 0000000000..88f113fd5b
--- /dev/null
+++ b/contrib/test_decoding/sql/truncate.sql
@@ -0,0 +1,10 @@
+SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_decoding');
+
+CREATE TABLE tab1 (id serial unique, data int);
+CREATE TABLE tab2 (a int primary key, b int);
+
+TRUNCATE tab1;
+TRUNCATE tab1, tab1 RESTART IDENTITY CASCADE;
+TRUNCATE tab1, tab2;
+
+SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
diff --git a/contrib/test_decoding/test_decoding.c b/contrib/test_decoding/test_decoding.c
index a94aeeae29..c238f12e66 100644
--- a/contrib/test_decoding/test_decoding.c
+++ b/contrib/test_decoding/test_decoding.c
@@ -52,6 +52,10 @@ static void pg_decode_commit_txn(LogicalDecodingContext *ctx,
static void pg_decode_change(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, Relation rel,
ReorderBufferChange *change);
+static void pg_decode_truncate(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ int nrelations, Relation relations[],
+ ReorderBufferChange *change);
static bool pg_decode_filter(LogicalDecodingContext *ctx,
RepOriginId origin_id);
static void pg_decode_message(LogicalDecodingContext *ctx,
@@ -74,6 +78,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->startup_cb = pg_decode_startup;
cb->begin_cb = pg_decode_begin_txn;
cb->change_cb = pg_decode_change;
+ cb->truncate_cb = pg_decode_truncate;
cb->commit_cb = pg_decode_commit_txn;
cb->filter_by_origin_cb = pg_decode_filter;
cb->shutdown_cb = pg_decode_shutdown;
@@ -480,6 +485,62 @@ pg_decode_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
OutputPluginWrite(ctx, true);
}
+static void
+pg_decode_truncate(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
+ int nrelations, Relation relations[], ReorderBufferChange *change)
+{
+ TestDecodingData *data;
+ MemoryContext old;
+ int i;
+
+ data = ctx->output_plugin_private;
+
+ /* output BEGIN if we haven't yet */
+ if (data->skip_empty_xacts && !data->xact_wrote_changes)
+ {
+ pg_output_begin(ctx, data, txn, false);
+ }
+ data->xact_wrote_changes = true;
+
+ /* Avoid leaking memory by using and resetting our own context */
+ old = MemoryContextSwitchTo(data->context);
+
+ OutputPluginPrepareWrite(ctx, true);
+
+ appendStringInfoString(ctx->out, "table ");
+
+ for (i = 0; i < nrelations; i++)
+ {
+ Oid relid = RelationGetRelid(relations[i]);
+
+ if (i > 0)
+ appendStringInfoString(ctx->out, ", ");
+
+ appendStringInfoString(ctx->out,
+ quote_qualified_identifier(
+ get_namespace_name(get_rel_namespace(relid)),
+ get_rel_name(relid)));
+ }
+
+ appendStringInfoString(ctx->out, ": TRUNCATE:");
+
+ if (change->data.truncate_msg.restart_seqs
+ || change->data.truncate_msg.cascade)
+ {
+ if (change->data.truncate_msg.restart_seqs)
+ appendStringInfo(ctx->out, " restart_seqs");
+ if (change->data.truncate_msg.cascade)
+ appendStringInfo(ctx->out, " cascade");
+ }
+ else
+ appendStringInfoString(ctx->out, " (no-flags)");
+
+ MemoryContextSwitchTo(old);
+ MemoryContextReset(data->context);
+
+ OutputPluginWrite(ctx, true);
+}
+
static void
pg_decode_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr lsn, bool transactional,
diff --git a/doc/src/sgml/logicaldecoding.sgml b/doc/src/sgml/logicaldecoding.sgml
index f6b14dccb0..b29cfe6fb4 100644
--- a/doc/src/sgml/logicaldecoding.sgml
+++ b/doc/src/sgml/logicaldecoding.sgml
@@ -383,6 +383,7 @@ <title>Initialization Function</title>
LogicalDecodeStartupCB startup_cb;
LogicalDecodeBeginCB begin_cb;
LogicalDecodeChangeCB change_cb;
+ LogicalDecodeTruncateCB truncate_cb;
LogicalDecodeCommitCB commit_cb;
LogicalDecodeMessageCB message_cb;
LogicalDecodeFilterByOriginCB filter_by_origin_cb;
@@ -394,8 +395,10 @@ <title>Initialization Function</title>
The <function>begin_cb</function>, <function>change_cb</function>
and <function>commit_cb</function> callbacks are required,
while <function>startup_cb</function>,
- <function>filter_by_origin_cb</function>
+ <function>filter_by_origin_cb</function>, <function>truncate_cb</function>,
and <function>shutdown_cb</function> are optional.
+ If <function>truncate_cb</function> is not set but a
+ <command>TRUNCATE</command> is to be decoded, the action will be ignored.
</para>
</sect2>
@@ -590,6 +593,28 @@ <title>Change Callback</title>
</note>
</sect3>
+ <sect3 id="logicaldecoding-output-plugin-truncate">
+ <title>Truncate Callback</title>
+
+ <para>
+ The <function>truncate_cb</function> callback is called for a
+ <command>TRUNCATE</command> command.
+<programlisting>
+typedef void (*LogicalDecodeTruncateCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ int nrelations,
+ Relation relations[],
+ ReorderBufferChange *change);
+</programlisting>
+ The parameters are analogous to the <function>change_cb</function>
+ callback. However, because <command>TRUNCATE</command> actions on
+ tables connected by foreign keys need to be executed together, this
+ callback receives an array of relations instead of just a single one.
+ See the description of the <xref linkend="sql-truncate"/> statement for
+ details.
+ </para>
+ </sect3>
+
<sect3 id="logicaldecoding-output-plugin-filter-origin">
<title>Origin Filter Callback</title>
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index f96567f5d5..0bafb4fefc 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -9260,6 +9260,13 @@ heap_redo(XLogReaderState *record)
case XLOG_HEAP_UPDATE:
heap_xlog_update(record, false);
break;
+ case XLOG_HEAP_TRUNCATE:
+ /*
+ * TRUNCATE is a no-op because the actions are already logged as
+ * SMGR WAL records. TRUNCATE WAL record only exists for logical
+ * decoding.
+ */
+ break;
case XLOG_HEAP_HOT_UPDATE:
heap_xlog_update(record, true);
break;
diff --git a/src/backend/access/rmgrdesc/heapdesc.c b/src/backend/access/rmgrdesc/heapdesc.c
index b00c071cb6..349feb6510 100644
--- a/src/backend/access/rmgrdesc/heapdesc.c
+++ b/src/backend/access/rmgrdesc/heapdesc.c
@@ -75,6 +75,17 @@ heap_desc(StringInfo buf, XLogReaderState *record)
xlrec->new_offnum,
xlrec->new_xmax);
}
+ else if (info == XLOG_HEAP_TRUNCATE)
+ {
+ xl_heap_truncate *xlrec = (xl_heap_truncate *) rec;
+
+ if (xlrec->flags & XLH_TRUNCATE_CASCADE)
+ appendStringInfo(buf, "cascade ");
+ if (xlrec->flags & XLH_TRUNCATE_RESTART_SEQS)
+ appendStringInfo(buf, "restart_seqs ");
+ appendStringInfo(buf, "nrelids %u", xlrec->nrelids);
+ /* skip the list of relids */
+ }
else if (info == XLOG_HEAP_CONFIRM)
{
xl_heap_confirm *xlrec = (xl_heap_confirm *) rec;
@@ -186,6 +197,9 @@ heap_identify(uint8 info)
case XLOG_HEAP_HOT_UPDATE | XLOG_HEAP_INIT_PAGE:
id = "HOT_UPDATE+INIT";
break;
+ case XLOG_HEAP_TRUNCATE:
+ id = "TRUNCATE";
+ break;
case XLOG_HEAP_CONFIRM:
id = "HEAP_CONFIRM";
break;
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index ec2f9616ed..077b954f36 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -16,6 +16,7 @@
#include "access/genam.h"
#include "access/heapam.h"
+#include "access/heapam_xlog.h"
#include "access/multixact.h"
#include "access/reloptions.h"
#include "access/relscan.h"
@@ -1322,11 +1323,7 @@ ExecuteTruncate(TruncateStmt *stmt)
{
List *rels = NIL;
List *relids = NIL;
- List *seq_relids = NIL;
- EState *estate;
- ResultRelInfo *resultRelInfos;
- ResultRelInfo *resultRelInfo;
- SubTransactionId mySubid;
+ List *relids_logged = NIL;
ListCell *cell;
/*
@@ -1350,6 +1347,9 @@ ExecuteTruncate(TruncateStmt *stmt)
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, myrelid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, myrelid);
if (recurse)
{
@@ -1370,6 +1370,9 @@ ExecuteTruncate(TruncateStmt *stmt)
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, childrelid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, childrelid);
}
}
else if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
@@ -1379,7 +1382,34 @@ ExecuteTruncate(TruncateStmt *stmt)
errhint("Do not specify the ONLY keyword, or use TRUNCATE ONLY on the partitions directly.")));
}
+ ExecuteTruncateGuts(rels, relids, relids_logged,
+ stmt->behavior, stmt->restart_seqs);
+
+ /* And close the rels */
+ foreach(cell, rels)
+ {
+ Relation rel = (Relation) lfirst(cell);
+
+ heap_close(rel, NoLock);
+ }
+}
+
+void
+ExecuteTruncateGuts(List *explicit_rels, List *relids, List *relids_logged,
+ DropBehavior behavior, bool restart_seqs)
+{
+ List *rels;
+ List *seq_relids = NIL;
+ EState *estate;
+ ResultRelInfo *resultRelInfos;
+ ResultRelInfo *resultRelInfo;
+ SubTransactionId mySubid;
+ ListCell *cell;
+ Oid *logrelids;
+
/*
+ * Open, exclusive-lock, and check all the explicitly-specified relations
+ *
* In CASCADE mode, suck in all referencing relations as well. This
* requires multiple iterations to find indirectly-dependent relations. At
* each phase, we need to exclusive-lock new rels before looking for their
@@ -1387,7 +1417,8 @@ ExecuteTruncate(TruncateStmt *stmt)
* soon as we open it, to avoid a faux pas such as holding lock for a long
* time on a rel we have no permissions for.
*/
- if (stmt->behavior == DROP_CASCADE)
+ rels = list_copy(explicit_rels);
+ if (behavior == DROP_CASCADE)
{
for (;;)
{
@@ -1409,6 +1440,9 @@ ExecuteTruncate(TruncateStmt *stmt)
truncate_check_rel(rel);
rels = lappend(rels, rel);
relids = lappend_oid(relids, relid);
+ /* Log this relation only if needed for logical decoding */
+ if (RelationIsLogicallyLogged(rel))
+ relids_logged = lappend_oid(relids_logged, relid);
}
}
}
@@ -1421,7 +1455,7 @@ ExecuteTruncate(TruncateStmt *stmt)
#ifdef USE_ASSERT_CHECKING
heap_truncate_check_FKs(rels, false);
#else
- if (stmt->behavior == DROP_RESTRICT)
+ if (behavior == DROP_RESTRICT)
heap_truncate_check_FKs(rels, false);
#endif
@@ -1431,7 +1465,7 @@ ExecuteTruncate(TruncateStmt *stmt)
* We want to do this early since it's pointless to do all the truncation
* work only to fail on sequence permissions.
*/
- if (stmt->restart_seqs)
+ if (restart_seqs)
{
foreach(cell, rels)
{
@@ -1586,6 +1620,38 @@ ExecuteTruncate(TruncateStmt *stmt)
ResetSequence(seq_relid);
}
+ /*
+ * Write a WAL record to allow this set of actions to be logically decoded.
+ *
+ * Assemble an array of relids so we can write a single WAL record for the
+ * whole action.
+ */
+ if (list_length(relids_logged) > 0)
+ {
+ xl_heap_truncate xlrec;
+ int i = 0;
+
+ logrelids = palloc(list_length(relids_logged) * sizeof(Oid));
+ foreach (cell, relids_logged)
+ logrelids[i++] = lfirst_oid(cell);
+
+ xlrec.dbId = MyDatabaseId;
+ xlrec.nrelids = list_length(relids_logged);
+ xlrec.flags = 0;
+ if (behavior == DROP_CASCADE)
+ xlrec.flags |= XLH_TRUNCATE_CASCADE;
+ if (restart_seqs)
+ xlrec.flags |= XLH_TRUNCATE_RESTART_SEQS;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfHeapTruncate);
+ XLogRegisterData((char *) logrelids, list_length(relids_logged) * sizeof(Oid));
+
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ (void) XLogInsert(RM_HEAP_ID, XLOG_HEAP_TRUNCATE);
+ }
+
/*
* Process all AFTER STATEMENT TRUNCATE triggers.
*/
@@ -1603,7 +1669,10 @@ ExecuteTruncate(TruncateStmt *stmt)
/* We can clean up the EState now */
FreeExecutorState(estate);
- /* And close the rels (can't do this while EState still holds refs) */
+ /* And close the eventual rels opened by CASCADE
+ * (can't do this while EState still holds refs)
+ */
+ rels = list_difference_ptr(rels, explicit_rels);
foreach(cell, rels)
{
Relation rel = (Relation) lfirst(cell);
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index 6eb0d5527e..c94ac29d83 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -65,6 +65,7 @@ static void DecodeLogicalMsgOp(LogicalDecodingContext *ctx, XLogRecordBuffer *bu
static void DecodeInsert(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeUpdate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeDelete(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
+static void DecodeTruncate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeMultiInsert(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
static void DecodeSpecConfirm(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
@@ -449,6 +450,11 @@ DecodeHeapOp(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
DecodeDelete(ctx, buf);
break;
+ case XLOG_HEAP_TRUNCATE:
+ if (SnapBuildProcessChange(builder, xid, buf->origptr))
+ DecodeTruncate(ctx, buf);
+ break;
+
case XLOG_HEAP_INPLACE:
/*
@@ -825,6 +831,41 @@ DecodeDelete(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
ReorderBufferQueueChange(ctx->reorder, XLogRecGetXid(r), buf->origptr, change);
}
+/*
+ * Parse XLOG_HEAP_TRUNCATE from wal
+ */
+static void
+DecodeTruncate(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+{
+ XLogReaderState *r = buf->record;
+ xl_heap_truncate *xlrec;
+ ReorderBufferChange *change;
+
+ xlrec = (xl_heap_truncate *) XLogRecGetData(r);
+
+ /* only interested in our database */
+ if (xlrec->dbId != ctx->slot->data.database)
+ return;
+
+ /* output plugin doesn't look for this origin, no need to queue */
+ if (FilterByOrigin(ctx, XLogRecGetOrigin(r)))
+ return;
+
+ change = ReorderBufferGetChange(ctx->reorder);
+ change->action = REORDER_BUFFER_CHANGE_TRUNCATE;
+ change->origin_id = XLogRecGetOrigin(r);
+ if (xlrec->flags & XLH_TRUNCATE_CASCADE)
+ change->data.truncate_msg.cascade = true;
+ if (xlrec->flags & XLH_TRUNCATE_RESTART_SEQS)
+ change->data.truncate_msg.restart_seqs = true;
+ change->data.truncate_msg.nrelids = xlrec->nrelids;
+ change->data.truncate_msg.relids = palloc(xlrec->nrelids * sizeof(Oid));
+ memcpy(change->data.truncate_msg.relids, xlrec->relids,
+ xlrec->nrelids * sizeof(Oid));
+ ReorderBufferQueueChange(ctx->reorder, XLogRecGetXid(r),
+ buf->origptr, change);
+}
+
/*
* Decode XLOG_HEAP2_MULTI_INSERT_insert record into multiple tuplebufs.
*
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index 3d8ad7ddf8..0737c7b1e7 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -62,6 +62,8 @@ static void commit_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr commit_lsn);
static void change_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
Relation relation, ReorderBufferChange *change);
+static void truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ int nrelations, Relation relations[], ReorderBufferChange *change);
static void message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
@@ -183,6 +185,7 @@ StartupDecodingContext(List *output_plugin_options,
/* wrap output plugin callbacks, so we can add error context information */
ctx->reorder->begin = begin_cb_wrapper;
ctx->reorder->apply_change = change_cb_wrapper;
+ ctx->reorder->apply_truncate = truncate_cb_wrapper;
ctx->reorder->commit = commit_cb_wrapper;
ctx->reorder->message = message_cb_wrapper;
@@ -734,6 +737,46 @@ change_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
error_context_stack = errcallback.previous;
}
+static void
+truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ int nrelations, Relation relations[], ReorderBufferChange *change)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ if (!ctx->callbacks.truncate_cb)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "truncate";
+ state.report_location = change->lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn->xid;
+
+ /*
+ * report this change's lsn so replies from clients can give an up2date
+ * answer. This won't ever be enough (and shouldn't be!) to confirm
+ * receipt of this transaction, but it might allow another transaction's
+ * commit to be confirmed with one message.
+ */
+ ctx->write_location = change->lsn;
+
+ ctx->callbacks.truncate_cb(ctx, txn, nrelations, relations, change);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
bool
filter_by_origin_cb_wrapper(LogicalDecodingContext *ctx, RepOriginId origin_id)
{
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index b4016ed52b..7691c1a8b4 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -415,6 +415,7 @@ ReorderBufferReturnChange(ReorderBuffer *rb, ReorderBufferChange *change)
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
break;
}
@@ -1491,6 +1492,38 @@ ReorderBufferCommit(ReorderBuffer *rb, TransactionId xid,
specinsert = change;
break;
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
+ {
+ int i;
+ int nrelids = change->data.truncate_msg.nrelids;
+ int nrelations = 0;
+ Relation *relations;
+
+ relations = palloc0(nrelids * sizeof(Relation));
+ for (i = 0; i < nrelids; i++)
+ {
+ Oid relid = change->data.truncate_msg.relids[i];
+ Relation relation;
+
+ relation = RelationIdGetRelation(relid);
+
+ if (relation == NULL)
+ elog(ERROR, "could not open relation with OID %u", relid);
+
+ if (!RelationIsLogicallyLogged(relation))
+ continue;
+
+ relations[nrelations++] = relation;
+ }
+
+ rb->apply_truncate(rb, txn, nrelations, relations, change);
+
+ for (i = 0; i < nrelations; i++)
+ RelationClose(relations[i]);
+
+ break;
+ }
+
case REORDER_BUFFER_CHANGE_MESSAGE:
rb->message(rb, txn, change->lsn, true,
change->data.msg.prefix,
@@ -2255,6 +2288,7 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
}
break;
}
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
@@ -2534,6 +2568,7 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
/* the base struct contains all the data, easy peasy */
+ case REORDER_BUFFER_CHANGE_TRUNCATE:
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
case REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID:
case REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID:
diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h
index 700e25c36a..0052e4c569 100644
--- a/src/include/access/heapam_xlog.h
+++ b/src/include/access/heapam_xlog.h
@@ -32,7 +32,7 @@
#define XLOG_HEAP_INSERT 0x00
#define XLOG_HEAP_DELETE 0x10
#define XLOG_HEAP_UPDATE 0x20
-/* 0x030 is free, was XLOG_HEAP_MOVE */
+#define XLOG_HEAP_TRUNCATE 0x30
#define XLOG_HEAP_HOT_UPDATE 0x40
#define XLOG_HEAP_CONFIRM 0x50
#define XLOG_HEAP_LOCK 0x60
@@ -109,6 +109,27 @@ typedef struct xl_heap_delete
#define SizeOfHeapDelete (offsetof(xl_heap_delete, flags) + sizeof(uint8))
+/*
+ * xl_heap_delete flag values, 8 bits are available.
+ */
+#define XLH_TRUNCATE_CASCADE (1<<0)
+#define XLH_TRUNCATE_RESTART_SEQS (1<<1)
+
+/*
+ * For truncate we list all truncated relids in an array, followed by all
+ * sequence relids that need to be restarted, if any.
+ * All rels are always within the same database, so we just list dbid once.
+ */
+typedef struct xl_heap_truncate
+{
+ Oid dbId;
+ uint32 nrelids;
+ uint8 flags;
+ Oid relids[FLEXIBLE_ARRAY_MEMBER];
+} xl_heap_truncate;
+
+#define SizeOfHeapTruncate (offsetof(xl_heap_truncate, relids))
+
/*
* We don't store the whole fixed part (HeapTupleHeaderData) of an inserted
* or updated tuple in WAL; we can save a few bytes by reconstructing the
diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h
index 04a961d383..70ee3da76b 100644
--- a/src/include/commands/tablecmds.h
+++ b/src/include/commands/tablecmds.h
@@ -54,6 +54,8 @@ extern void AlterRelationNamespaceInternal(Relation classRel, Oid relOid,
extern void CheckTableNotInUse(Relation rel, const char *stmt);
extern void ExecuteTruncate(TruncateStmt *stmt);
+extern void ExecuteTruncateGuts(List *explicit_rels, List *relids, List *relids_logged,
+ DropBehavior behavior, bool restart_seqs);
extern void SetRelationHasSubclass(Oid relationId, bool relhassubclass);
diff --git a/src/include/replication/output_plugin.h b/src/include/replication/output_plugin.h
index 82875d6b3d..1ee0a56f03 100644
--- a/src/include/replication/output_plugin.h
+++ b/src/include/replication/output_plugin.h
@@ -61,6 +61,15 @@ typedef void (*LogicalDecodeChangeCB) (struct LogicalDecodingContext *ctx,
Relation relation,
ReorderBufferChange *change);
+/*
+ * Callback for every TRUNCATE in a successful transaction.
+ */
+typedef void (*LogicalDecodeTruncateCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ int nrelations,
+ Relation relations[],
+ ReorderBufferChange *change);
+
/*
* Called for every (explicit or implicit) COMMIT of a successful transaction.
*/
@@ -98,6 +107,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeStartupCB startup_cb;
LogicalDecodeBeginCB begin_cb;
LogicalDecodeChangeCB change_cb;
+ LogicalDecodeTruncateCB truncate_cb;
LogicalDecodeCommitCB commit_cb;
LogicalDecodeMessageCB message_cb;
LogicalDecodeFilterByOriginCB filter_by_origin_cb;
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index aa430c843c..13a8798115 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -59,7 +59,8 @@ enum ReorderBufferChangeType
REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID,
REORDER_BUFFER_CHANGE_INTERNAL_TUPLECID,
REORDER_BUFFER_CHANGE_INTERNAL_SPEC_INSERT,
- REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM
+ REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM,
+ REORDER_BUFFER_CHANGE_TRUNCATE
};
/*
@@ -128,6 +129,18 @@ typedef struct ReorderBufferChange
CommandId cmax;
CommandId combocid;
} tuplecid;
+
+ /*
+ * Truncate data for REORDER_BUFFER_CHANGE_TRUNCATE representing
+ * one set of relations to be truncated.
+ */
+ struct
+ {
+ Size nrelids;
+ bool cascade;
+ bool restart_seqs;
+ Oid *relids;
+ } truncate_msg;
} data;
/*
@@ -283,6 +296,14 @@ typedef void (*ReorderBufferApplyChangeCB) (
Relation relation,
ReorderBufferChange *change);
+/* truncate callback signature */
+typedef void (*ReorderBufferApplyTruncateCB) (
+ ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ int nrelations,
+ Relation relations[],
+ ReorderBufferChange *change);
+
/* begin callback signature */
typedef void (*ReorderBufferBeginCB) (
ReorderBuffer *rb,
@@ -328,6 +349,7 @@ struct ReorderBuffer
*/
ReorderBufferBeginCB begin;
ReorderBufferApplyChangeCB apply_change;
+ ReorderBufferApplyTruncateCB apply_truncate;
ReorderBufferCommitCB commit;
ReorderBufferMessageCB message;
base-commit: 0a64b45152b593c5eb95f2e88fbce7fbfe84ae7b
--
2.17.0
v19-0002-Logical-replication-support-for-TRUNCATE.patchtext/plain; charset=UTF-8; name=v19-0002-Logical-replication-support-for-TRUNCATE.patch; x-mac-creator=0; x-mac-type=0Download
From 7ee846501f8532cc6b6ebc06b07b8555ae8a57f4 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <peter_e@gmx.net>
Date: Thu, 5 Apr 2018 11:47:06 -0400
Subject: [PATCH v19 2/2] Logical replication support for TRUNCATE
---
doc/src/sgml/catalogs.sgml | 8 +
doc/src/sgml/logical-replication.sgml | 13 +-
doc/src/sgml/protocol.sgml | 56 +++++++
doc/src/sgml/ref/create_publication.sgml | 10 +-
src/backend/catalog/pg_publication.c | 1 +
src/backend/commands/publicationcmds.c | 20 ++-
src/backend/replication/logical/proto.c | 55 +++++++
src/backend/replication/logical/worker.c | 68 +++++++++
src/backend/replication/pgoutput/pgoutput.c | 129 +++++++++++-----
src/backend/utils/cache/relcache.c | 3 +-
src/bin/pg_dump/pg_dump.c | 33 +++-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 2 +-
src/bin/psql/describe.c | 26 +++-
src/include/catalog/pg_publication.h | 7 +-
src/include/replication/logicalproto.h | 4 +
src/test/regress/expected/publication.out | 84 +++++-----
src/test/subscription/t/010_truncate.pl | 161 ++++++++++++++++++++
18 files changed, 571 insertions(+), 110 deletions(-)
create mode 100644 src/test/subscription/t/010_truncate.pl
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index d6a9d8c580..e8efa13e8d 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -5518,6 +5518,14 @@ <title><structname>pg_publication</structname> Columns</title>
<entry>If true, <command>DELETE</command> operations are replicated for
tables in the publication.</entry>
</row>
+
+ <row>
+ <entry><structfield>pubtruncate</structfield></entry>
+ <entry><type>bool</type></entry>
+ <entry></entry>
+ <entry>If true, <command>TRUNCATE</command> operations are replicated for
+ tables in the publication.</entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 75551d8ee1..151e773fc2 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -108,8 +108,8 @@ <title>Publication</title>
<para>
Publications can choose to limit the changes they produce to
- any combination of <command>INSERT</command>, <command>UPDATE</command>, and
- <command>DELETE</command>, similar to how triggers are fired by
+ any combination of <command>INSERT</command>, <command>UPDATE</command>,
+ <command>DELETE</command>, and <command>TRUNCATE</command>, similar to how triggers are fired by
particular event types. By default, all operation types are replicated.
</para>
@@ -364,15 +364,6 @@ <title>Restrictions</title>
</para>
</listitem>
- <listitem>
- <para>
- <command>TRUNCATE</command> commands are not replicated. This can, of
- course, be worked around by using <command>DELETE</command> instead. To
- avoid accidental <command>TRUNCATE</command> invocations, you can revoke
- the <literal>TRUNCATE</literal> privilege from tables.
- </para>
- </listitem>
-
<listitem>
<para>
Large objects (see <xref linkend="largeobjects"/>) are not replicated.
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index b94dd4ac65..004b36084f 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -6774,6 +6774,62 @@ <title>Logical Replication Message Formats</title>
</listitem>
</varlistentry>
+<varlistentry>
+<term>
+Truncate
+</term>
+<listitem>
+<para>
+
+<variablelist>
+<varlistentry>
+<term>
+ Byte1('T')
+</term>
+<listitem>
+<para>
+ Identifies the message as a truncate message.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+ Int32
+</term>
+<listitem>
+<para>
+ Number of relations
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+ Int8
+</term>
+<listitem>
+<para>
+ Option bits for <command>TRUNCATE</command>:
+ 1 for <literal>CASCADE</literal>, 2 for <literal>RESTART IDENTITY</literal>
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+ Int32
+</term>
+<listitem>
+<para>
+ ID of the relation corresponding to the ID in the relation
+ message. This field is repeated for each relation.
+</para>
+</listitem>
+</varlistentry>
+
+</variablelist>
+</para>
+</listitem>
+</varlistentry>
+
</variablelist>
<para>
diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml
index bfe12d5f41..99f87ca393 100644
--- a/doc/src/sgml/ref/create_publication.sgml
+++ b/doc/src/sgml/ref/create_publication.sgml
@@ -106,10 +106,11 @@ <title>Parameters</title>
This parameter determines which DML operations will be published by
the new publication to the subscribers. The value is
comma-separated list of operations. The allowed operations are
- <literal>insert</literal>, <literal>update</literal>, and
- <literal>delete</literal>. The default is to publish all actions,
+ <literal>insert</literal>, <literal>update</literal>,
+ <literal>delete</literal>, and <literal>truncate</literal>.
+ The default is to publish all actions,
and so the default value for this option is
- <literal>'insert, update, delete'</literal>.
+ <literal>'insert, update, delete, truncate'</literal>.
</para>
</listitem>
</varlistentry>
@@ -168,8 +169,7 @@ <title>Notes</title>
</para>
<para>
- <command>TRUNCATE</command> and <acronym>DDL</acronym> operations
- are not published.
+ <acronym>DDL</acronym> operations are not published.
</para>
</refsect1>
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index ba18258ebb..ec3bd1d22d 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -376,6 +376,7 @@ GetPublication(Oid pubid)
pub->pubactions.pubinsert = pubform->pubinsert;
pub->pubactions.pubupdate = pubform->pubupdate;
pub->pubactions.pubdelete = pubform->pubdelete;
+ pub->pubactions.pubtruncate = pubform->pubtruncate;
ReleaseSysCache(tup);
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 9c5aa9ebc2..29992d4a0e 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -62,7 +62,8 @@ parse_publication_options(List *options,
bool *publish_given,
bool *publish_insert,
bool *publish_update,
- bool *publish_delete)
+ bool *publish_delete,
+ bool *publish_truncate)
{
ListCell *lc;
@@ -72,6 +73,7 @@ parse_publication_options(List *options,
*publish_insert = true;
*publish_update = true;
*publish_delete = true;
+ *publish_truncate = true;
/* Parse options */
foreach(lc, options)
@@ -96,6 +98,7 @@ parse_publication_options(List *options,
*publish_insert = false;
*publish_update = false;
*publish_delete = false;
+ *publish_truncate = false;
*publish_given = true;
publish = defGetString(defel);
@@ -116,6 +119,8 @@ parse_publication_options(List *options,
*publish_update = true;
else if (strcmp(publish_opt, "delete") == 0)
*publish_delete = true;
+ else if (strcmp(publish_opt, "truncate") == 0)
+ *publish_truncate = true;
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -145,6 +150,7 @@ CreatePublication(CreatePublicationStmt *stmt)
bool publish_insert;
bool publish_update;
bool publish_delete;
+ bool publish_truncate;
AclResult aclresult;
/* must have CREATE privilege on database */
@@ -181,7 +187,8 @@ CreatePublication(CreatePublicationStmt *stmt)
parse_publication_options(stmt->options,
&publish_given, &publish_insert,
- &publish_update, &publish_delete);
+ &publish_update, &publish_delete,
+ &publish_truncate);
values[Anum_pg_publication_puballtables - 1] =
BoolGetDatum(stmt->for_all_tables);
@@ -191,6 +198,8 @@ CreatePublication(CreatePublicationStmt *stmt)
BoolGetDatum(publish_update);
values[Anum_pg_publication_pubdelete - 1] =
BoolGetDatum(publish_delete);
+ values[Anum_pg_publication_pubtruncate - 1] =
+ BoolGetDatum(publish_truncate);
tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
@@ -237,11 +246,13 @@ AlterPublicationOptions(AlterPublicationStmt *stmt, Relation rel,
bool publish_insert;
bool publish_update;
bool publish_delete;
+ bool publish_truncate;
ObjectAddress obj;
parse_publication_options(stmt->options,
&publish_given, &publish_insert,
- &publish_update, &publish_delete);
+ &publish_update, &publish_delete,
+ &publish_truncate);
/* Everything ok, form a new tuple. */
memset(values, 0, sizeof(values));
@@ -258,6 +269,9 @@ AlterPublicationOptions(AlterPublicationStmt *stmt, Relation rel,
values[Anum_pg_publication_pubdelete - 1] = BoolGetDatum(publish_delete);
replaces[Anum_pg_publication_pubdelete - 1] = true;
+
+ values[Anum_pg_publication_pubtruncate - 1] = BoolGetDatum(publish_truncate);
+ replaces[Anum_pg_publication_pubtruncate - 1] = true;
}
tup = heap_modify_tuple(tup, RelationGetDescr(rel), values, nulls,
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index 948343e4ae..edc97a7662 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -26,6 +26,9 @@
*/
#define LOGICALREP_IS_REPLICA_IDENTITY 1
+#define TRUNCATE_CASCADE (1<<0)
+#define TRUNCATE_RESTART_SEQS (1<<1)
+
static void logicalrep_write_attrs(StringInfo out, Relation rel);
static void logicalrep_write_tuple(StringInfo out, Relation rel,
HeapTuple tuple);
@@ -292,6 +295,58 @@ logicalrep_read_delete(StringInfo in, LogicalRepTupleData *oldtup)
return relid;
}
+/*
+ * Write TRUNCATE to the output stream.
+ */
+void
+logicalrep_write_truncate(StringInfo out,
+ int nrelids,
+ Oid relids[],
+ bool cascade, bool restart_seqs)
+{
+ int i;
+ uint8 flags = 0;
+
+ pq_sendbyte(out, 'T'); /* action TRUNCATE */
+
+ pq_sendint32(out, nrelids);
+
+ /* encode and send truncate flags */
+ if (cascade)
+ flags |= TRUNCATE_CASCADE;
+ if (restart_seqs)
+ flags |= TRUNCATE_RESTART_SEQS;
+ pq_sendint8(out, flags);
+
+ for (i = 0; i < nrelids; i++)
+ pq_sendint32(out, relids[i]);
+}
+
+/*
+ * Read TRUNCATE from stream.
+ */
+List *
+logicalrep_read_truncate(StringInfo in,
+ bool *cascade, bool *restart_seqs)
+{
+ int i;
+ int nrelids;
+ List *relids = NIL;
+ uint8 flags;
+
+ nrelids = pq_getmsgint(in, 4);
+
+ /* read and decode truncate flags */
+ flags = pq_getmsgint(in, 1);
+ *cascade = (flags & TRUNCATE_CASCADE) > 0;
+ *restart_seqs = (flags & TRUNCATE_RESTART_SEQS) > 0;
+
+ for (i = 0; i < nrelids; i++)
+ relids = lappend_oid(relids, pq_getmsgint(in, 4));
+
+ return relids;
+}
+
/*
* Write relation description to the output stream.
*/
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index fdace7eea2..18766364f7 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -30,10 +30,12 @@
#include "access/xact.h"
#include "access/xlog_internal.h"
+#include "catalog/catalog.h"
#include "catalog/namespace.h"
#include "catalog/pg_subscription.h"
#include "catalog/pg_subscription_rel.h"
+#include "commands/tablecmds.h"
#include "commands/trigger.h"
#include "executor/executor.h"
@@ -83,6 +85,7 @@
#include "utils/inval.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
+#include "utils/rel.h"
#include "utils/timeout.h"
#include "utils/tqual.h"
#include "utils/syscache.h"
@@ -888,6 +891,67 @@ apply_handle_delete(StringInfo s)
CommandCounterIncrement();
}
+/*
+ * Handle TRUNCATE message.
+ *
+ * TODO: FDW support
+ */
+static void
+apply_handle_truncate(StringInfo s)
+{
+ bool cascade = false;
+ bool restart_seqs = false;
+ List *remote_relids = NIL;
+ List *remote_rels = NIL;
+ List *rels = NIL;
+ List *relids = NIL;
+ List *relids_logged = NIL;
+ ListCell *lc;
+
+ ensure_transaction();
+
+ remote_relids = logicalrep_read_truncate(s, &cascade, &restart_seqs);
+
+ foreach(lc, remote_relids)
+ {
+ LogicalRepRelId relid = lfirst_oid(lc);
+ LogicalRepRelMapEntry *rel;
+
+ rel = logicalrep_rel_open(relid, RowExclusiveLock);
+ if (!should_apply_changes_for_rel(rel))
+ {
+ /*
+ * The relation can't become interesting in the middle of the
+ * transaction so it's safe to unlock it.
+ */
+ logicalrep_rel_close(rel, RowExclusiveLock);
+ continue;
+ }
+
+ remote_rels = lappend(remote_rels, rel);
+ rels = lappend(rels, rel->localrel);
+ relids = lappend_oid(relids, rel->localreloid);
+ if (RelationIsLogicallyLogged(rel->localrel))
+ relids_logged = lappend_oid(relids, rel->localreloid);
+ }
+
+ /*
+ * Even if we used CASCADE on the upstream master we explicitly
+ * default to replaying changes without further cascading.
+ * This might be later changeable with a user specified option.
+ */
+ ExecuteTruncateGuts(rels, relids, relids_logged, DROP_RESTRICT, restart_seqs);
+
+ foreach(lc, remote_rels)
+ {
+ LogicalRepRelMapEntry *rel = lfirst(lc);
+
+ logicalrep_rel_close(rel, NoLock);
+ }
+
+ CommandCounterIncrement();
+}
+
/*
* Logical replication protocol message dispatcher.
@@ -919,6 +983,10 @@ apply_dispatch(StringInfo s)
case 'D':
apply_handle_delete(s);
break;
+ /* TRUNCATE */
+ case 'T':
+ apply_handle_truncate(s);
+ break;
/* RELATION */
case 'R':
apply_handle_relation(s);
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index aa9cf5b54e..e31ad7e027 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -39,6 +39,9 @@ static void pgoutput_commit_txn(LogicalDecodingContext *ctx,
static void pgoutput_change(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, Relation rel,
ReorderBufferChange *change);
+static void pgoutput_truncate(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, int nrelations, Relation relations[],
+ ReorderBufferChange *change);
static bool pgoutput_origin_filter(LogicalDecodingContext *ctx,
RepOriginId origin_id);
@@ -77,6 +80,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->startup_cb = pgoutput_startup;
cb->begin_cb = pgoutput_begin_txn;
cb->change_cb = pgoutput_change;
+ cb->truncate_cb = pgoutput_truncate;
cb->commit_cb = pgoutput_commit_txn;
cb->filter_by_origin_cb = pgoutput_origin_filter;
cb->shutdown_cb = pgoutput_shutdown;
@@ -250,6 +254,46 @@ pgoutput_commit_txn(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
OutputPluginWrite(ctx, true);
}
+/*
+ * Write the relation schema if the current schema haven't been sent yet.
+ */
+static void
+maybe_send_schema(LogicalDecodingContext *ctx,
+ Relation relation, RelationSyncEntry *relentry)
+{
+ if (!relentry->schema_sent)
+ {
+ TupleDesc desc;
+ int i;
+
+ desc = RelationGetDescr(relation);
+
+ /*
+ * Write out type info if needed. We do that only for user created
+ * types.
+ */
+ for (i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (att->attisdropped)
+ continue;
+
+ if (att->atttypid < FirstNormalObjectId)
+ continue;
+
+ OutputPluginPrepareWrite(ctx, false);
+ logicalrep_write_typ(ctx->out, att->atttypid);
+ OutputPluginWrite(ctx, false);
+ }
+
+ OutputPluginPrepareWrite(ctx, false);
+ logicalrep_write_rel(ctx->out, relation);
+ OutputPluginWrite(ctx, false);
+ relentry->schema_sent = true;
+ }
+}
+
/*
* Sends the decoded DML over wire.
*/
@@ -288,40 +332,7 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
/* Avoid leaking memory by using and resetting our own context */
old = MemoryContextSwitchTo(data->context);
- /*
- * Write the relation schema if the current schema haven't been sent yet.
- */
- if (!relentry->schema_sent)
- {
- TupleDesc desc;
- int i;
-
- desc = RelationGetDescr(relation);
-
- /*
- * Write out type info if needed. We do that only for user created
- * types.
- */
- for (i = 0; i < desc->natts; i++)
- {
- Form_pg_attribute att = TupleDescAttr(desc, i);
-
- if (att->attisdropped)
- continue;
-
- if (att->atttypid < FirstNormalObjectId)
- continue;
-
- OutputPluginPrepareWrite(ctx, false);
- logicalrep_write_typ(ctx->out, att->atttypid);
- OutputPluginWrite(ctx, false);
- }
-
- OutputPluginPrepareWrite(ctx, false);
- logicalrep_write_rel(ctx->out, relation);
- OutputPluginWrite(ctx, false);
- relentry->schema_sent = true;
- }
+ maybe_send_schema(ctx, relation, relentry);
/* Send the data */
switch (change->action)
@@ -363,6 +374,51 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
MemoryContextReset(data->context);
}
+static void
+pgoutput_truncate(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
+ int nrelations, Relation relations[], ReorderBufferChange *change)
+{
+ PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
+ MemoryContext old;
+ RelationSyncEntry *relentry;
+ int i;
+ int nrelids;
+ Oid *relids;
+
+ old = MemoryContextSwitchTo(data->context);
+
+ relids = palloc0(nrelations * sizeof(Oid));
+ nrelids = 0;
+
+ for (i = 0; i < nrelations; i++)
+ {
+ Relation relation = relations[i];
+ Oid relid = RelationGetRelid(relation);
+
+ relentry = get_rel_sync_entry(data, relid);
+
+ if (!is_publishable_relation(relation))
+ continue;
+
+ if (!relentry->pubactions.pubtruncate)
+ continue;
+
+ relids[nrelids++] = relid;
+ maybe_send_schema(ctx, relation, relentry);
+ }
+
+ OutputPluginPrepareWrite(ctx, true);
+ logicalrep_write_truncate(ctx->out,
+ nrelids,
+ relids,
+ change->data.truncate_msg.cascade,
+ change->data.truncate_msg.restart_seqs);
+ OutputPluginWrite(ctx, true);
+
+ MemoryContextSwitchTo(old);
+ MemoryContextReset(data->context);
+}
+
/*
* Currently we always forward.
*/
@@ -504,7 +560,7 @@ get_rel_sync_entry(PGOutputData *data, Oid relid)
* we only need to consider ones that the subscriber requested.
*/
entry->pubactions.pubinsert = entry->pubactions.pubupdate =
- entry->pubactions.pubdelete = false;
+ entry->pubactions.pubdelete = entry->pubactions.pubtruncate = false;
foreach(lc, data->publications)
{
@@ -515,10 +571,11 @@ get_rel_sync_entry(PGOutputData *data, Oid relid)
entry->pubactions.pubinsert |= pub->pubactions.pubinsert;
entry->pubactions.pubupdate |= pub->pubactions.pubupdate;
entry->pubactions.pubdelete |= pub->pubactions.pubdelete;
+ entry->pubactions.pubtruncate |= pub->pubactions.pubtruncate;
}
if (entry->pubactions.pubinsert && entry->pubactions.pubupdate &&
- entry->pubactions.pubdelete)
+ entry->pubactions.pubdelete && entry->pubactions.pubtruncate)
break;
}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 69a2114a10..6a67c185b0 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -5339,6 +5339,7 @@ GetRelationPublicationActions(Relation relation)
pubactions->pubinsert |= pubform->pubinsert;
pubactions->pubupdate |= pubform->pubupdate;
pubactions->pubdelete |= pubform->pubdelete;
+ pubactions->pubtruncate |= pubform->pubtruncate;
ReleaseSysCache(tup);
@@ -5347,7 +5348,7 @@ GetRelationPublicationActions(Relation relation)
* other publications.
*/
if (pubactions->pubinsert && pubactions->pubupdate &&
- pubactions->pubdelete)
+ pubactions->pubdelete && pubactions->pubtruncate)
break;
}
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index d066f4f00b..69016a6c4d 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -3712,6 +3712,7 @@ getPublications(Archive *fout)
int i_pubinsert;
int i_pubupdate;
int i_pubdelete;
+ int i_pubtruncate;
int i,
ntups;
@@ -3723,12 +3724,20 @@ getPublications(Archive *fout)
resetPQExpBuffer(query);
/* Get the publications. */
- appendPQExpBuffer(query,
- "SELECT p.tableoid, p.oid, p.pubname, "
- "(%s p.pubowner) AS rolname, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete "
- "FROM pg_publication p",
- username_subquery);
+ if (fout->remoteVersion >= 110000)
+ appendPQExpBuffer(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "(%s p.pubowner) AS rolname, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate "
+ "FROM pg_publication p",
+ username_subquery);
+ else
+ appendPQExpBuffer(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "(%s p.pubowner) AS rolname, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate "
+ "FROM pg_publication p",
+ username_subquery);
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -3742,6 +3751,7 @@ getPublications(Archive *fout)
i_pubinsert = PQfnumber(res, "pubinsert");
i_pubupdate = PQfnumber(res, "pubupdate");
i_pubdelete = PQfnumber(res, "pubdelete");
+ i_pubtruncate = PQfnumber(res, "pubtruncate");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -3762,6 +3772,8 @@ getPublications(Archive *fout)
(strcmp(PQgetvalue(res, i, i_pubupdate), "t") == 0);
pubinfo[i].pubdelete =
(strcmp(PQgetvalue(res, i, i_pubdelete), "t") == 0);
+ pubinfo[i].pubtruncate =
+ (strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
if (strlen(pubinfo[i].rolname) == 0)
write_msg(NULL, "WARNING: owner of publication \"%s\" appears to be invalid\n",
@@ -3829,6 +3841,15 @@ dumpPublication(Archive *fout, PublicationInfo *pubinfo)
first = false;
}
+ if (pubinfo->pubtruncate)
+ {
+ if (!first)
+ appendPQExpBufferStr(query, ", ");
+
+ appendPQExpBufferStr(query, "truncate");
+ first = false;
+ }
+
appendPQExpBufferStr(query, "');\n");
ArchiveEntry(fout, pubinfo->dobj.catId, pubinfo->dobj.dumpId,
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index a4d6d926a8..c2314758de 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -595,6 +595,7 @@ typedef struct _PublicationInfo
bool pubinsert;
bool pubupdate;
bool pubdelete;
+ bool pubtruncate;
} PublicationInfo;
/*
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index acdfde2a1e..25852b903c 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2038,7 +2038,7 @@
create_order => 50,
create_sql => 'CREATE PUBLICATION pub1;',
regexp => qr/^
- \QCREATE PUBLICATION pub1 WITH (publish = 'insert, update, delete');\E
+ \QCREATE PUBLICATION pub1 WITH (publish = 'insert, update, delete, truncate');\E
/xm,
like => {
%full_runs,
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 0c3be1f504..75a1e42cee 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -5187,7 +5187,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -5207,13 +5207,17 @@ listPublications(const char *pattern)
" puballtables AS \"%s\",\n"
" pubinsert AS \"%s\",\n"
" pubupdate AS \"%s\",\n"
- " pubdelete AS \"%s\"\n",
+ " pubdelete AS \"%s\"",
gettext_noop("Name"),
gettext_noop("Owner"),
gettext_noop("All tables"),
gettext_noop("Inserts"),
gettext_noop("Updates"),
gettext_noop("Deletes"));
+ if (pset.sversion >= 110000)
+ appendPQExpBuffer(&buf,
+ ",\n pubtruncate AS \"%s\"",
+ gettext_noop("Truncates"));
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -5254,6 +5258,7 @@ describePublications(const char *pattern)
PQExpBufferData buf;
int i;
PGresult *res;
+ bool has_pubtruncate;
if (pset.sversion < 100000)
{
@@ -5265,13 +5270,19 @@ describePublications(const char *pattern)
return true;
}
+ has_pubtruncate = (pset.sversion >= 110000);
+
initPQExpBuffer(&buf);
printfPQExpBuffer(&buf,
"SELECT oid, pubname,\n"
" pg_catalog.pg_get_userbyid(pubowner) AS owner,\n"
- " puballtables, pubinsert, pubupdate, pubdelete\n"
- "FROM pg_catalog.pg_publication\n");
+ " puballtables, pubinsert, pubupdate, pubdelete");
+ if (has_pubtruncate)
+ appendPQExpBuffer(&buf,
+ ", pubtruncate");
+ appendPQExpBuffer(&buf,
+ "\nFROM pg_catalog.pg_publication\n");
processSQLNamePattern(pset.db, &buf, pattern, false, false,
NULL, "pubname", NULL,
@@ -5317,6 +5328,9 @@ describePublications(const char *pattern)
printTableOpt myopt = pset.popt.topt;
printTableContent cont;
+ if (has_pubtruncate)
+ ncols++;
+
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
printTableInit(&cont, &myopt, title.data, ncols, nrows);
@@ -5326,12 +5340,16 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Inserts"), true, align);
printTableAddHeader(&cont, gettext_noop("Updates"), true, align);
printTableAddHeader(&cont, gettext_noop("Deletes"), true, align);
+ if (has_pubtruncate)
+ printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 4), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 5), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 6), false, false);
+ if (has_pubtruncate)
+ printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (!puballtables)
{
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index 37e77b8be7..b643c489cd 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -49,6 +49,9 @@ CATALOG(pg_publication,6104)
/* true if deletes are published */
bool pubdelete;
+ /* true if truncates are published */
+ bool pubtruncate;
+
} FormData_pg_publication;
/* ----------------
@@ -63,19 +66,21 @@ typedef FormData_pg_publication *Form_pg_publication;
* ----------------
*/
-#define Natts_pg_publication 6
+#define Natts_pg_publication 7
#define Anum_pg_publication_pubname 1
#define Anum_pg_publication_pubowner 2
#define Anum_pg_publication_puballtables 3
#define Anum_pg_publication_pubinsert 4
#define Anum_pg_publication_pubupdate 5
#define Anum_pg_publication_pubdelete 6
+#define Anum_pg_publication_pubtruncate 7
typedef struct PublicationActions
{
bool pubinsert;
bool pubupdate;
bool pubdelete;
+ bool pubtruncate;
} PublicationActions;
typedef struct Publication
diff --git a/src/include/replication/logicalproto.h b/src/include/replication/logicalproto.h
index 116f16f42d..92e88d3127 100644
--- a/src/include/replication/logicalproto.h
+++ b/src/include/replication/logicalproto.h
@@ -97,6 +97,10 @@ extern void logicalrep_write_delete(StringInfo out, Relation rel,
HeapTuple oldtuple);
extern LogicalRepRelId logicalrep_read_delete(StringInfo in,
LogicalRepTupleData *oldtup);
+extern void logicalrep_write_truncate(StringInfo out, int nrelids, Oid relids[],
+ bool cascade, bool restart_seqs);
+extern List *logicalrep_read_truncate(StringInfo in,
+ bool *cascade, bool *restart_seqs);
extern void logicalrep_write_rel(StringInfo out, Relation rel);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
extern void logicalrep_write_typ(StringInfo out, Oid typoid);
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 0c86c647bc..afbbdd543d 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -21,20 +21,20 @@ ERROR: unrecognized publication parameter: foo
CREATE PUBLICATION testpub_xxx WITH (publish = 'cluster, vacuum');
ERROR: unrecognized "publish" value: "cluster"
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes
---------------------+--------------------------+------------+---------+---------+---------
- testpib_ins_trunct | regress_publication_user | f | t | f | f
- testpub_default | regress_publication_user | f | f | t | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates
+--------------------+--------------------------+------------+---------+---------+---------+-----------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes
---------------------+--------------------------+------------+---------+---------+---------
- testpib_ins_trunct | regress_publication_user | f | t | f | f
- testpub_default | regress_publication_user | f | t | t | t
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates
+--------------------+--------------------------+------------+---------+---------+---------+-----------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f
(2 rows)
--- adding tables
@@ -76,10 +76,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes
---------------------------+------------+---------+---------+---------
- regress_publication_user | t | t | t | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates
+--------------------------+------------+---------+---------+---------+-----------
+ regress_publication_user | t | t | t | f | f
(1 row)
DROP TABLE testpub_tbl2;
@@ -89,19 +89,19 @@ CREATE TABLE testpub_tbl3a (b text) INHERITS (testpub_tbl3);
CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes
---------------------------+------------+---------+---------+---------
- regress_publication_user | f | t | t | t
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates
+--------------------------+------------+---------+---------+---------+-----------
+ regress_publication_user | f | t | t | t | t
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes
---------------------------+------------+---------+---------+---------
- regress_publication_user | f | t | t | t
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates
+--------------------------+------------+---------+---------+---------+-----------
+ regress_publication_user | f | t | t | t | t
Tables:
"public.testpub_tbl3"
@@ -119,10 +119,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes
---------------------------+------------+---------+---------+---------
- regress_publication_user | f | t | t | t
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates
+--------------------------+------------+---------+---------+---------+-----------
+ regress_publication_user | f | t | t | t | t
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -165,10 +165,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes
---------------------------+------------+---------+---------+---------
- regress_publication_user | f | t | t | t
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates
+--------------------------+------------+---------+---------+---------+-----------
+ regress_publication_user | f | t | t | t | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -210,10 +210,10 @@ DROP TABLE testpub_parted;
DROP VIEW testpub_view;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes
---------------------------+------------+---------+---------+---------
- regress_publication_user | f | t | t | t
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates
+--------------------------+------------+---------+---------+---------+-----------
+ regress_publication_user | f | t | t | t | f
(1 row)
-- fail - must be owner of publication
@@ -223,20 +223,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes
--------------+--------------------------+------------+---------+---------+---------
- testpub_foo | regress_publication_user | f | t | t | t
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates
+-------------+--------------------------+------------+---------+---------+---------+-----------
+ testpub_foo | regress_publication_user | f | t | t | t | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes
------------------+---------------------------+------------+---------+---------+---------
- testpub_default | regress_publication_user2 | f | t | t | t
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates
+-----------------+---------------------------+------------+---------+---------+---------+-----------
+ testpub_default | regress_publication_user2 | f | t | t | t | f
(1 row)
DROP PUBLICATION testpub_default;
diff --git a/src/test/subscription/t/010_truncate.pl b/src/test/subscription/t/010_truncate.pl
new file mode 100644
index 0000000000..8ea4ab624f
--- /dev/null
+++ b/src/test/subscription/t/010_truncate.pl
@@ -0,0 +1,161 @@
+# Test TRUNCATE
+use strict;
+use warnings;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 9;
+
+# setup
+
+my $node_publisher = get_new_node('publisher');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->start;
+
+my $node_subscriber = get_new_node('subscriber');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->start;
+
+my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+
+$node_publisher->safe_psql('postgres',
+ "CREATE TABLE tab1 (a int PRIMARY KEY)");
+
+$node_subscriber->safe_psql('postgres',
+ "CREATE TABLE tab1 (a int PRIMARY KEY)");
+
+$node_publisher->safe_psql('postgres',
+ "CREATE TABLE tab2 (a int PRIMARY KEY)");
+
+$node_subscriber->safe_psql('postgres',
+ "CREATE TABLE tab2 (a int PRIMARY KEY)");
+
+$node_publisher->safe_psql('postgres',
+ "CREATE TABLE tab3 (a int PRIMARY KEY)");
+
+$node_subscriber->safe_psql('postgres',
+ "CREATE TABLE tab3 (a int PRIMARY KEY)");
+
+$node_publisher->safe_psql('postgres',
+ "CREATE TABLE tab4 (x int PRIMARY KEY, y int REFERENCES tab3)");
+
+$node_subscriber->safe_psql('postgres',
+ "CREATE TABLE tab4 (x int PRIMARY KEY, y int REFERENCES tab3)");
+
+$node_subscriber->safe_psql('postgres',
+ "CREATE SEQUENCE seq1 OWNED BY tab1.a"
+);
+$node_subscriber->safe_psql('postgres',
+ "ALTER SEQUENCE seq1 START 101"
+);
+
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION pub1 FOR TABLE tab1");
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION pub2 FOR TABLE tab2 WITH (publish = insert)");
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION pub3 FOR TABLE tab3, tab4");
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr application_name=sub1' PUBLICATION pub1");
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION sub2 CONNECTION '$publisher_connstr application_name=sub2' PUBLICATION pub2");
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION sub3 CONNECTION '$publisher_connstr application_name=sub3' PUBLICATION pub3");
+
+$node_publisher->wait_for_catchup('sub1');
+
+# insert data to truncate
+
+$node_subscriber->safe_psql('postgres', "INSERT INTO tab1 VALUES (1), (2), (3)");
+
+$node_publisher->wait_for_catchup('sub1');
+
+# truncate and check
+
+$node_publisher->safe_psql('postgres', "TRUNCATE tab1");
+
+$node_publisher->wait_for_catchup('sub1');
+
+my $result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab1");
+is($result, qq(0||),
+ 'truncate replicated');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq1')");
+is($result, qq(1),
+ 'sequence not restarted');
+
+# truncate with restart identity
+
+$node_publisher->safe_psql('postgres', "TRUNCATE tab1 RESTART IDENTITY");
+
+$node_publisher->wait_for_catchup('sub1');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT nextval('seq1')");
+is($result, qq(101),
+ 'truncate restarted identities');
+
+# test publication that does not replicate truncate
+
+$node_subscriber->safe_psql('postgres', "INSERT INTO tab2 VALUES (1), (2), (3)");
+
+$node_publisher->safe_psql('postgres', "TRUNCATE tab2");
+
+$node_publisher->wait_for_catchup('sub2');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab2");
+is($result, qq(3|1|3),
+ 'truncate not replicated');
+
+$node_publisher->safe_psql('postgres',
+ "ALTER PUBLICATION pub2 SET (publish = 'insert, truncate')");
+
+$node_publisher->safe_psql('postgres', "TRUNCATE tab2");
+
+$node_publisher->wait_for_catchup('sub2');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab2");
+is($result, qq(0||),
+ 'truncate replicated after publication change');
+
+# test multiple tables connected by foreign keys
+
+$node_subscriber->safe_psql('postgres', "INSERT INTO tab3 VALUES (1), (2), (3)");
+$node_subscriber->safe_psql('postgres', "INSERT INTO tab4 VALUES (11, 1), (111, 1), (22, 2)");
+
+$node_publisher->safe_psql('postgres', "TRUNCATE tab3, tab4");
+
+$node_publisher->wait_for_catchup('sub3');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab3");
+is($result, qq(0||),
+ 'truncate of multiple tables replicated');
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(x), max(x) FROM tab4");
+is($result, qq(0||),
+ 'truncate of multiple tables replicated');
+
+# test truncate of multiple tables, some of which are not published
+
+$node_subscriber->safe_psql('postgres', "DROP SUBSCRIPTION sub2");
+$node_publisher->safe_psql('postgres', "DROP PUBLICATION pub2");
+
+$node_subscriber->safe_psql('postgres', "INSERT INTO tab1 VALUES (1), (2), (3)");
+$node_subscriber->safe_psql('postgres', "INSERT INTO tab2 VALUES (1), (2), (3)");
+
+$node_publisher->safe_psql('postgres', "TRUNCATE tab1, tab2");
+
+$node_publisher->wait_for_catchup('sub1');
+
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab1");
+is($result, qq(0||),
+ 'truncate of multiple tables some not published');
+$result = $node_subscriber->safe_psql('postgres',
+ "SELECT count(*), min(a), max(a) FROM tab2");
+is($result, qq(3|1|3),
+ 'truncate of multiple tables some not published');
--
2.17.0
Re: [PATCH] Logical decoding of TRUNCATE
This sounds like a good approach.
+static void +pg_decode_truncate(LogicalDecodingContext *ctx, ReorderBufferTXN *txn, + int nrelations, Relation relations[], ReorderBufferChange *change) +{
+ for (i = 0; i < nrelations; i++) + { + Oid relid = RelationGetRelid(relations[i]); + + if (i > 0) + appendStringInfoString(ctx->out, ", "); + + appendStringInfoString(ctx->out, + quote_qualified_identifier( + get_namespace_name(get_rel_namespace(relid)), + get_rel_name(relid)));
Note that you start the loop having the Relation; yet you go extra
length to grab the relnamespace and relname from syscache instead of
just relations[i]->rd_rel->relname etc.
pgoutput doesn't do it that way, so it doesn't affect logical
replication, but I think it's best not to create awkward code in
test_decoding, since it's so widely copied.
+ else if (info == XLOG_HEAP_TRUNCATE) + { + xl_heap_truncate *xlrec = (xl_heap_truncate *) rec; + + if (xlrec->flags & XLH_TRUNCATE_CASCADE) + appendStringInfo(buf, "cascade "); + if (xlrec->flags & XLH_TRUNCATE_RESTART_SEQS) + appendStringInfo(buf, "restart_seqs "); + appendStringInfo(buf, "nrelids %u", xlrec->nrelids); + /* skip the list of relids */ + }
Maybe not a big deal, but for future pg_waldump users I'm sure it'll be
nice to have the OIDs here.
+void +ExecuteTruncateGuts(List *explicit_rels, List *relids, List *relids_logged, + DropBehavior behavior, bool restart_seqs) +{
Please add a comment atop this function.
+ /* + * Write a WAL record to allow this set of actions to be logically decoded. + * + * Assemble an array of relids so we can write a single WAL record for the + * whole action. + */ + if (list_length(relids_logged) > 0) + { + xl_heap_truncate xlrec; + int i = 0;
I wonder if this should happen only if logical decoding? (Maybe it
already occurs because relids_logged would be empty? Worth a comment in
that case)
--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [PATCH] Logical decoding of TRUNCATE
On 4/5/18 13:07, Alvaro Herrera wrote:
Note that you start the loop having the Relation; yet you go extra
length to grab the relnamespace and relname from syscache instead of
just relations[i]->rd_rel->relname etc.
fixed
Maybe not a big deal, but for future pg_waldump users I'm sure it'll be
nice to have the OIDs here.
done
+void +ExecuteTruncateGuts(List *explicit_rels, List *relids, List *relids_logged, + DropBehavior behavior, bool restart_seqs) +{Please add a comment atop this function.
done
+ /* + * Write a WAL record to allow this set of actions to be logically decoded. + * + * Assemble an array of relids so we can write a single WAL record for the + * whole action. + */ + if (list_length(relids_logged) > 0) + { + xl_heap_truncate xlrec; + int i = 0;I wonder if this should happen only if logical decoding? (Maybe it
already occurs because relids_logged would be empty? Worth a comment in
that case)
Added an assertion and a comment.
Committed with those changes.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [PATCH] Logical decoding of TRUNCATE
On Sat, Apr 07, 2018 at 07:40:11PM -0400, Peter Eisentraut wrote:
Committed with those changes.
Since commit 039eb6e added logical replication support for TRUNCATE, logical
apply of the TRUNCATE fails if it chooses a parallel index build:
cat >/tmp/most_parallel.conf <<EOCONF; make -C src/test/subscription check TEMP_CONFIG=/tmp/most_parallel.conf
# like noah@leadboat.com buildfarm members
log_line_prefix = '%m [%p:%l] %q%a '
log_connections = 'true'
log_disconnections = 'true'
log_statement = 'all'
fsync = off
authentication_timeout = '600s'
wal_sender_timeout = '18000s'
# not on v12
backtrace_functions = 'quickdie, GetTransactionSnapshot'
# as much parallelism as we can get
log_statement = all
wal_level = minimal
max_wal_senders = 0
force_parallel_mode = regress
min_parallel_index_scan_size = 0
min_parallel_table_scan_size = 0
parallel_setup_cost = 0
parallel_tuple_cost = 0
EOCONF
Symptom in src/test/subscription/tmp_check/log/010_truncate_subscriber.log:
2020-12-19 17:54:04.669 PST [3629509:1] LOG: logical replication apply worker for subscription "sub1" has started
2020-12-19 17:54:04.682 PST [3629509:2] ERROR: cannot take query snapshot during a parallel operation
2020-12-19 17:54:04.682 PST [3629509:3] BACKTRACE:
postgres: subscriber: logical replication worker for subscription 16411 (GetTransactionSnapshot+0x168) [0x951ce8]
postgres: subscriber: logical replication worker for subscription 16411 (InitializeParallelDSM+0x16) [0x52cf86]
postgres: subscriber: logical replication worker for subscription 16411 (btbuild+0x26a) [0x50905a]
postgres: subscriber: logical replication worker for subscription 16411 (index_build+0x14b) [0x569c1b]
postgres: subscriber: logical replication worker for subscription 16411 (reindex_index+0x19a) [0x56caea]
postgres: subscriber: logical replication worker for subscription 16411 (reindex_relation+0xc0) [0x56d090]
postgres: subscriber: logical replication worker for subscription 16411 (ExecuteTruncateGuts+0x376) [0x62f0d6]
postgres: subscriber: logical replication worker for subscription 16411 () [0x78d592]
postgres: subscriber: logical replication worker for subscription 16411 (ApplyWorkerMain+0x5ab) [0x78e4eb]
postgres: subscriber: logical replication worker for subscription 16411 (StartBackgroundWorker+0x23f) [0x75522f]
postgres: subscriber: logical replication worker for subscription 16411 () [0x762a6d]
postgres: subscriber: logical replication worker for subscription 16411 () [0x7635ee]
/lib64/libpthread.so.0(+0xf630) [0x7fe081e97630]
/lib64/libc.so.6(__select+0x13) [0x7fe0805c0983]
postgres: subscriber: logical replication worker for subscription 16411 () [0x4887ac]
postgres: subscriber: logical replication worker for subscription 16411 (PostmasterMain+0x1118) [0x764c88]
postgres: subscriber: logical replication worker for subscription 16411 (main+0x6f2) [0x48aae2]
/lib64/libc.so.6(__libc_start_main+0xf5) [0x7fe0804ed555]
postgres: subscriber: logical replication worker for subscription 16411 () [0x48ab49]
2020-12-19 17:54:04.683 PST [3629353:5] LOG: background worker "logical replication worker" (PID 3629509) exited with exit code 1
(To see that particular failure at commit 039eb6e, one would need to
cherry-pick f90e80b to avoid an earlier failure. One would also modify
PostgresNode to inject the parallelism settings, because PostgresNode did not
support TEMP_CONFIG in those days.)
Re: [PATCH] Logical decoding of TRUNCATE
Hi,
On 2020-12-20 04:13:19 +0000, Noah Misch wrote:
postgres: subscriber: logical replication worker for subscription 16411 (GetTransactionSnapshot+0x168) [0x951ce8]
postgres: subscriber: logical replication worker for subscription 16411 (InitializeParallelDSM+0x16) [0x52cf86]
postgres: subscriber: logical replication worker for subscription 16411 (btbuild+0x26a) [0x50905a]
postgres: subscriber: logical replication worker for subscription 16411 (index_build+0x14b) [0x569c1b]
postgres: subscriber: logical replication worker for subscription 16411 (reindex_index+0x19a) [0x56caea]
postgres: subscriber: logical replication worker for subscription 16411 (reindex_relation+0xc0) [0x56d090]
postgres: subscriber: logical replication worker for subscription 16411 (ExecuteTruncateGuts+0x376) [0x62f0d6]
postgres: subscriber: logical replication worker for subscription 16411 () [0x78d592]
postgres: subscriber: logical replication worker for subscription 16411 (ApplyWorkerMain+0x5ab) [0x78e4eb]
postgres: subscriber: logical replication worker for subscription 16411 (StartBackgroundWorker+0x23f) [0x75522f]
postgres: subscriber: logical replication worker for subscription 16411 () [0x762a6d]
postgres: subscriber: logical replication worker for subscription 16411 () [0x7635ee]
/lib64/libpthread.so.0(+0xf630) [0x7fe081e97630]
/lib64/libc.so.6(__select+0x13) [0x7fe0805c0983]
postgres: subscriber: logical replication worker for subscription 16411 () [0x4887ac]
postgres: subscriber: logical replication worker for subscription 16411 (PostmasterMain+0x1118) [0x764c88]
postgres: subscriber: logical replication worker for subscription 16411 (main+0x6f2) [0x48aae2]
/lib64/libc.so.6(__libc_start_main+0xf5) [0x7fe0804ed555]
postgres: subscriber: logical replication worker for subscription 16411 () [0x48ab49]
2020-12-19 17:54:04.683 PST [3629353:5] LOG: background worker "logical replication worker" (PID 3629509) exited with exit code 1
Hm. Do I understand correctly that this problem is hit solely because
the parallel mode code relies on there already have been a transaction
snapshot set, thus avoiding the error? And that the code normally only
works because GetTransactionSnapshot() will already have been called
somewhere, before EnterParallelMode()?
Greetings,
Andres Freund
Re: [PATCH] Logical decoding of TRUNCATE
On Sun, Dec 20, 2020 at 3:13 PM Andres Freund <andres@anarazel.de> wrote:
Hm. Do I understand correctly that this problem is hit solely because
the parallel mode code relies on there already have been a transaction
snapshot set, thus avoiding the error? And that the code normally only
works because GetTransactionSnapshot() will already have been called
somewhere, before EnterParallelMode()?
It seems unlikely that InitializeParallelDSM() was ever intended to be
run in a background worker.
--
Peter Geoghegan
Re: [PATCH] Logical decoding of TRUNCATE
On Sun, Dec 20, 2020 at 9:43 AM Noah Misch <noah@leadboat.com> wrote:
Since commit 039eb6e added logical replication support for TRUNCATE, logical
apply of the TRUNCATE fails if it chooses a parallel index build:
I think the TRUNCATE operation should not use parallelism either via
apply worker or without it because there is nothing to scan in heap.
Additionally, we can have an Assert or elog in InitializeParallelDSM
to ensure that it is never invoked by parallel worker.
--
With Regards,
Amit Kapila.
Re: [PATCH] Logical decoding of TRUNCATE
On Sun, Dec 20, 2020 at 03:54:31PM -0800, Peter Geoghegan wrote:
On Sun, Dec 20, 2020 at 3:13 PM Andres Freund <andres@anarazel.de> wrote:
Hm. Do I understand correctly that this problem is hit solely because
the parallel mode code relies on there already have been a transaction
snapshot set, thus avoiding the error? And that the code normally only
works because GetTransactionSnapshot() will already have been called
somewhere, before EnterParallelMode()?
I think so.
It seems unlikely that InitializeParallelDSM() was ever intended to be
run in a background worker.
That wouldn't surprise me. Nonetheless, when worker_spi runs parallel
queries, they work fine. The logical replication worker experiences novel
scenarios, because it calls ExecuteTruncateGuts() directly, not as part of an
actual TRUNCATE query. That bypasses some of the usual once-per-query setup.
On Mon, Dec 21, 2020 at 12:29:37PM +0530, Amit Kapila wrote:
I think the TRUNCATE operation should not use parallelism either via
apply worker or without it because there is nothing to scan in heap.
That's fair.
Additionally, we can have an Assert or elog in InitializeParallelDSM
to ensure that it is never invoked by parallel worker.
I don't know whether InitializeParallelDSM() operates correctly from inside a
parallel worker. That is orthogonal to the bug here.
Re: [PATCH] Logical decoding of TRUNCATE
Hi,
On 2020-12-20 15:54:31 -0800, Peter Geoghegan wrote:
On Sun, Dec 20, 2020 at 3:13 PM Andres Freund <andres@anarazel.de> wrote:
Hm. Do I understand correctly that this problem is hit solely because
the parallel mode code relies on there already have been a transaction
snapshot set, thus avoiding the error? And that the code normally only
works because GetTransactionSnapshot() will already have been called
somewhere, before EnterParallelMode()?It seems unlikely that InitializeParallelDSM() was ever intended to be
run in a background worker.
IDK, the environment in a bgworker shouldn't be that different from the
normal query environment in a normal connection. And it's far from
insane to want to be able to run a paralell query in a bgworker (and I
*think* I have seen that work before). This case here seems more like
an accidental dependency than anything to me, once that could perhaps
even hint at problems in normal backends too.
Greetings,
Andres Freund
Re: [PATCH] Logical decoding of TRUNCATE
On Mon, Dec 21, 2020 at 09:42:47AM -0800, Andres Freund wrote:
On 2020-12-20 15:54:31 -0800, Peter Geoghegan wrote:
On Sun, Dec 20, 2020 at 3:13 PM Andres Freund <andres@anarazel.de> wrote:
Hm. Do I understand correctly that this problem is hit solely because
the parallel mode code relies on there already have been a transaction
snapshot set, thus avoiding the error? And that the code normally only
works because GetTransactionSnapshot() will already have been called
somewhere, before EnterParallelMode()?It seems unlikely that InitializeParallelDSM() was ever intended to be
run in a background worker.IDK, the environment in a bgworker shouldn't be that different from the
normal query environment in a normal connection. And it's far from
insane to want to be able to run a paralell query in a bgworker (and I
*think* I have seen that work before). This case here seems more like
an accidental dependency than anything to me, once that could perhaps
even hint at problems in normal backends too.
Yeah. SPI_execute("TRUNCATE foo", false, 0) has no trouble doing a parallel
index build in a bgworker. Let's ignore intention and be pleased about that.