COPY vs. transition tables
Folks,
Using the script attached, I'm getting this very odd result set below.
Audit records from COPY to the "foo bar" table aren't getting
recorded, but audit records from COPY to the baz table are.
Best,
David.
\i copy_oddity.sql
CREATE TABLE
COMMENT
CREATE FUNCTION
CREATE FUNCTION
CREATE EVENT TRIGGER
COMMENT
psql:logging_infra.sql:165: NOTICE: Adding log table(s) for public.foo bar
CREATE TABLE
INSERT 0 1
UPDATE 1
DELETE 1
COPY 5
timestamp | user | action | table_schema | table_name | old_row | new_row
-------------------------------+---------+-------------------------------+--------------+------------+-----------------------+-----------------------
2017-07-08 01:36:52.368228-07 | shackle | 2017-07-08 01:36:52.368228-07 | public | foo bar | | {"t": null, "id": 1}
2017-07-08 01:36:52.368228-07 | shackle | 2017-07-08 01:36:52.368228-07 | public | foo bar | {"t": null, "id": 1} | {"t": "baz", "id": 1}
2017-07-08 01:36:52.368228-07 | shackle | 2017-07-08 01:36:52.368228-07 | public | foo bar | {"t": "baz", "id": 1} |
(3 rows)
psql:logging_infra.sql:180: NOTICE: Adding log table(s) for public.baz
psql:logging_infra.sql:180: NOTICE: relation "public_log" already exists, skipping
CREATE TABLE
COPY 5
timestamp | user | action | table_schema | table_name | old_row | new_row
-------------------------------+---------+-------------------------------+--------------+------------+---------+------------
2017-07-08 01:36:52.368228-07 | shackle | 2017-07-08 01:36:52.368228-07 | public | baz | | {"t": "a"}
2017-07-08 01:36:52.368228-07 | shackle | 2017-07-08 01:36:52.368228-07 | public | baz | | {"t": "b"}
2017-07-08 01:36:52.368228-07 | shackle | 2017-07-08 01:36:52.368228-07 | public | baz | | {"t": "c"}
2017-07-08 01:36:52.368228-07 | shackle | 2017-07-08 01:36:52.368228-07 | public | baz | | {"t": "d"}
2017-07-08 01:36:52.368228-07 | shackle | 2017-07-08 01:36:52.368228-07 | public | baz | | {"t": "e"}
(5 rows)
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david(dot)fetter(at)gmail(dot)com
Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate
Attachments:
On Sat, Jul 8, 2017 at 8:42 PM, David Fetter <david@fetter.org> wrote:
Using the script attached, I'm getting this very odd result set below.
Audit records from COPY to the "foo bar" table aren't getting
recorded, but audit records from COPY to the baz table are.
Thanks for the bug report. I think it's the presence of the index on
"foo bar", not the space in its name (but thanks for that curve
ball!), that causes these tuples not to be captured.
CopyFromInsertBatch takes a different path depending on whether there
are any indexes, and mistakenly passes NULL for transition_capture.
The attached seems to fix it, but I'll look more closely and send a
version with a regression test on Monday.
--
Thomas Munro
http://www.enterprisedb.com
Attachments:
copy-with-indexes-and-transitions.patchapplication/octet-stream; name=copy-with-indexes-and-transitions.patchDownload
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f391828e74f..fc5f4f66ead 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2915,7 +2915,7 @@ CopyFromInsertBatch(CopyState cstate, EState *estate, CommandId mycid,
estate, false, NULL, NIL);
ExecARInsertTriggers(estate, resultRelInfo,
bufferedTuples[i],
- recheckIndexes, NULL);
+ recheckIndexes, cstate->transition_capture);
list_free(recheckIndexes);
}
}
On Sun, Jul 09, 2017 at 11:46:03AM +1200, Thomas Munro wrote:
On Sat, Jul 8, 2017 at 8:42 PM, David Fetter <david@fetter.org> wrote:
Using the script attached, I'm getting this very odd result set below.
Audit records from COPY to the "foo bar" table aren't getting
recorded, but audit records from COPY to the baz table are.Thanks for the bug report. I think it's the presence of the index on
"foo bar", not the space in its name (but thanks for that curve
ball!), that causes these tuples not to be captured.
Thanks for getting this fixed, and apologies for not trimming this
down to a minimal repro.
Best,
David.
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david(dot)fetter(at)gmail(dot)com
Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Sun, Jul 9, 2017 at 11:46 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:
On Sat, Jul 8, 2017 at 8:42 PM, David Fetter <david@fetter.org> wrote:
Using the script attached, I'm getting this very odd result set below.
Audit records from COPY to the "foo bar" table aren't getting
recorded, but audit records from COPY to the baz table are.Thanks for the bug report. I think it's the presence of the index on
"foo bar", not the space in its name (but thanks for that curve
ball!), that causes these tuples not to be captured.
CopyFromInsertBatch takes a different path depending on whether there
are any indexes, and mistakenly passes NULL for transition_capture.
The attached seems to fix it, but I'll look more closely and send a
version with a regression test on Monday.
Here it is. Added to open items.
--
Thomas Munro
http://www.enterprisedb.com
Attachments:
copy-with-indexes-and-transitions-v2.patchapplication/octet-stream; name=copy-with-indexes-and-transitions-v2.patchDownload
From 214af32e9cbd942ba9b858beaa6779c2b61d352b Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@enterprisedb.com>
Date: Mon, 10 Jul 2017 10:45:10 +1200
Subject: [PATCH] Fix COPY's handling of transition tables with indexes.
Commit c46c0e5202e8cfe750c6629db7852fdb15d528f3 failed to pass the
TransitionCaptureState object to ExecARInsertTriggers() in the case
where it's using heap_multi_insert and there are indexes. Repair.
---
src/backend/commands/copy.c | 2 +-
src/test/regress/expected/triggers.out | 7 ++++++-
src/test/regress/sql/triggers.sql | 7 +++++++
3 files changed, 14 insertions(+), 2 deletions(-)
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f391828e74f..fc5f4f66ead 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2915,7 +2915,7 @@ CopyFromInsertBatch(CopyState cstate, EState *estate, CommandId mycid,
estate, false, NULL, NIL);
ExecARInsertTriggers(estate, resultRelInfo,
bufferedTuples[i],
- recheckIndexes, NULL);
+ recheckIndexes, cstate->transition_capture);
list_free(recheckIndexes);
}
}
diff --git a/src/test/regress/expected/triggers.out b/src/test/regress/expected/triggers.out
index aaee30219ad..ac132b042d5 100644
--- a/src/test/regress/expected/triggers.out
+++ b/src/test/regress/expected/triggers.out
@@ -2156,6 +2156,11 @@ NOTICE: trigger = child3_delete_trig, old table = (CCC,42,foo)
-- are really inserted into the parent)
copy parent (a, b) from stdin;
NOTICE: trigger = parent_insert_trig, new table = (AAA,42), (BBB,42), (CCC,42)
+-- same behavior for copy if there is an index (interesting because rows are
+-- captured by a different code path in copy.c if there are indexes)
+create index on parent(b);
+copy parent (a, b) from stdin;
+NOTICE: trigger = parent_insert_trig, new table = (DDD,42)
-- DML affecting parent sees tuples collected from children even if
-- there is no transition table trigger on the children
drop trigger child1_insert_trig on child1;
@@ -2168,7 +2173,7 @@ drop trigger child3_insert_trig on child3;
drop trigger child3_update_trig on child3;
drop trigger child3_delete_trig on child3;
delete from parent;
-NOTICE: trigger = parent_delete_trig, old table = (AAA,42), (BBB,42), (CCC,42)
+NOTICE: trigger = parent_delete_trig, old table = (AAA,42), (BBB,42), (CCC,42), (DDD,42)
drop table child1, child2, child3, parent;
--
-- Verify prohibition of row triggers with transition triggers on
diff --git a/src/test/regress/sql/triggers.sql b/src/test/regress/sql/triggers.sql
index 659a5a1422b..b10159a1cf2 100644
--- a/src/test/regress/sql/triggers.sql
+++ b/src/test/regress/sql/triggers.sql
@@ -1661,6 +1661,13 @@ BBB 42
CCC 42
\.
+-- same behavior for copy if there is an index (interesting because rows are
+-- captured by a different code path in copy.c if there are indexes)
+create index on parent(b);
+copy parent (a, b) from stdin;
+DDD 42
+\.
+
-- DML affecting parent sees tuples collected from children even if
-- there is no transition table trigger on the children
drop trigger child1_insert_trig on child1;
--
2.13.2
"Thomas" == Thomas Munro <thomas.munro@enterprisedb.com> writes:
Thomas> Here it is. Added to open items.
On it.
--
Andrew (irc:RhodiumToad)
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
"Andrew" == Andrew Gierth <andrew@tao11.riddles.org.uk> writes:
"Thomas" == Thomas Munro <thomas.munro@enterprisedb.com> writes:
Thomas> Here it is. Added to open items.
Andrew> On it.
Committed.
--
Andrew (irc:RhodiumToad)
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers