Update does not move row across foreign partitions in v11

Started by Derek Hansabout 7 years ago33 messageshackersgeneral
Jump to latest
#1Derek Hans
derek.hans@gmail.com
hackersgeneral

I've set up 2 instances of PostgreSQL 11. On instance A, I created a table
with 2 local partitions and 2 partitions on instance B using foreign data
wrappers, following https://pgdash.io/blog/postgres-11-sharding.html.
Inserting rows into this table works as expected, with rows ending up in
the appropriate partition. However, updating those rows only moves them
across partitions in some of the situations:

- From local partition to local partition
- From local partition to foreign partition

Rows are not moved

- From foreign partition to local partition
- From foreign partition to foreign partition

Is this the expected behavior? Am I missing something or configured
something incorrectly?

Thanks,
Derek

#2Derek Hans
derek.hans@gmail.com
In reply to: Derek Hans (#1)
hackersgeneral
Re: Update does not move row across foreign partitions in v11

Hi all,
This behavior makes the new data sharding functionality in v11 only
marginally useful as you can't shard across database instances.
Considering data sharding appeared to be one of the key improvements in
v11, I'm confused - am I misunderstanding the expected functionality?

Thanks!

On Fri, Feb 22, 2019 at 9:44 AM Derek Hans <derek.hans@gmail.com> wrote:

I've set up 2 instances of PostgreSQL 11. On instance A, I created a table
with 2 local partitions and 2 partitions on instance B using foreign data
wrappers, following https://pgdash.io/blog/postgres-11-sharding.html.
Inserting rows into this table works as expected, with rows ending up in
the appropriate partition. However, updating those rows only moves them
across partitions in some of the situations:

- From local partition to local partition
- From local partition to foreign partition

Rows are not moved

- From foreign partition to local partition
- From foreign partition to foreign partition

Is this the expected behavior? Am I missing something or configured
something incorrectly?

Thanks,
Derek

-- 
*Derek*
+1 (415) 754-0519 | derek.hans@gmail.com | Skype: derek.hans
#3Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Derek Hans (#1)
hackersgeneral
Re: Update does not move row across foreign partitions in v11

On 2019-Feb-22, Derek Hans wrote:

I've set up 2 instances of PostgreSQL 11. On instance A, I created a table
with 2 local partitions and 2 partitions on instance B using foreign data
wrappers, following https://pgdash.io/blog/postgres-11-sharding.html.
Inserting rows into this table works as expected, with rows ending up in
the appropriate partition. However, updating those rows only moves them
across partitions in some of the situations:

- From local partition to local partition
- From local partition to foreign partition

Rows are not moved

- From foreign partition to local partition
- From foreign partition to foreign partition

Is this the expected behavior? Am I missing something or configured
something incorrectly?

Sounds like a bug to me.

--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#4Derek Hans
derek.hans@gmail.com
In reply to: Alvaro Herrera (#3)
hackersgeneral
Re: Update does not move row across foreign partitions in v11

Based on a reply to reporting this as a bug, moving rows out of foreign
partitions is not yet implemented so this is behaving as expected. There's
a mention of this limitation in the Notes section of the Update docs.

On Wed, Feb 27, 2019 at 6:12 PM Alvaro Herrera <alvherre@2ndquadrant.com>
wrote:

On 2019-Feb-22, Derek Hans wrote:

I've set up 2 instances of PostgreSQL 11. On instance A, I created a

table

with 2 local partitions and 2 partitions on instance B using foreign data
wrappers, following https://pgdash.io/blog/postgres-11-sharding.html.
Inserting rows into this table works as expected, with rows ending up in
the appropriate partition. However, updating those rows only moves them
across partitions in some of the situations:

- From local partition to local partition
- From local partition to foreign partition

Rows are not moved

- From foreign partition to local partition
- From foreign partition to foreign partition

Is this the expected behavior? Am I missing something or configured
something incorrectly?

Sounds like a bug to me.

--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

-- 
*Derek*
+1 (415) 754-0519 | derek.hans@gmail.com | Skype: derek.hans
#5David Rowley
dgrowleyml@gmail.com
In reply to: Derek Hans (#4)
hackersgeneral
Re: Update does not move row across foreign partitions in v11

On Tue, 5 Mar 2019 at 03:01, Derek Hans <derek.hans@gmail.com> wrote:

Based on a reply to reporting this as a bug, moving rows out of foreign partitions is not yet implemented so this is behaving as expected. There's a mention of this limitation in the Notes section of the Update docs.

(Moving this discussion to -Hackers)

In [1]/messages/by-id/CAGrP7a3Xc1Qy_B2WJcgAD8uQTS_NDcJn06O5mtS_Ne1nYhBsyw@mail.gmail.com, Derek reports that once a row is inserted into a foreign
partition that an UPDATE does not correctly route it back out into the
correct partition.

I didn't really follow the foreign partition code when it went in, but
do recall being involved in the documentation about the limitations of
partitioned tables in table 5.10.2.3 in [2]https://www.postgresql.org/docs/devel/ddl-partitioning.html#DDL-PARTITIONING-DECLARATIVE-LIMITATIONS. Unfortunately, table
5.10.2.3 does not seem to mention this limitation at all. As Derek
mentions, there is a brief mention in [3]https://www.postgresql.org/docs/devel/sql-update.html in the form of:

"Currently, rows cannot be moved from a partition that is a foreign
table to some other partition, but they can be moved into a foreign
table if the foreign data wrapper supports it."

I don't quite understand what a "foreign table to some other
partition" is meant to mean. Partitions don't have foreign tables,
they can only be one themselves.

I've tried to put all this right again in the attached. However, I was
a bit unsure of what "but they can be moved into a foreign table if
the foreign data wrapper supports it." is referring to. Copying Robert
and Etsuro as this was all added in 3d956d9562aa. Hopefully, they can
confirm what is meant by this.

[1]: /messages/by-id/CAGrP7a3Xc1Qy_B2WJcgAD8uQTS_NDcJn06O5mtS_Ne1nYhBsyw@mail.gmail.com
[2]: https://www.postgresql.org/docs/devel/ddl-partitioning.html#DDL-PARTITIONING-DECLARATIVE-LIMITATIONS
[3]: https://www.postgresql.org/docs/devel/sql-update.html

--
David Rowley http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

Attachments:

doc_confirm_foreign_partition_limitations.patchapplication/octet-stream; name=doc_confirm_foreign_partition_limitations.patchDownload+13-3
#6Amit Langote
Langote_Amit_f8@lab.ntt.co.jp
In reply to: David Rowley (#5)
hackersgeneral
Re: Update does not move row across foreign partitions in v11

Hi David,

On 2019/03/06 11:06, David Rowley wrote:

On Tue, 5 Mar 2019 at 03:01, Derek Hans <derek.hans@gmail.com> wrote:

Based on a reply to reporting this as a bug, moving rows out of foreign partitions is not yet implemented so this is behaving as expected. There's a mention of this limitation in the Notes section of the Update docs.

(Moving this discussion to -Hackers)

In [1], Derek reports that once a row is inserted into a foreign
partition that an UPDATE does not correctly route it back out into the
correct partition.

I didn't really follow the foreign partition code when it went in, but
do recall being involved in the documentation about the limitations of
partitioned tables in table 5.10.2.3 in [2]. Unfortunately, table
5.10.2.3 does not seem to mention this limitation at all. As Derek
mentions, there is a brief mention in [3] in the form of:

"Currently, rows cannot be moved from a partition that is a foreign
table to some other partition, but they can be moved into a foreign
table if the foreign data wrapper supports it."

I don't quite understand what a "foreign table to some other
partition" is meant to mean. Partitions don't have foreign tables,
they can only be one themselves.

I've tried to put all this right again in the attached. However, I was
a bit unsure of what "but they can be moved into a foreign table if
the foreign data wrapper supports it." is referring to. Copying Robert
and Etsuro as this was all added in 3d956d9562aa. Hopefully, they can
confirm what is meant by this.

Did you miss my reply on that thread?

/messages/by-id/CA+HiwqF3gma5HfCJb4_cOk0_+LEpVc57EHdBfz_EKt+Nu0hNYg@mail.gmail.com

Thanks,
Amit

#7David Rowley
dgrowleyml@gmail.com
In reply to: Amit Langote (#6)
hackersgeneral
Re: Update does not move row across foreign partitions in v11

On Wed, 6 Mar 2019 at 15:26, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp> wrote:

I've tried to put all this right again in the attached. However, I was
a bit unsure of what "but they can be moved into a foreign table if
the foreign data wrapper supports it." is referring to. Copying Robert
and Etsuro as this was all added in 3d956d9562aa. Hopefully, they can
confirm what is meant by this.

Did you miss my reply on that thread?

/messages/by-id/CA+HiwqF3gma5HfCJb4_cOk0_+LEpVc57EHdBfz_EKt+Nu0hNYg@mail.gmail.com

Yes. I wasn't aware that there were two threads for this.

--
David Rowley http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

#8Amit Langote
Langote_Amit_f8@lab.ntt.co.jp
In reply to: David Rowley (#7)
hackersgeneral
Re: Update does not move row across foreign partitions in v11

On 2019/03/06 11:29, David Rowley wrote:

On Wed, 6 Mar 2019 at 15:26, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp> wrote:

I've tried to put all this right again in the attached. However, I was
a bit unsure of what "but they can be moved into a foreign table if
the foreign data wrapper supports it." is referring to. Copying Robert
and Etsuro as this was all added in 3d956d9562aa. Hopefully, they can
confirm what is meant by this.

Did you miss my reply on that thread?

/messages/by-id/CA+HiwqF3gma5HfCJb4_cOk0_+LEpVc57EHdBfz_EKt+Nu0hNYg@mail.gmail.com

Yes. I wasn't aware that there were two threads for this.

Ah, indeed. In the documentation fix patch I'd posted, I also made
changes to release-11.sgml to link to the limitations section. (I'm
attaching it here for your reference.)

Thanks,
Amit

Attachments:

document-update-row-movement-limitation.patchtext/plain; charset=UTF-8; name=document-update-row-movement-limitation.patchDownload+14-0
#9Etsuro Fujita
fujita.etsuro@lab.ntt.co.jp
In reply to: David Rowley (#5)
hackersgeneral
Re: Update does not move row across foreign partitions in v11

(2019/03/06 11:06), David Rowley wrote:

On Tue, 5 Mar 2019 at 03:01, Derek Hans<derek.hans@gmail.com> wrote:

Based on a reply to reporting this as a bug, moving rows out of foreign partitions is not yet implemented so this is behaving as expected. There's a mention of this limitation in the Notes section of the Update docs.

(Moving this discussion to -Hackers)

In [1], Derek reports that once a row is inserted into a foreign
partition that an UPDATE does not correctly route it back out into the
correct partition.

I didn't really follow the foreign partition code when it went in, but
do recall being involved in the documentation about the limitations of
partitioned tables in table 5.10.2.3 in [2]. Unfortunately, table
5.10.2.3 does not seem to mention this limitation at all. As Derek
mentions, there is a brief mention in [3] in the form of:

"Currently, rows cannot be moved from a partition that is a foreign
table to some other partition, but they can be moved into a foreign
table if the foreign data wrapper supports it."

I don't quite understand what a "foreign table to some other
partition" is meant to mean. Partitions don't have foreign tables,
they can only be one themselves.

I think "foreign table" is describing "partition" in front of that; "a
partition that is a foreign table".

I've tried to put all this right again in the attached. However, I was
a bit unsure of what "but they can be moved into a foreign table if
the foreign data wrapper supports it." is referring to. Copying Robert
and Etsuro as this was all added in 3d956d9562aa. Hopefully, they can
confirm what is meant by this.

That means that rows can be moved from a local partition to a foreign
partition if the FDW supports it.

IMO, I think the existing mention in [3] is good, so I would vote for
putting the same mention in table 5.10.2.3 in [2] as well.

Best regards,
Etsuro Fujita

#10David Rowley
dgrowleyml@gmail.com
In reply to: Etsuro Fujita (#9)
hackersgeneral
Re: Update does not move row across foreign partitions in v11

On Wed, 6 Mar 2019 at 16:29, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp> wrote:

(2019/03/06 11:06), David Rowley wrote:

I don't quite understand what a "foreign table to some other
partition" is meant to mean. Partitions don't have foreign tables,
they can only be one themselves.

I think "foreign table" is describing "partition" in front of that; "a
partition that is a foreign table".

I think I was reading this wrong:

- Currently, rows cannot be moved from a partition that is a
- foreign table to some other partition, but they can be moved into a foreign
- table if the foreign data wrapper supports it.

I parsed it as "cannot be moved from a partition, that is a foreign
table to some other partition"

and subsequently struggled with what "a foreign table to some other
partition" is.

but now looking at it, I think it's meant to mean:

"cannot be moved from a foreign table partition to another partition"

I've tried to put all this right again in the attached. However, I was
a bit unsure of what "but they can be moved into a foreign table if
the foreign data wrapper supports it." is referring to. Copying Robert
and Etsuro as this was all added in 3d956d9562aa. Hopefully, they can
confirm what is meant by this.

That means that rows can be moved from a local partition to a foreign
partition if the FDW supports it.

It seems a bit light on detail to me. If I was a user I'd want to know
what exactly the FDW needed to support this. Does it need a special
partition move function? Looking at ExecFindPartition(), this check
seems to be done in CheckValidResultRel() and is basically:

case RELKIND_FOREIGN_TABLE:
/* Okay only if the FDW supports it */
fdwroutine = resultRelInfo->ri_FdwRoutine;
switch (operation)
{
case CMD_INSERT:
if (fdwroutine->ExecForeignInsert == NULL)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot insert into foreign table \"%s\"",
RelationGetRelationName(resultRel))));

Alternatively, we could just remove the mention about "if the FDW
supports it", since it's probably unlikely for an FDW not to support
INSERT.

IMO, I think the existing mention in [3] is good, so I would vote for
putting the same mention in table 5.10.2.3 in [2] as well.

I think the sentence is unclear, at least I struggled to parse it the
first time. Happy for Amit to choose some better words and include in
his patch. I think it should be done in the same commit.

--
David Rowley http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

#11Etsuro Fujita
fujita.etsuro@lab.ntt.co.jp
In reply to: Amit Langote (#8)
hackersgeneral
Re: Update does not move row across foreign partitions in v11

(2019/03/06 11:34), Amit Langote wrote:

Ah, indeed. In the documentation fix patch I'd posted, I also made
changes to release-11.sgml to link to the limitations section. (I'm
attaching it here for your reference.)

I'm not sure it's a good idea to make changes to the release notes like
that, because 1) that would make the release notes verbose, and 2) it
might end up doing the same thing to items that have some limitations in
the existing/future release notes (eg, FOR EACH ROW triggers on
partitioned tables added to V11 has the limitation listed on the
limitation section, so the same link would be needed.), for consistency.

Best regards,
Etsuro Fujita

#12Amit Langote
Langote_Amit_f8@lab.ntt.co.jp
In reply to: Etsuro Fujita (#11)
hackersgeneral
Re: Update does not move row across foreign partitions in v11

Fujita-san,

On 2019/03/06 13:04, Etsuro Fujita wrote:

(2019/03/06 11:34), Amit Langote wrote:

Ah, indeed.  In the documentation fix patch I'd posted, I also made
changes to release-11.sgml to link to the limitations section.  (I'm
attaching it here for your reference.)

I'm not sure it's a good idea to make changes to the release notes like
that, because 1) that would make the release notes verbose, and 2) it
might end up doing the same thing to items that have some limitations in
the existing/future release notes (eg, FOR EACH ROW triggers on
partitioned tables added to V11 has the limitation listed on the
limitation section, so the same link would be needed.), for consistency.

OK, sure. It just seemed to me that the original complainer found it
quite a bit surprising that such a limitation is not mentioned in the
release notes, but maybe that's fine. It seems we don't normally list
feature limitations in the release notes, which as you rightly say, would
make them verbose.

The main problem here is indeed that the limitation is not listed under
the partitioning limitations in ddl.sgml, where it's easier to notice than
in the UPDATE's page. I've updated my patch to remove the release-11.sgml
changes.

Thanks,
Amit

Attachments:

document-update-row-movement-limitation.patchtext/plain; charset=UTF-8; name=document-update-row-movement-limitation.patchDownload+7-0
#13Amit Langote
Langote_Amit_f8@lab.ntt.co.jp
In reply to: David Rowley (#10)
hackersgeneral
Re: Update does not move row across foreign partitions in v11

On 2019/03/06 12:47, David Rowley wrote:

On Wed, 6 Mar 2019 at 16:29, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp> wrote:

That means that rows can be moved from a local partition to a foreign
partition if the FDW supports it.

It seems a bit light on detail to me. If I was a user I'd want to know
what exactly the FDW needed to support this. Does it need a special
partition move function? Looking at ExecFindPartition(), this check
seems to be done in CheckValidResultRel() and is basically:

case RELKIND_FOREIGN_TABLE:
/* Okay only if the FDW supports it */
fdwroutine = resultRelInfo->ri_FdwRoutine;
switch (operation)
{
case CMD_INSERT:
if (fdwroutine->ExecForeignInsert == NULL)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot insert into foreign table \"%s\"",
RelationGetRelationName(resultRel))));

Alternatively, we could just remove the mention about "if the FDW
supports it", since it's probably unlikely for an FDW not to support
INSERT.

AFAIK, there's no special support in FDWs for "tuple moving" as such. The
"if the FDW supports it" refers to the FDW's ability to handle tuple
routing. Note that moving/re-routing involves calling
ExecPrepareTupleRouting followed by ExecInsert on the new tupls after the
old tuple is deleted. If an FDW doesn't support tuple routing, then a
tuple cannot be moved into it. That's what that text is talking about.

Maybe, we should reword it as "if the FDW supports tuple routing", so that
a reader doesn't go looking around for "tuple moving support" in FDWs.

Thanks,
Amit

#14David Rowley
dgrowleyml@gmail.com
In reply to: Amit Langote (#13)
hackersgeneral
Re: Update does not move row across foreign partitions in v11

On Wed, 6 Mar 2019 at 17:20, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp> wrote:

On 2019/03/06 12:47, David Rowley wrote:

It seems a bit light on detail to me. If I was a user I'd want to know
what exactly the FDW needed to support this. Does it need a special
partition move function? Looking at ExecFindPartition(), this check
seems to be done in CheckValidResultRel() and is basically:

case RELKIND_FOREIGN_TABLE:
/* Okay only if the FDW supports it */
fdwroutine = resultRelInfo->ri_FdwRoutine;
switch (operation)
{
case CMD_INSERT:
if (fdwroutine->ExecForeignInsert == NULL)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot insert into foreign table \"%s\"",
RelationGetRelationName(resultRel))));

Alternatively, we could just remove the mention about "if the FDW
supports it", since it's probably unlikely for an FDW not to support
INSERT.

AFAIK, there's no special support in FDWs for "tuple moving" as such. The
"if the FDW supports it" refers to the FDW's ability to handle tuple
routing. Note that moving/re-routing involves calling
ExecPrepareTupleRouting followed by ExecInsert on the new tupls after the
old tuple is deleted. If an FDW doesn't support tuple routing, then a
tuple cannot be moved into it. That's what that text is talking about.

Maybe, we should reword it as "if the FDW supports tuple routing", so that
a reader doesn't go looking around for "tuple moving support" in FDWs.

I think you missed my point. If there's no special support for "tuple
moving", as you say, then what help is it to tell the user "if the FDW
supports tuple routing"? The answer is, it's not any help. How would
the user check such a fact?

As far as I can tell, this is just the requirements as defined in
CheckValidResultRel() for CMD_INSERT. Fragments of which I pasted
above.

--
David Rowley http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

#15Amit Langote
Langote_Amit_f8@lab.ntt.co.jp
In reply to: David Rowley (#14)
hackersgeneral
Re: Update does not move row across foreign partitions in v11

On 2019/03/06 13:30, David Rowley wrote:

On Wed, 6 Mar 2019 at 17:20, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp> wrote:

On 2019/03/06 12:47, David Rowley wrote:

It seems a bit light on detail to me. If I was a user I'd want to know
what exactly the FDW needed to support this. Does it need a special
partition move function? Looking at ExecFindPartition(), this check
seems to be done in CheckValidResultRel() and is basically:

case RELKIND_FOREIGN_TABLE:
/* Okay only if the FDW supports it */
fdwroutine = resultRelInfo->ri_FdwRoutine;
switch (operation)
{
case CMD_INSERT:
if (fdwroutine->ExecForeignInsert == NULL)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot insert into foreign table \"%s\"",
RelationGetRelationName(resultRel))));

Alternatively, we could just remove the mention about "if the FDW
supports it", since it's probably unlikely for an FDW not to support
INSERT.

AFAIK, there's no special support in FDWs for "tuple moving" as such. The
"if the FDW supports it" refers to the FDW's ability to handle tuple
routing. Note that moving/re-routing involves calling
ExecPrepareTupleRouting followed by ExecInsert on the new tupls after the
old tuple is deleted. If an FDW doesn't support tuple routing, then a
tuple cannot be moved into it. That's what that text is talking about.

Maybe, we should reword it as "if the FDW supports tuple routing", so that
a reader doesn't go looking around for "tuple moving support" in FDWs.

I think you missed my point. If there's no special support for "tuple
moving", as you say, then what help is it to tell the user "if the FDW
supports tuple routing"? The answer is, it's not any help. How would
the user check such a fact?

Hmm, maybe getting the following error, like one would get in PG 10 when
using postgres_fdw-managed partitions:

ERROR: cannot route inserted tuples to a foreign table

Getting the above error is perhaps not the best way for a user to learn of
this fact, but maybe we (and hopefully other FDW authors) mention this in
the documentation?

As far as I can tell, this is just the requirements as defined in
CheckValidResultRel() for CMD_INSERT. Fragments of which I pasted
above.

Only supporting INSERT doesn't suffice though. An FDW which intends to
support tuple routing and hence 1-way tuple moving needs to updated like
postgres_fdw was in PG 11.

Thanks,
Amit

#16Etsuro Fujita
fujita.etsuro@lab.ntt.co.jp
In reply to: Amit Langote (#12)
hackersgeneral
Re: Update does not move row across foreign partitions in v11

(2019/03/06 13:18), Amit Langote wrote:

The main problem here is indeed that the limitation is not listed under
the partitioning limitations in ddl.sgml, where it's easier to notice than
in the UPDATE's page.

Agreed.

I've updated my patch to remove the release-11.sgml
changes.

Thanks for the updated patch!

--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -3376,6 +3376,13 @@ ALTER TABLE measurement ATTACH PARTITION 
measurement_y2008m02
        </para>
       </listitem>
+     <listitem>
+      <para>
+       <command>UPDATE</command> row movement is not supported in the cases
+       where the old row is contained in a foreign table partition.
+      </para>
+     </listitem>

ISTM that it's also a limitation that rows can be moved from a local
partition to a foreign partition *if the FDW support tuple routing*, so
I would vote for mentioning that as well here.

Best regards,
Etsuro Fujita

#17Etsuro Fujita
fujita.etsuro@lab.ntt.co.jp
In reply to: Amit Langote (#15)
hackersgeneral
Re: Update does not move row across foreign partitions in v11

(2019/03/06 13:53), Amit Langote wrote:

On 2019/03/06 13:30, David Rowley wrote:

I think you missed my point. If there's no special support for "tuple
moving", as you say, then what help is it to tell the user "if the FDW
supports tuple routing"? The answer is, it's not any help. How would
the user check such a fact?

Hmm, maybe getting the following error, like one would get in PG 10 when
using postgres_fdw-managed partitions:

ERROR: cannot route inserted tuples to a foreign table

Getting the above error is perhaps not the best way for a user to learn of
this fact, but maybe we (and hopefully other FDW authors) mention this in
the documentation?

+1

As far as I can tell, this is just the requirements as defined in
CheckValidResultRel() for CMD_INSERT. Fragments of which I pasted
above.

Only supporting INSERT doesn't suffice though. An FDW which intends to
support tuple routing and hence 1-way tuple moving needs to updated like
postgres_fdw was in PG 11.

That's right; the "if the FDW supports it" in the documentation refers
to the FDW's support for the callback functions BeginForeignInsert() and
EndForeignInsert() described in 57.2.4. FDW Routines For Updating
Foreign Tables [1]https://www.postgresql.org/docs/current/fdw-callbacks.html#FDW-CALLBACKS-UPDATE in addition to ExecForeignInsert(), as stated there:

"Tuples inserted into a partitioned table by INSERT or COPY FROM are
routed to partitions. If an FDW supports routable foreign-table
partitions, it should also provide the following callback functions."

Best regards,
Etsuro Fujita

[1]: https://www.postgresql.org/docs/current/fdw-callbacks.html#FDW-CALLBACKS-UPDATE
https://www.postgresql.org/docs/current/fdw-callbacks.html#FDW-CALLBACKS-UPDATE

#18Amit Langote
Langote_Amit_f8@lab.ntt.co.jp
In reply to: Etsuro Fujita (#16)
hackersgeneral
Re: Update does not move row across foreign partitions in v11

Fujita-san,

On 2019/03/06 15:10, Etsuro Fujita wrote:

--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -3376,6 +3376,13 @@ ALTER TABLE measurement ATTACH PARTITION
measurement_y2008m02
       </para>
      </listitem>
+     <listitem>
+      <para>
+       <command>UPDATE</command> row movement is not supported in the cases
+       where the old row is contained in a foreign table partition.
+      </para>
+     </listitem>

ISTM that it's also a limitation that rows can be moved from a local
partition to a foreign partition *if the FDW support tuple routing*, so I
would vote for mentioning that as well here.

Thanks for checking.

I have updated the patch to include a line about this in the same
paragraph, because maybe we don't need to make a new <listitem> for it.

Thanks,
Amit

Attachments:

document-update-row-movement-limitation-v2.patchtext/plain; charset=UTF-8; name=document-update-row-movement-limitation-v2.patchDownload+9-0
#19Etsuro Fujita
fujita.etsuro@lab.ntt.co.jp
In reply to: Amit Langote (#18)
hackersgeneral
Re: Update does not move row across foreign partitions in v11

(2019/03/06 15:34), Amit Langote wrote:

On 2019/03/06 15:10, Etsuro Fujita wrote:

--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -3376,6 +3376,13 @@ ALTER TABLE measurement ATTACH PARTITION
measurement_y2008m02
</para>
</listitem>
+<listitem>
+<para>
+<command>UPDATE</command>  row movement is not supported in the cases
+       where the old row is contained in a foreign table partition.
+</para>
+</listitem>

ISTM that it's also a limitation that rows can be moved from a local
partition to a foreign partition *if the FDW support tuple routing*, so I
would vote for mentioning that as well here.

Thanks for checking.

I have updated the patch to include a line about this in the same
paragraph, because maybe we don't need to make a new<listitem> for it.

Thanks for the patch!

The patch looks good to me, but one thing I'm wondering is: as suggested
by David, it would be better to rephrase this mention in the UPDATE
reference page, in a single commit:

"Currently, rows cannot be moved from a partition that is a foreign
table to some other partition, but they can be moved into a foreign
table if the foreign data wrapper supports it."

I don't think it needs to be completely rephrased; it's enough for me to
rewrite it to something like this:

"Currently, rows cannot be moved from a foreign-table partition to some
other partition, but they can be moved into a foreign-table partition if
the foreign data wrapper supports tuple routing."

And to make maintenance work easy, I think it might be better to just
put this on the limitations section of 5.10. Table Partitioning. What
do you think about that?

Best regards,
Etsuro Fujita

#20Robert Haas
robertmhaas@gmail.com
In reply to: Etsuro Fujita (#19)
hackersgeneral
Re: Update does not move row across foreign partitions in v11

On Thu, Mar 7, 2019 at 7:35 AM Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:

Thanks for the patch!

The patch looks good to me, but one thing I'm wondering is: as suggested
by David, it would be better to rephrase this mention in the UPDATE
reference page, in a single commit:

"Currently, rows cannot be moved from a partition that is a foreign
table to some other partition, but they can be moved into a foreign
table if the foreign data wrapper supports it."

I don't think it needs to be completely rephrased; it's enough for me to
rewrite it to something like this:

"Currently, rows cannot be moved from a foreign-table partition to some
other partition, but they can be moved into a foreign-table partition if
the foreign data wrapper supports tuple routing."

I prefer David's wording.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#21Amit Langote
Langote_Amit_f8@lab.ntt.co.jp
In reply to: Robert Haas (#20)
hackersgeneral
#22Amit Langote
Langote_Amit_f8@lab.ntt.co.jp
In reply to: Etsuro Fujita (#19)
hackersgeneral
#23David Rowley
dgrowleyml@gmail.com
In reply to: Amit Langote (#22)
hackersgeneral
#24Etsuro Fujita
fujita.etsuro@lab.ntt.co.jp
In reply to: David Rowley (#23)
hackersgeneral
#25David Rowley
dgrowleyml@gmail.com
In reply to: Etsuro Fujita (#24)
hackersgeneral
#26Amit Langote
Langote_Amit_f8@lab.ntt.co.jp
In reply to: David Rowley (#25)
hackersgeneral
#27Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Amit Langote (#26)
hackersgeneral
#28Amit Langote
Langote_Amit_f8@lab.ntt.co.jp
In reply to: Alvaro Herrera (#27)
hackersgeneral
#29Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Amit Langote (#28)
hackersgeneral
#30Amit Langote
Langote_Amit_f8@lab.ntt.co.jp
In reply to: Alvaro Herrera (#29)
hackersgeneral
#31Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Amit Langote (#30)
hackersgeneral
#32Derek Hans
derek.hans@gmail.com
In reply to: Alvaro Herrera (#31)
hackersgeneral
#33Amit Langote
Langote_Amit_f8@lab.ntt.co.jp
In reply to: Alvaro Herrera (#31)
hackersgeneral