Re: Report error position in partition bound check
On 2 July 2020, at 06:39, Daniel Gustafsson <daniel@yesql.se> wrote:
On 10 Apr 2020, at 23:50, Alexandra Wang <lewang@pivotal.io> wrote:
On Fri, Apr 10, 2020 at 8:37 AM Ashutosh Bapat <
ashutosh.bapat@2ndquadrant.com <mailto:ashutosh.bapat@2ndquadrant.com>>
wrote:
for a multi-key value the ^
points to the first column and the reader may think that that's the
problematci column. Should it instead point to ( ?I attached a v2 of Amit's 0002 patch to also report the exact column
for the partition overlap errors.This patch fails to apply to HEAD due to conflicts in the create_table
expected
output. Can you please submit a rebased version? I'm marking the CF
entry
Waiting on Author in the meantime.
Thank you Daniel. Here's the rebased patch. I also squashed the two
patches into one so it's easier to review.
--
*Alexandra Wang*
Attachments:
v3-0001-Improve-check-new-partition-bound-error-position-rep.patchapplication/x-patch; name=v3-0001-Improve-check-new-partition-bound-error-position-rep.patchDownload
From 572f9086fc9d6e3e7cf1e58c3a6ee21dd8cd9f1b Mon Sep 17 00:00:00 2001
From: Alexandra Wang <walexandra@vmware.com>
Date: Fri, 10 Jul 2020 10:28:49 -0700
Subject: [PATCH] Improve check new partition bound error position report
We have been passing a dummy ParseState to ereport(). Without the source
text in the ParseState ereport does not report the error position even
if a error location is supplied. This patch passes a ParseState to
check_new_partition_bound() when it is available.
-- Create parent table
create table foo (a int, b int, c date) partition by range (b,c);
-- Before:
create table foo_part_1 partition of foo for values from (1, date '2007-01-01') to (1, date '2006-01-01');
ERROR: empty range bound specified for partition "foo_part_1"
DETAIL: Specified lower bound (1, '2007-01-01') is greater than or equal to upper bound (1, '2006-01-01').
-- After:
create table foo_part_1 partition of foo for values from (1, date '2007-01-01') to (1, date '2006-01-01');
ERROR: empty range bound specified for partition "foo_part_1"
LINE 1: ...e foo_part_1 partition of foo for values from (1, date '2007...
^
DETAIL: Specified lower bound (1, '2007-01-01') is greater than or equal to upper bound (1, '2006-01-01').
Co-authored-by: Ashwin Agrawal <aashwin@vmware.com>
Co-authored-by: Amit Langote <amitlangote09@gmail.com>
---
src/backend/commands/tablecmds.c | 15 ++++--
src/backend/parser/parse_utilcmd.c | 3 ++
src/backend/partitioning/partbounds.c | 63 ++++++++++++++--------
src/include/partitioning/partbounds.h | 4 +-
src/test/regress/expected/alter_table.out | 10 ++++
src/test/regress/expected/create_table.out | 30 +++++++++++
6 files changed, 98 insertions(+), 27 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 26700da278..8de7473ba7 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -541,7 +541,8 @@ static void ComputePartitionAttrs(ParseState *pstate, Relation rel, List *partPa
static void CreateInheritance(Relation child_rel, Relation parent_rel);
static void RemoveInheritance(Relation child_rel, Relation parent_rel);
static ObjectAddress ATExecAttachPartition(List **wqueue, Relation rel,
- PartitionCmd *cmd);
+ PartitionCmd *cmd,
+ AlterTableUtilityContext * context);
static void AttachPartitionEnsureIndexes(Relation rel, Relation attachrel);
static void QueuePartitionConstraintValidation(List **wqueue, Relation scanrel,
List *partConstraint,
@@ -1005,7 +1006,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
* Check first that the new partition's bound is valid and does not
* overlap with any of existing partitions of the parent.
*/
- check_new_partition_bound(relname, parent, bound);
+ check_new_partition_bound(relname, parent, bound, pstate);
/*
* If the default partition exists, its partition constraints will
@@ -4646,7 +4647,8 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
cur_pass, context);
Assert(cmd != NULL);
if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
- ATExecAttachPartition(wqueue, rel, (PartitionCmd *) cmd->def);
+ ATExecAttachPartition(wqueue, rel, (PartitionCmd *) cmd->def,
+ context);
else
ATExecAttachPartitionIdx(wqueue, rel,
((PartitionCmd *) cmd->def)->name);
@@ -16186,7 +16188,8 @@ QueuePartitionConstraintValidation(List **wqueue, Relation scanrel,
* Return the address of the newly attached partition.
*/
static ObjectAddress
-ATExecAttachPartition(List **wqueue, Relation rel, PartitionCmd *cmd)
+ATExecAttachPartition(List **wqueue, Relation rel, PartitionCmd *cmd,
+ AlterTableUtilityContext * context)
{
Relation attachrel,
catalog;
@@ -16201,6 +16204,7 @@ ATExecAttachPartition(List **wqueue, Relation rel, PartitionCmd *cmd)
const char *trigger_name;
Oid defaultPartOid;
List *partBoundConstraint;
+ ParseState *pstate = make_parsestate(NULL);
/*
* We must lock the default partition if one exists, because attaching a
@@ -16365,8 +16369,9 @@ ATExecAttachPartition(List **wqueue, Relation rel, PartitionCmd *cmd)
* of existing partitions of the parent - note that it does not return on
* error.
*/
+ pstate->p_sourcetext = context->queryString;
check_new_partition_bound(RelationGetRelationName(attachrel), rel,
- cmd->bound);
+ cmd->bound, pstate);
/* OK to create inheritance. Rest of the checks performed there */
CreateInheritance(attachrel, rel);
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 0e4caa6ad4..6a9ee15520 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -4170,5 +4170,8 @@ transformPartitionBoundValue(ParseState *pstate, Node *val,
if (!IsA(value, Const))
elog(ERROR, "could not evaluate partition bound expression");
+ /* Preserve parser location information. */
+ ((Const *) value)->location = exprLocation(val);
+
return (Const *) value;
}
diff --git a/src/backend/partitioning/partbounds.c b/src/backend/partitioning/partbounds.c
index 7553d55987..3cade9e68c 100644
--- a/src/backend/partitioning/partbounds.c
+++ b/src/backend/partitioning/partbounds.c
@@ -223,7 +223,8 @@ static int32 partition_rbound_cmp(int partnatts, FmgrInfo *partsupfunc,
static int partition_range_bsearch(int partnatts, FmgrInfo *partsupfunc,
Oid *partcollation,
PartitionBoundInfo boundinfo,
- PartitionRangeBound *probe, bool *is_equal);
+ PartitionRangeBound *probe, bool *is_equal,
+ int32 *cmpval);
static int get_partition_bound_num_indexes(PartitionBoundInfo b);
static Expr *make_partition_op_expr(PartitionKey key, int keynum,
uint16 strategy, Expr *arg1, Expr *arg2);
@@ -2807,14 +2808,14 @@ partitions_are_ordered(PartitionBoundInfo boundinfo, int nparts)
*/
void
check_new_partition_bound(char *relname, Relation parent,
- PartitionBoundSpec *spec)
+ PartitionBoundSpec *spec, ParseState *pstate)
{
PartitionKey key = RelationGetPartitionKey(parent);
PartitionDesc partdesc = RelationGetPartitionDesc(parent);
PartitionBoundInfo boundinfo = partdesc->boundinfo;
- ParseState *pstate = make_parsestate(NULL);
int with = -1;
bool overlap = false;
+ int overlap_location = 0;
if (spec->is_default)
{
@@ -2909,6 +2910,7 @@ check_new_partition_bound(char *relname, Relation parent,
if (boundinfo->indexes[remainder] != -1)
{
overlap = true;
+ overlap_location = spec->location;
with = boundinfo->indexes[remainder];
break;
}
@@ -2937,6 +2939,7 @@ check_new_partition_bound(char *relname, Relation parent,
{
Const *val = castNode(Const, lfirst(cell));
+ overlap_location = val->location;
if (!val->constisnull)
{
int offset;
@@ -2970,6 +2973,7 @@ check_new_partition_bound(char *relname, Relation parent,
{
PartitionRangeBound *lower,
*upper;
+ int cmpval;
Assert(spec->strategy == PARTITION_STRATEGY_RANGE);
lower = make_one_partition_rbound(key, -1, spec->lowerdatums, true);
@@ -2979,10 +2983,16 @@ check_new_partition_bound(char *relname, Relation parent,
* First check if the resulting range would be empty with
* specified lower and upper bounds
*/
- if (partition_rbound_cmp(key->partnatts, key->partsupfunc,
- key->partcollation, lower->datums,
- lower->kind, true, upper) >= 0)
+ cmpval = partition_rbound_cmp(key->partnatts,
+ key->partsupfunc,
+ key->partcollation, lower->datums,
+ lower->kind, true, upper);
+ if (cmpval >= 0)
{
+ /* Fetch the problem bound from lower datums list. */
+ PartitionRangeDatum *datum = list_nth(spec->lowerdatums,
+ cmpval - 1);
+
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("empty range bound specified for partition \"%s\"",
@@ -2990,7 +3000,7 @@ check_new_partition_bound(char *relname, Relation parent,
errdetail("Specified lower bound %s is greater than or equal to upper bound %s.",
get_range_partbound_string(spec->lowerdatums),
get_range_partbound_string(spec->upperdatums)),
- parser_errposition(pstate, spec->location)));
+ parser_errposition(pstate, datum->location)));
}
if (partdesc->nparts > 0)
@@ -3022,7 +3032,7 @@ check_new_partition_bound(char *relname, Relation parent,
key->partsupfunc,
key->partcollation,
boundinfo, lower,
- &equal);
+ &equal, &cmpval);
if (boundinfo->indexes[offset + 1] < 0)
{
@@ -3034,7 +3044,6 @@ check_new_partition_bound(char *relname, Relation parent,
*/
if (offset + 1 < boundinfo->ndatums)
{
- int32 cmpval;
Datum *datums;
PartitionRangeDatumKind *kind;
bool is_lower;
@@ -3056,6 +3065,8 @@ check_new_partition_bound(char *relname, Relation parent,
* offset + 2.
*/
overlap = true;
+ overlap_location = ((PartitionRangeDatum *)
+ list_nth(spec->upperdatums, -cmpval - 1))->location;
with = boundinfo->indexes[offset + 2];
}
}
@@ -3066,7 +3077,13 @@ check_new_partition_bound(char *relname, Relation parent,
* The new partition overlaps with the existing
* partition between offset and offset + 1.
*/
+ Datum *datum;
+
overlap = true;
+ Assert(cmpval >= 0);
+ datum = cmpval == 0 ? linitial(spec->lowerdatums):
+ list_nth(spec->lowerdatums, cmpval - 1);
+ overlap_location = ((PartitionRangeDatum *)datum)->location;
with = boundinfo->indexes[offset + 1];
}
}
@@ -3082,11 +3099,12 @@ check_new_partition_bound(char *relname, Relation parent,
if (overlap)
{
Assert(with >= 0);
+ Assert(overlap_location > 0);
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("partition \"%s\" would overlap partition \"%s\"",
relname, get_rel_name(partdesc->oids[with])),
- parser_errposition(pstate, spec->location)));
+ parser_errposition(pstate, overlap_location)));
}
}
@@ -3320,7 +3338,9 @@ make_one_partition_rbound(PartitionKey key, int index, List *datums, bool lower)
* partition_rbound_cmp
*
* Return for two range bounds whether the 1st one (specified in datums1,
- * kind1, and lower1) is <, =, or > the bound specified in *b2.
+ * kind1, and lower1) is <, =, or > the bound specified in *b2. 0 is returned if
+ * equal and the 1-based index of the first mismatching bound if unequal;
+ * multiplied by -1 if the 1st bound is smaller.
*
* partnatts, partsupfunc and partcollation give the number of attributes in the
* bounds to be compared, comparison function to be used and the collations of
@@ -3340,6 +3360,7 @@ partition_rbound_cmp(int partnatts, FmgrInfo *partsupfunc,
bool lower1, PartitionRangeBound *b2)
{
int32 cmpval = 0; /* placate compiler */
+ int result = 0;
int i;
Datum *datums2 = b2->datums;
PartitionRangeDatumKind *kind2 = b2->kind;
@@ -3347,6 +3368,8 @@ partition_rbound_cmp(int partnatts, FmgrInfo *partsupfunc,
for (i = 0; i < partnatts; i++)
{
+ result++;
+
/*
* First, handle cases where the column is unbounded, which should not
* invoke the comparison procedure, and should not consider any later
@@ -3354,9 +3377,9 @@ partition_rbound_cmp(int partnatts, FmgrInfo *partsupfunc,
* compare the same way as the values they represent.
*/
if (kind1[i] < kind2[i])
- return -1;
+ return -result;
else if (kind1[i] > kind2[i])
- return 1;
+ return result;
else if (kind1[i] != PARTITION_RANGE_DATUM_VALUE)
/*
@@ -3381,9 +3404,9 @@ partition_rbound_cmp(int partnatts, FmgrInfo *partsupfunc,
* two.
*/
if (cmpval == 0 && lower1 != lower2)
- cmpval = lower1 ? 1 : -1;
+ cmpval = lower1 ? result : -result;
- return cmpval;
+ return cmpval == 0 ? 0 : (cmpval < 0 ? -result : result);;
}
/*
@@ -3495,7 +3518,7 @@ static int
partition_range_bsearch(int partnatts, FmgrInfo *partsupfunc,
Oid *partcollation,
PartitionBoundInfo boundinfo,
- PartitionRangeBound *probe, bool *is_equal)
+ PartitionRangeBound *probe, bool *is_equal, int32 *cmpval)
{
int lo,
hi,
@@ -3505,19 +3528,17 @@ partition_range_bsearch(int partnatts, FmgrInfo *partsupfunc,
hi = boundinfo->ndatums - 1;
while (lo < hi)
{
- int32 cmpval;
-
mid = (lo + hi + 1) / 2;
- cmpval = partition_rbound_cmp(partnatts, partsupfunc,
+ *cmpval = partition_rbound_cmp(partnatts, partsupfunc,
partcollation,
boundinfo->datums[mid],
boundinfo->kind[mid],
(boundinfo->indexes[mid] == -1),
probe);
- if (cmpval <= 0)
+ if (*cmpval <= 0)
{
lo = mid;
- *is_equal = (cmpval == 0);
+ *is_equal = (*cmpval == 0);
if (*is_equal)
break;
diff --git a/src/include/partitioning/partbounds.h b/src/include/partitioning/partbounds.h
index dfc720720b..c82f77d02f 100644
--- a/src/include/partitioning/partbounds.h
+++ b/src/include/partitioning/partbounds.h
@@ -14,6 +14,7 @@
#include "fmgr.h"
#include "nodes/parsenodes.h"
#include "nodes/pg_list.h"
+#include "parser/parse_node.h"
#include "partitioning/partdefs.h"
#include "utils/relcache.h"
struct RelOptInfo; /* avoid including pathnodes.h here */
@@ -98,7 +99,8 @@ extern PartitionBoundInfo partition_bounds_merge(int partnatts,
List **inner_parts);
extern bool partitions_are_ordered(PartitionBoundInfo boundinfo, int nparts);
extern void check_new_partition_bound(char *relname, Relation parent,
- PartitionBoundSpec *spec);
+ PartitionBoundSpec *spec,
+ ParseState *pstate);
extern void check_default_partition_contents(Relation parent,
Relation defaultRel,
PartitionBoundSpec *new_spec);
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 002079601f..4fae37e022 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -3812,6 +3812,8 @@ SELECT conislocal, coninhcount FROM pg_constraint WHERE conrelid = 'part_1'::reg
CREATE TABLE fail_part (LIKE part_1 INCLUDING CONSTRAINTS);
ALTER TABLE list_parted ATTACH PARTITION fail_part FOR VALUES IN (1);
ERROR: partition "fail_part" would overlap partition "part_1"
+LINE 1: ...LE list_parted ATTACH PARTITION fail_part FOR VALUES IN (1);
+ ^
DROP TABLE fail_part;
-- check that an existing table can be attached as a default partition
CREATE TABLE def_part (LIKE list_parted INCLUDING CONSTRAINTS);
@@ -3821,6 +3823,8 @@ ALTER TABLE list_parted ATTACH PARTITION def_part DEFAULT;
CREATE TABLE fail_def_part (LIKE part_1 INCLUDING CONSTRAINTS);
ALTER TABLE list_parted ATTACH PARTITION fail_def_part DEFAULT;
ERROR: partition "fail_def_part" conflicts with existing default partition "def_part"
+LINE 1: ...ER TABLE list_parted ATTACH PARTITION fail_def_part DEFAULT;
+ ^
-- check validation when attaching list partitions
CREATE TABLE list_parted2 (
a int,
@@ -3890,6 +3894,8 @@ CREATE TABLE partr_def1 PARTITION OF range_parted DEFAULT;
CREATE TABLE partr_def2 (LIKE part1 INCLUDING CONSTRAINTS);
ALTER TABLE range_parted ATTACH PARTITION partr_def2 DEFAULT;
ERROR: partition "partr_def2" conflicts with existing default partition "partr_def1"
+LINE 1: ...LTER TABLE range_parted ATTACH PARTITION partr_def2 DEFAULT;
+ ^
-- Overlapping partitions cannot be attached, hence, following should give error
INSERT INTO partr_def1 VALUES (2, 10);
CREATE TABLE part3 (LIKE range_parted);
@@ -4010,8 +4016,12 @@ CREATE TABLE hpart_1 PARTITION OF hash_parted FOR VALUES WITH (MODULUS 4, REMAIN
CREATE TABLE fail_part (LIKE hpart_1);
ALTER TABLE hash_parted ATTACH PARTITION fail_part FOR VALUES WITH (MODULUS 8, REMAINDER 4);
ERROR: partition "fail_part" would overlap partition "hpart_1"
+LINE 1: ...hash_parted ATTACH PARTITION fail_part FOR VALUES WITH (MODU...
+ ^
ALTER TABLE hash_parted ATTACH PARTITION fail_part FOR VALUES WITH (MODULUS 8, REMAINDER 0);
ERROR: partition "fail_part" would overlap partition "hpart_1"
+LINE 1: ...hash_parted ATTACH PARTITION fail_part FOR VALUES WITH (MODU...
+ ^
DROP TABLE fail_part;
-- check validation when attaching hash partitions
-- check that violating rows are correctly reported
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 1c72f23bc9..41dce69cc4 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -677,6 +677,8 @@ LINE 1: ...BLE fail_part PARTITION OF list_parted FOR VALUES WITH (MODU...
CREATE TABLE part_default PARTITION OF list_parted DEFAULT;
CREATE TABLE fail_default_part PARTITION OF list_parted DEFAULT;
ERROR: partition "fail_default_part" conflicts with existing default partition "part_default"
+LINE 1: ...TE TABLE fail_default_part PARTITION OF list_parted DEFAULT;
+ ^
-- specified literal can't be cast to the partition column data type
CREATE TABLE bools (
a bool
@@ -702,6 +704,8 @@ CREATE TABLE bigintp_10 PARTITION OF bigintp FOR VALUES IN (10);
-- fails due to overlap:
CREATE TABLE bigintp_10_2 PARTITION OF bigintp FOR VALUES IN ('10');
ERROR: partition "bigintp_10_2" would overlap partition "bigintp_10"
+LINE 1: ...ABLE bigintp_10_2 PARTITION OF bigintp FOR VALUES IN ('10');
+ ^
DROP TABLE bigintp;
CREATE TABLE range_parted (
a date
@@ -823,8 +827,12 @@ CREATE TABLE part_ab PARTITION OF list_parted2 FOR VALUES IN ('a', 'b');
CREATE TABLE list_parted2_def PARTITION OF list_parted2 DEFAULT;
CREATE TABLE fail_part PARTITION OF list_parted2 FOR VALUES IN (null);
ERROR: partition "fail_part" would overlap partition "part_null_z"
+LINE 1: ...LE fail_part PARTITION OF list_parted2 FOR VALUES IN (null);
+ ^
CREATE TABLE fail_part PARTITION OF list_parted2 FOR VALUES IN ('b', 'c');
ERROR: partition "fail_part" would overlap partition "part_ab"
+LINE 1: ...ail_part PARTITION OF list_parted2 FOR VALUES IN ('b', 'c');
+ ^
-- check default partition overlap
INSERT INTO list_parted2 VALUES('X');
CREATE TABLE fail_part PARTITION OF list_parted2 FOR VALUES IN ('W', 'X', 'Y');
@@ -835,28 +843,42 @@ CREATE TABLE range_parted2 (
-- trying to create range partition with empty range
CREATE TABLE fail_part PARTITION OF range_parted2 FOR VALUES FROM (1) TO (0);
ERROR: empty range bound specified for partition "fail_part"
+LINE 1: ..._part PARTITION OF range_parted2 FOR VALUES FROM (1) TO (0);
+ ^
DETAIL: Specified lower bound (1) is greater than or equal to upper bound (0).
-- note that the range '[1, 1)' has no elements
CREATE TABLE fail_part PARTITION OF range_parted2 FOR VALUES FROM (1) TO (1);
ERROR: empty range bound specified for partition "fail_part"
+LINE 1: ..._part PARTITION OF range_parted2 FOR VALUES FROM (1) TO (1);
+ ^
DETAIL: Specified lower bound (1) is greater than or equal to upper bound (1).
CREATE TABLE part0 PARTITION OF range_parted2 FOR VALUES FROM (minvalue) TO (1);
CREATE TABLE fail_part PARTITION OF range_parted2 FOR VALUES FROM (minvalue) TO (2);
ERROR: partition "fail_part" would overlap partition "part0"
+LINE 1: ..._part PARTITION OF range_parted2 FOR VALUES FROM (minvalue) ...
+ ^
CREATE TABLE part1 PARTITION OF range_parted2 FOR VALUES FROM (1) TO (10);
CREATE TABLE fail_part PARTITION OF range_parted2 FOR VALUES FROM (9) TO (maxvalue);
ERROR: partition "fail_part" would overlap partition "part1"
+LINE 1: ..._part PARTITION OF range_parted2 FOR VALUES FROM (9) TO (max...
+ ^
CREATE TABLE part2 PARTITION OF range_parted2 FOR VALUES FROM (20) TO (30);
CREATE TABLE part3 PARTITION OF range_parted2 FOR VALUES FROM (30) TO (40);
CREATE TABLE fail_part PARTITION OF range_parted2 FOR VALUES FROM (10) TO (30);
ERROR: partition "fail_part" would overlap partition "part2"
+LINE 1: ...art PARTITION OF range_parted2 FOR VALUES FROM (10) TO (30);
+ ^
CREATE TABLE fail_part PARTITION OF range_parted2 FOR VALUES FROM (10) TO (50);
ERROR: partition "fail_part" would overlap partition "part2"
+LINE 1: ...art PARTITION OF range_parted2 FOR VALUES FROM (10) TO (50);
+ ^
-- Create a default partition for range partitioned table
CREATE TABLE range2_default PARTITION OF range_parted2 DEFAULT;
-- More than one default partition is not allowed, so this should give error
CREATE TABLE fail_default_part PARTITION OF range_parted2 DEFAULT;
ERROR: partition "fail_default_part" conflicts with existing default partition "range2_default"
+LINE 1: ... TABLE fail_default_part PARTITION OF range_parted2 DEFAULT;
+ ^
-- Check if the range for default partitions overlap
INSERT INTO range_parted2 VALUES (85);
CREATE TABLE fail_part PARTITION OF range_parted2 FOR VALUES FROM (80) TO (90);
@@ -870,17 +892,23 @@ CREATE TABLE range_parted3 (
CREATE TABLE part00 PARTITION OF range_parted3 FOR VALUES FROM (0, minvalue) TO (0, maxvalue);
CREATE TABLE fail_part PARTITION OF range_parted3 FOR VALUES FROM (0, minvalue) TO (0, 1);
ERROR: partition "fail_part" would overlap partition "part00"
+LINE 1: ..._part PARTITION OF range_parted3 FOR VALUES FROM (0, minvalu...
+ ^
CREATE TABLE part10 PARTITION OF range_parted3 FOR VALUES FROM (1, minvalue) TO (1, 1);
CREATE TABLE part11 PARTITION OF range_parted3 FOR VALUES FROM (1, 1) TO (1, 10);
CREATE TABLE part12 PARTITION OF range_parted3 FOR VALUES FROM (1, 10) TO (1, maxvalue);
CREATE TABLE fail_part PARTITION OF range_parted3 FOR VALUES FROM (1, 10) TO (1, 20);
ERROR: partition "fail_part" would overlap partition "part12"
+LINE 1: ...rt PARTITION OF range_parted3 FOR VALUES FROM (1, 10) TO (1,...
+ ^
CREATE TABLE range3_default PARTITION OF range_parted3 DEFAULT;
-- cannot create a partition that says column b is allowed to range
-- from -infinity to +infinity, while there exist partitions that have
-- more specific ranges
CREATE TABLE fail_part PARTITION OF range_parted3 FOR VALUES FROM (1, minvalue) TO (1, maxvalue);
ERROR: partition "fail_part" would overlap partition "part10"
+LINE 1: ..._part PARTITION OF range_parted3 FOR VALUES FROM (1, minvalu...
+ ^
-- check for partition bound overlap and other invalid specifications for the hash partition
CREATE TABLE hash_parted2 (
a varchar
@@ -892,6 +920,8 @@ CREATE TABLE h2part_4 PARTITION OF hash_parted2 FOR VALUES WITH (MODULUS 8, REMA
-- overlap with part_4
CREATE TABLE fail_part PARTITION OF hash_parted2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
ERROR: partition "fail_part" would overlap partition "h2part_4"
+LINE 1: ...LE fail_part PARTITION OF hash_parted2 FOR VALUES WITH (MODU...
+ ^
-- modulus must be greater than zero
CREATE TABLE fail_part PARTITION OF hash_parted2 FOR VALUES WITH (MODULUS 0, REMAINDER 1);
ERROR: modulus for hash partition must be a positive integer
--
2.27.0
On Fri, 10 Jul 2020 at 23:31, Alexandra Wang <alexandra.wanglei@gmail.com>
wrote:
Thank you Daniel. Here's the rebased patch. I also squashed the two
patches into one so it's easier to review.Thanks for rebasing patch. It applies cleanly still. Here are some comments
@@ -3320,7 +3338,9 @@ make_one_partition_rbound(PartitionKey key, int
index, List *datums, bool lower)
* partition_rbound_cmp
*
* Return for two range bounds whether the 1st one (specified in datums1,
I think it's better to reword it as. "For two range bounds decide whether
...
- * kind1, and lower1) is <, =, or > the bound specified in *b2.
+ * kind1, and lower1) is <, =, or > the bound specified in *b2. 0 is
returned if
+ * equal and the 1-based index of the first mismatching bound if unequal;
+ * multiplied by -1 if the 1st bound is smaller.
This sentence makes sense after the above correction. I liked this change,
requires very small changes in other parts.
/*
@@ -3495,7 +3518,7 @@ static int
partition_range_bsearch(int partnatts, FmgrInfo *partsupfunc,
Oid *partcollation,
PartitionBoundInfo boundinfo,
- PartitionRangeBound *probe, bool *is_equal)
+ PartitionRangeBound *probe, bool *is_equal, int32
*cmpval)
Please update the prologue explaining the new argument.
After this change, the patch will be ready for a committer.
--
Best Wishes,
Ashutosh
On Fri, Sep 04, 2020 at 07:42:27PM +0530, Ashutosh Bapat wrote:
After this change, the patch will be ready for a committer.
Alexandra, this patch is waiting on author after this review. Could
you answer to the points raised by Ashutosh and update this patch
accordingly?
--
Michael
Hi Ashutosh,
I had forgotten about this thread, but Michael's ping email brought it
to my attention.
On Fri, Sep 4, 2020 at 11:12 PM Ashutosh Bapat
<ashutosh.bapat@2ndquadrant.com> wrote:
Thanks for rebasing patch. It applies cleanly still. Here are some comments
Thanks for the review.
@@ -3320,7 +3338,9 @@ make_one_partition_rbound(PartitionKey key, int index, List *datums, bool lower)
* partition_rbound_cmp
*
* Return for two range bounds whether the 1st one (specified in datums1,I think it's better to reword it as. "For two range bounds decide whether ...
- * kind1, and lower1) is <, =, or > the bound specified in *b2. + * kind1, and lower1) is <, =, or > the bound specified in *b2. 0 is returned if + * equal and the 1-based index of the first mismatching bound if unequal; + * multiplied by -1 if the 1st bound is smaller.This sentence makes sense after the above correction. I liked this change,
requires very small changes in other parts.
+1 to your suggested rewording, although I wrote: "For two range
bounds this decides whether..."
/* @@ -3495,7 +3518,7 @@ static int partition_range_bsearch(int partnatts, FmgrInfo *partsupfunc, Oid *partcollation, PartitionBoundInfo boundinfo, - PartitionRangeBound *probe, bool *is_equal) + PartitionRangeBound *probe, bool *is_equal, int32 *cmpval)Please update the prologue explaining the new argument.
Done. Actually, I noticed that *is_equal was unused in this
function's only caller. *cmpval == 0 already gives that, so removed
is_equal parameter.
Attached updated version.
--
Amit Langote
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v4-0001-Improve-check-new-partition-bound-error-position-.patchapplication/octet-stream; name=v4-0001-Improve-check-new-partition-bound-error-position-.patchDownload
From 5e5c45eeecc3f5d56475e5d9b9eab4d1eca927a4 Mon Sep 17 00:00:00 2001
From: Alexandra Wang <walexandra@vmware.com>
Date: Fri, 10 Jul 2020 10:28:49 -0700
Subject: [PATCH v4] Improve check new partition bound error position report
We have been passing a dummy ParseState to ereport(). Without the source
text in the ParseState ereport does not report the error position even
if a error location is supplied. This patch passes a ParseState to
check_new_partition_bound() when it is available.
-- Create parent table
create table foo (a int, b int, c date) partition by range (b,c);
-- Before:
create table foo_part_1 partition of foo for values from (1, date '2007-01-01') to (1, date '2006-01-01');
ERROR: empty range bound specified for partition "foo_part_1"
DETAIL: Specified lower bound (1, '2007-01-01') is greater than or equal to upper bound (1, '2006-01-01').
-- After:
create table foo_part_1 partition of foo for values from (1, date '2007-01-01') to (1, date '2006-01-01');
ERROR: empty range bound specified for partition "foo_part_1"
LINE 1: ...e foo_part_1 partition of foo for values from (1, date '2007...
^
DETAIL: Specified lower bound (1, '2007-01-01') is greater than or equal to upper bound (1, '2006-01-01').
Co-authored-by: Ashwin Agrawal <aashwin@vmware.com>
Co-authored-by: Amit Langote <amitlangote09@gmail.com>
---
src/backend/commands/tablecmds.c | 15 ++++--
src/backend/parser/parse_utilcmd.c | 3 ++
src/backend/partitioning/partbounds.c | 81 +++++++++++++++++++-----------
src/include/partitioning/partbounds.h | 4 +-
src/test/regress/expected/alter_table.out | 10 ++++
src/test/regress/expected/create_table.out | 30 +++++++++++
6 files changed, 107 insertions(+), 36 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index eab570a..9038310 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -543,7 +543,8 @@ static void ComputePartitionAttrs(ParseState *pstate, Relation rel, List *partPa
static void CreateInheritance(Relation child_rel, Relation parent_rel);
static void RemoveInheritance(Relation child_rel, Relation parent_rel);
static ObjectAddress ATExecAttachPartition(List **wqueue, Relation rel,
- PartitionCmd *cmd);
+ PartitionCmd *cmd,
+ AlterTableUtilityContext * context);
static void AttachPartitionEnsureIndexes(Relation rel, Relation attachrel);
static void QueuePartitionConstraintValidation(List **wqueue, Relation scanrel,
List *partConstraint,
@@ -1007,7 +1008,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
* Check first that the new partition's bound is valid and does not
* overlap with any of existing partitions of the parent.
*/
- check_new_partition_bound(relname, parent, bound);
+ check_new_partition_bound(relname, parent, bound, pstate);
/*
* If the default partition exists, its partition constraints will
@@ -4718,7 +4719,8 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
cur_pass, context);
Assert(cmd != NULL);
if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
- ATExecAttachPartition(wqueue, rel, (PartitionCmd *) cmd->def);
+ ATExecAttachPartition(wqueue, rel, (PartitionCmd *) cmd->def,
+ context);
else
ATExecAttachPartitionIdx(wqueue, rel,
((PartitionCmd *) cmd->def)->name);
@@ -16280,7 +16282,8 @@ QueuePartitionConstraintValidation(List **wqueue, Relation scanrel,
* Return the address of the newly attached partition.
*/
static ObjectAddress
-ATExecAttachPartition(List **wqueue, Relation rel, PartitionCmd *cmd)
+ATExecAttachPartition(List **wqueue, Relation rel, PartitionCmd *cmd,
+ AlterTableUtilityContext * context)
{
Relation attachrel,
catalog;
@@ -16295,6 +16298,7 @@ ATExecAttachPartition(List **wqueue, Relation rel, PartitionCmd *cmd)
const char *trigger_name;
Oid defaultPartOid;
List *partBoundConstraint;
+ ParseState *pstate = make_parsestate(NULL);
/*
* We must lock the default partition if one exists, because attaching a
@@ -16459,8 +16463,9 @@ ATExecAttachPartition(List **wqueue, Relation rel, PartitionCmd *cmd)
* of existing partitions of the parent - note that it does not return on
* error.
*/
+ pstate->p_sourcetext = context->queryString;
check_new_partition_bound(RelationGetRelationName(attachrel), rel,
- cmd->bound);
+ cmd->bound, pstate);
/* OK to create inheritance. Rest of the checks performed there */
CreateInheritance(attachrel, rel);
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index ec94437..6626757 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -4256,5 +4256,8 @@ transformPartitionBoundValue(ParseState *pstate, Node *val,
if (!IsA(value, Const))
elog(ERROR, "could not evaluate partition bound expression");
+ /* Preserve parser location information. */
+ ((Const *) value)->location = exprLocation(val);
+
return (Const *) value;
}
diff --git a/src/backend/partitioning/partbounds.c b/src/backend/partitioning/partbounds.c
index 58f9b46..5cf1bf0 100644
--- a/src/backend/partitioning/partbounds.c
+++ b/src/backend/partitioning/partbounds.c
@@ -220,10 +220,10 @@ static int32 partition_rbound_cmp(int partnatts, FmgrInfo *partsupfunc,
Oid *partcollation, Datum *datums1,
PartitionRangeDatumKind *kind1, bool lower1,
PartitionRangeBound *b2);
-static int partition_range_bsearch(int partnatts, FmgrInfo *partsupfunc,
- Oid *partcollation,
- PartitionBoundInfo boundinfo,
- PartitionRangeBound *probe, bool *is_equal);
+static int partition_range_bsearch(int partnatts, FmgrInfo *partsupfunc,
+ Oid *partcollation,
+ PartitionBoundInfo boundinfo,
+ PartitionRangeBound *probe, int32 *cmpval);
static int get_partition_bound_num_indexes(PartitionBoundInfo b);
static Expr *make_partition_op_expr(PartitionKey key, int keynum,
uint16 strategy, Expr *arg1, Expr *arg2);
@@ -2805,14 +2805,14 @@ partitions_are_ordered(PartitionBoundInfo boundinfo, int nparts)
*/
void
check_new_partition_bound(char *relname, Relation parent,
- PartitionBoundSpec *spec)
+ PartitionBoundSpec *spec, ParseState *pstate)
{
PartitionKey key = RelationGetPartitionKey(parent);
PartitionDesc partdesc = RelationGetPartitionDesc(parent);
PartitionBoundInfo boundinfo = partdesc->boundinfo;
- ParseState *pstate = make_parsestate(NULL);
int with = -1;
bool overlap = false;
+ int overlap_location = 0;
if (spec->is_default)
{
@@ -2907,6 +2907,7 @@ check_new_partition_bound(char *relname, Relation parent,
if (boundinfo->indexes[remainder] != -1)
{
overlap = true;
+ overlap_location = spec->location;
with = boundinfo->indexes[remainder];
break;
}
@@ -2935,6 +2936,7 @@ check_new_partition_bound(char *relname, Relation parent,
{
Const *val = castNode(Const, lfirst(cell));
+ overlap_location = val->location;
if (!val->constisnull)
{
int offset;
@@ -2968,6 +2970,7 @@ check_new_partition_bound(char *relname, Relation parent,
{
PartitionRangeBound *lower,
*upper;
+ int cmpval;
Assert(spec->strategy == PARTITION_STRATEGY_RANGE);
lower = make_one_partition_rbound(key, -1, spec->lowerdatums, true);
@@ -2977,10 +2980,16 @@ check_new_partition_bound(char *relname, Relation parent,
* First check if the resulting range would be empty with
* specified lower and upper bounds
*/
- if (partition_rbound_cmp(key->partnatts, key->partsupfunc,
- key->partcollation, lower->datums,
- lower->kind, true, upper) >= 0)
+ cmpval = partition_rbound_cmp(key->partnatts,
+ key->partsupfunc,
+ key->partcollation, lower->datums,
+ lower->kind, true, upper);
+ if (cmpval >= 0)
{
+ /* Fetch the problem bound from lower datums list. */
+ PartitionRangeDatum *datum = list_nth(spec->lowerdatums,
+ cmpval - 1);
+
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("empty range bound specified for partition \"%s\"",
@@ -2988,13 +2997,12 @@ check_new_partition_bound(char *relname, Relation parent,
errdetail("Specified lower bound %s is greater than or equal to upper bound %s.",
get_range_partbound_string(spec->lowerdatums),
get_range_partbound_string(spec->upperdatums)),
- parser_errposition(pstate, spec->location)));
+ parser_errposition(pstate, datum->location)));
}
if (partdesc->nparts > 0)
{
int offset;
- bool equal;
Assert(boundinfo &&
boundinfo->strategy == PARTITION_STRATEGY_RANGE &&
@@ -3020,7 +3028,7 @@ check_new_partition_bound(char *relname, Relation parent,
key->partsupfunc,
key->partcollation,
boundinfo, lower,
- &equal);
+ &cmpval);
if (boundinfo->indexes[offset + 1] < 0)
{
@@ -3032,7 +3040,6 @@ check_new_partition_bound(char *relname, Relation parent,
*/
if (offset + 1 < boundinfo->ndatums)
{
- int32 cmpval;
Datum *datums;
PartitionRangeDatumKind *kind;
bool is_lower;
@@ -3054,6 +3061,8 @@ check_new_partition_bound(char *relname, Relation parent,
* offset + 2.
*/
overlap = true;
+ overlap_location = ((PartitionRangeDatum *)
+ list_nth(spec->upperdatums, -cmpval - 1))->location;
with = boundinfo->indexes[offset + 2];
}
}
@@ -3064,7 +3073,14 @@ check_new_partition_bound(char *relname, Relation parent,
* The new partition overlaps with the existing
* partition between offset and offset + 1.
*/
+ Datum *datum;
+
overlap = true;
+ /* New lower bound is certainly >= bound at offet. */
+ Assert(cmpval >= 0);
+ datum = cmpval == 0 ? linitial(spec->lowerdatums):
+ list_nth(spec->lowerdatums, cmpval - 1);
+ overlap_location = ((PartitionRangeDatum *) datum)->location;
with = boundinfo->indexes[offset + 1];
}
}
@@ -3080,11 +3096,12 @@ check_new_partition_bound(char *relname, Relation parent,
if (overlap)
{
Assert(with >= 0);
+ Assert(overlap_location > 0);
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("partition \"%s\" would overlap partition \"%s\"",
relname, get_rel_name(partdesc->oids[with])),
- parser_errposition(pstate, spec->location)));
+ parser_errposition(pstate, overlap_location)));
}
}
@@ -3317,8 +3334,10 @@ make_one_partition_rbound(PartitionKey key, int index, List *datums, bool lower)
/*
* partition_rbound_cmp
*
- * Return for two range bounds whether the 1st one (specified in datums1,
- * kind1, and lower1) is <, =, or > the bound specified in *b2.
+ * For two range bounds this decides whether the 1st one (specified in
+ * datums1, kind1, and lower1) is <, =, or > the bound specified in *b2. 0 is
+ * returned if equal, otherwise a non-zero integer whose absolute values gives
+ * the 1-based partition key number of the first mismatching column.
*
* partnatts, partsupfunc and partcollation give the number of attributes in the
* bounds to be compared, comparison function to be used and the collations of
@@ -3338,6 +3357,7 @@ partition_rbound_cmp(int partnatts, FmgrInfo *partsupfunc,
bool lower1, PartitionRangeBound *b2)
{
int32 cmpval = 0; /* placate compiler */
+ int result = 0;
int i;
Datum *datums2 = b2->datums;
PartitionRangeDatumKind *kind2 = b2->kind;
@@ -3345,6 +3365,8 @@ partition_rbound_cmp(int partnatts, FmgrInfo *partsupfunc,
for (i = 0; i < partnatts; i++)
{
+ result++;
+
/*
* First, handle cases where the column is unbounded, which should not
* invoke the comparison procedure, and should not consider any later
@@ -3352,9 +3374,9 @@ partition_rbound_cmp(int partnatts, FmgrInfo *partsupfunc,
* compare the same way as the values they represent.
*/
if (kind1[i] < kind2[i])
- return -1;
+ return -result;
else if (kind1[i] > kind2[i])
- return 1;
+ return result;
else if (kind1[i] != PARTITION_RANGE_DATUM_VALUE)
/*
@@ -3379,9 +3401,9 @@ partition_rbound_cmp(int partnatts, FmgrInfo *partsupfunc,
* two.
*/
if (cmpval == 0 && lower1 != lower2)
- cmpval = lower1 ? 1 : -1;
+ cmpval = lower1 ? result : -result;
- return cmpval;
+ return cmpval == 0 ? 0 : (cmpval < 0 ? -result : result);;
}
/*
@@ -3486,14 +3508,16 @@ partition_list_bsearch(FmgrInfo *partsupfunc, Oid *partcollation,
* equal to the given range bound or -1 if all of the range bounds are
* greater
*
- * *is_equal is set to true if the range bound at the returned index is equal
- * to the input range bound
+ * Upon return from this function, *cmpval is set to 0 if the bound at the
+ * returned index matches exactly with the input range bound, otherwise a
+ * non-zero integer whose absolute value gives the 1-based partition key number
+ * of the first mismatching column.
*/
static int
partition_range_bsearch(int partnatts, FmgrInfo *partsupfunc,
Oid *partcollation,
PartitionBoundInfo boundinfo,
- PartitionRangeBound *probe, bool *is_equal)
+ PartitionRangeBound *probe, int32 *cmpval)
{
int lo,
hi,
@@ -3503,21 +3527,18 @@ partition_range_bsearch(int partnatts, FmgrInfo *partsupfunc,
hi = boundinfo->ndatums - 1;
while (lo < hi)
{
- int32 cmpval;
-
mid = (lo + hi + 1) / 2;
- cmpval = partition_rbound_cmp(partnatts, partsupfunc,
+ *cmpval = partition_rbound_cmp(partnatts, partsupfunc,
partcollation,
boundinfo->datums[mid],
boundinfo->kind[mid],
(boundinfo->indexes[mid] == -1),
probe);
- if (cmpval <= 0)
+ if (*cmpval <= 0)
{
lo = mid;
- *is_equal = (cmpval == 0);
- if (*is_equal)
+ if (*cmpval == 0)
break;
}
else
@@ -3528,7 +3549,7 @@ partition_range_bsearch(int partnatts, FmgrInfo *partsupfunc,
}
/*
- * partition_range_bsearch
+ * partition_range_datum_bsearch
* Returns the index of the greatest range bound that is less than or
* equal to the given tuple or -1 if all of the range bounds are greater
*
diff --git a/src/include/partitioning/partbounds.h b/src/include/partitioning/partbounds.h
index dfc7207..c82f77d 100644
--- a/src/include/partitioning/partbounds.h
+++ b/src/include/partitioning/partbounds.h
@@ -14,6 +14,7 @@
#include "fmgr.h"
#include "nodes/parsenodes.h"
#include "nodes/pg_list.h"
+#include "parser/parse_node.h"
#include "partitioning/partdefs.h"
#include "utils/relcache.h"
struct RelOptInfo; /* avoid including pathnodes.h here */
@@ -98,7 +99,8 @@ extern PartitionBoundInfo partition_bounds_merge(int partnatts,
List **inner_parts);
extern bool partitions_are_ordered(PartitionBoundInfo boundinfo, int nparts);
extern void check_new_partition_bound(char *relname, Relation parent,
- PartitionBoundSpec *spec);
+ PartitionBoundSpec *spec,
+ ParseState *pstate);
extern void check_default_partition_contents(Relation parent,
Relation defaultRel,
PartitionBoundSpec *new_spec);
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index f566153..0ce6ee4 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -3868,6 +3868,8 @@ SELECT conislocal, coninhcount FROM pg_constraint WHERE conrelid = 'part_1'::reg
CREATE TABLE fail_part (LIKE part_1 INCLUDING CONSTRAINTS);
ALTER TABLE list_parted ATTACH PARTITION fail_part FOR VALUES IN (1);
ERROR: partition "fail_part" would overlap partition "part_1"
+LINE 1: ...LE list_parted ATTACH PARTITION fail_part FOR VALUES IN (1);
+ ^
DROP TABLE fail_part;
-- check that an existing table can be attached as a default partition
CREATE TABLE def_part (LIKE list_parted INCLUDING CONSTRAINTS);
@@ -3877,6 +3879,8 @@ ALTER TABLE list_parted ATTACH PARTITION def_part DEFAULT;
CREATE TABLE fail_def_part (LIKE part_1 INCLUDING CONSTRAINTS);
ALTER TABLE list_parted ATTACH PARTITION fail_def_part DEFAULT;
ERROR: partition "fail_def_part" conflicts with existing default partition "def_part"
+LINE 1: ...ER TABLE list_parted ATTACH PARTITION fail_def_part DEFAULT;
+ ^
-- check validation when attaching list partitions
CREATE TABLE list_parted2 (
a int,
@@ -3946,6 +3950,8 @@ CREATE TABLE partr_def1 PARTITION OF range_parted DEFAULT;
CREATE TABLE partr_def2 (LIKE part1 INCLUDING CONSTRAINTS);
ALTER TABLE range_parted ATTACH PARTITION partr_def2 DEFAULT;
ERROR: partition "partr_def2" conflicts with existing default partition "partr_def1"
+LINE 1: ...LTER TABLE range_parted ATTACH PARTITION partr_def2 DEFAULT;
+ ^
-- Overlapping partitions cannot be attached, hence, following should give error
INSERT INTO partr_def1 VALUES (2, 10);
CREATE TABLE part3 (LIKE range_parted);
@@ -4066,8 +4072,12 @@ CREATE TABLE hpart_1 PARTITION OF hash_parted FOR VALUES WITH (MODULUS 4, REMAIN
CREATE TABLE fail_part (LIKE hpart_1);
ALTER TABLE hash_parted ATTACH PARTITION fail_part FOR VALUES WITH (MODULUS 8, REMAINDER 4);
ERROR: partition "fail_part" would overlap partition "hpart_1"
+LINE 1: ...hash_parted ATTACH PARTITION fail_part FOR VALUES WITH (MODU...
+ ^
ALTER TABLE hash_parted ATTACH PARTITION fail_part FOR VALUES WITH (MODULUS 8, REMAINDER 0);
ERROR: partition "fail_part" would overlap partition "hpart_1"
+LINE 1: ...hash_parted ATTACH PARTITION fail_part FOR VALUES WITH (MODU...
+ ^
DROP TABLE fail_part;
-- check validation when attaching hash partitions
-- check that violating rows are correctly reported
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 1c72f23..41dce69 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -677,6 +677,8 @@ LINE 1: ...BLE fail_part PARTITION OF list_parted FOR VALUES WITH (MODU...
CREATE TABLE part_default PARTITION OF list_parted DEFAULT;
CREATE TABLE fail_default_part PARTITION OF list_parted DEFAULT;
ERROR: partition "fail_default_part" conflicts with existing default partition "part_default"
+LINE 1: ...TE TABLE fail_default_part PARTITION OF list_parted DEFAULT;
+ ^
-- specified literal can't be cast to the partition column data type
CREATE TABLE bools (
a bool
@@ -702,6 +704,8 @@ CREATE TABLE bigintp_10 PARTITION OF bigintp FOR VALUES IN (10);
-- fails due to overlap:
CREATE TABLE bigintp_10_2 PARTITION OF bigintp FOR VALUES IN ('10');
ERROR: partition "bigintp_10_2" would overlap partition "bigintp_10"
+LINE 1: ...ABLE bigintp_10_2 PARTITION OF bigintp FOR VALUES IN ('10');
+ ^
DROP TABLE bigintp;
CREATE TABLE range_parted (
a date
@@ -823,8 +827,12 @@ CREATE TABLE part_ab PARTITION OF list_parted2 FOR VALUES IN ('a', 'b');
CREATE TABLE list_parted2_def PARTITION OF list_parted2 DEFAULT;
CREATE TABLE fail_part PARTITION OF list_parted2 FOR VALUES IN (null);
ERROR: partition "fail_part" would overlap partition "part_null_z"
+LINE 1: ...LE fail_part PARTITION OF list_parted2 FOR VALUES IN (null);
+ ^
CREATE TABLE fail_part PARTITION OF list_parted2 FOR VALUES IN ('b', 'c');
ERROR: partition "fail_part" would overlap partition "part_ab"
+LINE 1: ...ail_part PARTITION OF list_parted2 FOR VALUES IN ('b', 'c');
+ ^
-- check default partition overlap
INSERT INTO list_parted2 VALUES('X');
CREATE TABLE fail_part PARTITION OF list_parted2 FOR VALUES IN ('W', 'X', 'Y');
@@ -835,28 +843,42 @@ CREATE TABLE range_parted2 (
-- trying to create range partition with empty range
CREATE TABLE fail_part PARTITION OF range_parted2 FOR VALUES FROM (1) TO (0);
ERROR: empty range bound specified for partition "fail_part"
+LINE 1: ..._part PARTITION OF range_parted2 FOR VALUES FROM (1) TO (0);
+ ^
DETAIL: Specified lower bound (1) is greater than or equal to upper bound (0).
-- note that the range '[1, 1)' has no elements
CREATE TABLE fail_part PARTITION OF range_parted2 FOR VALUES FROM (1) TO (1);
ERROR: empty range bound specified for partition "fail_part"
+LINE 1: ..._part PARTITION OF range_parted2 FOR VALUES FROM (1) TO (1);
+ ^
DETAIL: Specified lower bound (1) is greater than or equal to upper bound (1).
CREATE TABLE part0 PARTITION OF range_parted2 FOR VALUES FROM (minvalue) TO (1);
CREATE TABLE fail_part PARTITION OF range_parted2 FOR VALUES FROM (minvalue) TO (2);
ERROR: partition "fail_part" would overlap partition "part0"
+LINE 1: ..._part PARTITION OF range_parted2 FOR VALUES FROM (minvalue) ...
+ ^
CREATE TABLE part1 PARTITION OF range_parted2 FOR VALUES FROM (1) TO (10);
CREATE TABLE fail_part PARTITION OF range_parted2 FOR VALUES FROM (9) TO (maxvalue);
ERROR: partition "fail_part" would overlap partition "part1"
+LINE 1: ..._part PARTITION OF range_parted2 FOR VALUES FROM (9) TO (max...
+ ^
CREATE TABLE part2 PARTITION OF range_parted2 FOR VALUES FROM (20) TO (30);
CREATE TABLE part3 PARTITION OF range_parted2 FOR VALUES FROM (30) TO (40);
CREATE TABLE fail_part PARTITION OF range_parted2 FOR VALUES FROM (10) TO (30);
ERROR: partition "fail_part" would overlap partition "part2"
+LINE 1: ...art PARTITION OF range_parted2 FOR VALUES FROM (10) TO (30);
+ ^
CREATE TABLE fail_part PARTITION OF range_parted2 FOR VALUES FROM (10) TO (50);
ERROR: partition "fail_part" would overlap partition "part2"
+LINE 1: ...art PARTITION OF range_parted2 FOR VALUES FROM (10) TO (50);
+ ^
-- Create a default partition for range partitioned table
CREATE TABLE range2_default PARTITION OF range_parted2 DEFAULT;
-- More than one default partition is not allowed, so this should give error
CREATE TABLE fail_default_part PARTITION OF range_parted2 DEFAULT;
ERROR: partition "fail_default_part" conflicts with existing default partition "range2_default"
+LINE 1: ... TABLE fail_default_part PARTITION OF range_parted2 DEFAULT;
+ ^
-- Check if the range for default partitions overlap
INSERT INTO range_parted2 VALUES (85);
CREATE TABLE fail_part PARTITION OF range_parted2 FOR VALUES FROM (80) TO (90);
@@ -870,17 +892,23 @@ CREATE TABLE range_parted3 (
CREATE TABLE part00 PARTITION OF range_parted3 FOR VALUES FROM (0, minvalue) TO (0, maxvalue);
CREATE TABLE fail_part PARTITION OF range_parted3 FOR VALUES FROM (0, minvalue) TO (0, 1);
ERROR: partition "fail_part" would overlap partition "part00"
+LINE 1: ..._part PARTITION OF range_parted3 FOR VALUES FROM (0, minvalu...
+ ^
CREATE TABLE part10 PARTITION OF range_parted3 FOR VALUES FROM (1, minvalue) TO (1, 1);
CREATE TABLE part11 PARTITION OF range_parted3 FOR VALUES FROM (1, 1) TO (1, 10);
CREATE TABLE part12 PARTITION OF range_parted3 FOR VALUES FROM (1, 10) TO (1, maxvalue);
CREATE TABLE fail_part PARTITION OF range_parted3 FOR VALUES FROM (1, 10) TO (1, 20);
ERROR: partition "fail_part" would overlap partition "part12"
+LINE 1: ...rt PARTITION OF range_parted3 FOR VALUES FROM (1, 10) TO (1,...
+ ^
CREATE TABLE range3_default PARTITION OF range_parted3 DEFAULT;
-- cannot create a partition that says column b is allowed to range
-- from -infinity to +infinity, while there exist partitions that have
-- more specific ranges
CREATE TABLE fail_part PARTITION OF range_parted3 FOR VALUES FROM (1, minvalue) TO (1, maxvalue);
ERROR: partition "fail_part" would overlap partition "part10"
+LINE 1: ..._part PARTITION OF range_parted3 FOR VALUES FROM (1, minvalu...
+ ^
-- check for partition bound overlap and other invalid specifications for the hash partition
CREATE TABLE hash_parted2 (
a varchar
@@ -892,6 +920,8 @@ CREATE TABLE h2part_4 PARTITION OF hash_parted2 FOR VALUES WITH (MODULUS 8, REMA
-- overlap with part_4
CREATE TABLE fail_part PARTITION OF hash_parted2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
ERROR: partition "fail_part" would overlap partition "h2part_4"
+LINE 1: ...LE fail_part PARTITION OF hash_parted2 FOR VALUES WITH (MODU...
+ ^
-- modulus must be greater than zero
CREATE TABLE fail_part PARTITION OF hash_parted2 FOR VALUES WITH (MODULUS 0, REMAINDER 1);
ERROR: modulus for hash partition must be a positive integer
--
1.8.3.1
On Thu, 17 Sep 2020 at 13:06, Amit Langote <amitlangote09@gmail.com> wrote:
Hi Ashutosh,
I had forgotten about this thread, but Michael's ping email brought it
to my attention.Thanks Amit for addressing comments.
@@ -4256,5 +4256,8 @@ transformPartitionBoundValue(ParseState *pstate, Node
*val,
if (!IsA(value, Const))
elog(ERROR, "could not evaluate partition bound expression");
+ /* Preserve parser location information. */
+ ((Const *) value)->location = exprLocation(val);
+
return (Const *) value;
}
This caught my attention and I was wondering whether transformExpr() itself
should transfer the location from input expression to the output
expression. Some minions of transformExprRecurse() seem to be doing that.
The change here may be an indication that some of them are not doing this.
In that case may be it's better to find those and fix rather than a
white-wash fix here. In what case did we find that location was not set by
transformExpr? Sorry for not catching this earlier.
/* New lower bound is certainly >= bound at offet. */
offet/offset? But this comment is implied by the comment just two lines
above. So I am not sure it's really needed.
/* Fetch the problem bound from lower datums list. */
This is fetching problematic key value rather than the whole problematic
bound. I think the comment would be useful if it explains why cmpval -1 th
key is problematic but then that's evident from the prologue
of partition_rbound_cmp() so I am not sure if this comment is really
required. For example, we aren't adding a comment here
+ overlap_location = ((PartitionRangeDatum *)
+ list_nth(spec->upperdatums, -cmpval - 1))->location;
--
Best Wishes,
Ashutosh
Thanks Ashutosh.
On Fri, Sep 18, 2020 at 7:33 PM Ashutosh Bapat
<ashutosh.bapat@2ndquadrant.com> wrote:
Thanks Amit for addressing comments.
@@ -4256,5 +4256,8 @@ transformPartitionBoundValue(ParseState *pstate, Node *val,
if (!IsA(value, Const))
elog(ERROR, "could not evaluate partition bound expression");+ /* Preserve parser location information. */ + ((Const *) value)->location = exprLocation(val); + return (Const *) value; }This caught my attention and I was wondering whether transformExpr() itself should transfer the location from input expression to the output expression. Some minions of transformExprRecurse() seem to be doing that. The change here may be an indication that some of them are not doing this. In that case may be it's better to find those and fix rather than a white-wash fix here. In what case did we find that location was not set by transformExpr? Sorry for not catching this earlier.
AFAICS, transformExpr() is fine. What loses the location value is the
unconditional evaluate_expr() call which generates a fresh Const node,
possibly after evaluating a non-Const expression that is passed to it.
I don't find it very desirable to change evaluate_expr() to accept a
location value, because other callers of it don't seem to care.
Instead, in the updated patch, I have made calling evaluate_expr()
conditional on the expression at hand being a non-Const node and
assign location by hand on return. If the expression is already
Const, we don't need to update the location field as it should already
be correct. Though, I did notice that the evaluate_expr() call has an
additional responsibility which is to pass the partition key specified
collation to the bound expression, so we should not fail to update an
already-Const node's collation likewise.
/* New lower bound is certainly >= bound at offet. */
offet/offset? But this comment is implied by the comment just two lines above. So I am not sure it's really needed.
Given that cmpval is set all the way in partition_range_bsearch(), I
thought it would help to clarify why this code can assume it must be
= 0. It is because a valid offset returned by
partition_range_bsearch() must correspond to a bound that it found to
be <= the probe bound passed to it.
/* Fetch the problem bound from lower datums list. */ This is fetching problematic key value rather than the whole problematic bound. I think the comment would be useful if it explains why cmpval -1 th key is problematic but then that's evident from the prologue of partition_rbound_cmp() so I am not sure if this comment is really required. For example, we aren't adding a comment here + overlap_location = ((PartitionRangeDatum *) + list_nth(spec->upperdatums, -cmpval - 1))->location;
In the attached updated patch, I have tried to make the code and
comments for different cases consistent. Please have a look.
--
Amit Langote
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v5-0001-Improve-check-new-partition-bound-error-position-.patchapplication/octet-stream; name=v5-0001-Improve-check-new-partition-bound-error-position-.patchDownload
From 539f4c9c2cdb63d19e2fc635b2f3f35871de1ccb Mon Sep 17 00:00:00 2001
From: Alexandra Wang <walexandra@vmware.com>
Date: Fri, 10 Jul 2020 10:28:49 -0700
Subject: [PATCH v5] Improve check new partition bound error position report
We have been passing a dummy ParseState to ereport(). Without the source
text in the ParseState ereport does not report the error position even
if a error location is supplied. This patch passes a ParseState to
check_new_partition_bound() when it is available.
-- Create parent table
create table foo (a int, b int, c date) partition by range (b,c);
-- Before:
create table foo_part_1 partition of foo for values from (1, date '2007-01-01') to (1, date '2006-01-01');
ERROR: empty range bound specified for partition "foo_part_1"
DETAIL: Specified lower bound (1, '2007-01-01') is greater than or equal to upper bound (1, '2006-01-01').
-- After:
create table foo_part_1 partition of foo for values from (1, date '2007-01-01') to (1, date '2006-01-01');
ERROR: empty range bound specified for partition "foo_part_1"
LINE 1: ...e foo_part_1 partition of foo for values from (1, date '2007...
^
DETAIL: Specified lower bound (1, '2007-01-01') is greater than or equal to upper bound (1, '2006-01-01').
Co-authored-by: Ashwin Agrawal <aashwin@vmware.com>
Co-authored-by: Amit Langote <amitlangote09@gmail.com>
---
src/backend/commands/tablecmds.c | 15 +++--
src/backend/parser/parse_utilcmd.c | 45 +++++++++------
src/backend/partitioning/partbounds.c | 91 ++++++++++++++++++++----------
src/include/partitioning/partbounds.h | 4 +-
src/test/regress/expected/alter_table.out | 10 ++++
src/test/regress/expected/create_table.out | 30 ++++++++++
6 files changed, 142 insertions(+), 53 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index eab570a..9038310 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -543,7 +543,8 @@ static void ComputePartitionAttrs(ParseState *pstate, Relation rel, List *partPa
static void CreateInheritance(Relation child_rel, Relation parent_rel);
static void RemoveInheritance(Relation child_rel, Relation parent_rel);
static ObjectAddress ATExecAttachPartition(List **wqueue, Relation rel,
- PartitionCmd *cmd);
+ PartitionCmd *cmd,
+ AlterTableUtilityContext * context);
static void AttachPartitionEnsureIndexes(Relation rel, Relation attachrel);
static void QueuePartitionConstraintValidation(List **wqueue, Relation scanrel,
List *partConstraint,
@@ -1007,7 +1008,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
* Check first that the new partition's bound is valid and does not
* overlap with any of existing partitions of the parent.
*/
- check_new_partition_bound(relname, parent, bound);
+ check_new_partition_bound(relname, parent, bound, pstate);
/*
* If the default partition exists, its partition constraints will
@@ -4718,7 +4719,8 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
cur_pass, context);
Assert(cmd != NULL);
if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
- ATExecAttachPartition(wqueue, rel, (PartitionCmd *) cmd->def);
+ ATExecAttachPartition(wqueue, rel, (PartitionCmd *) cmd->def,
+ context);
else
ATExecAttachPartitionIdx(wqueue, rel,
((PartitionCmd *) cmd->def)->name);
@@ -16280,7 +16282,8 @@ QueuePartitionConstraintValidation(List **wqueue, Relation scanrel,
* Return the address of the newly attached partition.
*/
static ObjectAddress
-ATExecAttachPartition(List **wqueue, Relation rel, PartitionCmd *cmd)
+ATExecAttachPartition(List **wqueue, Relation rel, PartitionCmd *cmd,
+ AlterTableUtilityContext * context)
{
Relation attachrel,
catalog;
@@ -16295,6 +16298,7 @@ ATExecAttachPartition(List **wqueue, Relation rel, PartitionCmd *cmd)
const char *trigger_name;
Oid defaultPartOid;
List *partBoundConstraint;
+ ParseState *pstate = make_parsestate(NULL);
/*
* We must lock the default partition if one exists, because attaching a
@@ -16459,8 +16463,9 @@ ATExecAttachPartition(List **wqueue, Relation rel, PartitionCmd *cmd)
* of existing partitions of the parent - note that it does not return on
* error.
*/
+ pstate->p_sourcetext = context->queryString;
check_new_partition_bound(RelationGetRelationName(attachrel), rel,
- cmd->bound);
+ cmd->bound, pstate);
/* OK to create inheritance. Rest of the checks performed there */
CreateInheritance(attachrel, rel);
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index ec94437..4dcdd34 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -4177,6 +4177,13 @@ transformPartitionBoundValue(ParseState *pstate, Node *val,
value = transformExpr(pstate, val, EXPR_KIND_PARTITION_BOUND);
/*
+ * transformExpr() should have already rejected column references,
+ * subqueries, aggregates, window functions, and SRFs, based on the
+ * EXPR_KIND_ of a partition bound expression.
+ */
+ Assert(!contain_var_clause(value));
+
+ /*
* Check that the input expression's collation is compatible with one
* specified for the parent's partition key (partcollation). Don't throw
* an error if it's the default collation which we'll replace with the
@@ -4220,14 +4227,18 @@ transformPartitionBoundValue(ParseState *pstate, Node *val,
parser_errposition(pstate, exprLocation(value))));
}
- /* Coerce to correct type */
+ /*
+ * Coerce to the correct type. This might cause an explicit coercion step
+ * to be added on top of the expression, which must be evaluated before
+ * returning the expression to the caller.
+ */
value = coerce_to_target_type(pstate,
value, exprType(value),
colType,
colTypmod,
COERCION_ASSIGNMENT,
COERCE_IMPLICIT_CAST,
- -1);
+ exprLocation(value));
if (value == NULL)
ereport(ERROR,
@@ -4236,25 +4247,23 @@ transformPartitionBoundValue(ParseState *pstate, Node *val,
format_type_be(colType), colName),
parser_errposition(pstate, exprLocation(val))));
- /* Simplify the expression, in case we had a coercion */
- if (!IsA(value, Const))
- value = (Node *) expression_planner((Expr *) value);
-
/*
- * transformExpr() should have already rejected column references,
- * subqueries, aggregates, window functions, and SRFs, based on the
- * EXPR_KIND_ for a default expression.
- */
- Assert(!contain_var_clause(value));
-
- /*
- * Evaluate the expression, assigning the partition key's collation to the
- * resulting Const expression.
+ * Evaluate the expression, if needed, assigning the partition key's
+ * collation to the resulting Const expression.
*/
- value = (Node *) evaluate_expr((Expr *) value, colType, colTypmod,
- partCollation);
if (!IsA(value, Const))
- elog(ERROR, "could not evaluate partition bound expression");
+ {
+ value = (Node *) expression_planner((Expr *) value);
+ value = (Node *) evaluate_expr((Expr *) value, colType, colTypmod,
+ partCollation);
+ if (!IsA(value, Const))
+ elog(ERROR, "could not evaluate partition bound expression");
+
+ /* Preserve original node's location value. */
+ ((Const *) value)->location = exprLocation(val);
+ }
+ else
+ ((Const *) value)->constcollid = partCollation;
return (Const *) value;
}
diff --git a/src/backend/partitioning/partbounds.c b/src/backend/partitioning/partbounds.c
index 58f9b46..9f9ba8d 100644
--- a/src/backend/partitioning/partbounds.c
+++ b/src/backend/partitioning/partbounds.c
@@ -220,10 +220,10 @@ static int32 partition_rbound_cmp(int partnatts, FmgrInfo *partsupfunc,
Oid *partcollation, Datum *datums1,
PartitionRangeDatumKind *kind1, bool lower1,
PartitionRangeBound *b2);
-static int partition_range_bsearch(int partnatts, FmgrInfo *partsupfunc,
- Oid *partcollation,
- PartitionBoundInfo boundinfo,
- PartitionRangeBound *probe, bool *is_equal);
+static int partition_range_bsearch(int partnatts, FmgrInfo *partsupfunc,
+ Oid *partcollation,
+ PartitionBoundInfo boundinfo,
+ PartitionRangeBound *probe, int32 *cmpval);
static int get_partition_bound_num_indexes(PartitionBoundInfo b);
static Expr *make_partition_op_expr(PartitionKey key, int keynum,
uint16 strategy, Expr *arg1, Expr *arg2);
@@ -2805,14 +2805,14 @@ partitions_are_ordered(PartitionBoundInfo boundinfo, int nparts)
*/
void
check_new_partition_bound(char *relname, Relation parent,
- PartitionBoundSpec *spec)
+ PartitionBoundSpec *spec, ParseState *pstate)
{
PartitionKey key = RelationGetPartitionKey(parent);
PartitionDesc partdesc = RelationGetPartitionDesc(parent);
PartitionBoundInfo boundinfo = partdesc->boundinfo;
- ParseState *pstate = make_parsestate(NULL);
int with = -1;
bool overlap = false;
+ int overlap_location = 0;
if (spec->is_default)
{
@@ -2907,6 +2907,7 @@ check_new_partition_bound(char *relname, Relation parent,
if (boundinfo->indexes[remainder] != -1)
{
overlap = true;
+ overlap_location = spec->location;
with = boundinfo->indexes[remainder];
break;
}
@@ -2935,6 +2936,7 @@ check_new_partition_bound(char *relname, Relation parent,
{
Const *val = castNode(Const, lfirst(cell));
+ overlap_location = val->location;
if (!val->constisnull)
{
int offset;
@@ -2968,6 +2970,7 @@ check_new_partition_bound(char *relname, Relation parent,
{
PartitionRangeBound *lower,
*upper;
+ int cmpval;
Assert(spec->strategy == PARTITION_STRATEGY_RANGE);
lower = make_one_partition_rbound(key, -1, spec->lowerdatums, true);
@@ -2977,10 +2980,16 @@ check_new_partition_bound(char *relname, Relation parent,
* First check if the resulting range would be empty with
* specified lower and upper bounds
*/
- if (partition_rbound_cmp(key->partnatts, key->partsupfunc,
- key->partcollation, lower->datums,
- lower->kind, true, upper) >= 0)
+ cmpval = partition_rbound_cmp(key->partnatts,
+ key->partsupfunc,
+ key->partcollation, lower->datums,
+ lower->kind, true, upper);
+ if (cmpval >= 0)
{
+ /* Fetch the problematic key from the lower datums list. */
+ PartitionRangeDatum *datum = list_nth(spec->lowerdatums,
+ cmpval - 1);
+
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("empty range bound specified for partition \"%s\"",
@@ -2988,13 +2997,12 @@ check_new_partition_bound(char *relname, Relation parent,
errdetail("Specified lower bound %s is greater than or equal to upper bound %s.",
get_range_partbound_string(spec->lowerdatums),
get_range_partbound_string(spec->upperdatums)),
- parser_errposition(pstate, spec->location)));
+ parser_errposition(pstate, datum->location)));
}
if (partdesc->nparts > 0)
{
int offset;
- bool equal;
Assert(boundinfo &&
boundinfo->strategy == PARTITION_STRATEGY_RANGE &&
@@ -3020,7 +3028,7 @@ check_new_partition_bound(char *relname, Relation parent,
key->partsupfunc,
key->partcollation,
boundinfo, lower,
- &equal);
+ &cmpval);
if (boundinfo->indexes[offset + 1] < 0)
{
@@ -3032,7 +3040,6 @@ check_new_partition_bound(char *relname, Relation parent,
*/
if (offset + 1 < boundinfo->ndatums)
{
- int32 cmpval;
Datum *datums;
PartitionRangeDatumKind *kind;
bool is_lower;
@@ -3049,11 +3056,19 @@ check_new_partition_bound(char *relname, Relation parent,
if (cmpval < 0)
{
/*
+ * Fetch the problematic key from the upper
+ * datums list.
+ */
+ PartitionRangeDatum *datum =
+ list_nth(spec->upperdatums, -cmpval - 1);
+
+ /*
* The new partition overlaps with the
* existing partition between offset + 1 and
* offset + 2.
*/
overlap = true;
+ overlap_location = datum->location;
with = boundinfo->indexes[offset + 2];
}
}
@@ -3064,7 +3079,20 @@ check_new_partition_bound(char *relname, Relation parent,
* The new partition overlaps with the existing
* partition between offset and offset + 1.
*/
+ PartitionRangeDatum *datum;
+
+ /*
+ * Fetch the problematic key from the lower datums
+ * list. Given the way partition_range_bsearch()
+ * works, the new lower bound is certainly >= the bound
+ * at offset. If the bound matches exactly, we flag
+ * the 1st key.
+ */
+ Assert(cmpval >= 0);
+ datum = cmpval == 0 ? linitial(spec->lowerdatums):
+ list_nth(spec->lowerdatums, cmpval - 1);
overlap = true;
+ overlap_location = datum->location;
with = boundinfo->indexes[offset + 1];
}
}
@@ -3080,11 +3108,12 @@ check_new_partition_bound(char *relname, Relation parent,
if (overlap)
{
Assert(with >= 0);
+ Assert(overlap_location > 0);
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("partition \"%s\" would overlap partition \"%s\"",
relname, get_rel_name(partdesc->oids[with])),
- parser_errposition(pstate, spec->location)));
+ parser_errposition(pstate, overlap_location)));
}
}
@@ -3317,8 +3346,10 @@ make_one_partition_rbound(PartitionKey key, int index, List *datums, bool lower)
/*
* partition_rbound_cmp
*
- * Return for two range bounds whether the 1st one (specified in datums1,
- * kind1, and lower1) is <, =, or > the bound specified in *b2.
+ * For two range bounds this decides whether the 1st one (specified in
+ * datums1, kind1, and lower1) is <, =, or > the bound specified in *b2. 0 is
+ * returned if equal, otherwise a non-zero integer whose absolute values gives
+ * the 1-based partition key number of the first mismatching column.
*
* partnatts, partsupfunc and partcollation give the number of attributes in the
* bounds to be compared, comparison function to be used and the collations of
@@ -3338,6 +3369,7 @@ partition_rbound_cmp(int partnatts, FmgrInfo *partsupfunc,
bool lower1, PartitionRangeBound *b2)
{
int32 cmpval = 0; /* placate compiler */
+ int result = 0;
int i;
Datum *datums2 = b2->datums;
PartitionRangeDatumKind *kind2 = b2->kind;
@@ -3345,6 +3377,8 @@ partition_rbound_cmp(int partnatts, FmgrInfo *partsupfunc,
for (i = 0; i < partnatts; i++)
{
+ result++;
+
/*
* First, handle cases where the column is unbounded, which should not
* invoke the comparison procedure, and should not consider any later
@@ -3352,9 +3386,9 @@ partition_rbound_cmp(int partnatts, FmgrInfo *partsupfunc,
* compare the same way as the values they represent.
*/
if (kind1[i] < kind2[i])
- return -1;
+ return -result;
else if (kind1[i] > kind2[i])
- return 1;
+ return result;
else if (kind1[i] != PARTITION_RANGE_DATUM_VALUE)
/*
@@ -3381,7 +3415,7 @@ partition_rbound_cmp(int partnatts, FmgrInfo *partsupfunc,
if (cmpval == 0 && lower1 != lower2)
cmpval = lower1 ? 1 : -1;
- return cmpval;
+ return cmpval == 0 ? 0 : (cmpval < 0 ? -result : result);
}
/*
@@ -3486,14 +3520,16 @@ partition_list_bsearch(FmgrInfo *partsupfunc, Oid *partcollation,
* equal to the given range bound or -1 if all of the range bounds are
* greater
*
- * *is_equal is set to true if the range bound at the returned index is equal
- * to the input range bound
+ * Upon return from this function, *cmpval is set to 0 if the bound at the
+ * returned index matches exactly with the input range bound, otherwise a
+ * non-zero integer whose absolute value gives the 1-based partition key number
+ * of the first mismatching column.
*/
static int
partition_range_bsearch(int partnatts, FmgrInfo *partsupfunc,
Oid *partcollation,
PartitionBoundInfo boundinfo,
- PartitionRangeBound *probe, bool *is_equal)
+ PartitionRangeBound *probe, int32 *cmpval)
{
int lo,
hi,
@@ -3503,21 +3539,18 @@ partition_range_bsearch(int partnatts, FmgrInfo *partsupfunc,
hi = boundinfo->ndatums - 1;
while (lo < hi)
{
- int32 cmpval;
-
mid = (lo + hi + 1) / 2;
- cmpval = partition_rbound_cmp(partnatts, partsupfunc,
+ *cmpval = partition_rbound_cmp(partnatts, partsupfunc,
partcollation,
boundinfo->datums[mid],
boundinfo->kind[mid],
(boundinfo->indexes[mid] == -1),
probe);
- if (cmpval <= 0)
+ if (*cmpval <= 0)
{
lo = mid;
- *is_equal = (cmpval == 0);
- if (*is_equal)
+ if (*cmpval == 0)
break;
}
else
@@ -3528,7 +3561,7 @@ partition_range_bsearch(int partnatts, FmgrInfo *partsupfunc,
}
/*
- * partition_range_bsearch
+ * partition_range_datum_bsearch
* Returns the index of the greatest range bound that is less than or
* equal to the given tuple or -1 if all of the range bounds are greater
*
diff --git a/src/include/partitioning/partbounds.h b/src/include/partitioning/partbounds.h
index dfc7207..c82f77d 100644
--- a/src/include/partitioning/partbounds.h
+++ b/src/include/partitioning/partbounds.h
@@ -14,6 +14,7 @@
#include "fmgr.h"
#include "nodes/parsenodes.h"
#include "nodes/pg_list.h"
+#include "parser/parse_node.h"
#include "partitioning/partdefs.h"
#include "utils/relcache.h"
struct RelOptInfo; /* avoid including pathnodes.h here */
@@ -98,7 +99,8 @@ extern PartitionBoundInfo partition_bounds_merge(int partnatts,
List **inner_parts);
extern bool partitions_are_ordered(PartitionBoundInfo boundinfo, int nparts);
extern void check_new_partition_bound(char *relname, Relation parent,
- PartitionBoundSpec *spec);
+ PartitionBoundSpec *spec,
+ ParseState *pstate);
extern void check_default_partition_contents(Relation parent,
Relation defaultRel,
PartitionBoundSpec *new_spec);
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index f566153..0ce6ee4 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -3868,6 +3868,8 @@ SELECT conislocal, coninhcount FROM pg_constraint WHERE conrelid = 'part_1'::reg
CREATE TABLE fail_part (LIKE part_1 INCLUDING CONSTRAINTS);
ALTER TABLE list_parted ATTACH PARTITION fail_part FOR VALUES IN (1);
ERROR: partition "fail_part" would overlap partition "part_1"
+LINE 1: ...LE list_parted ATTACH PARTITION fail_part FOR VALUES IN (1);
+ ^
DROP TABLE fail_part;
-- check that an existing table can be attached as a default partition
CREATE TABLE def_part (LIKE list_parted INCLUDING CONSTRAINTS);
@@ -3877,6 +3879,8 @@ ALTER TABLE list_parted ATTACH PARTITION def_part DEFAULT;
CREATE TABLE fail_def_part (LIKE part_1 INCLUDING CONSTRAINTS);
ALTER TABLE list_parted ATTACH PARTITION fail_def_part DEFAULT;
ERROR: partition "fail_def_part" conflicts with existing default partition "def_part"
+LINE 1: ...ER TABLE list_parted ATTACH PARTITION fail_def_part DEFAULT;
+ ^
-- check validation when attaching list partitions
CREATE TABLE list_parted2 (
a int,
@@ -3946,6 +3950,8 @@ CREATE TABLE partr_def1 PARTITION OF range_parted DEFAULT;
CREATE TABLE partr_def2 (LIKE part1 INCLUDING CONSTRAINTS);
ALTER TABLE range_parted ATTACH PARTITION partr_def2 DEFAULT;
ERROR: partition "partr_def2" conflicts with existing default partition "partr_def1"
+LINE 1: ...LTER TABLE range_parted ATTACH PARTITION partr_def2 DEFAULT;
+ ^
-- Overlapping partitions cannot be attached, hence, following should give error
INSERT INTO partr_def1 VALUES (2, 10);
CREATE TABLE part3 (LIKE range_parted);
@@ -4066,8 +4072,12 @@ CREATE TABLE hpart_1 PARTITION OF hash_parted FOR VALUES WITH (MODULUS 4, REMAIN
CREATE TABLE fail_part (LIKE hpart_1);
ALTER TABLE hash_parted ATTACH PARTITION fail_part FOR VALUES WITH (MODULUS 8, REMAINDER 4);
ERROR: partition "fail_part" would overlap partition "hpart_1"
+LINE 1: ...hash_parted ATTACH PARTITION fail_part FOR VALUES WITH (MODU...
+ ^
ALTER TABLE hash_parted ATTACH PARTITION fail_part FOR VALUES WITH (MODULUS 8, REMAINDER 0);
ERROR: partition "fail_part" would overlap partition "hpart_1"
+LINE 1: ...hash_parted ATTACH PARTITION fail_part FOR VALUES WITH (MODU...
+ ^
DROP TABLE fail_part;
-- check validation when attaching hash partitions
-- check that violating rows are correctly reported
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 1c72f23..41dce69 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -677,6 +677,8 @@ LINE 1: ...BLE fail_part PARTITION OF list_parted FOR VALUES WITH (MODU...
CREATE TABLE part_default PARTITION OF list_parted DEFAULT;
CREATE TABLE fail_default_part PARTITION OF list_parted DEFAULT;
ERROR: partition "fail_default_part" conflicts with existing default partition "part_default"
+LINE 1: ...TE TABLE fail_default_part PARTITION OF list_parted DEFAULT;
+ ^
-- specified literal can't be cast to the partition column data type
CREATE TABLE bools (
a bool
@@ -702,6 +704,8 @@ CREATE TABLE bigintp_10 PARTITION OF bigintp FOR VALUES IN (10);
-- fails due to overlap:
CREATE TABLE bigintp_10_2 PARTITION OF bigintp FOR VALUES IN ('10');
ERROR: partition "bigintp_10_2" would overlap partition "bigintp_10"
+LINE 1: ...ABLE bigintp_10_2 PARTITION OF bigintp FOR VALUES IN ('10');
+ ^
DROP TABLE bigintp;
CREATE TABLE range_parted (
a date
@@ -823,8 +827,12 @@ CREATE TABLE part_ab PARTITION OF list_parted2 FOR VALUES IN ('a', 'b');
CREATE TABLE list_parted2_def PARTITION OF list_parted2 DEFAULT;
CREATE TABLE fail_part PARTITION OF list_parted2 FOR VALUES IN (null);
ERROR: partition "fail_part" would overlap partition "part_null_z"
+LINE 1: ...LE fail_part PARTITION OF list_parted2 FOR VALUES IN (null);
+ ^
CREATE TABLE fail_part PARTITION OF list_parted2 FOR VALUES IN ('b', 'c');
ERROR: partition "fail_part" would overlap partition "part_ab"
+LINE 1: ...ail_part PARTITION OF list_parted2 FOR VALUES IN ('b', 'c');
+ ^
-- check default partition overlap
INSERT INTO list_parted2 VALUES('X');
CREATE TABLE fail_part PARTITION OF list_parted2 FOR VALUES IN ('W', 'X', 'Y');
@@ -835,28 +843,42 @@ CREATE TABLE range_parted2 (
-- trying to create range partition with empty range
CREATE TABLE fail_part PARTITION OF range_parted2 FOR VALUES FROM (1) TO (0);
ERROR: empty range bound specified for partition "fail_part"
+LINE 1: ..._part PARTITION OF range_parted2 FOR VALUES FROM (1) TO (0);
+ ^
DETAIL: Specified lower bound (1) is greater than or equal to upper bound (0).
-- note that the range '[1, 1)' has no elements
CREATE TABLE fail_part PARTITION OF range_parted2 FOR VALUES FROM (1) TO (1);
ERROR: empty range bound specified for partition "fail_part"
+LINE 1: ..._part PARTITION OF range_parted2 FOR VALUES FROM (1) TO (1);
+ ^
DETAIL: Specified lower bound (1) is greater than or equal to upper bound (1).
CREATE TABLE part0 PARTITION OF range_parted2 FOR VALUES FROM (minvalue) TO (1);
CREATE TABLE fail_part PARTITION OF range_parted2 FOR VALUES FROM (minvalue) TO (2);
ERROR: partition "fail_part" would overlap partition "part0"
+LINE 1: ..._part PARTITION OF range_parted2 FOR VALUES FROM (minvalue) ...
+ ^
CREATE TABLE part1 PARTITION OF range_parted2 FOR VALUES FROM (1) TO (10);
CREATE TABLE fail_part PARTITION OF range_parted2 FOR VALUES FROM (9) TO (maxvalue);
ERROR: partition "fail_part" would overlap partition "part1"
+LINE 1: ..._part PARTITION OF range_parted2 FOR VALUES FROM (9) TO (max...
+ ^
CREATE TABLE part2 PARTITION OF range_parted2 FOR VALUES FROM (20) TO (30);
CREATE TABLE part3 PARTITION OF range_parted2 FOR VALUES FROM (30) TO (40);
CREATE TABLE fail_part PARTITION OF range_parted2 FOR VALUES FROM (10) TO (30);
ERROR: partition "fail_part" would overlap partition "part2"
+LINE 1: ...art PARTITION OF range_parted2 FOR VALUES FROM (10) TO (30);
+ ^
CREATE TABLE fail_part PARTITION OF range_parted2 FOR VALUES FROM (10) TO (50);
ERROR: partition "fail_part" would overlap partition "part2"
+LINE 1: ...art PARTITION OF range_parted2 FOR VALUES FROM (10) TO (50);
+ ^
-- Create a default partition for range partitioned table
CREATE TABLE range2_default PARTITION OF range_parted2 DEFAULT;
-- More than one default partition is not allowed, so this should give error
CREATE TABLE fail_default_part PARTITION OF range_parted2 DEFAULT;
ERROR: partition "fail_default_part" conflicts with existing default partition "range2_default"
+LINE 1: ... TABLE fail_default_part PARTITION OF range_parted2 DEFAULT;
+ ^
-- Check if the range for default partitions overlap
INSERT INTO range_parted2 VALUES (85);
CREATE TABLE fail_part PARTITION OF range_parted2 FOR VALUES FROM (80) TO (90);
@@ -870,17 +892,23 @@ CREATE TABLE range_parted3 (
CREATE TABLE part00 PARTITION OF range_parted3 FOR VALUES FROM (0, minvalue) TO (0, maxvalue);
CREATE TABLE fail_part PARTITION OF range_parted3 FOR VALUES FROM (0, minvalue) TO (0, 1);
ERROR: partition "fail_part" would overlap partition "part00"
+LINE 1: ..._part PARTITION OF range_parted3 FOR VALUES FROM (0, minvalu...
+ ^
CREATE TABLE part10 PARTITION OF range_parted3 FOR VALUES FROM (1, minvalue) TO (1, 1);
CREATE TABLE part11 PARTITION OF range_parted3 FOR VALUES FROM (1, 1) TO (1, 10);
CREATE TABLE part12 PARTITION OF range_parted3 FOR VALUES FROM (1, 10) TO (1, maxvalue);
CREATE TABLE fail_part PARTITION OF range_parted3 FOR VALUES FROM (1, 10) TO (1, 20);
ERROR: partition "fail_part" would overlap partition "part12"
+LINE 1: ...rt PARTITION OF range_parted3 FOR VALUES FROM (1, 10) TO (1,...
+ ^
CREATE TABLE range3_default PARTITION OF range_parted3 DEFAULT;
-- cannot create a partition that says column b is allowed to range
-- from -infinity to +infinity, while there exist partitions that have
-- more specific ranges
CREATE TABLE fail_part PARTITION OF range_parted3 FOR VALUES FROM (1, minvalue) TO (1, maxvalue);
ERROR: partition "fail_part" would overlap partition "part10"
+LINE 1: ..._part PARTITION OF range_parted3 FOR VALUES FROM (1, minvalu...
+ ^
-- check for partition bound overlap and other invalid specifications for the hash partition
CREATE TABLE hash_parted2 (
a varchar
@@ -892,6 +920,8 @@ CREATE TABLE h2part_4 PARTITION OF hash_parted2 FOR VALUES WITH (MODULUS 8, REMA
-- overlap with part_4
CREATE TABLE fail_part PARTITION OF hash_parted2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
ERROR: partition "fail_part" would overlap partition "h2part_4"
+LINE 1: ...LE fail_part PARTITION OF hash_parted2 FOR VALUES WITH (MODU...
+ ^
-- modulus must be greater than zero
CREATE TABLE fail_part PARTITION OF hash_parted2 FOR VALUES WITH (MODULUS 0, REMAINDER 1);
ERROR: modulus for hash partition must be a positive integer
--
1.8.3.1
On Wed, 23 Sep 2020 at 14:41, Amit Langote <amitlangote09@gmail.com> wrote:
Thanks Ashutosh.
On Fri, Sep 18, 2020 at 7:33 PM Ashutosh Bapat
<ashutosh.bapat@2ndquadrant.com> wrote:Thanks Amit for addressing comments.
@@ -4256,5 +4256,8 @@ transformPartitionBoundValue(ParseState *pstate,
Node *val,
if (!IsA(value, Const))
elog(ERROR, "could not evaluate partition bound expression");+ /* Preserve parser location information. */ + ((Const *) value)->location = exprLocation(val); + return (Const *) value; }This caught my attention and I was wondering whether transformExpr()
itself should transfer the location from input expression to the output
expression. Some minions of transformExprRecurse() seem to be doing that.
The change here may be an indication that some of them are not doing this.
In that case may be it's better to find those and fix rather than a
white-wash fix here. In what case did we find that location was not set by
transformExpr? Sorry for not catching this earlier.AFAICS, transformExpr() is fine. What loses the location value is the
unconditional evaluate_expr() call which generates a fresh Const node,
possibly after evaluating a non-Const expression that is passed to it.
I don't find it very desirable to change evaluate_expr() to accept a
location value, because other callers of it don't seem to care.
Instead, in the updated patch, I have made calling evaluate_expr()
conditional on the expression at hand being a non-Const node and
assign location by hand on return. If the expression is already
Const, we don't need to update the location field as it should already
be correct. Though, I did notice that the evaluate_expr() call has an
additional responsibility which is to pass the partition key specified
collation to the bound expression, so we should not fail to update an
already-Const node's collation likewise.
Thanks for the detailed explanation. I am not sure whether skipping one
evaluate_expr() call for a constant is better or reassigning the location.
This looks better than the last patch.
/* New lower bound is certainly >= bound at offet. */
offet/offset? But this comment is implied by the comment just two linesabove. So I am not sure it's really needed.
Given that cmpval is set all the way in partition_range_bsearch(), I
thought it would help to clarify why this code can assume it must be= 0. It is because a valid offset returned by
partition_range_bsearch() must correspond to a bound that it found to
be <= the probe bound passed to it.
/* Fetch the problem bound from lower datums list. */
This is fetching problematic key value rather than the whole problematicbound. I think the comment would be useful if it explains why cmpval -1 th
key is problematic but then that's evident from the prologue of
partition_rbound_cmp() so I am not sure if this comment is really required.
For example, we aren't adding a comment here+ overlap_location = ((PartitionRangeDatum *) + list_nth(spec->upperdatums, -cmpval - 1))->location;In the attached updated patch, I have tried to make the code and
comments for different cases consistent. Please have a look.
The comments look okay to me. I don't see a way to keep them short and yet
avoid reading the prologue of partition_range_bsearch(). And there is no
point in repeating a portion of that prologue at multiple places. So I am
fine with these set of comments.
Setting this CF entry as "RFC". Thanks.
--
Best Wishes,
Ashutosh
On Wed, Sep 23, 2020 at 10:22 PM Ashutosh Bapat
<ashutosh.bapat@2ndquadrant.com> wrote:
On Wed, 23 Sep 2020 at 14:41, Amit Langote <amitlangote09@gmail.com> wrote:
Setting this CF entry as "RFC". Thanks.
Great, thanks for your time on this.
--
Amit Langote
EnterpriseDB: http://www.enterprisedb.com
I looked this over and pushed it with some minor adjustments.
However, while I was looking at it I couldn't help noticing that
transformPartitionBoundValue's handling of collation concerns seems
less than sane. There are two things bugging me:
1. Why does it care about the expression's collation only when there's
a top-level CollateExpr? For example, that means we get an error for
regression=# create table p (f1 text collate "C") partition by list(f1);
CREATE TABLE
regression=# create table c1 partition of p for values in ('a' collate "POSIX");
ERROR: collation of partition bound value for column "f1" does not match partition key collation "C"
but not this:
regression=# create table c2 partition of p for values in ('a' || 'b' collate "POSIX");
CREATE TABLE
Given that we will override the expression's collation with the partition
column's collation anyway, I don't see why we have this check at all,
so my preference is to just rip out the entire stanza beginning with
"if (IsA(value, CollateExpr))". If we keep it, though, I think it needs
to do something else that is more general.
2. Nothing is doing assign_expr_collations() on the partition expression.
This can trivially be shown to cause problems:
regression=# create table p (f1 bool) partition by list(f1);
CREATE TABLE
regression=# create table cf partition of p for values in ('a' < 'b');
ERROR: could not determine which collation to use for string comparison
HINT: Use the COLLATE clause to set the collation explicitly.
If we want to rip out the collation mismatch error altogether, then
fixing #2 would just require inserting assign_expr_collations() before
the expression_planner() call. The other direction that would make
sense to me is to perform assign_expr_collations() after
coerce_to_target_type(), and then to complain if exprCollation()
isn't default and doesn't match the partition collation. In any
case a specific test for a CollateExpr seems quite wrong.
regards, tom lane
On Thu, Sep 24, 2020 at 7:19 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
I looked this over and pushed it with some minor adjustments.
Thank you.
However, while I was looking at it I couldn't help noticing that
transformPartitionBoundValue's handling of collation concerns seems
less than sane. There are two things bugging me:1. Why does it care about the expression's collation only when there's
a top-level CollateExpr? For example, that means we get an error forregression=# create table p (f1 text collate "C") partition by list(f1);
CREATE TABLE
regression=# create table c1 partition of p for values in ('a' collate "POSIX");
ERROR: collation of partition bound value for column "f1" does not match partition key collation "C"but not this:
regression=# create table c2 partition of p for values in ('a' || 'b' collate "POSIX");
CREATE TABLEGiven that we will override the expression's collation with the partition
column's collation anyway, I don't see why we have this check at all,
so my preference is to just rip out the entire stanza beginning with
"if (IsA(value, CollateExpr))". If we keep it, though, I think it needs
to do something else that is more general.2. Nothing is doing assign_expr_collations() on the partition expression.
This can trivially be shown to cause problems:regression=# create table p (f1 bool) partition by list(f1);
CREATE TABLE
regression=# create table cf partition of p for values in ('a' < 'b');
ERROR: could not determine which collation to use for string comparison
HINT: Use the COLLATE clause to set the collation explicitly.If we want to rip out the collation mismatch error altogether, then
fixing #2 would just require inserting assign_expr_collations() before
the expression_planner() call. The other direction that would make
sense to me is to perform assign_expr_collations() after
coerce_to_target_type(), and then to complain if exprCollation()
isn't default and doesn't match the partition collation. In any
case a specific test for a CollateExpr seems quite wrong.
I tried implementing that as attached and one test failed:
create table test_part_coll_posix (a text) partition by range (a
collate "POSIX");
...
create table test_part_coll_cast2 partition of test_part_coll_posix
for values from (name 's') to ('z');
+ERROR: collation of partition bound value for column "a" does not
match partition key collation "POSIX"
+LINE 1: ...ion of test_part_coll_posix for values from (name 's') to ('...
I dug up the discussion which resulted in this test being added and
found that Peter E had opined that this failure should not occur [1]/messages/by-id/04661508-b6f5-177e-6f6b-c4cb8426b9f0@2ndquadrant.com.
Maybe that is why I put that half-baked guard consisting of checking
if the erroneous collation comes from an explicit COLLATE clause. Now
I think maybe giving an error is alright but we should tell in the
DETAIL message what the expression's collation is, like as follows:
create table test_part_coll_cast2 partition of test_part_coll_posix
for values from (name 's') to ('z');
+ERROR: collation of partition bound value for column "a" does not
match partition key collation "POSIX"
+LINE 1: ...ion of test_part_coll_posix for values from (name 's') to ('...
+ ^
+DETAIL: The collation of partition bound value is "C".
--
Amit Langote
EnterpriseDB: http://www.enterprisedb.com
[1]: /messages/by-id/04661508-b6f5-177e-6f6b-c4cb8426b9f0@2ndquadrant.com
Attachments:
partition-bound-collation-check.patchapplication/octet-stream; name=partition-bound-collation-check.patchDownload
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 164312d..68a8a03 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -4172,6 +4172,7 @@ transformPartitionBoundValue(ParseState *pstate, Node *val,
Oid partCollation)
{
Node *value;
+ Oid exprCollOid;
/* Transform raw parsetree */
value = transformExpr(pstate, val, EXPR_KIND_PARTITION_BOUND);
@@ -4184,50 +4185,6 @@ transformPartitionBoundValue(ParseState *pstate, Node *val,
Assert(!contain_var_clause(value));
/*
- * Check that the input expression's collation is compatible with one
- * specified for the parent's partition key (partcollation). Don't throw
- * an error if it's the default collation which we'll replace with the
- * parent's collation anyway.
- */
- if (IsA(value, CollateExpr))
- {
- Oid exprCollOid = exprCollation(value);
-
- /*
- * Check we have a collation iff it is a collatable type. The only
- * expected failures here are (1) COLLATE applied to a noncollatable
- * type, or (2) partition bound expression had an unresolved
- * collation. But we might as well code this to be a complete
- * consistency check.
- */
- if (type_is_collatable(colType))
- {
- if (!OidIsValid(exprCollOid))
- ereport(ERROR,
- (errcode(ERRCODE_INDETERMINATE_COLLATION),
- errmsg("could not determine which collation to use for partition bound expression"),
- errhint("Use the COLLATE clause to set the collation explicitly.")));
- }
- else
- {
- if (OidIsValid(exprCollOid))
- ereport(ERROR,
- (errcode(ERRCODE_DATATYPE_MISMATCH),
- errmsg("collations are not supported by type %s",
- format_type_be(colType))));
- }
-
- if (OidIsValid(exprCollOid) &&
- exprCollOid != DEFAULT_COLLATION_OID &&
- exprCollOid != partCollation)
- ereport(ERROR,
- (errcode(ERRCODE_DATATYPE_MISMATCH),
- errmsg("collation of partition bound value for column \"%s\" does not match partition key collation \"%s\"",
- colName, get_collation_name(partCollation)),
- parser_errposition(pstate, exprLocation(value))));
- }
-
- /*
* Coerce to the correct type. This might cause an explicit coercion step
* to be added on top of the expression, which must be evaluated before
* returning the result to the caller.
@@ -4247,6 +4204,35 @@ transformPartitionBoundValue(ParseState *pstate, Node *val,
format_type_be(colType), colName),
parser_errposition(pstate, exprLocation(val))));
+ /* Fix up collation information */
+ assign_expr_collations(pstate, value);
+
+ /*
+ * Check that the input expression's assigned collation is compatible with
+ * one specified for the parent's partition key (partcollation). Don't
+ * throw an error if it's the default collation which we'll replace with
+ * the parent's collation anyway.
+ */
+ exprCollOid = exprCollation(value);
+
+ /* Complain if COLLATE seems to be applied to an uncollatable type. */
+ if (!type_is_collatable(colType) && OidIsValid(exprCollOid))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("collations are not supported by type %s",
+ format_type_be(colType))));
+
+ if (OidIsValid(exprCollOid) &&
+ exprCollOid != DEFAULT_COLLATION_OID &&
+ exprCollOid != partCollation)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("collation of partition bound value for column \"%s\" does not match partition key collation \"%s\"",
+ colName, get_collation_name(partCollation)),
+ parser_errposition(pstate, exprLocation(value)),
+ errdetail("The collation of partition bound value is \"%s\".",
+ get_collation_name(exprCollOid))));
+
/*
* Evaluate the expression, if needed, assigning the partition key's data
* type and collation to the resulting Const node.
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 41dce69..ec900ac 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -1034,6 +1034,7 @@ create table test_part_coll partition of test_part_coll_posix for values from ('
ERROR: collation of partition bound value for column "a" does not match partition key collation "POSIX"
LINE 1: ...artition of test_part_coll_posix for values from ('a' collat...
^
+DETAIL: The collation of partition bound value is "C".
-- ok
create table test_part_coll partition of test_part_coll_posix for values from ('a' collate "POSIX") to ('g');
-- ok
@@ -1044,10 +1045,15 @@ create table test_part_coll_cast partition of test_part_coll_posix for values fr
ERROR: collation of partition bound value for column "a" does not match partition key collation "POSIX"
LINE 1: ...ion of test_part_coll_posix for values from (name 'm' collat...
^
+DETAIL: The collation of partition bound value is "C".
-- ok
create table test_part_coll_cast partition of test_part_coll_posix for values from (name 'm' collate "POSIX") to ('s');
-- ok; partition collation silently overrides the default collation of type 'name'
create table test_part_coll_cast2 partition of test_part_coll_posix for values from (name 's') to ('z');
+ERROR: collation of partition bound value for column "a" does not match partition key collation "POSIX"
+LINE 1: ...ion of test_part_coll_posix for values from (name 's') to ('...
+ ^
+DETAIL: The collation of partition bound value is "C".
drop table test_part_coll_posix;
-- Partition bound in describe output
\d+ part_b
[ cc'ing Peter, since his opinion seems to have got us here in the first place ]
Amit Langote <amitlangote09@gmail.com> writes:
On Thu, Sep 24, 2020 at 7:19 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
However, while I was looking at it I couldn't help noticing that
transformPartitionBoundValue's handling of collation concerns seems
less than sane. There are two things bugging me:1. Why does it care about the expression's collation only when there's
a top-level CollateExpr? For example, that means we get an error forregression=# create table p (f1 text collate "C") partition by list(f1);
CREATE TABLE
regression=# create table c1 partition of p for values in ('a' collate "POSIX");
ERROR: collation of partition bound value for column "f1" does not match partition key collation "C"but not this:
regression=# create table c2 partition of p for values in ('a' || 'b' collate "POSIX");
CREATE TABLEGiven that we will override the expression's collation with the partition
column's collation anyway, I don't see why we have this check at all,
so my preference is to just rip out the entire stanza beginning with
"if (IsA(value, CollateExpr))". If we keep it, though, I think it needs
to do something else that is more general.
I dug up the discussion which resulted in this test being added and
found that Peter E had opined that this failure should not occur [1].
Well, I agree with Peter to that extent, but my opinion is that *none*
of these cases ought to be errors. What we're doing here is performing
an implicit assignment-level coercion of the expression to the type of
the column, and changing the collation is allowed as part of that:
regression=# create table foo (f1 text collate "C");
CREATE TABLE
regression=# insert into foo values ('a' COLLATE "POSIX");
INSERT 0 1
regression=# update foo set f1 = 'b' COLLATE "POSIX";
UPDATE 1
So I find it completely inconsistent that the partitioning logic
complains about equivalent cases. I think we should just rip the
whole thing out, as per the attached draft. This causes several
regression test results to change, but AFAICS those are only there
to exercise the error tests that I think we should get rid of.
regards, tom lane
Attachments:
remove-useless-collation-check-wip.patchtext/x-diff; charset=us-ascii; name=remove-useless-collation-check-wip.patchDownload
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 164312d60e..0dc03dd984 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -4183,50 +4183,6 @@ transformPartitionBoundValue(ParseState *pstate, Node *val,
*/
Assert(!contain_var_clause(value));
- /*
- * Check that the input expression's collation is compatible with one
- * specified for the parent's partition key (partcollation). Don't throw
- * an error if it's the default collation which we'll replace with the
- * parent's collation anyway.
- */
- if (IsA(value, CollateExpr))
- {
- Oid exprCollOid = exprCollation(value);
-
- /*
- * Check we have a collation iff it is a collatable type. The only
- * expected failures here are (1) COLLATE applied to a noncollatable
- * type, or (2) partition bound expression had an unresolved
- * collation. But we might as well code this to be a complete
- * consistency check.
- */
- if (type_is_collatable(colType))
- {
- if (!OidIsValid(exprCollOid))
- ereport(ERROR,
- (errcode(ERRCODE_INDETERMINATE_COLLATION),
- errmsg("could not determine which collation to use for partition bound expression"),
- errhint("Use the COLLATE clause to set the collation explicitly.")));
- }
- else
- {
- if (OidIsValid(exprCollOid))
- ereport(ERROR,
- (errcode(ERRCODE_DATATYPE_MISMATCH),
- errmsg("collations are not supported by type %s",
- format_type_be(colType))));
- }
-
- if (OidIsValid(exprCollOid) &&
- exprCollOid != DEFAULT_COLLATION_OID &&
- exprCollOid != partCollation)
- ereport(ERROR,
- (errcode(ERRCODE_DATATYPE_MISMATCH),
- errmsg("collation of partition bound value for column \"%s\" does not match partition key collation \"%s\"",
- colName, get_collation_name(partCollation)),
- parser_errposition(pstate, exprLocation(value))));
- }
-
/*
* Coerce to the correct type. This might cause an explicit coercion step
* to be added on top of the expression, which must be evaluated before
@@ -4253,6 +4209,7 @@ transformPartitionBoundValue(ParseState *pstate, Node *val,
*/
if (!IsA(value, Const))
{
+ assign_expr_collations(pstate, value);
value = (Node *) expression_planner((Expr *) value);
value = (Node *) evaluate_expr((Expr *) value, colType, colTypmod,
partCollation);
On Fri, Sep 25, 2020 at 12:02 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
[ cc'ing Peter, since his opinion seems to have got us here in the first place ]
Amit Langote <amitlangote09@gmail.com> writes:
On Thu, Sep 24, 2020 at 7:19 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
However, while I was looking at it I couldn't help noticing that
transformPartitionBoundValue's handling of collation concerns seems
less than sane. There are two things bugging me:1. Why does it care about the expression's collation only when there's
a top-level CollateExpr? For example, that means we get an error forregression=# create table p (f1 text collate "C") partition by list(f1);
CREATE TABLE
regression=# create table c1 partition of p for values in ('a' collate "POSIX");
ERROR: collation of partition bound value for column "f1" does not match partition key collation "C"but not this:
regression=# create table c2 partition of p for values in ('a' || 'b' collate "POSIX");
CREATE TABLEGiven that we will override the expression's collation with the partition
column's collation anyway, I don't see why we have this check at all,
so my preference is to just rip out the entire stanza beginning with
"if (IsA(value, CollateExpr))". If we keep it, though, I think it needs
to do something else that is more general.I dug up the discussion which resulted in this test being added and
found that Peter E had opined that this failure should not occur [1].Well, I agree with Peter to that extent, but my opinion is that *none*
of these cases ought to be errors. What we're doing here is performing
an implicit assignment-level coercion of the expression to the type of
the column, and changing the collation is allowed as part of that:regression=# create table foo (f1 text collate "C");
CREATE TABLE
regression=# insert into foo values ('a' COLLATE "POSIX");
INSERT 0 1
regression=# update foo set f1 = 'b' COLLATE "POSIX";
UPDATE 1So I find it completely inconsistent that the partitioning logic
complains about equivalent cases.
My perhaps wrong impression was that the bound expression that is
specified when creating a partition is not as such being *assigned* to
the key column, but now that I think about it some more, that doesn't
matter.
I think we should just rip the
whole thing out, as per the attached draft. This causes several
regression test results to change, but AFAICS those are only there
to exercise the error tests that I think we should get rid of.
Yeah, I can see no other misbehavior resulting from this.
--
Amit Langote
EnterpriseDB: http://www.enterprisedb.com
Amit Langote <amitlangote09@gmail.com> writes:
On Fri, Sep 25, 2020 at 12:02 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Well, I agree with Peter to that extent, but my opinion is that *none*
of these cases ought to be errors. What we're doing here is performing
an implicit assignment-level coercion of the expression to the type of
the column, and changing the collation is allowed as part of that:regression=# create table foo (f1 text collate "C");
CREATE TABLE
regression=# insert into foo values ('a' COLLATE "POSIX");
INSERT 0 1
regression=# update foo set f1 = 'b' COLLATE "POSIX";
UPDATE 1So I find it completely inconsistent that the partitioning logic
complains about equivalent cases.
My perhaps wrong impression was that the bound expression that is
specified when creating a partition is not as such being *assigned* to
the key column, but now that I think about it some more, that doesn't
matter.
I think we should just rip the
whole thing out, as per the attached draft. This causes several
regression test results to change, but AFAICS those are only there
to exercise the error tests that I think we should get rid of.
Yeah, I can see no other misbehavior resulting from this.
OK, I'll clean up the regression test cases and push that.
(Although this could be claimed to be a bug, I do not feel
a need to back-patch the behavioral change.)
regards, tom lane
On Tue, Sep 29, 2020 at 2:01 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Amit Langote <amitlangote09@gmail.com> writes:
On Fri, Sep 25, 2020 at 12:02 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Well, I agree with Peter to that extent, but my opinion is that *none*
of these cases ought to be errors. What we're doing here is performing
an implicit assignment-level coercion of the expression to the type of
the column, and changing the collation is allowed as part of that:regression=# create table foo (f1 text collate "C");
CREATE TABLE
regression=# insert into foo values ('a' COLLATE "POSIX");
INSERT 0 1
regression=# update foo set f1 = 'b' COLLATE "POSIX";
UPDATE 1So I find it completely inconsistent that the partitioning logic
complains about equivalent cases.My perhaps wrong impression was that the bound expression that is
specified when creating a partition is not as such being *assigned* to
the key column, but now that I think about it some more, that doesn't
matter.I think we should just rip the
whole thing out, as per the attached draft. This causes several
regression test results to change, but AFAICS those are only there
to exercise the error tests that I think we should get rid of.Yeah, I can see no other misbehavior resulting from this.
OK, I'll clean up the regression test cases and push that.
Thanks.
(Although this could be claimed to be a bug, I do not feel
a need to back-patch the behavioral change.)
Agreed. The assign_expr_collations() omission was indeed a bug.
--
Amit Langote
EnterpriseDB: http://www.enterprisedb.com