Default Partition for Range
Hello,
Many were in favour of the default partition for tables partitioned by
range [1]Discussion on default partition: */messages/by-id/CAOgcT0PLPge=5U6=GU5SnC3_8yutCbWWOiUva3Cw94M9zpbvgQ@mail.gmail.com </messages/by-id/CAOgcT0PLPge=5U6=GU5SnC3_8yutCbWWOiUva3Cw94M9zpbvgQ@mail.gmail.com>*.
Please find attached the WIP patch for the same which can be applied over
the default_partition_v12.patch.
Syntax: Same as agreed for list:
CREATE PARTITION <part_name> PARTITION OF <parent_tbl> DEFAULT;
Default Constraint:
Negation constraints of all existing partitions.
One case:
CREATE TABLE range1 (a int, b int) PARTITION by range (a);
CREATE TABLE range1_1 PARTITION OF range1 FOR VALUES FROM (1) TO (5);
CREATE TABLE range1_2 PARTITION OF range1 FOR VALUES FROM (7) TO (10);
CREATE TABLE range1_def PARTITION OF range1 DEFAULT;
\d+ range1_def
Table "public.range1_def"
Column | Type | Collation | Nullable | Default | Storage | Stats target
| Description
--------+---------+-----------+----------+---------+--------
-+--------------+-------------
a | integer | | not null | | plain |
|
b | integer | | | | plain |
|
Partition of: range1 DEFAULT
Partition constraint: (((a < 1) OR (a >= 5)) AND ((a < 7) OR (a >= 10)))
It still needs more work:
1. Handling addition of new partition after default, insertion of data, few
more bugs
2. Documentation
3. Regression tests
[1]: Discussion on default partition: */messages/by-id/CAOgcT0PLPge=5U6=GU5SnC3_8yutCbWWOiUva3Cw94M9zpbvgQ@mail.gmail.com </messages/by-id/CAOgcT0PLPge=5U6=GU5SnC3_8yutCbWWOiUva3Cw94M9zpbvgQ@mail.gmail.com>*
*/messages/by-id/CAOgcT0PLPge=5U6=GU5SnC3_8yutCbWWOiUva3Cw94M9zpbvgQ@mail.gmail.com
</messages/by-id/CAOgcT0PLPge=5U6=GU5SnC3_8yutCbWWOiUva3Cw94M9zpbvgQ@mail.gmail.com>*
--
Beena Emerson
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
default_range_partition_WIP.patchapplication/octet-stream; name=default_range_partition_WIP.patchDownload+283-112
On Mon, May 22, 2017 at 7:27 AM, Beena Emerson <memissemerson@gmail.com> wrote:
Hello,
Many were in favour of the default partition for tables partitioned by range
[1].
Please find attached the WIP patch for the same which can be applied over
the default_partition_v12.patch.Syntax: Same as agreed for list:
CREATE PARTITION <part_name> PARTITION OF <parent_tbl> DEFAULT;Default Constraint:
Negation constraints of all existing partitions.One case:
CREATE TABLE range1 (a int, b int) PARTITION by range (a);
CREATE TABLE range1_1 PARTITION OF range1 FOR VALUES FROM (1) TO (5);
CREATE TABLE range1_2 PARTITION OF range1 FOR VALUES FROM (7) TO (10);
CREATE TABLE range1_def PARTITION OF range1 DEFAULT;
\d+ range1_def
Table "public.range1_def"
Column | Type | Collation | Nullable | Default | Storage | Stats target
| Description
--------+---------+-----------+----------+---------+---------+--------------+-------------
a | integer | | not null | | plain |
|
b | integer | | | | plain |
|
Partition of: range1 DEFAULT
Partition constraint: (((a < 1) OR (a >= 5)) AND ((a < 7) OR (a >= 10)))
Would it be more readable if this reads as NOT
(constraint_for_partition_1 || constraint_for_partition_2 ||
constraint_for_partition_3)? That way, the code to create
constraint_for_partition_n can be reused and there's high chance that
we will end up with consistent constraints?
It still needs more work:
1. Handling addition of new partition after default, insertion of data, few
more bugs
2. Documentation
3. Regression tests
I think, the default partition support for all the strategies
supporting it should be a single patch since most of the code will be
shared?
--
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Hello,
On Mon, May 22, 2017 at 11:29 AM, Ashutosh Bapat <
ashutosh.bapat@enterprisedb.com> wrote:
On Mon, May 22, 2017 at 7:27 AM, Beena Emerson <memissemerson@gmail.com>
wrote:
Would it be more readable if this reads as NOT
(constraint_for_partition_1 || constraint_for_partition_2 ||
constraint_for_partition_3)? That way, the code to create
constraint_for_partition_n can be reused and there's high chance that
we will end up with consistent constraints?
PFA the patch which gives the default partition constraint as you have
suggested.
It still needs more work:
1. Handling addition of new partition after default, insertion of data,few
more bugs
2. Documentation
3. Regression testsI think, the default partition support for all the strategies
supporting it should be a single patch since most of the code will be
shared?
Dependency on list default patch:
gram.y : adding the syntax
partition.c:
- default_index member in PartitionBoundInfoData;
- check_new_partition_bound : the code for adding a partition after default
has been completely reused.
- isDefaultPartitionBound function is used.
The structures are same but the handling of list and range is very
different and the code mostly has the switch case construct to handle the
2 separately. I feel it could be taken separately.
As suggested in the default list thread, I have created
a partition_bound_has_default macro and avoided usage of has_default in
range code. This has to be used for list as well.
Another suggestion for list which has to be implemented in this patch in
removal of PARTITION_DEFAULT. Ii have not done this in this version.
--
Beena Emerson
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
default_range_partition.patchapplication/octet-stream; name=default_range_partition.patchDownload+272-101
Hi Beena,
I went through your patch, and here are some of my comments:
- For generating a qual for default range partition, instead of scanning
for all
the children and collecting all the boundspecs, how about creating and
negating
an expression from the lists of lowerdatums and upperdatums in boundinfo.
- Wrong comment:
+ int default_index; /* Index of the default list partition. */
- Suggested by Robert earlier on default list partitioning thread, instead
of
abbreviating is_def/found_def use is(found)_default etc.
- unrelated change:
- List *all_parts;
+ List *all_parts;
- typo: partiton should be partition, similar typo at other places too.
+ * A non-default range partiton table does not currently allow partition
keys
- Useless hunk for this patch:
- Oid defid;
+ Oid defid;
- better to use IsA here, instead of doing explicit check:
+ bound->content[i] = (datum->type == T_DefElem)
+ ? RANGE_DATUM_DEFAULT
- It is better to use head of either lowerdatums or upperdatums list to
verify
if its a default partition and get rid of the constant PARTITION_DEFAULT
altogether.
+ {
+ /*
+ * If the partition is the default partition switch back to
+ * PARTITION_STRATEGY_RANGE
+ */
+ if (spec->strategy == PARTITION_DEFAULT)
+ {
+ is_def = true;
+ result_spec->strategy = PARTITION_STRATEGY_RANGE;
+ }
- I am sorry, but I could not understand following hunk. Does this change
really
belongs to this patch? If not, it will be better to handle it separately.
@@ -2242,33 +2387,16 @@ get_partition_for_tuple(PartitionDispatch *pd,
ecxt->ecxt_scantuple = slot;
FormPartitionKeyDatum(parent, slot, estate, values, isnull);
- if (key->strategy == PARTITION_STRATEGY_RANGE)
+ if (key->strategy == PARTITION_STRATEGY_LIST && isnull[0])
{
/*
- * Since we cannot route tuples with NULL partition keys through
- * a range-partitioned table, simply return that no partition
- * exists
+ * A null partition key is only acceptable if null-accepting list
+ * partition exists.
*/
- for (i = 0; i < key->partnatts; i++)
- {
- if (isnull[i])
- {
- *failed_at = parent;
- *failed_slot = slot;
- result = -1;
- goto error_exit;
- }
- }
+ if (partition_bound_accepts_nulls(partdesc->boundinfo))
+ cur_index = partdesc->boundinfo->null_index;
- Change not related to this patch:
- List *all_parts;
+ List *all_parts;
- In the function get_qual_from_partbound() is_def is always going to be
false
for range partition, the function get_qual_for_range can be directly passed
false instead.
- Following comment for function get_qual_for_range_default() implies that
this
function returns bool, but the actually it returns a List.
+ *
+ * If DEFAULT is the only partiton for the table then this returns TRUE.
+ *
Regards,
Jeevan Ladhe
On Wed, May 24, 2017 at 12:52 AM, Beena Emerson <memissemerson@gmail.com>
wrote:
Show quoted text
Hello,
On Mon, May 22, 2017 at 11:29 AM, Ashutosh Bapat <
ashutosh.bapat@enterprisedb.com> wrote:On Mon, May 22, 2017 at 7:27 AM, Beena Emerson <memissemerson@gmail.com>
wrote:
Would it be more readable if this reads as NOT
(constraint_for_partition_1 || constraint_for_partition_2 ||
constraint_for_partition_3)? That way, the code to create
constraint_for_partition_n can be reused and there's high chance that
we will end up with consistent constraints?PFA the patch which gives the default partition constraint as you have
suggested.It still needs more work:
1. Handling addition of new partition after default, insertion of data,few
more bugs
2. Documentation
3. Regression testsI think, the default partition support for all the strategies
supporting it should be a single patch since most of the code will be
shared?Dependency on list default patch:
gram.y : adding the syntax
partition.c:
- default_index member in PartitionBoundInfoData;
- check_new_partition_bound : the code for adding a partition after
default has been completely reused.
- isDefaultPartitionBound function is used.The structures are same but the handling of list and range is very
different and the code mostly has the switch case construct to handle the
2 separately. I feel it could be taken separately.As suggested in the default list thread, I have created
a partition_bound_has_default macro and avoided usage of has_default in
range code. This has to be used for list as well.
Another suggestion for list which has to be implemented in this patch in
removal of PARTITION_DEFAULT. Ii have not done this in this version.--
Beena Emerson
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Mon, May 29, 2017 at 3:22 PM, Jeevan Ladhe
<jeevan.ladhe@enterprisedb.com> wrote:
Hi Beena,
I went through your patch, and here are some of my comments:
Thank you for your comments. I will take care of them in the next
version of patch.
- I am sorry, but I could not understand following hunk. Does this change
really
belongs to this patch? If not, it will be better to handle it separately.@@ -2242,33 +2387,16 @@ get_partition_for_tuple(PartitionDispatch *pd,
ecxt->ecxt_scantuple = slot;
FormPartitionKeyDatum(parent, slot, estate, values, isnull);- if (key->strategy == PARTITION_STRATEGY_RANGE) + if (key->strategy == PARTITION_STRATEGY_LIST && isnull[0]) { /* - * Since we cannot route tuples with NULL partition keys through - * a range-partitioned table, simply return that no partition - * exists + * A null partition key is only acceptable if null-accepting list + * partition exists. */
In RANGE, initially NULL was not allowed, now NULL is routed to
default. I have only removed the check for null in RANGE and kept the
check for null partition in case of list.
--
Beena Emerson
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Hello,
PFA the updated patch.
Dependent patch default_partition_v17.patch [1]http://www.mail-archive.com/pgsql-hackers@postgresql.org/msg315573.html
This patch adds regression tests and removes the separate function to
get default partition qual.
On Mon, May 29, 2017 at 3:22 PM, Jeevan Ladhe
<jeevan.ladhe@enterprisedb.com> wrote:
Hi Beena,
I went through your patch, and here are some of my comments:
- For generating a qual for default range partition, instead of scanning for
all
the children and collecting all the boundspecs, how about creating and
negating
an expression from the lists of lowerdatums and upperdatums in boundinfo.
Unlike list, range partition can be for multiple columns and the
expressions get complicated. I have used the same logic of looping
through different partitions and negating the ORed expr. However, this
is now done under get_qual_for_range.
- Wrong comment: + int default_index; /* Index of the default list partition. */
Comment is made generic in the dependent patch.
- Suggested by Robert earlier on default list partitioning thread, instead
of
abbreviating is_def/found_def use is(found)_default etc.
corrected.
- unrelated change: - List *all_parts; + List *all_parts;
undone.
- typo: partiton should be partition, similar typo at other places too. + * A non-default range partiton table does not currently allow partition keys
rectified.
- Useless hunk for this patch: - Oid defid; + Oid defid;
undone.
- better to use IsA here, instead of doing explicit check: + bound->content[i] = (datum->type == T_DefElem) + ? RANGE_DATUM_DEFAULT
Modified
- It is better to use head of either lowerdatums or upperdatums list to
verify
if its a default partition and get rid of the constant PARTITION_DEFAULT
altogether.
modified this part as necessary.
- In the function get_qual_from_partbound() is_def is always going to be
false
for range partition, the function get_qual_for_range can be directly passed
false instead.- Following comment for function get_qual_for_range_default() implies that this function returns bool, but the actually it returns a List. + * + * If DEFAULT is the only partiton for the table then this returns TRUE. + *
Updated.
[1]: http://www.mail-archive.com/pgsql-hackers@postgresql.org/msg315573.html
--
Beena Emerson
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
default_range_partition_v2.patchapplication/octet-stream; name=default_range_partition_v2.patchDownload+351-85
On Wed, May 31, 2017 at 5:53 PM, Beena Emerson <memissemerson@gmail.com> wrote:
Hello,
PFA the updated patch.
Dependent patch default_partition_v17.patch [1]
This patch adds regression tests and removes the separate function to
get default partition qual.On Mon, May 29, 2017 at 3:22 PM, Jeevan Ladhe
<jeevan.ladhe@enterprisedb.com> wrote:Hi Beena,
I went through your patch, and here are some of my comments:
- For generating a qual for default range partition, instead of scanning for
all
the children and collecting all the boundspecs, how about creating and
negating
an expression from the lists of lowerdatums and upperdatums in boundinfo.Unlike list, range partition can be for multiple columns and the
expressions get complicated. I have used the same logic of looping
through different partitions and negating the ORed expr. However, this
is now done under get_qual_for_range.- Wrong comment: + int default_index; /* Index of the default list partition. */Comment is made generic in the dependent patch.
- Suggested by Robert earlier on default list partitioning thread, instead
of
abbreviating is_def/found_def use is(found)_default etc.corrected.
- unrelated change: - List *all_parts; + List *all_parts;undone.
- typo: partiton should be partition, similar typo at other places too. + * A non-default range partiton table does not currently allow partition keysrectified.
- Useless hunk for this patch: - Oid defid; + Oid defid;undone.
- better to use IsA here, instead of doing explicit check: + bound->content[i] = (datum->type == T_DefElem) + ? RANGE_DATUM_DEFAULTModified
- It is better to use head of either lowerdatums or upperdatums list to
verify
if its a default partition and get rid of the constant PARTITION_DEFAULT
altogether.modified this part as necessary.
- In the function get_qual_from_partbound() is_def is always going to be
false
for range partition, the function get_qual_for_range can be directly passed
false instead.- Following comment for function get_qual_for_range_default() implies that this function returns bool, but the actually it returns a List. + * + * If DEFAULT is the only partiton for the table then this returns TRUE. + *Updated.
[1] http://www.mail-archive.com/pgsql-hackers@postgresql.org/msg315573.html
Hi Beena,
I had a look at the patch from the angle of aesthetics and there are a
few cosmetic changes I might suggest. Please have a look at the
attached patch and if you agree with those changes you may merge it on
your patch. The patch also fixes a couple of more spelling mistakes
unrelated to this patch.
--
Regards,
Rafia Sabih
EnterpriseDB: http://www.enterprisedb.com/
Attachments:
cosmetic_range_default_partition.patchapplication/octet-stream; name=cosmetic_range_default_partition.patchDownload+44-3
On Fri, Jun 2, 2017 at 4:12 PM, Rafia Sabih
<rafia.sabih@enterprisedb.com> wrote:
On Wed, May 31, 2017 at 5:53 PM, Beena Emerson <memissemerson@gmail.com> wrote:
Hi Beena,
I had a look at the patch from the angle of aesthetics and there are a
few cosmetic changes I might suggest. Please have a look at the
attached patch and if you agree with those changes you may merge it on
your patch. The patch also fixes a couple of more spelling mistakes
unrelated to this patch.
I think if you have found spelling mistakes unrelated to this patch,
then it is better to submit those as a separate patch in a new thread.
--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Fri, Jun 2, 2017 at 8:09 AM, Amit Kapila <amit.kapila16@gmail.com> wrote:
I think if you have found spelling mistakes unrelated to this patch,
then it is better to submit those as a separate patch in a new thread.
+1.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Fri, Jun 2, 2017 at 5:48 PM, Robert Haas <robertmhaas@gmail.com> wrote:
On Fri, Jun 2, 2017 at 8:09 AM, Amit Kapila <amit.kapila16@gmail.com> wrote:
I think if you have found spelling mistakes unrelated to this patch,
then it is better to submit those as a separate patch in a new thread.+1.
Sure, attached is the version without those changes.
--
Regards,
Rafia Sabih
EnterpriseDB: http://www.enterprisedb.com/
Attachments:
cosmetic_range_default_partition_v2.patchapplication/octet-stream; name=cosmetic_range_default_partition_v2.patchDownload+42-1
Hello,
On Sun, Jun 4, 2017 at 9:26 AM, Rafia Sabih
<rafia.sabih@enterprisedb.com> wrote:
On Fri, Jun 2, 2017 at 5:48 PM, Robert Haas <robertmhaas@gmail.com> wrote:
On Fri, Jun 2, 2017 at 8:09 AM, Amit Kapila <amit.kapila16@gmail.com> wrote:
I think if you have found spelling mistakes unrelated to this patch,
then it is better to submit those as a separate patch in a new thread.+1.
Sure, attached is the version without those changes.
Thank you for your comments. I have applied most of the changes. The
regression comment change from 'fail' -> ' Following statement should
fail' and 'ok' -> 'Following should complete successfully' is ignored
since other tests in the file had similar comments
The new patch is rebased over default_partition_v18.patch
[http://www.mail-archive.com/pgsql-hackers@postgresql.org/msg315831.html]
--
Beena Emerson
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
default_range_partition_v3.patchapplication/octet-stream; name=default_range_partition_v3.patchDownload+327-85
On Mon, Jun 5, 2017 at 1:43 AM, Beena Emerson <memissemerson@gmail.com> wrote:
The new patch is rebased over default_partition_v18.patch
[http://www.mail-archive.com/pgsql-hackers@postgresql.org/msg315831.html]
I have done the initial review of the patch, I have few comments.
+ if ((lower->content[0] == RANGE_DATUM_DEFAULT))
+ {
+ if (partition_bound_has_default(partdesc->boundinfo))
+ {
+ overlap = true;
+ with = partdesc->boundinfo->default_index;
+ }
I think we can change if ((lower->content[0] == RANGE_DATUM_DEFAULT))
check to if (spec->is_default) that way it will be consistent with the
check in the PARTITION_STRATEGY_LIST. Or we can move this complete
check outside of the switch.
-------------
- PartitionRangeDatum *datum = castNode(PartitionRangeDatum, lfirst(lc));
+ Node *node = lfirst(lc);
+ PartitionRangeDatum *datum;
+
+ datum = castNode(PartitionRangeDatum, node);
Why do you need to change this?
--------------
+ if (!datums)
+ bound->content[i] = RANGE_DATUM_DEFAULT;
+
Better to check if (datums != NULL) for non boolean types.
-------------
+ if (content1[i] == RANGE_DATUM_DEFAULT ||
+ content2[i] == RANGE_DATUM_DEFAULT)
+ {
+ if (content1[i] == content2[i])
+ return 0;
+ else if (content1[i] == RANGE_DATUM_DEFAULT)
+ return -1;
I don't see any comments why default partition should be considered
smallest in the index comparison. For negative infinity, it's pretty
obvious by the enum name itself.
-------------
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Hello Dilip,
On Mon, Jun 5, 2017 at 8:44 PM, Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Mon, Jun 5, 2017 at 1:43 AM, Beena Emerson <memissemerson@gmail.com> wrote:
The new patch is rebased over default_partition_v18.patch
[http://www.mail-archive.com/pgsql-hackers@postgresql.org/msg315831.html]I have done the initial review of the patch, I have few comments.
Thank you.
+ if ((lower->content[0] == RANGE_DATUM_DEFAULT)) + { + if (partition_bound_has_default(partdesc->boundinfo)) + { + overlap = true; + with = partdesc->boundinfo->default_index; + }I think we can change if ((lower->content[0] == RANGE_DATUM_DEFAULT))
check to if (spec->is_default) that way it will be consistent with the
check in the PARTITION_STRATEGY_LIST. Or we can move this complete
check outside of the switch.
I have now moved the is_default check for both list and range outside
the switch case.
-------------
- PartitionRangeDatum *datum = castNode(PartitionRangeDatum, lfirst(lc)); + Node *node = lfirst(lc); + PartitionRangeDatum *datum; + + datum = castNode(PartitionRangeDatum, node);Why do you need to change this?
I forgot to remove it.
It was needed for previous version of patch but no longer needed and
hence reverted this change.
--------------
+ if (!datums) + bound->content[i] = RANGE_DATUM_DEFAULT; +Better to check if (datums != NULL) for non boolean types.
done.
------------- + if (content1[i] == RANGE_DATUM_DEFAULT || + content2[i] == RANGE_DATUM_DEFAULT) + { + if (content1[i] == content2[i]) + return 0; + else if (content1[i] == RANGE_DATUM_DEFAULT) + return -1;I don't see any comments why default partition should be considered
smallest in the index comparison. For negative infinity, it's pretty
obvious by the enum name itself.
Default could be lowest or highest, no specific reason for putting it lowest.
I have not added any comments in this version.
--
Beena Emerson
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
default_range_partition_v4.patchapplication/octet-stream; name=default_range_partition_v4.patchDownload+319-106
Hello,
PFA the updated patch.
This is rebased over v21 patches of list partition.
(http://www.mail-archive.com/pgsql-hackers@postgresql.org/msg316818.html)
--
Beena Emerson
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
default_range_partition_v5.patchapplication/octet-stream; name=default_range_partition_v5.patchDownload+311-96
On Thu, Jun 15, 2017 at 11:20 AM, Beena Emerson <memissemerson@gmail.com> wrote:
Hello,
PFA the updated patch.
This is rebased over v21 patches of list partition.
(http://www.mail-archive.com/pgsql-hackers@postgresql.org/msg316818.html)
While testing I have noticed segmentation fault for a simple case.
create table test(a int, b int) partition by range(a,b);
create table test1 partition of test DEFAULT;
postgres=# drop table test1;
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
This is basically crashing in RelationBuildPartitionDesc, so I think
we don't have any test case for testing DEFAULT range partition where
partition key has more than one attribute. So I suggest we can add
such test case.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, Jun 20, 2017 at 6:57 PM, Dilip Kumar <dilipbalaut@gmail.com> wrote:
This is basically crashing in RelationBuildPartitionDesc, so I think
we don't have any test case for testing DEFAULT range partition where
partition key has more than one attribute. So I suggest we can add
such test case.
Some more comments.
<code>
- bound->datums = (Datum *) palloc0(key->partnatts * sizeof(Datum));
- bound->content = (RangeDatumContent *) palloc0(key->partnatts *
- sizeof(RangeDatumContent));
+ bound->datums = datums ? (Datum *) palloc0(key->partnatts * sizeof(Datum))
+ : NULL;
+ bound->content = (RangeDatumContent *) palloc0(
+ (datums ? key->partnatts : 1) * sizeof(RangeDatumContent));
bound->lower = lower;
i = 0;
+
+ /* If default, datums are NULL */
+ if (datums == NULL)
+ bound->content[i] = RANGE_DATUM_DEFAULT;
</code>
For the default partition we are only setting bound->content[0] to
default, but content for others key
attributes are not initialized. But later in the code, if the content
of the first key is RANGE_DATUM_DEFAULT then it should not access the
next content, but I see there are some exceptions. Which can access
uninitialized value?
for example see below code
<code>
for (i = 0; i < key->partnatts; i++)
{
+ if (rb_content[i] == RANGE_DATUM_DEFAULT) --> why it's going to
content for next parttion attribute, we never initialized that?
+ continue;
+
if (rb_content[i] != RANGE_DATUM_FINITE)
return rb_content[i] == RANGE_DATUM_NEG_INF ? -1 : 1;
</code>
Also
In RelatiobBuildPartitionDesc
<code>
for (j = 0; j < key->partnatts; j++)
{
-- Suppose first is RANGE_DATUM_DEFAULT then we should not check next
because that is never initialized. I think this is the cause of
the crash also what I have reported above.
----
if (rbounds[i]->content[j] == RANGE_DATUM_FINITE)
boundinfo->datums[i][j] =
datumCopy(rbounds[i]->datums[j],
key->parttypbyval[j],
key->parttyplen[j]);
/* Remember, we are storing the tri-state value. */
boundinfo->content[i][j] = rbounds[i]->content[j];
</code>
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Wed, Jun 21, 2017 at 8:57 AM, Dilip Kumar <dilipbalaut@gmail.com> wrote:
For the default partition we are only setting bound->content[0] to
default, but content for others key
attributes are not initialized. But later in the code, if the content
of the first key is RANGE_DATUM_DEFAULT then it should not access the
next content, but I see there are some exceptions. Which can access
uninitialized value?
I think somebody should do some testing of the existing code with
valgrind. And then apply the list-partitioning patch and this patch,
and do some more testing with valgrind. It seems to be really easy to
miss these uninitialized access problems during code review.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Wed, Jun 21, 2017 at 8:08 PM, Robert Haas <robertmhaas@gmail.com> wrote:
I think somebody should do some testing of the existing code with
valgrind. And then apply the list-partitioning patch and this patch,
and do some more testing with valgrind. It seems to be really easy to
miss these uninitialized access problems during code review.
I think it will help, but it may not help in all the scenarios
because most of the places we are allocating memory with palloc0 (
initialized with 0) but it never initialized with RANGE_DATUM_DEFAULT
except the first element (in the case of DEFAULT partition). And,
later they may be considered as RANGE_DATUM_FINITE (because its value
is 0).
One solution can be that if bound is DEFAULT then initialize with
RANGE_DATUM_DEFAULT for the complete content array for that partition
bound instead of just first. Otherwise, we need to be careful of
early exiting wherever we are looping the content array of the DEFAULT
bound.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Hi Beena,
I started testing and reviewing the patch. Can you update the patch as v5
patch does not apply cleanly on master?
Thank you,
Rahila Syed
On Wed, Jun 21, 2017 at 8:43 PM, Dilip Kumar <dilipbalaut@gmail.com> wrote:
Show quoted text
On Wed, Jun 21, 2017 at 8:08 PM, Robert Haas <robertmhaas@gmail.com>
wrote:I think somebody should do some testing of the existing code with
valgrind. And then apply the list-partitioning patch and this patch,
and do some more testing with valgrind. It seems to be really easy to
miss these uninitialized access problems during code review.I think it will help, but it may not help in all the scenarios
because most of the places we are allocating memory with palloc0 (
initialized with 0) but it never initialized with RANGE_DATUM_DEFAULT
except the first element (in the case of DEFAULT partition). And,
later they may be considered as RANGE_DATUM_FINITE (because its value
is 0).One solution can be that if bound is DEFAULT then initialize with
RANGE_DATUM_DEFAULT for the complete content array for that partition
bound instead of just first. Otherwise, we need to be careful of
early exiting wherever we are looping the content array of the DEFAULT
bound.--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Wed, Jun 28, 2017 at 12:34 PM, Rahila Syed <rahilasyed90@gmail.com> wrote:
Hi Beena,
I started testing and reviewing the patch. Can you update the patch as v5
patch does not apply cleanly on master?
I am currently working on Dilip's comments, I will update the patch soon.
--
Beena Emerson
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers