bugs or my fault

Started by Yudianto Prasetyoover 4 years ago8 messagesbugs
Jump to latest
#1Yudianto Prasetyo
mr.yudianto@gmail.com

Hello,

can you explain if this is a bug or my fault ?

I'm making a simple example of hash partition

CREATE TABLE myschema.customers(
id int,
status text,
arr numeric
) PARTITION BY HASH(id);

CREATE TABLE myschema.cust0 PARTITION OF myschema.customers
FOR VALUES WITH (modulus 3, remainder 0);

CREATE TABLE myschema.cust1 PARTITION OF myschema.customers
FOR VALUES WITH (modulus 3, remainder 1);

CREATE TABLE myschema.cust2 PARTITION OF myschema.customers
FOR VALUES WITH (modulus 3, remainder 2);

INSERT INTO myschema.customers VALUES (1,'ACTIVE',100), (2,'RECURRING',20),
(3,'EXPIRED',38), (4,'REACTIVATED',144);

when I run the query, the result is:

2 "RECURRING" 20
4 "REACTIVATED" 144

shouldn't be

3 "EXPIRED" 38

i run this in version

PostgreSQL 13.4, compiled by Visual C++ build 1914, 64-bit

Please help me

#2Yudianto Prasetyo
mr.yudianto@gmail.com
In reply to: Yudianto Prasetyo (#1)
Re: bugs or my fault

sorry I miss the query

select * from myschema.cust0;

On Wed, Sep 15, 2021 at 1:28 PM Yudianto Prasetyo <mr.yudianto@gmail.com>
wrote:

Show quoted text

Hello,

can you explain if this is a bug or my fault ?

I'm making a simple example of hash partition

CREATE TABLE myschema.customers(
id int,
status text,
arr numeric
) PARTITION BY HASH(id);

CREATE TABLE myschema.cust0 PARTITION OF myschema.customers
FOR VALUES WITH (modulus 3, remainder 0);

CREATE TABLE myschema.cust1 PARTITION OF myschema.customers
FOR VALUES WITH (modulus 3, remainder 1);

CREATE TABLE myschema.cust2 PARTITION OF myschema.customers
FOR VALUES WITH (modulus 3, remainder 2);

INSERT INTO myschema.customers VALUES (1,'ACTIVE',100),
(2,'RECURRING',20), (3,'EXPIRED',38), (4,'REACTIVATED',144);

when I run the query, the result is:

2 "RECURRING" 20
4 "REACTIVATED" 144

shouldn't be

3 "EXPIRED" 38

i run this in version

PostgreSQL 13.4, compiled by Visual C++ build 1914, 64-bit

Please help me

#3Peter Smith
smithpb2250@gmail.com
In reply to: Yudianto Prasetyo (#2)
Re: bugs or my fault

On Wed, Sep 15, 2021 at 7:08 PM Yudianto Prasetyo <mr.yudianto@gmail.com> wrote:

sorry I miss the query

select * from myschema.cust0;

On Wed, Sep 15, 2021 at 1:28 PM Yudianto Prasetyo <mr.yudianto@gmail.com> wrote:

Hello,

can you explain if this is a bug or my fault ?

I'm making a simple example of hash partition

CREATE TABLE myschema.customers(
id int,
status text,
arr numeric
) PARTITION BY HASH(id);

CREATE TABLE myschema.cust0 PARTITION OF myschema.customers
FOR VALUES WITH (modulus 3, remainder 0);

CREATE TABLE myschema.cust1 PARTITION OF myschema.customers
FOR VALUES WITH (modulus 3, remainder 1);

CREATE TABLE myschema.cust2 PARTITION OF myschema.customers
FOR VALUES WITH (modulus 3, remainder 2);

INSERT INTO myschema.customers VALUES (1,'ACTIVE',100), (2,'RECURRING',20), (3,'EXPIRED',38), (4,'REACTIVATED',144);

when I run the query, the result is:

2 "RECURRING" 20
4 "REACTIVATED" 144

shouldn't be

3 "EXPIRED" 38

i run this in version

PostgreSQL 13.4, compiled by Visual C++ build 1914, 64-bit

Please help me

I could be wrong but IIRC I have been tripped up by this before. The
docs [1]https://www.postgresql.org/docs/13/ddl-partitioning.html says "Each partition will hold the rows for which the hash
value of the partition key divided by the specified modulus will
produce the specified remainder.". Perhaps you were expecting it to do
the do modulus on the key, but not on the hash value of the key?

------
[1]: https://www.postgresql.org/docs/13/ddl-partitioning.html

Kind Regards,
Peter Smith.
Fujitsu Australia.

#4Yudianto Prasetyo
mr.yudianto@gmail.com
In reply to: Peter Smith (#3)
Re: bugs or my fault

hallo,

from the document, I think the determinant of each row in the partition is
the residual value (partition key divided by the specified modulus)

So I think the value that will come out is not what I have listed.

The value that should come out like below ( as in the mods column )

SELECT *,(id % 3)as mods FROM myschema.customers;

id status arr mods
2 "RECURRING" 20 2
4 "REACTIVATED" 144 1
3 "EXPIRED" 38 0
1 "ACTIVE" 100 1

table cust0
id = 3

table cust1
id = 4,1

table cust2
id = 2

best regards

Yudianto

On Wed, Sep 15, 2021 at 5:16 PM Peter Smith <smithpb2250@gmail.com> wrote:

Show quoted text

On Wed, Sep 15, 2021 at 7:08 PM Yudianto Prasetyo <mr.yudianto@gmail.com>
wrote:

sorry I miss the query

select * from myschema.cust0;

On Wed, Sep 15, 2021 at 1:28 PM Yudianto Prasetyo <mr.yudianto@gmail.com>

wrote:

Hello,

can you explain if this is a bug or my fault ?

I'm making a simple example of hash partition

CREATE TABLE myschema.customers(
id int,
status text,
arr numeric
) PARTITION BY HASH(id);

CREATE TABLE myschema.cust0 PARTITION OF myschema.customers
FOR VALUES WITH (modulus 3, remainder 0);

CREATE TABLE myschema.cust1 PARTITION OF myschema.customers
FOR VALUES WITH (modulus 3, remainder 1);

CREATE TABLE myschema.cust2 PARTITION OF myschema.customers
FOR VALUES WITH (modulus 3, remainder 2);

INSERT INTO myschema.customers VALUES (1,'ACTIVE',100),

(2,'RECURRING',20), (3,'EXPIRED',38), (4,'REACTIVATED',144);

when I run the query, the result is:

2 "RECURRING" 20
4 "REACTIVATED" 144

shouldn't be

3 "EXPIRED" 38

i run this in version

PostgreSQL 13.4, compiled by Visual C++ build 1914, 64-bit

Please help me

I could be wrong but IIRC I have been tripped up by this before. The
docs [1] says "Each partition will hold the rows for which the hash
value of the partition key divided by the specified modulus will
produce the specified remainder.". Perhaps you were expecting it to do
the do modulus on the key, but not on the hash value of the key?

------
[1] https://www.postgresql.org/docs/13/ddl-partitioning.html

Kind Regards,
Peter Smith.
Fujitsu Australia.

In reply to: Yudianto Prasetyo (#4)
Re: bugs or my fault

Hello

from the document, I think the determinant of each row in the partition is the residual value (partition key divided by the specified modulus)

No, the hash from partition key is used, not partition key itself. (How to divide text "hello!" by the specified modulus?)

regards, Sergei

#6Yudianto Prasetyo
mr.yudianto@gmail.com
In reply to: Sergei Kornilov (#5)
Re: bugs or my fault

"No, the hash from partition key is used, not partition key itself."

Sorry, I still don't understand the meaning of the sentence above. can you
give a simple example.

best regards,

yudianto

On Wed, Sep 15, 2021 at 7:36 PM Sergei Kornilov <sk@zsrv.org> wrote:

Show quoted text

Hello

from the document, I think the determinant of each row in the partition

is the residual value (partition key divided by the specified modulus)

No, the hash from partition key is used, not partition key itself. (How to
divide text "hello!" by the specified modulus?)

regards, Sergei

In reply to: Yudianto Prasetyo (#6)
Re: bugs or my fault

We don't do just id % 3, we calculate some hash value from partition key (id) and route the tuple according this hash value. Such condition for your example:

select *, satisfies_hash_partition('myschema.customers'::regclass, 3, 0, id) as modulus3_remainder0_target_partition from myschema.cust0;

Can't illustrate more at the SQL level due to the lack of a uint64 calculations. At C level we are here: https://github.com/postgres/postgres/blob/REL_13_STABLE/src/backend/partitioning/partbounds.c#L4595

regards, Sergei

#8Yudianto Prasetyo
mr.yudianto@gmail.com
In reply to: Sergei Kornilov (#7)
Re: bugs or my fault

Hello,

thanks for the information about hash partitions.

I just think that the hash partition will not speed up the Query, because
we don't know the location of certain rows such as the range partition or
the list partition. For example, transactions in 2000 with a range
partition, we can query the "transaction2000" partition table faster than
we can query the "transaction" table.

best regards

yudianto

On Wed, Sep 15, 2021 at 8:38 PM Sergei Kornilov <sk@zsrv.org> wrote:

Show quoted text

We don't do just id % 3, we calculate some hash value from partition key
(id) and route the tuple according this hash value. Such condition for your
example:

select *, satisfies_hash_partition('myschema.customers'::regclass, 3, 0,
id) as modulus3_remainder0_target_partition from myschema.cust0;

Can't illustrate more at the SQL level due to the lack of a uint64
calculations. At C level we are here:
https://github.com/postgres/postgres/blob/REL_13_STABLE/src/backend/partitioning/partbounds.c#L4595

regards, Sergei