Random note of encouragement
Hey All,
I've been doing some (futile) work trying to speed up aggregates with a
group by in PostgreSQL 9.5.
I installed PostgreSQL 9.6 on the same machine to see if I could get
anything running in parallel when using partitioning - which didn't work.
But - I did find this:
With the following setup:
CREATE TABLE base(
view_time TIMESTAMP WITHOUT time ZONE,
view_time_day TIMESTAMP WITHOUT time ZONE,
count_n numeric);
INSERT INTO base
SELECT view_time,
date_trunc('day', view_time),
COUNT::numeric,
FROM
(SELECT
TIMESTAMP '2015-12-01' + random() * interval '30 days' AS view_time,
trunc(random() * 99 + 1) AS COUNT
FROM generate_series(1,30000000)) a;
analyze base;
Now when I run the following SQL (multiple times to allow for getting
everything into shared buffers, which is 4GB on my machine):
select sum(count_n) from base group by view_time_day;
I get the following results:
PSQL 9.5 - *~21 seconds*
PSQL 9.6 devel - *~8.5 seconds*
I think that's pretty good!
I know this is a devel release, things may change, blah blah. But still,
something has changed for the better here!
I get the same plan on both nodes:
HashAggregate (cost=670590.56..670590.95 rows=31 width=13)
Group Key: view_time_day
-> Seq Scan on base (cost=0.00..520590.04 rows=30000104 width=13)
Cheers,
James Sewell,
PostgreSQL Team Lead / Solutions Architect
______________________________________
Level 2, 50 Queen St, Melbourne VIC 3000
*P *(+61) 3 8370 8000 *W* www.lisasoft.com *F *(+61) 3 8370 8099
--
------------------------------
The contents of this email are confidential and may be subject to legal or
professional privilege and copyright. No representation is made that this
email is free of viruses or other defects. If you have received this
communication in error, you may not copy or distribute any part of it or
otherwise disclose its contents to anyone. Please advise the sender of your
incorrect receipt of this correspondence.
On Thu, Feb 25, 2016 at 10:06:34AM +1100, James Sewell wrote:
Now when I run the following SQL (multiple times to allow for getting
everything into shared buffers, which is 4GB on my machine):select sum(count_n) from base group by view_time_day;
I get the following results:
PSQL 9.5 - ~21 seconds
PSQL 9.6 devel - ~8.5 secondsI think that's pretty good!
I know this is a devel release, things may change, blah blah. But still,
something has changed for the better here!
Wow, that is cool. Can anyone suggest which commit improved this?
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Roman grave inscription +--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Thu, Feb 25, 2016 at 12:26 PM, Bruce Momjian <bruce@momjian.us> wrote:
On Thu, Feb 25, 2016 at 10:06:34AM +1100, James Sewell wrote:
Now when I run the following SQL (multiple times to allow for getting
everything into shared buffers, which is 4GB on my machine):select sum(count_n) from base group by view_time_day;
I get the following results:
PSQL 9.5 - ~21 seconds
PSQL 9.6 devel - ~8.5 secondsI think that's pretty good!
I know this is a devel release, things may change, blah blah. But still,
something has changed for the better here!Wow, that is cool. Can anyone suggest which commit improved this?
Since it sums numerics, maybe integer transition functions from commit
959277a4f579da5243968c750069570a58e92b38 helped?
--
Thomas Munro
http://www.enterprisedb.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
I've actually just tested this on 9.3 - and I get roughly the same as
9.6devel.
Now going back to make sure my 9.5 environment is sane.
Hopefully this isn't me jumping the gun.
Cheers,
James Sewell,
PostgreSQL Team Lead / Solutions Architect
______________________________________
Level 2, 50 Queen St, Melbourne VIC 3000
*P *(+61) 3 8370 8000 *W* www.lisasoft.com *F *(+61) 3 8370 8099
On Thu, Feb 25, 2016 at 10:50 AM, Thomas Munro <
thomas.munro@enterprisedb.com> wrote:
On Thu, Feb 25, 2016 at 12:26 PM, Bruce Momjian <bruce@momjian.us> wrote:
On Thu, Feb 25, 2016 at 10:06:34AM +1100, James Sewell wrote:
Now when I run the following SQL (multiple times to allow for getting
everything into shared buffers, which is 4GB on my machine):select sum(count_n) from base group by view_time_day;
I get the following results:
PSQL 9.5 - ~21 seconds
PSQL 9.6 devel - ~8.5 secondsI think that's pretty good!
I know this is a devel release, things may change, blah blah. But still,
something has changed for the better here!Wow, that is cool. Can anyone suggest which commit improved this?
Since it sums numerics, maybe integer transition functions from commit
959277a4f579da5243968c750069570a58e92b38 helped?--
Thomas Munro
http://www.enterprisedb.com
--
------------------------------
The contents of this email are confidential and may be subject to legal or
professional privilege and copyright. No representation is made that this
email is free of viruses or other defects. If you have received this
communication in error, you may not copy or distribute any part of it or
otherwise disclose its contents to anyone. Please advise the sender of your
incorrect receipt of this correspondence.
On 25 February 2016 at 12:50, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:
On Thu, Feb 25, 2016 at 12:26 PM, Bruce Momjian <bruce@momjian.us> wrote:
On Thu, Feb 25, 2016 at 10:06:34AM +1100, James Sewell wrote:
I get the following results:
PSQL 9.5 - ~21 seconds
PSQL 9.6 devel - ~8.5 secondsI think that's pretty good!
I know this is a devel release, things may change, blah blah. But still,
something has changed for the better here!Wow, that is cool. Can anyone suggest which commit improved this?
Since it sums numerics, maybe integer transition functions from commit
959277a4f579da5243968c750069570a58e92b38 helped?
Those changes were only really to allow bigint to use int128
internally instead of NUMERIC, it didn't make any changes to any
NUMERIC aggregate functions.
It would be interesting to see the explain analyze buffers for both.
Perhaps 9.5 just read more buffers from disk than 9.6 did.
--
David Rowley http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Argh seems like a false alarm for now.
I installed 9.5 from RPM source (the other was one I had installed
previously) and the performance matched 9.6
Sorry about that, I must have *something* screwed up on the other one.
Cheers,
James Sewell,
PostgreSQL Team Lead / Solutions Architect
______________________________________
Level 2, 50 Queen St, Melbourne VIC 3000
*P *(+61) 3 8370 8000 *W* www.lisasoft.com *F *(+61) 3 8370 8099
--
------------------------------
The contents of this email are confidential and may be subject to legal or
professional privilege and copyright. No representation is made that this
email is free of viruses or other defects. If you have received this
communication in error, you may not copy or distribute any part of it or
otherwise disclose its contents to anyone. Please advise the sender of your
incorrect receipt of this correspondence.
On Thu, Feb 25, 2016 at 12:50:06PM +1300, Thomas Munro wrote:
On Thu, Feb 25, 2016 at 12:26 PM, Bruce Momjian <bruce@momjian.us> wrote:
On Thu, Feb 25, 2016 at 10:06:34AM +1100, James Sewell wrote:
Now when I run the following SQL (multiple times to allow for getting
everything into shared buffers, which is 4GB on my machine):select sum(count_n) from base group by view_time_day;
I get the following results:
PSQL 9.5 - ~21 seconds
PSQL 9.6 devel - ~8.5 secondsI think that's pretty good!
I know this is a devel release, things may change, blah blah. But still,
something has changed for the better here!Wow, that is cool. Can anyone suggest which commit improved this?
Since it sums numerics, maybe integer transition functions from commit
959277a4f579da5243968c750069570a58e92b38 helped?
Seems it was a wrong report, but anyway, this commit was in 9.5, while
the user reported a speedup in 9.6.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Roman grave inscription +--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Thu, Feb 25, 2016 at 1:43 PM, Bruce Momjian <bruce@momjian.us> wrote:
On Thu, Feb 25, 2016 at 12:50:06PM +1300, Thomas Munro wrote:
On Thu, Feb 25, 2016 at 12:26 PM, Bruce Momjian <bruce@momjian.us> wrote:
On Thu, Feb 25, 2016 at 10:06:34AM +1100, James Sewell wrote:
Now when I run the following SQL (multiple times to allow for getting
everything into shared buffers, which is 4GB on my machine):select sum(count_n) from base group by view_time_day;
I get the following results:
PSQL 9.5 - ~21 seconds
PSQL 9.6 devel - ~8.5 secondsI think that's pretty good!
I know this is a devel release, things may change, blah blah. But still,
something has changed for the better here!Wow, that is cool. Can anyone suggest which commit improved this?
Since it sums numerics, maybe integer transition functions from commit
959277a4f579da5243968c750069570a58e92b38 helped?Seems it was a wrong report, but anyway, this commit was in 9.5, while
the user reported a speedup in 9.6.
Oops, right, and as David said it's also irrelevant.
FWIW I couldn't reproduce this comparing 9.5 with 9.6, but the numbers
reported just happen to match nearly exactly what I get comparing -O2
and -O0 builds here...
--
Thomas Munro
http://www.enterprisedb.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers