PostgresSQL 13.0 Beta 1 on Phoronix
Hi,
There was news in Phoronix about the Beta 1 Release of Postgres (1).
Unfortunately for Postgres advocacy it does not bring good news,
it is showing regressions in the benchmarks compared to version 12.
Without going into the technical merits of how the test was done,
they have no way of knowing whether such regressions actually exist or if
it is a failure of how the tests were done.
But it would be nice to have arguments to counter, for the sake of
Postgres' promotion.
I'm using the development version (latest), and for now, it seems to be
faster than version 12.
regards,
Ranier Vilela
1. https://www.phoronix.com/scan.php?page=news_item&px=PostgreSQL-13-Beta
On Sun, May 24, 2020 at 2:52 AM Ranier Vilela <ranier.vf@gmail.com> wrote:
There was news in Phoronix about the Beta 1 Release of Postgres (1).
Unfortunately for Postgres advocacy it does not bring good news,
it is showing regressions in the benchmarks compared to version 12.
Without going into the technical merits of how the test was done,
they have no way of knowing whether such regressions actually exist or if it is a failure of how the tests were done.
This shellscript appears to be used by Phoronix to run pgbench:
It looks like they're only running pgbench for 60 second runs in all
configurations -- notice that "-T 60" is passed to pgbench. I'm not
entirely sure that that's all that there is to it. Still, there isn't
any real attempt to make it clear what's going on here. I have my
doubts about how representative these numbers are for that reason.
--
Peter Geoghegan
Em dom., 24 de mai. de 2020 às 14:34, Peter Geoghegan <pg@bowt.ie> escreveu:
On Sun, May 24, 2020 at 2:52 AM Ranier Vilela <ranier.vf@gmail.com> wrote:
There was news in Phoronix about the Beta 1 Release of Postgres (1).
Unfortunately for Postgres advocacy it does not bring good news,
it is showing regressions in the benchmarks compared to version 12.
Without going into the technical merits of how the test was done,
they have no way of knowing whether such regressions actually exist orif it is a failure of how the tests were done.
This shellscript appears to be used by Phoronix to run pgbench:
It looks like they're only running pgbench for 60 second runs in all
configurations -- notice that "-T 60" is passed to pgbench. I'm not
entirely sure that that's all that there is to it. Still, there isn't
any real attempt to make it clear what's going on here. I have my
doubts about how representative these numbers are for that reason.
I also find it very suspicious.V12 seems to be better at read-only
workloads (at least it shows the graphics).
I'm using V13, in normal mode (read / write), medium load.
regards,
Ranier VIlela
On Sun, May 24, 2020 at 02:50:08PM -0300, Ranier Vilela wrote:
Em dom., 24 de mai. de 2020 às 14:34, Peter Geoghegan <pg@bowt.ie> escreveu:
It looks like they're only running pgbench for 60 second runs in all
configurations -- notice that "-T 60" is passed to pgbench. I'm not
entirely sure that that's all that there is to it. Still, there isn't
any real attempt to make it clear what's going on here. I have my
doubts about how representative these numbers are for that reason.I also find it very suspicious.
I don't know, but what seems pretty clear to me is this benchmark does
zero customization of postgresql.conf (it disables autovacuum!?), and
that the number of connections is calculated based on the number of
cores while the scaling factor is visibly calculated from the amount
of memory available in the environment. Perhaps the first part is
wanted, but we are very conservative to allow PG to work on small-ish
machines with the default configuration, and a 56-core machine with
378GB of memory is not something I would define as small-ish.
--
Michael
Em seg., 25 de mai. de 2020 às 03:57, Michael Paquier <michael@paquier.xyz>
escreveu:
On Sun, May 24, 2020 at 02:50:08PM -0300, Ranier Vilela wrote:
Em dom., 24 de mai. de 2020 às 14:34, Peter Geoghegan <pg@bowt.ie>
escreveu:
It looks like they're only running pgbench for 60 second runs in all
configurations -- notice that "-T 60" is passed to pgbench. I'm not
entirely sure that that's all that there is to it. Still, there isn't
any real attempt to make it clear what's going on here. I have my
doubts about how representative these numbers are for that reason.I also find it very suspicious.
I don't know, but what seems pretty clear to me is this benchmark does
zero customization of postgresql.conf (it disables autovacuum!?), and
that the number of connections is calculated based on the number of
cores while the scaling factor is visibly calculated from the amount
of memory available in the environment. Perhaps the first part is
wanted, but we are very conservative to allow PG to work on small-ish
machines with the default configuration, and a 56-core machine with
378GB of memory is not something I would define as small-ish.
Does this mean that V13 would need additional settings in postgresql.conf,
to perform better than V12, out of the box?
If there is any new feature in V13 that needs some configuration in
postgresql.conf,
Would be bettert it should be documented or configured in the installation
itself,
to avoid this type of misunderstanding, which harms the perception of
Postgres.
regards,
Ranier Vilela