max_parallel_degree > 0 for 9.6 beta

Started by Andres Freundalmost 10 years ago27 messageshackers
Jump to latest
#1Andres Freund
andres@anarazel.de

Hi,

max_parallel_degree currently defaults to 0. I think we should enable
it by default for at least the beta period. Otherwise we're primarily
going to get reports back after the release.

Then, at the end of beta, we can decide what the default should be.

- Andres

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#2Andreas Joseph Krogh
andreas@visena.com
In reply to: Andres Freund (#1)
Re: max_parallel_degree > 0 for 9.6 beta

På onsdag 20. april 2016 kl. 19:46:31, skrev Andres Freund <andres@anarazel.de
<mailto:andres@anarazel.de>>:
Hi,

max_parallel_degree currently defaults to 0.  I think we should enable
it by default for at least the beta period. Otherwise we're primarily
going to get reports back after the release.

Then, at the end of beta, we can decide what the default should be.
 
+1
 
Not enabling it per default gives the signal "It's not safe".
 
-- Andreas Joseph Krogh
CTO / Partner - Visena AS
Mobile: +47 909 56 963
andreas@visena.com <mailto:andreas@visena.com>
www.visena.com <https://www.visena.com&gt;
<https://www.visena.com&gt;

 

#3Tom Lane
tgl@sss.pgh.pa.us
In reply to: Andres Freund (#1)
Re: max_parallel_degree > 0 for 9.6 beta

Andres Freund <andres@anarazel.de> writes:

max_parallel_degree currently defaults to 0. I think we should enable
it by default for at least the beta period. Otherwise we're primarily
going to get reports back after the release.

Then, at the end of beta, we can decide what the default should be.

+1, but let's put an entry on the 9.6 open-items page to remind us to
make that decision at the right time.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#4Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#3)
Re: max_parallel_degree > 0 for 9.6 beta

On Wed, Apr 20, 2016 at 2:28 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Andres Freund <andres@anarazel.de> writes:

max_parallel_degree currently defaults to 0. I think we should enable
it by default for at least the beta period. Otherwise we're primarily
going to get reports back after the release.

Then, at the end of beta, we can decide what the default should be.

+1, but let's put an entry on the 9.6 open-items page to remind us to
make that decision at the right time.

So, I suggest that the only sensible non-zero values here are probably
"1" or "2", given a default pool of 8 worker processes system-wide.
Andres told me yesterday he'd vote for "2". Any other opinions?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#5Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#4)
Re: max_parallel_degree > 0 for 9.6 beta

Robert Haas <robertmhaas@gmail.com> writes:

On Wed, Apr 20, 2016 at 2:28 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Andres Freund <andres@anarazel.de> writes:

max_parallel_degree currently defaults to 0. I think we should enable
it by default for at least the beta period. Otherwise we're primarily
going to get reports back after the release.

So, I suggest that the only sensible non-zero values here are probably
"1" or "2", given a default pool of 8 worker processes system-wide.
Andres told me yesterday he'd vote for "2". Any other opinions?

It has to be at least 2 for beta purposes, else you are not testing
situations with more than one worker process at all, which would be
rather a large omission no?

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#6Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#5)
Re: max_parallel_degree > 0 for 9.6 beta

On Thu, Apr 21, 2016 at 1:48 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Robert Haas <robertmhaas@gmail.com> writes:

On Wed, Apr 20, 2016 at 2:28 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Andres Freund <andres@anarazel.de> writes:

max_parallel_degree currently defaults to 0. I think we should enable
it by default for at least the beta period. Otherwise we're primarily
going to get reports back after the release.

So, I suggest that the only sensible non-zero values here are probably
"1" or "2", given a default pool of 8 worker processes system-wide.
Andres told me yesterday he'd vote for "2". Any other opinions?

It has to be at least 2 for beta purposes, else you are not testing
situations with more than one worker process at all, which would be
rather a large omission no?

That's what Andres, thought, too. From my point of view, the big
thing is to be using workers at all. It is of course possible that
there could be some bugs where a single worker is not enough, but
there's a lot of types of bug where even one worker would probably
find the problem. But I'm OK with changing the default to 2.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#7Gavin Flower
GavinFlower@archidevsys.co.nz
In reply to: Robert Haas (#6)
Re: max_parallel_degree > 0 for 9.6 beta

On 22/04/16 06:07, Robert Haas wrote:

On Thu, Apr 21, 2016 at 1:48 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Robert Haas <robertmhaas@gmail.com> writes:

On Wed, Apr 20, 2016 at 2:28 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Andres Freund <andres@anarazel.de> writes:

max_parallel_degree currently defaults to 0. I think we should enable
it by default for at least the beta period. Otherwise we're primarily
going to get reports back after the release.

So, I suggest that the only sensible non-zero values here are probably
"1" or "2", given a default pool of 8 worker processes system-wide.
Andres told me yesterday he'd vote for "2". Any other opinions?

It has to be at least 2 for beta purposes, else you are not testing
situations with more than one worker process at all, which would be
rather a large omission no?

That's what Andres, thought, too. From my point of view, the big
thing is to be using workers at all. It is of course possible that
there could be some bugs where a single worker is not enough, but
there's a lot of types of bug where even one worker would probably
find the problem. But I'm OK with changing the default to 2.

I'm curious.

Why not 4? As most processors now have at least 4 physical cores, &
surely it be more likely to flush out race conditions.

Cheers,
Gavin

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#8Robert Haas
robertmhaas@gmail.com
In reply to: Gavin Flower (#7)
Re: max_parallel_degree > 0 for 9.6 beta

On Thu, Apr 21, 2016 at 4:01 PM, Gavin Flower
<GavinFlower@archidevsys.co.nz> wrote:

On 22/04/16 06:07, Robert Haas wrote:

On Thu, Apr 21, 2016 at 1:48 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Robert Haas <robertmhaas@gmail.com> writes:

On Wed, Apr 20, 2016 at 2:28 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Andres Freund <andres@anarazel.de> writes:

max_parallel_degree currently defaults to 0. I think we should enable
it by default for at least the beta period. Otherwise we're primarily
going to get reports back after the release.

So, I suggest that the only sensible non-zero values here are probably
"1" or "2", given a default pool of 8 worker processes system-wide.
Andres told me yesterday he'd vote for "2". Any other opinions?

It has to be at least 2 for beta purposes, else you are not testing
situations with more than one worker process at all, which would be
rather a large omission no?

That's what Andres, thought, too. From my point of view, the big
thing is to be using workers at all. It is of course possible that
there could be some bugs where a single worker is not enough, but
there's a lot of types of bug where even one worker would probably
find the problem. But I'm OK with changing the default to 2.

I'm curious.

Why not 4? As most processors now have at least 4 physical cores, & surely
it be more likely to flush out race conditions.

Because if we did that, then it's extremely likely that people would
end up writing queries that are faster only if workers are present,
and then not get any workers.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#9Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#8)
Re: max_parallel_degree > 0 for 9.6 beta

Robert Haas <robertmhaas@gmail.com> writes:

On Thu, Apr 21, 2016 at 4:01 PM, Gavin Flower
<GavinFlower@archidevsys.co.nz> wrote:

Why not 4? As most processors now have at least 4 physical cores, & surely
it be more likely to flush out race conditions.

Because if we did that, then it's extremely likely that people would
end up writing queries that are faster only if workers are present,
and then not get any workers.

Is that because max_worker_processes is only 8 by default? Maybe we
need to raise that, at least for beta purposes?

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#10Amit Kapila
amit.kapila16@gmail.com
In reply to: Gavin Flower (#7)
Re: max_parallel_degree > 0 for 9.6 beta

On Fri, Apr 22, 2016 at 1:31 AM, Gavin Flower <GavinFlower@archidevsys.co.nz

wrote:

On 22/04/16 06:07, Robert Haas wrote:

On Thu, Apr 21, 2016 at 1:48 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Robert Haas <robertmhaas@gmail.com> writes:

On Wed, Apr 20, 2016 at 2:28 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Andres Freund <andres@anarazel.de> writes:

max_parallel_degree currently defaults to 0. I think we should enable
it by default for at least the beta period. Otherwise we're primarily
going to get reports back after the release.

So, I suggest that the only sensible non-zero values here are probably

"1" or "2", given a default pool of 8 worker processes system-wide.
Andres told me yesterday he'd vote for "2". Any other opinions?

It has to be at least 2 for beta purposes, else you are not testing
situations with more than one worker process at all, which would be
rather a large omission no?

That's what Andres, thought, too. From my point of view, the big
thing is to be using workers at all. It is of course possible that
there could be some bugs where a single worker is not enough, but
there's a lot of types of bug where even one worker would probably
find the problem. But I'm OK with changing the default to 2.

I'm curious.

Why not 4?

IIUC, the idea to change max_parallel_degree for beta is to catch any bugs
in parallelism code, not to do any performance testing of same. So, I
think either 1 or 2 should be sufficient to hit the bugs if there are any.
Do you have any reason to think that we might miss some category of bugs if
we don't use higher max_parallel_degree?

With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

#11Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#9)
Re: max_parallel_degree > 0 for 9.6 beta

On Thu, Apr 21, 2016 at 7:20 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Robert Haas <robertmhaas@gmail.com> writes:

On Thu, Apr 21, 2016 at 4:01 PM, Gavin Flower
<GavinFlower@archidevsys.co.nz> wrote:

Why not 4? As most processors now have at least 4 physical cores, & surely
it be more likely to flush out race conditions.

Because if we did that, then it's extremely likely that people would
end up writing queries that are faster only if workers are present,
and then not get any workers.

Is that because max_worker_processes is only 8 by default? Maybe we
need to raise that, at least for beta purposes?

I'm not really in favor of that. I mean, almost all of our default
settings are optimized for running PostgreSQL on, for example, a
Raspberry Pi 2, so it would seem odd to suddenly swing the other
direction and assume that there are more than 8 unused CPU cores. It
doesn't make sense to me to roll out settings in beta that we wouldn't
be willing to release with if they work out. That's why, honestly, I
would prefer max_parallel_degree=1, which I think would be practical
for many real-world deployments. max_parallel_degree=2 is OK. Beyond
that, we're just setting people up to fail, I think. Higher settings
should probably only be used on substantial hardware, and not
everybody has that.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#12Andreas Joseph Krogh
andreas@visena.com
In reply to: Robert Haas (#11)
Re: max_parallel_degree > 0 for 9.6 beta

På fredag 22. april 2016 kl. 14:56:33, skrev Robert Haas <robertmhaas@gmail.com
<mailto:robertmhaas@gmail.com>>:
On Thu, Apr 21, 2016 at 7:20 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Robert Haas <robertmhaas@gmail.com> writes:

On Thu, Apr 21, 2016 at 4:01 PM, Gavin Flower
<GavinFlower@archidevsys.co.nz> wrote:

Why not 4?  As most processors now have at least 4 physical cores, &

surely

it be more likely to flush out race conditions.

Because if we did that, then it's extremely likely that people would
end up writing queries that are faster only if workers are present,
and then not get any workers.

Is that because max_worker_processes is only 8 by default?  Maybe we
need to raise that, at least for beta purposes?

I'm not really in favor of that.  I mean, almost all of our default
settings are optimized for running PostgreSQL on, for example, a
Raspberry Pi 2, so it would seem odd to suddenly swing the other
direction and assume that there are more than 8 unused CPU cores.  It
doesn't make sense to me to roll out settings in beta that we wouldn't
be willing to release with if they work out.  That's why, honestly, I
would prefer max_parallel_degree=1, which I think would be practical
for many real-world deployments.  max_parallel_degree=2 is OK.  Beyond
that, we're just setting people up to fail, I think.  Higher settings
should probably only be used on substantial hardware, and not
everybody has that.
 
Maybe it's time to ask the question if the settings should be optimized more
for high-end HW and not som matchstick-box? I mean, most of the people I know
who are responsible for databases run them on HW colser to high-end than
low-end. I'm not sure why optimizing for low-end is such a great choice.
 
-- Andreas Joseph Krog

 

#13Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#11)
Re: max_parallel_degree > 0 for 9.6 beta

Robert Haas <robertmhaas@gmail.com> writes:

On Thu, Apr 21, 2016 at 7:20 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Is that because max_worker_processes is only 8 by default? Maybe we
need to raise that, at least for beta purposes?

I'm not really in favor of that. I mean, almost all of our default
settings are optimized for running PostgreSQL on, for example, a
Raspberry Pi 2, so it would seem odd to suddenly swing the other
direction and assume that there are more than 8 unused CPU cores.

I'm not following why you think that max_worker_processes cannot be
set higher than the number of cores. By that argument, it's insane
that we ship with max_connections = 100. In practice it's generally
fine, and people can get away with oversubscribing their core count
even more than that, because it's seldom that all those processes
are actually contending for CPU at the same time. There are enough
inefficiencies in our parallel-query design that the same will most
certainly be true for parallel workers.

So what I'm concerned about for beta purposes is that we have a setup that
can exercise cases like, say, varying orders in which different workers
return tuples, or potential deadlocks between sibling workers. We'd get
no coverage of that behavioral space at max_parallel_degree=1. I'm not
really convinced that we'll get adequate coverage at
max_parallel_degree=2.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#14Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#13)
Re: max_parallel_degree > 0 for 9.6 beta

On Fri, Apr 22, 2016 at 9:33 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Robert Haas <robertmhaas@gmail.com> writes:

On Thu, Apr 21, 2016 at 7:20 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Is that because max_worker_processes is only 8 by default? Maybe we
need to raise that, at least for beta purposes?

I'm not really in favor of that. I mean, almost all of our default
settings are optimized for running PostgreSQL on, for example, a
Raspberry Pi 2, so it would seem odd to suddenly swing the other
direction and assume that there are more than 8 unused CPU cores.

I'm not following why you think that max_worker_processes cannot be
set higher than the number of cores. By that argument, it's insane
that we ship with max_connections = 100. In practice it's generally
fine, and people can get away with oversubscribing their core count
even more than that, because it's seldom that all those processes
are actually contending for CPU at the same time. There are enough
inefficiencies in our parallel-query design that the same will most
certainly be true for parallel workers.

It is much less likely to be true for parallel workers. The reason
why those processes aren't contending for the CPU at the same time is
generally that most of the connections are in fact idle. But a
parallel worker is never idle. It is launched when it is needed to
run a query and exits immediately afterward. If it's not contending
for the CPU, it will be contending for I/O bandwidth, or a lock.

So what I'm concerned about for beta purposes is that we have a setup that
can exercise cases like, say, varying orders in which different workers
return tuples, or potential deadlocks between sibling workers. We'd get
no coverage of that behavioral space at max_parallel_degree=1. I'm not
really convinced that we'll get adequate coverage at
max_parallel_degree=2.

The right solution to that is for people who have the right hardware
to raise the settings, not to unleash a ridiculous set of defaults on
everyone. I really hope that some people do serious destruction
testing of parallel query and try to break it. For example, you could
use the parallel_degree reloption to force 100 parallel workers to
scan the same relation. That's likely to be dog slow, but it might
well turn up some bugs.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#15Joshua D. Drake
jd@commandprompt.com
In reply to: Robert Haas (#14)
Re: max_parallel_degree > 0 for 9.6 beta

On 04/22/2016 06:47 AM, Robert Haas wrote:

On Fri, Apr 22, 2016 at 9:33 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Robert Haas <robertmhaas@gmail.com> writes:

On Thu, Apr 21, 2016 at 7:20 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Is that because max_worker_processes is only 8 by default? Maybe we
need to raise that, at least for beta purposes?

I'm not really in favor of that. I mean, almost all of our default
settings are optimized for running PostgreSQL on, for example, a
Raspberry Pi 2, so it would seem odd to suddenly swing the other
direction and assume that there are more than 8 unused CPU cores.

This is the problem right here.

We should be shipping for a reasonable production configuration. It is
not reasonable to assume that someone is going to be running on a
Rasberry Pi 2. Yes, we can effectively run on that platform that doesn't
mean it should be our default target configuration. Consider that a
5.00/mo Digital Ocean VM is going to outperform a Rasberry Pi.

It is much less likely to be true for parallel workers. The reason
why those processes aren't contending for the CPU at the same time is
generally that most of the connections are in fact idle. But a
parallel worker is never idle. It is launched when it is needed to
run a query and exits immediately afterward. If it's not contending
for the CPU, it will be contending for I/O bandwidth, or a lock.

True, but isn't that also what context switching and (possibly)
hyperthreading are for?

So what I'm concerned about for beta purposes is that we have a setup that
can exercise cases like, say, varying orders in which different workers
return tuples, or potential deadlocks between sibling workers. We'd get
no coverage of that behavioral space at max_parallel_degree=1. I'm not
really convinced that we'll get adequate coverage at
max_parallel_degree=2.

The right solution to that is for people who have the right hardware
to raise the settings, not to unleash a ridiculous set of defaults on
everyone. I really hope that some people do serious destruction
testing of parallel query and try to break it. For example, you could
use the parallel_degree reloption to force 100 parallel workers to
scan the same relation. That's likely to be dog slow, but it might
well turn up some bugs.

I think your argument sounds more like a production solution, not a Beta
solution. We should be pushing it a little bit in Beta.

JD

--
Command Prompt, Inc. http://the.postgres.company/
+1-503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Everyone appreciates your honesty, until you are honest with them.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#16Robert Haas
robertmhaas@gmail.com
In reply to: Joshua D. Drake (#15)
Re: max_parallel_degree > 0 for 9.6 beta

On Fri, Apr 22, 2016 at 10:07 AM, Joshua D. Drake <jd@commandprompt.com> wrote:

This is the problem right here.

We should be shipping for a reasonable production configuration. It is not
reasonable to assume that someone is going to be running on a Rasberry Pi 2.
Yes, we can effectively run on that platform that doesn't mean it should be
our default target configuration. Consider that a 5.00/mo Digital Ocean VM
is going to outperform a Rasberry Pi.

I don't disagree with that, and I think there is a considerable amount
of work that could be done to create a saner "out of the box"
configuration. But I don't think that the two weeks before beta is
the right time to start building a consensus around what that might
look like.

True, but isn't that also what context switching and (possibly)
hyperthreading are for?

Sure. What you should expect, though, is that overall system
throughput will be higher if the system is not oversubscribed. You
can use parallel query selectively to speed up certain queries even if
that takes you above the number of CPUs you have; if those queries are
on a deadline, finishing them sooner may be worth whatever you lose in
overall throughput.

I think your argument sounds more like a production solution, not a Beta
solution. We should be pushing it a little bit in Beta.

Shipping with max_parallel_workers=2 *is* pushing it a little bit.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#17Gavin Flower
GavinFlower@archidevsys.co.nz
In reply to: Robert Haas (#11)
Re: max_parallel_degree > 0 for 9.6 beta

On 23/04/16 00:56, Robert Haas wrote:

On Thu, Apr 21, 2016 at 7:20 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Robert Haas <robertmhaas@gmail.com> writes:

On Thu, Apr 21, 2016 at 4:01 PM, Gavin Flower
<GavinFlower@archidevsys.co.nz> wrote:

Why not 4? As most processors now have at least 4 physical cores, & surely
it be more likely to flush out race conditions.

Because if we did that, then it's extremely likely that people would
end up writing queries that are faster only if workers are present,
and then not get any workers.

Is that because max_worker_processes is only 8 by default? Maybe we
need to raise that, at least for beta purposes?

I'm not really in favor of that. I mean, almost all of our default
settings are optimized for running PostgreSQL on, for example, a
Raspberry Pi 2, so it would seem odd to suddenly swing the other
direction and assume that there are more than 8 unused CPU cores. It
doesn't make sense to me to roll out settings in beta that we wouldn't
be willing to release with if they work out. That's why, honestly, I
would prefer max_parallel_degree=1, which I think would be practical
for many real-world deployments. max_parallel_degree=2 is OK. Beyond
that, we're just setting people up to fail, I think. Higher settings
should probably only be used on substantial hardware, and not
everybody has that.

If Java can find out how many processors there are available to it,
since JDK1.4, then surely PostgreSQL can do the same?

So how about the default being half the available processors rounded up
to the nearest integer?

Perhaps the GUC for workers should be a percentage of the available
processors, with the minimum & maximum workers optionally specified - or
something of that nature?

Cheers,
Gavin

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#18Gavin Flower
GavinFlower@archidevsys.co.nz
In reply to: Amit Kapila (#10)
Re: max_parallel_degree > 0 for 9.6 beta

On 22/04/16 17:36, Amit Kapila wrote:

On Fri, Apr 22, 2016 at 1:31 AM, Gavin Flower
<GavinFlower@archidevsys.co.nz <mailto:GavinFlower@archidevsys.co.nz>>
wrote:

On 22/04/16 06:07, Robert Haas wrote:

On Thu, Apr 21, 2016 at 1:48 PM, Tom Lane <tgl@sss.pgh.pa.us
<mailto:tgl@sss.pgh.pa.us>> wrote:

Robert Haas <robertmhaas@gmail.com
<mailto:robertmhaas@gmail.com>> writes:

On Wed, Apr 20, 2016 at 2:28 PM, Tom Lane
<tgl@sss.pgh.pa.us <mailto:tgl@sss.pgh.pa.us>> wrote:

Andres Freund <andres@anarazel.de
<mailto:andres@anarazel.de>> writes:

max_parallel_degree currently defaults to 0.
I think we should enable
it by default for at least the beta period.
Otherwise we're primarily
going to get reports back after the release.

So, I suggest that the only sensible non-zero values
here are probably
"1" or "2", given a default pool of 8 worker processes
system-wide.
Andres told me yesterday he'd vote for "2". Any other
opinions?

It has to be at least 2 for beta purposes, else you are
not testing
situations with more than one worker process at all, which
would be
rather a large omission no?

That's what Andres, thought, too. From my point of view, the big
thing is to be using workers at all. It is of course possible
that
there could be some bugs where a single worker is not enough, but
there's a lot of types of bug where even one worker would probably
find the problem. But I'm OK with changing the default to 2.

I'm curious.

Why not 4?

IIUC, the idea to change max_parallel_degree for beta is to catch any
bugs in parallelism code, not to do any performance testing of same.
So, I think either 1 or 2 should be sufficient to hit the bugs if
there are any. Do you have any reason to think that we might miss
some category of bugs if we don't use higher max_parallel_degree?

With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com <http://www.enterprisedb.com/&gt;

No. Just felt that 4 would not be too great for the type of processor
chips used on servers to handle.

For complications, such as race conditions and implied logical
assumptions - I tend to think of 0, 1, 2, 3, many.

Essentially just a gut feeling that 4 might reveal more corner cases.

Cheers,
Gavin

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#19Noah Misch
noah@leadboat.com
In reply to: Tom Lane (#3)
Re: max_parallel_degree > 0 for 9.6 beta

On Wed, Apr 20, 2016 at 02:28:15PM -0400, Tom Lane wrote:

Andres Freund <andres@anarazel.de> writes:

max_parallel_degree currently defaults to 0. I think we should enable
it by default for at least the beta period. Otherwise we're primarily
going to get reports back after the release.

Then, at the end of beta, we can decide what the default should be.

+1, but let's put an entry on the 9.6 open-items page to remind us to
make that decision at the right time.

It's that time. Do we restore the max_parallel_workers_per_gather=0 default,
or is enabling this by default the right thing after all?

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#20Tom Lane
tgl@sss.pgh.pa.us
In reply to: Noah Misch (#19)
Re: max_parallel_degree > 0 for 9.6 beta

Noah Misch <noah@leadboat.com> writes:

On Wed, Apr 20, 2016 at 02:28:15PM -0400, Tom Lane wrote:

+1, but let's put an entry on the 9.6 open-items page to remind us to
make that decision at the right time.

It's that time. Do we restore the max_parallel_workers_per_gather=0 default,
or is enabling this by default the right thing after all?

At this point I'd have to vote against enabling by default in 9.6. The
fact that in the past week we've found bugs as bad as e1a93dd6a does not
give me a warm fuzzy feeling about the parallel-query code being ready
for prime time.

Of course the question is how do we ever get to that point if we chicken
out with enabling it by default now. Maybe we could keep it turned on
in HEAD.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#21Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#20)
#22David G. Johnston
david.g.johnston@gmail.com
In reply to: Robert Haas (#21)
#23Michael Paquier
michael@paquier.xyz
In reply to: David G. Johnston (#22)
#24Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Michael Paquier (#23)
#25Robert Haas
robertmhaas@gmail.com
In reply to: Alvaro Herrera (#24)
#26Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#25)
#27Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#26)