Rename max_parallel_degree?
Why is the parallelism variable called "max_parallel_degree"? Is that a
descriptive name? What does "degree" mean? Why is it not called
"max_parallel_workers"?
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Sun, Apr 24, 2016 at 9:28 AM, Bruce Momjian <bruce@momjian.us> wrote:
Why is the parallelism variable called "max_parallel_degree"? Is that a
descriptive name? What does "degree" mean?
It is to denote amount of parallelism at node level.
Why is it not called
"max_parallel_workers"?
Degree of Parallelism is a term used in many of the other popular databases
for the similar purpose, so I think that is another reason to prefer
max_parallel_degree.
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com
On Sat, Apr 23, 2016 at 8:58 PM, Bruce Momjian <bruce@momjian.us> wrote:
Why is the parallelism variable called "max_parallel_degree"? Is that a
descriptive name? What does "degree" mean? Why is it not called
"max_parallel_workers"?
I also think that "max_parallel_workers" works better.
While certain other systems have a concept of a "maximum parallel
degree", it is not the same. Those other systems disable parallelism
altogether when "max parallel degree" is 1, whereas Postgres uses 1
parallel worker along with a leader process. And so, parallelism
*will* still be used on Postgres. That's a pretty significant
difference.
--
Peter Geoghegan
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 04/23/2016 11:00 PM, Peter Geoghegan wrote:
On Sat, Apr 23, 2016 at 8:58 PM, Bruce Momjian <bruce@momjian.us> wrote:
Why is the parallelism variable called "max_parallel_degree"? Is that a
descriptive name? What does "degree" mean? Why is it not called
"max_parallel_workers"?I also think that "max_parallel_workers" works better.
+1.
JD
--
Command Prompt, Inc. http://the.postgres.company/
+1-503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Everyone appreciates your honesty, until you are honest with them.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Sat, Apr 23, 2016 at 11:58 PM, Bruce Momjian <bruce@momjian.us> wrote:
Why is the parallelism variable called "max_parallel_degree"? Is that a
descriptive name? What does "degree" mean? Why is it not called
"max_parallel_workers"?
Because "degree of parallelism" is standard terminology, I guess.
Also, consider that we have the related but actually sorta different
GUC max_worker_processes. I think max_parallel_workers to control the
per-query behavior and max_worker_processes to control the global
system behavior would be confusing - those names are very close
together.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Robert Haas <robertmhaas@gmail.com> writes:
On Sat, Apr 23, 2016 at 11:58 PM, Bruce Momjian <bruce@momjian.us> wrote:
Why is the parallelism variable called "max_parallel_degree"? Is that a
descriptive name? What does "degree" mean? Why is it not called
"max_parallel_workers"?
Because "degree of parallelism" is standard terminology, I guess.
FWIW, I agree with Bruce that using "degree" here is a poor choice.
It's an unnecessary dependence on technical terminology that many people
will not be familiar with.
Also, consider that we have the related but actually sorta different
GUC max_worker_processes. I think max_parallel_workers to control the
per-query behavior and max_worker_processes to control the global
system behavior would be confusing - those names are very close
together.
Well, that just says that we'd better reconsider *both* of those names.
Frankly, neither of them is well chosen, and the fact that they currently
sound unrelated is a bug not a feature. What about something like
"max_overall_worker_processes" and "max_session_worker_processes"?
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Sun, Apr 24, 2016 at 8:23 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Robert Haas <robertmhaas@gmail.com> writes:
On Sat, Apr 23, 2016 at 11:58 PM, Bruce Momjian <bruce@momjian.us>
wrote:
Why is the parallelism variable called "max_parallel_degree"? Is that a
descriptive name? What does "degree" mean? Why is it not called
"max_parallel_workers"?Because "degree of parallelism" is standard terminology, I guess.
FWIW, I agree with Bruce that using "degree" here is a poor choice.
It's an unnecessary dependence on technical terminology that many people
will not be familiar with.
FWIW, SQL Server calls it "degree of parallelism" as well (
https://technet.microsoft.com/en-us/library/ms188611(v=sql.105).aspx). And
their configuration option is "max degree of parallelism":
https://technet.microsoft.com/en-us/library/ms181007(v=sql.105).aspx.
Oracle also call it "degree of parallelism" (
https://docs.oracle.com/cd/E18283_01/server.112/e17110/initparams176.htm).
So it's certainly not a made-up term. And I'd go as far as to say that most
people coming from other databases would be familiar with it. It may not be
a standard, but clearly it's very close to being that.
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
Magnus Hagander <magnus@hagander.net> writes:
On Sun, Apr 24, 2016 at 8:23 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
FWIW, I agree with Bruce that using "degree" here is a poor choice.
It's an unnecessary dependence on technical terminology that many people
will not be familiar with.
FWIW, SQL Server calls it "degree of parallelism" as well (
https://technet.microsoft.com/en-us/library/ms188611(v=sql.105).aspx). And
their configuration option is "max degree of parallelism":
https://technet.microsoft.com/en-us/library/ms181007(v=sql.105).aspx.
Yes, but both they and Oracle appear to consider "degree" to mean the
total number of processors used, not the number of secondary jobs in
addition to the main one. The only thing worse than employing obscure
technical terminology is employing it incorrectly: that way, you get to
confuse both the users who know what it means and those who don't.
The fact that we couldn't get this right seems to me to be sufficient
evidence that we should stay away from the term "degree".
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Sun, Apr 24, 2016 at 02:23:43PM -0400, Tom Lane wrote:
Robert Haas <robertmhaas@gmail.com> writes:
Also, consider that we have the related but actually sorta different
GUC max_worker_processes. I think max_parallel_workers to control the
per-query behavior and max_worker_processes to control the global
system behavior would be confusing - those names are very close
together.Well, that just says that we'd better reconsider *both* of those names.
Frankly, neither of them is well chosen, and the fact that they currently
sound unrelated is a bug not a feature. What about something like
"max_overall_worker_processes" and "max_session_worker_processes"?
I don't think "overall" works. I am think
"max_cluster_worker_processes" and "max_session_worker_processes" works.
I guess you could also use "max_server_worker_processes", but that is
referring to the database server, not the OS-level server.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Sun, Apr 24, 2016 at 2:23 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
FWIW, I agree with Bruce that using "degree" here is a poor choice.
It's an unnecessary dependence on technical terminology that many people
will not be familiar with.
And many others will. Some made-up term that is entirely
PostgreSQL-specific is not going to be better.
Well, that just says that we'd better reconsider *both* of those names.
Frankly, neither of them is well chosen, and the fact that they currently
sound unrelated is a bug not a feature. What about something like
"max_overall_worker_processes" and "max_session_worker_processes"?
The first one is fine except for the IMHO-fatal defect that
max_worker_processes has been around since 9.4, and I think it's far
too late to rename it. Should we rename max_connections to
max_overall_connections at the same time?
The second one is wrong for at least two reasons: it's not a
per-session maximum, and it's not a maximum of all worker processes,
just parallel workers.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Sun, Apr 24, 2016 at 2:42 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Magnus Hagander <magnus@hagander.net> writes:
On Sun, Apr 24, 2016 at 8:23 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
FWIW, I agree with Bruce that using "degree" here is a poor choice.
It's an unnecessary dependence on technical terminology that many people
will not be familiar with.FWIW, SQL Server calls it "degree of parallelism" as well (
https://technet.microsoft.com/en-us/library/ms188611(v=sql.105).aspx). And
their configuration option is "max degree of parallelism":
https://technet.microsoft.com/en-us/library/ms181007(v=sql.105).aspx.Yes, but both they and Oracle appear to consider "degree" to mean the
total number of processors used, not the number of secondary jobs in
addition to the main one. The only thing worse than employing obscure
technical terminology is employing it incorrectly: that way, you get to
confuse both the users who know what it means and those who don't.
This is not so clear-cut as you are making it out to be. For example,
see http://www.akadia.com/services/ora_parallel_processing.html - viz
"The number of parallel slave processes associated with an operation
is called its degree of parallelism", which is pretty close to what
the parameter currently called max_parallel_degree actually does.
See also http://searchitchannel.techtarget.com/feature/Using-parallel-SQL-to-improve-Oracle-database-performance
- "The Degree of Parallelism (DOP) defines the number of parallel
streams of execution that will be created. In the simplest case, this
translates to the number of parallel slave processes enlisted to
support your SQL's execution. However, the number of parallel
processes is more often twice the DOP. This is because each step in a
nontrivial execution plan needs to feed data into the subsequent step,
so two sets of processes are required to maintain the parallel stream
of processing."
Other sources disagree, of course, as is often the case with Oracle. Wahoo.
Of course, we could make this value 1-based rather than 0-based, as
Peter Geoghegan suggested a while back. But as I think I said at the
time, I think that's more misleading than helpful. The leader
participates in the parallel plan, but typically does far less of the
work beneath the Gather node than the other nodes involved in the
query, often almost none. In short, the leader is special.
Pretending that it's just another process involved in the parallel
group isn't doing anyone a favor.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Mon, Apr 25, 2016 at 5:01 AM, Robert Haas <robertmhaas@gmail.com> wrote:
On Sun, Apr 24, 2016 at 2:42 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Magnus Hagander <magnus@hagander.net> writes:
On Sun, Apr 24, 2016 at 8:23 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
FWIW, I agree with Bruce that using "degree" here is a poor choice.
It's an unnecessary dependence on technical terminology that manypeople
will not be familiar with.
FWIW, SQL Server calls it "degree of parallelism" as well (
https://technet.microsoft.com/en-us/library/ms188611(v=sql.105).aspx).And
their configuration option is "max degree of parallelism":
https://technet.microsoft.com/en-us/library/ms181007(v=sql.105).aspx.Yes, but both they and Oracle appear to consider "degree" to mean the
total number of processors used, not the number of secondary jobs in
addition to the main one. The only thing worse than employing obscure
technical terminology is employing it incorrectly: that way, you get to
confuse both the users who know what it means and those who don't.This is not so clear-cut as you are making it out to be. For example,
see http://www.akadia.com/services/ora_parallel_processing.html - viz
"The number of parallel slave processes associated with an operation
is called its degree of parallelism", which is pretty close to what
the parameter currently called max_parallel_degree actually does.
So maybe something like session_parallel_degree, to add another color to
the bikeshed?
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
On 25 April 2016 at 03:44, Robert Haas <robertmhaas@gmail.com> wrote:
On Sun, Apr 24, 2016 at 2:23 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
FWIW, I agree with Bruce that using "degree" here is a poor choice.
It's an unnecessary dependence on technical terminology that many people
will not be familiar with.And many others will. Some made-up term that is entirely
PostgreSQL-specific is not going to be better.
Just to add my 2c, "degree" implies some sort of graded scale. So
setting it to (say) 10 would be "maximum", setting to 0 would be
"none" and setting it to anything in between would be relative to the
maximum.
eg in Vol26 "Encyclopedia of Computer Science and Technology" (the
first compsci reference that appeared for a google search) there are
three levels of granularity of degrees of parallelism.
Frankly it seems that the SQL crowd stole the computer science term
and applied it incorrectly.
Having a configuration option "_workers" makes much more sense to me.
It absolutely describes what it does without needing to refer to a
manual, and it removes ambiguity.
Geoff
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Mon, Apr 25, 2016 at 12:17:56PM +0200, Magnus Hagander wrote:
On Mon, Apr 25, 2016 at 5:01 AM, Robert Haas <robertmhaas@gmail.com> wrote:
On Sun, Apr 24, 2016 at 2:42 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Magnus Hagander <magnus@hagander.net> writes:
On Sun, Apr 24, 2016 at 8:23 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
FWIW, I agree with Bruce that using "degree" here is a poor choice.
It's an unnecessary dependence on technical terminology that manypeople
will not be familiar with.
FWIW, SQL Server calls it "degree of parallelism" as well (
https://technet.microsoft.com/en-us/library/ms188611(v=sql.105).aspx).And
their configuration option is "max degree of parallelism":
https://technet.microsoft.com/en-us/library/ms181007(v=sql.105).aspx.Yes, but both they and Oracle appear to consider "degree" to mean the
total number of processors used, not the number of secondary jobs in
addition to the main one.� The only thing worse than employing obscure
technical terminology is employing it incorrectly: that way, you get to
confuse both the users who know what it means and those who don't.This is not so clear-cut as you are making it out to be.� For example,
see http://www.akadia.com/services/ora_parallel_processing.html - viz
"The number of parallel slave processes associated with an operation
is called its degree of parallelism", which is pretty close to what
the parameter currently called max_parallel_degree actually does.So maybe something like �session_parallel_degree, to add another color to the
bikeshed?
I think Robert said it is per-executor node, not per session, similar to
work_mem.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Sun, Apr 24, 2016 at 11:21:01AM +0530, Amit Kapila wrote:
On Sun, Apr 24, 2016 at 9:28 AM, Bruce Momjian <bruce@momjian.us> wrote:
Why is the parallelism variable called "max_parallel_degree"?� Is that a
descriptive name?� What does "degree" mean?It is to denote amount of parallelism at node level.
----------
Sorry, it was Amit who said the parallelism is per-node, meaning calling
the variable "session" would not work.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 04/25/2016 07:08 AM, Bruce Momjian wrote:
On Sun, Apr 24, 2016 at 11:21:01AM +0530, Amit Kapila wrote:
On Sun, Apr 24, 2016 at 9:28 AM, Bruce Momjian <bruce@momjian.us> wrote:
Why is the parallelism variable called "max_parallel_degree"? Is that a
descriptive name? What does "degree" mean?It is to denote amount of parallelism at node level.
----------
Sorry, it was Amit who said the parallelism is per-node, meaning calling
the variable "session" would not work.
max_parallel_nodes
jD
--
Command Prompt, Inc. http://the.postgres.company/
+1-503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Everyone appreciates your honesty, until you are honest with them.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Mon, Apr 25, 2016 at 11:27 AM, Joshua D. Drake <jd@commandprompt.com> wrote:
max_parallel_nodes
I hope you are trolling me. It does not bound the maximum number of
parallel nodes, but rather the maximum number of workers per parallel
node. In most cases, a query is only going to have one parallel node,
but sometimes it could have more than one. In all of the actual cases
tested by me thus far, the workers for one node terminate before we
try to launch workers for another node, but it is theoretically
possible to have a query plan where this isn't the case. It is
getting a bit irritating to me to people who seem not even to have
read the existing documentation for this GUC, let alone tried it out
and gotten familiar with what it does, are convinced they know how it
should be changed.
On the underlying issue, the max_parallel_degree naming is tied into
the parallel_degree reloption and the parallel_degree field that is
part of each Path. If you want to rename any of those, you are going
to need to rename them all, I think. So we could do
max_parallel_degree -> max_parallel_workers_per_executor_node and
parallel_degree -> target_number_of_parallel_workers, but that is
rather longwinded and it's very difficult to shorten those very much
without losing clarity. I grant that there is some learning curve in
getting familiar with new terms of art, like "parallel degree", but in
the long run it's better to ask people to learn a few new terms than
to trying to repeat the whole description of what the thing is every
time you refer to it. We've got existing terms of art like
"multixact" or even "wraparound" that are far, far more obscure than
this, and those are entirely PostgreSQL-specific things without a hint
of any parallel in other products.
The concept of parallel degree is not going away, and its reach
extends well beyond the GUC. We can rename it to some other term, but
this one is pretty standard. Trying to avoid having a term for it is
not going to work.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 04/25/2016 09:04 AM, Robert Haas wrote:
On Mon, Apr 25, 2016 at 11:27 AM, Joshua D. Drake <jd@commandprompt.com> wrote:
max_parallel_nodes
I hope you are trolling me.
Actually I wasn't. It is usually a little more obvious when I troll,
subtlety is not exactly my strong suit ;)
The only reason I suggested that name was because of this comment:
"I think Robert said it is per-executor node, not per session, similar
to work_mem." (from Bruce)
If it doesn't make sense, I apologize for the noise.
Sincerely,
JD
--
Command Prompt, Inc. http://the.postgres.company/
+1-503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Everyone appreciates your honesty, until you are honest with them.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Tom Lane wrote:
Magnus Hagander <magnus@hagander.net> writes:
On Sun, Apr 24, 2016 at 8:23 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
FWIW, I agree with Bruce that using "degree" here is a poor choice.
It's an unnecessary dependence on technical terminology that many people
will not be familiar with.FWIW, SQL Server calls it "degree of parallelism" as well (
https://technet.microsoft.com/en-us/library/ms188611(v=sql.105).aspx). And
their configuration option is "max degree of parallelism":
https://technet.microsoft.com/en-us/library/ms181007(v=sql.105).aspx.Yes, but both they and Oracle appear to consider "degree" to mean the
total number of processors used, not the number of secondary jobs in
addition to the main one. The only thing worse than employing obscure
technical terminology is employing it incorrectly:
What about calling it something even simpler, such as "max_parallelism"?
This avoids such cargo cult, and there's no implication that it's
per-query.
--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Mon, Apr 25, 2016 at 2:58 PM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:
Tom Lane wrote:
Magnus Hagander <magnus@hagander.net> writes:
On Sun, Apr 24, 2016 at 8:23 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
FWIW, I agree with Bruce that using "degree" here is a poor choice.
It's an unnecessary dependence on technical terminology that many people
will not be familiar with.FWIW, SQL Server calls it "degree of parallelism" as well (
https://technet.microsoft.com/en-us/library/ms188611(v=sql.105).aspx). And
their configuration option is "max degree of parallelism":
https://technet.microsoft.com/en-us/library/ms181007(v=sql.105).aspx.Yes, but both they and Oracle appear to consider "degree" to mean the
total number of processors used, not the number of secondary jobs in
addition to the main one. The only thing worse than employing obscure
technical terminology is employing it incorrectly:What about calling it something even simpler, such as "max_parallelism"?
This avoids such cargo cult, and there's no implication that it's
per-query.
So what would we call the "parallel_degree" member of the Path data
structure, and the "parallel_degree" reloption? I don't think
renaming either of those to "parallelism" is going to be an
improvement.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
