Fractions in GUC variables
This patch implements the first wiki/Todo Configuration Files item
"Consider normalizing fractions in postgresql.conf, perhaps using '%'".
The "Fractions in GUC variables" discussion is here.
/messages/by-id/467132CF.9020501@enterprisedb.com
This patch implements expressing GUC variables as percents in
postgresql.conf.
autovacuum_vacuum_scale_factor = 20% # percent of table size before vacuum
autovacuum_analyze_scale_factor = 10% # percent of table size before
analyze
As you can see the postgresql.conf file and the documentation read more
naturally. I added a regression test to guc.sql. The sql interface also
accepts both numeric and percent forms although the percent form must be
quoted because '%' is an operator.
show cursor_tuple_fraction; --> 10%
set cursor_tuple_fraction = .15; --> 15%
set cursor_tuple_fraction = '33%'; --> 33%
I tagged four configuration variables to display as percents.
The attached patch applies cleanly against master and passes all regression
tests including two new tests in guc.sql.
Attachments:
guc_percent-v1.patchapplication/octet-stream; name=guc_percent-v1.patchDownload
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
new file mode 100644
index d607eca..dfc482f
*** a/doc/src/sgml/config.sgml
--- b/doc/src/sgml/config.sgml
***************
*** 24,30 ****
<para>
All parameter names are case-insensitive. Every parameter takes a
value of one of five types: boolean, integer, floating point,
! string, or enum.
</para>
<itemizedlist>
--- 24,30 ----
<para>
All parameter names are case-insensitive. Every parameter takes a
value of one of five types: boolean, integer, floating point,
! string, or enum. Floating point values may be expressed as a percentage.
</para>
<itemizedlist>
*************** include_dir 'conf.d'
*** 2411,2418 ****
</term>
<listitem>
<para>
! Specifies the target of checkpoint completion, as a fraction of
! total time between checkpoints. The default is 0.5.
This parameter can only be set in the <filename>postgresql.conf</>
file or on the server command line.
</para>
--- 2411,2418 ----
</term>
<listitem>
<para>
! Specifies the target of checkpoint completion, as a percentage of
! total time between checkpoints. The default is 50%.
This parameter can only be set in the <filename>postgresql.conf</>
file or on the server command line.
</para>
*************** SELECT * FROM parent WHERE key = 2400;
*** 3589,3601 ****
</term>
<listitem>
<para>
! Sets the planner's estimate of the fraction of a cursor's rows that
! will be retrieved. The default is 0.1. Smaller values of this
setting bias the planner towards using <quote>fast start</> plans
for cursors, which will retrieve the first few rows quickly while
perhaps taking a long time to fetch all rows. Larger values
put more emphasis on the total estimated time. At the maximum
! setting of 1.0, cursors are planned exactly like regular queries,
considering only the total estimated time and not how soon the
first rows might be delivered.
</para>
--- 3589,3601 ----
</term>
<listitem>
<para>
! Sets the planner's estimate of the percentage of a cursor's rows that
! will be retrieved. The default is 10%. Smaller values of this
setting bias the planner towards using <quote>fast start</> plans
for cursors, which will retrieve the first few rows quickly while
perhaps taking a long time to fetch all rows. Larger values
put more emphasis on the total estimated time. At the maximum
! setting of 100%, cursors are planned exactly like regular queries,
considering only the total estimated time and not how soon the
first rows might be delivered.
</para>
*************** COPY postgres_log FROM '/full/path/to/lo
*** 5221,5230 ****
</term>
<listitem>
<para>
! Specifies a fraction of the table size to add to
<varname>autovacuum_vacuum_threshold</varname>
when deciding whether to trigger a <command>VACUUM</>.
! The default is 0.2 (20% of table size).
This parameter can only be set in the <filename>postgresql.conf</>
file or on the server command line.
This setting can be overridden for individual tables by
--- 5221,5230 ----
</term>
<listitem>
<para>
! Specifies a percentage of the table size to add to
<varname>autovacuum_vacuum_threshold</varname>
when deciding whether to trigger a <command>VACUUM</>.
! The default is 20% of table size.
This parameter can only be set in the <filename>postgresql.conf</>
file or on the server command line.
This setting can be overridden for individual tables by
*************** COPY postgres_log FROM '/full/path/to/lo
*** 5241,5250 ****
</term>
<listitem>
<para>
! Specifies a fraction of the table size to add to
<varname>autovacuum_analyze_threshold</varname>
when deciding whether to trigger an <command>ANALYZE</>.
! The default is 0.1 (10% of table size).
This parameter can only be set in the <filename>postgresql.conf</>
file or on the server command line.
This setting can be overridden for individual tables by
--- 5241,5250 ----
</term>
<listitem>
<para>
! Specifies a percentage of the table size to add to
<varname>autovacuum_analyze_threshold</varname>
when deciding whether to trigger an <command>ANALYZE</>.
! The default is 10% of table size.
This parameter can only be set in the <filename>postgresql.conf</>
file or on the server command line.
This setting can be overridden for individual tables by
diff --git a/src/backend/utils/misc/guc-file.l b/src/backend/utils/misc/guc-file.l
new file mode 100644
index 6dca5df..dc2171b
*** a/src/backend/utils/misc/guc-file.l
--- b/src/backend/utils/misc/guc-file.l
*************** UNIT_LETTER [a-zA-Z]
*** 74,80 ****
INTEGER {SIGN}?({DIGIT}+|0x{HEXDIGIT}+){UNIT_LETTER}*
EXPONENT [Ee]{SIGN}?{DIGIT}+
! REAL {SIGN}?{DIGIT}*"."{DIGIT}*{EXPONENT}?
LETTER [A-Za-z_\200-\377]
LETTER_OR_DIGIT [A-Za-z_0-9\200-\377]
--- 74,83 ----
INTEGER {SIGN}?({DIGIT}+|0x{HEXDIGIT}+){UNIT_LETTER}*
EXPONENT [Ee]{SIGN}?{DIGIT}+
!
! FLOAT {SIGN}?{DIGIT}*"."{DIGIT}*{EXPONENT}?
! PERCENT {SIGN}?{DIGIT}*"."?{DIGIT}*"%"
! REAL {FLOAT}|{PERCENT}
LETTER [A-Za-z_\200-\377]
LETTER_OR_DIGIT [A-Za-z_0-9\200-\377]
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
new file mode 100644
index b1bff7f..fb67a90
*** a/src/backend/utils/misc/guc.c
--- b/src/backend/utils/misc/guc.c
*************** static struct config_real ConfigureNames
*** 2635,2643 ****
{
{"cursor_tuple_fraction", PGC_USERSET, QUERY_TUNING_OTHER,
! gettext_noop("Sets the planner's estimate of the fraction of "
"a cursor's rows that will be retrieved."),
! NULL
},
&cursor_tuple_fraction,
DEFAULT_CURSOR_TUPLE_FRACTION, 0.0, 1.0,
--- 2635,2644 ----
{
{"cursor_tuple_fraction", PGC_USERSET, QUERY_TUNING_OTHER,
! gettext_noop("Sets the planner's estimate of the percentage of "
"a cursor's rows that will be retrieved."),
! NULL,
! GUC_SHOW_PERCENT
},
&cursor_tuple_fraction,
DEFAULT_CURSOR_TUPLE_FRACTION, 0.0, 1.0,
*************** static struct config_real ConfigureNames
*** 2687,2694 ****
{
{"autovacuum_vacuum_scale_factor", PGC_SIGHUP, AUTOVACUUM,
! gettext_noop("Number of tuple updates or deletes prior to vacuum as a fraction of reltuples."),
! NULL
},
&autovacuum_vac_scale,
0.2, 0.0, 100.0,
--- 2688,2696 ----
{
{"autovacuum_vacuum_scale_factor", PGC_SIGHUP, AUTOVACUUM,
! gettext_noop("Number of tuple updates or deletes prior to vacuum as a percentage of reltuples."),
! NULL,
! GUC_SHOW_PERCENT
},
&autovacuum_vac_scale,
0.2, 0.0, 100.0,
*************** static struct config_real ConfigureNames
*** 2696,2703 ****
},
{
{"autovacuum_analyze_scale_factor", PGC_SIGHUP, AUTOVACUUM,
! gettext_noop("Number of tuple inserts, updates, or deletes prior to analyze as a fraction of reltuples."),
! NULL
},
&autovacuum_anl_scale,
0.1, 0.0, 100.0,
--- 2698,2706 ----
},
{
{"autovacuum_analyze_scale_factor", PGC_SIGHUP, AUTOVACUUM,
! gettext_noop("Number of tuple inserts, updates, or deletes prior to analyze as a percentage of reltuples."),
! NULL,
! GUC_SHOW_PERCENT
},
&autovacuum_anl_scale,
0.1, 0.0, 100.0,
*************** static struct config_real ConfigureNames
*** 2706,2713 ****
{
{"checkpoint_completion_target", PGC_SIGHUP, WAL_CHECKPOINTS,
! gettext_noop("Time spent flushing dirty buffers during checkpoint, as fraction of checkpoint interval."),
! NULL
},
&CheckPointCompletionTarget,
0.5, 0.0, 1.0,
--- 2709,2717 ----
{
{"checkpoint_completion_target", PGC_SIGHUP, WAL_CHECKPOINTS,
! gettext_noop("Time spent flushing dirty buffers during checkpoint, as a percentage of checkpoint interval."),
! NULL,
! GUC_SHOW_PERCENT
},
&CheckPointCompletionTarget,
0.5, 0.0, 1.0,
*************** parse_int(const char *value, int *result
*** 5253,5258 ****
--- 5257,5263 ----
/*
* Try to parse value as a floating point number in the usual format.
+ * If the string ends in '%' divide by 100.
* If the string parses okay, return true, else false.
* If okay and result is not NULL, return the value in *result.
*/
*************** parse_real(const char *value, double *re
*** 5270,5275 ****
--- 5275,5287 ----
if (endptr == value || errno == ERANGE)
return false;
+ /* Check for percentage. */
+ if (*endptr == '%')
+ {
+ val /= 100.0;
+ endptr++;
+ }
+
/* allow whitespace after number */
while (isspace((unsigned char) *endptr))
endptr++;
*************** _ShowOption(struct config_generic * reco
*** 8208,8215 ****
val = (*conf->show_hook) ();
else
{
! snprintf(buffer, sizeof(buffer), "%g",
! *conf->variable);
val = buffer;
}
}
--- 8220,8235 ----
val = (*conf->show_hook) ();
else
{
! if (record->flags & GUC_SHOW_PERCENT)
! {
! snprintf(buffer, sizeof(buffer), "%g%%",
! *conf->variable * 100.0);
! }
! else
! {
! snprintf(buffer, sizeof(buffer), "%g",
! *conf->variable);
! }
val = buffer;
}
}
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
new file mode 100644
index b053659..0352c2d
*** a/src/backend/utils/misc/postgresql.conf.sample
--- b/src/backend/utils/misc/postgresql.conf.sample
***************
*** 199,205 ****
#checkpoint_segments = 3 # in logfile segments, min 1, 16MB each
#checkpoint_timeout = 5min # range 30s-1h
! #checkpoint_completion_target = 0.5 # checkpoint target duration, 0.0 - 1.0
#checkpoint_warning = 30s # 0 disables
# - Archiving -
--- 199,205 ----
#checkpoint_segments = 3 # in logfile segments, min 1, 16MB each
#checkpoint_timeout = 5min # range 30s-1h
! #checkpoint_completion_target = 50% # checkpoint target duration, 0-100%
#checkpoint_warning = 30s # 0 disables
# - Archiving -
***************
*** 303,309 ****
#default_statistics_target = 100 # range 1-10000
#constraint_exclusion = partition # on, off, or partition
! #cursor_tuple_fraction = 0.1 # range 0.0-1.0
#from_collapse_limit = 8
#join_collapse_limit = 8 # 1 disables collapsing of explicit
# JOIN clauses
--- 303,309 ----
#default_statistics_target = 100 # range 1-10000
#constraint_exclusion = partition # on, off, or partition
! #cursor_tuple_fraction = 10% # range 0-100%
#from_collapse_limit = 8
#join_collapse_limit = 8 # 1 disables collapsing of explicit
# JOIN clauses
***************
*** 483,490 ****
# vacuum
#autovacuum_analyze_threshold = 50 # min number of row updates before
# analyze
! #autovacuum_vacuum_scale_factor = 0.2 # fraction of table size before vacuum
! #autovacuum_analyze_scale_factor = 0.1 # fraction of table size before analyze
#autovacuum_freeze_max_age = 200000000 # maximum XID age before forced vacuum
# (change requires restart)
#autovacuum_multixact_freeze_max_age = 400000000 # maximum multixact age
--- 483,490 ----
# vacuum
#autovacuum_analyze_threshold = 50 # min number of row updates before
# analyze
! #autovacuum_vacuum_scale_factor = 20% # percent of table size before vacuum
! #autovacuum_analyze_scale_factor = 10% # percent of table size before analyze
#autovacuum_freeze_max_age = 200000000 # maximum XID age before forced vacuum
# (change requires restart)
#autovacuum_multixact_freeze_max_age = 400000000 # maximum multixact age
diff --git a/src/include/utils/guc.h b/src/include/utils/guc.h
new file mode 100644
index 80813d2..fd7bde5
*** a/src/include/utils/guc.h
--- b/src/include/utils/guc.h
*************** typedef enum
*** 215,220 ****
--- 215,221 ----
#define GUC_NOT_WHILE_SEC_REST 0x8000 /* can't set if security restricted */
#define GUC_DISALLOW_IN_AUTO_FILE 0x00010000 /* can't set in
* PG_AUTOCONF_FILENAME */
+ #define GUC_SHOW_PERCENT 0x00020000 /* Display as percent. */
/* GUC vars that are actually declared in guc.c, rather than elsewhere */
extern bool log_duration;
diff --git a/src/test/regress/expected/guc.out b/src/test/regress/expected/guc.out
new file mode 100644
index 4f0065c..95feb8c
*** a/src/test/regress/expected/guc.out
--- b/src/test/regress/expected/guc.out
*************** SHOW datestyle;
*** 9,14 ****
--- 9,15 ----
-- SET to some nondefault value
SET vacuum_cost_delay TO 40;
SET datestyle = 'ISO, YMD';
+ SET cursor_tuple_fraction = '15%';
SHOW vacuum_cost_delay;
vacuum_cost_delay
-------------------
*************** SHOW datestyle;
*** 21,26 ****
--- 22,33 ----
ISO, YMD
(1 row)
+ SHOW cursor_tuple_fraction;
+ cursor_tuple_fraction
+ -----------------------
+ 15%
+ (1 row)
+
SELECT '2006-08-13 12:34:56'::timestamptz;
timestamptz
------------------------
diff --git a/src/test/regress/sql/guc.sql b/src/test/regress/sql/guc.sql
new file mode 100644
index 3de8a6b..4fef222
*** a/src/test/regress/sql/guc.sql
--- b/src/test/regress/sql/guc.sql
*************** SHOW datestyle;
*** 5,12 ****
--- 5,14 ----
-- SET to some nondefault value
SET vacuum_cost_delay TO 40;
SET datestyle = 'ISO, YMD';
+ SET cursor_tuple_fraction = '15%';
SHOW vacuum_cost_delay;
SHOW datestyle;
+ SHOW cursor_tuple_fraction;
SELECT '2006-08-13 12:34:56'::timestamptz;
-- SET LOCAL has no effect outside of a transaction
On 12/07/2014 11:48 AM, John Gorman wrote:
This patch implements the first wiki/Todo Configuration Files item
"Consider normalizing fractions in postgresql.conf, perhaps using '%'".The "Fractions in GUC variables" discussion is here.
Oh, this is nice! Thanks for working on it.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Import Notes
Reply to msg id not found: WMc1e6af7dc999140af04b7ddef171f33d549b5a18ab74b1d84b3b92266f0a90dce5d9da710ffb225e1a29d888184b403e@asav-1.01.com
On Mon, Dec 8, 2014 at 4:48 AM, John Gorman <johngorman2@gmail.com> wrote:
The attached patch applies cleanly against master and passes all regression
tests including two new tests in guc.sql.
Please be sure to register your patch to the upcoming commit fest,
this way it will not fall into oblivion and will get some feedback:
https://commitfest.postgresql.org/action/commitfest_view?id=25
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 12/07/2014 09:48 PM, John Gorman wrote:
This patch implements the first wiki/Todo Configuration Files item
"Consider normalizing fractions in postgresql.conf, perhaps using '%'".The "Fractions in GUC variables" discussion is here.
Ah, flash from the past :-). Thanks for looking into this.
The bgwriter_lru_percent and bgwriter_all_percent settings are gone now,
so the original reason to do this is gone. We consistently use fractions
in the GUCs now.
This patch implements expressing GUC variables as percents in
postgresql.conf.autovacuum_vacuum_scale_factor = 20% # percent of table size before vacuum
autovacuum_analyze_scale_factor = 10% # percent of table size before
analyzeAs you can see the postgresql.conf file and the documentation read more
naturally. I added a regression test to guc.sql. The sql interface also
accepts both numeric and percent forms although the percent form must be
quoted because '%' is an operator.show cursor_tuple_fraction; --> 10%
set cursor_tuple_fraction = .15; --> 15%
set cursor_tuple_fraction = '33%'; --> 33%I tagged four configuration variables to display as percents.
I'm not sure I agree that percentages are better than fractions. I'd
vote a -0.5 for changing the default display format. There isn't much
harm in accepting them as a secondary format, though.
We should only accept percentages for settings where that makes sense.
It is sensible for most of the PGC_REAL settings, but not so much for
geqo_selection_bias or seed, for example.
Overall, I feel that this isn't really worth the trouble. We use
fractions consistently now, so there isn't much room for confusion over
what the current values mean. Using a percentage might be more familiar
for some people, but OTOH you'll have to get used to the fractions
anyway, unless we change the default output format too, and I'm not in
favour of doing that. I suggest that we just drop this, and remove the
TODO item.
- Heikki
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 12/15/14 8:56 AM, Heikki Linnakangas wrote:
show cursor_tuple_fraction; --> 10%
set cursor_tuple_fraction = .15; --> 15%
set cursor_tuple_fraction = '33%'; --> 33%I tagged four configuration variables to display as percents.
I'm not sure I agree that percentages are better than fractions. I'd
vote a -0.5 for changing the default display format. There isn't much
harm in accepting them as a secondary format, though.
Agreed with not changing the default output. Everyone is used to it,
and there is no reason why the percent output would be intrinsically better.
We should only accept percentages for settings where that makes sense.
It is sensible for most of the PGC_REAL settings, but not so much for
geqo_selection_bias or seed, for example.
Right. But then this feature would get more complicated, as opposed to
supposedly making things simpler.
Overall, I feel that this isn't really worth the trouble. We use
fractions consistently now, so there isn't much room for confusion over
what the current values mean. Using a percentage might be more familiar
for some people, but OTOH you'll have to get used to the fractions
anyway, unless we change the default output format too, and I'm not in
favour of doing that. I suggest that we just drop this, and remove the
TODO item.
Agreed.
The patch is sound as far as it goes (I might be inclined to accept
whitespace between number and % sign), but given the above points and
the original reason for it having been eliminated, I'm inclined to drop it.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Mon, Dec 15, 2014 at 10:19:19AM -0500, Peter Eisentraut wrote:
Overall, I feel that this isn't really worth the trouble. We use
fractions consistently now, so there isn't much room for confusion over
what the current values mean. Using a percentage might be more familiar
for some people, but OTOH you'll have to get used to the fractions
anyway, unless we change the default output format too, and I'm not in
favour of doing that. I suggest that we just drop this, and remove the
TODO item.Agreed.
The patch is sound as far as it goes (I might be inclined to accept
whitespace between number and % sign), but given the above points and
the original reason for it having been eliminated, I'm inclined to drop it.
TODO item removed.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 12/7/14, 1:48 PM, John Gorman wrote:
This patch implements the first wiki/Todo Configuration Files item "Consider normalizing fractions in postgresql.conf, perhaps using '%'".
FWIW, I've reviewed this (should have read the thread first :/). It looks clean, passes make check and works as advertised. I also looked at what config options were set to be % and they make sense. Looking for .[0-9] in postgresql.conf, the only GUCs I saw where % didn't make sense was the two geco GUCs Heikki mentioned.
One thing I don't like is the need to wrap a %-based SET in quotes, but we need to do that for all GUCs that include units, so presumably there's no good way around it.
--
Jim Nasby, Data Architect, Blue Treble Consulting
Data in Trouble? Get it in Treble! http://BlueTreble.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, Dec 16, 2014 at 12:19 AM, Peter Eisentraut <peter_e@gmx.net> wrote:
On 12/15/14 8:56 AM, Heikki Linnakangas wrote:
Overall, I feel that this isn't really worth the trouble. We use
fractions consistently now, so there isn't much room for confusion over
what the current values mean. Using a percentage might be more familiar
for some people, but OTOH you'll have to get used to the fractions
anyway, unless we change the default output format too, and I'm not in
favour of doing that. I suggest that we just drop this, and remove the
TODO item.Agreed.
The patch is sound as far as it goes (I might be inclined to accept
whitespace between number and % sign), but given the above points and
the original reason for it having been eliminated, I'm inclined to drop it.
+1.
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers